12017-11-16T00:02:56  *** meshcollider has quit IRC
  22017-11-16T00:11:55  *** grio_ has joined #bitcoin-core-dev
  32017-11-16T00:12:07  *** grio has quit IRC
  42017-11-16T00:12:19  *** Victor_sueca has joined #bitcoin-core-dev
  52017-11-16T00:13:18  *** Khunbish has quit IRC
  62017-11-16T00:13:38  *** Victorsueca has quit IRC
  72017-11-16T00:22:13  *** meshcollider has joined #bitcoin-core-dev
  82017-11-16T00:23:45  *** goatpig has quit IRC
  92017-11-16T00:31:35  <bitcoin-git> [bitcoin] willyko opened pull request #11700: Add gitian PGP key: willyko (master...master) https://github.com/bitcoin/bitcoin/pull/11700
 102017-11-16T00:40:38  *** jsfour has quit IRC
 112017-11-16T00:45:52  *** jsfour has joined #bitcoin-core-dev
 122017-11-16T00:48:17  *** shtirlic has quit IRC
 132017-11-16T00:49:05  *** shtirlic has joined #bitcoin-core-dev
 142017-11-16T00:53:56  *** shtirlic has quit IRC
 152017-11-16T00:54:23  *** jb55 has quit IRC
 162017-11-16T00:54:45  *** shtirlic has joined #bitcoin-core-dev
 172017-11-16T00:58:11  *** dabura667 has joined #bitcoin-core-dev
 182017-11-16T01:04:28  *** shtirlic has quit IRC
 192017-11-16T01:05:19  *** shtirlic has joined #bitcoin-core-dev
 202017-11-16T01:14:40  *** shtirlic has quit IRC
 212017-11-16T01:21:47  *** Antoinette31Kuhl has joined #bitcoin-core-dev
 222017-11-16T01:21:48  *** intcat has quit IRC
 232017-11-16T01:22:29  *** shtirlic has joined #bitcoin-core-dev
 242017-11-16T01:22:46  *** spinza has quit IRC
 252017-11-16T01:22:46  *** Kirstin82Ledner has quit IRC
 262017-11-16T01:22:46  *** Cory has quit IRC
 272017-11-16T01:22:46  *** BGL has quit IRC
 282017-11-16T01:22:46  *** davec has quit IRC
 292017-11-16T01:22:46  *** d9b4bef9 has quit IRC
 302017-11-16T01:22:46  *** puff has quit IRC
 312017-11-16T01:22:46  *** Lauda has quit IRC
 322017-11-16T01:22:46  *** victorSN has quit IRC
 332017-11-16T01:22:46  *** newbie-- has quit IRC
 342017-11-16T01:22:46  *** Apocalyptic has quit IRC
 352017-11-16T01:22:46  *** neha has quit IRC
 362017-11-16T01:22:46  *** gmaxwell has quit IRC
 372017-11-16T01:22:46  *** CryptAxe has quit IRC
 382017-11-16T01:22:46  *** isis has quit IRC
 392017-11-16T01:22:46  *** newbold_ has quit IRC
 402017-11-16T01:22:46  *** MarcoFalke has quit IRC
 412017-11-16T01:22:46  *** waxwing has quit IRC
 422017-11-16T01:22:46  *** chainhead has quit IRC
 432017-11-16T01:22:47  *** nanotube has quit IRC
 442017-11-16T01:22:47  *** kanzure has quit IRC
 452017-11-16T01:24:36  *** intcat has joined #bitcoin-core-dev
 462017-11-16T01:25:05  *** LumberCartel has quit IRC
 472017-11-16T01:28:16  *** spinza has joined #bitcoin-core-dev
 482017-11-16T01:28:17  *** Cory has joined #bitcoin-core-dev
 492017-11-16T01:28:17  *** BGL has joined #bitcoin-core-dev
 502017-11-16T01:28:17  *** davec has joined #bitcoin-core-dev
 512017-11-16T01:28:17  *** d9b4bef9 has joined #bitcoin-core-dev
 522017-11-16T01:28:17  *** puff has joined #bitcoin-core-dev
 532017-11-16T01:28:17  *** Lauda has joined #bitcoin-core-dev
 542017-11-16T01:28:17  *** victorSN has joined #bitcoin-core-dev
 552017-11-16T01:28:17  *** newbie-- has joined #bitcoin-core-dev
 562017-11-16T01:28:17  *** Apocalyptic has joined #bitcoin-core-dev
 572017-11-16T01:28:17  *** neha has joined #bitcoin-core-dev
 582017-11-16T01:28:17  *** gmaxwell has joined #bitcoin-core-dev
 592017-11-16T01:28:17  *** CryptAxe has joined #bitcoin-core-dev
 602017-11-16T01:28:17  *** isis has joined #bitcoin-core-dev
 612017-11-16T01:28:17  *** newbold_ has joined #bitcoin-core-dev
 622017-11-16T01:28:17  *** MarcoFalke has joined #bitcoin-core-dev
 632017-11-16T01:28:17  *** waxwing has joined #bitcoin-core-dev
 642017-11-16T01:28:17  *** chainhead has joined #bitcoin-core-dev
 652017-11-16T01:28:17  *** nanotube has joined #bitcoin-core-dev
 662017-11-16T01:28:17  *** kanzure has joined #bitcoin-core-dev
 672017-11-16T01:28:26  *** Cory has quit IRC
 682017-11-16T01:28:26  *** BGL has quit IRC
 692017-11-16T01:30:33  *** shtirlic has quit IRC
 702017-11-16T01:31:20  *** shtirlic has joined #bitcoin-core-dev
 712017-11-16T01:35:07  *** Cory has joined #bitcoin-core-dev
 722017-11-16T01:39:41  *** Aaronvan_ has joined #bitcoin-core-dev
 732017-11-16T01:40:04  *** StopAndDecrypt_ has joined #bitcoin-core-dev
 742017-11-16T01:40:22  *** Aaronvan_ has quit IRC
 752017-11-16T01:40:52  *** StopAndDecrypt has quit IRC
 762017-11-16T01:42:17  *** AaronvanW has quit IRC
 772017-11-16T01:45:05  *** sanada has joined #bitcoin-core-dev
 782017-11-16T01:49:08  *** Ylbam has quit IRC
 792017-11-16T01:58:53  *** chartractegg has joined #bitcoin-core-dev
 802017-11-16T01:59:39  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 812017-11-16T02:03:46  *** jb55 has joined #bitcoin-core-dev
 822017-11-16T02:06:32  *** chartractegg has quit IRC
 832017-11-16T02:08:20  *** chartractegg has joined #bitcoin-core-dev
 842017-11-16T02:14:27  *** jtimon has quit IRC
 852017-11-16T02:15:37  <meshcollider> sipa: re #11693 do you know if there is any way to currently signrawtransaction for P2SH-P2WSH ? Or should I open a PR to add a witness program key to that call
 862017-11-16T02:15:38  <gribble> https://github.com/bitcoin/bitcoin/issues/11693 | Signing raw transaction that has p2sh-p2wsh input · Issue #11693 · bitcoin/bitcoin · GitHub
 872017-11-16T02:15:49  *** chartractegg has left #bitcoin-core-dev
 882017-11-16T02:15:53  <meshcollider> Or is there something else planned
 892017-11-16T02:18:38  <sipa> meshcollider: hmm, that is surprising
 902017-11-16T02:19:08  <sipa> meshcollider: maybe we should just automatically add the p2sh version of each script passed to signrawtransaction
 912017-11-16T02:19:17  <sipa> that should work without api break
 922017-11-16T02:19:53  <meshcollider> Mmm so you would just pass the internal WSH script in the scriptSig?
 932017-11-16T02:21:54  *** jsfour has quit IRC
 942017-11-16T02:22:57  *** owowo has quit IRC
 952017-11-16T02:24:42  *** Murch has quit IRC
 962017-11-16T02:27:35  *** BGL has joined #bitcoin-core-dev
 972017-11-16T02:27:51  *** owowo has joined #bitcoin-core-dev
 982017-11-16T02:27:52  *** owowo has joined #bitcoin-core-dev
 992017-11-16T02:27:52  *** owowo has joined #bitcoin-core-dev
1002017-11-16T02:37:39  <meshcollider> JSON*
1012017-11-16T02:38:15  <sipa> right
1022017-11-16T02:38:38  <sipa> hmm, but it would need to guess the witness version
1032017-11-16T02:38:49  <meshcollider> sipa: it's already a JASON object so adding an extra key wouldn't break it anyway would it
1042017-11-16T02:38:55  <meshcollider> Oops did that resend
1052017-11-16T02:42:15  <sipa> okay
1062017-11-16T02:42:17  <sipa> good
1072017-11-16T02:42:23  <gmaxwell> from a usability perspective, having to have some extra key isn't super friendly, if it can be reasonably avoided.
1082017-11-16T02:42:48  <gmaxwell> (these interfaces do exist for thigns other than driving test harnesses, you know)
1092017-11-16T02:46:15  <meshcollider> I don't know how it could be avoided without instead passing in the witness version, like sipa mentioned
1102017-11-16T02:47:46  *** LumberCartel has joined #bitcoin-core-dev
1112017-11-16T02:49:49  <sipa> or alternatively, doing it automatically for all known witness types and versions
1122017-11-16T02:50:31  <meshcollider> Why does the signrawtransaction help text already say "redeemScript": "hex",   (string, required for P2SH or P2WSH) redeem script
1132017-11-16T02:50:47  <meshcollider> Oh don't worry, that's P2WSH not P2SH-P2WSH
1142017-11-16T02:51:02  *** go1111111 has quit IRC
1152017-11-16T02:52:40  <gmaxwell> whatever we do, it should be the case the you can form a valid signraw line using nothing other than a simple reformatting of listunspent's output.
1162017-11-16T02:56:20  <meshcollider> What is the redeemScript output from listunspent for P2SH-P2WSH
1172017-11-16T02:57:12  <meshcollider> I assume it doesn't currently output the witness script at all, which will need to be modified too
1182017-11-16T02:57:19  <sipa> gmaxwell: that's not really possible, and already not true for P2SH
1192017-11-16T02:58:47  <sipa> at least not for watch-only outputs
1202017-11-16T02:59:12  <gmaxwell> sipa: if you've imported the script it should be possible (I have no watch only p2sh so I don't know if it does)
1212017-11-16T02:59:30  <sipa> gmaxwell: it won't work
1222017-11-16T02:59:38  <gmaxwell> We should fix that then.
1232017-11-16T03:00:06  <sipa> i mean it won't work when passing in explicit private keys
1242017-11-16T03:00:27  <sipa> because then it uses the keystore consteucted from the rpc arguments rather than your wallet's keystore
1252017-11-16T03:00:58  <sipa> part of this confusion is solved by finally splitting up signrawtransaction into a wallet version and a utility version (achow has a pr i think)
1262017-11-16T03:01:06  <gmaxwell> the flaw here is that our private key encodings don't represent (or imply) the redeemscript.
1272017-11-16T03:01:30  <sipa> in the wallet version, things already work fine and nothing is needed for p2sh-p2wsk
1282017-11-16T03:01:53  <sipa> in the non-wallet version you inherently need to pass in the solving information somehow
1292017-11-16T03:02:03  <sipa> indeed one way is to encode it in the private key
1302017-11-16T03:02:30  <gmaxwell> or just declare that it's already encoded in the private key, and expand each private key all known ways; but that doesn't work e.g. for multisig.
1312017-11-16T03:02:49  <sipa> but that's not generally possible for all output types (and specifically won't work for.any multisig kinda thing, which is exactly where p2wsh is used), so other ways must exist as well
1322017-11-16T03:02:58  <sipa> right
1332017-11-16T03:03:32  <gmaxwell> e.g. current private keys = {p2pk,p2pkh,p2sh-one-key,p2wpkhv0,p2wpkhv0-p2sh}
1342017-11-16T03:04:27  <gmaxwell> we have the redeemscript for each input we're going to sign as an argument, so it could meet in the middle with the private keys. ugh.
1352017-11-16T03:05:49  <sipa> yeah
1362017-11-16T03:06:22  <sipa> whatwver encoding is used for the solving information, listunspent should probably be made to report it
1372017-11-16T03:06:28  <sipa> if known
1382017-11-16T03:08:44  <Chris_Stewart_5> gmaxwell: So basically this would look like attaching an extra byte of data to the current format to indicate the script it corresponds to>
1392017-11-16T03:08:47  <Chris_Stewart_5> ?
1402017-11-16T03:09:42  <Chris_Stewart_5> and some sort of standardized scheme for standard script types I guess
1412017-11-16T03:17:16  *** m8tion01 has joined #bitcoin-core-dev
1422017-11-16T03:20:31  *** m8tion03 has quit IRC
1432017-11-16T03:25:05  *** checksauce has joined #bitcoin-core-dev
1442017-11-16T03:25:21  <meshcollider> So if I just made listunspent just report witness script for P2SH-P2WSH outputs and then signrawtransaction accept it that would be simple right
1452017-11-16T03:26:33  <meshcollider> That's the cleanest way i can see to ensure P2SH-P2WSH multisig works for example, which is what brought on this discussion
1462017-11-16T03:26:39  <sipa> while you're at it, also add redeemscript for P2SH?
1472017-11-16T03:30:05  *** bule has joined #bitcoin-core-dev
1482017-11-16T03:32:47  *** Chris_Stewart_5 has quit IRC
1492017-11-16T03:38:57  <meshcollider> To listunspent?
1502017-11-16T03:39:24  <sipa> yes, it's also needed in signrawtransaction when giving private keys manually
1512017-11-16T03:39:53  <meshcollider> sipa: I thought listunspent already had it
1522017-11-16T03:42:05  <sipa> oh, indeed!
1532017-11-16T04:01:30  *** checksauce has quit IRC
1542017-11-16T04:08:55  *** checksauce has joined #bitcoin-core-dev
1552017-11-16T04:13:49  *** go1111111 has joined #bitcoin-core-dev
1562017-11-16T04:26:11  *** justan0theruser has joined #bitcoin-core-dev
1572017-11-16T04:26:46  *** justan0theruser has joined #bitcoin-core-dev
1582017-11-16T04:27:00  *** justanotheruser has quit IRC
1592017-11-16T04:28:51  *** lex11 has joined #bitcoin-core-dev
1602017-11-16T04:31:48  *** lex11 has quit IRC
1612017-11-16T04:32:01  *** grio_ has quit IRC
1622017-11-16T04:32:50  *** BCBot has quit IRC
1632017-11-16T04:49:18  *** checksauce has quit IRC
1642017-11-16T04:49:45  *** checksauce has joined #bitcoin-core-dev
1652017-11-16T04:50:27  *** nickler has quit IRC
1662017-11-16T04:51:45  *** nickler has joined #bitcoin-core-dev
1672017-11-16T05:00:10  <bitcoin-git> [bitcoin] jamesob opened pull request #11702: [build] Add a script for installing db4 (master...install-db4-script) https://github.com/bitcoin/bitcoin/pull/11702
1682017-11-16T05:00:24  *** grio has joined #bitcoin-core-dev
1692017-11-16T05:08:28  *** bule has quit IRC
1702017-11-16T05:11:54  *** jsfour has joined #bitcoin-core-dev
1712017-11-16T05:23:00  *** BCBot has joined #bitcoin-core-dev
1722017-11-16T05:27:51  *** checksauce has quit IRC
1732017-11-16T05:32:47  *** go1111111 has quit IRC
1742017-11-16T05:39:00  *** intcat has quit IRC
1752017-11-16T05:39:50  *** intcat has joined #bitcoin-core-dev
1762017-11-16T06:05:21  *** tripleslash has quit IRC
1772017-11-16T06:08:46  *** tripleslash has joined #bitcoin-core-dev
1782017-11-16T06:09:29  *** go1111111 has joined #bitcoin-core-dev
1792017-11-16T06:14:47  *** Taco has joined #bitcoin-core-dev
1802017-11-16T06:19:10  *** Taco has quit IRC
1812017-11-16T06:34:15  *** Ylbam has joined #bitcoin-core-dev
1822017-11-16T06:35:39  *** checksauce has joined #bitcoin-core-dev
1832017-11-16T06:41:49  *** indistylo has joined #bitcoin-core-dev
1842017-11-16T07:21:51  *** jl2012 has quit IRC
1852017-11-16T07:23:02  *** sugarpuff has quit IRC
1862017-11-16T07:25:08  *** sugarpuff has joined #bitcoin-core-dev
1872017-11-16T07:33:20  *** Victor_sueca is now known as Victorsueca
1882017-11-16T07:44:57  *** jsfour has quit IRC
1892017-11-16T07:51:08  *** intcat has quit IRC
1902017-11-16T07:51:09  *** arubi has quit IRC
1912017-11-16T07:53:38  *** intcat has joined #bitcoin-core-dev
1922017-11-16T07:55:39  *** BashCo has quit IRC
1932017-11-16T07:56:15  *** BashCo has joined #bitcoin-core-dev
1942017-11-16T07:56:25  *** arubi has joined #bitcoin-core-dev
1952017-11-16T07:58:57  *** checksauce has quit IRC
1962017-11-16T08:00:27  *** BashCo has quit IRC
1972017-11-16T08:06:31  *** dat_ has joined #bitcoin-core-dev
1982017-11-16T08:23:55  *** checksauce has joined #bitcoin-core-dev
1992017-11-16T08:25:35  *** LumberCartel has quit IRC
2002017-11-16T08:25:59  *** BashCo has joined #bitcoin-core-dev
2012017-11-16T08:26:14  <eck> random question: if leveldb already has its own WAL, then why does bitcoin need an in-memory cache/buffer for utxo updates
2022017-11-16T08:27:04  <wumpus> eck: because we can do it more efficient because we know the properties of the data and access patterns
2032017-11-16T08:27:21  <wumpus> eck: leveldb's caching is virtually useless for our use case, so we minimize their caches
2042017-11-16T08:28:19  <wumpus> why does a database implement its own caching if the OS already has a page cache'
2052017-11-16T08:28:24  *** qoqsj has joined #bitcoin-core-dev
2062017-11-16T08:28:28  <wumpus> would be similar :)
2072017-11-16T08:28:50  <eck> i don't care/know that much about how leveldb implements read caches, but for writes, it seems really problematic you have multiple caching layrs
2082017-11-16T08:28:53  *** LumberCartel has joined #bitcoin-core-dev
2092017-11-16T08:29:19  <eck> in this example you have bitcoin cache + leveldb cache + kernel page cache
2102017-11-16T08:29:20  *** checksauce has quit IRC
2112017-11-16T08:29:27  <eck> that's a lot of caching layers
2122017-11-16T08:29:58  <wumpus> why would that be problematic? it's pretty much how modern platforms work, your CPU also has caches as different levels
2132017-11-16T08:30:12  <sipa> eck: there are a number of reasons why our caching provides functionality on top of LevelDB's
2142017-11-16T08:30:37  <sipa> one is that LevelDB inherently deals with serialized records, while our caching layer stores information using our native in-memory formats
2152017-11-16T08:30:46  <sipa> serialization and deserialization have a nontrivial cost
2162017-11-16T08:30:53  <wumpus> in any case, if you find a way to improve performance, go for it
2172017-11-16T08:31:45  <sipa> furthermore, we're able to exploit a very significant property of our data set: entries are written once, and deleted once - and almost never overwritten in between
2182017-11-16T08:32:02  <eck> maybe I can or cannot, just trying to make sure I understand the current state of affaris:
2192017-11-16T08:32:26  <eck> as i understand it, by default a full node today will flush to disk as infrequently as once every 24h
2202017-11-16T08:32:35  <sipa> we keep track in our cache whether an entry exists in the caching layer below, and if not (= it's a freshly created entry) which gets deleted (utxo gets spent), we simply delete it from memory, without it ever needing to touch disk
2212017-11-16T08:33:11  <sipa> this optimization saves us 90% of all I/O or more, depending on cache sizes
2222017-11-16T08:33:27  <sipa> because many UTXO entries get spent quickly after being created
2232017-11-16T08:33:41  <bitcoin-git> [bitcoin] laanwj pushed 9 new commits to master: https://github.com/bitcoin/bitcoin/compare/54aedc013744...3c098a8aa078
2242017-11-16T08:33:42  <bitcoin-git> bitcoin/master 1a44534 MeshCollider: scripted-diff: Replace #include "" with #include <> (ryanofsky)...
2252017-11-16T08:33:43  <bitcoin-git> bitcoin/master 5b56ec9 Wladimir J. van der Laan: qt: refactor: Use absolute include paths in .ui files
2262017-11-16T08:33:43  <eck> could you point me (rougly) to the part of the code i woudl oook at?
2272017-11-16T08:33:44  <bitcoin-git> bitcoin/master 0c71521 Wladimir J. van der Laan: build: Remove -I for everything but project root...
2282017-11-16T08:33:46  <wumpus> yes, leveldb does not do this for batches
2292017-11-16T08:34:07  <sipa> eck: coins.cpp
2302017-11-16T08:34:12  <wumpus> so if you queue both adds, updates, and deletes, they will actually get executed
2312017-11-16T08:34:16  <bitcoin-git> [bitcoin] laanwj closed pull request #11651: refactor: Make all #includes relative to project root (laanwj, MeshCollider, ryanofsky) (master...201711_absolute_includes) https://github.com/bitcoin/bitcoin/pull/11651
2322017-11-16T08:34:42  <sipa> yes, LevelDB's "batch" structure is literally an std::string
2332017-11-16T08:34:47  <wumpus> also leveldb's batch storage format is really inefficient with regard to memroy allocation, large consecutive memory areas
2342017-11-16T08:34:48  <wumpus> right
2352017-11-16T08:34:51  <sipa> with serialized write/erase operations in it
2362017-11-16T08:35:22  <gmaxwell> eck: the infrequency is basically unrelated to the structure, it could continiously flush, now that we have non-atomic flushing... that just hasn't been implemented yet.
2372017-11-16T08:35:48  <eck> thanks! in the past I wrote my own (simpler obviously) C++ implementation of an ss-table-like structure, and I'm trying to confirm that the whole caching i/o layer in bitcoind works as I expect it would
2382017-11-16T08:35:48  <gmaxwell> and in general we want to delay flushing as much as possible in order to suppress writes from ever happening in the first place.
2392017-11-16T08:35:50  <sipa> right, the idea is to move the flushing to a background process that continuously flushes the oldest dirty cache entries
2402017-11-16T08:36:24  <sipa> but never gets too close to 'now', as to not interfere with our freshness optimization
2412017-11-16T08:36:39  <sipa> eck: cool
2422017-11-16T08:36:57  *** jouke has quit IRC
2432017-11-16T08:39:48  *** jouke has joined #bitcoin-core-dev
2442017-11-16T08:39:55  <eck> thanks everyone for your help, I will report back if I have further conclusions or questions
2452017-11-16T08:41:28  <sipa> yw!
2462017-11-16T08:50:09  *** ekrok_ has quit IRC
2472017-11-16T08:50:28  *** ekrok_ has joined #bitcoin-core-dev
2482017-11-16T08:51:30  *** laurentmt has joined #bitcoin-core-dev
2492017-11-16T08:52:11  *** whphhg has quit IRC
2502017-11-16T08:52:19  *** whphhg_ has joined #bitcoin-core-dev
2512017-11-16T08:52:45  *** whphhg_ is now known as whphhg
2522017-11-16T08:53:46  <eck> related, if I wanted to follow up with someone speifically about the storage layer, who is an expert?
2532017-11-16T08:54:11  <gmaxwell> just ask here.
2542017-11-16T08:54:16  <gmaxwell> other people would learn from your questions.
2552017-11-16T08:55:58  <eck> alright, at a high level i want to know why bitcoin has a caching layer at all: there is already some caching in the kernel page cache, leveldb has some caching og it its own, and then bitcoin itself will cache data for up to 24h
2562017-11-16T08:56:56  <wumpus> that's what sipa just explained
2572017-11-16T08:57:00  <eck> from my superfcial understanding, i would just write directly to leveldb and let it take care of the rest
2582017-11-16T08:57:20  <wumpus> that was the first thing that was tried, and perf was abysmal
2592017-11-16T08:57:36  <gmaxwell> eck: you can do that, it takes about a week to sync the chain that way on typical hardware.
2602017-11-16T08:57:58  <wumpus> that might work better with other databases, but not leveldb
2612017-11-16T08:58:15  <gmaxwell> everything else we've tried was basically an order of magnitude slower than leveldb.
2622017-11-16T08:58:28  <wumpus> the best (only) way to get feeling for it is to experiment, it's really hard to beat the performance of the current solution
2632017-11-16T08:58:33  <eck> interesting, I would like to / am willin g to repeat the experiment to verify it locally
2642017-11-16T08:59:08  <gmaxwell> eck: you can just set the dbcache to a minimal value and see the result for 95% of the effect.
2652017-11-16T08:59:42  <eck> ok, thanks
2662017-11-16T08:59:58  <wumpus> I have some old utxo database experiments here: https://github.com/laanwj/bitcoin
2672017-11-16T09:00:02  <gmaxwell> eck: syncing the chain with every operation going to the database requires about 1 billion database updates.
2682017-11-16T09:00:10  <wumpus> maybe that's useful at least for seeing what files to touch...
2692017-11-16T09:00:45  *** qoqsj has quit IRC
2702017-11-16T09:01:10  <eck> generally though,i would account for the wallet db taking basically 0 time, right?
2712017-11-16T09:01:21  <wumpus> LMDB seemed promising but I never got spectacular results, maybe I was just using it wrong, and maybe the approach of not caching at the bitcoin level would work better with it
2722017-11-16T09:02:02  <gmaxwell> eck: I can't figure out why you think the wallet database would be involved at all.
2732017-11-16T09:02:12  <wumpus> wallet has nothing to do with this, when you benchmark this, I suggest you disable it compile-time
2742017-11-16T09:02:21  <gmaxwell> The wallet database uses reasources when you have wallets with loads of transactions and keys, otherwise it does nothing.
2752017-11-16T09:02:43  <eck> i don't care about the wallet database, it's slow but it's smalll
2762017-11-16T09:03:13  <wumpus> it's also only read at startup, and written when the wallet actually changes, which is infrequent for most people
2772017-11-16T09:03:15  <eck> my interest recently is syncing a g1-small GCE instance
2782017-11-16T09:03:35  <gmaxwell> in any case, to sync in three hours (which we do, on a fast desktop at least with dbcache cranked) without a dbcache would require sustaining 100k operations per second, which is not realistic except on specialized hardware (e.g. nvme raid or whatever).
2792017-11-16T09:03:42  <eck> it has 1.5GB of memory, and essentially 0 disk I/O
2802017-11-16T09:03:44  *** inimaxedwards has joined #bitcoin-core-dev
2812017-11-16T09:04:32  <wumpus> it has terrible I/O, disabling caching is certainly not what you should look at
2822017-11-16T09:04:41  <eck> my instance on core right now is synicin at < 5% a *day*, which seems like there is some fundamental horkage in some layer
2832017-11-16T09:04:42  <wumpus> sync on a faster machine then copy over the state
2842017-11-16T09:04:50  <eck> yeah maybe
2852017-11-16T09:05:37  <gmaxwell> I think you might just be underestimating how much work is involved in syncing and how slow those instances are. :)
2862017-11-16T09:05:38  <wumpus> w/ the cloud thing, you could just hire one of the large instances for a day or so
2872017-11-16T09:06:00  <gmaxwell> with 1.5GB memory you may be able to increase the cache further without crashing, it will speed it up.
2882017-11-16T09:06:44  <eck> from what I can tell, on GCE they give you I/O capacity base on the disk size
2892017-11-16T09:07:02  <wumpus> this is comparable to running a node on e.g. rpi, they can keep up, but doing initial sync on them takes extreme amounts of patience
2902017-11-16T09:07:09  <eck> so if you want 200 GB, you're basically on teh bottom wrung, even if you have a ton of cpu/memory
2912017-11-16T09:07:34  <gmaxwell> But there is just an utterly stupendous amount of work required, ... the software is highly optimized (and sure, there are also still many things that could be done to make it faster). .. but e.g. compared to ethereum the amount of blockchain bitcoin core processes per second is something like two orders of magnitude greater. ( https://anduck.net/bitcoincore_vs_geth_full_node_stats.png )
2922017-11-16T09:07:48  <wumpus> if it's a vm maybe you can temporary increase the amount if memory, then sync with dbcache of 4000, then i/o will not be touched until it's done
2932017-11-16T09:09:06  <eck> thanks all, I will definitely be in here asking about this gain
2942017-11-16T09:09:12  <gmaxwell> indeed, the only IO with a huge db cache is just writing blocks out to disk and the final flush.
2952017-11-16T09:09:43  <eck> I will ask more once I've done more research
2962017-11-16T09:10:16  <wumpus> oh yes it will need to write out the blocks, but that's linear and granular, not seek-heavy database i/o I meant
2972017-11-16T09:11:32  *** Antoinette31Kuhl has quit IRC
2982017-11-16T09:17:57  *** jl2012 has joined #bitcoin-core-dev
2992017-11-16T09:19:04  *** Edward48Beier has joined #bitcoin-core-dev
3002017-11-16T09:20:44  *** intcat has quit IRC
3012017-11-16T09:21:57  *** intcat has joined #bitcoin-core-dev
3022017-11-16T09:29:43  *** jadox has joined #bitcoin-core-dev
3032017-11-16T09:30:52  *** Edward48Beier has quit IRC
3042017-11-16T09:40:56  *** jsfour has joined #bitcoin-core-dev
3052017-11-16T09:43:39  *** Yasmeen76Emard has joined #bitcoin-core-dev
3062017-11-16T09:45:09  *** jsfour has quit IRC
3072017-11-16T09:45:54  *** AaronvanW has joined #bitcoin-core-dev
3082017-11-16T09:50:28  *** qposdf has joined #bitcoin-core-dev
3092017-11-16T10:01:02  *** Ylbam has quit IRC
3102017-11-16T10:02:56  *** promag has joined #bitcoin-core-dev
3112017-11-16T10:04:15  *** roconnor_ has quit IRC
3122017-11-16T10:05:29  *** timothy has joined #bitcoin-core-dev
3132017-11-16T10:14:19  *** dabura667 has quit IRC
3142017-11-16T10:14:50  *** jadox has quit IRC
3152017-11-16T10:28:16  <meshcollider> To add witnessScript to listunspent output, how do we retrieve a CScript from the wallet using WitnessV0ScriptHash if the CScripts are indexed by CScriptID which is Hash160 not SHA?
3162017-11-16T10:28:46  <sipa> aha!
3172017-11-16T10:29:04  <sipa> Hash160(x) = RIPEMD160(SHA256(x))
3182017-11-16T10:29:14  *** Yasmeen76Emard has quit IRC
3192017-11-16T10:29:27  <sipa> so given y=SHA256(x) you can compute Hash160(x) as RIPEMD160(y)
3202017-11-16T10:29:31  <meshcollider> Oh! Perfect :)
3212017-11-16T10:36:10  *** Edgardo10Toy has joined #bitcoin-core-dev
3222017-11-16T10:49:13  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/3c098a8aa078...66d46c7901b7
3232017-11-16T10:49:14  <bitcoin-git> bitcoin/master ec85248 John Newbery: [travis-ci] Only run linters on Pull Requests...
3242017-11-16T10:49:14  <bitcoin-git> bitcoin/master 66d46c7 Wladimir J. van der Laan: Merge #11699: [travis-ci] Only run linters on Pull Requests...
3252017-11-16T10:49:46  <bitcoin-git> [bitcoin] laanwj closed pull request #11699: [travis-ci] Only run linters on Pull Requests (master...lint_only_prs) https://github.com/bitcoin/bitcoin/pull/11699
3262017-11-16T10:50:06  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/66d46c7901b7...084f52f38dc2
3272017-11-16T10:50:06  <bitcoin-git> bitcoin/master 069215e practicalswift: Initialize recently introduced non-static class member lastCycles to zero in constructor...
3282017-11-16T10:50:06  <bitcoin-git> bitcoin/master 084f52f Wladimir J. van der Laan: Merge #11654: tests: Initialize recently introduced non-static class member lastCycles to zero in constructor...
3292017-11-16T10:50:37  <bitcoin-git> [bitcoin] laanwj closed pull request #11654: tests: Initialize recently introduced non-static class member lastCycles to zero in constructor (master...uninitialized-members) https://github.com/bitcoin/bitcoin/pull/11654
3302017-11-16T10:57:39  *** asu has joined #bitcoin-core-dev
3312017-11-16T10:58:25  <asu> hi, all
3322017-11-16T10:58:44  <asu> i has a question.
3332017-11-16T10:59:50  <asu> i want to build a bitcoin p2p trade web
3342017-11-16T11:00:48  <asu> buy i found if my customer send 0.01bitcoin to another, the fee is 0.00027 bitcoin
3352017-11-16T11:01:59  <asu> but in localbitcoins.com, the fee is zero
3362017-11-16T11:02:27  <asu> how can i do it? thanks
3372017-11-16T11:02:46  <asu> who can help me
3382017-11-16T11:03:07  <kinlo> asu: that question is for #bitcoin, not for here
3392017-11-16T11:03:43  <asu> ok, thanks
3402017-11-16T11:04:18  <asu> where is the bitcoin irc?
3412017-11-16T11:04:29  <kinlo> just join #bitcoin
3422017-11-16T11:04:39  <asu> join #bitcoin
3432017-11-16T11:05:05  <kinlo> it's /join #bitcoin
3442017-11-16T11:05:35  <asu> thanks
3452017-11-16T11:11:02  <kgc> Hi, I updated issue https://github.com/bitcoin/bitcoin/issues/11693 with some more information, while I can proceed with my implementation it's slightly tricky/confusing to use.
3462017-11-16T11:11:49  <meshcollider> sipa: It appears there is a way to use signrawtransaction with P2SH-P2WSH as-is, check out https://bitcoin.stackexchange.com/a/62746/51948
3472017-11-16T11:12:48  <meshcollider> basically put the same input twice, once with the P2SH redeemScript and once with the witness redeemScript
3482017-11-16T11:13:20  <meshcollider> which is good to know but certainly not the cleanest way to do it
3492017-11-16T11:14:09  <kgc> yeah
3502017-11-16T11:14:39  <kgc> I made a suggestion how to potentially clean it up a bit, but that's already for you to decide change or not :)
3512017-11-16T11:16:02  <meshcollider> kgc: I'm working on something right now here: github.com/MeshCollider/bitcoin/tree/201711_signrawtransaction_wsh
3522017-11-16T11:16:20  <meshcollider> haven't tested yet
3532017-11-16T11:18:21  <kgc> good to know
3542017-11-16T11:19:37  <kgc> if it makes it to official release most probably will switch it using to avoid confusion in the future when looking at my own code
3552017-11-16T11:20:00  *** ghost43 has quit IRC
3562017-11-16T11:20:22  *** ghost43 has joined #bitcoin-core-dev
3572017-11-16T11:21:57  *** shesek has quit IRC
3582017-11-16T11:24:28  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/084f52f38dc2...99bc0b428b03
3592017-11-16T11:24:29  <bitcoin-git> bitcoin/master 28f8b66 Eelis: Diagnose unsuitable outputs in lockunspent()....
3602017-11-16T11:24:29  <bitcoin-git> bitcoin/master 99bc0b4 Wladimir J. van der Laan: Merge #11087: Diagnose unsuitable outputs in lockunspent()....
3612017-11-16T11:24:53  <bitcoin-git> [bitcoin] laanwj closed pull request #11087: Diagnose unsuitable outputs in lockunspent(). (master...lockunspent) https://github.com/bitcoin/bitcoin/pull/11087
3622017-11-16T11:41:46  *** jsfour has joined #bitcoin-core-dev
3632017-11-16T11:43:06  <wumpus> re: https://github.com/bitcoin/bitcoin/pull/11281#discussion_r151390218
3642017-11-16T11:43:25  <wumpus> is there anything that makes scanning blocks out of order go wrong?
3652017-11-16T11:43:49  <wumpus> (maybe when there are chains of transactions depending on each other?)
3662017-11-16T11:44:56  <wumpus> in that case we'd want to disable scanning of the wallet of incoming blocks when it is rescanning
3672017-11-16T11:45:55  <wumpus> and we would need logic to backtrack in case of reorgs
3682017-11-16T11:45:57  *** jsfour has quit IRC
3692017-11-16T11:48:10  *** asu has quit IRC
3702017-11-16T11:56:20  *** qposdf has quit IRC
3712017-11-16T12:02:47  *** jsfour has joined #bitcoin-core-dev
3722017-11-16T12:05:03  *** shesek has joined #bitcoin-core-dev
3732017-11-16T12:05:03  *** shesek has joined #bitcoin-core-dev
3742017-11-16T12:07:15  *** jsfour has quit IRC
3752017-11-16T12:18:21  *** jsfour has joined #bitcoin-core-dev
3762017-11-16T12:20:36  *** shesek has quit IRC
3772017-11-16T12:21:55  *** mxg has joined #bitcoin-core-dev
3782017-11-16T12:22:57  *** jsfour has quit IRC
3792017-11-16T12:40:52  *** SopaXorzTaker has joined #bitcoin-core-dev
3802017-11-16T12:45:00  *** lex11 has joined #bitcoin-core-dev
3812017-11-16T12:47:45  *** lex11 has quit IRC
3822017-11-16T12:51:04  *** mxg has quit IRC
3832017-11-16T12:51:11  *** d9b4bef9 has quit IRC
3842017-11-16T12:52:24  *** d9b4bef9 has joined #bitcoin-core-dev
3852017-11-16T12:57:35  <bitcoin-git> [bitcoin] sipsorcery opened pull request #11704: Windows build doc update (master...windoc) https://github.com/bitcoin/bitcoin/pull/11704
3862017-11-16T13:02:47  *** promag has quit IRC
3872017-11-16T13:18:35  *** Gnof has joined #bitcoin-core-dev
3882017-11-16T13:39:40  *** goatpig has joined #bitcoin-core-dev
3892017-11-16T13:44:59  *** Gnof has quit IRC
3902017-11-16T13:46:22  *** jtimon has joined #bitcoin-core-dev
3912017-11-16T13:49:03  *** roconnor_ has joined #bitcoin-core-dev
3922017-11-16T14:00:18  *** promag has joined #bitcoin-core-dev
3932017-11-16T14:14:32  *** bitstr3am has joined #bitcoin-core-dev
3942017-11-16T14:15:45  *** dat_ has quit IRC
3952017-11-16T14:16:46  *** blockchain has joined #bitcoin-core-dev
3962017-11-16T14:17:15  *** bitstr3am has quit IRC
3972017-11-16T14:19:12  *** jsfour has joined #bitcoin-core-dev
3982017-11-16T14:21:53  *** indistylo has quit IRC
3992017-11-16T14:22:56  *** meshcollider has quit IRC
4002017-11-16T14:23:08  *** lukedashjr has joined #bitcoin-core-dev
4012017-11-16T14:23:24  *** jsfour has quit IRC
4022017-11-16T14:24:01  *** luke-jr has quit IRC
4032017-11-16T14:27:32  *** lukedashjr is now known as luke-jr
4042017-11-16T14:45:43  *** jsfour has joined #bitcoin-core-dev
4052017-11-16T14:46:22  <bitcoin-git> [bitcoin] laanwj closed pull request #10772: Implementation of BIP8 (master...bip8-height) https://github.com/bitcoin/bitcoin/pull/10772
4062017-11-16T14:47:25  *** andytoshi has quit IRC
4072017-11-16T14:47:25  *** andytoshi has joined #bitcoin-core-dev
4082017-11-16T14:49:16  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4092017-11-16T14:49:41  *** coin_trader has quit IRC
4102017-11-16T14:50:07  *** jsfour has quit IRC
4112017-11-16T14:50:22  *** coin_trader has joined #bitcoin-core-dev
4122017-11-16T14:51:45  *** promag has quit IRC
4132017-11-16T14:59:41  *** lex11 has joined #bitcoin-core-dev
4142017-11-16T15:02:33  *** lex11 has quit IRC
4152017-11-16T15:16:07  <bitcoin-git> [bitcoin] zhaokexun opened pull request #11705: 0.15 (master...0.15) https://github.com/bitcoin/bitcoin/pull/11705
4162017-11-16T15:17:27  <bitcoin-git> [bitcoin] laanwj closed pull request #11705: 0.15 (master...0.15) https://github.com/bitcoin/bitcoin/pull/11705
4172017-11-16T15:19:29  *** Chris_Stewart_5 has quit IRC
4182017-11-16T15:27:24  *** lex11 has joined #bitcoin-core-dev
4192017-11-16T15:33:10  *** lex11 has quit IRC
4202017-11-16T15:33:55  *** lex11 has joined #bitcoin-core-dev
4212017-11-16T15:34:57  *** jcorgan has quit IRC
4222017-11-16T15:34:57  *** jcorgan has joined #bitcoin-core-dev
4232017-11-16T15:36:12  *** lex11 has quit IRC
4242017-11-16T15:40:08  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4252017-11-16T15:57:57  *** jb55 has quit IRC
4262017-11-16T16:01:21  *** Chris_Stewart_5 has quit IRC
4272017-11-16T16:21:04  *** indistylo has joined #bitcoin-core-dev
4282017-11-16T16:26:36  <BlueMatt> wumpus: do you want a followup pr to #11686 ?
4292017-11-16T16:26:38  <gribble> https://github.com/bitcoin/bitcoin/issues/11686 | Make ISSUE_TEMPLATE a bit shorter, mention hardware tests by TheBlueMatt · Pull Request #11686 · bitcoin/bitcoin · GitHub
4302017-11-16T16:28:11  *** d9b4bef9 has quit IRC
4312017-11-16T16:28:30  <wumpus> BlueMatt: I think it'd make sense, but I don't really want to turn it into a controversial topic
4322017-11-16T16:29:01  <BlueMatt> oh I dont think anyone cares *that* much, I was just curious if you want more pr volume
4332017-11-16T16:29:12  <BlueMatt> its also a github md file, not like we need to have big review cycles on it.......
4342017-11-16T16:29:16  *** d9b4bef9 has joined #bitcoin-core-dev
4352017-11-16T16:34:54  *** Khunbish has joined #bitcoin-core-dev
4362017-11-16T16:35:41  *** m8tion01 has quit IRC
4372017-11-16T16:37:07  *** LumberCartel has quit IRC
4382017-11-16T16:37:25  <wumpus> right, a change to a github md isn't so bad with regards to PR volume
4392017-11-16T16:37:47  <BlueMatt> k, I'll tweak it again
4402017-11-16T16:37:56  <wumpus> thanks :)
4412017-11-16T16:44:13  *** cbag has joined #bitcoin-core-dev
4422017-11-16T16:46:09  *** jsfour has joined #bitcoin-core-dev
4432017-11-16T16:50:32  *** jsfour has quit IRC
4442017-11-16T16:50:59  *** sunday-afternoon has joined #bitcoin-core-dev
4452017-11-16T16:51:44  <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #11706: Make default issue text all comments to make issues more readable (master...2017-11-shorter-default-issue-redux) https://github.com/bitcoin/bitcoin/pull/11706
4462017-11-16T16:51:46  <BlueMatt> wumpus: ^
4472017-11-16T16:55:46  *** chartractegg has joined #bitcoin-core-dev
4482017-11-16T17:04:12  *** BashCo has quit IRC
4492017-11-16T17:04:34  *** indistylo has quit IRC
4502017-11-16T17:06:18  *** LumberCartel has joined #bitcoin-core-dev
4512017-11-16T17:24:06  *** jsfour has joined #bitcoin-core-dev
4522017-11-16T17:25:39  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4532017-11-16T17:29:02  *** jsfour has quit IRC
4542017-11-16T17:29:05  *** kk_ has joined #bitcoin-core-dev
4552017-11-16T17:33:15  *** BashCo has joined #bitcoin-core-dev
4562017-11-16T17:43:01  *** LumberCartel has quit IRC
4572017-11-16T17:44:07  *** RubenSomsen has joined #bitcoin-core-dev
4582017-11-16T17:47:21  *** jsfour has joined #bitcoin-core-dev
4592017-11-16T17:55:05  *** cbag has quit IRC
4602017-11-16T18:01:55  *** LumberCartel has joined #bitcoin-core-dev
4612017-11-16T18:04:38  *** Murch has joined #bitcoin-core-dev
4622017-11-16T18:34:12  *** kk_ has quit IRC
4632017-11-16T18:36:26  *** timothy has quit IRC
4642017-11-16T18:36:37  *** sh4ft has joined #bitcoin-core-dev
4652017-11-16T18:39:34  *** Ylbam has joined #bitcoin-core-dev
4662017-11-16T18:42:03  *** RubenSomsen has quit IRC
4672017-11-16T18:42:08  *** sipa has quit IRC
4682017-11-16T18:42:09  *** sipa has joined #bitcoin-core-dev
4692017-11-16T18:43:01  *** tux3do has joined #bitcoin-core-dev
4702017-11-16T18:43:32  *** laurentmt has quit IRC
4712017-11-16T18:54:00  *** JackH has joined #bitcoin-core-dev
4722017-11-16T18:55:26  <bitcoin-git> [bitcoin] jnewbery opened pull request #11707: [tests] Fix sendheaders (master...fix_sendheaders) https://github.com/bitcoin/bitcoin/pull/11707
4732017-11-16T18:55:34  *** meshcollider has joined #bitcoin-core-dev
4742017-11-16T19:00:18  <achow101> meeting?
4752017-11-16T19:00:45  <luke-jr> hajimemashite?
4762017-11-16T19:01:04  <wumpus> #startmeeting
4772017-11-16T19:01:04  <lightningbot> Meeting started Thu Nov 16 19:01:04 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
4782017-11-16T19:01:04  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
4792017-11-16T19:01:22  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag
4802017-11-16T19:01:30  <sipa> present
4812017-11-16T19:01:33  <achow101> hi
4822017-11-16T19:01:34  <jtimon> hi
4832017-11-16T19:01:35  <meshcollider> hello
4842017-11-16T19:01:41  <gmaxwell> hi
4852017-11-16T19:01:45  <sdaftuar> ack
4862017-11-16T19:01:48  <jonasschnelli> hi
4872017-11-16T19:01:56  <gmaxwell> will be back in 10 minutes, maybe the meeting won't be over by then. :P
4882017-11-16T19:02:11  <wumpus> #topic high priority for review
4892017-11-16T19:02:16  <BlueMatt> new high-priority for me: #11639
4902017-11-16T19:02:18  <gribble> https://github.com/bitcoin/bitcoin/issues/11639 | Rewrite the interface between validation and net_processing wrt DoS by TheBlueMatt · Pull Request #11639 · bitcoin/bitcoin · GitHub
4912017-11-16T19:02:32  <kanzure> hi.
4922017-11-16T19:02:55  <wumpus> only four things left https://github.com/bitcoin/bitcoin/projects/8
4932017-11-16T19:03:23  <BlueMatt> also probably worth a post-merge review: #10286 (note that this will likely make lots of open wallet-rpc change conflict silently - you need to add the new BlockUntilSyncedToCurrentChain call in some wallet rpc functions as boiler plate, see dev docs for more)
4942017-11-16T19:03:27  <gribble> https://github.com/bitcoin/bitcoin/issues/10286 | Call wallet notify callbacks in scheduler thread (without cs_main) by TheBlueMatt · Pull Request #10286 · bitcoin/bitcoin · GitHub
4952017-11-16T19:03:31  *** promag has joined #bitcoin-core-dev
4962017-11-16T19:03:37  <wumpus> added 11639
4972017-11-16T19:03:42  <promag> Hi
4982017-11-16T19:04:15  <luke-jr> should #11383 be on there? I can rebase after the meeting
4992017-11-16T19:04:18  <gribble> https://github.com/bitcoin/bitcoin/issues/11383 | Basic Multiwallet GUI support by luke-jr · Pull Request #11383 · bitcoin/bitcoin · GitHub
5002017-11-16T19:04:34  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
5012017-11-16T19:04:45  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
5022017-11-16T19:05:12  <wumpus> luke-jr: added
5032017-11-16T19:05:16  <bitcoin-git> [bitcoin] luke-jr closed pull request #10391: OP_CHECKBLOCKATHEIGHT anti-replay (BIP 115; logic only) (master...cbah) https://github.com/bitcoin/bitcoin/pull/10391
5042017-11-16T19:06:07  <promag> Rpc console still only for 1st wallet right?
5052017-11-16T19:06:33  <luke-jr> promag: that PR has an independent combobox for the debug window
5062017-11-16T19:06:46  <luke-jr> (including a "no wallet" option)
5072017-11-16T19:07:01  <promag> Should rebase on dynamic wallet loading? Or vice-versa?
5082017-11-16T19:07:01  <wumpus> #topic rpc console for multi wallet
5092017-11-16T19:07:16  <jonasschnelli> The dropdown seems okay isch.
5102017-11-16T19:07:23  <luke-jr> promag: IMO GUI should go before dynamic loading
5112017-11-16T19:07:28  <jonasschnelli> Ideally we would have a higher-level visual selector
5122017-11-16T19:07:31  <promag> Kk
5132017-11-16T19:07:38  <luke-jr> jonasschnelli: ?
5142017-11-16T19:07:41  <jonasschnelli> luke-jr: agree
5152017-11-16T19:08:09  <jonasschnelli> luke-jr: it confusing to have a wallet level switch in the console
5162017-11-16T19:08:24  <jtimon> what's wrong with the combobox ?
5172017-11-16T19:08:30  <jonasschnelli> But I don't see another simple way
5182017-11-16T19:08:41  <promag> One thing that bothers me with the combo is that the gui state is lost
5192017-11-16T19:08:45  <luke-jr> maybe improvements there can be made after merge, if someone thinks of a better way
5202017-11-16T19:08:52  <luke-jr> promag: ?
5212017-11-16T19:08:56  <achow101> I think it might be confusing to users to have the debug window possibly be for a different wallet than the main wallet gui
5222017-11-16T19:09:12  <wumpus> the combobox is ok
5232017-11-16T19:09:16  <jonasschnelli> I think its an acceptable first step
5242017-11-16T19:09:22  <promag> Like list scroll position, selection, focus, etc
5252017-11-16T19:09:24  <sipa> pieter was here
5262017-11-16T19:09:38  <wumpus> the debug window is supposed to be separate from the main GUI, having it influence what wallet is selected is even more confusing
5272017-11-16T19:09:40  <jtimon> I think it's perfectly fine for the debug console to be flexible like this. seems just handy to put it there
5282017-11-16T19:09:50  <wumpus> yes
5292017-11-16T19:09:56  <promag> Another option is one tab per wallet
5302017-11-16T19:10:02  <wumpus> no, please not
5312017-11-16T19:10:14  <luke-jr> maybe (post-merge) an idea might be to have a red alert icon next to the combobox if it doesn't match the main window
5322017-11-16T19:10:43  <achow101> I was thinking that when you first opened the debug window it could default to the wallet that was in use in the main window
5332017-11-16T19:10:44  <wumpus> meh
5342017-11-16T19:10:54  <achow101> then users can change the wallet if they want to
5352017-11-16T19:11:13  <jonasschnelli> I think the dropbox is still the best solution on the table,... (even if not ideal)
5362017-11-16T19:11:14  <jtimon> that sounds reasonable to me
5372017-11-16T19:11:18  <wumpus> I really think having the debug window and main window interact in that way is a mess both in code and in interaction, but anyhow
5382017-11-16T19:11:40  <wumpus> okay, any other topic?
5392017-11-16T19:11:41  <luke-jr> sounds like we at least agree it's a post-merge topic XD
5402017-11-16T19:11:45  <jonasschnelli> ack
5412017-11-16T19:11:48  <jtimon> oh, if it's a mess in the code, I'm not sure it's worth it. I'll shut up
5422017-11-16T19:11:52  <promag> wumpus: btw why not tabs?
5432017-11-16T19:12:07  <wumpus> promag: multiple tabs with the same console just pointing at a different wallet sounds terrible to me
5442017-11-16T19:12:08  <jtimon> spaces are better
5452017-11-16T19:12:11  <jonasschnelli> promag: most calls are pure node calls...
5462017-11-16T19:12:11  * jtimon hidfdes
5472017-11-16T19:12:22  <wumpus> promag: the tabs are supposed to be for essentially different things
5482017-11-16T19:12:33  <promag> At least you keep track of the correct log
5492017-11-16T19:12:36  <wumpus> e.g. more charts, more pages of debug info, etc
5502017-11-16T19:12:42  <jnewbery> promag: multiwallet comes first, dynamic loading later
5512017-11-16T19:13:04  <jnewbery> *multiwallet GUI comes first
5522017-11-16T19:13:17  <promag> Anyway, ack on the order
5532017-11-16T19:13:24  <luke-jr> promag: perhaps a log entry after you execute a command on a different wallet than previously (post-merge stuff)
5542017-11-16T19:13:42  <promag> Ok ok
5552017-11-16T19:13:47  <wumpus> why not a command to switch between wallets, btw?
5562017-11-16T19:14:10  <wumpus> the combobox is great to show what the current wallet is, but shouldn't the wallet be switchable with typing?
5572017-11-16T19:14:12  <luke-jr> /wallet <name> ?
5582017-11-16T19:14:16  <wumpus> for ex.
5592017-11-16T19:14:23  <jonasschnelli> yes... ideally it would be stateless.
5602017-11-16T19:14:30  <jonasschnelli> to ensure one is not executing on the wrong wallet
5612017-11-16T19:14:31  <achow101> so it would be a gui only command?
5622017-11-16T19:14:37  <jonasschnelli> wallet:xyz getnewaddress
5632017-11-16T19:14:38  <wumpus> achow101: yes
5642017-11-16T19:14:54  <jonasschnelli> if wallet:<filename> is missing, we get the standard rpcish reject
5652017-11-16T19:14:55  <wumpus> jonasschnelli: type the wallet name for every command? yes, maybe
5662017-11-16T19:14:57  <promag> It could be part of the "prompt"
5672017-11-16T19:15:08  <MarcoFalke> Needs autocomplete!
5682017-11-16T19:15:17  <jonasschnelli> I think the wallet-selected-state can be dangerous
5692017-11-16T19:15:18  <wumpus> jonasschnelli: that's absolutely safest
5702017-11-16T19:15:23  <wumpus> jonasschnelli: agree
5712017-11-16T19:15:26  <jonasschnelli> and it's RPC like
5722017-11-16T19:15:35  <wumpus> yes
5732017-11-16T19:15:39  <jonasschnelli> one can still use arrow-up edit
5742017-11-16T19:15:43  <jtimon> a gui only command doesn't feel right
5752017-11-16T19:16:07  <luke-jr> nesting is already GUI-only
5762017-11-16T19:16:21  <wumpus> jtimon: no, agree, jonasschnelli's proposal to make it stateless and have to provide it for every command is better, that's the same as needs tobe done for bitcoin-cli
5772017-11-16T19:16:22  <jonasschnelli> yes. It's fine
5782017-11-16T19:16:59  <luke-jr> is this still post-merge, or have we un-concept-ack'd the MW GUI PR?
5792017-11-16T19:17:01  <promag> Even when there is 1 wallet only?
5802017-11-16T19:17:13  <MarcoFalke> promag: No
5812017-11-16T19:17:17  <wumpus> promag: that's the exception
5822017-11-16T19:17:22  <jonasschnelli> luke-jr: both would be okay for me (post merge or now)
5832017-11-16T19:17:24  <wumpus> if it is unambigious then why not
5842017-11-16T19:17:36  <wumpus> wallet needs to be provided if multiple wallets are loaded
5852017-11-16T19:17:47  <promag> Ack
5862017-11-16T19:17:53  <luke-jr> wumpus: because it'd be really annoying to use?
5872017-11-16T19:17:54  <wumpus> if no wallet is loaded, there's no problem, if one wallet is loaded, then it's clear which one is meant
5882017-11-16T19:18:14  <wumpus> if mutliple are loaded then wallet commands are ambigious
5892017-11-16T19:18:19  <promag> It's the same with cli
5902017-11-16T19:18:26  <wumpus> yes, it's the same with bitcoin-cli
5912017-11-16T19:18:28  <jonasschnelli> It's maybe annoying... but it's the wallet. Safety first
5922017-11-16T19:18:36  <luke-jr> cli is just a testing tool though; it doesn't need to be convenient
5932017-11-16T19:18:36  <gmaxwell> why wouldn't the debug window just have a combo box
5942017-11-16T19:18:49  <luke-jr> gmaxwell: that's the current code
5952017-11-16T19:18:57  <jonasschnelli> gmaxwell: I think you will quickly choose the wrong wallet
5962017-11-16T19:19:11  <wumpus> gmaxwell: it's somewhat dangerous; easy to type a command with the wrong one selected
5972017-11-16T19:19:17  <jtimon> gmaxwell: some people are worried about a state, not sure what the problem is either
5982017-11-16T19:19:23  <gmaxwell> luke-jr: that is not true. cli is probably about as frequently used for using the software as the gui (this probably says some unfortunate things about the gui, but.. :P )
5992017-11-16T19:19:33  <luke-jr> could do both, I guess
6002017-11-16T19:19:37  <luke-jr> gmaxwell: I highly doubt that!
6012017-11-16T19:19:46  <achow101> luke-jr: I think there could be some weird interactions with doing both
6022017-11-16T19:19:55  <wumpus> gmaxwell: there is no clear visual link between what you type and the combobox, though it could be somehow improved by logging in big colorful letters when a different wallet is selected
6032017-11-16T19:19:58  <luke-jr> both = combobox with in-command override only when no-wallet selected
6042017-11-16T19:20:03  <wumpus> e.g. ============ current wallet: blabla.dat ===============
6052017-11-16T19:20:10  <gmaxwell> wumpus: thats a point, the prompt to could also show the wallet.
6062017-11-16T19:20:21  <wumpus> gmaxwell: yes, indeed
6072017-11-16T19:20:21  <gmaxwell> and there could be a line written in when it chages, like that.
6082017-11-16T19:20:31  <jtimon> how is it going to be with the cli again?
6092017-11-16T19:20:41  <luke-jr> jtimon: no changes needed there
6102017-11-16T19:20:57  <wumpus> jtimon: for the cli you have to provide the wallet name on every call to select the endpoint ,if it's ambigious, nothing will change there
6112017-11-16T19:21:11  <gmaxwell> jtimon: cli makes you specify it as a dashed argument to bitcoin-cli, which is a bit obnoxious but works.
6122017-11-16T19:21:12  <wumpus> decision is to be made about the console
6132017-11-16T19:21:16  <wumpus> but seems a combobox will do for now
6142017-11-16T19:21:21  <wumpus> so leave it like that for now luke-jr
6152017-11-16T19:21:24  <luke-jr> k
6162017-11-16T19:21:33  *** laurentmt has joined #bitcoin-core-dev
6172017-11-16T19:21:41  <promag> Next?
6182017-11-16T19:21:41  <jtimon> I see, thanks. just like you have to provide testnet or regtest every time but you don't need that in the GUI
6192017-11-16T19:21:49  <wumpus> jtimon: yep
6202017-11-16T19:21:58  <wumpus> GUI can keep state for you that the cli cannot
6212017-11-16T19:22:11  <wumpus> because it 'captures' the user, unlike a command that's launched every time
6222017-11-16T19:22:25  <wumpus> yes, other topics?
6232017-11-16T19:22:50  <achow101> topic suggestion: encrypted wallets by default
6242017-11-16T19:22:51  <promag> Flat options in rpc?
6252017-11-16T19:23:11  <wumpus> #topic encrypted wallets by default
6262017-11-16T19:23:18  <jtimon> I wanted to ask jl2012 about #11398
6272017-11-16T19:23:20  <gribble> https://github.com/bitcoin/bitcoin/issues/11398 | Hardcode CSV and SEGWIT deployment by jl2012 · Pull Request #11398 · bitcoin/bitcoin · GitHub
6282017-11-16T19:23:26  <wumpus> ... why??
6292017-11-16T19:23:38  <morcos> can someone open an issue about deciding wallet access from the console, i think shipping with it as i understand it to be now seems terrible, but i agree no reason to hold up progress on merging
6302017-11-16T19:23:43  <jonasschnelli> achow101: with an option to unencrypt later?
6312017-11-16T19:23:59  <achow101> jonasschnelli: I guess?
6322017-11-16T19:24:03  <sipa> wumpus: why what?
6332017-11-16T19:24:09  <wumpus> why encrypt the wallet by default?
6342017-11-16T19:24:14  <jonasschnelli> achow101: I think that would be great.
6352017-11-16T19:24:24  <gmaxwell> If you have users encrypt wallets when they open one without any value in it they will reliably lose the key.  The positive confirmation that the user is backed up like electrum has reduces that sort of risk.
6362017-11-16T19:24:31  <wumpus> it forces people to choose a passphrase which they'll probably forget
6372017-11-16T19:24:33  <achow101> a lot of wallet software do this now and I don't think people necessarily realize that their wallets are unencrypted until they go to the encrypt wallet option or rpc
6382017-11-16T19:24:40  <wumpus> I think most people lose money because of losing wallets or losing passphrases not theft
6392017-11-16T19:24:51  <wumpus> what thread model does encrypting wallets protect against anyhow?
6402017-11-16T19:24:52  <jonasschnelli> that true on the other hand
6412017-11-16T19:25:10  <jonasschnelli> Those who have access to support ticket systems of consumer wallets do know that
6422017-11-16T19:25:21  <luke-jr> wumpus: bad PR
6432017-11-16T19:25:29  <gmaxwell> Wallet encryption is mostly a tool for people to lose their money but feel better about it because its their own fault.    The great advantage of wallet encryption by default, as I'd see it, is resolving this mess of having to preserve unencrypted keys.
6442017-11-16T19:25:34  <morcos> couldn't we encrypt the wallet by default but not create the wallet by default
6452017-11-16T19:25:45  <morcos> so you solve the problem of them just clicking through the encryption aspect
6462017-11-16T19:25:50  <achow101> morcos: that was the idea I was thinking about
6472017-11-16T19:25:57  <gmaxwell> But for that advantage I would recommend a late initilization that doesn't create a wallet until you ask for an address... or go to encrypt it.
6482017-11-16T19:26:06  <achow101> you don't make the wallet until it is actually used, and only then do you prompt the user to make a wallet
6492017-11-16T19:26:11  <wumpus> I mean, the only use for encrypting wallets I see is: other people use your computer, and you're afraid of them copying the wallet but not installing a keylogger
6502017-11-16T19:26:20  <gmaxwell> +1 on the late initilization.
6512017-11-16T19:26:24  <wumpus> I don't think it protects against any other attacks
6522017-11-16T19:26:46  <morcos> wumpus: you dont think its useful for backups?
6532017-11-16T19:26:48  <gmaxwell> wumpus: well I really like encryption so that I know that I'm not accidentally going to send funds, but for that it's sufficient to make the key "yes" :P
6542017-11-16T19:26:59  <luke-jr> morcos: for backups you really want to encrypt the whole thing anyway
6552017-11-16T19:27:06  <gmaxwell> morcos: ^
6562017-11-16T19:27:07  <achow101> I have a branch for late initialiation: https://github.com/achow101/bitcoin/tree/start-no-wallet
6572017-11-16T19:27:09  <morcos> i suppose, maybe backups wasn't the right word
6582017-11-16T19:27:10  <achow101> it doesn't work right now
6592017-11-16T19:27:30  <morcos> maybe i meant having the wallet to check on things but not worrying too much about it
6602017-11-16T19:27:34  <wumpus> or maybe the case where e.g. malware in the browser sandbox can grab a fixed file from your computer, but there's no persistent access
6612017-11-16T19:27:39  <achow101> also encryption reduces the file size by like half because unencrypted keys are massive for some reason
6622017-11-16T19:28:08  <wumpus> another thing that will cause confusion is that for other wallets, the passphrase is the seed
6632017-11-16T19:28:15  <luke-jr> wumpus: even when it was introduced, it was acknowledged as mostly just a PR stunt
6642017-11-16T19:28:20  <wumpus> so people will think that only keeping the passphrase is enough to keep access to their funds
6652017-11-16T19:28:20  <jtimon> gmaxwell: I was actually scared to suggest a default key for "resolving this mess of having to preserve unencrypted keys"
6662017-11-16T19:28:31  <wumpus> there are already peple making that mistake now but it's rarer
6672017-11-16T19:28:37  <wumpus> (because you only have to choose it explicitly)
6682017-11-16T19:28:40  <luke-jr> achow101: huh? how?
6692017-11-16T19:28:44  *** pugsh has joined #bitcoin-core-dev
6702017-11-16T19:28:45  <gmaxwell> +1 for late init,  +1 for positive confirmation recovery backup (like electrum);  -1 for more pressure to encrypt unless the last step is done, +1 for it if the last step is done.
6712017-11-16T19:28:50  <morcos> also, this might sound stupid, but if you have a Core-encrypted wallet, you at least know the balance, so you know whether it's worth trying to figure out how to unencrypt it
6722017-11-16T19:29:01  <wumpus> so no, I think focing people to choose a passphrase when first creating their wallet is a bad idea
6732017-11-16T19:29:07  <achow101> luke-jr: encrypted keys are way smaller than unencrypted ones
6742017-11-16T19:29:14  <morcos> +1 gmaxwells +/-1's
6752017-11-16T19:29:20  <luke-jr> how is that even possible?
6762017-11-16T19:29:33  <promag> Sorry have to be afk
6772017-11-16T19:29:38  <gmaxwell> luke-jr: because the unencryted keys use some brain damaged openssl encoding
6782017-11-16T19:29:43  <gmaxwell> that encludes all the curve parameters.
6792017-11-16T19:29:44  <wumpus> that's just an implementation detail htough; unencrypted keys could be stored smaller, too
6802017-11-16T19:29:51  <achow101> luke-jr: the format. unencrypted keys are DER format or something. they have the curve params in them
6812017-11-16T19:29:55  <wumpus> we could encrypt the wallet by default, with an empty passphrase
6822017-11-16T19:30:02  <luke-jr> ew
6832017-11-16T19:30:04  <gmaxwell> right, thats a reason to change the format, not a reason to encrypt.
6842017-11-16T19:30:09  <sipa> achow101: they have field params, curve params, generator, public key and private key in them :)
6852017-11-16T19:30:17  <sipa> and all of that in inefficient DER
6862017-11-16T19:30:24  <sipa> 279 bytes total, iirc
6872017-11-16T19:30:45  <wumpus> yes it's terrible
6882017-11-16T19:31:26  <wumpus> and doesn't help with anything, if you're going to store the keys in redundant format at least pad it with something that provides error correction
6892017-11-16T19:31:54  <luke-jr> XD
6902017-11-16T19:31:54  <BlueMatt> I mean its error correction in case we forget our curve parameters...or something
6912017-11-16T19:32:03  <sipa> BlueMatt: we actually hardly look at it
6922017-11-16T19:32:05  <gmaxwell> wumpus: What are your thoughts on, long term:  delayed creation, at create time in the GUI force the user to write down a recovery code (like electrum does; force via reentry and copy/paste jamming).. and have a checkbox to encrypt there too?   recovery code would greatly offset all risks of loss, including lost the passphrase.
6932017-11-16T19:32:17  <luke-jr> at that size, just store 8 copies of it
6942017-11-16T19:32:48  <wumpus> gmaxwell: the recovery code would be the HD seed?
6952017-11-16T19:32:53  <gmaxwell> luke-jr: storing N copies of a key right next to each other hardly helps since disks tend to die a physical sector at a time.
6962017-11-16T19:32:56  <achow101> gmaxwell: recovery code as in something like bip39?
6972017-11-16T19:32:58  <gmaxwell> wumpus: yea, an encoding of the HD seed.
6982017-11-16T19:33:03  <wumpus> gmaxwell: that sounds great to me
6992017-11-16T19:33:06  <morcos> gmaxwell: encryption using recovery code?
7002017-11-16T19:33:17  <luke-jr> gmaxwell: sure, but in that case you're screwed with checksums too
7012017-11-16T19:33:20  <gmaxwell> achow101: not bip39 as it's a brainwallet scheme that can't encode arbritary data, but yes.
7022017-11-16T19:33:30  <morcos> i also like that idea, but i worry about the importing of private keys...  we'd have to put in a whole lot of warnings about that
7032017-11-16T19:33:36  <wumpus> achow101: more like other wallets lke electrum's seed phrase
7042017-11-16T19:33:49  <achow101> wumpus: yes, I would prefer using Electrum's scheme
7052017-11-16T19:33:53  <wumpus> achow101: (there's a BIP for it but I don't know the number)
7062017-11-16T19:33:54  <gmaxwell> morcos: I think we need to get to having an import tainted flag on wallets, and warnings about that.
7072017-11-16T19:33:54  <achow101> that's what we plan to do for Armory
7082017-11-16T19:34:08  <luke-jr> morcos: importing private keys is already considered dangerous and "never do this"
7092017-11-16T19:34:21  <wumpus> gmaxwell: I also greatly like the idea of not creating a wallet by default, so starting in no-wallet mode
7102017-11-16T19:34:22  <jtimon> so what's wrong with the "yes"/default/empty passphrase/key?
7112017-11-16T19:34:32  <jonasschnelli> the recovery phrase would be unencrypted?
7122017-11-16T19:34:40  <gmaxwell> achow101: ugg electrum itself. can't encode arbritary data, so it can't work with existing wallets. at least it's better than bip39.
7132017-11-16T19:34:55  <achow101> jonasschnelli: it would have to be to be able to recover from forgotten passwords
7142017-11-16T19:35:25  <achow101> gmaxwell: it can't? (I haven't really looked at it)
7152017-11-16T19:35:27  <jtimon> jonasschnelli: yes, would be public knowledge (and for the user it would be like if none was set) unless you actively set one
7162017-11-16T19:35:32  <jonasschnelli> achow101: I just worry about people storing those recovery phrases on phones and "plaintext "papers
7172017-11-16T19:35:34  <gmaxwell> jonasschnelli: I have mixed feelings about that.  I think a best practice is to have your recovery keys encrypted with a WEAK key,  like that insecure password your whole family knows; and there is no risk of it being forgotten... but which a burgler would likely be thwarted, but thats too complex to communicate.
7182017-11-16T19:36:12  <gmaxwell> jonasschnelli: but we should realize that risk of users losing a strong password is likely orders of magnitude more likely than a local in person attack.
7192017-11-16T19:36:27  <wumpus> it gets quite complex to manage if the recovery key is encrypted too
7202017-11-16T19:36:33  * BlueMatt notes that if we do some kind of default-encryption-with-weak-password we should have a clear tag on keys to help out the "shitsihitshit, please scan entire raw disk for anything that looks like a key" use-case
7212017-11-16T19:36:37  <jonasschnelli> gmaxwell: Indeed. Though people who can take care of a passphase should not be punished
7222017-11-16T19:36:40  <wumpus> there's the recovery key passphrase, the wallet passphrase,...
7232017-11-16T19:36:46  <gmaxwell> achow101: unless I'm confused (always likely) it's just a minor fixup of BIP39.
7242017-11-16T19:37:15  <luke-jr> BlueMatt: +1
7252017-11-16T19:37:16  <wumpus> BlueMatt: that's where the redundant key format is useful :)
7262017-11-16T19:37:32  <wumpus> BlueMatt: it greatly helps efficiently scanning for private keys on a disk :p
7272017-11-16T19:37:38  <BlueMatt> heh, I know
7282017-11-16T19:37:49  <gmaxwell> BlueMatt: yea, sure, anything key format should have e.g. somethin like the network magic then the private key then a 64 bit crc... and then its cheap to scan the media looking for it.
7292017-11-16T19:37:54  <wumpus> I don't think you can do a similar thing for the encrypted keys right now
7302017-11-16T19:38:00  <wumpus> not that they're any use without the master key
7312017-11-16T19:38:27  <BlueMatt> i mean ideally we'd have a clear tag on both so that such software can prompt the user with "found a wallet, please enter passphrase"
7322017-11-16T19:38:35  <BlueMatt> but now we're going down a rewrite-wallet-format rabbit hole
7332017-11-16T19:38:38  <gmaxwell> jonasschnelli: I don't know how to manage the multiple keys case. One possiblity would be to make the recovery key unencrypted by default, and have an advanced dialog that lets you set encryption for it. And support reading in encrypted ones.
7342017-11-16T19:38:54  <jonasschnelli> Yes. That would be great
7352017-11-16T19:39:04  <gmaxwell> jonasschnelli: I have a lovely suggestion for hardware wallet friendly KDFs for these things too.
7362017-11-16T19:39:06  <achow101> BlueMatt: I propose that we just deprecate the wallet :p
7372017-11-16T19:39:09  <morcos> May I make a meta suggestion.. I think we often lose progress on ideas like this by not having someone document what we discussed.  could we ask for volunteer every time we have a good discussion like this to draft up a plan.
7382017-11-16T19:39:16  <luke-jr> achow101: I get the feeling often
7392017-11-16T19:39:16  <jtimon> ack on starting in no-wallet mode
7402017-11-16T19:39:22  <jonasschnelli> gmaxwell: +1 (happy to hear)
7412017-11-16T19:39:33  <achow101> morcos: meeting notes writer
7422017-11-16T19:39:49  <achow101> morcos: he'll write the meeting notes sometime after exams this week
7432017-11-16T19:39:52  <wumpus> achow101: and then what, change it into an art project where you can look at blocks drifting by, without being able to do anything? :p
7442017-11-16T19:40:22  <luke-jr> wumpus: write a new one :p
7452017-11-16T19:40:36  <morcos> yeah but i mena more a focused thing... like after SF devcore -> plan for Segwit wallet  ;   this meeting -> plan for wallet encryption recovery code
7462017-11-16T19:40:36  <wumpus> luke-jr: you can do that without deprecating anything
7472017-11-16T19:40:51  <gmaxwell> the block drifting UI should play https://www.youtube.com/watch?v=8Z-fyNdnOKE in a loop.
7482017-11-16T19:41:00  <achow101> I'm scared to click that link
7492017-11-16T19:41:05  <gmaxwell> it's just music.
7502017-11-16T19:41:17  <gmaxwell> but we've trained you well.
7512017-11-16T19:41:50  *** promag has quit IRC
7522017-11-16T19:41:52  <sipa> morcos: i've just posted a bit of a writeup/rant on wallet design and segwit support: https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2
7532017-11-16T19:41:54  <wumpus> morcos: yes, we shouldn't forget segwit wallet
7542017-11-16T19:41:59  <morcos> woohoo!
7552017-11-16T19:42:02  <wumpus> morcos: that's the thing people are actually waiting for now :)
7562017-11-16T19:42:05  <sdaftuar> sipa: thanks!
7572017-11-16T19:42:08  <gmaxwell> FWIW, sipa has been working on a stronger base=32 BCH code for things like private keys and stealth addresses; which could be an option for recovery codes.
7582017-11-16T19:42:12  <wumpus> sipa: nice!
7592017-11-16T19:42:13  <luke-jr> clicking that link won't have permissions for my audio :p
7602017-11-16T19:42:22  <BlueMatt> when segwit wallet
7612017-11-16T19:42:27  *** d_t has joined #bitcoin-core-dev
7622017-11-16T19:42:30  <achow101> sipa: cool!
7632017-11-16T19:42:51  <achow101> BlueMatt: soon(tm)
7642017-11-16T19:43:20  <luke-jr> (mini rant: using #include <…> for our own files is stupid)
7652017-11-16T19:44:23  <wumpus> luke-jr: sigh, the other alternative would have been to fix all relative includes, but that was discussed in detail in the PR and the one before it
7662017-11-16T19:44:29  *** go1111111 has quit IRC
7672017-11-16T19:45:23  <wumpus> luke-jr: so using #include "../primitive/block.h" in e.g. the wallet. This roots everything at the project root, which is just as unambigious and shorter...
7682017-11-16T19:45:32  *** Dizzle has joined #bitcoin-core-dev
7692017-11-16T19:45:54  <luke-jr> I just hope /usr/include/primitive/block.h gets ignored
7702017-11-16T19:46:08  <wumpus> that doesn't get ignored either with ""
7712017-11-16T19:46:20  <luke-jr> :|
7722017-11-16T19:47:06  <gmaxwell> obviously we need to rename every header file to filename_bitcoin_core_is_awesome.h
7732017-11-16T19:47:08  <wumpus> at least not the way we were using it, which is essentially as <>, I think if you use "" relatively you can avoid it
7742017-11-16T19:48:03  <wumpus> yep!
7752017-11-16T19:48:03  <luke-jr> gmaxwell: I'm thinking more of malware infecting builds this way
7762017-11-16T19:48:19  <wumpus> well if your build root is infected you're fucked anyway
7772017-11-16T19:48:29  <jnewbery> luke-jr: in any case, the PRs were open for a few months (much longer than I would have liked in fact). There was opportunity to comment on those PRs. I think the ship has sailed now.
7782017-11-16T19:48:40  <luke-jr> true
7792017-11-16T19:48:57  <wumpus> protecting against that is even more questionable than encrypting your wallet, against any possible realistic threat model
7802017-11-16T19:49:45  <gmaxwell> luke-jr: well if your host is compromised it's pretty unlikely that it would be limited to only tripping you up with shadowed include files.
7812017-11-16T19:49:47  <wumpus> jnewbery: indeed, it's almost as if he waited for it to be merged
7822017-11-16T19:49:57  * BlueMatt nominates #11363 for cfields' high-priority-for-review, cause he apparently isnt here...
7832017-11-16T19:49:59  <gribble> https://github.com/bitcoin/bitcoin/issues/11363 | net: Split socket create/connect by theuni · Pull Request #11363 · bitcoin/bitcoin · GitHub
7842017-11-16T19:50:10  <luke-jr> wumpus: just didn't notice it until rebasing on top of the merged code
7852017-11-16T19:51:05  <luke-jr> anyhow, #11383 rebase is done
7862017-11-16T19:51:07  <gribble> https://github.com/bitcoin/bitcoin/issues/11383 | Basic Multiwallet GUI support by luke-jr · Pull Request #11383 · bitcoin/bitcoin · GitHub
7872017-11-16T19:51:16  <jonasschnelli> ^^
7882017-11-16T19:52:33  <jonasschnelli> thanks.. will test
7892017-11-16T19:52:34  <Dizzle> I like multiwallet. Thanks for working on it, luke-jr. I miss the classic multibit bulk walletting.
7902017-11-16T19:52:41  <wumpus> luke-jr: anyhow C/C++ including is fragile that way; possible modules https://clang.llvm.org/docs/Modules.html will improve that in the future
7912017-11-16T19:53:13  <sipa> yay c++20... which we'll switch to in 20125?
7922017-11-16T19:53:19  <sipa> *2025
7932017-11-16T19:53:24  <wumpus> yes, in 20125
7942017-11-16T19:53:31  <luke-jr> :x
7952017-11-16T19:53:33  <Chris_Stewart_5> ack
7962017-11-16T19:53:36  <meshcollider>  lol
7972017-11-16T19:53:39  <jonasschnelli> heh
7982017-11-16T19:53:53  <gmaxwell> change in topic, anyone have recent stats for the number of remaining btc1 nodes-- which are likely about to become a distributed DOS attack on the bitcoin network?
7992017-11-16T19:53:56  <wumpus> BlueMatt: will add that one
8002017-11-16T19:54:40  <wumpus> #topic DDoS network stats
8012017-11-16T19:54:59  <meshcollider> gmaxwell:  https://coin.dance/nodes says 139 but maybe not what you're after?
8022017-11-16T19:55:12  <luke-jr> (I'm going to drop as soon as the meeting is officially over. I'll be back a few minutes later in case there's stuff to talk about)
8032017-11-16T19:55:19  <jonasschnelli> I can filter my seed crawler for uagent string?
8042017-11-16T19:55:23  <gmaxwell> meshcollider: ha. I didn't expect them to shut off that fast, I guess they were really almost all just a couple people sybling.
8052017-11-16T19:55:35  <gmaxwell> meshcollider: okay, probably not much to worry about.
8062017-11-16T19:56:24  <meshcollider> gmaxwell yeah lol there was a Reddit post which went into some detail showing 90% were hosted by AWS
8072017-11-16T19:56:35  <wumpus> PSA before the meeting is over: I want to collect corrupted leveldb files, if you have a leveldb corruption please patch https://github.com/bitcoin/bitcoin/pull/11674 and send me the indicated corrupted file.
8082017-11-16T19:57:10  <jonasschnelli> 861 peers with "Bitcoin ABC" and 100% uptime during last two hours.
8092017-11-16T19:57:24  <luke-jr> jonasschnelli: that's just BCH
8102017-11-16T19:57:25  <achow101> gmaxwell: my btc1 node is connected to 34 other btc1 nodes, so at least 35
8112017-11-16T19:57:26  <meshcollider> ABC is not btc1
8122017-11-16T19:57:35  <BlueMatt> i mean its what we did 0.15.1 for, no?
8132017-11-16T19:58:10  <gmaxwell> BlueMatt: yes, sure I wanted to know how many there weer because if there were thousands I'd make a post on reddit to urge people to upgrade to 0.15+ seems it might not be needed.
8142017-11-16T19:58:11  <jonasschnelli> meshcollider: what uagent does btc1 uses?
8152017-11-16T19:58:25  <luke-jr> jonasschnelli: /Satoshi:1.*/
8162017-11-16T19:58:30  <jonasschnelli> ah
8172017-11-16T19:58:41  <achow101> jonasschnelli: most have a uacomment with "2x"
8182017-11-16T19:58:46  <jonasschnelli> 107
8192017-11-16T19:58:55  <gmaxwell> with only 140ish it's pretty unlikely many nodes will get isolated behind them.
8202017-11-16T19:59:25  <achow101> what block were they forking at?
8212017-11-16T19:59:30  <achow101> (I need to add it to my site)
8222017-11-16T19:59:34  <gmaxwell> 494784
8232017-11-16T19:59:55  <jtimon> weren't they using a naming just the same as bitcoin core but increasing a version? (ie 0.14.3)
8242017-11-16T20:00:03  <gmaxwell> hopefully someone will mine a couple blocks on that fork to help get those nodes disconnected.
8252017-11-16T20:00:21  <gmaxwell> jtimon: they made the major version 1.
8262017-11-16T20:00:26  <achow101> gmaxwell: a mining pool announced that they would go with the 2x fork regardless
8272017-11-16T20:00:27  <jtimon> oh, right
8282017-11-16T20:00:27  <jnewbery> won't they disconnect themselves once a valid block is found?
8292017-11-16T20:00:34  <wumpus> ding dong
8302017-11-16T20:00:40  <gmaxwell> achow101: that was 'bitpico' who is crazy.
8312017-11-16T20:00:53  <gmaxwell> it's meaningless.
8322017-11-16T20:00:59  <achow101> oh
8332017-11-16T20:01:11  <wumpus> #endmeeting
8342017-11-16T20:01:11  <lightningbot> Meeting ended Thu Nov 16 20:01:11 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
8352017-11-16T20:01:11  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-11-16-19.01.html
8362017-11-16T20:01:11  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-11-16-19.01.txt
8372017-11-16T20:01:11  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-11-16-19.01.log.html
8382017-11-16T20:01:52  *** Dizzle has quit IRC
8392017-11-16T20:25:01  *** luke-jr has quit IRC
8402017-11-16T20:26:16  *** luke-jr has joined #bitcoin-core-dev
8412017-11-16T20:27:12  *** jtimon has quit IRC
8422017-11-16T20:30:50  *** roadcrap has quit IRC
8432017-11-16T20:54:00  *** LumberCartel has quit IRC
8442017-11-16T21:01:03  *** laurentmt has quit IRC
8452017-11-16T21:01:08  *** BashCo_ has joined #bitcoin-core-dev
8462017-11-16T21:04:10  *** BashCo has quit IRC
8472017-11-16T21:05:33  *** Cheeseo has joined #bitcoin-core-dev
8482017-11-16T21:06:18  *** wunpunch has joined #bitcoin-core-dev
8492017-11-16T21:06:28  *** promag has joined #bitcoin-core-dev
8502017-11-16T21:11:06  *** promag has joined #bitcoin-core-dev
8512017-11-16T21:11:55  *** promag has quit IRC
8522017-11-16T21:12:15  *** sh4ft has quit IRC
8532017-11-16T21:21:10  *** promag has joined #bitcoin-core-dev
8542017-11-16T21:22:27  *** promag has quit IRC
8552017-11-16T21:31:24  *** torkel_ has joined #bitcoin-core-dev
8562017-11-16T21:38:58  *** torkel_ has quit IRC
8572017-11-16T21:39:43  *** PaulCapestany has quit IRC
8582017-11-16T21:39:48  *** torkel_ has joined #bitcoin-core-dev
8592017-11-16T21:43:22  *** AaronvanW has quit IRC
8602017-11-16T21:43:58  *** AaronvanW has joined #bitcoin-core-dev
8612017-11-16T21:44:46  <jonasschnelli> gmaxwell: I'm happy to hear your bip39 successor HWW KDF idea...
8622017-11-16T21:44:57  *** cryptapus has quit IRC
8632017-11-16T21:45:06  *** torkel_ is now known as torkelrogstad
8642017-11-16T21:45:16  <jonasschnelli> PBKDF2 with 2048 rounds seems not ideal (BIP39)
8652017-11-16T21:46:16  <sipa> jonasschnelli: the idea is a mechanism that allows you to enter the passphrase on a HW device, have the HW device outsource the hardening to a desktop computer (with more power) without revealing the passphrase
8662017-11-16T21:46:26  <sipa> and then being able to verify the computer did the hardening correctly
8672017-11-16T21:46:51  <jonasschnelli> +1
8682017-11-16T21:47:02  <sipa> adam back proposed a scheme for this a while ago, but it's purely CPU dependent
8692017-11-16T21:47:17  <sipa> whether it can be combined with memory hard hardening is an open question i think
8702017-11-16T21:47:33  <jonasschnelli> purely CPU is still much better then 2048-PBKDF
8712017-11-16T21:47:40  <sipa> haha, yes
8722017-11-16T21:49:17  <jonasschnelli> Somethine we (HWW company) do discuss regularly is how we can make the backup situation better.. a lot of things are involved. Bip39, sdcard, shamir's secret, notary services, etc.
8732017-11-16T21:49:37  <jonasschnelli> I'm not sure if a plain text seed dump (or BIP39) is something you want in a bank tresor
8742017-11-16T21:54:10  *** PaulCapestany has joined #bitcoin-core-dev
8752017-11-16T21:55:02  *** jsfour has quit IRC
8762017-11-16T22:10:04  *** LumberCartel has joined #bitcoin-core-dev
8772017-11-16T22:11:47  <goatpig> mdisc?
8782017-11-16T22:12:19  <jonasschnelli> mdisc?
8792017-11-16T22:12:31  <goatpig> it's basically a cdrom made out of rock
8802017-11-16T22:12:35  <goatpig> really really durable
8812017-11-16T22:12:42  *** Dizzle has joined #bitcoin-core-dev
8822017-11-16T22:14:38  *** ok7685 has joined #bitcoin-core-dev
8832017-11-16T22:16:33  *** LumberCartel has quit IRC
8842017-11-16T22:18:45  *** sunday-afternoon has quit IRC
8852017-11-16T22:21:07  <jcorgan> i'm not sure if the durability is really demonstrated, but i do have quite a few encrypted live boot images burned to them
8862017-11-16T22:21:29  *** harrymm has quit IRC
8872017-11-16T22:22:16  <goatpig> it's hard to demonstrate in practice
8882017-11-16T22:24:27  *** jsfour has joined #bitcoin-core-dev
8892017-11-16T22:25:58  *** harrymm has joined #bitcoin-core-dev
8902017-11-16T22:29:14  <jcorgan> it's a bit like closed-source software, the media is trade secret, but independent testing was pretty good
8912017-11-16T22:29:24  <jcorgan> http://www.esystor.com/images/China_Lake_Full_Report.pdf
8922017-11-16T22:30:11  <goatpig> at any rate it's far superior to plain cd/dvds or nvram
8932017-11-16T22:31:39  <jcorgan> certainly. they're great for "encrypted live boot cold-storage resurrection system" discs scattered in a few places
8942017-11-16T22:34:07  *** Chris_Stewart_5 has quit IRC
8952017-11-16T22:38:06  *** JackH has quit IRC
8962017-11-16T22:38:34  *** JackH has joined #bitcoin-core-dev
8972017-11-16T22:45:21  *** wxss has quit IRC
8982017-11-16T22:47:54  <cfields> whoops, totally forgot about today's meeting :\
8992017-11-16T22:56:54  *** wxss has joined #bitcoin-core-dev
9002017-11-16T22:58:33  *** PaulCapestany has quit IRC
9012017-11-16T22:58:38  *** PaulCape_ has joined #bitcoin-core-dev
9022017-11-16T22:58:46  *** cryptapus has joined #bitcoin-core-dev
9032017-11-16T22:58:47  *** cryptapus has joined #bitcoin-core-dev
9042017-11-16T23:02:56  *** meshcollider has quit IRC
9052017-11-16T23:10:30  *** torkelrogstad has quit IRC
9062017-11-16T23:10:47  *** torkelrogstad has joined #bitcoin-core-dev
9072017-11-16T23:11:39  *** tux3do has quit IRC
9082017-11-16T23:20:25  *** torkelrogstad has quit IRC
9092017-11-16T23:20:46  *** torkelrogstad has joined #bitcoin-core-dev
9102017-11-16T23:24:16  *** LumberCartel has joined #bitcoin-core-dev
9112017-11-16T23:32:33  *** jtimon has joined #bitcoin-core-dev
9122017-11-16T23:33:57  *** torkelrogstad has quit IRC
9132017-11-16T23:41:00  *** torkelrogstad has joined #bitcoin-core-dev
9142017-11-16T23:41:27  *** LumberCartel has quit IRC
9152017-11-16T23:42:09  *** Cogito_Ergo_Sum has quit IRC
9162017-11-16T23:50:49  *** goatpig has quit IRC
9172017-11-16T23:51:20  *** torkelrogstad has quit IRC
9182017-11-16T23:51:34  *** torkelrogstad has joined #bitcoin-core-dev
9192017-11-16T23:51:50  *** meshcollider has joined #bitcoin-core-dev
9202017-11-16T23:56:33  *** jsfour has quit IRC