12017-02-23T00:14:09  <jtimon> can we please just get rid of accounts for 0.15 ?
  22017-02-23T00:14:54  <luke-jr> ryanofsky: accounts don't have coins
  32017-02-23T00:15:10  <luke-jr> jtimon: IIRC there's still some things we lack substitutes for? dunno
  42017-02-23T00:15:46  <jtimon> not sure what the progress on that was, but I thought we were close
  52017-02-23T00:41:33  <jtimon> sorry for repeating the question, why are all these functions declared as extern? https://github.com/bitcoin/bitcoin/blob/master/src/rpc/server.h#L188
  62017-02-23T00:52:51  *** abpa has quit IRC
  72017-02-23T01:01:23  *** jannes has quit IRC
  82017-02-23T01:07:17  *** Giszmo has quit IRC
  92017-02-23T01:34:01  *** vicenteH has quit IRC
 102017-02-23T01:42:25  *** goksinen has quit IRC
 112017-02-23T01:48:49  *** goksinen has joined #bitcoin-core-dev
 122017-02-23T01:55:52  *** goksinen has quit IRC
 132017-02-23T01:56:17  *** fanquake has joined #bitcoin-core-dev
 142017-02-23T02:05:52  *** AaronvanW has quit IRC
 152017-02-23T02:14:04  *** jtimon has quit IRC
 162017-02-23T02:21:04  *** goksinen has joined #bitcoin-core-dev
 172017-02-23T02:24:01  *** Ylbam has quit IRC
 182017-02-23T02:25:47  *** MarcoFalke has quit IRC
 192017-02-23T02:25:48  *** goksinen has quit IRC
 202017-02-23T02:26:35  *** jtimon has joined #bitcoin-core-dev
 212017-02-23T02:43:22  *** goksinen has joined #bitcoin-core-dev
 222017-02-23T02:46:37  <jtimon> luke-jr: ok, sorry, it was -disablewallet, not wallet=0, right now, with that option, and with either -help or invalid args it says: error code: -32601 error message: Method not found, it could instead
 232017-02-23T02:49:28  *** jtimon has quit IRC
 242017-02-23T02:53:01  *** goksinen has quit IRC
 252017-02-23T03:07:13  *** justan0theruser has joined #bitcoin-core-dev
 262017-02-23T03:08:50  *** goksinen has joined #bitcoin-core-dev
 272017-02-23T03:09:40  *** justanotheruser has quit IRC
 282017-02-23T03:13:27  *** goksinen has quit IRC
 292017-02-23T03:19:30  *** goksinen has joined #bitcoin-core-dev
 302017-02-23T03:23:57  *** goksinen has quit IRC
 312017-02-23T03:36:22  *** goksinen has joined #bitcoin-core-dev
 322017-02-23T03:43:53  *** goksinen has joined #bitcoin-core-dev
 332017-02-23T03:51:32  *** goksinen_ has joined #bitcoin-core-dev
 342017-02-23T04:00:57  *** goksinen_ has quit IRC
 352017-02-23T04:14:22  *** goksinen has quit IRC
 362017-02-23T04:42:22  *** goksinen has joined #bitcoin-core-dev
 372017-02-23T04:46:57  *** goksinen has quit IRC
 382017-02-23T04:51:31  *** chris2000 has joined #bitcoin-core-dev
 392017-02-23T04:53:44  *** davec has quit IRC
 402017-02-23T04:54:31  *** chris200_ has quit IRC
 412017-02-23T04:55:23  *** davec has joined #bitcoin-core-dev
 422017-02-23T04:56:15  *** fanquake has quit IRC
 432017-02-23T05:00:09  *** dermoth has quit IRC
 442017-02-23T05:00:56  *** dermoth has joined #bitcoin-core-dev
 452017-02-23T05:05:42  *** chjj_ is now known as chjj
 462017-02-23T05:28:09  *** goksinen has joined #bitcoin-core-dev
 472017-02-23T05:32:41  *** goksinen has quit IRC
 482017-02-23T05:37:05  <luke-jr> https://github.com/iancoleman/bip39/issues/58
 492017-02-23T05:37:22  <luke-jr> not sure if it's a real problem or not yet, but some crypto experts may want to look (sipa, gmaxwell)
 502017-02-23T05:39:13  *** rafalcpp has quit IRC
 512017-02-23T05:52:54  *** fanquake has joined #bitcoin-core-dev
 522017-02-23T06:01:56  <bitcoin-git> [bitcoin] NicolasDorier opened pull request #9830: Add trusted flag to listunspent result (master...listunspenttrusted) https://github.com/bitcoin/bitcoin/pull/9830
 532017-02-23T06:10:24  *** goksinen has joined #bitcoin-core-dev
 542017-02-23T06:15:27  *** goksinen has quit IRC
 552017-02-23T06:18:35  *** goksinen has joined #bitcoin-core-dev
 562017-02-23T06:25:20  *** afk11 has quit IRC
 572017-02-23T06:26:01  *** afk11 has joined #bitcoin-core-dev
 582017-02-23T06:31:57  *** goksinen has quit IRC
 592017-02-23T06:33:44  <bitcoin-git> [bitcoin] theuni opened pull request #9831: build: force a c++ standard to be specified (master...no-default-std) https://github.com/bitcoin/bitcoin/pull/9831
 602017-02-23T06:42:39  *** ryankung has joined #bitcoin-core-dev
 612017-02-23T06:46:38  <midnightmagic> That feeler-connections PR is a very nice read.
 622017-02-23T06:50:26  <gmaxwell> luke-jr: crazy js code treating binary data as strings. :(
 632017-02-23T06:52:50  <luke-jr> >_<
 642017-02-23T06:57:37  <gmaxwell> should be recoverable at least... better than prior bitcoinjs / bn.js bugs.
 652017-02-23T06:59:51  *** goksinen has joined #bitcoin-core-dev
 662017-02-23T07:00:27  *** cannon-c has joined #bitcoin-core-dev
 672017-02-23T07:04:27  *** goksinen has quit IRC
 682017-02-23T07:05:53  <cannon-c> Does bitcoin core have bip39 for backing up wallet? Having trouble seeming to find it.
 692017-02-23T07:06:07  <cannon-c> If not, any objection to including bip39?
 702017-02-23T07:10:56  <gmaxwell> It is a terrible spec which shouldn't be implemented by anyone, in my opinion.
 712017-02-23T07:12:12  <gmaxwell> It forces the use of public derrivation which results in security issues when used with software that allows public key export. It has no versioning, so wallets cannot change the schemes without creating a potential for funds loss.  The 'check values' for it are optional and can't really be enforced for general compatiblity, which then also means that most uses of it end up unintentioally promot
 722017-02-23T07:12:18  <gmaxwell> ing insecure brainwallet usage.
 732017-02-23T07:12:44  <gmaxwell> The authors of the sepcification were uninterested in resolving these and other shortcomings, resulting in one of the authors having their name pulled from it.
 742017-02-23T07:13:22  <luke-jr> I thought BIP 39 didn't dictate anything re derivation?
 752017-02-23T07:14:05  <cannon-c> what is "public derrivation"?
 762017-02-23T07:14:25  * luke-jr observes people are actually using https://github.com/bitcoin/bips/wiki/Comments:BIP-0039
 772017-02-23T07:15:01  <cannon-c> I thought bip39 was just using dictionary wordlist in 2048 length list to convert 256 bit entropy to human readable words (256bit being 24 words)
 782017-02-23T07:15:54  <gmaxwell> indeed you're right it's BIP 44 that does but it doesn't matter because the lack of versioning means that you're stuck with the defacto standard.
 792017-02-23T07:16:51  <cannon-c> I do like idea of words for backup, I can write down backup on paper using words instead of key that is prone to misspelling and funds lost.
 802017-02-23T07:17:03  <cannon-c> gmaxwell what do you prefer above word based mnemonic then?
 812017-02-23T07:18:55  <gmaxwell> words are fine, but should be a scheme that can encode versioning and bidirectionally convert.
 822017-02-23T07:19:23  *** lclc has joined #bitcoin-core-dev
 832017-02-23T07:20:09  * luke-jr wonders if Electrum's scheme is saner, considering they had similar criticism of BIP 39
 842017-02-23T07:20:34  *** goksinen has joined #bitcoin-core-dev
 852017-02-23T07:20:57  <cannon-c> If I udnerstand correctly, Electrum's scheme even if dictionary is lost can still recover direct from words themselves?
 862017-02-23T07:21:19  <cannon-c> Just something I heard, not familiar with Electrum's scheme though.
 872017-02-23T07:23:24  <bitcoin-git> [bitcoin] NicolasDorier opened pull request #9832: [qa] assert_start_raises_init_error (master...assert_start_raises_init_error) https://github.com/bitcoin/bitcoin/pull/9832
 882017-02-23T07:24:57  *** goksinen has quit IRC
 892017-02-23T07:34:47  *** city22 has joined #bitcoin-core-dev
 902017-02-23T07:37:49  <wumpus> what's up with travis?  https://github.com/bitcoin/bitcoin/issues/9825   How can there be a threading error *before* the tests of test_bitcoin even start?
 912017-02-23T07:38:30  <wumpus> errors such as "test_bitcoin: tpp.c:62: __pthread_tpp_change_priority: Assertion `new_prio == -1 || (new_prio >= __sched_fifo_min_prio && new_prio <= __sched_fifo_max_prio)' failed."
 922017-02-23T07:38:59  <wumpus> and "../include/boost/thread/pthread/recursive_mutex.hpp:113: void boost::recursive_mutex::lock(): Assertion `!pthread_mutex_lock(&m)' failed."
 932017-02-23T07:40:12  <wumpus> sounds like locking is being done before locking is initialized, e.g. before we even get to main()?
 942017-02-23T07:40:52  <bitcoin-git> [bitcoin] benma opened pull request #9833: Trivial: fix comments referencing AppInit2 (master...stalecomments) https://github.com/bitcoin/bitcoin/pull/9833
 952017-02-23T07:45:33  *** BashCo has quit IRC
 962017-02-23T07:50:42  <jonasschnelli> wumpus: you think this is a travis issue or did we break the context initialisation in some way?
 972017-02-23T07:51:06  <wumpus> I don't think it's a travis issue, but I can't (which is always terribly frustrating with these issues) reproduce it locally
 982017-02-23T07:51:26  <wumpus> but yes it's probably something we do differently in our code now
 992017-02-23T07:51:38  *** udiWertheimer has quit IRC
1002017-02-23T07:52:33  *** city22 has quit IRC
1012017-02-23T07:52:34  *** goksinen has joined #bitcoin-core-dev
1022017-02-23T07:52:34  *** chris2000 has quit IRC
1032017-02-23T07:52:36  <wumpus> current master simply passes https://travis-ci.org/bitcoin/bitcoin/builds/204167316 - so it's intermittent
1042017-02-23T07:53:19  <jonasschnelli> wumpus: we recently changes something close to that,,,
1052017-02-23T07:53:27  <jonasschnelli> the missing bitcoin-tx ECC_Start()
1062017-02-23T07:53:33  <jonasschnelli> Not sure if this is related
1072017-02-23T07:53:46  <wumpus> unknown location(0): fatal error: in "rpc_tests/rpc_rawsign": signal: illegal operand; address of failing instruction: 0x2b73cbf4cf3b
1082017-02-23T07:53:49  <wumpus> WAT??
1092017-02-23T07:54:35  <jonasschnelli> :/
1102017-02-23T07:55:08  <midnightmagic> I wonder if it would be worthwhile to include a "legal participation" flag of some sort.. so that people who use the services provided by your bitcoin node must agree to the EULA of your node, in order to use it.
1112017-02-23T07:55:55  <wumpus> jonasschnelli: yes, might be related, I don't know. Although I think the test setup happens after the "Running 228 test cases...", not before it
1122017-02-23T07:56:02  <wumpus> not sure tho
1132017-02-23T07:56:30  <wumpus> midnightmagic: you could include a url in your subversion
1142017-02-23T07:56:57  *** goksinen has quit IRC
1152017-02-23T07:57:12  <wumpus> (though it's a bit challenging with all those characters sanitized out :-)
1162017-02-23T07:57:18  <midnightmagic> Then, technically, mass-sybil'ing could be prosecuted if the sybils didn't disconnect immediately and refuse to connect again.
1172017-02-23T07:58:00  <wumpus> depending on your country you can set any terms of use when you run a service
1182017-02-23T08:00:35  *** norotartagen has quit IRC
1192017-02-23T08:00:56  <midnightmagic> auto-EULA might not be strong enough.
1202017-02-23T08:01:50  <wumpus> so you imagine something where you have to read an EULA and click for every node you connect to?
1212017-02-23T08:02:05  <wumpus> uhm, no, I'll pass
1222017-02-23T08:02:12  *** norotartagen has joined #bitcoin-core-dev
1232017-02-23T08:02:50  <gmaxwell> midnightmagic: proposed previously was an alternative protocol for transaction relay that has standard requirements like that as just part of the protocol spec.
1242017-02-23T08:03:03  <gmaxwell> midnightmagic: anyone is free to create such a thing.
1252017-02-23T08:03:20  *** BashCo has joined #bitcoin-core-dev
1262017-02-23T08:03:22  <midnightmagic> well, triangulation of an idea is nice to hear anyway.
1272017-02-23T08:03:31  <gmaxwell> Because so much of the abusive behavior we can detect is commercially motivated-- its an intresting question.
1282017-02-23T08:03:56  <midnightmagic> commercial motivation implies pockets, and corporation behavioural laws.
1292017-02-23T08:04:02  <wumpus> yes, separating transaction submission/relay would make sense
1302017-02-23T08:04:08  <gmaxwell> But it's not something that would make sense as the general Bitcoin protocol I think. Or at least it would be too easily trolled. But people are free to try it out and I hope they do.
1312017-02-23T08:04:11  <wumpus> that's the only thing people care to spy on anyway
1322017-02-23T08:04:26  <wumpus> for relaying blocks, it doesn't matter nearly as much
1332017-02-23T08:05:27  <midnightmagic> I was thinking just connecting at all. I wonder if it would devolve into horros e.g. to authorize your client to agree to EULA semi-automatically on your behalf based on acceptable terms.
1342017-02-23T08:06:12  <gmaxwell> midnightmagic: I think it only makes sense if the terms are basically just part of the protocol.
1352017-02-23T08:06:59  <midnightmagic> hrm.
1362017-02-23T08:07:39  <wumpus> making it harder to spy on transactions by using onion routing / some kind of deaddrop system, would do a lot against spying
1372017-02-23T08:07:54  <wumpus> better than getting involved inthe vagaries of international law
1382017-02-23T08:13:17  *** goksinen has joined #bitcoin-core-dev
1392017-02-23T08:17:57  *** goksinen has quit IRC
1402017-02-23T08:18:23  <wumpus> what are the exact specifications of the travis VMs?
1412017-02-23T08:19:33  <wumpus> ubuntu trusty at least, will try with that locally
1422017-02-23T08:25:17  <bitcoin-git> [bitcoin] benma opened pull request #9834: qt: clean up initialize/shutdown signals (master...exitcode) https://github.com/bitcoin/bitcoin/pull/9834
1432017-02-23T08:44:09  *** goksinen has joined #bitcoin-core-dev
1442017-02-23T08:46:17  <wumpus> of course, it all passes perfectly in my own trusty vm...
1452017-02-23T08:47:21  <gmaxwell> make your vm slower? :P
1462017-02-23T08:48:27  *** goksinen has quit IRC
1472017-02-23T08:48:53  <wumpus> I guess running a few instances of test_bitcoin in parallel could simulate that
1482017-02-23T08:54:12  <wumpus> looks like test_bitcoin doesn't clean up its UUID directories (/tmp/92bc-0a4a-f496-6b40)
1492017-02-23T08:54:35  <wumpus> while the /tmp/test_bitcoin_1487840020_6012 are being cleaned up
1502017-02-23T08:55:55  <wumpus> (yes, the tests finish succesfully)
1512017-02-23T09:02:00  *** chjj has quit IRC
1522017-02-23T09:11:46  *** Ylbam has joined #bitcoin-core-dev
1532017-02-23T09:15:23  *** goksinen has joined #bitcoin-core-dev
1542017-02-23T09:19:57  *** goksinen has quit IRC
1552017-02-23T09:35:35  *** ryankung has quit IRC
1562017-02-23T09:39:12  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/bed5b30a5622...d14555de3de9
1572017-02-23T09:39:13  <bitcoin-git> bitcoin/master 874c736 Russell Yanofsky: Fix pruning test broken by 2 hour manual prune window...
1582017-02-23T09:39:13  <bitcoin-git> bitcoin/master d14555d Wladimir J. van der Laan: Merge #9820: Fix pruning test broken by 2 hour manual prune window...
1592017-02-23T09:39:32  <bitcoin-git> [bitcoin] laanwj closed pull request #9820: Fix pruning test broken by 2 hour manual prune window (master...pr/prunewind) https://github.com/bitcoin/bitcoin/pull/9820
1602017-02-23T09:39:52  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/599c69abe3101512eeee13733c0f58eb3e363eae
1612017-02-23T09:39:53  <bitcoin-git> bitcoin/0.14 599c69a Russell Yanofsky: Fix pruning test broken by 2 hour manual prune window...
1622017-02-23T09:41:06  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d14555de3de9...1d2a57e9fd2c
1632017-02-23T09:41:06  <bitcoin-git> bitcoin/master fa4cd2e MarcoFalke: qa: Check return code when stopping nodes...
1642017-02-23T09:41:07  <bitcoin-git> bitcoin/master 1d2a57e Wladimir J. van der Laan: Merge #9824: qa: Check return code when stopping nodes...
1652017-02-23T09:41:22  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/260c71cbb857d12540a2f53372282d11f2b401a8
1662017-02-23T09:41:23  <bitcoin-git> bitcoin/0.14 260c71c MarcoFalke: qa: Check return code when stopping nodes...
1672017-02-23T09:41:31  <bitcoin-git> [bitcoin] laanwj closed pull request #9824: qa: Check return code when stopping nodes (master...Mf1702-qaRet) https://github.com/bitcoin/bitcoin/pull/9824
1682017-02-23T09:47:18  *** goksinen has joined #bitcoin-core-dev
1692017-02-23T09:49:09  <bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/1d2a57e9fd2c...e68c266f3d56
1702017-02-23T09:49:10  <bitcoin-git> bitcoin/master b602fe0 Cory Fields: build: warn about variable length arrays
1712017-02-23T09:49:10  <bitcoin-git> bitcoin/master 205830a Cory Fields: build: add --enable-werror option...
1722017-02-23T09:49:11  <bitcoin-git> bitcoin/master e68c266 Wladimir J. van der Laan: Merge #9789: build: add --enable-werror and warn on vla's...
1732017-02-23T09:49:29  <bitcoin-git> [bitcoin] laanwj closed pull request #9789: build: add --enable-werror and warn on vla's (master...no-vla) https://github.com/bitcoin/bitcoin/pull/9789
1742017-02-23T09:49:44  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to 0.14: https://github.com/bitcoin/bitcoin/compare/260c71cbb857...05e906dbc68e
1752017-02-23T09:49:44  <bitcoin-git> bitcoin/0.14 749fe95 Cory Fields: build: warn about variable length arrays...
1762017-02-23T09:49:45  <bitcoin-git> bitcoin/0.14 05e906d Cory Fields: build: add --enable-werror option...
1772017-02-23T09:51:57  *** goksinen has quit IRC
1782017-02-23T09:55:22  *** Alina-malina has quit IRC
1792017-02-23T09:56:53  *** rafalcpp has joined #bitcoin-core-dev
1802017-02-23T10:03:51  *** vicenteH has joined #bitcoin-core-dev
1812017-02-23T10:08:21  *** vicenteH has quit IRC
1822017-02-23T10:10:44  *** vicenteH has joined #bitcoin-core-dev
1832017-02-23T10:18:23  *** goksinen has joined #bitcoin-core-dev
1842017-02-23T10:20:24  *** chjj has joined #bitcoin-core-dev
1852017-02-23T10:22:57  *** goksinen has quit IRC
1862017-02-23T11:01:14  *** BashCo has quit IRC
1872017-02-23T11:08:28  *** lclc has quit IRC
1882017-02-23T11:09:21  *** BashCo has joined #bitcoin-core-dev
1892017-02-23T11:15:59  *** MarcoFalke has joined #bitcoin-core-dev
1902017-02-23T11:22:37  <wumpus> I've been running test_bitcoin with five threads, for three hours on a trusty VM - no single failure
1912017-02-23T11:46:48  *** lclc has joined #bitcoin-core-dev
1922017-02-23T11:48:32  *** chjj has quit IRC
1932017-02-23T12:16:43  *** AaronvanW has joined #bitcoin-core-dev
1942017-02-23T12:19:05  *** goksinen has joined #bitcoin-core-dev
1952017-02-23T12:23:27  *** goksinen has quit IRC
1962017-02-23T12:40:37  <warren> Hmm, the blk*.dat files exported by linearize-data.py is slightly modified by bitcoind if you drop them in your blocks/ directory.  http://pastebin.com/PH99QYdm   diff of two hexdumps shows a tiny bit at the end of each file is truncated.
1972017-02-23T12:49:54  *** goksinen has joined #bitcoin-core-dev
1982017-02-23T12:54:27  *** goksinen has quit IRC
1992017-02-23T13:21:50  *** goksinen has joined #bitcoin-core-dev
2002017-02-23T13:26:27  *** goksinen has quit IRC
2012017-02-23T13:34:52  *** BashCo has quit IRC
2022017-02-23T13:35:32  *** BashCo has joined #bitcoin-core-dev
2032017-02-23T13:38:50  *** goksinen has joined #bitcoin-core-dev
2042017-02-23T13:38:57  <wumpus> warren: hm that;s a bug
2052017-02-23T13:39:20  <wumpus> only the last blk.dat file should be modified
2062017-02-23T13:39:45  <wumpus> oh, of course, it scans them again from the beginning, so every file is 'last file' for a while
2072017-02-23T13:40:20  <wumpus> so the truncating behavior makes some sense - don't think it necessarily needs to do that during reindexing, but it's not unexpected
2082017-02-23T13:41:24  <wumpus> this is different from the bootstrap.dat file which was really an external file which was imported
2092017-02-23T13:49:27  *** goksinen has quit IRC
2102017-02-23T14:05:53  *** goksinen has joined #bitcoin-core-dev
2112017-02-23T14:06:42  *** goksinen has quit IRC
2122017-02-23T14:07:13  *** goksinen has joined #bitcoin-core-dev
2132017-02-23T14:08:40  *** goksinen_ has joined #bitcoin-core-dev
2142017-02-23T14:11:22  *** cannon-c has quit IRC
2152017-02-23T14:12:57  *** goksinen_ has quit IRC
2162017-02-23T14:14:12  *** nickler has quit IRC
2172017-02-23T14:20:14  *** goksinen_ has joined #bitcoin-core-dev
2182017-02-23T14:22:25  *** goksinen has quit IRC
2192017-02-23T14:26:30  *** nickler has joined #bitcoin-core-dev
2202017-02-23T14:29:22  *** goksinen has joined #bitcoin-core-dev
2212017-02-23T14:33:20  *** jnewbery_ has joined #bitcoin-core-dev
2222017-02-23T14:35:57  *** goksinen has quit IRC
2232017-02-23T14:43:01  *** Guyver2 has joined #bitcoin-core-dev
2242017-02-23T14:50:19  *** PaulCapestany has joined #bitcoin-core-dev
2252017-02-23T15:03:03  *** goksinen has joined #bitcoin-core-dev
2262017-02-23T15:07:27  *** goksinen has quit IRC
2272017-02-23T15:08:39  *** city22 has joined #bitcoin-core-dev
2282017-02-23T15:09:15  *** city22 has quit IRC
2292017-02-23T15:16:43  *** goksinen has joined #bitcoin-core-dev
2302017-02-23T15:21:27  *** goksinen has quit IRC
2312017-02-23T15:23:12  *** Chris_Stewart_5 has quit IRC
2322017-02-23T15:27:05  *** goksinen has joined #bitcoin-core-dev
2332017-02-23T15:29:54  *** Chris_Stewart_5 has joined #bitcoin-core-dev
2342017-02-23T15:35:06  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e68c266f3d56...7146d96de3e1
2352017-02-23T15:35:06  <bitcoin-git> bitcoin/master c578408 John Newbery: Add exclude option to rpc-tests.py
2362017-02-23T15:35:07  <bitcoin-git> bitcoin/master 7146d96 MarcoFalke: Merge #9766: Add --exclude option to rpc-tests.py...
2372017-02-23T15:35:31  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9766: Add --exclude option to rpc-tests.py (master...rpctestexclude) https://github.com/bitcoin/bitcoin/pull/9766
2382017-02-23T15:38:03  *** Giszmo has joined #bitcoin-core-dev
2392017-02-23T15:40:18  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/7146d96de3e1...d6064a89ac97
2402017-02-23T15:40:19  <bitcoin-git> bitcoin/master 3f95a80 John Newbery: Fix docstrings in qa tests...
2412017-02-23T15:40:19  <bitcoin-git> bitcoin/master d6064a8 MarcoFalke: Merge #9577: Fix docstrings in qa tests...
2422017-02-23T15:40:33  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9577: Fix docstrings in qa tests (master...docstrings) https://github.com/bitcoin/bitcoin/pull/9577
2432017-02-23T15:40:47  <MarcoFalke> Great work on the test framework jnewbery!
2442017-02-23T15:41:12  <jnewbery> Thanks MarcoFalke!
2452017-02-23T15:46:11  *** Giszmo has quit IRC
2462017-02-23T15:46:54  *** Giszmo has joined #bitcoin-core-dev
2472017-02-23T15:55:31  *** laurentmt has joined #bitcoin-core-dev
2482017-02-23T16:02:29  *** laurentmt has quit IRC
2492017-02-23T16:15:02  *** BashCo has quit IRC
2502017-02-23T16:15:37  *** BashCo has joined #bitcoin-core-dev
2512017-02-23T16:20:12  *** BashCo has quit IRC
2522017-02-23T16:34:18  *** BashCo has joined #bitcoin-core-dev
2532017-02-23T16:37:00  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d6064a89ac97...a13a417cdcfd
2542017-02-23T16:37:00  <bitcoin-git> bitcoin/master 3333ad0 MarcoFalke: qa: Set correct path for binaries in rpc tests
2552017-02-23T16:37:01  <bitcoin-git> bitcoin/master a13a417 MarcoFalke: Merge #9823: qa: Set correct path for binaries in rpc tests...
2562017-02-23T16:37:20  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9823: qa: Set correct path for binaries in rpc tests (master...Mf1702-qaPath) https://github.com/bitcoin/bitcoin/pull/9823
2572017-02-23T16:37:35  *** kyletorpey has joined #bitcoin-core-dev
2582017-02-23T16:38:04  *** kyletorpey has left #bitcoin-core-dev
2592017-02-23T16:47:33  *** chjj has joined #bitcoin-core-dev
2602017-02-23T16:50:23  *** abpa has joined #bitcoin-core-dev
2612017-02-23T16:57:21  <bitcoin-git> [bitcoin] DannyHamilton opened pull request #9838: Update COPYING (master...patch-1) https://github.com/bitcoin/bitcoin/pull/9838
2622017-02-23T16:58:46  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9838: Update COPYING (master...patch-1) https://github.com/bitcoin/bitcoin/pull/9838
2632017-02-23T17:01:29  <gmaxwell> https://tradeblock.com/bitcoin/tx/8d31992805518fd62daa3bdd2a5c4fd2cd3054c9b3dca1d78055e9528cff6adc > sha1 bounty redeemed.
2642017-02-23T17:03:44  <jouke_> :)
2652017-02-23T17:07:41  <BlueMatt> gmaxwell: is that a second sha1 collision, then?
2662017-02-23T17:10:11  <BlueMatt> gmaxwell: indeed, a second sha1 collision
2672017-02-23T17:11:04  <gmaxwell> smartbits is much nicer in its display: https://www.smartbit.com.au/tx/8d31992805518fd62daa3bdd2a5c4fd2cd3054c9b3dca1d78055e9528cff6adc
2682017-02-23T17:12:07  <BlueMatt> gmaxwell: oh, nvm, same sha1 collision
2692017-02-23T17:12:37  <BlueMatt> same prefix, just dropped the postfix
2702017-02-23T17:20:32  <Chris_Stewart_5> all of the inputs on that tx correspond to outputs that had SHA1 bounties?
2712017-02-23T17:23:26  <gmaxwell> Yes.
2722017-02-23T17:32:18  *** niska has quit IRC
2732017-02-23T17:39:56  *** atroxes has quit IRC
2742017-02-23T17:41:13  *** atroxes has joined #bitcoin-core-dev
2752017-02-23T17:55:58  *** niska has joined #bitcoin-core-dev
2762017-02-23T17:57:58  *** goksinen has quit IRC
2772017-02-23T17:59:58  *** goksinen has joined #bitcoin-core-dev
2782017-02-23T18:03:50  <warren> wumpus: I'd like behavior where during -reindex it can handle any existing blk*.dat files to be read-only, do not truncate during scan, and create new after it scanned whatever already exists there. That way I can hardlink shared files between multiple full archival node instances on the same machine.
2792017-02-23T18:04:02  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/a13a417cdcfd...692c9eddba67
2802017-02-23T18:04:03  <bitcoin-git> bitcoin/master 9829c54 Cory Fields: build: force a c++ standard to be specified...
2812017-02-23T18:04:06  <bitcoin-git> bitcoin/master 692c9ed Wladimir J. van der Laan: Merge #9831: build: force a c++ standard to be specified...
2822017-02-23T18:04:22  <bitcoin-git> [bitcoin] laanwj closed pull request #9831: build: force a c++ standard to be specified (master...no-default-std) https://github.com/bitcoin/bitcoin/pull/9831
2832017-02-23T18:04:44  <wumpus> warren: heh yes, hardlinking block files works as long as you don't reindex, apparently
2842017-02-23T18:05:07  <wumpus> warren: I tend to hardlink both the block/utxo database files and the block files, so have never noticed that
2852017-02-23T18:05:52  <wumpus> warren: though: truncating the files to not contain any zero space at the end shouldn't affect any other users
2862017-02-23T18:05:57  <warren> which are the "block/utxo database files"
2872017-02-23T18:06:27  <wumpus> the ldb files: https://gist.github.com/laanwj/3c4614a23e072cbb3d39090da1834a68
2882017-02-23T18:07:31  <warren> wumpus: currently during *any* startup it fails if it cannot open a blk*.dat file writable, which is unnecessary.  If they can be read it should be adequate as long as new blk.dat files can be written after the read-only files.
2892017-02-23T18:07:47  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/99fd85cb44fe9983c64cfa376299e67b18568304
2902017-02-23T18:07:47  <bitcoin-git> bitcoin/0.14 99fd85c Cory Fields: build: force a c++ standard to be specified...
2912017-02-23T18:07:48  <wumpus> I agree, but that's not the case right now
2922017-02-23T18:07:59  <gmaxwell> wumpus: sounds like a trivial change to make it not open them writable.
2932017-02-23T18:08:03  <gmaxwell> er warren
2942017-02-23T18:08:39  <sipa> we could mark files in the block index as unwritable as well
2952017-02-23T18:09:05  <sipa> and then skip them when looking for files to append new blocks to
2962017-02-23T18:09:17  <wumpus> I remember luke-jr has an issue open about this https://github.com/bitcoin/bitcoin/issues/2039
2972017-02-23T18:09:24  <warren> I know.  currently it goes through a generic abstraction to open files
2982017-02-23T18:09:54  *** lclc has quit IRC
2992017-02-23T18:10:12  <wumpus> sipa: the problem in this case is not appending, but truncating. But yeah that'd help too
3002017-02-23T18:10:19  *** rndguy has joined #bitcoin-core-dev
3012017-02-23T18:11:02  <wumpus> I like how pruning at least works with hardlinked block files, by pruning a block file at a time instead of messing with the contents
3022017-02-23T18:11:21  <warren> fopen failing on read-only and truncation are two related problems, following by marking which files should be not writable.  Anything else?
3032017-02-23T18:11:44  *** MarcoFalke has quit IRC
3042017-02-23T18:12:21  <warren> where should that read-only marking be written?
3052017-02-23T18:12:37  <wumpus> in the block index db
3062017-02-23T18:19:25  <bitcoin-git> [bitcoin] ryanofsky opened pull request #9839: [qa] Make import-rescan.py watchonly check reliable (master...pr/travis-rescan) https://github.com/bitcoin/bitcoin/pull/9839
3072017-02-23T18:29:41  *** mturquette has quit IRC
3082017-02-23T18:31:24  *** mturquette has joined #bitcoin-core-dev
3092017-02-23T18:32:31  *** Sosumi has joined #bitcoin-core-dev
3102017-02-23T18:43:41  *** jnewbery_ has quit IRC
3112017-02-23T18:43:41  *** jnewbery has quit IRC
3122017-02-23T18:49:04  *** laurentmt has joined #bitcoin-core-dev
3132017-02-23T18:49:05  <wumpus> almost time to tag rc2?
3142017-02-23T18:49:20  <wumpus> well I guess it can wait until the meeting :)
3152017-02-23T18:49:24  <bitcoin-git> [bitcoin] ryanofsky opened pull request #9840: Update sendfrom RPC help to correct coin selection misconception (master...pr/fromacct) https://github.com/bitcoin/bitcoin/pull/9840
3162017-02-23T18:49:41  <BlueMatt> heh
3172017-02-23T18:49:44  <BlueMatt> wumpus: i think so
3182017-02-23T18:52:10  <BlueMatt> cfields: where are we on #9787
3192017-02-23T18:52:17  <gribble> https://github.com/bitcoin/bitcoin/issues/9787 | release: add a few performance-related notes by theuni · Pull Request #9787 · bitcoin/bitcoin · GitHub
3202017-02-23T18:52:31  <BlueMatt> oh, wait, missed you updated part of it
3212017-02-23T18:52:35  <sdaftuar> #9814 looks potentially concerning?  unclear if its reproducible i guess
3222017-02-23T18:52:45  <gribble> https://github.com/bitcoin/bitcoin/issues/9814 | 0.14rc1 coredump in QScreen::handle() · Issue #9814 · bitcoin/bitcoin · GitHub
3232017-02-23T18:53:02  <BlueMatt> oh, did we not fix that yet? ugh, yea, probably need a fix for that before rc2
3242017-02-23T18:53:19  <wumpus> sdaftuar: I don't know. haven't heard any reports about that from anyone else
3252017-02-23T18:53:56  <wumpus> also no one seems to be able to reproduce it
3262017-02-23T18:54:02  <wumpus> so no, I don't think it should block rc2
3272017-02-23T18:54:13  <cfields> BlueMatt: yea, i need some clarification from you. Also, I assume there are some more obvious user-facing perf improvements that should be added, i just can't come up with any
3282017-02-23T18:54:25  <BlueMatt> wumpus: hmmm....I mean I'd tend to trust dooglus to not have some obviously-broken shit
3292017-02-23T18:54:31  <cfields> BlueMatt: we could do a quick call for additions in the meeting
3302017-02-23T18:54:44  <BlueMatt> wumpus: but, ok
3312017-02-23T18:54:54  <BlueMatt> wumpus: (I dont see much of a different option)
3322017-02-23T18:55:08  <wumpus> BlueMatt: also it seems like a qt issue
3332017-02-23T18:55:14  <BlueMatt> yes
3342017-02-23T18:55:38  <wumpus> and seems it's a custom built binary, not our static one from bitcoin.org
3352017-02-23T18:56:10  <BlueMatt> well i hope we support custom binaries with non-static qt :p
3362017-02-23T18:56:28  <BlueMatt> cfields: i think just the spelling issue sdaftuar mentioned and we can get it in for rc2?
3372017-02-23T18:56:30  <wumpus> but we can't support all broken configurations out there
3382017-02-23T18:56:43  <cfields> BlueMatt: sure, if you're good with the rest
3392017-02-23T18:56:45  <wumpus> well at least I can't. Maybe you want to take that up, fine with me
3402017-02-23T18:56:46  <wumpus> :)
3412017-02-23T18:57:04  *** lclc has joined #bitcoin-core-dev
3422017-02-23T18:57:21  <BlueMatt> wumpus: heh, indeed, if no one can figure out the issue then it shouldnt block rc2, I just want to make sure someone who actually knows qt has given it a super good look-over, because i certainliy cant help there :/
3432017-02-23T18:57:25  <cfields> wumpus: did you see my comment the other day about that smelling like an old touch/wacom bug? Not sure if you dug that up, but I'll have a look now if not
3442017-02-23T18:58:00  <wumpus> cfields: I vaguely remember that one, yes.
3452017-02-23T18:58:06  <gmaxwell> if it's an issue in an older QT we could disable building with versions that old.
3462017-02-23T18:58:08  <wumpus> cfields: it needed a qt patch
3472017-02-23T18:58:37  <cfields> wumpus: right. My thought was that because the crash comes from older system libs, that bug may be present
3482017-02-23T18:58:45  <cfields> looking it up to see if it could be the same thing
3492017-02-23T18:59:11  <wumpus> looks like even the filer cannot reproduce it
3502017-02-23T18:59:26  <wumpus> "Edit: I've tried repeating the same steps again and wasn't able to get the crash to happen again."
3512017-02-23T19:00:31  <wumpus> #startmeeting
3522017-02-23T19:00:31  <lightningbot> Meeting started Thu Feb 23 19:00:31 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
3532017-02-23T19:00:31  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
3542017-02-23T19:00:32  <wumpus> #meetingstart
3552017-02-23T19:01:08  <gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
3562017-02-23T19:01:13  <jonasschnelli> hi
3572017-02-23T19:01:25  <wumpus> cfields: we could ask if dooglus is using a tablet/touch screen, or was using it at that time.
3582017-02-23T19:01:43  <cfields> wumpus: oh, haha, i didn't notice it was the same reporter
3592017-02-23T19:01:53  <kanzure> hi.
3602017-02-23T19:02:14  <sipa> present
3612017-02-23T19:02:15  <petertodd> hi
3622017-02-23T19:02:26  <jonasschnelli> wumpus: did you reproduce 9814 with the same setup? Ubuntu and Qt 5.3?
3632017-02-23T19:02:28  <cfields> oh, nm
3642017-02-23T19:02:46  <petertodd> so quick note re: git and SHA1 collisions: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-February/013600.html
3652017-02-23T19:02:54  <wumpus> jonasschnelli: no, I did not reproduce it
3662017-02-23T19:02:58  <sipa> petertodd: i wanted to bring that up
3672017-02-23T19:03:04  <wumpus> #topic git and sha collisions
3682017-02-23T19:03:29  <petertodd> so while many people will say git isn't vulnerable if you trust a git repo's maintainers, that is *not* true as a pull-req could add a colliding file
3692017-02-23T19:03:38  <luke-jr> sounds like MD5 breakage
3702017-02-23T19:03:38  <achow101> does git and github only support sha1?
3712017-02-23T19:03:42  <sipa> achow101: yes
3722017-02-23T19:03:50  <sipa> i only read about this new sha1 attack an hour ago... how hard is it to create a collision?
3732017-02-23T19:04:01  <petertodd> probably only relevant with binary files, but we don't know 100% yet because details of the attack haven't been published
3742017-02-23T19:04:03  *** MarcoFalke has joined #bitcoin-core-dev
3752017-02-23T19:04:03  <luke-jr> achow101: IIRC, git is designed in such a way that changing SHA1 is very difficult
3762017-02-23T19:04:04  <achow101> sipa: still very hard
3772017-02-23T19:04:20  <sipa> i see
3782017-02-23T19:04:20  <gmaxwell> sipa: it's not a new attack its the one published from a coule years ago with about 2^64 complexity.  There were some estimates of a cost on EC2 of about $110k.
3792017-02-23T19:04:26  <wumpus> 110 GPU-years or so
3802017-02-23T19:04:31  <gmaxwell> It's just actually been executed now.
3812017-02-23T19:04:48  <sipa> gmaxwell: no, it seems new research was done that simplifies it further
3822017-02-23T19:04:50  <gmaxwell> (at least thats my understanding)
3832017-02-23T19:04:53  <petertodd> gmaxwell: is it confirmed that google's efforts don't include any advances on what was already known to be possible?
3842017-02-23T19:04:55  <gmaxwell> oh. I missed that, okay!
3852017-02-23T19:05:26  <luke-jr> is the git project taking any action now?
3862017-02-23T19:05:36  <BlueMatt> luke-jr: doesnt look like it
3872017-02-23T19:05:37  <wumpus> Google and CWI, the dutch center for mathetmatics and informatics
3882017-02-23T19:05:44  <BlueMatt> luke-jr: (at least no movement as of this morning)
3892017-02-23T19:05:52  <gmaxwell> I think they already said they wouldn't before?
3902017-02-23T19:06:09  <sipa> i wonder how hard it is to create an overlay... that goes back in history, computes a sha256 for every tree and commit, and then include gpg signatures on those?
3912017-02-23T19:06:23  <jonasschnelli> sipa: great idea
3922017-02-23T19:06:38  <petertodd> sipa: I have code for something very similar to that in OpenTimestamps actually, and people have written tools to do that as well other than opentimestamps
3932017-02-23T19:06:40  <btcdrak> The exploit has a fancy name as usual http://shattered.io/
3942017-02-23T19:06:41  *** jtimon has joined #bitcoin-core-dev
3952017-02-23T19:06:44  <gmaxwell> http://marc.info/?l=git&m=115678778717621&w=2 "The only _dangerous_ kind of collision is the inadvertent kind,
3962017-02-23T19:06:47  <gmaxwell> but that's obviously also the very very unlikely kind."
3972017-02-23T19:06:52  <BlueMatt> sipa: I was looking this morning to see if you could reasonably patch git to do so directly, looks hard.....still, we can update our dev scripts to do a sha256sum of all committed files and sign that as well
3982017-02-23T19:07:47  <sipa> BlueMatt: signing just the trees i guess is an easy first step
3992017-02-23T19:07:47  <btcdrak> according to the site "How is GIT affected?
4002017-02-23T19:07:47  <btcdrak> GIT strongly relies on SHA-1 for the identification and integrity checking of all file objects and commits. It is essentially possible to create two GIT repositories with the same head commit hash and different contents, say a benign source code and a backdoored one. An attacker could potentially selectively serve either repository to targeted users. This
4012017-02-23T19:07:47  <btcdrak> will require attackers to compute their own collision."
4022017-02-23T19:07:47  <wumpus> but the tree is a sha1 hash too, isn't it?
4032017-02-23T19:07:47  <BlueMatt> gmaxwell: yup, great, linus and associated kernel folks continue to willfully ignore that security matters even the slightest bit
4042017-02-23T19:07:47  <gmaxwell> what does the signed commit stuff sign? if it signs the commit message, then we can include a sha256 in it and have the verify check that too.
4052017-02-23T19:07:47  <BlueMatt> wumpus: yes, it would have to be a separate thing
4062017-02-23T19:07:47  <wumpus> if you don't trust SHA1 anymore, that rabbit hole goes deep
4072017-02-23T19:07:47  * luke-jr votes banning binary files in any case :p
4082017-02-23T19:07:53  <sdaftuar> is it worth periodically running https://github.com/cr-marcstevens/sha1collisiondetection on commits in our repo?
4092017-02-23T19:07:54  <BlueMatt> wumpus: eg the merge scripts could include a hash of sha256sum * in the commit message
4102017-02-23T19:08:07  <petertodd> gmaxwell: the signed commit stuff signs a SHA1 hash, but it's easy to extend that with a gnupg wrapper; I could take the code from OpenTimestamps and do that
4112017-02-23T19:08:08  <BlueMatt> sdaftuar: I have a patch for git to use that as the hash function
4122017-02-23T19:08:16  <BlueMatt> and abort() if it detects a potentially-bad hash
4132017-02-23T19:08:22  <sipa> BlueMatt: sounds like a great idea
4142017-02-23T19:08:39  <kanzure> off-topic, but i wonder what git could change to make upgrading backwards-compatible in the future. for now i think it must be an incompatible upgrade to switch to another hash function?
4152017-02-23T19:08:39  <sipa> (including the sha256sum * in the merge commit message)
4162017-02-23T19:08:42  <wumpus> BlueMatt: if bad hashes are rare, that makes sense
4172017-02-23T19:08:47  <petertodd> gmaxwell: the one thing OpenTimestamps doesn't already do is recalculate hashes of *prior* commits with SHA256, but that'd be easy to add
4182017-02-23T19:09:12  <kanzure> i would also like the gh-meta repository maintainer to consider timestamping with opentimestamps at some point, i dunno who he is and i haven't pestered him yet
4192017-02-23T19:09:25  <BlueMatt> wumpus: the authors of that hash code claim 2**-90
4202017-02-23T19:09:32  <kanzure> er, the bitcoin-gh-meta repository person
4212017-02-23T19:09:32  <BlueMatt> (for random hits)
4222017-02-23T19:09:32  <luke-jr> detecting it after the fact seems like it would still be irrepairable?
4232017-02-23T19:10:03  <wumpus> kanzure: iwilcox on IRC
4242017-02-23T19:10:10  <petertodd> kanzure: git could easily have git commits simultaneously generate and sign two different hash functions; quite feasible to implement. I'll bet you I could pull that off in a week or two of work.
4252017-02-23T19:10:12  <kanzure> oh good, he is easily pesterable.
4262017-02-23T19:10:46  <kanzure> petertodd: yea but needs to be backwards compatible; and the bloat from two commits does not sound ideal. are they considering that btw?
4272017-02-23T19:10:53  <wumpus> BlueMatt: if the chance of an inadvertent match is that low, in that case, abort() /reject is a great solution
4282017-02-23T19:11:25  <luke-jr> kanzure: I'd guess you simply list the parent commit hashes twice?
4292017-02-23T19:11:27  <petertodd> kanzure: no bloat involved - the data can be shared across both commits
4302017-02-23T19:12:03  <kanzure> ah okay. well, i hadn't heard that proposal today, someone should check if git upstream is thinking about that.
4312017-02-23T19:12:09  <gmaxwell> probably said enough on this for now.
4322017-02-23T19:12:15  <btcdrak> There's a conversation on the git mailing list https://public-inbox.org/git/xmqqk28g92h7.fsf@gitster.mtv.corp.google.com/T/#t
4332017-02-23T19:12:33  <petertodd> gmaxwell: ack
4342017-02-23T19:12:42  <wumpus> yes, agreed
4352017-02-23T19:12:58  <petertodd> next topic
4362017-02-23T19:13:06  <wumpus> #topic help cfields with adding performance-related release notes
4372017-02-23T19:13:08  <wumpus> https://github.com/bitcoin/bitcoin/pull/9787
4382017-02-23T19:13:13  <cfields> quick call for 0.14 perf improvements on 9897
4392017-02-23T19:13:18  <cfields> heh, thanks :)
4402017-02-23T19:13:52  <cfields> err... #9787
4412017-02-23T19:13:54  <gribble> https://github.com/bitcoin/bitcoin/issues/9787 | release: add a few performance-related notes by theuni · Pull Request #9787 · bitcoin/bitcoin · GitHub
4422017-02-23T19:13:57  <jtimon> kanzure: yeah I assume changing the hash function would be a hardfork for old repositories :p
4432017-02-23T19:14:16  <cfields> if anyone added a big user-facing performance improvement for 0.14, please speak up
4442017-02-23T19:14:46  <gmaxwell> jtimon: please don't use bitcoin terms for other things especially non-consensus systems. The classic words "backwards incompatible" are fine. :P
4452017-02-23T19:15:03  <jtimon> gmaxwell: yep, sorry, was a bad joke
4462017-02-23T19:15:29  <wumpus> jtimon: dependso n how close the new hash function is to SHA-1. If it gets the same output 1-2**90 of the time, most repositories won't be affected
4472017-02-23T19:15:30  <gmaxwell> cfields: no measurements in the notes?
4482017-02-23T19:16:37  <cfields> gmaxwell: see the pr description. I've taken some measurements on the net stuff, but they're highly localized.
4492017-02-23T19:17:20  <gmaxwell> wish I realized no one was going to benchmark I could have started one last night. :P
4502017-02-23T19:17:40  <jeremyrubin> I would also note that the cuckoocache means nodes wanting to use more cores but had performance degrade should try more cores again
4512017-02-23T19:18:13  <sipa> right, cuckoocache has no effect at low parallellism, i think?
4522017-02-23T19:18:50  <morcos> sipa: no it's smarter about keepign the right signatures in the cache due to the generations and lazy eviction
4532017-02-23T19:18:54  <cfields> gmaxwell: i've done lots of benchmarking. But as I said, I'm not sure how to translate that into helpful figures
4542017-02-23T19:19:21  <gmaxwell> sipa: e.g. it will make reorgs much faster!
4552017-02-23T19:19:35  <cfields> well the alternative is to use a reference system/environment for each improvement
4562017-02-23T19:19:46  <gmaxwell> cfields: users don't care about each improvement.
4572017-02-23T19:20:15  <wumpus> it also doesn't need to be super precise
4582017-02-23T19:20:25  <gmaxwell> cfields: "Sync with least release takes N hours now, sync with new release takes Y now." would be nice.
4592017-02-23T19:20:53  <jonasschnelli> or syncs are rougle XYZ% faster...
4602017-02-23T19:20:54  *** lclc_ has joined #bitcoin-core-dev
4612017-02-23T19:21:13  <jonasschnelli> use the ~ and nobody will blame you afterwards. :)
4622017-02-23T19:21:30  <jeremyrubin> use two ~~ to be extra aproximate
4632017-02-23T19:21:40  *** lclc has quit IRC
4642017-02-23T19:21:43  <wumpus> it's marketing not science :p hehe
4652017-02-23T19:21:48  <gmaxwell> but ~~ will just give you the same number you put in!
4662017-02-23T19:22:44  <jeremyrubin> The is-approximately operator is non-involutive ;)
4672017-02-23T19:22:51  <gmaxwell> Well people just have no general idea of the impact.  Marketing would be saying that it's 2x faster rathern the 3x faster because 2x is more plausable. :P
4682017-02-23T19:22:51  <jeremyrubin> anyways
4692017-02-23T19:23:21  <cfields> ok, i'll add a vague 0.13 vs 0.14 sync time. The 0.13 will take time though, I haven't had the patience to finish one yet
4702017-02-23T19:23:28  <jeremyrubin> Could be cool to spin up a few different EC2 instances to compare...
4712017-02-23T19:23:39  <wumpus> EC is not a good comparison environment
4722017-02-23T19:23:45  <wumpus> sloooow i/o
4732017-02-23T19:23:56  <sipa> i'm syncing on a RPi3
4742017-02-23T19:24:02  <sipa> sloooow
4752017-02-23T19:24:09  <wumpus> what storage?
4762017-02-23T19:24:13  <luke-jr> lol
4772017-02-23T19:24:27  <sipa> microsd card
4782017-02-23T19:24:27  <jonasschnelli> uh
4792017-02-23T19:24:27  <cfields> i used EC for my sync benching because i figured it represented a very typical use-case
4802017-02-23T19:24:33  <wumpus> that's the cause of the slowness, likely
4812017-02-23T19:24:34  <jeremyrubin> Actually I like that. We should publish the worst system that can sync :p
4822017-02-23T19:24:47  <sipa> wumpus: absolutely
4832017-02-23T19:24:50  * luke-jr ponders trying on his USB Armory again
4842017-02-23T19:25:01  <jtimon> other topics?
4852017-02-23T19:25:07  <wumpus> sipa: if it's really CPU bound, don't forget to use the experimental ARM assembly secp256k1 :)
4862017-02-23T19:25:23  <wumpus> sipa: right, as I expected
4872017-02-23T19:25:31  <cfields> For reference, 0.14 sync took roughly 3 hours on ec2 w/ 4 cpu cores
4882017-02-23T19:25:42  <sipa> wumpus: i enabled it, but it's not nearly CPU bound
4892017-02-23T19:25:51  <achow101> topic: rc2 status?
4902017-02-23T19:26:10  <wumpus> #topic rc2 status
4912017-02-23T19:26:24  <wumpus> I think it should be ready to tag
4922017-02-23T19:26:57  <wumpus> well, need to update the translations and release notes to include changes since rc1, but I don't think there's anything that still needs to be solved
4932017-02-23T19:27:04  <achow101> there are still open prs in the milestone though
4942017-02-23T19:27:22  <achow101> (well 1 that actually does something)
4952017-02-23T19:27:30  <wumpus> achow101: one is release notes - that can go in any time before final
4962017-02-23T19:27:58  <wumpus> #9829 would be nice to get in, but it's breaking travis
4972017-02-23T19:28:00  <gribble> https://github.com/bitcoin/bitcoin/issues/9829 | Fix importmulti returning rescan errors for wrong keys by ryanofsky · Pull Request #9829 · bitcoin/bitcoin · GitHub
4982017-02-23T19:28:49  <jtimon> ryanofsky: ping
4992017-02-23T19:29:20  <ryanofsky> should i do something? bump travis?
5002017-02-23T19:29:23  <wumpus> (I've already tried to retrigger it so it's not one of today's intermittent problems)
5012017-02-23T19:29:56  <wumpus> ryanofsky: I don't think that helps
5022017-02-23T19:30:16  <gmaxwell> Does it pass locally?
5032017-02-23T19:30:39  <ryanofsky> yeah, the errors are the same "__pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed" things i've seen on other prs
5042017-02-23T19:31:07  <wumpus> ryanofsky: oh, https://github.com/bitcoin/bitcoin/issues/9825
5052017-02-23T19:31:17  <ryanofsky> i had to bump one of my prs last week 5-6 times to make travis pass
5062017-02-23T19:31:22  <wumpus> ryanofsky: okay, will respin
5072017-02-23T19:32:34  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/847e3753a6d6a6ab4dd081e2945cb200faf730d4
5082017-02-23T19:32:34  <bitcoin-git> bitcoin/0.14 847e375 Wladimir J. van der Laan: qt: pre-rc2 translations update
5092017-02-23T19:33:00  <morcos> Can we figure out when these travis errors started?  Do we get them on personal travis instances?  Can we bisect?  Seems likely somethign changed on our end right?
5102017-02-23T19:33:12  <wumpus> #topic travis issues
5112017-02-23T19:33:16  <gmaxwell> do we have any idea whats causing that? I assume no one has hit it locally?  I left test bitcoin running in a loop when I first saw it on the off chance it was an ASLR mediated thing that was only hitting travis sometimes.
5122017-02-23T19:33:32  <gmaxwell> (no results, of course)
5132017-02-23T19:33:37  <jonasschnelli> We recently added a missing ecc_start to bitcoin-tx... but I can't see any relation
5142017-02-23T19:33:50  <gmaxwell> jonasschnelli: I don't even see why you mention that?
5152017-02-23T19:33:51  <wumpus> I tried to reproduce #9825 for hours on a trusty vm, with five threads for a few hours... but no dice
5162017-02-23T19:33:52  <gribble> https://github.com/bitcoin/bitcoin/issues/9825 | Intermittent FAIL: test/test_bitcoin in Travis · Issue #9825 · bitcoin/bitcoin · GitHub
5172017-02-23T19:34:01  <wumpus> to bitcoin-tx? yea, that won't make a difference
5182017-02-23T19:34:26  <jonasschnelli> gmaxwell: becuase of that "test_bitcoin: key.cpp:300: void ECC_Start(): Assertion `secp256k1_context_sign == __null' failed."?
5192017-02-23T19:34:31  <gmaxwell> It looks like test_bitcoin fails before it even really starts. So global constructors or somehting in the C libraries.
5202017-02-23T19:34:43  <wumpus> jonasschnelli: that's only a effect of the failure
5212017-02-23T19:34:44  <jeremyrubin> I noticed that one of the builds seems to have some additional bounds checks enabled -- might be nice to enable those across travis tests? Hopefully no code relies on bounds checks/not
5222017-02-23T19:34:49  <wumpus> jonasschnelli: *everything* after the initial failure fails
5232017-02-23T19:34:55  <wumpus> jonasschnelli: secp256k1 is innocent
5242017-02-23T19:35:00  <jonasschnelli> ah.. I see.
5252017-02-23T19:35:23  <jtimon> perhaps we're forgetting some EEC_stop() somewhere?
5262017-02-23T19:35:32  <gmaxwell> no. geesh
5272017-02-23T19:35:47  <wumpus> no, I'm fairly sure it doesn't have to do with secp256k1
5282017-02-23T19:35:57  <wumpus> it's something done with mutexes before mutexes are initialized
5292017-02-23T19:36:20  <jtimon> ok, I really have no idea what's happening
5302017-02-23T19:36:24  <wumpus> looks like some kind of race condition, but I have no idea where or what
5312017-02-23T19:36:47  <wumpus> if it happens it *always* happens in test_bitcoin, never in bitcoind, bitcoin-cli, bitcoin-tx
5322017-02-23T19:37:05  <gmaxwell> I wonder if we could make our travis build script rerun any failing case under gdb so it would print a backtrace?
5332017-02-23T19:37:27  <BlueMatt> gmaxwell: probably using coredumps?
5342017-02-23T19:37:40  <wumpus> if you rerun it it will probably pass
5352017-02-23T19:37:42  <jtimon> perhaps some of the globals initialized in test_bitcoin.cpp ?
5362017-02-23T19:37:43  <gmaxwell> or that.
5372017-02-23T19:37:48  <jeremyrubin> in theory boost test runner can start gdb
5382017-02-23T19:37:53  <jtimon> just thinking out loud again...
5392017-02-23T19:37:54  <jonasschnelli> Could it be related to the boost test framework?
5402017-02-23T19:37:55  <BlueMatt> eg if it crashes coredump and gdb print all backtraces
5412017-02-23T19:38:03  *** jnewbery has joined #bitcoin-core-dev
5422017-02-23T19:38:10  <wumpus> but yes, getting a core file would be useful
5432017-02-23T19:38:14  <jeremyrubin> boost test runner can start gdb. I wonder if it reads .gdbinit
5442017-02-23T19:38:15  <gmaxwell> BlueMatt: thats probably the right way to go there.
5452017-02-23T19:38:25  <wumpus> although you need the executable too, to debug it
5462017-02-23T19:38:28  <gmaxwell> jeremyrubin: yes, but if it crashes its probably not in a good state to do so. :P
5472017-02-23T19:38:32  <wumpus> and uploading from travis :(
5482017-02-23T19:38:40  <BlueMatt> wumpus: i mean we can at least print stack traces
5492017-02-23T19:38:47  <gmaxwell> wumpus: just detect the crash in the script and run gdb.
5502017-02-23T19:38:53  <gmaxwell> and print the backtraces.
5512017-02-23T19:39:05  <wumpus> ok, yes, backtrace would help
5522017-02-23T19:40:01  *** lclc_ has quit IRC
5532017-02-23T19:40:02  <gmaxwell> do we know when the first of these errors was?
5542017-02-23T19:40:07  <wumpus> jtimon: yes, it sounds ilke a global initialisation order fiasco
5552017-02-23T19:40:28  <jeremyrubin> could be nice to detect the failing case, and then re-run it under say, valgrind
5562017-02-23T19:40:28  *** lclc has joined #bitcoin-core-dev
5572017-02-23T19:40:47  <wumpus> no, I don't know. I started getting annoyed by them today, but it's been going on yesterday at least as well
5582017-02-23T19:41:39  <wumpus> bisecting this with travis would take *a lot* of patience
5592017-02-23T19:41:53  <gmaxwell> that kind of suggests that it's a change on travis' end, no? we haven't had any changes on 0.14 in the past two days that could have caused this?
5602017-02-23T19:42:06  <wumpus> I don't think so either.
5612017-02-23T19:42:16  <wumpus> the big locking changes are longer ago
5622017-02-23T19:42:27  <wumpus> and it didn't start after that
5632017-02-23T19:42:50  <jeremyrubin> topic suggest -- code reorganizing (renaming rpc tests, etc)
5642017-02-23T19:43:05  <gmaxwell> does something in the travis log report what hardware it ran on?
5652017-02-23T19:43:06  <wumpus> still, the question is whether it is a change at travis's end that exposes something in our code
5662017-02-23T19:43:15  <gmaxwell> e.g. so we could correlate failures with a specific host?
5672017-02-23T19:43:22  <wumpus> or something that is completely broken on their end
5682017-02-23T19:43:30  <wumpus> gmaxwell: I'm afraid not. Wasn't able to find it
5692017-02-23T19:43:38  *** jtimon has quit IRC
5702017-02-23T19:43:49  <wumpus> I don't think they give access to that information
5712017-02-23T19:43:59  <jnewbery> wumpus: why is uploading from travis :( ? is it something you've tried before?
5722017-02-23T19:44:14  <wumpus> jnewbery: it's a pit of snakes, security-wise
5732017-02-23T19:44:21  <ryanofsky> this is older than 2 days, i first noticed these last friday on 9773: https://github.com/bitcoin/bitcoin/pull/9773#issuecomment-280684263
5742017-02-23T19:44:38  *** jtimon has joined #bitcoin-core-dev
5752017-02-23T19:44:40  <wumpus> jnewbery: we could temporary have it submit files somewhere to debug an issue, I guess
5762017-02-23T19:45:24  <jnewbery> It can be configured it to upload artifacts to S3: https://docs.travis-ci.com/user/uploading-artifacts/
5772017-02-23T19:45:30  <gmaxwell> jeremyrubin: the issue there is just in terms of provisioning travis with secrets.
5782017-02-23T19:45:37  <kanzure> no private environment variables?
5792017-02-23T19:45:40  <gmaxwell> er darnit, too many js in here.
5802017-02-23T19:45:54  <wumpus> private environment variables don't work for pull requests, the test script could be replaced with anything
5812017-02-23T19:45:57  <jeremyrubin> gmaxwell: i was wat-ing :)
5822017-02-23T19:46:06  <gmaxwell> kanzure: PR's can steal them.
5832017-02-23T19:46:07  <jeremyrubin> Can you get a shell to travis
5842017-02-23T19:46:20  <wumpus> including echo $SECRET_CODE or wget https://host/secretcode?$SECRETCODE
5852017-02-23T19:46:22  <kanzure> cool. hm.
5862017-02-23T19:46:30  <jeremyrubin> (probably against TOS)
5872017-02-23T19:46:51  <MarcoFalke> I think you can specify secrets that are valid on branches only, but I might be wrong
5882017-02-23T19:46:56  <gmaxwell> jeremyrubin: A assume just open a PR that connects back to you. :P
5892017-02-23T19:47:08  <wumpus> lol a reverse shell in a PR
5902017-02-23T19:47:13  <BlueMatt> I assume you can, but, yea ToS (not to mention monopolizing their service....)
5912017-02-23T19:47:32  <gmaxwell> wait, you're all not mining bitcoins there already?
5922017-02-23T19:47:41  <jeremyrubin> I heard someone did that recently right?
5932017-02-23T19:47:48  *** harrymm has quit IRC
5942017-02-23T19:48:10  <gmaxwell> in any case, we've wasted most of a perfectly good hour on this. :P
5952017-02-23T19:48:18  <wumpus> I like the idea of uploading the executable and core files more. They could be pushed to a private server, no need to openly host them, that meanst here's less reason for people up to no good to inject anything
5962017-02-23T19:48:56  <wumpus> the biggest security worry was with hosting the built binaries
5972017-02-23T19:49:07  <jtimon> topic code reorganizing?
5982017-02-23T19:49:13  <wumpus> bleh
5992017-02-23T19:49:15  <wumpus> okay :p
6002017-02-23T19:49:23  <wumpus> #topic code reorganizing
6012017-02-23T19:49:25  <jnewbery> suggested (marginally related) topic: code coverage and benchmark tracking
6022017-02-23T19:49:27  <gmaxwell> we should ask travis for a feature that sets an enviroment variable to H(project secret || commit hash) ... and then we could have something whitelist uploads from specific PRs (e.g. by known people)
6032017-02-23T19:49:36  <jtimon> it's 10 mins, so no time for a big topic I think
6042017-02-23T19:50:04  <jeremyrubin> I have a POC branch which moves most of the pure data structures  to a datastructures dir.
6052017-02-23T19:50:06  <gmaxwell> Rename rpctests to tests rename test_bitcoin to useless_thing_that_crashes_in_travis. Done.
6062017-02-23T19:50:36  <wumpus> gmaxwell: yep
6072017-02-23T19:50:39  <jtimon> jeremyrubin: you mean including things like CTransaction and CBlockHeader?
6082017-02-23T19:50:41  <luke-jr> jeremyrubin: that doesn't sound like the right direction
6092017-02-23T19:50:43  <jeremyrubin> no
6102017-02-23T19:50:49  <jeremyrubin> non-bitcoin specific ones
6112017-02-23T19:50:51  <gmaxwell> jeremyrubin: really? datastructures?  shall we have a file called functions.cpp and a file called variables.cpp too? :P
6122017-02-23T19:50:57  <jeremyrubin> E.g., prevector
6132017-02-23T19:50:58  <luke-jr> XD
6142017-02-23T19:51:06  *** echonaut1 has quit IRC
6152017-02-23T19:51:07  <luke-jr> jeremyrubin: oh, okay, that sounds less bad
6162017-02-23T19:51:08  <gmaxwell> oh you mean things like effective language extensions.
6172017-02-23T19:51:12  <jtimon> I would prefer to move prevector to the consensus dir
6182017-02-23T19:51:20  *** echonaut has joined #bitcoin-core-dev
6192017-02-23T19:51:28  <wumpus> yea, prevector is consensus critical
6202017-02-23T19:51:32  <gmaxwell> suggests that the name still isn't good in that luke and I misunderstood it immediately. :P
6212017-02-23T19:51:41  <jeremyrubin> so is secp256k1 ;)
6222017-02-23T19:51:50  <sipa> so is libc
6232017-02-23T19:51:50  *** echonaut has quit IRC
6242017-02-23T19:51:52  <jtimon> like in https://github.com/bitcoin/bitcoin/pull/8328
6252017-02-23T19:51:56  *** harrymm has joined #bitcoin-core-dev
6262017-02-23T19:51:58  <wumpus> yes, we're not renaming secp256k1 are we
6272017-02-23T19:52:00  <luke-jr> move prevector to boost
6282017-02-23T19:52:01  * luke-jr hides
6292017-02-23T19:52:02  <gmaxwell> Okay, so step one we move libc under consensus/ ...
6302017-02-23T19:52:32  <jeremyrubin> Anyways, I think it would be nice to move things like that, which could later be made upstream repos
6312017-02-23T19:52:38  <wumpus> I don't think this is going anywhere, everyone has a different opinoin on what file to put where
6322017-02-23T19:52:47  <gmaxwell> lpad.js.
6332017-02-23T19:52:54  *** echonaut has joined #bitcoin-core-dev
6342017-02-23T19:53:11  <luke-jr> Does prevector have any usage outside consensus?
6352017-02-23T19:53:19  <jeremyrubin> I think the reason I'd want that is it makes it easier to have more exhaustive testing and also allows other users to more easily use such datastructures
6362017-02-23T19:53:29  <sipa> luke-jr: it probably should :)
6372017-02-23T19:53:40  <wumpus> do we have any unique data structures that anyone else inthe world would want to use?
6382017-02-23T19:53:41  <jtimon> jeremyrubin: yeah I would like libconsensus to be an "upstream repo" like libsecp256k1, but we would need to complete it first
6392017-02-23T19:53:55  <gmaxwell> jeremyrubin: I don't think any of us care to maintain things like prevector for other usage. Making a good library takes a tremendous amount of work.
6402017-02-23T19:54:00  <wumpus> I don't think a bitcoin-datastructures library makes sense
6412017-02-23T19:54:01  <wumpus> right
6422017-02-23T19:54:17  <wumpus> if we offer a library it should be something useful to bitcoin users
6432017-02-23T19:54:39  <gmaxwell> Obviously if the author of something like that wants to create a library for other usage, thats fine! but not a project priority.
6442017-02-23T19:54:40  <wumpus> like a wallet library, or extend libconsensus, ...
6452017-02-23T19:55:15  <wumpus> sure, you can do whatever you want with the files in the repository, if you need it in your project you can make a library out of it for your own use, or just copy the file, etc
6462017-02-23T19:55:18  <gmaxwell> from a code org perspective I don't see a problem with moving things like prevector, limitedmap into a utility function directory.. With just the understanding that many of those are used in consensus code too.
6472017-02-23T19:55:32  <wumpus> not everything needs to be maintained as a library
6482017-02-23T19:55:32  <jtimon> but since each dev seems to have a different idea of what a "complete libconsensus" should look like...
6492017-02-23T19:56:04  <wumpus> gmaxwell: me neither
6502017-02-23T19:56:07  <kallewoof> Compartmentalizing bitcoin into separate modules seemed like a plan but maybe not shared by everyone. If it is a shared plan restructuring file places seems important.
6512017-02-23T19:56:08  <luke-jr> I don't mind it, but IMO more important to work toward more useful library splitting
6522017-02-23T19:56:14  <sipa> gmaxwell: agree, some things are generic enough that they can be seen as extensions of the standard libraries
6532017-02-23T19:56:42  <gmaxwell> sipa: e.g. someday libstdc++ could get something that generalizes prevector, if it did, we'd drop prevector and use that.
6542017-02-23T19:56:59  <sipa> c++23
6552017-02-23T19:57:03  <jonasschnelli> heh
6562017-02-23T19:57:04  <gmaxwell> (well STL technically, I guess, point remains)
6572017-02-23T19:57:32  <gmaxwell> sipa: C++23 will just integrate libconsensus of course.  template cryprocurrency.
6582017-02-23T19:57:39  <wumpus> hehe
6592017-02-23T19:57:42  <jeremyrubin> Well that's my point kinda
6602017-02-23T19:57:52  <jnewbery> suggested topic: code coverage and benchmark tracking
6612017-02-23T19:57:59  <jeremyrubin> the consensus things that aren't overly bitcoin-specific
6622017-02-23T19:57:59  <gmaxwell> Except I was making it as a joke. :P
6632017-02-23T19:58:01  <wumpus> jnewbery: next week
6642017-02-23T19:58:11  <jeremyrubin> Facebook's folly lib is kinda like that
6652017-02-23T19:58:13  <wumpus> the meeting is about to close
6662017-02-23T19:58:22  <sipa> anyone have any last words?
6672017-02-23T19:58:27  <gmaxwell> jnewbery: we can talk about about it after the meeting and discuss further next week. :)
6682017-02-23T19:58:44  <jnewbery> gmaxwell: sounds good
6692017-02-23T19:58:51  <wumpus> #endmeeting
6702017-02-23T19:58:51  <lightningbot> Meeting ended Thu Feb 23 19:58:51 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
6712017-02-23T19:58:51  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.html
6722017-02-23T19:58:51  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.txt
6732017-02-23T19:58:51  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.log.html
6742017-02-23T19:58:55  *** laurentmt has quit IRC
6752017-02-23T19:59:03  <BlueMatt> ok, #topic code coverage and benchmark tracking
6762017-02-23T19:59:20  <jeremyrubin> I wrote a benchdiff tool a while back
6772017-02-23T19:59:22  <MarcoFalke> ACK on benchmark tracking
6782017-02-23T19:59:27  <gmaxwell> jeremyrubin: the history of corporate soup libraries isn't really good, a lot of them just become abandoneware... and we have some many things to worry about that cooking up more libraries is really ... not the project's priority even if it would be good in an abstract sense.
6792017-02-23T19:59:35  *** lclc_ has joined #bitcoin-core-dev
6802017-02-23T19:59:59  <gmaxwell> In some cases like libsecp256k1 it can be strategic to build something for general use. But thats case by case.
6812017-02-23T20:00:04  <wumpus> folly is unfocused
6822017-02-23T20:00:16  <jnewbery> is anyone doing coverage? Anyone tried using coveralls or codecov or ... for tracking code coverage?
6832017-02-23T20:00:18  <jeremyrubin> Sure, the point was not really about having it be a separate library
6842017-02-23T20:00:25  <jeremyrubin> but for increasing testing
6852017-02-23T20:00:29  <wumpus> 'random grabbag of data structures' libraries don't tend to be used a lot
6862017-02-23T20:00:31  <kanzure> i thought it was already known that code coverage is low
6872017-02-23T20:00:38  <gmaxwell> jnewbery: we have configure script setup for lcov. It works.
6882017-02-23T20:00:41  <wumpus> even if the data structures in it are genius
6892017-02-23T20:01:19  <luke-jr> wumpus: we'd use it
6902017-02-23T20:01:24  <wumpus> jeremyrubin: the problem is that you also get lots of issues for unrelated uses
6912017-02-23T20:01:25  <jnewbery> gmaxwell: it works is a good start. Which way does code coverage go over time?
6922017-02-23T20:01:39  <gmaxwell> And our expirence from libsecp256k1 suggests that there has been relatively little project value from making it generally usable.  :(  basically many things that really should use it simply do not. And what does use it almost never reports interesting feedback or provides useful testing. :(
6932017-02-23T20:01:40  <luke-jr> wumpus: if nothing else, libraries can reduce the binary download size by not duplicating the code between our 4 or 5 programs
6942017-02-23T20:01:48  <kanzure> jnewbery: there's a lot of missing unit tests. they don't exist; go write them.
6952017-02-23T20:01:55  <wumpus> jeremyrubin: right now the code only supports our use case, it doens't have to balloon to everyone's features and favorite other things
6962017-02-23T20:01:57  <gmaxwell> jnewbery: it's been going up. well at least the 'rpc tests' improvements increased it considerably.
6972017-02-23T20:02:01  <luke-jr> gmaxwell: some things which really should use it seem to?
6982017-02-23T20:02:05  <cfields> kanzure: he has been :)
6992017-02-23T20:02:12  *** lclc has quit IRC
7002017-02-23T20:02:12  <kanzure> cfields: shows what i know
7012017-02-23T20:02:19  <jnewbery> cfields: :)
7022017-02-23T20:02:45  <wumpus> luke-jr:sure, internal libraries for organization without exposing them to the outside or making them official APIs
7032017-02-23T20:02:51  <wumpus> luke-jr: we already have those
7042017-02-23T20:03:05  <cfields> kanzure: you're right to point out that we're missing lots of coverage, ofc.
7052017-02-23T20:03:13  <gmaxwell> luke-jr: e.g. the horrific python stuff is used much more commonly.. and various embedded devices reinvented the wheel creating big sidechannel vulnerablities along the way.
7062017-02-23T20:03:15  <kanzure> cfields: prolly not a good excuse to not do constant coverage reports, hehe
7072017-02-23T20:03:16  <wumpus> luke-jr: turn them to dlls and you could reduce download size a bit
7082017-02-23T20:03:27  <cfields> heh
7092017-02-23T20:03:32  <jnewbery> kanzure: I'd like to do more. Doing coverage once is good. Tracking it across releases is better
7102017-02-23T20:03:42  <luke-jr> gmaxwell: eg JoinMarket seems to use it?
7112017-02-23T20:03:52  <wumpus> luke-jr: not by much I think as most of the duplication will tend to get compressed out
7122017-02-23T20:03:52  <luke-jr> or was it Electrum
7132017-02-23T20:03:52  <jnewbery> I'd also like to have historic data about benchmarks
7142017-02-23T20:03:52  <cfields> jnewbery: i'd be very curious to see if something like coveralls could be hooked up easily-ish
7152017-02-23T20:03:58  *** jtimon has quit IRC
7162017-02-23T20:04:04  <gmaxwell> jnewbery: ideally we'd be at a point where travis could fail builds that drop coverage too much, but the existing tests do not have high enough coverage reliably to make the workthwhile.
7172017-02-23T20:04:13  <luke-jr> wumpus: IMO libconsensus will be more significant
7182017-02-23T20:04:16  <jeremyrubin> Benchdiff tool is here: https://github.com/JeremyRubin/bitcoin/tree/benchdiff
7192017-02-23T20:04:24  <gmaxwell> luke-jr: yes, after I nagged them to move off some crazily vulnerable python code that was found on the internet.
7202017-02-23T20:04:31  <luke-jr> XD
7212017-02-23T20:04:42  <gmaxwell> luke-jr: e.g. code they were using would accept the signature 0,0 as valid for any pubkey+message.
7222017-02-23T20:04:48  <jnewbery> gmaxwell: I'm not aiming for ideal just yet. Just a general sense of which direction we're moving in
7232017-02-23T20:05:08  <wumpus> comparing benchmarks would need a machine dedicated to just that
7242017-02-23T20:05:49  <jnewbery> cfields: do you know if anyone has tried hooking up coveralls?
7252017-02-23T20:05:50  <gmaxwell> wumpus: we could run benchmarks in a cpu simulator... at glacial speed but get consistent results. :P (this is actually reasonable to do for things like in the benchmark tool)
7262017-02-23T20:05:50  <BlueMatt> benchmarks would have to happen on custom infrastructure, but coveralls or something would be nice
7272017-02-23T20:05:53  <wumpus> comparing results from different machines or from VMs would be pointless
7282017-02-23T20:06:02  <BlueMatt> if its easy to hook up (and doesnt require stupid shit like write permissions on your repo), i dont see why not?
7292017-02-23T20:06:34  <cfields> jnewbery: not that i'm aware of. I think the gcov stuff on our side needs a bit of love first, I'd be happy to help out there.
7302017-02-23T20:06:35  <wumpus> gmaxwell: ah yes a cycle correct simulator
7312017-02-23T20:06:40  <gmaxwell> I've been trying to get a simulator restup for libsecp256k1, where we can use it to verify constant-timeness of the compiled result.
7322017-02-23T20:07:17  <sipa> restup?
7332017-02-23T20:07:18  <jnewbery> ok, I'll try to hook up coveralls on my repo before next week's meeting
7342017-02-23T20:07:21  <gmaxwell> (I have the test working locally but haven't figured out how to automate it, will likely need to modify the simulator)
7352017-02-23T20:07:26  <gmaxwell> sipa: setup.
7362017-02-23T20:07:28  <jeremyrubin> diff
7372017-02-23T20:07:31  <wumpus> riscv has a cycle-accurate simulator
7382017-02-23T20:07:35  <jeremyrubin> crap sorry
7392017-02-23T20:08:16  <gmaxwell> wumpus: yes, though it doesn't simulate dram so I found it wasn't good enough for my libsecp256k1 testing. :P there is a 'cycle accurate' simulator of abstract x86_64 which doesn't exactly match any particular cpu but it close enough.
7402017-02-23T20:08:29  <Chris_Stewart_5> jnewbery: would love to see that on our repo
7412017-02-23T20:08:32  <jeremyrubin> here's the POC of the move datastructures/utils to a separate dir btw https://github.com/JeremyRubin/bitcoin/tree/move-to-libraries (may need rebase)
7422017-02-23T20:08:55  <gmaxwell> (I also tried the riscv one though before realizing it didn't do dram... it's crazy, it's a seperate compile target for the HDL)
7432017-02-23T20:09:00  <wumpus> gmaxwell: I'll shut up, seems you did much more research into this :)
7442017-02-23T20:09:22  <Chris_Stewart_5> jnewbery: I know isle2983 is interested in this as well -- not sure how much free time he has but might be worth collaborating
7452017-02-23T20:09:22  <jeremyrubin> i only did a few things just as an example -- editing lots of files isn't super fun :p
7462017-02-23T20:09:27  <gmaxwell> wumpus: well my research is mostly for a weird thing, trying to setup a CI test that catches the compiler undermining the constant time behavior of libsecp256k1's private key operations.
7472017-02-23T20:10:11  <jnewbery> Chris_Stewart_5: thanks. I'll ping him
7482017-02-23T20:10:27  <wumpus> in any case +1 for hooking up coveralls, seems like a small thing to do with potentially large payoff
7492017-02-23T20:10:40  <Chris_Stewart_5> wumpus: strongly agree.
7502017-02-23T20:11:03  <jeremyrubin> gmaxwell: isn't that pretty tough to do given modern pipelining?
7512017-02-23T20:12:30  <wumpus> gmaxwell: that HDL tool is pretty neat in itself, allowing generating customized riscv cores on demand, but yes using it to generate c++ seems a bit crazy
7522017-02-23T20:13:43  <wumpus> jeremyrubin: and don't forget branch prediction
7532017-02-23T20:14:16  <jeremyrubin> wumpus: that's a part of the pipeline
7542017-02-23T20:14:33  <gmaxwell> jeremyrubin: libsecp256k1 private key operations are constant time on current intel cpus. It was not that hard. The code just has to be completely free of data dependant branches, data dependant memory accesses, etc.
7552017-02-23T20:15:36  <gmaxwell> for example, sha256's compression function is a function that is naturally constant time, even a trivial implementation gets that right.
7562017-02-23T20:15:42  <gmaxwell> Just write all your code to look like that. :P
7572017-02-23T20:16:10  <gmaxwell> Though the compiler could still screw you over if it gets too smart, which is why I want ci support for testing it.
7582017-02-23T20:17:12  <bitcoin-git> [bitcoin] jnewbery opened pull request #9842: Fix RPC failure testing (continuation of #9707) (master...rpctestassert2) https://github.com/bitcoin/bitcoin/pull/9842
7592017-02-23T20:18:23  <MarcoFalke> BlueMatt: Is coveralls the thing that spams a comment on every pull request?
7602017-02-23T20:18:34  <BlueMatt> MarcoFalke: I sure hope not?
7612017-02-23T20:18:45  <wumpus> I don't think that's necessary
7622017-02-23T20:18:49  <BlueMatt> I'd hope it can integrate with github's ci status thing
7632017-02-23T20:18:50  <wumpus> it can integrate into github
7642017-02-23T20:19:38  <MarcoFalke> Sound good when this is possible.
7652017-02-23T20:19:59  <MarcoFalke> re: test_bitcoin failures
7662017-02-23T20:20:07  <MarcoFalke> It is an uninitialized read
7672017-02-23T20:20:20  <MarcoFalke> because g_conman is not set up in the unit tests, I assume
7682017-02-23T20:20:30  <MarcoFalke> *g_connman
7692017-02-23T20:23:43  <wumpus> MarcoFalke: mh that could be it
7702017-02-23T20:24:22  <gmaxwell> perhaps someday bitcoin core can have coverage like: https://people.xiph.org/~greg/libsecp256k1-coverage/src/index.html
7712017-02-23T20:25:15  <wumpus> could you add it? :)
7722017-02-23T20:26:05  <wumpus> that's pretty similar to coveralls output
7732017-02-23T20:26:12  <Chris_Stewart_5> MarcoFalke: I think coverall is what you are thinking of wrt comments on pull requests, see: https://github.com/bitcoin-s/bitcoin-s-core/pull/60
7742017-02-23T20:26:23  <BlueMatt> wumpus: I think he meant the mostly-100% coverage indicators there :p
7752017-02-23T20:26:28  <Chris_Stewart_5> not sure if it is configurable or not, honestly haven't played with it too much
7762017-02-23T20:26:44  <wumpus> BlueMatt: oh okay
7772017-02-23T20:26:58  <Chris_Stewart_5> but I think it is extremely helpful for beginners to figure out where to contribute too
7782017-02-23T20:26:58  <MarcoFalke> Chris_Stewart_5: Yes, this is terrible
7792017-02-23T20:26:58  <jeremyrubin> Can implement the trump policy: every line of new code must test 2 other lines of code
7802017-02-23T20:27:20  <BlueMatt> jeremyrubin: do we, then, need to test tests?
7812017-02-23T20:27:36  <gmaxwell> the code is the test of the tests.
7822017-02-23T20:27:44  <gmaxwell> go break the code, tests should fail.
7832017-02-23T20:27:54  <BlueMatt> gmaxwell: yes, we should have test harnesses for that :P
7842017-02-23T20:28:18  <gmaxwell> I still have not found any good tools for C.  I have something that belongs in a lovecraftian horror novel...
7852017-02-23T20:28:39  <jeremyrubin> can we get rid of the code
7862017-02-23T20:28:48  <jeremyrubin> and just gen it from tests
7872017-02-23T20:29:05  <wumpus> mauling the source code in unspeakable ways to make tests fail
7882017-02-23T20:29:13  <jnewbery> gmaxwell: that report is a beautiful thing. So much green
7892017-02-23T20:29:17  <gmaxwell> (A script that systemtatically makes 1 byte changes to the source code, then sees if it compiles, and if it does checks if the stripped binary matches the original, and skips it.. and otherwise checks if tests fail....)
7902017-02-23T20:29:36  <BlueMatt> gmaxwell: yea..........
7912017-02-23T20:29:55  <gmaxwell> jnewbery: it's actually better than the report shows.. that report is only with running the tests at the defaults, they get a bit more coverage when cranked up.
7922017-02-23T20:35:03  <jeremyrubin> Is there any coverage tools that let you see if you execute a branch with multiple values?
7932017-02-23T20:35:17  <jeremyrubin> E.g., doing concolic execution of some kind would be cool
7942017-02-23T20:35:31  <jeremyrubin> klee?
7952017-02-23T20:38:39  *** jtimon has joined #bitcoin-core-dev
7962017-02-23T20:40:13  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/3b2f7fdcaea36c8c5c93b21030d4e2327d2b6e8c
7972017-02-23T20:40:14  <bitcoin-git> bitcoin/0.14 3b2f7fd Wladimir J. van der Laan: doc: Add authors and changes since rc1 to release notes
7982017-02-23T20:41:53  <wumpus>  * [new tag]         v0.14.0rc2 -> v0.14.0rc2
7992017-02-23T20:42:08  <BlueMatt> hey-o
8002017-02-23T20:42:32  <achow101> yay
8012017-02-23T20:42:56  <sipa>     wee
8022017-02-23T20:43:19  <achow101> I hope gitian works this time (but it probably won't)
8032017-02-23T20:43:53  <wumpus> well there were no changes to gitian since 0.14.0rc1, so if it didn't work for you it probably still won't
8042017-02-23T20:44:08  <wumpus> but you can try
8052017-02-23T20:44:19  <sipa> gmaxwell: it looks like the sha1 collision that google produced took 2^63 sha invocations
8062017-02-23T20:44:41  <cfields> damn, just pushed release notes spelling fix :(
8072017-02-23T20:45:00  <wumpus> cfields: there's no hurry ,release notes changes can go in until final
8082017-02-23T20:45:10  * luke-jr tries his gitian automation script again :p
8092017-02-23T20:45:18  <wumpus> and between the last rc and final
8102017-02-23T20:45:33  <cfields> ok
8112017-02-23T20:47:34  *** lclc has joined #bitcoin-core-dev
8122017-02-23T20:47:51  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to 0.14: https://github.com/bitcoin/bitcoin/compare/3b2f7fdcaea3...f00429666c60
8132017-02-23T20:47:52  <bitcoin-git> bitcoin/0.14 95e68df Cory Fields: release: add a few performance-related notes
8142017-02-23T20:47:52  <cfields> I'm doing a vanilla, all defaults, public p2p sync of 0.13/0.14 nodes on ec2 with 4cores/16gb ram. I think that's a common enough use-case. If anyone would like to bench under different conditions, please go for it
8152017-02-23T20:47:52  <bitcoin-git> bitcoin/0.14 f004296 Wladimir J. van der Laan: Merge #9787: release: add a few performance-related notes...
8162017-02-23T20:49:00  *** lclc_ has quit IRC
8172017-02-23T20:50:26  <wumpus> cfields: at least it's kind of standardized
8182017-02-23T20:50:55  <achow101> cfields: I think 8gb ram is a more likely scenario
8192017-02-23T20:51:38  <luke-jr> note 32-bit builds can't use much RAM, but I don't think we care so much about that
8202017-02-23T20:54:41  <wumpus> let's not start arguing his choice of benchmark machine
8212017-02-23T20:54:59  <wumpus> if you want to benchmark on something else, go ahead
8222017-02-23T20:58:14  *** lclc has quit IRC
8232017-02-23T20:59:11  *** lclc has joined #bitcoin-core-dev
8242017-02-23T21:06:35  *** pavel_ has joined #bitcoin-core-dev
8252017-02-23T21:08:46  *** lclc has quit IRC
8262017-02-23T21:09:04  *** paveljanik has quit IRC
8272017-02-23T21:25:15  <achow101> woohoo! First again! (but there's probably an issue with osx)
8282017-02-23T21:47:02  *** jtimon has quit IRC
8292017-02-23T21:47:52  *** jtimon has joined #bitcoin-core-dev
8302017-02-23T21:49:50  <jtimon> re style, I wish I could uncomment ;; (add-hook 'before-save-hook 'delete-trailing-whitespace)
8312017-02-23T22:05:21  *** Madars has quit IRC
8322017-02-23T22:13:09  *** Madars has joined #bitcoin-core-dev
8332017-02-23T22:20:12  *** rndguy has quit IRC
8342017-02-23T22:22:43  <bitcoin-git> [bitcoin] jnewbery opened pull request #9843: Fix segwit getblocktemplate test (master...fixsegwitgetblocktemplate) https://github.com/bitcoin/bitcoin/pull/9843
8352017-02-23T22:26:05  <gmaxwell> I think we should not have that behavior. We should not make mining catch fire when segwit activates.
8362017-02-23T22:33:29  <luke-jr> gmaxwell: in other words, we should support non-segwit mining after it activates?
8372017-02-23T22:33:56  <luke-jr> (that'd be somewhat non-trivial FWIW)
8382017-02-23T22:34:14  *** Guyver2 has quit IRC
8392017-02-23T22:34:59  *** marcoagner has joined #bitcoin-core-dev
8402017-02-23T22:35:28  <luke-jr> basically we'd need to teach CreateNewBlock to filter out segwit txs, and then do some magic with GBT caching
8412017-02-23T22:38:00  <luke-jr> IIRC last time it was discussed, the conclusion was to not support it; I forget where that convo was tho
8422017-02-23T22:44:46  *** MarcoFalke has quit IRC
8432017-02-23T22:44:56  *** MarcoFalke has joined #bitcoin-core-dev
8442017-02-23T22:47:19  *** jtimon has quit IRC
8452017-02-23T22:50:14  *** jnewbery has quit IRC
8462017-02-23T22:50:37  *** goksinen has quit IRC
8472017-02-23T22:54:46  *** jtimon has joined #bitcoin-core-dev
8482017-02-23T22:57:43  <cfields> achow101: no match :(
8492017-02-23T23:02:40  <achow101> cfields: on osx? again?
8502017-02-23T23:02:52  <cfields> achow101: yep :(
8512017-02-23T23:02:55  <cfields> still building the rest
8522017-02-23T23:03:16  <achow101> *sigh* it's still a problem
8532017-02-23T23:05:08  <achow101> I'll see if it does the alternating thing like last time
8542017-02-23T23:11:04  *** goksinen has joined #bitcoin-core-dev
8552017-02-23T23:13:04  *** e4xit has joined #bitcoin-core-dev
8562017-02-23T23:15:57  *** goksinen has quit IRC
8572017-02-23T23:18:19  <gmaxwell> luke-jr: yes, we should.
8582017-02-23T23:18:38  <gmaxwell> luke-jr: the only harm is you won't include segwit transactions... and we can whine at miners later to fix themselves.
8592017-02-23T23:18:45  <gmaxwell> better to have some segwit txn rather than none.
8602017-02-23T23:19:02  <gmaxwell> We should also set the version bit, even if you're not sending the flag.
8612017-02-23T23:19:36  *** goksinen has joined #bitcoin-core-dev
8622017-02-23T23:20:13  *** goksinen has joined #bitcoin-core-dev
8632017-02-23T23:25:59  <bitcoin-git> [bitcoin] jtimon opened pull request #9845: RPC: cleanups in rpc/server (master...0.15-extern-rpc-server) https://github.com/bitcoin/bitcoin/pull/9845
8642017-02-23T23:26:26  <jtimon> jonasschnelli: am I missing something about the wallet in ^^, I hope not
8652017-02-23T23:40:51  *** AaronvanW has quit IRC
8662017-02-23T23:55:12  *** neha has quit IRC
8672017-02-23T23:55:35  *** fanquake has joined #bitcoin-core-dev
8682017-02-23T23:56:37  <fanquake> Speaking of benchmarking, can anyone tell me on "average" how much slower reindexing is compared to sync?
8692017-02-23T23:56:42  *** neha has joined #bitcoin-core-dev
8702017-02-23T23:57:27  <fanquake> Ive managed to get qt reindexing at ~3%/hr, which seems very slow. This machine has performed a -reindex-chainstate in <3 hrs before.
8712017-02-23T23:57:58  <fanquake> * less than 3 hrs
8722017-02-23T23:58:56  *** vicenteH has quit IRC
8732017-02-23T23:59:22  <sipa> fanquake: reindex-chainstate should be strictly faster than sync