12017-06-29T00:13:08  *** IRC-Source_13446 has joined #bitcoin-core-dev
  22017-06-29T00:21:52  *** Victorsueca has quit IRC
  32017-06-29T00:32:52  *** Chris_Stewart_5 has quit IRC
  42017-06-29T00:35:26  *** Victorsueca has joined #bitcoin-core-dev
  52017-06-29T00:36:07  *** mol has joined #bitcoin-core-dev
  62017-06-29T01:00:48  *** dabura667 has joined #bitcoin-core-dev
  72017-06-29T01:16:21  *** dabura667 has quit IRC
  82017-06-29T01:16:34  *** dabura667 has joined #bitcoin-core-dev
  92017-06-29T01:18:35  <phantomcircuit> gmaxwell, btw master compiles for me
 102017-06-29T01:31:05  <phantomcircuit> why did the pull testing stuff get changed?
 112017-06-29T01:31:11  <phantomcircuit> oh travis changed
 122017-06-29T02:04:40  *** elkalamar has joined #bitcoin-core-dev
 132017-06-29T02:19:02  *** mol has quit IRC
 142017-06-29T02:30:35  *** goatpig has joined #bitcoin-core-dev
 152017-06-29T03:11:17  *** belcher_ has quit IRC
 162017-06-29T03:27:52  *** SopaXorzTaker has quit IRC
 172017-06-29T03:32:06  *** SopaXorzTaker has joined #bitcoin-core-dev
 182017-06-29T03:37:33  *** jamesob has quit IRC
 192017-06-29T03:38:07  *** jamesob has joined #bitcoin-core-dev
 202017-06-29T03:42:22  *** jamesob has quit IRC
 212017-06-29T04:11:22  *** movrcx has quit IRC
 222017-06-29T04:12:27  *** fizzwont has quit IRC
 232017-06-29T04:12:30  *** j_ybt has joined #bitcoin-core-dev
 242017-06-29T04:13:39  *** fizzwont has joined #bitcoin-core-dev
 252017-06-29T04:14:03  *** fizzwont is now known as Guest7920
 262017-06-29T04:16:46  *** owowo has quit IRC
 272017-06-29T04:18:01  *** d9b4bef9 has quit IRC
 282017-06-29T04:19:07  *** d9b4bef9 has joined #bitcoin-core-dev
 292017-06-29T04:20:01  *** d9b4bef9 has quit IRC
 302017-06-29T04:21:09  *** d9b4bef9 has joined #bitcoin-core-dev
 312017-06-29T04:23:14  *** owowo has joined #bitcoin-core-dev
 322017-06-29T04:26:40  *** Guest7920 has quit IRC
 332017-06-29T04:27:09  *** fizzwont_ has joined #bitcoin-core-dev
 342017-06-29T04:28:29  *** j_ybt has quit IRC
 352017-06-29T04:30:16  *** j_ybt has joined #bitcoin-core-dev
 362017-06-29T04:54:38  *** Giszmo has quit IRC
 372017-06-29T05:00:50  *** fizzwont_ is now known as fizzwont
 382017-06-29T05:00:56  *** fizzwont has joined #bitcoin-core-dev
 392017-06-29T05:03:46  *** ProfMac has joined #bitcoin-core-dev
 402017-06-29T05:10:31  *** unholymachine has quit IRC
 412017-06-29T06:03:52  *** harrymm has quit IRC
 422017-06-29T06:25:44  *** harrymm has joined #bitcoin-core-dev
 432017-06-29T06:40:36  <bitcoin-git> [bitcoin] jonasschnelli closed pull request #9722: GUI: Display warning when attempting address reuse (master...reuse) https://github.com/bitcoin/bitcoin/pull/9722
 442017-06-29T07:00:05  *** dermoth has quit IRC
 452017-06-29T07:00:35  *** dermoth has joined #bitcoin-core-dev
 462017-06-29T07:04:24  *** j_ybt has quit IRC
 472017-06-29T07:09:03  *** j_ybt has joined #bitcoin-core-dev
 482017-06-29T07:19:40  *** harrymm1 has joined #bitcoin-core-dev
 492017-06-29T07:19:48  *** j_ybt has quit IRC
 502017-06-29T07:21:09  *** harrymm has quit IRC
 512017-06-29T07:24:05  *** arowser has joined #bitcoin-core-dev
 522017-06-29T07:24:22  *** harrymm1 has quit IRC
 532017-06-29T07:39:29  *** harrymm has joined #bitcoin-core-dev
 542017-06-29T07:41:31  *** Ylbam has joined #bitcoin-core-dev
 552017-06-29T07:43:26  *** baldur has quit IRC
 562017-06-29T08:02:50  *** chjj has quit IRC
 572017-06-29T08:04:54  *** chjj has joined #bitcoin-core-dev
 582017-06-29T08:14:37  *** timothy has joined #bitcoin-core-dev
 592017-06-29T08:16:05  *** AaronvanW has joined #bitcoin-core-dev
 602017-06-29T08:18:06  *** Aaronvan_ has joined #bitcoin-core-dev
 612017-06-29T08:21:31  *** AaronvanW has quit IRC
 622017-06-29T08:30:07  *** JackH has joined #bitcoin-core-dev
 632017-06-29T08:42:32  *** riemann has joined #bitcoin-core-dev
 642017-06-29T08:53:21  *** sjums has quit IRC
 652017-06-29T08:56:30  *** sjums has joined #bitcoin-core-dev
 662017-06-29T08:57:58  *** timothy has quit IRC
 672017-06-29T08:59:31  *** timothy has joined #bitcoin-core-dev
 682017-06-29T09:05:35  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/90a002ea647d...080ec5209172
 692017-06-29T09:05:35  <bitcoin-git> bitcoin/master 3c85332 Wladimir J. van der Laan: contrib: Update laanwj key...
 702017-06-29T09:05:36  <bitcoin-git> bitcoin/master 080ec52 MarcoFalke: Merge #10688: contrib: Update laanwj key...
 712017-06-29T09:06:06  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #10688: contrib: Update laanwj key (master...2017_06_laanwj_key) https://github.com/bitcoin/bitcoin/pull/10688
 722017-06-29T09:13:36  *** jannes has joined #bitcoin-core-dev
 732017-06-29T09:50:31  *** baldur has joined #bitcoin-core-dev
 742017-06-29T09:56:09  <wumpus> cfields: btw any idea what to do here? https://github.com/bitcoin/bitcoin/issues/10670   arguably the openbsd case is worst-case, they use an ancient GNU binutils
 752017-06-29T09:57:39  <wumpus> would be possible to detect this in configure, I guess, and then not disable the sse-reliant stuff
 762017-06-29T09:57:47  <wumpus> s/disable/enable
 772017-06-29T10:10:08  *** bincap has joined #bitcoin-core-dev
 782017-06-29T10:10:28  *** bincap has quit IRC
 792017-06-29T10:25:13  *** LeMiner2 has joined #bitcoin-core-dev
 802017-06-29T10:26:28  *** LeMiner has quit IRC
 812017-06-29T10:26:28  *** LeMiner2 is now known as LeMiner
 822017-06-29T10:36:20  *** Guest20283 is now known as haakonn
 832017-06-29T10:36:31  *** haakonn has joined #bitcoin-core-dev
 842017-06-29T11:09:03  <bitcoin-git> [bitcoin] practicalswift opened pull request #10701: Remove the virtual specifier for functions with the override specifier (master...virtual-override) https://github.com/bitcoin/bitcoin/pull/10701
 852017-06-29T11:22:55  *** talmai has joined #bitcoin-core-dev
 862017-06-29T11:35:33  *** jannes has quit IRC
 872017-06-29T11:35:59  *** jannes has joined #bitcoin-core-dev
 882017-06-29T11:38:06  *** Guyver2 has joined #bitcoin-core-dev
 892017-06-29T11:44:16  *** laurentmt has joined #bitcoin-core-dev
 902017-06-29T11:44:55  *** laurentmt has quit IRC
 912017-06-29T11:45:20  *** jannes has quit IRC
 922017-06-29T11:46:17  *** dabura667 has quit IRC
 932017-06-29T11:54:59  *** jannes has joined #bitcoin-core-dev
 942017-06-29T11:59:58  *** jannes has quit IRC
 952017-06-29T12:00:15  *** jannes has joined #bitcoin-core-dev
 962017-06-29T12:05:30  *** unholymachine has joined #bitcoin-core-dev
 972017-06-29T12:09:40  *** jannes has quit IRC
 982017-06-29T12:09:57  *** jannes has joined #bitcoin-core-dev
 992017-06-29T12:26:07  <bitcoin-git> [bitcoin] MeshCollider opened pull request #10702: [Trivial] Improve end-of-namespace comment consistency (master...improve-end-of-namespace-comment-consistence) https://github.com/bitcoin/bitcoin/pull/10702
1002017-06-29T12:28:38  *** arubi has quit IRC
1012017-06-29T12:29:02  *** arubi has joined #bitcoin-core-dev
1022017-06-29T12:33:14  <wumpus> what is people's obsession with en-of-namespace comments
1032017-06-29T12:33:52  <wumpus> we've just merged one three days ago (#9544), and now we need another PR?
1042017-06-29T12:33:53  <gribble> https://github.com/bitcoin/bitcoin/issues/9544 | [trivial] Add end of namespace comments. Improve consistency. by practicalswift · Pull Request #9544 · bitcoin/bitcoin · GitHub
1052017-06-29T12:55:37  *** belcher_ has joined #bitcoin-core-dev
1062017-06-29T13:04:02  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/080ec5209172...4c72cc33ebcc
1072017-06-29T13:04:02  <bitcoin-git> bitcoin/master fd9599b practicalswift: [qt] Avoid potential null pointer dereference in TransactionView::exportClicked()
1082017-06-29T13:04:03  <bitcoin-git> bitcoin/master 4c72cc3 Wladimir J. van der Laan: Merge #10673: [qt] Avoid potential null pointer dereference in TransactionView::exportClicked()...
1092017-06-29T13:04:32  <bitcoin-git> [bitcoin] laanwj closed pull request #10673: [qt] Avoid potential null pointer dereference in TransactionView::exportClicked() (master...null-pointer-dereference) https://github.com/bitcoin/bitcoin/pull/10673
1102017-06-29T13:05:18  *** Chris_Stewart_5 has joined #bitcoin-core-dev
1112017-06-29T13:36:06  <bitcoin-git> [bitcoin] jnewbery opened pull request #10703: [tests] Allow tests to pass when stderr is non-empty (master...test_stderr) https://github.com/bitcoin/bitcoin/pull/10703
1122017-06-29T13:57:49  <bitcoin-git> [bitcoin] jnewbery opened pull request #10704: [tests] nits in dbcrash.py (master...dbcrashtestnits) https://github.com/bitcoin/bitcoin/pull/10704
1132017-06-29T14:11:25  *** nemgun has joined #bitcoin-core-dev
1142017-06-29T14:16:56  *** Giszmo has joined #bitcoin-core-dev
1152017-06-29T14:23:46  *** Guest8908 has quit IRC
1162017-06-29T14:24:49  *** arubi has quit IRC
1172017-06-29T14:24:49  *** dermoth has quit IRC
1182017-06-29T14:25:04  *** tripleslash has joined #bitcoin-core-dev
1192017-06-29T14:25:27  *** tripleslash is now known as Guest86952
1202017-06-29T14:28:40  *** arubi has joined #bitcoin-core-dev
1212017-06-29T14:29:32  *** dermoth has joined #bitcoin-core-dev
1222017-06-29T14:30:17  *** mol has joined #bitcoin-core-dev
1232017-06-29T14:36:52  *** Dyaheon has quit IRC
1242017-06-29T14:37:39  *** Dyaheon has joined #bitcoin-core-dev
1252017-06-29T14:40:22  *** mol- has joined #bitcoin-core-dev
1262017-06-29T14:43:07  *** mol has quit IRC
1272017-06-29T14:43:08  *** jannes has quit IRC
1282017-06-29T14:43:53  *** jannes has joined #bitcoin-core-dev
1292017-06-29T14:58:28  *** jannes has quit IRC
1302017-06-29T14:59:53  *** Murch has joined #bitcoin-core-dev
1312017-06-29T15:03:50  *** talmai has quit IRC
1322017-06-29T15:11:17  *** jannes has joined #bitcoin-core-dev
1332017-06-29T15:14:14  *** mol- has quit IRC
1342017-06-29T15:15:40  *** Dizzle has joined #bitcoin-core-dev
1352017-06-29T15:18:28  *** riemann has quit IRC
1362017-06-29T15:18:52  *** jannes has quit IRC
1372017-06-29T15:20:02  <jonasschnelli> Should we crate a "trivial" label?
1382017-06-29T15:20:17  *** molz has joined #bitcoin-core-dev
1392017-06-29T15:20:39  <jonasschnelli> or something that point concludes "typo only fixes / comments / etc."?
1402017-06-29T15:25:13  <wumpus> we had a trivial label in the past, I removed that at some point because it didn't add anything that 'docs/output' or 'refactoring' doesn't do
1412017-06-29T15:25:46  <wumpus> it made people have the idea that labeling something 'trivial' would make it accepted sooner, prompting the creating of tons of trivial changes
1422017-06-29T15:26:17  <wumpus> in any case, better to label specifically. 'trivial' doesn't really tell anything about the change
1432017-06-29T15:27:19  <wumpus> e.g. a comment change would be a documentation change
1442017-06-29T15:31:12  *** jannes has joined #bitcoin-core-dev
1452017-06-29T15:32:56  *** eenoch_ has quit IRC
1462017-06-29T15:33:49  *** eenoch has joined #bitcoin-core-dev
1472017-06-29T15:37:32  *** sanada has quit IRC
1482017-06-29T15:37:41  *** sanada has joined #bitcoin-core-dev
1492017-06-29T15:39:29  <wumpus> dbcrash.py travis troubles https://github.com/bitcoin/bitcoin/pull/10704#issuecomment-312005841
1502017-06-29T15:40:58  <bitcoin-git> [bitcoin] MarcoFalke pushed 7 new commits to master: https://github.com/bitcoin/bitcoin/compare/4c72cc33ebcc...65cc7aacfbfc
1512017-06-29T15:41:00  <bitcoin-git> bitcoin/master 37065d2 John Newbery: [tests] remove unused imports from utils.py
1522017-06-29T15:41:00  <bitcoin-git> bitcoin/master f1fe536 John Newbery: [tests] fix flake8 warnings in test_framework.py and util.py
1532017-06-29T15:41:01  <bitcoin-git> bitcoin/master cad967a John Newbery: [tests] Move stop_node and start_node methods to BitcoinTestFramework...
1542017-06-29T15:41:23  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #10556: Move stop/start functions from utils.py into BitcoinTestFramework (master...testframeworkstopstart) https://github.com/bitcoin/bitcoin/pull/10556
1552017-06-29T15:44:39  *** abpa has joined #bitcoin-core-dev
1562017-06-29T15:49:18  <jonasschnelli> wumpus: I see, good point about the "trivial" label.
1572017-06-29T15:59:01  <sdaftuar> wumpus: thanks for the heads up, i'll investigate the dbcrash issue
1582017-06-29T16:00:20  *** talmai has joined #bitcoin-core-dev
1592017-06-29T16:00:22  <wumpus> sdaftuar: they look like different problems; 248398016 seems a race issue (sending command to terminated process), 248398018 is a timeout while running `generate`. It's a tad strange that both problems happen to be in dbcrash.
1602017-06-29T16:00:57  <sdaftuar> wumpus: it looks like the issue in 248398016 may just be that i got the exception name wrong somehow
1612017-06-29T16:01:08  <sdaftuar> https://travis-ci.org/bitcoin/bitcoin/jobs/248398016#L3321
1622017-06-29T16:03:50  <sdaftuar> we can bump the rpctimeout for the test to fix that second problem
1632017-06-29T16:04:59  <wumpus> sdaftuar: oops, good catch, didn't notice that between all the errors
1642017-06-29T16:06:47  <wumpus> yes that seems just a case of 'travis VM too slow for timeout'
1652017-06-29T16:10:18  *** mol has joined #bitcoin-core-dev
1662017-06-29T16:12:49  *** molz has quit IRC
1672017-06-29T16:24:41  *** talmai has quit IRC
1682017-06-29T16:32:34  *** Guyver2_ has joined #bitcoin-core-dev
1692017-06-29T16:34:48  *** Guyver2 has quit IRC
1702017-06-29T16:36:24  <cfields> wumpus: hmm
1712017-06-29T16:36:59  <cfields> wumpus: re the openbsd assembler, i think we could patch it to use the hex, same as rdrand? :(
1722017-06-29T16:38:21  <cfields> (i don't know enough about assemblers, but i figured that was done that way for exactly this reason)
1732017-06-29T16:38:54  <cfields> but yes, we could also check during configure
1742017-06-29T16:39:29  *** abpa has quit IRC
1752017-06-29T16:42:08  *** bincap has joined #bitcoin-core-dev
1762017-06-29T16:48:58  <sipa> cfields: yeah, i tried using rdrand as a mnemonic, but the osx assembler didn't accept it on travis
1772017-06-29T16:49:26  <sipa> hex always works, but the downside is that you must hardcode the registers
1782017-06-29T16:50:33  <wumpus> cfields: I don't particularly care if the sse-accelerated code is not used on openbsd
1792017-06-29T16:50:43  <wumpus> cfields: they have to use openssl -no-asm as well
1802017-06-29T16:51:15  <cfields> heh, really? That's a big hit
1812017-06-29T16:51:30  <wumpus> we definitely shouldn't dumb down the code just for this case, there's a large chance that openbsd will switch to clang at some pointa different as
1822017-06-29T16:51:33  <wumpus> yes, really
1832017-06-29T16:52:20  <cfields> yikes, ok
1842017-06-29T16:52:56  <wumpus> I don't want to get involved in the politics here, they choose to use an old as that doens't support modern instructions, they get slower code. It does need to compile, though.
1852017-06-29T16:53:42  *** timothy has quit IRC
1862017-06-29T16:53:42  <cfields> ok. let's just add an inline-compile check rather than trying to hunt down the assembler though. some (clang, at least) let you choose between an internal assembler or external
1872017-06-29T16:53:53  <sipa> using hex asm also has the advantage of not needing separately compiled objects just to have access to one asm instruction
1882017-06-29T16:53:58  <wumpus> their gcc is also -  by choice - an ancient version
1892017-06-29T16:54:17  <wumpus> ok, just don't do this for openbsd, it's likely a temporary issue
1902017-06-29T16:54:24  <sipa> agree
1912017-06-29T17:12:37  *** JackH has quit IRC
1922017-06-29T17:19:10  *** Aaronvan_ is now known as AaronvanW
1932017-06-29T17:21:43  *** Guyver2_ has quit IRC
1942017-06-29T17:24:57  *** owowo has quit IRC
1952017-06-29T17:30:06  *** owowo has joined #bitcoin-core-dev
1962017-06-29T17:35:00  *** Yogaqueef has joined #bitcoin-core-dev
1972017-06-29T17:49:13  *** sdaftuar has quit IRC
1982017-06-29T17:49:29  <bitcoin-git> [bitcoin] ka7 opened pull request #10705: Trivial: spelling fixes (master...feature/spelling) https://github.com/bitcoin/bitcoin/pull/10705
1992017-06-29T17:56:27  <bitcoin-git> [bitcoin] laanwj pushed 7 new commits to master: https://github.com/bitcoin/bitcoin/compare/65cc7aacfbfc...0c3542e5dec3
2002017-06-29T17:56:28  <bitcoin-git> bitcoin/master 00cb69b Jonas Schnelli: [Qt] allow to execute a callback during splashscreen progress
2012017-06-29T17:56:28  <bitcoin-git> bitcoin/master ae09d45 Jonas Schnelli: Allow to shut down during txdb upgrade
2022017-06-29T17:56:29  <bitcoin-git> bitcoin/master 316fcb5 Jonas Schnelli: Allow to cancel the txdb upgrade via splashscreen callback
2032017-06-29T17:57:01  <bitcoin-git> [bitcoin] laanwj closed pull request #10660: Allow to cancel the txdb upgrade via splashscreen keypress 'q' (master...2017/06/chainstate_update_prog) https://github.com/bitcoin/bitcoin/pull/10660
2042017-06-29T18:17:50  *** shesek has quit IRC
2052017-06-29T18:18:37  <bitcoin-git> [bitcoin] morcos opened pull request #10706: Improve wallet fee logic and fix GUI bugs (master...improveWalletFeeLogic) https://github.com/bitcoin/bitcoin/pull/10706
2062017-06-29T18:20:06  <bitcoin-git> [bitcoin] laanwj pushed 8 new commits to master: https://github.com/bitcoin/bitcoin/compare/0c3542e5dec3...2935b469ae96
2072017-06-29T18:20:07  <bitcoin-git> bitcoin/master 6d22b2b Matt Corallo: Pull script verify flags calculation out of ConnectBlock
2082017-06-29T18:20:07  <bitcoin-git> bitcoin/master b5fea8d Matt Corallo: Cache full script execution results in addition to signatures...
2092017-06-29T18:20:08  <bitcoin-git> bitcoin/master eada04e Matt Corallo: Do not print soft-fork-script warning with -promiscuousmempool
2102017-06-29T18:20:19  <bitcoin-git> [bitcoin] laanwj closed pull request #10192: Cache full script execution results in addition to signatures (master...2017-04-cache-scriptchecks) https://github.com/bitcoin/bitcoin/pull/10192
2112017-06-29T18:24:34  <sipa> w00t
2122017-06-29T18:24:47  <BlueMatt> hey, neat
2132017-06-29T18:24:52  <BlueMatt> 0.15 is coming together :)
2142017-06-29T18:25:27  <Dizzle> Definitely. I'm sad unix sockets aren't in. Looking forward to that for my electrum server.
2152017-06-29T18:29:00  <wumpus> Dizzle: good to hear at least someone was waiting for the UNIX sockets stuff, did you review/test the PR?
2162017-06-29T18:29:33  <Dizzle> wumpus: I will be this weekend, am getting an electrumx patch ready for it.
2172017-06-29T18:29:47  <wumpus> awesome
2182017-06-29T18:33:21  <wumpus> Dizzle: are you going to use P2P or RPC over unix sockets?
2192017-06-29T18:33:29  <wumpus> (or both)
2202017-06-29T18:33:29  <Dizzle> RPC
2212017-06-29T18:35:21  <Dizzle> Electrum servers tend to maintain their own utxo DB. Syncing with your node's db over RPC is a bit of a bottleneck. Using asynchronous i/o speeds things along but there is plenty of overhead using the TCP loopback when both softwares are in the same operating environment.
2222017-06-29T18:37:04  <bitcoin-git> [bitcoin] morcos opened pull request #10707: Better API for estimatesmartfee RPC  (master...bettersmartfeeapi) https://github.com/bitcoin/bitcoin/pull/10707
2232017-06-29T18:37:12  <wumpus> oh interesting, I had mostly seen the UNIX sockets as security improvement, not so much for performance, but yes avoiding local TCP might save some overhead
2242017-06-29T18:48:42  <gmaxwell> wumpus: well for p2p it would be a performance improvement if we changed the protocol and dropped the 'crc', but I don't think we were planning on doing that.
2252017-06-29T18:49:26  <jcorgan> i have a non-bitcoin related application that communicates between two processes using a unix socket at a continuous 5Gbps data rate
2262017-06-29T18:49:43  <jcorgan> uses zmq over unix socket
2272017-06-29T18:50:10  <gmaxwell> jcorgan: yea, right now though for us the fact that every p2p message gets sha2ed is way more overhead than TCP.
2282017-06-29T18:50:31  <wumpus> http://bhavin.directi.com/unix-domain-sockets-vs-tcp-sockets/
2292017-06-29T18:50:38  <jcorgan> oh, sure, i was just commenting that unix sockets are pretty fast
2302017-06-29T18:51:00  <wumpus> so there's really a performance gain in both latency and throughput because no local network needs to be emulates, I never realized that
2312017-06-29T18:51:16  <wumpus> makes sense though
2322017-06-29T18:51:33  <gmaxwell> perhaps an input to the BIP151 stuff... that it should be possible to run it in a mode that turns off the auth for use over a domain socket at least.
2332017-06-29T18:52:16  <wumpus> well one of the uses for UNIX would be for tor; in that case we certainly don't want to disable the crcing, or auth. But agree it'd be nice to have it as possibility.
2342017-06-29T18:52:57  <gmaxwell> yea. and one always needs to worry about downgrade attacks.
2352017-06-29T18:53:03  <jonasschnelli> gmaxwell: great idea
2362017-06-29T18:53:30  <wumpus> it'd be another kind of whitebind 'nocrcbind' 'noauthbind'
2372017-06-29T18:53:37  <jonasschnelli> gmaxwell: But then there would be no checksum... if you disable the poly1305 mac?
2382017-06-29T18:53:39  <wumpus> bindflags extension
2392017-06-29T18:53:58  <gmaxwell> jonasschnelli: right but there isn't any need for one with a purely local unix domain socket.
2402017-06-29T18:54:07  <wumpus> jonasschnelli: over a local socket, when communicating with local software, there's no point
2412017-06-29T18:54:20  <jonasschnelli> Indeed.
2422017-06-29T18:54:25  *** Murch has quit IRC
2432017-06-29T18:54:29  <jonasschnelli> Corruption through socket coms is not possible I guess?
2442017-06-29T18:54:38  <wumpus> and the permissions of the socket itself are used for authentication
2452017-06-29T18:54:40  <jonasschnelli> *domain sockets
2462017-06-29T18:54:44  *** SopaXorzTaker has quit IRC
2472017-06-29T18:54:49  <gmaxwell> jonasschnelli: no, not any more than any random memory anywhere.
2482017-06-29T18:54:53  <wumpus> no, unless memory/cpu corruption, in which case there's other issues
2492017-06-29T18:55:23  <wumpus> this is a SOCK_STREAM AF_UNIX socket, so neither reordering nor corruption nor packet loss should happen
2502017-06-29T18:55:28  <jonasschnelli> I see... yes. That mode would be awesome... especially for wallets (stuff I'm writing into libbtc) that downloads everything from the local peer
2512017-06-29T18:55:31  *** talmai has joined #bitcoin-core-dev
2522017-06-29T18:56:15  <wumpus> it's theoretically possible for SOCK_DGRAM AF_UNIX sockets to deliver packets out of order, though I've never heard of an OS that does that (but anyhow not an issue for us)
2532017-06-29T18:57:20  <jcorgan> one nice thing about ZMQ is that with a parameter change I can do the same thing between network endpoints on different hosts or two processes on the same host over a unix socket
2542017-06-29T18:57:24  <jcorgan> no code changes
2552017-06-29T18:57:29  *** lifeofguenter has quit IRC
2562017-06-29T18:57:46  <jcorgan> (or even two threads using zmq over shared memory)
2572017-06-29T18:57:58  <wumpus> yes, that's nice
2582017-06-29T18:58:28  <jcorgan> anyway, carry on
2592017-06-29T18:59:03  *** lifeofguenter has joined #bitcoin-core-dev
2602017-06-29T18:59:14  *** Murch has joined #bitcoin-core-dev
2612017-06-29T19:00:02  <gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
2622017-06-29T19:00:12  <wumpus> #startmeeting
2632017-06-29T19:00:12  <lightningbot> Meeting started Thu Jun 29 19:00:12 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
2642017-06-29T19:00:12  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
2652017-06-29T19:00:41  <sipa> oi
2662017-06-29T19:00:46  <instagibbs> #meetingbegin
2672017-06-29T19:00:58  <wumpus> topics?
2682017-06-29T19:00:59  <morcos> i'd like to discuss the fee changes needed for 0.15
2692017-06-29T19:01:06  <sipa> i have a few topics
2702017-06-29T19:01:16  <instagibbs> morcos, ack
2712017-06-29T19:01:19  <sipa> short update on signature aggregation
2722017-06-29T19:01:25  *** sdaftuar has joined #bitcoin-core-dev
2732017-06-29T19:01:29  * BlueMatt wants to change priority review to #10652
2742017-06-29T19:01:31  <gribble> https://github.com/bitcoin/bitcoin/issues/10652 | Small step towards demangling cs_main from CNodeState by TheBlueMatt · Pull Request #10652 · bitcoin/bitcoin · GitHub
2752017-06-29T19:01:34  <gmaxwell> hurray for merges.
2762017-06-29T19:01:44  <sipa> the need for the watchonly rpc flag after multiwallet
2772017-06-29T19:01:47  <cfields> hi, here
2782017-06-29T19:02:03  *** praxeology has joined #bitcoin-core-dev
2792017-06-29T19:02:09  <sipa> rolling utxo hashes
2802017-06-29T19:02:22  <wumpus> thanks for the topic suggestions, yes let's as usual start with high priority for review
2812017-06-29T19:02:33  <wumpus> #topic high priority for review
2822017-06-29T19:02:35  <wumpus> https://github.com/bitcoin/bitcoin/projects/8
2832017-06-29T19:02:38  <jonasschnelli> suggesting: again multiwallet endpoint vs json parameter
2842017-06-29T19:02:58  *** shesek has joined #bitcoin-core-dev
2852017-06-29T19:03:08  <wumpus> BlueMatt: instead of #10179?
2862017-06-29T19:03:11  <gribble> https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub
2872017-06-29T19:03:15  <BlueMatt> correct
2882017-06-29T19:03:17  <kanzure> hi.
2892017-06-29T19:03:22  <BlueMatt> well, actually, its built on
2902017-06-29T19:03:22  <jonasschnelli> Replaced BlueMatt's 10179 with 10652
2912017-06-29T19:03:23  <BlueMatt> so...ehh
2922017-06-29T19:03:29  <BlueMatt> but, yea
2932017-06-29T19:03:46  <jonasschnelli> aha.. double pull binding strategy. :)
2942017-06-29T19:04:04  <BlueMatt> i mean 10179 is like one ack away, just want cfields to confirm i addressed his feedback sufficiently
2952017-06-29T19:04:25  <morcos> So I don't think I've had any there for a couple weeks, if I could add two?  It would be the first two of the fee changes, both have been open a little while, #10543 and #10589
2962017-06-29T19:04:26  <gribble> https://github.com/bitcoin/bitcoin/issues/10543 | Change API to estimaterawfee by morcos · Pull Request #10543 · bitcoin/bitcoin · GitHub
2972017-06-29T19:04:27  <gribble> https://github.com/bitcoin/bitcoin/issues/10589 | More economical fee estimates for RBF and RPC options to control by morcos · Pull Request #10589 · bitcoin/bitcoin · GitHub
2982017-06-29T19:04:35  <morcos> I apologize I have not been around to do more reviewing recently
2992017-06-29T19:05:09  <wumpus> BlueMatt: yes, as we discussed: it should still be merged, but it's no longer high-priority because you don't expect the dependent PR to get in in time to be safe for 0.15
3002017-06-29T19:05:22  <jonasschnelli> morcos: which one di you want to add to the high-prio list?
3012017-06-29T19:05:29  <jonasschnelli> *do
3022017-06-29T19:05:31  <wumpus> both
3032017-06-29T19:05:55  <morcos> both! :)  but i suppose 10589, if i can only have one
3042017-06-29T19:05:56  <jonasschnelli> Good
3052017-06-29T19:06:00  <BlueMatt> wumpus: well I want some glances at 10652 pre-15 to see if its too much or if it can go ahead...if its small enough for 15 I do want it for 15
3062017-06-29T19:06:19  <cfields> BlueMatt: yes, good enough. Will ACK it.
3072017-06-29T19:06:19  <jonasschnelli> We need both for 0.15
3082017-06-29T19:06:20  <BlueMatt> (since it fixes the kinda-not-a-big-deal provide-invalid-block attack thing)
3092017-06-29T19:07:15  <wumpus> ok - any other suggestions?
3102017-06-29T19:07:23  <wumpus> enough other topics otherwise
3112017-06-29T19:07:40  <wumpus> #topic short update on signature aggregation
3122017-06-29T19:07:43  <sipa> hi
3132017-06-29T19:07:43  <wumpus> (sipa)
3142017-06-29T19:07:54  <praxeology> Whats the status on the mempool data structure change?
3152017-06-29T19:08:05  <praxeology> woops not mempool
3162017-06-29T19:08:07  <sipa> this is just a status update of what gmaxwell, apoelstra and me have been working on lately
3172017-06-29T19:08:08  <praxeology> utxo
3182017-06-29T19:08:11  <wumpus> praxeology: you're interrupting a meeting
3192017-06-29T19:08:31  <sipa> i presented on this in milan, and later we wrote a paper for bitcoin17
3202017-06-29T19:08:37  <gmaxwell> praxeology: long since done.
3212017-06-29T19:08:50  <sipa> the paper was rejected with the very valuable feedback that a solution already existed
3222017-06-29T19:09:05  <sipa> namely a paper by Bellare & Neven from 2006
3232017-06-29T19:09:42  <sipa> it only solves one of the problems we were trying to solve (signature aggregation, not key aggregation)... but that's the only consensus-critical part if we'd want aggregation in bitcoin trnasactions
3242017-06-29T19:09:53  <gmaxwell> (which irritatingly never turned up in eons of searching for us)
3252017-06-29T19:10:15  <wumpus> so that solution is usable for bitcoin?
3262017-06-29T19:10:18  <sipa> yes
3272017-06-29T19:10:24  <sipa> the advantage is that this is peer-reviewed scheme with a strong security proof under very wide assumptions
3282017-06-29T19:10:26  <wumpus> nice!
3292017-06-29T19:10:29  <gmaxwell> Their solution is almost equivilent to ours (or is equivient with the right kind of squinting about hash function definitions).
3302017-06-29T19:10:31  <jonasschnelli> https://eprint.iacr.org/2006/285.pdf
3312017-06-29T19:10:37  *** ivan has joined #bitcoin-core-dev
3322017-06-29T19:10:38  <gmaxwell> so thats good too.
3332017-06-29T19:11:08  <wumpus> yes
3342017-06-29T19:11:12  <wumpus> good news
3352017-06-29T19:11:16  <gmaxwell> jonasschnelli: doesn't look like the right paper (though maybe its one they published to another venue)
3362017-06-29T19:11:17  <BlueMatt> cool!
3372017-06-29T19:11:48  <sipa> so what this scheme gives us is a way for transactions to have a single signature (as long as all signers cooperate, so even in the case of coinjoin) overall... regardless of the number of inputs or multisig
3382017-06-29T19:11:51  <wumpus> do we have a good link?
3392017-06-29T19:12:08  <sipa> https://cseweb.ucsd.edu/~mihir/papers/multisignatures-ccs.pdf
3402017-06-29T19:12:14  <sipa> ^
3412017-06-29T19:12:24  <gmaxwell> ^ thats it.
3422017-06-29T19:12:26  <wumpus> #link https://cseweb.ucsd.edu/~mihir/papers/multisignatures-ccs.pdf
3432017-06-29T19:12:35  <sipa> what it does not do is an ability to turn multisig into signle sig (but that could be added on top later, as it's purely a wallet interaction thing)
3442017-06-29T19:12:43  <sipa> it also supports batch validation
3452017-06-29T19:12:53  <cfields> ooh
3462017-06-29T19:12:56  <sipa> meaning that a whole block (or even multiple blocks) could be validated at once
3472017-06-29T19:13:16  <sipa> the speedup depends on the size of the batch, but may go as high as 5x (for 4000 signatures)
3482017-06-29T19:13:21  <gmaxwell> Unfortunately our paper isn't available because we need to update it to reflect that work,  but it is much more targeted for the Bitcoin application (and would probably be much more clear for people here).
3492017-06-29T19:13:50  <sipa> in the batch validation case (without aggregated signatures) the speedup would likely be restricted to 3.5x or so
3502017-06-29T19:13:55  <morcos> gmaxwell: is that something that'll happen?  can we just wait to read yours?
3512017-06-29T19:14:08  <sipa> yes, we'll definitely finish up the paper
3522017-06-29T19:14:18  <sipa> and discuss the change more widely
3532017-06-29T19:14:24  <sipa> just wanted to give a heads up here
3542017-06-29T19:14:31  <wumpus> yes, thanks for the update!
3552017-06-29T19:14:37  <morcos> if i could have next topic, i have to leave early
3562017-06-29T19:14:40  <cfields> sipa: what about that per-block aggregation that was briefly discussed? does this get us any closer to that?
3572017-06-29T19:14:46  *** jannes has quit IRC
3582017-06-29T19:14:50  <cfields> nm, will follow-up after meeting
3592017-06-29T19:15:07  <wumpus> morcos: what was your topic?
3602017-06-29T19:15:13  <gmaxwell> ~2.3x speedup for 32 signatures in the aggregate, fwiw.
3612017-06-29T19:15:17  <morcos> Fee changes for 0.15
3622017-06-29T19:15:27  <wumpus> #topic fee changes needed for 0.15
3632017-06-29T19:15:31  <wumpus> morcos: sorry, missed that one
3642017-06-29T19:15:50  <wumpus> morcos: you were actually first to propose a topic :)
3652017-06-29T19:16:05  <morcos> I'll be relatively quick for my part, I think I've got all the PR's out now that I think need to go in for 0.15, but I want to encourage people to think about a bunch of the RPC API changes so they are good in their first release
3662017-06-29T19:16:24  <morcos> But the other thign is there is one piece of missing functionality wheich I think is needed
3672017-06-29T19:16:26  *** goatpig has quit IRC
3682017-06-29T19:16:36  <morcos> #10590
3692017-06-29T19:16:36  <gribble> https://github.com/bitcoin/bitcoin/issues/10590 | Access to longer fee estimates using GUI · Issue #10590 · bitcoin/bitcoin · GitHub
3702017-06-29T19:17:15  <morcos> Given how volatile fee estimates are and how much they change between short targets and long, I think it's important to give the GUI access to longer fee estimates
3712017-06-29T19:17:31  <morcos> But someone more familar with QT can probably whip that up a lot quicker than me
3722017-06-29T19:17:53  <morcos> Might be best to build it on top of all my other changes, #10707 shoudl have everything in one
3732017-06-29T19:17:54  <gribble> https://github.com/bitcoin/bitcoin/issues/10707 | Better API for estimatesmartfee RPC by morcos · Pull Request #10707 · bitcoin/bitcoin · GitHub
3742017-06-29T19:17:55  <jonasschnelli> I'll have a look.
3752017-06-29T19:18:13  <morcos> Thanks!  That's what I was hoping for. :)  instagibbs might have more on this topic?
3762017-06-29T19:18:21  <instagibbs> #10333 for fee bug fix :)
3772017-06-29T19:18:22  <gribble> https://github.com/bitcoin/bitcoin/issues/10333 | [wallet] fee fixes: always create change, adjust value, and p… by instagibbs · Pull Request #10333 · bitcoin/bitcoin · GitHub
3782017-06-29T19:18:36  <instagibbs> not much else, maybe I'll summon energy to review your PRs
3792017-06-29T19:18:39  <morcos> is that the only thing you feel is critical for 0.15?
3802017-06-29T19:18:46  <instagibbs> only realistic merge yeah
3812017-06-29T19:18:51  <morcos> most of mine are now easy to review i think
3822017-06-29T19:18:59  <gmaxwell> Do we have a feel for when bump will be generally usable enough to start making replacability a default? (or at least more visible?)
3832017-06-29T19:19:03  <instagibbs> some of my other work has been sucked up by achow101 so waiting on 0.16 for that stuff
3842017-06-29T19:19:54  <morcos> gmaxwell: at least it'll be easily accessible to choose RBF given the RPC and GUI options in 0.15
3852017-06-29T19:20:09  <instagibbs> I'd really like it to be able to add additional inputs as needed
3862017-06-29T19:20:13  <gmaxwell> Oh!  I often miss gui changes, I'll check that out.
3872017-06-29T19:20:20  <jonasschnelli> Yes. Not sure if we persist the RBF state across restarts (in the GUI)
3882017-06-29T19:20:21  <morcos> Might help us learn if there are other bump features needed...
3892017-06-29T19:20:23  <instagibbs> but it seems to me the logic is much simpler with effective value....
3902017-06-29T19:20:26  <jonasschnelli> Ideally we should
3912017-06-29T19:20:28  <instagibbs> some disagree
3922017-06-29T19:20:43  <gmaxwell> instagibbs: IIRC that was the big usability blocker for further use.
3932017-06-29T19:21:05  <wumpus> gmaxwell: https://github.com/bitcoin/bitcoin/pull/9527#issuecomment-311659024
3942017-06-29T19:21:09  <instagibbs> it randomly doesn't work which is disappointing UX
3952017-06-29T19:21:20  <morcos> gmaxwell: the ability to addother inputs?  isn't it pretty rare to not have change?
3962017-06-29T19:21:45  <jonasschnelli> But can happen...
3972017-06-29T19:21:46  <wumpus> no, we don't persist RBF state, it has to be selected per transaction
3982017-06-29T19:22:02  <jonasschnelli> wumpus: maybe the GUI should remember it
3992017-06-29T19:22:03  <instagibbs> morcos, we are going to target more exact matches in future, fwiw
4002017-06-29T19:22:04  <wumpus> the only way to make it persist is the command line option
4012017-06-29T19:22:15  <morcos> wumpus: the gui initializes with the the command line argument, and then persists during the session
4022017-06-29T19:22:15  <wumpus> jonasschnelli: meh, better to have it as "option" then
4032017-06-29T19:22:24  <gmaxwell> FWIW, I believe electrum defaults to replacable now and pushes pretty hard in that direction, though users can flip it off on a per tx basis.
4042017-06-29T19:22:29  <morcos> via checkbox
4052017-06-29T19:22:47  <wumpus> jonasschnelli: persisting non-option settings between restarts would be unexpected
4062017-06-29T19:23:03  <jonasschnelli> Yes. I guess your right..
4072017-06-29T19:23:03  <gmaxwell> In any case, I think the default is kind of moot until bumping is sufficiently mature.
4082017-06-29T19:23:21  <wumpus> between transactions in the same session makes sense I guess
4092017-06-29T19:23:25  <morcos> I suppose I have one more question on that
4102017-06-29T19:23:31  <jonasschnelli> Yes. If the bump won't work because it can't add another input the default should remain at the current state
4112017-06-29T19:23:37  <wumpus> yes
4122017-06-29T19:23:51  <jonasschnelli> It can happen quickly when fees are rising
4132017-06-29T19:23:52  *** DrOlmer has joined #bitcoin-core-dev
4142017-06-29T19:23:59  <achow101> hi. I'm late
4152017-06-29T19:24:01  <morcos> Right now there are no options to the "Increase transaction fee" option in the GUI and it uses the default tx confirm target.  Should it instead use whatever the slider is set to?
4162017-06-29T19:24:19  <jonasschnelli> Yes
4172017-06-29T19:24:19  <morcos> If the slider is not in use and custom fee is set, shoudl it use that?
4182017-06-29T19:24:23  <wumpus> morcos: the slider is on another tab
4192017-06-29T19:24:24  <jonasschnelli> I'd like to work on the replacability in the GUI for 0.16
4202017-06-29T19:24:31  <morcos> Those would be easy changes to make after my PR
4212017-06-29T19:24:39  <BlueMatt> the slider is in another tab, thats strange
4222017-06-29T19:24:47  <wumpus> morcos: not sure that would be intuitive, people assume the slider is for new transactions, the bump option should probably have its own choice dialog
4232017-06-29T19:24:50  <jonasschnelli> First I though of bringing back to tx to the original send-tx screen (you could even add recipients... ) but meh
4242017-06-29T19:25:01  <morcos> wumpus: that seems maybe too much optionality
4252017-06-29T19:25:08  <jonasschnelli> The bump window should just be lager and has the slider
4262017-06-29T19:25:16  <wumpus> jonasschnelli: yes
4272017-06-29T19:25:27  <morcos> ok, thats fine.. so leave it as the wallet default confirm target for now?
4282017-06-29T19:25:35  <wumpus> yes
4292017-06-29T19:25:41  <BlueMatt> yea, sucks, but its easy and reasonable
4302017-06-29T19:25:47  <jonasschnelli> And also we have never really discussed the pre-signed bumps.. but that we should probably do in another meeting
4312017-06-29T19:25:57  <BlueMatt> yea, that sounds like a 16
4322017-06-29T19:25:59  <instagibbs> jonasschnelli, that will involve new strategy
4332017-06-29T19:26:00  <instagibbs> :)
4342017-06-29T19:26:17  <instagibbs> reasonably diff from after the fact fix imo
4352017-06-29T19:26:43  <jonasschnelli> I'd say focus on fee opt. in 0.15, rbf in 0.16
4362017-06-29T19:27:15  <wumpus> agreed
4372017-06-29T19:27:22  <wumpus> #topic the need for the watchonly rpc flag after multiwallet (sipa)
4382017-06-29T19:27:27  <sipa> hi!
4392017-06-29T19:27:38  <wumpus> (we need to move forward a bit, lots of topics)
4402017-06-29T19:27:43  <sipa> currently many RPCs have an optional flag "include watchonly"
4412017-06-29T19:27:44  <jonasschnelli> is that similar to the -disablehot?
4422017-06-29T19:27:53  * jonasschnelli is listening
4432017-06-29T19:28:14  <sipa> at the time the need for that flag existed because of a desire to keep your "hot" wallet separated from your "watch only" wallet
4442017-06-29T19:28:20  <sipa> i think that was a mistake
4452017-06-29T19:28:24  <wumpus> disablehot: #9662
4462017-06-29T19:28:25  <gribble> https://github.com/bitcoin/bitcoin/issues/9662 | Add `-disablehot` mode: a sane mode for watchonly-wallets by jonasschnelli · Pull Request #9662 · bitcoin/bitcoin · GitHub
4472017-06-29T19:28:41  <wumpus> sipa: yes, on the long term I agree with you
4482017-06-29T19:28:50  <jonasschnelli> sipa: you think with multiwallet the wallet should either be watch or hot?
4492017-06-29T19:28:55  <sipa> jonasschnelli: no
4502017-06-29T19:29:03  <wumpus> sipa: makes more sense to have a wallet either full-watchonly or has-keys
4512017-06-29T19:29:16  <sipa> wumpus: perhaps, but that's orthogonal
4522017-06-29T19:29:22  <wumpus> sipa: I don't understand you then
4532017-06-29T19:29:24  <instagibbs> ok get to the point :)
4542017-06-29T19:29:25  <BlueMatt> why is that a mistake?
4552017-06-29T19:29:32  <jonasschnelli> Let sipa explain...
4562017-06-29T19:29:33  <sipa> what i'm trying to get at is that the within-a-wallet separation is no longer needed
4572017-06-29T19:29:51  <wumpus> how is that different from what I said?
4582017-06-29T19:30:11  <wumpus> instead of watchonly within a wallet you'd have a watchonly wallet and a normal wallet
4592017-06-29T19:30:18  <sipa> i'm not arguing to remove the ability to have both keys and watchonly in one wallet
4602017-06-29T19:30:18  <gmaxwell> because if you want to have a mixed thing thats fine too, then you just have a mixed thing. No need to flag, if you want seperation, use two wallets.
4612017-06-29T19:30:19  <jonasschnelli> but I fail to see the difference then between only allowing watch-only or hot
4622017-06-29T19:30:32  <sipa> just that there is no need to just select coins that affect one part
4632017-06-29T19:30:38  <gmaxwell> you're suggesting an extra restriction.
4642017-06-29T19:30:38  <sipa> or see a 'balance' of just one part
4652017-06-29T19:30:51  <sipa> a wallet is a wallet, and has a single balance
4662017-06-29T19:31:01  <sipa> some of the keys may require decrypting your wallet
4672017-06-29T19:31:05  <wumpus> oh, right
4682017-06-29T19:31:06  <sipa> some of the keys may require a hardware wallet
4692017-06-29T19:31:18  <jonasschnelli> I see... yes.
4702017-06-29T19:31:21  <sipa> some of the key may be just watchonly and you need to use raw transactions to interact with thing
4712017-06-29T19:31:26  <BlueMatt> fair, this sounds like an 0.17 or 0.18 thing, though
4722017-06-29T19:31:30  <gmaxwell> Now, logically you probably will seperate or something, for convience, but I don't see a particular reason to require that right now.
4732017-06-29T19:31:32  <BlueMatt> are you asking if we should deprecate?
4742017-06-29T19:31:33  <sipa> i was hoping 0.15
4752017-06-29T19:31:33  <wumpus> BlueMatt: agree, long term
4762017-06-29T19:31:47  <sipa> just make the watchonly flag ignored and always set it to true
4772017-06-29T19:31:49  <wumpus> this is not something we're going to change in the RPC interface pre-0.15
4782017-06-29T19:31:54  <sipa> ok
4792017-06-29T19:31:55  <wumpus> peopel rely on this
4802017-06-29T19:32:01  <wumpus> we could document it as deprecated
4812017-06-29T19:32:03  <BlueMatt> we'd need to mark it deprecated
4822017-06-29T19:32:04  <morcos> sipa: that seems reasonable except what about identifying which things you have keys for and which you dont..
4832017-06-29T19:32:11  <BlueMatt> probably deprecate after we have working multiwallet that is stable
4842017-06-29T19:32:19  <wumpus> then remove the flag for 0.16 or 0.17, but this seems over-hurried
4852017-06-29T19:32:20  <BlueMatt> so maybe deprecate in 0.16...
4862017-06-29T19:32:24  <morcos> that seems a useful distinction to keep to me
4872017-06-29T19:32:25  <gmaxwell> with 0.15 and multiwallet we can start deprication at least-- e.g. advise that this will happen in the future, suggest people use seperate wallets. . The one problem with that however is that your seperate watchonly wallet still needs the stupid flag everywhere. :(
4882017-06-29T19:32:26  <BlueMatt> remove in 17 or 18
4892017-06-29T19:32:46  <wumpus> let's focus on actually getting multiwallet into 0.15
4902017-06-29T19:32:48  <jonasschnelli> I somehow think mixed wallets can be a footgun source... but right, it orthogonal
4912017-06-29T19:32:48  <instagibbs> related topic: some way to signal that the funds are "safe" when you expect a hardware wallet to have the privkey
4922017-06-29T19:32:52  <instagibbs> post-0.15 ofc
4932017-06-29T19:32:57  <sipa> maybe i haven't made this clear, but how do you deal with hardware wallets, for example?
4942017-06-29T19:33:07  <jonasschnelli> we need a standard!
4952017-06-29T19:33:08  <sipa> add a 2nd option everyone 'include hw wallet keys'
4962017-06-29T19:33:09  <morcos> +1 better support for hardware wallets!
4972017-06-29T19:33:14  <sipa> jonasschnelli: orthogonal
4982017-06-29T19:33:18  <wumpus> hardware wallets in bitcoin core is a different topic
4992017-06-29T19:33:23  <BlueMatt> we dont need to add a flag for hw wallets
5002017-06-29T19:33:30  <sipa> BlueMatt: then why do we need a flag for watchonly?
5012017-06-29T19:33:35  <wumpus> important, but certainly not one that's going to make it into 0.15
5022017-06-29T19:33:38  <BlueMatt> we can say "hw wallets are always included in balance, flag for watchonly is deprecated" starting in the version that supports hw wallets
5032017-06-29T19:33:40  <gmaxwell> sipa is pointing out that the model of 'watch only' when applied to also having hardware wallets starts adding combinitoric blowup.
5042017-06-29T19:33:54  <sipa> BlueMatt: fair enough
5052017-06-29T19:34:01  <jonasschnelli> If a wallet has no clear cur between hot and cold (watch-only), a code-level guarantee, I would not use it for hot funds...
5062017-06-29T19:34:02  <BlueMatt> yes, agreed, we should not make it worse, but we dont need to worry about this until at least 16, I think
5072017-06-29T19:34:06  <jonasschnelli> *cut
5082017-06-29T19:34:14  <wumpus> agree on not making it worse
5092017-06-29T19:34:15  <BlueMatt> need useable working good multiwallet first, which likely wont be 15
5102017-06-29T19:34:18  <gmaxwell> BlueMatt: thats a point. now just give me a flag for importmulti that gives me a watching key imported that way and it's good to go. :P
5112017-06-29T19:34:19  <sipa> jonasschnelli: again, orthogonal
5122017-06-29T19:34:34  <instagibbs> I have a working Core+Ledger system, and have a couple thoughts, but this is a different topic yep
5132017-06-29T19:34:39  <gmaxwell> BlueMatt: uhh, it's like done.
5142017-06-29T19:34:50  <jonasschnelli> sipa: but why not just separating pure watch-only wallets from hot wallets? Why would that be orthogonal?
5152017-06-29T19:35:03  <BlueMatt> gmaxwell: I know, but we need a cycle of finding more use-cases and making sure we've got it all covered, was my piont
5162017-06-29T19:35:11  <wumpus> yes multiwallet is almost done, but in 0.15 it will at least be experimental
5172017-06-29T19:35:15  <BlueMatt> eg createwallet flows within rpc, disconectwallet, etc
5182017-06-29T19:35:16  <sipa> jonasschnelli: "orthogonal" means you can still do that
5192017-06-29T19:35:23  <sipa> jonasschnelli: it has nothing to do with this issue
5202017-06-29T19:35:24  <wumpus> it's the first release it is in, after all
5212017-06-29T19:35:27  <gmaxwell> jonasschnelli: because that is an additional restriction that AFAIK isn't needed. maybe later its needed to not support mixed but it seems like a seperate issue to me.
5222017-06-29T19:35:46  <jonasschnelli> Okay
5232017-06-29T19:36:01  <BlueMatt> ok, so we all agree, eventually push people towards multiwallet away from watchonly :)
5242017-06-29T19:36:05  <BlueMatt> next topic? :p
5252017-06-29T19:36:07  <sipa> what i want to get add is that a wallet is just a collection of keys it considers "mine" - independent of its ability to actually fully sign
5262017-06-29T19:36:12  <sipa> BlueMatt: yes, agree
5272017-06-29T19:36:19  <wumpus> #topic rolling utxo hashes
5282017-06-29T19:36:22  <wumpus> (sipa again)
5292017-06-29T19:36:22  <sipa> hi!
5302017-06-29T19:36:30  <instagibbs> sipa, ISMINE_* tho :)
5312017-06-29T19:36:34  <instagibbs> ok next topic
5322017-06-29T19:36:54  <sipa> with pertxout we changed the serialized_hash because the new format no longer maintains the tx version of the utxo
5332017-06-29T19:37:11  <sipa> i posted about rolling utxo hashes a while ago on the ML
5342017-06-29T19:37:35  <sipa> i'm not proposing actually implementing that, but would it be worthwhile to immediately switch to a scheme that is compatible with it?
5352017-06-29T19:37:43  <sipa> so that there is no need to break the API again
5362017-06-29T19:38:01  <gmaxwell> sipa: as in don't do the rolling thing, but have the oneshot thing compute the same hash?
5372017-06-29T19:38:06  <sipa> yes
5382017-06-29T19:38:16  <sipa> downside: makes gettxoutsetinfo slower
5392017-06-29T19:38:30  <wumpus> how much slower?
5402017-06-29T19:38:32  <sipa> upside: allows us to make gettxoutsetinfo super fast in the future
5412017-06-29T19:38:42  <gmaxwell> lots slower.
5422017-06-29T19:38:44  <sipa> several times
5432017-06-29T19:38:56  <wumpus> could add a new RPC for it
5442017-06-29T19:39:02  <gmaxwell> sipa: Well a challenge there is that I'm not sure that we've settled on the field. So that isn't a guarentee of compatiblity.
5452017-06-29T19:39:03  <wumpus> instead of gettxoutsetinfo
5462017-06-29T19:39:27  <sipa> interesting, i hadn't considered that
5472017-06-29T19:39:30  <sipa> gmaxwell: yeah, i know
5482017-06-29T19:39:47  <gmaxwell> actually if we drop the hash from gettxoutsetinfo i think thats the only thing now that requires scanning the whole thing.
5492017-06-29T19:39:55  <sipa> no, everything does
5502017-06-29T19:40:01  <sipa> (txout count etc)
5512017-06-29T19:40:03  <wumpus> yes it's all aggregate statistics
5522017-06-29T19:40:06  <gmaxwell> yes but it wouldn't have to with rather trivial changes.
5532017-06-29T19:40:08  <sipa> though those things can be maintained on the fly
5542017-06-29T19:40:20  <gmaxwell> which would be robust and wouldn't change.
5552017-06-29T19:40:27  <sdaftuar> i think we will want an RPC that can scan the disk to calculate the answer, even if we are able to calculate everything on the fly
5562017-06-29T19:40:34  <sdaftuar> so that we know our on-disk data is correct
5572017-06-29T19:40:35  <sipa> sdaftuar: good point
5582017-06-29T19:40:39  <gmaxwell> sdaftuar: restart your node. :P
5592017-06-29T19:41:02  <sipa> an advantage of the fast hash is that you can compare it with a recompute-the-whole-thing
5602017-06-29T19:41:17  <wumpus> that'd be very nice
5612017-06-29T19:41:17  <gmaxwell> okay interesting points.
5622017-06-29T19:41:45  <wumpus> a utxo hash that would be quick to compute for every block would be very nice to have
5632017-06-29T19:42:20  <gmaxwell> (I was momentarily overestimating how easy it would be to switch to summary statistics, I forgot that they have to be saved and loaded across restart... or otherwise every startup needs the equal of a stats call)
5642017-06-29T19:43:03  <gmaxwell> wumpus: right thats the goal of pieter's work. It's just a bit immature now, and if we implmenet it at the moment we may want to switch to an incompatible version later.
5652017-06-29T19:43:39  <wumpus> I like to check that all my nodes have the same utxo hash, but calling getutxosetinfo for every block takes too much time, I've tried and given up :)
5662017-06-29T19:43:52  <gmaxwell> Assuming we stay with the multiplicative group hash,  we need to pick a prime where multiplication mod that prime is as fast as possible.  Sipa has done some work there, but it's a research project that can sink as much time as we want to put into it.
5672017-06-29T19:44:32  <sipa> or we could just use the elliptic curve version, which can probably be made ~2 slower than the GMP-based MuHash
5682017-06-29T19:44:38  <sipa> which is just a few lines of code
5692017-06-29T19:44:57  <wumpus> now doing it intermittently, but that means that when it fails we don't know exacly where it started to diverge
5702017-06-29T19:45:17  <gmaxwell> right, I want to have UpdateTip log the value.
5712017-06-29T19:45:23  <sipa> ^ that
5722017-06-29T19:45:31  <sdaftuar> wumpus: it's actually not clear to me how much the fast utxo hash calculation helps in comparing running nodes
5732017-06-29T19:46:11  <sipa> well the fast utxo hash lets you do a consistency check on just a single node
5742017-06-29T19:46:31  <sdaftuar> but what is exactly being compared as consistent?
5752017-06-29T19:46:32  <sipa> by having a fast incrementally-updated version, and a slow recompute-from-scratch one
5762017-06-29T19:46:33  <gmaxwell> sdaftuar: because you can log the utxo hash at each point, and so if they diverge in a way that the hash sees (e.g. not underlying disk corruption) you'll learn.   Also you could run a command that checks the disk against the running value to catch that disk corrouption.
5772017-06-29T19:46:56  <gmaxwell> so  your disk <> your running <> my running <> my disk
5782017-06-29T19:47:13  <sdaftuar> yes, i agree if you do the comparison with disk, then you get something valuable
5792017-06-29T19:47:25  <sdaftuar> but just comparing the fast calculation between nodes doesn't seem like it does much, does it?
5802017-06-29T19:47:28  <wumpus> hm yes good point
5812017-06-29T19:47:31  <gmaxwell> right now it is a PITA to compare you and I at disk because we have to do it at the same time (and hope there isn't a block at that instant. :P )
5822017-06-29T19:47:40  <sdaftuar> gmaxwell: agreed
5832017-06-29T19:47:47  <gmaxwell> sdaftuar: it depends on where the errors you're concerned about are happening.
5842017-06-29T19:48:23  <wumpus> gmaxwell: yes, even when you time the RPC command on blocknotify, it sometimes misses the block :)
5852017-06-29T19:48:31  <gmaxwell> if they're below the layer where the running hash runs you only gain if you also do periodic checks between it and the disk.  Above it, however, you have constant checking.
5862017-06-29T19:49:08  <gmaxwell> but the nice thing is that disk and running can be async checked... You and I don't need to do our disk comparisons at the same time.
5872017-06-29T19:49:36  <wumpus> indeed
5882017-06-29T19:49:49  <gmaxwell> sdaftuar: this is all also machinery we almost certantly need for a reasonable UTXO-assume-valid kind of sync in the future.
5892017-06-29T19:49:55  <wumpus> all in all a rolling utxo hash is an improvement, it creates more options, but you can still do the same as now if you want
5902017-06-29T19:50:03  <sdaftuar> gmaxwell: yeah i agree and that's the use case i'm most excited about :)
5912017-06-29T19:50:19  <sdaftuar> i was just trying to figure out exactly how i'd use to compare my own nodes, and wasn't sure of the utility
5922017-06-29T19:50:44  <gmaxwell> wumpus: the challange though is that it isn't free. muhash on the whole utxo set takes CPU minutes.
5932017-06-29T19:51:02  <wumpus> gmaxwell: yes I'm not sure it should replace the faster hash
5942017-06-29T19:51:15  <wumpus> maybe it should justb e an additional thing
5952017-06-29T19:51:23  <gmaxwell> well once it's a running hash its very fast. :P
5962017-06-29T19:51:41  <sdaftuar> hash_serialized_3? :P
5972017-06-29T19:51:41  <wumpus> OTOH we're already breaking the hash for 0.15
5982017-06-29T19:51:56  <wumpus> (which is kind of sad, as it makes it impossible to compare against older versions)
5992017-06-29T19:52:32  <gmaxwell> sipa backported the new hash to the old system for development testing, FWIW.
6002017-06-29T19:52:54  <gmaxwell> (it's a pretty trivial change, IIRC, just drop the version from it)
6012017-06-29T19:53:03  <wumpus> cool, that'd be useful, especially with the 0.14 to 0.15 database change it's important to be able to check synchronization
6022017-06-29T19:54:05  <gmaxwell> This patch existed at one point already, dunno if sipa still has it.
6032017-06-29T19:54:07  <sipa> the problem is that #10434 is quite a bit of intricate code
6042017-06-29T19:54:09  <gribble> https://github.com/bitcoin/bitcoin/issues/10434 | [WIP] 3072-bit MuHash based hash_serialized by sipa · Pull Request #10434 · bitcoin/bitcoin · GitHub
6052017-06-29T19:54:43  <sipa> the EC version would be many times less code (given that we already have secp256k1), but be a few times slower
6062017-06-29T19:55:36  <wumpus> I don't have a strong opinion on it
6072017-06-29T19:55:37  <sipa> on the other hand, MuHash is very simple to implement in anything that already has big integers (it's a few lines in python)
6082017-06-29T19:55:58  <sipa> ok
6092017-06-29T19:56:00  *** tmddzk has joined #bitcoin-core-dev
6102017-06-29T19:56:03  *** justanotheruser has quit IRC
6112017-06-29T19:56:06  <wumpus> though in general I'd say higher performance seems preferable to the ability to re-use code
6122017-06-29T19:56:23  <sipa> in that case, some review would be welcome :)
6132017-06-29T19:56:32  <wumpus> but I haven't seen the code
6142017-06-29T19:56:42  <wumpus> yeah, hope to get around to it
6152017-06-29T19:56:59  <sipa> i can drop the asm optimized version from the first PR if wanted
6162017-06-29T19:57:06  <praxeology> Couldn't you put a delay on insert/remove from the rolling hash... say only for utxos that are 1 day of blocks old? isn't a hash for N blocks ago just about as good as the current hash?
6172017-06-29T19:57:18  <sipa> praxeology: totally irrelevant
6182017-06-29T19:57:43  <sipa> that would mean you need to keep those utxos around for processing later
6192017-06-29T19:57:59  <sipa> we have an approach that can combine them into a running hash in _microseconds_
6202017-06-29T19:58:06  <gmaxwell> All doing that does is perhaps saves you 1% of computation for blocks that are reorged out but at the expense of complexifying everything because the data is inconsistently available.
6212017-06-29T19:58:16  *** justanotheruser has joined #bitcoin-core-dev
6222017-06-29T19:58:29  <praxeology> What percent of utxos are spent within a day?
6232017-06-29T19:58:37  <instagibbs> 2 minutes left
6242017-06-29T19:58:41  <instagibbs> if anyone has microtopic
6252017-06-29T19:58:43  <wumpus> that seems irrelevant to this discussion
6262017-06-29T19:58:54  <wumpus> (although it's interesting to know in its own right)
6272017-06-29T19:59:07  <praxeology> Sounds like you guys are concerned about performance on the rolling hash
6282017-06-29T19:59:24  <wumpus> #endmeeting
6292017-06-29T19:59:24  <lightningbot> Meeting ended Thu Jun 29 19:59:24 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
6302017-06-29T19:59:24  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-29-19.00.html
6312017-06-29T19:59:24  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-29-19.00.txt
6322017-06-29T19:59:24  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-29-19.00.log.html
6332017-06-29T19:59:31  <gmaxwell> praxeology: delaying it doesn't change that.
6342017-06-29T19:59:32  <instagibbs> sipa, you still around?
6352017-06-29T19:59:36  <sipa> instagibbs: sure
6362017-06-29T19:59:45  <instagibbs> ok let me toss a wallet q to you
6372017-06-29T19:59:54  <gmaxwell> praxeology: its the same amount of computation if you do it now or 100 blocks ago.
6382017-06-29T20:00:00  <midnightmagic> Man I wish people would end meetings elsewhere with that kind of precision..
6392017-06-29T20:00:04  <wumpus> yes it'd just move the computation in time
6402017-06-29T20:00:04  * midnightmagic shudders
6412017-06-29T20:00:22  <wumpus> midnightmagic: I guess they should hire me as meeting chair :p
6422017-06-29T20:00:27  <praxeology> gmaxwell:  delaying will reduce the number of things that are xored (err whatever math op you doing), since utxos that were spent before the delay window are never added
6432017-06-29T20:00:29  <gmaxwell> praxeology: if you imagine something which is never consistent with any actual state of the system, then that really isn't all that useful.
6442017-06-29T20:00:36  <BlueMatt> I was gonna say something about the swiss leading the meeting, but its wumpus, not jonasschnelli
6452017-06-29T20:00:42  <instagibbs> sipa, so for hw wallets, would it be reasonable to have a new ISMINE type that means the wallet expects the output to be spendable by the wallet even though privkeys aren't physically present in the wallet.dat
6462017-06-29T20:00:58  <sipa> instagibbs: in my opinion, ISMINE should become a bool
6472017-06-29T20:01:03  <sipa> it's yours or not
6482017-06-29T20:01:07  <gmaxwell> praxeology: that isn't delayed.
6492017-06-29T20:01:08  <instagibbs> ah ok, so no
6502017-06-29T20:01:14  <midnightmagic> wumpus: :-)  Your nickname shall be The Guillotine
6512017-06-29T20:01:19  <instagibbs> well in that case at least some of the logic could be moved elsewhere
6522017-06-29T20:01:22  <sipa> instagibbs: the ability to spend should be independent from what is considered yours
6532017-06-29T20:01:30  <sipa> instagibbs: (note, that's my personal opinion)
6542017-06-29T20:01:43  <gmaxwell> praxeology: I believe what you are suggesting doing is computing it sparsely so that there isn't a value computed at every block.
6552017-06-29T20:01:43  <wumpus> midnightmagic: :-)
6562017-06-29T20:01:45  <instagibbs> no it's fair, I'm just trying to get my wallet to understand what I consider mine
6572017-06-29T20:01:49  <instagibbs> and right now it's janky mess
6582017-06-29T20:02:05  <sipa> instagibbs: perhaps the solvable distinction is still useful
6592017-06-29T20:02:30  <instagibbs> so a new "consider yours" function would fix my current issue in a cleaner way
6602017-06-29T20:02:49  <sipa> cfields: still around?
6612017-06-29T20:02:52  <achow101> instagibbs: isn't there some combination of ISMINE_ types that indicate "no privkey in wallet but I can still spend it"
6622017-06-29T20:02:53  <gmaxwell> praxeology: this would reduce computation somewhat, but at the expense creating coordination points... and then you only perhaps get a 2x speedup, but you also add bursts of letency rather than being able to compute it continually.
6632017-06-29T20:02:57  <cfields> sipa: yes
6642017-06-29T20:03:00  <praxeology> gmaxwell: I'm suggesting only adding a utxo to the hash on the 144th confirmation
6652017-06-29T20:03:27  <instagibbs> achow101, there is solvable+watchonly
6662017-06-29T20:03:31  <instagibbs> but that doesn't mean you can spend it
6672017-06-29T20:03:37  <gmaxwell> praxeology: that would make something which is _never_ consistent with the utxo set at any point, I think this would be completely useless to us.
6682017-06-29T20:03:49  <instagibbs> so if you have a hw wallet, you expect to be able to sign for it, but there's no enum for it
6692017-06-29T20:03:54  <gmaxwell> praxeology: it couldn't be used to check database consistency, it couldn't be used to perform a sync from utxo.
6702017-06-29T20:04:07  <sipa> praxeology: that sort of approach may be useful for UTXO/TXO commitment like approaches, where updating the commitment is very expensive and cheaper when batched
6712017-06-29T20:04:08  <instagibbs> so in corner cases you consider that output untrusted, and get "insufficient amount"
6722017-06-29T20:04:22  <sipa> praxeology: but the rolling UTXO idea was specifically intended to not need that, because it's so fast
6732017-06-29T20:04:30  <praxeology> gmaxwell: if you have the last 144 blocks then you can do the remainder of the utxos from those blocks to finish the hash at a particular point
6742017-06-29T20:04:33  <achow101> instagibbs: oh
6752017-06-29T20:04:47  <bitcoin-git> [bitcoin] narula opened pull request #10708: Connecttrace fewer blocks (master...connecttrace-fewer-blocks) https://github.com/bitcoin/bitcoin/pull/10708
6762017-06-29T20:05:01  <sipa> praxeology: just looking up a utxo spent from disk is more work than updating the hash
6772017-06-29T20:05:08  *** Dyaheon has quit IRC
6782017-06-29T20:05:17  <cfields> neha__: eh? :)
6792017-06-29T20:05:20  <instagibbs> achow101, I coded a fix for this corner issue, but only for p2sh multisig... current methodology kind of forces me to do janky fix or make another ismine enum :/
6802017-06-29T20:06:12  <praxeology> sipa: yes, well not sure the use case of when such would be needed
6812017-06-29T20:06:15  <achow101> just make ismine a bool :p
6822017-06-29T20:07:07  *** Dyaheon has joined #bitcoin-core-dev
6832017-06-29T20:08:17  <praxeology> earlier you guys were talking about a tx just having one signature, even for multiple things that need to be signed.  You talked about computation performance.  What about impact on tx size?
6842017-06-29T20:08:45  <praxeology> particularly since... i hear that network bandwidth is the main bottlneck
6852017-06-29T20:09:23  <instagibbs> N signatures in 1 signature space possible, across inputs. Still need the pubkeys
6862017-06-29T20:10:52  *** neha__ is now known as neha
6872017-06-29T20:11:19  <praxeology> sure, still need public keys.  but what about the signature size?
6882017-06-29T20:11:42  <neha> cfields: BlueMatt gave me an issue to fix!
6892017-06-29T20:11:46  <sipa> 64 bytes instead of N*72
6902017-06-29T20:11:48  <sipa> praxeology: ^
6912017-06-29T20:11:53  <gmaxwell> praxeology: instagibbs said: one signature.
6922017-06-29T20:12:28  <sipa> praxeology: and bandwidth isn't all that much of a factor anymore single compact blocks etc
6932017-06-29T20:12:31  <cfields> neha: good to see! that's a rabbit hole. I'd be nervous if you didn't find more things to fix while you're down there :)
6942017-06-29T20:12:40  <instagibbs> achow101, sounds simple, but let's see all the interaction with current wallet stuff
6952017-06-29T20:12:47  <gmaxwell> (the signatures in bitcoin today are 72 instead of 64.125 just because they use a dumb encoding.
6962017-06-29T20:13:05  <gmaxwell> )
6972017-06-29T20:13:44  <praxeology> sipa: do you have a layman link for "single compact block"? or anyone?
6982017-06-29T20:13:52  <sipa> praxeology: bip 152
6992017-06-29T20:14:12  <sipa> i'm sure there are better explanation online than the bip, though
7002017-06-29T20:14:21  <gmaxwell> I dunno the bit is pretty good.
7012017-06-29T20:14:27  <gmaxwell> bip
7022017-06-29T20:14:38  <instagibbs> gmaxwell, speaking of which how is 0.5RTT going these days, any change?
7032017-06-29T20:14:44  *** justanotheruser has quit IRC
7042017-06-29T20:14:53  <sipa> cfields: so the block-wide aggregation that adiabat proposed a while ago on the ML still applies to Bellare-Neven... allowing to have only 32 bytes of the signature in every tx, and another 32 byte block wide
7052017-06-29T20:14:54  <instagibbs> (if you're monitoring)
7062017-06-29T20:14:57  *** AaronvanW has quit IRC
7072017-06-29T20:15:06  <sipa> cfields: the downside is that it doesn't play nicely with cached signature validation
7082017-06-29T20:15:12  *** justanotheruser has joined #bitcoin-core-dev
7092017-06-29T20:15:18  <gmaxwell> instagibbs: meh, we need the skip-recent-txn things in mining.  It's gone up and down (in particular utility of the extrapool has gone up and down)
7102017-06-29T20:15:23  <sipa> cfields: as wtxids would change after inclusion in a block
7112017-06-29T20:15:34  *** AaronvanW has joined #bitcoin-core-dev
7122017-06-29T20:15:44  <gmaxwell> instagibbs: during periods of long backlogs the extrapool is too small-- I see misses that my node had seen before.
7132017-06-29T20:15:49  *** RoyceX has joined #bitcoin-core-dev
7142017-06-29T20:16:11  *** cheese_ has quit IRC
7152017-06-29T20:16:33  <cfields> sipa: hmm. Does parallel validation still apply as well?
7162017-06-29T20:16:39  <praxeology> sipa: oh, that is just where txs are not re-relayed with blocks.  Something  like a 1/2 bandwidth used improvement.
7172017-06-29T20:16:51  <cfields> *the parallel validation improvements
7182017-06-29T20:17:04  <praxeology> or is there something else I missed when skimming? something on the order of mimblewimble improvement?
7192017-06-29T20:17:18  <sipa> praxeology: yes, but the bandwidth needed to propagate a block quickly is massively reduced
7202017-06-29T20:17:27  <sipa> praxeology: overall data volume is reduced by 2
7212017-06-29T20:17:42  <gmaxwell> praxeology: typically at the tip blocks are transmitted with about 16kbytes.
7222017-06-29T20:18:08  <sipa> cfields: yes
7232017-06-29T20:18:29  <sipa> cfields: you can just shard and do the computation for a number of groups
7242017-06-29T20:18:44  <sipa> and then do a simple cheap combine operation
7252017-06-29T20:19:08  <gmaxwell> you lose some of the asymtoptic gains though we've been expirementing with parallel versions of the aggregate validation operation.
7262017-06-29T20:20:30  <gmaxwell> instagibbs: in the last 144 blocks I see 23% requiring a round trip. 13% were saved from needing one by the extra pool.  A week ago the extrapool saves were about half that I think.
7272017-06-29T20:21:21  *** owowo has quit IRC
7282017-06-29T20:22:43  <cfields> sipa: if we cached as much as possible otherwise (hashing mainly, i suppose) and completely dropped the pre-validated cache, do you have a sense of how it'd compare? I realize it'd be worth keeping the cache as it'd still have a good hit rate, i'm just curious.
7292017-06-29T20:23:17  <sipa> cfields: i don't understand
7302017-06-29T20:23:19  <gmaxwell> cfields: I'm unclear about what you're asking.
7312017-06-29T20:24:18  <cfields> trying to weight the benefits of parallel validation against losing some pre-cached hits
7322017-06-29T20:24:26  <sipa> well, do both
7332017-06-29T20:24:45  <gmaxwell> yea, there isn't any conflict. You parallel validate the things that miss the cache.
7342017-06-29T20:24:57  <sipa> oh, you mean with the block-wide aggregation
7352017-06-29T20:24:59  <gmaxwell> do you mean batch validation instead of parallel btw?
7362017-06-29T20:25:06  <sipa> my concern there is mostly the layering violation
7372017-06-29T20:25:13  <gmaxwell> the block wide aggregation stuff is ugh
7382017-06-29T20:25:27  <cfields> yes, sorry. i meant batch.
7392017-06-29T20:26:22  <BlueMatt> instagibbs: I think we'd actually see a bigger improvement by responding to getblocktxn requests in the background while connecting a block than making 0.5rtt more common
7402017-06-29T20:26:29  <BlueMatt> network-wide that is
7412017-06-29T20:26:39  <BlueMatt> though 0.15 may speed up block connection enough.....anyway
7422017-06-29T20:26:52  *** owowo has joined #bitcoin-core-dev
7432017-06-29T20:28:00  <instagibbs> BlueMatt, i was hoping for lazy improvements, like "it got better" :P
7442017-06-29T20:28:09  <instagibbs> yeah I reviewed a PR for that a long while ago
7452017-06-29T20:29:05  <sipa> cfields: so for batch validation... batch validate the txn in a block you haven't seen yet, and ignore the rest
7462017-06-29T20:29:15  <sipa> (assuming there is no block-wide anything going on)
7472017-06-29T20:29:38  <gmaxwell> BlueMatt: if you want to make that even faster: create a ReadBlockFromDisk that returns a blockblob (don't deseralize the transactions in it).
7482017-06-29T20:29:56  <gmaxwell> and use that for all getblocky kinds of requests.
7492017-06-29T20:30:39  <gmaxwell> (the cases not covered by the cached getblocktxn are whole block requests...)
7502017-06-29T20:30:51  <gmaxwell> (and we waste time deseralizing then reserializing the blocks...)
7512017-06-29T20:30:59  <cfields> sipa: ah, makes sense
7522017-06-29T20:31:37  <gmaxwell> cfields: there is a bit of a conflict right now because batch and parallel are in competition with each other... bigger batches get more speedup, but you want to use all your cores....
7532017-06-29T20:34:47  <cfields> is the batch operation itself not parallelizable?
7542017-06-29T20:34:55  <BlueMatt> gmaxwell: I'm less concerned about that w/ parallell ProcessMessages - if you ask for an old block its gonna be slow (though I agree that we should fix that, just less interesting for the latency improvements)
7552017-06-29T20:35:03  <BlueMatt> instagibbs: well that one got dropped in favor of "doing it cleanly"
7562017-06-29T20:35:15  <cfields> i suspect I'm misunderstanding the conflict
7572017-06-29T20:35:28  <BlueMatt> instagibbs: now its 10652 + the two other PRs that make up that branch, then an actual parallellization PR
7582017-06-29T20:35:33  <BlueMatt> so....16? maybe 17
7592017-06-29T20:35:39  <instagibbs> BlueMatt, ah k
7602017-06-29T20:35:53  <instagibbs> clearly behind the times
7612017-06-29T20:38:20  <sipa> cfields: a batch of 4000 signatures takes less than 8 times as much CPU as a batch of 500 signatures
7622017-06-29T20:38:46  <sipa> cfields: if you split the batch up in 8, and run those 8 on separate CPUs, you're going to do 8*batch(500) work, not 1*batch(4000) work
7632017-06-29T20:39:26  <gmaxwell> cfields: the algorithim is not naturally parallizable, though with lots of synchronization traffic it can be made parallel.
7642017-06-29T20:40:00  <gmaxwell> how much of the batch(4000) speed we can get out of something pushed to be made N-way parallel is an open question.
7652017-06-29T20:40:12  <gmaxwell> If synchronization between threads is free the answer is "almost all of it"
7662017-06-29T20:40:23  <cfields> ok that's what i was missing, thanks
7672017-06-29T20:40:27  <gmaxwell> If it is very expensive, the answer appears to be "almost none of it".
7682017-06-29T20:40:31  *** Yogaqueef has quit IRC
7692017-06-29T20:40:56  <cfields> i'll read the paper before discussing further
7702017-06-29T20:41:01  *** Murch has quit IRC
7712017-06-29T20:41:07  <gmaxwell> None of this is in the paper.
7722017-06-29T20:41:41  <cfields> oh. in that case, I already read the paper :p
7732017-06-29T20:41:47  <sipa> cfields: the 'hard' part of the computation is doing a huge n1*P1 + n2*P2 + n3*P3 + ... (where the n's are integers and the Ps are EC points)
7742017-06-29T20:42:21  <sipa> turns out, there is a very neat algorithm (multiple, in fact) that do this whole computation many times faster than just multiplying individually and adding
7752017-06-29T20:42:57  <gmaxwell> Same as for a normal signature validation except there it's  just n1 * P + n2 * G  ... so only two ecpoints.  in batch and aggregate validation there is  pubkeys + 1.
7762017-06-29T20:46:36  <gmaxwell> done simply, a n1*P requires 256 EC additions (technically 256 doublings and 128 additions, but doublings are about twice as fast as an addition)-- using grade school long multiplication (in base 2).  The batch computation of n1*P1 + n2*P2 + n3*P3 + ... can do the job in about 26 additions per point for 4096 inputs.
7772017-06-29T20:46:46  *** Murch has joined #bitcoin-core-dev
7782017-06-29T20:47:10  <cfields> whoa
7792017-06-29T20:47:28  <cfields> oh, misread :)
7802017-06-29T20:47:56  <gmaxwell> yes per point, but still thats almost a 10x speedup over a dumb algorithim.
7812017-06-29T20:50:24  <cfields> are there further speedups if the result is known ahead of time and you're just attempting to verify correctness?
7822017-06-29T20:50:56  <gmaxwell> What we do in secp256k1 for validation (which is two points) is far from simplisic and takes much less than 256 adds worth of work per each.  I believe it's about equal to about 84 per point.
7832017-06-29T20:51:19  <gmaxwell> cfields: the aggregate and batch both count on a property like that.
7842017-06-29T20:51:40  <gmaxwell> the R value in the signature is the result of this calculation.
7852017-06-29T20:52:08  <cfields> ah, so that's the 32bytes-per-block
7862017-06-29T20:54:01  <sipa> cfields: eh, i think you're confused
7872017-06-29T20:54:07  <sipa> cfields: perhaps we should move to #secp256k1
7882017-06-29T20:54:22  <cfields> sure
7892017-06-29T20:55:14  <gmaxwell> To be clear: Batch validation exploits ' the result is known ahead of time and you're just attempting to verify correctness'   --- because the signature (or each signature) has an R value that comes with it, and the signature validation is trying to verify that an R value it computes is the same as the provided one.
7902017-06-29T20:56:22  <gmaxwell> You can also encode signatures another way, like the wikipedia article on schnorr signatures does-- using "e,s" which is a hash and a scalar; and this form cannot be batch verified because you don't know the result of that multiexp equation in advance.
7912017-06-29T20:58:36  *** DrOlmer has quit IRC
7922017-06-29T21:07:52  *** awsom82 has joined #bitcoin-core-dev
7932017-06-29T21:36:58  *** Chris_Stewart_5 has quit IRC
7942017-06-29T21:39:41  *** laurentmt has joined #bitcoin-core-dev
7952017-06-29T21:41:54  *** laurentmt has quit IRC
7962017-06-29T21:53:36  <cfields> BlueMatt: ok, i have the shared_ptr change hacked in, and it's pretty huge. Lots of stuff has to change at the same time... it's kinda hard to avoid a giant commit.
7972017-06-29T21:54:15  <cfields> BlueMatt: i have no problem doing your refcount change first, then undoing it with this big change next if you'd prefer.
7982017-06-29T22:07:39  *** justan0theruser has joined #bitcoin-core-dev
7992017-06-29T22:10:40  *** justanotheruser has quit IRC
8002017-06-29T22:15:27  *** vicenteH has quit IRC
8012017-06-29T22:22:40  *** AaronvanW has quit IRC
8022017-06-29T22:27:16  *** AaronvanW has joined #bitcoin-core-dev
8032017-06-29T22:41:33  *** justan0theruser has quit IRC
8042017-06-29T22:41:51  *** justanotheruser has joined #bitcoin-core-dev
8052017-06-29T22:42:38  *** Murch has quit IRC
8062017-06-29T22:44:11  *** Murch has joined #bitcoin-core-dev
8072017-06-29T22:50:21  *** nemgun has quit IRC
8082017-06-29T23:01:03  *** awsom82 has left #bitcoin-core-dev
8092017-06-29T23:22:17  *** jimhash has joined #bitcoin-core-dev
8102017-06-29T23:23:27  *** rabidus has quit IRC
8112017-06-29T23:25:23  *** rabidus has joined #bitcoin-core-dev
8122017-06-29T23:28:29  *** Dizzle has quit IRC
8132017-06-29T23:41:37  *** jimhash has left #bitcoin-core-dev
8142017-06-29T23:42:35  *** Giszmo has quit IRC
8152017-06-29T23:57:33  *** Giszmo has joined #bitcoin-core-dev