12017-11-27T00:01:45  *** daedal_ has joined #bitcoin-core-dev
  22017-11-27T00:01:52  *** qrest has quit IRC
  32017-11-27T00:05:35  *** daedal__ has quit IRC
  42017-11-27T00:06:33  *** AaronvanW has joined #bitcoin-core-dev
  52017-11-27T00:08:45  *** intcat has quit IRC
  62017-11-27T00:08:54  *** Aaronvan_ has joined #bitcoin-core-dev
  72017-11-27T00:09:37  *** intcat has joined #bitcoin-core-dev
  82017-11-27T00:12:27  *** AaronvanW has quit IRC
  92017-11-27T00:17:56  *** wxss has quit IRC
 102017-11-27T00:20:42  *** qrest has joined #bitcoin-core-dev
 112017-11-27T00:26:50  *** promag has quit IRC
 122017-11-27T00:27:59  *** intcat has quit IRC
 132017-11-27T00:28:59  *** intcat has joined #bitcoin-core-dev
 142017-11-27T00:30:40  *** pandabull has joined #bitcoin-core-dev
 152017-11-27T00:31:22  *** pandabull has quit IRC
 162017-11-27T00:33:24  *** JamesAU has joined #bitcoin-core-dev
 172017-11-27T00:35:04  *** JamesAU has quit IRC
 182017-11-27T00:35:27  *** JamesAU has joined #bitcoin-core-dev
 192017-11-27T00:35:29  *** Chris_Stewart_5 has quit IRC
 202017-11-27T00:35:43  *** intcat has quit IRC
 212017-11-27T00:36:26  *** intcat has joined #bitcoin-core-dev
 222017-11-27T00:43:46  *** cyber55 has joined #bitcoin-core-dev
 232017-11-27T00:48:23  *** karelb has quit IRC
 242017-11-27T00:51:29  *** jtimon has quit IRC
 252017-11-27T00:53:32  *** jb55 has joined #bitcoin-core-dev
 262017-11-27T00:59:21  *** dabura667 has joined #bitcoin-core-dev
 272017-11-27T01:03:26  *** jb55 has quit IRC
 282017-11-27T01:05:45  *** Randolf has joined #bitcoin-core-dev
 292017-11-27T01:09:57  *** Randolf has quit IRC
 302017-11-27T01:10:32  *** Randolf has joined #bitcoin-core-dev
 312017-11-27T01:17:06  *** Victorsueca has quit IRC
 322017-11-27T01:25:05  *** Victorsueca has joined #bitcoin-core-dev
 332017-11-27T01:29:35  *** Randolf has quit IRC
 342017-11-27T01:43:47  *** satwo has quit IRC
 352017-11-27T01:53:01  *** Emcy_ has joined #bitcoin-core-dev
 362017-11-27T01:54:53  *** Emcy has quit IRC
 372017-11-27T01:55:44  *** wunpunch has quit IRC
 382017-11-27T01:56:30  *** quantbot has joined #bitcoin-core-dev
 392017-11-27T01:57:05  *** quantbot has joined #bitcoin-core-dev
 402017-11-27T01:57:25  *** quantbot has joined #bitcoin-core-dev
 412017-11-27T01:58:05  *** quantbot has joined #bitcoin-core-dev
 422017-11-27T02:01:45  *** l3gion has quit IRC
 432017-11-27T02:01:58  *** Emcy has joined #bitcoin-core-dev
 442017-11-27T02:02:39  *** quantbot has quit IRC
 452017-11-27T02:03:11  *** Emcy_ has quit IRC
 462017-11-27T02:04:46  *** justanotheruser has joined #bitcoin-core-dev
 472017-11-27T02:06:47  *** daedal__ has joined #bitcoin-core-dev
 482017-11-27T02:09:41  *** daedal_ has quit IRC
 492017-11-27T02:13:50  *** justanotheruser has quit IRC
 502017-11-27T02:14:07  *** justanotheruser has joined #bitcoin-core-dev
 512017-11-27T02:15:52  *** satwo has joined #bitcoin-core-dev
 522017-11-27T02:15:57  *** Ylbam has quit IRC
 532017-11-27T02:21:51  *** quantbot has joined #bitcoin-core-dev
 542017-11-27T02:22:35  *** quantbot has quit IRC
 552017-11-27T02:26:05  *** StopAndDecrypt has quit IRC
 562017-11-27T02:26:38  *** StopAndDecrypt has joined #bitcoin-core-dev
 572017-11-27T02:26:38  *** StopAndDecrypt has quit IRC
 582017-11-27T02:26:38  *** StopAndDecrypt has joined #bitcoin-core-dev
 592017-11-27T02:46:23  *** go1111111 has quit IRC
 602017-11-27T02:46:27  *** go11111111111 has quit IRC
 612017-11-27T02:49:47  *** meshcollider has quit IRC
 622017-11-27T02:59:21  *** jcorgan has quit IRC
 632017-11-27T02:59:51  *** Aaronvan_ has quit IRC
 642017-11-27T03:00:30  *** AaronvanW has joined #bitcoin-core-dev
 652017-11-27T03:03:13  *** jcorgan has joined #bitcoin-core-dev
 662017-11-27T03:04:41  *** AaronvanW has quit IRC
 672017-11-27T03:05:03  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 682017-11-27T03:06:41  *** Emcy_ has joined #bitcoin-core-dev
 692017-11-27T03:08:50  *** pbase has joined #bitcoin-core-dev
 702017-11-27T03:08:53  *** Emcy has quit IRC
 712017-11-27T03:17:03  *** meshcollider has joined #bitcoin-core-dev
 722017-11-27T03:22:02  *** dabura667 has quit IRC
 732017-11-27T03:32:27  *** Chris_Stewart_5 has quit IRC
 742017-11-27T03:38:52  *** Randolf has joined #bitcoin-core-dev
 752017-11-27T03:51:26  *** quantbot has joined #bitcoin-core-dev
 762017-11-27T03:58:54  *** daedal__ has quit IRC
 772017-11-27T03:59:18  *** AaronvanW has joined #bitcoin-core-dev
 782017-11-27T04:04:15  *** qrest has quit IRC
 792017-11-27T04:05:26  *** AaronvanW has quit IRC
 802017-11-27T04:09:13  *** qrestlove has joined #bitcoin-core-dev
 812017-11-27T04:37:15  *** goatpig has quit IRC
 822017-11-27T04:38:09  *** digifis has joined #bitcoin-core-dev
 832017-11-27T04:41:03  *** sin_ has quit IRC
 842017-11-27T04:43:45  *** pbase has quit IRC
 852017-11-27T04:55:35  *** jack__ has joined #bitcoin-core-dev
 862017-11-27T05:03:49  *** hoge_ has joined #bitcoin-core-dev
 872017-11-27T05:12:15  *** qrestlove has quit IRC
 882017-11-27T05:15:09  *** musalbas has quit IRC
 892017-11-27T05:20:01  *** plazar has joined #bitcoin-core-dev
 902017-11-27T05:23:58  *** qrestlove has joined #bitcoin-core-dev
 912017-11-27T05:24:17  *** plazar has quit IRC
 922017-11-27T05:25:26  *** plazar has joined #bitcoin-core-dev
 932017-11-27T05:28:39  *** plazar has joined #bitcoin-core-dev
 942017-11-27T05:29:48  *** meshcollider has quit IRC
 952017-11-27T05:33:37  *** plazar has quit IRC
 962017-11-27T05:37:31  *** plazar has joined #bitcoin-core-dev
 972017-11-27T05:40:25  *** plazar has quit IRC
 982017-11-27T05:40:50  *** plazar has joined #bitcoin-core-dev
 992017-11-27T05:43:21  *** plazar has quit IRC
1002017-11-27T05:43:40  *** plazar has joined #bitcoin-core-dev
1012017-11-27T05:46:57  *** plazar has quit IRC
1022017-11-27T05:47:11  *** plazar has joined #bitcoin-core-dev
1032017-11-27T05:48:59  *** plazar has quit IRC
1042017-11-27T05:49:16  *** hellxcode has joined #bitcoin-core-dev
1052017-11-27T06:08:55  *** jack__ has quit IRC
1062017-11-27T06:13:01  *** hoge_ has quit IRC
1072017-11-27T06:21:17  *** hoge_ has joined #bitcoin-core-dev
1082017-11-27T06:21:33  *** hoge_ has joined #bitcoin-core-dev
1092017-11-27T06:23:54  *** jon888 has joined #bitcoin-core-dev
1102017-11-27T06:26:09  *** hellxcode has quit IRC
1112017-11-27T06:26:42  *** AaronvanW has joined #bitcoin-core-dev
1122017-11-27T06:28:43  *** hellxcode has joined #bitcoin-core-dev
1132017-11-27T06:29:01  *** sanjeev has quit IRC
1142017-11-27T06:33:54  *** esotericnonsense has quit IRC
1152017-11-27T06:41:23  *** Ylbam has joined #bitcoin-core-dev
1162017-11-27T06:45:06  *** hellxcode has quit IRC
1172017-11-27T07:03:15  *** sanjeev has joined #bitcoin-core-dev
1182017-11-27T07:04:28  *** sanjeev_ has joined #bitcoin-core-dev
1192017-11-27T07:05:12  *** sanjeev has quit IRC
1202017-11-27T07:05:32  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
1212017-11-27T07:05:32  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
1222017-11-27T07:09:56  *** hoge_ has quit IRC
1232017-11-27T07:12:20  *** hoge_ has joined #bitcoin-core-dev
1242017-11-27T07:12:38  *** StopAndDecrypt has quit IRC
1252017-11-27T07:13:03  *** StopAndDecrypt has joined #bitcoin-core-dev
1262017-11-27T07:13:03  *** StopAndDecrypt has quit IRC
1272017-11-27T07:13:03  *** StopAndDecrypt has joined #bitcoin-core-dev
1282017-11-27T07:16:35  *** hoge_ has quit IRC
1292017-11-27T07:18:54  *** JamesAU has quit IRC
1302017-11-27T07:22:58  *** hoge_ has joined #bitcoin-core-dev
1312017-11-27T07:24:19  *** JamesAU has joined #bitcoin-core-dev
1322017-11-27T07:29:10  *** JamesAU has quit IRC
1332017-11-27T07:34:52  *** dabura667 has joined #bitcoin-core-dev
1342017-11-27T07:50:37  *** jon888 has left #bitcoin-core-dev
1352017-11-27T07:51:01  *** BashCo has quit IRC
1362017-11-27T08:05:42  *** JamesAU has joined #bitcoin-core-dev
1372017-11-27T08:11:12  *** digifis has quit IRC
1382017-11-27T08:12:29  *** BashCo has joined #bitcoin-core-dev
1392017-11-27T08:16:02  *** nickler has quit IRC
1402017-11-27T08:16:31  *** promag has joined #bitcoin-core-dev
1412017-11-27T08:17:06  *** nickler has joined #bitcoin-core-dev
1422017-11-27T08:19:21  *** promag has quit IRC
1432017-11-27T08:43:46  *** Cogito_Ergo_Sum has quit IRC
1442017-11-27T08:45:06  *** sanjeev_ has quit IRC
1452017-11-27T08:52:27  *** sanjeev has joined #bitcoin-core-dev
1462017-11-27T09:09:48  *** atroxes has quit IRC
1472017-11-27T09:14:45  *** timothy has joined #bitcoin-core-dev
1482017-11-27T09:21:11  *** laurentmt has joined #bitcoin-core-dev
1492017-11-27T09:23:39  *** laurentmt has quit IRC
1502017-11-27T09:25:31  *** hoge_ has quit IRC
1512017-11-27T09:29:21  *** promag has joined #bitcoin-core-dev
1522017-11-27T09:41:45  *** promag has quit IRC
1532017-11-27T09:43:00  *** atroxes has joined #bitcoin-core-dev
1542017-11-27T09:57:58  *** Guyver2 has joined #bitcoin-core-dev
1552017-11-27T10:05:34  *** Ylbam has quit IRC
1562017-11-27T10:11:21  *** Ayana59Smitham has quit IRC
1572017-11-27T10:24:27  *** promag has joined #bitcoin-core-dev
1582017-11-27T10:27:10  *** promag has quit IRC
1592017-11-27T10:35:44  *** jtimon has joined #bitcoin-core-dev
1602017-11-27T10:45:52  <wallet42> What are the counter arguments to have a switch where the miner can set a couple of vbytes (10-50K) aside in a getblocktemplate that will include lowfee, oldest first transactions instead of highest paying?
1612017-11-27T10:59:35  *** AndyS2 has joined #bitcoin-core-dev
1622017-11-27T11:01:33  *** AaronvanW has quit IRC
1632017-11-27T11:01:54  *** AaronvanW has joined #bitcoin-core-dev
1642017-11-27T11:08:40  <sipa> wallet42: complicates the block construction code significantly, would likely not be used by a significant portion of the hashrate
1652017-11-27T11:09:46  <sipa> further, having it available even if it was used wouldn't really benefit anyone; it's essentially introducing a lottery in the mining construction from the perspective of wallets
1662017-11-27T11:13:20  <Provoostenator> I was thinking about creating an RPC method to add a new destination + amount to an existing RBF transaction. Or to add a "append-tx" argument to sendtoaddress. Has anyone worked on that in past?
1672017-11-27T11:13:51  <Provoostenator> (or explained why that's a terrible idea)
1682017-11-27T11:14:13  <sipa> Provoostenator: i've thought a bit about a more invasive idea that's related
1692017-11-27T11:14:51  <sipa> i think it would be nice to have in the wallet, independent from the sendtoaddress/sendmany/... RPCs, a list of "outputs i would like to pay to" (with address/amount)
1702017-11-27T11:15:21  <sipa> and any time that list is changed, or a transaction confirms, a new single transaction is constructed that pays out to all the entries in that list that haven't been paid to already
1712017-11-27T11:15:42  <Provoostenator> And then have the wallet figure out if a new transaction is cheaper than amending one?
1722017-11-27T11:15:57  <sipa> no, there is always just a single outstanding unconfirmed transaction
1732017-11-27T11:16:07  <sipa> for any change, that transaction is modified
1742017-11-27T11:16:16  <Provoostenator> That makes sense.
1752017-11-27T11:16:33  <sipa> the cost of bumping for RBF reasons is usually trivial compared to its actual fee
1762017-11-27T11:16:47  <sipa> (just add 1sat/vbyte)
1772017-11-27T11:17:14  <Provoostenator> Right, unless it's a very large transaction. But that would be trivial to check.
1782017-11-27T11:17:18  <sipa> there are complications to this... for example you need coin selection that guarantees conflicts with all previously attempted unconfirmed transaction
1792017-11-27T11:17:47  <sipa> or you may need to pre-sign multiple candidate follow-up transactions based on which of your earlier attempts confirms
1802017-11-27T11:17:59  <sipa> (as you can't automatically get access to encrypted keys later)
1812017-11-27T11:18:27  <sipa> and it likely conflicts with how people actually use bitcoin in the real world now - where they expect a txid to be something you can track
1822017-11-27T11:18:28  <Provoostenator> Oh, so in that case you'd use SIGHASH flags?
1832017-11-27T11:18:58  <sipa> just predict the possible sceniarios, and sign a full follow-up transaction for each (or at least for the few most likely ones)
1842017-11-27T11:22:48  *** qrestlove has quit IRC
1852017-11-27T11:22:48  <aj> sipa, wallet42: can't you just use prioritisetransaction rpc on the 50 oldest/whatever transactions to get that behaviour anyway?
1862017-11-27T11:23:25  <Provoostenator> Although it sounds more elegant, I don't think I can pull that off, yet :-)
1872017-11-27T11:24:14  <sipa> aj: yes
1882017-11-27T11:24:37  <aj> sipa: predict the possible scenarios? should be called fund-xanatos-transaction...
1892017-11-27T11:24:48  <sipa> xanatos?
1902017-11-27T11:25:04  <aj> http://tvtropes.org/pmwiki/pmwiki.php/Main/XanatosGambit
1912017-11-27T11:27:44  <sipa> Provoostenator: well i think this would be pretty much independent from all existing transaction logic
1922017-11-27T11:27:56  <sipa> it'd reuse coin selection, but that's it :)
1932017-11-27T11:29:25  *** mumux has joined #bitcoin-core-dev
1942017-11-27T11:30:26  *** mumux has joined #bitcoin-core-dev
1952017-11-27T11:30:48  <wallet42> sipa: are the wallet RPC capabilities currently enough to pull of something like that with an external python app?
1962017-11-27T11:31:12  <wallet42> As a prototype
1972017-11-27T11:31:54  <sipa> yes, just use prioritisetransaction
1982017-11-27T11:32:02  <sipa> and getrawmempool
1992017-11-27T11:35:34  *** meshcollider has joined #bitcoin-core-dev
2002017-11-27T11:40:26  *** StopAndDecrypt has quit IRC
2012017-11-27T11:41:10  *** StopAndDecrypt has joined #bitcoin-core-dev
2022017-11-27T11:41:10  *** StopAndDecrypt has quit IRC
2032017-11-27T11:41:10  *** StopAndDecrypt has joined #bitcoin-core-dev
2042017-11-27T11:41:18  *** qrestlove has joined #bitcoin-core-dev
2052017-11-27T11:42:19  <sipa> ugh, master allocates enormous amounts of memory for me while syncing from scratch from network
2062017-11-27T11:42:38  <sipa> 0.15.1 with the same configuration works fine
2072017-11-27T11:43:02  <sipa> (it made my system with 32G RAM OOM)
2082017-11-27T11:44:28  <Provoostenator> sipa: any way Travis could catch something like that?
2092017-11-27T11:44:46  <Provoostenator> Or does it take too long before memory usage gets out of hand?
2102017-11-27T11:45:06  <sipa> it OOMed when my debug.log said the dbcache was 1.7G
2112017-11-27T11:45:17  <sipa> which is substantial but not excessive
2122017-11-27T11:45:31  <sipa> and the overhead on top of what the dbcache predicts should only be a few 100 MB
2132017-11-27T11:51:45  *** JamesAU has quit IRC
2142017-11-27T11:52:17  *** JamesAU has joined #bitcoin-core-dev
2152017-11-27T11:53:53  *** aspect_ has quit IRC
2162017-11-27T11:53:53  *** Muis has quit IRC
2172017-11-27T11:54:22  *** michagogo has quit IRC
2182017-11-27T11:54:32  *** hsmiths has quit IRC
2192017-11-27T11:54:33  *** aspect_ has joined #bitcoin-core-dev
2202017-11-27T11:54:44  *** jl2012 has quit IRC
2212017-11-27T11:54:58  *** nejon has quit IRC
2222017-11-27T11:55:38  *** Muis has joined #bitcoin-core-dev
2232017-11-27T11:55:38  *** nejon has joined #bitcoin-core-dev
2242017-11-27T11:55:55  *** michagogo has joined #bitcoin-core-dev
2252017-11-27T11:56:49  *** JamesAU has quit IRC
2262017-11-27T11:56:57  *** lifeofguenter has quit IRC
2272017-11-27T11:58:48  *** cyber55 has quit IRC
2282017-11-27T11:59:06  *** cyber55 has joined #bitcoin-core-dev
2292017-11-27T11:59:20  *** cyber55 has quit IRC
2302017-11-27T11:59:40  *** lifeofguenter has joined #bitcoin-core-dev
2312017-11-27T12:00:33  *** go1111111 has joined #bitcoin-core-dev
2322017-11-27T12:09:00  *** wxss has joined #bitcoin-core-dev
2332017-11-27T12:09:23  *** JamesAU has joined #bitcoin-core-dev
2342017-11-27T12:10:08  *** AaronvanW has quit IRC
2352017-11-27T12:11:05  *** isis has quit IRC
2362017-11-27T12:11:50  *** isis has joined #bitcoin-core-dev
2372017-11-27T12:13:29  *** nickler has quit IRC
2382017-11-27T12:14:00  *** nickler has joined #bitcoin-core-dev
2392017-11-27T12:14:10  *** JamesAU has quit IRC
2402017-11-27T12:18:28  *** dabura667 has quit IRC
2412017-11-27T12:19:22  *** JamesAU has joined #bitcoin-core-dev
2422017-11-27T12:24:04  <Provoostenator> sipa: the second log link in the channel topic seems a bit stale...
2432017-11-27T12:24:33  <aj> Provoostenator: yeah, the cron job broke and i haven't updated it since botbot seems better
2442017-11-27T12:28:19  *** JamesAU has quit IRC
2452017-11-27T12:32:53  <Provoostenator> Any idea why "test/functional/test_runner.py --loglevel=INFO bumpfee" doesn't result in "Mining blocks..." appearing my console?
2462017-11-27T12:33:09  <Provoostenator> It should according to: https://github.com/bitcoin/bitcoin/blob/master/test/functional/bumpfee.py#L52
2472017-11-27T12:34:54  *** quantbot has quit IRC
2482017-11-27T12:35:30  *** quantbot has joined #bitcoin-core-dev
2492017-11-27T12:38:07  <aj> Provoostenator: okay cron job fixed, that wasn't so hard
2502017-11-27T12:38:32  *** AaronvanW has joined #bitcoin-core-dev
2512017-11-27T12:39:17  <Provoostenator> aj: nice, that seems to work
2522017-11-27T12:39:59  *** quantbot has quit IRC
2532017-11-27T12:41:38  *** Chris_Stewart_5 has joined #bitcoin-core-dev
2542017-11-27T12:42:13  <Provoostenator> Is anyone timestamping them?
2552017-11-27T12:48:58  <aj> Provoostenator: each line has a timestamp :) but if you mean cryptographically, no
2562017-11-27T12:49:02  *** JamesAU has joined #bitcoin-core-dev
2572017-11-27T12:53:27  *** JamesAU has quit IRC
2582017-11-27T13:09:37  *** JamesAU has joined #bitcoin-core-dev
2592017-11-27T13:14:07  *** JamesAU has quit IRC
2602017-11-27T13:16:54  <morcos> sipa: that sounds bad
2612017-11-27T13:17:10  <morcos> re: wallet replacement, i think privacy concerns would also be substantial
2622017-11-27T13:17:56  <morcos> you can assume chain analysis companies would start tracking unconfirmed txs, and use that to further group outputs as related
2632017-11-27T13:19:20  *** JamesAU has joined #bitcoin-core-dev
2642017-11-27T13:20:59  <Provoostenator> That's one reason I would make this RPC-only initially.
2652017-11-27T13:22:25  <Provoostenator> morcos: but does it really matter much privacy wise if you spend the next transaction from your change or by amending the original transaction?
2662017-11-27T13:23:52  *** JamesAU has quit IRC
2672017-11-27T13:24:27  *** JamesAU has joined #bitcoin-core-dev
2682017-11-27T13:26:29  <morcos> Provoostenator: well if you spend from your change, it MAY be unclear that it was your change or the person you paid spending (agreed that right now change is probably pretty easily identifiable though)
2692017-11-27T13:26:55  <morcos> also if you spend unconfirmed, then you have a good point
2702017-11-27T13:28:50  *** JamesAU has quit IRC
2712017-11-27T13:28:57  <Provoostenator> I can see a use case for low priority and smaller transactions, e.g. paying back friends, where you intentionally set the fee too low, so you can keep amending it. A poor mans payment channel. Obviously there's a privacy cost.
2722017-11-27T13:29:27  <morcos> or rich mans payment channel as the case may be.  :)
2732017-11-27T13:29:42  *** Cheeseo has joined #bitcoin-core-dev
2742017-11-27T13:31:01  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
2752017-11-27T13:31:01  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
2762017-11-27T13:33:06  <Provoostenator> "TestFramework (INFO): Mining blocks..." does show up in the test log file.
2772017-11-27T13:35:38  *** hoge_ has joined #bitcoin-core-dev
2782017-11-27T13:35:59  <Provoostenator> Even stranger given that the default is INFO.
2792017-11-27T13:43:43  <Provoostenator> Ah, so I need to call "test/functional/bumpfee.py --loglevel=INFO" for this to work.
2802017-11-27T13:49:41  *** Chris_Stewart_5 has quit IRC
2812017-11-27T13:50:56  *** jl2012 has joined #bitcoin-core-dev
2822017-11-27T13:51:29  *** StopAndDecrypt has quit IRC
2832017-11-27T13:51:47  *** hsmiths has joined #bitcoin-core-dev
2842017-11-27T13:52:28  *** StopAndDecrypt has joined #bitcoin-core-dev
2852017-11-27T13:52:29  *** StopAndDecrypt has quit IRC
2862017-11-27T13:52:29  *** StopAndDecrypt has joined #bitcoin-core-dev
2872017-11-27T13:55:22  *** meshcollider has quit IRC
2882017-11-27T14:00:48  *** kbgg_ has joined #bitcoin-core-dev
2892017-11-27T14:03:38  *** Chris_Stewart_5 has joined #bitcoin-core-dev
2902017-11-27T14:06:34  *** goatpig has joined #bitcoin-core-dev
2912017-11-27T14:12:22  *** Chris_Stewart_5 has quit IRC
2922017-11-27T14:12:47  *** SopaXorzTaker has joined #bitcoin-core-dev
2932017-11-27T14:13:09  *** hoge_ has quit IRC
2942017-11-27T14:19:03  *** quantbot has joined #bitcoin-core-dev
2952017-11-27T14:19:10  *** JamesAU_ has joined #bitcoin-core-dev
2962017-11-27T14:20:10  *** quantbot has quit IRC
2972017-11-27T14:20:34  *** Cogito_Ergo_Sum has quit IRC
2982017-11-27T14:20:36  *** quantbot has joined #bitcoin-core-dev
2992017-11-27T14:23:27  *** JamesAU_ has quit IRC
3002017-11-27T14:24:16  *** JamesAU_ has joined #bitcoin-core-dev
3012017-11-27T14:25:54  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
3022017-11-27T14:25:54  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
3032017-11-27T14:29:26  *** JamesAU_ has quit IRC
3042017-11-27T14:32:02  *** SopaXorzTaker has quit IRC
3052017-11-27T14:32:38  *** SopaXorzTaker has joined #bitcoin-core-dev
3062017-11-27T14:35:13  *** sanjeev has quit IRC
3072017-11-27T14:36:13  *** promag has joined #bitcoin-core-dev
3082017-11-27T14:48:30  <bitcoin-git> [bitcoin] joemphilips opened pull request #11770: [REST] add a rest endpoint for estimatesmartfee, docs, and test (master...rest_fee) https://github.com/bitcoin/bitcoin/pull/11770
3092017-11-27T14:55:36  *** Chris_Stewart_5 has joined #bitcoin-core-dev
3102017-11-27T15:00:58  *** Chris_Stewart_5 has quit IRC
3112017-11-27T15:01:54  *** Chris_Stewart_5 has joined #bitcoin-core-dev
3122017-11-27T15:27:27  *** Randolf has quit IRC
3132017-11-27T15:35:50  *** StopAndDecrypt_ has joined #bitcoin-core-dev
3142017-11-27T15:36:01  *** StopAndDecrypt has quit IRC
3152017-11-27T15:52:40  *** archaeal_ has joined #bitcoin-core-dev
3162017-11-27T15:53:07  *** tim has joined #bitcoin-core-dev
3172017-11-27T15:53:55  *** tim has quit IRC
3182017-11-27T15:53:57  *** dcousens has quit IRC
3192017-11-27T15:54:58  *** dcousens has joined #bitcoin-core-dev
3202017-11-27T15:55:03  *** StopAndDecrypt_ has quit IRC
3212017-11-27T15:55:23  *** StopAndDecrypt has joined #bitcoin-core-dev
3222017-11-27T15:55:23  *** StopAndDecrypt has quit IRC
3232017-11-27T15:55:23  *** StopAndDecrypt has joined #bitcoin-core-dev
3242017-11-27T16:00:50  *** cheese_ has joined #bitcoin-core-dev
3252017-11-27T16:04:51  <promag> archaeal_: it it was me, I would have the PR title: Add REST endpoint blockhash
3262017-11-27T16:05:11  *** RoyceX has joined #bitcoin-core-dev
3272017-11-27T16:05:31  <promag> then one commit for the implementation, one for the test and other for the doc
3282017-11-27T16:05:41  *** satwo has quit IRC
3292017-11-27T16:06:29  <promag> ^ 11765
3302017-11-27T16:07:06  <archaeal_> i see...yeah that makes sense
3312017-11-27T16:07:57  *** cheese_ has quit IRC
3322017-11-27T16:09:05  <Lauda> sipa I can confirm very high memory usage. I had a fresh sync (albeit a fork of 0.15.x) and it was using over 6 GB with 4GB dbcache and swap was 100% utilized.
3332017-11-27T16:18:41  *** RoyceX has quit IRC
3342017-11-27T16:25:16  *** zaxar has joined #bitcoin-core-dev
3352017-11-27T16:25:48  *** zaxar has quit IRC
3362017-11-27T16:31:31  *** zaxaar has joined #bitcoin-core-dev
3372017-11-27T16:33:51  *** zaxaar has quit IRC
3382017-11-27T16:33:58  *** RoyceX has joined #bitcoin-core-dev
3392017-11-27T16:33:59  *** DvdKhl has joined #bitcoin-core-dev
3402017-11-27T16:35:32  <bitcoin-git> [bitcoin] jnewbery opened pull request #11771:  [tests] Change invalidtxrequest to use BitcoinTestFramework (master...refactor_invalidtxrequest) https://github.com/bitcoin/bitcoin/pull/11771
3412017-11-27T16:35:57  *** Randolf has joined #bitcoin-core-dev
3422017-11-27T16:36:42  *** BashCo has quit IRC
3432017-11-27T16:36:50  *** shesek has quit IRC
3442017-11-27T16:37:18  *** BashCo has joined #bitcoin-core-dev
3452017-11-27T16:37:30  *** cheese_ has joined #bitcoin-core-dev
3462017-11-27T16:37:37  *** promag has quit IRC
3472017-11-27T16:40:12  *** RoyceX has quit IRC
3482017-11-27T16:40:50  *** Ylbam has joined #bitcoin-core-dev
3492017-11-27T16:41:32  *** BashCo has quit IRC
3502017-11-27T16:43:24  *** satwo has joined #bitcoin-core-dev
3512017-11-27T16:50:09  *** cheese_ has quit IRC
3522017-11-27T16:54:27  *** Randolf has quit IRC
3532017-11-27T17:01:22  *** Emcy has joined #bitcoin-core-dev
3542017-11-27T17:02:58  *** Emcy_ has quit IRC
3552017-11-27T17:05:18  *** Khunbish has joined #bitcoin-core-dev
3562017-11-27T17:05:58  *** Khunbish has quit IRC
3572017-11-27T17:09:49  *** JackH has joined #bitcoin-core-dev
3582017-11-27T17:11:31  *** cheese_ has joined #bitcoin-core-dev
3592017-11-27T17:11:36  *** musalbas has joined #bitcoin-core-dev
3602017-11-27T17:18:49  *** Guyver2 has quit IRC
3612017-11-27T17:21:09  *** Randolf has joined #bitcoin-core-dev
3622017-11-27T17:22:41  *** cheese_ has quit IRC
3632017-11-27T17:24:39  *** BashCo has joined #bitcoin-core-dev
3642017-11-27T17:25:27  *** Randolf has quit IRC
3652017-11-27T17:26:31  *** jb55 has joined #bitcoin-core-dev
3662017-11-27T17:30:19  *** dqx_ has joined #bitcoin-core-dev
3672017-11-27T17:31:12  *** JackH has quit IRC
3682017-11-27T17:33:15  *** Chasity76Ortiz has joined #bitcoin-core-dev
3692017-11-27T17:37:31  <bitcoin-git> [bitcoin] jnewbery closed pull request #11047: [tests] rename functional tests (master...rename_functional_tests) https://github.com/bitcoin/bitcoin/pull/11047
3702017-11-27T17:40:20  *** Emcy_ has joined #bitcoin-core-dev
3712017-11-27T17:41:35  *** Emcy has quit IRC
3722017-11-27T17:41:50  *** dqx_ has quit IRC
3732017-11-27T17:42:40  *** archaeal_ has quit IRC
3742017-11-27T17:42:46  <morcos> sipa: have a second to talk about SW wallet?
3752017-11-27T17:42:52  *** dqx_ has joined #bitcoin-core-dev
3762017-11-27T17:43:37  <morcos> i thought one of the cases we wanted to cover (which is mostly a version of 2 i think) is that you have a backup of a 0.15.0 wallet now, and you addwitnessaddress, but then later you have to restore from backup
3772017-11-27T17:43:49  <morcos> this is what the 2(a) case was meant to handle
3782017-11-27T17:43:59  *** Giszmo has quit IRC
3792017-11-27T17:44:32  *** dqx_ has quit IRC
3802017-11-27T17:44:43  *** dqx_ has joined #bitcoin-core-dev
3812017-11-27T17:44:46  <morcos> i dont think i particularly care if we don't handle that in 0.16, but it does seem like at some point we should handle that...  with SW out, people are going to be doing that work flow.
3822017-11-27T17:45:19  *** Giszmo has joined #bitcoin-core-dev
3832017-11-27T17:46:31  <morcos> never mind, i didn't read carefully enough..  the 1(a) case handles that if you are running 0.16, its only the downgrade case that isn't handled
3842017-11-27T17:46:35  <morcos> i think that is good enough
3852017-11-27T17:47:02  *** arubi has quit IRC
3862017-11-27T17:48:28  <sipa> morcos: ok
3872017-11-27T17:48:28  <morcos> i'm fine with 1(a), 2(c), 3(a).  But I wonder if there is some way we can warn the user about Problem 2, like could the backup have a higher walelt version (ick)
3882017-11-27T17:48:53  <sipa> right, that would be one solution
3892017-11-27T17:49:13  <sipa> but it's hard to prevent backups by just file copying
3902017-11-27T17:50:09  *** dqx_ has quit IRC
3912017-11-27T17:50:43  *** dqx_ has joined #bitcoin-core-dev
3922017-11-27T17:51:05  *** JackH has joined #bitcoin-core-dev
3932017-11-27T17:51:23  <sipa> you could have some system where a wallet is 'loaded' into a node, and the wallet file itself becomes marked as a backup, but something else in the node's configuration contains knowledge to avoid treating it as a backup
3942017-11-27T17:51:28  <sipa> but... yuck
3952017-11-27T17:51:33  *** dqx_ has quit IRC
3962017-11-27T17:51:44  *** dqx_ has joined #bitcoin-core-dev
3972017-11-27T17:54:21  *** arubi has joined #bitcoin-core-dev
3982017-11-27T17:54:39  <morcos> well the good thing is there is a workaround right.. so maybe  warnings are enough, saying that if you want to downgrade and you only have a backup, you first need to run 0.16 with the backup and make a fresh backup, and then you can downgrade
3992017-11-27T17:55:13  <morcos> and that is true regardless of whether you backup is from 0.15 or 0.16
4002017-11-27T18:04:35  *** Giszmo has quit IRC
4012017-11-27T18:07:21  *** Randolf has joined #bitcoin-core-dev
4022017-11-27T18:07:37  *** Emcy has joined #bitcoin-core-dev
4032017-11-27T18:09:05  *** Emcy_ has quit IRC
4042017-11-27T18:09:16  *** timothy has quit IRC
4052017-11-27T18:11:59  <BlueMatt> sipa: I really cant say I'm a fan of 2c
4062017-11-27T18:14:56  <sipa> the alternatives aren't great either though
4072017-11-27T18:15:42  *** Jochem has joined #bitcoin-core-dev
4082017-11-27T18:17:23  <BlueMatt> sipa: what did you think of my suggestion of a segwit-wallet-bestblockhash field
4092017-11-27T18:17:27  <BlueMatt> in addition to wallet-bestblockhash
4102017-11-27T18:17:39  *** JackH has quit IRC
4112017-11-27T18:17:52  <BlueMatt> which allows you to rescan for segwit addresses if you downgraded to 15 for a while (and havent fully pruned those blocks)
4122017-11-27T18:19:59  <sipa> BlueMatt: sounds doable, but it's not really something we can continue
4132017-11-27T18:20:35  <sipa> it's a bit of an accident that segwit was introduced in 0.13.1, but will only be actually supported 3 major versions later
4142017-11-27T18:20:47  <BlueMatt> hmm?
4152017-11-27T18:21:09  <sipa> i don't think we should continue to expect that it's at all feasible that a new feature introduced will actually work at all after downgrading
4162017-11-27T18:21:23  <BlueMatt> I didnt claim it would/should?
4172017-11-27T18:21:38  <sipa> right, but the approach you suggest really only works for segwit
4182017-11-27T18:21:46  <sipa> not for anything later
4192017-11-27T18:21:49  <sipa> (likely)
4202017-11-27T18:21:59  <BlueMatt> yes, this is specifically a segwit thing in large part cause we're not doing a full -walletupgrade for it
4212017-11-27T18:22:21  <BlueMatt> if we were doing a -walletupgrade (which I still think is the "right" way to do this, but obviously with a ton more complication) then I dont think we'd need to do this
4222017-11-27T18:22:49  *** Giszmo has joined #bitcoin-core-dev
4232017-11-27T18:22:49  <BlueMatt> a non-default clear statement of "I'm moving my wallet to 0.16" should be sufficient, but absent that, we should be more careful to handle some downgrade cases
4242017-11-27T18:24:14  <gmaxwell> how is walletupgrade right when it leaves current wallets people are using today unrecoverable?
4252017-11-27T18:24:21  <BlueMatt> hmm?
4262017-11-27T18:24:44  <gmaxwell> people use addwitnessaddress, then their backups are instantly no good-- no lookahead, no nothing.
4272017-11-27T18:25:07  <gmaxwell> that screup is fixed by 'upgrading' by default
4282017-11-27T18:25:21  <sipa> gmaxwell: i don't see how that has anything to do with explicit upgrade or not
4292017-11-27T18:25:24  <BlueMatt> I mean addwitnessaddress is barely supported and largely unsupportable, at least without something like upgradewallet
4302017-11-27T18:25:35  <BlueMatt> I mean you can auto-upgrade a user if they've previously used addwitnessaddress
4312017-11-27T18:25:45  <BlueMatt> cause an addwitnessaddress wallet is already a mess (at least wrt backups)
4322017-11-27T18:26:04  <sipa> explicit upgrade just guarantees that old versions will give you an error rather than silently missing things _for addresses generated in the future_
4332017-11-27T18:26:16  <sipa> obviously any addresses generated in the must should always continue to work
4342017-11-27T18:26:52  <BlueMatt> of course, but explicit upgrade also has a clear definition for the user: you cannot open this wallet anymore in old versions
4352017-11-27T18:27:12  <BlueMatt> which if we *dont* do an explicit upgrade, we end up in this quagmire of backward compatibility/downgrade hell
4362017-11-27T18:27:14  <gmaxwell> BlueMatt: if you don't autoupgrade someone then you won't make their backups good again.
4372017-11-27T18:27:34  <BlueMatt> "make their backups good again"?
4382017-11-27T18:28:06  <BlueMatt> I do not think we should *ever* auto-upgrade someone, with the possible exception of folks who've previously addwitnessaddress'd, cause they're kinda fucked if we dont, or at least we should nag them
4392017-11-27T18:28:30  <sipa> i think at some point we should do a required uograde
4402017-11-27T18:28:35  <gmaxwell> Right now, joe blow believes our claims of supporting segwit, runs the software, follows instructions to use it... unknown to him, his backup will miss coins if recovered.  Post upgrading to a release that autoupgrades his backup will no longer miss coins when recovered.
4412017-11-27T18:28:55  <gmaxwell> _You cannot tell if they've used addwitnessaddress because they could be loading a backup which hasn't seen it_
4422017-11-27T18:29:27  <sipa> well 1a solves that
4432017-11-27T18:30:03  <gmaxwell> if you try, you get even more awesome effects like one backup works for address X and another doesn't for reasons entirely unrelated to address X.
4442017-11-27T18:30:07  <BlueMatt> yes, I think for that reason 1a is probably good
4452017-11-27T18:30:48  <sipa> the only case that is not covered is downgrading + restoring a backup simuktaneously
4462017-11-27T18:31:15  <sipa> which is inherently impossible to solve completely
4472017-11-27T18:31:31  <BlueMatt> yes, and hence my suggestion to do a segwit-wallet-bestblockhash, cause then we at least fix it for non-pruned or only-briefly-run-as-downgraded nodes
4482017-11-27T18:31:32  <sipa> but some solutions are possible for the case where the backup was made after uograding
4492017-11-27T18:31:37  <BlueMatt> sidestepping the version rabbithole
4502017-11-27T18:32:08  *** dqx_ has quit IRC
4512017-11-27T18:32:21  * BlueMatt would also like to further discuss 2a
4522017-11-27T18:32:32  <BlueMatt> whats default keypool size? its not *that* bad
4532017-11-27T18:32:39  <sipa> 2000 keys
4542017-11-27T18:32:51  <BlueMatt> and, when we go to do things "the right way", we can clean those out, maybe
4552017-11-27T18:32:55  <sipa> + more logic and edge cases
4562017-11-27T18:33:02  <BlueMatt> with a -walletupgrade can clean out those entries
4572017-11-27T18:33:24  <sipa> = something that still only works as long as you don't actually use the wallet too much after downgrading
4582017-11-27T18:34:44  <BlueMatt> hmm? no?
4592017-11-27T18:34:57  <BlueMatt> your backups will still work cause they have the witnesses added
4602017-11-27T18:35:08  <BlueMatt> and the upgraded wallet obviously wont cause you set the version to be un-openable with the old version
4612017-11-27T18:35:36  <sipa> right
4622017-11-27T18:36:11  <sipa> but if the wallet was used a lot in the new version after the backup was made, you'll hit problems when the backup's keypool runs out
4632017-11-27T18:36:29  <BlueMatt> yes, but that is always true
4642017-11-27T18:36:38  <BlueMatt> if your keypool runs out you cant run from a backup or you lose funds
4652017-11-27T18:36:39  <BlueMatt> period.
4662017-11-27T18:37:03  <BlueMatt> (I suppose with the exception of HD, but....ugh)
4672017-11-27T18:37:27  <sipa> no, not period
4682017-11-27T18:37:43  <sipa> this is exactly the assumption we don't need to make with HD wallets anymore
4692017-11-27T18:38:03  <sipa> relying on the keypool to deal with downgraded backups reintroduces that assumption
4702017-11-27T18:38:32  <BlueMatt> yes, with the exception of HD
4712017-11-27T18:38:40  <sipa> that's not a small exception
4722017-11-27T18:38:48  <sipa> that's the norm
4732017-11-27T18:39:13  <BlueMatt> ok, so now we're back to segwit-wallet-bestblockhash
4742017-11-27T18:39:17  <BlueMatt> thats the only way to fix that issue
4752017-11-27T18:39:52  <wallet42> sipa: ah yes. prioritisetransaction is exactly what im looking for.
4762017-11-27T18:40:09  <sipa> BlueMatt: right, but only in the case the user actually uogrades again
4772017-11-27T18:40:21  <BlueMatt> well if they dont upgrade again there is literally nothing we can do
4782017-11-27T18:40:27  <sipa> that's my point
4792017-11-27T18:40:32  <wallet42> spia: my python prototype question was in the context of the output list wallet amend tx amend idea you talked about with provoost
4802017-11-27T18:40:41  <BlueMatt> (and I think thats reasonable, if you use segwit in a version with a headline of "SegWit Wallet" and then downgrade, you'd kinda expect to not have your segwit funds)
4812017-11-27T18:41:04  <sipa> BlueMatt: so why do we need to support the downgrade + restore case again?
4822017-11-27T18:41:04  <BlueMatt> well the current proposal does *not* fully work in the case you upgrade again, without a rescan
4832017-11-27T18:41:13  <sipa> it's without guarantees at best
4842017-11-27T18:41:23  <BlueMatt> if you are non-pruned its guaranteed
4852017-11-27T18:41:23  <sipa> in whatever case
4862017-11-27T18:41:50  <sipa> it's without guarantees at best as long as you don't upgrade
4872017-11-27T18:41:57  <BlueMatt> huh?
4882017-11-27T18:42:18  <sipa> don't upgrade after downgrading, i mean
4892017-11-27T18:42:46  <BlueMatt> yes, if you dont ever switch back to 0.16+, then you are maybe not safe in this case
4902017-11-27T18:42:56  <sipa> not maybe
4912017-11-27T18:42:58  <BlueMatt> but so what? we can still improve things
4922017-11-27T18:43:03  <sipa> in that case nothing can be guaranteed
4932017-11-27T18:43:15  <BlueMatt> doing a partial-rescan is a very, very big improvement for many of the cases we're talking about
4942017-11-27T18:43:19  <BlueMatt> even if its not a guarantee
4952017-11-27T18:43:34  <BlueMatt> assuming you may never upgrade to 0.16+ and trying to support that seems like nonsense
4962017-11-27T18:43:38  <BlueMatt> but thats not what I'm arguing here
4972017-11-27T18:43:41  <sipa> ok, so let's walk through yhis
4982017-11-27T18:43:48  <sipa> you upgrade to 0.16
4992017-11-27T18:43:56  <sipa> there is no segwitwallet record
5002017-11-27T18:44:29  <sipa> you do what? assume that the existing non-segwit wallet bestblock record is accurate?
5012017-11-27T18:45:13  <BlueMatt> a) ignore, b) rescan from segwit activation date if you have P2SH-P2WPKH entries c) always rescan from segwit activation date
5022017-11-27T18:45:57  <sipa> b assumes the file is recent enough to have those records
5032017-11-27T18:46:00  <BlueMatt> d) rescan in case -findmissingsegwitoutputs=true is set
5042017-11-27T18:46:09  <BlueMatt> indeed, b kinda sucks
5052017-11-27T18:46:17  *** promag has joined #bitcoin-core-dev
5062017-11-27T18:46:32  <sipa> i think the best solution is just telling people to run with 0.16 and -rescan in that case
5072017-11-27T18:46:42  <BlueMatt> do we have a rescan from date rpc?
5082017-11-27T18:46:45  <BlueMatt> i think we do now, right?
5092017-11-27T18:47:06  <sipa> unsure
5102017-11-27T18:47:22  <BlueMatt> I mean in that case we also need 2a, I think
5112017-11-27T18:48:00  <sipa> how so?
5122017-11-27T18:48:13  <sipa> 1a solves everything during a rescan
5132017-11-27T18:48:24  *** windsok has quit IRC
5142017-11-27T18:48:44  *** arubi has quit IRC
5152017-11-27T18:49:00  <BlueMatt> yea, sorry, 1a obviously fixes it for rescan, maybe I'm just more worried about problem 2
5162017-11-27T18:49:39  <gmaxwell> unrelated to this, through some combination of multiwallet, -rescan, restarting my node.. I ended up in a situation with a bunch of wallets that lost all their transactions but think they're in sync.
5172017-11-27T18:49:39  <BlueMatt> more like "oops, da server with wallet blew up, bring in the one we upgraded from last week, running 0.15.1, will have to do for now"
5182017-11-27T18:50:02  <BlueMatt> gmaxwell: thats....frightening....is it 0.15.1 with the copied-db issue?
5192017-11-27T18:50:07  <morcos> i basically agree with sipa
5202017-11-27T18:50:14  *** dqx_ has joined #bitcoin-core-dev
5212017-11-27T18:50:16  <sipa> my solution to problem 2 is tell people to avoid downgrading and restoring, and if in exceptional cases that misses something, rescan
5222017-11-27T18:50:18  <gmaxwell> w/ recent-ish master.
5232017-11-27T18:50:39  <BlueMatt> sipa: but that doesnt solve the problem of not upgrading right away? at least 2a will help significantly
5242017-11-27T18:50:45  <morcos> i think 1a, 3a and a guide for acceptable workflows and how to recover is the way to go..  we should not make things too difficult.
5252017-11-27T18:51:31  <morcos> however i think we will need to do something like 2a when we have the upgrade wallet option for 0.17.  but then it will i think be fully optional as to whether you upgrade or not.
5262017-11-27T18:51:45  <gmaxwell> I _think_ what I did was start with two dozen -wallets and -rescan then realized it would sequentially rescan each, killed the process, then went on to try using them directly.
5272017-11-27T18:52:09  *** digifis has joined #bitcoin-core-dev
5282017-11-27T18:52:12  <sipa> BlueMatt: but even if we do 2a, there is no solution for 4a
5292017-11-27T18:52:16  *** windsok has joined #bitcoin-core-dev
5302017-11-27T18:52:25  *** promag has quit IRC
5312017-11-27T18:52:27  <sipa> recovery options are great, but they exist
5322017-11-27T18:52:45  <BlueMatt> yea, issue 4 is unsolveable, I agree
5332017-11-27T18:52:52  <morcos> sorry, not actually 2a for 0.17, but something that takes all your keys in your keypool and stores them as things that implicitly might have any style address
5342017-11-27T18:52:56  <BlueMatt> with the exception of 1a + rescan
5352017-11-27T18:53:14  <sipa> 2a is essentially trying to avoid a recovery procedure in a situation which already is something that should never be part of normal workflow
5362017-11-27T18:54:15  *** arubi has joined #bitcoin-core-dev
5372017-11-27T18:54:20  <gmaxwell> if we make rescan faster we could be more liberal in expecting its use.
5382017-11-27T18:55:01  <gmaxwell> not tooo liberal, since pruned nodes can't rescan...
5392017-11-27T18:55:48  <BlueMatt> gmaxwell: do you have an intuition in why rescan is slow? I havent looked into it
5402017-11-27T18:56:58  <sipa> BlueMatt: we should do a benchmark for reindex with and without wallet
5412017-11-27T18:58:16  *** promag has joined #bitcoin-core-dev
5422017-11-27T18:58:43  <sipa> if there is a significant different between the two, there is a totally obvious improvement from not calling Solver on each output in order to match it with the wallet
5432017-11-27T18:58:53  <sipa> (which construct a vector of vectors ffs)
5442017-11-27T18:58:54  <gmaxwell> well we fully read and deseralize each block and each transaction then pass them through the wallet code, so that implhes 1001 memory allocations plus sha256ing each transaction (and reserializing to do so)... looking at IO we also seem to create a lot of excess  IO with opening the file, seeking to the block closing it.
5452017-11-27T18:59:29  <sipa> then indeed there is the cost of just loading blocks, which could be improved by having a block deserializer that doesn't build/verify the merkle tree
5462017-11-27T18:59:57  <gmaxwell> at least from rough looking at it, we do both a bunch of dump computation and IO and all of it serialized.
5472017-11-27T19:00:21  <gmaxwell> it's worth noting that it's not remarkably faster with the blockchain all in cache.
5482017-11-27T19:01:18  <BlueMatt> sipa: ok, so I guess I feel a lot more comfortable about things if we do #11281 and recommend rescanblockchain segwit_activation_height
5492017-11-27T19:01:20  <gribble> https://github.com/bitcoin/bitcoin/issues/11281 | Avoid permanent cs_main/cs_wallet lock during RescanFromTime by jonasschnelli · Pull Request #11281 · bitcoin/bitcoin · GitHub
5502017-11-27T19:02:32  <BlueMatt> sipa: I'd still be more comfortable with 2a...its far from a hard guarantee of anything, but I really could see someone ending up in the state where that'd save them
5512017-11-27T19:02:47  <BlueMatt> if for no other reason than someone restoring a backup may load in 0.15, realize they need 0.16, then open in 0.16 right away
5522017-11-27T19:02:51  <BlueMatt> and end up screwing themselves
5532017-11-27T19:04:06  <BlueMatt> the segwit-wallet-best-block-hash would also similarly solve the same issue even if you do my option a where you just ignore it if it isnt there
5542017-11-27T19:04:19  *** promag has quit IRC
5552017-11-27T19:04:36  <BlueMatt> though I agree that kinda feels gross given its solving such a specific edge case without any hard guarantees
5562017-11-27T19:04:41  *** windsok has quit IRC
5572017-11-27T19:06:32  *** JamesAU has joined #bitcoin-core-dev
5582017-11-27T19:08:12  *** windsok has joined #bitcoin-core-dev
5592017-11-27T19:08:12  *** windsok has joined #bitcoin-core-dev
5602017-11-27T19:08:43  <BlueMatt> can we at least high-priority #11281 for 0.16?
5612017-11-27T19:08:45  <gribble> https://github.com/bitcoin/bitcoin/issues/11281 | Avoid permanent cs_main/cs_wallet lock during RescanFromTime by jonasschnelli · Pull Request #11281 · bitcoin/bitcoin · GitHub
5622017-11-27T19:08:52  <BlueMatt> so that rescan is at least slightly less painful from the qt command window
5632017-11-27T19:08:56  *** Emcy_ has joined #bitcoin-core-dev
5642017-11-27T19:09:00  <sipa> sgtm
5652017-11-27T19:09:28  <BlueMatt> and add a "I lost my segwit coins" button in the debug window
5662017-11-27T19:09:30  * BlueMatt *ducks*
5672017-11-27T19:10:34  *** Emcy has quit IRC
5682017-11-27T19:10:57  *** JamesAU has quit IRC
5692017-11-27T19:13:19  <gmaxwell> any kind of scanning uncertanty will cause untold amounts of prophylactic scanning.
5702017-11-27T19:13:21  *** bobl has joined #bitcoin-core-dev
5712017-11-27T19:13:40  <BlueMatt> yea, but I dont think we can really solve that......
5722017-11-27T19:13:51  *** digifis has quit IRC
5732017-11-27T19:14:06  <BlueMatt> in part cause 0.16 docs are going to have to have a bunch of "in case you did X, do this rescan"
5742017-11-27T19:14:47  *** bobl has quit IRC
5752017-11-27T19:15:11  <gmaxwell> thats partly why I was commenting about making rescanning faster.
5762017-11-27T19:15:37  <BlueMatt> yes, though dunno about making that a requirement for 0.16 :p
5772017-11-27T19:15:44  <gmaxwell> oh another point on speed-- I opened an issue for this-- right now if you load up a dozen wallets and -rescan. ... it rescans a dozen times, one for each wallet by itself...
5782017-11-27T19:15:58  <BlueMatt> yes, that should be fixed as well
5792017-11-27T19:16:14  <gmaxwell> my first idea about not spending the rest of my life rescanning was thwarted by that.
5802017-11-27T19:22:41  *** jb55 has quit IRC
5812017-11-27T19:28:09  *** jb55 has joined #bitcoin-core-dev
5822017-11-27T19:30:51  <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #11775: Move fee estimator into validationinterface/cscheduler thread (master...2017-09-background-feeest) https://github.com/bitcoin/bitcoin/pull/11775
5832017-11-27T19:33:35  *** Emcy_ has quit IRC
5842017-11-27T19:34:22  *** Emcy has joined #bitcoin-core-dev
5852017-11-27T19:47:24  *** Emcy_ has joined #bitcoin-core-dev
5862017-11-27T19:48:15  *** meshcollider has joined #bitcoin-core-dev
5872017-11-27T19:48:35  *** Emcy has quit IRC
5882017-11-27T20:03:01  *** laurentmt has joined #bitcoin-core-dev
5892017-11-27T20:06:08  *** Emcy has joined #bitcoin-core-dev
5902017-11-27T20:07:32  *** Emcy_ has quit IRC
5912017-11-27T20:11:56  <jonasschnelli> BlueMatt: added 11281 to the project 7
5922017-11-27T20:11:59  *** Chasity76Ortiz has quit IRC
5932017-11-27T20:13:02  <BlueMatt> jonasschnelli: thanks!
5942017-11-27T20:15:07  *** ekrok has quit IRC
5952017-11-27T20:15:28  *** ekrok has joined #bitcoin-core-dev
5962017-11-27T20:17:16  <cluelessperson> BlueMatt: are you updating the fee estimator?
5972017-11-27T20:17:19  *** ekrok_ has joined #bitcoin-core-dev
5982017-11-27T20:17:40  *** ekrok has quit IRC
5992017-11-27T20:18:35  *** Emcy has quit IRC
6002017-11-27T20:20:27  <BlueMatt> cluelessperson: it doesnt change the fee estimator logic, except to handle a few edge cases better
6012017-11-27T20:33:46  *** Sinclair6 has joined #bitcoin-core-dev
6022017-11-27T20:37:33  *** laurentmt has quit IRC
6032017-11-27T20:42:54  <sipa> Lauda: using 6GB with 4GB dbcache sounds bad but not nearly the same thing i was seeing
6042017-11-27T20:43:20  <sipa> i was seeing 32GB usage with 1.7G dbcache
6052017-11-27T20:43:52  *** Randolf has quit IRC
6062017-11-27T21:07:13  *** Cheeseo has quit IRC
6072017-11-27T21:07:21  <Lauda> Well, add 4 gb of SWAP to that so probably ~10ish (I think I allocated 7 maximum for the VM). However, yes. That is much worse :<
6082017-11-27T21:07:48  <Lauda> Any ideas as to what is causing that?
6092017-11-27T21:10:01  <sipa> by "using" i mean RSS size, not physical usage
6102017-11-27T21:10:32  <sipa> it's pretty hard to map VM usage to physical usage, as processes can share memory etc
6112017-11-27T21:11:49  <sipa> (by VM i mean virtual memory, not virtual machine - it's probably confusing in this context)
6122017-11-27T21:12:17  <Lauda> Oh, then 6 GB yeah.
6132017-11-27T21:13:55  *** JackH has joined #bitcoin-core-dev
6142017-11-27T21:14:13  <Lauda> Your behavior is only in current master, not in 0.15.1?
6152017-11-27T21:14:45  <sipa> indeed
6162017-11-27T21:15:28  <sipa> in 0.15.1 memory usage stayed nicely within a few 100 MB of the cache size
6172017-11-27T21:15:39  <sipa> though i now fail to reproduce with master
6182017-11-27T21:16:55  <Lauda> That's strange.
6192017-11-27T21:27:33  *** Randolf has joined #bitcoin-core-dev
6202017-11-27T21:35:50  *** Randolf has quit IRC
6212017-11-27T21:36:36  *** cryptapus_ has joined #bitcoin-core-dev
6222017-11-27T21:36:37  *** cryptapus_ has joined #bitcoin-core-dev
6232017-11-27T21:36:40  *** cryptapus has quit IRC
6242017-11-27T21:36:54  *** cryptapus_ is now known as cryptapus
6252017-11-27T21:39:05  *** Dizzle has joined #bitcoin-core-dev
6262017-11-27T21:45:56  *** Emcy has joined #bitcoin-core-dev
6272017-11-27T21:55:01  *** JamesAU has joined #bitcoin-core-dev
6282017-11-27T21:57:58  *** archaeal_ has joined #bitcoin-core-dev
6292017-11-27T21:59:32  <cluelessperson> BlueMatt: do you know who might be best to redo the logic?
6302017-11-27T21:59:41  <cluelessperson> reason being that fees seem to jump up faster than volume
6312017-11-27T21:59:53  <cluelessperson> https://jochen-hoenicke.de/queue/#2h
6322017-11-27T21:59:56  <gribble> https://github.com/bitcoin/bitcoin/issues/2 | Long-term, safe, store-of-value · Issue #2 · bitcoin/bitcoin · GitHub
6332017-11-27T22:00:31  <BlueMatt> cluelessperson: if there were a clear and easy fix it would have been done already :/
6342017-11-27T22:00:39  <BlueMatt> sadly good fee estimation is super non-trivial
6352017-11-27T22:01:05  <BlueMatt> *especially* for the non-economical (eg doing non-rbf so want high confidence) case
6362017-11-27T22:01:30  <cluelessperson> BlueMatt: agreed.  I could only recommend experimenting with "lower estimates slightly" and see how the mempool responds.
6372017-11-27T22:01:35  *** pgupta has joined #bitcoin-core-dev
6382017-11-27T22:01:38  <cluelessperson> that would take >years to tune.
6392017-11-27T22:01:44  <BlueMatt> doing it with rbf in economical mode could probably get improvements and go down faster, etc
6402017-11-27T22:02:05  <BlueMatt> "lower estimates slightly"? what does that mean?
6412017-11-27T22:02:26  <BlueMatt> in steady-state the fee estimator does incredibly well, it also happens to do incredibly well for very-high-confidence-of-inclusion
6422017-11-27T22:02:40  <cluelessperson> BlueMatt:   (existing_fee_estimate) - 1 sat,    -5 sat,   -10 sat
6432017-11-27T22:02:48  * cluelessperson chuckles at the absurdity
6442017-11-27T22:02:53  <BlueMatt> though fee estimation also ends up being a bit of a self-fuffilling prophecy
6452017-11-27T22:03:05  <BlueMatt> yea, I dont think that is a good approach :p
6462017-11-27T22:05:32  *** Chris_Stewart_5 has quit IRC
6472017-11-27T22:06:19  *** Cheeseo has joined #bitcoin-core-dev
6482017-11-27T22:06:20  <cluelessperson> BlueMatt: I'm taking classes.  Hopefully I can help in the future, but right now I'm a moronw. :/
6492017-11-27T22:06:55  *** ekrok_ has quit IRC
6502017-11-27T22:08:01  *** ekrok has joined #bitcoin-core-dev
6512017-11-27T22:16:28  *** ekrok has quit IRC
6522017-11-27T22:16:35  *** ekrok_ has joined #bitcoin-core-dev
6532017-11-27T22:18:14  <mlz> lol
6542017-11-27T22:28:01  *** meshcollider has quit IRC
6552017-11-27T22:48:57  *** Cheeseo has quit IRC
6562017-11-27T22:49:10  *** AaronvanW has quit IRC
6572017-11-27T22:52:46  *** bule has joined #bitcoin-core-dev
6582017-11-27T22:58:57  *** JamesAU has quit IRC
6592017-11-27T22:59:26  *** btcdrak has joined #bitcoin-core-dev
6602017-11-27T23:02:49  *** cyber55 has joined #bitcoin-core-dev
6612017-11-27T23:07:49  *** meshcollider has joined #bitcoin-core-dev
6622017-11-27T23:10:29  *** pgupta has quit IRC
6632017-11-27T23:11:11  *** js___ has joined #bitcoin-core-dev
6642017-11-27T23:11:11  *** pgupta has joined #bitcoin-core-dev
6652017-11-27T23:11:43  *** js___ has quit IRC
6662017-11-27T23:12:10  *** dqx_ has quit IRC
6672017-11-27T23:14:29  *** dqx_ has joined #bitcoin-core-dev
6682017-11-27T23:14:39  *** quantbot has quit IRC
6692017-11-27T23:15:56  *** pgupta has quit IRC
6702017-11-27T23:19:11  *** dqx_ has quit IRC
6712017-11-27T23:19:46  *** shesek has joined #bitcoin-core-dev
6722017-11-27T23:19:46  *** shesek has joined #bitcoin-core-dev
6732017-11-27T23:19:50  *** Randolf has joined #bitcoin-core-dev
6742017-11-27T23:22:26  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #11779: qa: Combine logs on travis by default (master...Mf1711-travisQaLogs) https://github.com/bitcoin/bitcoin/pull/11779
6752017-11-27T23:23:23  *** pgupta has joined #bitcoin-core-dev
6762017-11-27T23:26:55  *** Randolf has quit IRC
6772017-11-27T23:28:09  *** pgupta has quit IRC
6782017-11-27T23:29:25  *** JamesAU has joined #bitcoin-core-dev
6792017-11-27T23:35:17  *** Cogito_Ergo_Sum has quit IRC
6802017-11-27T23:35:47  *** jb55 has quit IRC
6812017-11-27T23:38:46  *** JamesAU has quit IRC
6822017-11-27T23:40:39  *** jb55 has joined #bitcoin-core-dev
6832017-11-27T23:46:06  *** esotericnonsense has joined #bitcoin-core-dev
6842017-11-27T23:48:09  *** brianhoffman_ has joined #bitcoin-core-dev
6852017-11-27T23:50:06  *** Randolf has joined #bitcoin-core-dev
6862017-11-27T23:50:59  *** brianhoffman has quit IRC
6872017-11-27T23:50:59  *** brianhoffman_ is now known as brianhoffman
6882017-11-27T23:56:37  *** dqx_ has joined #bitcoin-core-dev
6892017-11-27T23:56:50  *** dqx_ has joined #bitcoin-core-dev
6902017-11-27T23:58:12  *** bule2 has joined #bitcoin-core-dev
6912017-11-27T23:59:30  *** Dizzle has quit IRC