12017-08-08T00:00:16  *** chjj has quit IRC
  22017-08-08T00:01:38  *** promag has joined #bitcoin-core-dev
  32017-08-08T00:12:32  *** chjj has joined #bitcoin-core-dev
  42017-08-08T00:22:12  *** Chris_Stewart_5 has joined #bitcoin-core-dev
  52017-08-08T00:27:35  *** AaronvanW has quit IRC
  62017-08-08T00:30:18  *** jimpo has joined #bitcoin-core-dev
  72017-08-08T00:32:38  <jimpo> What is HSNG? (mentioned in discussion of new addr message)
  82017-08-08T00:33:26  *** abpa has quit IRC
  92017-08-08T00:35:08  <gmaxwell> Hidden services NG. Tor HS addresses will be becoming 256 bits long.
 102017-08-08T00:35:43  <gmaxwell> NG = next gen, I assume
 112017-08-08T00:36:06  <jimpo> interesting. thx
 122017-08-08T00:38:27  *** veleiro has quit IRC
 132017-08-08T00:39:10  *** chjj has quit IRC
 142017-08-08T00:43:03  *** veleiro has joined #bitcoin-core-dev
 152017-08-08T00:45:20  *** AaronvanW has joined #bitcoin-core-dev
 162017-08-08T00:49:38  *** Cheeseo has quit IRC
 172017-08-08T00:52:19  *** chjj has joined #bitcoin-core-dev
 182017-08-08T01:00:10  *** chjj has quit IRC
 192017-08-08T01:05:58  *** jimpo has quit IRC
 202017-08-08T01:13:56  *** chjj has joined #bitcoin-core-dev
 212017-08-08T01:18:30  *** chjj has quit IRC
 222017-08-08T01:29:01  *** shaolinfry has quit IRC
 232017-08-08T01:29:17  *** dgenr8 has quit IRC
 242017-08-08T01:31:57  *** chjj has joined #bitcoin-core-dev
 252017-08-08T01:33:28  *** jimpo has joined #bitcoin-core-dev
 262017-08-08T01:34:55  <jimpo> is there anyone working/focussing on extracting libbitcoinconsensus?
 272017-08-08T01:36:07  <Chris_Stewart_5> jimpo: Last I knew jtimon was, not sure if he is still working on it though
 282017-08-08T01:39:02  <bitcoin-git> [bitcoin] eklitzke opened pull request #11005: Tests: Add a lint check for trailing whitespace (master...whitespace) https://github.com/bitcoin/bitcoin/pull/11005
 292017-08-08T02:04:38  *** Alina-malina has quit IRC
 302017-08-08T02:07:38  *** jimpo has quit IRC
 312017-08-08T02:08:38  *** Alina-malina has joined #bitcoin-core-dev
 322017-08-08T02:17:19  *** windsok has quit IRC
 332017-08-08T02:17:22  *** Alina-malina has quit IRC
 342017-08-08T02:18:37  *** windsok has joined #bitcoin-core-dev
 352017-08-08T02:29:48  *** Alina-malina has joined #bitcoin-core-dev
 362017-08-08T02:31:59  *** Murch has quit IRC
 372017-08-08T02:34:27  *** jamesob has quit IRC
 382017-08-08T02:35:56  *** Alina-malina has quit IRC
 392017-08-08T02:37:14  *** veleiro has quit IRC
 402017-08-08T02:38:30  *** Alina-malina has joined #bitcoin-core-dev
 412017-08-08T02:42:51  *** veleiro has joined #bitcoin-core-dev
 422017-08-08T02:44:27  *** Alina-malina has quit IRC
 432017-08-08T02:47:00  *** Alina-malina has joined #bitcoin-core-dev
 442017-08-08T02:53:37  *** jouke has quit IRC
 452017-08-08T02:55:27  *** Alina-malina has quit IRC
 462017-08-08T02:55:45  *** jouke has joined #bitcoin-core-dev
 472017-08-08T02:59:25  *** Alina-malina has joined #bitcoin-core-dev
 482017-08-08T03:02:40  <bitcoin-git> [bitcoin] promag opened pull request #11006: WIP: Improve shutdown process (master...201708-fast-shutdown) https://github.com/bitcoin/bitcoin/pull/11006
 492017-08-08T03:05:40  *** Alina-malina has quit IRC
 502017-08-08T03:06:22  *** Alina-malina has joined #bitcoin-core-dev
 512017-08-08T03:12:39  *** veleiro1 has joined #bitcoin-core-dev
 522017-08-08T03:12:51  *** veleiro has quit IRC
 532017-08-08T03:14:57  *** Alina-malina has quit IRC
 542017-08-08T03:26:42  *** Alina-malina has joined #bitcoin-core-dev
 552017-08-08T03:29:26  *** ula has quit IRC
 562017-08-08T03:40:37  *** promag has quit IRC
 572017-08-08T03:42:47  *** veleiro1 has left #bitcoin-core-dev
 582017-08-08T03:54:16  *** chjj has quit IRC
 592017-08-08T04:14:35  *** Chris_Stewart_5 has quit IRC
 602017-08-08T04:22:38  <luke-jr> gmaxwell: hmm, but it makes sense to show them up top in the question page :/
 612017-08-08T04:27:31  <gmaxwell> luke-jr: so make the order different for each, questions by least answers answers by most
 622017-08-08T04:35:56  *** Alina-malina has quit IRC
 632017-08-08T04:35:56  *** Alina-malina has joined #bitcoin-core-dev
 642017-08-08T04:52:28  *** d_t has quit IRC
 652017-08-08T05:20:35  *** dcousens has joined #bitcoin-core-dev
 662017-08-08T05:20:39  <luke-jr> gmaxwell: done
 672017-08-08T05:29:17  *** chjj has joined #bitcoin-core-dev
 682017-08-08T05:37:53  *** corinrose_ has joined #bitcoin-core-dev
 692017-08-08T06:12:00  *** miknotauro has joined #bitcoin-core-dev
 702017-08-08T06:12:00  *** Austindoggie has joined #bitcoin-core-dev
 712017-08-08T06:17:03  *** Austindoggie has left #bitcoin-core-dev
 722017-08-08T06:25:18  *** dermoth has quit IRC
 732017-08-08T06:25:38  *** dermoth has joined #bitcoin-core-dev
 742017-08-08T06:48:32  *** BashCo has quit IRC
 752017-08-08T06:49:06  *** BashCo has joined #bitcoin-core-dev
 762017-08-08T06:53:53  *** BashCo has quit IRC
 772017-08-08T07:12:10  *** BashCo has joined #bitcoin-core-dev
 782017-08-08T07:15:03  *** Deacydal has joined #bitcoin-core-dev
 792017-08-08T07:17:07  *** Deacyde has quit IRC
 802017-08-08T07:18:06  *** promag has joined #bitcoin-core-dev
 812017-08-08T07:20:15  *** miknotauro has quit IRC
 822017-08-08T07:45:02  <luke-jr> hm, it's a shame we're releasing 0.15 without bech32 sending :x
 832017-08-08T07:45:53  <gmaxwell> luke-jr: when does segwit activate?
 842017-08-08T07:46:17  <luke-jr> gmaxwell: Aug 21 it looks like
 852017-08-08T07:46:37  <gmaxwell> luke-jr: also it's just not that mature yet,... a serious problem was found in the test vectors of the spec and ref implementations just a week ago.
 862017-08-08T07:46:46  <luke-jr> :<
 872017-08-08T07:47:09  <gmaxwell> luke-jr: dunno if you read the minutes from the last meeting (or was it the one before last) but we were talkign about a short cycle release with segwit wallet support right after 15 in any case.
 882017-08-08T07:47:36  <luke-jr> I didn't, but newbery did mention it to me. I agree a rapid 0.16 would be good.
 892017-08-08T07:47:50  <gmaxwell> https://github.com/sipa/bech32/pull/21 also re: bech32 support.
 902017-08-08T07:49:04  <luke-jr> is there a reason to have a C++ implementation at all? seems to me the proper route to go would be to make a C libbech32, which can be used by Core…?
 912017-08-08T07:51:24  <gmaxwell> a library for 50 lines of code... what are you, a nodejs developer? lpad() ftw? :P  But seriously, a native C++ interface is typesafe. Wrappign a C implementation to give it a C++ friendly interface would probably end up being the same size as the library itself.
 922017-08-08T07:51:49  <gmaxwell> FWIW it's much simpler to implement bech32 than base58.
 932017-08-08T07:52:30  <luke-jr> hmm
 942017-08-08T08:23:06  *** SopaXorzTaker has joined #bitcoin-core-dev
 952017-08-08T08:25:16  <achow101> I suspect that #10982 is going to need to be locked soon due to brigading and trolling
 962017-08-08T08:25:20  <gribble> https://github.com/bitcoin/bitcoin/issues/10982 | Disconnect network service bits 6 and 8 until Aug 1, 2018 by TheBlueMatt · Pull Request #10982 · bitcoin/bitcoin · GitHub
 972017-08-08T08:25:26  <jonasschnelli> How do we deal with the service bits 6 and 8? Should NODE_NETWORK_LIMITED avoid bit6 because something else claimed it (although not via BIP process)?
 982017-08-08T08:27:18  <achow101> It was actually bits 5 and 7 that were claimed
 992017-08-08T08:30:09  <luke-jr> kidnapped*? :P
1002017-08-08T08:33:08  <jonasschnelli> achow101: Bip 7 is SW2x, right? Whats 5?
1012017-08-08T08:35:32  <luke-jr> jonasschnelli: BCash I assume
1022017-08-08T08:37:59  *** AaronvanW has quit IRC
1032017-08-08T08:41:56  *** promag has quit IRC
1042017-08-08T08:42:46  *** Bartholome has joined #bitcoin-core-dev
1052017-08-08T08:46:50  *** Bartholome has quit IRC
1062017-08-08T08:51:29  <luke-jr> achow101: sigh, would have been nice to get the commit message fixed before merging :/
1072017-08-08T08:54:22  <gmaxwell> unfortunately s2x people were spamming their slack with it and such the moment it went up.
1082017-08-08T08:54:24  *** AaronvanW has joined #bitcoin-core-dev
1092017-08-08T08:55:10  <luke-jr> it would also be nice if we didn't merge things because of trolls :x
1102017-08-08T08:56:49  *** corinrose_ has quit IRC
1112017-08-08T09:03:29  <luke-jr> jonasschnelli: perhaps: uint16_t archival_data_flag = (servicebits & NODE_NETWORK_FLAG_MASK); if (servicebits & NODE_NETWORK) { … } else if (archival_data_flag == NODE_NETWORK_LIMITED_HIGH) { … } else if (archival_data_flag == NODE_NETWORK_LIMITED_LOW) { … } would fit nicely
1122017-08-08T09:08:03  *** timothy has joined #bitcoin-core-dev
1132017-08-08T09:18:50  <MarcoFalke> Reminder to call gpg --keyserver pgp.mit.edu --refresh-keys F4316B9B
1142017-08-08T09:21:06  <gmaxwell> achow101: depends on if you count the bits starting at 0 or 1
1152017-08-08T09:25:22  <luke-jr> gmaxwell: which has always been counted from 0..
1162017-08-08T09:26:13  <jnewbery> NODE_NONE must surely be zero?
1172017-08-08T09:26:22  <gmaxwell> depends on your language,  "first bit" is 1<<0.
1182017-08-08T09:28:34  <jonasschnelli> From what I read you usually start a 0
1192017-08-08T09:28:36  <jonasschnelli> *at
1202017-08-08T09:28:40  <bitcoin-git> [bitcoin] laanwj pushed 5 new commits to master: https://github.com/bitcoin/bitcoin/compare/fa8a0639f7b0...627c3c0e495b
1212017-08-08T09:28:41  <bitcoin-git> bitcoin/master dac3782 Wladimir J. van der Laan: doc: Correct AmountFromValue/ValueFromAmount names
1222017-08-08T09:28:41  <bitcoin-git> bitcoin/master 46347ad Wladimir J. van der Laan: rpc: Move ValueFromAmount to core_write...
1232017-08-08T09:28:42  <bitcoin-git> bitcoin/master ec05c50 Wladimir J. van der Laan: rpc: Use ValueFromAmount instead of FormatMoney in TxToUniv...
1242017-08-08T09:28:56  <bitcoin-git> [bitcoin] laanwj closed pull request #10999: Fix amounts formatting in `decoderawtransaction` (master...2017_08_decoderawtx_amount) https://github.com/bitcoin/bitcoin/pull/10999
1252017-08-08T09:42:08  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/627c3c0e495b...4268426b4500
1262017-08-08T09:42:09  <bitcoin-git> bitcoin/master 055d95f John Newbery: [wallet] return correct error code from resendwallettransaction
1272017-08-08T09:42:09  <bitcoin-git> bitcoin/master 4268426 Wladimir J. van der Laan: Merge #11002: [wallet] return correct error code from resendwallettransaction...
1282017-08-08T09:42:38  <bitcoin-git> [bitcoin] laanwj closed pull request #11002: [wallet] return correct error code from resendwallettransaction (master...resendwallettransaction_error_code) https://github.com/bitcoin/bitcoin/pull/11002
1292017-08-08T09:58:54  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/4268426b4500...2507fd55568b
1302017-08-08T09:58:54  <bitcoin-git> bitcoin/master 861f9a2 Matt Corallo: Skip remainder of init if upgrade is cancelled
1312017-08-08T09:58:55  <bitcoin-git> bitcoin/master 2507fd5 Wladimir J. van der Laan: Merge #10998: Fix upgrade cancel warnings...
1322017-08-08T09:59:44  <bitcoin-git> [bitcoin] laanwj closed pull request #10998: Fix upgrade cancel warnings (master...2017-08-fix-upgrade-cancel-warnings) https://github.com/bitcoin/bitcoin/pull/10998
1332017-08-08T10:00:47  *** JackH has joined #bitcoin-core-dev
1342017-08-08T10:10:29  *** johnpark_pj has quit IRC
1352017-08-08T10:11:40  *** johnpark_pj has joined #bitcoin-core-dev
1362017-08-08T10:17:17  *** jrayhawk has quit IRC
1372017-08-08T10:17:43  *** jrayhawk has joined #bitcoin-core-dev
1382017-08-08T10:18:11  *** comboy_ has quit IRC
1392017-08-08T10:19:06  *** Guyver2 has joined #bitcoin-core-dev
1402017-08-08T10:19:49  *** comboy has joined #bitcoin-core-dev
1412017-08-08T10:29:57  <wumpus> doing last checks on 10882, after which I'll merge it. After that I'm planning to tag 0.15.0rc1. Any objections?
1422017-08-08T10:30:55  <wumpus> (ah yes #10483 and #10607 too)
1432017-08-08T10:30:57  <gribble> https://github.com/bitcoin/bitcoin/issues/10483 | scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL by practicalswift · Pull Request #10483 · bitcoin/bitcoin · GitHub
1442017-08-08T10:30:58  <gribble> https://github.com/bitcoin/bitcoin/issues/10607 | scripted-diff: stop using the gArgs wrappers by benma · Pull Request #10607 · bitcoin/bitcoin · GitHub
1452017-08-08T10:31:46  <wumpus> then branching off 0.15, bumping version numbers, then tagging the release
1462017-08-08T10:32:51  <gmaxwell> I think 10882 is still in a state where if you run getnewaddress to the pont where it exhausts your keypool then recieve a transaction your node shuts off.
1472017-08-08T10:33:45  <wumpus> can you coment jnewbery?
1482017-08-08T10:33:51  <wumpus> if so, i'm going to bump it to 0.15.1 instead
1492017-08-08T10:34:21  <jnewbery> Yes, that's the case - I reverted to the commit that had ACKs
1502017-08-08T10:35:08  <wumpus> is it enough of an improvement to merge it like this?
1512017-08-08T10:35:43  <jnewbery> I think it's an improvement
1522017-08-08T10:35:54  <wumpus> (or does it make things worse? I don't know anymore)
1532017-08-08T10:36:16  <gmaxwell> It's a critical improvement, but I think the sequence of events I described should block a release, just like absense of this PR.  Keep in mind that most existing already encrypted wallets have less than 500 keys in their keypool already.
1542017-08-08T10:36:31  <wumpus> you want to block the release on this?
1552017-08-08T10:37:11  <gmaxwell> I think we need to turn HD off without this. We currently have non-trivial funds loss exposure without this.
1562017-08-08T10:37:21  <wumpus> is it a regression since 0.14?
1572017-08-08T10:37:46  <gmaxwell> It was introduced in 0.14 with hdwallet, though we only realized over time all the ways it could fail.
1582017-08-08T10:38:06  <wumpus> I'd really dislike blocking 0.15
1592017-08-08T10:38:21  <wumpus> anyone else think we should delay 0.15?
1602017-08-08T10:38:44  <wumpus> as I understand, the plan is to do a 0.15.1 fairly quick after it anyway
1612017-08-08T10:39:01  <gmaxwell> Agreed! but I don't think it's acceptable to keep shipping stuff we know will basically make people lose funds, and yet I don't think we can ship something that will just fall over on most encrypted wallets in use right now.
1622017-08-08T10:39:03  <wumpus> so every day we delay here delays that, too
1632017-08-08T10:39:09  <gmaxwell> I know.
1642017-08-08T10:39:27  <wumpus> and if it was already broken in 0.14, and still broken in 0.15, releaseing at least won't make things worse
1652017-08-08T10:39:50  <wumpus> delaying just means people will keep using 0.14.x in which it is broken too
1662017-08-08T10:40:23  <gmaxwell> whatever we do next might be fast but it's not going to be days.
1672017-08-08T10:40:27  <wumpus> anyhow, going back to other things then, we can postpone for another day at least
1682017-08-08T10:40:50  <gmaxwell> jnewbery: can you make a second PR with the remaining parts
1692017-08-08T10:41:01  <gmaxwell> I don't understand why that fix was removed.. it's really pretty broken without it.
1702017-08-08T10:41:05  <jnewbery> I plan to
1712017-08-08T10:41:09  <gmaxwell> OK.
1722017-08-08T10:41:39  <jnewbery> because it and the other changes broke a test, and it was suggested I revert to a commit that already had lots of ACKs
1732017-08-08T10:41:52  <wumpus> jnewbery: agree that doing a new PR is better
1742017-08-08T10:42:19  <wumpus> let's just merge 10882 before the review there turns even more into a mess, if some state of a PR is sufficiently accepted and reviewed ideally no new things are added
1752017-08-08T10:42:27  <gmaxwell> (It's not that I don't think we can merge whats there, but I think we can't release with it because it will fail very quickly for basically anyone with a pre-existing encrypted wallet)
1762017-08-08T10:42:45  <wumpus> it will fail for anyone?
1772017-08-08T10:43:03  <wumpus> ok, then I'm not going to merge it, I already had half a hart attack with a 'corrupted' wallet yesterday :p
1782017-08-08T10:43:12  <gmaxwell> wumpus: right now with that PR if you get a transaction while your keypool has under 500 addresses in it, you'll shut down.
1792017-08-08T10:43:13  <jnewbery> yes, I realise now that the combination of 10882 and increasing the keypool defaults probably breaks this for pre-existing wallets
1802017-08-08T10:43:41  <gmaxwell> jnewbery: well this was going to need a look ahead greater than 50 regardless.
1812017-08-08T10:44:33  <wumpus> then why does everyone ack it if it creates a broken situation
1822017-08-08T10:44:42  <gmaxwell> jnewbery: sorry for my earlier comments on it not making it adequately clear that this applies to almost everyone, I just thought that pointing out that it would go down after a getnewaddress loop was enough. :)
1832017-08-08T10:46:24  <luke-jr> wumpus: my only possibly-not-ignored objection I think, is to breaking backward compatibility for RPC :/  (#10989)
1842017-08-08T10:46:30  <gribble> https://github.com/bitcoin/bitcoin/issues/10989 | RPC: Restore backward compatibility, in multiwallet mode by luke-jr · Pull Request #10989 · bitcoin/bitcoin · GitHub
1852017-08-08T10:47:16  <gmaxwell> luke-jr: I think that objection is acknoweldged and set aside. Making MW require the wallet was an intentional decision at least for now that basically everyone supported.
1862017-08-08T10:47:28  <jnewbery> wumpus - I think because it's been open for a long time and gone through many iterations, during which time the keypool defaults were increased. I've only just realised the implications from what gmaxwell said just now
1872017-08-08T10:48:47  <jnewbery> I can hopefully get a minimal PR on top of #10882 in the next hour that includes juse the getnewaddress fix. gmaxwell, will you be able to review today?
1882017-08-08T10:48:50  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
1892017-08-08T10:49:19  <luke-jr> so people will just be forced to switch to Knots to actually use MW I guess, whatever
1902017-08-08T10:50:01  <luke-jr> I suppose that's the same as with previous versions
1912017-08-08T10:50:55  <gmaxwell> luke-jr: come on dude, having to specify which wallet you want is not "having to use".  It's perhaps arguably less useful, but the feature is expiremental in this version.
1922017-08-08T10:51:05  <wumpus> multiwallet is experimental
1932017-08-08T10:51:18  <wumpus> even if it's completely useless in rc1, I'm not going to block the release on that
1942017-08-08T10:51:40  <wumpus> (and I know for sure it's not compltely useless)
1952017-08-08T10:51:58  <luke-jr> gmaxwell: no existing software uses the new API, and as you point out, the new API is experimental.. so there's basically no way to get the primary "just keep wallets in sync" benefit without using a new experimental API
1962017-08-08T10:52:31  <gmaxwell> luke-jr: bitcoin cli works, and many other things are easy to get working...
1972017-08-08T10:52:37  <luke-jr> anyway, I acknowledge that this isn't going to be fixed; I'm just not happy about it.
1982017-08-08T10:52:45  <gmaxwell> I agree it dimishes the wallet warming usecase.
1992017-08-08T10:54:13  <gmaxwell> luke-jr: the concern people have about accidentally interacting with the wrong wallet is a good one though. It's an obvious footgun and it's more conservative to avoid it for now.
2002017-08-08T10:54:53  <gmaxwell> luke-jr: also, perhaps that walletwarming usecase would better be met with a seperate state e.g. a scanwallet=file  where there is no risk of accidentally using it beyond getting some status about it.
2012017-08-08T10:55:09  <luke-jr> gmaxwell: yes, I agree that's a reasonable concern. I don't know a way to address it.
2022017-08-08T10:55:25  <luke-jr> hopefully MW will be more than warming by 0.16 anyway
2032017-08-08T10:55:54  <luke-jr> (I actually think it might have been better to release without *any* RPC MW support, than with what we have now)
2042017-08-08T10:56:24  <luke-jr> (that'd solve the footgun issue, while not breaking the warming use case)
2052017-08-08T10:56:28  <gmaxwell> luke-jr: fwiw, I think it would be more constructive on concerns like this if instead of just saying it's wrong, if you walked us through the specific case that you care about that it screws up.  It's a lot easier to sympatize with your concern if someone understands it, and they can't unless you explain it.  When you just state matter of factly that it's currently wrong or broken, it's random c
2062017-08-08T10:56:34  <gmaxwell> hance if any given person will understand why you think that.
2072017-08-08T10:56:59  <gmaxwell> luke-jr: For many people, updating to make the calls (e.g. cli users) is trivial.
2082017-08-08T10:57:05  <luke-jr> gmaxwell: my understanding was that it's basically the only real use case 0.15 will support MW for
2092017-08-08T10:57:08  <gmaxwell> and being able to actually use MW will be nice.
2102017-08-08T10:57:45  <gmaxwell> luke-jr: that wasn't my understanding. Actually, I didn't even really know people were thinking that much about "warming but not using" until I read the release note text re: GUI.  Which I assume you wrote.
2112017-08-08T10:58:25  <gmaxwell> Though I agree it's a useful thing to do, esp in a world with pruning.. (also rescan times are a seriously negative part of my life these days)
2122017-08-08T10:59:01  <luke-jr> (FWIW, Newbery wrote it)
2132017-08-08T10:59:04  <gmaxwell> I manage cold wallets for myself, for blockstream, for varrious projects... and it's not uncommon that I lose hours of waiting due to having to rescan something.
2142017-08-08T10:59:08  <gmaxwell> ah okay!
2152017-08-08T10:59:49  <jnewbery> yeah - I think for qt-only users warming is the only use in v0.15
2162017-08-08T10:59:59  <luke-jr> I suppose a reasonable compromise is to have a command-line option to set a default wallet, and only support it in that case. But probably too late to get it into 0.15
2172017-08-08T11:00:05  <jnewbery> but for RPC users, multiwallet is fully supported
2182017-08-08T11:00:22  <luke-jr> hmm. can a Qt user even access their wallet via debug console now?
2192017-08-08T11:01:45  <jnewbery> yes, just wallet 0
2202017-08-08T11:01:55  <jnewbery> #10870
2212017-08-08T11:01:56  <gribble> https://github.com/bitcoin/bitcoin/issues/10870 | [Qt] Use wallet 0 in rpc console if running with multiple wallets by jonasschnelli · Pull Request #10870 · bitcoin/bitcoin · GitHub
2222017-08-08T11:03:42  <luke-jr> ah
2232017-08-08T11:05:05  <jnewbery> gmaxwell - I don't think the getnewaddress changes you're asking for in 10882 address the encrypted-wallet-with-fewer-than-500-keys issue that you just raised
2242017-08-08T11:05:29  <jnewbery> basically every HD wallet now has fewer than 500 keys
2252017-08-08T11:06:20  <jnewbery> so on upgrade, everyone with an encrypted HD wallet will have their node shut down and be presented with the 'restart with bypasskeypoolcritical blah blah blah' error
2262017-08-08T11:07:12  <jnewbery> it's not a loss of funds risk, but it's a terrible first user impression of v0.15
2272017-08-08T11:08:52  *** Aaronvan_ has joined #bitcoin-core-dev
2282017-08-08T11:08:52  <gmaxwell> the whole thing is a bit weird. When the wallet is current there is no need for the keypool to have lookahead.
2292017-08-08T11:10:20  <gmaxwell> But I didn't want to get into dictating major changes to it.
2302017-08-08T11:11:40  *** AaronvanW has quit IRC
2312017-08-08T11:13:08  <wumpus> maybe it should only enlarge the keypool the first time the user unlocks the wallet
2322017-08-08T11:13:09  <wumpus> after upgrading
2332017-08-08T11:13:17  <Lauda> Does anyone know what the limiting factor for rescan speed is? I have been running a few imports on a fairly good machine, and I don't see everything being utilized whilst this is taking up a lot of time (single key 2+ hours). Disk I/O maybe?
2342017-08-08T11:13:56  <Lauda> CPU <20% on average, 2 GB of RAM out of 32 (with dbcache 12GB).
2352017-08-08T11:14:05  *** rjak has joined #bitcoin-core-dev
2362017-08-08T11:14:37  <wumpus> rescan could be a lot faster if you'd not care about transaction history, then it'd only have to scan over the utxo set instead of every single block
2372017-08-08T11:15:15  <Lauda> I don't, I just care about the final balance (in this case). Is there an option to ignore the TX history?
2382017-08-08T11:15:48  <Lauda> Seems like rescan speed will keep getting much worse with time, no?
2392017-08-08T11:16:35  <jnewbery> wumpus, the problem is that keypool_critical was changed to 500 during the iterations of this PR (when keypool was changed to 1000). We'd need a way for keypool_critical to be lowered to 50 on the first run after upgrading, or similar. I can't think of a good way to do that
2402017-08-08T11:20:17  <Lauda> I should also note that it is taking over 2 hours on a VPS with an SSD. I wonder how much of a difference it would make with an HDD (% duration increase).
2412017-08-08T11:25:25  *** Aaronvan_ has quit IRC
2422017-08-08T11:31:54  *** AaronvanW has joined #bitcoin-core-dev
2432017-08-08T11:33:22  *** Aaronvan_ has joined #bitcoin-core-dev
2442017-08-08T11:34:42  <gmaxwell> the rescan is slow presumably because it deseralizes each block and each transaction and hashes each trasnaction and does dozens of memory allocations for each transactions.
2452017-08-08T11:35:27  <gmaxwell> It could probably take your scriptpubkeys and txns as bytes and scan the raw block data without deseralizing things and be 500 times faster... but it would be a fair amount of work to implement that.
2462017-08-08T11:36:08  *** AaronvanW has quit IRC
2472017-08-08T11:36:23  <gmaxwell> jnewbery: keypool_criticial being 50 is not really reasonable in any case. it would miss transactions in many of my existing wallets, for example, and I doubt my usage is that atypical.
2482017-08-08T11:36:46  <gmaxwell> jnewbery: I think the bigger point though is that when the wallet is current there is no reason to shut down.
2492017-08-08T11:37:10  <gmaxwell> It doesn't matter if there is look ahead if the wallet hasn't fallen behind.
2502017-08-08T11:37:12  <wumpus> yes, it will get slower with time, assuming you're donig a full rescan
2512017-08-08T11:37:30  <wumpus> if you rescan a more or less fixed number of blocks it should stay at the same speed
2522017-08-08T11:37:41  <wumpus> or at least not become much slower...
2532017-08-08T11:38:10  *** promag has joined #bitcoin-core-dev
2542017-08-08T11:38:44  <wumpus> and yes, rescan as it is could certainly be optimized, though I doubt anyone sees it as that much of a priority
2552017-08-08T11:38:58  <jnewbery> "when the wallet is current there is no reason to shut down." - the getnewaddress change doesn't fix that
2562017-08-08T11:39:04  *** berndj has quit IRC
2572017-08-08T11:39:32  <Lauda> gmaxwell that sounds like a good optimization for the future. Thanks for the responses.
2582017-08-08T11:39:53  <wumpus> you're welcome to give it a try of course
2592017-08-08T11:40:56  <gmaxwell> jnewbery: I realize this. When you suggested that I thought it would be simpler than another solution.
2602017-08-08T11:41:13  *** berndj has joined #bitcoin-core-dev
2612017-08-08T11:44:08  <jnewbery> I must have misunderstood you in the issue. I suggested the getnewaddress change because I thought you were talking about running getnewaddress 500 times. I didn't realise there was an upgrade issue until you mentioned it today
2622017-08-08T11:46:26  <gmaxwell> I was just trying to give an example where the keypool would be below the threshold. I don't really see them as different... end result is the same regardless of how you get there. Sorry.
2632017-08-08T11:47:07  <gmaxwell> e.g. imagine that the crit threshold were still 50, you'd still have existing wallets shut down immediately, or after the user executes a dozen getnewaddresses. :)
2642017-08-08T11:47:33  <gmaxwell> Just fewer of them.
2652017-08-08T11:47:55  <gmaxwell> but they'd mostly all die eventually except for users that unlock frequently before running out.
2662017-08-08T11:50:41  *** promag has quit IRC
2672017-08-08T11:54:59  <jnewbery> ok, so how do we fix this?
2682017-08-08T11:55:12  <jnewbery> Offline keypool topup would be *really* useful
2692017-08-08T11:58:12  <gmaxwell> jnewbery: Can we distinguish the case where we are current vs rescanning? If so, then we just shouldn't perform the shutdown when we're current.
2702017-08-08T12:04:49  <gmaxwell> I think it would still be good to stop newaddresses short of the limit, so that we don't immediately go into shutdown on any attempt to rescan when the pool is empty.. but that is a less criticial improvement.
2712017-08-08T12:05:16  <jnewbery> We rescan from the wallet's best block on startup. So even with that change, we're going to barf on upgrade
2722017-08-08T12:06:36  <jnewbery> (unless you shutdown v0.14, upgrade, start v0.15 before a block has been produced)
2732017-08-08T12:07:00  *** dermoth has quit IRC
2742017-08-08T12:07:01  <gmaxwell> hm. but we don't have those blocks at startup.
2752017-08-08T12:07:13  *** arubi has quit IRC
2762017-08-08T12:07:29  <gmaxwell> when we start, the best block should be the same... though yea, IIRC we do a rescan in any case. I think just as a belt and suspenders check.
2772017-08-08T12:08:00  <jnewbery> oh, ok. Makes sense
2782017-08-08T12:08:05  *** dermoth has joined #bitcoin-core-dev
2792017-08-08T12:09:55  *** arubi has joined #bitcoin-core-dev
2802017-08-08T12:11:42  <morcos> jnewbery: gmaxwell: i dont have all the code handy, but just brainstorming a quick fix for 0.15
2812017-08-08T12:11:53  <morcos> wouldn't it be possible to take advantage of the two different limits
2822017-08-08T12:12:18  <gmaxwell> I think that initial rescan can be disabled, but it's been so long since I've thought about it, there may be reasons that I am not remembering.
2832017-08-08T12:12:24  <morcos> if we set keypoolmin to 500 and keypoolcritical to 25, then don't we not miss anything since our best block has stopped advancing?
2842017-08-08T12:13:00  <gmaxwell> oh. that is an interesting point, but what about wallets with empty keypools
2852017-08-08T12:13:09  <gmaxwell> (below 25)
2862017-08-08T12:13:19  <morcos> then with the release notes we can recommend running walletpasphrase as soon as you startup  so your keypool will top up to the new bigger amounts
2872017-08-08T12:13:42  <morcos> oh wait, that doesnt work exactly...
2882017-08-08T12:13:44  <morcos> shoot
2892017-08-08T12:14:05  <morcos> also it would be potentially a problem for pruned nodes
2902017-08-08T12:14:17  <gmaxwell> need a third number, but then I think it does except for but what if there isn't even 25.. and pruned.
2912017-08-08T12:15:09  <morcos> i need to look at the code, but right now i actually think it is a broken state to set keypoolmin different from keypoolcritical
2922017-08-08T12:15:23  <gmaxwell> morcos: If instead we just make it so that the only time this applies is when the wallet has fallen behind the node tip (or is otherwise rescanned)-- then I think that would be fine.
2932017-08-08T12:15:55  <gmaxwell> You'd still get forced into topping up anytime you imported keys, moved wallets between nodes, etc.. but normal operation wouldn't run into it.
2942017-08-08T12:17:37  *** promag has joined #bitcoin-core-dev
2952017-08-08T12:17:41  <gmaxwell> I believe the reason we do rescanning at tip even when we're in sync (assuming we still do it) was a concern about reorgs and or the wallet not getting flushed consistently with blocks in an unclean shutdown or something like that.
2962017-08-08T12:17:58  <morcos> The scary thing to me is the way we turn off setting the best block, but then potentially turn it back on if we haven't rescanned (thereby missing scanning a bunch of blocks)
2972017-08-08T12:18:20  <morcos> i haven't looked at the most recent code, and maybe ryanofsky's suggestion fixed that i don't remember
2982017-08-08T12:18:59  <morcos> i'm going to stop giving out ideas until i look more closely at the code again
2992017-08-08T12:19:23  <morcos> but for the record wumpus, my view is it would be better to delay 0.15 (even by a week or two) to get this right
3002017-08-08T12:20:13  <morcos> i think we're getting pretty well into understanding all the issues now, so would be good to fix it while it's fresh, and it also seems like a big enough problem, that would be nice to fix before release
3012017-08-08T12:24:34  <sdaftuar> i think #10982 should be locked
3022017-08-08T12:24:36  <gribble> https://github.com/bitcoin/bitcoin/issues/10982 | Disconnect network service bits 6 and 8 until Aug 1, 2018 by TheBlueMatt · Pull Request #10982 · bitcoin/bitcoin · GitHub
3032017-08-08T12:27:24  <gmaxwell> :(
3042017-08-08T12:30:41  *** Aaronvan_ is now known as AaronvanW
3052017-08-08T12:32:11  <jnewbery> morcos - I think ryanofsky's change is helpful. I tried including it but it broke tests, so I rolled back to a previous commit
3062017-08-08T12:34:38  <jnewbery> gmaxwell - ShutdownIfKeypoolCritical() is only called in CreateWalletFromFile() and AddToWalletIfInvolvingMe(). I think your change would be to remove it from CreateWalletFromFile(), and only call it from AddToWalletIfInvolvingMe() if that function was called by SyncTransaction() (and not by ScanForWalletTransactions())
3072017-08-08T12:34:46  *** Aaronvan_ has joined #bitcoin-core-dev
3082017-08-08T12:35:50  <jnewbery> I think I know what I need to do - re-add the getnewaddress changes, re-add ryanofksy's changes about setting best block, and make the change above
3092017-08-08T12:36:36  *** Aaronvan_ is now known as AaronvanW_
3102017-08-08T12:36:38  *** AaronvanW has quit IRC
3112017-08-08T12:40:26  *** AaronvanW_ is now known as AaronvanW
3122017-08-08T12:41:41  <sdaftuar> MarcoFalke: thank you
3132017-08-08T12:41:50  <wumpus> MarcoFalke: you just beat me to it :)
3142017-08-08T12:42:06  <MarcoFalke> Too much mail spam :(
3152017-08-08T12:43:31  <gmaxwell> :-( sorry that i even commented on it at all, it just made it worse
3162017-08-08T12:47:03  *** BashCo_ has joined #bitcoin-core-dev
3172017-08-08T12:49:42  *** BashCo has quit IRC
3182017-08-08T12:55:47  *** promag has quit IRC
3192017-08-08T13:03:37  <jnewbery> ~.
3202017-08-08T13:03:49  <sipa> agree.
3212017-08-08T13:04:50  * sdaftuar wonders what sipa's reasoning is
3222017-08-08T13:07:17  *** Chris_Stewart_5 has joined #bitcoin-core-dev
3232017-08-08T13:09:51  *** promag has joined #bitcoin-core-dev
3242017-08-08T13:33:21  *** justanotheruser has quit IRC
3252017-08-08T13:39:23  *** Austindoggie has joined #bitcoin-core-dev
3262017-08-08T13:45:51  *** justanotheruser has joined #bitcoin-core-dev
3272017-08-08T13:45:56  *** justanotheruser has quit IRC
3282017-08-08T13:46:18  *** justanotheruser has joined #bitcoin-core-dev
3292017-08-08T13:52:50  *** laurentmt has joined #bitcoin-core-dev
3302017-08-08T13:56:48  *** laurentmt has quit IRC
3312017-08-08T14:00:39  *** SopaXorzTaker has quit IRC
3322017-08-08T14:01:35  *** timothy has quit IRC
3332017-08-08T14:01:35  *** drizztbsd has joined #bitcoin-core-dev
3342017-08-08T14:19:26  *** promag has quit IRC
3352017-08-08T14:23:29  *** btcdrak has quit IRC
3362017-08-08T14:25:45  *** Guyver2_ has joined #bitcoin-core-dev
3372017-08-08T14:26:15  *** Murch has joined #bitcoin-core-dev
3382017-08-08T14:26:50  *** promag has joined #bitcoin-core-dev
3392017-08-08T14:28:17  *** Guyver2 has quit IRC
3402017-08-08T14:28:18  *** Guyver2_ is now known as Guyver2
3412017-08-08T14:32:00  *** miknotauro has joined #bitcoin-core-dev
3422017-08-08T14:34:49  *** SopaXorzTaker has joined #bitcoin-core-dev
3432017-08-08T14:40:07  *** promag has quit IRC
3442017-08-08T14:46:41  *** miknotauro has quit IRC
3452017-08-08T14:48:30  <wumpus> jnewbery: maybe reverting the default mempool size increase for 0.15.0 will make this easier?
3462017-08-08T14:50:10  *** miknotauro has joined #bitcoin-core-dev
3472017-08-08T14:50:34  *** promag has joined #bitcoin-core-dev
3482017-08-08T15:02:03  *** timothy has joined #bitcoin-core-dev
3492017-08-08T15:03:01  *** drizztbsd has quit IRC
3502017-08-08T15:03:10  <Chris_Stewart_5> How long do we support gcc versions? Curious for #8469
3512017-08-08T15:03:12  <gribble> https://github.com/bitcoin/bitcoin/issues/8469 | [POC] Introducing property based testing to Core by Christewart · Pull Request #8469 · bitcoin/bitcoin · GitHub
3522017-08-08T15:04:17  <wumpus> Chris_Stewart_5: there's no fixed policy on that - the preference is to support all compilers unless there's a good reason not to, such as lack of c++11 support
3532017-08-08T15:15:32  *** dcousens has quit IRC
3542017-08-08T15:19:02  <jnewbery> wumpus: yes, that would resolve the upgrade issue, although I'm sure gmaxwell will argue against that
3552017-08-08T15:33:22  <wumpus> sure, I just mean maybe we can't have everything in a timeframe soon enough for 0.15.0 and we need to make choices
3562017-08-08T15:35:11  <luke-jr> Chris_Stewart_5: it would be pretty bad to not build on popular stable OSs
3572017-08-08T15:36:07  *** Aaronvan_ has joined #bitcoin-core-dev
3582017-08-08T15:36:12  <luke-jr> so I'd check Fedora, Debian, RedHat/CentOS, Gentoo, Ubuntu at least
3592017-08-08T15:36:28  <luke-jr> IIRC typically the hold-back is CentOS
3602017-08-08T15:37:32  <wumpus> luke-jr: it doesn't even work on travis
3612017-08-08T15:37:54  <luke-jr> heh
3622017-08-08T15:38:16  <wumpus> gcc 4.8 isn't *that* old and crappy yet, I doubt we should drop support just to be able to do tests in a certain way
3632017-08-08T15:38:24  *** AaronvanW has quit IRC
3642017-08-08T15:38:52  <wumpus> and the tests working on travis is kind of important, so disabling them on gcc 4.8 isn't going to fly either
3652017-08-08T15:39:02  <luke-jr> right
3662017-08-08T15:39:30  <bitcoin-git> [bitcoin] practicalswift opened pull request #11007: [qt] Fix memory leak when loading a corrupted wallet file (master...wallet-corrupted-leak) https://github.com/bitcoin/bitcoin/pull/11007
3672017-08-08T15:39:54  <Chris_Stewart_5> Yes, 4.8.4 was released in Dec of 2014
3682017-08-08T15:40:22  <Chris_Stewart_5> ancient in terms of bitcoin! :P
3692017-08-08T15:40:22  <sam_c> gentoo is about to mask 4.8
3702017-08-08T15:43:46  <wumpus> besides that, we already got lots of complaints that we're requiring a c++11 compiler, I"d prefer to keep the compiler requirement the same for the forseeable future
3712017-08-08T15:46:01  *** BashCo_ has quit IRC
3722017-08-08T15:46:37  *** BashCo has joined #bitcoin-core-dev
3732017-08-08T15:46:44  <Chris_Stewart_5> wumpus: Do we have a 'rough' estimate on when 0.15.0 will be done?
3742017-08-08T15:46:56  *** Austindoggie_ has joined #bitcoin-core-dev
3752017-08-08T15:50:16  *** Austindoggie_ has quit IRC
3762017-08-08T15:50:52  *** Austindoggie has quit IRC
3772017-08-08T15:51:00  *** BashCo has quit IRC
3782017-08-08T15:52:45  <wumpus> Chris_Stewart_5: yes, rc1 was planned for the 6th, we've slipped two days now
3792017-08-08T15:54:41  <wumpus> (which is perfectly normal)
3802017-08-08T15:54:59  *** ekerstein has joined #bitcoin-core-dev
3812017-08-08T15:56:28  *** jamesob has joined #bitcoin-core-dev
3822017-08-08T16:04:56  <luke-jr> sam_c: is it? why?
3832017-08-08T16:05:30  <luke-jr> actually, it looks like it's been masked since May, just because it's old O.o
3842017-08-08T16:08:02  <Chris_Stewart_5> cool, was just curious
3852017-08-08T16:09:32  *** BashCo has joined #bitcoin-core-dev
3862017-08-08T16:30:01  *** promag has quit IRC
3872017-08-08T16:31:07  *** Aaronvan_ is now known as AaronvanW
3882017-08-08T16:32:04  *** miknotauro has quit IRC
3892017-08-08T16:43:36  *** JackH has quit IRC
3902017-08-08T16:49:23  *** promag has joined #bitcoin-core-dev
3912017-08-08T16:55:52  *** jimpo has joined #bitcoin-core-dev
3922017-08-08T16:56:32  *** timothy has quit IRC
3932017-08-08T16:57:14  *** d_t has joined #bitcoin-core-dev
3942017-08-08T16:58:10  *** promag has quit IRC
3952017-08-08T17:03:30  *** JackH has joined #bitcoin-core-dev
3962017-08-08T17:16:03  *** corinrose_ has joined #bitcoin-core-dev
3972017-08-08T17:32:49  *** chjj has quit IRC
3982017-08-08T17:38:54  *** Aaronvan_ has joined #bitcoin-core-dev
3992017-08-08T17:39:13  *** AaronvanW has quit IRC
4002017-08-08T17:41:16  *** Chris_Stewart_5 has quit IRC
4012017-08-08T17:54:17  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4022017-08-08T18:12:52  *** Chris_Stewart_5 has quit IRC
4032017-08-08T18:18:15  *** chjj has joined #bitcoin-core-dev
4042017-08-08T18:21:28  *** clarkmoody has joined #bitcoin-core-dev
4052017-08-08T18:23:32  *** wvr has joined #bitcoin-core-dev
4062017-08-08T18:25:55  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4072017-08-08T18:26:37  *** jannes has quit IRC
4082017-08-08T18:27:06  *** Aaronvan_ is now known as AaronvanW
4092017-08-08T18:32:59  *** kexkey has joined #bitcoin-core-dev
4102017-08-08T18:33:07  *** jannes has joined #bitcoin-core-dev
4112017-08-08T18:33:23  *** clarkmoody_ has joined #bitcoin-core-dev
4122017-08-08T18:37:07  *** clarkmoody has quit IRC
4132017-08-08T18:37:41  *** kexkey has quit IRC
4142017-08-08T18:38:36  *** chjj has quit IRC
4152017-08-08T18:39:24  *** kexkey has joined #bitcoin-core-dev
4162017-08-08T18:43:40  *** abpa has joined #bitcoin-core-dev
4172017-08-08T18:49:07  *** kexkey has quit IRC
4182017-08-08T18:51:57  *** chjj has joined #bitcoin-core-dev
4192017-08-08T18:53:34  *** Dizzle has joined #bitcoin-core-dev
4202017-08-08T18:56:21  *** echonaut has quit IRC
4212017-08-08T18:56:36  *** echonaut has joined #bitcoin-core-dev
4222017-08-08T18:58:16  <jcorgan> congrats
4232017-08-08T18:59:10  *** ekerstein has quit IRC
4242017-08-08T18:59:43  <AaronvanW> +1
4252017-08-08T19:00:40  <arubi> when's "status" going to change to "locked-in" ?
4262017-08-08T19:00:40  <arubi> I mean, congrats!  :)  but also that
4272017-08-08T19:00:46  <gmaxwell> oh I guess, barring no reorgs segwit lockin is now guarenteed.
4282017-08-08T19:01:21  <gmaxwell> arubi: the thing happening now is that there aren't enough blocks left to prevent a lock in, but it won't lock in for another day or so IIRC.
4292017-08-08T19:01:40  <arubi> so I was just informed it happens at the end of the period
4302017-08-08T19:01:48  *** timothy has joined #bitcoin-core-dev
4312017-08-08T19:01:55  <gmaxwell> so even if all blocks for the next of the window don't signal segwit it will still lock in.
4322017-08-08T19:02:31  <jcorgan> something like 16-17 hours from now
4332017-08-08T19:02:41  <arubi> right, my confusion was re. the 'getblockchaininfo' "status" filed to changing after 95% rather than at the end of the period
4342017-08-08T19:02:43  <jcorgan> for end of period
4352017-08-08T19:03:04  <Chris_Stewart_5> Mmmm ok
4362017-08-08T19:04:02  *** chjj has quit IRC
4372017-08-08T19:16:18  *** chjj has joined #bitcoin-core-dev
4382017-08-08T19:19:58  <gmaxwell> 12:19:19 <@betawaffle> gmaxwell: can you congratulate everyone involved in segwit for us?
4392017-08-08T19:20:10  <sdaftuar> i guess we can cross "activate segwit" off the list of action items
4402017-08-08T19:21:47  <gmaxwell> \O/
4412017-08-08T19:22:16  <jnewbery> gmaxwell morcos wumpus (and anyone else who's interested in wallet topup). I've pushed a branch which I think covers gmaxwell's requested logic:
4422017-08-08T19:22:19  <jnewbery> - don't shutdown the node on startup if keypool is < keypool_critical
4432017-08-08T19:22:21  <jnewbery>     - don't shutdown the node if wallet is current
4442017-08-08T19:22:24  <jnewbery>     - DO shutdown the node if rescanning and keypool drops below
4452017-08-08T19:22:26  <jnewbery>     keypool_critical
4462017-08-08T19:22:27  <jnewbery> #10882
4472017-08-08T19:22:30  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
4482017-08-08T19:22:32  <gmaxwell> When does the period end? exactly, it would kinda be nice if rc1 lands just after that, so that miners running BIP-91 code can safely try rc1.
4492017-08-08T19:23:00  <jnewbery> getnewaddress should also be safe
4502017-08-08T19:23:28  <gmaxwell> sdaftuar: now we just need to get people to modulate their hashrate so that the activation coincides with the solar eclipse. It's pretty close.
4512017-08-08T19:23:29  <jnewbery> also includes ryanofsky's m_update_best_block
4522017-08-08T19:23:39  <gmaxwell> (quick, someone make more spinoffs)
4532017-08-08T19:23:54  <luke-jr> lol
4542017-08-08T19:24:57  <gmaxwell> jnewbery: where is the branch?
4552017-08-08T19:25:36  <jnewbery> #10882
4562017-08-08T19:25:39  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
4572017-08-08T19:26:07  <gmaxwell> jnewbery: oh great
4582017-08-08T19:27:08  <jnewbery> This should fix the upgrade issue. bitcoind v0.15 won't shutdown first time you run it with an old encrypted HD wallet
4592017-08-08T19:31:17  <jnewbery> but if you don't unlock and topup your keypool before you shutdown the node, then I expect it'll refuse to start the next time (since your node tip will advance but your wallet best block won't)
4602017-08-08T19:31:42  <gmaxwell> Right. We may need to do a bit of warning related to this pattern still:
4612017-08-08T19:31:55  <gmaxwell> wallet is below 500 because it's old and encrypted.
4622017-08-08T19:32:09  <gmaxwell> you start, life is good, but you are not updating the best block.
4632017-08-08T19:32:29  <gmaxwell> then you restart, and now the wallet is behind.
4642017-08-08T19:32:39  <gmaxwell> it rescans, and life becomes sad.
4652017-08-08T19:33:59  <jnewbery> yes, sad, but better than any alternative as far as I can see
4662017-08-08T19:34:35  <jnewbery> there is a warning that your keypool is below keypool_min and your best block isn't advancing
4672017-08-08T19:35:30  <jnewbery> 2017-08-08 19:19:20 Keypool has fallen below keypool_min (500). Wallet will no longer watch for new transactions and best block height will not be advanced.
4682017-08-08T19:35:33  <jnewbery> 2017-08-08 19:19:20 Unlock wallet, top up keypool and rescan to resume watching for new transactions.
4692017-08-08T19:35:55  <jnewbery> Perhaps that could be made more explicit "TOP UP KEYPOOL BEFORE NEXT RESTART!"
4702017-08-08T19:39:33  <luke-jr> Can we add a new checkpoint? (Minimally invasive way to lock Segwit in hard)
4712017-08-08T19:40:04  <gmaxwell> luke-jr: No.
4722017-08-08T19:40:18  <gmaxwell> Thats not a proper or interesting thing to do.
4732017-08-08T19:40:37  <luke-jr> just to eliminate the FUD around miners potentially reorging it out
4742017-08-08T19:41:11  <gmaxwell> luke-jr: you should do what we would eventually do, make a patch that removes the activation and just replaces it with a height comparison.
4752017-08-08T19:41:24  <luke-jr> Isn't it too late for that in 0.15?
4762017-08-08T19:42:37  <gmaxwell> probably, but we should still write the patch.
4772017-08-08T19:43:03  <luke-jr> to be clear: I meant the new checkpoint specifically for 0.15.0 only
4782017-08-08T19:43:26  <gmaxwell> luke-jr: but just pinning the consensus isn't a right thing to do, and especially violating the rules for setting the things.
4792017-08-08T19:43:39  <luke-jr> "violating the rules for setting the things."?
4802017-08-08T19:44:10  <gmaxwell> luke-jr: there is a spec for setting that requires them to be set at least 2016 blocks in the past.
4812017-08-08T19:44:23  <luke-jr> it will be by the time 0.15.0 is released
4822017-08-08T19:44:23  <gmaxwell> to avoid legislating the ledger state.
4832017-08-08T19:44:59  <gmaxwell> also neighter of these things will prevent the reorg around the activation and steal outputs along the way.
4842017-08-08T19:45:21  <gmaxwell> luke-jr: it's important to enforce these things by the time we'd issue SW addresses, less so before.
4852017-08-08T19:45:37  <luke-jr> hmm, true, I guess we'd need to be checkpointing the actual activation block for that. and there isn't 2016 blocks there.
4862017-08-08T19:46:02  <luke-jr> gmaxwell: Core is more than just a wallet, and wallets besides Core exist..
4872017-08-08T19:46:07  <sdaftuar> while on the subject of cleaning up technical debt associated with prior soft fork deployments: #10695 could use review
4882017-08-08T19:46:08  <gribble> https://github.com/bitcoin/bitcoin/issues/10695 | [qa] Rewrite BIP65/BIP66 functional tests by sdaftuar · Pull Request #10695 · bitcoin/bitcoin · GitHub
4892017-08-08T19:46:13  <sdaftuar> or merge
4902017-08-08T19:46:49  <sdaftuar> we should write the csv-height-activation patch first imo
4912017-08-08T19:46:59  <jnewbery> 10695 looks good to me
4922017-08-08T19:48:07  <sdaftuar> jnewbery: did we ever fix the ComparisonTestFramework for the bug i mentioned in the OP of that PR?
4932017-08-08T19:48:11  *** Giszmo has joined #bitcoin-core-dev
4942017-08-08T19:49:37  *** chjj has quit IRC
4952017-08-08T19:50:03  <jnewbery> I think we had buy-in for rewriting the ComparisonTestFramework scripts using the standard BitcoinTestFramework
4962017-08-08T19:50:34  <jnewbery> In fact I've already done that for some of them, but it depends on TestNode and will several months of rebasing
4972017-08-08T19:50:45  <jnewbery> *will need
4982017-08-08T19:50:49  <sdaftuar> re-reading my message there though i think there was an immediate bug that makes our comparisonTestFramework tests (perhaps including p2p-fullblocktest?) not test anything.
4992017-08-08T19:50:52  <gmaxwell> jnewbery: I'm still feeling this solution is incomplete. For example, one of my wallets (though not hd) is a encrypted wallet I use for watching cold funds. I'm never going to unlock the darn thing, its keypool is empty right now.   Maybe what we should do for this release is merge the scan and mark as use and topup if possible but instead of shutting down, spam errors that the keypool ran out a
5002017-08-08T19:50:58  <gmaxwell> nd your wallet balance may be incorrect.
5012017-08-08T19:51:50  <jnewbery> we don't shutdown for non-HD wallets
5022017-08-08T19:52:18  <gmaxwell> jnewbery: I know we don't, but my useage pattern wouldn't be any different if this wallet were HD.
5032017-08-08T19:53:52  <gmaxwell> I still don't think we have this handling cooked, in that we'll make 0.15 unusable for at least some people or very awkward.  I think we can do better even if I don't know how yet. But it's critical we do something, because without the mark use thing there are several ways to result in serious funds loss. (including giving customers addresses you already gave to other customers, then crediting t
5042017-08-08T19:53:58  <gmaxwell> he wrong accounts when people make payments..)
5052017-08-08T19:58:08  *** spudowiar has joined #bitcoin-core-dev
5062017-08-08T19:58:12  *** spudowiar has left #bitcoin-core-dev
5072017-08-08T19:59:32  *** paveljanik has joined #bitcoin-core-dev
5082017-08-08T19:59:32  *** paveljanik has joined #bitcoin-core-dev
5092017-08-08T20:00:00  <sdaftuar> gmaxwell: in your watching cold funds example, would restarting with a -keypoolmin=0 be sufficient for the use case?
5102017-08-08T20:01:17  <sdaftuar> (perhaps i don't quite understand the PR though, so ignore me if my question doesn't make sense)
5112017-08-08T20:01:49  *** chjj has joined #bitcoin-core-dev
5122017-08-08T20:03:38  <gmaxwell> Yes. It is. I suppose we can just release note that. "If you want to run like this, set that." good point.
5132017-08-08T20:04:52  <jnewbery> thanks sdaftuar. I think that covers greg's use case
5142017-08-08T20:05:48  <jnewbery> gmaxwell can we try to evealuate whether this fixes the immediate problem (potential funds loss). We can fix up awkward behaviour post-branch
5152017-08-08T20:06:42  <gmaxwell> my concern there is that we also need to not introduce something that leaves lots (?) of people stuck. But sdaftuar answers how it doesn't.
5162017-08-08T20:13:12  <sdaftuar> has anyone given thought to how these keypool parameters work with multiwallet?
5172017-08-08T20:13:25  *** somenick_29345 has joined #bitcoin-core-dev
5182017-08-08T20:14:04  *** chjj has quit IRC
5192017-08-08T20:14:44  *** chjj has joined #bitcoin-core-dev
5202017-08-08T20:15:33  <sdaftuar> eg if we advise people to use -keypoolmin=0 to solve one situation, and they load up two wallets, are they jeopardizing themselves?
5212017-08-08T20:17:25  *** SopaXorzTaker has quit IRC
5222017-08-08T20:17:54  <jnewbery> yes, if you use -keypoolmin=0 you can get into a state where your best block has advanced while your keypool is depleted
5232017-08-08T20:17:55  *** clarkmoody_ has quit IRC
5242017-08-08T20:18:08  <jnewbery> and so potentially miss transactions
5252017-08-08T20:18:19  <bitcoin-git> [bitcoin] gmaxwell opened pull request #11008: Enable disablesafemode by default. (master...no-safe-mode) https://github.com/bitcoin/bitcoin/pull/11008
5262017-08-08T20:19:12  <jnewbery> but keypoolmin is a hidden debug option. I don't think we're going to be advertising it widely
5272017-08-08T20:20:14  <bitcoin-git> [bitcoin] rawodb opened pull request #11009: Add information about the next state in getblockchaininfo rpc request (master...master) https://github.com/bitcoin/bitcoin/pull/11009
5282017-08-08T20:23:44  <jonasschnelli> sdaftuar, luke-jr: about NODE_NETWORK_LIMITED_*. If we would treat them independently, would this mean, a peer signalling NODE_NETWORK_LIMITED_HIGH will always singal NODE_NETWORK_LIMITED_LOW?
5292017-08-08T20:23:55  <jonasschnelli> And there would be no special case for a peer signaling both bits?
5302017-08-08T20:24:10  <jonasschnelli> I agree that would make most sense.
5312017-08-08T20:24:16  <sdaftuar> jonasschnelli: in my opinion, a peer should signal both if it offers both
5322017-08-08T20:24:24  <bitcoin-git> [bitcoin] rawodb closed pull request #11009: Add information about the next state in getblockchaininfo rpc request (master...master) https://github.com/bitcoin/bitcoin/pull/11009
5332017-08-08T20:24:57  <sdaftuar> that way if someone wants to write a bitcoin network client that only worries about one of those properties, they don't need to do any complicated reasoning
5342017-08-08T20:25:01  <jonasschnelli> We would just loose the third state for the two new bits for pure logical consistence reasons. But I guess this makes sense.
5352017-08-08T20:25:42  <sdaftuar> i would actually suggest that we rename the service bits too, if it's not too much bikeshedding...  NODE_NETWORK is a non-descriptive term for full node to begin with
5362017-08-08T20:25:57  *** jimpo has quit IRC
5372017-08-08T20:25:57  <sdaftuar> and putting LIMITED in the name implies denial of "full" services, which i think isn't want we want
5382017-08-08T20:25:58  <jonasschnelli> We only have 24 useful service bits (~10 are gone)
5392017-08-08T20:26:19  <bitcoin-git> [bitcoin] rawodb opened pull request #11010: Add information about the next state in getblockchaininfo rpc request (master...pr/getblockchaininfo) https://github.com/bitcoin/bitcoin/pull/11010
5402017-08-08T20:26:46  <jonasschnelli> sdaftuar: Agree with renaming
5412017-08-08T20:26:56  <jonasschnelli> sdaftuar: any proposals?
5422017-08-08T20:26:57  <sdaftuar> i guess i haven't come up with good alternatives yet, but something which means "NODE_RECENT_BLOCKS" or "NODE_REALLY_RECENT_BLOCKS" is along the lines of what i'm thinking
5432017-08-08T20:27:07  <sdaftuar> (those are not actual proposals, just intuition)
5442017-08-08T20:28:02  <jonasschnelli> The question is also if NODE_NETWORK implies more then just historical block responses...
5452017-08-08T20:29:50  <sdaftuar> yeah that's a good question, it's not obvious to me
5462017-08-08T20:30:09  <sdaftuar> ie we have non-NODE_NETWORK nodes (pruning nodes) which do all the rest, already
5472017-08-08T20:31:00  <jonasschnelli> I guess there is no concrete definition what a peer must confirm to to allow signalling NODE_NETWORK (expect the reference implementation)
5482017-08-08T20:31:31  <sipa> yeah, node_network just means "all the things" :)
5492017-08-08T20:31:37  <jonasschnelli> also, SPV is also ~NODE_NETWORK (!= pruned)
5502017-08-08T20:31:53  <jonasschnelli> ~(== bit unset)
5512017-08-08T20:32:07  <sdaftuar> right, but SPV is certainly allowed to eg relay addresses... not sure if any do.
5522017-08-08T20:32:21  <jonasschnelli> Which results in that NODE_NETWORK_LIMITED somehow includes tx/block relay
5532017-08-08T20:35:30  <sipa> i guess it's reasonable to suggest that every node can participate in address murmuring, regardless of service flags
5542017-08-08T20:35:36  <sipa> it's an ootional thing regardless
5552017-08-08T20:36:04  <sipa> since compact blocks, it doesn't really make sense to have separate services for tx relay and block relay
5562017-08-08T20:36:47  <sipa> so the only question is whether we want to have a separate bit for relay vs historical fetch (even if it's just 144 blocks deep)
5572017-08-08T20:37:38  <sipa> any client right now will want both
5582017-08-08T20:38:16  <jonasschnelli> sipa: for a possible use case where a peer serves historical blocks but won't relay (pure block storage)?
5592017-08-08T20:38:36  <sdaftuar> if no clients currently offer one but not the other, i don't think it makes sense to split that now.  we can wait til someone wants to build that.
5602017-08-08T20:39:05  <jonasschnelli> Agree
5612017-08-08T20:40:21  *** ula|2 has joined #bitcoin-core-dev
5622017-08-08T20:41:00  <sipa> sdaftuar: right... the only downside is that it'd cost another bit later
5632017-08-08T20:41:07  *** AaronvanW has quit IRC
5642017-08-08T20:42:07  <sdaftuar> sipa: we either take that 1 bit cost now or later, right?  unless there's a proposal for distinguishing now that saves a bit, which i missed?
5652017-08-08T20:43:37  *** Aaronvan_ has joined #bitcoin-core-dev
5662017-08-08T20:46:46  <sipa> sdaftuar: if we now have 3 bits (relay_tx_and_blocks, blocks_144, blocks_1008), we're good. if we only have (relay_tx_and_blocks_up_to_144, blocks_1008), we may need 2 extra bits later (relay_tx_blocks, blocks_144) totalling 4 bits, where one just implies two other ones
5672017-08-08T20:47:18  *** timothy has quit IRC
5682017-08-08T20:48:58  <jonasschnelli> I think the current definition of NODE_NETWORK_LIMITED_LOW is:
5692017-08-08T20:49:05  <jonasschnelli> NODE_NETWORK_LIMITED_LOW means the same as NODE_NETWORK with the limitation of only serving the last 288 (2 day) blocks
5702017-08-08T20:49:23  <jonasschnelli> (coupled to the definition of NODE_NETWORK)
5712017-08-08T20:49:42  <jonasschnelli> Changing the block value is possible though the client/protocol version?
5722017-08-08T20:50:46  <jonasschnelli> If we want relay as an extra option (assume full block SPV may want to service historical blocks but not relay) another bit would be required
5732017-08-08T20:51:47  *** ula|2 has quit IRC
5742017-08-08T20:54:13  <sdaftuar> sipa: i think blocks_144 and blocks_1008 without relay_tx_and_blocks are sort of nonsensical; by definition you're always willing to provide the current tip, and you never have guarantees that transactions will be relayed to you eg due to policy differences, so i don't think of that as much of a promise.
5752017-08-08T20:54:18  <sdaftuar> but putting that aside:
5762017-08-08T20:54:35  <sdaftuar> i think we could downgrade those service bits in the future to "may or may not relay blocks and transactions"
5772017-08-08T20:54:43  <sdaftuar> if we wanted to save a bit
5782017-08-08T20:54:48  <sdaftuar> oh
5792017-08-08T20:55:02  <sdaftuar> i guess i'm not sure what i'm comparing...
5802017-08-08T20:55:58  <sdaftuar> i'm taking jonas' proposal as relay_tx_and_blocks_up_to_144, relay_tx_and_blocks_up_to_1008
5812017-08-08T20:56:04  <sipa> sdaftuar: but there is a technical difference
5822017-08-08T20:56:21  <sdaftuar> and suggesting we could in the future add a single bit saying relay_tx_and_blocks
5832017-08-08T20:56:51  <sdaftuar> and at the point we add that, we say that there are no promises on the quality of service you get from older nodes that aren't setting the new bit
5842017-08-08T20:57:03  <sipa> sdaftuar: for example relay_blocks_tx could imply support for tx/inv(tx)/cmpctblock/blocktxn/getdata(tx), while the other implies support for getblock/getdata(block)/inv(block)
5852017-08-08T20:57:21  <sdaftuar> cmpctblock and blocktxn are already separately negotiated
5862017-08-08T20:57:55  <sipa> but not advertized
5872017-08-08T20:58:01  <sipa> (right?)
5882017-08-08T20:58:04  <sdaftuar> right
5892017-08-08T20:58:15  <sipa> they're implied by NODE_NETWORK + sufficient protocol version
5902017-08-08T20:58:32  <sdaftuar> well they're implied by NODE_SEGWIT, for all practical purposes
5912017-08-08T20:58:39  <sdaftuar> but interesting point
5922017-08-08T20:58:50  *** Giszmo has quit IRC
5932017-08-08T20:59:04  <sdaftuar> i'm not sure it's reasonable for service bits to keep up with p2p protocol improvements
5942017-08-08T20:59:09  <sipa> agree
5952017-08-08T21:00:46  <gmaxwell> I'm not enthused by all these flags because in general we should not be segmenting up the topology if we can avoid it.
5962017-08-08T21:01:43  <sdaftuar> i agree with that.  i also generally want to punt on features we are not sure anyone will implement.
5972017-08-08T21:02:26  <sipa> agree
5982017-08-08T21:02:57  <sipa> just pointing out that it may cost us an extra bit in the future - perhaps that's ok
5992017-08-08T21:03:05  *** Giszmo has joined #bitcoin-core-dev
6002017-08-08T21:06:16  *** ula has joined #bitcoin-core-dev
6012017-08-08T21:07:21  <bitcoin-git> [bitcoin] MarcoFalke pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/2507fd55568b...929fd7276c0f
6022017-08-08T21:07:22  <bitcoin-git> bitcoin/master d4f0d87 Suhas Daftuar: [qa] Rewrite BIP65 functional tests...
6032017-08-08T21:07:22  <bitcoin-git> bitcoin/master 4ccc12a Suhas Daftuar: [qa] Rewrite BIP66 functional tests...
6042017-08-08T21:07:23  <bitcoin-git> bitcoin/master 929fd72 MarcoFalke: Merge #10695: [qa] Rewrite BIP65/BIP66 functional tests...
6052017-08-08T21:07:46  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #10695: [qa] Rewrite BIP65/BIP66 functional tests (master...2017-06-fix-bip65-test) https://github.com/bitcoin/bitcoin/pull/10695
6062017-08-08T21:11:09  *** Guyver2 has quit IRC
6072017-08-08T21:27:37  <jonasschnelli> Should peers avoid serving blocks above the NODE_NETWORK_LIMITED_LOW|HIGH threshold to reduce finger printing opportunities?
6082017-08-08T21:28:49  <jonasschnelli> Assume you have pruned down to 10k blocks and signalling NODE_NETWORK_LIMITED_HIGH, one could fingerprint by checking how deep down you can serve blocks
6092017-08-08T21:30:22  <sipa> yes, i think so
6102017-08-08T21:30:36  *** somenick_29345 has quit IRC
6112017-08-08T21:30:37  <jonasschnelli> Okay. Will update the BIP
6122017-08-08T21:31:06  <jonasschnelli> Or maybe just the implementation... need to check
6132017-08-08T21:31:34  *** clarkmoody has joined #bitcoin-core-dev
6142017-08-08T21:36:02  <bitcoin-git> [bitcoin] rawodb closed pull request #11010: Add information about the next state in getblockchaininfo rpc request (master...pr/getblockchaininfo) https://github.com/bitcoin/bitcoin/pull/11010
6152017-08-08T22:02:37  *** instagibbs has quit IRC
6162017-08-08T22:07:16  *** instagibbs has joined #bitcoin-core-dev
6172017-08-08T22:23:25  *** Yogaqueef has quit IRC
6182017-08-08T22:25:17  *** chjj has quit IRC
6192017-08-08T22:26:38  *** Cheeseo has joined #bitcoin-core-dev
6202017-08-08T22:27:03  <molz> Dear Core Devs, Thank you for all your hard work!  https://i.imgflip.com/1trxh9.jpg
6212017-08-08T22:29:55  <sipa> molz: yw!
6222017-08-08T22:32:16  <mryandao> could anyone from here answer this: https://bitcoin.stackexchange.com/questions/57416/since-bitcoin-core-0-14-0-how-does-a-node-with-default-settings-compute-the-dus
6232017-08-08T22:35:52  <Murch> sipa: Transaction overhead is 10 bytes for P2PKH transactions, since segwit adds flag and marker, which however go into the witness part am I correct to calculate the weight of the tx overhead as 42?
6242017-08-08T22:36:24  <Murch> For a transaction that spends at least one P2SHP2PWSH input
6252017-08-08T22:37:03  <sipa> Murch: looks right to me
6262017-08-08T22:37:10  <sipa> mryandao: yes, will do if i find the time
6272017-08-08T22:37:13  <Murch> thank you :)
6282017-08-08T22:37:34  *** chjj has joined #bitcoin-core-dev
6292017-08-08T22:40:16  *** Chris_Stewart_5 has quit IRC
6302017-08-08T22:53:03  <sipa> mryandao: achow101 beat me to it :)
6312017-08-08T22:53:12  <achow101> :)
6322017-08-08T22:54:39  <bitcoin-git> [bitcoin] gmaxwell opened pull request #11011: [Trivial] Add a comment on the use of prevector in script. (master...201708-prevector-comment) https://github.com/bitcoin/bitcoin/pull/11011
6332017-08-08T23:11:06  *** jouke has quit IRC
6342017-08-08T23:13:13  *** jouke has joined #bitcoin-core-dev
6352017-08-08T23:13:13  *** jouke has quit IRC
6362017-08-08T23:13:13  *** jouke has joined #bitcoin-core-dev