12017-01-03T00:27:26  *** justan0theruser has joined #bitcoin-core-dev
  22017-01-03T00:30:31  *** justanotheruser has quit IRC
  32017-01-03T00:35:20  *** dviola has joined #bitcoin-core-dev
  42017-01-03T00:37:28  *** lejitz has joined #bitcoin-core-dev
  52017-01-03T00:50:27  *** justan0theruser is now known as justanotheruser
  62017-01-03T00:55:15  <dviola> hello, the archlinux bitcoin-qt package maintainer builds bitcoin-qt using "--with-incompatible-bdb" and bdb 5.3.28, I have heard there could be potential problems by taking a wallet.dat that was created with newer bdb and then using it on a bitcoin-qt that was compiled with an older version of bdb, are there any tests I could run to know I won't run into issues?
  72017-01-03T00:57:26  *** Ylbam has quit IRC
  82017-01-03T01:00:23  <phantomcircuit> dviola, 5.x has a different format than 4.8 which is not backwards compatible
  92017-01-03T01:00:33  <phantomcircuit> if you open a wallet in 5.x it wont open in 4.x
 102017-01-03T01:00:43  <phantomcircuit> so that's a one way trip
 112017-01-03T01:08:09  <dviola> phantomcircuit: I see, so would it be better if I create the wallet using the bitcoin-qt builds from bitcoin.org?
 122017-01-03T01:09:21  <dviola> I'm a little confused because I created a wallet.dat using Arch's build, then used the wallet.dat with bitcoin-qt from bitcoin.org, and it seems to have worked just fine, but I don't have funds in this wallet of course
 132017-01-03T01:09:34  <dviola> I could still see the address and such
 142017-01-03T01:09:41  <dviola> which was the same
 152017-01-03T01:13:28  *** Giszmo has quit IRC
 162017-01-03T01:14:49  *** MarcoFalke has quit IRC
 172017-01-03T01:15:10  <dviola> just tested again
 182017-01-03T01:15:23  <luke-jr> dviola: *using* the wallet.dat with bdb5 should break using it with bdb4.. no matter how it was created.
 192017-01-03T01:17:57  <dviola> hrm
 202017-01-03T01:19:13  <dviola> here's what I did: I installed bitcoin-qt with pacman, started a fresh ~/.bitcoin and wallet.dat, I created 5 addresses with name, etc. I copied the wallet.dat to my home dir, I downloaded bitcoin-qt from bitcoin.org, started that with the old wallet.dat and all the address were there
 212017-01-03T01:22:30  <dviola> luke-jr: I understand, but isn't what I did the equivalent of what you said? I'm a little confused over this
 222017-01-03T01:27:24  <dviola> let me show you something...
 232017-01-03T01:28:43  <dviola> this is bitcoin-qt from archlinux (bdb 5.3.28): https://i.imgur.com/iL6MVB2.png
 242017-01-03T01:28:54  <dviola> this is bitcoin-qt from bitcoin.org (bdb 4.8.x): https://i.imgur.com/G50bTJi.png
 252017-01-03T01:28:57  <dviola> same wallet.dat
 262017-01-03T01:29:08  <dviola> I can still read the data from wallet
 272017-01-03T01:29:40  <dviola> I'm confused and I apologize if I should have asked in #bitcoin instead, please let me know if I should go there
 282017-01-03T01:31:02  <luke-jr> dviola: are you sure Arch isn't using bdb4 for the wallet?
 292017-01-03T01:31:19  <luke-jr> if both are installed, I think we prefer it even if incompatible-bdb is allowed via option
 302017-01-03T01:35:23  <dviola> I'm sure, this is how they build it: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/bitcoin#n166
 312017-01-03T01:35:30  <dviola> ldd shows this also: libdb_cxx-5.3.so => /usr/lib/libdb_cxx-5.3.so (0x00007f3aaa8bc000)
 322017-01-03T01:36:38  <dviola> I asked Timothy Redaelli (archlinux bitcoin maintainer) and he said this: https://i.imgur.com/bfvsEcZ.png
 332017-01-03T01:42:53  <dviola> not saying I don't believe what you just said, I guess I'm puzzled as to why it works
 342017-01-03T01:55:28  <dviola> luke-jr: hrm, I only have bdb5 system-wide
 352017-01-03T02:01:17  *** abpa has quit IRC
 362017-01-03T02:04:42  <dviola> even with an intact ~/.bitcoin it still works fine with both binaries
 372017-01-03T02:04:44  <dviola> bdb4 and 5
 382017-01-03T02:05:10  <dviola> I can see the info from getwalletinfo and do dumpwallet, etc
 392017-01-03T02:06:17  <dviola> I wonder if there is a way to exhaust this and see where it starts to fail
 402017-01-03T02:14:54  *** lejitz has quit IRC
 412017-01-03T02:14:56  *** lejitz1 has joined #bitcoin-core-dev
 422017-01-03T02:21:06  *** AaronvanW has quit IRC
 432017-01-03T03:07:32  *** dviola has quit IRC
 442017-01-03T03:08:11  *** abpa has joined #bitcoin-core-dev
 452017-01-03T03:09:21  *** abpa has quit IRC
 462017-01-03T03:17:59  *** Victorsueca has joined #bitcoin-core-dev
 472017-01-03T03:20:20  *** Victor_sueca has quit IRC
 482017-01-03T03:27:07  <gmaxwell> anyone interested in picking up #8660 ? it's a nice feature, but has gone fallow after main was killed.
 492017-01-03T03:27:09  <gribble> https://github.com/bitcoin/bitcoin/issues/8660 | txoutsbyaddress index (take 2) by djpnewton · Pull Request #8660 · bitcoin/bitcoin · GitHub
 502017-01-03T03:28:51  <gmaxwell> droark: it should just fail to even open, always did in the past.
 512017-01-03T03:29:42  <gmaxwell> maybe some time in BDB 5 they changed the format to be backwards compatible if shut down cleanly?
 522017-01-03T03:34:45  *** justanotheruser has quit IRC
 532017-01-03T03:35:03  *** justanotheruser has joined #bitcoin-core-dev
 542017-01-03T03:35:10  *** justanotheruser has quit IRC
 552017-01-03T03:39:06  *** justanotheruser has joined #bitcoin-core-dev
 562017-01-03T03:54:22  *** robert__ has quit IRC
 572017-01-03T03:59:21  <phantomcircuit> gmaxwell, does it have to handle reorgs correctly?
 582017-01-03T04:00:45  <phantomcircuit> i dont think it would actually if you were saving address -> [block hash/tx index/out index]
 592017-01-03T04:01:07  <phantomcircuit> but that's maybe slightly bigger than just address > txout
 602017-01-03T04:06:59  <phantomcircuit> oh it's just for the utxo?
 612017-01-03T04:07:01  <phantomcircuit> yeah i guess
 622017-01-03T04:10:27  *** robert__ has joined #bitcoin-core-dev
 632017-01-03T04:19:39  *** Squidicc has joined #bitcoin-core-dev
 642017-01-03T04:22:49  *** squidicuz has quit IRC
 652017-01-03T05:31:14  <gmaxwell> phantomcircuit: it's just for the utxo set, the PR is somewhat poorly named!
 662017-01-03T05:48:10  <CodeShark> utxosbyaddress? :)
 672017-01-03T05:48:54  <CodeShark> utxosbyscriptpubkey might even be more useful
 682017-01-03T05:53:59  <CodeShark> I might take a look at it, it is a pretty nice feature
 692017-01-03T05:55:26  <gmaxwell> assuming the database is constructed correctly (lets hope) the difference between by address vs by scriptpubkey is purely a RPC parameter one.
 702017-01-03T05:55:53  <CodeShark> yeah, indeed :)
 712017-01-03T05:56:06  <CodeShark> but we don't have address formats for all scriptpubkey types
 722017-01-03T05:58:13  <CodeShark> how big is the extra index here?
 732017-01-03T06:00:14  <gmaxwell> CodeShark: should be roughly similar in size to the utxo set.
 742017-01-03T06:00:40  <CodeShark> it's something of a reverse lookup, no?
 752017-01-03T06:00:56  <CodeShark> when validating we need to grab scriptpubkey from blockhash:txindex
 762017-01-03T06:01:02  <gmaxwell> right.
 772017-01-03T06:01:13  *** windsok has quit IRC
 782017-01-03T06:01:20  <gmaxwell> I don't recall what it implements, but
 792017-01-03T06:01:32  <CodeShark> or txhash:outindex, rather ;)
 802017-01-03T06:01:35  <gmaxwell> e.g. it could map(h(scrippubkey)[64 bits]) -> [txid:vout,...]  which would potentially equal in size or smaller even though it's going to be more sparse.
 812017-01-03T06:01:52  <gmaxwell> Well my right was responding to what you meant, of course.
 822017-01-03T06:02:57  <gmaxwell> at least if I'd written this from scratch I would have done that kind of hash, probably with a per database salt to keep clowns from intentionally making your queries slow by causing collisions.
 832017-01-03T06:05:34  <CodeShark> I'm a little ambivalent about using an in-process storage engine for things not required for full validation. I'd love to see an architecture with an external storage engine that can be configured for different kinds of queries. But I guess this index isn't too huge
 842017-01-03T06:07:19  <CodeShark> pulling the consensus-critical stuff out to an external storage engine is perhaps a bit risky, though
 852017-01-03T06:07:36  <CodeShark> and perhaps there are optimization issues as well here with the caching
 862017-01-03T06:07:45  <CodeShark> I'm not too familiar with the cache stuff
 872017-01-03T06:09:21  *** veleiro has joined #bitcoin-core-dev
 882017-01-03T06:09:23  *** robert__ is now known as spinNspiral
 892017-01-03T06:09:29  <CodeShark> the main advantage here is that different people could extend the functionality of the storage engine and queries supported without having to touch any consensus code at all
 902017-01-03T06:09:42  <CodeShark> nor having to rebuild the validation engine
 912017-01-03T06:11:36  <CodeShark> but we already have the existing RPC framework...which encourages just adding more in-process RPC calls
 922017-01-03T06:23:12  <CodeShark> people always talk about keeping these indices external to bitcoind...but without a framework for it that people can easily extend everyone has to reinvent the wheel
 932017-01-03T06:24:18  <CodeShark> ...poorly :p
 942017-01-03T06:30:07  <gmaxwell> You can always index whatever you want talking to things over the rpc. but there are plenty of uses for this just from the commandline.
 952017-01-03T06:32:38  <CodeShark> no question it's useful. but there could be a bunch other useful queries for app developers (or even for interactive use from commandline) - and having to merge another in-process RPC call everytime is a bottleneck
 962017-01-03T06:33:49  <CodeShark> or it encourages people to have to maintain their own forks of the entire project so they can customize it
 972017-01-03T06:35:15  <CodeShark> as for an external index, we could get far better performance foregoing the RPC and using a different IPC
 982017-01-03T06:35:58  <CodeShark> also, the indices could be built up during IBD
 992017-01-03T06:36:33  <CodeShark> but these are longer term objectives - I still like this PR :)
1002017-01-03T06:48:55  *** windsok has joined #bitcoin-core-dev
1012017-01-03T06:52:37  *** spinNspiral has quit IRC
1022017-01-03T06:52:38  *** Ylbam has joined #bitcoin-core-dev
1032017-01-03T07:06:27  *** Squidicc is now known as squidicuz
1042017-01-03T07:13:39  *** udiWertheimer has joined #bitcoin-core-dev
1052017-01-03T07:27:31  *** tunafizz has quit IRC
1062017-01-03T07:48:32  *** udiWertheimer has quit IRC
1072017-01-03T07:58:15  *** veleiro has quit IRC
1082017-01-03T07:59:51  <jonasschnelli> Is it okay to use GPG's (v2.1+) Ed25516 or even secp256k1 keys for gitian signing?
1092017-01-03T08:00:11  <jonasschnelli> *25519
1102017-01-03T08:01:31  <wumpus> well, it's not forbidden, but I can't count them right now (my gpg is at 1.4.20, ubuntu 16.04 lts)
1112017-01-03T08:02:27  <jonasschnelli> Yes. It's probably to early.
1122017-01-03T08:02:29  <wumpus> I can look at compiling my own for 0.14, but not for 0.13.2 today
1132017-01-03T08:02:44  <phantomcircuit> wumpus, you can install gnupg 2.x
1142017-01-03T08:02:49  <phantomcircuit> it's like gnupg-2 or something
1152017-01-03T08:02:53  <jonasschnelli> gpg2
1162017-01-03T08:02:57  <wumpus> phantomcircuit: oh!
1172017-01-03T08:03:24  <jonasschnelli> I may start with signing with Ed25519, just to have some variety.
1182017-01-03T08:03:44  <gmaxwell> jonasschnelli: distributions ship the table version of gpg, mostly (exclusively?) so AFAICT relatively few people have it.
1192017-01-03T08:03:55  <wumpus> phantomcircuit: jonasschnelli: that gets me 2.1.11
1202017-01-03T08:05:35  <jonasschnelli> I guess for all ecdsa curves (including secp256k1) you need 2.1.16 (or 2.1.17?)
1212017-01-03T08:05:35  <wumpus> gah, it doesn't use the debian alternatives system though, so I have to manually patch up scripts to use gpg2 instead of gpg
1222017-01-03T08:05:54  <jonasschnelli> But watch out when using gpg2
1232017-01-03T08:06:00  <jonasschnelli> It will upgrade you .gpg folder
1242017-01-03T08:06:08  <jonasschnelli> And will no longer be backward comp.
1252017-01-03T08:06:14  <jonasschnelli> So,... make a backup first
1262017-01-03T08:06:21  <wumpus> oops
1272017-01-03T08:06:24  <jcorgan> eww
1282017-01-03T08:06:35  <jonasschnelli> maybe this was too late. :)
1292017-01-03T08:06:45  <CodeShark> lol
1302017-01-03T08:06:51  <wumpus> let's see :) I only looked at the help message, didn't encrypt anything yet
1312017-01-03T08:07:57  <wumpus> good, v1 still works for verifying signatures
1322017-01-03T08:08:05  <wumpus> I'll look at this for 0.14 okay
1332017-01-03T08:08:15  <jonasschnelli> Yes. No hurry.
1342017-01-03T08:08:43  <jonasschnelli> If secp256k1 is supported through GPG, this would allow signing over a hardware wallet without new curves added to them.
1352017-01-03T08:11:01  <wumpus> yea that could be interesting, though it would have to assign a single purpose to keys, don't use keys for signing that are used for bitcoin
1362017-01-03T08:11:14  <jonasschnelli> Yes. Indeed.
1372017-01-03T08:11:56  *** BashCo has quit IRC
1382017-01-03T08:12:25  <jonasschnelli> I'd also prefer Ed25519,... not supported by (all?) available hardware wallets I guess.
1392017-01-03T08:12:33  *** BashCo has joined #bitcoin-core-dev
1402017-01-03T08:14:59  <wumpus> ed25519 would have the advantage it can be used for ssh too
1412017-01-03T08:15:14  <jonasschnelli> Oh. Yes.
1422017-01-03T08:15:33  <wumpus> (even if you compile it without openssl :p)
1432017-01-03T08:16:03  <jonasschnelli> Right. ed25519 & chacha20Poly1305@openssh. Nice combo.
1442017-01-03T08:17:25  *** BashCo has quit IRC
1452017-01-03T08:17:54  <wumpus> yes
1462017-01-03T08:18:37  *** udiWertheimer has joined #bitcoin-core-dev
1472017-01-03T08:32:26  *** BashCo has joined #bitcoin-core-dev
1482017-01-03T08:32:32  <bitcoin-git> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/53442af0aac3...510c0d9c79a5
1492017-01-03T08:32:32  <bitcoin-git> bitcoin/master 9e351c9 Jonas Schnelli: SetMerkleBranch: remove unused code, remove cs_main lock requirement
1502017-01-03T08:32:33  <bitcoin-git> bitcoin/master 510c0d9 Jonas Schnelli: Merge #9446: SetMerkleBranch: remove unused code, remove cs_main lock requirement...
1512017-01-03T08:32:48  <bitcoin-git> [bitcoin] jonasschnelli closed pull request #9446: SetMerkleBranch: remove unused code, remove cs_main lock requirement (master...2016/12/merklebranch) https://github.com/bitcoin/bitcoin/pull/9446
1522017-01-03T08:59:30  <wumpus> uploaded binaries and pushed 0.13.2 release announcement to the mailing lists, waiting for travis on bitcoin.org (https://github.com/bitcoin-dot-org/bitcoin.org/pull/1470), feel free to get the rest of the announcement cycle started
1532017-01-03T09:02:09  <gmaxwell> jonasschnelli: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/013397.html  A spv wallet should _never_ be showing transactions from untrusted peers.
1542017-01-03T09:02:28  <gmaxwell> jonasschnelli: because the peers can feed them any garbage they want and the spv client has no idea about it.
1552017-01-03T09:02:51  <jonasschnelli> (phonecall, will response soon)
1562017-01-03T09:04:01  <gmaxwell> and you get lovely results like https://people.xiph.org/~greg/21mbtc.png
1572017-01-03T09:08:10  *** paveljanik has quit IRC
1582017-01-03T09:08:16  <sipa> gmaxwell: but what if you trust a peer enough to not relay invalid txn, but not enough to not spy on you?
1592017-01-03T09:08:42  <jonasschnelli> gmaxwell: I see.
1602017-01-03T09:09:07  <jonasschnelli> I mentioned this in the bitcoin-dev mail, because users relay on it,... even if its fundamentally broken.
1612017-01-03T09:09:21  <sipa> rely
1622017-01-03T09:09:35  <jonasschnelli> rely. Thanks sipa. :)
1632017-01-03T09:09:50  <jonasschnelli> Also,... it looked like, that my feeler PR: https://github.com/bitcoin/bitcoin/pull/9238 did get some rejects.
1642017-01-03T09:11:11  <jonasschnelli> In a long shot, I could imagine, BFD (filter digest / filter commitments) for filtering from untrusted peers, and BIP39 for filtering from trusted peers (once BIP150 has ben established).
1652017-01-03T09:11:23  <jonasschnelli> Even if BIP39 is not the most efficient way.
1662017-01-03T09:11:30  <sipa> BIP39?
1672017-01-03T09:11:33  <sipa> are you sure?
1682017-01-03T09:11:35  <jonasschnelli> 37! argh
1692017-01-03T09:11:56  <jonasschnelli> Bloom Filter
1702017-01-03T09:13:06  *** jannes has joined #bitcoin-core-dev
1712017-01-03T09:13:17  <jonasschnelli> But I think there is no solution for a non-BIP37 0-conf "filtering" without running into these privacy issues related to bip37
1722017-01-03T09:13:33  <gmaxwell> jonasschnelli: for your wokrin in 9238 just transfer all transactions, any filtering scheme will trash privacy, and transfering transactions is an aggregate of 14 kilobit/s.
1732017-01-03T09:13:53  <gmaxwell> But displaying unconfirmed transactions in any non-validating wallet is just an invite to abuse.
1742017-01-03T09:14:35  <gmaxwell> Many lite wallets that do this today are utterly crippled by it, because they also spend unconfirmed coins, so a single troublemaker giving them some fake payments makes them unusable. (or at least they did a half year ago.)
1752017-01-03T09:15:11  <jonasschnelli> gmaxwell: Yes. Agree. But the user-experience without displaying incoming funds will probably be not acceptable "in the field".
1762017-01-03T09:15:25  <gmaxwell> please.
1772017-01-03T09:16:39  <jonasschnelli> I don't know. I also don't like the 0-conf displaying. While working on the Core wallets SPV mode, I just had the feeling that people will not use it because of the missing 0-conf display.
1782017-01-03T09:16:47  <gmaxwell> I think it's a bad idea to release features that would require an emergency software update to disable them as soon as one board person spends an hour making a few like hack that sends out garbage transactions and spins up a bunch of sybil nodes.
1792017-01-03T09:17:56  <jonasschnelli> 0-conf / spv should probably only be possible in conjunction with BIP150 and a trusted peer (at home or somewhere)
1802017-01-03T09:18:35  <gmaxwell> In any case, just send all the data. As mentioned it's 14kbit/sec.  The history is really the only annoying part about that.
1812017-01-03T09:19:26  <jonasschnelli> gmaxwell: do you mean send all the data if someone request a filtered mempool?
1822017-01-03T09:19:34  <jonasschnelli> or just for tv invs?
1832017-01-03T09:19:38  <jonasschnelli> *tx
1842017-01-03T09:20:10  <gmaxwell> I mean don't do any tx filtering.
1852017-01-03T09:21:25  <gmaxwell> Same as we do today. Please. implementing spv mode in core doesn't mean implementing all the flaws and vulnerabilities in other software slavishly. Thats pointless. If someone wants something that is going to blast out there addresses, validate nothing, etc.. they can already use bitcoinj based software.
1862017-01-03T09:21:41  <jonasschnelli> Heh. True.
1872017-01-03T09:22:01  <gmaxwell> :) at least the spv behavior in core can be private.
1882017-01-03T09:22:10  <jonasschnelli> I have implemented the BFD relevant hooks. IsBlockRelevantToWallet(pindex).
1892017-01-03T09:23:39  <jonasschnelli> gmaxwell: The only usability issues are: catch-up a couple of weeks (144*<day> MBs, take a while) and the missing 0-conf. Maybe this is acceptable to prevent privacy.
1902017-01-03T09:24:50  <gmaxwell> I think it's not bad, especially if we had better facilities for keeping it caught up in the background.
1912017-01-03T09:24:55  <jonasschnelli> Though, I haven't fully understood the semi.-trusted oracle idea for the BDF. I would expect to have the BDF in the coinbase...
1922017-01-03T09:26:00  <jonasschnelli> But somehow the BFD is required for older blocks as well
1932017-01-03T09:26:09  <gmaxwell> It's not really great to go from nothing to consensus rule if it can be avoided: No design survives first contact with users.  Better to prove out the idea and refine it before its required, which is possible here.
1942017-01-03T09:27:29  <jonasschnelli> From where would you get the BFD (maybe its not bloom filters in the end)? A new p2p command?
1952017-01-03T09:28:13  <gmaxwell> just another type of element that you can getdata, perhaps more than one.
1962017-01-03T09:29:53  <jonasschnelli> this would result in trusting the peer, right? The whole signing thing of BFDs would still be a thing? Or just compare BFDs from the data retrived from other peers?
1972017-01-03T09:30:20  <jonasschnelli> Probably easy to sybil you with fake data
1982017-01-03T09:30:57  <gmaxwell> I thought you were referring to the filter itselt and not the hash root... Unclear, multiple models are possible and _all_ are better than BIP37.
1992017-01-03T09:31:38  <jonasschnelli> Right, tx withhold is also possible with BIP37.
2002017-01-03T09:32:18  <gmaxwell> and asking multiple peers is very inefficient with BIP37.. needs n-times the bandwidth.
2012017-01-03T09:33:19  <jonasschnelli> good point.
2022017-01-03T09:36:22  *** slsdhl has joined #bitcoin-core-dev
2032017-01-03T09:36:34  *** slsdhl has quit IRC
2042017-01-03T09:37:14  <btcdrak> Requesting review for release blog post https://github.com/bitcoin-core/bitcoincore.org/pull/305
2052017-01-03T09:40:23  <gmaxwell> jonasschnelli: but pulling data from peers is just one option, you could-- for example-- go get a hash root (root of a tree of digest hashes) from a website and pop that into a configuration.
2062017-01-03T09:41:09  <gmaxwell> (and then use that for filtering the past-- then the future, past that root, you just fetch whole blocks)
2072017-01-03T09:49:31  *** jeremias has quit IRC
2082017-01-03T09:50:22  *** jeremias has joined #bitcoin-core-dev
2092017-01-03T09:52:28  *** timothy has quit IRC
2102017-01-03T09:54:18  *** Guyver2 has joined #bitcoin-core-dev
2112017-01-03T10:01:23  *** timothy has joined #bitcoin-core-dev
2122017-01-03T10:12:32  *** timothy has quit IRC
2132017-01-03T10:13:13  *** timothy has joined #bitcoin-core-dev
2142017-01-03T10:19:03  *** timothy has quit IRC
2152017-01-03T10:19:32  *** MarcoFalke has joined #bitcoin-core-dev
2162017-01-03T10:23:41  <BlueMatt> cfields: ohhh, static CNodeState::cs....missed the static part
2172017-01-03T10:24:28  <BlueMatt> cfields: ok, I'll make it static in the pr and add a TODO: do something smarter
2182017-01-03T10:24:38  <BlueMatt> cfields: oh, wait, no, really dont want to do that
2192017-01-03T10:25:40  *** Giszmo has joined #bitcoin-core-dev
2202017-01-03T10:26:35  <BlueMatt> cfields: that would mean that #9375 + multithreaded processmessages wouldnt accomplish anything :/
2212017-01-03T10:26:37  <gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
2222017-01-03T10:28:06  <sipa> BlueMatt: but the same is true for the current code, no?
2232017-01-03T10:28:13  <sipa> which uses cs_main
2242017-01-03T10:31:40  <BlueMatt> sipa: no, I dont believe so, #9375 + #9419 + a fix for https://github.com/bitcoin/bitcoin/pull/9375#discussion_r94180477 (which just means making fWantsCmpctWitness atomic) + multithreaded ProcessMessages will work
2252017-01-03T10:31:42  <gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
2262017-01-03T10:31:44  <gribble> https://github.com/bitcoin/bitcoin/issues/9419 | Stop Using cs_main for CNodeState/State() by TheBlueMatt · Pull Request #9419 · bitcoin/bitcoin · GitHub
2272017-01-03T10:33:09  <sipa> BlueMatt: ah, i see
2282017-01-03T10:33:14  <sipa> right
2292017-01-03T10:33:14  <BlueMatt> it may be the case that we dont need State() at all for that, though
2302017-01-03T10:35:38  <BlueMatt> yea, no, so to make it all work we need to have a State()->fWantsCmpctWitness and pfrom->GetSendVersion() call without cs_main
2312017-01-03T10:40:05  *** timothy has joined #bitcoin-core-dev
2322017-01-03T10:51:53  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.13: https://github.com/bitcoin/bitcoin/commit/77eaadb6c9619370b09513fecba13cfe656d63d6
2332017-01-03T10:51:54  <bitcoin-git> bitcoin/0.13 77eaadb Wladimir J. van der Laan: doc: Clean out release notes on 0.13.x branch...
2342017-01-03T10:52:56  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to master: https://github.com/bitcoin/bitcoin/commit/03e1d6ce349cc83b92140fec7d0c5f88893c0a9c
2352017-01-03T10:52:56  <bitcoin-git> bitcoin/master 03e1d6c Wladimir J. van der Laan: doc: Add historical release notes for 0.13.2
2362017-01-03T10:54:45  <jonasschnelli> gmaxwell: thanks.
2372017-01-03T10:55:22  <jonasschnelli> gmaxwell: you mentioned here (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-May/012637.html) a rough specs for a better filter structure then bloom
2382017-01-03T10:55:30  <jonasschnelli> Do you have any specification for that?
2392017-01-03T10:59:30  <BlueMatt> cfields: hmmm, regarding #9441, do we really want to run the entire ProcessMessages loop (calling SendMessages potentially umpteen times) just to process multiple messages from the same node?
2402017-01-03T10:59:32  <gribble> https://github.com/bitcoin/bitcoin/issues/9441 | Net: Massive speedup. Net locks overhaul by theuni · Pull Request #9441 · bitcoin/bitcoin · GitHub
2412017-01-03T10:59:49  <BlueMatt> (ie remove the loop inside ProcessMessages and move it to ThreadProcessMessages)
2422017-01-03T11:01:02  <timothy> buiding bitcoin-core 0.13.2 on archlinux...
2432017-01-03T11:03:18  <sipa> BlueMatt: i was wondering about that too
2442017-01-03T11:03:52  <sipa> BlueMatt: but there is hardly any (contentious) locking going on anymore in between
2452017-01-03T11:04:04  <BlueMatt> yea, but SendMessages.....
2462017-01-03T11:04:09  <sipa> Ah.
2472017-01-03T11:04:31  <sipa> i hadn't considered that
2482017-01-03T11:04:37  <sipa> that defeats the purpose
2492017-01-03T11:05:10  <sipa> hmm, i was about to say that it negatively impacts batching of invs and addrs
2502017-01-03T11:05:27  <sipa> but since we have explicit random delays for those, i don't think that's really an issue anymore
2512017-01-03T11:05:40  <BlueMatt> yea, I dont think it breaks anything
2522017-01-03T11:05:47  <BlueMatt> just repeatedly calls SendMessages to do nothing
2532017-01-03T11:06:12  <sipa> oh, it won't break anything
2542017-01-03T11:06:53  <BlueMatt> I assume you didnt touch cs_vSend due to https://github.com/bitcoin/bitcoin/pull/9419/commits/c214d120a363a05ba9afdccff6b4bda6e29ae7c4, cfields?
2552017-01-03T11:10:06  <BlueMatt> cfields: other than that, I got through #9441 and it looks good to me
2562017-01-03T11:10:08  <gribble> https://github.com/bitcoin/bitcoin/issues/9441 | Net: Massive speedup. Net locks overhaul by theuni · Pull Request #9441 · bitcoin/bitcoin · GitHub
2572017-01-03T11:10:38  * BlueMatt will copy discussion into pr thread
2582017-01-03T11:10:45  *** AaronvanW has joined #bitcoin-core-dev
2592017-01-03T11:10:46  *** AaronvanW has quit IRC
2602017-01-03T11:10:46  *** AaronvanW has joined #bitcoin-core-dev
2612017-01-03T11:17:14  <sipa> thanks
2622017-01-03T11:17:22  * sipa -> NYC
2632017-01-03T11:17:45  <BlueMatt> sipa: cool, see you tomorrow afternoon
2642017-01-03T11:17:55  <BlueMatt> anyone else at RWC? sipa, cfields and I likely will be
2652017-01-03T11:24:38  *** bitcfan has joined #bitcoin-core-dev
2662017-01-03T11:25:34  *** Saucery has quit IRC
2672017-01-03T11:45:22  *** tiago_ has joined #bitcoin-core-dev
2682017-01-03T11:51:21  <timothy> is MALLOC_ARENA_MAX=1 still "recommended" on linux?
2692017-01-03T12:01:18  <sipa> timothy: i expect that setting it so low may come with lower performancr
2702017-01-03T12:01:27  <sipa> but i don't think anyone ever benchmarked it
2712017-01-03T12:03:16  <sipa> it's certainly recommended if you are very tight on memory
2722017-01-03T12:03:59  <timothy> is -reindex-chainstate is faster than -reindex?
2732017-01-03T12:04:06  <timothy> without the second is :P
2742017-01-03T12:05:26  *** windsok has quit IRC
2752017-01-03T12:09:32  <sipa> yes
2762017-01-03T12:09:53  <sipa> but it only works when your blocks and blocks/index are not corrupted
2772017-01-03T12:19:23  *** kadoban has quit IRC
2782017-01-03T12:19:24  *** jl2012 has quit IRC
2792017-01-03T12:19:24  *** kinlo has quit IRC
2802017-01-03T12:19:51  *** kinlo has joined #bitcoin-core-dev
2812017-01-03T12:19:51  *** kadoban has joined #bitcoin-core-dev
2822017-01-03T12:20:02  *** tiago_ has quit IRC
2832017-01-03T12:20:53  *** jl2012 has joined #bitcoin-core-dev
2842017-01-03T12:32:52  *** BashCo_ has joined #bitcoin-core-dev
2852017-01-03T12:36:01  *** BashCo has quit IRC
2862017-01-03T12:36:16  *** windsok has joined #bitcoin-core-dev
2872017-01-03T13:21:37  *** JackH has quit IRC
2882017-01-03T13:25:50  *** ryanofsky_ has quit IRC
2892017-01-03T13:27:28  *** ryanofsky has joined #bitcoin-core-dev
2902017-01-03T13:30:04  *** goregrind has quit IRC
2912017-01-03T13:30:09  *** goregrin1 has joined #bitcoin-core-dev
2922017-01-03T13:30:56  *** belcher has quit IRC
2932017-01-03T13:33:14  *** belcher has joined #bitcoin-core-dev
2942017-01-03T13:46:55  *** juscamarena has quit IRC
2952017-01-03T13:47:16  *** juscamarena has joined #bitcoin-core-dev
2962017-01-03T13:47:23  *** Aleph0 has quit IRC
2972017-01-03T13:47:23  *** ybit has quit IRC
2982017-01-03T13:47:23  *** jrayhawk has quit IRC
2992017-01-03T13:47:23  *** aj has quit IRC
3002017-01-03T13:47:30  *** aj has joined #bitcoin-core-dev
3012017-01-03T13:47:31  *** Aleph0 has joined #bitcoin-core-dev
3022017-01-03T13:47:32  *** jrayhawk has joined #bitcoin-core-dev
3032017-01-03T13:47:37  *** ybit has joined #bitcoin-core-dev
3042017-01-03T13:47:41  <jonasschnelli> bitcoin-qt tells me today when running with a fresh datadir: "Last received block was generated 7 year(s) and 52 week(s) ago."
3052017-01-03T13:47:44  *** ibrightly has quit IRC
3062017-01-03T13:48:24  *** blkdb has quit IRC
3072017-01-03T13:49:31  <wumpus> jonasschnelli: so it regards it as an edge case, 52 weeks but not a year yet. curious :)
3082017-01-03T13:50:02  <jonasschnelli> qint64 remainder = secs % YEAR_IN_SECONDS;
3092017-01-03T13:50:09  <jonasschnelli> Na. I don't fix that.
3102017-01-03T13:50:30  <wumpus> yeah it's silly but not a serious issue
3112017-01-03T13:51:01  *** ibrightly has joined #bitcoin-core-dev
3122017-01-03T13:55:32  *** roasbeef has quit IRC
3132017-01-03T13:56:59  *** roasbeef has joined #bitcoin-core-dev
3142017-01-03T13:57:51  *** laurentmt has joined #bitcoin-core-dev
3152017-01-03T13:58:30  *** laurentmt has quit IRC
3162017-01-03T13:58:55  *** laurentmt has joined #bitcoin-core-dev
3172017-01-03T14:07:37  <bitcoin-git> [bitcoin] laanwj opened pull request #9460: Fix a few typos in translated strings (master...2017_01_messages_update) https://github.com/bitcoin/bitcoin/pull/9460
3182017-01-03T14:13:27  *** dermoth has joined #bitcoin-core-dev
3192017-01-03T14:15:40  *** laurentmt has quit IRC
3202017-01-03T14:18:11  *** bsm117532 has quit IRC
3212017-01-03T14:18:12  *** fengling has quit IRC
3222017-01-03T14:41:13  <luke-jr> wumpus: btcdrak: bad signature on sendy release ann
3232017-01-03T14:42:03  <wumpus> luke-jr: are you checking the html or text attachment?
3242017-01-03T14:42:57  *** helo has quit IRC
3252017-01-03T14:43:09  <wumpus> in case of the html you should run gpg on the raw html file
3262017-01-03T14:43:19  <wumpus> (both attachments verify OK here)
3272017-01-03T14:44:05  <luke-jr> both
3282017-01-03T14:44:20  <luke-jr> how do you run GPG on the raw HTML file, seeing as the HTML is outside the signature? :/
3292017-01-03T14:44:37  <bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/03e1d6ce349c...e0dc3d70c6ec
3302017-01-03T14:44:38  <bitcoin-git> bitcoin/master a9d6151 Wladimir J. van der Laan: qt,wallet: Fix a few typos in messages...
3312017-01-03T14:44:39  <bitcoin-git> bitcoin/master d45b21e Wladimir J. van der Laan: qt: Fill in English numerusforms...
3322017-01-03T14:44:39  <bitcoin-git> bitcoin/master e0dc3d7 Wladimir J. van der Laan: Merge #9460: Fix a few typos in translated strings...
3332017-01-03T14:44:53  <btcdrak> luke-jr: matches for me
3342017-01-03T14:44:53  <bitcoin-git> [bitcoin] laanwj closed pull request #9460: Fix a few typos in translated strings (master...2017_01_messages_update) https://github.com/bitcoin/bitcoin/pull/9460
3352017-01-03T14:45:07  <wumpus> that mail has a text/plain as well as a text/html mime attachment - just save them to a file and run gpg on them?
3362017-01-03T14:46:30  <luke-jr> hmm, that works. I wonder what copy/paste is changing
3372017-01-03T14:46:55  <btcdrak> Thunderbird also automatically validated it for me.
3382017-01-03T14:47:03  <jonasschnelli> Same here
3392017-01-03T14:47:11  <luke-jr> my mail client normally auto-validates, but it didn't like the formatting or something
3402017-01-03T14:53:42  <bitcoin-git> [bitcoin] jonasschnelli opened pull request #9461: [Qt] Improve progress display during headers-sync and peer-finding (master...2017/01/qt_sync) https://github.com/bitcoin/bitcoin/pull/9461
3412017-01-03T15:14:12  *** Cheeseo has joined #bitcoin-core-dev
3422017-01-03T15:14:12  *** Cheeseo has joined #bitcoin-core-dev
3432017-01-03T15:14:23  *** Cheeseo has left #bitcoin-core-dev
3442017-01-03T15:14:49  <jonasschnelli> luke-jr: I think squashing makes sense when one commit overwrites many lines from a former commit in the same PR.
3452017-01-03T15:14:56  <jonasschnelli> But Okay for #8877
3462017-01-03T15:14:58  <gribble> https://github.com/bitcoin/bitcoin/issues/8877 | Qt RPC console: history sensitive-data filter, and saving input line when browsing history by luke-jr · Pull Request #8877 · bitcoin/bitcoin · GitHub
3472017-01-03T15:15:36  <luke-jr> jonasschnelli: there are some cases where I think it may make sense, yes
3482017-01-03T15:15:56  *** andytoshi has left #bitcoin-core-dev
3492017-01-03T15:16:05  <jonasschnelli> But your right, squashing and loosing "logical history" is not ideal
3502017-01-03T15:17:13  *** bsm117532 has joined #bitcoin-core-dev
3512017-01-03T15:20:09  <jonasschnelli> can anyone give https://github.com/bitcoin/bitcoin/pull/8877 a final review?
3522017-01-03T15:21:22  *** chris2000 has joined #bitcoin-core-dev
3532017-01-03T15:34:52  *** helo has joined #bitcoin-core-dev
3542017-01-03T15:37:29  *** blkdb has joined #bitcoin-core-dev
3552017-01-03T15:40:41  *** BashCo_ has quit IRC
3562017-01-03T15:41:18  *** BashCo has joined #bitcoin-core-dev
3572017-01-03T15:45:35  *** BashCo has quit IRC
3582017-01-03T15:50:29  *** fengling has joined #bitcoin-core-dev
3592017-01-03T15:54:46  <cfields> BlueMatt: yes, cs_vSend wasn't touched because of your PR. I've just been operating under the assumption that the lock around SendMessages will be removed by one PR or another.
3602017-01-03T15:55:44  <btcdrak> jonasschnelli: done.
3612017-01-03T15:56:30  *** jtimon has joined #bitcoin-core-dev
3622017-01-03T15:57:40  <jonasschnelli> thanks btcdrak
3632017-01-03T15:57:52  <jonasschnelli> now I own you a review. :)
3642017-01-03T15:57:53  <bitcoin-git> [bitcoin] jonasschnelli pushed 12 new commits to master: https://github.com/bitcoin/bitcoin/compare/e0dc3d70c6ec...6dc4c43d326e
3652017-01-03T15:57:54  <bitcoin-git> bitcoin/master fc95daa Jonas Schnelli: Qt/RPCConsole: Save current command entry when browsing history...
3662017-01-03T15:57:54  <bitcoin-git> bitcoin/master 9044908 Jonas Schnelli: Qt/RPCConsole: Don't store commands with potentially sensitive information in the history...
3672017-01-03T15:57:55  <bitcoin-git> bitcoin/master de8980d Luke Dashjr: Bugfix: Do not add sensitive information to history for real...
3682017-01-03T15:58:05  <bitcoin-git> [bitcoin] jonasschnelli closed pull request #8877: Qt RPC console: history sensitive-data filter, and saving input line when browsing history (master...qt_console_history_filter) https://github.com/bitcoin/bitcoin/pull/8877
3692017-01-03T16:02:08  <jtimon> #9271 needs rebase but could still get some general feedback
3702017-01-03T16:02:11  <gribble> https://github.com/bitcoin/bitcoin/issues/9271 | An error has occurred and has been logged. Please contact this bot's administrator for more information.
3712017-01-03T16:02:14  <instagibbs> can someone explain ProcessBlockAvailability? The comment for it is unclear: "/** Check whether the last unknown block a peer advertised is not yet known. */"
3722017-01-03T16:02:28  <instagibbs> check whether an unknown block is unknown... ok!
3732017-01-03T16:05:04  <jtimon> also, friendly reminder https://github.com/bitcoin/bitcoin/pull/8855 https://github.com/bitcoin/bitcoin/pull/9279 and https://github.com/bitcoin/bitcoin/pull/8498 still *don't* need rebase
3742017-01-03T16:07:18  *** udiWertheimer has quit IRC
3752017-01-03T16:07:21  <instagibbs> nevermind, think I got it
3762017-01-03T16:07:44  *** xiangfu has quit IRC
3772017-01-03T16:07:49  *** jannes has quit IRC
3782017-01-03T16:07:58  *** fengling has quit IRC
3792017-01-03T16:08:56  *** xiangfu has joined #bitcoin-core-dev
3802017-01-03T16:09:11  *** BashCo has joined #bitcoin-core-dev
3812017-01-03T16:13:07  * luke-jr rebases #9152 to add history filter before he forgets ☺
3822017-01-03T16:13:09  <gribble> https://github.com/bitcoin/bitcoin/issues/9152 | Wallet/RPC: sweepprivkeys method to scan UTXO set and send to local wallet by luke-jr · Pull Request #9152 · bitcoin/bitcoin · GitHub
3832017-01-03T16:19:47  *** jannes has joined #bitcoin-core-dev
3842017-01-03T16:25:56  <BlueMatt> cfields: hmmm...yes, re: moving the loop into net_processing I figured that was just due to not complicating that pr too much, thanks
3852017-01-03T16:26:30  <cfields> BlueMatt: np, thanks for review
3862017-01-03T16:26:55  <BlueMatt> cfields: as for not having a per-node loop....hum...can we re-add it in net.cpp with two lines so that its not a (proabbly barely noticeable) regression?
3872017-01-03T16:27:58  <BlueMatt> bool fMoreNodeWork = ... while (fMoreNodeWork) ProcessMessages
3882017-01-03T16:28:07  <cfields> BlueMatt: I'm not sure there's any actual change from the current behavior
3892017-01-03T16:28:11  <sdaftuar> instagibbs: ProcessBlockAvailability is for when you get block announcements via inv (before you have the header)
3902017-01-03T16:28:38  <BlueMatt> cfields: you mean like from current master or from current pr?
3912017-01-03T16:29:02  <BlueMatt> sdaftuar: we still have crap for that? can we just remove it and replace with "getheaders"
3922017-01-03T16:29:10  <cfields> BlueMatt: oh.. it doesn't swallow all messages at once for fairness. I'm pretty sure emptying the node's full queue before moving on would be a DDoS vector
3932017-01-03T16:29:15  <cfields> BlueMatt: from master
3942017-01-03T16:29:42  <sdaftuar> BlueMatt: i think we could!
3952017-01-03T16:29:54  <cfields> BlueMatt: see the comment here: https://github.com/bitcoin/bitcoin/pull/9441/commits/99599884147ea4db3f960b0e706b039ba1553c80#r94270593
3962017-01-03T16:32:36  <BlueMatt> cfields: I believe the improvements against master come largely from removing lock contention between the message processing loop and the read-from-wire loop?
3972017-01-03T16:32:42  <BlueMatt> or do I have that backwards?
3982017-01-03T16:33:20  <BlueMatt> cfields: re: swallowing from one node...that isnt how I read the old code?
3992017-01-03T16:34:01  <BlueMatt> I mean certainly if we got a packet which only had one message we'd not process more than one message at a time, but if the loop is slow to go around (it appears to be sometimes due to various messages being slow) then we might have multiple to process per-peer
4002017-01-03T16:34:14  <cfields> BlueMatt: sorry, by no change in current behavior, i meant the swallowing-one-message part. Obviously fixing the lock contention is a behavioural change :)
4012017-01-03T16:34:26  <BlueMatt> and in that case I think we'd prefer to only call SendMessages less often (though, to be fair, doing them back-to-back is less likely to be slow than otherwise)
4022017-01-03T16:35:06  *** fengling has joined #bitcoin-core-dev
4032017-01-03T16:35:09  <cfields> BlueMatt: take a look at the current code. I really don't think the new behavior is different, other than the case of corrupted messages
4042017-01-03T16:35:15  <BlueMatt> cfields: to answer in a different way: why do we need to change from the current behavior
4052017-01-03T16:35:25  <BlueMatt> cfields: whats the break you're referring to in that comment?
4062017-01-03T16:35:29  <BlueMatt> (line numbers are confusing here)
4072017-01-03T16:35:40  <BlueMatt> the !message.complete() one?
4082017-01-03T16:36:00  <cfields> and yes, sendmessages needs to be broken up and fixed in lots of different ways, but imo not here
4092017-01-03T16:36:01  <cfields> sec
4102017-01-03T16:36:12  <BlueMatt> definitely agreed, not here
4112017-01-03T16:36:31  <cfields> BlueMatt: whoops, that should be 2538
4122017-01-03T16:36:31  <BlueMatt> just trying to figure out if re-adding a while somewhere in that pr is better or worse, since this is all we can hope for in 0.14 :)
4132017-01-03T16:36:58  <BlueMatt> ohhhhhh, I hadnt seen /that/ break
4142017-01-03T16:37:23  <cfields> yes. So the loop only continues in 2 cases, iirc.
4152017-01-03T16:37:37  <cfields> bad hash, or uhmm.. bad start chars maybe?
4162017-01-03T16:37:42  <BlueMatt> wait, so on current master we will continue the loop if the message is somehow corrupted, but otherwise wont? wtf.....
4172017-01-03T16:37:57  <BlueMatt> oh, no, just bad hash
4182017-01-03T16:38:00  <BlueMatt> yea, this makes no sense
4192017-01-03T16:38:38  <cfields> BlueMatt: indeed, those are the cases that the node should be punished for, but instead they get a free pass.
4202017-01-03T16:38:39  <cfields> go figure.
4212017-01-03T16:38:47  *** xiangfu has quit IRC
4222017-01-03T16:39:15  <BlueMatt> well, i guess if your checksum is bad its not like we did any work for you anyway...
4232017-01-03T16:39:33  <cfields> sure, but that should at least toss you to the back of the line
4242017-01-03T16:39:57  *** xiangfu has joined #bitcoin-core-dev
4252017-01-03T16:40:29  <cfields> BlueMatt: so you agree now that the loop behavior isn't changed here significantly? (or, at least, only for the better?)
4262017-01-03T16:41:14  <BlueMatt> yes, agreed
4272017-01-03T16:41:25  <cfields> ok
4282017-01-03T16:41:27  <BlueMatt> I suppose I wouldnt want to go audit for whether it was a dos vuln to change it in this pr
4292017-01-03T16:41:29  <BlueMatt> so, good :)
4302017-01-03T16:42:03  <cfields> heh
4312017-01-03T16:42:32  <cfields> BlueMatt: as a follow-up for 0.14, if you're concerned about it, we could add a quick hack to quickly exit SendMessages if it hasn't been at least x msec since the last call
4322017-01-03T16:42:43  <cfields> but i don't think there's any real need, since this has been the behavior all along
4332017-01-03T16:44:05  <BlueMatt> yea, I'm less concerned about that...still hoping for parallel processmessages but....that seems stretchy now :/
4342017-01-03T16:44:30  <cfields> :(
4352017-01-03T16:45:11  <BlueMatt> depends on how many reviewers pop their heads up this week, I suppose
4362017-01-03T16:46:25  <cfields> BlueMatt: other than the current PRs, how much remains there? I think that 9441 should cut down on the scary races a good bit?
4372017-01-03T16:46:55  <BlueMatt> yea, I think after current PRs its just one or five std::atomics in CNode and then the one commit which fuckes around with the ProcessMessages loop
4382017-01-03T16:47:06  <BlueMatt> which would need rebased on your stuff, but probably wouldnt be too hard
4392017-01-03T16:47:53  <cfields> these are atomics that fix more than just CNodeStats which has always been racy anyway?
4402017-01-03T16:48:14  <BlueMatt> I havent looked as of the latest stuff so you'd know better than I
4412017-01-03T16:48:22  <BlueMatt> I mean we should probably atomic-ify CNodeStats...
4422017-01-03T16:48:26  <BlueMatt> easy enough
4432017-01-03T16:48:28  <cfields> I'll check out your branch again
4442017-01-03T16:48:53  <BlueMatt> cool
4452017-01-03T16:49:16  <BlueMatt> I'll probably not get much done until thurs....early flight tomorrow so will likely be super tired during the day at rwc
4462017-01-03T16:49:30  <cfields> yea, I was thinking about that yesterday. I think it may make sense to just actually keep the stats in CNodeStats with one lock. See nRecvBytes as an example
4472017-01-03T16:49:41  <cfields> then when we need the stats, just do a quick lock and copy out
4482017-01-03T16:49:54  <BlueMatt> iirc that like automagically ended up with lock inversions
4492017-01-03T16:50:04  <BlueMatt> i think i looked into that, but dont remember now
4502017-01-03T16:50:12  <BlueMatt> seems weird that it would, but....something something...
4512017-01-03T16:50:40  <cfields> hmm, not sure how. I'll look again
4522017-01-03T16:50:57  <cfields> BlueMatt: btw, you're good with the boost changes that 9441 include?
4532017-01-03T16:51:28  <BlueMatt> which boost changes? you mean #9289?
4542017-01-03T16:51:30  <cfields> if so, mind re-acking #9289 ?
4552017-01-03T16:51:30  <gribble> https://github.com/bitcoin/bitcoin/issues/9289 | net: drop boost::thread_group by theuni · Pull Request #9289 · bitcoin/bitcoin · GitHub
4562017-01-03T16:51:33  <gribble> https://github.com/bitcoin/bitcoin/issues/9289 | net: drop boost::thread_group by theuni · Pull Request #9289 · bitcoin/bitcoin · GitHub
4572017-01-03T16:51:34  <cfields> yes
4582017-01-03T16:53:02  <BlueMatt> aww fuck, got rebased, yea, I'm pretty sure nothing changed that I didnt think was reasonable or good...I'll try to take another look tomorrow, but I'm pretty sure that ack is still valid
4592017-01-03T16:53:05  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #9462: [qt] Do not translate tilde character (master...Mf1701-qtTransTilde) https://github.com/bitcoin/bitcoin/pull/9462
4602017-01-03T16:54:59  <cfields> BlueMatt: np, thanks
4612017-01-03T16:56:08  <cfields> BlueMatt: if i could fixup the racy stuff that would affect parallel processing, want me to PR? Or does that just complicate the current PR maze?
4622017-01-03T16:57:01  <BlueMatt> cfields: up to you...I had assumed you were waiting on the current prs to clean up a bit, but if you can do it without stepping on your own toes feel free
4632017-01-03T16:58:22  <cfields> ok. I'll at least try to get something ready on top of the queued-up changes. If it happens to not depend on them, I'll go ahead and submit it.
4642017-01-03T16:58:46  <BlueMatt> sounds good, thanks
4652017-01-03T16:59:16  <BlueMatt> I'll look into the same tomorrow on the flight if I'm not asleep the whole time
4662017-01-03T16:59:40  <cfields> heh, ok
4672017-01-03T17:00:08  <BlueMatt> ehh, by "the same" I meant redoing the parallel-processmessages pr
4682017-01-03T17:00:12  <BlueMatt> that was super unclear....
4692017-01-03T17:08:26  *** abpa has joined #bitcoin-core-dev
4702017-01-03T17:08:27  *** davec has quit IRC
4712017-01-03T17:08:58  *** udiWertheimer has joined #bitcoin-core-dev
4722017-01-03T17:08:59  *** davec has joined #bitcoin-core-dev
4732017-01-03T17:09:08  *** udiWertheimer has quit IRC
4742017-01-03T17:17:23  *** lejitz1 has quit IRC
4752017-01-03T17:17:25  *** lejitz has joined #bitcoin-core-dev
4762017-01-03T17:27:37  *** xiangfu has quit IRC
4772017-01-03T17:30:50  <instagibbs> is there a plan for multiwallet for 0.14, or is it too late
4782017-01-03T17:31:13  *** fengling has quit IRC
4792017-01-03T17:35:00  <gmaxwell> we're not at the feature freeze yet, and it seems done +/- things that need to be fixed due to review.
4802017-01-03T17:35:14  <gmaxwell> Several people have said they would review it.
4812017-01-03T17:35:43  <gmaxwell> It would be really great it multiwallet and the utxo scriptpubkey index could make it in.
4822017-01-03T17:39:53  *** fengling has joined #bitcoin-core-dev
4832017-01-03T17:40:30  *** lejitz has quit IRC
4842017-01-03T17:42:18  <instagibbs> #8694 needs rebase though
4852017-01-03T17:42:20  <gribble> https://github.com/bitcoin/bitcoin/issues/8694 | Basic multiwallet support by luke-jr · Pull Request #8694 · bitcoin/bitcoin · GitHub
4862017-01-03T17:42:27  *** xiangfu has joined #bitcoin-core-dev
4872017-01-03T17:44:10  <luke-jr> instagibbs: #8776 goes first
4882017-01-03T17:44:12  <gribble> https://github.com/bitcoin/bitcoin/issues/8776 | Wallet refactoring leading up to multiwallet by luke-jr · Pull Request #8776 · bitcoin/bitcoin · GitHub
4892017-01-03T17:44:26  <instagibbs> ok, ill review
4902017-01-03T17:45:28  <luke-jr> actually #8776 looks ready to merge. 3 reviews already
4912017-01-03T17:45:30  <gribble> https://github.com/bitcoin/bitcoin/issues/8776 | Wallet refactoring leading up to multiwallet by luke-jr · Pull Request #8776 · bitcoin/bitcoin · GitHub
4922017-01-03T17:45:36  <luke-jr> yes we know gribble --.
4932017-01-03T17:46:55  *** jannes has quit IRC
4942017-01-03T17:49:24  <BlueMatt> luke-jr: good time to go rebase 8694 then :p
4952017-01-03T17:49:34  <luke-jr> sure
4962017-01-03T17:51:23  <luke-jr> oh, need #8775 still as well
4972017-01-03T17:51:25  <gribble> https://github.com/bitcoin/bitcoin/issues/8775 | RPC refactoring: Access wallet using new GetWalletForJSONRPCRequest by luke-jr · Pull Request #8775 · bitcoin/bitcoin · GitHub
4982017-01-03T17:52:20  <luke-jr> instagibbs: ^ could use review
4992017-01-03T18:03:36  <luke-jr> (once those two get in, looks reasonable to split up 8694 further into base/Qt/RPC PRs)
5002017-01-03T18:17:57  *** paveljanik has joined #bitcoin-core-dev
5012017-01-03T18:17:57  *** paveljanik has joined #bitcoin-core-dev
5022017-01-03T18:20:05  *** bsm117532 has quit IRC
5032017-01-03T18:36:36  *** dermoth has quit IRC
5042017-01-03T18:46:14  *** arubi has quit IRC
5052017-01-03T18:46:33  *** arubi has joined #bitcoin-core-dev
5062017-01-03T18:50:28  *** lainknight has joined #bitcoin-core-dev
5072017-01-03T18:55:22  *** Sosumi has joined #bitcoin-core-dev
5082017-01-03T19:05:58  *** lejitz has joined #bitcoin-core-dev
5092017-01-03T19:09:55  *** jtimon has quit IRC
5102017-01-03T19:11:49  *** arubi has quit IRC
5112017-01-03T19:12:04  *** arubi has joined #bitcoin-core-dev
5122017-01-03T19:14:40  *** LeMiner has quit IRC
5132017-01-03T19:31:58  *** Sosumi has quit IRC
5142017-01-03T19:35:44  *** jl2012 has quit IRC
5152017-01-03T19:35:44  *** kadoban has quit IRC
5162017-01-03T19:35:44  *** kinlo has quit IRC
5172017-01-03T19:35:44  *** Ylbam has quit IRC
5182017-01-03T19:35:44  *** Victorsueca has quit IRC
5192017-01-03T19:35:44  *** nOgAnOo has quit IRC
5202017-01-03T19:35:44  *** limpkin has quit IRC
5212017-01-03T19:35:44  *** PaulCape_ has quit IRC
5222017-01-03T19:35:44  *** luke-jr has quit IRC
5232017-01-03T19:35:44  *** murr4y has quit IRC
5242017-01-03T19:35:45  *** _mn3monic has quit IRC
5252017-01-03T19:35:45  *** Taek has quit IRC
5262017-01-03T19:35:53  *** _mn3monic has joined #bitcoin-core-dev
5272017-01-03T19:35:58  *** luke-jr has joined #bitcoin-core-dev
5282017-01-03T19:36:01  *** luke-jr has quit IRC
5292017-01-03T19:36:01  *** luke-jr has joined #bitcoin-core-dev
5302017-01-03T19:36:23  *** murr4y has joined #bitcoin-core-dev
5312017-01-03T19:36:54  *** Taek has joined #bitcoin-core-dev
5322017-01-03T19:36:55  *** kadoban has joined #bitcoin-core-dev
5332017-01-03T19:37:32  *** Victorsueca has joined #bitcoin-core-dev
5342017-01-03T19:37:40  *** PaulCapestany has joined #bitcoin-core-dev
5352017-01-03T19:38:22  *** profall has quit IRC
5362017-01-03T19:39:40  *** jl2012 has joined #bitcoin-core-dev
5372017-01-03T19:40:29  *** limpkin has joined #bitcoin-core-dev
5382017-01-03T19:42:05  *** Ylbam has joined #bitcoin-core-dev
5392017-01-03T19:46:20  *** profall has joined #bitcoin-core-dev
5402017-01-03T19:48:58  *** nOgAnOo has joined #bitcoin-core-dev
5412017-01-03T19:49:15  *** lejitz_ has joined #bitcoin-core-dev
5422017-01-03T20:00:37  *** Chris_Stewart_5 has quit IRC
5432017-01-03T20:09:48  <sdaftuar> BlueMatt: if I understand 9375 correctly, I think that when there is a fork, and some nodes have tip A and others tip B, #9375 would cause all nodes to end up downloading both A and B.
5442017-01-03T20:09:51  <gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
5452017-01-03T20:10:12  <sdaftuar> i guess that could result in a testnet DoS vector?
5462017-01-03T20:10:30  <sdaftuar> maybe that's the least of our testnet problems though
5472017-01-03T20:11:14  *** lejitz1 has joined #bitcoin-core-dev
5482017-01-03T20:12:31  *** lejitz has quit IRC
5492017-01-03T20:12:31  *** lejitz_ is now known as lejitz
5502017-01-03T20:12:44  *** wvr has joined #bitcoin-core-dev
5512017-01-03T20:16:53  *** Chris_Stewart_5 has joined #bitcoin-core-dev
5522017-01-03T20:17:01  *** CubicEarth has joined #bitcoin-core-dev
5532017-01-03T20:19:56  *** lainknight has quit IRC
5542017-01-03T20:20:40  *** lejitz1 has quit IRC
5552017-01-03T20:40:51  *** CubicEarth has quit IRC
5562017-01-03T20:41:06  *** bsm117532 has joined #bitcoin-core-dev
5572017-01-03T20:43:00  *** CubicEarth has joined #bitcoin-core-dev
5582017-01-03T20:53:04  *** udiWertheimer has joined #bitcoin-core-dev
5592017-01-03T20:53:53  *** dermoth has joined #bitcoin-core-dev
5602017-01-03T20:58:33  *** laurentmt has joined #bitcoin-core-dev
5612017-01-03T21:01:13  *** squidicuz has quit IRC
5622017-01-03T21:01:42  *** squidicuz has joined #bitcoin-core-dev
5632017-01-03T21:05:04  *** Squidicc has joined #bitcoin-core-dev
5642017-01-03T21:05:55  *** laurentmt has quit IRC
5652017-01-03T21:08:25  *** squidicuz has quit IRC
5662017-01-03T21:52:40  <bitcoin-git> [bitcoin] sipa pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/6dc4c43d326e...ce5c1f4acae4
5672017-01-03T21:52:41  <bitcoin-git> bitcoin/master 680b0c0 Suhas Daftuar: Release cs_main before calling ProcessNewBlock (cmpctblock handling)
5682017-01-03T21:52:41  <bitcoin-git> bitcoin/master bd02bdd Suhas Daftuar: Release cs_main before processing cmpctblock as header
5692017-01-03T21:52:42  <bitcoin-git> bitcoin/master ce5c1f4 Pieter Wuille: Merge #9252: Release cs_main before calling ProcessNewBlock, or processing headers (cmpctblock handling)...
5702017-01-03T21:52:55  <bitcoin-git> [bitcoin] sipa closed pull request #9252: Release cs_main before calling ProcessNewBlock, or processing headers (cmpctblock handling) (master...cb-lock) https://github.com/bitcoin/bitcoin/pull/9252
5712017-01-03T22:11:48  <bitcoin-git> [bitcoin] sipa pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/ce5c1f4acae4...2a524b8e8fe6
5722017-01-03T22:11:48  <bitcoin-git> bitcoin/master fb0c934 Luke Dashjr: Wallet: Let the interval-flushing thread figure out the filename
5732017-01-03T22:11:49  <bitcoin-git> bitcoin/master 5394b39 Luke Dashjr: Wallet: Split main logic from InitLoadWallet into CreateWalletFromFile
5742017-01-03T22:11:50  <bitcoin-git> bitcoin/master 2a524b8 Pieter Wuille: Merge #8776: Wallet refactoring leading up to multiwallet...
5752017-01-03T22:15:53  <gmaxwell> \O/
5762017-01-03T22:16:29  *** Squidicuz has joined #bitcoin-core-dev
5772017-01-03T22:18:37  *** Squidicc has quit IRC
5782017-01-03T22:19:04  *** Squidicc has joined #bitcoin-core-dev
5792017-01-03T22:21:37  *** Squidicuz has quit IRC
5802017-01-03T22:30:10  *** Guyver2 has quit IRC
5812017-01-03T22:31:12  <Chris_Stewart_5> Can we have OP_1NEGATE used as a push op code for witness program versioning?
5822017-01-03T22:32:57  <sipa> no, only OP_0, and OP_1..OP_16
5832017-01-03T22:34:16  <Chris_Stewart_5> Interesting, thanks sipa
5842017-01-03T22:35:08  <Chris_Stewart_5> Actually, is there any reason for this?
5852017-01-03T22:35:16  <sipa> not really
5862017-01-03T22:35:24  <sipa> it was considered sufficient :)
5872017-01-03T22:38:57  <luke-jr> by some* maybe; IMO it was overlooked.
5882017-01-03T22:40:02  <Chris_Stewart_5> Defintely a protocol 'quirk' since the BIP does say '1 byte push opcodes'
5892017-01-03T22:40:27  <luke-jr> not worth the effort to fix at this point though
5902017-01-03T22:41:02  <sipa> fix the bip :)
5912017-01-03T22:41:11  <sipa> (to match the code)
5922017-01-03T22:41:30  <Chris_Stewart_5> but it is so much easier to criticize! :P. I'll submit a pull req later..
5932017-01-03T22:44:05  *** fanquake has joined #bitcoin-core-dev
5942017-01-03T22:45:00  <sipa> thanks!
5952017-01-03T23:12:12  <sipa> Chris_Stewart_5: the BIP clearly says "for 0 to 16", no?
5962017-01-03T23:12:25  <sipa> (i missed that when suggesting you send a PR, sorry)
5972017-01-03T23:12:56  <Chris_Stewart_5> sipa: What is '0 to 16'? Bytes? Technically OP_1 is 0x51
5982017-01-03T23:13:12  <Chris_Stewart_5> I don't like the wording in that regard either.
5992017-01-03T23:13:22  <sipa> the value being pushed
6002017-01-03T23:13:29  <Chris_Stewart_5> Why not explicitly state what ops they are?
6012017-01-03T23:13:33  <sipa> sure
6022017-01-03T23:13:45  <sipa> i trying to find the misunderstanding
6032017-01-03T23:13:47  <Chris_Stewart_5> instead having to do the conversion in your head?
6042017-01-03T23:13:53  <sipa> if it isn't clear, reformulating helps
6052017-01-03T23:14:21  <Chris_Stewart_5> I don't think I have a misunderstanding per se, but I think can be clearer. Instead of having to infer the op codes from the text I think they should just be explicitly stated
6062017-01-03T23:14:49  <Chris_Stewart_5> Plus it falls more inline with the actual implementation IMO
6072017-01-03T23:14:54  <sipa> fair enough
6082017-01-03T23:17:06  <fanquake> sipa You shouldn't see significant speedup with #8610 if you have a large -dbcache set (like 2048), should you?
6092017-01-03T23:17:08  <gribble> https://github.com/bitcoin/bitcoin/issues/8610 | Share unused mempool memory with coincache by sipa · Pull Request #8610 · bitcoin/bitcoin · GitHub
6102017-01-03T23:17:47  <sipa> fanquake: depends how large, but the effect should certainly be less dramativ
6112017-01-03T23:17:50  <sipa> *dramatic
6122017-01-03T23:19:09  <fanquake> Ok. Just ran through with -dbcache=2048 on master and that PR. Both basically the same (~800s) to reindex to 280k blocks.
6132017-01-03T23:19:37  <fanquake> I think I'll run through again without uping the dbcache at all. That seemed to show the most significant speedup.
6142017-01-03T23:20:20  *** droark has quit IRC
6152017-01-03T23:21:54  <sipa> if you set dbcache to 10 i'm sure the difference will be spectacular :D
6162017-01-03T23:22:23  <luke-jr> Chris_Stewart_5: adding "valid" seems to make it less clear IMO
6172017-01-03T23:23:00  <Chris_Stewart_5> luke-jr: How so? A invalid 1 byte op code is OP_1NEGATE
6182017-01-03T23:23:44  <sipa> how about "a select subset of 1-byte push opcodes" ?
6192017-01-03T23:23:46  <Chris_Stewart_5> before it was a blanket statement, a 'one byte push op code' or something like that
6202017-01-03T23:23:54  <fanquake> heh yes I'm sure it would. When I first tested with 300mb dbcache there was something like a 30% speedup.
6212017-01-03T23:23:54  <sipa> oh, and OP_0 is not a 1-byte push
6222017-01-03T23:24:03  <Chris_Stewart_5> oo interesting
6232017-01-03T23:24:09  <Chris_Stewart_5> good point :/
6242017-01-03T23:24:42  <sipa> fanquake: if you feel like benchmarking, i have a few more utxo cache changes to test coming up :)
6252017-01-03T23:26:07  <fanquake> sipa sure
6262017-01-03T23:26:41  <Chris_Stewart_5> sipa: Then maybe just a 'select set of op codes' and then enumerate them like I have?
6272017-01-03T23:27:20  <sipa> Chris_Stewart_5: sgtm
6282017-01-03T23:28:27  <luke-jr> Chris_Stewart_5: OP_1NEGATE is valid though
6292017-01-03T23:28:50  <luke-jr> it's just not matched for witness purposes
6302017-01-03T23:29:03  <Chris_Stewart_5> (*this)[0] < OP_1 does it fail this predicate?
6312017-01-03T23:31:07  <Chris_Stewart_5> luke-jr: Here: https://github.com/bitcoin/bitcoin/blob/14d01309bed59afb08651f2b701ff90371b15b20/src/script/script.cpp#L228
6322017-01-03T23:32:34  <sipa> Chris_Stewart_5: depends what *this is...
6332017-01-03T23:34:05  *** justanotheruser has quit IRC
6342017-01-03T23:34:36  <BlueMatt> sdaftuar: yes, your comment about only calling NewPoWValidBlock on things that are better than our current tip is a good one
6352017-01-03T23:34:45  <BlueMatt> sdaftuar: and build on our tip
6362017-01-03T23:36:00  <BlueMatt> sdaftuar: re: https://github.com/bitcoin/bitcoin/pull/9375#discussion_r94483881 I think I'm missing your point here? did you mean to comment a bit further down where the check is to set bool send?
6372017-01-03T23:37:09  *** justanotheruser has joined #bitcoin-core-dev
6382017-01-03T23:43:49  *** droark has joined #bitcoin-core-dev
6392017-01-03T23:46:44  *** kinlo has joined #bitcoin-core-dev