12015-11-17T00:03:34 <dcousens> also, unrelated, but the total parse time (I realised that, before due to lazy evaluation, it was skipping ahead), when the result is piped to /dev/null, its parsing at ~210MiB/s, otherwise its blocked by output to about 110MiB/s
22015-11-17T00:04:26 <dcousens> (meaning, about 7 minutes for a full 50G parse w/ ~50G output)
32015-11-17T00:05:43 *** guest234234 has joined #bitcoin-core-dev
42015-11-17T00:16:01 <dcousens> sipa: I also found a blog post detailing what I'm running into, tl;dr all the garbage is just a long run of 0's
52015-11-17T00:16:58 <sipa> dcousens: that's preallocation
62015-11-17T00:17:30 <dcousens> sipa: is it meant to be written though?
72015-11-17T00:17:36 <sipa> dcousens: we preallocate the block files when first creating them, and then overwrite pieces of it with blocks; at the end, when leaving the file, we truncate
82015-11-17T00:18:16 <dcousens> sipa: well, considering my bitcoind is closed, and the file still has a huge run of 0's at the end
92015-11-17T00:18:16 <sipa> but things can go wrong i guess (for example, if the first write after restart is to a new file, you never 'leave' the old file)
102015-11-17T00:18:26 <dcousens> I'm guessing the truncate isn't 100% solid
112015-11-17T00:18:28 <sipa> dcousens: the last file will always have zeroes at the end
122015-11-17T00:18:34 <sipa> as it isn't full yet
132015-11-17T00:18:44 <dcousens> ok, by leaving the file you meant moving onto the next
142015-11-17T00:18:50 <sipa> right, indeed
152015-11-17T00:20:31 <gmaxwell> we prefill with zeros to prevent fragmenting the crap out of non-fragmentation resistant file systems with our incremental writes.
162015-11-17T00:20:47 <dcousens> gmaxwell: gotcha
172015-11-17T00:21:01 <dcousens> though, it probably shouldn't be in the middle of a .dat though
182015-11-17T00:25:08 <dcousens> a 10MiB gap infact, then 252 more blocks after
192015-11-17T00:27:09 <gmaxwell> dcousens: are these old files? We previously had bugs that might cause behavior like that, but I think they're all fixed.
202015-11-17T00:27:15 <GitHub175> [bitcoin] ptschip opened pull request #7034: Fixed Windows secp256k1 compilation issue #7018 (master...secp256k1_compile) https://github.com/bitcoin/bitcoin/pull/7034
212015-11-17T00:27:32 <dcousens> gmaxwell: its a fresh IBD with a just-compiled 4 days ago bitcoin/bitcoin master
222015-11-17T00:28:00 <dcousens> at most the build would be within 2 weeks
232015-11-17T00:28:49 <gmaxwell> okay, that sounds like a bug then.
242015-11-17T00:29:05 <dcousens> Happy to donate time to debug it, but, not sure where to start :S
252015-11-17T00:29:30 *** Anduck has quit IRC
262015-11-17T00:30:46 *** Anduck has joined #bitcoin-core-dev
272015-11-17T00:35:23 <GitHub69> [bitcoin] gmaxwell pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e54ebbf60097...0a547d2d5501
282015-11-17T00:35:23 <GitHub69> bitcoin/master 4d29032 Eric Lombrozo: Fixed integer comparison warning.
292015-11-17T00:35:24 <GitHub69> bitcoin/master 0a547d2 Gregory Maxwell: Merge pull request #7023...
302015-11-17T00:35:33 <GitHub106> [bitcoin] gmaxwell closed pull request #7023: Fixed integer comparison warning. (master...signed_int_comparison_fix) https://github.com/bitcoin/bitcoin/pull/7023
312015-11-17T00:39:57 <GitHub66> [bitcoin] gmaxwell pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/0a547d2d5501...972bf9c52987
322015-11-17T00:39:57 <GitHub66> bitcoin/master f6d9d5e Jonas Schnelli: add (max)uploadtarget infos to getnettotals RPC help
332015-11-17T00:39:58 <GitHub66> bitcoin/master 972bf9c Gregory Maxwell: Merge pull request #6999...
342015-11-17T00:40:02 <GitHub149> [bitcoin] gmaxwell closed pull request #6999: add (max)uploadtarget infos to getnettotals RPC help (master...2015/11/maxuploadtarget_rpc_help) https://github.com/bitcoin/bitcoin/pull/6999
352015-11-17T00:41:49 <GitHub25> [bitcoin] ptschip closed pull request #7034: Fixed Windows secp256k1 compilation issue #7018 (master...secp256k1_compile) https://github.com/bitcoin/bitcoin/pull/7034
362015-11-17T00:48:42 *** Guest23423 has joined #bitcoin-core-dev
372015-11-17T00:48:59 *** zooko has quit IRC
382015-11-17T00:52:23 *** guest234234 has quit IRC
392015-11-17T00:54:27 *** Ylbam has quit IRC
402015-11-17T01:00:49 <GitHub18> [bitcoin] gmaxwell pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/972bf9c52987...87ee0e2dbc12
412015-11-17T01:00:50 <GitHub18> bitcoin/master 598e494 Jorge Timón: Chainparams: Explicit CChainParams arg for main (pre miner):...
422015-11-17T01:00:50 <GitHub18> bitcoin/master 6bc9e40 Jorge Timón: Chainparams: Explicit CChainParams arg for miner:...
432015-11-17T01:00:51 <GitHub18> bitcoin/master 87ee0e2 Gregory Maxwell: Merge pull request #6986...
442015-11-17T01:00:54 <GitHub68> [bitcoin] gmaxwell closed pull request #6986: Globals: Don't call Params() from miner.cpp (master...global-chainparams-miner) https://github.com/bitcoin/bitcoin/pull/6986
452015-11-17T01:13:22 <GitHub122> [bitcoin] dcousens opened pull request #7035: torcontrol: only output disconnect if -debug=tor (master...torlog) https://github.com/bitcoin/bitcoin/pull/7035
462015-11-17T01:15:15 *** Anduck has quit IRC
472015-11-17T01:15:33 *** Anduck has joined #bitcoin-core-dev
482015-11-17T01:15:59 *** zooko has joined #bitcoin-core-dev
492015-11-17T01:16:03 <gmaxwell> wumpus: so valgrind is telling me the event_new TorController::disconnected_cb is creating a leak on clean shutdown with stop. But I've confirmed the destructor that calls event_del is correctly firing. Is there some extra libevent magic needed to cause this to get freed?
502015-11-17T01:16:33 <dcousens> gmaxwell: wumpus isn't here? :S
512015-11-17T01:16:55 <sipa> it seems he got truncated into wump
522015-11-17T01:17:27 <gmaxwell> I wondered why my client completed 'wump' :) (but apparently I didn't wonder that much)
532015-11-17T01:37:44 *** Guest23423 has quit IRC
542015-11-17T01:49:57 <morcos> gmaxwell: This is not a complaint but just a question, because I do think we should merge more of Jorge's refactoring changes, but we planning on trying to find a time to do a bunch of them so that we can consolidate the merge conflicts?
552015-11-17T01:53:23 <gmaxwell> Feel free to complain in any case; I was mindful of it creating some conflict-- but it didn't move any code and I thought refactors that created structural changes were the big concern there. I looked at existing open PRs (including yours) and concluded that the disruption from this would be small. (and if I felt that I could have merged yours first now and asked Jorge to rebase, I would have d
562015-11-17T01:53:29 <gmaxwell> one that).
572015-11-17T01:55:04 <morcos> yeah like i said, i'm not complaining, rebasing #7020 took a second, and #6898 is going to need some more work depending on #7020 and #7008 anyway. But I just wanted to be aware if we were about to have a slew of these, since we're also trying to get a bunch of stuff in before the freeze.
582015-11-17T01:55:15 <gmaxwell> morcos: if you go and fix your pull req, please provide feedback by screaming my name on IRC in proportion to what a pain in the ass iti is. :)
592015-11-17T01:56:10 <gmaxwell> Ah, no I wasn't planning on pulling most of the other ways. (there was another very minor change from jtimon that I just asked him to rebase wrt the one I just merged, that I think won't break anything)
602015-11-17T01:56:44 <morcos> speaking of, #6898 , the CreateNewBlock pull, it is quite a risky change, although with a big benefit. What else do you think we should do to get comfortable with it.
612015-11-17T01:57:23 <morcos> For instance, I called it out in the initial PR comment, but I didn't realize until reading your comments on IRC this weekend that not stopping to add txs at some minimum feerate is actually a pretty big negative change
622015-11-17T01:58:22 <morcos> I've since fixed that... But it worries me to change the code that miners everywhere will be relying on without extensive testing. What should that be?
632015-11-17T02:03:52 <gmaxwell> A couple things: we can probably add some tests that hand it simulated mempools and check some additional invarients, like that the selection is feerate monotone. We can deploy it early 'mining' (doesn't have to start actually mining, just call createnewblock and sanity check the output) on the main network. We can probably get some miners with non-trival hashpower to deploy it post merge durin
642015-11-17T02:03:58 <gmaxwell> g the 0.12 testing cycle; allowing us to catch an extreme stupidity before release.
652015-11-17T02:04:37 *** belcher has quit IRC
662015-11-17T02:04:41 <gmaxwell> It's also bad news that we have so many speedups in 0.12. That means miners will likely upgrade very quickly.
672015-11-17T02:04:54 <sipa> morcos: would it be possible to add the new getblocktemplate as a new RPC (getblocktemplatebeta or so) ?
682015-11-17T02:05:03 <sipa> and leaving the old one as-is for now
692015-11-17T02:05:04 <gmaxwell> Otherwise we would have some buffering from "well, it'll take a while for everyone to deploy it."
702015-11-17T02:05:59 <gmaxwell> hm. if we do that I'd want to do something to identify the blocks created with it... but I don't know that we can. :(
712015-11-17T02:06:57 <sipa> it probably makes little difference
722015-11-17T02:07:08 *** zooko has quit IRC
732015-11-17T02:07:31 <gmaxwell> well I just mean that if we see miners making pathalogically stupid blocks it would be nice to be able to tell if the new code is to blame. :)
742015-11-17T02:08:44 <sipa> gmaxwell: i mean that miners will likely use it anyway, even if it's called beta (although that does mean changing some software...)
752015-11-17T02:09:24 <morcos> sipa: i'm sure thats easy enough to do. Maybe that makes sense in that if there is a problem its easy to revert back without having to stop using 0.12
762015-11-17T02:09:31 <dcousens> a rescan shouldn't modify the .dat files, should it?
772015-11-17T02:09:59 <gmaxwell> sipa: that could also be a flag: oldcreatenewblock=0
782015-11-17T02:10:26 <sipa> dcousens: no
792015-11-17T02:10:28 <gmaxwell> Another reason to keep the old one around is if the miners are carrying patches to the old one, they can upgrade to 0.12 and keep them in place... OTOH, more for us to test.
802015-11-17T02:10:59 <morcos> I'd tested previously that it generated the exact same blocks, modulo the comments I put at the top of the pull. But I did forget one more. I'll go update that. I stop trying to fill the last little spot in the block early, but if I increase how much it'll search for that , they were the same
812015-11-17T02:12:00 <morcos> I think I like this idea though. If its a command line option then all we need is my duplicated CreateNewBlock code , and getblocktemplate checks the argument to see which CreateNewBlock it should call?
822015-11-17T02:12:04 <phantomcircuit> gmaxwell, the coinbase sig flags returned by getblocktemplate could be modified
832015-11-17T02:12:12 <phantomcircuit> im not sure if any miners actually use that though
842015-11-17T02:12:54 <morcos> Hmm.. kind of annoying for unit tests and rpc tests though....
852015-11-17T02:13:43 <phantomcircuit> when is feature freeze for 0.12 ?
862015-11-17T02:13:43 <sipa> morcos: or CreateNewBlock itself dispatches based on the command-line flag?
872015-11-17T02:13:58 <gmaxwell> phantomcircuit: lots of people ignore the coinbaseflags coming out of gbt. :(
882015-11-17T02:14:36 *** guest234234 has joined #bitcoin-core-dev
892015-11-17T02:15:30 <gmaxwell> morcos: it could just be a commandline flag, doubt we want rpc callers picking it on the fly.
902015-11-17T02:15:48 <morcos> One other comment I'll make about 6898 is that its clearly a WIP in the larger sense too. If we can rip out priority for 0.13, it'll clean up the code a lot, and also we might rework it yet again to do some intelligent CPFP
912015-11-17T02:16:19 <morcos> so we should think a bit about how we might wan tto make future major changes to the code as well?
922015-11-17T02:17:28 <morcos> gmaxwell: yes thats what i meant (checks the command line argument) i agree you should not be able to toggle back and forth
932015-11-17T02:17:29 <gmaxwell> Well I do kind of like having the ability to swap in new algorithims with runtime selection.
942015-11-17T02:17:54 <gmaxwell> I don't know how realistic that is generally, since I think different algorithims will require different indexes for efficient implementation.
952015-11-17T02:18:36 <morcos> yes and keeping two algorithms properly tested is basically impossible
962015-11-17T02:20:41 <gmaxwell> well right now with the current one there is a montone property I think it will obey, in that it will always select the highest feerate transactions, so anything omitted must be equal to or lower than the bottom feerate. So testing on real and simulated loads will at least tell us if its behaving crazily (like failing to mine a particular transaction causing it to clog the mempool)
972015-11-17T02:20:48 <gmaxwell> But fancier algorithms will lose that simple property.
982015-11-17T02:20:55 *** guest234234 has quit IRC
992015-11-17T02:34:24 <morcos> gmaxwell: you're assuming wiht a priority space of 0? Also that's not exactly correct, in that a high fee rate child could only have become elgible to be included when there was no longer enough space for it.
1002015-11-17T02:35:32 <gmaxwell> Yes, that was assuming priority space of 0.
1012015-11-17T02:35:35 <morcos> But I was suggesting we hack up the existing code to change the tie breaker and require that the blocks are identical for testing. I think that's easy enough to achieve. The problem is whether there is some degenerate situation where something bad happens
1022015-11-17T02:35:43 <gmaxwell> ah, indeed, it still needs a topological sort.
1032015-11-17T02:36:30 <phantomcircuit> morcos, so the answer i would like to give is "it doesn't matter they can run both and the block verification at the end will protect them from bugs"
1042015-11-17T02:36:44 <phantomcircuit> but i know that lots of them dont have proper setups so that isn't really true
1052015-11-17T02:36:45 <phantomcircuit> :(
1062015-11-17T02:37:25 <gmaxwell> phantomcircuit: if you note above I mentioned the case of some high feerate txn not getting mined when it should.
1072015-11-17T02:38:15 *** davec has quit IRC
1082015-11-17T02:39:51 <morcos> So there are some very weird edge case differences I guess. For instance, one of the versions has the sigop check before the block priority space check and the other has it after. so if you have a transaction that fills the block priority space, you now switch to fee rate sorted, but if it fails the sigop test, then code A will have moved on to fee txs, but code B will still try for another priority tx
1092015-11-17T02:40:42 <morcos> i can't remember which is which, and i haven't encountered that yet, nor do i expect to... but at a certain point, mimicking the old behavior is not what we want
1102015-11-17T02:41:02 <gmaxwell> no reason to think much of the old behavior is any good.
1112015-11-17T02:43:44 *** zooko has joined #bitcoin-core-dev
1122015-11-17T02:45:20 *** davec has joined #bitcoin-core-dev
1132015-11-17T02:57:58 <phantomcircuit> gmaxwell, i assume that was from a block maxing out checksigs?
1142015-11-17T03:32:06 <dcousens> gmaxwell: "One thought I had on this (before realizing there was already a pool"
1152015-11-17T03:32:18 <dcousens> did you mean s/pool/pull/?
1162015-11-17T03:36:07 *** davec has quit IRC
1172015-11-17T03:38:57 *** davec has joined #bitcoin-core-dev
1182015-11-17T03:51:12 *** jtimon has quit IRC
1192015-11-17T03:58:35 <BlueMatt> gmaxwell: nope, actually, master still verifies with all the pgp sigs
1202015-11-17T04:02:01 <BlueMatt> ofc jonasschnelli's key is completely unsigned, so......
1212015-11-17T04:02:21 <gmaxwell> dcousens: yep
1222015-11-17T04:02:47 <sipa> BlueMatt: i'll go see him when i'm in .ch (assuming he doesn't hate my face)
1232015-11-17T04:26:13 <BlueMatt> sipa: sign wump's key while you're up north :p
1242015-11-17T04:52:47 *** Squidicuz has joined #bitcoin-core-dev
1252015-11-17T04:56:30 *** sipa_ has joined #bitcoin-core-dev
1262015-11-17T04:56:41 *** cfields_ has joined #bitcoin-core-dev
1272015-11-17T04:59:58 *** dgenr8 has quit IRC
1282015-11-17T05:00:01 *** jonasschnelli has quit IRC
1292015-11-17T05:00:03 *** zmanian_ has quit IRC
1302015-11-17T05:00:03 *** Squidicc has quit IRC
1312015-11-17T05:00:04 *** da2ce7 has quit IRC
1322015-11-17T05:00:05 *** sipa has quit IRC
1332015-11-17T05:00:06 *** andytoshi has quit IRC
1342015-11-17T05:00:09 *** teward has quit IRC
1352015-11-17T05:00:10 *** cfields has quit IRC
1362015-11-17T05:00:10 *** phantomcircuit has quit IRC
1372015-11-17T05:00:11 *** amiller_ has quit IRC
1382015-11-17T05:00:12 *** BlueMatt has quit IRC
1392015-11-17T05:00:22 *** dgenr8 has joined #bitcoin-core-dev
1402015-11-17T05:00:29 *** Guest802 has joined #bitcoin-core-dev
1412015-11-17T05:00:40 *** jonasschnelli has joined #bitcoin-core-dev
1422015-11-17T05:00:57 *** phantomcircuit_ has joined #bitcoin-core-dev
1432015-11-17T05:02:08 *** BlueMatt has joined #bitcoin-core-dev
1442015-11-17T05:02:15 *** zooko has quit IRC
1452015-11-17T05:03:29 *** zmanian_ has joined #bitcoin-core-dev
1462015-11-17T05:03:54 *** andytoshi has joined #bitcoin-core-dev
1472015-11-17T05:05:41 *** teward has joined #bitcoin-core-dev
1482015-11-17T05:09:46 *** Guest802 has quit IRC
1492015-11-17T05:09:46 *** Guest802 has joined #bitcoin-core-dev
1502015-11-17T05:09:46 *** Guest802 is now known as amiller
1512015-11-17T05:10:00 *** da2ce7 has joined #bitcoin-core-dev
1522015-11-17T05:15:07 *** phantomcircuit_ is now known as phantomcircuit
1532015-11-17T05:17:14 *** zmanian_ has quit IRC
1542015-11-17T05:18:51 *** btcdrak has quit IRC
1552015-11-17T05:20:09 *** CodeShark has quit IRC
1562015-11-17T05:25:21 *** amiller has quit IRC
1572015-11-17T05:25:21 *** zarathustra has quit IRC
1582015-11-17T05:26:01 *** amiller_ has joined #bitcoin-core-dev
1592015-11-17T05:29:07 *** CodeShark has joined #bitcoin-core-dev
1602015-11-17T05:35:47 *** btcdrak has joined #bitcoin-core-dev
1612015-11-17T05:37:11 *** zmanian_ has joined #bitcoin-core-dev
1622015-11-17T06:07:38 *** zarathustra has joined #bitcoin-core-dev
1632015-11-17T06:07:38 *** zarathustra has joined #bitcoin-core-dev
1642015-11-17T06:07:49 *** pigeons has quit IRC
1652015-11-17T06:08:37 *** pigeons has joined #bitcoin-core-dev
1662015-11-17T06:09:00 *** pigeons is now known as Guest96832
1672015-11-17T06:12:20 <dcousens> sipa_: so
1682015-11-17T06:12:49 <dcousens> I'm still debugging this, and the garbage in my blk00289
1692015-11-17T06:13:00 <dcousens> is random transactions, but its not at all an actual block
1702015-11-17T06:13:17 <dcousens> like, its not prefixed by a header
1712015-11-17T06:14:59 <dcousens> http://pastie.org/10561905
1722015-11-17T06:15:26 <dcousens> Any ideas?
1732015-11-17T06:16:08 <dcousens> that pastie is hard to decipher, but tldr I printed some bytes from the previous block to see if maybe the length was off, but, again, that block verifies so :/
1742015-11-17T06:16:26 <dcousens> no idea, maybe theres a loose buffer overwrite somewhere?
1752015-11-17T06:17:33 <gmaxwell> dcousens: while syncing, we write to the file and only flush the indexes periodically, if you shut off prematurely, it'll continue at the last index write, and overwrite whatever blocks were there. Might this be consistent with what you're seeing?
1762015-11-17T06:18:46 <dcousens> gmaxwell: so you're thinking it had written a block, of say size N, it was then shutdown prematurely, and on re-open, it then wrote block of size M at that same index, where M < N?
1772015-11-17T06:20:19 <gmaxwell> Yes.
1782015-11-17T06:20:55 <gmaxwell> though unless it were the last block placed in the file (according to the block index), I don't know why it wouldn't have overwrote the lest.
1792015-11-17T06:21:44 <midnightmagic> out-of-order block storage
1802015-11-17T06:21:49 <midnightmagic> makes sense.
1812015-11-17T06:22:27 <midnightmagic> sort of..
1822015-11-17T06:29:43 <dcousens> gmaxwell: if I nuke this file
1832015-11-17T06:29:45 <dcousens> and rescan
1842015-11-17T06:29:50 <dcousens> Will it survive? :P
1852015-11-17T06:30:07 <gmaxwell> it'll rescan up to that part and begin downloading the rest after it.
1862015-11-17T06:30:24 <gmaxwell> (by rescan I mean reindex)
1872015-11-17T06:30:27 <dcousens> uh yeah
1882015-11-17T06:30:30 <dcousens> reindex*
1892015-11-17T06:30:36 <dcousens> won't use any of the other .dat files?
1902015-11-17T06:30:56 <dcousens> (as in, it'll abandon all blocks after it and re-download the remaining 15G?)
1912015-11-17T06:31:33 <gmaxwell> it'll use all the earlier ones, and none of the later ones.
1922015-11-17T06:32:00 <dcousens> ok
1932015-11-17T06:32:07 <gmaxwell> It doesn't have a sophicated indirect membership management because ... that only makes sense if you're going to do crazy things like randomly delete a file in the middle. :)
1942015-11-17T06:34:06 <dcousens> gmaxwell: :), well, at this point I can do this, or just ignore bytes until I reach a magic int then verify the block :/
1952015-11-17T06:34:50 <dcousens> the 2.5 minute blockchain parse is too good to give up now haha
1962015-11-17T06:35:10 *** Anduck has quit IRC
1972015-11-17T06:35:23 *** Anduck has joined #bitcoin-core-dev
1982015-11-17T06:36:10 <gmaxwell> dcousens: what are you doing this parse for?
1992015-11-17T06:37:28 <dcousens> gmaxwell: custom index building for various dbs, its the fastest way to bootstrap, 5minutes to write it all out then just use the RPC to get anything else
2002015-11-17T06:37:36 *** tulip has joined #bitcoin-core-dev
2012015-11-17T06:38:02 <dcousens> much nicer then waiting 40+hrs for an RPC level sync, or hacking bitcoind to do it for me
2022015-11-17T06:38:33 <gmaxwell> You benchmark that rpc scan since we got libevent?
2032015-11-17T06:38:57 <dcousens> Its just become a bit of a timesink now because of this weird run of 000's and this issue.
2042015-11-17T06:39:15 <dcousens> gmaxwell: actually I haven't
2052015-11-17T06:40:09 <gmaxwell> I don't know which changes made it faster, but its much faster than it used to be; still going to be slow compared to scanning the files directly probably.
2062015-11-17T06:40:24 <dcousens> gmaxwell: but maybe not slow enough to warrant this, point
2072015-11-17T06:41:01 <gmaxwell> (well also, if its slow but fast enough to use; I'd like to optimize it further!)
2082015-11-17T06:41:14 <dcousens> gmaxwell: the point was also so I had a tooling to be able to do things like check ancestor/depths super quick etc
2092015-11-17T06:42:07 <dcousens> for service level bootstrapping, the new RPC speed may be enough,but analytically I'd still like a fast parse.
2102015-11-17T06:43:30 <gmaxwell> just take care, there have been some things that lost funds because they were doing block scanning and didn't know some blocks were orphans... :-/
2112015-11-17T06:44:02 <dcousens> gmaxwell: indeed, noticed the orphans
2122015-11-17T06:44:34 <tulip> there's also traps for people trying to do block parsing as well, some transactions contain blocks.
2132015-11-17T06:45:13 <dcousens> haha, like an 80byte OP_RETURN?
2142015-11-17T06:45:52 <tulip> the minimum block size is more than 80 bytes, but something like that.
2152015-11-17T06:48:40 <dcousens> gmaxwell: I'll give a play at the RPC sync speed
2162015-11-17T06:48:43 <dcousens> let you know :)
2172015-11-17T06:59:50 <gmaxwell> anyone around that understands the ZMQ interface? I'm trying to figure out how it hooks in to get a signal of a new block. I see nothing in the codebase that references CZMQNotificationInterface::UpdatedBlockTip at all.
2182015-11-17T07:00:22 <dcousens> gmaxwell: give me an hour and I might be sharing that question with you
2192015-11-17T07:04:03 <aj> gmaxwell: src/validationinterface.cpp sets it up as a signal, and src/main.cpp triggers the signal?
2202015-11-17T07:05:18 <aj> gmaxwell: or src/init.cpp has RegisterValidationInterface(pzmqNotificationInterface); to set it up in the first place?
2212015-11-17T07:10:06 <gmaxwell> thanks!
2222015-11-17T07:12:48 *** Ylbam has joined #bitcoin-core-dev
2232015-11-17T07:13:55 <GitHub177> [bitcoin] gmaxwell opened pull request #7037: Move the blocknotify callback ahead of peer announcement. (master...notify_early) https://github.com/bitcoin/bitcoin/pull/7037
2242015-11-17T07:15:51 *** wump is now known as wumpus
2252015-11-17T07:29:08 <GitHub2> [bitcoin] gmaxwell pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/87ee0e2dbc12...eac53ec99201
2262015-11-17T07:29:09 <GitHub2> bitcoin/master 141c44e MarcoFalke: [contrib] Update versionprefix to "bitcoin-core" in verify.sh
2272015-11-17T07:29:09 <GitHub2> bitcoin/master a6d5a65 MarcoFalke: [trivial] contrib: Fix `echo`s in verify.sh
2282015-11-17T07:29:10 <GitHub2> bitcoin/master eac53ec Gregory Maxwell: Merge pull request #7026...
2292015-11-17T07:29:13 <GitHub74> [bitcoin] gmaxwell closed pull request #7026: [contrib] Update versionprefix to "bitcoin-core" in verify.sh (master...MarcoFalke-2015-verify) https://github.com/bitcoin/bitcoin/pull/7026
2302015-11-17T07:33:26 <gmaxwell> Luke-Jr: https://github.com/bitcoin/bitcoin/pull/6851 wants rebase
2312015-11-17T07:42:04 *** MarcoFalke has joined #bitcoin-core-dev
2322015-11-17T07:51:47 *** jonasschnelli has quit IRC
2332015-11-17T07:51:48 *** jonasschnelli has joined #bitcoin-core-dev
2342015-11-17T08:07:47 <jonasschnelli> sipa: BlueMatt: i'll go see him when i'm in .ch (assuming he doesn't hate my face) <--- Hah. The question is, if you don't hate my face after bothering your with tons of wallet/main.cpp questions. :)
2352015-11-17T08:08:32 <jonasschnelli> sipa: I would be in Zurich next Tuesday 24., if you want to sign my key
2362015-11-17T08:13:55 <btcdrak> I see wumpus has been untruncated thankfully :-P
2372015-11-17T08:21:29 *** Guyver2 has joined #bitcoin-core-dev
2382015-11-17T08:38:57 *** MarcoFalke has quit IRC
2392015-11-17T09:02:42 *** guest234234 has joined #bitcoin-core-dev
2402015-11-17T09:14:43 *** MarcoFalke has joined #bitcoin-core-dev
2412015-11-17T09:15:53 *** guest234234 has quit IRC
2422015-11-17T09:32:33 *** rubensayshi has joined #bitcoin-core-dev
2432015-11-17T09:39:14 *** MarcoFalke has quit IRC
2442015-11-17T09:40:32 *** GAit has joined #bitcoin-core-dev
2452015-11-17T09:43:28 *** tulip has quit IRC
2462015-11-17T09:56:59 *** d_t has quit IRC
2472015-11-17T09:58:47 <CodeShark> The usual procedure for BIP proposals is to discuss on mailing list first. But given that at least a few important reviewers/critics either have unsubscribed to the ML entirely or else just ignore it, should we discuss it here or in bitcoin-dev?
2482015-11-17T10:01:09 <wumpus> well definitely not here, this channel is just for bitcoin core related programming work. The proper answer to your question is stil 'the mailing list'. #bitcoin-dev is fine, but IRC discussion tends to be more ephermal than mailing lists
2492015-11-17T10:01:42 *** tulip has joined #bitcoin-core-dev
2502015-11-17T10:01:59 <wumpus> (or #bitcoin-wizards if it is moonmath related :-) )
2512015-11-17T10:05:00 *** GAit has quit IRC
2522015-11-17T10:05:15 *** GAit has joined #bitcoin-core-dev
2532015-11-17T10:05:26 *** GAit has quit IRC
2542015-11-17T10:10:14 <wumpus> btcdrak: tends to happen when I get disconnected from the IRC server, the client will automatically butcher my name when it is already used
2552015-11-17T10:10:31 <btcdrak> wumpus: it have us all a giggle yesterday :)
2562015-11-17T10:10:44 <btcdrak> I was going to suggest we all truncate in solidarity!
2572015-11-17T10:10:49 <btcdrak> :)
2582015-11-17T10:12:37 <wumpus> lol :)
2592015-11-17T10:20:38 <wumpus> gmaxwell: re: valgrind tests, what version of libevent are you using?
2602015-11-17T10:21:20 <wumpus> please use an up-to-date version of libevent, I know for fact a few (minor) memory leaks have been fixed over time
2612015-11-17T10:23:33 <wumpus> (it's very possible that there is an actual leak caused by our code, but to avoid getting stuck in unnecessary rabbit holes I'd like to rule out issues that were already fixed years ago :-) )
2622015-11-17T10:28:30 *** davec has quit IRC
2632015-11-17T10:34:47 *** Anduck has quit IRC
2642015-11-17T10:35:04 *** Anduck has joined #bitcoin-core-dev
2652015-11-17T10:49:38 <btcdrak> who maintains the Ubuntu ppa packages for bitcoin? there are issues being reported https://www.reddit.com/r/Bitcoin/comments/3t04fp/psa_please_upgrade_to_bitcoin_core_0112_to/cx33vpw?context=3
2662015-11-17T10:51:38 *** GAit has joined #bitcoin-core-dev
2672015-11-17T10:52:15 *** tulip has quit IRC
2682015-11-17T11:00:06 *** randy-waterhouse has quit IRC
2692015-11-17T11:04:11 <sipa_> btcdrak: BlueMatt
2702015-11-17T11:04:17 *** sipa_ is now known as sipa
2712015-11-17T11:04:41 <btcdrak> hi
2722015-11-17T11:05:33 *** davec has joined #bitcoin-core-dev
2732015-11-17T11:08:43 <BlueMatt> btcdrak: huh? ubuntu 11.04???
2742015-11-17T11:09:04 <BlueMatt> 11.04 has been fully deprecated since 2013?!
2752015-11-17T11:12:12 <wumpus> right - 11.04 is not a LTS release
2762015-11-17T11:12:25 <wumpus> they should upgrade to 12.04, at the least
2772015-11-17T11:33:17 *** guest234234 has joined #bitcoin-core-dev
2782015-11-17T11:42:40 *** GAit has quit IRC
2792015-11-17T11:45:06 *** GAit has joined #bitcoin-core-dev
2802015-11-17T11:46:16 *** rav3n_pl has joined #bitcoin-core-dev
2812015-11-17T11:48:29 <rav3n_pl> hello. im looking to simple way to check for double spends of unconfirmed transaction. ive found bitcoinXT that return some info on "gettransaction" rpc, but it can be used only for in-wallet transactions. is there a way, to check it for non-wallet transactions?
2822015-11-17T11:51:52 <rav3n_pl> i can parse mempool transactions, but trying to avoid it
2832015-11-17T11:54:38 <phantomcircuit> rav3n_pl, that's impossible
2842015-11-17T11:55:02 <rav3n_pl> :/ bad
2852015-11-17T11:55:17 <phantomcircuit> rav3n_pl, also you're way offtopic
2862015-11-17T11:55:21 <rav3n_pl> :D
2872015-11-17T11:55:28 <rav3n_pl> sorry
2882015-11-17T11:55:45 <rav3n_pl> not found anythin closer in channel list
2892015-11-17T11:55:51 <wumpus> #bitcoin please
2902015-11-17T11:56:01 <rav3n_pl> okay, ty
2912015-11-17T12:44:53 *** jtimon has joined #bitcoin-core-dev
2922015-11-17T12:48:32 *** ParadoxSpiral has joined #bitcoin-core-dev
2932015-11-17T12:49:52 *** rav3n_pl is now known as rav3n_pl_away
2942015-11-17T12:52:39 *** ParadoxSpiral has quit IRC
2952015-11-17T12:56:52 *** Guest96832 is now known as pigeons
2962015-11-17T13:03:55 *** MarcoFalke has joined #bitcoin-core-dev
2972015-11-17T13:09:58 *** GAit has quit IRC
2982015-11-17T13:12:10 *** Anduck has quit IRC
2992015-11-17T13:13:48 *** Anduck has joined #bitcoin-core-dev
3002015-11-17T13:15:29 *** ParadoxSpiral has joined #bitcoin-core-dev
3012015-11-17T13:17:58 <jtimon> what I was supposed to do with "undefined reference to `secp256k1_ecdsa_sign_recoverable'", git clean -f and make clean didn't do it...
3022015-11-17T13:18:39 <phantomcircuit> jtimon, git clean -fdx
3032015-11-17T13:19:37 <jtimon> phantomcircuit: thanks
3042015-11-17T13:22:19 *** MarcoFalke has quit IRC
3052015-11-17T13:31:48 *** GAit has joined #bitcoin-core-dev
3062015-11-17T13:42:48 *** AtashiCon has quit IRC
3072015-11-17T13:42:54 *** Arnavion3 has joined #bitcoin-core-dev
3082015-11-17T13:42:58 *** Arnavion3 is now known as AtashiCon
3092015-11-17T13:44:06 *** murr4y has quit IRC
3102015-11-17T13:48:04 *** murr4y has joined #bitcoin-core-dev
3112015-11-17T13:49:36 *** fanquake has joined #bitcoin-core-dev
3122015-11-17T13:50:10 <fanquake> Do you still need to pass âwith-gui=qt5 to configure to get a qt5 build, or do we change it to default to qt5 when available?
3132015-11-17T13:59:22 <jtimon> andytoshi: from what we talked previously, I think you like #6907
3142015-11-17T14:00:20 <wumpus> fanquake: no longer necessary to provide that
3152015-11-17T14:00:34 <wumpus> if you have qt4 and qt5 development headers installed it will now default to qt5
3162015-11-17T14:03:26 <fanquake> wumpus nice; thought I'd seen a pull to change it. Good that we've made that change.
3172015-11-17T14:03:58 <jtimon> fanquake: wumpus: maybe something for the release notes?
3182015-11-17T14:05:21 <sipa> jtimon: the releases use qt5 for a while already, this only applies to people building from scratch, i think
3192015-11-17T14:08:11 *** GAit has quit IRC
3202015-11-17T14:08:40 <wumpus> jtimon: there's already so many "notable changes" for 0.12 that I doubt this one will make the cut - it'll be in the long list of pulls/commits anyway
3212015-11-17T14:08:46 <wumpus> sipa: right
3222015-11-17T14:09:01 <jtimon> sipa: wumpus fair enough
3232015-11-17T14:11:00 <fanquake> You use [skip-ci] in the body of a pull to skip travis right?
3242015-11-17T14:12:18 <wumpus> yes, that should be possible
3252015-11-17T14:12:26 <wumpus> (I'm not sure what the exact incantation is)
3262015-11-17T14:15:17 <GitHub159> [bitcoin] fanquake opened pull request #7040: [doc] Update OS X build notes for new qt5 configure (master...patch-4) https://github.com/bitcoin/bitcoin/pull/7040
3272015-11-17T14:23:25 <GitHub45> [bitcoin] fanquake opened pull request #7041: [doc][trivial] Update OS X install instructions (master...patch-5) https://github.com/bitcoin/bitcoin/pull/7041
3282015-11-17T15:03:32 *** guest234234 has quit IRC
3292015-11-17T15:09:29 <GitHub64> [bitcoin] fanquake opened pull request #7042: [doc] Update Debian watch & control (master...update-debian) https://github.com/bitcoin/bitcoin/pull/7042
3302015-11-17T15:13:15 *** zooko has joined #bitcoin-core-dev
3312015-11-17T15:13:41 *** Anduck has quit IRC
3322015-11-17T15:30:36 *** treehug88 has joined #bitcoin-core-dev
3332015-11-17T15:33:12 *** go1111111 has quit IRC
3342015-11-17T15:36:24 *** MarcoFalke has joined #bitcoin-core-dev
3352015-11-17T15:38:40 *** paveljanik has joined #bitcoin-core-dev
3362015-11-17T15:38:40 *** paveljanik has joined #bitcoin-core-dev
3372015-11-17T15:58:27 *** MarcoFalke has quit IRC
3382015-11-17T16:00:51 <sdaftuar> sipa: wumpus: any thoughts on 7020? morcos and i both have pulls that change the CTxMemPoolEntry constructor which causes a lot of work updating unit tests.
3392015-11-17T16:01:21 <sdaftuar> if 7020 is likely to go in i can rebase #6915 on it though
3402015-11-17T16:02:08 <sdaftuar> (it needs to be rebased anyway)
3412015-11-17T16:06:25 *** MarcoFalke has joined #bitcoin-core-dev
3422015-11-17T16:06:32 *** moli has quit IRC
3432015-11-17T16:07:22 *** moli has joined #bitcoin-core-dev
3442015-11-17T16:15:23 *** zooko has quit IRC
3452015-11-17T16:19:27 *** fanquake has quit IRC
3462015-11-17T16:22:17 <Luke-Jr> gmaxwell: I can rebase it easily, but I wonder if I should try to get master to actually build first :/
3472015-11-17T16:28:34 <Luke-Jr> looks like I had to manually re-configure. more build system issues, sigh
3482015-11-17T16:42:23 *** rubensayshi has quit IRC
3492015-11-17T16:42:24 <MarcoFalke> Trying to set up gitian in fedora. What should I do when ./bin/gbuild asks for a password for ubuntu@localhost?
3502015-11-17T16:42:28 <MarcoFalke> wumpus ?
3512015-11-17T16:42:58 <sipa> MarcoFalke: try 'ubuntu'
3522015-11-17T16:43:37 <wumpus> with lxc? you could add a sudoers rule that allows 'lxc-start' without password, there's an example in doc/gitian-building.md
3532015-11-17T16:43:37 <MarcoFalke> nice
3542015-11-17T16:44:41 <MarcoFalke> Why would I need to type the password 'ubuntu' five times? I tried it up to three times and assumed it failed because there was no response.
3552015-11-17T16:44:54 <MarcoFalke> Anyway, why isn't it using the keys I created?
3562015-11-17T16:45:10 <MarcoFalke> I am using kvm
3572015-11-17T16:45:56 <wumpus> with kvm it shouldn't be asking passwords at all, kvm sandboxes can be run as unprivileged users, IIRC you only need a few root commands to make the VM in make-base-vm
3582015-11-17T16:46:13 <wumpus> but not on gbuild, gbuild asking passwords is absolutely wrong
3592015-11-17T16:46:35 <MarcoFalke> make-base-vm asked for my password on the fedora machine
3602015-11-17T16:46:44 <wumpus> yes, that's normal
3612015-11-17T16:47:06 <wumpus> if gbuild asks for a password *inside* the image there's certainly something wrong with it
3622015-11-17T16:47:15 <wumpus> what vmbuilder version did you use?
3632015-11-17T16:48:17 <MarcoFalke> 0.12.4
3642015-11-17T16:49:00 *** d_t has joined #bitcoin-core-dev
3652015-11-17T16:49:36 <wumpus> installed from source or from a package?
3662015-11-17T16:50:10 <MarcoFalke> wget http://archive.ubuntu.com/ubuntu/pool/universe/v/vm-builder/vm-builder_0.12.4+bzr494.orig.tar.gz
3672015-11-17T16:50:15 <MarcoFalke> Now it's asking for root@localhost password
3682015-11-17T16:50:44 <wumpus> ok, shouldn't be anything wrong with that
3692015-11-17T16:50:58 <wumpus> not sure what causes your issues then
3702015-11-17T16:53:44 <wumpus> but my guess is that something went wrong while building the image
3712015-11-17T16:55:35 <MarcoFalke> The var/id_dsa key is supposed to be used by the ubuntu@localhost user?
3722015-11-17T16:58:25 <GitHub86> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/eac53ec99201...e8df8a5077df
3732015-11-17T16:58:25 <GitHub86> bitcoin/master e587bc3 Alex Morcos: Implement helper class for CTxMemPoolEntry constructor...
3742015-11-17T16:58:26 <GitHub86> bitcoin/master e8df8a5 Wladimir J. van der Laan: Merge pull request #7020...
3752015-11-17T16:58:30 <GitHub93> [bitcoin] laanwj closed pull request #7020: Implement helper class for CTxMemPoolEntry constructor (master...EntryHelper) https://github.com/bitcoin/bitcoin/pull/7020
3762015-11-17T17:00:06 <wumpus> yes, those keys are passed into vmbuilder
3772015-11-17T17:00:28 <wumpus> I think they are supposed to be installed inside the VM for both the root and ubuntu user
3782015-11-17T17:01:02 <wumpus> --ssh-key=var/id_dsa.pub --ssh-user-key=var/id_dsa.pub
3792015-11-17T17:03:06 <morcos> thanks wumpus!
3802015-11-17T17:11:03 <MarcoFalke> Trying with precise, now.
3812015-11-17T17:22:12 *** MarcoFalke has quit IRC
3822015-11-17T17:25:54 *** MarcoFalke has joined #bitcoin-core-dev
3832015-11-17T17:51:55 *** btcdrak has quit IRC
3842015-11-17T17:53:08 *** MarcoFalke has quit IRC
3852015-11-17T17:53:40 *** CodeShark has quit IRC
3862015-11-17T17:55:53 *** zmanian_ has quit IRC
3872015-11-17T17:59:59 *** davec has quit IRC
3882015-11-17T18:00:16 *** davec has joined #bitcoin-core-dev
3892015-11-17T18:05:06 *** d4de has quit IRC
3902015-11-17T18:06:47 *** zooko has joined #bitcoin-core-dev
3912015-11-17T18:13:27 *** rav3n_pl_away is now known as rav3n_pl
3922015-11-17T18:43:26 *** andytoshi has quit IRC
3932015-11-17T18:51:06 <gmaxwell> wumpus: I'll repro on something very new (though this particular system isn't an old one); mostly making sure that it wasn't known and harmless.
3942015-11-17T19:04:10 *** zooko has quit IRC
3952015-11-17T19:11:23 *** zmanian_ has joined #bitcoin-core-dev
3962015-11-17T19:15:31 *** btcdrak has joined #bitcoin-core-dev
3972015-11-17T19:17:40 *** andytoshi has joined #bitcoin-core-dev
3982015-11-17T19:39:27 *** trippysalmon has joined #bitcoin-core-dev
3992015-11-17T19:40:31 *** CodeShark has joined #bitcoin-core-dev
4002015-11-17T19:44:03 *** sdaftuar has quit IRC
4012015-11-17T19:51:56 <GitHub126> [bitcoin] instagibbs opened pull request #7044: RPC: Added additional config option for multiple RPC users. (master...multrpc) https://github.com/bitcoin/bitcoin/pull/7044
4022015-11-17T20:02:06 *** evoskuil has quit IRC
4032015-11-17T20:21:03 *** trippysalmon has quit IRC
4042015-11-17T20:38:46 *** evoskuil has joined #bitcoin-core-dev
4052015-11-17T20:53:02 <GitHub121> [bitcoin] luke-jr opened pull request #7045: Bugfix: Use unique autostart filenames on Linux for testnet/regtest (master...linux_autostart_unique) https://github.com/bitcoin/bitcoin/pull/7045
4062015-11-17T20:53:36 *** zooko has joined #bitcoin-core-dev
4072015-11-17T21:02:31 *** moli has quit IRC
4082015-11-17T21:02:46 *** evoskuil has quit IRC
4092015-11-17T21:08:53 *** go1111111 has joined #bitcoin-core-dev
4102015-11-17T21:17:37 *** paveljanik has quit IRC
4112015-11-17T21:21:58 *** randy-waterhouse has joined #bitcoin-core-dev
4122015-11-17T21:30:18 <Luke-Jr> I wonder if #6978 (qt -fPIC stuff) should be backported?
4132015-11-17T21:32:15 *** SatoshisCat has joined #bitcoin-core-dev
4142015-11-17T21:32:24 *** MarcoFalke has joined #bitcoin-core-dev
4152015-11-17T21:37:30 *** tulip has joined #bitcoin-core-dev
4162015-11-17T21:47:18 *** belcher has joined #bitcoin-core-dev
4172015-11-17T21:51:18 <GitHub102> [bitcoin] pstratem opened pull request #7046: [WIP] Net: Ignore "tx" messages in blocks only mode. (master...2015-11-17-blocksonly) https://github.com/bitcoin/bitcoin/pull/7046
4182015-11-17T21:58:26 *** d_t has quit IRC
4192015-11-17T22:00:38 <phantomcircuit> gmaxwell, oops i just realized i messed up the whitelistalways stuff
4202015-11-17T22:00:40 * phantomcircuit goes to fix
4212015-11-17T22:01:04 <phantomcircuit> (it technically works but it drops the inv so the whitelisted peer has to actually push the tx blindly)
4222015-11-17T22:01:47 *** treehug88 has quit IRC
4232015-11-17T22:02:46 <phantomcircuit> sipa, i end up calculating bool fAcceptTxs, is that something you would see being in CNodeState? (currently it's constant during runtime)
4242015-11-17T22:03:08 <phantomcircuit> (unless you can remove the whitelist flag at runtime, can you?)
4252015-11-17T22:03:27 <sipa> no, whitelist won't change at runtime
4262015-11-17T22:05:35 <phantomcircuit> BlueMatt, can i disable the mempool by limiting the size to 0?
4272015-11-17T22:05:42 <phantomcircuit> morcos, same question^
4282015-11-17T22:06:17 <BlueMatt> i think you have to set it to some big number
4292015-11-17T22:06:20 <BlueMatt> but i dont recall now
4302015-11-17T22:06:33 <phantomcircuit> although i guess if im going to be relaying whitelisted peers transactions i might as well also be keeping them in the mempool
4312015-11-17T22:06:39 <phantomcircuit> gmaxwell, any opinion there?
4322015-11-17T22:06:44 <sipa> BlueMatt: he means disabling the mempool, not disabling limiting :)
4332015-11-17T22:06:46 <morcos> phantomcircuit: also don't recall, sorry
4342015-11-17T22:06:52 <morcos> oh
4352015-11-17T22:07:16 <phantomcircuit> yes
4362015-11-17T22:08:29 <tulip> Leaving block file 375: CBlockFileInfo(blocks=0, size=0, heights=0...0, time=1970-01-01...1970-01-01)
4372015-11-17T22:08:37 <morcos> yikes, it looks like you can't because of bad interaction with descendantlimit
4382015-11-17T22:08:56 <phantomcircuit> so blocksonly=1 whitelistalwaysrelay=1 should the node be adding the transaction to it's mempool or just blindly passing it to every peer
4392015-11-17T22:09:09 <sipa> tulip: oh, i think that's harmless
4402015-11-17T22:09:30 <morcos> BlueMatt: do you think it should be completely prohibited from violating the desired ratio, or just warn
4412015-11-17T22:10:15 <BlueMatt> phantomcircuit: I assume you'll break things if you do, but it should disable mempool, yes
4422015-11-17T22:10:32 <BlueMatt> oh, yea, what morcos said
4432015-11-17T22:10:45 <phantomcircuit> BlueMatt, im not sure what i'd break doing that though since it's already relaying whitelisted peers transactions even if they're rejected from the mempool entirely
4442015-11-17T22:10:48 <BlueMatt> morcos: hmm? which ratio?
4452015-11-17T22:11:12 <morcos> BlueMatt: if you look at 6557, i implemented that check differently, so the descendentsizelimit is automatically maxed at the right ratio of your mempool size unless you explicitly set it
4462015-11-17T22:11:17 <BlueMatt> phantomcircuit: mostly "there be dragons" (of unspecified type)
4472015-11-17T22:11:19 <tulip> sipa: nothing seems to be on fire otherwise. is it caused by something broken in my local state, or a recent change in master?
4482015-11-17T22:11:19 <morcos> that might be a better parameter interaction
4492015-11-17T22:12:11 <BlueMatt> morcos: sorry, I've been behind on bitcoin core...people were starting to notice the relay network had been broken for a long time so i figured i needed to fix it
4502015-11-17T22:12:41 <morcos> BlueMatt: you return error if the mempool is less than 40 * the descendant size limit. i'm saying if the descendant size limit isn't set, you should just silently cap it at 1/40 of mempool. and if it is explicitly set, let any two values go, evne if lmiting will be adversely affected
4512015-11-17T22:13:01 <michagogo> 16:11:00 <fanquake> You use [skip-ci] in the body of a pull to skip travis right?
4522015-11-17T22:13:09 <morcos> 6557 was just our version of the limiting pull, its closed now, i'm just pointing out a different way to do the parameter interaction
4532015-11-17T22:13:17 <michagogo> I think it's [skip ci] or [ci skip], both work
4542015-11-17T22:13:23 <michagogo> I don't know if - works
4552015-11-17T22:13:24 <sipa> tulip: i believe it's due to a recent change in master
4562015-11-17T22:13:45 <michagogo> And I think it's the commit message, though I'm not sure about that
4572015-11-17T22:14:07 <phantomcircuit> BlueMatt, hmm well then i'll have to muck about with the "tx" ProcessMessage logic a lot more than i wanted to
4582015-11-17T22:14:16 <phantomcircuit> would anybody be opposed to moving it to it's own function?
4592015-11-17T22:14:29 <phantomcircuit> (indeed most of the logic in ProcessMessage would do well in it's own function)
4602015-11-17T22:16:50 <morcos> phantomcircuit: ha ha, sdaftuar and i would LOVE that. its the most annoying part of our historical data simulator right now
4612015-11-17T22:18:02 *** pmienk_ has quit IRC
4622015-11-17T22:18:28 <phantomcircuit> is anybody going to kill me if i submit a pr that moves each of the message handling routines into it's own function?
4632015-11-17T22:18:35 <phantomcircuit> speak now or yell at me later
4642015-11-17T22:19:47 <BlueMatt> morcos: hmm...I'm inclined to say leave it as is...I prefer the documented parameters to just refuse to run if it's gonna end up doing something dumb (people will shoot themselves in the foot if you let them
4652015-11-17T22:19:55 <BlueMatt> phantomcircuit: please god do
4662015-11-17T22:20:19 <BlueMatt> morcos: i wouldnt complain loudly if it was changed, but its not like the limit enforced now is crazy
4672015-11-17T22:20:43 <phantomcircuit> bool static ProcessVersionMessage(CNode* pfrom, CDataStream& vRecv, int64_t nTimeReceived) EXCLUSIVE_LOCKS_REQUIRED(cs_main)
4682015-11-17T22:20:46 <phantomcircuit> etc
4692015-11-17T22:22:23 <phantomcircuit> hmm need strCommand too
4702015-11-17T22:23:01 <tulip> sipa: thanks.
4712015-11-17T22:24:20 * BlueMatt -> lunch
4722015-11-17T22:24:29 <sipa> phantomcircuit: i've argued against that before, because it will conflict with pretty much every PR there is
4732015-11-17T22:24:55 <sipa> phantomcircuit: though that was under the assumption that some decent modularization of msg handling would happen instead, and so far, it hasn't
4742015-11-17T22:25:18 <phantomcircuit> sipa, it probably will, and it probably always will
4752015-11-17T22:25:55 <sipa> yes, but moving things now, and then doing clean modularization later means it breaks everything twice :)
4762015-11-17T22:25:58 <sipa> for the same benefit
4772015-11-17T22:26:00 *** sdaftuar_ has joined #bitcoin-core-dev
4782015-11-17T22:26:33 <phantomcircuit> sipa, it'll at least be easier to modularize once it's "cleaned" up a bit
4792015-11-17T22:26:50 <sipa> hardly, i think
4802015-11-17T22:27:54 <phantomcircuit> sipa, when you say modularization you're referring to having a ping module etc
4812015-11-17T22:27:55 <phantomcircuit> right?
4822015-11-17T22:29:03 <sipa> yes
4832015-11-17T22:29:40 *** sdaftuar_ has quit IRC
4842015-11-17T22:31:10 <phantomcircuit> sipa, i would be worried about such a system being too complicated
4852015-11-17T22:31:24 <phantomcircuit> the basic protocol is really very simple
4862015-11-17T22:31:45 <phantomcircuit> the only complexity is in scheduling block download really
4872015-11-17T22:32:18 <phantomcircuit> (which i assume is part of your interest in doing that?)
4882015-11-17T22:37:17 *** devpo has joined #bitcoin-core-dev
4892015-11-17T22:37:37 <devpo> hi is anybdy here to help me a little
4902015-11-17T22:37:38 <devpo> ?
4912015-11-17T22:38:56 <devpo> :'(
4922015-11-17T22:39:01 <btcdrak> devpo: what's up?
4932015-11-17T22:41:18 <devpo> i need a php script
4942015-11-17T22:41:31 <btcdrak> better move to #bitcoin
4952015-11-17T22:41:45 <devpo> ok
4962015-11-17T22:42:02 *** devpo has quit IRC
4972015-11-17T22:45:32 <sipa> phantomcircuit: no, my interest is allowing the message handler to run in multiple threads
4982015-11-17T22:46:25 <sipa> (and making things cleaner in the process)
4992015-11-17T22:48:11 <phantomcircuit> sipa, assuming things are holding the correct locks i dont see why we cant do that now
5002015-11-17T22:48:23 *** molly has joined #bitcoin-core-dev
5012015-11-17T22:48:43 <sipa> phantomcircuit: the problem is that it isn't clear what those locks are
5022015-11-17T22:49:13 <sipa> if all handler ran in their own module, with their own state, with their own encapsulated lock, it would be trivially correct
5032015-11-17T22:50:13 <sipa> anyway, no big objection if you want to move code out of processblock into smaller functions, but not everything at once...
5042015-11-17T22:50:40 <phantomcircuit> sipa, it would work today with very if any problems because of cs_vRecvMsg/cs_vSend
5052015-11-17T22:50:57 <phantomcircuit> (assuming for a second that there's nothing in SendMessages looking at vRecvMsg
5062015-11-17T22:51:00 <phantomcircuit> )
5072015-11-17T22:51:12 <sipa> you can't do that now
5082015-11-17T22:51:21 <sipa> all shared handler state is locked by cs_main
5092015-11-17T22:51:33 <phantomcircuit> it doesn't get you much of anything though since virtually everything they're doing that's cpu intensive requires cs_main
5102015-11-17T22:51:43 <sipa> yes
5112015-11-17T22:52:13 <sipa> but many of those cs_main only protect handler state, and not consensus/block related structures
5122015-11-17T22:52:25 <sipa> those need to be replaced by their own handler-spoecific locks
5132015-11-17T22:53:19 <phantomcircuit> what would be an appropriate DoS level for a peer that's sending transactions despite fRelayTxs=false (keeping in mind fRelayTxs will be true for whitelisted peers)
5142015-11-17T22:53:22 *** d_t has joined #bitcoin-core-dev
5152015-11-17T22:54:03 <phantomcircuit> (and it's sending a protocol version high enough to support fRelayTxs in version)
5162015-11-17T22:55:01 <gmaxwell> phantomcircuit: I think we shouldn't DOS now.
5172015-11-17T22:55:31 <gmaxwell> not without studying more what it'll do. I know BTCD in the past wanted the protocol to define sending unsolicited transactions as misbehavior.
5182015-11-17T22:56:54 <tulip> phantomcircuit: a least one piece of SPV wallet software pushes all of their transactions directly without inv'ing them.
5192015-11-17T23:00:34 <gmaxwell> perhaps in the blocks only context is a fine way to introduce that behavior, but I think we shouldn't rush into it.
5202015-11-17T23:00:50 <gmaxwell> We can try asking davec if he has any insight into observed behavior.
5212015-11-17T23:01:20 <gmaxwell> phantomcircuit: One thing you could do is log when a non-whitelisted peer sends a tx message in blocksonly mode.
5222015-11-17T23:11:27 <phantomcircuit> gmaxwell, is it important that GetMainSignals().Inventory(inv.hash); runs regardless of whether we call pfrom->AskFor(inv); or not?
5232015-11-17T23:11:28 *** Guyver2 has quit IRC
5242015-11-17T23:11:41 <phantomcircuit> (if so it needs to be moved)
5252015-11-17T23:12:33 <gmaxwell> phantomcircuit: my belief was no, if we're not relaying we don't care what others have relayed to us.
5262015-11-17T23:13:09 <gmaxwell> (and in particular, we don't want to know: just another way for a peer to use our memory)
5272015-11-17T23:13:24 <phantomcircuit> the version in master actually does call GetMainSignals().Inventory(inv.hash); even if we dont ask anybody for the data
5282015-11-17T23:13:36 <phantomcircuit> yeah that's what i thought
5292015-11-17T23:27:38 <phantomcircuit> im afraid to ask but, GetBoolArg has it's own locks... right?
5302015-11-17T23:27:54 <sipa> phantomcircuit: no, but mapArgs etc are immutable :)
5312015-11-17T23:28:17 <sipa> (apart from initialization which happens single-threadedly)
5322015-11-17T23:28:24 <phantomcircuit> sipa, so... practically safe but technically a threading violation
5332015-11-17T23:28:33 <sipa> phantomcircuit: no, no threading violation
5342015-11-17T23:28:39 <sipa> reading data that can't change is safe
5352015-11-17T23:30:48 <gmaxwell> phantomcircuit: the memory model is that shared reading is okay, but as soon as there is a potential write (even of the same data) then there must be locks that exclude all other readers and writers.
5362015-11-17T23:32:29 <gmaxwell> as the complier is allowed to do things like "oh I can see in this block I will, for sure, overwrite location X.. that means I have exclusive access.. so until then I can spill random registers into it."
5372015-11-17T23:32:38 *** MarcoFalke has quit IRC
5382015-11-17T23:37:40 *** ParadoxSpiral has quit IRC
5392015-11-17T23:41:01 <phantomcircuit> gmaxwell, ah right
5402015-11-17T23:45:20 *** zooko has quit IRC
5412015-11-17T23:46:41 <phantomcircuit> gmaxwell, pr updated
5422015-11-17T23:47:09 <phantomcircuit> only thing im not happy with is that it's not obvious why transactions dont end up in the orphan pool
5432015-11-17T23:50:04 <gmaxwell> phantomcircuit: you should improve your commit messages some; e.g. https://github.com/pstratem/bitcoin/commit/17e6157c1975a4f5e0afa97d632ca0310b227158 should explain that previously if we recieved an unsolicited TX message (which we shouldn't) we'd process it anyways, even in blocks only mode.
5442015-11-17T23:50:21 *** SatoshisCat has quit IRC
5452015-11-17T23:51:32 <davec> Regarding the DoS banning, in general I've never seen much point in allowing protocol level misbehavior. It's TCP, so if the peer isn't following the protocol, they should be disconnected with prejudice.
5462015-11-17T23:52:31 <phantomcircuit> gmaxwell, can do
5472015-11-17T23:52:51 <phantomcircuit> er is there an easy way to edit an arbitrary commit message?
5482015-11-17T23:53:13 <sipa> git commit --amend
5492015-11-17T23:53:18 <davec> git commit --amend for the top commit
5502015-11-17T23:53:26 <davec> if it's further back, git rebase -i
5512015-11-17T23:53:32 <davec> and use 'r'
5522015-11-17T23:56:08 <phantomcircuit> oh rebase e
5532015-11-17T23:56:11 <phantomcircuit> r*
5542015-11-17T23:56:12 <phantomcircuit> that's nice
5552015-11-17T23:56:32 <tulip> davec: in 0.8.x nodes were banned for their own internal misbehaviour which was distinctly non-optimal. it's good to be strict, but there's definitely some risk to it.
5562015-11-17T23:57:53 <sipa> davec, tulip: i don't think the question is whether we should allow protocol misbehaviour (imho, we shouldn't), but what is considered misbehaviour :)