12016-12-06T00:04:41 <dcousens> gmaxwell: https://0bin.net/paste/GS3r5T8zbK5PWDmo#a97Ti6KI4ePCUa6tCJC0RWTDYa3oMxrM4dYJASGuga9 for gdb
22016-12-06T00:05:16 <dcousens> let me know if you need anything else
32016-12-06T00:05:38 <dcousens> I was considering compiling it without clang, just to see
42016-12-06T00:08:35 *** droark has quit IRC
52016-12-06T00:15:48 <dcousens> I forgot how much slower gcc builds :S
62016-12-06T00:16:33 <gmaxwell> dcousens: build with O1.
72016-12-06T00:16:57 <gmaxwell> Clang default optimization levels are less effective than GCC's it compiles faster but for most code produces slower binaries.
82016-12-06T00:17:23 <dcousens> gmaxwell: did you need me to build with O1 for debugging? Or just future reference
92016-12-06T00:17:25 <sipa> i sometimes configure with -O1 to compile faster
102016-12-06T00:17:34 <gmaxwell> no, just if the compile times bother you-- :)
112016-12-06T00:17:39 <sipa> you could compile with "-Og -ggdb3"
122016-12-06T00:18:26 <dcousens> brb, making breakfast ha
132016-12-06T00:18:42 <gmaxwell> I see things waiting on cs_main in that backtrace... supports the locking theory but I don't see where.
142016-12-06T00:26:42 <cfields> BlueMatt: around?
152016-12-06T00:27:33 <cfields> BlueMatt: something I'm hacking on just started coming together nicely, but would like to make sure we're on the same page before I get in too deep
162016-12-06T00:27:42 <dcousens> behaviour unchanged with gcc
172016-12-06T00:27:58 <dcousens> wait no
182016-12-06T00:28:02 <dcousens> no, RPC is responding now
192016-12-06T00:28:42 <bitcoin-git> [bitcoin] gmaxwell opened pull request #9286: Time-based expiration in LimitOrphanTxSize only when map is full. (master...defer_orphan_exp) https://github.com/bitcoin/bitcoin/pull/9286
202016-12-06T00:28:54 <dcousens> though, only on requests not waiting CS_MAIN :P
212016-12-06T00:29:00 <gmaxwell> hah
222016-12-06T00:29:07 <gmaxwell> dcousens: do you know how to compile with debug lockorder?
232016-12-06T00:29:20 <cfields> dcousens: any chance it's sitting in an fread?
242016-12-06T00:29:42 <gmaxwell> My WAG is that something about your configuration has revealed a lock inversion introduced -- likely by one of Matt's recent race condition fixes -- but other people aren't seeing it.
252016-12-06T00:29:46 <dcousens> gmaxwell: no, a few commands away from doing it
262016-12-06T00:29:58 <cfields> dcousens: if so, i get that on one of my machines when i use a certain combo of clang/libc and hardening enabled
272016-12-06T00:30:06 <dcousens> gmaxwell: did you want me to start culling params?
282016-12-06T00:30:22 <gmaxwell> dcousens: no, I'd like you to build with the lockorder debugging enabled.
292016-12-06T00:30:34 <gmaxwell> if it is an inversion like I'm guessing it should immediately tell us exactly where.
302016-12-06T00:31:13 <cfields> dcousens: oh, nm, i see the bt
312016-12-06T00:31:40 <dcousens> gmaxwell: what do I need to do to enable it?
322016-12-06T00:32:35 <gmaxwell> dcousens: run ./configure CXXFLAGS="-DDEBUG_LOCKORDER -g" then build like normal
332016-12-06T00:33:00 <dcousens> OK, interestingly this doesn't happen on a fresh .bitcoin (tried locally)
342016-12-06T00:33:09 <dcousens> running the configure now
352016-12-06T00:35:13 <dcousens> scratch that, still happens on a fresh .bitcoin :) (yay)
362016-12-06T00:38:59 <BlueMatt> cfields: yea, I can take a look
372016-12-06T00:39:32 <cfields> BlueMatt: well, not much to show yet, just an idea...
382016-12-06T00:39:43 <BlueMatt> sure
392016-12-06T00:40:03 <cfields> BlueMatt: using https://github.com/theuni/bitcoin/commit/3f598dbe7100c7c6c7bfb7e10210585327ed9d31 as a starting point
402016-12-06T00:40:36 <BlueMatt> cfields: didnt you already show me this?
412016-12-06T00:40:38 <cfields> with that, CConnman holds an instance of the message processor interface. Taking that a step further...
422016-12-06T00:41:01 <cfields> BlueMatt: yea, just giving the context
432016-12-06T00:41:05 <BlueMatt> ok
442016-12-06T00:41:20 <dcousens> while its compiling, i've binary searched the arguments to when it doesn't happen...
452016-12-06T00:41:28 <dcousens> and, I have the argument that causes it :P
462016-12-06T00:41:44 <dcousens> -minrelaytxfee=0.00000001
472016-12-06T00:41:58 <cfields> BlueMatt: er, jump to pm?
482016-12-06T00:42:07 <dcousens> remove that argf
492016-12-06T00:42:10 <dcousens> everything works
502016-12-06T00:42:25 <BlueMatt> cfields: ok
512016-12-06T00:42:29 <dcousens> ping gmaxwell
522016-12-06T00:42:53 <gmaxwell> dcousens: sweet
532016-12-06T00:43:09 <dcousens> and the lock order is compiled
542016-12-06T00:43:15 <dcousens> want a full bt?
552016-12-06T00:43:26 <gmaxwell> just run it.. is it spitting out any errors?
562016-12-06T00:45:13 <dcousens> gmaxwell: no errors yet
572016-12-06T00:45:42 <dcousens> nothing seen, would it be a hard crash or a debug log error?
582016-12-06T00:45:59 <dcousens> in any case, neither
592016-12-06T00:46:44 <gmaxwell> It logs with the normal logging process.
602016-12-06T00:47:27 <dcousens> gmaxwell: for what its worth, you could probably check it out with just the above arg, a fresh compile off master causes that deadlock with that arg
612016-12-06T00:47:50 <gmaxwell> yep. will be doing as soon as it compiles here.
622016-12-06T00:47:53 <dcousens> like, fresh compile, fresh .bitcoin, deadlocks on -minrelaytxfee=0.00000001
632016-12-06T00:48:07 <gmaxwell> (takes like 15 minutes to comile bitcoin on my laptop)
642016-12-06T00:48:23 <dcousens> haha, the joy
652016-12-06T00:48:51 *** alpalp has joined #bitcoin-core-dev
662016-12-06T00:48:51 *** alpalp has joined #bitcoin-core-dev
672016-12-06T00:49:07 <gmaxwell> (and all my other systems are busy with longer running expirements at the moment)
682016-12-06T00:57:48 *** alpalp has quit IRC
692016-12-06T01:01:19 *** alpalp has joined #bitcoin-core-dev
702016-12-06T01:01:19 *** alpalp has joined #bitcoin-core-dev
712016-12-06T01:06:20 <gmaxwell> okay longer than 15 minutes.
722016-12-06T01:09:06 <gmaxwell> dcousens: your misbehavior doesn't reproduce for me. :(
732016-12-06T01:09:20 <dcousens> gmaxwell: :( - really?
742016-12-06T01:09:36 <dcousens> Dumb question
752016-12-06T01:10:14 <gmaxwell> dcousens: well it could require interaction with some other option, or some library/compiler version.
762016-12-06T01:10:18 <dcousens> It definitely fixes my issue, I mean, all my nodes are back up haha
772016-12-06T01:10:38 <dcousens> but, I'll retry local repro
782016-12-06T01:14:14 <dcousens> rm -rf ~/.bitcoin, git clean -xdf, ./autogen.sh, ./configure --disable-bench --disable-tests --disable-wallet --without-miniupnpc, make -j4, ./src/bitcoind -minrelaytxfee=0.00000001 ---- that was my previous steps, I'm retrying them now
792016-12-06T01:16:18 <dcousens> Otherwise, I guess I could maybe grab a list of some local package versions for you if you want, or try it out on a fresh ubuntu install
802016-12-06T01:16:39 *** TomMc has quit IRC
812016-12-06T01:16:52 <dcousens> but, I probably don't have that time atm :(
822016-12-06T01:16:58 <dcousens> (for the fresh install etc)
832016-12-06T01:18:02 *** Chris_Stewart_5 has joined #bitcoin-core-dev
842016-12-06T01:18:53 <dcousens> gmaxwell: reproduced with the above commands
852016-12-06T01:19:34 <gmaxwell> hm. disable wallet. I could try that.
862016-12-06T01:19:43 <dcousens> commit 7d5d449
872016-12-06T01:21:36 *** Squidicuz has joined #bitcoin-core-dev
882016-12-06T01:22:19 <dcousens> gmaxwell: interesting
892016-12-06T01:22:29 <dcousens> -minrelaytxfee=0.1 does not deadlock
902016-12-06T01:22:49 <dcousens> -minrelaytxfee=0.00000001 does
912016-12-06T01:23:44 <dcousens> -minrelaytxfee=0.00000002 does not deadlock
922016-12-06T01:23:48 <dcousens> lol
932016-12-06T01:24:05 <dcousens> for god sake, I reckon the binary just has it in for my configuration
942016-12-06T01:25:53 <dcousens> rounding error I suppose?
952016-12-06T01:26:07 <dcousens> Want me to bisect back to the source commit?
962016-12-06T01:29:52 <gmaxwell> dcousens: would be useful, since only you can break it right now.
972016-12-06T01:30:40 <dcousens> gmaxwell: must be platform related, since the hardware is completely different (EC2 instance, RPi, and my Laptop)
982016-12-06T01:30:45 <dcousens> but they all run Arch
992016-12-06T01:30:47 <dcousens> :P
1002016-12-06T01:31:39 <dcousens> and yeah, only in the last... 1.5 weeks of dev, if that
1012016-12-06T01:32:41 *** Ylbam has quit IRC
1022016-12-06T01:36:30 *** TomMc has joined #bitcoin-core-dev
1032016-12-06T01:42:07 <gmaxwell> dcousens: disable wallet didn't break it for me.
1042016-12-06T01:42:36 <dcousens> gmaxwell: awesome, the fact this happens with me for both gcc+clang makes me think it must be some other dependency?
1052016-12-06T01:42:45 <dcousens> And cross-hardware
1062016-12-06T01:53:07 <dcousens> man, net_processing touches so many files haha
1072016-12-06T01:53:54 <dcousens> none the less, found the source of the bug so far, seems to be that static FeeFilterRounder
1082016-12-06T01:54:06 <dcousens> if I change that to a constant fee != 1, it doesnt deadlock
1092016-12-06T01:54:22 <dcousens> the investigation continues...
1102016-12-06T01:55:00 <sipa> dcousens: try with https://github.com/bitcoin/bitcoin/pull/9268/files
1112016-12-06T01:56:51 <dcousens> sipa: success
1122016-12-06T01:56:53 <dcousens> that fixes the issue
1132016-12-06T01:57:40 <sipa> dcousens: can you briefly comment on the PR?
1142016-12-06T01:57:45 <dcousens> sipa: did
1152016-12-06T01:57:47 <sipa> ah, i'm too late
1162016-12-06T01:58:45 <dcousens> gmaxwell: welp, should I investigate further into why it deadlocks for only me or?
1172016-12-06T02:00:07 *** abpa has quit IRC
1182016-12-06T02:00:49 <dcousens> I feel like this is probably a red-herring
1192016-12-06T02:01:25 <dcousens> Because nothing in that very small code block aludes to a deadlock issue, from what I can see
1202016-12-06T02:05:52 *** davec has quit IRC
1212016-12-06T02:10:18 <morcos> sipa: do you have any suspicion of what's going on? that doesn't make much sense
1222016-12-06T02:10:37 <dcousens> morcos: not at this point, but, it repros quite solidly
1232016-12-06T02:10:49 <dcousens> -- on my machines, ha
1242016-12-06T02:13:39 <morcos> that first stack trace looks kind of crazy
1252016-12-06T02:14:56 <gmaxwell> I see the bug.
1262016-12-06T02:15:17 <dcousens> gmaxwell: oh?
1272016-12-06T02:15:20 *** wasi0 has joined #bitcoin-core-dev
1282016-12-06T02:15:27 <gmaxwell> well one sec. might have spoke too soon.
1292016-12-06T02:15:38 <sipa> morcos: my guess would be something related to initialization order
1302016-12-06T02:15:52 <gmaxwell> yea, okay not what I thought.
1312016-12-06T02:15:55 <gmaxwell> FeeFilterRounder::FeeFilterRounder(const CFeeRate& minIncrementalFee)
1322016-12-06T02:15:55 <gmaxwell> { CAmount minFeeLimit = minIncrementalFee.GetFeePerK() / 2;
1332016-12-06T02:16:21 <gmaxwell> so his rate will go to zero there, and then I expect to find a loop immediately after it with that rate as an increment.
1342016-12-06T02:16:24 <gmaxwell> but it wasn't so. :)
1352016-12-06T02:16:43 <gmaxwell> oh wait, there it goes.
1362016-12-06T02:16:46 <dcousens> gmaxwell: still sounds plausible
1372016-12-06T02:16:51 *** wasi0 has quit IRC
1382016-12-06T02:17:00 <gmaxwell> for (double bucketBoundary = minFeeLimit; bucketBoundary <= MAX_FEERATE; bucketBoundary *= FEE_SPACING) {
1392016-12-06T02:17:08 *** wasi0 has joined #bitcoin-core-dev
1402016-12-06T02:17:24 <gmaxwell> so there you go bucketBoundary = 0 ; bucketBoundary <= MAX_FEERATE; bucketBoundary *= FEE_SPACING.. infinite loop.
1412016-12-06T02:17:39 *** wasi has quit IRC
1422016-12-06T02:17:58 <gmaxwell> though this doesn't explain why he's seeing it and I'm not.
1432016-12-06T02:18:00 <morcos> oops
1442016-12-06T02:18:25 <morcos> hmm
1452016-12-06T02:18:33 <dcousens> gmaxwell: and now the fun starts haha
1462016-12-06T02:18:35 <morcos> that's backwrds then
1472016-12-06T02:18:57 <gmaxwell> oh perhaps some artifact of mempool saving, which I wasn't clearing?
1482016-12-06T02:19:09 <morcos> oh nm
1492016-12-06T02:19:11 <gmaxwell> hm. no we don't save the feerates.
1502016-12-06T02:19:15 <dcousens> gmaxwell: mine was loading mempools from previous builds
1512016-12-06T02:19:18 <dcousens> that wouldn't be it
1522016-12-06T02:19:21 <morcos> ok so right now it interprets the command line argument and 9268 makes it not
1532016-12-06T02:19:24 <morcos> ok
1542016-12-06T02:19:34 <morcos> your explanation makes sense
1552016-12-06T02:19:47 <gmaxwell> right and apparently it used to not. So the unexplained part is why is it not happening to everyone.
1562016-12-06T02:20:11 *** wasi0 has quit IRC
1572016-12-06T02:20:16 <gmaxwell> dcousens: can you insert some instrumentation in that function to log the minFeeLimit before the loop and bucketBoundary inside the loop?
1582016-12-06T02:20:23 <gmaxwell> just to confirm its really infinitely looping there?
1592016-12-06T02:20:31 <dcousens> gmaxwell: sure
1602016-12-06T02:21:21 <dcousens> 304 or 490? 490 right
1612016-12-06T02:21:27 <dcousens> (line number0
1622016-12-06T02:21:28 <dcousens> )
1632016-12-06T02:21:50 <morcos> yeah 490
1642016-12-06T02:22:38 <morcos> gmaxwell: yeah wait, why would it get rounded to 0
1652016-12-06T02:22:53 <morcos> oh, b/c its an integer number of satoshis?
1662016-12-06T02:23:17 <gmaxwell> 1/2 = 0 yes. CAmounts are integers.
1672016-12-06T02:26:05 <dcousens> gmaxwell: thats the deadlock
1682016-12-06T02:26:37 <dcousens> bucketBou;w
1692016-12-06T02:26:43 <dcousens> ah, ignore that
1702016-12-06T02:27:03 <dcousens> anyway, yeah, that is the deadlock*, minFeeLimit == 0, bucketBoundary = 0.0000
1712016-12-06T02:27:22 <gmaxwell> dcousens: great, but now why you and not anyone else.
1722016-12-06T02:27:36 * gmaxwell checks my config.
1732016-12-06T02:28:43 <dcousens> gmaxwell: let me know if you need me to check anything, have to return to a different task but happy to run anything
1742016-12-06T02:29:00 <gmaxwell> dcousens: thanks so much for all your help. I'm sure we've got it from here.
1752016-12-06T02:30:59 <morcos> dcousens: yes, thanks and sorry
1762016-12-06T02:31:16 <dcousens> morcos: all good, no need to be sorry, I run master for this exact reason :)
1772016-12-06T02:36:39 *** wasi has joined #bitcoin-core-dev
1782016-12-06T02:40:24 <morcos> gmaxwell: compiler isn't allowed to optimize that out is it?
1792016-12-06T02:41:13 <morcos> nm, stupid question, it can't get out of the loop...
1802016-12-06T02:41:17 <gmaxwell> morcos: if the compiler is able to prove a loop is infinite it can.
1812016-12-06T02:41:28 <gmaxwell> But it can't prove that loop is infinite, it depends on the argument.
1822016-12-06T02:41:42 <gmaxwell> but yea, actually you're probably right.
1832016-12-06T02:42:10 <gmaxwell> The loop uses floating point, god knows there is probably some way to rewrite that loop which is accurate except for 0.
1842016-12-06T02:42:41 <gmaxwell> but I've reproduced it.
1852016-12-06T02:42:45 <gmaxwell> so theres that.
1862016-12-06T02:42:57 *** TomMc has quit IRC
1872016-12-06T02:42:57 <morcos> oh good
1882016-12-06T02:43:11 <gmaxwell> I yanked everything out of my configuration except txindex=1, and dcousens's setting.
1892016-12-06T02:43:49 <morcos> oh , so you sometimes don't get it still... that is super weird
1902016-12-06T02:43:58 <gmaxwell> sipa: thoughts on merging https://github.com/bitcoin/bitcoin/pull/9268 to unbust master while the actual algorithim here gets fixed?
1912016-12-06T02:44:13 <gmaxwell> morcos: I think it might be a product of which peers I connect to.
1922016-12-06T02:44:21 <gmaxwell> e.g. doesn't get triggered until I send a fee filter.
1932016-12-06T02:44:38 <morcos> should we fix it by doing anything other than max(1,)
1942016-12-06T02:46:07 <gmaxwell> I don't see any reason to do anything but that. But thats not my code. :) I'd have to understand it to answer!
1952016-12-06T02:47:02 <morcos> yeah , i mean we could do something else entirely, its probably not the cleanest way to accomplish the goal, but i don't know that its worth doing anyting other than that, i'll pr
1962016-12-06T02:48:37 <gmaxwell> we could keep the behavior "this is what you get for setting you relayfee to 10 nanobitcoin and spamming other folks" :P
1972016-12-06T02:50:23 <morcos> Its an optimization itself, this is your nodes expected performance profile
1982016-12-06T02:52:18 <gmaxwell> "We've decided to save you the bandwidth and just give you a simulation of the expected cpu usage."
1992016-12-06T02:53:11 <bitcoin-git> [bitcoin] morcos opened pull request #9288: Fix a bug if the min fee is 0 for FeeFilterRounder (master...fixFFRbug) https://github.com/bitcoin/bitcoin/pull/9288
2002016-12-06T02:54:03 <morcos> How snarky would it be if that was just BlueMatt's way of pointing out the underlying bug
2012016-12-06T02:54:38 <gmaxwell> well true, that fact that your node is down makes you pretty distinguishable from other users.
2022016-12-06T02:55:01 *** alpalp has quit IRC
2032016-12-06T02:57:21 <gmaxwell> I guess its good? news that we already had a PR open that would have covered this bug up...
2042016-12-06T02:57:31 * gmaxwell continues to curse configuration options for making software untestable.
2052016-12-06T03:02:08 <bitcoin-git> [bitcoin] theuni closed pull request #8631: Nuke boost::thread and boost::thread_group (master...no-interrupt-threads) https://github.com/bitcoin/bitcoin/pull/8631
2062016-12-06T03:02:27 <bitcoin-git> [bitcoin] theuni opened pull request #9289: net: drop boost::thread_group (master...connman-threads) https://github.com/bitcoin/bitcoin/pull/9289
2072016-12-06T03:05:23 <dcousens> gmaxwell: haha
2082016-12-06T03:05:50 <dcousens> good to know you repro'd it :)
2092016-12-06T03:05:53 *** alpalp has joined #bitcoin-core-dev
2102016-12-06T03:06:09 <gmaxwell> really since the introduction of fee filter, low minrelay fees aren't so anti-social anymore at least.
2112016-12-06T03:07:27 <dcousens> gmaxwell: IMHO, all I really would want as policy is IsStandard, and a maximum in-memory size, sorted by fee descending (accounting for CPFP etc)
2122016-12-06T03:08:20 <dcousens> hence, the setting of 1 haha
2132016-12-06T03:10:32 <dcousens> gmaxwell: I wouldn't mind seeing double spends in the mempool too, but, I guess that would annoy too many people
2142016-12-06T03:11:24 <dcousens> and probably opens too many DOS vectors
2152016-12-06T03:14:47 <gmaxwell> well double spends in the mempool would break varrious assumptions in the code. :)
2162016-12-06T03:15:03 <gmaxwell> dcousens: thats not unreasonable since the advent of feefilter.
2172016-12-06T03:16:27 <dcousens> morcos: line 304 could probably be bounded too, just incase MIN_FEERATE was changed
2182016-12-06T03:41:04 *** davec has joined #bitcoin-core-dev
2192016-12-06T03:41:39 *** abpa has joined #bitcoin-core-dev
2202016-12-06T03:53:16 *** justanotheruser has joined #bitcoin-core-dev
2212016-12-06T03:54:53 *** justan0theruser has quit IRC
2222016-12-06T04:15:42 *** Giszmo has quit IRC
2232016-12-06T05:15:16 *** aalex has quit IRC
2242016-12-06T05:15:35 *** aalex has joined #bitcoin-core-dev
2252016-12-06T05:19:06 *** owowo has quit IRC
2262016-12-06T05:20:05 *** moli has quit IRC
2272016-12-06T05:20:05 *** molz has joined #bitcoin-core-dev
2282016-12-06T05:21:52 *** owowo has joined #bitcoin-core-dev
2292016-12-06T05:21:58 *** Chris_Stewart_5 has quit IRC
2302016-12-06T05:26:51 *** owowo has quit IRC
2312016-12-06T05:27:53 *** owowo has joined #bitcoin-core-dev
2322016-12-06T05:29:35 *** dcousens has quit IRC
2332016-12-06T05:30:50 *** dcousens has joined #bitcoin-core-dev
2342016-12-06T05:42:30 *** larrysalibra has joined #bitcoin-core-dev
2352016-12-06T06:02:20 *** jtimon has quit IRC
2362016-12-06T06:19:35 *** larrysalibra has quit IRC
2372016-12-06T06:21:10 <wumpus> so did anyone compile bitcoin for risc-v yet? :)
2382016-12-06T06:23:08 *** protomar has joined #bitcoin-core-dev
2392016-12-06T06:45:16 *** Alopex has quit IRC
2402016-12-06T06:46:22 *** Alopex has joined #bitcoin-core-dev
2412016-12-06T06:49:57 <bitcoin-git> [bitcoin] gmaxwell opened pull request #9290: Make RelayWalletTransaction attempt to AcceptToMemoryPool. (master...resend_retries_mempool) https://github.com/bitcoin/bitcoin/pull/9290
2422016-12-06T07:08:02 *** Alopex has quit IRC
2432016-12-06T07:08:44 <bitcoin-git> [bitcoin] sipa opened pull request #9291: Remove mapOrphanTransactionsByPrev from DoS_tests (master...reallyoneorphan) https://github.com/bitcoin/bitcoin/pull/9291
2442016-12-06T07:09:07 *** Alopex has joined #bitcoin-core-dev
2452016-12-06T07:10:36 <warren> wumpus: there's machines available for it?
2462016-12-06T07:11:12 <gmaxwell> Does anyone recall why we decided to not backport #9053 ? I was just thinking about maybe it should be backported, then saw it was tagged and untagged.
2472016-12-06T07:11:14 <gribble> https://github.com/bitcoin/bitcoin/issues/9053 | IBD using chainwork instead of height and not using header timestamps by gmaxwell · Pull Request #9053 · bitcoin/bitcoin · GitHub
2482016-12-06T07:11:24 <wumpus> warren: I just ordered a hifive board, but that's arduino-class, nothing that can handle bitcoind IIRC. There's a toolchain and qemu (also linux user) though so it'd be possible to compile and run it
2492016-12-06T07:11:45 <sipa> gmaxwell: i think we considered it for 0.13.1, but decided it wasn't important enough for that?
2502016-12-06T07:11:46 <wumpus> warren: maybe if you use a RiscV core on one of the high-end FPGAs
2512016-12-06T07:12:35 <gmaxwell> TD-Linux: was telling me recently he planned on having an actual RiscV chip soon... but not one that would be anywhere near powerful enough to run bitcoin core. :P
2522016-12-06T07:13:14 <wumpus> gmaxwell: yeah, if there's no discussion on the issue it probably means it was discussed in a meeting
2532016-12-06T07:13:42 <wumpus> nov 3's meeting I'd guess
2542016-12-06T07:14:14 <wumpus> gmaxwell: yep this one probably: https://www.sifive.com/products/hifive1/ ordered one as well
2552016-12-06T07:14:15 <jonasschnelli> <*highlight> [18:40:09] <morcos:#bitcoin-core-dev> jonasschnelli: you around?
2562016-12-06T07:14:16 <jonasschnelli> Now. You?
2572016-12-06T07:14:24 <gmaxwell> 12:57 <@wumpus> I think personally I'd prefer to keep it for 0.14, so the new rule/logic can prove itself a while
2582016-12-06T07:14:29 <jonasschnelli> ref: [18:40:09] <morcos:#bitcoin-core-dev> jonasschnelli: you around?
2592016-12-06T07:14:31 <gmaxwell> okay, yep. sorry, hamster brain here.
2602016-12-06T07:14:55 <wumpus> gmaxwell: well it has been a while in master now ...
2612016-12-06T07:15:09 <wumpus> gmaxwell: I think that was just last-minute before a release
2622016-12-06T07:16:34 <jonasschnelli> gmaxwell: I have a solution for the HD chain-split (95% complete). I think it could be backwards compatible. Means, if a 0.13 wallet will be loaded, only external (m/0'/0'/k) will be generated. Creating a new wallet will respect the 0'/|1'/ switch.
2632016-12-06T07:16:35 <jonasschnelli> https://github.com/bitcoin/bitcoin/compare/master...jonasschnelli:2016/12/hd_split?expand=1
2642016-12-06T07:16:38 <gmaxwell> I don't have a really strong opinion, but I did just have a 0.13.1 testnet node act goofy today in a way that change fixes, so it came back to mind.
2652016-12-06T07:17:11 <jonasschnelli> Loading the 0.14 wallet in 0.13 will result in using only the external chain.. or we disable that.
2662016-12-06T07:17:16 <jonasschnelli> (probably better to disable)
2672016-12-06T07:17:19 <gmaxwell> jonasschnelli: but if you go back to an older version do funds silently disappear? or .. I guess we'll just not be backwards compatible?
2682016-12-06T07:17:33 <wumpus> I do think we should only backport critical things, not things that 'would be nice' but are risky. Also if it *only* affected testnet it'd be a no-brainer.
2692016-12-06T07:17:47 <jonasschnelli> Once the keypool is filled, the keypath doesn't matter.
2702016-12-06T07:18:30 <jonasschnelli> I think we should probably disable loading 0.14 (hd splitI) wallets in 0.13. Becuase once you load them in 0.13, you take the risk of having chain outputs in the external chain.
2712016-12-06T07:18:33 <jonasschnelli> This can be confusing.
2722016-12-06T07:18:51 <wumpus> this also has an effect on mainnet, it'd be a pain if it introduced some regression, though that chance is lower than at Nov 3 which was two days after it was openened :)
2732016-12-06T07:19:05 <gmaxwell> wumpus: well it fixes bad behavior that is possible on mainnet too just really unlikely and not something we're currently observing.
2742016-12-06T07:19:53 <wumpus> it's been merged in master for more than a month and no one at least observed any wacky behavior due to it, so I'd trust it for backport now
2752016-12-06T07:22:07 <sipa> jonasschnelli: it's 2:22 am for morcos
2762016-12-06T07:22:20 <bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/7d5d44969b4a...ed8d693c71b0
2772016-12-06T07:22:21 <bitcoin-git> bitcoin/master bc7ff8d Gregory Sanders: Add option to return non-segwit serialization via rpc
2782016-12-06T07:22:21 <bitcoin-git> bitcoin/master 412bab2 instagibbs: Adapt ZMQ/rest serialization to take rpcserialversion arg
2792016-12-06T07:22:22 <bitcoin-git> bitcoin/master ed8d693 Wladimir J. van der Laan: Merge #9194: Add option to return non-segwit serialization via rpc...
2802016-12-06T07:22:34 <bitcoin-git> [bitcoin] laanwj closed pull request #9194: Add option to return non-segwit serialization via rpc (master...nonswserialrpc) https://github.com/bitcoin/bitcoin/pull/9194
2812016-12-06T07:22:35 <wumpus> sipa: ugh, your comment on #9194 was just too late
2822016-12-06T07:22:37 <gribble> https://github.com/bitcoin/bitcoin/issues/9194 | Add option to return non-segwit serialization via rpc by instagibbs · Pull Request #9194 · bitcoin/bitcoin · GitHub
2832016-12-06T07:22:55 <sipa> wumpus: can easily be fixed later
2842016-12-06T07:23:03 <wumpus> had already hit enter then I saw it appear at the same time :p
2852016-12-06T07:23:27 <wumpus> yes luckily just a help message
2862016-12-06T07:23:44 *** ClockCat has quit IRC
2872016-12-06T07:23:55 <sipa> well the comment is more about showing an error when an unknown value is passed
2882016-12-06T07:24:21 <sipa> i'll PR my suggested change
2892016-12-06T07:25:17 <jonasschnelli> sipa: Oh. Yes.
2902016-12-06T07:25:41 *** owowo has quit IRC
2912016-12-06T07:27:55 <jonasschnelli> For the chain-split, what would be preferable, a keypool=100 results in 50 internal and 50 external keys or in 100/100?
2922016-12-06T07:28:08 <bitcoin-git> [bitcoin] sipa opened pull request #9292: Complain when unknown rpcserialversion is specified (master...nofutureserial) https://github.com/bitcoin/bitcoin/pull/9292
2932016-12-06T07:28:22 <sipa> jonasschnelli: internal keys need way fewer
2942016-12-06T07:28:22 <jonasschnelli> sipa: I guess its 2.30am you as well?
2952016-12-06T07:28:37 <sipa> jonasschnelli: no, i'm on the west coast, 23:28 here
2962016-12-06T07:28:39 <gmaxwell> no, the US is a big place.
2972016-12-06T07:28:49 <jonasschnelli> Ah. Right. He's NY.
2982016-12-06T07:28:59 <gmaxwell> US timezones span 7 hours IIRC.
2992016-12-06T07:29:00 *** owowo has joined #bitcoin-core-dev
3002016-12-06T07:29:43 <sipa> there's a tiny part of the US on the other side of the international date line
3012016-12-06T07:30:11 <jonasschnelli> sipa: But in general, should the keypool=<value> results in <value>=<internal>+<external> or <value>=<internalamout>, <value>=<externalamount>?
3022016-12-06T07:30:20 *** murchandamus has quit IRC
3032016-12-06T07:30:41 <gmaxwell> yea, I wasn't counting guam.
3042016-12-06T07:30:54 <gmaxwell> in that case they span like 20 hours. :P
3052016-12-06T07:31:47 <sipa> gmaxwell: you don't need to count territories. there is a tiny part of alaska across the date line
3062016-12-06T07:32:02 *** murchandamus has joined #bitcoin-core-dev
3072016-12-06T07:32:59 <sipa> ah, but it still uses western time. ignore me
3082016-12-06T07:51:07 <sipa> https://github.com/bitcoin/bitcoin/pulse/monthly
3092016-12-06T07:51:10 <sipa> that's pretty cool
3102016-12-06T07:51:20 <sipa> Excluding merges, 33 authors have pushed 160 commits to master and 175 commits to all branches. On master, 288 files have changed and there have been 13,187 additions and 10,679 deletions.
3112016-12-06T07:53:04 *** BashCo has quit IRC
3122016-12-06T07:53:11 <wumpus> nice!
3132016-12-06T07:53:30 <wumpus> BlueMatt is clearly winning this month :)
3142016-12-06T07:54:43 <sipa> also the last week and the last 3 days
3152016-12-06T07:55:20 <sipa> but over a month... 112 PRs merged, and 50 PRs opened
3162016-12-06T07:55:23 <sipa> that's pretty good
3172016-12-06T08:01:40 *** abpa has quit IRC
3182016-12-06T08:07:58 *** Ylbam has joined #bitcoin-core-dev
3192016-12-06T08:13:54 *** BashCo has joined #bitcoin-core-dev
3202016-12-06T08:19:46 *** warren has quit IRC
3212016-12-06T08:19:46 *** adam3us_ has quit IRC
3222016-12-06T08:22:13 *** adam3us has joined #bitcoin-core-dev
3232016-12-06T08:23:08 *** warren has joined #bitcoin-core-dev
3242016-12-06T08:44:25 *** Guyver2 has joined #bitcoin-core-dev
3252016-12-06T08:54:22 *** NielsvG has joined #bitcoin-core-dev
3262016-12-06T08:57:53 <bitcoin-git> [bitcoin] gmaxwell opened pull request #9293: [0.13 Backport] IBD using chainwork instead of height and not using header timestamp (#9053) (0.13...9053_backport) https://github.com/bitcoin/bitcoin/pull/9293
3272016-12-06T08:58:34 *** larrysalibra has joined #bitcoin-core-dev
3282016-12-06T08:58:50 <gmaxwell> based on the prior discussion ^, if we decide we don't want to-- I won't be offened. But I figured I wouldn't ask about it then fail to do the work.
3292016-12-06T08:59:45 <gmaxwell> (it wasn't any work, however)
3302016-12-06T08:59:54 *** larrysalibra has quit IRC
3312016-12-06T09:00:04 <wumpus> great
3322016-12-06T09:13:43 *** jannes has joined #bitcoin-core-dev
3332016-12-06T09:33:08 *** _biO_ has quit IRC
3342016-12-06T09:50:40 *** MarcoFalke has joined #bitcoin-core-dev
3352016-12-06T10:06:30 *** rafalcpp has quit IRC
3362016-12-06T10:08:31 <bitcoin-git> [bitcoin] jonasschnelli opened pull request #9294: Use internal HD chain for change outputs (hd split) (master...2016/12/hd_split) https://github.com/bitcoin/bitcoin/pull/9294
3372016-12-06T10:13:59 <bitcoin-git> [bitcoin] jonasschnelli closed pull request #9238: Ignore BIP35 mempool command by default (master...2016/11/dis_mempool) https://github.com/bitcoin/bitcoin/pull/9238
3382016-12-06T10:15:17 <bitcoin-git> [bitcoin] jonasschnelli closed pull request #8182: [Qt] Add simple opt-in-RBF support (master...2016/04/qt_rbf_set_new) https://github.com/bitcoin/bitcoin/pull/8182
3392016-12-06T10:15:49 <bitcoin-git> [bitcoin] jonasschnelli closed pull request #7685: Add bloom filter usage statistics (master...2016/03/bf_stats) https://github.com/bitcoin/bitcoin/pull/7685
3402016-12-06T10:16:44 *** molz has quit IRC
3412016-12-06T10:21:07 *** _biO_ has joined #bitcoin-core-dev
3422016-12-06T10:28:27 *** CubicEarth has joined #bitcoin-core-dev
3432016-12-06T10:37:16 *** AaronvanW has quit IRC
3442016-12-06T10:41:10 *** AaronvanW has joined #bitcoin-core-dev
3452016-12-06T10:41:11 *** AaronvanW has quit IRC
3462016-12-06T10:41:11 *** AaronvanW has joined #bitcoin-core-dev
3472016-12-06T10:48:07 <MarcoFalke> wumpus: Can you add instagibbs to our team?
3482016-12-06T10:48:27 <MarcoFalke> https://github.com/orgs/bitcoin/people
3492016-12-06T10:48:55 <timothy> MarcoFalke: core developers?
3502016-12-06T10:49:21 <MarcoFalke> Just as a member, so I can select pull requests by name :)
3512016-12-06T10:50:15 <MarcoFalke> I think it is some read-only group
3522016-12-06T10:50:22 <wumpus> sure
3532016-12-06T10:53:45 <jonasschnelli> wumpus: Yes. The fundrawtx exception is strange. Currently checking...
3542016-12-06T10:54:31 <wumpus> MarcoFalke: I've invited him to bitcoin and bitcoin-core (as read-only member), team page should still be updated
3552016-12-06T10:56:07 <MarcoFalke> ok, thx. Often I know who opened a pull request and to check it I use the dropdown on the pull request page.
3562016-12-06T10:58:19 <jonasschnelli> Interesting, can you call fundrawtx on a locked wallet?!
3572016-12-06T10:58:55 <wumpus> jonasschnelli: yes you should be able to, only signing requires unlocking
3582016-12-06T10:59:25 <wumpus> funding requires the wallet's utxo set which is not encrypted
3592016-12-06T10:59:43 <jonasschnelli> Not only, also retrieving a change output key
3602016-12-06T10:59:51 <jonasschnelli> Keypool can be empty, needs unlocking.
3612016-12-06T11:00:29 <wumpus> oh sure - if the keypool is empty it needs new keys, and for that the wallet needs to be unlocked
3622016-12-06T11:01:17 <jonasschnelli> But there is nothing that prevents that. Only an assertion
3632016-12-06T11:01:23 <wumpus> then can you either generate new keys and re-lock the wallet or call the RPC with wallet unlocked - there's more RPCs which work like that
3642016-12-06T11:01:41 <wumpus> it should just raise an exception in that case
3652016-12-06T11:02:01 <wumpus> which IIRC trying to get a new key with empty keypool (and locked wallet) does
3662016-12-06T11:02:23 <jonasschnelli> https://github.com/bitcoin/bitcoin/blob/master/src/wallet/wallet.cpp#L2392
3672016-12-06T11:02:34 <jonasschnelli> If no key is available, the app crashes. :)
3682016-12-06T11:02:39 <jonasschnelli> (over the assertion)
3692016-12-06T11:03:04 <jonasschnelli> And I can't find a code block that ensures we check if the wallet is unlocked first.
3702016-12-06T11:05:02 * jonasschnelli afk
3712016-12-06T11:09:11 <bitcoin-git> [bitcoin] MarcoFalke pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/ed8d693c71b0...919db037f1f5
3722016-12-06T11:09:12 <bitcoin-git> bitcoin/master fa2ecc4 MarcoFalke: [qa] pruning: Use cached utxo set to run faster
3732016-12-06T11:09:13 <bitcoin-git> bitcoin/master fab1af3 MarcoFalke: [qa] maxuploadtarget: Use cached utxo set
3742016-12-06T11:09:13 <bitcoin-git> bitcoin/master 919db03 MarcoFalke: Merge #9274: [qa] Use cached utxo set to fix performance regression...
3752016-12-06T11:09:26 <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9274: [qa] Use cached utxo set to fix performance regression (master...Mf1612-qaPruningCacheUtxos) https://github.com/bitcoin/bitcoin/pull/9274
3762016-12-06T11:37:23 *** dcousens has quit IRC
3772016-12-06T11:37:49 <wumpus> jonasschnelli: yeah that comment is definitely wrong
3782016-12-06T11:38:00 <wumpus> jonasschnelli: it should raise an exception, not crash the application
3792016-12-06T11:39:11 *** dcousens has joined #bitcoin-core-dev
3802016-12-06T11:46:23 *** paveljanik has joined #bitcoin-core-dev
3812016-12-06T11:46:23 *** paveljanik has joined #bitcoin-core-dev
3822016-12-06T11:48:56 <jonasschnelli> wumpus: Yes. Will fix that soon,
3832016-12-06T11:51:46 * wumpus has built a riscv64 toolchain, then a riscv64 bitcoind and is running it in qemu-riscv64 (linux userspace), now syncing the chain with it. This was surprisingly easy with the depends system, bar 1 line I had to add for openssl configuration and -lpthreads in the LDFLAGS it just worked!
3842016-12-06T11:55:16 *** paveljanik has quit IRC
3852016-12-06T11:55:26 *** paveljanik has joined #bitcoin-core-dev
3862016-12-06T11:55:42 *** paveljanik has joined #bitcoin-core-dev
3872016-12-06T12:03:53 <bitcoin-git> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/919db037f1f5...e15660c16f75
3882016-12-06T12:03:53 <bitcoin-git> bitcoin/master 89a3723 Jonas Schnelli: [Qt] Show ModalOverlay by pressing the progress bar, disabled show() in sync mode
3892016-12-06T12:03:54 <bitcoin-git> bitcoin/master e15660c Jonas Schnelli: Merge #9280: [Qt] Show ModalOverlay by pressing the progress bar, allow hiding...
3902016-12-06T12:04:04 <bitcoin-git> [bitcoin] jonasschnelli closed pull request #9280: [Qt] Show ModalOverlay by pressing the progress bar, allow hiding (master...2016/12/qt_modal) https://github.com/bitcoin/bitcoin/pull/9280
3912016-12-06T12:06:02 *** e4xit has quit IRC
3922016-12-06T12:06:44 *** e4xit_ has joined #bitcoin-core-dev
3932016-12-06T12:21:47 *** laurentmt has joined #bitcoin-core-dev
3942016-12-06T12:25:31 *** laurentmt has quit IRC
3952016-12-06T12:47:11 *** Ylbam has quit IRC
3962016-12-06T12:47:30 *** Ylbam has joined #bitcoin-core-dev
3972016-12-06T12:50:41 *** jl2012 has joined #bitcoin-core-dev
3982016-12-06T12:51:31 <bitcoin-git> [bitcoin] jonasschnelli opened pull request #9295: [Wallet] Bugfix: Fundrawtransaction: don't terminate when keypool is empty (master...2016/12/fix_frt) https://github.com/bitcoin/bitcoin/pull/9295
3992016-12-06T12:51:50 <MarcoFalke> jonasschnelli: Does this fix the segfault on master?
4002016-12-06T12:52:12 <jonasschnelli> Which Segfault?
4012016-12-06T12:52:20 <jonasschnelli> It should fix an assert terminiation
4022016-12-06T12:52:27 <jonasschnelli> not sure if this can lead to a segfault.
4032016-12-06T12:53:01 <MarcoFalke> oh, maybe unrelated.
4042016-12-06T12:53:10 <MarcoFalke> I can't send any coins: bitcoin-qt: ./primitives/transaction.h:331: void SerializeTransaction(const TxType&, Stream&) [with Stream = CDataStream; TxType = CTransaction]: Assertion `tx.wit.vtxinwit.size() <= tx.vin.size()' failed.
4052016-12-06T12:53:55 <jonasschnelli> Hmm... that one probably needs further investigation.
4062016-12-06T12:57:46 *** jl2012 has quit IRC
4072016-12-06T12:58:15 *** jl2012 has joined #bitcoin-core-dev
4082016-12-06T13:00:17 <paveljanik> MarcoFalke, I can confirm the same problem here.
4092016-12-06T13:05:55 <jonasschnelli> paveljanik MarcoFalke: do we have steps to reproduce?
4102016-12-06T13:06:03 <jonasschnelli> Or a failing rpc test snipped?
4112016-12-06T13:06:18 <MarcoFalke> Interestingly the rpc tests do not fail, it seems
4122016-12-06T13:06:25 <paveljanik> jonasschnelli, only in GUI.
4132016-12-06T13:06:27 <MarcoFalke> only the gui, really odd
4142016-12-06T13:06:36 <paveljanik> Just Send something somewhere.
4152016-12-06T13:06:50 <paveljanik> after waiting for Yes(2) ... Yes. Click on Yes, BUMP
4162016-12-06T13:10:13 <afk11> there are some others that wind up exposed through bitcoinconsensus.h.. anyone wish to advise if we should have those functions check if an impossible situation is about to arise before calling VerifyScript?
4172016-12-06T13:10:18 <afk11> or consumers of bitcoinconsensus.h?
4182016-12-06T13:10:26 <afk11> sorry, others = assertions
4192016-12-06T13:10:37 *** dcousens has quit IRC
4202016-12-06T13:11:35 <jonasschnelli> paveljanik MarcoFalke: wait, it happens >always< on any tx in the GUI?
4212016-12-06T13:12:20 <paveljanik> both two my tries finished with SEIGSEGV
4222016-12-06T13:13:08 <jonasschnelli> I can send to myself.. but not directly on master (some commits behind)
4232016-12-06T13:13:16 <jonasschnelli> compiling now
4242016-12-06T13:13:23 <paveljanik> testnet e.g...
4252016-12-06T13:14:06 <paveljanik> it is #8580
4262016-12-06T13:14:08 <gribble> https://github.com/bitcoin/bitcoin/issues/8580 | Make CTransaction actually immutable by sipa · Pull Request #8580 · bitcoin/bitcoin · GitHub
4272016-12-06T13:15:49 <jonasschnelli> Ah. I'm regtesting..
4282016-12-06T13:16:31 <jonasschnelli> paveljanik, MarcoFalke: has someone opened an issue for that?
4292016-12-06T13:17:14 <paveljanik> not yet IIRC.
4302016-12-06T13:17:28 <paveljanik> I can't right now, about to leave..
4312016-12-06T13:17:48 <jonasschnelli> Okay. Sure.
4322016-12-06T13:18:02 <paveljanik> https://github.com/bitcoin/bitcoin/pull/8580#pullrequestreview-11582301
4332016-12-06T13:18:08 <paveljanik> but leaving now, sorry
4342016-12-06T13:23:05 *** Giszmo has joined #bitcoin-core-dev
4352016-12-06T13:25:11 *** rafalcpp has joined #bitcoin-core-dev
4362016-12-06T13:34:15 *** Chris_Stewart_5 has joined #bitcoin-core-dev
4372016-12-06T13:39:55 <morcos> jonasschnelli: paveljanik: found the bug. walletmodel.cpp:338
4382016-12-06T13:40:08 <morcos> i'm just going to do a quick scan for other occurences and then PR a fix
4392016-12-06T14:02:21 <jonasschnelli> morcos: great! Thanks.
4402016-12-06T14:03:01 <bitcoin-git> [bitcoin] morcos opened pull request #9296: Fix missed change to WalletTx structure (master...fixwallettx) https://github.com/bitcoin/bitcoin/pull/9296
4412016-12-06T14:04:15 <morcos> wumpus: we should get a quick review of 9296 and merge that or whatever the correct fix is
4422016-12-06T14:04:43 <morcos> Since anyone using master will have transactions committed and then crash, it could inadverntantly lead to multiple payments or something
4432016-12-06T14:05:12 <jonasschnelli> MarcoFalke: Oops, I think i change the tags you made on https://github.com/bitcoin/bitcoin/pull/9295
4442016-12-06T14:06:06 <MarcoFalke> The milestone says what is the lowest version the patch should go to.
4452016-12-06T14:06:27 <jonasschnelli> morcos: we haven't really discussed #8501...
4462016-12-06T14:06:29 <gribble> https://github.com/bitcoin/bitcoin/issues/8501 | Add mempool statistics collector by jonasschnelli · Pull Request #8501 · bitcoin/bitcoin · GitHub
4472016-12-06T14:06:29 <morcos> I panicked a little because I thought it was potentially corrupting the wallet at first, but what is getting committed to the wallet is fine, it was just serialization for the GUI that was messed up..
4482016-12-06T14:06:30 <MarcoFalke> So "needs backport" and "0.14.0" does not make sense :)
4492016-12-06T14:06:57 *** laurentmt has joined #bitcoin-core-dev
4502016-12-06T14:07:08 <morcos> jonasschnelli: yes, i left a few comments on irc yesterday re: 8501.
4512016-12-06T14:07:17 *** laurentmt has quit IRC
4522016-12-06T14:07:40 <jonasschnelli> morcos: I think there can be a problem with storing to many timestamp... is that what you thought?
4532016-12-06T14:08:09 <morcos> it just seemed to me that the idea of storing 1000 seconds, 1000 minutes, 1000 hours and 1000 days (with no time stamps) is perfectly adequate for any stat, but maybe not? (or 2000)
4542016-12-06T14:08:53 <jonasschnelli> Yes. But that would required fix frequency and active lock occupy. Not?
4552016-12-06T14:09:00 <morcos> i think the question would be what to do if some number of seconds go by before you record a new data point... i would argue that you want to differentiate between recording a repeated data point and not updating the stat
4562016-12-06T14:09:11 <morcos> jonasschnelli: not the way i'm imagining it
4572016-12-06T14:09:44 <morcos> lets say you had a way to indicate no value recorded.
4582016-12-06T14:10:10 <jonasschnelli> morcos: Yes. But I guess you still blow memory up to the size of the samples structure?...
4592016-12-06T14:10:13 <morcos> and look for a moment and the 1000 seconds data points... you have a circular buffer of 1000 entries
4602016-12-06T14:10:37 <morcos> and every time you call it, say you call it 17 seconds later.. you put in 16 NA's and then the latest data point
4612016-12-06T14:10:46 <morcos> so then graphing can be smart about interpolating
4622016-12-06T14:10:47 <jonasschnelli> hm...
4632016-12-06T14:11:09 <morcos> but if you call it every second with the same data point, it would put it in , you would distinguish between a step function and slow increase
4642016-12-06T14:11:10 <jonasschnelli> So you would not care if the 17second samples was done at 17.5s?
4652016-12-06T14:11:15 <morcos> yes, it would use constant memory
4662016-12-06T14:11:21 <morcos> but not that much
4672016-12-06T14:11:25 <morcos> a few MB
4682016-12-06T14:12:20 <jonasschnelli> I think the possible time offset could matter at the 1000 hours circular buffer.
4692016-12-06T14:12:40 <morcos> right, so you'd have to figure out how to cover that.. but i think thats doable.. suppose the data point for 12:14:16 means the latest data point that you have before 12:14:16.00000000
4702016-12-06T14:12:50 <jonasschnelli> Assume sample 1 was done at 0h, sample 2 and 0.6h (=1h sample), sample three at 1.4.
4712016-12-06T14:13:16 <jonasschnelli> What speaks against storing a sample with a delta timestamp (1byte) to the last sample?
4722016-12-06T14:13:25 <jonasschnelli> Could be something like a linked list with NA's
4732016-12-06T14:13:41 <jonasschnelli> *linked list
4742016-12-06T14:13:42 <morcos> then if you call it at 14.93 (that goes in the 12:14:15 data point) then you save just the latest call every time if you call it at 15.03, 15.37, 15.87, and then when you call it at 16.05, you put the 15.87 data point in for 12:14:16
4752016-12-06T14:14:47 <morcos> i think you could also do linked lists with delta time stamps, but i think the delta time stamps don't add much
4762016-12-06T14:15:20 <morcos> i guess i'm envisioning that for things with 1h samplign or 1day sampling you'll be calling it much more frequently anyway
4772016-12-06T14:15:27 <jonasschnelli> Maybe the advantage of delta timestamps is: more accurate (no interpolation in the recorded data), flexible memory cap-
4782016-12-06T14:15:31 <morcos> so you'll get a data point pretty close to the cut off
4792016-12-06T14:16:18 <jonasschnelli> sipa once had the idea to remove in-between samples once they are old and stats-memory gets drained.
4802016-12-06T14:16:18 <morcos> but are you envisioning that the part of the program thats making the call to record the sample is doing it separately for each timescale?
4812016-12-06T14:17:00 <morcos> well the design i'm trying to describe would never even keep "in-between" samples
4822016-12-06T14:17:02 <jonasschnelli> morcos: IMO the stats infrastructure should not actively record stuff. It should offer an interface to store records based
4832016-12-06T14:17:19 <morcos> yes, i understand
4842016-12-06T14:17:24 <jonasschnelli> "in-between" samples is probably the wrong term. Sorry.
4852016-12-06T14:17:33 <morcos> so in your mempool code, you call stats.record(SomeStats)
4862016-12-06T14:17:39 <jonasschnelli> Let's assume you record samples in a 1s frequency...
4872016-12-06T14:17:47 <jonasschnelli> then you reserved stats memory gets low
4882016-12-06T14:17:49 <morcos> but i'm saying that you can then record those however often you want from the mempool
4892016-12-06T14:17:59 <jonasschnelli> You could remove some of the old 1s samples...
4902016-12-06T14:18:07 <morcos> and the stats class will save the last 1000 seconds, 1000 mins, 1000 hrs and 1000 days of data poitns for you
4912016-12-06T14:18:23 <morcos> it removes the old 1s samples b/c its a circular buffer
4922016-12-06T14:18:58 <jonasschnelli> Yes. But maybe someone once to keep 1s stats longer.. in 8501 I introduces a -statsmaxmemory
4932016-12-06T14:19:23 <morcos> ok, well that was my argument, is that no one would care about keeping 1s stats for longer than say 1000 or 2000 secs
4942016-12-06T14:19:32 <morcos> if they wanted to look at a longer time period, they would look at minutes
4952016-12-06T14:19:36 <jonasschnelli> Thats a point. Right
4962016-12-06T14:19:39 <morcos> if that is false, then i agree
4972016-12-06T14:20:14 <jonasschnelli> My experience tells me, record as much as possible, :-)
4982016-12-06T14:20:28 <jonasschnelli> I would even say a stats to disk dumper would be great
4992016-12-06T14:20:44 <jonasschnelli> Especially stuff that can't be reconstructred
5002016-12-06T14:20:50 <jonasschnelli> (freerates, etc.)
5012016-12-06T14:21:11 <morcos> the concern i had about your design was that if we are recording lots of stats
5022016-12-06T14:21:24 <morcos> then we lose the ability to look over a long time period without using tons of memory
5032016-12-06T14:21:31 <morcos> i don't really care how we do it
5042016-12-06T14:21:42 <jonasschnelli> Yes. That is not done well in 8501
5052016-12-06T14:21:53 <morcos> but i think the stats class should automatically keep less fine-grained data for a longer time in order to keep the memory limited
5062016-12-06T14:21:59 <morcos> i was just presenting one way to do that
5072016-12-06T14:22:06 <jonasschnelli> The idea was to slowly stretch the frequency over time, depending on the available stats ram.
5082016-12-06T14:22:24 <jonasschnelli> Not just cut the back
5092016-12-06T14:22:26 <morcos> but don't you think you might want to look at both?
5102016-12-06T14:22:36 <jonasschnelli> Yes.
5112016-12-06T14:22:42 <morcos> high frequency recent and low frequency long time horizon
5122016-12-06T14:22:48 <morcos> ok... so thats my goal...
5132016-12-06T14:22:51 <jonasschnelli> I think you convinced me to do the 1000s 1000m 1000h 1000d approach.
5142016-12-06T14:23:01 <jonasschnelli> maybe the 1000 is configurable.
5152016-12-06T14:23:06 <morcos> doesn't matter to me how we do it... i think a delta version coudl be just as good
5162016-12-06T14:23:16 <morcos> and you could just be smart about trimming the delta list or something
5172016-12-06T14:23:30 <morcos> yes, 1000 should be configurable i thik... actually maybe isn't enough for a default
5182016-12-06T14:23:47 <jonasschnelli> also, I liked the configurability of the buffer in MB.
5192016-12-06T14:23:54 <jonasschnelli> That's what you probably care about.
5202016-12-06T14:23:58 <jonasschnelli> Not the 1000
5212016-12-06T14:24:01 <morcos> 1000 secs is just 16 minutes... you would not want to have to only have 16 data points
5222016-12-06T14:24:14 <jonasschnelli> You would say, I reserve 300MB for stats.
5232016-12-06T14:24:38 <jonasschnelli> Right... just en
5242016-12-06T14:24:46 <jonasschnelli> just as an example
5252016-12-06T14:25:16 <jonasschnelli> So,.. you convinced me for high frequency recent and low frequency long time horizon,...
5262016-12-06T14:25:23 <morcos> ok cool... any approach that automatically keeps both recent fine-grained and long time horizon bigger step, is fine with me
5272016-12-06T14:25:27 <jonasschnelli> And I think the delta timeoffset thing could work.
5282016-12-06T14:25:28 <morcos> thanks
5292016-12-06T14:25:40 <jonasschnelli> It can be tricky if you cut sample not at the back
5302016-12-06T14:25:51 <jonasschnelli> You need to re-adjust the delta.
5312016-12-06T14:26:14 <jonasschnelli> Losing a sample will corrupt the timestream
5322016-12-06T14:26:43 <morcos> i'll let you worry about that. :)
5332016-12-06T14:26:48 <jonasschnelli> heh.
5342016-12-06T14:27:00 <jonasschnelli> Yes. I try to overhaul 8501
5352016-12-06T14:27:18 <jonasschnelli> But lets first fix Bitcoin
5362016-12-06T14:42:07 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e15660c16f75...fde7d99c4d7d
5372016-12-06T14:42:08 <bitcoin-git> bitcoin/master 28f8ae8 Alex Morcos: Fix missed change to WalletTx structure
5382016-12-06T14:42:08 <bitcoin-git> bitcoin/master fde7d99 Wladimir J. van der Laan: Merge #9296: Fix missed change to WalletTx structure...
5392016-12-06T14:42:20 <bitcoin-git> [bitcoin] laanwj closed pull request #9296: Fix missed change to WalletTx structure (master...fixwallettx) https://github.com/bitcoin/bitcoin/pull/9296
5402016-12-06T14:46:16 *** laurentmt has joined #bitcoin-core-dev
5412016-12-06T14:46:19 *** laurentmt has quit IRC
5422016-12-06T14:54:36 <kanzure> reproducible builds for windows https://github.com/jasonwhite/ducible
5432016-12-06T15:02:24 <timothy> kanzure: you can still build using mingw32 on linux :P
5442016-12-06T15:03:35 *** TomMc has joined #bitcoin-core-dev
5452016-12-06T15:41:32 *** protomar has quit IRC
5462016-12-06T15:45:44 *** JackH has joined #bitcoin-core-dev
5472016-12-06T15:52:50 *** abpa has joined #bitcoin-core-dev
5482016-12-06T15:57:21 *** Ylbam has quit IRC
5492016-12-06T15:58:50 *** laurentmt has joined #bitcoin-core-dev
5502016-12-06T16:05:54 *** laurentmt has quit IRC
5512016-12-06T16:09:01 <instagibbs> is there a written down explanation of the lock macros anywhere
5522016-12-06T16:10:45 *** limpkin has quit IRC
5532016-12-06T16:11:12 *** limpkin has joined #bitcoin-core-dev
5542016-12-06T16:25:06 *** echonaut8 has joined #bitcoin-core-dev
5552016-12-06T16:25:09 *** echonaut has quit IRC
5562016-12-06T16:31:02 *** Sosumi has joined #bitcoin-core-dev
5572016-12-06T16:32:40 *** aalex has quit IRC
5582016-12-06T16:33:00 *** aalex has joined #bitcoin-core-dev
5592016-12-06T16:39:24 *** moli has joined #bitcoin-core-dev
5602016-12-06T16:42:13 *** BashCo has quit IRC
5612016-12-06T16:42:45 *** BashCo has joined #bitcoin-core-dev
5622016-12-06T16:48:05 *** abpa has joined #bitcoin-core-dev
5632016-12-06T16:55:13 *** BashCo has quit IRC
5642016-12-06T17:32:42 *** aalex has quit IRC
5652016-12-06T17:33:01 *** aalex has joined #bitcoin-core-dev
5662016-12-06T17:40:31 *** BashCo has joined #bitcoin-core-dev
5672016-12-06T17:40:53 <Chris_Stewart_5> sipa: re: yesterdays convo about syncing, seems like another guy had a similar experience that I did in #bitcoin, not sure if this is a concern or not
5682016-12-06T17:45:24 *** jtimon has joined #bitcoin-core-dev
5692016-12-06T17:50:17 <sipa> Chris_Stewart_5: that sync is slow? of course it is a concern
5702016-12-06T17:50:40 <sipa> but we can't just go double our memory requirements
5712016-12-06T17:54:02 <gmaxwell> the speeds Chris_Stewart_5 were reporting sounded reasonably slow, but yes-- the processing is slowing down as the chain grows and will continue to do so.
5722016-12-06T17:54:50 <gmaxwell> We have been fighting that effect for years and will continue to do so, but nothing changes the fact that more state means more work.
5732016-12-06T17:54:54 *** atroxes has joined #bitcoin-core-dev
5742016-12-06T18:02:55 <Chris_Stewart_5> gmaxwell: Yes, I was wondering if this was average or extremely slow. I thought I read something about it being on the order of magnitude of 12 hours -- everything you read online is accurate right? :P
5752016-12-06T18:05:02 *** atroxes_ has joined #bitcoin-core-dev
5762016-12-06T18:08:13 *** aalex has quit IRC
5772016-12-06T18:08:35 *** aalex has joined #bitcoin-core-dev
5782016-12-06T18:08:53 <owowo> I synced today, was about 4k block behind, but it was quite fast.
5792016-12-06T18:09:10 *** atroxes has quit IRC
5802016-12-06T18:09:10 *** atroxes_ is now known as atroxes
5812016-12-06T18:25:10 *** protomar has joined #bitcoin-core-dev
5822016-12-06T18:32:34 *** protomar has quit IRC
5832016-12-06T18:38:10 *** Ylbam has joined #bitcoin-core-dev
5842016-12-06T18:40:44 *** alpalp has quit IRC
5852016-12-06T19:04:42 *** jannes has quit IRC
5862016-12-06T19:12:40 *** Madars has quit IRC
5872016-12-06T19:15:13 *** TomMc has quit IRC
5882016-12-06T19:19:45 *** laurentmt has joined #bitcoin-core-dev
5892016-12-06T19:20:04 *** laurentmt has quit IRC
5902016-12-06T19:33:03 *** Madars has joined #bitcoin-core-dev
5912016-12-06T19:34:19 *** jtimon has quit IRC
5922016-12-06T19:42:26 *** MarcoFalke has quit IRC
5932016-12-06T19:47:36 *** belcher has quit IRC
5942016-12-06T19:52:13 *** atroxes has quit IRC
5952016-12-06T19:52:56 *** atroxes has joined #bitcoin-core-dev
5962016-12-06T19:53:47 *** MarcoFalke has joined #bitcoin-core-dev
5972016-12-06T19:57:44 *** atroxes has quit IRC
5982016-12-06T19:59:32 *** belcher has joined #bitcoin-core-dev
5992016-12-06T20:03:47 *** atroxes has joined #bitcoin-core-dev
6002016-12-06T20:15:08 *** atroxes has quit IRC
6012016-12-06T20:16:10 *** atroxes has joined #bitcoin-core-dev
6022016-12-06T20:55:41 *** Sosumi has quit IRC
6032016-12-06T20:58:23 <morcos> gmaxwell: can we duel about SelectCoinsMinConf(.., #ancestors, ..)?
6042016-12-06T20:58:50 <morcos> both sdaftuar and i think that the most important thing for your goal is to try low values for number of ancestors
6052016-12-06T20:59:33 <morcos> and lets for the moment just worry abou tthe devault limit of 25... then you are suggesting 1,6,13,25
6062016-12-06T21:01:13 <morcos> sorry 0,6,13,25 (eh, 0 or 1 doesn't mean anything anyway though b/c you skip anything >= n and everything has at least 1 , it counts itself)
6072016-12-06T21:02:29 <morcos> so i think emphasizing more tries in the low limits helps keep any individual chain small and therefore helps avoid polluting small chains by combining them with big ones
6082016-12-06T21:03:10 <morcos> so trying 2,3,4,25 might even be better than 2,6,13,25
6092016-12-06T21:03:27 <morcos> anyway thats the tradeoff sdaftuar and i were trying to suggest, emphasize trying the low numbers
6102016-12-06T21:04:26 <gmaxwell> morcos: I agree with you, I didn't perform that bit of shed painting myself because I didn't think it mattered _that_ much. The only reason I commented there was because instagibbs changed it in a way that was clearly wrong for some values. :)
6112016-12-06T21:04:53 <morcos> yeah but all it is is a bit inefficient for not default limits
6122016-12-06T21:04:58 <morcos> no one has non-default limits
6132016-12-06T21:04:59 <morcos> who cares
6142016-12-06T21:05:03 <morcos> its not broken
6152016-12-06T21:05:24 <gmaxwell> I don't care about inefficient, but when he changed the patch he harcoded one to 3 which could be higher than your limit.
6162016-12-06T21:05:36 <morcos> but that doesn't hurt anything
6172016-12-06T21:06:46 *** ville-- has quit IRC
6182016-12-06T21:06:46 <gmaxwell> say you've set your limit to two, it will cause you to go past it.
6192016-12-06T21:06:52 *** ville-- has joined #bitcoin-core-dev
6202016-12-06T21:06:53 <morcos> it just lets you potentially pick coins which will let you create a tx which will fail the final test.. thats already a possibility anyway..
6212016-12-06T21:07:27 *** ville-- has quit IRC
6222016-12-06T21:07:33 *** ville- has joined #bitcoin-core-dev
6232016-12-06T21:07:51 <morcos> the actual test is way more restrictive, none of your ancestors can have more than the descendant limit
6242016-12-06T21:08:01 <gmaxwell> Personally, if I wrote it, I would have done 1,2,3,max-2,max-1,max or something like that.
6252016-12-06T21:08:04 <morcos> we're nto checking that in this quick heuristic anyway
6262016-12-06T21:08:23 <morcos> so you always might be picking coins with which you have no chance , you just don't know
6272016-12-06T21:08:39 <morcos> but better to have multiple tries with things which have a good heuristic
6282016-12-06T21:08:46 <gmaxwell> you might be, but that isn't the pratical case people are encountering.
6292016-12-06T21:09:03 <gmaxwell> I don't know where we disagree.
6302016-12-06T21:09:34 <morcos> mostly i think its fine as is, and your suggestion is a slight downgrade.. but i agree its mostly bikeshedding i suppose
6312016-12-06T21:10:52 <gmaxwell> hm. if the limit is set to 2.. why should it check 3? would you agree that min(3,limit) wouldn't be better?
6322016-12-06T21:11:29 <morcos> sure
6332016-12-06T21:11:53 <gmaxwell> I think it absolutely should spend more tries on low numbers, because say it runs with limit/4, then it combines two inputs a depth 2 and a depth limit/2 .. great, now you've just trashed a short chain.
6342016-12-06T21:12:04 <morcos> exactly
6352016-12-06T21:13:03 <gmaxwell> but for that e.g. spacing my a multiplicave factor at least won't screw you by more than that factor.
6362016-12-06T21:13:09 <morcos> i'm fine adding a min, i just think making it work well for a limit of 25 is fine, as long as its not horribly broken for other limits, which it wont' be b/c it'll still try 2 first
6372016-12-06T21:13:35 <gmaxwell> okay we agree then.
6382016-12-06T21:14:07 <morcos> 2, min(4,limit), something in between, limit
6392016-12-06T21:14:24 <morcos> :)
6402016-12-06T21:18:02 <gmaxwell> I posted min(2,limit/4), min(4, limit/3), limit/2
6412016-12-06T21:18:06 <gmaxwell> , limit
6422016-12-06T21:18:19 <instagibbs> is it safe to come out of my bikeshelter :)
6432016-12-06T21:18:27 <sipa> cfields: i've been playing with -flto lately, and you can use -flto=jobserver at link time to make gcc itself perform the linking step in multiple processes... the problem is that this requires prefixing the compiler invocation in make with a '+' so make knows to expose its jobserver interface to it
6442016-12-06T21:18:47 <gmaxwell> instagibbs: haha
6452016-12-06T21:18:48 <sipa> cfields: any idea how to accomplish that in automake. it doesn't seem that CXX="+g++" works
6462016-12-06T21:19:49 <gmaxwell> morcos: I really try to not bikeshed this stuff too much, because it's just a hop-skip-and-jump away from arguing that we should be linking GLPK.
6472016-12-06T21:20:09 <morcos> heh
6482016-12-06T21:20:37 <morcos> well sdaftuar and i played around with testing the PR, and it kind of performed poorly
6492016-12-06T21:20:49 <morcos> if you started with two inputs
6502016-12-06T21:21:02 <morcos> instead of getting 50 txs, you only got 30 (whereas before you got 25)
6512016-12-06T21:21:53 <morcos> this was because we were sending to ourselves so the outputs were being combined quickly and we were trashing chains, so led us to prefer this small change approach.. obviously as sdaftuar reported, it works much better when the test doesn't pay yourself
6522016-12-06T21:22:24 <morcos> s/small change approach/emphasis on small limits approach/
6532016-12-06T21:23:27 <gmaxwell> I'm glad you tested it-- I anticipated that result but I was trying to resist bikesheding it and making it not go into 0.13.2.
6542016-12-06T21:24:30 *** paveljanik has quit IRC
6552016-12-06T21:24:32 <morcos> i thik with a large set of coins though , it'll really shine as it'll stop you from just being stupid
6562016-12-06T21:25:12 <morcos> btw, it would be helpful for me if more people went through and tagged things for 0.14, i could review some more things, and would like to know where to concentrate...
6572016-12-06T21:25:25 <morcos> i think i've done what i can on the existing milestones
6582016-12-06T21:26:23 <instagibbs> ok, putting final touches on the selection, thanks for all the feedback
6592016-12-06T21:26:35 <instagibbs> (also am I the only one getting spammed to hell by phising PMs?)
6602016-12-06T21:27:23 *** AaronvanW has quit IRC
6612016-12-06T21:27:41 <gmaxwell> instagibbs: yes.. you can /mode instagibbs +R to turn off pms from unregistered users.
6622016-12-06T21:28:01 <instagibbs> ah thx
6632016-12-06T21:28:17 <gmaxwell> remember to turn it back off again later, if you want j-random-people to pm you later.
6642016-12-06T21:28:52 <sipa> what about j-invariant-people?
6652016-12-06T21:30:13 <gmaxwell> sipa: thats complex.
6662016-12-06T21:33:23 <bitcoin-git> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/fde7d99c4d7d...09c4fd157c5b
6672016-12-06T21:33:24 <bitcoin-git> bitcoin/master 9b9324e Matt Corallo: Fix rounding privacy leak introduced in #9260
6682016-12-06T21:33:24 <bitcoin-git> bitcoin/master 09c4fd1 Pieter Wuille: Merge #9268: Fix rounding privacy leak introduced in #9260...
6692016-12-06T21:33:39 <bitcoin-git> [bitcoin] sipa closed pull request #9268: Fix rounding privacy leak introduced in #9260 (master...2016-12-feefilterrounder) https://github.com/bitcoin/bitcoin/pull/9268
6702016-12-06T21:55:00 *** Guyver2 has quit IRC
6712016-12-06T21:58:40 *** grubles_ has joined #bitcoin-core-dev
6722016-12-06T22:18:25 *** wvr has joined #bitcoin-core-dev
6732016-12-06T22:19:48 *** cryptapus_afk has quit IRC
6742016-12-06T22:35:48 *** LeMiner has quit IRC
6752016-12-06T22:40:10 *** LeMiner has joined #bitcoin-core-dev
6762016-12-06T23:00:01 *** tmkexpress has joined #bitcoin-core-dev
6772016-12-06T23:11:34 *** MarcoFalke has left #bitcoin-core-dev
6782016-12-06T23:26:09 <bitcoin-git> [bitcoin] Mirobit opened pull request #9297: Various RPC help outputs updated (master...rpchelp12/16) https://github.com/bitcoin/bitcoin/pull/9297
6792016-12-06T23:54:44 *** jtimon has joined #bitcoin-core-dev