12017-02-25T00:02:22 <cfields> sipa: want me to PR that? Or you have something else in mind?
22017-02-25T00:02:59 <BlueMatt> sipa: ok, here's a theory......"10:39:08 Prune:" is the prune called by AcceptBlock's FlushStateToDisk call (seems obvious given the Pre-allocating blk just above)....now, that flush call hit fDoFullFlush triggering a pcoinsTip->Flush(). OK, so now in CCoinsViewDB::BatchWrite we iterate over the entries in mapCoins, allocating memory for each one in the leveldb batch, erasing *AS WE GO*...at some point it ran out of memory, throwing
32017-02-25T00:02:59 <BlueMatt> bad_alloc and leaving pcoinsTip in an inconsistent state, but std::bad_alloc is not an std::runtime_error so the catch at the end of FlushStateToDisk didnt catch it. OK, phew, so now we go to connect the next block and fail to connect its inputs (without allocating new disk space). I /think/ that explains your debug.log without hand waving about hardware errors
42017-02-25T00:04:06 *** alpalp has joined #bitcoin-core-dev
52017-02-25T00:04:06 *** alpalp has joined #bitcoin-core-dev
62017-02-25T00:04:36 <sipa> BlueMatt: sounds like a plausible theory
72017-02-25T00:04:40 <sipa> cfields: PR what?
82017-02-25T00:04:49 <cfields> BlueMatt: that was my reading of it as well
92017-02-25T00:05:07 *** MarcoFalke has quit IRC
102017-02-25T00:05:11 <cfields> sipa: https://github.com/theuni/bitcoin/commit/28afe574567ba838d46959047282460dbab39b91
112017-02-25T00:05:24 <sipa> if that's the problem (and that seems to become more plausible as dbcache gets set higher, as you get the duplication only at flush time)
122017-02-25T00:05:43 <sipa> we should probably make the batch construction not erase things for ccoinscache
132017-02-25T00:06:12 <BlueMatt> sipa: yes, an easy way to make this particular, very specific issue, less likely is to not erase as we go
142017-02-25T00:06:19 <BlueMatt> ofc that means more memory usage, but...better than blowing up
152017-02-25T00:08:15 <sipa> BlueMatt: alternatively, we could fail to flush, and still delete everything
162017-02-25T00:08:18 <sipa> from the cache
172017-02-25T00:08:36 <BlueMatt> sipa: hmm?
182017-02-25T00:08:57 <sipa> ... and then reset pcoinsTip->hashBlock and chainActive to the last succesfully flushed block (which may be 1000s ago)
192017-02-25T00:09:21 <BlueMatt> sipa: in that case we must AbortNode, I think?
202017-02-25T00:09:28 <sipa> oh, absolutely
212017-02-25T00:10:18 <sipa> but i'm worried we may not be able to to instantly abort (due to locks being held all over the place while the OOM occurs)
222017-02-25T00:11:10 <BlueMatt> yea, I mean ideally we'd broaden the catch in FlushStateToDisk and use the AbortNode there
232017-02-25T00:11:23 <BlueMatt> just this time dont let the coins map get into an inconsistent state
242017-02-25T00:12:20 <sipa> cfields: looks good
252017-02-25T00:13:17 <BlueMatt> cfields: oh, somehow I missed that that /is/ only for bad_alloc
262017-02-25T00:13:32 <BlueMatt> yea, lets totally do that!
272017-02-25T00:13:33 <cfields> <BlueMatt> wait, does this apply to more than bad_alloc?
282017-02-25T00:13:33 <cfields> <cfields> no
292017-02-25T00:13:35 <cfields> :)
302017-02-25T00:13:44 <BlueMatt> oh, reading comprehension fail
312017-02-25T00:18:08 <cfields> sipa: want to switch prevector to new[] and ditch realloc()? Or call get_new_handler()() if they return null?
322017-02-25T00:18:23 <cfields> (or just leave it alone and let something write to 0 :p)
332017-02-25T00:18:41 <sipa> cfields: just switch it to new[]
342017-02-25T00:18:57 <cfields> ok
352017-02-25T00:18:59 <BlueMatt> it would suck to drop realloc?
362017-02-25T00:19:20 <BlueMatt> just assert(new_ptr)
372017-02-25T00:19:39 <sipa> how often does realloc actually help?
382017-02-25T00:20:02 <BlueMatt> no idea, but the difference between get_new_handler()() and assert(new_ptr) is minimal
392017-02-25T00:20:51 <sipa> well i don't like making prevector assume that OOMs are fatal
402017-02-25T00:20:59 <sipa> that's something application level
412017-02-25T00:21:29 <BlueMatt> sipa: yea, and in our application they always are :p
422017-02-25T00:22:46 <sipa> still, it's nicer to have that knowledge in only one place
432017-02-25T00:26:14 <BlueMatt> sipa: internet says realloc is useful, though in our usage of prevector its likely not super noticeable
442017-02-25T00:26:22 <BlueMatt> so I'm fine with whatever
452017-02-25T00:30:42 <sipa> BlueMatt: oh, i just realized... calling get_new_handler()() in prevector doesn't imply it's going to fail
462017-02-25T00:31:58 <BlueMatt> huh? you mean doesnt imply its going to std::terminate instead of throwing or so?
472017-02-25T00:31:58 <sipa> yes
482017-02-25T00:32:05 <sipa> so, i don't care.
492017-02-25T00:32:05 <BlueMatt> ok, if you prefer that, do that
502017-02-25T00:41:07 <bitcoin-git> [bitcoin] jtimon opened pull request #9855: RPC: Use integer satoshis instead BTC with decimals (master...0.15-rpc-satoshis) https://github.com/bitcoin/bitcoin/pull/9855
512017-02-25T00:48:00 *** jl2012 has quit IRC
522017-02-25T00:54:25 *** PaulCape_ has quit IRC
532017-02-25T00:56:11 *** PaulCapestany has joined #bitcoin-core-dev
542017-02-25T00:56:50 *** gk_-1wm_- has joined #bitcoin-core-dev
552017-02-25T00:56:51 *** gk_-1wm_- has left #bitcoin-core-dev
562017-02-25T01:00:51 *** echonaut has quit IRC
572017-02-25T01:01:00 *** abpa has quit IRC
582017-02-25T01:01:07 *** echonaut has joined #bitcoin-core-dev
592017-02-25T01:17:28 *** bsm1175322 has quit IRC
602017-02-25T01:18:02 *** CodeShark has quit IRC
612017-02-25T01:18:17 *** CodeShark has joined #bitcoin-core-dev
622017-02-25T01:25:11 *** Chris_Stewart_5 has quit IRC
632017-02-25T01:30:09 *** dodomojo has joined #bitcoin-core-dev
642017-02-25T01:33:13 *** Chris_Stewart_5 has joined #bitcoin-core-dev
652017-02-25T01:35:02 *** arowser has quit IRC
662017-02-25T01:39:34 *** arowser has joined #bitcoin-core-dev
672017-02-25T01:56:55 *** arowser has quit IRC
682017-02-25T01:58:40 *** arowser has joined #bitcoin-core-dev
692017-02-25T02:11:33 *** jtimon has quit IRC
702017-02-25T02:20:02 *** gk_-1wm has joined #bitcoin-core-dev
712017-02-25T02:20:03 *** gk_-1wm has left #bitcoin-core-dev
722017-02-25T02:34:54 *** droark has joined #bitcoin-core-dev
732017-02-25T02:36:43 *** droark has quit IRC
742017-02-25T02:39:22 *** alpalp has quit IRC
752017-02-25T02:43:25 *** Giszmo has quit IRC
762017-02-25T02:44:53 *** jl2012 has joined #bitcoin-core-dev
772017-02-25T02:57:55 *** wasi has quit IRC
782017-02-25T02:58:32 *** wasi has joined #bitcoin-core-dev
792017-02-25T03:02:09 *** AaronvanW has quit IRC
802017-02-25T03:02:43 *** AaronvanW has joined #bitcoin-core-dev
812017-02-25T03:08:23 *** alpalp has joined #bitcoin-core-dev
822017-02-25T03:08:23 *** alpalp has joined #bitcoin-core-dev
832017-02-25T03:35:24 <sipa> cfields, BlueMatt: crap, it seems i reported this exact same bug a year ago already...
842017-02-25T03:59:44 *** droark has joined #bitcoin-core-dev
852017-02-25T04:01:58 *** dodomojo has quit IRC
862017-02-25T04:06:44 <cfields> sipa: heh. Well, at least we can get it in for 0.14
872017-02-25T04:06:59 <cfields> sipa: does that mean it takes 1+ years to sync your rpi? :)
882017-02-25T04:10:48 *** wudayoda has quit IRC
892017-02-25T04:12:03 *** wudayoda has joined #bitcoin-core-dev
902017-02-25T04:14:01 *** Ylbam has quit IRC
912017-02-25T04:18:34 <sipa> cfields: it's a new one
922017-02-25T04:18:35 *** alpalp has quit IRC
932017-02-25T04:18:48 <sipa> it took 2 days to sync to the point where it was
942017-02-25T04:18:54 *** alpalp has joined #bitcoin-core-dev
952017-02-25T04:18:54 *** alpalp has joined #bitcoin-core-dev
962017-02-25T04:29:22 *** alpalp has quit IRC
972017-02-25T04:31:12 *** dodomojo has joined #bitcoin-core-dev
982017-02-25T04:43:54 *** dodomojo has quit IRC
992017-02-25T04:49:03 *** dodomojo has joined #bitcoin-core-dev
1002017-02-25T04:49:56 *** dodomojo has quit IRC
1012017-02-25T05:06:28 <cfields> sipa: i have no clue how representative the CCheckQueueSpeedPrevectorJob bench is, but there's a noticible difference when switching to new[] and dropping realloc
1022017-02-25T05:07:37 <cfields> sipa: for the sake of being conservative this late in the release cycle, i think get_new_handler()() is likely to have much less impact
1032017-02-25T05:10:31 *** mr_burdell has joined #bitcoin-core-dev
1042017-02-25T05:13:00 *** BashCo_ has joined #bitcoin-core-dev
1052017-02-25T05:14:08 *** roasbeef_ has joined #bitcoin-core-dev
1062017-02-25T05:14:12 *** davec__ has joined #bitcoin-core-dev
1072017-02-25T05:14:17 *** gmaxwell_ has joined #bitcoin-core-dev
1082017-02-25T05:14:32 *** AdrianG_ has joined #bitcoin-core-dev
1092017-02-25T05:14:47 *** wump has joined #bitcoin-core-dev
1102017-02-25T05:16:42 *** davec has quit IRC
1112017-02-25T05:16:43 *** Chris_Stewart_5 has quit IRC
1122017-02-25T05:16:44 *** jamoes has quit IRC
1132017-02-25T05:16:44 *** jcorgan has quit IRC
1142017-02-25T05:16:45 *** Aleph0 has quit IRC
1152017-02-25T05:16:45 *** haakonn has quit IRC
1162017-02-25T05:16:48 *** murchandamus has quit IRC
1172017-02-25T05:16:48 *** adiabat has quit IRC
1182017-02-25T05:16:48 *** wumpus has quit IRC
1192017-02-25T05:16:49 *** goregrin1 has quit IRC
1202017-02-25T05:16:49 *** Guest74039 has quit IRC
1212017-02-25T05:16:50 *** achow101 has quit IRC
1222017-02-25T05:16:53 *** _mn3monic has quit IRC
1232017-02-25T05:16:54 *** roasbeef has quit IRC
1242017-02-25T05:16:54 *** midnightmagic has quit IRC
1252017-02-25T05:16:54 *** gmaxwell has quit IRC
1262017-02-25T05:16:54 *** so has quit IRC
1272017-02-25T05:16:55 *** BashCo has quit IRC
1282017-02-25T05:16:55 *** justanotheruser has quit IRC
1292017-02-25T05:16:55 *** waxwing has quit IRC
1302017-02-25T05:16:56 *** gluytium_ has quit IRC
1312017-02-25T05:16:57 *** Squidicc has quit IRC
1322017-02-25T05:16:58 *** warren has quit IRC
1332017-02-25T05:16:59 *** Squidicuz has joined #bitcoin-core-dev
1342017-02-25T05:17:00 *** so_ has joined #bitcoin-core-dev
1352017-02-25T05:17:00 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1362017-02-25T05:17:07 *** warren has joined #bitcoin-core-dev
1372017-02-25T05:17:10 *** waxwing has joined #bitcoin-core-dev
1382017-02-25T05:17:21 *** achow101 has joined #bitcoin-core-dev
1392017-02-25T05:17:29 *** goregrind has joined #bitcoin-core-dev
1402017-02-25T05:17:32 *** gmaxwell_ has quit IRC
1412017-02-25T05:17:33 *** gmaxwell_ has joined #bitcoin-core-dev
1422017-02-25T05:17:36 *** murchandamus has joined #bitcoin-core-dev
1432017-02-25T05:17:44 *** gmaxwell_ is now known as gmaxwell
1442017-02-25T05:18:01 *** adiabat has joined #bitcoin-core-dev
1452017-02-25T05:18:21 *** jcorgan has joined #bitcoin-core-dev
1462017-02-25T05:18:57 *** midnightmagic has joined #bitcoin-core-dev
1472017-02-25T05:19:01 *** jamoes has joined #bitcoin-core-dev
1482017-02-25T05:19:11 *** gluytium has joined #bitcoin-core-dev
1492017-02-25T05:19:53 *** justanotheruser has joined #bitcoin-core-dev
1502017-02-25T05:20:23 *** michagogo has quit IRC
1512017-02-25T05:21:17 *** haakonn has joined #bitcoin-core-dev
1522017-02-25T05:21:41 *** haakonn is now known as Guest48161
1532017-02-25T05:24:44 *** michagogo has joined #bitcoin-core-dev
1542017-02-25T05:29:19 <sipa> cfields: ack
1552017-02-25T05:30:30 <bitcoin-git> [bitcoin] theuni opened pull request #9856: Terminate immediately when allocation fails (master...bad_alloc_terminate2) https://github.com/bitcoin/bitcoin/pull/9856
1562017-02-25T05:32:36 <cfields> damn fanquake, you're fast!
1572017-02-25T05:35:04 *** fanquake has joined #bitcoin-core-dev
1582017-02-25T05:40:13 *** fanquake1 has joined #bitcoin-core-dev
1592017-02-25T05:41:15 *** fanquake has quit IRC
1602017-02-25T05:41:15 *** Guest48161 has quit IRC
1612017-02-25T05:41:16 *** so_ has quit IRC
1622017-02-25T05:42:30 *** Chris_Stewart_5 has quit IRC
1632017-02-25T05:42:31 *** Squidicuz has quit IRC
1642017-02-25T05:42:31 *** Squidicuz has joined #bitcoin-core-dev
1652017-02-25T05:42:34 *** so has joined #bitcoin-core-dev
1662017-02-25T05:43:12 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1672017-02-25T05:46:40 *** haakonn_ has joined #bitcoin-core-dev
1682017-02-25T05:59:35 <cfields> sipa: well i guess that discussion has been rendered moot. assert for 0.14, and do an efficient specialized unsigned char* prevector for master?
1692017-02-25T05:59:49 <sipa> cfields: ok
1702017-02-25T06:20:04 *** chjj has joined #bitcoin-core-dev
1712017-02-25T06:54:11 *** roasbeef_ is now known as roasbeef
1722017-02-25T07:00:04 *** dermoth has quit IRC
1732017-02-25T07:01:01 *** dermoth has joined #bitcoin-core-dev
1742017-02-25T07:09:06 *** _mn3monic has joined #bitcoin-core-dev
1752017-02-25T07:12:07 *** lclc has joined #bitcoin-core-dev
1762017-02-25T07:15:22 *** nickler has quit IRC
1772017-02-25T07:17:10 *** lclc has quit IRC
1782017-02-25T07:20:15 *** nickler has joined #bitcoin-core-dev
1792017-02-25T07:36:50 *** lclc has joined #bitcoin-core-dev
1802017-02-25T07:45:15 *** lclc has quit IRC
1812017-02-25T07:46:07 <wump> bah, #9856 is really ugly :/
1822017-02-25T07:46:09 <gribble> https://github.com/bitcoin/bitcoin/issues/9856 | Terminate immediately when allocation fails by theuni · Pull Request #9856 · bitcoin/bitcoin · GitHub
1832017-02-25T07:46:15 *** wump is now known as wumpus
1842017-02-25T07:47:19 *** lclc has joined #bitcoin-core-dev
1852017-02-25T07:48:35 *** fanquake1 has quit IRC
1862017-02-25T07:52:00 <wumpus> I understand wanting to terminate immediately on a memory error in some cases, but can't you just catch the exception at some point instead of registereing a global handler?
1872017-02-25T07:52:43 <sipa> wumpus: the problem is exactly that it is being caught... and then we continue in an inconsistent state
1882017-02-25T07:52:59 <wumpus> well then crash the program at the catch site!
1892017-02-25T07:53:12 <sipa> that's what this does
1902017-02-25T07:53:34 <sipa> tell the program to crash whenever new fails
1912017-02-25T07:53:39 <wumpus> no, this uses some poorly documented feature of c++ to change a global callback for every failed allocation
1922017-02-25T07:53:48 <wumpus> instead of just in the consensus code or wherever it is actually dangerous
1932017-02-25T07:55:06 <wumpus> some allocation errors, either in bitcoin itself or in libraries may actually be recoverable, and they would now cause a crash
1942017-02-25T07:56:13 <sipa> oh, sure, longer term i think we can find something more specific
1952017-02-25T07:56:39 <sipa> but as long as we have "catch std::exception& e" everywhere, which catches everything, that's pretty hard
1962017-02-25T07:56:42 *** lclc has quit IRC
1972017-02-25T07:56:54 <wumpus> I think this is a bad direction to go in with the code base
1982017-02-25T07:56:59 <wumpus> shoudln't we be addressing that then?
1992017-02-25T07:57:14 <sipa> in 0.14?
2002017-02-25T07:57:28 *** nohup has joined #bitcoin-core-dev
2012017-02-25T07:57:40 <wumpus> for 0.14 this is ok, but this is a PR to master
2022017-02-25T07:58:17 <sipa> hmm, you think this should go in just 0.14?
2032017-02-25T07:58:28 *** lclc has joined #bitcoin-core-dev
2042017-02-25T07:58:42 <wumpus> the way the PR is now? yes, I think so. This just too aspecific an hack
2052017-02-25T07:58:47 <sipa> i think terminating when OOM is a very reasonable thing, and certainly more reasonable than trying to continuing
2062017-02-25T07:59:01 <wumpus> I think terminating on OOM in select cases is a reasonable thing
2072017-02-25T08:00:07 <sipa> there may be more user-friendly things to do in some cases, but it's very hard... allocation failures can occur pretty much everywhere, and it seems really hard to avoid inconsistent states
2082017-02-25T08:00:36 <wumpus> sure, it's difficult
2092017-02-25T08:00:45 <wumpus> but this just feels like giving up on any kind of sane error handling
2102017-02-25T08:01:20 <sipa> i don't think so... OOM is different from disk errors, for example
2112017-02-25T08:01:29 <sipa> you can cleanly recover from that, usually
2122017-02-25T08:01:33 <wumpus> I really dislike global handlers like this, and I don't think much software uses this
2132017-02-25T08:01:52 <wumpus> it just changes the expectation of the c++ standard that allocations return an exception
2142017-02-25T08:02:17 <wumpus> which is adding surprising behavior at a low level
2152017-02-25T08:04:27 <sipa> i'm surprised to read that
2162017-02-25T08:04:46 <gmaxwell> on many systems allocation failures are just silently ignored and the program just crashes when it tries to use the memory.
2172017-02-25T08:04:52 *** zrts_ has joined #bitcoin-core-dev
2182017-02-25T08:04:59 <sipa> i think that most software just doesn't try to catch bad_alloc
2192017-02-25T08:05:13 <sipa> and it just crashes
2202017-02-25T08:05:18 <gmaxwell> In virtually none of the software could an allocation even possibly be handled, esp as almost everything we do involves allocation.
2212017-02-25T08:05:44 <wumpus> ok ,never mind
2222017-02-25T08:06:00 <wumpus> seems you all agree, just do this
2232017-02-25T08:06:35 <gmaxwell> but I agree that just fixing the badalloc is not a full solution. What about other exceptions? If bad alloc can cause spurrious block invalidations how long until the next thing.
2242017-02-25T08:07:17 <sipa> well, i think we should get rid of the catch std::exception and catch ... everywhere
2252017-02-25T08:07:20 <wumpus> yes if the code is not exception-safe, that should be addressed
2262017-02-25T08:07:28 <sipa> and only catch expected exceptions
2272017-02-25T08:07:32 <gmaxwell> So I am of the opinion that bad alloc should just crash but also that the software needs to deal better with unexpected exceptions in block validation... as I mentioned before this is the third case of exceptions causing bogus block rejection that we've had.
2282017-02-25T08:07:56 <sipa> but by doing that, we'd still just crash in case of an OOM
2292017-02-25T08:08:06 <sipa> at least with this PR, we write to debug.log what happened
2302017-02-25T08:08:26 <wumpus> but unlike AbortNode() no attempt is made to show a dialog to the user
2312017-02-25T08:08:47 <gmaxwell> can't show a dialog without allocating.
2322017-02-25T08:09:06 <sipa> well in many cases, that would be possible
2332017-02-25T08:09:06 <wumpus> well the point is: maybe it needs less allocation than the original allocation request
2342017-02-25T08:09:10 <wumpus> not all allocations are equal
2352017-02-25T08:09:13 <sipa> indeed
2362017-02-25T08:09:22 <sipa> typically those that trip an OOM are large ones
2372017-02-25T08:09:32 <wumpus> someone might be trying to allocate a 4GB buffer, which fails, then after that the program could continue keeping in mind that it can't have that buffer
2382017-02-25T08:09:37 <gmaxwell> That is a fair point. Also if it did something like shut down the mempool it might free enough to display a message.
2392017-02-25T08:10:14 <gmaxwell> I wonder what the distribution of vm_overcommit settings is, it used to be ubiquitious, the fact that sipa even saw this suggests that it isn't anymore.
2402017-02-25T08:11:04 <sipa> for this specific issue, it may be possible to catch the bad_alloc inside the leveldb wrapper, and if so, mark the db object as read-only
2412017-02-25T08:11:11 <gmaxwell> apparently some hurestic is now the default.
2422017-02-25T08:11:16 <wumpus> in many cases showing a simple modal dialog woudl still work, especially on windows where it's sort of a direct sytem operation. It should at least we attempted
2432017-02-25T08:11:24 <sipa> that's fair
2442017-02-25T08:11:39 <sipa> but i think it requires too many changes for 0.14
2452017-02-25T08:11:42 <gmaxwell> I have no issue with that, so long as if it fails the system still shuts down.
2462017-02-25T08:11:42 <wumpus> and we have a general 'A fatal issuee has occured check debug.log' already in AbortNode
2472017-02-25T08:12:16 <sipa> i really want to think harder about possible inconsistent states
2482017-02-25T08:12:27 <wumpus> well an allocation failure in qt will just propagate the std::alloc_error
2492017-02-25T08:12:31 <wumpus> qt is exception safe
2502017-02-25T08:12:41 <gmaxwell> sipa: I'm kinda surprised that cfield's patch works. IIRC from gdbing boost asio it used to try to allocate 4TB of memory and the fail. I'm surprised nothing else in boost does.
2512017-02-25T08:12:57 <wumpus> so just catch that and abort() if there is one
2522017-02-25T08:13:03 <sipa> gmaxwell: it's c++11
2532017-02-25T08:13:07 <sipa> boost does not assume c++11
2542017-02-25T08:13:29 <sipa> for example, do we want to prevent writing to mempool.dat after some unhandled system error, to prevent corrupting it?
2552017-02-25T08:14:02 <wumpus> yes
2562017-02-25T08:14:16 <sipa> if catching of the exception occurs higher up, there is less risk of those things
2572017-02-25T08:14:19 *** chjj has quit IRC
2582017-02-25T08:14:26 <sipa> but what about writes that happen in unrelated background threads
2592017-02-25T08:14:34 <sipa> the wallet gets flushed in a background thread
2602017-02-25T08:14:44 <wumpus> set a flag and don't flush anything anymore
2612017-02-25T08:14:56 <wumpus> I mean all the databases are atomic right?
2622017-02-25T08:15:09 <wumpus> you can call writes, and as long as you don't flush, they'll be ignored on shutdown
2632017-02-25T08:15:28 <sipa> but a flag where? an exception can only propagate up to its threads' main function
2642017-02-25T08:15:36 <wumpus> a global flag
2652017-02-25T08:15:41 <wumpus> checked before each database flag
2662017-02-25T08:15:43 <wumpus> flush*
2672017-02-25T08:16:15 <sipa> if an OOM occurs inside the background runner thread, it will - if uncaught - just kill the background runner
2682017-02-25T08:16:18 <wumpus> a kind of alarm state, which makes sure nothing is permanently committed
2692017-02-25T08:16:36 <wumpus> catch the exception and call a handler to set that flag?
2702017-02-25T08:16:47 <sipa> that's really ugly...
2712017-02-25T08:17:34 <wumpus> why is that ugly? you should catch exceptions such as std::alloc anyway to warn the user, log something , and initiaite shutdown
2722017-02-25T08:17:38 <sipa> doesn't leveldb have some background writer thread?
2732017-02-25T08:17:58 <wumpus> why is leveldb's background thread a problem here?
2742017-02-25T08:18:06 <wumpus> we're concerned with corruptions of our own state, right?
2752017-02-25T08:18:22 <wumpus> leveldb won't be writing anything that we haven't committed
2762017-02-25T08:18:48 <wumpus> so as soon as there is the suspicion of inconsistent state, don't write anything to leveldb anymore
2772017-02-25T08:20:29 <sipa> i'm worried about an OOM caused by a thread we don't control
2782017-02-25T08:20:44 <wumpus> then the inconsistent state won't be committed to disk, and leveldb should be fine. Same for the wallet
2792017-02-25T08:20:45 <sipa> but i assume that would just cause a crash, as we can't catch a bad_alloc there anyway
2802017-02-25T08:21:26 <wumpus> well yes the writer thread could die in the middle of writing, sure, but there's no way to avoid that. The database needs t obe able to handle that, also for other crash robustness
2812017-02-25T08:22:19 <wumpus> what it won't do, however, is write inconsistent state if we haven't told it to
2822017-02-25T08:22:29 <sipa> yes, indeed
2832017-02-25T08:23:34 <sipa> i think you're right - with some changes it may be possible to do this safely
2842017-02-25T08:23:53 <sipa> if we make sure there is no code that may unintentionally catch a bad_alloc
2852017-02-25T08:24:02 <wumpus> right
2862017-02-25T08:24:12 <sipa> and that all the catches that do, also call AbortNode, which sets this flag
2872017-02-25T08:24:24 <wumpus> I think there should be a rule: if you catch a general std::exception, you can't continue there, and need to stop the program
2882017-02-25T08:24:47 <wumpus> as you have no idea what kind of error you have and whether it's recoverable
2892017-02-25T08:24:51 <sipa> right
2902017-02-25T08:24:58 <wumpus> (so that's for top-level handlers only)
2912017-02-25T08:25:52 <sipa> we have 53 places where std::exception is caught
2922017-02-25T08:26:06 <sipa> and 22 where ... is caught (even worse, as you can't even inspect the object)
2932017-02-25T08:26:13 <wumpus> the ptential worry is in the networking code... some parsing problems throw exceptions, we need to be really sure to catch them and continue otherwise there's a DoS
2942017-02-25T08:26:56 <wumpus> e.g. serialization throws std::runtime_error instead of something specific
2952017-02-25T08:27:29 <sipa> serialization throws std::ios_base
2962017-02-25T08:27:36 <sipa> serialization throws std::ios_base::failure
2972017-02-25T08:27:59 <wumpus> okay That's always an i/o error. Better than I thought
2982017-02-25T08:28:12 <sipa> there are some other things that can throw that
2992017-02-25T08:28:31 <wumpus> catching that and just reporting it and continuing in the network code would be ok
3002017-02-25T08:28:52 <sipa> network code or net_handling?
3012017-02-25T08:29:06 *** panicstr has joined #bitcoin-core-dev
3022017-02-25T08:29:18 <sipa> net handling should catch serialization failures, and disconnect the offending peer
3032017-02-25T08:29:29 <wumpus> well the packet processing. It should make sure that one node is disconnected and not thewhole program comes tumbling down
3042017-02-25T08:29:35 <wumpus> yes
3052017-02-25T08:29:43 <sipa> right
3062017-02-25T08:29:53 <sipa> in fact, it's that except catch this is the culprit here
3072017-02-25T08:30:02 <sipa> *that exact
3082017-02-25T08:30:24 <wumpus> right, it should be more specific
3092017-02-25T08:30:42 <wumpus> catching ios::base and potentially a few others instead of std::exception would go a long way
3102017-02-25T08:30:48 <sipa> agree
3112017-02-25T08:33:22 <wumpus> and some exceptions like std::bad_alloc should initiate a quick shutdown. It could still be a DoS, but the program is probably in such a bad state when it happens that continuing with the other nodes is hopeless.
3122017-02-25T08:33:53 <wumpus> (though theoretically it could stil be a botched alloc due to the remote peer specifying a very large buffer which would never occur in reality)
3132017-02-25T08:36:44 <wumpus> so I guess uploading executables for rc2 is no longer necessary and we should merge #9856 and do rc3?
3142017-02-25T08:36:45 <gribble> https://github.com/bitcoin/bitcoin/issues/9856 | Terminate immediately when allocation fails by theuni · Pull Request #9856 · bitcoin/bitcoin · GitHub
3152017-02-25T08:42:58 <wumpus> speaking about memory usage: another report of running out of memory with 0.14. #9857
3162017-02-25T08:42:59 <gribble> https://github.com/bitcoin/bitcoin/issues/9857 | Sync problem crashes Bitcoin: EXCEPTION: St9bad_alloc · Issue #9857 · bitcoin/bitcoin · GitHub
3172017-02-25T08:43:09 <wumpus> maybe 0.14 increased steady-state memory usage?
3182017-02-25T08:49:22 <sipa> maybe
3192017-02-25T08:49:37 <sipa> the reuse of mempool memory by coincache may contribute to it
3202017-02-25T08:49:59 <sipa> though that doesn't affect worst-case memory usage, it may increase the average
3212017-02-25T08:50:02 *** lclc has quit IRC
3222017-02-25T08:50:55 <sipa> for RPi we should recommend lowering the default settings, i'm afraid
3232017-02-25T08:51:58 *** lclc has joined #bitcoin-core-dev
3242017-02-25T08:56:43 <wumpus> bah, what really helps when creating a gdb parachute is propagating the exit code, if not it will just always return success https://github.com/bitcoin/bitcoin/pull/9851/commits/6d7a789fb510d540906955c377143b92acb1fbb6 . Forgot this and might have been spinning for nothing for a while :p
3252017-02-25T09:05:32 *** lclc has quit IRC
3262017-02-25T09:07:02 <wumpus> uhm. Caught that issue in the act now. Unfortunately, the traceback is useless. Gdb near-crashes while generating it!
3272017-02-25T09:07:10 *** lclc has joined #bitcoin-core-dev
3282017-02-25T09:07:13 *** nabu has joined #bitcoin-core-dev
3292017-02-25T09:07:27 <wumpus> "Error while running hook_stop: Cannot access memory at address 0x24"
3302017-02-25T09:08:01 <wumpus> okay, so much for gdb...
3312017-02-25T09:11:07 <wumpus> that leaves trying to make it cough up a core dump, and uploading that somewhere...
3322017-02-25T09:12:22 <jeremyrubin> wumpus: cuckoocache in theory may have increase memory usage a bit
3332017-02-25T09:13:06 <jeremyrubin> (if you're curious for all steadystate memory usage increases)
3342017-02-25T09:13:29 <wumpus> by how much?
3352017-02-25T09:13:35 <jeremyrubin> unclear
3362017-02-25T09:14:00 <jeremyrubin> it always uses 100% of whatever's it's allocated
3372017-02-25T09:14:25 <jeremyrubin> whereas the previous one only uses space when it has entries (in theory)
3382017-02-25T09:14:42 <sipa> the default max is 32 MB instead of 40 MB though
3392017-02-25T09:14:47 <sipa> so at peak, it's smaller
3402017-02-25T09:14:48 <jeremyrubin> so at startup, would be using 32 MB whereas the prior was 0
3412017-02-25T09:15:53 <jeremyrubin> it's just unclear how long you have to be on the network to experience a full cache w/ old code
3422017-02-25T09:16:32 <jeremyrubin> anyways I'm pretty sure this isn't the cause :)
3432017-02-25T09:17:23 <sipa> 1GB of memory is not much
3442017-02-25T09:18:10 <wumpus> indeed, runninga node with 1GB of memory has always required tweaking
3452017-02-25T09:18:26 <sipa> it's more surprising to me that 0.13.2 worked, though
3462017-02-25T09:18:33 <sipa> he's even reducing dbcache
3472017-02-25T09:18:47 *** lclc has quit IRC
3482017-02-25T09:19:04 <wumpus> I do see increased reporting of out-of-memory errors with 0.14
3492017-02-25T09:19:41 <jeremyrubin> yeah, totally could be the sigcache, could definitely put someone over-edge
3502017-02-25T09:19:49 <jeremyrubin> especially if they set the value higher
3512017-02-25T09:19:57 <jeremyrubin> e.g., 128 mb
3522017-02-25T09:20:14 <jeremyrubin> prior version you could set it to a GB and run happily for a while
3532017-02-25T09:20:28 <wumpus> they don't touch that setting, just dbcache
3542017-02-25T09:20:43 <wumpus> possibly they should. We should update the recommedations for low-memory nodes.
3552017-02-25T09:21:28 <jeremyrubin> I don't think practically they'd need to adjust it down, just be less agressive adjusting it up -- not much benefit in making it too large
3562017-02-25T09:22:32 <jeremyrubin> Can we (by any chance) fast track sipa's hack to get cuckoocache back to be non pow 2?
3572017-02-25T09:23:14 <jeremyrubin> it sucks if they're picking between 8 16 32 MB sizes, some more resolution there would be good
3582017-02-25T09:23:40 <wumpus> which # is that?
3592017-02-25T09:24:09 <jeremyrubin> #9533
3602017-02-25T09:24:11 <gribble> https://github.com/bitcoin/bitcoin/issues/9533 | Allow non-power-of-2 signature cache sizes by sipa · Pull Request #9533 · bitcoin/bitcoin · GitHub
3612017-02-25T09:26:05 <wumpus> jeremyrubin: would help to have more than a concept ACK there from you
3622017-02-25T09:26:43 <jeremyrubin> k I'll do some more thorough review
3632017-02-25T09:26:47 <wumpus> thanks
3642017-02-25T09:28:59 <wumpus> also like gmaxwell I wonder what the performance difference would be, 8 multiplication would be slower than 8 logical AND
3652017-02-25T09:31:37 <jeremyrubin> I'm not currently set up to run great performance tests currently (will take some work to reset up my benching environment)
3662017-02-25T09:31:57 <jeremyrubin> But honestly performance was fine even when it was %
3672017-02-25T09:32:11 <wumpus> 'fine' hehe
3682017-02-25T09:32:17 <jeremyrubin> *shrug*
3692017-02-25T09:32:20 <sipa> jeremyrubin: i'm sure that for the sigcache it doesn't matter
3702017-02-25T09:32:33 <sipa> but maybe it does for other things in the future
3712017-02-25T09:32:36 <wumpus> well it's obviously a compromise between memory usage and performance
3722017-02-25T09:32:47 <jeremyrubin> I mostly changed it to hash_mask to make sipa happy :p
3732017-02-25T09:33:02 <wumpus> neither '&' or '* >>' is 'better', but we need to be careful what to choose
3742017-02-25T09:33:15 <sipa> well you violated the core tenet of never using integer division by a variable!
3752017-02-25T09:33:23 <sipa> it invokes the wrath of maxwell
3762017-02-25T09:33:31 <jeremyrubin> It was checked to not be 0 ;)
3772017-02-25T09:33:58 <jeremyrubin> anyways...
3782017-02-25T09:34:04 <wumpus> division? where?
3792017-02-25T09:34:18 <jeremyrubin> (old version of cuckoocache used % in the hash computation)
3802017-02-25T09:34:33 <wumpus> I saw only multiplies, if there's a division, I'm going to NACK this one
3812017-02-25T09:34:33 <wumpus> ooh :p
3822017-02-25T09:35:08 <sipa> and that PR brings back the flexibility of variable sizes, but with a * and a >> instead of a %
3832017-02-25T09:35:22 <jeremyrubin> I recall benchmarking and not being able to see a difference between %, &, >>
3842017-02-25T09:35:39 <wumpus> yes using % would be more straightforward code readability wise but division is even worse than multiplication
3852017-02-25T09:35:44 <jeremyrubin> yeah
3862017-02-25T09:35:54 <jeremyrubin> so I think it's fairly safe to take this one.
3872017-02-25T09:36:00 <wumpus> yes
3882017-02-25T09:36:07 <jeremyrubin> My one suggestion sipa is to convert size to a uint64_t
3892017-02-25T09:36:11 <jeremyrubin> like the member
3902017-02-25T09:36:20 <jeremyrubin> so you can get rid of all the casts in the hash part
3912017-02-25T09:36:34 <wumpus> would this result in more efficient code on 32-bit though?
3922017-02-25T09:36:41 <wumpus> as it guarantees that the top bits are zero
3932017-02-25T09:36:50 <jeremyrubin> I'm not sure
3942017-02-25T09:37:01 <jeremyrubin> but they have to be cast to uint64_t in hashing part now
3952017-02-25T09:37:17 <jeremyrubin> so my thought is not having to do that cast every hash is slightly more efficient code
3962017-02-25T09:37:34 <wumpus> casts don't generally generate code
3972017-02-25T09:37:34 *** panicstr has quit IRC
3982017-02-25T09:37:48 <jeremyrubin> uint32_t to uint64_t even?
3992017-02-25T09:37:59 <jeremyrubin> there are instructions that auto-0 it to the alu?
4002017-02-25T09:38:02 <jeremyrubin> that's fine then
4012017-02-25T09:38:14 <wumpus> no. on 32-bit, it just means 'assume top bits are 0'. On 64 bit evrything is stored in 64-bit registers and the top bits are always 0
4022017-02-25T09:38:37 <jeremyrubin> cool, then that's fine as is
4032017-02-25T09:38:43 <wumpus> at least on x86_64 you can't load something into the lower 32 bits without zeroing the upper bits
4042017-02-25T09:39:11 <jeremyrubin> I'd consider it an Ack from me then.
4052017-02-25T09:39:59 <jeremyrubin> The tests are pretty agressive on cuckoocache, so if the hashing idea were broken in some way it would either catch it or it would be broken in a very minor way.
4062017-02-25T09:40:00 <wumpus> of course the considereations depend on the specific architecture, but in general casts are very cheap or generate no extra code at all especially between big types. Arithmetic with 8-bit and 16-bit values can result in extra 'ANDs' in some architectures you're right
4072017-02-25T09:40:18 <wumpus> but then it's not so much the cast that generates the code
4082017-02-25T09:40:53 <wumpus> yes, having good tests is great!
4092017-02-25T09:41:40 <wumpus> the state of testing in bitcoin core improved a lot last year
4102017-02-25T09:42:06 <jeremyrubin> *cough* merge my checkqueue tests *cough* :P
4112017-02-25T09:42:21 <wumpus> which reminds me I should merge #9847
4122017-02-25T09:42:23 <gribble> https://github.com/bitcoin/bitcoin/issues/9847 | Extra test vector for BIP32 by sipa · Pull Request #9847 · bitcoin/bitcoin · GitHub
4132017-02-25T09:42:26 <wumpus> jeremyrubin: oh, aren't they yet?
4142017-02-25T09:42:54 <jeremyrubin> #9497
4152017-02-25T09:42:56 <gribble> https://github.com/bitcoin/bitcoin/issues/9497 | CCheckQueue Unit Tests by JeremyRubin · Pull Request #9497 · bitcoin/bitcoin · GitHub
4162017-02-25T09:42:58 <wumpus> usually test-only pulls are merged very quickly
4172017-02-25T09:43:02 *** d9b4bef9 has quit IRC
4182017-02-25T09:43:25 <wumpus> then again I cannot really keep up anymore with the volume of pulls
4192017-02-25T09:43:52 <jeremyrubin> It's actually fine that they weren't, I found a mixed up loop condition (such that one test wasn't doing much) when I was sitting with kalle
4202017-02-25T09:43:55 <wumpus> which is good and bad
4212017-02-25T09:43:59 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f19afdbfb4cb...6206252e5073
4222017-02-25T09:44:00 <bitcoin-git> bitcoin/master 30aedcb Pieter Wuille: BIP32 extra test vector
4232017-02-25T09:44:00 <bitcoin-git> bitcoin/master 6206252 Wladimir J. van der Laan: Merge #9847: Extra test vector for BIP32...
4242017-02-25T09:44:01 <sipa> but division is even worse than multiplication -> nehalem can issue a 64-bit mul every 2 cycles; a division every 19-69 cycles
4252017-02-25T09:44:08 *** d9b4bef9 has joined #bitcoin-core-dev
4262017-02-25T09:44:14 <jeremyrubin> But that's all fixed now
4272017-02-25T09:44:19 <bitcoin-git> [bitcoin] laanwj closed pull request #9847: Extra test vector for BIP32 (master...bip32up) https://github.com/bitcoin/bitcoin/pull/9847
4282017-02-25T09:44:33 <wumpus> sipa: yeah, that would classify as 'bad' in my eyes :)
4292017-02-25T09:44:37 <sipa> wumpus: the mul or the div?
4302017-02-25T09:44:40 <wumpus> the div
4312017-02-25T09:44:46 <jeremyrubin> nehalem?
4322017-02-25T09:44:51 <sipa> yes, absolutely
4332017-02-25T09:45:08 <sipa> but 'even worse' sounds like multiplication is already pretty bad... every 2 cycles is pretty good i'd say :)
4342017-02-25T09:45:23 <wumpus> divs have historically been really bad, some CPUs don't even have a specific instruction for it, but even those that have it's generally terrible
4352017-02-25T09:45:42 <wumpus> because of the underlying computational structure of course, not because those CPUs are badly implemented or so :)
4362017-02-25T09:45:49 <jeremyrubin> ah, nehalem: an intel arch name I'd thoroughly forgotten :)
4372017-02-25T09:46:32 <wumpus> 2 cycles is pretty good, espeically if it's *every* 2 cycles
4382017-02-25T09:46:47 <jeremyrubin> and that hash routine should pipeline well
4392017-02-25T09:46:54 <sipa> the latency is higher than 2 cycles
4402017-02-25T09:46:55 <wumpus> usually mul is quite quick but the number of multiplication units is limited, so issuing a lot will bottleneck on that
4412017-02-25T09:47:01 <wumpus> but not on nehalem, apparently :)
4422017-02-25T09:47:05 <sipa> but the throughput is once every 2 cycles
4432017-02-25T09:47:19 <sipa> same on skylaky, btw
4442017-02-25T09:47:22 <wumpus> what would be the throughput for logical and?
4452017-02-25T09:47:53 <sipa> 0.5 or 1, depending on the type of inputs
4462017-02-25T09:47:54 <wumpus> can't be much faster than per 2 cycles
4472017-02-25T09:47:55 <wumpus> oh!
4482017-02-25T09:48:05 <sipa> 0.25 for and with a constant, even
4492017-02-25T09:48:21 <sipa> http://www.agner.org/optimize/instruction_tables.pdf
4502017-02-25T09:48:22 <wumpus> modern CPU architectures are so fast we can almost ignore computation overhead. Too bad memory didn't keep up :)
4512017-02-25T09:48:42 <jeremyrubin> Also if you want to really make that hashing faster, could switch to balke2b or something
4522017-02-25T09:49:34 <wumpus> sipa: thanks , that seems like a very useful document
4532017-02-25T09:50:09 <jeremyrubin> wumpus: +1 thanks!
4542017-02-25T09:50:58 <sipa> LOCK ADD... 18 cycles!
4552017-02-25T09:51:15 *** Ylbam has joined #bitcoin-core-dev
4562017-02-25T09:53:25 <wumpus> ieeeh
4572017-02-25T09:57:38 <jeremyrubin> so I'm trying to write some test software
4582017-02-25T09:57:47 <jeremyrubin> and I want to force segwit active
4592017-02-25T09:58:05 <jeremyrubin> seems that isn't trivial to do?
4602017-02-25T09:58:10 <sipa> regtest?
4612017-02-25T09:59:19 <jeremyrubin> it seems to still require some blocks to vote for it before it can be made active?
4622017-02-25T09:59:36 <sipa> yes
4632017-02-25T09:59:55 <sipa> 3*144 blocks
4642017-02-25T10:00:47 <jeremyrubin> hmm... would be kinda nice to not have to do that, similar to how I can set BIPxxxHeight = 0.
4652017-02-25T10:01:39 *** lclc has joined #bitcoin-core-dev
4662017-02-25T10:06:35 *** MarcoFalke has joined #bitcoin-core-dev
4672017-02-25T10:08:00 *** jl2012 has quit IRC
4682017-02-25T10:14:13 <jeremyrubin> would it be safe to change VersionBitsState to read VersionBitsDeploymentInfo for an override state parameter?
4692017-02-25T10:17:10 *** lclc has quit IRC
4702017-02-25T10:19:02 <jeremyrubin> no I guess I'd neet to change GetStateFor
4712017-02-25T10:19:06 *** lclc has joined #bitcoin-core-dev
4722017-02-25T10:34:04 *** Squidicc has joined #bitcoin-core-dev
4732017-02-25T10:34:55 *** lclc has quit IRC
4742017-02-25T10:36:42 *** Squidicuz has quit IRC
4752017-02-25T10:36:50 *** lclc has joined #bitcoin-core-dev
4762017-02-25T10:39:19 *** nohup has quit IRC
4772017-02-25T10:46:53 *** Victor_sueca is now known as Victorsueca
4782017-02-25T10:50:53 *** panicstr has joined #bitcoin-core-dev
4792017-02-25T11:01:22 *** lclc has quit IRC
4802017-02-25T11:08:13 *** MarcoFalke has left #bitcoin-core-dev
4812017-02-25T11:22:43 *** MarcoFalke has joined #bitcoin-core-dev
4822017-02-25T11:35:20 *** panicstr has quit IRC
4832017-02-25T11:56:11 *** lclc has joined #bitcoin-core-dev
4842017-02-25T12:05:08 *** lclc has quit IRC
4852017-02-25T12:06:57 *** lclc has joined #bitcoin-core-dev
4862017-02-25T12:13:59 *** AaronvanW has quit IRC
4872017-02-25T12:14:16 *** AaronvanW has joined #bitcoin-core-dev
4882017-02-25T12:14:16 *** AaronvanW has joined #bitcoin-core-dev
4892017-02-25T12:17:36 *** AaronvanW has quit IRC
4902017-02-25T12:18:10 *** AaronvanW has joined #bitcoin-core-dev
4912017-02-25T12:18:10 *** AaronvanW has joined #bitcoin-core-dev
4922017-02-25T12:29:01 *** alpalp has joined #bitcoin-core-dev
4932017-02-25T12:29:01 *** alpalp has joined #bitcoin-core-dev
4942017-02-25T12:31:41 *** wudayoda has quit IRC
4952017-02-25T12:33:51 *** wudayoda has joined #bitcoin-core-dev
4962017-02-25T12:42:13 <btcdrak> jeremyrubin you could just turn down the number of blocks required on regtest from 144 to something lower like 10.
4972017-02-25T12:51:52 *** lclc has quit IRC
4982017-02-25T13:01:53 *** jl2012 has joined #bitcoin-core-dev
4992017-02-25T13:28:16 *** AaronvanW has quit IRC
5002017-02-25T13:28:17 *** Aaronvan_ has joined #bitcoin-core-dev
5012017-02-25T13:54:09 *** vFSgrcFGBJHg has joined #bitcoin-core-dev
5022017-02-25T13:55:15 *** lclc has joined #bitcoin-core-dev
5032017-02-25T14:00:02 *** lclc has quit IRC
5042017-02-25T14:02:25 *** lclc has joined #bitcoin-core-dev
5052017-02-25T14:06:02 *** Aaronvan_ has quit IRC
5062017-02-25T14:06:36 *** AaronvanW has joined #bitcoin-core-dev
5072017-02-25T14:06:36 *** AaronvanW has joined #bitcoin-core-dev
5082017-02-25T14:11:12 *** AaronvanW has quit IRC
5092017-02-25T14:14:46 *** lclc has quit IRC
5102017-02-25T14:16:02 *** alpalp has quit IRC
5112017-02-25T14:16:59 *** lclc has joined #bitcoin-core-dev
5122017-02-25T14:32:07 *** AaronvanW has joined #bitcoin-core-dev
5132017-02-25T14:32:08 *** AaronvanW has joined #bitcoin-core-dev
5142017-02-25T14:38:28 *** Guyver2 has joined #bitcoin-core-dev
5152017-02-25T14:46:46 *** panicstr has joined #bitcoin-core-dev
5162017-02-25T14:51:56 *** panicstr has quit IRC
5172017-02-25T15:03:20 *** arowser has quit IRC
5182017-02-25T15:04:56 *** arowser has joined #bitcoin-core-dev
5192017-02-25T15:13:22 *** alpalp has joined #bitcoin-core-dev
5202017-02-25T15:24:58 *** alpalp has quit IRC
5212017-02-25T15:30:08 *** alpalp has joined #bitcoin-core-dev
5222017-02-25T15:30:08 *** alpalp has joined #bitcoin-core-dev
5232017-02-25T15:34:35 *** Giszmo has joined #bitcoin-core-dev
5242017-02-25T15:42:40 *** MarcoFalke has quit IRC
5252017-02-25T15:45:25 <cfields> woohoo, 0.13 ec2 sync finally finished :)
5262017-02-25T15:51:16 *** AaronvanW has quit IRC
5272017-02-25T15:58:45 <cfields> for those interested... ec2 xlarge 4core, 16gb ram. All default settings:
5282017-02-25T15:59:02 <cfields> v0.13.2: 01:12:39:43
5292017-02-25T15:59:02 <cfields> v0.14.0: 00:06:24:17
5302017-02-25T16:01:09 <paveljanik> nice!
5312017-02-25T16:03:53 <cfields> Setting a reasonable dbcache cuts the 0.14 time in half. Unsure what it does for 0.13. Similar, I assume
5322017-02-25T16:05:02 <cfields> Actually no... 0.13 is doing lots more validation. So it'd still be much more than half.
5332017-02-25T16:05:36 *** AaronvanW has joined #bitcoin-core-dev
5342017-02-25T16:10:43 *** lclc has quit IRC
5352017-02-25T16:18:05 *** str4d has joined #bitcoin-core-dev
5362017-02-25T16:48:24 <wumpus> FYI rc2 executables are online: https://bitcoin.org/bin/bitcoin-core-0.14.0/test.rc2/
5372017-02-25T17:06:36 *** AaronvanW has quit IRC
5382017-02-25T17:06:47 <Lauda> sync is faster with 0.14.0?
5392017-02-25T17:10:26 *** AaronvanW has joined #bitcoin-core-dev
5402017-02-25T17:10:27 *** AaronvanW has joined #bitcoin-core-dev
5412017-02-25T17:15:23 *** Squidicc is now known as squidicuz
5422017-02-25T17:41:22 *** alpalp has quit IRC
5432017-02-25T18:14:27 *** alpalp has joined #bitcoin-core-dev
5442017-02-25T18:14:28 *** alpalp has joined #bitcoin-core-dev
5452017-02-25T18:54:22 *** Chris_Stewart_5 has quit IRC
5462017-02-25T18:57:22 *** CodeShark has quit IRC
5472017-02-25T18:57:23 *** GreenIsMyPepper has quit IRC
5482017-02-25T18:58:26 *** CodeShark has joined #bitcoin-core-dev
5492017-02-25T18:59:56 *** GreenIsMyPepper has joined #bitcoin-core-dev
5502017-02-25T19:11:03 *** Chris_Stewart_5 has joined #bitcoin-core-dev
5512017-02-25T19:23:49 *** LeMiner has joined #bitcoin-core-dev
5522017-02-25T19:31:23 *** wudayoda has quit IRC
5532017-02-25T19:32:00 *** wudayoda has joined #bitcoin-core-dev
5542017-02-25T19:57:47 <bitcoin-git> [bitcoin] jameshilliard opened pull request #9858: remove TestBlockValidity from CreateNewBlock critical path (master...remove-tbv) https://github.com/bitcoin/bitcoin/pull/9858
5552017-02-25T20:16:45 *** laurentmt has joined #bitcoin-core-dev
5562017-02-25T20:36:30 *** chjj has joined #bitcoin-core-dev
5572017-02-25T20:39:30 *** chjj has quit IRC
5582017-02-25T20:39:52 *** chjj has joined #bitcoin-core-dev
5592017-02-25T20:54:18 *** crudel has quit IRC
5602017-02-25T20:57:55 *** justan0theruser has joined #bitcoin-core-dev
5612017-02-25T21:00:14 *** str4d has quit IRC
5622017-02-25T21:01:02 *** justanotheruser has quit IRC
5632017-02-25T21:01:53 *** lclc has joined #bitcoin-core-dev
5642017-02-25T21:08:01 *** AaronvanW has quit IRC
5652017-02-25T21:11:47 *** laurentmt has quit IRC
5662017-02-25T21:12:04 *** AaronvanW has joined #bitcoin-core-dev
5672017-02-25T21:17:42 *** lclc has quit IRC
5682017-02-25T21:19:54 *** lclc has joined #bitcoin-core-dev
5692017-02-25T21:22:50 *** crudel has joined #bitcoin-core-dev
5702017-02-25T21:40:27 *** lclc has quit IRC
5712017-02-25T21:42:43 *** AaronvanW has quit IRC
5722017-02-25T21:44:57 *** owowo has quit IRC
5732017-02-25T21:48:48 <cfields> luke-jr: jinx :)
5742017-02-25T21:48:54 <luke-jr> ?
5752017-02-25T21:49:08 <cfields> luke-jr: 9858
5762017-02-25T21:54:11 *** owowo has joined #bitcoin-core-dev
5772017-02-25T21:54:20 <luke-jr> i c âº
5782017-02-25T22:09:11 *** wumpus has quit IRC
5792017-02-25T22:11:29 *** wumpus has joined #bitcoin-core-dev
5802017-02-25T22:19:29 *** owowo has quit IRC
5812017-02-25T22:24:45 <luke-jr> would be nice if everything that went into stable branches were daggy fixes.
5822017-02-25T22:25:35 <sipa> daggy?
5832017-02-25T22:27:33 <luke-jr> sipa: http://wiki.secondlife.com/wiki/Creating_a_version_control_repository#The_.22Daggy.22_Fix
5842017-02-25T22:29:04 *** Guyver2 has quit IRC
5852017-02-25T22:31:17 <luke-jr> basically means bugfix commits are parented to the bug-introducing commit, allowing a merge into each branch of the same fix commit
5862017-02-25T22:35:35 *** JackH has joined #bitcoin-core-dev
5872017-02-25T22:37:16 <gmaxwell> are really that many of our fixes ones where there is a obvious bug inducing commit which is recent enough for that to be useful?
5882017-02-25T22:41:34 <luke-jr> gmaxwell: maybe not. as a middle-ground, I typically just try to rebase to the most recent branching point that is a clean merge to master
5892017-02-25T22:42:01 <luke-jr> eg, everything I do now would be based on 9828f9a at the latest (if possible)
5902017-02-25T22:42:24 <luke-jr> (which I have simple-tagged as "branch-0.14" in my repo)
5912017-02-25T22:56:47 *** owowo has joined #bitcoin-core-dev
5922017-02-25T22:56:47 *** owowo has joined #bitcoin-core-dev
5932017-02-25T23:17:02 *** justanotheruser has joined #bitcoin-core-dev
5942017-02-25T23:19:27 *** justan0theruser has quit IRC