12018-06-09T00:03:25  *** Krellan has joined #bitcoin-core-dev
  22018-06-09T00:13:46  *** Krellan_ has joined #bitcoin-core-dev
  32018-06-09T00:14:21  *** Krellan__ has joined #bitcoin-core-dev
  42018-06-09T00:14:34  *** Krellan has quit IRC
  52018-06-09T00:15:14  *** Chris_Stewart_5 has joined #bitcoin-core-dev
  62018-06-09T00:18:44  *** Krellan_ has quit IRC
  72018-06-09T00:48:55  *** Chris_Stewart_5 has quit IRC
  82018-06-09T01:03:50  *** fanquake has joined #bitcoin-core-dev
  92018-06-09T01:19:12  *** nmnkgl has quit IRC
 102018-06-09T01:19:51  *** nmnkgl has joined #bitcoin-core-dev
 112018-06-09T01:24:22  *** nmnkgl has quit IRC
 122018-06-09T01:26:26  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 132018-06-09T01:29:45  <fanquake> Empact Am alive and well :p
 142018-06-09T01:35:27  *** Chris_Stewart_5 has quit IRC
 152018-06-09T01:40:54  *** jhfrontz has quit IRC
 162018-06-09T02:02:35  *** Krellan__ has quit IRC
 172018-06-09T02:08:34  *** drexl has quit IRC
 182018-06-09T02:17:23  *** nmnkgl has joined #bitcoin-core-dev
 192018-06-09T02:19:27  *** AaronvanW has quit IRC
 202018-06-09T02:31:04  *** snickerfritz has joined #bitcoin-core-dev
 212018-06-09T02:47:50  *** fanquake has quit IRC
 222018-06-09T02:54:58  *** pipi has joined #bitcoin-core-dev
 232018-06-09T02:57:12  *** pipi has quit IRC
 242018-06-09T03:24:42  *** nmnkgl has quit IRC
 252018-06-09T03:25:17  *** nmnkgl has joined #bitcoin-core-dev
 262018-06-09T03:29:51  *** nmnkgl has quit IRC
 272018-06-09T03:41:46  *** CubicEarths has quit IRC
 282018-06-09T03:52:55  *** CubicEarths has joined #bitcoin-core-dev
 292018-06-09T04:03:57  *** CubicEarths has quit IRC
 302018-06-09T04:30:37  *** Deadhandd has quit IRC
 312018-06-09T04:33:49  *** Deadhand has joined #bitcoin-core-dev
 322018-06-09T04:51:45  *** CubicEarths has joined #bitcoin-core-dev
 332018-06-09T05:34:02  *** d9b4bef9 has quit IRC
 342018-06-09T05:35:00  *** harrymm has quit IRC
 352018-06-09T05:35:07  *** d9b4bef9 has joined #bitcoin-core-dev
 362018-06-09T05:37:26  *** harrymm has joined #bitcoin-core-dev
 372018-06-09T05:37:51  <murrayn> Is there a reason -O2 is specifically enabled in configure with --enable-debug? Should this not be -Og?
 382018-06-09T06:10:11  *** bitconner has quit IRC
 392018-06-09T06:19:33  *** Randolf has joined #bitcoin-core-dev
 402018-06-09T06:49:57  *** Tralfaz has quit IRC
 412018-06-09T07:15:00  *** Guyver2 has joined #bitcoin-core-dev
 422018-06-09T07:21:36  *** sud_ has joined #bitcoin-core-dev
 432018-06-09T07:23:06  *** sud_ has quit IRC
 442018-06-09T07:23:42  *** CubicEarths has quit IRC
 452018-06-09T07:54:39  *** CubicEarths has joined #bitcoin-core-dev
 462018-06-09T08:07:26  *** bitconner has joined #bitcoin-core-dev
 472018-06-09T08:10:52  *** CubicEarths has quit IRC
 482018-06-09T08:11:22  *** Guyver2 has quit IRC
 492018-06-09T08:40:03  *** SopaXorzTaker has joined #bitcoin-core-dev
 502018-06-09T08:55:34  *** Madars has quit IRC
 512018-06-09T08:57:57  *** bitconner has quit IRC
 522018-06-09T09:01:05  *** bitconner has joined #bitcoin-core-dev
 532018-06-09T09:13:36  *** Madars has joined #bitcoin-core-dev
 542018-06-09T09:39:22  *** Madars has quit IRC
 552018-06-09T09:45:20  *** SopaXorzTaker has quit IRC
 562018-06-09T09:45:48  *** SopaXorzTaker has joined #bitcoin-core-dev
 572018-06-09T10:00:24  *** echonaut has quit IRC
 582018-06-09T10:00:42  *** echonaut has joined #bitcoin-core-dev
 592018-06-09T10:18:52  *** Madars has joined #bitcoin-core-dev
 602018-06-09T10:22:41  *** grafcaps has joined #bitcoin-core-dev
 612018-06-09T10:27:22  *** grafcaps has quit IRC
 622018-06-09T10:42:22  *** Madars has quit IRC
 632018-06-09T10:59:50  *** SopaXorzTaker has quit IRC
 642018-06-09T11:00:09  *** SopaXorzTaker has joined #bitcoin-core-dev
 652018-06-09T11:01:56  *** AaronvanW has joined #bitcoin-core-dev
 662018-06-09T11:03:52  *** Aaronvan_ has joined #bitcoin-core-dev
 672018-06-09T11:06:57  *** AaronvanW has quit IRC
 682018-06-09T11:08:01  *** d9b4bef9 has quit IRC
 692018-06-09T11:08:51  *** SopaXorzTaker has quit IRC
 702018-06-09T11:09:08  *** d9b4bef9 has joined #bitcoin-core-dev
 712018-06-09T11:09:47  *** promag has joined #bitcoin-core-dev
 722018-06-09T11:17:22  *** Madars has joined #bitcoin-core-dev
 732018-06-09T11:18:32  *** promag has quit IRC
 742018-06-09T11:19:01  *** promag has joined #bitcoin-core-dev
 752018-06-09T11:19:55  *** SopaXorzTaker has joined #bitcoin-core-dev
 762018-06-09T11:19:59  *** promag has quit IRC
 772018-06-09T11:20:22  *** promag has joined #bitcoin-core-dev
 782018-06-09T11:20:26  *** promag has quit IRC
 792018-06-09T11:20:47  *** jhfrontz has joined #bitcoin-core-dev
 802018-06-09T11:42:32  *** Madars has quit IRC
 812018-06-09T11:57:52  *** Victorsueca has quit IRC
 822018-06-09T11:59:14  *** Victorsueca has joined #bitcoin-core-dev
 832018-06-09T12:05:44  *** Aaronvan_ has quit IRC
 842018-06-09T12:06:20  *** AaronvanW has joined #bitcoin-core-dev
 852018-06-09T12:10:55  *** AaronvanW has quit IRC
 862018-06-09T12:22:29  *** Madars has joined #bitcoin-core-dev
 872018-06-09T12:39:49  *** games_ has quit IRC
 882018-06-09T12:41:10  *** drexl has joined #bitcoin-core-dev
 892018-06-09T12:43:14  *** AaronvanW has joined #bitcoin-core-dev
 902018-06-09T12:53:07  *** Madars has quit IRC
 912018-06-09T13:16:30  *** ula has joined #bitcoin-core-dev
 922018-06-09T13:20:33  *** bitconner has quit IRC
 932018-06-09T13:32:33  *** SopaXorzTaker has quit IRC
 942018-06-09T13:42:04  *** jcohen has joined #bitcoin-core-dev
 952018-06-09T13:43:52  *** Madars has joined #bitcoin-core-dev
 962018-06-09T13:54:09  *** jcohen has quit IRC
 972018-06-09T13:58:05  *** ken2812221 has joined #bitcoin-core-dev
 982018-06-09T14:07:50  *** Guyver2 has joined #bitcoin-core-dev
 992018-06-09T14:13:34  *** justanotheruser has quit IRC
1002018-06-09T14:15:31  *** jcohen has joined #bitcoin-core-dev
1012018-06-09T14:19:52  *** nekotribal has quit IRC
1022018-06-09T14:33:12  *** jcohen has quit IRC
1032018-06-09T14:56:15  *** jcohen has joined #bitcoin-core-dev
1042018-06-09T15:05:01  *** nekotribal has joined #bitcoin-core-dev
1052018-06-09T15:09:12  *** SopaXorzTaker has joined #bitcoin-core-dev
1062018-06-09T15:12:08  *** AaronvanW has quit IRC
1072018-06-09T15:19:46  *** jcohen has quit IRC
1082018-06-09T15:24:09  *** SopaXorzTaker has quit IRC
1092018-06-09T15:32:44  *** justanotheruser has joined #bitcoin-core-dev
1102018-06-09T15:37:37  *** andytoshi has quit IRC
1112018-06-09T15:38:26  *** Chris_Stewart_5 has joined #bitcoin-core-dev
1122018-06-09T15:49:58  *** SopaXorzTaker has joined #bitcoin-core-dev
1132018-06-09T15:58:30  *** justanotheruser has quit IRC
1142018-06-09T16:10:08  *** grafcaps has joined #bitcoin-core-dev
1152018-06-09T16:14:38  *** jcohen has joined #bitcoin-core-dev
1162018-06-09T16:21:39  *** Randolf has quit IRC
1172018-06-09T16:25:55  *** AaronvanW has joined #bitcoin-core-dev
1182018-06-09T16:34:09  <gmaxwell> sipa: linked on your SHANI pr is an implementation where someone else noticed the throuput/latency relationship that I noticed.. they also do a 4way and it's faster (by a small amount) than 2-way.
1192018-06-09T16:34:27  *** Chris_Stewart_5 has quit IRC
1202018-06-09T16:34:42  <gmaxwell> they get 18% speedup for 2way over 1way, and 21% for 4-way over two-way.
1212018-06-09T16:35:15  <gmaxwell> I'm not sure if that difference is even worth it, though perhaps throughput might increase for later cpus.
1222018-06-09T16:37:23  <sipa> interesting, i'll try that too
1232018-06-09T16:37:47  <gmaxwell> Their implementation might be interesting to look at to see if they had some smarter way of dealing with register pressure.
1242018-06-09T16:38:07  <sipa> another remarkable thing i noticed: the speedup of 64-specialized shani over variable length shani was close to 2x
1252018-06-09T16:38:19  <sipa> far higher than the ratio observed elsewhere
1262018-06-09T16:38:33  <sipa> gmaxwell: from what i can see it's just interleaving
1272018-06-09T16:39:17  <gmaxwell> (presumably register churn is why their attempt at 8-way was slower 2/4 way)
1282018-06-09T16:40:26  <gmaxwell> sipa: The 64-specialized saves expander work, which I guess isn't as fast with shani?  or maybe it's just that shani is faster so calling overhead (which the specialized reduces) matters more?
1292018-06-09T16:41:12  <provoostenator> Memory management is a pain. I have a device with 1 GB RAM, trying to squeeze as much as possible out of it during IBD. Without swap, if I set it slight too high, it crashes when dbcache gets too large. With swap, it starts using the swap, which presumably defeats the purpose. Is there any way to _have_ swap but prevent dbcache from using it?
1302018-06-09T16:42:05  <gmaxwell> provoostenator: I doubt swapping is actually defeating the purpose, at least if it isn't doing it heavily.
1312018-06-09T16:42:25  <gmaxwell> The data that gets swapped is infrequently used stuff first...
1322018-06-09T16:45:15  <sipa> gmaxwell: SHANI has special instructions both for expansion and transform
1332018-06-09T16:45:19  <provoostenator> It indeed didn't seem very slow, so maybe it's not too bad in practice then. 450 MB dbcache (with maxmempool=5) seems about the max without swap.
1342018-06-09T16:51:22  *** jcohen has quit IRC
1352018-06-09T16:52:59  *** jcohen has joined #bitcoin-core-dev
1362018-06-09T16:57:10  *** laurentmt has joined #bitcoin-core-dev
1372018-06-09T17:01:10  *** Sinclair6 has joined #bitcoin-core-dev
1382018-06-09T17:01:52  *** berndj has quit IRC
1392018-06-09T17:04:37  *** berndj has joined #bitcoin-core-dev
1402018-06-09T17:09:43  <sipa> gmaxwell: 4-way seems a bit slower here, but that may be due to less than perfectly interleaved code being emitted
1412018-06-09T17:11:41  *** laurentmt1 has joined #bitcoin-core-dev
1422018-06-09T17:12:50  *** laurentmt has quit IRC
1432018-06-09T17:12:50  *** laurentmt1 is now known as laurentmt
1442018-06-09T17:16:51  *** laurentmt has quit IRC
1452018-06-09T17:18:43  *** laurentmt has joined #bitcoin-core-dev
1462018-06-09T17:33:10  *** ctrlbreak has joined #bitcoin-core-dev
1472018-06-09T17:36:48  *** Guyver2 has quit IRC
1482018-06-09T17:37:02  *** Guyver2 has joined #bitcoin-core-dev
1492018-06-09T17:38:23  *** laurentmt has quit IRC
1502018-06-09T17:49:28  *** grafcaps has quit IRC
1512018-06-09T17:51:40  *** grafcaps has joined #bitcoin-core-dev
1522018-06-09T17:51:46  *** ToURniQue has joined #bitcoin-core-dev
1532018-06-09T17:59:23  *** jimmysong_ has joined #bitcoin-core-dev
1542018-06-09T18:03:12  *** TheV01d_ has joined #bitcoin-core-dev
1552018-06-09T18:03:39  *** kanzure_ has joined #bitcoin-core-dev
1562018-06-09T18:04:04  *** TheV01d has quit IRC
1572018-06-09T18:04:04  *** ula has quit IRC
1582018-06-09T18:04:05  *** kanzure has quit IRC
1592018-06-09T18:04:05  *** belcher has quit IRC
1602018-06-09T18:04:06  *** jimmysong has quit IRC
1612018-06-09T18:04:13  *** TheV01d_ is now known as TheV01d
1622018-06-09T18:04:26  *** belcher has joined #bitcoin-core-dev
1632018-06-09T18:06:00  *** stepa[m] has quit IRC
1642018-06-09T18:06:27  *** thaumavorio has quit IRC
1652018-06-09T18:06:42  *** kewde[m] has quit IRC
1662018-06-09T18:06:52  *** joshb[m] has quit IRC
1672018-06-09T18:07:14  *** squarfed[m] has quit IRC
1682018-06-09T18:07:17  *** ajtowns[m] has quit IRC
1692018-06-09T18:07:17  *** herzmeister[m] has quit IRC
1702018-06-09T18:08:54  *** thaumavorio has joined #bitcoin-core-dev
1712018-06-09T18:15:40  *** jcohen has quit IRC
1722018-06-09T18:17:33  *** SopaXorzTaker has quit IRC
1732018-06-09T18:18:13  *** AaronvanW has quit IRC
1742018-06-09T18:18:50  *** AaronvanW has joined #bitcoin-core-dev
1752018-06-09T18:20:00  *** SopaXorzTaker has joined #bitcoin-core-dev
1762018-06-09T18:23:29  *** AaronvanW has quit IRC
1772018-06-09T18:28:54  *** AaronvanW has joined #bitcoin-core-dev
1782018-06-09T18:32:13  *** justanotheruser has joined #bitcoin-core-dev
1792018-06-09T18:33:26  *** AaronvanW has quit IRC
1802018-06-09T18:34:07  *** intcat has quit IRC
1812018-06-09T18:36:28  *** intcat has joined #bitcoin-core-dev
1822018-06-09T18:51:43  *** SopaXorzTaker has quit IRC
1832018-06-09T19:05:02  *** TheRec_ has joined #bitcoin-core-dev
1842018-06-09T19:05:02  *** TheRec_ has joined #bitcoin-core-dev
1852018-06-09T19:05:50  *** TheRec has quit IRC
1862018-06-09T19:29:40  *** AaronvanW has joined #bitcoin-core-dev
1872018-06-09T19:34:32  *** AaronvanW has quit IRC
1882018-06-09T19:42:18  *** Chris_Stewart_5 has joined #bitcoin-core-dev
1892018-06-09T20:04:40  *** justanotheruser has quit IRC
1902018-06-09T20:13:57  *** GoldenBear has quit IRC
1912018-06-09T20:20:12  *** justanotheruser has joined #bitcoin-core-dev
1922018-06-09T20:21:59  *** Guyver2 has quit IRC
1932018-06-09T20:28:53  *** joshb[m] has joined #bitcoin-core-dev
1942018-06-09T20:43:07  *** justanotheruser has quit IRC
1952018-06-09T20:44:48  *** Victorsueca has quit IRC
1962018-06-09T20:45:20  *** ajtowns[m] has joined #bitcoin-core-dev
1972018-06-09T20:45:20  *** herzmeister[m] has joined #bitcoin-core-dev
1982018-06-09T20:45:20  *** kewde[m] has joined #bitcoin-core-dev
1992018-06-09T20:45:21  *** squarfed[m] has joined #bitcoin-core-dev
2002018-06-09T20:45:21  *** stepa[m] has joined #bitcoin-core-dev
2012018-06-09T20:46:02  *** Victorsueca has joined #bitcoin-core-dev
2022018-06-09T20:59:38  *** justanotheruser has joined #bitcoin-core-dev
2032018-06-09T21:04:53  <provoostenator> I have a new theory as to why my aggresive pruning IBD branch is _slower_ than master. Namely that dirty CCoinsCacheEntry read/write doesn't perform well for very large cache sizes. See See also https://github.com/bitcoin/bitcoin/pull/12404#issuecomment-395998702
2042018-06-09T21:05:04  <provoostenator> (theory, still have to measure this)
2052018-06-09T21:27:29  <phantomcircuit> provoostenator, aggressive pruning?
2062018-06-09T21:27:34  *** kanzure_ is now known as kanzure
2072018-06-09T21:27:46  <sipa> phantomcircuit: #12404
2082018-06-09T21:27:48  <gribble> https://github.com/bitcoin/bitcoin/issues/12404 | Prune more aggressively during IBD by Sjors · Pull Request #12404 · bitcoin/bitcoin · GitHub
2092018-06-09T21:28:52  <phantomcircuit> oh
2102018-06-09T21:30:28  *** AaronvanW has joined #bitcoin-core-dev
2112018-06-09T21:34:38  <phantomcircuit> sipa, does flushing the cache still remove everything?
2122018-06-09T21:35:00  <sipa> yes
2132018-06-09T21:36:58  *** AaronvanW has quit IRC
2142018-06-09T21:37:38  <phantomcircuit> sipa, and there's no way to flush "upto block x" right?
2152018-06-09T21:38:59  <sipa> phantomcircuit: indeed, because there may have been entries created before x, but spent after x, which wouldn't be present on disk
2162018-06-09T21:39:38  <sipa> it is possible with the non-atomic flushing since 0.15 (which writes to disk a range of blocks rather than a single up-to-x point)
2172018-06-09T21:39:48  <sipa> though it's pretty complicated to reason about
2182018-06-09T21:59:25  <phantomcircuit> sipa, so to enable that you'd need to keep around entries that are a record of an entry being deleted?
2192018-06-09T22:01:37  <sipa> phantomcircuit: you actually don't
2202018-06-09T22:02:27  <sipa> you just need to accurately keep track of (a) the block up to which you've flushed everything and (b) the block up to which effects may be present on disk, and at startup replay the blocks' UTXO effects between those 2
2212018-06-09T22:02:34  <sipa> that's already implemented even
2222018-06-09T22:02:53  <sipa> however, once you introduce partial flushing during reorgs which may overlap etc... it becomes far more complicated
2232018-06-09T22:03:34  <phantomcircuit> yeah wasn't thinking about reorgs
2242018-06-09T22:04:27  <sipa> all of this is doable, and i think i know all the algorithms necessary to implement it
2252018-06-09T22:05:05  <sipa> with the goal of being able to have a background process that just periodically (and asynchronously) flushes the oldest dirty UTXO entries (and wipes the oldest non-dirty ones)
2262018-06-09T22:05:27  <sipa> but it's a pretty big amount of work without knowing if it'll actually speed things up :)
2272018-06-09T22:06:17  <phantomcircuit> sipa, i had a patch which did this, but broke consensus across reorgs
2282018-06-09T22:06:22  <phantomcircuit> it was a substantial speed up
2292018-06-09T22:06:33  <phantomcircuit> but that was a while ago, so possibly it wouldn't be as large anymore?
2302018-06-09T22:06:54  <sipa> since per-txout in 0.15 performance profiles of such things may have shifted drastically
2312018-06-09T22:07:00  <sipa> it could be less or more of a speedup now :)
2322018-06-09T22:11:37  <phantomcircuit> yeah
2332018-06-09T22:11:47  <phantomcircuit> iirc it was really simple to do
2342018-06-09T22:38:23  *** jhfrontz has quit IRC
2352018-06-09T22:38:48  *** Chris_Stewart_5 has quit IRC
2362018-06-09T22:52:26  *** jhfrontz has joined #bitcoin-core-dev
2372018-06-09T22:56:15  *** andytoshi has joined #bitcoin-core-dev
2382018-06-09T22:59:41  *** andytoshi has joined #bitcoin-core-dev
2392018-06-09T22:59:42  *** andytoshi has joined #bitcoin-core-dev
2402018-06-09T23:03:20  *** andytoshi has joined #bitcoin-core-dev
2412018-06-09T23:23:00  *** Giszmo has joined #bitcoin-core-dev
2422018-06-09T23:37:44  *** Chris_Stewart_5 has joined #bitcoin-core-dev
2432018-06-09T23:39:59  <phantomcircuit> sipa, the FRESH flag looks a bit confusing
2442018-06-09T23:41:04  <phantomcircuit> the idea is that if an entry is added and spent before a flush it's effectively a noop ?
2452018-06-09T23:41:22  <sipa> it just means "this entry does not exist in the parent cache, so if it is spent, we can just forget about it"
2462018-06-09T23:41:54  <sipa> phantomcircuit: it's *the* major performance gain our cache gives
2472018-06-09T23:41:55  <phantomcircuit> ok i get that
2482018-06-09T23:42:15  <phantomcircuit> yeah
2492018-06-09T23:43:36  <sipa> because it avoids entries ever hitting disk at all
2502018-06-09T23:45:07  *** ToURniQue has quit IRC
2512018-06-09T23:47:26  <phantomcircuit> sipa, yup i definitely get it