12018-04-06T00:00:03 *** weez17 has quit IRC
22018-04-06T00:00:47 *** weez17 has joined #bitcoin-core-dev
32018-04-06T00:04:13 *** promag has quit IRC
42018-04-06T00:04:55 *** dafunkiz_ has quit IRC
52018-04-06T00:06:27 *** Lauda has quit IRC
62018-04-06T00:13:22 *** Lauda has joined #bitcoin-core-dev
72018-04-06T00:13:38 *** dafunkiz_ has joined #bitcoin-core-dev
82018-04-06T00:16:16 *** drexl has quit IRC
92018-04-06T00:22:07 *** Sinclair6 has joined #bitcoin-core-dev
102018-04-06T00:22:37 *** Victorsueca has quit IRC
112018-04-06T00:22:50 *** Randolf has joined #bitcoin-core-dev
122018-04-06T00:23:53 *** Victorsueca has joined #bitcoin-core-dev
132018-04-06T00:25:57 *** grafcaps has quit IRC
142018-04-06T00:25:59 *** Madars has quit IRC
152018-04-06T00:28:35 *** Lauda has quit IRC
162018-04-06T00:30:12 *** Lauda has joined #bitcoin-core-dev
172018-04-06T00:30:30 *** AaronvanW has quit IRC
182018-04-06T00:31:04 *** AaronvanW has joined #bitcoin-core-dev
192018-04-06T00:33:51 *** arbitrary_guy has quit IRC
202018-04-06T00:34:11 *** arbitrary_guy has joined #bitcoin-core-dev
212018-04-06T00:37:06 *** AaronvanW has quit IRC
222018-04-06T00:40:07 *** nitramiz has joined #bitcoin-core-dev
232018-04-06T00:40:49 *** moneyball has quit IRC
242018-04-06T00:43:09 *** grafcaps has joined #bitcoin-core-dev
252018-04-06T01:06:34 *** dafunkiz_ has quit IRC
262018-04-06T01:08:46 *** Randolf has quit IRC
272018-04-06T01:10:03 *** isis_ is now known as isis
282018-04-06T01:11:48 *** Randolf has joined #bitcoin-core-dev
292018-04-06T01:15:21 *** andytoshi has quit IRC
302018-04-06T01:16:34 *** DrOlmer has quit IRC
312018-04-06T01:16:53 *** DrOlmer has joined #bitcoin-core-dev
322018-04-06T01:20:42 <aj> right, different VMs, but same test_bitcion.exe, under wine coinselector_tests/knapsack_solver_test takes 581.8 seconds, under windows server 2016 w/ updates it takes 31.4s
332018-04-06T01:21:40 <sipa> is there a way to do profiling in wine?
342018-04-06T01:22:59 *** Madars has joined #bitcoin-core-dev
352018-04-06T01:22:59 <aj> i figure i'll try a newer version of wine than what's in trusty first
362018-04-06T01:29:34 *** dafunkiz_ has joined #bitcoin-core-dev
372018-04-06T01:31:11 *** shesek` has joined #bitcoin-core-dev
382018-04-06T01:39:41 <aj> same version of wine on xenial seems no better, unsurprisingly; but, wine 3.0 at home seems much faster
392018-04-06T01:42:05 *** AaronvanW has joined #bitcoin-core-dev
402018-04-06T01:43:24 <aj> total of 236s for the entire test suite under wine 3.0 vs 193s under windows server; different hardware though
412018-04-06T01:45:57 *** shesek` has quit IRC
422018-04-06T01:47:40 *** AaronvanW has quit IRC
432018-04-06T01:49:22 *** Madars has quit IRC
442018-04-06T01:49:33 *** zigen has joined #bitcoin-core-dev
452018-04-06T01:51:32 *** meshcollider has quit IRC
462018-04-06T02:07:59 <aj> wine1.8 from some random ppa on xenial also gets the entire test suite in 377s or so, different hardware yet again
472018-04-06T02:08:58 <aj> oh, bugger, that's wine3.0 as well, should have paid more attention when installing
482018-04-06T02:10:22 <bitcoin-git> [bitcoin] lachlangreenbank opened pull request #12898: Comment cleanup and consistency (master...patch-1) https://github.com/bitcoin/bitcoin/pull/12898
492018-04-06T02:10:52 *** Krellan has quit IRC
502018-04-06T02:15:02 *** nitramiz has quit IRC
512018-04-06T02:15:34 *** dafuq_ has quit IRC
522018-04-06T02:15:38 *** dafunkiz_ has quit IRC
532018-04-06T02:16:06 *** murrayn has joined #bitcoin-core-dev
542018-04-06T02:22:44 *** unholymachine has joined #bitcoin-core-dev
552018-04-06T02:26:01 *** Murch has quit IRC
562018-04-06T02:28:53 *** booyah has quit IRC
572018-04-06T02:30:34 *** booyah has joined #bitcoin-core-dev
582018-04-06T02:32:25 *** nitramiz has joined #bitcoin-core-dev
592018-04-06T02:34:05 *** moneyball has joined #bitcoin-core-dev
602018-04-06T02:34:43 *** Madars has joined #bitcoin-core-dev
612018-04-06T02:35:21 *** cryptojanitor has quit IRC
622018-04-06T02:50:37 *** Derrekito has quit IRC
632018-04-06T02:51:28 *** Derrekito has joined #bitcoin-core-dev
642018-04-06T02:53:00 *** karimofthecrop has joined #bitcoin-core-dev
652018-04-06T02:57:44 *** Madars has quit IRC
662018-04-06T02:58:17 *** Derrekito has quit IRC
672018-04-06T03:12:52 *** moneyball has quit IRC
682018-04-06T03:52:50 *** zigen has quit IRC
692018-04-06T03:53:22 *** zigen has joined #bitcoin-core-dev
702018-04-06T03:54:43 *** zigen has quit IRC
712018-04-06T03:54:57 *** zigen has joined #bitcoin-core-dev
722018-04-06T03:55:47 *** moneyball has joined #bitcoin-core-dev
732018-04-06T03:56:02 *** Madars has joined #bitcoin-core-dev
742018-04-06T04:00:19 *** zigen has quit IRC
752018-04-06T04:00:47 *** zigen has joined #bitcoin-core-dev
762018-04-06T04:05:19 *** zigen has quit IRC
772018-04-06T04:05:56 *** zigen has joined #bitcoin-core-dev
782018-04-06T04:11:48 *** dafunkiz_ has joined #bitcoin-core-dev
792018-04-06T04:14:26 *** zigen has quit IRC
802018-04-06T04:26:19 *** dafunkiz_ has quit IRC
812018-04-06T04:36:09 *** indistylo has joined #bitcoin-core-dev
822018-04-06T04:45:03 *** nitramiz has quit IRC
832018-04-06T04:45:49 *** dafunkiz_ has joined #bitcoin-core-dev
842018-04-06T04:58:32 *** rex4539 has joined #bitcoin-core-dev
852018-04-06T05:00:32 *** arubi has quit IRC
862018-04-06T05:01:30 *** arubi has joined #bitcoin-core-dev
872018-04-06T05:02:54 <karl2> sipa: you can use gprof with wine binaries as normal, I found out the other day. It wasn't particularly exciting stuff when I tried to track down job #2 timeout thing the other day, though. Maybe running different wine versions will trigger it, like aj is saying!
882018-04-06T05:02:58 <gribble> https://github.com/bitcoin/bitcoin/issues/2 | Long-term, safe, store-of-value · Issue #2 · bitcoin/bitcoin · GitHub
892018-04-06T05:03:30 *** karl2 is now known as kallewoof
902018-04-06T05:06:35 *** geezas has quit IRC
912018-04-06T05:08:31 <bitcoin-git> [bitcoin] AkioNak opened pull request #12899: macOS: Prevent Xcode 9.3 build warnings (master...preventxcodebuildwarnings) https://github.com/bitcoin/bitcoin/pull/12899
922018-04-06T05:14:40 *** clarkmoody has quit IRC
932018-04-06T05:16:23 *** rex4539 has quit IRC
942018-04-06T05:18:53 *** karimofthecrop has quit IRC
952018-04-06T05:21:20 *** zigen has joined #bitcoin-core-dev
962018-04-06T05:23:42 *** zigen has quit IRC
972018-04-06T05:23:54 *** zigen has joined #bitcoin-core-dev
982018-04-06T05:25:34 *** Bridgette4Marks has quit IRC
992018-04-06T05:27:55 *** rex4539 has joined #bitcoin-core-dev
1002018-04-06T05:29:27 *** Soligor has quit IRC
1012018-04-06T05:31:59 *** dafunkiz_ has quit IRC
1022018-04-06T05:32:07 *** moneyball has quit IRC
1032018-04-06T05:32:23 *** Soligor has joined #bitcoin-core-dev
1042018-04-06T05:34:47 *** dafunkiz_ has joined #bitcoin-core-dev
1052018-04-06T05:36:11 *** Evel-Knievel has quit IRC
1062018-04-06T05:42:19 *** aruns has joined #bitcoin-core-dev
1072018-04-06T05:42:42 *** indistylo has quit IRC
1082018-04-06T06:00:58 *** jtimon has quit IRC
1092018-04-06T06:04:38 *** Madisen45Reynold has joined #bitcoin-core-dev
1102018-04-06T06:12:46 *** zigen has quit IRC
1112018-04-06T06:13:21 *** zigen has joined #bitcoin-core-dev
1122018-04-06T06:17:17 <bitcoin-git> [bitcoin] martintibor40 opened pull request #12900: Update README.md (master...patch-1) https://github.com/bitcoin/bitcoin/pull/12900
1132018-04-06T06:17:54 <bitcoin-git> [bitcoin] fanquake closed pull request #12900: Update README.md (master...patch-1) https://github.com/bitcoin/bitcoin/pull/12900
1142018-04-06T06:17:56 *** zigen has quit IRC
1152018-04-06T06:24:45 *** zigen has joined #bitcoin-core-dev
1162018-04-06T06:26:04 *** zigen has quit IRC
1172018-04-06T06:26:39 *** zigen has joined #bitcoin-core-dev
1182018-04-06T06:26:48 *** cheese_ has joined #bitcoin-core-dev
1192018-04-06T06:29:51 *** Cheeseo has quit IRC
1202018-04-06T06:30:57 *** zigen has quit IRC
1212018-04-06T06:34:15 *** zigen has joined #bitcoin-core-dev
1222018-04-06T06:38:46 *** zigen has quit IRC
1232018-04-06T06:44:51 *** dafunkiz_ has quit IRC
1242018-04-06T07:03:18 *** rods1 has quit IRC
1252018-04-06T07:03:46 *** rods1 has joined #bitcoin-core-dev
1262018-04-06T07:07:27 *** aruns has quit IRC
1272018-04-06T07:24:14 *** grafcaps has quit IRC
1282018-04-06T07:25:51 *** booyah_ has joined #bitcoin-core-dev
1292018-04-06T07:26:03 *** booyah has quit IRC
1302018-04-06T07:29:27 *** dermoth has joined #bitcoin-core-dev
1312018-04-06T07:34:30 *** zarez has joined #bitcoin-core-dev
1322018-04-06T07:49:37 *** timothy has joined #bitcoin-core-dev
1332018-04-06T08:05:03 *** setpill has joined #bitcoin-core-dev
1342018-04-06T08:07:47 *** Madisen45Reynold has quit IRC
1352018-04-06T08:13:17 *** atroxes has quit IRC
1362018-04-06T08:15:43 *** atroxes has joined #bitcoin-core-dev
1372018-04-06T08:20:56 *** murrayn has quit IRC
1382018-04-06T08:21:20 *** murrayn has joined #bitcoin-core-dev
1392018-04-06T08:21:21 *** murrayn has joined #bitcoin-core-dev
1402018-04-06T08:34:09 *** unholymachine has quit IRC
1412018-04-06T08:35:28 *** SopaXorzTaker has joined #bitcoin-core-dev
1422018-04-06T08:37:05 *** unholymachine has joined #bitcoin-core-dev
1432018-04-06T08:40:15 *** promag has joined #bitcoin-core-dev
1442018-04-06T08:42:10 *** promag has quit IRC
1452018-04-06T08:43:47 *** rex4539 has quit IRC
1462018-04-06T08:46:13 *** rex4539 has joined #bitcoin-core-dev
1472018-04-06T08:47:13 *** lman has joined #bitcoin-core-dev
1482018-04-06T08:47:36 *** murrayn has quit IRC
1492018-04-06T08:48:02 *** murrayn has joined #bitcoin-core-dev
1502018-04-06T08:58:13 *** Victorsueca has quit IRC
1512018-04-06T08:59:23 *** Victorsueca has joined #bitcoin-core-dev
1522018-04-06T09:02:11 *** zigen has joined #bitcoin-core-dev
1532018-04-06T09:02:36 *** Eldora75Hettinge has joined #bitcoin-core-dev
1542018-04-06T09:06:37 *** zigen has quit IRC
1552018-04-06T09:11:39 <bitcoin-git> [bitcoin] practicalswift opened pull request #12901: build: Show enabled sanitizers in configure output (master...print-enabled-sanitizers-in-configure-output) https://github.com/bitcoin/bitcoin/pull/12901
1562018-04-06T09:13:25 *** Eldora75Hettinge has quit IRC
1572018-04-06T09:15:57 *** davec_ has quit IRC
1582018-04-06T09:17:21 *** DrOlmer has quit IRC
1592018-04-06T09:17:38 *** zigen has joined #bitcoin-core-dev
1602018-04-06T09:18:10 *** DrOlmer has joined #bitcoin-core-dev
1612018-04-06T09:23:22 *** grafcaps has joined #bitcoin-core-dev
1622018-04-06T09:25:24 *** cryptojanitor has joined #bitcoin-core-dev
1632018-04-06T09:27:13 *** zigen has quit IRC
1642018-04-06T09:27:59 *** grafcaps has quit IRC
1652018-04-06T09:36:56 *** davec has joined #bitcoin-core-dev
1662018-04-06T09:37:43 *** zigen has joined #bitcoin-core-dev
1672018-04-06T09:39:58 *** gwillen has quit IRC
1682018-04-06T09:42:13 *** gwillen has joined #bitcoin-core-dev
1692018-04-06T09:44:20 *** zigen has quit IRC
1702018-04-06T09:55:35 *** arbitrary_guy has quit IRC
1712018-04-06T10:02:37 *** Conner40VonRuede has joined #bitcoin-core-dev
1722018-04-06T10:08:11 *** jtimon has joined #bitcoin-core-dev
1732018-04-06T10:17:27 *** maxmatics has joined #bitcoin-core-dev
1742018-04-06T10:18:00 <maxmatics> hello world.
1752018-04-06T10:32:01 *** arbitrary_guy has joined #bitcoin-core-dev
1762018-04-06T10:39:57 *** arbitrary_guy has quit IRC
1772018-04-06T10:41:32 *** maxmatics has quit IRC
1782018-04-06T10:44:02 *** Victorsueca has quit IRC
1792018-04-06T10:45:03 *** Victorsueca has joined #bitcoin-core-dev
1802018-04-06T11:04:20 *** pyericz has joined #bitcoin-core-dev
1812018-04-06T11:05:21 *** pyericz has quit IRC
1822018-04-06T11:10:03 *** AaronvanW has joined #bitcoin-core-dev
1832018-04-06T11:11:42 *** Aaronvan_ has joined #bitcoin-core-dev
1842018-04-06T11:15:42 *** AaronvanW has quit IRC
1852018-04-06T11:19:53 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1862018-04-06T11:23:27 *** Giszmo has quit IRC
1872018-04-06T11:26:58 *** Giszmo has joined #bitcoin-core-dev
1882018-04-06T11:31:57 *** jojeyh has joined #bitcoin-core-dev
1892018-04-06T11:37:34 *** mrannanay has joined #bitcoin-core-dev
1902018-04-06T11:43:15 *** pyericz has joined #bitcoin-core-dev
1912018-04-06T11:44:03 *** grafcaps has joined #bitcoin-core-dev
1922018-04-06T11:48:26 *** grafcaps has quit IRC
1932018-04-06T11:49:07 *** Conner40VonRuede has quit IRC
1942018-04-06T11:50:00 *** esotericnonsense has quit IRC
1952018-04-06T11:50:02 *** esotericnonsens_ has joined #bitcoin-core-dev
1962018-04-06T11:58:27 *** zautomata1 has quit IRC
1972018-04-06T12:02:36 *** Jaleel25Walter has joined #bitcoin-core-dev
1982018-04-06T12:18:27 *** Giszmo has quit IRC
1992018-04-06T12:27:09 *** Evel-Knievel has joined #bitcoin-core-dev
2002018-04-06T12:27:15 *** pyericz has quit IRC
2012018-04-06T12:29:35 *** Jaleel25Walter has quit IRC
2022018-04-06T12:36:24 *** Chris_Stewart_5 has quit IRC
2032018-04-06T12:40:23 *** esotericnonsens_ is now known as esotericnonsense
2042018-04-06T12:51:50 *** jojeyh has quit IRC
2052018-04-06T12:51:58 *** zarez has quit IRC
2062018-04-06T13:02:44 *** Blaze52Cormier has joined #bitcoin-core-dev
2072018-04-06T13:09:08 *** arubi_ has joined #bitcoin-core-dev
2082018-04-06T13:10:44 <bitcoin-git> [bitcoin] MarcoFalke closed pull request #12898: Comment cleanup and consistency (master...patch-1) https://github.com/bitcoin/bitcoin/pull/12898
2092018-04-06T13:12:20 *** arubi has quit IRC
2102018-04-06T13:14:20 *** zarez has joined #bitcoin-core-dev
2112018-04-06T13:17:14 *** Blaze52Cormier has quit IRC
2122018-04-06T13:18:14 *** qu4ku has joined #bitcoin-core-dev
2132018-04-06T13:18:35 *** zarez has quit IRC
2142018-04-06T13:24:10 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2152018-04-06T13:27:02 *** wxss has joined #bitcoin-core-dev
2162018-04-06T13:30:05 *** satwo has joined #bitcoin-core-dev
2172018-04-06T13:32:10 *** Aaronvan_ has quit IRC
2182018-04-06T13:34:40 *** zarez has joined #bitcoin-core-dev
2192018-04-06T13:39:17 *** zarez has quit IRC
2202018-04-06T13:40:39 *** Deinogalerix21 has joined #bitcoin-core-dev
2212018-04-06T13:43:13 *** qu4ku_ has joined #bitcoin-core-dev
2222018-04-06T13:43:27 *** AaronvanW has joined #bitcoin-core-dev
2232018-04-06T13:44:18 *** Aaronvan_ has joined #bitcoin-core-dev
2242018-04-06T13:44:37 *** Aaronvan_ has quit IRC
2252018-04-06T13:45:15 *** Aaronvan_ has joined #bitcoin-core-dev
2262018-04-06T13:46:54 *** qu4ku has quit IRC
2272018-04-06T13:48:08 *** AaronvanW has quit IRC
2282018-04-06T13:50:04 <bitcoin-git> [bitcoin] sdaftuar opened pull request #12902: [qa] Handle potential cookie race when starting node (master...2018-04-improve-dbcrash-restarts) https://github.com/bitcoin/bitcoin/pull/12902
2292018-04-06T13:51:46 *** Deinogalerix21 has quit IRC
2302018-04-06T13:53:16 *** satwo has quit IRC
2312018-04-06T14:02:10 *** satwo has joined #bitcoin-core-dev
2322018-04-06T14:02:41 *** Hillary82Hartman has joined #bitcoin-core-dev
2332018-04-06T14:03:37 *** griswaalt has joined #bitcoin-core-dev
2342018-04-06T14:04:27 *** Victorsueca has quit IRC
2352018-04-06T14:05:43 *** Victorsueca has joined #bitcoin-core-dev
2362018-04-06T14:09:06 *** Dizzle has joined #bitcoin-core-dev
2372018-04-06T14:10:14 *** qu4ku_ has quit IRC
2382018-04-06T14:10:31 *** qu4ku has joined #bitcoin-core-dev
2392018-04-06T14:13:19 *** satwo has quit IRC
2402018-04-06T14:16:50 *** grafcaps has joined #bitcoin-core-dev
2412018-04-06T14:20:14 *** Hillary82Hartman has quit IRC
2422018-04-06T14:21:05 *** grafcaps has quit IRC
2432018-04-06T14:22:38 *** griswaalt has quit IRC
2442018-04-06T14:28:35 *** geezas has joined #bitcoin-core-dev
2452018-04-06T14:33:02 *** d9b4bef9 has quit IRC
2462018-04-06T14:33:24 *** moneyball has joined #bitcoin-core-dev
2472018-04-06T14:34:08 *** d9b4bef9 has joined #bitcoin-core-dev
2482018-04-06T14:36:26 <stevenroose> Does bitcoin core have a utxo cache? If so, could anyone point me to the file where it is defined?
2492018-04-06T14:36:36 *** satwo has joined #bitcoin-core-dev
2502018-04-06T14:38:54 <stevenroose> Aha, I found -dbcache, which is the size in MiB for the utxo db cache
2512018-04-06T14:40:04 <stevenroose> I thought the cuckoocache was used as a sigcache.
2522018-04-06T14:40:15 <stevenroose> Is it also used for the utxo cache?
2532018-04-06T14:41:31 <sdaftuar> stevenroose: no, it is not. see src/coins.h and src/coins.cpp for more information about the utxo cache.
2542018-04-06T14:44:52 <stevenroose> sdaftuar: thanks!
2552018-04-06T14:45:17 <stevenroose> from the init code, I see that the -dbcache is split in 3, of which one is "chain state cache", what is that one for?
2562018-04-06T14:49:37 *** satwo has quit IRC
2572018-04-06T14:52:13 *** Giszmo has joined #bitcoin-core-dev
2582018-04-06T14:59:42 *** zarez has joined #bitcoin-core-dev
2592018-04-06T15:00:49 *** nitramiz has joined #bitcoin-core-dev
2602018-04-06T15:07:02 *** rex4539 has quit IRC
2612018-04-06T15:07:22 *** rex4539 has joined #bitcoin-core-dev
2622018-04-06T15:09:26 <bitcoin-git> [bitcoin] sdaftuar opened pull request #12904: [qa] Ensure bitcoind processes are cleaned up when tests end (master...2018-04-always-kill-bitcoind) https://github.com/bitcoin/bitcoin/pull/12904
2632018-04-06T15:09:32 *** nitramiz has quit IRC
2642018-04-06T15:20:21 *** Murch has joined #bitcoin-core-dev
2652018-04-06T15:24:47 *** Guyver2 has joined #bitcoin-core-dev
2662018-04-06T15:25:38 *** grafcaps has joined #bitcoin-core-dev
2672018-04-06T15:28:40 *** CubicEar_ has quit IRC
2682018-04-06T15:29:09 *** CubicEarths has joined #bitcoin-core-dev
2692018-04-06T15:30:18 *** grafcaps has quit IRC
2702018-04-06T15:33:57 *** CubicEarths has quit IRC
2712018-04-06T15:36:20 <sipa> stevenroose: for the UTXO set the layers are: (1) disk (with OS cache etc) (2) LevelDB's cache (3) Bitcoin Core's pcoinsTip (a CCoinsViewCache object)
2722018-04-06T15:36:40 <setpill> is there any way to actively check for conflicting transactions in the mempool for a given tx?
2732018-04-06T15:37:39 <sipa> stevenroose: there is another data set, the block index which is loaded entirely in memory, but is stored in a separate LevelDB database
2742018-04-06T15:38:09 <sipa> so the 3 pieces -dbcache is split over is the blockindex leveldb cache, the chainstate leveldb cache (=utxo), and pcoinTip
2752018-04-06T15:38:28 <setpill> i suppose trying to rebroadcast would give an error in some cases, but i am not sure how reliable that is, and i would prefer just simply checking without broadcasting
2762018-04-06T15:39:22 *** qu4ku has quit IRC
2772018-04-06T15:41:46 *** zarez has quit IRC
2782018-04-06T15:44:14 *** Randolf has quit IRC
2792018-04-06T15:45:52 *** qu4ku has joined #bitcoin-core-dev
2802018-04-06T15:57:53 *** qu4ku_ has joined #bitcoin-core-dev
2812018-04-06T15:57:54 *** qu4ku has quit IRC
2822018-04-06T16:04:43 *** Victorsueca has quit IRC
2832018-04-06T16:05:54 *** Victorsueca has joined #bitcoin-core-dev
2842018-04-06T16:06:08 *** Dizzle has quit IRC
2852018-04-06T16:09:34 *** unholymachine has quit IRC
2862018-04-06T16:09:55 *** grafcaps has joined #bitcoin-core-dev
2872018-04-06T16:12:08 *** Aaronvan_ is now known as AaronvanW
2882018-04-06T16:12:09 *** moneyball has quit IRC
2892018-04-06T16:14:21 *** zarez has joined #bitcoin-core-dev
2902018-04-06T16:16:09 *** dgenr8 has quit IRC
2912018-04-06T16:16:22 *** dgenr8 has joined #bitcoin-core-dev
2922018-04-06T16:20:06 *** zarez has quit IRC
2932018-04-06T16:20:55 *** moneyball has joined #bitcoin-core-dev
2942018-04-06T16:25:53 *** qu4ku_ has quit IRC
2952018-04-06T16:31:04 *** promag has joined #bitcoin-core-dev
2962018-04-06T16:31:29 *** Giszmo has quit IRC
2972018-04-06T16:32:55 *** wxss has quit IRC
2982018-04-06T16:38:13 *** promag has quit IRC
2992018-04-06T16:38:14 *** geezas has quit IRC
3002018-04-06T16:40:17 *** Randolf has joined #bitcoin-core-dev
3012018-04-06T16:41:51 *** twistedline_ has quit IRC
3022018-04-06T16:42:10 *** setpill has quit IRC
3032018-04-06T16:47:01 *** Giszmo has joined #bitcoin-core-dev
3042018-04-06T16:48:05 *** jb55 has joined #bitcoin-core-dev
3052018-04-06T16:48:18 *** promag has joined #bitcoin-core-dev
3062018-04-06T16:49:27 *** twistedline has joined #bitcoin-core-dev
3072018-04-06T17:01:41 *** rex4539 has quit IRC
3082018-04-06T17:13:54 *** williamayd has joined #bitcoin-core-dev
3092018-04-06T17:14:15 *** promag has quit IRC
3102018-04-06T17:16:35 *** promag has joined #bitcoin-core-dev
3112018-04-06T17:17:05 *** DrOlmer has quit IRC
3122018-04-06T17:17:20 *** promag has quit IRC
3132018-04-06T17:17:37 *** DrOlmer has joined #bitcoin-core-dev
3142018-04-06T17:18:32 *** williamayd has quit IRC
3152018-04-06T17:19:14 *** lman has quit IRC
3162018-04-06T17:22:21 *** rex4539 has joined #bitcoin-core-dev
3172018-04-06T17:25:31 *** promag has joined #bitcoin-core-dev
3182018-04-06T17:29:48 *** promag has quit IRC
3192018-04-06T17:29:50 *** Murch has quit IRC
3202018-04-06T17:32:57 *** Murch has joined #bitcoin-core-dev
3212018-04-06T17:35:03 *** Victorsueca has quit IRC
3222018-04-06T17:35:47 *** promag has joined #bitcoin-core-dev
3232018-04-06T17:36:15 *** Victorsueca has joined #bitcoin-core-dev
3242018-04-06T17:36:30 *** williamayd has joined #bitcoin-core-dev
3252018-04-06T17:39:53 *** williamayd has quit IRC
3262018-04-06T17:40:52 *** dafunkiz_ has joined #bitcoin-core-dev
3272018-04-06T17:43:25 *** promag has quit IRC
3282018-04-06T17:43:32 *** isis is now known as isis_
3292018-04-06T17:47:32 *** vicenteH has quit IRC
3302018-04-06T17:47:41 *** jnewbery has quit IRC
3312018-04-06T17:49:11 *** Murch has quit IRC
3322018-04-06T17:50:27 *** Murch has joined #bitcoin-core-dev
3332018-04-06T17:52:27 *** Chris_Stewart_5 has quit IRC
3342018-04-06T17:57:45 *** arubi_ is now known as arubi
3352018-04-06T17:59:42 *** jb551 has joined #bitcoin-core-dev
3362018-04-06T17:59:45 *** Murch has quit IRC
3372018-04-06T18:00:19 *** Murch has joined #bitcoin-core-dev
3382018-04-06T18:02:18 *** SopaXorzTaker has quit IRC
3392018-04-06T18:05:22 *** arubi has quit IRC
3402018-04-06T18:05:53 *** arubi has joined #bitcoin-core-dev
3412018-04-06T18:08:31 *** dafunkiz_ has quit IRC
3422018-04-06T18:10:50 *** dafunkiz_ has joined #bitcoin-core-dev
3432018-04-06T18:11:11 *** booyah_ is now known as booyah
3442018-04-06T18:13:59 *** promag has joined #bitcoin-core-dev
3452018-04-06T18:14:05 *** jnewbery has joined #bitcoin-core-dev
3462018-04-06T18:15:19 *** jb551 has quit IRC
3472018-04-06T18:16:32 *** dafunki__ has joined #bitcoin-core-dev
3482018-04-06T18:19:17 *** promag has quit IRC
3492018-04-06T18:19:29 *** dafunkiz_ has quit IRC
3502018-04-06T18:21:01 *** drexl has joined #bitcoin-core-dev
3512018-04-06T18:24:38 *** moneyball has quit IRC
3522018-04-06T18:26:16 *** moneyball has joined #bitcoin-core-dev
3532018-04-06T18:28:25 *** promag has joined #bitcoin-core-dev
3542018-04-06T18:28:45 <bitcoin-git> [bitcoin] sdaftuar opened pull request #12905: [rpcwallet] Clamp walletpassphrase value at 100M seconds (master...2018-04-wallet-encryption-timeout) https://github.com/bitcoin/bitcoin/pull/12905
3552018-04-06T18:33:42 *** AhNo1 has joined #bitcoin-core-dev
3562018-04-06T18:34:05 *** AhNo1 has quit IRC
3572018-04-06T18:35:02 *** d9b4bef9 has quit IRC
3582018-04-06T18:38:08 *** d9b4bef9 has joined #bitcoin-core-dev
3592018-04-06T18:39:28 *** Aaronvan_ has joined #bitcoin-core-dev
3602018-04-06T18:42:44 *** AaronvanW has quit IRC
3612018-04-06T18:47:59 *** commavir_ has quit IRC
3622018-04-06T18:48:33 *** commavir has joined #bitcoin-core-dev
3632018-04-06T18:49:00 *** AaronvanW has joined #bitcoin-core-dev
3642018-04-06T18:49:37 *** Murch has quit IRC
3652018-04-06T18:50:57 <sdaftuar> setpill: unfortunately i don't think we have great tools right now. in the next release, we'll have an rpc called "testmempoolaccept" which you could use to determine whether a given transaction would be accepted to your mempool, which might be along the lines of what you'd want?
3662018-04-06T18:51:09 <sdaftuar> but it's tricky in general, because dealing with transaction chains is not easy
3672018-04-06T18:51:35 *** Aaronvan_ has quit IRC
3682018-04-06T18:52:38 <sdaftuar> for instance, if someone sends you a transaction that depends on another unconfirmed transaction, and then a third transaction conflicts with the parent and evicts it from the mempool, it's hard to tell that your transaction is indirectly conflicted as well
3692018-04-06T18:53:16 *** Murch has joined #bitcoin-core-dev
3702018-04-06T18:55:33 *** Randolf has quit IRC
3712018-04-06T18:59:10 *** sturles has quit IRC
3722018-04-06T19:00:37 *** promag has quit IRC
3732018-04-06T19:00:56 *** promag has joined #bitcoin-core-dev
3742018-04-06T19:02:34 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3752018-04-06T19:03:50 <stevenroose> sipa: the block index I understand
3762018-04-06T19:04:02 <stevenroose> "the chainstate leveldb cache (=utxo), and pcoinTip" -> so what exactly is the difference there?
3772018-04-06T19:04:16 <stevenroose> they are both used to cache the utxo set, right?
3782018-04-06T19:04:37 *** promag has quit IRC
3792018-04-06T19:07:28 *** sturles has joined #bitcoin-core-dev
3802018-04-06T19:10:59 <jamesob> does POTENTIAL DEADLOCK DETECTED being logged by a node during functional test runs indicate something is definitely out of the ordinary?
3812018-04-06T19:19:16 *** promag has joined #bitcoin-core-dev
3822018-04-06T19:20:10 *** Chris_Stewart_5 has quit IRC
3832018-04-06T19:21:10 *** promag has quit IRC
3842018-04-06T19:21:21 *** dafunki__ has quit IRC
3852018-04-06T19:21:32 <stevenroose> sipa: oh you meant with (=utxo) that it's just the leveldb cache for the store that has the utxo data
3862018-04-06T19:21:58 <stevenroose> ok, so the only thing core maintains itself is the CCoinsViewCache, right?
3872018-04-06T19:23:43 *** dafunkiz_ has joined #bitcoin-core-dev
3882018-04-06T19:26:11 *** dafunkiz_ has quit IRC
3892018-04-06T19:32:06 *** jnewbery has quit IRC
3902018-04-06T19:32:52 *** jamesob has quit IRC
3912018-04-06T19:33:58 *** jnewbery has joined #bitcoin-core-dev
3922018-04-06T19:35:27 *** jamesob has joined #bitcoin-core-dev
3932018-04-06T19:45:13 *** cryptojanitor has quit IRC
3942018-04-06T19:45:13 *** Victorsueca has quit IRC
3952018-04-06T19:46:24 *** Victorsueca has joined #bitcoin-core-dev
3962018-04-06T19:47:15 <sipa> stevenroose: yes, but that's thr most important source of speedups
3972018-04-06T19:47:35 <sipa> the leveldb cache helps ob systems with very slow i/o
3982018-04-06T19:48:08 <stevenroose> sipa: the coinsCache map is from outpoint to utxo entry, right?
3992018-04-06T19:48:32 <stevenroose> Doesn't that mean that for a new tx, the txhash is potentially added a lot of times?
4002018-04-06T19:49:22 <stevenroose> in btcd, we have a structure where we map txid to utxoentry that has a (potentially sparse) map from index to output
4012018-04-06T19:49:32 <stevenroose> but it
4022018-04-06T19:49:53 <stevenroose> 's currently not cached, and indeed, that's one of the mayor performance bottlenecks
4032018-04-06T19:51:34 <drexl> when an opcode has inputs, do these come from the stack?
4042018-04-06T19:51:58 *** moneyball has quit IRC
4052018-04-06T19:52:41 <sipa> stevenroose: i believe btcd's design was based on bitcoin core's previous
4062018-04-06T19:52:57 <sipa> we switched from per-tx to per-txout in 0.15
4072018-04-06T19:53:37 <sipa> leveldb deduplicates multiple key-value pairs with keys that share a prefix anyway, so on disk it's not all that impactful
4082018-04-06T19:53:47 *** jamesob has quit IRC
4092018-04-06T19:54:14 <sipa> and it simplifies the in-memory cache and serialization overhead for read/writes significantly
4102018-04-06T19:54:53 <stevenroose> simplifies it a lot indeed
4112018-04-06T19:55:16 *** rex4539 has quit IRC
4122018-04-06T19:55:21 <stevenroose> any numbers on extra memory usage /utxo stored?
4132018-04-06T19:55:28 <sipa> 500 MB on disk extra
4142018-04-06T19:56:02 <stevenroose> is disk more important than memory?
4152018-04-06T19:56:09 <sipa> imho, no
4162018-04-06T19:56:16 *** jamesob has joined #bitcoin-core-dev
4172018-04-06T19:56:17 <sipa> it's a small constant factor extra disk
4182018-04-06T19:56:30 <stevenroose> I mean for initila sync mostly, a big memory cache can be very significant, no?
4192018-04-06T19:56:30 <sipa> but it makes memory usage for the cache faster, more effective, and more efficient
4202018-04-06T19:56:48 <stevenroose> hmm
4212018-04-06T19:57:15 <sipa> because now we don't need to load unrelated other unspent outputs into memory
4222018-04-06T19:57:19 <sipa> when one output is being spent
4232018-04-06T19:57:30 *** Krellan has joined #bitcoin-core-dev
4242018-04-06T19:59:16 <stevenroose> oh, but you don't have to do that anyway, right?
4252018-04-06T19:59:26 <stevenroose> the entry map (index -> output) is sparce
4262018-04-06T19:59:50 <stevenroose> also on disk, you don't store entire txs, only the unspent outputs
4272018-04-06T19:59:55 <sipa> yes
4282018-04-06T20:00:02 *** moneyball has joined #bitcoin-core-dev
4292018-04-06T20:00:05 <sipa> but then you need a complex operation to write changes to disk
4302018-04-06T20:00:17 <sipa> and i don't see how you can easily perform the freshness optimization on that
4312018-04-06T20:00:36 *** andytoshi has joined #bitcoin-core-dev
4322018-04-06T20:01:04 *** andytoshi has joined #bitcoin-core-dev
4332018-04-06T20:01:07 *** cryptojanitor has joined #bitcoin-core-dev
4342018-04-06T20:01:08 <sipa> (that's the idea that if you create a utxo in memory, and then later spend it, in memory, it can be deleted from the cache, and nothing ever needs to hit disk, because both the create-txout and spent-txout operations are idempotent)
4352018-04-06T20:01:27 *** rex4539 has joined #bitcoin-core-dev
4362018-04-06T20:02:14 <stevenroose> hmm, I only just started looking into this more deeply, but I don't see how storing utxos grouped per tx change that
4372018-04-06T20:02:38 <sipa> well it means you can only do that optimization if all utxos of a tx are spent before a flush
4382018-04-06T20:04:17 <sipa> i guess you can have a hybrid where you store them per-txout on disk, but with shared txids in memory
4392018-04-06T20:04:27 <sipa> i looked into doing that, but the memory usage savings are tiny
4402018-04-06T20:05:17 <stevenroose> I don't know what optimization you talk about tbh. let's say you add a tx with two outputs (you always add whole txs right? I don't think single outpoints make much sense), so you git txid -> (o1, o2)
4412018-04-06T20:05:42 <sipa> yup
4422018-04-06T20:06:01 <stevenroose> then, before a flush, o2 gets spent, so you keep txid -> (o1) and then when you flush, I don't see the overhead over writing txid1 -> o over txid -> (o1)
4432018-04-06T20:06:18 <stevenroose> (opposite overhead)
4442018-04-06T20:06:23 *** meshcollider has joined #bitcoin-core-dev
4452018-04-06T20:06:38 <sipa> that's actually a pretty significant overhead
4462018-04-06T20:06:53 <sipa> you need a dynamically allocated structure for variable-length output array
4472018-04-06T20:07:47 <sipa> it's hard to compare what we're talking about because there are so many variations, depending on how you do things in memory vs on disk
4482018-04-06T20:10:38 *** harrymm has quit IRC
4492018-04-06T20:10:58 <stevenroose> I realize, trying to see if it's worth turning the whole structure here upside down :)
4502018-04-06T20:11:19 <sipa> https://github.com/bitcoin/bitcoin/pull/10195
4512018-04-06T20:18:28 <stevenroose> Hmm, not reassuring :)
4522018-04-06T20:19:09 *** dafunkiz_ has joined #bitcoin-core-dev
4532018-04-06T20:20:21 <sipa> what could be done is change the in-memory representation slightly where you have two maps, one txid->(int,coinbase,height,#unspents) and another (int,vout)->(amount,script)
4542018-04-06T20:20:43 <sipa> rather than the single map we currently have (txid,vout)->(amount,script,coinbase,height)
4552018-04-06T20:21:13 <sipa> but it becomes a lot more complicated in the presence of more complicated cache flushing strategies
4562018-04-06T20:22:11 <sipa> perhaps if average txn over time gain more txouts this becomes better
4572018-04-06T20:23:38 *** harrymm has joined #bitcoin-core-dev
4582018-04-06T20:23:42 <stevenroose> well, mostly to avoid changing all the db code etc, I will probably start with a cache on the current system
4592018-04-06T20:24:05 <stevenroose> the argument of txs becoming bigger could make sense, though
4602018-04-06T20:24:47 <stevenroose> if we want actual unlinkability with CT, coinjoin-like structures will become increasingly common
4612018-04-06T20:25:45 <stevenroose> but yeah the simplicity of a txout based structure is also very compelling
4622018-04-06T20:26:02 <sipa> in any case, i'm very skeptical that any attempts to share the txids and other tx metadata in memory are worthwhile
4632018-04-06T20:26:50 <sipa> and unfortunately the CCoinsViewCache code is pretty complicated as it takes advantage of a bunch of tricks that are specific to utxo data
4642018-04-06T20:27:15 <sipa> so it's not trivial to just drop in another cache design
4652018-04-06T20:27:30 <sipa> you may want to talk to eklitzke
4662018-04-06T20:28:21 <stevenroose> "a bunch of tricks that are specific to utxo data" hmm
4672018-04-06T20:28:56 <sipa> well in particular the freshness optimization
4682018-04-06T20:29:00 <sipa> that i mentioned above
4692018-04-06T20:29:08 <stevenroose> I was going through it a bit, will certainly do some more consideration before i dive into coding
4702018-04-06T20:29:21 <stevenroose> yeah, I'll look into that
4712018-04-06T20:29:37 <sipa> it seems like a very hard first project if you're not already somewhat familiar with the codebase :)
4722018-04-06T20:29:51 <stevenroose> how does it handle crashes? keeping latest flushed height or so and rebuilding newer blocks from disk in case of crash?
4732018-04-06T20:29:58 <sipa> ah!
4742018-04-06T20:31:18 <sipa> https://github.com/bitcoin/bitcoin/pull/10148
4752018-04-06T20:31:31 *** ThinkOfANick has joined #bitcoin-core-dev
4762018-04-06T20:32:37 <stevenroose> thanks!
4772018-04-06T20:33:02 <stevenroose> when talking about cached "chain state", I suppose new blocks are always directly written to disk, no?
4782018-04-06T20:33:12 <sipa> the chain state is the utxo set
4792018-04-06T20:33:22 <sipa> blocks are stored completely independently
4802018-04-06T20:33:23 <stevenroose> ok, that clears it up
4812018-04-06T20:34:40 *** mohsen_ has joined #bitcoin-core-dev
4822018-04-06T20:34:52 <sipa> basically on disk there is a marker with a hash of a block that means "utxos created or spent after this block MAY be present" and another that means "all utxos created or spent before this block MUST be on disk"
4832018-04-06T20:34:54 *** mohsen_ has quit IRC
4842018-04-06T20:35:12 <sipa> and at startup all blocks before the first and the second hash are replayed and applied to the UTXO set
4852018-04-06T20:35:42 <sipa> this approach means we don't need to write the whole cache utxo set in one atomic operation, and also don't need to remember old deleted entries
4862018-04-06T20:37:01 *** setpill has joined #bitcoin-core-dev
4872018-04-06T20:38:08 <ThinkOfANick> sipa: Wait, why not remember old entries?
4882018-04-06T20:38:27 <sipa> ThinkOfANick: because you want to save memory
4892018-04-06T20:38:43 *** ThinkOfANick has quit IRC
4902018-04-06T20:38:59 <stevenroose> sipa: I'm having difficulty to see how those two block locators are not the same
4912018-04-06T20:39:16 <setpill> sdaftuar: perhaps ill have to resort to adding all the "from" addresses of the entire chain as watch-only addresses...
4922018-04-06T20:39:29 <sipa> stevenroose: you have 1 million modified UTXO entries in memory
4932018-04-06T20:39:32 *** mohsen_ has joined #bitcoin-core-dev
4942018-04-06T20:39:50 *** ThinkOfANick has joined #bitcoin-core-dev
4952018-04-06T20:39:50 <sipa> stevenroose: you can't construct a single batch to write them all at once to memory, as that batch would be gigabytes in side
4962018-04-06T20:39:56 <sipa> so you write part of it
4972018-04-06T20:39:58 <sipa> *size
4982018-04-06T20:40:20 <sipa> say you last full flush was at block 400000
4992018-04-06T20:40:21 <sipa> the current tip is 450000
5002018-04-06T20:40:36 <sipa> that means your cache may include entries from up to block 450000
5012018-04-06T20:40:39 <stevenroose> (partial flush, right?)
5022018-04-06T20:40:48 <stevenroose> ah full
5032018-04-06T20:40:52 <stevenroose> srry didn't see
5042018-04-06T20:41:11 <sipa> you're now doing a partial flush, writing just some subset of UTXO cache entries to disk
5052018-04-06T20:41:39 <sipa> that may include UTXOs created up to block 450000 (or may miss things that were created between 400000 and 450000)
5062018-04-06T20:41:58 <sipa> but it's not guaranteed to contain everything up to 450000 (in fact, because you know it's partial, it can't be)
5072018-04-06T20:42:07 <sipa> so the range you write is 400000..450000
5082018-04-06T20:42:12 <sipa> and if a crash happens then and there
5092018-04-06T20:42:24 <sipa> at startup you'll need to replay everything between 400000 and 450000
5102018-04-06T20:43:07 <sipa> because all utxo creation and spending operations are idempotent, it never hurts to replay an operation that was already applied
5112018-04-06T20:43:21 <stevenroose> I see, I just don't see how the 450000 is not just the same as the tip.. you also said "utxos created or spent after this block MAY be present" that must have been "before" then?
5122018-04-06T20:43:38 <sipa> 450000 is the tip
5132018-04-06T20:43:41 <sipa> in this scenario
5142018-04-06T20:43:43 <stevenroose> yeah
5152018-04-06T20:43:50 <sipa> no
5162018-04-06T20:44:00 <sipa> utxos created or spent after 400000 MAY be present
5172018-04-06T20:44:08 <sipa> utxos created or spent before 450000 MUSt be present
5182018-04-06T20:44:13 <sipa> so the range is 400000..450000
5192018-04-06T20:44:27 *** ThinkOfANick has left #bitcoin-core-dev
5202018-04-06T20:44:35 <sipa> eh, i guess this is vague
5212018-04-06T20:44:35 <stevenroose> > utxos created or spent before 450000 MUSt be present
5222018-04-06T20:44:44 <stevenroose> yeah that's where you lost me
5232018-04-06T20:44:46 <sipa> let me reformulate
5242018-04-06T20:45:06 <sipa> all operations up to block 400000 are guaranteed to be on disk
5252018-04-06T20:45:07 <stevenroose> you said "MUST be before" and "MAY be after" so I assume that is the same one
5262018-04-06T20:45:20 <sipa> all operations between 400000 and 450000 may be present on disk, but are not guaranteed to be
5272018-04-06T20:45:35 <sipa> my "may" and "must" were very confusing before
5282018-04-06T20:45:44 <stevenroose> yeah, that's what confused me
5292018-04-06T20:45:56 <sipa> but things after 450000 are guaranteed to not be on disk
5302018-04-06T20:46:05 <stevenroose> so now let's assume there is already a persistent chain tip indicator, then you only need to keep one, right?
5312018-04-06T20:46:17 <sipa> well this is the chain tip indicator
5322018-04-06T20:46:24 <sipa> instead of a tip, it's a range
5332018-04-06T20:46:43 <stevenroose> that's why I was confused for it to be two. ok ok, yeah I got it. I just assumed you would always need a chaintip anyways
5342018-04-06T20:46:55 <sipa> in the ideal scenario the two are the same
5352018-04-06T20:46:58 <sipa> after a full flush
5362018-04-06T20:47:03 <stevenroose> one last thing I'm missing
5372018-04-06T20:47:35 <stevenroose> how to know when to update that consistence height
5382018-04-06T20:47:38 <stevenroose> (the first one of the range)
5392018-04-06T20:47:45 <sipa> ah, right now it's very simple
5402018-04-06T20:48:19 <sipa> when we start a flush operation, we check what the previous lower-height (the 400000) was, and update it to (that lower height, current tip)
5412018-04-06T20:48:34 <sipa> after a flush operation completes, it's replaced with (current tip, current tip)
5422018-04-06T20:49:10 <stevenroose> oh, but then you still need full flushes?
5432018-04-06T20:49:16 <stevenroose> so you can't update in partial writes?
5442018-04-06T20:49:29 <sipa> well we only have full flushes now
5452018-04-06T20:49:36 <sipa> but they're implemented as a sequence of partial flushes
5462018-04-06T20:49:53 <sipa> longer term i want a system where we have a background thread that's constantly flushing
5472018-04-06T20:50:09 <sipa> and is always "running behind" on the tip
5482018-04-06T20:50:25 <sipa> to give the memory db a chance to cache creates/spends that cancel each other out before writing
5492018-04-06T20:50:46 <sipa> but keeping track of which is the lower hash in the range in that system is more complicated
5502018-04-06T20:51:18 <sipa> it's basically the lowest height of which you either have an unwritten create or unwritten spend
5512018-04-06T20:51:33 <sipa> but it's more tricky in the presence of reorganizations
5522018-04-06T20:51:44 <sipa> https://github.com/bitcoin/bitcoin/blob/master/src/txdb.cpp#L102L137 <- current flushing to disk logic
5532018-04-06T20:52:37 <stevenroose> oooooooh
5542018-04-06T20:53:05 <stevenroose> ok, I thought you were doing a partial flush when the cache was full
5552018-04-06T20:53:18 <stevenroose> (like LRI fashion or so)
5562018-04-06T20:53:22 <sipa> nope
5572018-04-06T20:53:29 <stevenroose> but it's just a way to reduce the size of the transaction
5582018-04-06T20:53:38 <stevenroose> ldb transaction
5592018-04-06T20:53:49 <sipa> i've experimented with various approaches for MRU eviction from the cache etc
5602018-04-06T20:53:51 <stevenroose> oh yeah than you can just do all batches and update the pointer
5612018-04-06T20:54:03 *** dafunkiz_ has quit IRC
5622018-04-06T20:54:03 <sipa> but they're basically all slower than what we're doing now (on fast hw at least)
5632018-04-06T20:55:01 <stevenroose> sipa: yeah you'd have the problem of knowing to what hash it's consistent. you'd need to keep heights in entries and iterate over all entries once and a while to see what the most recent dirty one is
5642018-04-06T20:55:26 <sipa> the *least* recent dirty one, yes
5652018-04-06T20:55:40 <sipa> thankfully, utxo entries already have a height
5662018-04-06T20:55:43 <stevenroose> excuseer :p
5672018-04-06T20:56:07 <sipa> unfortunately, that's the creation height and not really the modification height (which may differ in the case of a spend or a reorg)
5682018-04-06T20:56:54 <stevenroose> yeah I keep having a hard time picturing reorg handling there
5692018-04-06T20:57:10 <stevenroose> because when you don't have a txindex and you delete an entry, it's impossible to get it back :D
5702018-04-06T20:57:21 <sipa> oh, we have undo data
5712018-04-06T20:57:24 <sipa> the *.rev files
5722018-04-06T20:57:26 <stevenroose> do you keep like "revert objects" for the last X blocks
5732018-04-06T20:57:33 <stevenroose> ah
5742018-04-06T20:57:40 *** dafunkiz_ has joined #bitcoin-core-dev
5752018-04-06T20:58:20 <stevenroose> how many are there? (thinking hardfork races here where two chains constantly catch up with each other and fuck old nodes)
5762018-04-06T20:58:39 <sipa> there is one per block
5772018-04-06T20:58:46 <sipa> and we prune them along with the blocks themselves
5782018-04-06T20:59:04 <stevenroose> (technically wouldnt be a hardfork in that case though)
5792018-04-06T20:59:44 <stevenroose> wait, why keep for all? or are they only like pointers to the actual data?
5802018-04-06T21:00:05 <sipa> because we need to be able to reorg?
5812018-04-06T21:00:22 <stevenroose> I haven't seen any of that data in btcd's codebase, let me dig to that tomorrow :)
5822018-04-06T21:00:24 *** jigawatt has left #bitcoin-core-dev
5832018-04-06T21:00:40 <stevenroose> yeah I know, but well, reorging over half the chain is kinda unlikely, isn't it?
5842018-04-06T21:00:48 <sipa> yes
5852018-04-06T21:00:55 <stevenroose> I'd say at least before the last checkpoint makes no sense..
5862018-04-06T21:01:04 <sipa> checkpoints need to go away
5872018-04-06T21:01:09 <stevenroose> oh
5882018-04-06T21:01:28 <sipa> but yes, sure, it's unlikely that deep reorgs happen
5892018-04-06T21:01:42 <stevenroose> what's the fundamental problem with checkpoints?
5902018-04-06T21:01:52 <sipa> they confuse people
5912018-04-06T21:01:54 <sipa> :)
5922018-04-06T21:02:09 <sipa> they're seen as a security measure
5932018-04-06T21:02:42 <stevenroose> more as an efficiency tool :p I mean you can skip verification up to that point
5942018-04-06T21:03:02 <sipa> we have assumevalid for that now, which is far less invasive
5952018-04-06T21:03:12 <stevenroose> (f.e. when syncing a node I always "ask my friends for the latest block they trust" (i.e. check some explorers) and do --addcheckpoint
5962018-04-06T21:03:16 <sipa> it doesn't restrict which chain is valid
5972018-04-06T21:03:30 <sipa> it just skips validation for any block that is an ancestor of a known valid block
5982018-04-06T21:03:44 <sipa> but if the best chain we see is different than the assumevalid one, we'll accept it (after validating)
5992018-04-06T21:04:05 <stevenroose> ok yeah that's a better version of a checkpoint
6002018-04-06T21:04:31 <sipa> yes, assumevalid is updated from time to time, but we haven't modified checkpoints in years
6012018-04-06T21:04:39 <stevenroose> but checkpoints are also usefull against eclipse attacks when you're just getting started
6022018-04-06T21:04:46 <sipa> no
6032018-04-06T21:05:10 <stevenroose> at least they let you know something is up, no?
6042018-04-06T21:05:16 <sipa> they're useful against being spammed with low difficulty headers
6052018-04-06T21:05:25 <sipa> but that's independent of eclipse attacks
6062018-04-06T21:05:48 <sipa> we need backward headers sync to remove that dependency on checkpoints
6072018-04-06T21:06:12 <stevenroose> backward header sync?
6082018-04-06T21:06:23 *** laurentmt has joined #bitcoin-core-dev
6092018-04-06T21:06:28 <sipa> first learn the best header, and only then download headers along that path
6102018-04-06T21:06:51 <sipa> as opposed to downloading whatever header people give you, hoping that indeed it'll turn out to be one with more work than your current one
6112018-04-06T21:07:01 <sipa> (that's how it works now)
6122018-04-06T21:07:53 <stevenroose> how can you learn the best header?
6132018-04-06T21:08:17 <sipa> using a yet to be devised protocol :)
6142018-04-06T21:08:18 <stevenroose> asking all peers? (heighest checkpoint? :p)
6152018-04-06T21:08:22 <stevenroose> oh
6162018-04-06T21:08:29 *** laurentmt has quit IRC
6172018-04-06T21:09:06 <sipa> there are some ideas about random sampling, where someone can send you a merkle sum tree over all their headers, and then you randomly query it a number of times to see if they indeed have the distribution of pow they claim
6182018-04-06T21:09:09 <stevenroose> I recently thought about a backwards sync mechanism for initila utxo building. but I guess it's kinda not worth the effort when there is a good utxo cache
6192018-04-06T21:10:20 <sipa> and once you've done enough queries, you know they actually have a chain with a certain amount of work
6202018-04-06T21:10:31 <sipa> and if that amount of work is good enough, you can start downloading the actual headers
6212018-04-06T21:10:59 <setpill> sipa: wouldn't just believing the claimed amount of accumulated work + ban on lie work?
6222018-04-06T21:11:10 <stevenroose> are checkpoints really that bad?
6232018-04-06T21:11:36 <sipa> stevenroose: people seem to misunderstand that if checkpoints ever have an effect, bitcoin is broken
6242018-04-06T21:11:56 <sipa> i'm much more comfortable to have much weaker assumptions about correctness of the code
6252018-04-06T21:12:06 <sipa> (which includes the checkpoints)
6262018-04-06T21:12:17 <sipa> setpill: how is that better than what we have now?
6272018-04-06T21:13:15 <sipa> stevenroose: i don't think they're terrible, but we also don't really need them anymore, except for this tiny DoS concern
6282018-04-06T21:13:16 <setpill> sipa: last blocks are likely to have more pow behind them so are more expensive to maliciously craft
6292018-04-06T21:13:29 <stevenroose> "if checkpoints ever have an effect, bitcoin is broken" you mean that when code breaks validity of an old block that no one will validate because checkpoints?
6302018-04-06T21:14:10 <sipa> stevenroose: i mean that if checkpoints ever prevent the network from reorging to an attacker chain, it's clear that the concept of PoW itself is brokenb
6312018-04-06T21:15:38 *** promag has joined #bitcoin-core-dev
6322018-04-06T21:15:41 <stevenroose> and what if they prevent *a new node* from syncing a wrong chain? that's what their main use is imho
6332018-04-06T21:16:08 *** dafunkiz_ has quit IRC
6342018-04-06T21:16:10 <sipa> stevenroose: that's not what they do
6352018-04-06T21:16:18 <sipa> stevenroose: they just prevent OOM
6362018-04-06T21:16:30 <stevenroose> I mean even as a spam vector, a decent miner right now can prob create a quite significantly long chain that is use in size (1MB blocks) and has legit work
6372018-04-06T21:16:46 <stevenroose> s/use/huge/
6382018-04-06T21:16:46 <sipa> yes, absolutely - that's exactly the one thing they still do
6392018-04-06T21:17:00 <stevenroose> OOM?
6402018-04-06T21:17:07 <sipa> out of memory
6412018-04-06T21:17:15 <sipa> also, not actually blocks, just headers
6422018-04-06T21:17:31 <sipa> we don't download block data until a chain of validated headers is actually the best chain
6432018-04-06T21:18:18 <stevenroose> yeah true, so that would only work if eclipsed
6442018-04-06T21:18:25 <sipa> oh, and there is a known min amount of work
6452018-04-06T21:18:31 <sipa> independently of checkpoints
6462018-04-06T21:18:52 <stevenroose> "min amount of work at height x"?
6472018-04-06T21:18:53 <sipa> so we never accept a headers chain until it passes that point
6482018-04-06T21:19:00 <stevenroose> oh llike that
6492018-04-06T21:19:02 <stevenroose> cumulative
6502018-04-06T21:19:06 *** dafunkiz_ has joined #bitcoin-core-dev
6512018-04-06T21:19:09 <stevenroose> thats neat
6522018-04-06T21:19:59 <setpill> sipa: interesting, i hadnt heard of that; is that documented somewhere?
6532018-04-06T21:21:51 <sipa> -minimumchainwork cmdline option
6542018-04-06T21:22:59 <stevenroose> well, thanks for the insights :)
6552018-04-06T21:23:54 <sipa> yw!
6562018-04-06T21:26:22 <setpill> sipa: ahh, so its another "checkpoint-esque" thing, as in a hardcoded value that gets updated periodically?
6572018-04-06T21:27:05 <setpill> for a second i was under the impression pow inflation somehow had a lower bound ^^'
6582018-04-06T21:27:55 <sipa> setpill: there is a minimum difficulty, but it's trivial
6592018-04-06T21:28:19 <setpill> yeah and wont help much against a malicious chain
6602018-04-06T21:28:34 *** owowo has quit IRC
6612018-04-06T21:29:18 <sipa> a single modern CPU thread can create a minimum-difficulty block in a few minute
6622018-04-06T21:29:41 <sipa> modern HW miners can make 1000s per second
6632018-04-06T21:30:55 <setpill> yeah, so i thought there was a higher-than-that, actually useful lower bound on difficulty based on something i was unaware of
6642018-04-06T21:31:21 <sipa> there is also the max-divide-by-4 rule for difficulty changes
6652018-04-06T21:33:34 <stevenroose> sipa: is that a consensus rule??
6662018-04-06T21:33:40 <sipa> yes
6672018-04-06T21:33:42 <stevenroose> was not aware of that
6682018-04-06T21:33:47 <sipa> always has been
6692018-04-06T21:33:51 <stevenroose> is there a max-multiply-by-x one?
6702018-04-06T21:33:55 <sipa> yes, by 4
6712018-04-06T21:34:07 <stevenroose> aha
6722018-04-06T21:34:07 <sipa> neither rule has ever been hit
6732018-04-06T21:34:16 <sipa> on mainnet at least
6742018-04-06T21:34:33 <stevenroose> I hate the zero-diff-after-20-mins testnet rule
6752018-04-06T21:34:42 <sipa> haha
6762018-04-06T21:34:43 *** owowo has joined #bitcoin-core-dev
6772018-04-06T21:35:08 <stevenroose> I used to try and testnet mine with an old butterfly labs jalapeno but somehow was never hitting one
6782018-04-06T21:35:26 <sipa> you need to set your timestamp in the future :)
6792018-04-06T21:35:27 <stevenroose> btcd didn't notify gbt long polls on the 20 minute hit
6802018-04-06T21:35:50 *** promag has quit IRC
6812018-04-06T21:35:52 <stevenroose> also
6822018-04-06T21:36:57 *** Giszmo has quit IRC
6832018-04-06T21:43:49 *** timothy has quit IRC
6842018-04-06T21:44:20 *** Guyver2 has quit IRC
6852018-04-06T21:47:05 *** grafcaps has quit IRC
6862018-04-06T21:47:06 *** Randolf has joined #bitcoin-core-dev
6872018-04-06T21:54:47 *** jb55 has quit IRC
6882018-04-06T21:56:02 *** Giszmo has joined #bitcoin-core-dev
6892018-04-06T22:10:16 *** grafcaps has joined #bitcoin-core-dev
6902018-04-06T22:12:34 *** dafunkiz_ has quit IRC
6912018-04-06T22:18:02 *** dafunkiz_ has joined #bitcoin-core-dev
6922018-04-06T22:24:44 *** vicenteH has joined #bitcoin-core-dev
6932018-04-06T22:26:09 *** meshcollider has quit IRC
6942018-04-06T22:34:01 *** d9b4bef9 has quit IRC
6952018-04-06T22:34:04 *** promag has joined #bitcoin-core-dev
6962018-04-06T22:35:08 *** d9b4bef9 has joined #bitcoin-core-dev
6972018-04-06T22:36:02 *** d9b4bef9 has quit IRC
6982018-04-06T22:37:14 *** d9b4bef9 has joined #bitcoin-core-dev
6992018-04-06T22:39:13 *** cryptapus has quit IRC
7002018-04-06T22:45:27 *** dafunkiz_ has quit IRC
7012018-04-06T22:47:14 *** grafcaps has quit IRC
7022018-04-06T22:51:26 *** dafunkiz_ has joined #bitcoin-core-dev
7032018-04-06T22:51:43 *** cryptapus has joined #bitcoin-core-dev
7042018-04-06T22:51:43 *** cryptapus has quit IRC
7052018-04-06T22:51:43 *** cryptapus has joined #bitcoin-core-dev
7062018-04-06T22:53:28 *** meshcollider has joined #bitcoin-core-dev
7072018-04-06T22:58:26 *** rex4539 has quit IRC
7082018-04-06T22:58:39 *** CubicEarths has joined #bitcoin-core-dev
7092018-04-06T23:03:20 *** CubicEarths has quit IRC
7102018-04-06T23:03:49 *** CubicEarths has joined #bitcoin-core-dev
7112018-04-06T23:05:44 <jtimon> sipa: re protocol to know the best chain: couldn't compact pow proofs be better for that than random sampling?
7122018-04-06T23:06:20 <jtimon> hi stevenroose
7132018-04-06T23:06:34 *** dafunkiz_ has quit IRC
7142018-04-06T23:08:44 *** CubicEarths has quit IRC
7152018-04-06T23:08:48 <sipa> jtimon: compact pow proofs need consensus rules
7162018-04-06T23:08:52 <sipa> bitcoin doesn't have those
7172018-04-06T23:09:27 *** dafunkiz_ has joined #bitcoin-core-dev
7182018-04-06T23:09:54 <jtimon> sure, I mean assuming a compact proofs sf
7192018-04-06T23:14:07 *** setpill has quit IRC
7202018-04-06T23:16:25 <intcat> is there any (u)txo commitment design (close to becoming) a BIP?
7212018-04-06T23:16:45 <intcat> ive read some ml interaction between peter todd and bram cohen but it's a few years old and im not sure how much has happened on that since
7222018-04-06T23:33:38 *** Randolf has quit IRC
7232018-04-06T23:35:36 *** grafcaps has joined #bitcoin-core-dev
7242018-04-06T23:37:41 *** promag has quit IRC
7252018-04-06T23:40:47 *** dafunkiz_ has quit IRC
7262018-04-06T23:41:08 *** harrymm has quit IRC
7272018-04-06T23:44:49 *** arbitrary_guy has joined #bitcoin-core-dev
7282018-04-06T23:46:00 *** dafunkiz_ has joined #bitcoin-core-dev
7292018-04-06T23:48:29 *** Victorsueca has quit IRC
7302018-04-06T23:49:55 *** Victorsueca has joined #bitcoin-core-dev
7312018-04-06T23:51:16 *** harrymm has joined #bitcoin-core-dev
7322018-04-06T23:54:54 *** Sinclair6 has quit IRC