12018-12-13T00:10:53 *** Victorsueca has quit IRC
22018-12-13T00:26:04 *** Chris_Stewart_5 has quit IRC
32018-12-13T00:32:15 *** smaho has joined #bitcoin-core-dev
42018-12-13T00:35:46 *** smaho has quit IRC
52018-12-13T00:41:49 *** _cryptodesktop_i has joined #bitcoin-core-dev
62018-12-13T00:53:27 *** shesek has quit IRC
72018-12-13T00:53:49 *** shesek has joined #bitcoin-core-dev
82018-12-13T00:54:34 *** shesek has quit IRC
92018-12-13T00:54:56 *** shesek has joined #bitcoin-core-dev
102018-12-13T01:01:56 *** dviola has joined #bitcoin-core-dev
112018-12-13T01:06:22 *** miknotauro has quit IRC
122018-12-13T01:15:54 *** _cryptodesktop_i has quit IRC
132018-12-13T01:22:56 *** millerti has quit IRC
142018-12-13T01:38:31 *** promag has quit IRC
152018-12-13T01:40:41 <gmaxwell> promag: maybe everywhere we should split min_conf into min_conf_trusted and min_conf_untrusted. Most wallet behaior defaults to 0,1.
162018-12-13T01:41:55 <gmaxwell> (or more specifically, the coinselection code by default does something like, 6,6 and if that fails 1,1 and if that fails 0,1.)
172018-12-13T01:46:01 *** bitcoinjunior has joined #bitcoin-core-dev
182018-12-13T02:30:02 *** rh0nj has quit IRC
192018-12-13T02:30:58 *** ap4lmtree has quit IRC
202018-12-13T02:31:08 *** rh0nj has joined #bitcoin-core-dev
212018-12-13T02:31:23 *** ap4lmtree has joined #bitcoin-core-dev
222018-12-13T02:32:05 <phantomcircuit> gmaxwell, what?
232018-12-13T02:32:15 <phantomcircuit> of min_conf for our own transactions nvm
242018-12-13T02:34:28 *** DougieBot5000_ has joined #bitcoin-core-dev
252018-12-13T02:37:55 *** DougieBot5000 has quit IRC
262018-12-13T02:43:08 *** DougieBot5000_ is now known as DougieBot5000
272018-12-13T03:15:06 *** ap4lmtree has quit IRC
282018-12-13T03:15:35 *** ap4lmtree has joined #bitcoin-core-dev
292018-12-13T03:15:48 *** jonasschnelli has quit IRC
302018-12-13T03:17:47 *** AaronvanW has quit IRC
312018-12-13T03:22:57 *** jonasschnelli has joined #bitcoin-core-dev
322018-12-13T03:25:20 *** ap4lmtree has quit IRC
332018-12-13T03:28:07 *** ap4lmtree has joined #bitcoin-core-dev
342018-12-13T03:28:15 *** lopp has quit IRC
352018-12-13T03:38:54 *** bitcoinjunior has quit IRC
362018-12-13T03:53:06 *** Murch has quit IRC
372018-12-13T03:58:48 *** ghost43 has quit IRC
382018-12-13T03:59:49 *** ghost43 has joined #bitcoin-core-dev
392018-12-13T04:08:32 *** miknotauro has joined #bitcoin-core-dev
402018-12-13T04:20:40 *** Murch has joined #bitcoin-core-dev
412018-12-13T04:31:40 *** spinza has quit IRC
422018-12-13T04:42:32 *** adam3us has quit IRC
432018-12-13T04:44:47 *** spinza has joined #bitcoin-core-dev
442018-12-13T04:46:35 *** shesek has quit IRC
452018-12-13T04:48:01 *** shesek has joined #bitcoin-core-dev
462018-12-13T04:48:02 *** adam3us has joined #bitcoin-core-dev
472018-12-13T05:04:39 *** EREVAN has joined #bitcoin-core-dev
482018-12-13T05:04:47 *** EREVAN is now known as Guest24281
492018-12-13T05:05:35 *** droark has joined #bitcoin-core-dev
502018-12-13T05:18:49 *** ossifrage has quit IRC
512018-12-13T05:23:09 <droark> Hi. I have a question. On mainnet, about how long does Core need to run before the fee estimation code is considered accurate?
522018-12-13T05:23:51 <sipa> about twice as long as the delay you want to estimate for
532018-12-13T05:26:55 *** bitcoinsushi has joined #bitcoin-core-dev
542018-12-13T05:39:25 *** nelsonhb has joined #bitcoin-core-dev
552018-12-13T05:51:07 <droark> Thank you.
562018-12-13T05:55:46 *** midnightmagic has quit IRC
572018-12-13T06:01:18 <droark> Also, I seem to recall there being some reason why the code can't provide an estimate for TX insertion within one block. I can't remember it, though. Does it have to do with statistical variance?
582018-12-13T06:01:44 <sipa> yes, the next block may be 1 second away
592018-12-13T06:01:58 <sipa> no fee can give you a 95% chance of being included in that
602018-12-13T06:03:00 <droark> Thanks. One last question for now. To clarify, when you say twice as long as the delay, do you mean actual time or # of blocks? I assume the latter but it sounds like I might be wrong.
612018-12-13T06:04:10 <sipa> #blocks
622018-12-13T06:04:29 <droark> Thanks.
632018-12-13T06:07:10 *** hebasto has joined #bitcoin-core-dev
642018-12-13T06:12:07 *** midnightmagic has joined #bitcoin-core-dev
652018-12-13T06:14:04 *** jonasschnelli has quit IRC
662018-12-13T06:14:05 *** jonasschnelli has joined #bitcoin-core-dev
672018-12-13T06:24:28 *** ghost43 has quit IRC
682018-12-13T06:24:39 *** ghost43 has joined #bitcoin-core-dev
692018-12-13T07:01:57 *** Cory has quit IRC
702018-12-13T07:07:17 *** Pasha has joined #bitcoin-core-dev
712018-12-13T07:10:19 *** CodeBlue1776 has quit IRC
722018-12-13T07:10:28 *** Pasha is now known as Cory
732018-12-13T07:11:22 *** CodeBlue1776 has joined #bitcoin-core-dev
742018-12-13T07:38:33 *** shesek has quit IRC
752018-12-13T07:38:53 *** shesek has joined #bitcoin-core-dev
762018-12-13T07:38:53 *** shesek has joined #bitcoin-core-dev
772018-12-13T07:49:11 *** dviola has quit IRC
782018-12-13T07:51:34 *** shesek has quit IRC
792018-12-13T07:51:58 *** BGL has quit IRC
802018-12-13T07:54:42 *** shesek has joined #bitcoin-core-dev
812018-12-13T07:55:43 *** bitcoinsushi has quit IRC
822018-12-13T08:07:13 *** shesek has quit IRC
832018-12-13T08:08:35 *** shesek has joined #bitcoin-core-dev
842018-12-13T08:08:35 *** shesek has joined #bitcoin-core-dev
852018-12-13T08:13:31 *** shesek has quit IRC
862018-12-13T08:13:56 *** Murch has quit IRC
872018-12-13T08:15:00 *** nelsonhb has quit IRC
882018-12-13T08:49:40 *** booyah_ has joined #bitcoin-core-dev
892018-12-13T08:49:46 *** booyah has quit IRC
902018-12-13T08:57:09 *** TX1683 has quit IRC
912018-12-13T08:59:49 *** shesek has joined #bitcoin-core-dev
922018-12-13T08:59:49 *** shesek has joined #bitcoin-core-dev
932018-12-13T09:03:01 *** TX1683 has joined #bitcoin-core-dev
942018-12-13T09:08:30 *** owowo has joined #bitcoin-core-dev
952018-12-13T09:10:21 *** TheRec has quit IRC
962018-12-13T09:35:45 *** BGL has joined #bitcoin-core-dev
972018-12-13T10:02:16 *** Guyver2 has joined #bitcoin-core-dev
982018-12-13T10:04:34 *** Sentineo has quit IRC
992018-12-13T10:07:57 *** TheRec has joined #bitcoin-core-dev
1002018-12-13T10:07:57 *** TheRec has joined #bitcoin-core-dev
1012018-12-13T10:27:39 *** promag has joined #bitcoin-core-dev
1022018-12-13T10:34:43 *** timothy has joined #bitcoin-core-dev
1032018-12-13T10:35:06 *** spinza has quit IRC
1042018-12-13T11:05:01 *** rh0nj has quit IRC
1052018-12-13T11:06:08 *** rh0nj has joined #bitcoin-core-dev
1062018-12-13T11:25:54 *** spinza has joined #bitcoin-core-dev
1072018-12-13T11:29:02 *** Sentineo has joined #bitcoin-core-dev
1082018-12-13T11:35:24 *** Guyver2_ has joined #bitcoin-core-dev
1092018-12-13T11:37:09 <wumpus> unknown options in the config file are ignored, unknown command line options give an error, I think that's a good middle ground
1102018-12-13T11:38:07 <wumpus> if you explicitly provide -wallet= on *the command line* with a -disablewallet build I think it's fair to error out, you're requesting something it cannot do
1112018-12-13T11:38:28 *** Guyver2 has quit IRC
1122018-12-13T11:38:30 <wumpus> while if there's still left-over wallet configuration in the configuration file, well that probably shouldn't trip up too bad
1132018-12-13T11:42:50 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1142018-12-13T11:47:08 <promag> wumpus: I tend to agree with that
1152018-12-13T11:51:25 *** shesek has quit IRC
1162018-12-13T11:54:00 *** shesek has joined #bitcoin-core-dev
1172018-12-13T11:54:00 *** shesek has joined #bitcoin-core-dev
1182018-12-13T12:16:15 *** fanquake has joined #bitcoin-core-dev
1192018-12-13T12:16:55 <fanquake> wumpus I think #14741 and #14319 can go in
1202018-12-13T12:16:56 <gribble> https://github.com/bitcoin/bitcoin/issues/14741 | doc: Indicate -rpcauth option password hashing alg by dongcarl · Pull Request #14741 · bitcoin/bitcoin · GitHub
1212018-12-13T12:16:58 <gribble> https://github.com/bitcoin/bitcoin/issues/14319 | doc: Fix PSBT howto and example parameters by priscoan · Pull Request #14319 · bitcoin/bitcoin · GitHub
1222018-12-13T12:17:12 <fanquake> Both trivialish doc changes
1232018-12-13T12:18:56 *** shesek has quit IRC
1242018-12-13T12:20:21 *** shesek has joined #bitcoin-core-dev
1252018-12-13T12:20:21 *** shesek has joined #bitcoin-core-dev
1262018-12-13T12:25:50 *** shesek has quit IRC
1272018-12-13T12:26:11 *** shesek has joined #bitcoin-core-dev
1282018-12-13T12:27:35 *** shesek has quit IRC
1292018-12-13T12:28:06 <wumpus> fanquake: looks like it!
1302018-12-13T12:28:16 *** shesek has joined #bitcoin-core-dev
1312018-12-13T12:29:54 *** Chris_Stewart_5 has quit IRC
1322018-12-13T12:43:16 *** midnightmagic has quit IRC
1332018-12-13T12:46:58 *** midnightmagic has joined #bitcoin-core-dev
1342018-12-13T12:52:51 *** midnightmagic has quit IRC
1352018-12-13T12:55:18 *** miknotauro has quit IRC
1362018-12-13T12:58:41 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1372018-12-13T13:00:48 *** midnightmagic has joined #bitcoin-core-dev
1382018-12-13T13:03:12 <fanquake> Anyone found any issues testing rc1?
1392018-12-13T13:03:33 <wumpus> haven't seen any issues reported ye
1402018-12-13T13:15:41 <wumpus> not sure we've given it enough time yet, though
1412018-12-13T13:19:13 <wumpus> though rc period for backport releases needs to be less than for major releases
1422018-12-13T13:20:19 *** promag has quit IRC
1432018-12-13T13:20:30 <fanquake> fair enough
1442018-12-13T13:20:54 <fanquake> was just wondering, because there's at least one more fix, plus the PSBT documentation I'm about to backport we could get into .1
1452018-12-13T13:21:08 <fanquake> Otherwise we'll just wait for .2
1462018-12-13T13:21:25 <wumpus> but could discuss that at the meeting, along with whether to drop vista support in 0.18.0
1472018-12-13T13:21:38 <wumpus> right
1482018-12-13T13:22:56 <fanquake> I was actually considering getting up for the meeting this morning. Had a few things to bring up
1492018-12-13T13:24:02 <fanquake> Also qt 5.12 for 0.18.0
1502018-12-13T13:26:05 *** Guyver2__ has joined #bitcoin-core-dev
1512018-12-13T13:28:52 *** Guyver2_ has quit IRC
1522018-12-13T13:30:52 <wumpus> right
1532018-12-13T13:31:03 <wumpus> ok that'd be awesome :)
1542018-12-13T13:32:08 <wumpus> re: qt, we probably want #14849 (5.9.7) first
1552018-12-13T13:32:10 <gribble> https://github.com/bitcoin/bitcoin/issues/14849 | depends: qt 5.9.7 by fanquake · Pull Request #14849 · bitcoin/bitcoin · GitHub
1562018-12-13T13:32:37 <wumpus> that looks ready
1572018-12-13T13:32:44 <fanquake> wumpus yep. I was going to add that as my "high priority" PR. Once that's in I'll backport to 0.17, then start looking at 5.12
1582018-12-13T13:36:29 <fanquake> Guess I'll add something else to HP
1592018-12-13T13:39:39 <wumpus> heh! it's good to have this in, I think, so that it (hopefully) gets some manual testing before release, qt relies mostly on manual testing afterall
1602018-12-13T13:41:23 <fanquake> hopefully nobody has been trying to use the (now disabled) lcd number functionality :p
1612018-12-13T13:43:00 *** Guyver2__ has quit IRC
1622018-12-13T13:44:38 <wumpus> hah just looked what that does and it really emulates a LCD display on the screen, no I don't think we plan on using that :p
1632018-12-13T13:44:54 <wumpus> always good to speed up qt compile by removing unnecessary modules
1642018-12-13T13:46:32 <wumpus> I'm happy how fast we got the boost compile already in the same way
1652018-12-13T13:47:08 <fanquake> Yes. A NO_QT=1 depends build is quite fast
1662018-12-13T13:47:29 <wumpus> yep even on slow hw
1672018-12-13T13:58:10 *** shesek has quit IRC
1682018-12-13T13:58:11 *** Chris_Stewart_5 has quit IRC
1692018-12-13T13:58:35 *** shesek has joined #bitcoin-core-dev
1702018-12-13T14:12:39 *** shesek has quit IRC
1712018-12-13T14:13:51 *** shesek has joined #bitcoin-core-dev
1722018-12-13T14:14:36 <wumpus> speaking of which, my RISC-V node still runs fine <3
1732018-12-13T14:18:06 *** shesek has quit IRC
1742018-12-13T14:19:41 *** shesek has joined #bitcoin-core-dev
1752018-12-13T14:21:04 *** shesek has joined #bitcoin-core-dev
1762018-12-13T14:21:17 <fanquake> nice
1772018-12-13T14:26:26 *** shesek has quit IRC
1782018-12-13T14:26:45 *** shesek has joined #bitcoin-core-dev
1792018-12-13T14:29:02 *** wxss has joined #bitcoin-core-dev
1802018-12-13T14:29:06 *** shesek has joined #bitcoin-core-dev
1812018-12-13T14:30:30 <fanquake> wumpus have you found a RISC-V board that will run Qt yet :o
1822018-12-13T14:33:16 <wumpus> fanquake: I'm fairly sure it would run qt (using remote X server) :-) but no, I don't have the extension board with PCI-E functionality so I can't actually connect a monitor
1832018-12-13T14:33:29 *** shesek has quit IRC
1842018-12-13T14:33:58 *** shesek has joined #bitcoin-core-dev
1852018-12-13T14:33:58 *** shesek has joined #bitcoin-core-dev
1862018-12-13T14:35:09 <wumpus> SiFive Unleashed + Extension board + AMD gfx card could run qt, but the former two are really hard to get hold of. I think that's going to improve though, more RISC-V cores are in the works, maybe next year...
1872018-12-13T14:35:36 *** shesek has quit IRC
1882018-12-13T14:35:54 *** shesek has joined #bitcoin-core-dev
1892018-12-13T14:39:54 <fanquake> wumpus cool. Will have to track something down to experiment with.
1902018-12-13T14:44:40 *** shesek has quit IRC
1912018-12-13T14:45:25 *** shesek has joined #bitcoin-core-dev
1922018-12-13T14:45:35 *** shesek has quit IRC
1932018-12-13T14:45:36 *** shesek has joined #bitcoin-core-dev
1942018-12-13T14:48:55 <wumpus> I really hope there will be more affordable boards that can run Linux, next
1952018-12-13T14:53:51 *** belcher has quit IRC
1962018-12-13T14:54:49 *** belcher has joined #bitcoin-core-dev
1972018-12-13T15:01:56 *** shesek has quit IRC
1982018-12-13T15:03:01 *** shesek has joined #bitcoin-core-dev
1992018-12-13T15:03:01 *** shesek has joined #bitcoin-core-dev
2002018-12-13T15:06:40 *** millerti has joined #bitcoin-core-dev
2012018-12-13T15:08:22 *** shesek has quit IRC
2022018-12-13T15:09:37 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2032018-12-13T15:10:54 *** shesek has joined #bitcoin-core-dev
2042018-12-13T15:10:54 *** shesek has joined #bitcoin-core-dev
2052018-12-13T15:13:05 *** shesek has quit IRC
2062018-12-13T15:13:45 *** shesek has joined #bitcoin-core-dev
2072018-12-13T15:18:56 *** shesek has joined #bitcoin-core-dev
2082018-12-13T15:25:41 *** shesek has quit IRC
2092018-12-13T15:28:10 *** shesek has joined #bitcoin-core-dev
2102018-12-13T15:28:10 *** shesek has joined #bitcoin-core-dev
2112018-12-13T15:29:40 <luke-jr> would be nice to get POWER gitian builds merged, considering it's a much better deal than RISC-V etc
2122018-12-13T15:30:00 *** jarthur has joined #bitcoin-core-dev
2132018-12-13T15:30:35 <luke-jr> shesek: please fix your connection problems sometime in the next 4 hours
2142018-12-13T15:34:07 *** michaelsdunn1 has quit IRC
2152018-12-13T15:34:54 <wumpus> luke-jr: what's blocking that?
2162018-12-13T15:34:58 *** shesek has quit IRC
2172018-12-13T15:36:59 *** shesek has joined #bitcoin-core-dev
2182018-12-13T15:37:08 <luke-jr> wumpus: no idea; there's a minor rebase needed, but it sat idle for some time before that
2192018-12-13T15:39:34 <luke-jr> #14066
2202018-12-13T15:39:37 <gribble> https://github.com/bitcoin/bitcoin/issues/14066 | gitian-linux: Build binaries for 64-bit POWER by luke-jr · Pull Request #14066 · bitcoin/bitcoin · GitHub
2212018-12-13T15:40:01 <wumpus> I guess we need some more testers
2222018-12-13T15:40:33 <wumpus> also: did you fix cfields's remarks yet?
2232018-12-13T15:41:19 <luke-jr> I answered them, at least
2242018-12-13T15:41:42 <wumpus> ok
2252018-12-13T15:43:54 *** davec has quit IRC
2262018-12-13T15:44:28 *** shesek has quit IRC
2272018-12-13T15:45:23 *** shesek has joined #bitcoin-core-dev
2282018-12-13T15:45:35 *** shesek has joined #bitcoin-core-dev
2292018-12-13T15:46:49 *** promag has joined #bitcoin-core-dev
2302018-12-13T15:51:55 *** davec has joined #bitcoin-core-dev
2312018-12-13T15:52:53 *** shesek has quit IRC
2322018-12-13T15:53:00 *** michaelsdunn1 has joined #bitcoin-core-dev
2332018-12-13T15:53:56 *** shesek has joined #bitcoin-core-dev
2342018-12-13T16:02:03 <luke-jr> kinda wish I knew what was going on here: https://bugzilla.mozilla.org/show_bug.cgi?id=1512162
2352018-12-13T16:02:54 *** shesek has quit IRC
2362018-12-13T16:07:16 *** jungly has joined #bitcoin-core-dev
2372018-12-13T16:08:14 <wumpus> that's a *weird* issue, why would that only happen on one platform
2382018-12-13T16:08:54 *** shesek has joined #bitcoin-core-dev
2392018-12-13T16:08:54 *** shesek has joined #bitcoin-core-dev
2402018-12-13T16:10:03 *** promag has quit IRC
2412018-12-13T16:11:57 *** fanquake has quit IRC
2422018-12-13T16:12:12 <wumpus> could be a compiler bug
2432018-12-13T16:17:50 <luke-jr> yes, it's not very clear if the compiler or Linux is screwing up here
2442018-12-13T16:18:04 <luke-jr> since the VDSO is provided at runtime, I would expect Linux
2452018-12-13T16:18:32 <luke-jr> but this VDSO code hasn't changed since 2010
2462018-12-13T16:21:04 *** shesek has quit IRC
2472018-12-13T16:24:25 *** shesek has joined #bitcoin-core-dev
2482018-12-13T16:26:24 <cjd> Could have been interesting to see the backtrace on the gettimeofday() to see if there is perhaps something weird about where the timeval is placed
2492018-12-13T16:26:25 *** shesek has joined #bitcoin-core-dev
2502018-12-13T16:26:25 *** shesek has joined #bitcoin-core-dev
2512018-12-13T16:27:24 <cjd> I could imagine something like timeval placed on a 32 bit boundry, linux thinks it's supposed to be on the 64, trashes the stack cookie
2522018-12-13T16:27:49 <cjd> but I only looked at it for 5 min so my 2c is probably overpriced
2532018-12-13T16:31:31 *** shesek has quit IRC
2542018-12-13T16:32:45 *** shesek has joined #bitcoin-core-dev
2552018-12-13T16:34:27 *** shesek has quit IRC
2562018-12-13T16:36:59 *** shesek has joined #bitcoin-core-dev
2572018-12-13T16:43:59 *** promag has joined #bitcoin-core-dev
2582018-12-13T16:47:42 *** shesek has quit IRC
2592018-12-13T16:49:18 *** shesek has joined #bitcoin-core-dev
2602018-12-13T16:52:22 *** shesek has quit IRC
2612018-12-13T16:52:30 *** Tralfaz has joined #bitcoin-core-dev
2622018-12-13T16:55:03 *** shesek has joined #bitcoin-core-dev
2632018-12-13T16:55:03 *** shesek has quit IRC
2642018-12-13T16:55:03 *** shesek has joined #bitcoin-core-dev
2652018-12-13T16:56:59 *** shesek has quit IRC
2662018-12-13T16:58:03 *** timothy has quit IRC
2672018-12-13T16:58:26 *** promag has quit IRC
2682018-12-13T17:02:19 *** shesek has joined #bitcoin-core-dev
2692018-12-13T17:02:19 *** shesek has joined #bitcoin-core-dev
2702018-12-13T17:04:55 *** miknotauro has joined #bitcoin-core-dev
2712018-12-13T17:13:10 *** miknotauro has quit IRC
2722018-12-13T17:31:46 *** wxss has quit IRC
2732018-12-13T17:39:43 *** jungly has quit IRC
2742018-12-13T17:40:19 *** promag has joined #bitcoin-core-dev
2752018-12-13T17:45:27 *** danra has joined #bitcoin-core-dev
2762018-12-13T17:48:17 *** booyah_ is now known as booyah
2772018-12-13T17:53:27 *** spinza has quit IRC
2782018-12-13T17:54:03 *** wxss has joined #bitcoin-core-dev
2792018-12-13T18:02:09 *** ChanServ sets mode: +o wumpus
2802018-12-13T18:02:21 *** ChanServ sets mode: -o wumpus
2812018-12-13T18:03:02 *** kmels has joined #bitcoin-core-dev
2822018-12-13T18:10:02 *** promag has quit IRC
2832018-12-13T18:16:43 *** Chris_Stewart_5 has quit IRC
2842018-12-13T18:17:44 *** Murch has joined #bitcoin-core-dev
2852018-12-13T18:20:43 *** Giszmo has quit IRC
2862018-12-13T18:25:54 *** miknotauro has joined #bitcoin-core-dev
2872018-12-13T18:34:57 *** bitcoinjunior has joined #bitcoin-core-dev
2882018-12-13T18:39:56 *** Giszmo has joined #bitcoin-core-dev
2892018-12-13T18:41:40 *** spinza has joined #bitcoin-core-dev
2902018-12-13T18:42:44 <wumpus> looks like travis is failing on all new PRs
2912018-12-13T18:43:34 <wumpus> some extrememly scary error
2922018-12-13T18:43:39 <wumpus> https://travis-ci.org/bitcoin/bitcoin/jobs/467611982
2932018-12-13T18:44:34 <wumpus> (this is for #14951 which is only a RPC tests change and definitely shouldn't cause like this)
2942018-12-13T18:44:36 <gribble> https://github.com/bitcoin/bitcoin/issues/14951 | Revert "tests: Support calling add_nodes more than once" by MarcoFalke · Pull Request #14951 · bitcoin/bitcoin · GitHub
2952018-12-13T18:48:49 <gmaxwell> it appears to be saying that two threads are using the same fastrandomcontext at once.
2962018-12-13T18:49:05 *** hebasto has quit IRC
2972018-12-13T18:49:48 <gmaxwell> I didn't know we were running tsan in travis.
2982018-12-13T18:49:53 <wumpus> oof likely caused by #14624 then
2992018-12-13T18:49:56 <gribble> https://github.com/bitcoin/bitcoin/issues/14624 | Some simple improvements to the RNG code by sipa · Pull Request #14624 · bitcoin/bitcoin · GitHub
3002018-12-13T18:50:03 <jonasschnelli> probably...
3012018-12-13T18:50:42 *** chenpo has joined #bitcoin-core-dev
3022018-12-13T18:50:56 <gmaxwell> wumpus: looking at src/test/validation_block_tests.cpp it's totally misusing the rng in the test.
3032018-12-13T18:52:38 <sipa> why did travis not detect this in the pr?
3042018-12-13T18:54:00 <gmaxwell> I don't even really get why this paticular test exists-- like it's just starting lots of threads that call ProcessNewBlock at once. But ProcessNewBlock pretty much instantly takes a lock.
3052018-12-13T18:54:12 <wumpus> yeh it's clearly sharing a FastRandomContext between multiple threads
3062018-12-13T18:54:40 <sipa> actually, it's good to know tsan catches this
3072018-12-13T18:54:45 *** brianhoffman_ has joined #bitcoin-core-dev
3082018-12-13T18:54:46 <wumpus> sipa: hmm, maybe because your PR was from before TSAN checking was added
3092018-12-13T18:54:58 *** brianhoffman has quit IRC
3102018-12-13T18:54:59 *** brianhoffman_ is now known as brianhoffman
3112018-12-13T18:55:03 <sipa> ah
3122018-12-13T18:55:18 <gmaxwell> if so, how come it wasn't caught when tsan checking was added?
3132018-12-13T18:55:27 <gmaxwell> simple fix, and just a test bug regardless.
3142018-12-13T18:55:29 *** brianhoffman has quit IRC
3152018-12-13T18:55:42 <wumpus> it doesn't re-run all the travis runs for allPRs on every merge
3162018-12-13T18:55:48 <gmaxwell> ah.
3172018-12-13T18:55:50 <wumpus> might be an old cache, or who knows
3182018-12-13T18:55:51 <gmaxwell> though I still don't really get the utility of that test.
3192018-12-13T18:56:19 <wumpus> it's pretty nice that this was aught and that you were able to decipher the error :)
3202018-12-13T18:57:23 <gmaxwell> for some reason the tsan output is a little weird, it's only giving one side's backtrace.
3212018-12-13T18:57:44 <gmaxwell> Not my first time decyphering tsan output. :P
3222018-12-13T18:57:47 *** hex17or has quit IRC
3232018-12-13T19:00:06 *** hex17or has joined #bitcoin-core-dev
3242018-12-13T19:00:08 <provoostenator> Meeting?
3252018-12-13T19:00:13 <wumpus> I guess it tests the locking in ProcessNewBlock in case that becomes less granular inthe future
3262018-12-13T19:00:20 <wumpus> #startmeeting
3272018-12-13T19:00:20 <lightningbot> Meeting started Thu Dec 13 19:00:20 2018 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
3282018-12-13T19:00:20 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
3292018-12-13T19:00:37 <moneyball> hi
3302018-12-13T19:00:56 <wumpus> proposed topic: drop Windows Vista support, make minimum supported Windows 7
3312018-12-13T19:01:03 <moneyball> just one topic proposed during the week - jamesob
3322018-12-13T19:01:06 <gmaxwell> wumpus: ping people.
3332018-12-13T19:01:35 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb
3342018-12-13T19:01:38 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3352018-12-13T19:01:42 <provoostenator> (ah well, good way to get more people to review that RNG PR post merge)
3362018-12-13T19:01:43 <achow101> hi
3372018-12-13T19:01:44 <meshcollider> hi
3382018-12-13T19:01:45 <jonasschnelli> hi
3392018-12-13T19:02:04 <luke-jr> hi
3402018-12-13T19:02:09 <chenpo> hi
3412018-12-13T19:02:41 <wumpus> #topic High priority for review
3422018-12-13T19:02:45 <provoostenator> Let's call the topic Asta la la Vista :-)
3432018-12-13T19:03:19 <wumpus> https://github.com/bitcoin/bitcoin/projects/8 three PRs left on the list right now: #14336 #14565 #14573
3442018-12-13T19:03:22 <gribble> https://github.com/bitcoin/bitcoin/issues/14336 | net: implement poll by pstratem · Pull Request #14336 · bitcoin/bitcoin · GitHub
3452018-12-13T19:03:24 <gribble> https://github.com/bitcoin/bitcoin/issues/14565 | Overhaul importmulti logic by sipa · Pull Request #14565 · bitcoin/bitcoin · GitHub
3462018-12-13T19:03:26 <gribble> https://github.com/bitcoin/bitcoin/issues/14573 | qt: Add Window menu by promag · Pull Request #14573 · bitcoin/bitcoin · GitHub
3472018-12-13T19:03:27 <wumpus> provoostenator: good idea
3482018-12-13T19:03:58 <sipa> hi
3492018-12-13T19:03:59 <wumpus> the poll one should be (almost?) ready to merge
3502018-12-13T19:04:09 <provoostenator> Nominating #11082
3512018-12-13T19:04:11 <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
3522018-12-13T19:04:41 <provoostenator> Needs rebase, but really could use more review before that.
3532018-12-13T19:04:48 <wumpus> provoostenator: already needs rebase now
3542018-12-13T19:04:58 <wumpus> (and has, for 24 days)
3552018-12-13T19:05:12 <jonasschnelli> provoostenator: Maybe take that over from luke-jr
3562018-12-13T19:05:23 <sipa> does it need concept discussion?
3572018-12-13T19:05:34 <jonasschnelli> probably...
3582018-12-13T19:05:38 <provoostenator> I will eventually, but I have 15 other thing on my list. I also think it needs concept discussion.
3592018-12-13T19:06:02 <luke-jr> it doesn't need taking over, although if someone wants to rebase it sooner, I can push that
3602018-12-13T19:06:13 <provoostenator> Though we've had IRC chats about it before and everyone seemed to think it was at least a reasonable approach, compared to alterantives.
3612018-12-13T19:06:18 <wumpus> if it needs concept discussion, it definitely doesn't belong on the high priority for review list
3622018-12-13T19:06:26 <wumpus> yes, I think it's a reasonable approach
3632018-12-13T19:06:54 <provoostenator> Alright, maybe add it after rebase?
3642018-12-13T19:06:55 <wumpus> I dread making init option parsing even more involved, but it's better than the alternatives
3652018-12-13T19:07:16 <provoostenator> Dynamic wallet loading also needs this to make it good UX.
3662018-12-13T19:07:17 <sipa> yeah, fair
3672018-12-13T19:07:25 <wumpus> this needs *very* good tests
3682018-12-13T19:07:27 <provoostenator> Otherwise you'd have to manually load each time you start.
3692018-12-13T19:07:44 <sipa> provoostenator: i was expecting that to go in the qt settings
3702018-12-13T19:07:50 <wumpus> especially for the qt case ast hat's even crazier
3712018-12-13T19:08:14 <wumpus> how many levels of overlapping options can you have :)
3722018-12-13T19:08:21 <provoostenator> It has a bunch of Boost tests, but can probably use some QT tests and functional tests.
3732018-12-13T19:08:38 <wumpus> though this is supposed to get rid of most of the qt settings I guess?
3742018-12-13T19:08:46 <gmaxwell> obviously we should store what wallets get loaded in wallet.dat...
3752018-12-13T19:08:50 <jonasschnelli> Overlapping options is already problematic on the QT layer...
3762018-12-13T19:08:54 <provoostenator> Well, fewer levels thanks to my followup #12833
3772018-12-13T19:08:54 <luke-jr> wumpus: in a subsequent PR
3782018-12-13T19:08:55 <sipa> gmaxwell: wahaha
3792018-12-13T19:08:57 <gribble> https://github.com/bitcoin/bitcoin/issues/12833 | [qt] move QSettings to bitcoin_rw.conf where possible by Sjors · Pull Request #12833 · bitcoin/bitcoin · GitHub
3802018-12-13T19:09:01 <wumpus> gmaxwell: yesss
3812018-12-13T19:09:17 <wumpus> luke-jr: right
3822018-12-13T19:09:35 <provoostenator> But I'm reluctant to add more settings, as that makes rebasing 12833 a pain.
3832018-12-13T19:09:54 <wumpus> luke-jr: so eventually it'll replace non-pure-user-interface settings in the qt settings, I mean?
3842018-12-13T19:10:01 <provoostenator> (and that one also needs tests, I haven't delved into how to write QT tests yet, can someone make me a Python framework?)
3852018-12-13T19:10:04 <luke-jr> wumpus: ideally
3862018-12-13T19:10:28 <provoostenator> Yeah the idea is to get rid of QTsettings, with the exception of the datadir.
3872018-12-13T19:10:34 <jonasschnelli> I guess the QT settings files will always be there for window sizes and such... but non critical settings for sure
3882018-12-13T19:10:39 <wumpus> qtsettings is fine for *ui* settings
3892018-12-13T19:10:45 <wumpus> such as the last position of the window
3902018-12-13T19:10:46 <provoostenator> And what jonasschnelli says.
3912018-12-13T19:10:48 <wumpus> and things like that
3922018-12-13T19:10:48 <jonasschnelli> indeed
3932018-12-13T19:10:57 <wumpus> but not for settings shared with the core
3942018-12-13T19:11:02 <provoostenator> But not stuff that's shared with bitcoind like prune=
3952018-12-13T19:11:03 <jonasschnelli> yes
3962018-12-13T19:11:07 <wumpus> such as dbcache, etc
3972018-12-13T19:11:10 <wumpus> right.
3982018-12-13T19:11:25 <jonasschnelli> That was just a bypass of not having a way to write to a config section
3992018-12-13T19:11:45 <luke-jr> ?
4002018-12-13T19:12:09 <jonasschnelli> We wanted dbcache, proxys in Qt but didn't had a way to write to the config, so we added another layer
4012018-12-13T19:12:15 <wumpus> yep
4022018-12-13T19:12:37 <jonasschnelli> Which is no longer a clever approach once we have rw_config
4032018-12-13T19:12:46 <sipa> i'd be more comfortable if the set of modifiable settings (the list of wallets?) were distinct from those that aren't
4042018-12-13T19:12:53 <wumpus> we definitely don't want to write to bitcoin.conf directly, but having a separate configuration file that's writable is fine
4052018-12-13T19:13:02 <sipa> and have the GUI (where you really want everything to be modifable) directly modify bitcoin.conf
4062018-12-13T19:13:07 <wumpus> noooooo
4072018-12-13T19:13:14 <wumpus> don't write to bitcoin.conf, never
4082018-12-13T19:13:18 <sipa> if you use the GUI, the GUI manages the settings
4092018-12-13T19:13:23 <wumpus> conf should be read only
4102018-12-13T19:13:24 <sipa> if you don't, the config file does
4112018-12-13T19:13:26 <wumpus> we've been over this, please
4122018-12-13T19:13:35 <sipa> right, or have a separate config entirely
4132018-12-13T19:13:48 <wumpus> let's not change the idea now
4142018-12-13T19:13:56 <sipa> i really dislike having a UI edit things at the same level as the user is supposed to
4152018-12-13T19:13:59 <sipa> okay.
4162018-12-13T19:14:00 * sipa shuts up
4172018-12-13T19:14:40 <wumpus> we can have this discussion for a few years then never decide to do anything
4182018-12-13T19:14:50 <provoostenator> There's also a bunch of settings that can go straight into the wallet payload, like RBF and address type (though not their defaults).
4192018-12-13T19:14:58 <provoostenator> (or maybe even their defaults)
4202018-12-13T19:15:25 <provoostenator> #13044
4212018-12-13T19:15:26 <gribble> https://github.com/bitcoin/bitcoin/issues/13044 | [RFC] Long term plan for wallet command-line args · Issue #13044 · bitcoin/bitcoin · GitHub
4222018-12-13T19:15:39 <jonasschnelli> maybe this also raises the question how to distinct global settings between wallets
4232018-12-13T19:15:45 <provoostenator> That also simplifies QT, because wallet settings can be updated without any of the QTSettings stuff.
4242018-12-13T19:16:01 <wumpus> I'm unconfortable with writing to bitcoin.conf directly, as it is specified with -conf, which might point to a read-only configuration file, it should not be assumed it is writable
4252018-12-13T19:16:32 <wumpus> putting settings in the wallet again?
4262018-12-13T19:16:37 <gmaxwell> ugh
4272018-12-13T19:16:41 <sipa> ugh
4282018-12-13T19:16:42 <wumpus> yea deja vu...
4292018-12-13T19:16:50 <gmaxwell> putting settings in the wallet I think is even worse now that we have multiwallet.
4302018-12-13T19:16:51 <jonasschnelli> wumpus: there are the wallet flags now...
4312018-12-13T19:16:55 <provoostenator> wumpus: yes, that was the idea. Wallets have meta data entries. We already put binary flags in the wallet.
4322018-12-13T19:16:57 <jonasschnelli> but yeah...not settings
4332018-12-13T19:17:07 <sipa> provoostenator: originally all settings were in the wallet, it was pretty bad :)
4342018-12-13T19:17:12 <luke-jr> it might make sense for some subset of settings
4352018-12-13T19:17:14 <wumpus> sure, wallet-specific metadata should be stored in the wallet
4362018-12-13T19:17:18 <wumpus> but not settings
4372018-12-13T19:17:19 <gmaxwell> so you load another wallet and then your software starts behaving differently, madness!
4382018-12-13T19:17:23 <provoostenator> Yes, obviously only wallet settings.
4392018-12-13T19:17:27 <jamesob> write to wallet.conf.qt in datadir, have that override -conf settings?
4402018-12-13T19:17:30 <gmaxwell> yea sure wallet specific metadata sure fine whatever.
4412018-12-13T19:17:43 <jamesob> *bitcoin.conf.qt
4422018-12-13T19:17:44 <wumpus> jamesob: that's exactly the idea behind the bitcoin_rw.conf
4432018-12-13T19:17:44 <jonasschnelli> I just though about address type per wallet,... how do we handle different address types with different wallets?
4442018-12-13T19:17:46 <luke-jr> eg, you might have a non-segwit wallet, and a segwit wallet
4452018-12-13T19:17:49 <provoostenator> See the list in the issue I linked to above.
4462018-12-13T19:17:50 <jonasschnelli> *thought
4472018-12-13T19:18:07 <jamesob> wumpus: my bad, getting to the meetin glate
4482018-12-13T19:18:07 <wumpus> jamesob: it contains writable settings which override what is in bitcoin.conf
4492018-12-13T19:18:34 <wumpus> jamesob: this means being able to edit settings through RPC as well as the GUI
4502018-12-13T19:18:47 <jamesob> oh cool
4512018-12-13T19:18:49 <provoostenator> Some of these make sense int he wallet: addresstype, changetype, discardfee, fallbackfee, keypool, mintxfee, paytxfee, txconfirmtarget, etc. For others it doesn't make sense.
4522018-12-13T19:19:07 <gmaxwell> I don't see how fee settings make much sense in a wallet.
4532018-12-13T19:19:10 <wumpus> I'm not sure
4542018-12-13T19:19:17 <gmaxwell> Addresstype I could buy. maybe.
4552018-12-13T19:19:23 <provoostenator> gmaxwell: to make them behave consistently.
4562018-12-13T19:19:30 <jonasschnelli> Addresstype certenly,... fees, probably not
4572018-12-13T19:19:38 <sipa> addresstype will go away in the brave new world future anyway
4582018-12-13T19:19:43 <wumpus> yes
4592018-12-13T19:19:43 <gmaxwell> keypool no, it almost could go away.
4602018-12-13T19:19:44 <jonasschnelli> But I know a lot of people running a SW and a non-SW wallet in parallel
4612018-12-13T19:19:45 <provoostenator> But maybe fee preferences are more mempool weather dependent than wallet specific.
4622018-12-13T19:19:59 <gmaxwell> jonasschnelli: sounds brain damaged.
4632018-12-13T19:20:15 <gmaxwell> jonasschnelli: do you mean they just want to get keys with different address types?
4642018-12-13T19:20:18 <jonasschnelli> It may be,... but it's just how transition phases happens
4652018-12-13T19:20:42 <provoostenator> "keypool" might be replaced with a descriptor specific keypool, but I don't think that changes anything. Unless we stop using hardened derivation at the address level.
4662018-12-13T19:20:43 <wumpus> jamesob: please review #11082 :)
4672018-12-13T19:20:46 <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
4682018-12-13T19:20:53 <gmaxwell> in any case, changing wallets doesn't change what network you're on, so changing txfee settings doesn't really follow logically.
4692018-12-13T19:20:55 <sipa> provoostenator: yes keypool will go away as well
4702018-12-13T19:21:11 <wumpus> gmaxwell: agree
4712018-12-13T19:21:18 <luke-jr> gmaxwell: maybe if you're worried the wallets will get linked by behaviour?
4722018-12-13T19:21:21 <gmaxwell> provoostenator: keypool was configurable in the first place because it interacted with backup durability.
4732018-12-13T19:21:23 <jonasschnelli> gmaxwell: I guess they want a SW wallet that derives P2SH(P2WPKH) and and their old wallet that derives P2PKH... (and not mix them)
4742018-12-13T19:21:28 <jamesob> wumpus: roger that
4752018-12-13T19:21:33 <provoostenator> I do like the idea of giving config / rw_config wallet specific sections.
4762018-12-13T19:21:52 <wumpus> scoping settings on the wallet will be confusing for users, should imo be avoided if possible
4772018-12-13T19:22:14 *** riemann has joined #bitcoin-core-dev
4782018-12-13T19:22:16 <gmaxwell> provoostenator: so it's not clear to me that forward caching of keys would be configurable in the future, or at least that it would be anything but an advanced thing that users usually shouldn't mess with.
4792018-12-13T19:22:26 <wumpus> if anything it's very difficult to come up with a good user interface for that
4802018-12-13T19:22:33 <sipa> i think all of those can either be reasonably done at the node level, or turn into descriptor-specific things in the wallet (and thus not need config)
4812018-12-13T19:22:43 <provoostenator> gmaxwell: ah I see, making it "just work" could make sense
4822018-12-13T19:22:45 * luke-jr suggests ignoring wallet-specific settings for initial rwconf purposes <.<
4832018-12-13T19:22:52 <wumpus> luke-jr: yes, I agree
4842018-12-13T19:22:59 <gmaxwell> yea agreed.
4852018-12-13T19:23:01 <wumpus> luke-jr: one step at a time
4862018-12-13T19:23:06 <wumpus> we've been on this step for years
4872018-12-13T19:23:45 <wumpus> every time it's the same discussion, though I haven't heard the idea of settings in the wallet seriously proposed for a while :)
4882018-12-13T19:23:49 <gmaxwell> to the extent that there are wallet specific things they probably should be in the wallet so they move with the wallet. Regardless, they don't need to be part of the rwconf discussion.
4892018-12-13T19:24:27 <provoostenator> Agree regarding getting rwconf out the door without adding anything to it.
4902018-12-13T19:24:43 <jonasschnelli> wumpus: since its only addresstype that may be relevant and will go away over time,... I take back the argument that settings per wallet are relevant
4912018-12-13T19:24:53 <wumpus> jonasschnelli: thank you
4922018-12-13T19:25:01 <gmaxwell> (I'm just doubtful that there actually are more than a couple wallet specific things, addresstypes seem the most realistic of the ones mentioned here)
4932018-12-13T19:25:34 <provoostenator> Indeed, that seems the most important thing to keep consistent per wallet for privacy reasons.
4942018-12-13T19:25:50 <jonasschnelli> boolean things like disable-privatekeys can be handles with the 64bit wallet flags
4952018-12-13T19:25:54 <jonasschnelli> *handled
4962018-12-13T19:26:00 <provoostenator> One day everyone will send to bech32 addresses, and then we'll do v1 SegWit to start the problem over again :-)
4972018-12-13T19:26:11 <sipa> jonasschnelli: even that we don't need post descriptors
4982018-12-13T19:26:14 <wumpus> at the least, settings that are part of the wallet *should not* be part of the global options sytem
4992018-12-13T19:26:23 <wumpus> otherwise it just becomes too many levels
5002018-12-13T19:26:26 <sipa> you'd just not a have a descriptor with private keys if you don't want private keys
5012018-12-13T19:26:37 <gmaxwell> provoostenator: no, because v1 segwit doesnt' change the address type, bech32 already supports it.
5022018-12-13T19:26:49 <gmaxwell> (shocking, we already anticipated that problem... :P )
5032018-12-13T19:27:03 *** timothy has joined #bitcoin-core-dev
5042018-12-13T19:27:05 <luke-jr> gmaxwell: but p2sh^2 could (independnetly of segwitv1)
5052018-12-13T19:27:22 <gmaxwell> provoostenator: I'm sure there will be some broken sites, but it should be much less of an issue.
5062018-12-13T19:27:26 <jonasschnelli> sipa: is that similar of saying "you don't need to import private keys if you don't want to have private keys"?
5072018-12-13T19:28:08 <jonasschnelli> disable-private key seems to be another layer,... a footgun protection. But seems like we are going OT
5082018-12-13T19:28:12 <sipa> jonasschnelli: it would turn no-private-keys into a flag for wallet creation perhaps, to determine what descriptors a new wallet is born with; but it wouldn't affect anything later
5092018-12-13T19:28:17 <provoostenator> gmaxwell: but you still don't want to e.g. consistently use or not use schnorr, so you still need to maintain a preference. Even if it's all bech32.
5102018-12-13T19:28:24 <provoostenator> *do want
5112018-12-13T19:28:33 <luke-jr> so back to rwconf.. <.<
5122018-12-13T19:28:34 <sipa> anyway, sure - we could keep it as a footgun protection, but it seems kind of pointless
5132018-12-13T19:28:40 <gmaxwell> provoostenator: yes but thats just a property of which keys/descriptors are in the wallet.
5142018-12-13T19:28:42 <sipa> yeah, getting offtopic, sorry
5152018-12-13T19:28:51 <provoostenator> gmaxwell: good point
5162018-12-13T19:28:51 <wumpus> I think it's time to move to the next topic
5172018-12-13T19:29:06 <provoostenator> Descriptors are very useful.
5182018-12-13T19:29:10 <meshcollider> sipa: I assume you'd still need a way to fail if you try and import private keys though
5192018-12-13T19:29:43 <provoostenator> That's the foodgun protection.
5202018-12-13T19:29:46 <provoostenator> footgun
5212018-12-13T19:30:21 <provoostenator> I see two topics proposed here https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a
5222018-12-13T19:30:29 <provoostenator> jamesob and moneyball
5232018-12-13T19:30:32 <wumpus> #topic Asta la la Vista
5242018-12-13T19:30:51 <jamesob> Terminator movies?
5252018-12-13T19:31:03 <provoostenator> Dropping Vista support
5262018-12-13T19:31:07 <wumpus> so the idea to move the minimum supported windows to W7 recently came up in #14922
5272018-12-13T19:31:09 <gribble> https://github.com/bitcoin/bitcoin/issues/14922 | [WIP] windows: Set _WIN32_WINNT to 0x0601 (Windows 7) by ken2812221 · Pull Request #14922 · bitcoin/bitcoin · GitHub
5282018-12-13T19:31:29 <sipa> vista extended support ended on april 11, 2017
5292018-12-13T19:31:36 <wumpus> apparently this was already changed in the release notes #12546
5302018-12-13T19:31:38 <gribble> https://github.com/bitcoin/bitcoin/issues/12546 | [docs] Minor improvements to Compatibility Notes by randolf · Pull Request #12546 · bitcoin/bitcoin · GitHub
5312018-12-13T19:31:46 <sipa> end of mainstream support was in 2009
5322018-12-13T19:31:51 <luke-jr> for Linux, we just support latest stable distros, so if we mirror that for Windows, we can move to Win10 :P
5332018-12-13T19:32:05 <sipa> it was also one of the least popular windows releases...
5342018-12-13T19:32:14 <wumpus> so I think dropping support for Vista is pretty non-controversial
5352018-12-13T19:32:15 <wumpus> yes
5362018-12-13T19:32:22 <wumpus> it wasn't even popular at the time
5372018-12-13T19:32:29 <achow101> +1
5382018-12-13T19:32:32 <jonasschnelli> ack
5392018-12-13T19:32:34 <meshcollider> Yep
5402018-12-13T19:32:41 <sipa> so i think there is no real to keep it
5412018-12-13T19:32:51 <provoostenator> I'm a Mac fanboy, so conflict of interest :-)
5422018-12-13T19:32:54 <sipa> i actually expect less opposition than the opposition we got from dropping XP support
5432018-12-13T19:33:08 <gmaxwell> People are still using XP, I think less so than vista.
5442018-12-13T19:33:13 <luke-jr> sipa: this may be the first time we actually break XP, note
5452018-12-13T19:33:15 <wumpus> luke-jr: W7 is still used quite a lot by people unwilling to move to W10
5462018-12-13T19:33:16 <gmaxwell> er more so than vista.
5472018-12-13T19:33:38 <gmaxwell> the bigger issue will be W10 which is kinda spyware land windows from what I'm told.
5482018-12-13T19:33:50 <wumpus> yes
5492018-12-13T19:33:51 <meshcollider> Most people consider W7 as a strict improvement over vista so there should be minimal resistance :)
5502018-12-13T19:33:52 <sipa> is there anything between w7 and w10?
5512018-12-13T19:33:59 <luke-jr> 8.1
5522018-12-13T19:34:08 <wumpus> there's a windows 8 but I don't think it ever caught on much
5532018-12-13T19:34:40 <sipa> ah yes
5542018-12-13T19:34:47 <sipa> anyway, ack dropping vista support
5552018-12-13T19:34:52 <wumpus> ok
5562018-12-13T19:35:02 <sipa> based on what i read; not a windows user myself
5572018-12-13T19:35:04 <gmaxwell> that PR will it make it actually not work when it does otherwise?
5582018-12-13T19:35:24 *** bitcoin-git has joined #bitcoin-core-dev
5592018-12-13T19:35:24 <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14953: test: Make g_insecure_rand_ctx thread_local (master...Mf1812-testThreadLocal) https://github.com/bitcoin/bitcoin/pull/14953
5602018-12-13T19:35:24 *** bitcoin-git has left #bitcoin-core-dev
5612018-12-13T19:35:25 <MarcoFalke> Should fix the thread sanitizer issue ^
5622018-12-13T19:35:30 <gmaxwell> That isn't what we mean by supported on other platforms, on linux if you want to try running on really old stuff, you can try but you're own your own.
5632018-12-13T19:35:34 <wumpus> yes it sets the minimum API level so that it won't start in <W7, and makes it possible to use newer APIs
5642018-12-13T19:35:52 * luke-jr pokes MarcoFalke
5652018-12-13T19:36:13 <gmaxwell> ah it gates newer APIs. Is there a reason to not announce we don't support vista, but only bump that flag when we actually go to use one of those apis?
5662018-12-13T19:36:34 <gmaxwell> (just blocking software where it otherwise works doesn't really feel in the spirit of open source)
5672018-12-13T19:36:47 <wumpus> you're pedaling back now?
5682018-12-13T19:36:58 <luke-jr> gmaxwell: IIRC some PR wants to use Win7 API
5692018-12-13T19:37:05 <gmaxwell> luke-jr: okay.
5702018-12-13T19:37:05 <wumpus> FWIW qt is also going to break vista support
5712018-12-13T19:37:16 <luke-jr> I did suggest making the change in *that* PR..;
5722018-12-13T19:37:24 <wumpus> this is for 0.18
5732018-12-13T19:37:31 <wumpus> which is still a few months away
5742018-12-13T19:37:38 <gmaxwell> wumpus: no wumpus, I'm not "pedaling back" but the logic given above that we don't support older linux doesn't actually apply to actively breaking older windows.
5752018-12-13T19:37:50 <gmaxwell> If it's going to be broken by other changes in any case, thats fine then.
5762018-12-13T19:37:53 <luke-jr> eg, ionice_win bumps the Windows API version too, but wasn't merged yet
5772018-12-13T19:37:58 <wumpus> gmaxwell: fair enough but luke-jr was talking about supporting *only* windows 10
5782018-12-13T19:38:05 <wumpus> which is kind of extreme
5792018-12-13T19:38:18 <wumpus> W7 as bottom should be in line with all modern software
5802018-12-13T19:38:29 <luke-jr> wumpus: just mentioning it as a comparison to how we handle Linux distros ;)
5812018-12-13T19:38:35 <gmaxwell> from all I've heard running bitcoin on windows 10 sounds like it's probably a bad idea in general.
5822018-12-13T19:38:38 <meshcollider> From the PR, "#14881 is using inet_pton and it's only for Vista or later. So I create this PR just for that."
5832018-12-13T19:38:40 <gribble> https://github.com/bitcoin/bitcoin/issues/14881 | Tests: Contract testing for the procedure AddTimeData by mmachicao · Pull Request #14881 · bitcoin/bitcoin · GitHub
5842018-12-13T19:39:05 <gmaxwell> wumpus: okay, concern withdrawn. I just thought it would be sad to gratitiously break it, but since we have a reason to to do, good to go.
5852018-12-13T19:39:07 <luke-jr> gmaxwell: but only Windows 10 is actually supports anymore IIRC
5862018-12-13T19:39:16 <wumpus> luke-jr: yes, I know it was not seriously, but would be in line with what we do for wsay, MacOSX, but mac users like upgrading a lot more than windows users
5872018-12-13T19:39:41 <luke-jr> supported*
5882018-12-13T19:39:57 <wumpus> luke-jr: what does microsoft still officially support? only W10? are you sure?
5892018-12-13T19:40:03 * luke-jr double checks
5902018-12-13T19:40:16 <luke-jr> Win 8.1: Mainstream support ended on January 9, 2018
5912018-12-13T19:40:23 <luke-jr> apparently you can pay for support until 2023
5922018-12-13T19:40:42 <achow101> windows 7 will have security updates until jan 2020
5932018-12-13T19:40:56 <wumpus> great
5942018-12-13T19:41:06 <luke-jr> maybe I misunderstand extended support
5952018-12-13T19:41:18 <luke-jr> Win 7: Mainstream support ended on January 13, 2015. Extended support until January 14, 2020 (January 2023 for Professional and Enterprise, users will need to pay for security updates).[5][6]
5962018-12-13T19:41:39 <wumpus> so w7 is still more or less relevant
5972018-12-13T19:41:49 <achow101> yes
5982018-12-13T19:42:04 <achow101> and i'm pretty sure a lot of people still use it too
5992018-12-13T19:42:09 <wumpus> right
6002018-12-13T19:42:26 <wumpus> let's move to next topic
6012018-12-13T19:42:51 <wumpus> #topic "add address index" PRs (jamesob)
6022018-12-13T19:43:19 <jamesob> I've talked to a number of companies who're clamoring for an address index and we've got four attempts buuuut
6032018-12-13T19:43:49 <jamesob> sipa has potentially disabused me of the notion that an addr index is a legitimate approach to just about anything that isn't debugging or analysis
6042018-12-13T19:43:49 <jonasschnelli> jamesob: you mean unspent address index?
6052018-12-13T19:44:03 <jamesob> sure, or a more general script descriptor index
6062018-12-13T19:44:06 <jonasschnelli> I agree with sipa
6072018-12-13T19:44:22 <achow101> jamesob: for what are the companies trying to use an address index for?
6082018-12-13T19:44:32 <wumpus> an address index over the full block chain never was a good idea, one over the utxo set was considered but never made itin yet
6092018-12-13T19:44:41 <jamesob> in any case, I think we should have some sort of recommendation (and ideally a maintained piece of software) to recommend to the folks who want to do addr-indexy thing
6102018-12-13T19:44:44 <jamesob> s
6112018-12-13T19:45:04 <luke-jr> jamesob: "don't do that" :p
6122018-12-13T19:45:08 <gmaxwell> I think unspents indexes are fine, generally, but shouldn't be a priority for the project over other concerns.. if someone wants to come and do the work for them, and to do them well, great.
6132018-12-13T19:45:15 <jamesob> achow101: afaict mostly watching for activity on various addresses
6142018-12-13T19:45:23 <jonasschnelli> jamesob: you could do the address index externaly via p2p..... look at https://github.com/jonasschnelli/bitcoincore-indexd (experiment)
6152018-12-13T19:45:24 <provoostenator> Always happy to test and review those address index PRs. I think that on top of the new index scheme it's not too bad to maintain.
6162018-12-13T19:45:40 <luke-jr> what if we have a full TXO/script index, and only expose it in the GUI as a block explorer? (ie, no RPC to be abused)
6172018-12-13T19:45:42 <gmaxwell> jamesob: our general recommendation for watching is to import them into a watching wallet.
6182018-12-13T19:45:42 <wumpus> *watcing activiity* shouldn't require an index of everything over the whole block chain
6192018-12-13T19:45:44 <provoostenator> Or we could come up with a plugin system for indexes, though that's also a can of worms.
6202018-12-13T19:45:47 <jamesob> jonasschnelli: indeed, as well as projects like electrs
6212018-12-13T19:46:10 <wumpus> wasn't there some external project addressing this?
6222018-12-13T19:46:11 <jonasschnelli> I think we should not load a complete address index on the shoulders of Core
6232018-12-13T19:46:14 <gmaxwell> jamesob: and I think if there are limitations on importing for watching, we'd like to improve those.
6242018-12-13T19:46:18 <luke-jr> provoostenator: I thought we had that now
6252018-12-13T19:46:25 <sipa> gmaxwell: yup!
6262018-12-13T19:46:31 <wumpus> it's a *huge* database in any case
6272018-12-13T19:46:33 <jonasschnelli> But I agree, externally makes it a bit more complex
6282018-12-13T19:46:41 <jamesob> in the next few weeks I'm going to be getting in touch with companies to get a more concrete idea of the usecases, so I'll report back with what I find
6292018-12-13T19:46:59 <provoostenator> luke-jr: have what? People can compile their own client of course, but that's pretty high bar and can easily lead to rebase nightmares.
6302018-12-13T19:47:01 <wumpus> bitcoin core is not meant as a chain analysis platform
6312018-12-13T19:47:06 <luke-jr> wumpus: I did it in ~2 GB a year or so ago, IIRC
6322018-12-13T19:47:07 <wumpus> you can do forensics using your own tools
6332018-12-13T19:47:13 <jamesob> wumpus: very much agree
6342018-12-13T19:47:20 <gmaxwell> jonasschnelli: right the problem for externally, is that implementing blockchain consistency is non-trivial and in my expirence most people who want to build an index don't want to deal with that.
6352018-12-13T19:47:44 <gmaxwell> jamesob: more information on the actual usecases would be very helpful.
6362018-12-13T19:47:49 <wumpus> it's non trivial but not rocket science either
6372018-12-13T19:47:54 <jonasschnelli> Sadly people expect fast BIP44 recovery (incl. history). This seems to be the most prominent real-world usecase for an address index
6382018-12-13T19:48:07 <wumpus> it shouldn't be that anything non-trivial needs to end up[ in bitcoin core
6392018-12-13T19:48:12 <wumpus> we're not the world of non-trivial solutions
6402018-12-13T19:48:26 <gmaxwell> jonasschnelli: most (I believe most, many at least) electrum servers don't do history, so I'm not entirely clear on the including history part of your comment.
6412018-12-13T19:48:35 <provoostenator> Let's also not forget to update #14053 with concept ACK / NACK so people don't waste time on trying it if it's undesirable.
6422018-12-13T19:48:38 <gmaxwell> wumpus: heh.
6432018-12-13T19:48:39 <gribble> https://github.com/bitcoin/bitcoin/issues/14053 | Add address-based index (attempt 4?) by marcinja · Pull Request #14053 · bitcoin/bitcoin · GitHub
6442018-12-13T19:48:43 <jonasschnelli> gmaxwell: IMO they do (index everything)
6452018-12-13T19:49:04 <jonasschnelli> (not electrum personal server though)
6462018-12-13T19:49:04 <wumpus> I mean this is another thing that's been *years*
6472018-12-13T19:49:09 <gmaxwell> jonasschnelli: no, I know that for a fact-- many electrum servers don't index history.
6482018-12-13T19:49:11 <wumpus> really, no one has been able to build a solution to this?
6492018-12-13T19:49:14 <gmaxwell> (I just don't know if its most of them)
6502018-12-13T19:49:18 <wumpus> not one of all those companies that want it?
6512018-12-13T19:49:24 <jamesob> provoostenator: and its sibling #14035
6522018-12-13T19:49:25 <luke-jr> wumpus: people capable of it don't want it :p
6532018-12-13T19:49:26 <gribble> https://github.com/bitcoin/bitcoin/issues/14035 | Utxoscriptindex by mgrychow · Pull Request #14035 · bitcoin/bitcoin · GitHub
6542018-12-13T19:49:37 <sipa> wumpus: blockstream.info uses electrs, which seems to work very well
6552018-12-13T19:49:43 <gmaxwell> wumpus: I think the process of learning enough to do it well does a pretty good job of convincing you that you don't want it.
6562018-12-13T19:49:51 <wumpus> gmaxwell: heh
6572018-12-13T19:49:59 <provoostenator> jonasschnelli: BIP44 recovery can be handled once we have descriptor support for importmulti and slightly more sane behavior (or a replacement for) the keypool.
6582018-12-13T19:50:02 <wumpus> sipa: that's a good suggestion then!
6592018-12-13T19:50:31 <gmaxwell> provoostenator: history recovery requires a scan of the chain, or a phenominally expensive index.
6602018-12-13T19:50:31 <jonasschnelli> provoostenator: you can't recover the history without an address index or a complete rescan (which is not possible for pruned peers)
6612018-12-13T19:50:43 <wumpus> https://github.com/romanz/electrs
6622018-12-13T19:50:51 <provoostenator> Ok, rescan for pruned nodes isn't there yet.
6632018-12-13T19:50:53 <gmaxwell> jonasschnelli: note that an index is also not compatible with pruning. :P
6642018-12-13T19:50:56 <wumpus> alrady had it bookmarked apparently :)
6652018-12-13T19:50:59 <jonasschnelli> The only light here is the scantxoutset where you can recover within seconds but not the spent-history
6662018-12-13T19:51:19 <provoostenator> Though you could use the neutrino filters for that and then refetch the relevant blocks.
6672018-12-13T19:51:24 <jonasschnelli> gmaxwell: it could be,... if we just keep the pointers and load the data via p2p once requested
6682018-12-13T19:51:32 <gmaxwell> jonasschnelli: I think we'd get a lot more bang for our buck working on making 'wallet without history before x' (pruned wallet) a first class supported thing.
6692018-12-13T19:51:48 <jonasschnelli> gmaxwell: completely agree on this
6702018-12-13T19:51:54 <gmaxwell> jonasschnelli: transfering 200GB of data to do the input is not really reasonable...
6712018-12-13T19:52:24 <gmaxwell> (and then not saving it... this would be pretty harmful to the p2p network if people were actually patient enough to use it. :) )
6722018-12-13T19:52:28 <jonasschnelli> gmaxwell: I mean there is a PR that enabled txindex with pruning... and fetches the tx over p2p once requested outside the kept block area
6732018-12-13T19:52:39 <jonasschnelli> which is slow.... but for debugging proposes okay
6742018-12-13T19:52:43 <luke-jr> (this makes me think again of the concept of prune-to-slow-storage where you can plug in a USB drive when/if you need old data..)
6752018-12-13T19:52:50 <gmaxwell> jonasschnelli: oh, I didn't recall that PR.
6762018-12-13T19:53:06 <wumpus> luke-jr: prune-to-tape *hides*
6772018-12-13T19:53:11 <provoostenator> luke-jr: slow storage being the internet? :-)
6782018-12-13T19:53:17 <jonasschnelli> #13014
6792018-12-13T19:53:19 <gribble> https://github.com/bitcoin/bitcoin/issues/13014 | Allow txindex in prune mode by jonasschnelli · Pull Request #13014 · bitcoin/bitcoin · GitHub
6802018-12-13T19:53:30 <luke-jr> provoostenator: could be local
6812018-12-13T19:53:53 <gmaxwell> jonasschnelli: bip 157 filtering could be used similarly.
6822018-12-13T19:54:15 <gmaxwell> jonasschnelli: e.g. saved the bip157 filters locally, and scan against them.
6832018-12-13T19:54:15 <jonasschnelli> yes... less disk space, more time spent on filtering... but same idea
6842018-12-13T19:54:37 <luke-jr> hmm
6852018-12-13T19:54:55 <luke-jr> what data do they want returned from the index? maybe we don't need to retain/fetch the block
6862018-12-13T19:54:58 *** dviola has joined #bitcoin-core-dev
6872018-12-13T19:55:01 *** rh0nj has quit IRC
6882018-12-13T19:55:08 <luke-jr> eg, if the client would be happy with a list of block numbers or txids..
6892018-12-13T19:55:08 <jamesob> are any of the BIP157/158 PRs on the high prio list? if not, they should be
6902018-12-13T19:55:11 <gmaxwell> wumpus: sadly, freeking tape costs about as much as HDDs today... LTO7 tapes (6TB) cost about $70.
6912018-12-13T19:55:28 <wumpus> gmaxwell: ouch
6922018-12-13T19:55:38 <provoostenator> So the wallet recovery flow would be: add descriptors, download BIP-157 filters (if you didn't keep them), process filters, fetch a few blocks. User can immediately use their wallet to receive and send.
6932018-12-13T19:55:48 <wumpus> gmaxwell: I guess they're more reliable though
6942018-12-13T19:55:49 <luke-jr> jamesob: making light wallets stronger has a negative impact on Bitcoin :/
6952018-12-13T19:56:08 *** rh0nj has joined #bitcoin-core-dev
6962018-12-13T19:56:23 <gmaxwell> provoostenator: I don't really think thats a viable long term workflow either... rather if you couldn't afford the space to keep them you probably don't want the time/bandwidth to download them.
6972018-12-13T19:56:27 <sipa> luke-jr: BIP158 helps our local wallet too
6982018-12-13T19:56:28 <provoostenator> luke-jr: not making them possible sends most people to web wallets, not to full nodes.
6992018-12-13T19:56:35 <jamesob> luke-jr: better light wallets might help adoption
7002018-12-13T19:56:39 <gmaxwell> luke-jr: I'm only interested in it for the local wallets.
7012018-12-13T19:57:00 <luke-jr> jamesob: better to not have that adoption..
7022018-12-13T19:57:05 <luke-jr> sipa / gmaxwell: yes, that's a point
7032018-12-13T19:57:22 <provoostenator> gmaxwell: recovery should be a rare thing anyway, I assume it only happens after a disaster. In which case you'd probably download the whole chain again anyway.
7042018-12-13T19:57:32 <sipa> provoostenator: it should be.
7052018-12-13T19:57:34 <gmaxwell> jamesob: history has shown otherwise, bip158 doesn't make lite wallets fundimentally more usable than they are now. They're still massively worse than server driven wallets like electrum or web wallets.
7062018-12-13T19:58:05 <gmaxwell> provoostenator: right but thats also a reason that fetching them when you don't have them isn't that interesting, IMO.
7072018-12-13T19:58:22 <jamesob> good point - but I guess if existing light wallets switched to bip157 it'd at least ease load on existing full nodes
7082018-12-13T19:58:41 <gmaxwell> jamesob: what light wallets?
7092018-12-13T19:58:55 <provoostenator> gmawell is that _because_ of bip158 or just because there aren't that many developers working on light (non web, non electrum) wallets? That could change over time.
7102018-12-13T19:58:59 <wumpus> 1 minute to go
7112018-12-13T19:59:04 <jamesob> anything using bloom-filter-based SPV
7122018-12-13T19:59:12 <gmaxwell> jamesob: at least connections I see there is virtually no acutal usage of p2p lite wallets anymore.
7132018-12-13T19:59:15 <wumpus> please wrap up the meeting
7142018-12-13T19:59:44 <jamesob> I'll report on any compelling usecases I find for addr index, but I suspect sipa et al are right that that's usually just the Wrong way
7152018-12-13T19:59:48 <gmaxwell> provoostenator: P2p lite wallets that scan the chain just end up with a very poor user expirence.
7162018-12-13T20:00:06 <jamesob> in the meantime I recommend we give concept ACK/NACK to outstanding PRs which are related
7172018-12-13T20:00:11 <gmaxwell> and that doesn't really have anything to do with bip158 vs bip37.
7182018-12-13T20:00:25 *** timothy has quit IRC
7192018-12-13T20:00:36 <gmaxwell> (in fact BIP158 is somewhat worse, but slightly less of a privacy disaster)
7202018-12-13T20:00:37 <wumpus> #endmeeting
7212018-12-13T20:00:37 <lightningbot> Meeting ended Thu Dec 13 20:00:37 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
7222018-12-13T20:00:37 <lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.html
7232018-12-13T20:00:37 <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.txt
7242018-12-13T20:00:37 <lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.log.html
7252018-12-13T20:01:03 <provoostenator> gmaxwell: I was thinking mostly about Lightning wallets that rely on a lightweight wallet but don't have support for on chain (other than topping up channels). Those don't need to care about unconfirmed transactions.
7262018-12-13T20:01:10 <wumpus> jamesob: I think poeople have become tired of arguing, or even explicitly NACKing those things
7272018-12-13T20:01:11 <gmaxwell> (it's worse because it takes a client more time to scan the chain than with BIP37, as it has to get quite a bit more data)
7282018-12-13T20:01:51 <gmaxwell> wumpus: +1
7292018-12-13T20:02:18 <jamesob> wumpus: makes sense. I'll come up with a reply to both PRs that tries to explain why they're not a fit for core
7302018-12-13T20:02:30 <wumpus> at least I have, I mean, if you've been in this since 2011 or so, and have to keep telling people something is abad idea while it becomes ever a worse idea (due to block chain growth)
7312018-12-13T20:02:32 <jamesob> and maybe we can think about adding something to the developer guidelines about indexing
7322018-12-13T20:02:41 <gmaxwell> UTXO-iyx indexes I generally concept ack (at least if they're ones that aren't specific about 'addresses' but work for spk's generally), history ones, I have NAKed.
7332018-12-13T20:02:54 <wumpus> gmaxwell: yep
7342018-12-13T20:03:30 <sipa> provoostenator: yes, i think BIP157 may be useful for a new class of clients that may become popular
7352018-12-13T20:03:56 <jamesob> gmaxwell: so a scriptpubkey index on UTXOs is worthwhile? is that a generally useful index to have for use by core?
7362018-12-13T20:03:58 <gmaxwell> jamesob: we have a problem that there aren't good instructions on watchign without an index (and we likely have bugs and limitations that make it unattractive that haven't been adequately reported).. And for the analysis usecases what people want is an analysis platform, which we're not going to provide but don't really have much to recommend.
7372018-12-13T20:04:29 <jamesob> gmaxwell: agree, I don't want to spend my time on the analysis usecases
7382018-12-13T20:05:02 <sipa> jamesob: i dislike UTXO index equally, but it's certainly less of a burden
7392018-12-13T20:05:05 <gmaxwell> jamesob: It's both useful (esp if we gain pruned wallet support-- but also scantxoutset speedup), but also it's not incompatiable with scalablity.
7402018-12-13T20:05:39 <jamesob> good point
7412018-12-13T20:05:40 <sipa> gmaxwell: assuming having a local UTXO set is forever
7422018-12-13T20:05:52 <gmaxwell> jamesob: it's my personal belief that within 10 years most users won't ever bother fetching the far back history (or only doing so as a background test and not keeping the results).
7432018-12-13T20:06:03 <gmaxwell> sipa: yes, well when otherwise becomes viable my thinking may change!
7442018-12-13T20:06:09 <sipa> (which at least for the foreseeable future it will be)
7452018-12-13T20:06:23 <jamesob> #utreexo :)
7462018-12-13T20:06:44 <gmaxwell> jamesob: and moreover, with the exception of ultra high end commercial installs, few users would have the resources or patience to fetch 15 years of bitcoin history.
7472018-12-13T20:07:19 <wumpus> doesn't need to be 'forever', it's useful *now*
7482018-12-13T20:07:23 <gmaxwell> jamesob: so any of indexes on that history are is trouble for the ecosystem, as the indexes are many times less viable resource wise than the history itself.
7492018-12-13T20:07:34 <jamesob> gmaxwell: so we'd expect "hobbyists" to run pruned nodes, or do you see some more sophisticated compaction of chain history?
7502018-12-13T20:07:57 <sipa> jamesob: providing bulk access to sequential data is *cheap*
7512018-12-13T20:08:14 <sipa> i expect nearly everyone to not bother with having the full chain
7522018-12-13T20:08:14 <gmaxwell> jamesob: My expirence is that commerical parties are often less tolerant of resource usage and more willing to accept "query blockchain.info" than hobbists.
7532018-12-13T20:08:17 <wumpus> UTXO db=sophisticaed compaction of chain history
7542018-12-13T20:08:35 <gmaxwell> jamesob: so I think you probably have that part backwards. :)
7552018-12-13T20:08:49 <jamesob> wumpus: right but you have to validate the construction of a utxo db...
7562018-12-13T20:09:13 <gmaxwell> jamesob: I expect that in the future nodes will download a historical UTXO set, verifying it against an assumevalid in their code, and then sync forward from that.
7572018-12-13T20:09:36 <wumpus> jamesob: oh sure if you really want to bypass validation
7582018-12-13T20:09:41 <sipa> (or in the even further future, in a utreexo or magical accumulator world, not even have that UTXO set...)
7592018-12-13T20:10:12 <gmaxwell> sipa: yes, but so far all those proposals are too slow to use or increase bandwidth 10x fold, which isn't a winning tradeoff.
7602018-12-13T20:10:13 <jamesob> gmaxwell wumpus: okay so that leads back into the discussion of some kind of utxo set commitment
7612018-12-13T20:10:15 <wumpus> jamesob: you can have my UTXO set if you want, no guarantees though :<
7622018-12-13T20:10:24 <jamesob> wumpus: ha generous
7632018-12-13T20:10:26 <gmaxwell> jamesob: no, it has nothing to do with a utxo set commitment in fact.
7642018-12-13T20:10:27 <sipa> gmaxwell: utreexo is at 2x bandwidth or so
7652018-12-13T20:11:06 <sipa> (according to simulation results i heard)
7662018-12-13T20:11:12 <jamesob> gmaxwell: you're talking about "assumevalid" in the existing sense and not an assumevalid on a utxo hash, right?
7672018-12-13T20:11:22 <provoostenator> What does "utreexo" stand for again?
7682018-12-13T20:11:29 <sipa> provoostenator: utxo + tree
7692018-12-13T20:11:36 <provoostenator> uTreeXO?
7702018-12-13T20:11:38 <jamesob> provoostenator: https://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-utxo-accumulators-and-utreexo/
7712018-12-13T20:11:43 <gmaxwell> jamesob: a utxo has, but "utxo commitment" has generally ment putting it in blocks which is irrelevant here.
7722018-12-13T20:12:14 <sipa> unfortunately it needs blocks that commit to the utxo *proofs* (not the root) for DoS resistance
7732018-12-13T20:13:37 <sipa> imo
7742018-12-13T20:13:38 <jamesob> gmaxwell: so what you're saying is that 15 years from now, you think that we'll have a hardcoded hash of some historical utxo snapshot (a la assumevalid) and users will start their chain validation from there?
7752018-12-13T20:13:51 <gmaxwell> jamesob: doing an assumevalid merely needs a hash, which we already have... though it would be better if nodes could record their hash for each block to make review easier.
7762018-12-13T20:14:20 <gmaxwell> jamesob: Yes, short of breakthroughs in ZKPs I don't think anything else will be viable.
7772018-12-13T20:15:38 <gmaxwell> We needed to constrain blocksize to a fraction of the current size to avoid time to initial sync constantly growing up even assuming computers/bandwidth/disk improving by 18% or whatever year over year. Since that isn't the case the validation time will continue to grow.
7782018-12-13T20:16:01 <jamesob> gmaxwell: makes sense
7792018-12-13T20:16:54 <gmaxwell> And it's already large enough that it pushes many companies to prefer hosted APIs... (even if they don't mind the initial sync time, the risk of having to take it again while the service is down is pretty unattractive). Some end users are less tolerant of loading delays, some more.
7802018-12-13T20:17:40 <gmaxwell> And in any case, the same security argument for assumevalid-scripts would apply: if someone can get away with making really obvious bad changes to the code you're running, all bets are off.
7812018-12-13T20:18:20 <gmaxwell> Main gaps in implementation now are that it would be really good to make it easier to audit an assumevalid hash, which maybe implies adding an incrementally updatable hash.
7822018-12-13T20:18:32 <gmaxwell> And then just storing and fetching snapshots for the p2p protocol.
7832018-12-13T20:19:30 <jamesob> i.e. sipa's rolling MUHASH
7842018-12-13T20:20:05 <sipa> i'm leaning towards the ECMH approach now, as it's easier to cache for prevalidated txn etc
7852018-12-13T20:20:34 <sipa> it's more CPU, but in generally CPU time that can easily move to separate threads etc
7862018-12-13T20:21:00 <jamesob> (for anyone curious about rolling UTXO set hashes: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014337.html)
7872018-12-13T20:24:43 <luke-jr> will it upset anyone's reviewing if I rebase #11082 now?
7882018-12-13T20:24:45 <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
7892018-12-13T20:25:00 <gmaxwell> I don't disagree, however for the 'sync from a snapshot and everyone can validate the hash', we could do something more boring and just compute a conventional hash of the utxo set in the background every 25000 blocks.
7902018-12-13T20:25:38 <gmaxwell> There is no particular need to know the hash at every single block... sync logistics mean that it wouldn't be realistic to support syncing from an arbritary block anyways.
7912018-12-13T20:27:55 <gmaxwell> The other issue with using muash/ecmh with a sync is that it doesn't really lend itself to incremental validation of fetched chunks. So it would probably need to be used in addition to another hash in any case.
7922018-12-13T20:28:39 <gmaxwell> incremental validation of chunks-- I mean you're gonna download the 2gb utxo set from 8 peers... okay, great, you get the whole thing and the echm doesn't match. What now? You don't even know what peer gave you the bad data.
7932018-12-13T20:28:49 <luke-jr> btw, it's not too late to reduce block sizes; if we get an organized dev-supported proposal following Lightning going production-ready, I think we can get enough support
7942018-12-13T20:29:14 <gmaxwell> luke-jr: it is too late because it already takes much larger to sync the chain than many people will tolerate.
7952018-12-13T20:29:30 <luke-jr> gmaxwell: we can catch up with time
7962018-12-13T20:29:44 <gmaxwell> Also, the harm from the significant reduction required would be considerable.
7972018-12-13T20:29:54 <luke-jr> not post-Lightning
7982018-12-13T20:30:14 <gmaxwell> luke-jr: yes post lightning too, since channel creations/closers still use capacity.
7992018-12-13T20:30:33 <luke-jr> less than we need
8002018-12-13T20:30:39 <gmaxwell> So you probably want a tree hash so that you can have each peer prove that the chunk they're sending you is consistent with the download you're expecting.
8012018-12-13T20:31:57 <sipa> luke-jr: i don't expect there to be a "lightning going production-ready"; there may or may not be a phase where it grows sufficiently mature, and there may or may not be a phase where large scale adoption happens
8022018-12-13T20:32:11 <sipa> don't put all eggs in one basket
8032018-12-13T20:32:31 <gmaxwell> luke-jr: I don't think it's at all obvious that the result is less than we need.
8042018-12-13T20:32:33 <luke-jr> one basket is better than none
8052018-12-13T20:32:57 <gmaxwell> Current blocksizes are perfectly survivable if we get out of this fetch the whole history on every install mode.
8062018-12-13T20:33:23 *** Murch has quit IRC
8072018-12-13T20:33:29 <luke-jr> I don't think changing the security model is an appropriate avenue for that.
8082018-12-13T20:34:01 <gmaxwell> I don't agree that its a meaninful change to the security model, but even if it were, it would be a good tradeoff.
8092018-12-13T20:34:28 <gmaxwell> having users and businesses not willing to run nodes because bringing one up takes days of syncing is a change to the security model.
8102018-12-13T20:34:31 <gmaxwell> unambigiously.
8112018-12-13T20:34:46 *** Murch has joined #bitcoin-core-dev
8122018-12-13T20:35:32 <gmaxwell> and we're already there, and no reduction will eliminate that. (you're right that assuming (cough) continued computer speedups it would eventually get better if blocks were significantly limited today, but even that would take a long time)
8132018-12-13T20:36:16 *** phwalkr has joined #bitcoin-core-dev
8142018-12-13T20:37:46 <gmaxwell> luke-jr: not to mention that the political viablity of a decrease is essentially 0, -- something you always knew, as that was also the reason that "the limit can be removed and restored if there are issues!" was a dumb idea on arrival.
8152018-12-13T20:39:06 <luke-jr> it's not zero. It just requires more education, less block space need (ie, Lightning), and ideally developer support.
8162018-12-13T20:39:54 *** shesek has quit IRC
8172018-12-13T20:40:28 *** shesek has joined #bitcoin-core-dev
8182018-12-13T20:40:42 <gwillen> given the massive political consequences of merely _not raising_ the size, I can hardly even imagine the idea of lowering it being taken seriously.
8192018-12-13T20:43:18 <luke-jr> gwillen: the trolls are off having fun with their altcoin now, no need to appease them anymore (in fact, they will probably encourage it because they think it's a bad idea)
8202018-12-13T20:44:08 <gmaxwell> The fundimental issue that many people can't wrap their heads around a limit having any purpose at all still remains. Also the issue of it being totally unclear how much is enough.
8212018-12-13T20:44:24 <luke-jr> gmaxwell: that's where education comes in
8222018-12-13T20:44:36 <gmaxwell> Education only goes so far.
8232018-12-13T20:45:07 *** gabridome has joined #bitcoin-core-dev
8242018-12-13T20:45:10 <gwillen> you've proposed reductions in size before, and I do not have the sense that they have ever gotten traction
8252018-12-13T20:45:17 <gmaxwell> in any case, the system inherently resists changes, so the deck is stacked against you... this is a good thing usually.
8262018-12-13T20:45:20 <gwillen> even among people opposed to increasing it
8272018-12-13T20:45:21 <luke-jr> gwillen: not really
8282018-12-13T20:45:45 <gmaxwell> luke-jr: I don't personally see a need or use to decrease it now.
8292018-12-13T20:46:01 <luke-jr> gmaxwell: because you've given up, it sounds like
8302018-12-13T20:46:17 <gmaxwell> Improvements to initial sync are required regardless! and I don't see them as regretable.
8312018-12-13T20:46:50 <luke-jr> real improvements to initial sync only work in combination with a block size decrease
8322018-12-13T20:46:51 <gmaxwell> and with them I'm not aware of an argument as to why the current size is particularly problematic, we have _radically_ improved things like block propagation.
8332018-12-13T20:47:01 <gmaxwell> (for example)
8342018-12-13T20:47:56 <gmaxwell> and with the work on tx relay improvements we'll be able to cut relay overheads by maybe 40x.
8352018-12-13T20:48:10 *** Guyver2 has joined #bitcoin-core-dev
8362018-12-13T20:48:21 <luke-jr> that's unrelated to initial sync..
8372018-12-13T20:49:31 <gmaxwell> exactly, I am pointing out that the current size is okay except for initial sync. And initial sync needs to be safely shortcutted regardless of the ongoing future size. And once it is, initial sync would no longer be a huge issue.
8382018-12-13T20:50:53 <luke-jr> I don't agree. That's what I mean by just giving up.
8392018-12-13T20:51:18 *** bitcoin-git has joined #bitcoin-core-dev
8402018-12-13T20:51:19 <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14954: WIP: build: Require python 3.5 (master...Mf1811-buildPython3) https://github.com/bitcoin/bitcoin/pull/14954
8412018-12-13T20:51:19 *** bitcoin-git has left #bitcoin-core-dev
8422018-12-13T20:51:40 *** Murch has quit IRC
8432018-12-13T20:51:42 <gmaxwell> I don't think thats giving up at all.
8442018-12-13T20:52:21 <gmaxwell> unless you think that downloading bitcoin node software written by other people and not doing the work of writing it yourself is giving up. :P
8452018-12-13T20:52:34 <luke-jr> code can be verified
8462018-12-13T20:53:51 <gmaxwell> with an extreme amount of work, and an AV can be verified, with considerably less cost/work than code.
8472018-12-13T20:53:55 *** rex4539 has joined #bitcoin-core-dev
8482018-12-13T20:54:07 <luke-jr> not if we give up on a full sync
8492018-12-13T20:55:18 <gmaxwell> I'm confused by that view.
8502018-12-13T20:56:09 <gmaxwell> set -assumevalid=0 and you'll not use it, you'll take a lot longer to sync. It's still radically cheaper than doing pratically any code audit. (especially since in the common case of someone bringing up a node when they already had one running, their review need only be to compare between their nodes)
8512018-12-13T20:56:17 *** rockhouse has quit IRC
8522018-12-13T20:56:18 *** victorSN has quit IRC
8532018-12-13T20:56:37 <luke-jr> gmaxwell: but we're talking about leaving in place an infinitely growing initial sync time
8542018-12-13T20:56:53 <luke-jr> right _now_ it's easier, but it won't be for long
8552018-12-13T20:57:07 *** phwalkr_ has joined #bitcoin-core-dev
8562018-12-13T20:58:01 <gmaxwell> The cost of an outsider finding a subtly introduced 'bug' in the code is phenomial, maybe the cost of a review that could reliably catch something like that is on the order of $100,000 ... and that cost goes up over time as wages for experts seem to go up and the codebase increases in size.
8572018-12-13T20:58:34 <gmaxwell> It would take a LONG time of growth before the cost of a non-AV sync matches the cost of review that would reliably find a subtle bugdoor.
8582018-12-13T20:58:42 *** Murch has joined #bitcoin-core-dev
8592018-12-13T20:59:11 <luke-jr> therefore everyone should just trust developers /s
8602018-12-13T20:59:29 <gmaxwell> I don't see how that follows.
8612018-12-13T21:00:38 *** phwalkr has quit IRC
8622018-12-13T21:00:45 <gmaxwell> Verification is costly, there isn't any way around that.
8632018-12-13T21:01:27 <luke-jr> it's something people can still do today, for the blockchain
8642018-12-13T21:05:37 *** phwalkr has joined #bitcoin-core-dev
8652018-12-13T21:08:07 *** phwalkr_ has quit IRC
8662018-12-13T21:09:42 <gmaxwell> luke-jr: and they will still be able to do it in 10 years at current growth rates. it will just continue to be burdensome enough that most people will choose not to do so, exactly as is the case for the rest of the code today.
8672018-12-13T21:10:43 <gmaxwell> Worrying that it'll take 40 hours of computer time instead of 8 seems penny-wise-and-pound foolish, when it'll take 40 hours of _human_ time to do even a light review of the code.
8682018-12-13T21:11:03 <gmaxwell> in terms of aggregate validation, political efforts would probably be better spent making implementations more verifyable.
8692018-12-13T21:20:56 <treyzania> inb4 bitcoin node in coq
8702018-12-13T21:21:25 <aj> fwiw, with the "do nothing" approach, assuming IBD speeds increase by 18% per year, and chain length grows by 200GB/year, IBD maxes out at around 2.6x current length in 5 years, and finally gets back to current time after ~18 years
8712018-12-13T21:23:55 *** jarthur has quit IRC
8722018-12-13T21:26:20 <gmaxwell> aj: thanks.
8732018-12-13T21:29:38 *** phwalkr has quit IRC
8742018-12-13T21:30:32 *** phwalkr has joined #bitcoin-core-dev
8752018-12-13T21:31:37 <aj> oh, except 100GB/year is more reasonable? 2MB * 1000 blks/week * 52 weeks/year?
8762018-12-13T21:32:27 <aj> that's max of 1.5x in 4 years, back to same time as now in 12 years
8772018-12-13T21:33:44 <phantomcircuit> gmaxwell, the only reason i can think you'd want an addrindex is if you have a pile of private keys and cant remember any metadata about them
8782018-12-13T21:34:27 <gmaxwell> phantomcircuit: even there, do you really want an addrindex or a utxo spk index? Also if it's actually a pile, scantxout set will be faster.
8792018-12-13T21:35:13 <phantomcircuit> gmaxwell, for whatever accounting reason you need the history not just the current utxo
8802018-12-13T21:35:28 <aj> decreasing the weight limit to 2Mweight under those assumptions (so 50GB/year) keeps IBD time increase under 10% from now
8812018-12-13T21:36:34 <gmaxwell> phantomcircuit: indeed. But for that case, you'd still probably be just as well off with bip157 filters on disk, and sequential scanning those.
8822018-12-13T21:45:41 *** bitcoin-git has joined #bitcoin-core-dev
8832018-12-13T21:45:42 <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/9a4334443085...a5aea9654f36
8842018-12-13T21:45:42 <bitcoin-git> bitcoin/master faead93 MarcoFalke: test: Make g_insecure_rand_ctx thread_local
8852018-12-13T21:45:43 <bitcoin-git> bitcoin/master a5aea96 MarcoFalke: Merge #14953: test: Make g_insecure_rand_ctx thread_local...
8862018-12-13T21:45:43 *** bitcoin-git has left #bitcoin-core-dev
8872018-12-13T21:45:48 <phantomcircuit> gmaxwell, that wasn't an option when people were clamoring for an addrindex though
8882018-12-13T21:46:24 *** bitcoin-git has joined #bitcoin-core-dev
8892018-12-13T21:46:24 <bitcoin-git> [bitcoin] MarcoFalke closed pull request #14953: test: Make g_insecure_rand_ctx thread_local (master...Mf1812-testThreadLocal) https://github.com/bitcoin/bitcoin/pull/14953
8902018-12-13T21:46:24 *** bitcoin-git has left #bitcoin-core-dev
8912018-12-13T21:46:37 <gmaxwell> well it's not an option yet. yea, without more inforation on what people are asking for I'm not sure if people would be satisified by disk-filter powered rescan.
8922018-12-13T21:47:22 <phantomcircuit> gmaxwell, im 99% certain what people want is an easy way to import historic payments to an address for accounting reasons
8932018-12-13T21:47:42 <phantomcircuit> and have that work with whatever custom wallet stuff they've written
8942018-12-13T21:47:48 <phantomcircuit> (ie no creation timestamp)
8952018-12-13T21:47:53 <gmaxwell> that absoluely is a reason, I doubt its the only one.
8962018-12-13T21:48:16 <phantomcircuit> im sure the other 1% is blockexplorers calling rpc directly
8972018-12-13T21:48:38 <gmaxwell> e.g. like I've talked to an exchange who was asking for an index because they thought it would make the daemon into a block explorer and they could use it to answer 'sent from' for blacklisting purposes.
8982018-12-13T21:48:54 <gmaxwell> But none of the addrindex proposals would have made that possible, regardless...
8992018-12-13T21:49:01 <gmaxwell> because they index outputs, not inputs.
9002018-12-13T21:52:06 *** fabianfabian has joined #bitcoin-core-dev
9012018-12-13T21:52:10 *** AaronvanW has joined #bitcoin-core-dev
9022018-12-13T21:53:56 <adiabat> sorry just got to this but a couple maybe relevant points:
9032018-12-13T21:54:23 <kanzure> late hi
9042018-12-13T21:54:35 <adiabat> I'm hoping the utreexo thing will help clients sync faster, even without any checkpoints. If you're running on an HDD, disk I/O is the main delay for IBD
9052018-12-13T21:55:00 *** miknotauro has quit IRC
9062018-12-13T21:55:12 <adiabat> and with utreexo / other utxo accumulator, you can avoid touching the disk and stay in ram, so you are just cpu limited by signatures
9072018-12-13T21:55:36 <adiabat> (which fast computers with SSDs are now anyway, but lots of people run nodes with spinning drives)
9082018-12-13T21:56:14 *** chenpo has quit IRC
9092018-12-13T21:56:26 <adiabat> also separate point is that the need for a bridge node; these nodes can attach proofs to inputs for the nodes which hage pruned the utxos set away and only hold the accumulator
9102018-12-13T21:56:29 <treyzania> I wonder how long it would take to sync off of a bunch of RAIDed floppies
9112018-12-13T21:57:09 <adiabat> the bridge node needs a utxo index; not of addresses though. But I'm not sure if it makes sense to build that into core or keep it as a separate server / node / software
9122018-12-13T21:57:45 <adiabat> so that might be a place to put address index software if people don't want it in bitcoind
9132018-12-13T21:58:06 <luke-jr> oh wow, I totally forgot about adiabat's thing after Tokyo >_<
9142018-12-13T21:58:47 <adiabat> yeah I need to put stuff up publicly. I've been working on it but it takes longer than I would expect, like evertyhing else :)
9152018-12-13T21:58:54 <instagibbs> (btw he quoted 15% overhead during IBD in other channel)
9162018-12-13T21:59:40 <adiabat> instagibbs: yeah, overhead depends on how much ram you dedicate to caching the proofs
9172018-12-13T22:00:12 <adiabat> with no caching I have 250GB of proofs, so not great
9182018-12-13T22:00:49 <adiabat> but it goes down quickly with increased caching. Also I have a maybe controversial way to use shorter hashes.
9192018-12-13T22:01:21 <luke-jr> did kanzure transcribe the discussions in Tokyo? I feel like I should review them now :/
9202018-12-13T22:01:24 <adiabat> in that you can argue that you're only needing 2nd preimage resistance, not collision resistance. But I'm not 100% set on that, even though I'm pretty convinced
9212018-12-13T22:02:20 <adiabat> yeah kanzure's transcript is here: http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-utxo-accumulators-and-utreexo/
9222018-12-13T22:03:53 <luke-jr> thanks
9232018-12-13T22:05:03 <kanzure> :)
9242018-12-13T22:05:08 <kanzure> you beat me to it
9252018-12-13T22:05:16 <sipa> a rare event.
9262018-12-13T22:08:42 *** Tralfaz has quit IRC
9272018-12-13T22:11:12 <gmaxwell> adiabat: you can sync entirely in ram today, it just takes 8GB of cache... so you can measure the performance impact of that vs defaults.
9282018-12-13T22:12:00 <gmaxwell> adiabat: it's non-trivial, but I am not expecting that it would be better for sync time to double bandwidth usage but avoid the in memory state.
9292018-12-13T22:12:21 <sipa> gmaxwell: with a couple hundred MB of cache it seems the overhead is just 15%
9302018-12-13T22:13:01 <sipa> if that's true, it starts to sound appealing for far more deployments
9312018-12-13T22:13:07 <gmaxwell> well currently.
9322018-12-13T22:14:42 <gmaxwell> I'm confused about that, it seems improbable to me. 15% would mean only on the order of _one_ extra hash per input.
9332018-12-13T22:15:10 * sipa looks at adiabat
9342018-12-13T22:15:28 <gmaxwell> 2x I believed. :)
9352018-12-13T22:15:51 <treyzania> it works *really* great for embedded devices which might have a few hundred MiB memory and flash storage
9362018-12-13T22:15:53 *** chenpo has joined #bitcoin-core-dev
9372018-12-13T22:16:06 <treyzania> or like rpis
9382018-12-13T22:16:08 *** phwalkr has quit IRC
9392018-12-13T22:16:49 <adiabat> gmaxwell: it's because so many utxos are short-lived, and this is with relying on "hints" from the archive node you IBD from
9402018-12-13T22:17:17 <adiabat> the archive node knows what's happening next and can give TTL values for all the utxos created in a block
9412018-12-13T22:17:19 <gmaxwell> Oh I see the few hundred mb cache is basically a cache of recent outputs, plus the top of a tree for non-recent outputs (or similar)?
9422018-12-13T22:17:31 <adiabat> better than recent -- upcoming
9432018-12-13T22:17:56 <sipa> adiabat: but that doesn't apply for chain tip updates, only historical sync
9442018-12-13T22:17:57 <adiabat> that part is trusted, so they could lie and say long-lived utxos will be consumed soon
9452018-12-13T22:18:02 <adiabat> right, this is only for IBD
9462018-12-13T22:18:18 <sipa> i think the affect on bandwidth to keep in sync is more important
9472018-12-13T22:18:23 <adiabat> these optimizations don't help once you're synced up; I've mostly been focusing on IBD so far
9482018-12-13T22:19:06 <adiabat> there are other ideas for in-sync, but I haven't worked on that much; with IBD I have a nice data set to work off of to get performance numbers
9492018-12-13T22:20:36 <adiabat> archive nodes indicating which utxos will be consumed soon could potentially be useful for IBD with the current levelDB as well I guess
9502018-12-13T22:21:11 <adiabat> worst case if the node you're downloading from lies to you, your cache is full of the wrong things and you have to go to disk more and hang up on the lying node
9512018-12-13T22:22:37 <gmaxwell> I don't think this is currently an interesting route of optimizations, from an engineering perspective. Right now the existing software doesn't implement a simple fifo for management of the cache, in part due to the complexity of doing so. Fifoing inputs would probably have 98% of the locality gain (a question I guess your research could answer), but still less complexity than cache
9522018-12-13T22:22:38 <gmaxwell> management hints.
9532018-12-13T22:23:28 <adiabat> yeah, fair, it makes more sense with the accumulator proofs in that you prevent having to download a lot of data
9542018-12-13T22:23:46 <gmaxwell> Right sounds more useful there.
9552018-12-13T22:25:34 <gmaxwell> But even there in IBD you're essentially trading off (say) 200gb * .15 = 30GB more data transfered, in order to avoid having to store 2GB durign sync. I think under most relaistic models of the relative costs of storage and bandwidth, this tradeoff isn't that amazing. Having the option of making it would be nice, but I would expect relatively few systems to actually be better off with that
9562018-12-13T22:25:35 <gmaxwell> choice.
9572018-12-13T22:28:09 <adiabat> for powerful computers it doesn't really help, but for computers with HDDs / space constraints it can help
9582018-12-13T22:28:28 <sipa> or with I/O-starved VPSes
9592018-12-13T22:28:43 <sipa> and perhaps it's also just faster
9602018-12-13T22:28:46 <adiabat> but yeah if you have a fast computer and slow internet, it makes things worse
9612018-12-13T22:28:48 <sipa> (to be seen)
9622018-12-13T22:29:31 <sipa> and it also makes assumevalid-utxo models trivial
9632018-12-13T22:29:40 <sipa> (no snapshot to transfer...)
9642018-12-13T22:29:56 <adiabat> well a bech32-encodable snapshot
9652018-12-13T22:30:09 <adiabat> hm actually no, it's a few hundred bytes so not quite... :)
9662018-12-13T22:30:09 *** shesek has quit IRC
9672018-12-13T22:30:54 *** shesek has joined #bitcoin-core-dev
9682018-12-13T22:32:34 <gmaxwell> if you're assuming that the node knows the top of the tree, in order to avoid an absurd bandwidth blowup, then you'll have to transfer that.
9692018-12-13T22:33:27 <sipa> that just gets relayed along with the first transactions that need it
9702018-12-13T22:33:30 <adiabat> you could tranfer the top along with the root, but as soon as you see a few proofs in the mempool, you'll get the top
9712018-12-13T22:33:49 <adiabat> right, what sipa said
9722018-12-13T22:33:56 <sipa> it's of course still bandwidth; just saying it doesn't need a special store-and-transfer-a-snapshot protocol
9732018-12-13T22:34:01 <gmaxwell> Okay, but thats just hiding the cost.
9742018-12-13T22:34:38 <sipa> by trivial i meant complexity, not bandwidth - but even there you're probably right that it's complexity we're paying anyway in a more complex block/tx sync protocl
9752018-12-13T22:34:53 <gmaxwell> sipa: it does need a snapshot, because you cannot run from the tip without wreaking security, you'll need the state as of the point you know is valid.
9762018-12-13T22:35:20 <adiabat> yeah, network bandwidth is the big cost to this, CPU cost is low
9772018-12-13T22:35:23 <gmaxwell> (I almost agreed with you for a moment there though! :P )
9782018-12-13T22:35:44 <sipa> gmaxwell: ok, sure - but it's a few hundred bytes
9792018-12-13T22:38:20 *** Guyver2 has quit IRC
9802018-12-13T22:38:22 *** michaelsdunn1 has quit IRC
9812018-12-13T22:38:47 <gmaxwell> really? there are 68m utxo or so, 1000 bytes would allow for 32 hashes. for 68m hashes we need a tree of depth 29. or 24 if 5 levels are saved from the top, so that results in 768 bytes per input of membership proof.
9822018-12-13T22:39:16 <gmaxwell> so I think not a few hundreds bytes... certantly a lot smaller than the whole utxo set though... assuming you don't want to be able to offer proofs yourself.
9832018-12-13T22:40:29 *** sipa has quit IRC
9842018-12-13T22:41:44 *** sipa has joined #bitcoin-core-dev
9852018-12-13T22:41:59 <sipa> what did i miss?
9862018-12-13T22:42:05 <luke-jr> pizza
9872018-12-13T22:42:10 <gmaxwell> dunno if you missed anything.
9882018-12-13T22:42:14 <sipa> that's okay
9892018-12-13T22:42:17 <gmaxwell> did you see
9902018-12-13T22:42:18 <gmaxwell> 14:38:47 < gmaxwell> really? there are 68m utxo or so, 1000 bytes would allow for 32 hashes. for 68m hashes we need a tree of depth 29. or 24 if 5
9912018-12-13T22:42:18 <gmaxwell> levels are saved from the top, so that results in 768 bytes per input of membership proof.
9922018-12-13T22:42:18 <gmaxwell> 14:39:16 < gmaxwell> so I think not a few hundreds bytes... certantly a lot smaller than the whole utxo set though... assuming you don't want to be
9932018-12-13T22:42:18 <gmaxwell> able to offer proofs yourself.
9942018-12-13T22:42:21 <gmaxwell> ?
9952018-12-13T22:43:11 <adiabat> I'll clean up the code a bit, but the code I have is not a single tree; it's a forest of perfect binary trees
9962018-12-13T22:43:21 <sipa> gmaxwell: the "state" of the utxo set is just the roots of each of the trees in the forest
9972018-12-13T22:43:22 <adiabat> so the worst case proof yeah is like 28 high
9982018-12-13T22:43:37 <sipa> it's distinct from the cache of elements in the trees (which just gets transferred along with the blocks)
9992018-12-13T22:43:55 <adiabat> but you can try to put most of the old utxos on the left in the big tree, and most proofs that you need to transfer are in the smaller trees with high turnover
10002018-12-13T22:44:52 <gmaxwell> sipa ... adding 28 hashes per input is a non-starter for general usage. There are some applications, vps in a datacenter where internal bandwidth is free, where it might be attractive. but for most systems _bandwidth_ is the most limited resource.
10012018-12-13T22:45:18 <sipa> gmaxwell: of course
10022018-12-13T22:45:37 <adiabat> gmaxwell: 28 hashes per input is too much, agreed. There's ways to bring that number down significantly
10032018-12-13T22:45:43 <sipa> gmaxwell: i think you're missing some of the ideas
10042018-12-13T22:46:08 <adiabat> (which I haven't written up, so not you're fault :)
10052018-12-13T22:46:37 <gmaxwell> adiabat: it can be brought down by caching more of the top...
10062018-12-13T22:46:37 <sipa> but in general i think it's a semantics discussion about what you call state and what you call cache
10072018-12-13T22:46:41 <adiabat> but a big part is exploiting the power-law dureation of utxos
10082018-12-13T22:46:44 <sipa> gmaxwell: adiabat is well aware
10092018-12-13T22:47:17 <gmaxwell> adiabat: That doesn't come about as a result of any physical law or even economic incentive in the current system. :(
10102018-12-13T22:47:48 <adiabat> true, a lot of is is a heuristic, so who knows. if you have uniform random access across the utxo set, it gets worse.
10112018-12-13T22:48:55 <adiabat> fitting it to the current 200GB data set we have though, it's quite consistent that many utxos persist for only a few blocks
10122018-12-13T22:49:26 <gmaxwell> sipa: please we cannot continue to fail to do something about sync times in favor of conjecture about future ideas with uncertian performance. Like here "It's 2x" but only under assumptions about spending patterns, that is _really_ misleading and I think it undermines productive discussion.
10132018-12-13T22:49:27 <adiabat> (many get created & destroyed in the same block which means they never even touch the accumulator)
10142018-12-13T22:50:08 *** wxss has quit IRC
10152018-12-13T22:50:34 <sipa> gmaxwell: for the sake of discussion, please just treat the caching we're talking about as what you're calling keeping the top levels of the tree as state
10162018-12-13T22:50:40 <adiabat> gmaxwell: I certainly don't mean to over-sell this; it's not a silver bullet, but I think it can help in some cases
10172018-12-13T22:50:59 <gmaxwell> adiabat: Sure.
10182018-12-13T22:51:38 <sipa> gmaxwell: fwiw, our current performance for sync with dbcache less than the entire UTXO set is also dependent on spending patterns
10192018-12-13T22:52:18 <gmaxwell> sipa: sure. (though less than it would be if the dbcache's managment weren't kinda stupid wrt locality)
10202018-12-13T22:53:04 <gmaxwell> But "oh look spending patterns changed and with no impact in fees, bandwidth needed by bitcoin nodes just went up 5x" isn't the same as saying dbcache works better with more locality.
10212018-12-13T22:53:27 <sipa> yeah, agree
10222018-12-13T22:54:07 <sipa> we need a separate average case and worst case analysis w.r.t. bandwidth impact
10232018-12-13T22:55:13 <gmaxwell> I am frustrated because these kind of far off researchy things seem to keep getting pulled out as a reason to not do something more near term about the initial synctime problem.
10242018-12-13T22:56:16 <sipa> i don't think so
10252018-12-13T22:56:32 <gmaxwell> Well, that is how I saw the discussion today.
10262018-12-13T22:57:06 <sipa> as far as i'm concerned, the potential prospect of utreexo or similar techniques are only a reason to not support a utxo-address index for me
10272018-12-13T22:57:58 <gmaxwell> That makes sense. I wasn't connecting it to that discussion.
10282018-12-13T22:58:47 *** Tralfaz has joined #bitcoin-core-dev
10292018-12-13T22:59:12 <gmaxwell> We can debate on the necessity of users getting historical data except for research purposes, no such debate exists for utxo though... so some scanning mechenism must be provided even in a utxotree world. ... Ideally one that preserves user privacy. But there can be ways of doing that.
10302018-12-13T22:59:13 <adiabat> fwiw I think it's towards the practical / implementable side of research-land. I certainly don't want to say "this solves everything, don't worry about sync times, I got this" though.
10312018-12-13T23:00:15 *** Tralfaz has quit IRC
10322018-12-13T23:01:10 <gmaxwell> adiabat: Towards perhaps, but I think you overestimate that. The gap between what we can imagine and build for testing and what we can safely put into production is pretty big. Right now Bitcoind's dbcache grows up to some limit then is flushed completely, rather than acting as a fifo queue that exploits locality. "Make it fifo" sounds trivial, but actually getting it done, and Right, and
10332018-12-13T23:01:10 <gmaxwell> being confident that its right is not.
10342018-12-13T23:01:23 <sipa> adiabat: well, we'll see - i think you may be underestimating the amount of work needed to deploy archive nodes, and the practically implementing a dos-resistant stateful sync protocol for parts of the utxo tree
10352018-12-13T23:02:03 <sipa> adiabat: even after you've characterized all the memory and cpu requirements of a close to fully written up spec
10362018-12-13T23:02:12 <adiabat> agreed, it's always harder than it seems. Also, I don't mean to put it into bitcoind, I'm going to start with more of a server / client model
10372018-12-13T23:02:54 <adiabat> I guess in comparison to the class group accumulator it's more implementable though
10382018-12-13T23:03:00 <sipa> wahaha
10392018-12-13T23:03:14 *** Tralfaz has joined #bitcoin-core-dev
10402018-12-13T23:03:29 <sipa> yes
10412018-12-13T23:03:29 <gmaxwell> sipa: in any case, I was connecting this discussion back more to the assume valid discusion instead of the indexing discussion. So to me it sounded like an argument to do nothing because utxotree will solve everything... and I'm dubious that the bandwidth overheads will make it a _general_ solution (obviously a big win in some cases), and I'm dubious as to how fast it can be made available.
10422018-12-13T23:03:50 <gmaxwell> adiabat: yea, funny I dunno about that! so like a class group accumulator is an isolated number theory programming project.
10432018-12-13T23:04:07 <adiabat> oh yeah definitely don't count on anything I'm doing time-wise, takes forever :)
10442018-12-13T23:04:15 <gmaxwell> so just from a software eng I think CGA might be more implementable, but it's just too slow to make useful regardless.
10452018-12-13T23:04:21 <sipa> gmaxwell: my current estimate for updating an archival node with a new block is about a year of CPU time
10462018-12-13T23:04:30 <sipa> gmaxwell: with class group based accumulators
10472018-12-13T23:04:40 <gmaxwell> sipa: lol yes, thats my 'too slow to make useful'
10482018-12-13T23:04:53 <sipa> oh of course
10492018-12-13T23:05:02 <gmaxwell> but like, _ignoring that_ (lol), in terms of actually switching bitcoin to use it, I expect it would be easier than utxotree.
10502018-12-13T23:05:26 <sipa> also ignoring the novel cryptographic assumption? :)
10512018-12-13T23:05:34 <adiabat> huh, yeah the proofs are tiny, everything's constant sized, etc. The utreexo accumulator stuff does have a whole bunch of weird logic.
10522018-12-13T23:05:37 <gmaxwell> Even including the cost of having to write a "fast" classgroup acc library (assuming fast doesn't mean anything more than well optimized)
10532018-12-13T23:06:43 <sipa> but sure, everything constant size makes a lot of logistical problems go away
10542018-12-13T23:07:07 <gmaxwell> including need for protocols that make roundtrips... the bane of everyone's existance.
10552018-12-13T23:08:28 *** Giszmo has quit IRC
10562018-12-13T23:11:50 <gmaxwell> sipa: in any case, going back to the indexing question... for history indexing part of my concern there is that for some use cases there is a "just don't do that" or "look in the utxo instead" option. Like "We need a history index to look up from addresses!" :P for the utxo set that is much less the case.
10572018-12-13T23:11:51 <gmaxwell> so if you imagine that in the future everything is using utxotree, we'll still need to provide some way to look up scriptpubkeys... maybe it's PIR queries to nodes what have an N of M shared copy of the whole database or whatever... but something will have to be provided.
10582018-12-13T23:11:54 <gmaxwell> as rescan is already pretty unrealistic and contantly getting worse.
10592018-12-13T23:12:29 <sipa> gmaxwell: i'm not sure what use cases really need access to the full UTXO set
10602018-12-13T23:12:40 *** promag has joined #bitcoin-core-dev
10612018-12-13T23:12:41 <sipa> of course there is a convenience in having it
10622018-12-13T23:12:51 <sipa> but there is convenience in having an address index for the whole chain too
10632018-12-13T23:13:05 <gmaxwell> sipa: I need to take my wallet offline for a long time or recover a backup and want to spend my funds.
10642018-12-13T23:13:36 <gmaxwell> To actually _use_ bitcoin I need that data... vs history is needed for analysis/auditing/etc purposes.
10652018-12-13T23:14:06 *** Chris_Stewart_5 has quit IRC
10662018-12-13T23:14:11 <gmaxwell> (and for those analysis you probably don't just want an address index in history you want all sorts of other indexes)
10672018-12-13T23:15:12 <sipa> gmaxwell: assuming the UTXO set was equally expensive to look things up as in the blockchain (it's obviously not, but just as a hypothetical), would you still say that?
10682018-12-13T23:15:37 <gmaxwell> yes, because you can't spend your bitcoins without the utxo data!
10692018-12-13T23:16:07 <sipa> sure, but we need access to the chain anyway in recovery scenarios
10702018-12-13T23:16:09 <gmaxwell> (I mean if they were equal, I suppose you could just use the blockchain instead. but the utxo part is special because it's actually needed to make any use of all of your coins)
10712018-12-13T23:16:39 <gmaxwell> sipa: we don't, the fact that we recover tx history is an artifact of how the software was implemented.
10722018-12-13T23:16:53 <gmaxwell> Considering we can't recover tx medatadata, its not like we're actually recovering the real history.
10732018-12-13T23:17:06 <gmaxwell> The history can well be useful, for sure, but it is not needed to make use of your bitcoins.
10742018-12-13T23:17:19 <gmaxwell> (and other wallets, e.g. electrum, can run without history)
10752018-12-13T23:17:25 *** lopp has joined #bitcoin-core-dev
10762018-12-13T23:17:27 <sipa> gmaxwell: for a business knowing history may be far more important than the ability to spend coins
10772018-12-13T23:18:08 <sipa> my point is that what you're talking about is pretty much a recovery scenario that should be rare; and the only reason you consider is more reasonable than seeing whole history is because it's currently (and presumably later) much cheaper
10782018-12-13T23:18:08 <gmaxwell> sipa: I disagree because the thing we can recover from the blockchain is not history, it's partial history-- it lacks metadata which is actually the important information.
10792018-12-13T23:18:24 <gmaxwell> moreover, that data is there-- go use a block explorer, there is no need a bitcoin wallet or node need to provide it.
10802018-12-13T23:18:29 <sipa> also true; generally you also need information you'd have backed up separately
10812018-12-13T23:18:51 * gmaxwell bbl
10822018-12-13T23:22:32 *** Giszmo has joined #bitcoin-core-dev
10832018-12-13T23:23:09 <lopp> question regarding maintaining code integrity during the development process: is there any way that an adversarial maintainer could merge an altered PR commit? just trying to fully visualize the "chain of code custody" between the steps of peer review and actually being merged
10842018-12-13T23:23:58 *** chenpo has quit IRC
10852018-12-13T23:25:20 *** fabianfabian has quit IRC
10862018-12-13T23:25:45 *** hex17or has quit IRC
10872018-12-13T23:38:22 *** Giszmo has quit IRC
10882018-12-13T23:42:49 *** shesek has quit IRC
10892018-12-13T23:59:19 *** Giszmo has joined #bitcoin-core-dev