12018-08-24T00:05:20  *** plankers has quit IRC
  22018-08-24T00:06:49  *** plankers has joined #bitcoin-core-dev
  32018-08-24T00:09:34  *** zivl has quit IRC
  42018-08-24T00:20:56  *** phwalkr has quit IRC
  52018-08-24T00:32:19  *** lnostdal has joined #bitcoin-core-dev
  62018-08-24T00:37:46  *** profmac has joined #bitcoin-core-dev
  72018-08-24T00:38:08  *** promag has quit IRC
  82018-08-24T00:38:36  *** Krellan has joined #bitcoin-core-dev
  92018-08-24T00:46:11  *** promag has joined #bitcoin-core-dev
 102018-08-24T01:07:09  *** rex4539 has quit IRC
 112018-08-24T01:07:15  *** Rootsudo has quit IRC
 122018-08-24T01:07:44  *** rex4539 has joined #bitcoin-core-dev
 132018-08-24T01:08:00  *** Rootsudo has joined #bitcoin-core-dev
 142018-08-24T01:08:03  *** AaronvanW has quit IRC
 152018-08-24T01:10:47  *** promag has quit IRC
 162018-08-24T01:17:35  *** promag has joined #bitcoin-core-dev
 172018-08-24T01:22:12  *** promag has quit IRC
 182018-08-24T01:46:43  *** Magma has quit IRC
 192018-08-24T01:46:48  *** dongcarl has quit IRC
 202018-08-24T01:46:49  *** adam3us has quit IRC
 212018-08-24T01:46:49  *** warren has quit IRC
 222018-08-24T01:46:49  *** pindarhk_ has quit IRC
 232018-08-24T01:46:55  *** warren has joined #bitcoin-core-dev
 242018-08-24T01:47:02  *** dongcarl has joined #bitcoin-core-dev
 252018-08-24T01:47:04  *** nanotube has quit IRC
 262018-08-24T01:50:57  *** adam3us has joined #bitcoin-core-dev
 272018-08-24T01:52:04  *** nanotube has joined #bitcoin-core-dev
 282018-08-24T01:58:48  *** lone3lf has quit IRC
 292018-08-24T02:28:44  *** vexbuy has quit IRC
 302018-08-24T02:28:44  *** rafalcpp has quit IRC
 312018-08-24T02:28:44  *** Lightsword has quit IRC
 322018-08-24T02:28:45  *** helo has quit IRC
 332018-08-24T02:28:45  *** instagibbs has quit IRC
 342018-08-24T02:28:46  *** niska has quit IRC
 352018-08-24T02:30:03  *** vexbuy has joined #bitcoin-core-dev
 362018-08-24T02:30:03  *** rafalcpp has joined #bitcoin-core-dev
 372018-08-24T02:30:03  *** Lightsword has joined #bitcoin-core-dev
 382018-08-24T02:30:03  *** helo has joined #bitcoin-core-dev
 392018-08-24T02:30:03  *** instagibbs has joined #bitcoin-core-dev
 402018-08-24T02:30:03  *** niska has joined #bitcoin-core-dev
 412018-08-24T02:31:56  *** keymone has quit IRC
 422018-08-24T02:31:56  *** BashCo has quit IRC
 432018-08-24T02:31:56  *** wumpus has quit IRC
 442018-08-24T02:31:56  *** gribble has quit IRC
 452018-08-24T02:31:56  *** thaumavorio has quit IRC
 462018-08-24T02:31:57  *** treyzania has quit IRC
 472018-08-24T02:31:57  *** BlueMatt has quit IRC
 482018-08-24T02:31:57  *** bitbee has quit IRC
 492018-08-24T02:31:57  *** Madars has quit IRC
 502018-08-24T02:31:57  *** berndj has quit IRC
 512018-08-24T02:31:57  *** andytoshi has quit IRC
 522018-08-24T02:32:02  *** GAit has quit IRC
 532018-08-24T02:32:02  *** ossifrage has quit IRC
 542018-08-24T02:32:02  *** justanotheruser has quit IRC
 552018-08-24T02:32:02  *** Lauda has quit IRC
 562018-08-24T02:32:02  *** phantomcircuit has quit IRC
 572018-08-24T02:32:02  *** gwillen has quit IRC
 582018-08-24T02:32:02  *** rev_strangehope has quit IRC
 592018-08-24T02:32:02  *** jamesob has quit IRC
 602018-08-24T02:32:02  *** dqx has quit IRC
 612018-08-24T02:43:32  *** sipa has quit IRC
 622018-08-24T02:47:32  *** BGL has quit IRC
 632018-08-24T02:49:39  *** gribble has joined #bitcoin-core-dev
 642018-08-24T02:50:36  *** berndj has joined #bitcoin-core-dev
 652018-08-24T02:50:41  *** bitbee has joined #bitcoin-core-dev
 662018-08-24T02:53:51  *** rev_strangehope has joined #bitcoin-core-dev
 672018-08-24T02:55:36  *** BashCo has joined #bitcoin-core-dev
 682018-08-24T02:58:23  *** justan0theruser has joined #bitcoin-core-dev
 692018-08-24T03:01:33  *** justan0theruser is now known as justanotheruser
 702018-08-24T03:05:50  *** lnostdal has quit IRC
 712018-08-24T03:05:50  *** Varunram has quit IRC
 722018-08-24T03:05:50  *** Giszmo has quit IRC
 732018-08-24T03:05:50  *** e4xit has quit IRC
 742018-08-24T03:05:50  *** farmerwampum has quit IRC
 752018-08-24T03:05:51  *** games_ has quit IRC
 762018-08-24T03:05:51  *** xHire has quit IRC
 772018-08-24T03:05:51  *** ryanofsky has quit IRC
 782018-08-24T03:05:51  *** molz has quit IRC
 792018-08-24T03:05:58  *** dongcarl has quit IRC
 802018-08-24T03:05:59  *** luke-jr has quit IRC
 812018-08-24T03:05:59  *** unholymachine has quit IRC
 822018-08-24T03:05:59  *** MarcoFalke has quit IRC
 832018-08-24T03:05:59  *** atroxes has quit IRC
 842018-08-24T03:05:59  *** windsok has quit IRC
 852018-08-24T03:05:59  *** petertodd has quit IRC
 862018-08-24T03:05:59  *** jl2012 has quit IRC
 872018-08-24T03:05:59  *** karelb has quit IRC
 882018-08-24T03:05:59  *** meshcollider has quit IRC
 892018-08-24T03:05:59  *** sturles has quit IRC
 902018-08-24T03:05:59  *** davex__ has quit IRC
 912018-08-24T03:06:37  *** lnostdal has joined #bitcoin-core-dev
 922018-08-24T03:06:37  *** Varunram has joined #bitcoin-core-dev
 932018-08-24T03:06:37  *** Giszmo has joined #bitcoin-core-dev
 942018-08-24T03:06:37  *** e4xit has joined #bitcoin-core-dev
 952018-08-24T03:06:37  *** farmerwampum has joined #bitcoin-core-dev
 962018-08-24T03:06:37  *** games_ has joined #bitcoin-core-dev
 972018-08-24T03:06:37  *** xHire has joined #bitcoin-core-dev
 982018-08-24T03:06:37  *** ryanofsky has joined #bitcoin-core-dev
 992018-08-24T03:06:37  *** molz has joined #bitcoin-core-dev
1002018-08-24T03:06:55  *** grubles has quit IRC
1012018-08-24T03:07:57  *** dongcarl has joined #bitcoin-core-dev
1022018-08-24T03:07:57  *** luke-jr has joined #bitcoin-core-dev
1032018-08-24T03:07:57  *** unholymachine has joined #bitcoin-core-dev
1042018-08-24T03:07:57  *** MarcoFalke has joined #bitcoin-core-dev
1052018-08-24T03:07:57  *** meshcollider has joined #bitcoin-core-dev
1062018-08-24T03:07:57  *** windsok has joined #bitcoin-core-dev
1072018-08-24T03:07:57  *** atroxes has joined #bitcoin-core-dev
1082018-08-24T03:07:57  *** petertodd has joined #bitcoin-core-dev
1092018-08-24T03:07:57  *** jl2012 has joined #bitcoin-core-dev
1102018-08-24T03:07:57  *** karelb has joined #bitcoin-core-dev
1112018-08-24T03:07:57  *** sturles has joined #bitcoin-core-dev
1122018-08-24T03:07:57  *** davex__ has joined #bitcoin-core-dev
1132018-08-24T03:08:14  *** plankers has quit IRC
1142018-08-24T03:08:19  *** grubles has joined #bitcoin-core-dev
1152018-08-24T03:08:26  *** grubles has quit IRC
1162018-08-24T03:08:26  *** grubles has joined #bitcoin-core-dev
1172018-08-24T03:12:21  *** Guest46194 has joined #bitcoin-core-dev
1182018-08-24T03:21:41  *** Krellan has quit IRC
1192018-08-24T03:22:28  *** plankers has joined #bitcoin-core-dev
1202018-08-24T03:25:25  *** cornfeedhobo has quit IRC
1212018-08-24T03:28:57  *** jhfrontz has quit IRC
1222018-08-24T03:41:02  *** cornfeedhobo has joined #bitcoin-core-dev
1232018-08-24T03:43:14  *** esotericnonsense has quit IRC
1242018-08-24T03:43:22  *** BlueMatt has joined #bitcoin-core-dev
1252018-08-24T03:48:17  *** BlueMatt has quit IRC
1262018-08-24T03:50:06  *** BlueMatt has joined #bitcoin-core-dev
1272018-08-24T03:57:45  *** moxuan has joined #bitcoin-core-dev
1282018-08-24T04:01:06  *** plankers has quit IRC
1292018-08-24T04:07:43  *** ken2812221 has quit IRC
1302018-08-24T04:07:54  *** BGL has joined #bitcoin-core-dev
1312018-08-24T04:08:20  *** esotericnonsense has joined #bitcoin-core-dev
1322018-08-24T04:15:31  *** ken2812221 has joined #bitcoin-core-dev
1332018-08-24T04:17:51  <ken2812221> Is there any existing document that teach us how to add new source files?
1342018-08-24T04:20:23  <ken2812221> I think I can add those for MSVC.
1352018-08-24T04:26:33  *** moxuan has quit IRC
1362018-08-24T04:27:45  *** moxuan has joined #bitcoin-core-dev
1372018-08-24T04:28:46  *** plankers has joined #bitcoin-core-dev
1382018-08-24T04:29:44  *** ossifrage has joined #bitcoin-core-dev
1392018-08-24T04:40:14  *** echonaut9 has quit IRC
1402018-08-24T04:40:31  *** echonaut has joined #bitcoin-core-dev
1412018-08-24T04:54:39  *** harrymm has quit IRC
1422018-08-24T04:56:13  *** moxuan has quit IRC
1432018-08-24T04:57:29  *** vexbuy has quit IRC
1442018-08-24T04:58:05  *** vexbuy has joined #bitcoin-core-dev
1452018-08-24T04:58:16  *** harrymm has joined #bitcoin-core-dev
1462018-08-24T05:00:00  *** plankers has quit IRC
1472018-08-24T05:00:44  *** Rootsudo has quit IRC
1482018-08-24T05:01:57  *** ken2812221 has quit IRC
1492018-08-24T05:02:35  *** vexbuy has quit IRC
1502018-08-24T05:03:13  *** harrymm has quit IRC
1512018-08-24T05:05:17  *** vexbuy has joined #bitcoin-core-dev
1522018-08-24T05:08:01  *** moxuan has joined #bitcoin-core-dev
1532018-08-24T05:09:54  *** sipa has joined #bitcoin-core-dev
1542018-08-24T05:14:24  *** moxuan has quit IRC
1552018-08-24T05:15:22  *** moxuan has joined #bitcoin-core-dev
1562018-08-24T05:16:23  *** moxuan has quit IRC
1572018-08-24T05:18:41  *** moxuan has joined #bitcoin-core-dev
1582018-08-24T05:40:54  *** kallewoof has quit IRC
1592018-08-24T06:08:27  *** vexbuy has quit IRC
1602018-08-24T06:09:03  *** vexbuy has joined #bitcoin-core-dev
1612018-08-24T06:10:21  *** ken2812221 has joined #bitcoin-core-dev
1622018-08-24T06:13:39  *** vexbuy has quit IRC
1632018-08-24T06:17:08  *** sipa has quit IRC
1642018-08-24T06:17:57  *** owowo has joined #bitcoin-core-dev
1652018-08-24T06:18:44  *** sipa has joined #bitcoin-core-dev
1662018-08-24T06:22:26  *** vexbuy has joined #bitcoin-core-dev
1672018-08-24T06:24:12  *** harrymm_ has joined #bitcoin-core-dev
1682018-08-24T06:54:06  *** SopaXorzTaker has joined #bitcoin-core-dev
1692018-08-24T06:56:35  *** vexbuy has quit IRC
1702018-08-24T06:57:14  *** vexbuy has joined #bitcoin-core-dev
1712018-08-24T07:00:28  *** ken2812221 has quit IRC
1722018-08-24T07:00:53  *** ken2812221 has joined #bitcoin-core-dev
1732018-08-24T07:01:22  *** vexbuy has quit IRC
1742018-08-24T07:03:41  *** Krellan has joined #bitcoin-core-dev
1752018-08-24T07:05:50  *** profmac has quit IRC
1762018-08-24T07:08:01  *** d9b4bef9 has quit IRC
1772018-08-24T07:09:07  *** d9b4bef9 has joined #bitcoin-core-dev
1782018-08-24T07:15:22  *** csknk has joined #bitcoin-core-dev
1792018-08-24T07:19:14  *** profmac has joined #bitcoin-core-dev
1802018-08-24T07:22:21  *** Guyver2 has joined #bitcoin-core-dev
1812018-08-24T07:33:24  *** ken2812221 has quit IRC
1822018-08-24T07:41:24  *** gribble has quit IRC
1832018-08-24T07:41:46  *** ken2812221 has joined #bitcoin-core-dev
1842018-08-24T07:44:13  *** gribble has joined #bitcoin-core-dev
1852018-08-24T08:02:35  *** IGHOR has joined #bitcoin-core-dev
1862018-08-24T08:06:11  *** setpill has joined #bitcoin-core-dev
1872018-08-24T08:06:22  *** vexbuy has joined #bitcoin-core-dev
1882018-08-24T08:08:14  *** Ylbam_ has joined #bitcoin-core-dev
1892018-08-24T08:09:24  *** Ylbam_ has quit IRC
1902018-08-24T08:11:54  *** Krellan has quit IRC
1912018-08-24T08:12:13  *** Krellan has joined #bitcoin-core-dev
1922018-08-24T08:14:11  *** ylbam has joined #bitcoin-core-dev
1932018-08-24T08:26:19  * luke-jr wonders why gitian builds seem to ignore cached depends sometimes
1942018-08-24T08:32:43  *** vexbuy has quit IRC
1952018-08-24T08:40:17  *** moxuan has quit IRC
1962018-08-24T08:40:27  *** vexbuy has joined #bitcoin-core-dev
1972018-08-24T08:40:28  *** Zenton has joined #bitcoin-core-dev
1982018-08-24T08:40:42  *** profmac has quit IRC
1992018-08-24T08:45:27  *** vexbuy has quit IRC
2002018-08-24T09:04:38  *** promag has joined #bitcoin-core-dev
2012018-08-24T09:08:53  *** zivl has joined #bitcoin-core-dev
2022018-08-24T09:09:29  *** vexbuy has joined #bitcoin-core-dev
2032018-08-24T09:31:24  *** vexbuy has quit IRC
2042018-08-24T09:32:20  *** AaronvanW has joined #bitcoin-core-dev
2052018-08-24T09:51:47  *** vexbuy has joined #bitcoin-core-dev
2062018-08-24T09:52:32  *** vexbuy has joined #bitcoin-core-dev
2072018-08-24T09:59:04  *** rex4539 has quit IRC
2082018-08-24T10:20:43  *** Krellan has quit IRC
2092018-08-24T10:22:01  *** Krellan has joined #bitcoin-core-dev
2102018-08-24T10:32:48  *** profmac has joined #bitcoin-core-dev
2112018-08-24T10:55:44  *** rex4539 has joined #bitcoin-core-dev
2122018-08-24T11:07:01  *** d9b4bef9 has quit IRC
2132018-08-24T11:08:07  *** d9b4bef9 has joined #bitcoin-core-dev
2142018-08-24T11:12:54  *** ken2812221 has quit IRC
2152018-08-24T11:37:55  *** spinza has joined #bitcoin-core-dev
2162018-08-24T12:01:38  *** profmac has quit IRC
2172018-08-24T12:06:11  *** promag has quit IRC
2182018-08-24T12:13:55  *** berndj has quit IRC
2192018-08-24T12:16:46  *** berndj has joined #bitcoin-core-dev
2202018-08-24T12:24:28  *** ken2812221 has joined #bitcoin-core-dev
2212018-08-24T12:39:31  *** vexbuy has quit IRC
2222018-08-24T12:49:08  *** Guyver2 has quit IRC
2232018-08-24T12:51:56  *** vexbuy has joined #bitcoin-core-dev
2242018-08-24T12:54:52  *** profmac has joined #bitcoin-core-dev
2252018-08-24T12:57:10  *** ylbam has quit IRC
2262018-08-24T12:58:05  *** Chris_Stewart_5 has joined #bitcoin-core-dev
2272018-08-24T13:02:11  *** setpill has quit IRC
2282018-08-24T13:37:49  <cfields> luke-jr: rc2 required rebuilding qt to fix determinism. Is that what you're seeing?
2292018-08-24T13:38:08  <cfields> jonasschnelli: macOS detached signature reminder :)
2302018-08-24T14:15:27  *** Chris_Stewart_5 has quit IRC
2312018-08-24T14:17:33  <MarcoFalke> ken2812221: A new section in developer notes?
2322018-08-24T14:18:06  *** vexbuy_ has joined #bitcoin-core-dev
2332018-08-24T14:21:46  <luke-jr> cfields: I'm seeing all the deps being rebuilt for basically the same commit
2342018-08-24T14:21:48  *** vexbuy has quit IRC
2352018-08-24T14:29:20  *** Chris_Stewart_5 has joined #bitcoin-core-dev
2362018-08-24T14:32:44  *** jhfrontz has joined #bitcoin-core-dev
2372018-08-24T14:37:36  *** vexbuy_ has quit IRC
2382018-08-24T14:37:36  *** Krellan has quit IRC
2392018-08-24T14:38:12  *** vexbuy has joined #bitcoin-core-dev
2402018-08-24T14:38:17  *** Krellan has joined #bitcoin-core-dev
2412018-08-24T14:39:41  *** vexbuy has quit IRC
2422018-08-24T14:39:43  *** michaelsdunn1 has joined #bitcoin-core-dev
2432018-08-24T14:39:52  <jonasschnelli> cfields: oh. One sec
2442018-08-24T14:41:54  *** profmac has quit IRC
2452018-08-24T14:44:05  <cfields> luke-jr: hmm, yea, that's not right
2462018-08-24T14:45:01  *** Victorsueca has quit IRC
2472018-08-24T14:45:08  <jonasschnelli> cfields: done -> https://github.com/bitcoin-core/bitcoin-detached-sigs/pull/11
2482018-08-24T14:45:19  <luke-jr> cfields: is there any way to get debug output to see why it's not using the cached builds?
2492018-08-24T14:45:25  <cfields> jonasschnelli: thanks!
2502018-08-24T14:46:15  *** Victorsueca has joined #bitcoin-core-dev
2512018-08-24T14:46:19  <cfields> luke-jr: sure, but gitian makes it kinda tough. Can you drop to a shell in the builder?
2522018-08-24T14:47:00  <cfields> luke-jr: first thing, try building one more time and seeing if it happens again
2532018-08-24T14:47:16  <cfields> IIRC bionic's glibc was updated this/last week, so that might've triggered it
2542018-08-24T14:49:46  <luke-jr> oh
2552018-08-24T14:52:21  <luke-jr> pretty sure I've seen it multiple times in the last few days, but I haven't been watching closely enough to be sure
2562018-08-24T14:52:22  <cfields> (depends does something like COMPILER_SEED=`$CC --version --verbose`
2572018-08-24T14:52:26  <luke-jr> so I'll do a few more just in case
2582018-08-24T14:52:26  <cfields> )
2592018-08-24T14:52:43  <cfields> and if it changes, it invalidates all current builds
2602018-08-24T14:52:48  <cfields> ok
2612018-08-24T14:53:22  <luke-jr> (also, the cache files are replaced with the same filename)
2622018-08-24T14:53:33  <cfields> oh, that's weird
2632018-08-24T14:53:51  <cfields> the filename contains the hash of the determined build recipe. So that shouldn't be happening.
2642018-08-24T14:54:19  <cfields> maybe some filesystem thing keeps them from sending correctly to the builder?
2652018-08-24T14:55:03  <luke-jr> no sign of trouble there that I could tell
2662018-08-24T14:56:30  <luke-jr> hmm
2672018-08-24T14:57:01  <luke-jr> I have cache/bitcoin-linux-0.17/bitcoin-linux-0.17/, possibly from manual copy-from-target on failures
2682018-08-24T14:57:22  <luke-jr> it's possible since the glibc bump, I've only had failures..
2692018-08-24T14:57:47  <cfields> hmm?
2702018-08-24T14:58:11  <luke-jr> cfields: when gitian fails, it doesn't copy cache changes back to the host; I'd intended to manually do that, but it's possible they ended up in the wrong place
2712018-08-24T14:58:23  <luke-jr> combined with the glibc update you mentioned, that may explain why it's rebuilding every time
2722018-08-24T14:58:29  <cfields> ah, makes sense
2732018-08-24T15:03:16  *** keymone has joined #bitcoin-core-dev
2742018-08-24T15:04:46  *** jamesob has joined #bitcoin-core-dev
2752018-08-24T15:11:15  <cfields> gitian builders: detached sigs for 0.17.0rc2 are up
2762018-08-24T15:11:34  *** as1nc has joined #bitcoin-core-dev
2772018-08-24T15:11:34  *** MaxHastings_ has joined #bitcoin-core-dev
2782018-08-24T15:11:49  *** Dizzle has joined #bitcoin-core-dev
2792018-08-24T15:12:03  *** rex4539 has quit IRC
2802018-08-24T15:13:02  <cfields> fanquake/achow101/jonasschnelli/provoostenator/luke-jr/wumpus/MarcoFalke/fivepiece: ^^
2812018-08-24T15:13:41  <jonasschnelli>  /o/
2822018-08-24T15:14:24  <MaxHastings_> Hey. I just have a quick question. Are all the known vulnerabilities for SPV wallets listed here? https://bitcoin.org/en/developer-guide#simplified-payment-verification-spv I've been told on the Bitcoin slack that there are more.
2832018-08-24T15:18:43  <as1nc> Hello guys, just published a 2018 simple guide to set up a bitcoin core node on, feedback really appreciated. https://gist.github.com/larafale/b4df34a97c7134cf1579539caf2db2c2
2842018-08-24T15:19:23  <jonasschnelli> MaxHastings_: does it mention that the current used bloomfilter false-positive-rates do not protect privacy? (See Jonas Nicks master thesis from 2015)
2852018-08-24T15:19:52  *** profmac has joined #bitcoin-core-dev
2862018-08-24T15:20:06  <jonasschnelli> MaxHastings_: there are eventually non-protocol vulnerabilities (implementation issues)
2872018-08-24T15:20:54  <jonasschnelli> as1nc: nice. You could mention pruning?
2882018-08-24T15:21:09  <jonasschnelli> (and or electrum personal server)
2892018-08-24T15:21:25  <jonasschnelli> But its OT in this channel... so use #bitcoin
2902018-08-24T15:24:16  <luke-jr> cfields: can you sign PPC Windows builds too?
2912018-08-24T15:24:22  <luke-jr> jk ;)
2922018-08-24T15:25:22  <cfields> luke-jr: eh? does that exist?
2932018-08-24T15:25:31  <luke-jr> cfields: dunno, hence jk
2942018-08-24T15:25:37  <cfields> oh, phew
2952018-08-24T15:25:50  <luke-jr> XD
2962018-08-24T15:26:05  <luke-jr> (working on POWER8+ gitian binaries)
2972018-08-24T15:26:08  <luke-jr> (for Linux)
2982018-08-24T15:26:14  <cfields> because if they couldn't pull off small-endian arm, BE ppc wouldn't stand a chance :)
2992018-08-24T15:26:23  <luke-jr> who said BE?
3002018-08-24T15:26:31  <luke-jr> afaik even when Windows did support PPC, it was LE
3012018-08-24T15:26:38  <cfields> yeah yeah, that's why I specified
3022018-08-24T15:26:46  <cfields> wait, they actually did support PPC at one point?
3032018-08-24T15:26:50  <MaxHastings_> jonasschnelli: Ah I see I did not know bloom filters were ineffective for keeping user privacy. Are there any other missing vulnerabilities on that page other than privacy concerns? I was told on Bitcoin slack that the SPV client could be tricked to change its consensus rules by malicious nodes.
3042018-08-24T15:27:30  <luke-jr> cfields: NT supported a lot of archs
3052018-08-24T15:27:58  <sipa> MaxHastings_: by definition, yes
3062018-08-24T15:28:02  <cfields> huh. I used win embedded a few places, so I guess that makes sense. I never would've thought ppc though.
3072018-08-24T15:28:14  <sipa> MaxHastings_: they rely on the majority of the hashrate
3082018-08-24T15:28:54  <luke-jr> "Windows NT 3.51 added support for the PowerPC processor in 1995, specifically PReP-compliant systems such as the IBM Power Series desktops/laptops and Motorola PowerStack series; but despite meetings between Michael Spindler and Bill Gates, not on the Power Macintosh as the PReP compliant Power Macintosh project failed to ship."
3092018-08-24T15:29:19  <luke-jr> not really sure what that means XD
3102018-08-24T15:29:38  <as1nc> jonasschnelli, yes but i really want to incite people to txindex their chain so they can benefit the full spectrum of bitcoin capabilities. lightning for exemple require a txindexed chain right ?
3112018-08-24T15:30:14  <sipa> as1nc: i don't think so (or that's temporary in any case)
3122018-08-24T15:30:17  <jonasschnelli> Not sure. C-Lightning doesn't IMO
3132018-08-24T15:30:59  <as1nc> lnd does i think
3142018-08-24T15:31:38  <sipa> i'm sure that's temporary until better protocols to communicate exist
3152018-08-24T15:32:22  <sipa> (like bip157)
3162018-08-24T15:34:05  <MaxHastings_> sipa: Good to know.
3172018-08-24T15:37:00  <gmaxwell> c-lightning doesn't.
3182018-08-24T15:37:08  <gmaxwell> as1nc: txindex is non-viable long term.
3192018-08-24T15:37:44  <gmaxwell> Anytime you think "X _requires_ txindex" you should think "X will eventually force people to depend on a centeralized service".
3202018-08-24T15:38:52  <molz> lnd doesn't require txindex anymore
3212018-08-24T15:40:42  *** Zenton has quit IRC
3222018-08-24T15:41:00  <as1nc> ok interesting, I'm txindexing because I want to provide services for people. but the only case i find indexing worth it for personnal use is SPV wallets. does it makes sense ?
3232018-08-24T15:41:26  <as1nc> i use electrum pointed to my node
3242018-08-24T15:41:43  <sipa> as1nc: txindex is purely a local feature; it is not exposed over the network
3252018-08-24T15:43:31  <as1nc> sipa, yes
3262018-08-24T15:44:53  *** phwalkr has joined #bitcoin-core-dev
3272018-08-24T15:47:37  *** phwalkr has quit IRC
3282018-08-24T16:06:34  *** BashCo has quit IRC
3292018-08-24T16:07:16  *** BashCo has joined #bitcoin-core-dev
3302018-08-24T16:09:56  <Dizzle> as1nc: yep, it makes sense, for both personal use or providing centralized services. As for lnd, if you don't have a txindex, lnd basically falls back to rescanning blocks when it needs to know a tx's status, so a bit of a performance drag in some cases.
3312018-08-24T16:10:09  *** rex4539 has joined #bitcoin-core-dev
3322018-08-24T16:11:30  <gmaxwell> how is scanning blocks going to be a performance drag? thats how a node works anyways.
3332018-08-24T16:21:46  <Dizzle> gmaxwell: at lnd level, it's done manually instead of regularly i.e. specifically to find what may have befallen the tx - https://github.com/lightningnetwork/lnd/blob/c9bead7c21b1b83d69894935f4e1269ceffbecc7/chainntnfs/bitcoindnotify/bitcoind.go#L535 - so instead of a quick index lookup in Core or btcd, it's back and forth RPC and comparison looping.
3342018-08-24T16:25:39  <as1nc> Dizzle: thx, so it make sense to txindex your chain for personnal use, even if, like gmaxwell said, its not viable long run
3352018-08-24T16:32:09  *** BashCo has quit IRC
3362018-08-24T16:34:29  *** BashCo has joined #bitcoin-core-dev
3372018-08-24T16:39:54  *** phwalkr has joined #bitcoin-core-dev
3382018-08-24T16:42:36  <gmaxwell> Dizzle: I don't see any back and forth there, it just gets the block.
3392018-08-24T16:43:10  <gmaxwell> txindex lowers performance a lot, so I'm still not seeing how you're saying the lnd performance is worth without it.
3402018-08-24T16:44:18  <luke-jr> if only bech32 addresses were shorter, we could have called it belch32
3412018-08-24T16:44:54  *** phwalkr has quit IRC
3422018-08-24T16:52:34  *** peevsie has joined #bitcoin-core-dev
3432018-08-24T16:55:45  <Dizzle> gmaxwell: only saying the performance is worse without txindex for the cases when historical lookup of a tx's status is needed. I mean, that's what the index is useful for. The back and forth I'm referring to is getblockhash+getblock RPC calls for each height in turn, check for a match, repeat.
3442018-08-24T17:01:45  <gmaxwell> Dizzle: and my point is that you're missing the forrest for the trees. fetching the list of transactions in a block is cheap. maintaining the txindex slows down block processing a lot.
3452018-08-24T17:02:10  <gmaxwell> So I think what you're describing costs several milliseconds per block to save a millisecond on some blocks.
3462018-08-24T17:06:39  *** Urgo has joined #bitcoin-core-dev
3472018-08-24T17:10:44  <luke-jr> how does symbol-check not complain about aarch using symbols from glibc 2.17?
3482018-08-24T17:10:55  *** peevsie has quit IRC
3492018-08-24T17:14:47  <luke-jr> …oh, gitian disables it for aarch XD
3502018-08-24T17:18:50  *** Chris_Stewart_5 has quit IRC
3512018-08-24T17:26:33  *** AaronvanW has quit IRC
3522018-08-24T17:30:23  *** Victorsueca has quit IRC
3532018-08-24T17:31:46  *** Victorsueca has joined #bitcoin-core-dev
3542018-08-24T17:36:51  *** AaronvanW has joined #bitcoin-core-dev
3552018-08-24T17:41:56  *** AaronvanW has quit IRC
3562018-08-24T17:42:48  *** Zenton has joined #bitcoin-core-dev
3572018-08-24T17:48:53  *** BashCo has quit IRC
3582018-08-24T17:50:09  *** BashCo has joined #bitcoin-core-dev
3592018-08-24T17:53:21  *** esotericnonsense has quit IRC
3602018-08-24T17:53:32  *** BashCo has quit IRC
3612018-08-24T17:59:31  *** BashCo has joined #bitcoin-core-dev
3622018-08-24T18:02:25  <jnewbery> gmaxell: "txindex lowers performance a lot" - is that still true after #13033 ?
3632018-08-24T18:02:28  <gribble> https://github.com/bitcoin/bitcoin/issues/13033 | Build txindex in parallel with validation by jimpo · Pull Request #13033 · bitcoin/bitcoin · GitHub
3642018-08-24T18:03:39  *** esotericnonsense has joined #bitcoin-core-dev
3652018-08-24T18:08:13  *** michaelsdunn1 has quit IRC
3662018-08-24T18:08:48  <gmaxwell> jnewbery: maybe it is no longer "a lot", but it will still be more costly than checking the txids in a block, since it does that and a lot more.
3672018-08-24T18:09:21  <gmaxwell> jnewbery: I missed that PR, so thanks for pointing that otu.
3682018-08-24T18:09:45  <BlueMatt> I mean I dunno what the IOPS cost of it is, but if you're running with huge dbcache it should be almost free modulo the fact that we fsync the block files all the time (which we should stop doing, really)
3692018-08-24T18:10:13  <sipa> BlueMatt: once per hour
3702018-08-24T18:10:29  <sipa> and only the ones written to afaik
3712018-08-24T18:10:31  *** BashCo_ has joined #bitcoin-core-dev
3722018-08-24T18:10:45  <BlueMatt> not for block files, no, every time we move to a new one
3732018-08-24T18:10:47  <BlueMatt> last I checked
3742018-08-24T18:11:25  <BlueMatt> yea, every time we move to a new file we FlushBlockFile
3752018-08-24T18:11:30  <BlueMatt> which fsync's, afaics
3762018-08-24T18:11:38  <gmaxwell> BlueMatt: your comment is frustrating me. Above we are having a discussion where the argument was made that a whole txindex should be enabled to replace simply checking each block for a transaction.
3772018-08-24T18:11:54  <BlueMatt> ah, I missed context, sorry
3782018-08-24T18:11:56  <BlueMatt> yea, obviously not
3792018-08-24T18:11:57  *** BashCo has quit IRC
3802018-08-24T18:12:10  *** michaelsdunn1 has joined #bitcoin-core-dev
3812018-08-24T18:12:13  <gmaxwell> perhaps txindex no longer slows down reindex by 20% or whatever. but yea...
3822018-08-24T18:12:38  <sipa> performance isn't really the reason not to use it - it just doesn't scale well
3832018-08-24T18:12:44  <BlueMatt> anyway, as a separate point, spinning-rust-ibd-with-huge-dbcache the hang after each "Leaving block file" entry is really annoying
3842018-08-24T18:12:58  <BlueMatt> sipa: well, *also* performance :p
3852018-08-24T18:18:58  *** BashCo_ has quit IRC
3862018-08-24T18:19:57  *** Krellan has quit IRC
3872018-08-24T18:22:43  *** BashCo has joined #bitcoin-core-dev
3882018-08-24T18:23:12  *** as1nc has quit IRC
3892018-08-24T18:23:14  <BlueMatt> sipa: wait, isnt extended-encoding the *only* way you can encode a 0-input 1-output transaction?
3902018-08-24T18:23:34  <BlueMatt> reliably, at least
3912018-08-24T18:23:50  <BlueMatt> short of psbt
3922018-08-24T18:25:32  <sipa> BlueMatt: there is no reliable way to encode 0-input transactions
3932018-08-24T18:25:55  <BlueMatt> sure, if you use extended-encoding
3942018-08-24T18:26:08  <sipa> that's still potentially ambiguous
3952018-08-24T18:26:14  <BlueMatt> the use-all-data heuristic should handle it fine
3962018-08-24T18:26:35  <BlueMatt> and doesnt-need-to-be-backwards-compatible APIs (eg rust-bitcoin's deserializer) can *only* handle it that way
3972018-08-24T18:27:02  <sipa> i'm confused
3982018-08-24T18:27:17  <sipa> it is not legal to encode a tx without witnesses using extended encoding
3992018-08-24T18:27:23  <sipa> so there should never be any ambiguity
4002018-08-24T18:27:47  <BlueMatt> but every deserializer I've ever seen handles it fine, and its the only way to have no ambiguity in decoding 0-input txn
4012018-08-24T18:28:00  <BlueMatt> so I think we should revise the spec to match implementations?
4022018-08-24T18:28:03  <BlueMatt> or am I confusing myself
4032018-08-24T18:28:34  *** leishman has joined #bitcoin-core-dev
4042018-08-24T18:28:59  <sipa> sigh, so make an exception to the only-extended-if-has-witness rule in case there are 0 inputs?
4052018-08-24T18:29:16  <sipa> that's fine by me - i mostly care about only having a single encoding for actual transactions
4062018-08-24T18:29:29  <sipa> using the raw tx format for partial transactions was always a hack at best
4072018-08-24T18:29:51  <BlueMatt> yes, agreed, but the only reliably hack is always-use-extended, sadly :(
4082018-08-24T18:30:03  <BlueMatt> but, ok, fair, only doing it for 0-inputs seems reasonable to me
4092018-08-24T18:30:15  <sipa> we can also make the serializer do the same
4102018-08-24T18:30:24  <sipa> always use extended encoding when 0 inputs
4112018-08-24T18:30:27  <BlueMatt> would seem prudent to me :(
4122018-08-24T18:30:33  <sipa> ... or psbt
4132018-08-24T18:30:36  *** BashCo has quit IRC
4142018-08-24T18:30:39  <sipa> but everything at its time
4152018-08-24T18:30:43  <BlueMatt> yea, well hopefully people migrate to psbt
4162018-08-24T18:30:45  <BlueMatt> yea
4172018-08-24T18:30:52  *** andytoshi has joined #bitcoin-core-dev
4182018-08-24T18:37:57  *** AaronvanW has joined #bitcoin-core-dev
4192018-08-24T18:40:23  *** leishman has quit IRC
4202018-08-24T18:41:07  *** Empact has joined #bitcoin-core-dev
4212018-08-24T18:42:26  *** AaronvanW has quit IRC
4222018-08-24T18:45:38  <andytoshi> achow101: how do you use PSBT when nobody knows the full set of inputs up front?
4232018-08-24T18:46:11  <andytoshi> in particular, can you use PSBT to avoid the 0-input serialization ambiguity
4242018-08-24T18:46:58  <sipa> right
4252018-08-24T18:47:00  *** SopaXorzTaker has quit IRC
4262018-08-24T18:47:14  <sipa> for 0-input things people should just use a list of amounts/scriptpubkeys
4272018-08-24T18:47:50  <sipa> (i've mentioned extended psbt to support 0 inputs just as a container format, though)
4282018-08-24T18:48:26  <andytoshi> if we're going to propose an extension BIP to add p2c fields and industry/private reserved IDs, we should throw that in the pile
4292018-08-24T18:49:24  <sipa> in any case, PSBT's serialization supports 0 inputs fine
4302018-08-24T18:49:46  <sipa> (because it specifies using non-witness serialization for the tx, there is never any ambiguity)
4312018-08-24T18:49:51  <andytoshi> oh right
4322018-08-24T18:50:06  <andytoshi> ok i forgot that, never mind me, sorry
4332018-08-24T18:50:22  <sipa> it's not really usable to use for the various PSBT roles just because there isn't anything to be done with it
4342018-08-24T18:50:41  <gmaxwell> https://bitcoinperf.com/timeline/#/?exe=3,4,2,1&base=1+23&ben=reindex.522000.dbcache=2048.mem-usage&env=1&revs=50&equid=off&quarts=on&extr=on  < I think this should be blocking 0.17.
4352018-08-24T18:50:53  <gmaxwell> 20% memory usage increase for unknown reasons isn't cool.
4362018-08-24T18:51:57  <BlueMatt> sipa: wait, so psbt is not usable with the existing deserializer?
4372018-08-24T18:52:04  <sipa> BlueMatt: yes it is
4382018-08-24T18:52:07  <BlueMatt> cause 0-input non-witness is ambiguous?
4392018-08-24T18:52:15  <jamesob> gmaxwell: still bisecting
4402018-08-24T18:52:22  <sipa> BlueMatt: not if it is known to be non-witness
4412018-08-24T18:52:31  <BlueMatt> unless you give it a flag saying "it is 0-input, and not-witness-encoded, i promise"
4422018-08-24T18:52:46  <BlueMatt> ah, i guess we have that flag, other deserializers need it, though
4432018-08-24T18:52:49  <sipa> BlueMatt: which is what the implementation in core does
4442018-08-24T18:52:50  <BlueMatt> which is super obnoxious
4452018-08-24T18:52:55  <jamesob> running some tests to ensure increased mem usage isn't some quirk with the benchmarking framework
4462018-08-24T18:53:13  <sipa> BlueMatt: well, deal with it
4472018-08-24T18:54:12  <BlueMatt> lolk, but why doesnt it just serialize a list of outputs?
4482018-08-24T18:54:31  <sipa> BlueMatt: because it needs to serialize inputs too
4492018-08-24T18:55:07  <BlueMatt> I mean in no-input cases
4502018-08-24T18:55:17  <sipa> PSBT isn't designed for no-input cases
4512018-08-24T18:55:23  <BlueMatt> I thought psbt was supposed to simplify the deserialization ambiguity, not work around it with more flags :(
4522018-08-24T18:55:35  <sipa> it's to simplify the signing procedure
4532018-08-24T18:55:43  <sipa> BlueMatt: i don't understand why this is hard
4542018-08-24T18:55:53  <BlueMatt> its not "hard", just really annoying
4552018-08-24T18:55:53  <sipa> you know there is no witness, don't try to deserialize as a witness
4562018-08-24T18:56:08  <BlueMatt> yes, well I presume most deserializers dont have a flag for that
4572018-08-24T18:56:10  <BlueMatt> cause they dont need it
4582018-08-24T18:56:14  <BlueMatt> at least rust-bitcoin currently doesnt
4592018-08-24T18:56:18  <sipa> ?
4602018-08-24T18:56:20  <BlueMatt> which means plumbing through a ton of flags stuff
4612018-08-24T18:56:28  <sipa> how do you deal with a tx from a non-witness peer?
4622018-08-24T18:56:40  <gmaxwell> jamesob: well I also recently noticed the infinite dbcache case using much more dbcache size than I remember.
4632018-08-24T18:56:42  <BlueMatt> you never see 0-input txn on peer network?
4642018-08-24T18:56:45  <BlueMatt> so no ambiguity there
4652018-08-24T18:57:00  <sipa> yes, but you should reject if it were to contain a witness
4662018-08-24T18:57:02  <jamesob> gmaxwell: ah okay, so that corroborates
4672018-08-24T18:57:10  <sipa> i guess that can be done after deserialization, fine
4682018-08-24T18:57:41  <sipa> but really, is this worth arguing over?
4692018-08-24T18:57:56  <sipa> deserialization of 0-input is a pain, i'm sorry for that
4702018-08-24T18:58:00  <BlueMatt> I mean is it too late to fix the psbt spec? its just annoying to have so many serialization/deserialization modes
4712018-08-24T18:58:07  <sipa> yes, it is far too late
4722018-08-24T18:58:25  <BlueMatt> if the thing already has a "this is a 0-input tx" indicator, then why not have just used a list of outputs instead
4732018-08-24T18:58:31  <BlueMatt> plus nVserion/nLockTime
4742018-08-24T18:58:43  <sipa> no it doesn't have a this is a 0-input tx
4752018-08-24T18:59:10  <sipa> that indicator is the lack of inputs
4762018-08-24T18:59:36  <BlueMatt> ehh, whatever, I guess can be dealt with
4772018-08-24T18:59:37  <sipa> and again, psbt isn't designed for 0 inputs; it's designed to simplify coordination of signing, which by definition needs at least 1 input
4782018-08-24T18:59:46  <sipa> it just so happens that it is unambiguous even with 0 inputs
4792018-08-24T19:00:21  <gmaxwell> sipa: uh, isn't "I create a PSBT with the outputs I want, and give it to you so you can add inputs (and your change)" a supported use case
4802018-08-24T19:00:22  <achow101> BlueMatt: there are no flags in psbt. the global unsigned tx is always non-witness
4812018-08-24T19:00:40  <achow101> the input txs are self descriptive as to witness or not as they are always valid network txs
4822018-08-24T19:01:03  <sipa> gmaxwell: not initially no, but there is no reason why it couldn't
4832018-08-24T19:01:05  <BlueMatt> achow101: wait, huh? 0-input isnt "valid network txs"
4842018-08-24T19:01:27  <achow101> BlueMatt: input txs are txs that already exist on the network .they cannot be 0-input
4852018-08-24T19:01:28  <gmaxwell> BlueMatt: he's talking about prevout inputs.
4862018-08-24T19:01:29  <sipa> BlueMatt: the inputs
4872018-08-24T19:01:35  <gmaxwell> for checking fees.
4882018-08-24T19:01:39  <BlueMatt> ah
4892018-08-24T19:02:10  <BlueMatt> anyway, I'll stop complaining since I obviously didnt read the spec in detail
4902018-08-24T19:02:41  <sipa> gmaxwell: in PSBT role terminology that would be someone who takes the output of a Creator, and then "creates" a new PSBT for a different transaction
4912018-08-24T19:02:57  <sipa> gmaxwell: but in the workflow, all steps operate on the same "proposed transaction"
4922018-08-24T19:03:10  <sipa> when there are 0 inputs, there isn't really a proposed transaction
4932018-08-24T19:03:22  <sipa> though the same format could easily be used to support that
4942018-08-24T19:04:02  <achow101> gmaxwell: psbt's workflows are currently only defined for the signing stage. it assumes that all inputs and outputs have been decided and negotiated in some previous step
4952018-08-24T19:04:36  <BlueMatt> ah, so there are never any 0-input txn encoded in it at all?
4962018-08-24T19:04:46  <sipa> BlueMatt: in the PSBT workflow, no
4972018-08-24T19:04:47  <achow101> but you could still use psbt for negotiating the inputs. I'm not sure why you would though
4982018-08-24T19:05:05  <BlueMatt> sipa: why didnt you say that in the first place? :)
4992018-08-24T19:05:06  <sipa> but the format can be used for that, if you want
5002018-08-24T19:05:14  <achow101> BlueMatt: no, there is not. but it technically still works and will be deserialized properly
5012018-08-24T19:05:25  <sipa> 18:55:17 < sipa> PSBT isn't designed for no-input cases
5022018-08-24T19:05:29  <sipa> 18:55:35 < sipa> it's to simplify the signing procedure
5032018-08-24T19:07:44  <sipa> it's kind of annyong to think about PSBT-the-container-format (which supports 0 input fine, if you're willing to do the decode-knowing-there-is-no-witness thing) and PSBT-the-procedure-for-signing (which clearly doesn't) as separate
5042018-08-24T19:08:48  <BlueMatt> well I mean tell me to shut up cause I havent read it in detail, but I think in a new container-format we should be encoding 0-input with extended serialization
5052018-08-24T19:09:11  <BlueMatt> cause thats what we just said, above, too
5062018-08-24T19:09:38  <sipa> i think that's insane.
5072018-08-24T19:10:04  <sipa> PSBT implementations aren't even required to support segwit
5082018-08-24T19:10:06  <achow101> BlueMatt: why? there are no witnesses to serialize ever
5092018-08-24T19:10:37  <sipa> just add a flag to your deserializer
5102018-08-24T19:10:47  <achow101> BlueMatt: a psbt has two types of txs: the unsigned tx (the one you want to sign) and the input txs (ones already on the network).
5112018-08-24T19:10:58  <achow101> the unsigned tx, by definition, has no signatures. therefore it never has witnesses
5122018-08-24T19:11:26  <achow101> thus it is completely unnecessary to serialize the unsigned tx with extended serialization as it can never have a witness
5132018-08-24T19:11:38  <BlueMatt> sipa: we just discussed, above, encoding 0-input with extended serialization, since thats the *only* way to do it that is non-ambiguous
5142018-08-24T19:11:50  <BlueMatt> and otherwise createrawtransaction+fundrawtransaction are...not ideal
5152018-08-24T19:11:59  <gmaxwell> I'm just really confused by this discussion.  I expected that PSBT was the fix for the createrawtransaction ambiguity case.
5162018-08-24T19:12:13  <gmaxwell> and now you're saying that it isn't but it is?
5172018-08-24T19:12:14  <achow101> BlueMatt: it is completely unambiguous because the unsigned tx is always non-witness
5182018-08-24T19:12:34  <gmaxwell> so this discussion is pointless.
5192018-08-24T19:12:40  <sipa> BlueMatt: this is the *first* time ever in my life i hear the suggestion to use extended serialization for 0 input
5202018-08-24T19:12:44  <luke-jr> s/unsigned/no-input/
5212018-08-24T19:13:06  <sipa> BlueMatt: it's a cool trick, i admit
5222018-08-24T19:13:07  <BlueMatt> sipa: well its either that or we have to pass a flag into signrawtransaction (and any equivalent APIs in any bitcoin library anywhere)
5232018-08-24T19:13:24  <sipa> but i had always lived under the assumption that our deserializer failed on that
5242018-08-24T19:13:34  <sipa> it was just accidentally removed in a refactor
5252018-08-24T19:13:34  <BlueMatt> of which there are many, any I generally dont trust most bitcoin libraries...for obvious reasons
5262018-08-24T19:13:41  <achow101> BlueMatt: why are you signing a 0-input tx?
5272018-08-24T19:13:45  <achow101> what is there to even sign?
5282018-08-24T19:13:51  <sipa> achow101: not the point
5292018-08-24T19:13:54  <BlueMatt> sipa: for p2p, it probably should, because, you know, 0-input is bogus on the p2p/consensus layer
5302018-08-24T19:14:01  <BlueMatt> achow101: not signing, funding
5312018-08-24T19:14:25  <BlueMatt> achow101: right now the suggested workflow of createrawtransaction, fundrawtransaction, signrawtransaction, sendrawtransaction is hit by the ambiguity bug
5322018-08-24T19:14:29  <gmaxwell> achow101: See our rpc, "fundrawtransaction"  --- that isn't  "Fun! Draw Transaction." :P
5332018-08-24T19:14:46  <sipa> BlueMatt: createraw and fundraw are merged into one in PSBT workflow
5342018-08-24T19:14:57  <BlueMatt> its (mostly) fine right now, but my understanding was the psbt was addressing exactly that use-case but across multiple signers like multisig and the like
5352018-08-24T19:15:15  <luke-jr> achow101: you don't know it's zero input yet
5362018-08-24T19:15:20  <gmaxwell> the reason they exist seperately to begin with is that there are workflows where they need to be seperate. E.g. coinjoins.
5372018-08-24T19:15:43  <sipa> FFS people
5382018-08-24T19:15:44  <BlueMatt> sipa: fair, but you just mentioned that the existing container could be extended to split that out, and my point is, "is the container well-defined for the ambiguity bug"
5392018-08-24T19:15:46  <gmaxwell> or partial coincontrols. ("spend _this_ coin plus, whatever is needed")
5402018-08-24T19:15:48  <sipa> can you have commented on this 9 months ago?
5412018-08-24T19:16:06  <sipa> BlueMatt: YES IT IS. NO WITNESSES IN THE UNSIGNED PSBT TRANSACTION
5422018-08-24T19:16:11  <sipa> i'm tired of this discussion
5432018-08-24T19:16:29  <gmaxwell> sipa:  I read the spec and saw no evidence that 0 inputs wasn't a supported case!
5442018-08-24T19:16:36  <sipa> the fact that you want to reuse your heuristic decoder for raw transactions in PSBT may mean you need to add a flag to disable that feature
5452018-08-24T19:16:50  <sipa> in a post-PSBT world, nobody ever needs to implement such a heuristic again
5462018-08-24T19:17:20  <achow101> "Type: Unsigned Transaction PSBT_GLOBAL_UNSIGNED_TX = 0x00 ... The transaction must be in the old serialization format (without witnesses)"
5472018-08-24T19:17:24  <sipa> because actual transactions will always have witnesses (or use unambiguously old serialization for things without)
5482018-08-24T19:17:30  <sipa> and PSBTs will never have witnesses
5492018-08-24T19:17:44  <gmaxwell> it's surprising to me that you and achow didn't except PSBT to be a superset of the applications of the existing raw transactions.
5502018-08-24T19:17:48  *** profmac has quit IRC
5512018-08-24T19:18:06  <gmaxwell> achow101: great, and the old tx format supports 0 inputs.
5522018-08-24T19:18:13  <achow101> gmaxwell: exactly
5532018-08-24T19:18:20  <sipa> gmaxwell: not... really
5542018-08-24T19:18:22  <achow101> gmaxwell: but it is _always_ the old serialization. no ambiguity there
5552018-08-24T19:19:03  *** MaxHastings_ has quit IRC
5562018-08-24T19:19:42  <sipa> this discussion only happened because someone only has a heuristic tx decoder in their software, and finds it annoying to add a way to disable that heuristic
5572018-08-24T19:19:51  <gmaxwell> achow101: sure, I agree.  Sipa was asking why wasn't this commented on 9 months ago.  And my reply is because the shortfall (that 0 input PSBT aren't a supported usecase) isn't something I could extract from the spec.
5582018-08-24T19:20:11  <luke-jr> achow101: the ambiguity is because you don't know if it's 0-input or 1+ input segwit until you parse it correctly
5592018-08-24T19:20:20  <sipa> luke-jr: you're missing context
5602018-08-24T19:20:22  <gmaxwell> luke-jr: it's not ambigius in PSBT.
5612018-08-24T19:20:24  <luke-jr> ok
5622018-08-24T19:20:38  <gmaxwell> In PSBT the internal tx never has witnesses.
5632018-08-24T19:21:30  <achow101> gmaxwell: the format supports 0-inputs just fine. I think there's even a test for it. the processes that we also defined in the bip do not.
5642018-08-24T19:21:56  <sipa> gmaxwell: i think i always thought of PSBT as a better unambiguous serialization... but the BIP describes both a serialization and the procedures around it
5652018-08-24T19:22:08  <achow101> but the processes don't matter that much. you can use the psbt format for whatever steps that occur before signing and then use the processes defined in the bip
5662018-08-24T19:22:19  <sipa> and those procedures don't really map well to the part where you're still deciding what exactly to sign
5672018-08-24T19:22:23  <gmaxwell> I never read that stuff, it's not normative (doesn't change the meaning of the datastructures), and expirence from other BIPs suggests tha people ignore them. (see, for example BIP32)
5682018-08-24T19:22:37  <gmaxwell> (or at least don't read it carefully)
5692018-08-24T19:22:39  <sipa> gmaxwell: well, then you were right
5702018-08-24T19:22:49  <sipa> that part of PSBT supports 0 inputs fine :)
5712018-08-24T19:33:08  *** csknk has quit IRC
5722018-08-24T19:41:15  <gmaxwell> sipa: thanks! confusion resolved!!
5732018-08-24T19:47:01  *** vexbuy has joined #bitcoin-core-dev
5742018-08-24T19:53:43  *** plankers has joined #bitcoin-core-dev
5752018-08-24T20:01:00  *** BlueMatt has left #bitcoin-core-dev
5762018-08-24T20:03:10  *** vexbuy has quit IRC
5772018-08-24T20:04:20  *** vexbuy has joined #bitcoin-core-dev
5782018-08-24T20:13:59  <jnewbery> sipa: performance isn't really the reason not to use it - [txindex] just doesn't scale well" - scale well in what regard? Space requirements should be linear with blockchain size for a full archival node, right?
5792018-08-24T20:15:17  <sipa> jnewbery: i don't think it is realistic that everyone needs to store the whole chain
5802018-08-24T20:16:26  <sipa> or have fast access to it
5812018-08-24T20:16:38  <jnewbery> I agree! But for those that do, txindex is just a constant factor increase in space requirements
5822018-08-24T20:16:43  <sipa> sure
5832018-08-24T20:16:57  <jnewbery> ie txindex scales as well as archival nodes scale
5842018-08-24T20:17:14  <sipa> but building solutions that rely on having a fully-indexed blockchain available isn't scalable
5852018-08-24T20:17:31  <sipa> you don't need many archival nodes
5862018-08-24T20:18:10  <sipa> furthermore, nothing really _needs_ access to the full chain if designed well, in my opinion
5872018-08-24T20:18:27  <sipa> except for debugging purposes
5882018-08-24T20:18:46  <jnewbery> a block explorer?
5892018-08-24T20:18:56  <sipa> for example
5902018-08-24T20:19:02  <sipa> (i count that as debug purposes)
5912018-08-24T20:19:19  <sipa> not something to rely upon for production
5922018-08-24T20:20:22  <jnewbery> I agree that products shouldn't *rely* on block explorers, but there's demand for them now and I can't imagine there'd ever not be demand for them
5932018-08-24T20:20:38  <sipa> sadly enough :(
5942018-08-24T20:22:25  <sipa> however, in general i think we should discourage building on things like txindex
5952018-08-24T20:22:39  <jamesob> and in any case it seems like there'll always be use-cases which require rescan; ie when you become interested in an output you weren't keeping tabs on before
5962018-08-24T20:23:02  <sipa> as i fear we may end up with a less scalable ecosystem if everyone assumes it's easy to just look up whatever transaction
5972018-08-24T20:23:40  <sipa> and of course there are use cases - it wouldn't be there otherwise
5982018-08-24T20:24:34  <sipa> jamesob: i think that's either sign of a badly designed system, or an emergency (for which i think it's fine to need some index, you could also pay someone to look things up for you)
5992018-08-24T20:25:26  <jamesob> yeah, maybe so
6002018-08-24T20:25:30  *** roasbeef has joined #bitcoin-core-dev
6012018-08-24T20:26:41  <roasbeef> lnd doesn't require the txindex, it'll detect if the full node has it on or not, and fall back to just fetching blocks it needs manually, eventually if the gcs filters are exposed on RPC it could also fetch those before going for the full block it needs
6022018-08-24T20:29:26  <jnewbery> It's fair to say that 'assuming that it's easy to just look up whatever transaction' isn't a scalable architecture to build a product on, but I don't think the txindex in itself is unscalable now that we #13033
6032018-08-24T20:29:28  <gribble> https://github.com/bitcoin/bitcoin/issues/13033 | Build txindex in parallel with validation by jimpo · Pull Request #13033 · bitcoin/bitcoin · GitHub
6042018-08-24T20:31:33  <roasbeef> jnewbery: things could also be split up into distinct database, or even really just like memory mapped direct hash tables, shoving everything in leveldb makes other routine stuff slower as you end up with thousands of files that all need to be memory mapped and also to watch out for fd limits, then on top of that you end up with a handfull of disk seeks due to the file and sstable structure to read something like the offset of a txid in a block
6052018-08-24T20:33:20  <jamesob> roasbeef: jimpo's txindex change already moves indices into separate leveldbs
6062018-08-24T20:33:22  <roasbeef> Dizzle: we now continually also update a "height hint" so really a checkpoint of the state of that outpoint/txid of its best known state, so like if we know it isn't conf'd at height 100, but it was broadcast at height 10 we know now to start at height 10
6072018-08-24T20:33:47  <sipa> roasbeef: BIP157 will improve things further, i assume?
6082018-08-24T20:33:55  <Dizzle> roasbeef: nice
6092018-08-24T20:35:09  <jnewbery> ok, yes - that is fair, although in practice I'm not sure how much performance impact there is. If it's truly a performance bottleneck, then I assume #13243 gives us the infrastructure to store txindex somewhere else
6102018-08-24T20:35:11  <gribble> https://github.com/bitcoin/bitcoin/issues/13243 | Make reusable base class for auxiliary indices by jimpo · Pull Request #13243 · bitcoin/bitcoin · GitHub
6112018-08-24T20:35:35  <roasbeef> sipa: yeh, either if it's going over p2p network, or just even getting the filters over rpc from the full node, as it can fetch the filters in batch, and also cache filters on its end
6122018-08-24T20:35:59  *** profmac has joined #bitcoin-core-dev
6132018-08-24T20:37:07  <jamesob> jnewbery: indexes are already isolated from other dbs in /datadir/indexes/[index name]
6142018-08-24T20:37:12  <roasbeef> also eventually the stuff about creating larger like hierarchical multi-block filters can help as well when the node has been offline for a long-ish time
6152018-08-24T20:45:50  <gmaxwell> Under a no-address reuse assumption (lol), hierarchical filters are never a win.
6162018-08-24T20:46:02  *** d9b4bef9 has quit IRC
6172018-08-24T20:46:27  <gmaxwell> 13:16:57 < jnewbery> ie txindex scales as well as archival nodes scale
6182018-08-24T20:46:48  <gmaxwell> I don't expect that archive nodes will be a thing in 10 years, outside of academic curiousity and what not.
6192018-08-24T20:47:08  *** d9b4bef9 has joined #bitcoin-core-dev
6202018-08-24T20:47:19  <gmaxwell> No more than traces of all announced transactions from all peers in the network are a thing now.
6212018-08-24T20:48:33  <gmaxwell> I expect that new nodes will sync up using a months old UTXO state thats vallidated according to the assumevalid model, then catch up.
6222018-08-24T20:52:18  <jnewbery> Seems possible, but for the next 10 years I expect at least some people will want to run archival nodes!
6232018-08-24T20:52:33  <gmaxwell> oh absolutely, and we should do what we can to make that work well too.
6242018-08-24T20:52:33  <sipa> i hope they do.
6252018-08-24T20:52:41  <sipa> but i hope no services realpy need them
6262018-08-24T20:53:06  <gmaxwell> I've pushed a lot on optimizations related to e.g. reindex/ibd in the last year specifically because once we're syncing from AV-UTXO there will be much less motivation to improve these things.
6272018-08-24T20:54:14  <jnewbery> to be clear - I'm not arguing that we should *encourage* the use of txindex or needing a full blockchain. I just wanted understand what was meant by "txindex is unscalable".
6282018-08-24T20:56:21  <sipa> right; i think it's more "an ecosystem relying on txindex doesn't scale well"
6292018-08-24T20:56:30  <jnewbery> yes, makes sense
6302018-08-24T20:56:36  <sipa> txindex as an algorithmic thing itself scales as well as it could i think
6312018-08-24T20:57:03  <gmaxwell> Well I'm sure it's lacking a ton of possible optimizations.
6322018-08-24T20:57:30  <sipa> sure, but no dramatic complexity changes
6332018-08-24T20:58:07  <gmaxwell> no, just constant factors.
6342018-08-24T20:58:31  <gmaxwell> it's not the txindex structure itself isn't scable, it's that operations over the chain history aren't scalable.
6352018-08-24T21:00:37  <jnewbery> sipa, has your opinion changed at all since this: I both hate and love this patch. I hate making it easy to build infrastructure that relies on a fully-indexed blockchain (which shouldn't be necessary), as it may remove the incentive to build more scalable solutions. On the other hand, in cases where the alternative is relying on a trusted third party, this approach is certainly preferable, and
6362018-08-24T21:00:43  <jnewbery> would allow an RPC-based blockexplorer, for example.
6372018-08-24T21:00:44  <jnewbery> (#2802)
6382018-08-24T21:00:46  <gribble> https://github.com/bitcoin/bitcoin/issues/2802 | Add an address-based transaction index by sipa · Pull Request #2802 · bitcoin/bitcoin · GitHub
6392018-08-24T21:01:27  <gmaxwell> RPC based explorer also needs a reversed spentness index.
6402018-08-24T21:03:41  <gmaxwell> jnewbery: it's an interesting question of if people would actually use it if it was provided.  If they would I think it would be good to have. ... at least for the next 10 years, _some_ users would stop bleeding privacy wise.
6412018-08-24T21:07:29  *** Krellan has joined #bitcoin-core-dev
6422018-08-24T21:07:58  <sipa> jnewbery: i have the same love-hate relation with it :)
6432018-08-24T21:09:47  <jnewbery> good to know. Thanks!
6442018-08-24T21:09:58  <instagibbs> https://github.com/bitcoin/bitcoin/pull/14055 :( another RC?
6452018-08-24T21:10:42  <instagibbs> we(not me clearly, someone else) kind of broke the call. Probably is a way around it(walletupdatepsbt?)
6462018-08-24T21:10:55  <instagibbs> walletprocesspsbt
6472018-08-24T21:11:12  <gmaxwell> Well I think unless it turns out that the memory usage increase is fictional, we'll have another RC anyways.
6482018-08-24T21:12:35  <instagibbs> phew
6492018-08-24T21:16:26  *** farmerwampum has quit IRC
6502018-08-24T21:33:45  *** andytoshi has quit IRC
6512018-08-24T21:49:27  *** Chris_Stewart_5 has joined #bitcoin-core-dev
6522018-08-24T21:51:34  *** marcinja has quit IRC
6532018-08-24T21:56:29  *** dendisuhubdy has joined #bitcoin-core-dev
6542018-08-24T21:59:42  *** Victorsueca has quit IRC
6552018-08-24T22:00:54  *** Victorsueca has joined #bitcoin-core-dev
6562018-08-24T22:05:48  <luke-jr> woo finally got powerpc64 builds out of gitian working
6572018-08-24T22:06:23  <luke-jr> just in time for bluematt to close his PR :x
6582018-08-24T22:14:39  *** dendisuhubdy has quit IRC
6592018-08-24T22:16:32  *** michaelsdunn1 has quit IRC
6602018-08-24T22:19:11  *** jungly has joined #bitcoin-core-dev
6612018-08-24T22:19:38  *** meshcollider_ has joined #bitcoin-core-dev
6622018-08-24T22:34:54  *** Chris_Stewart_5 has quit IRC
6632018-08-24T22:35:20  *** murrayn has joined #bitcoin-core-dev
6642018-08-24T22:36:53  *** promag has joined #bitcoin-core-dev
6652018-08-24T22:38:47  *** jungly has quit IRC
6662018-08-24T22:56:18  *** profmac has quit IRC
6672018-08-24T23:00:23  *** leishman has joined #bitcoin-core-dev
6682018-08-24T23:01:18  *** AaronvanW has joined #bitcoin-core-dev
6692018-08-24T23:09:22  *** Dizzle has quit IRC
6702018-08-24T23:16:45  *** leishman has quit IRC
6712018-08-24T23:18:31  *** Chris_Stewart_5 has joined #bitcoin-core-dev
6722018-08-24T23:24:47  <luke-jr> is it preferable to de-bundle libpng from depends/qt, or patch the libpng bundled with depends/qt?
6732018-08-24T23:25:44  <luke-jr> cfields: ^
6742018-08-24T23:36:54  *** plankers has quit IRC
6752018-08-24T23:48:13  <luke-jr> probably debundling, since Qt's copy is missing the files needed :/
6762018-08-24T23:48:55  * luke-jr ponders using virtfs for gitian cache
6772018-08-24T23:51:54  <gmaxwell> jonasschnelli: so this protocol's use of a seperate chacha20 for the sizes is effectively halving the performance of our cypher.
6782018-08-24T23:53:03  <gmaxwell> jonasschnelli: the use of a message number as the nonce (instead of using a continious stream) will also decrease our performance for AVX by maybe 20%.
6792018-08-24T23:54:06  <sipa> gmaxwell: that's a strange gratuitous performance reduction, indeed
6802018-08-24T23:54:12  <sipa> i wonder why openssh did that
6812018-08-24T23:55:44  <gmaxwell> maybe because they thought they would read ahead lengths and thus schedule the other runs more efficiently?
6822018-08-24T23:57:29  <gmaxwell> https://tools.ietf.org/html/draft-josefsson-ssh-chacha20-poly1305-openssh-00#section-3
6832018-08-24T23:59:14  <sipa> the separation of the length and data cipher makes sense, but there's no need to discard 28 bytes from each output of K_1