12016-11-10T00:00:49 <jtimon> between 10775775 and d59a5184 my computer says, but I've risen false alarms before
22016-11-10T00:07:14 <jtimon> but if people want to try to reproduce...again it's walletbackup that fails, with "python3 ./qa/pull-tester/rpc-tests.py" (with or without -parallel=N), but not python3 "./qa/pull-tester/rpc-tests.py backupwallet"
32016-11-10T00:09:05 <jtimon> this happens in d59a5184 but not in 10775775 (to me)
42016-11-10T00:10:24 <gmaxwell> cfields: you've been helping people with segwit mining updates: does this all sound good to you? https://bitcointalk.org/index.php?topic=1674590.0
52016-11-10T00:11:22 *** JackH has quit IRC
62016-11-10T00:13:49 <cfields> gmaxwell: sounds about right. Excluding the obvious possible serialization pitfalls
72016-11-10T00:13:59 <BlueMatt> cfields: wanna respond to him there? :p
82016-11-10T00:14:20 <sipa> gmaxwell: either he uses default_witness_commitmnt, and he does not need to care about txid or hash or anything
92016-11-10T00:14:20 <BlueMatt> cfields: have anything that needs review? happy to trade for an ack on #9075 :p
102016-11-10T00:14:22 <gribble> https://github.com/bitcoin/bitcoin/issues/9075 | Decouple peer-processing-logic from block-connection-logic (#3) by TheBlueMatt · Pull Request #9075 · bitcoin/bitcoin · GitHub
112016-11-10T00:14:35 <sipa> or he does care about it, and then he needs to compute the witness root himself
122016-11-10T00:15:21 <cfields> BlueMatt: heh, i'd have to find/dust off my bitcointalk credentials
132016-11-10T00:16:14 <cfields> BlueMatt: not yet, I'm rethinking some of mine to take encryption into account. Happy to finish up with 9075 and ack though, I think you already addressed my concerns
142016-11-10T00:16:38 <BlueMatt> cool, I'll move on and look at #8895, then
152016-11-10T00:16:40 <gribble> https://github.com/bitcoin/bitcoin/issues/8895 | Better SigCache Implementation by JeremyRubin · Pull Request #8895 · bitcoin/bitcoin · GitHub
162016-11-10T00:17:13 <BlueMatt> splitting main.cpp is so close I can taste it :p
172016-11-10T00:17:40 <cfields> hehe
182016-11-10T00:18:38 <sipa> woot
192016-11-10T00:19:08 <cfields> gmaxwell: oh, default_witness_commitment contains the OP_RETURN + magic already. so as written, it's not clear if he'll dupe it
202016-11-10T00:19:39 <sipa> cfields: yeah, see what i said abpve
212016-11-10T00:23:56 *** davec has quit IRC
222016-11-10T00:24:10 *** AaronvanW has quit IRC
232016-11-10T00:25:45 *** davec has joined #bitcoin-core-dev
242016-11-10T00:42:02 *** harrymm has quit IRC
252016-11-10T00:58:14 <jtimon> it seem to magically happen in 50e8a9cc ...
262016-11-10T00:58:45 <jtimon> e9847303 works for me
272016-11-10T00:58:46 <morcos> jtimon: i really think you mean someone else.. i have no idea what you are talking about
282016-11-10T00:59:00 <gmaxwell> sipa: he still needs the txid to compute the normal hashroot.
292016-11-10T00:59:04 <gmaxwell> cfields: good spotting.
302016-11-10T01:00:11 <jtimon> morcos: no, once we established your pr was not related, I was talking in general: these pr_ids are before your pr
312016-11-10T01:00:29 <jtimon> s/pr_ids/commit ids/
322016-11-10T01:01:54 <jtimon> if anyone can confirm that does or doesn't have any problems with 50e8a9cc running the rpc tests, please tell me
332016-11-10T01:06:51 <jtimon> morcos: yep, sorry, my fault, I confused you with sdaftuar
342016-11-10T01:07:43 <sipa> gmaxwell: right!
352016-11-10T01:08:37 <jtimon> morcos: whose pr is unrelated to my comments as well, I just tried his pr to fix my problem and it didn't work (but it's not his pr's fault)
362016-11-10T01:12:13 <jtimon> and talking in general, if anyone besides travis can confirm that he run the rpc tests after 50e8a9cc that would be helpful
372016-11-10T01:36:52 *** fengling has joined #bitcoin-core-dev
382016-11-10T01:57:20 *** abpa has quit IRC
392016-11-10T01:59:59 <kanzure> sipa: have segwit bugfxies been backported(? forward-ported?) to elements?
402016-11-10T02:02:51 <sipa> kanzure: elements alpha has not been touched in a long time
412016-11-10T02:03:23 <sipa> we're working on a successor which will be based on segwit in core
422016-11-10T02:05:02 *** d9b4bef9 has quit IRC
432016-11-10T02:06:09 *** d9b4bef9 has joined #bitcoin-core-dev
442016-11-10T02:08:39 *** PaulCapestany has quit IRC
452016-11-10T02:09:17 *** PaulCapestany has joined #bitcoin-core-dev
462016-11-10T02:10:38 *** PaulCapestany has quit IRC
472016-11-10T02:11:23 *** PaulCapestany has joined #bitcoin-core-dev
482016-11-10T03:02:24 *** baldur has quit IRC
492016-11-10T03:05:25 *** harrymm has joined #bitcoin-core-dev
502016-11-10T03:21:57 <cfields> gmaxwell: thanks for relaying
512016-11-10T03:28:21 *** baldur has joined #bitcoin-core-dev
522016-11-10T03:31:33 *** jtimon has quit IRC
532016-11-10T03:32:31 *** btcdrak has joined #bitcoin-core-dev
542016-11-10T03:50:04 *** Chris_Stewart_5 has quit IRC
552016-11-10T04:05:55 *** Chris_Stewart_5 has joined #bitcoin-core-dev
562016-11-10T04:25:33 *** Chris_Stewart_5 has quit IRC
572016-11-10T04:29:35 *** abpa has joined #bitcoin-core-dev
582016-11-10T04:34:03 *** kadoban has quit IRC
592016-11-10T05:04:19 *** Giszmo has quit IRC
602016-11-10T05:08:55 *** PaulCape_ has joined #bitcoin-core-dev
612016-11-10T05:11:47 *** PaulCapestany has quit IRC
622016-11-10T06:11:13 *** DigiByteDev has joined #bitcoin-core-dev
632016-11-10T06:20:10 *** Ylbam has quit IRC
642016-11-10T06:20:44 *** baldur has quit IRC
652016-11-10T06:22:05 *** baldur has joined #bitcoin-core-dev
662016-11-10T06:22:46 *** asoltys_ is now known as asoltys
672016-11-10T06:25:17 *** Alopex has quit IRC
682016-11-10T06:26:22 *** Alopex has joined #bitcoin-core-dev
692016-11-10T06:27:51 <wumpus> oops, the harddisk of one of my nodes just broke down
702016-11-10T06:29:21 <wumpus> Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
712016-11-10T06:29:24 <wumpus> # 1 Extended offline Completed: read failure 40% 14648 1999472904
722016-11-10T06:29:37 *** face has quit IRC
732016-11-10T06:31:00 <wumpus> I've had harddisks die on me before, but never in this way, there's a huge range where reading just fails. Usually they just stop spinning up at all.
742016-11-10T06:41:36 <sipa> i don't think my SSD ever spinned at all :(
752016-11-10T06:44:06 <wumpus> the user needs to provide the momentum
762016-11-10T06:45:00 <sipa> :D
772016-11-10T06:45:08 <sipa> that's very SMART
782016-11-10T06:46:11 <paveljanik> and more fitness :-)
792016-11-10T06:47:22 *** LeMiner has joined #bitcoin-core-dev
802016-11-10T07:02:06 *** Ylbam has joined #bitcoin-core-dev
812016-11-10T07:15:04 <bitcoin-git> [bitcoin] paveljanik opened pull request #9121: Initialize variable to prevent compiler warning (master...20161110_rpcdump_uninitialized) https://github.com/bitcoin/bitcoin/pull/9121
822016-11-10T07:17:58 *** btcdrak has quit IRC
832016-11-10T08:28:23 *** Cheeseo has quit IRC
842016-11-10T08:28:44 *** rubensayshi has joined #bitcoin-core-dev
852016-11-10T08:55:02 *** Cheeseo has joined #bitcoin-core-dev
862016-11-10T08:55:02 *** Cheeseo has joined #bitcoin-core-dev
872016-11-10T09:02:10 *** btcdrak has joined #bitcoin-core-dev
882016-11-10T09:08:03 *** molz has joined #bitcoin-core-dev
892016-11-10T09:09:53 *** JackH has joined #bitcoin-core-dev
902016-11-10T09:11:49 *** moli has quit IRC
912016-11-10T09:25:15 <bitcoin-git> [bitcoin] visvirial opened pull request #9122: fix getnettotals RPC description about timemillis. (master...fix-getnettotals-rpc-desc) https://github.com/bitcoin/bitcoin/pull/9122
922016-11-10T09:31:24 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/faec09bc7f03...537e0cb252a3
932016-11-10T09:31:24 <bitcoin-git> bitcoin/master 45d372f UdjinM6: Missed one "return false" in recent refactoring in #9067
942016-11-10T09:31:25 <bitcoin-git> bitcoin/master 537e0cb Wladimir J. van der Laan: Merge #9120: bug: Missed one "return false" in recent refactoring in #9067...
952016-11-10T09:31:35 <bitcoin-git> [bitcoin] laanwj closed pull request #9120: bug: Missed one "return false" in recent refactoring in #9067 (master...fixExitCodesBitcoinTx) https://github.com/bitcoin/bitcoin/pull/9120
962016-11-10T09:32:40 *** AaronvanW has joined #bitcoin-core-dev
972016-11-10T09:33:06 *** AaronvanW has quit IRC
982016-11-10T09:33:06 *** AaronvanW has joined #bitcoin-core-dev
992016-11-10T09:35:29 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/537e0cb252a3...aab102cbae6b
1002016-11-10T09:35:30 <bitcoin-git> bitcoin/master bdcba6d Pavel JanÃk: Initialize variable to prevent compiler warning
1012016-11-10T09:35:30 <bitcoin-git> bitcoin/master aab102c Wladimir J. van der Laan: Merge #9121: Initialize variable to prevent compiler warning...
1022016-11-10T09:35:44 <bitcoin-git> [bitcoin] laanwj closed pull request #9121: Initialize variable to prevent compiler warning (master...20161110_rpcdump_uninitialized) https://github.com/bitcoin/bitcoin/pull/9121
1032016-11-10T09:38:39 *** Guyver2 has joined #bitcoin-core-dev
1042016-11-10T09:41:49 *** nickler has quit IRC
1052016-11-10T09:43:54 *** DigiByteDev has quit IRC
1062016-11-10T09:47:59 *** owowo has quit IRC
1072016-11-10T09:48:50 *** afk11 has quit IRC
1082016-11-10T09:48:54 *** testnet has quit IRC
1092016-11-10T09:55:45 *** afk11 has joined #bitcoin-core-dev
1102016-11-10T09:55:46 *** afk11 has quit IRC
1112016-11-10T09:55:46 *** afk11 has joined #bitcoin-core-dev
1122016-11-10T09:57:46 *** DigiByteDev has joined #bitcoin-core-dev
1132016-11-10T10:01:02 *** DigiByteDev has quit IRC
1142016-11-10T10:01:51 *** paveljanik has quit IRC
1152016-11-10T10:06:29 *** MarcoFalke has joined #bitcoin-core-dev
1162016-11-10T10:06:50 <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/aab102cbae6b...4a2b170c075c
1172016-11-10T10:06:51 <bitcoin-git> bitcoin/master a79f864 Masahiko Hyuga: fix getnettotals RPC description about timemillis.
1182016-11-10T10:06:51 <bitcoin-git> bitcoin/master 4a2b170 MarcoFalke: Merge #9122: fix getnettotals RPC description about timemillis....
1192016-11-10T10:07:05 <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9122: fix getnettotals RPC description about timemillis. (master...fix-getnettotals-rpc-desc) https://github.com/bitcoin/bitcoin/pull/9122
1202016-11-10T10:08:23 *** Lauda has quit IRC
1212016-11-10T10:17:38 *** Lauda has joined #bitcoin-core-dev
1222016-11-10T10:24:24 <bitcoin-git> [bitcoin] MarcoFalke pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/4a2b170c075c...e5364991daec
1232016-11-10T10:24:25 <bitcoin-git> bitcoin/master fac1141 MarcoFalke: [qa] preciousblock: Use assert_equal and BitcoinTestFramework.__init__
1242016-11-10T10:24:25 <bitcoin-git> bitcoin/master fa97ccb MarcoFalke: [qa] util: Rework sync_*()...
1252016-11-10T10:24:26 <bitcoin-git> bitcoin/master e536499 MarcoFalke: Merge #9097: [qa] Rework sync_* and preciousblock.py...
1262016-11-10T10:24:39 <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9097: [qa] Rework sync_* and preciousblock.py (master...Mf1611-qaSyncAndPrecious) https://github.com/bitcoin/bitcoin/pull/9097
1272016-11-10T10:25:18 *** Lauda has joined #bitcoin-core-dev
1282016-11-10T10:29:25 *** Lauda has quit IRC
1292016-11-10T10:33:37 *** laurentmt has joined #bitcoin-core-dev
1302016-11-10T10:35:29 *** Lauda has joined #bitcoin-core-dev
1312016-11-10T10:38:33 *** fengling has quit IRC
1322016-11-10T10:38:46 *** fanquake has joined #bitcoin-core-dev
1332016-11-10T10:38:48 *** paveljanik has joined #bitcoin-core-dev
1342016-11-10T10:43:24 *** laurentmt has quit IRC
1352016-11-10T10:48:38 *** Guyver2 has quit IRC
1362016-11-10T10:50:17 *** laurentmt has joined #bitcoin-core-dev
1372016-11-10T10:52:56 *** laurentmt has quit IRC
1382016-11-10T10:57:52 *** whphhg has quit IRC
1392016-11-10T11:13:03 *** fanquake has quit IRC
1402016-11-10T11:36:13 *** cdecker has joined #bitcoin-core-dev
1412016-11-10T11:54:45 *** MarcoFalke has left #bitcoin-core-dev
1422016-11-10T11:58:41 *** nickler has joined #bitcoin-core-dev
1432016-11-10T12:11:57 <jonasschnelli> would it make sense to use a 1byte command code instead of the 12byte char in the new bip151 message structure?
1442016-11-10T12:12:29 <jonasschnelli> Or would that lead to problems defining new commands and possible overlaps?
1452016-11-10T12:12:45 <jonasschnelli> 0x01 = inv, 0x02 = getaddr, etc.
1462016-11-10T12:13:33 <jonasschnelli> It would just save another 3-11 bytes per message.
1472016-11-10T12:15:28 *** AtashiCon has quit IRC
1482016-11-10T12:15:41 *** AtashiCon has joined #bitcoin-core-dev
1492016-11-10T12:15:59 *** tunafizz has quit IRC
1502016-11-10T12:16:00 *** moli has joined #bitcoin-core-dev
1512016-11-10T12:16:30 *** tunafizz has joined #bitcoin-core-dev
1522016-11-10T12:17:54 *** molz has quit IRC
1532016-11-10T12:21:15 *** cryptapus has joined #bitcoin-core-dev
1542016-11-10T12:44:59 *** jannes has joined #bitcoin-core-dev
1552016-11-10T13:08:55 <wumpus> jonasschnelli: I disagree with doing that: using 12-byte identifiers gives an enormouse scope for different parties to define commands that don't overlap
1562016-11-10T13:09:20 <wumpus> jonasschnelli: using a 1 or 4 byte command code implies a central authority dealing out command codes, and we've had many collisions already
1572016-11-10T13:09:39 <wumpus> (with other things like inv enums)
1582016-11-10T13:10:16 <wumpus> let's not throw away the baby with the bath water while (over) optimizing
1592016-11-10T13:24:29 <jonasschnelli> Okay.
1602016-11-10T13:33:47 <wumpus> you could bring it down from 12 to, say, 8 I guess, 64-bit identifiers can be pretty unique
1612016-11-10T13:34:22 *** harrymm has quit IRC
1622016-11-10T13:34:30 <wumpus> heck even 4 bytes probably won't cause a lot of fighting, this is not about about bits but enums
1632016-11-10T13:35:01 <wumpus> but 256 options is just too few and will guarantee fighting, that will end in fire :)
1642016-11-10T13:39:02 <jonasschnelli> It looks like that 256 options should be sufficient from the current perspective... but right,.. maybe we better keep it
1652016-11-10T13:40:21 <jonasschnelli> An INV would go down from 60bytes to 58bytes. :(
1662016-11-10T13:40:26 <jonasschnelli> I guess it's not worth it.
1672016-11-10T13:40:44 <wumpus> INVs can already combine right?
1682016-11-10T13:41:03 <wumpus> ideally send a whole bunch of them at once, not one per message
1692016-11-10T13:41:24 <jonasschnelli> A right. INV is a vector.
1702016-11-10T13:41:46 <wumpus> and yes, 256 bytes is 'sufficient from the current perspective' but that's not my point, my point is to make it feature proof and allow permissionless innovation
1712016-11-10T13:41:53 <wumpus> future proof*
1722016-11-10T13:42:07 <jonasschnelli> Yes. I think this is wise.
1732016-11-10T13:43:02 <jonasschnelli> I wonder if reducing the poly1305 mac tag (currently 16 bytes) would be something that could be considered.
1742016-11-10T13:43:12 <wumpus> is there ever a use case where lots of small messages are sent?
1752016-11-10T13:43:12 <jonasschnelli> Probably not.
1762016-11-10T13:43:17 *** jtimon has joined #bitcoin-core-dev
1772016-11-10T13:43:24 <jonasschnelli> wumpus: spv?
1782016-11-10T13:44:19 <wumpus> any that are not vector-able?
1792016-11-10T13:44:58 <wumpus> if so it may make sense to have a 'group' message which just concatenates a bunch of messages (of the same type, say, or at least with types RLE'd), under one mac tag and outer header and such
1802016-11-10T13:45:02 <jonasschnelli> Probably not.
1812016-11-10T13:45:26 <wumpus> that would make all messages vector-able. Although it's a design choice at which level to do that...
1822016-11-10T13:45:28 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1832016-11-10T13:46:03 <jonasschnelli> Bip151 encrypted message packaging (https://github.com/bitcoin/bips/blob/master/bip-0151.mediawiki#encrypted-messages-structure) allows that. Because from the protocol perspective, combining them seems to have no downside
1842016-11-10T13:46:13 <jonasschnelli> Which is different when we look at the implementation
1852016-11-10T13:46:14 <wumpus> oh that's already possible
1862016-11-10T13:46:34 <wumpus> I don't see the problem then, or motivation for looking at few-byte wins in the header
1872016-11-10T13:47:07 <wumpus> the implementation could be smart about it: queue up messages until a certain threshold is reached or timeout
1882016-11-10T13:47:14 <wumpus> like TCP nagle algorithm
1892016-11-10T13:47:28 <jonasschnelli> I think switching to encrypted traffic gives us kind of a one-time chance to improve the message structure. I just try to consider every possible optimization.
1902016-11-10T13:47:37 *** justanotheruser has quit IRC
1912016-11-10T13:47:56 <jonasschnelli> Yes. I have discussed that yesterday with cfields. A timeout until messaged gets combined..
1922016-11-10T13:47:57 <wumpus> do beware scope creep
1932016-11-10T13:48:08 <jonasschnelli> Good point.
1942016-11-10T13:50:21 *** Chris_Stewart_5 has quit IRC
1952016-11-10T13:51:02 <wumpus> in any case: those implementations are possible, they have been done many times before. I wouldn't worry too much about optimizing current implementtions when desiging a protocol
1962016-11-10T13:51:13 <wumpus> optimizing for
1972016-11-10T13:51:38 <wumpus> this needs to be future proof because we don't want to be in the same situation again in say, 5 years
1982016-11-10T13:51:45 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1992016-11-10T13:52:37 <wumpus> also means security should be adequate: don't reduce the MAC tags to near current security bounds
2002016-11-10T13:54:53 <jonasschnelli> ack
2012016-11-10T13:55:48 *** harrymm has joined #bitcoin-core-dev
2022016-11-10T13:56:42 *** striker227 has joined #bitcoin-core-dev
2032016-11-10T13:56:56 *** striker227 has quit IRC
2042016-11-10T14:01:21 *** paveljanik has quit IRC
2052016-11-10T14:17:42 *** Guyver2 has joined #bitcoin-core-dev
2062016-11-10T14:21:53 *** owowo has joined #bitcoin-core-dev
2072016-11-10T14:21:54 *** owowo has joined #bitcoin-core-dev
2082016-11-10T14:21:54 *** owowo has joined #bitcoin-core-dev
2092016-11-10T14:24:42 *** Guyver2__ has joined #bitcoin-core-dev
2102016-11-10T14:27:38 *** Guyver2 has quit IRC
2112016-11-10T14:27:45 *** Guyver2__ is now known as Guyver2
2122016-11-10T14:58:32 *** aalex_ has joined #bitcoin-core-dev
2132016-11-10T14:59:59 *** Chris_Stewart_5 has quit IRC
2142016-11-10T15:00:48 *** aalex_ has quit IRC
2152016-11-10T15:03:30 <bitcoin-git> [bitcoin] laanwj pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/e5364991daec...71bc39eb7483
2162016-11-10T15:03:31 <bitcoin-git> bitcoin/master b2e178a Matt Corallo: Add deserialize + CheckBlock benchmarks, and a full block hex
2172016-11-10T15:03:31 <bitcoin-git> bitcoin/master eecffe5 Matt Corallo: Remove redundant duplicate-input check from CheckTransaction
2182016-11-10T15:03:32 <bitcoin-git> bitcoin/master e2b3fb3 Matt Corallo: Optimize vInOutPoints insertion a bit
2192016-11-10T15:03:42 <bitcoin-git> [bitcoin] laanwj closed pull request #9049: Remove duplicatable duplicate-input check from CheckTransaction (master...2016-10-bench-checkblock) https://github.com/bitcoin/bitcoin/pull/9049
2202016-11-10T15:05:11 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2212016-11-10T15:17:18 *** abpa has quit IRC
2222016-11-10T15:22:38 *** Giszmo has joined #bitcoin-core-dev
2232016-11-10T16:00:10 *** Ylbam has quit IRC
2242016-11-10T16:13:19 *** PaulCape_ has quit IRC
2252016-11-10T16:15:59 *** Ylbam has joined #bitcoin-core-dev
2262016-11-10T16:18:33 *** PaulCapestany has joined #bitcoin-core-dev
2272016-11-10T16:34:00 *** testnet has joined #bitcoin-core-dev
2282016-11-10T16:39:51 <sipa> jonasschnelli: openssl just reported a dos attack bug in their chacha20-poly1305, we may want to verify if we'd be susceptible
2292016-11-10T16:40:37 <sipa> jonasschnelli: if you want to shave off some bytes, make the command codes variable length
2302016-11-10T16:42:47 <sipa> or they could be made lowercase only, and use bitpacking to map 12-character strings to 64-bit values ;)
2312016-11-10T17:01:44 *** Guyver2 has quit IRC
2322016-11-10T17:14:08 *** rubensayshi has quit IRC
2332016-11-10T17:18:56 *** laurentmt has joined #bitcoin-core-dev
2342016-11-10T17:20:57 *** Chris_Stewart_5 has quit IRC
2352016-11-10T17:22:02 *** Guyver2 has joined #bitcoin-core-dev
2362016-11-10T17:22:31 *** laurentmt has quit IRC
2372016-11-10T17:26:59 <cfields> BlueMatt: ping. I still need help understanding the race comment in #9075.
2382016-11-10T17:27:01 <gribble> https://github.com/bitcoin/bitcoin/issues/9075 | Decouple peer-processing-logic from block-connection-logic (#3) by TheBlueMatt · Pull Request #9075 · bitcoin/bitcoin · GitHub
2392016-11-10T17:30:05 <gmaxwell> The TCP/IP overhead is already pretty large, so reducing the command size is less relative improvement than you might think.
2402016-11-10T17:32:01 <gmaxwell> I think in gneral we'd be better off defining batch operations, like how an inv contains many items.
2412016-11-10T17:32:42 *** jannes has quit IRC
2422016-11-10T17:34:20 <gmaxwell> sipa: lol base-32, 5*12 = 60.
2432016-11-10T17:34:38 *** aalex has quit IRC
2442016-11-10T17:36:53 *** zmanian____ has quit IRC
2452016-11-10T17:39:33 *** zmanian____ has joined #bitcoin-core-dev
2462016-11-10T17:40:58 <sipa> gmaxwell: base 26 * 12 = 57 bits
2472016-11-10T17:45:54 *** abpa has joined #bitcoin-core-dev
2482016-11-10T17:50:32 *** aalex has joined #bitcoin-core-dev
2492016-11-10T17:52:06 *** laurentmt has joined #bitcoin-core-dev
2502016-11-10T17:52:15 *** laurentmt has quit IRC
2512016-11-10T17:55:57 *** paveljanik has joined #bitcoin-core-dev
2522016-11-10T17:55:57 *** paveljanik has joined #bitcoin-core-dev
2532016-11-10T18:00:44 <gmaxwell> sipa: you think people complain about endianness, make them pack a non-power of two range... :P
2542016-11-10T18:01:37 *** whphhg has joined #bitcoin-core-dev
2552016-11-10T18:02:31 *** Guyver2 has quit IRC
2562016-11-10T18:36:51 <BlueMatt> cfields: I think its dependant on multithreaded-peer-logic
2572016-11-10T18:37:53 <cfields> BlueMatt: ok, so the issue is that it drops the lock and then re-takes it, and in the meantime, the nodeid could've changed (if multi-threaded) ?
2582016-11-10T18:38:12 <BlueMatt> cfields: if you have two block sources at once
2592016-11-10T18:38:21 <cfields> right
2602016-11-10T18:38:23 <BlueMatt> or, eg, you have a node which gives you a block and then you submitblock the same block
2612016-11-10T18:39:06 <jtimon> ping https://github.com/bitcoin/bitcoin/pull/8855
2622016-11-10T18:39:43 <cfields> BlueMatt: in that case though, if it's a full block that's submitted, the ban bool could be switched as well, so it'd be possible to skip punishing a peer who's sent a bad full block?
2632016-11-10T18:39:50 <cfields> or am i misreading?
2642016-11-10T18:40:47 <cfields> (i'm just wondering if it makes sense to switch to a multimap, so that every node gets the correct response)
2652016-11-10T18:41:10 <BlueMatt> cfields: huh? no, I'm saying in thread a) a peer adds itself to mapBlockSource as the person to punish, then in thread b) another peer adds itself, then the block is processed from peer a...peer b doesnt get processed as "already rejected" but doesnt get the punishment for invalid block, either
2662016-11-10T18:41:23 <BlueMatt> i highly dont think it matters
2672016-11-10T18:44:57 <sipa> turn mapBlockSource into a multimap?
2682016-11-10T18:45:19 <cfields> sipa: heh, see 3 lines up :)
2692016-11-10T18:46:09 <cfields> i just don't see why we'd complicate possible future logic if we could easily guarantee that everyone gets the correct response
2702016-11-10T18:48:02 <BlueMatt> huh? I mean if they're on an invalid chain we'll ban them on the next block anyway
2712016-11-10T18:49:11 *** Guyver2 has joined #bitcoin-core-dev
2722016-11-10T18:53:30 *** kadoban has joined #bitcoin-core-dev
2732016-11-10T18:53:47 <cfields> BlueMatt: Is there some advantage to leaving it this way? "we'll get it eventually" seems like a strange argument here.
2742016-11-10T18:54:05 <BlueMatt> code diff? ease of change?
2752016-11-10T18:54:25 *** droark has quit IRC
2762016-11-10T18:55:03 <jonasschnelli> sipa: BIP151 has varlen command codes
2772016-11-10T18:55:25 <sipa> jonasschnelli: oh!
2782016-11-10T18:55:30 <sipa> jonasschnelli: then who cares
2792016-11-10T18:55:40 <jonasschnelli> Yes. I think this should be sufficient.
2802016-11-10T18:55:42 <BlueMatt> cfields: also, DoS needs to die
2812016-11-10T18:55:46 <BlueMatt> cfields: so we can fix it then
2822016-11-10T18:57:15 <jonasschnelli> sipa: I have started extracting ChaCha20 and Poly1305 from openssh (IEFT ChaCha20Poly1305 as well as ChaCha20Poly1305@openssh) https://github.com/jonasschnelli/chacha20poly1305
2832016-11-10T18:57:30 <jonasschnelli> I will pass it over to you soon.. :)
2842016-11-10T18:57:45 <jonasschnelli> I just wanted to make some benachmarks
2852016-11-10T18:57:46 <sipa> jonasschnelli: i'm not going to even look at it until the network refactors are done
2862016-11-10T18:57:50 <sipa> jonasschnelli: i expect it to be very easy
2872016-11-10T18:57:50 <cfields> BlueMatt: i suppose it's not getting worked up about if it requires multithreading anyway
2882016-11-10T18:58:17 <BlueMatt> cfields: my target for multithreading is 0.14, though I highly doubt I'll make that
2892016-11-10T18:59:30 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2902016-11-10T19:02:46 <sipa> meetingh?
2912016-11-10T19:02:50 <jonasschnelli> yes
2922016-11-10T19:03:00 <wumpus> #startmeeting
2932016-11-10T19:03:00 <lightningbot> Meeting started Thu Nov 10 19:03:00 2016 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
2942016-11-10T19:03:00 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
2952016-11-10T19:03:26 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012
2962016-11-10T19:03:35 <kanzure> hi.
2972016-11-10T19:03:36 <wumpus> proposed topics?
2982016-11-10T19:03:41 <paveljanik> Hi
2992016-11-10T19:03:41 <btcdrak> holidays
3002016-11-10T19:03:48 *** MarcoF____ has joined #bitcoin-core-dev
3012016-11-10T19:03:58 <BlueMatt> reasons why there is no way in hell we could multithread ProcessMessages in 0.14
3022016-11-10T19:04:00 <BlueMatt> :)
3032016-11-10T19:04:03 <achow101> oh, meeting already?
3042016-11-10T19:04:05 <jonasschnelli> Not sure if it's OT or not, but if possible, it like to propose a short topic "concept of hybrid SPV"
3052016-11-10T19:04:27 <BlueMatt> anyone else just want a bloom filter commitment soft fork for that?
3062016-11-10T19:04:36 <cfields> jonasschnelli: yes please
3072016-11-10T19:04:51 <wumpus> #topic hybrid SPV
3082016-11-10T19:04:55 <jonasschnelli> I wanted to ask if we'd like to see some form of "SPV" (wrong term, i agree) mode in Bitcoin-Core, if it's worth to continue the work
3092016-11-10T19:05:08 <jonasschnelli> IMO it could affect the userbase
3102016-11-10T19:05:14 <kanzure> was this about initial block download?
3112016-11-10T19:05:17 <jonasschnelli> (from expertish to novicish)
3122016-11-10T19:05:50 <wumpus> I think the idea has always been to have some kind of SPV mode in bitcoin core, yes
3132016-11-10T19:05:53 <sipa> why would it be a wrong term?
3142016-11-10T19:05:59 <kanzure> "SPV" was the one about fraud proofs etc
3152016-11-10T19:06:17 <sipa> ok, "client mode" as bitcoin's creator called it then
3162016-11-10T19:06:22 <jonasschnelli> I like SPV, ... but some people told me SPV implies bloom filters. I somehow disagree with that though
3172016-11-10T19:06:25 <wumpus> even if full block SPV without bloom filters
3182016-11-10T19:06:47 <jonasschnelli> I think full block hybrid SPV sounds ideal IMO
3192016-11-10T19:06:52 <BlueMatt> ^ that
3202016-11-10T19:06:58 <wumpus> no, it doesn't need bloom filters. Say you want to do SPV against a local node which stores the block chain anyway, there's no need for bloom filters
3212016-11-10T19:07:10 <jonasschnelli> Once we have 150/151, we could add bloomfiltering options against bip150 authed peers.
3222016-11-10T19:07:21 <BlueMatt> preferably not
3232016-11-10T19:07:23 <sdaftuar> jonasschnelli: hybrid spv means it starts out as spv, but eventually becomes fully validating?
3242016-11-10T19:07:31 <jonasschnelli> sdaftuar: right.
3252016-11-10T19:07:34 <wumpus> and yes could always be added later, if there is a sane way of filtering that doesn't throw all your privacy out of thew indow
3262016-11-10T19:07:34 <BlueMatt> better designs are possible, but for full blocks agreed
3272016-11-10T19:07:36 <sdaftuar> jonasschnelli: sounds awesome!
3282016-11-10T19:07:41 <NicolasDorier> I'm bit lost here, why is it called SPV if the node store the blockchain oO
3292016-11-10T19:07:56 <BlueMatt> NicolasDorier: SPV here is referring to the trust model of trusting miners
3302016-11-10T19:07:56 <NicolasDorier> is there some past discussion somewhere about it I can read ?
3312016-11-10T19:08:00 <wumpus> NicolasDorier: no, it doesn't need to store it
3322016-11-10T19:08:02 <jonasschnelli> For those who haven seen it, there a full working PR for the hybrid SPV: https://github.com/bitcoin/bitcoin/pull/9076
3332016-11-10T19:08:09 <kanzure> the level of confusion around naming and name-implications is really high. it's sort of amusing.
3342016-11-10T19:08:19 <wumpus> NicolasDorier: it just receives the blocks, filters them locally intead of setting a bloom filter
3352016-11-10T19:08:41 <jonasschnelli> Haven't got conceptual ACKs and wasen't sure if its worth to continue, ... make it clean and stable, etc.
3362016-11-10T19:08:42 <NicolasDorier> how is it different from a pruned node?
3372016-11-10T19:08:47 <kanzure> if it's not immediately obvious to most of us that feature x is included (like block downloading or something) then we need better names :P
3382016-11-10T19:08:55 <jonasschnelli> NicolasDorier: it does no validation
3392016-11-10T19:09:00 <wumpus> NicolasDorier: it would allow receiving transactions while validation is not complete yet, for example
3402016-11-10T19:09:10 <jonasschnelli> NicolasDorier: just header chainwork check and pass the transaction to the wallet
3412016-11-10T19:09:12 <wumpus> a pruned node is a full node
3422016-11-10T19:09:14 *** Victorsueca has quit IRC
3432016-11-10T19:09:34 <NicolasDorier> ah ok, so basically a node without UTXO ?
3442016-11-10T19:09:37 <jonasschnelli> The main difference is, that we load blocks during IBD first from the wallet brthday.
3452016-11-10T19:09:43 <jonasschnelli> NicolasDorier: yes. No UTXO set
3462016-11-10T19:09:53 <jonasschnelli> No mempool, same fee problem that all SPV wallets have to deal with
3472016-11-10T19:09:55 <NicolasDorier> ah ok got it
3482016-11-10T19:10:07 <wumpus> pruning has nothing to do with SPV, full node has nothing to do with 'storing the block chain' or not
3492016-11-10T19:10:08 <jonasschnelli> But with the option of slowing becomming a full node
3502016-11-10T19:10:08 <btcdrak> jonasschnelli: I dont think anyone concept ack because it's so obviously a good feature.
3512016-11-10T19:10:11 <wumpus> right, it is about the UTXO set
3522016-11-10T19:10:25 <wumpus> jonasschnelli: I've been very busy, sorry
3532016-11-10T19:10:27 <jtimon> oh, I undesrtand the details now, headers first, then you are an spv node that fetches the full block instead of using bloom filter but you keep syncing on the background to become a full node, sounds all fine
3542016-11-10T19:10:30 <wumpus> jonasschnelli: it's obviously a good idea
3552016-11-10T19:10:32 <BlueMatt> jonasschnelli: but still fill-blocks-in-background?
3562016-11-10T19:10:39 <BlueMatt> or are we moving away from that?
3572016-11-10T19:10:41 <BlueMatt> but, yes, obviously good
3582016-11-10T19:10:48 *** Victorsueca has joined #bitcoin-core-dev
3592016-11-10T19:10:52 <jonasschnelli> BlueMatt: there are two modes, purespv and hybridspv
3602016-11-10T19:10:53 <wumpus> BlueMatt: that should certainly be an option
3612016-11-10T19:10:59 <jonasschnelli> purespv will just throw away blocks...
3622016-11-10T19:11:08 <BlueMatt> oh, only optional? hum, not a fan of it only being optional
3632016-11-10T19:11:11 <jonasschnelli> hybrid spv will keep the blocks and does IBD in the background (maybe throttled)
3642016-11-10T19:11:20 <wumpus> hybrid stores them for later
3652016-11-10T19:11:28 <wumpus> BlueMatt: why not?
3662016-11-10T19:11:29 <morcos> jonasschnelli: +1 on the ability to throttle, think that would make it very useful
3672016-11-10T19:11:31 <BlueMatt> i mean you dont have to keep blocks, but you still want to ibd and build the utxo set
3682016-11-10T19:11:40 <BlueMatt> /validate
3692016-11-10T19:11:58 <BlueMatt> wumpus: because if you're not gonna be upgrading to full trust model why are you running bitcoin core as a wallet?
3702016-11-10T19:11:58 <wumpus> being able to use bitcoin core as a full SPV node would be useful, especially with a local node that does validate
3712016-11-10T19:12:03 <wumpus> BlueMatt: why not?
3722016-11-10T19:12:05 <jonasschnelli> BlueMatt: if you throw away blocks with the long term goal of getting a full node your wasting bandwith
3732016-11-10T19:12:15 <BlueMatt> jonasschnelli: no, you're not!
3742016-11-10T19:12:21 <BlueMatt> jonasschnelli: you're upgrading your security model
3752016-11-10T19:12:28 <jonasschnelli> BlueMatt:: you have to download the block again!
3762016-11-10T19:12:50 <BlueMatt> jonasschnelli: huh? oh, you mean keeping the blocks at the tip that you're using to fill wallet...ahh, misunderstanding
3772016-11-10T19:12:58 <jonasschnelli> purespv is interesting when connecting to a trusted node.
3782016-11-10T19:13:25 <wumpus> right
3792016-11-10T19:13:35 <BlueMatt> I'm not sure if its worth the effort to make bitcoin core a competitive spv wallet - they mostly all already support connecting to a trusted node (though need auth)
3802016-11-10T19:13:39 <jonasschnelli> fullblock SPV gets uninteresting if you import keys or watchonlys.. because we set the timestamp to 0 = download the whole blockchain
3812016-11-10T19:13:40 <achow101> so hybrid keeps blocks for later and purespv doesn't? Is that the only diference?
3822016-11-10T19:13:42 <BlueMatt> if someone wants to work on it I wont complain, but...
3832016-11-10T19:13:57 <jonasschnelli> achow101: right.
3842016-11-10T19:14:00 <BlueMatt> jonasschnelli: argh, I'm lost in your terms here
3852016-11-10T19:14:05 <btcdrak> Yeah I also think purespv seems wrong for Bitcoin Core.
3862016-11-10T19:14:09 <btcdrak> but w/e
3872016-11-10T19:14:11 <sipa> i don't see why
3882016-11-10T19:14:17 <jonasschnelli> achow101: hybrid means, your doing IBD with all its downsides...
3892016-11-10T19:14:17 <NicolasDorier> well bitcoin core has a wallet part
3902016-11-10T19:14:18 <sipa> bitcoin core has a perfectly good wallet
3912016-11-10T19:14:19 <BlueMatt> purespv? fullblock spv? define?
3922016-11-10T19:14:31 <sipa> seems stupid to restrict it to full node use only
3932016-11-10T19:14:31 <CodeShark> We really should drop the term spv
3942016-11-10T19:14:33 <jtimon> achow101: it seems also hybridspv does background ibd while pure doesn't
3952016-11-10T19:14:41 <achow101> ok
3962016-11-10T19:14:43 <jonasschnelli> purespv = no IBD, hybridspv = SPV during IDB
3972016-11-10T19:15:00 <BlueMatt> jonasschnelli: wait, so what is fullblockspv?
3982016-11-10T19:15:08 <wumpus> both
3992016-11-10T19:15:12 <btcdrak> I thought the point of the hybride SPV is just to make the wallet usable immediately during IDB, and put an end to the poor user experience for new users.
4002016-11-10T19:15:12 <BlueMatt> huh
4012016-11-10T19:15:15 <jonasschnelli> Both. Yes.
4022016-11-10T19:15:20 <BlueMatt> what does both mean?
4032016-11-10T19:15:28 <jtimon> isn't hybridspv the same as "both"?
4042016-11-10T19:15:32 <sipa> BlueMatt: both hybrid and pure download full blocks
4052016-11-10T19:15:33 <wumpus> no bloom filters are used in either case, it is all retrieving full blocks
4062016-11-10T19:15:33 <jonasschnelli> Both modes (pure and hybrid) are full block SPV modes.
4072016-11-10T19:15:41 <achow101> fullblock spv is where the full block is downloaded and the transaction is pulled. watever happens to the block is depended on hybrid or pure
4082016-11-10T19:15:41 <wumpus> what is so hard to understand about that?
4092016-11-10T19:15:50 <wumpus> achow101: right
4102016-11-10T19:15:52 <jonasschnelli> let me write a short post on bitcoincore ML
4112016-11-10T19:15:52 <BlueMatt> wumpus/sipa: ahh, just referring to the way we download in either case
4122016-11-10T19:16:04 <BlueMatt> wumpus: the way it was referenced it sounded like a different mode
4132016-11-10T19:16:24 <jonasschnelli> From the enduser perspective, the modes are very different
4142016-11-10T19:16:34 <wumpus> the only difference is that in pure SPV mode, the blocks are thrown away and there is no IBD, in hybrid mode the blocks are backfilled and IBD happens
4152016-11-10T19:16:51 <BlueMatt> wumpus: yes, i was confused and thought there was a third mode
4162016-11-10T19:17:04 <jtimon> I see no problem with allowing purespv optionally, the default is going to be hybridspv, right?
4172016-11-10T19:17:10 <wumpus> ok
4182016-11-10T19:17:15 <wumpus> jtimon: yes
4192016-11-10T19:17:20 <ryanofsky> in the pr, i suggested calling hybrid spv "prioritized download"
4202016-11-10T19:17:23 <jonasschnelli> purespv can solve an interesting usecase where one likes to decouple the wallet from the node
4212016-11-10T19:17:32 <wumpus> jonasschnelli: exactly
4222016-11-10T19:17:33 <BlueMatt> jtimon: agreed, just saying that it seems a strange thing to focus work on...there are already good options for consumers on the purespv front
4232016-11-10T19:17:45 <BlueMatt> like, lots of good options
4242016-11-10T19:17:46 <wumpus> jonasschnelli: it's basically a stand-alone wallet
4252016-11-10T19:18:05 <jonasschnelli> BlueMatt: purespv would be the only fullblock SPV wallet in the wild
4262016-11-10T19:18:07 <jtimon> BlueMatt: I suspect the reasoning is that it will be relatively cheap to add that extra mode
4272016-11-10T19:18:12 <wumpus> BlueMatt: it is not *focusing* work on, it's just a by-effect of allowing it
4282016-11-10T19:18:29 <BlueMatt> wumpus: yup, I'm fine with it being a free feature, just making a note
4292016-11-10T19:18:39 <jtimon> maybe a pr with the hybrid default one and a later one adding the purespv mode would make sense
4302016-11-10T19:18:40 <achow101> purespv would be great for privacy
4312016-11-10T19:18:57 <jonasschnelli> not for bandwidth thought
4322016-11-10T19:19:03 <sipa> BlueMatt: fair enough, but if we want hybrid spv because of the terrible syncing experience, we have all the pieces in place for purespv as well
4332016-11-10T19:19:07 <CodeShark> I still don't quite get purespv - sorry, scrolling back and keeping up is hard
4342016-11-10T19:19:20 <wumpus> sigh.\
4352016-11-10T19:19:23 <jonasschnelli> purespv = plain full block SPV
4362016-11-10T19:19:25 <wumpus> any other topics?
4372016-11-10T19:19:32 <wumpus> we're starting to repeat ourselves and that is annoying
4382016-11-10T19:19:35 <gmaxwell> hah
4392016-11-10T19:19:36 <jonasschnelli> download block, no validation, extract relevavnt txs
4402016-11-10T19:19:47 <CodeShark> ok
4412016-11-10T19:19:48 <jonasschnelli> Okay. I'll continue and try to make small PRs (could be hard though). Thanks
4422016-11-10T19:20:03 <wumpus> please just read back if you want to know things that have been discussed before, we can't repeat every single thing specifically for everyone
4432016-11-10T19:20:25 <BlueMatt> topics?
4442016-11-10T19:20:27 <MarcoF____> topic suggestion: 0.14
4452016-11-10T19:20:33 <CodeShark> I'd rather discuss bloom filter commitments or clientside bloom filtering from trusted nodes
4462016-11-10T19:20:34 <wumpus> #topic multithread ProcessMessages
4472016-11-10T19:20:44 <MarcoF____> Personally I mostly care about multiwallet refactoring for 0.14
4482016-11-10T19:21:00 <MarcoF____> but there are plenty of other pulls open, so we might want to prioritize?
4492016-11-10T19:21:04 <wumpus> BlueMatt ^^
4502016-11-10T19:21:33 <BlueMatt> i mean i think we all want multiple message-processing threads
4512016-11-10T19:21:40 <jonasschnelli> +1
4522016-11-10T19:21:51 <gmaxwell> Sorry, I delayed matt by asking him about it offline. My only comment was "don't reorder messages from a single peer!" :P but apparently I'm behind the times.
4532016-11-10T19:21:53 <BlueMatt> and while it probably wont be so useful with cs_main at the head of like every message, I'd like to see plumbing for it sooner rather than later
4542016-11-10T19:21:58 <wumpus> well if cs_main usage is brought down enough, that will start to make sense
4552016-11-10T19:22:10 <BlueMatt> then people can remove cs_main one message at a time and it will be useful
4562016-11-10T19:22:32 <BlueMatt> I'd be happy to see, but dont know if we'll make, cs_main split-outs too much for 0.14 (thats next after main.cpp split)
4572016-11-10T19:22:48 <BlueMatt> but I'd like to see a few free wins like being able to respond to getblocktxn requests while a block is processing
4582016-11-10T19:23:05 <BlueMatt> anyone have any thoughts on blockers for this? things likely to break? etc?
4592016-11-10T19:23:05 <gmaxwell> okay, thanks for the concrete example of something pretty useful.
4602016-11-10T19:23:21 <cfields> BlueMatt: seems unlikely to have any real-world benefit without breaking out CNodeState? Maybe we should try to knock that out first?
4612016-11-10T19:23:28 <gmaxwell> I was struggling to come up with one, but that one is good.
4622016-11-10T19:23:44 <wumpus> being able to service multiple nodes at once would be very useful
4632016-11-10T19:24:01 <BlueMatt> cfields: why do you have to break out CNodeState? you'll never call ProcessMessages in two threads for a single peer at the same time
4642016-11-10T19:24:07 <BlueMatt> cfields: that would violate our serialization stuff
4652016-11-10T19:24:30 <gmaxwell> Yes, in particular being able to reply to getblocktxn multiple times in parallel should reduce block relaying delays.
4662016-11-10T19:25:02 <BlueMatt> gmaxwell: yes, i really want it for FIBRE-based relay network - respond to getblocktxn from a shared_ptr<const CBlock> block_currently_processing
4672016-11-10T19:25:06 <wumpus> I mean looking at it from the other side, there's no good reason for keeping doing message processing only in one thread
4682016-11-10T19:25:16 <morcos> BlueMatt: I think its going to be important to do a thorough review of synchronization issues first. Sometimes I feel like things just happen to not be a problem b/c they are only accessed from the single thread.
4692016-11-10T19:25:34 <BlueMatt> morcos: yea, thats my concern :(
4702016-11-10T19:25:41 <cfields> BlueMatt: well when each one is grabbing for cs_main, it only takes one long lock to drop us back to single-threaded
4712016-11-10T19:25:42 <morcos> We do an ok job of fixing these missing locks when we find them, but if we're goign to make multiple message processing threads, we need to make sure we've really got them all
4722016-11-10T19:25:44 <BlueMatt> morcos: any concrete things i should be looking for
4732016-11-10T19:25:59 <morcos> I think we should make a list of what needs to be protected by cs_main
4742016-11-10T19:26:10 <BlueMatt> cfields: yea, I mean I'm ok with switching to a multi-threaded model that does ~nothing and then slowly reducing the locks
4752016-11-10T19:26:15 <gmaxwell> So I think making process message concurrent may create greater exposure for data races around the nodestats. Right now the locking around stats is _widely_ incorrect, leading to undefined behavior. (I haven't been watching closely and cfields likely improved a lot of it recently, but I expect there are still problems)
4762016-11-10T19:26:36 *** gabridome has joined #bitcoin-core-dev
4772016-11-10T19:26:47 <gmaxwell> (this is responsive to matt's question about likely sources of problems)
4782016-11-10T19:27:18 <sipa> gmaxwell: for stats we can just change them all to atomics
4792016-11-10T19:27:19 <gmaxwell> I recommend that we run testing harnesses with valgrind DRD and try to get this change to be data race free at least according to DRD.
4802016-11-10T19:27:34 <BlueMatt> gmaxwell: good suggestion, indeed
4812016-11-10T19:27:47 <gmaxwell> sipa: I made that suggestion previously. I think we access them infrequently enough that it's not an awful idea.
4822016-11-10T19:28:14 <sipa> gmaxwell: you can also cache per peer, and flush to a locked global occasionally if it's a concern.
4832016-11-10T19:28:20 <sipa> stats are easy, no consistency requirements
4842016-11-10T19:28:45 <gmaxwell> last time I ran DRD I saw some races around stats but there wasn't a wall of errors.
4852016-11-10T19:28:54 <wumpus> and the per-node stats should not be an issue as we likely don't want to be processing two messages from the same peer at the same time?
4862016-11-10T19:28:56 <cfields> sipa: that sounds like a good model
4872016-11-10T19:29:11 <sipa> wumpus: indeed
4882016-11-10T19:29:14 <BlueMatt> wumpus: yes, that would reduce lots of issues
4892016-11-10T19:29:19 <gmaxwell> wumpus: I suspect we can get a message from peer A that causes use to iterate over all the other peers stats.
4902016-11-10T19:29:28 <sdaftuar> BlueMatt: have you given much thought to how the threading logic would work?
4912016-11-10T19:29:41 <sdaftuar> i mean, how you'd decide how to split work across threads
4922016-11-10T19:29:46 <gmaxwell> (just an example why single peer at a time doesn't automatically mean thread safty of per node stats)
4932016-11-10T19:29:53 <wumpus> gmaxwell: that sounds scary
4942016-11-10T19:30:35 <gmaxwell> (maybe we don't actually do that anywhere, at least in response to a message... a connection, for example, causes that--)
4952016-11-10T19:30:35 <BlueMatt> sdaftuar: hum? do we need anything more complicated than just a thread pool each looking for peers that have available work and arent being worked on that just wait on a cv when there is nothing to do
4962016-11-10T19:30:50 *** MarcoF____ has quit IRC
4972016-11-10T19:30:51 <wumpus> gmaxwell: no, but keeping to rules like that will make it easier manageable
4982016-11-10T19:31:04 *** Sosumi has joined #bitcoin-core-dev
4992016-11-10T19:31:17 <gmaxwell> Another point to keep in mind is thread prolifervation and address space usage on 32-bit hosts. Not a reason to not do this, but just a cost.
5002016-11-10T19:31:19 <sdaftuar> well, my thought was that since most messages are tx's, we might find ourselves tying up all our threads but unable to continue, as they're all contending on cs_main
5012016-11-10T19:31:34 <sdaftuar> so you wouldn't necessarily be able to respond to a getblocktxn while processing a block
5022016-11-10T19:31:42 <sdaftuar> you could do somethign smarter though...
5032016-11-10T19:31:55 <wumpus> gmaxwell: it would still be possible to run with only one thread I suppose
5042016-11-10T19:31:59 <wumpus> gmaxwell: I'm not terribly worried
5052016-11-10T19:32:01 <BlueMatt> sdaftuar: yes, i was thinking for v1 we ignore that issue
5062016-11-10T19:32:08 <sdaftuar> ok :)
5072016-11-10T19:32:09 <gmaxwell> wumpus: I'm not either.
5082016-11-10T19:32:21 <BlueMatt> sdaftuar: indeed, eventually we could have some "oh, these messages take cs_main" list to make it smart
5092016-11-10T19:32:37 <gmaxwell> Besides, this is a better use of threads (actual concurrency) than some others we have had.
5102016-11-10T19:32:47 <BlueMatt> gmaxwell: very true :(
5112016-11-10T19:32:58 <wumpus> gmaxwell: we certainly shouldn't be restricting work towards better performance because there happen to be a few nodes on small systems, those can be handled specifically
5122016-11-10T19:33:10 <gmaxwell> (e.g. connect has a thread, wallet flush thread, etc.)
5132016-11-10T19:33:17 <BlueMatt> we can make up the difference by moving wallet flush into cscheduler thread :)
5142016-11-10T19:33:24 <gmaxwell> wumpus: agreed, I am just exposing areas of consideration. I support the work.
5152016-11-10T19:33:26 <wumpus> (and I run most of my nodes on 32-bit ARM so it's not like I don't care)
5162016-11-10T19:33:30 <BlueMatt> well, and like three net threads to cscheduler
5172016-11-10T19:33:52 *** Guyver2 has quit IRC
5182016-11-10T19:33:59 <wumpus> yes, we can do thread-stack accounting like that, but let's not get lost in details
5192016-11-10T19:34:04 <morcos> and while we're at it we can increase script checking threads :)
5202016-11-10T19:34:10 <petertodd> wumpus: oh, on rpi's and the like?
5212016-11-10T19:34:22 <wumpus> petertodd: cubox-i's are my favourite
5222016-11-10T19:34:28 <petertodd> wumpus: interesting, thanks!
5232016-11-10T19:35:23 <wumpus> next topic, I suppose
5242016-11-10T19:35:30 <gmaxwell> (there are other possibilties like decresing the stack size on some threads that don't do much, firefox uses 128k or 256k (I forget) stacks for media processing threads. So I really don't at all think it's a blocker, just something to keep in mind.)
5252016-11-10T19:36:14 <wumpus> gmaxwell: indeed you can do that through an environment variable IIRC, I think I even list it in my "reducing bitcoind memory usage" document ,if not I should
5262016-11-10T19:36:19 *** Guyver2 has joined #bitcoin-core-dev
5272016-11-10T19:36:29 <wumpus> #topic 0.14
5282016-11-10T19:36:49 <wumpus> @MarcoFalke
5292016-11-10T19:37:07 <jonasschnelli> multiwallet support would be nice
5302016-11-10T19:37:16 <wumpus> oh he left?
5312016-11-10T19:37:27 <gmaxwell> wumpus: yes, well you can control it for all threads in an env variable --- though that can result in security problems. :( But there are some threads where their stack usage is precisely decidable easily. (e.g. the connect thread).
5322016-11-10T19:37:27 <jtimon> yep, it seems so
5332016-11-10T19:37:43 *** MarcoFalk_ has joined #bitcoin-core-dev
5342016-11-10T19:37:52 <jonasschnelli> [20:36:29] <@wumpus> #topic 0.14
5352016-11-10T19:37:52 <jonasschnelli> [20:36:49] <@wumpus> @MarcoFalke
5362016-11-10T19:38:00 <BlueMatt> MarcoFalk_:
5372016-11-10T19:38:08 <bitcoin-git> [bitcoin] morcos opened pull request #9123: Remove extraneous LogPrint from fee estimation (master...eliminateFeeWarning) https://github.com/bitcoin/bitcoin/pull/9123
5382016-11-10T19:38:19 <MarcoFalk_> Jup, I'd like to hear what is a priority to get in to 0.14
5392016-11-10T19:38:21 <jtimon> multiwallet refactoring for 0.14
5402016-11-10T19:38:26 <MarcoFalk_> ^
5412016-11-10T19:38:26 <wumpus> gmaxwell: I'm not sure about security problems, though yes it can cause crashes if you set it that low that the stack underflows
5422016-11-10T19:38:28 * jonasschnelli gives MarcoFalk a IRC bouncer for birthday
5432016-11-10T19:38:42 <BlueMatt> main.cpp split, but thats guaranteed at this point, pretty much
5442016-11-10T19:38:57 <sdaftuar> i think the validation speedups from jeremyrubin would be great for 0.14
5452016-11-10T19:39:00 <MarcoFalk_> Ok, what is the progess on the net refactor. Is it almost done?
5462016-11-10T19:39:01 <wumpus> gmaxwell: then again the thread stacks are extremely small on some platforms
5472016-11-10T19:39:11 <wumpus> gmaxwell: linux assigns quite a lot by default
5482016-11-10T19:39:12 <BlueMatt> sdaftuar: yea, at least the cucku cache thing
5492016-11-10T19:39:16 <jonasschnelli> I think there is not much left for the multi-wallet support. Though, not sure if we can make this happen for 0.14
5502016-11-10T19:39:30 <jonasschnelli> Also.. we should add a private-key free mode.
5512016-11-10T19:39:43 <jonasschnelli> A safe-mode
5522016-11-10T19:39:44 <gmaxwell> sdaftuar: I agree, all of them.
5532016-11-10T19:39:48 <wumpus> if we all agree to start reviewing wallet changes, I'm sure we can get multi wallet support in 0.14
5542016-11-10T19:39:52 <jtimon> I would like to move on libconsensus, but so far nothing seems to be happening on that front
5552016-11-10T19:40:00 <MarcoFalk_> Some of the wallet changes need rebase
5562016-11-10T19:40:12 <jonasschnelli> wumpus: Nice! I can review some stuff next week.
5572016-11-10T19:40:15 <gmaxwell> In terms of wallet features I think some kind of multiwallet support would have the greatest ratio of benefit to cost+risk.
5582016-11-10T19:40:23 <gmaxwell> It's just software eng.
5592016-11-10T19:40:23 <jonasschnelli> gmaxwell: ack!
5602016-11-10T19:41:01 <jonasschnelli> Also, with the current fundrawtransaction options, you can perfectly run watch-only wallets
5612016-11-10T19:41:56 <cfields> I'm aiming to get net.h/cpp split in half in the next ~week also.
5622016-11-10T19:41:58 <jonasschnelli> So.. theres a project: https://github.com/bitcoin/bitcoin/projects/2
5632016-11-10T19:42:02 <jonasschnelli> for MW support
5642016-11-10T19:42:03 <cfields> (for the 0.14 list)
5652016-11-10T19:42:04 <gmaxwell> So I think multiwallet should be a greater priority. Also while 0.14 has a lot of really great infrastructural changes, I think it has relatively few ordinary user visible improvements.
5662016-11-10T19:42:26 <paveljanik> Network on/off in GUI please :-)
5672016-11-10T19:42:37 <jonasschnelli> paveljanik: Yes. I'll make that happen soon.
5682016-11-10T19:42:49 <wumpus> because most of the focus has been on 0.13.1, 0.14 will be somewhat of a smaller release, I don't think that's a problem
5692016-11-10T19:43:05 <gmaxwell> There are a number of other things I'd like to work on, like more bandwidth usage controls. Improvement to header fetching logic... but I think it's not useful to speak to work that hasn't even started.
5702016-11-10T19:43:13 *** gabridome has quit IRC
5712016-11-10T19:43:19 <jonasschnelli> I was hoping the mempool stats can be in 0.14... but not sure if there are enough reviews
5722016-11-10T19:43:21 <BlueMatt> bumpfee
5732016-11-10T19:43:22 <MarcoFalk_> I was pinging luke-jr on #8776, but haven't heard anything about luke-jr recently
5742016-11-10T19:43:23 <gribble> https://github.com/bitcoin/bitcoin/issues/8776 | Wallet refactoring leading up to multiwallet by luke-jr · Pull Request #8776 · bitcoin/bitcoin · GitHub
5752016-11-10T19:43:26 <MarcoFalk_> Hope hes doing fine
5762016-11-10T19:43:54 <paveljanik> jonasschnelli, mempool stats: count with me for review!
5772016-11-10T19:44:12 <jonasschnelli> BlueMatt: yeah! Bumpfee should get some review. Guys! https://github.com/bitcoin/bitcoin/pull/8456
5782016-11-10T19:44:26 <paveljanik> it would be nice to have a estimatefee 2, 3, 4 graph also...
5792016-11-10T19:44:29 <wumpus> cfields: weren't you working on bandwidth throttling w/ the P2P libevent switch?
5802016-11-10T19:44:30 <MarcoFalk_> paveljanik: I think the review-intense part is #8501
5812016-11-10T19:44:32 <gribble> https://github.com/bitcoin/bitcoin/issues/8501 | Add mempool statistics collector by jonasschnelli · Pull Request #8501 · bitcoin/bitcoin · GitHub
5822016-11-10T19:44:36 <gmaxwell> We'll see what shows up, in any case.
5832016-11-10T19:44:51 <wumpus> yeah there's not much point in listing all major open pulls here now
5842016-11-10T19:44:54 <MarcoFalk_> paveljanik: The gui change is rel. simple
5852016-11-10T19:45:06 <jonasschnelli> Yes. #8501 is groundwork for mempool stats. Has no review so far
5862016-11-10T19:45:08 <gribble> https://github.com/bitcoin/bitcoin/issues/8501 | Add mempool statistics collector by jonasschnelli · Pull Request #8501 · bitcoin/bitcoin · GitHub
5872016-11-10T19:45:11 <wumpus> we know about them, what they need is more review and testing :)
5882016-11-10T19:45:25 <MarcoFalk_> (and rebase) *ducks*
5892016-11-10T19:45:26 <morcos> I'm going to do at least some minor (and needed) improvements to fee estimation. Is there any interest on estimates for longer than 25 blocks? Or should I continue to punt on that?
5902016-11-10T19:45:38 <cfields> wumpus: yes, i don't think that's likely for 0.14 though. The libevent changes are going to be quite different from what I originally intended, I think
5912016-11-10T19:45:41 <gmaxwell> WRT review, I believe that multiwallet review is fundimentally easier than bumpfee for example, because in bumpfee I need to reason about a complex state machine, the effect on the network, corner cases with double spends, etc. And multiwallet is 98% "does this crash or not". :)
5922016-11-10T19:45:55 <wumpus> cfields: it's taking quite longer than we expected :)
5932016-11-10T19:46:13 <wumpus> gmaxwell: so start reviewing multiwallet pulls then :D
5942016-11-10T19:46:18 <BlueMatt> wumpus: lol, doesnt everything
5952016-11-10T19:46:21 <wumpus> (or did you already?)
5962016-11-10T19:46:30 <gmaxwell> morcos: mempool backlog levels are not great enough where estimates >25 blocks matter that much I think, currently.
5972016-11-10T19:46:35 <jtimon> btw maybe we can talk about the "custom mode" (separately from the blocksigning mode which I think it's unlikely to get to master with the union and the global, but we can just rebase every release)
5982016-11-10T19:46:39 <cfields> wumpus: yes, sorry for that. I vastly underestimated the refactor impact.
5992016-11-10T19:46:59 <wumpus> BlueMatt: yes, I'm kind of overwhelmed already, so I don't care that much some things are going slower
6002016-11-10T19:47:11 <morcos> gmaxwell: But its often possible to pay a much lower fee if you are willing to wait 100 blocks or whatever... So in the interest of keeping fees down.., but I just don't know if thats much of a use case.
6012016-11-10T19:47:39 <gmaxwell> morcos: fun graph, fee availble at the time of a block (red) vs immediately after (green): https://people.xiph.org/~greg/temp/fee_avail.png thats a week of data, also a different reason during the high fee floods a couple weeks ago: https://people.xiph.org/~greg/temp/fee_avail2.png
6022016-11-10T19:48:04 *** gabridome has joined #bitcoin-core-dev
6032016-11-10T19:48:09 <gmaxwell> morcos: interesting, if it would make a big difference I think it would be interesting. I wasn't thinking it would make much of a difference.
6042016-11-10T19:48:27 <wumpus> cfields: my worry is that some things are blocked on it, e.g. gmaxwell talks about wanting to do things with bandwidth management, those things would all be easier if we had switched to libevent, instead of trying to cram it into the current networking backend
6052016-11-10T19:48:29 <gmaxwell> morcos: I still wouldn't priortize it over general improvements.
6062016-11-10T19:48:50 *** gabridome has quit IRC
6072016-11-10T19:49:06 <morcos> gmaxwell: tremendous difference. i'm pretty sure you could always pay about 2 sat/byte if you are wiling to wait a couple days.. limiting factor might be the 3 day eviction.
6082016-11-10T19:49:13 <gmaxwell> Yes, I've held off on doing too much more with bandwidth management due to that. There are somethings we can do, for example delaying inv relays when bandwidth starved.
6092016-11-10T19:49:36 <cfields> wumpus: ACK. I'll pick up the pace.
6102016-11-10T19:49:49 <gmaxwell> morcos: my publically reachable nodes don't really expirence 3 day eviction, -- some clown or another inevitably connects and restores the evicted stuff. :P
6112016-11-10T19:50:09 <sipa> gmaxwell, cfields: it sounds to me like the bandwidth management you're talking about is very different and probably won't have much code conflicts
6122016-11-10T19:50:36 <sipa> as gmaxwell is talking about application level decisions that afaik don't even affect anything at the network layer
6132016-11-10T19:50:43 *** jcorgan has joined #bitcoin-core-dev
6142016-11-10T19:50:53 <gmaxwell> Well I specifically just mentioned the part of the management scope that wouldn't have conflicts. (and I think should probably be done first: to quickly offer something but then relay it out slowly is somewhat peer abusive. :P )
6152016-11-10T19:51:41 <cfields> sipa: indeed
6162016-11-10T19:53:44 <wumpus> other topics?
6172016-11-10T19:54:19 <gmaxwell> we should say hello to all the americans that missed the timezone change.
6182016-11-10T19:54:25 <gmaxwell> and are just arriving now. :P
6192016-11-10T19:54:33 <jcorgan> heh
6202016-11-10T19:54:48 <petertodd> gmaxwell: canadians too :)
6212016-11-10T19:54:56 <gmaxwell> Hi guys.
6222016-11-10T19:55:00 <achow101> hi
6232016-11-10T19:55:00 <gmaxwell> Welcome to the end of the meeting.
6242016-11-10T19:55:46 <wumpus> yes, welcome
6252016-11-10T19:55:50 <sipa> kthxbye
6262016-11-10T19:55:51 <wumpus> #endmeeting
6272016-11-10T19:55:51 <lightningbot> Meeting ended Thu Nov 10 19:55:51 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
6282016-11-10T19:55:51 <lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-11-10-19.03.html
6292016-11-10T19:55:51 <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-11-10-19.03.txt
6302016-11-10T19:55:51 <lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-11-10-19.03.log.html
6312016-11-10T19:56:04 <sipa> lunch?
6322016-11-10T19:56:14 <btcdrak> petertodd: you mean snow mexicans right?
6332016-11-10T19:56:34 <petertodd> btcdrak: nah, we're the ones building a wall
6342016-11-10T19:56:40 <jcorgan> no, they are 51st state-ians
6352016-11-10T19:56:54 <petertodd> sipa: sure, there's a nice diner near me
6362016-11-10T19:57:41 <paveljanik> MarcoFalk_, 8501 is part of 8550...
6372016-11-10T19:57:57 <petertodd> jcorgan: in a few years you're going to be saying canadafornia's
6382016-11-10T19:58:35 *** Guyver2 has quit IRC
6392016-11-10T19:58:36 <jonasschnelli> petertodd: yes. 8501 can work without 8550 and has the most bitcoin-core wide impact.
6402016-11-10T19:58:37 <jcorgan> true dat
6412016-11-10T19:58:46 <jonasschnelli> Ahm,.. meant paveljanik
6422016-11-10T19:58:52 <jcorgan> actually, i like how that sounds
6432016-11-10T19:58:54 *** Chris_Stewart_5 has quit IRC
6442016-11-10T19:59:28 <instagibbs> lol timechange :)
6452016-11-10T19:59:34 <BlueMatt> #8501, #8550
6462016-11-10T19:59:36 <gribble> https://github.com/bitcoin/bitcoin/issues/8501 | Add mempool statistics collector by jonasschnelli · Pull Request #8501 · bitcoin/bitcoin · GitHub
6472016-11-10T19:59:37 <gribble> https://github.com/bitcoin/bitcoin/issues/8550 | [Qt] Add interactive mempool graph by jonasschnelli · Pull Request #8550 · bitcoin/bitcoin · GitHub
6482016-11-10T20:00:30 <instagibbs> " we should say hello to all the americans that missed the timezone change." hmmm
6492016-11-10T20:01:04 <paveljanik> HELLo ;-)
6502016-11-10T20:01:28 *** gabridome has joined #bitcoin-core-dev
6512016-11-10T20:02:06 <instagibbs> I usually miss it anyways since it's not on my calendar and thursdays are my most productive day :P
6522016-11-10T20:02:20 <instagibbs> will add to cal
6532016-11-10T20:02:48 *** Chris_Stewart_5 has joined #bitcoin-core-dev
6542016-11-10T20:02:49 * jcorgan starts putting all his calendar events in UTC
6552016-11-10T20:02:56 <BlueMatt> jcorgan: you want iceland time
6562016-11-10T20:03:06 <MarcoFalk_> Maybe trump drops DST
6572016-11-10T20:03:22 <instagibbs> at blockstream we have this issue a lot so everything is on iceland time
6582016-11-10T20:03:37 <achow101> why iceland?
6592016-11-10T20:03:41 <jonasschnelli> MarcoFalk_: lol
6602016-11-10T20:03:57 <paveljanik> there will be TST soon...
6612016-11-10T20:04:01 <Chris_Stewart_5> oh damn, meeting moved up by an hour now for Americans?
6622016-11-10T20:04:11 <achow101> Chris_Stewart_5: daylight savings...
6632016-11-10T20:04:13 <sipa> achow101: iceland is 100% of the year in UTC
6642016-11-10T20:04:14 <BlueMatt> achow101: iceland == utc
6652016-11-10T20:04:17 <BlueMatt> no dst
6662016-11-10T20:04:23 <achow101> oh. til
6672016-11-10T20:04:33 <sipa> (which makes no sense, geographically they should be at +1 all the time)
6682016-11-10T20:04:44 <jcorgan> wish they'd just drop time zones altogether
6692016-11-10T20:04:55 <sipa> jcorgan: one per continent may make sense
6702016-11-10T20:05:28 <jcorgan> hmm, maybe we can change our time zone when we become New Canadafornia
6712016-11-10T20:05:52 *** harrymm has quit IRC
6722016-11-10T20:06:02 <instagibbs> ok, set to 7pm iceland time, should be good now
6732016-11-10T20:06:26 <jtimon> instagibbs: not everything is on iceland time :(
6742016-11-10T20:06:37 <gmaxwell> achow101: google software doesn't have a 'UTC' option (because braindamage, I guess), fortuately iceland is equivilent.
6752016-11-10T20:07:41 *** sanada has quit IRC
6762016-11-10T20:08:17 *** droark has joined #bitcoin-core-dev
6772016-11-10T20:08:36 <jcorgan> thanks for the iceland tip
6782016-11-10T20:08:46 <achow101> ohh.
6792016-11-10T20:08:50 <sdaftuar> BlueMatt: let's get #9058 merged, these spurious test failures are annoying...
6802016-11-10T20:08:52 <gribble> https://github.com/bitcoin/bitcoin/issues/9058 | Fixes for p2p-compactblocks.py test timeouts on travis (#8842) by ryanofsky · Pull Request #9058 · bitcoin/bitcoin · GitHub
6812016-11-10T20:19:01 *** Cory has quit IRC
6822016-11-10T20:25:19 *** Cory has joined #bitcoin-core-dev
6832016-11-10T20:28:12 *** harrymm has joined #bitcoin-core-dev
6842016-11-10T20:38:54 <wumpus> what's holding #9058 back?
6852016-11-10T20:38:56 <gribble> https://github.com/bitcoin/bitcoin/issues/9058 | Fixes for p2p-compactblocks.py test timeouts on travis (#8842) by ryanofsky · Pull Request #9058 · bitcoin/bitcoin · GitHub
6862016-11-10T20:39:52 <bitcoin-git> [bitcoin] paveljanik opened pull request #9124: Use better name for local variable to prevent -Wshadow compiler warning (master...20161110_Wshadow_benchcheckblock) https://github.com/bitcoin/bitcoin/pull/9124
6872016-11-10T20:40:15 *** gabridome has quit IRC
6882016-11-10T20:42:07 *** gabridome has joined #bitcoin-core-dev
6892016-11-10T20:42:13 <sdaftuar> wumpus: i think it's good to go, but thought bluematt might want to review since it is a (very small) change to BIP 152
6902016-11-10T20:46:29 *** gabridome has quit IRC
6912016-11-10T20:50:10 *** Ylbam has quit IRC
6922016-11-10T20:54:28 *** MarcoFalk_ has quit IRC
6932016-11-10T20:59:37 *** go1111111 has quit IRC
6942016-11-10T21:01:18 <bsm1175321> What is the plan for the MSG_FILTERED_WITNESS_BLOCK message? (spv + segwit...)
6952016-11-10T21:01:32 <bsm1175321> or block type rather in inv/getdata
6962016-11-10T21:04:16 *** cryptapus has quit IRC
6972016-11-10T21:06:24 <gmaxwell> bsm1175321: for most SPV wallets the witness is useless and not desirable.
6982016-11-10T21:06:38 <gmaxwell> Without the pubkey they can't verify it.
6992016-11-10T21:06:52 *** gabridome has joined #bitcoin-core-dev
7002016-11-10T21:07:16 <gmaxwell> bsm1175321: I think many people would be glad to work on a spec for that with clear applications.
7012016-11-10T21:07:27 <gmaxwell> But I expect most (though not all) SPV segwit wallets to not use it.
7022016-11-10T21:07:37 <bsm1175321> Which pubkey? The witness contains the pubkeys no?
7032016-11-10T21:08:09 <gmaxwell> The scritpubkey.
7042016-11-10T21:08:20 <bsm1175321> I was surprised to find the bcoin library setting this flag on testnet...spent a day tracking down why 0.13.1 wasn't responding to my getdata...
7052016-11-10T21:08:55 <gmaxwell> Setting what flag?
7062016-11-10T21:09:06 <bsm1175321> MSG_FILTERED_WITNESS_BLOCK
7072016-11-10T21:09:17 <bsm1175321> on getdata queries
7082016-11-10T21:09:43 <bsm1175321> Clearly a bug for bcoin...anyhoo
7092016-11-10T21:10:17 <bsm1175321> What do you have in mind for "a spec with clear applications"?
7102016-11-10T21:15:25 *** Sosumi has quit IRC
7112016-11-10T21:15:36 *** gabridome has quit IRC
7122016-11-10T21:16:37 *** tulip has joined #bitcoin-core-dev
7132016-11-10T21:18:14 <tulip> bsm1175321: similarly BIP37 hashes items in the scriptpubkey individually and adds them to the filter.
7142016-11-10T21:23:42 *** gabridome has joined #bitcoin-core-dev
7152016-11-10T21:23:43 <tulip> MSG_FILTERED_WITNESS_BLOCK would regrettably have to do the thing where the witness of a matching transaction is serialised and inserted back into the filter as well.
7162016-11-10T21:24:44 <bsm1175321> Woah I had no idea the BIP37 filters are applied to *any* data element in your script.
7172016-11-10T21:27:33 <tulip> the transaction, the TXID, the serialised inputs, every data element in any script.
7182016-11-10T21:31:28 <Chris_Stewart_5> tulip: I don't think the entire txs is checked, and I think it is only the outpoints in the inputs?
7192016-11-10T21:32:47 <Chris_Stewart_5> + date elements in the script like you said
7202016-11-10T21:33:23 <tulip> Chris_Stewart_5: you're right that it doesn't match the entire transaction, just the hash. I think the other parts of my comment are correct looking at the specification.
7212016-11-10T21:33:55 *** juscamarena has joined #bitcoin-core-dev
7222016-11-10T21:43:22 <gmaxwell> yea, that 'feature' really contributes to the lack of privacy and utiliy of BIP37.
7232016-11-10T21:43:24 *** FreeUser has joined #bitcoin-core-dev
7242016-11-10T21:43:35 <FreeUser> Hello.
7252016-11-10T21:43:57 <FreeUser> The alert system will be removed, yes?
7262016-11-10T21:44:23 <FreeUser> But how users will receive info about critical bugs?
7272016-11-10T21:45:03 <gmaxwell> it was removed a long time ago, it is only now being deactivated for older software.
7282016-11-10T21:45:23 <gmaxwell> FreeUser: same way they recieve news about anything.
7292016-11-10T21:47:00 <FreeUser> How to disable alerts in old Bitcoin Core versions?
7302016-11-10T21:48:20 <gmaxwell> FreeUser: there is a final alert which will be sent which causes a final alert to be displayed which cannot be overridden by any other alert.
7312016-11-10T21:49:25 <FreeUser> Are alerts stored in blockchain?
7322016-11-10T21:49:44 <gmaxwell> No.
7332016-11-10T21:50:49 <FreeUser> But how users received these alerts?
7342016-11-10T21:50:55 <gmaxwell> bsm1175321: there is only one application that I'm currently aware of for a SPV wallet to see witness data: so a multisignature wallet can track which of the signers signed their own coins.
7352016-11-10T21:51:17 <gmaxwell> FreeUser: they were sent peer to peer between indivigual nodes.
7362016-11-10T21:53:15 <FreeUser> What will happens with Litecoin/other altcoin?
7372016-11-10T21:53:19 <FreeUser> *altcoins
7382016-11-10T21:53:55 <gmaxwell> nothing, they're totally seperate systems
7392016-11-10T21:54:17 <FreeUser> Why Litecoin not updating Litecoin Core?
7402016-11-10T21:55:02 <bsm1175321> gmaxwell I'm interested in SPV clients getting witness data because I'm using the spent pubkeys for out-of-band auth and communication.
7412016-11-10T21:55:06 <FreeUser> Is it hard to fork newer version and change it like old?
7422016-11-10T21:55:12 <sipa> FreeUser: ask them
7432016-11-10T21:55:25 <sipa> litecoin is offtopic here
7442016-11-10T21:56:29 <gmaxwell> bsm1175321: the public keys are in the paying transactions txouts.
7452016-11-10T21:56:39 <FreeUser> What is difference between 0.13.1 and 0.13.99?
7462016-11-10T21:56:46 <FreeUser> And why 99?
7472016-11-10T21:57:00 <sipa> FreeUser: 0.13.99 is the version number used in the 0.14 branch before 0.14 is released
7482016-11-10T21:57:26 <sipa> FreeUser: at this point we have 0.14 branch with more significant changes, and a stable 0.13 branch that only receives bug fixes
7492016-11-10T21:57:56 <FreeUser> Is Bitcoin Knots an official wallet?
7502016-11-10T21:58:02 <sipa> there is no 'official'
7512016-11-10T21:58:09 *** MarcoFalke has joined #bitcoin-core-dev
7522016-11-10T21:58:22 <sipa> Knots is a fork of Bitcoin Core, maintained by different people
7532016-11-10T21:58:27 <FreeUser> Official means from Bitcoin Core devs
7542016-11-10T21:58:41 <sipa> i'm not involved in Knots
7552016-11-10T21:58:42 <gmaxwell> whats a Bitcoin Core devs? there is no such thing as membership.
7562016-11-10T21:58:50 <sipa> and i develop for Core.
7572016-11-10T21:58:53 <bsm1175321> gmaxwell: correct, I'm interested in obtaining that paying transaction, with witness data, by SPV clients.
7582016-11-10T21:58:56 <gmaxwell> Perhaps you're a Bitcoin Core dev, have you submitted a patch? :)
7592016-11-10T21:58:57 <Chris_Stewart_5> Damn! I paid that guy all my doge coin for a bitcoin core dev sticker..
7602016-11-10T21:59:06 <FreeUser> I can see "Bitcoin Core Developers" while loading Bitcoin Core.
7612016-11-10T21:59:22 <sipa> FreeUser: you too can contribute to either, none, or both
7622016-11-10T21:59:33 <gmaxwell> FreeUser: yes, that just means all the people who have contributed.
7632016-11-10T21:59:55 <MarcoFalke> FreeUser: The "Bitcoin Core Developers" is just the short version of all the credits (which you can find in the release notes)
7642016-11-10T22:00:58 <MarcoFalke> E.g. https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.1.md#credits
7652016-11-10T22:01:56 <FreeUser> Can I find Satoshi Nakamoto in release notes?
7662016-11-10T22:02:45 <sipa> he has not contributed to any release since 0.3.19
7672016-11-10T22:02:53 <sipa> (at least not under that name)
7682016-11-10T22:04:03 <Lauda> I'm somewhat inclinded to believe that sipa is satoshi
7692016-11-10T22:04:08 <Lauda> they both start with an 's'.
7702016-11-10T22:04:11 <FreeUser> Can Satoshi Nakamoto be killed?
7712016-11-10T22:04:26 <FreeUser> So
7722016-11-10T22:04:34 <sipa> wth are you talking about?
7732016-11-10T22:04:35 <Lauda> Oops wrong chat, thought this was #bitcoin. Sorry.
7742016-11-10T22:04:41 <sipa> please take this elsewhere
7752016-11-10T22:05:00 <FreeUser> Some cracker cracked his P2P Foundation account.
7762016-11-10T22:05:08 <sipa> off topic
7772016-11-10T22:05:22 <sipa> you're free to ask questions about Bitcoin and Bitcoin Core's development model here
7782016-11-10T22:05:45 *** FreeUser has quit IRC
7792016-11-10T22:05:56 * gmaxwell waits for the rbtc headline
7802016-11-10T22:06:09 *** juscamarena has quit IRC
7812016-11-10T22:07:48 *** Satoshin has joined #bitcoin-core-dev
7822016-11-10T22:07:51 <Satoshin> Hello.
7832016-11-10T22:08:35 *** sipa has quit IRC
7842016-11-10T22:08:35 *** sipa has joined #bitcoin-core-dev
7852016-11-10T22:08:38 *** ChanServ sets mode: +o sipa
7862016-11-10T22:09:04 <Satoshin> .help
7872016-11-10T22:09:07 *** Satoshin has left #bitcoin-core-dev
7882016-11-10T22:09:36 <gmaxwell> I recommend setting a wildcard on that.
7892016-11-10T22:11:33 <BlueMatt> sdaftuar: sounds good, will put it on the review-stack :)
7902016-11-10T22:19:44 <BlueMatt> sdaftuar: up next after cuckoo
7912016-11-10T22:27:28 *** MarcoFalke has left #bitcoin-core-dev
7922016-11-10T22:27:31 *** Guyver2 has joined #bitcoin-core-dev
7932016-11-10T22:52:24 <cfields> hmm, is it intended that feeler connections are allowed to send more than just version messages out?
7942016-11-10T22:56:40 *** gabridome has quit IRC
7952016-11-10T22:57:59 *** btcdrak has quit IRC
7962016-11-10T23:03:27 *** gabridome has joined #bitcoin-core-dev
7972016-11-10T23:15:57 <gmaxwell> cfields: not really, though I noticed that to and concluded that we'll probably like to expand their behavior over time.
7982016-11-10T23:16:43 <cfields> gmaxwell: ok. I suppose at worst, it just makes it look more like a normal node
7992016-11-10T23:16:48 <gmaxwell> E.g.e treat them more like like regular connections, and potentially disconnect another connection that hasn't been so useful.
8002016-11-10T23:18:17 <cfields> gmaxwell: right. I kinda assumed they did some of that already. For ex, noting nodes with unexpected services in addrman.
8012016-11-10T23:19:37 <cfields> Seems a shame to learn that, then mark it as up/fresh anyway
8022016-11-10T23:19:46 <gmaxwell> they do update the services before disconnecting. (though we avoid trying to feeler to things without relevant services right now, which diminishes the usefulness somewhat. :) )
8032016-11-10T23:20:42 <cfields> ok, roger. As long as it's not unintentional behavior :)
8042016-11-10T23:23:44 *** gabridome has quit IRC
8052016-11-10T23:33:04 *** justanotheruser has joined #bitcoin-core-dev
8062016-11-10T23:37:00 *** aalex has quit IRC
8072016-11-10T23:42:12 *** cdecker has quit IRC
8082016-11-10T23:44:21 *** gabridome has joined #bitcoin-core-dev
8092016-11-10T23:45:43 *** gabridome has quit IRC
8102016-11-10T23:49:33 *** Ylbam has joined #bitcoin-core-dev
8112016-11-10T23:55:01 *** jcorgan has quit IRC