12018-12-26T00:00:30  *** fabianfabian has quit IRC
  22018-12-26T00:06:14  *** reardencode has joined #bitcoin-core-dev
  32018-12-26T00:08:29  *** jarthur has joined #bitcoin-core-dev
  42018-12-26T00:09:26  *** Zenton has quit IRC
  52018-12-26T00:12:17  *** CodeBlue1776 has quit IRC
  62018-12-26T00:12:35  *** CodeBlue1776 has joined #bitcoin-core-dev
  72018-12-26T00:13:07  <sipa> jimmysong: "block store" ?
  82018-12-26T00:18:11  <jimmysong> storing that data in the db for a full node so it can be given to a peer that requests it
  92018-12-26T00:18:14  <sipa> a bloomfilter (or gcd filter) can't store any data, it's just a probabilistic structure to let you query for set elements
 102018-12-26T00:18:29  <sipa> a full node has no need for that data
 112018-12-26T00:19:03  <jimmysong> a full node doesn't but it'd be useful for a light client wanting to verify a cfilter hash
 122018-12-26T00:19:07  <sipa> it is store on disk though, in the undo block data in bitcoin core, for as long as the block itself is stored
 132018-12-26T00:19:21  <sipa> ah, for verification it should just be committed to
 142018-12-26T00:20:09  <jimmysong> a coinbase commitment is better, but is that something that's got a proposal?
 152018-12-26T00:21:24  <sipa> it's pretty trivial to do, but mucb easier to get agreement on once bip157 itself is deployed and used
 162018-12-26T00:21:35  <sipa> i expect
 172018-12-26T00:21:38  <jimmysong> makes sense
 182018-12-26T00:22:00  <jimmysong> i'm asking because i'm trying to write a wallet that uses neutrino
 192018-12-26T00:22:10  <jimmysong> the annoying part is the verification
 202018-12-26T00:22:51  <jimmysong> but getting the filter, checking against my script pubkeys, etc is great
 212018-12-26T00:23:04  <jimmysong> much better for privacy
 222018-12-26T00:23:48  <sipa> the node giving you prevour scripts along with the filter doesn't let you verify anything
 232018-12-26T00:23:57  <sipa> they can forge anything
 242018-12-26T00:23:58  *** fabianfabian has joined #bitcoin-core-dev
 252018-12-26T00:24:11  <jimmysong> it lets you verify that the filter bytes are what they said
 262018-12-26T00:24:21  <jimmysong> and you can check that the scripts for the inputs verify
 272018-12-26T00:24:28  <sipa> no, because the prevout scripts can be forged
 282018-12-26T00:24:44  <sipa> there is nothing committing to them
 292018-12-26T00:24:51  <jimmysong> wait, if you have a signature, you can make a pubkey that works with it?
 302018-12-26T00:25:10  <jimmysong> err, rather, a pubkey, you can make a hash preimage?
 312018-12-26T00:25:39  <jimmysong> the scriptsig/witness for p2pkh/p2wpkh is pubkey, sig
 322018-12-26T00:25:47  <jimmysong> so you'd have to forge a preimage of the pubkey
 332018-12-26T00:25:56  <sipa> oh, you mean full script validation?
 342018-12-26T00:26:08  <jimmysong> yes
 352018-12-26T00:26:14  <sipa> that shouldn't be something a light client cares about
 362018-12-26T00:26:32  <jimmysong> if it helps verify the cfilter hash, why not?
 372018-12-26T00:26:49  <jimmysong> commitment is obviously preferable
 382018-12-26T00:26:56  <jimmysong> but if we don't have that?
 392018-12-26T00:27:08  <roasbeef> jimmysong: yeh there's a half proposal on the ML to add that, the prior versions had the outpoint so they were fully verifiable, as is now you can verify half of it
 402018-12-26T00:27:21  <sipa> what is your attack model?
 412018-12-26T00:27:38  <roasbeef> differntiate honest peers from dishonest peers
 422018-12-26T00:27:55  <sipa> you don't want to get the prevout script for every block, right?
 432018-12-26T00:28:01  <roasbeef> it's not full script validation, it's verifying the filter is correct
 442018-12-26T00:28:21  <sipa> sure, but you'd need a full script interpreter for doing so
 452018-12-26T00:28:22  <roasbeef> yeh you'd fetch the block with the prior scripts and value why not while you're at it, w/ value you can verify fees to a degree as well
 462018-12-26T00:28:54  <roasbeef> no need to verify the script, just to verify a filter actually fully matchs a given block
 472018-12-26T00:29:13  <jimmysong> i was thinking full script interpreter
 482018-12-26T00:29:29  <jimmysong> which isn't trivial
 492018-12-26T00:29:44  <sipa> roasbeef: sure, that makes sense; but jimmysong was suggesring script validation, as without it, the prevout data can be trivially forged
 502018-12-26T00:29:44  <roasbeef> txscript.VM ;)
 512018-12-26T00:30:14  <sipa> jimmysong: but for which blocks would you get this prevout data?
 522018-12-26T00:30:26  <jimmysong> the blocks you download when you search for your utxos
 532018-12-26T00:30:31  <roasbeef> blocks for which you get conflicting advertisements
 542018-12-26T00:30:38  <jimmysong> that too
 552018-12-26T00:30:51  <sipa> jimmysong: a peer can still lie and give you a filter with no matches at all
 562018-12-26T00:30:56  *** sipa has quit IRC
 572018-12-26T00:31:14  <roasbeef> you can fetch the block and verify the outputs are properly included, but not fully inputs
 582018-12-26T00:31:15  *** sipa has joined #bitcoin-core-dev
 592018-12-26T00:31:43  <sipa> yes, for confloct detection it makes sense, and you need no script validation etc
 602018-12-26T00:31:53  <roasbeef> main purpose if conflict detection
 612018-12-26T00:31:59  <roasbeef> is*
 622018-12-26T00:32:17  <roasbeef> harding: the prev outs are in the undo blocks
 632018-12-26T00:32:52  <roasbeef> this was a concern during the switch over (that it would be slower to construct our possibly impossible), but the data is still around
 642018-12-26T00:34:29  <jimmysong> what's the objection to writing a script interpreter?
 652018-12-26T00:34:34  <jimmysong> is it because of all the edge cases?
 662018-12-26T00:34:41  <roasbeef> many already exist
 672018-12-26T00:34:55  <jimmysong> i've written a partial one
 682018-12-26T00:35:04  <jimmysong> i still don't get op_codeseparator
 692018-12-26T00:35:20  <roasbeef> code sep should die, ppl should stop trying to save it lol
 702018-12-26T00:36:01  <jimmysong> btw, roasbeef, those test vectors for neutrino broke my script interpreter =)
 712018-12-26T00:36:36  <sipa> jimmysong: in my view script interpreting is so easy to get wrong, it should only be done by full nodes
 722018-12-26T00:37:03  <jimmysong> it is easy to get wrong, but isn't there value to at least interpreting some subset?
 732018-12-26T00:37:25  <jimmysong> p2wsh/p2sh stuff can get pretty complicated
 742018-12-26T00:37:32  <sipa> i don't think so
 752018-12-26T00:37:50  <sipa> a wallet can recognize its own outputs, bit that doesn't need interpretatiom
 762018-12-26T00:38:18  <roasbeef> you can't even verify the outputs fully exist, so script verification is w/e, main reason for getting the prevouts is filter verification and fee awareness
 772018-12-26T00:38:18  <sipa> for debugging purposes it can be useful of course, but imo that's it
 782018-12-26T00:39:24  <jimmysong> so wallets should stick to standard scripts, or whatever has been analyzed beforehand
 792018-12-26T00:39:49  <sipa> well they construct them themselves
 802018-12-26T00:40:26  <sipa> so you don't need an interpreter to find its semantics
 812018-12-26T00:40:44  <jimmysong> right, the ones analyzed beforehand by the programmer
 822018-12-26T00:40:47  <sipa> and you won't trigger any edge cases unless you go out of your way to trigger them
 832018-12-26T00:42:17  <jimmysong> in any case, if the outpoint data is in the undo blocks, that can be made available?
 842018-12-26T00:42:32  <roasbeef> yep, just add a new inv type, and message format
 852018-12-26T00:43:41  <jimmysong> sure, you need the full tx's to verify against the hashes, which as harding pointed out can get pathologically large in edge cases
 862018-12-26T00:44:09  <jimmysong> similar to jj's attack from breaking bitcoin last year
 872018-12-26T00:44:30  <sipa> jimmysong: needing the full txn is unrealistic
 882018-12-26T00:44:49  <sipa> that data isn't available in general, and expensive to index
 892018-12-26T00:44:49  <roasbeef> why do you care about the hashes? get the script+value, verify the sig+fee, reconstruct filter, dunzo
 902018-12-26T00:45:24  <roasbeef> ofc if the witness commitment was to a merkalized version of the transaction, you could fetch that proof as well, but mo bytes there, what we have available atm is good enough to achieve the goal of cross checking
 912018-12-26T00:46:02  <sipa> in general we should aim to only expose data that is verifiable or can be made verifiable
 922018-12-26T00:46:34  <sipa> and adding comitments to undo data seems overkill, when all that's needed is a commitmemt to the filters
 932018-12-26T00:46:53  <jimmysong> yes, that's the preferred solution
 942018-12-26T00:47:00  <roasbeef> yeh not undo, just filters eventually, mean as in if the wtxid was a merkle tree with leaves of the inputs/outputs etc
 952018-12-26T00:47:59  <roasbeef> would also let you have a more compact proof of spentnesss, especially if things are also mega super coin-joiny in the future
 962018-12-26T00:48:24  <jimmysong> wow, good point
 972018-12-26T00:49:05  <sipa> i think all these adhoc methods of verifying filters break down if you look at them in an adverserial setting... you can use heuristics and download from multiple peers, and detect conflicts, and spot check here and there... but those approaches all add complexity and bandwidth proportionl to the level of guarantee they give
 982018-12-26T00:49:30  <roasbeef> yep those other appraoches are just plain easier to mess up, i prefer dead simple methods
 992018-12-26T00:49:58  <sipa> the only foolproof, cheap, ... way is a filter comitment in blocks
1002018-12-26T00:50:19  <jimmysong> yep, which costs something for a miner, but not very much. but it is a soft fork
1012018-12-26T00:50:28  <jimmysong> any idea how that would be received?
1022018-12-26T00:51:05  <gmaxwell> the bigger limit there is just that there needs to be a higher level of confidence in the design, since removing the requirement is a hardfork. Fortunately, as it is now has actually had a fair level of refinement.
1032018-12-26T00:51:51  <roasbeef> jimmysong: filter commit in op return, message to fetch header along w/ path for coinbase transaction? i think the filter chain would prob still have some use in that case as well
1042018-12-26T00:52:11  <gmaxwell> I've suggested in the past potentially doing a kind of rolling softfork, where the commitment is required to be valid only if its provided and only until some height.  And so long as people keep using it and it isn't replaced with something better, we just keep soft-forking in updated heights.. but if ever it gets replaced, it can just be allowed to expire.
1052018-12-26T00:52:46  <jimmysong> that would be a nice escape clause
1062018-12-26T00:52:46  <roasbeef> oooOOo, i guess the witness commitment is kinda like that for blocks w/o any witness data
1072018-12-26T00:53:01  <sipa> jimmysong: i think it's necessary; without commitment to the filters you can basically only use it locally or with a trusted full node
1082018-12-26T00:53:23  <sipa> which is pretty useful on itself, but nearly the same
1092018-12-26T00:53:33  <sipa> but not nearly
1102018-12-26T00:53:35  <gmaxwell> I'm very glad, e.g. that nothing about BIP37 ended up softforked in... that protocol turned out to be a lemon in a number of ways, but it took a couple years of use to realize that.
1112018-12-26T00:54:07  <roasbeef> well you can still use it in the open net, you have the same "one honest peer" assumption going on, the scripts just help you to distinguish between zero and 1 honest peer
1122018-12-26T00:55:39  <gmaxwell> roasbeef: even 1 honest peer requires you to have some kind of complex resolution to deal with disagreement, also-- because the p2p network is trivally sybable 'one honest peer' probably doesn't mean much unless you're doing something like manually configuring a trusted one.
1132018-12-26T00:55:45  *** fanquake has quit IRC
1142018-12-26T00:56:36  <gmaxwell> One should also consider the effect of incentiving varrious kinds of trouble making.  (like generally we've found that when we added vulnerablities like BIP37 attackers emerged that didn't exist previously, and then made more trouble even for people that didn't care about using BIP37)
1152018-12-26T00:57:58  <roasbeef> gmaxwell: you'd just fetch the blocks+scripts and reconstruct the filter, the commitment to the filters (the filter chain) helps you notice if something is funky at a glanec
1162018-12-26T00:58:19  <roasbeef> but then again, for smaller values you're prob not super concerned about this stuff
1172018-12-26T00:59:16  <gmaxwell> roasbeef: matching the spent inputs too requires fetching the inputs, which as sipa pointed out above is intractable in the worst case (and also astronishingly expensive even in the normal case)
1182018-12-26T00:59:49  <roasbeef> yeh the assumption is a new inv type to allow you to fetch the the input scripts, i guess i miss this worst case scenario?
1192018-12-26T00:59:57  <roasbeef> fetch with the block*
1202018-12-26T01:00:04  <gmaxwell> keeping also in mind that this stuff is saving you 30kbit/sec of bandwidth over downloading the whole blocks... in the ongoing case.
1212018-12-26T01:00:40  <gmaxwell> roasbeef: we have no way to provide that, and even if we did it couldn't be validated without fetching potentially gigabytes of additional data (you actually have to fetch the whole transactions).
1222018-12-26T01:03:48  *** fanquake has joined #bitcoin-core-dev
1232018-12-26T01:07:11  <roasbeef> ahh ok yeh i was missing the outpoints in the equation
1242018-12-26T01:07:18  <sipa> roasbeef: what if you receive two different filters, and request block/prevout data for both, and they are both correct? (and it turns out you receiced one true prevout data and one false prevout data, but with correct filters for that data)
1252018-12-26T01:07:21  <roasbeef> well can still verify half of it today ;)
1262018-12-26T01:07:43  <gmaxwell> roasbeef: yes, half indeed. and really the more useful half.
1272018-12-26T01:07:52  <gmaxwell> but at the expense of fetching the whole block.
1282018-12-26T01:08:09  <sipa> that's still detectable, but you're rediced to a majoroty vote kind of model, and at a possibly very high bandwidh cost
1292018-12-26T01:08:45  <gmaxwell> which also means that it really only takes one clown with an aws instance to cause a lot of users to fetch the blocks anyways.  I mean, perhaps not totally useless.  But as I mentioned, vulnerablties seem to attract attackers.
1302018-12-26T01:10:03  <gmaxwell> so then you're left with the work of implementing the validation, testing it, dealing with vulnerablities in it... and some clown spins up a bunch of sybils and everyone is downloading the entire blocks anyways. And-- the sybils themselves act as a lasting nussance.  I don't mean to argue against the non-commited usage, but considering the all in effects it's not the obvious big win that it
1312018-12-26T01:10:04  <gmaxwell> might seem at a glance.
1322018-12-26T01:11:11  <gmaxwell> and the historical rate of protocol additions introducing vulnerablties (either in implementation or design) is really high...
1332018-12-26T01:12:22  <roasbeef> it's a win for full nodes at least, serving the filters is much less intensive (and also stateless) compared to serving bip37, you also can't trigger worst case matching behavior over the entire chain
1342018-12-26T01:13:12  <gmaxwell> roasbeef: quite a few nodes just disable BIP37 completely (which seemed to stop the BIP37 based attacks)
1352018-12-26T01:14:28  <roasbeef> yeh i ended up doing that on my testnet nodes, seemed somsone was practicing their attacks on testnet lol
1362018-12-26T01:14:33  <gmaxwell> (I'm not disagreeing with your point though)
1372018-12-26T01:15:36  <gmaxwell> roasbeef: they were doing it on mainnet for a while. Though they seemed to give up after even a small set of their targets started disabling them. To the extent that they might have been targeting miners to cause block orphaning, that makes sense, but otherwise it's not really clear why it stopped.
1382018-12-26T01:16:00  <gmaxwell> Perhaps because they realized if they kept it up BIP37 was just going to end up removed and then they'd lose their toy. Who knows.
1392018-12-26T01:31:02  *** jpe_ has joined #bitcoin-core-dev
1402018-12-26T02:07:08  *** EagleTM has quit IRC
1412018-12-26T02:13:06  *** e4xit has quit IRC
1422018-12-26T02:19:57  *** jpe___ has joined #bitcoin-core-dev
1432018-12-26T02:20:32  *** jpe___ is now known as jpe
1442018-12-26T02:21:33  *** jpe_ has quit IRC
1452018-12-26T02:24:04  *** fabianfabian has quit IRC
1462018-12-26T02:24:49  *** morcos has quit IRC
1472018-12-26T02:27:13  *** morcos has joined #bitcoin-core-dev
1482018-12-26T02:29:21  *** ghost43 has quit IRC
1492018-12-26T02:29:38  *** ghost43 has joined #bitcoin-core-dev
1502018-12-26T02:30:51  *** peevsie has joined #bitcoin-core-dev
1512018-12-26T02:34:22  *** fabianfabian has joined #bitcoin-core-dev
1522018-12-26T02:54:42  *** mistergold has quit IRC
1532018-12-26T03:24:49  *** booyah has quit IRC
1542018-12-26T03:26:18  *** booyah has joined #bitcoin-core-dev
1552018-12-26T03:29:01  *** rh0nj has quit IRC
1562018-12-26T03:30:08  *** rh0nj has joined #bitcoin-core-dev
1572018-12-26T04:12:12  *** fabianfabian has quit IRC
1582018-12-26T04:49:33  *** hebasto has joined #bitcoin-core-dev
1592018-12-26T04:56:28  *** jpe has quit IRC
1602018-12-26T05:22:42  *** bitcoin-git has joined #bitcoin-core-dev
1612018-12-26T05:22:43  <bitcoin-git> [bitcoin] markaw67 opened pull request #15036: Mwortham (master...patch-2) https://github.com/bitcoin/bitcoin/pull/15036
1622018-12-26T05:22:43  *** bitcoin-git has left #bitcoin-core-dev
1632018-12-26T05:33:39  *** justanotheruser has quit IRC
1642018-12-26T05:49:03  *** peevsie has quit IRC
1652018-12-26T05:55:02  <fanquake> Wondering if I should even bother in #15036. The guy has turned up to the repo, spammed in one PR, then opened 15036.
1662018-12-26T05:55:03  <gribble> https://github.com/bitcoin/bitcoin/issues/15036 | Mwortham by markaw67 · Pull Request #15036 · bitcoin/bitcoin · GitHub
1672018-12-26T05:55:39  <fanquake> Clearly not bothering to read my comment, or from what I can tell, anything related to actually contributing to the project.
1682018-12-26T06:01:18  <gmaxwell> I'd recommend just closing and locking the PR. They aren't following the guidelines, and there is a good chance that its just trolling.
1692018-12-26T06:01:26  <gmaxwell> no different than any other driveby pr
1702018-12-26T06:07:37  *** notmandatory has joined #bitcoin-core-dev
1712018-12-26T06:09:41  <gmaxwell> and especially due to the principle that the project isn't a place that we'll tolerate other people turning into their performance art or battleground.
1722018-12-26T06:22:07  *** spinza has quit IRC
1732018-12-26T06:34:00  *** spinza has joined #bitcoin-core-dev
1742018-12-26T06:34:45  *** bitcoin-git has joined #bitcoin-core-dev
1752018-12-26T06:34:46  <bitcoin-git> [bitcoin] fanquake closed pull request #15036: Mwortham (master...patch-2) https://github.com/bitcoin/bitcoin/pull/15036
1762018-12-26T06:34:46  *** bitcoin-git has left #bitcoin-core-dev
1772018-12-26T06:39:09  *** rex4539 has joined #bitcoin-core-dev
1782018-12-26T06:40:23  *** jarthur has quit IRC
1792018-12-26T06:53:57  <gwillen> fanquake: if you are able, it seems like preemptively locking might be a good idea.
1802018-12-26T06:55:03  <fanquake> I was going to leave it for a reply, but have done it now.
1812018-12-26T06:57:57  <gwillen> the quality of his previous comment doesn't make me sanguine
1822018-12-26T06:58:09  <gwillen> also, the account that hit 'approve' looks like a sockpuppet
1832018-12-26T07:47:16  *** harrymm has quit IRC
1842018-12-26T08:04:59  *** harrymm has joined #bitcoin-core-dev
1852018-12-26T08:32:27  *** tiagotrs has joined #bitcoin-core-dev
1862018-12-26T08:37:36  *** harrymm has quit IRC
1872018-12-26T08:50:31  *** harrymm has joined #bitcoin-core-dev
1882018-12-26T09:01:43  *** elichai2 has joined #bitcoin-core-dev
1892018-12-26T09:04:07  *** justanotheruser has joined #bitcoin-core-dev
1902018-12-26T09:34:02  *** Zenton has joined #bitcoin-core-dev
1912018-12-26T09:50:02  *** e4xit has joined #bitcoin-core-dev
1922018-12-26T10:02:36  *** spinza has quit IRC
1932018-12-26T10:47:02  *** spinza has joined #bitcoin-core-dev
1942018-12-26T10:56:07  *** harrymm has quit IRC
1952018-12-26T11:02:01  *** rh0nj has quit IRC
1962018-12-26T11:03:10  *** rh0nj has joined #bitcoin-core-dev
1972018-12-26T11:35:20  *** spinza has quit IRC
1982018-12-26T11:40:47  *** spinza has joined #bitcoin-core-dev
1992018-12-26T11:56:51  *** EagleTM has joined #bitcoin-core-dev
2002018-12-26T12:06:25  *** ghost43 has quit IRC
2012018-12-26T12:06:36  *** ghost43 has joined #bitcoin-core-dev
2022018-12-26T12:14:09  *** Yaggi has joined #bitcoin-core-dev
2032018-12-26T12:25:48  *** AaronvanW has joined #bitcoin-core-dev
2042018-12-26T12:29:19  <fanquake> wumpus are you around tonight?
2052018-12-26T12:36:19  *** Yaggi has quit IRC
2062018-12-26T12:46:55  *** YAGGI has joined #bitcoin-core-dev
2072018-12-26T13:10:54  *** AaronvanW has quit IRC
2082018-12-26T13:17:07  *** YAGGI has quit IRC
2092018-12-26T13:26:48  *** EagleTM has quit IRC
2102018-12-26T13:30:46  *** bsm117532 has joined #bitcoin-core-dev
2112018-12-26T13:37:54  *** spaced0ut has joined #bitcoin-core-dev
2122018-12-26T13:52:01  *** mistergold has joined #bitcoin-core-dev
2132018-12-26T13:54:27  *** cluelessperson has joined #bitcoin-core-dev
2142018-12-26T13:54:52  *** cluelessperson is now known as Guest88093
2152018-12-26T13:56:05  *** bitcoin-git has joined #bitcoin-core-dev
2162018-12-26T13:56:05  <bitcoin-git> [bitcoin] hebasto opened pull request #15038: docs: Get more info about GUI-related issue on Linux (master...20181226-issue-template-gui-linux) https://github.com/bitcoin/bitcoin/pull/15038
2172018-12-26T13:56:05  *** bitcoin-git has left #bitcoin-core-dev
2182018-12-26T14:04:14  *** e4xit has quit IRC
2192018-12-26T14:04:26  *** Guest88093 has quit IRC
2202018-12-26T14:07:00  *** e4xit has joined #bitcoin-core-dev
2212018-12-26T14:08:13  *** cluelessperson has joined #bitcoin-core-dev
2222018-12-26T14:31:59  *** belcher has quit IRC
2232018-12-26T14:54:12  *** jpe has joined #bitcoin-core-dev
2242018-12-26T14:55:36  *** mistergold has quit IRC
2252018-12-26T14:59:14  *** fabianfabian has joined #bitcoin-core-dev
2262018-12-26T15:03:43  <wumpus> fanquake: maybe a bit
2272018-12-26T15:04:01  *** rh0nj has quit IRC
2282018-12-26T15:05:08  *** rh0nj has joined #bitcoin-core-dev
2292018-12-26T15:05:56  *** flex has joined #bitcoin-core-dev
2302018-12-26T15:07:56  *** bitcoin-git has joined #bitcoin-core-dev
2312018-12-26T15:07:56  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #15039: wallet: Avoid leaking nLockTime fingerprint when anti-fee-sniping (master...Mf1812-walletLocktimeFingerprint) https://github.com/bitcoin/bitcoin/pull/15039
2322018-12-26T15:07:56  *** bitcoin-git has left #bitcoin-core-dev
2332018-12-26T15:08:14  <fanquake> wumpus np, bunch of PRs mergable, but can deal with em later
2342018-12-26T15:13:17  <wumpus> PSA: there's no meeting today (there was confusion about this last week)
2352018-12-26T15:15:58  <hebasto> wumpus: today is Wednesday. Do you mean tomorrow?
2362018-12-26T15:18:25  <wumpus> oh sorry yes I mean tomorrow
2372018-12-26T15:19:25  <fanquake> only Wednesday for another 41 minutes anyway
2382018-12-26T15:20:53  <sipa> wumpus: no, you're right, no meeting today!
2392018-12-26T15:21:07  *** ryanofsky has quit IRC
2402018-12-26T15:24:10  *** ryanofsky has joined #bitcoin-core-dev
2412018-12-26T15:28:21  *** peevsie has joined #bitcoin-core-dev
2422018-12-26T15:39:36  *** tiagotrs has quit IRC
2432018-12-26T15:53:43  *** fanquake has quit IRC
2442018-12-26T15:59:59  *** kinlo has quit IRC
2452018-12-26T16:03:55  *** tiagotrs has joined #bitcoin-core-dev
2462018-12-26T16:06:25  *** kinlo has joined #bitcoin-core-dev
2472018-12-26T16:07:05  *** jpe has quit IRC
2482018-12-26T16:13:23  *** ddustin has joined #bitcoin-core-dev
2492018-12-26T16:16:32  *** cyber55 has quit IRC
2502018-12-26T16:34:18  *** mistergold has joined #bitcoin-core-dev
2512018-12-26T16:47:01  *** Murch has joined #bitcoin-core-dev
2522018-12-26T17:10:14  <andytoshi> are there any guarantees about the order of `getrawmempool` output? in particular do ancestors always precede descendants?
2532018-12-26T17:12:20  *** grubles has quit IRC
2542018-12-26T17:19:27  *** achow101_ has joined #bitcoin-core-dev
2552018-12-26T17:19:40  <instagibbs> from my cursory reading they are indexed in 4 different ways, none of which are related to ancestors/decendants
2562018-12-26T17:19:45  *** Murch has quit IRC
2572018-12-26T17:20:13  <andytoshi> kk thanks, i won't rely on that then
2582018-12-26T17:20:21  <instagibbs> salted txid, feerate(including desc), entry time, and sorted feerate(including ancestor
2592018-12-26T17:20:30  *** achow101 has quit IRC
2602018-12-26T17:20:46  <andytoshi> i mean, somehow when creating blocks they have to wind up in ancestor order ... is there an explicit sorting step then?
2612018-12-26T17:21:07  <instagibbs> they grab "packages" as they sort via package feerate
2622018-12-26T17:22:05  <instagibbs> there are internal links between the entries, just not exposed here afaik
2632018-12-26T17:22:14  <andytoshi> ah, yep, that makes sense
2642018-12-26T17:22:28  <andytoshi> my goal here is to recreate the packages from the output of getrawmempool
2652018-12-26T17:22:34  *** achow101_ is now known as achow101
2662018-12-26T17:22:41  *** Murch has joined #bitcoin-core-dev
2672018-12-26T17:26:06  <sipa> andytoshi: i'm pretty sure they're sorted by increasing total number of (recursive) unconfirmed dependencies, and then by feerate
2682018-12-26T17:26:17  <sipa> and then by txid as tiebreaker or so
2692018-12-26T17:26:41  <sipa> which guarantees that dependencies always come before the dependendees (?)
2702018-12-26T17:27:14  <sipa> that code is not shared with the block creation code, btw
2712018-12-26T17:30:12  <instagibbs> andytoshi, also try getmempoolentry which comes with more details?
2722018-12-26T17:32:26  <andytoshi> instagibbs: oh, yeah (or `getrawmempool true` which gives me the same details). will take a look at that to see if it's useful. i suspect not, i think i need to manually recreate a lot of this data in my code because i need a bunch of extra details, like the set of yet-ununspent inputs/outputs for the whole package
2732018-12-26T17:32:43  <andytoshi> sipa: cool, thanks. but i guess that's an implementation detail and if i were to write production software that depended on it, you'd be annoyed :)
2742018-12-26T17:35:07  <andytoshi> i wish there was some way i could signal core that i don't want certain outputs to be 0conf-spendable (or if they are, that i don't want cpfp rules to be applied)
2752018-12-26T17:36:25  <gmaxwell> andytoshi: encountering the RBF pinning problem?
2762018-12-26T17:37:32  <andytoshi> gmaxwell: roughly ... but i think it doesn't even need RBF to be a problem .. e.g. see russell's mailing list post https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016519.html
2772018-12-26T17:38:11  <andytoshi> the issue is that if my software is making a package A of transactions and doing all sorts of CPFP logic to make that sensible, and meanwhile some customer of mine is creating a package B with a massive low-fee sweep or whatever
2782018-12-26T17:38:19  *** Murch has quit IRC
2792018-12-26T17:38:23  <andytoshi> and then that customer creates a tx spending an output of A alongside the output of B...
2802018-12-26T17:38:33  <andytoshi> ...those packages become merged and suddenly my logic has been blindsided
2812018-12-26T17:38:44  <gmaxwell> that isn't how the implementation works.
2822018-12-26T17:39:04  <gmaxwell> I think Russell is like... reasoning from the feature's name.
2832018-12-26T17:39:32  *** jarthur has joined #bitcoin-core-dev
2842018-12-26T17:40:23  <andytoshi> well, the code is not super straightforward to someone uninvolved with it. the mempool logic related to the descendant limit looks kinda like it would do stuff like what i described
2852018-12-26T17:41:10  <gmaxwell> The descndant limit stuff can do things along those lines, but requires actually hitting the descendant limit.
2862018-12-26T17:45:52  <gmaxwell> If the limits are getting hit by ordnary usage we should look into fixing that. They were set so they were never hit when established (except for some obviously dumb floody crap), and only exist to prevent a bad computational blowup in the tracking code.
2872018-12-26T17:48:32  <instagibbs> one thing to note is that a single ~100kvB sweep can hose the descendant size limit pretty much
2882018-12-26T17:54:17  <gmaxwell> so fix it?
2892018-12-26T17:55:30  <gmaxwell> IIRC the reason for the size limits in the tracking is just so it doesn't falsely credit parents for feerate coming from transactions that are never going to fit in the same block.
2902018-12-26T18:00:00  *** peevsie has quit IRC
2912018-12-26T18:00:23  <andytoshi> maybe i'm confused about cpfp. my understanding is that if i make a tx with outputs controlled by other people, those other people are able to grief me and undermine my ability to use cpfp
2922018-12-26T18:00:31  <andytoshi> by extending the package such that i'd be hitting limits
2932018-12-26T18:02:16  <gmaxwell> yes, though also at the expense of delaying their own transaction confirmation.
2942018-12-26T18:02:34  <gmaxwell> I don't believe we've ever seen cpfp 'griefing' reported.
2952018-12-26T18:02:58  <gmaxwell> RBF pinning for sure, because a common usage pattern immediately causes it.
2962018-12-26T18:04:33  <gmaxwell> The limits exist only because there are computational overheads in the tracking, e.g. when removing a transaction its ancestors and descendants need to be walked to update their tracking.
2972018-12-26T18:11:59  *** jarthur has quit IRC
2982018-12-26T18:12:15  *** gelmutshmidt has joined #bitcoin-core-dev
2992018-12-26T18:13:03  *** gelmutshmidt has quit IRC
3002018-12-26T18:13:29  *** gelmutshmidt has joined #bitcoin-core-dev
3012018-12-26T18:14:03  *** tiagotrs has quit IRC
3022018-12-26T18:16:01  *** IGHOR has quit IRC
3032018-12-26T18:16:25  <andytoshi> sorry for the dumb questions, but can you clarify - if i'm making cpfp packages, and one of my customers 0conf spends one of the outputs i create, will their transaction pull my effective feerate toward the feerate of that tx? if so, then i need logic to reason about that, and by nature that logic needs to know about the limits (even though in practice i don't expect anyone to pull me close to
3042018-12-26T18:16:31  <andytoshi> them)
3052018-12-26T18:16:40  <andytoshi> or is it safe if i just make my own transactions that chain off each others' change outputs, and ignore everything else?
3062018-12-26T18:17:30  <andytoshi> maybe i should just pester sipa in person in a couple of weeks :)
3072018-12-26T18:17:44  <andytoshi> and i will try to write down what i'm learning as i do this
3082018-12-26T18:18:57  <gmaxwell> No, it will not.
3092018-12-26T18:20:28  <gmaxwell> The parents effective feerate is the highest feerate you can construct with it.
3102018-12-26T18:20:45  <andytoshi> ok, i think i've got it
3112018-12-26T18:21:09  <gmaxwell> They can, if they spam out to the limits, prevent new descendants from being taken.
3122018-12-26T18:21:23  <gmaxwell> But thats only in the case that the tracking limits are hit.
3132018-12-26T18:21:47  <andytoshi> so is this a rough high-level view of cpfp in core?: (a) "packages" only exist during miners' transaction selection; in the case that a transaction might be in multiple packages, they're computed greedily to maximize feerate; (b) but when accepting to the mempool, Core checks whether a transaction might cause a limit-violating package to exist, and if it would, the tx is rejected
3142018-12-26T18:26:06  <gmaxwell> close enough;  the limits aren't really limits on 'packages', they're more limits on the tracking datastructures used to create packages.
3152018-12-26T18:26:55  <andytoshi> yep, that's what i meant
3162018-12-26T18:29:17  *** IGHOR has joined #bitcoin-core-dev
3172018-12-26T18:39:59  *** Guyver2 has joined #bitcoin-core-dev
3182018-12-26T18:40:07  *** Zenton has quit IRC
3192018-12-26T18:48:36  *** Guyver2_ has joined #bitcoin-core-dev
3202018-12-26T18:51:25  *** peevsie has joined #bitcoin-core-dev
3212018-12-26T18:51:40  *** Guyver2 has quit IRC
3222018-12-26T19:06:02  *** rh0nj has quit IRC
3232018-12-26T19:08:20  *** AaronvanW has joined #bitcoin-core-dev
3242018-12-26T19:09:08  *** rh0nj has joined #bitcoin-core-dev
3252018-12-26T19:35:57  *** mistergold has quit IRC
3262018-12-26T19:49:21  *** tiagotrs has joined #bitcoin-core-dev
3272018-12-26T19:56:30  *** tiagotrs has quit IRC
3282018-12-26T20:00:22  *** gelmutshmidt has quit IRC
3292018-12-26T20:06:30  *** IGHOR has quit IRC
3302018-12-26T20:11:26  *** elichai2 has quit IRC
3312018-12-26T20:21:56  *** fabianfabian has quit IRC
3322018-12-26T20:24:37  *** IGHOR has joined #bitcoin-core-dev
3332018-12-26T20:27:24  *** bitcoin-git has joined #bitcoin-core-dev
3342018-12-26T20:27:24  <bitcoin-git> [bitcoin] hebasto opened pull request #15040: qt: Add workaround for QProgressDialog bug on macOS (master...20181226-fix-macos-qprogressdialog) https://github.com/bitcoin/bitcoin/pull/15040
3352018-12-26T20:27:24  *** bitcoin-git has left #bitcoin-core-dev
3362018-12-26T20:29:55  *** EagleTM has joined #bitcoin-core-dev
3372018-12-26T20:46:06  *** bsm117532 has quit IRC
3382018-12-26T20:47:11  *** bsm117532 has joined #bitcoin-core-dev
3392018-12-26T20:55:57  *** Krellan has quit IRC
3402018-12-26T21:03:03  *** hebasto has quit IRC
3412018-12-26T21:28:49  *** Zenton has joined #bitcoin-core-dev
3422018-12-26T21:36:59  *** Guyver2_ has quit IRC
3432018-12-26T21:41:54  *** promag has joined #bitcoin-core-dev
3442018-12-26T21:46:24  *** promag has quit IRC
3452018-12-26T21:56:10  *** jcorgan has quit IRC
3462018-12-26T21:56:41  *** jcorgan has joined #bitcoin-core-dev
3472018-12-26T22:03:34  *** spinza has quit IRC
3482018-12-26T22:10:00  *** spinza has joined #bitcoin-core-dev
3492018-12-26T22:10:56  *** tiagotrs has joined #bitcoin-core-dev
3502018-12-26T22:15:24  *** rex4539 has quit IRC
3512018-12-26T22:16:57  *** tiagotrs has quit IRC
3522018-12-26T22:42:23  *** promag has joined #bitcoin-core-dev
3532018-12-26T23:10:01  *** rh0nj has quit IRC
3542018-12-26T23:16:10  <jnewbery> andytoshi: I think that's mostly right. In (a), the miner is ordering by ancestor feerate (see BlockAssembler::addPackageTxs in miner.cpp). In (b), all of the {(ancestor|descedant) (count|size)} are taken into account (see CTxMemPool::CalculateMemPoolAncestors() in txmempool.cpp)
3552018-12-26T23:18:11  <jnewbery> "and i will try to write down what i'm learning as i do this". Please consider contributing that to https://github.com/bitcoinops/scaling-book/blob/master/1.fee_bumping/fee_bumping.md if you think you can document it!
3562018-12-26T23:31:33  <gmaxwell> jnewbery: your last statement sounds confusing, and plays into roconnor's misunderstanding.
3572018-12-26T23:32:31  <gmaxwell> Basically what roconnor was thinking was that if there is an unconfirmed txn and then I add a gigantic low feerate child to it,  I lower the feerate of the txn because the "package" has a lower feerate.. And that is not how it works, because of the max operation in the combining.
3582018-12-26T23:37:14  *** spinza has quit IRC
3592018-12-26T23:42:24  *** mistergold has joined #bitcoin-core-dev
3602018-12-26T23:44:32  <jnewbery> (b) is not looking at feerate. Just tx count and size
3612018-12-26T23:48:37  *** spinza has joined #bitcoin-core-dev
3622018-12-26T23:54:07  *** EagleTM has quit IRC