19:00:28 <wumpus> #startmeeting
19:00:28 <lightningbot> Meeting started Thu Jul 18 19:00:28 2019 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:28 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:00:31 <jnewbery> hi
19:00:40 <kanzure> hi
19:00:42 <jonasschnelli> hi
19:00:44 <kallewoof> hello
19:00:52 <hebasto> hi
19:00:57 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball kvaciral
19:01:05 <achow101> hi
19:01:07 <sipa> hi
19:01:07 <meshcollider> hi
19:01:10 * dongcarl would like to talk about addrv2 some more after the meeting if people have time
19:01:29 <luke-jr> hi
19:01:31 <jamesob> hi
19:02:19 <moneyball> hi
19:02:32 <wumpus> three topics proposed in https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a this time:  Remove mempool expiry, treat txs as replaceable instead , 0.18.1 and "-blocksonly not being a hidden option in 0.18.1"
19:02:35 <moneyball> https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a
19:02:49 <cfields> hi
19:04:06 <wumpus> #topic High priority for review
19:04:24 <warren> hi
19:04:31 <wumpus> current: 4 blockers, 5 things chasing concept ACK
19:04:36 <wumpus> https://github.com/bitcoin/bitcoin/projects/8
19:04:36 <gleb> hi
19:04:54 <jamesob> was hoping to get #16355 added
19:04:56 <gribble> https://github.com/bitcoin/bitcoin/issues/16355 | refactor: move CCoinsViewErrorCatcher out of init.cpp by jamesob · Pull Request #16355 · bitcoin/bitcoin · GitHub
19:04:56 <wumpus> anything to add, anything ready for merge?
19:05:16 <jonasschnelli> May I add #16202 to the list?
19:05:18 <gribble> https://github.com/bitcoin/bitcoin/issues/16202 | Refactor network message deserialization by jonasschnelli · Pull Request #16202 · bitcoin/bitcoin · GitHub
19:05:28 <kallewoof> wumpus: I would like #16411 in Chasing Concept ACK, if possible
19:05:31 <gribble> https://github.com/bitcoin/bitcoin/issues/16411 | Signet support by kallewoof · Pull Request #16411 · bitcoin/bitcoin · GitHub
19:05:52 <wumpus> jamesob jonasschnelli kallewoof sure
19:06:01 <jamesob> thanks!
19:07:39 <wumpus> ok added
19:07:55 <wumpus> there are *many* things chasing concept ACK, are there any making progress (maybe enough to remove them?)
19:09:56 <luke-jr> I almost wonder if concept-acks should be done separately from PRs, with more user input
19:09:56 <wumpus> #topic Remove mempool expiry, treat txs as replaceable instead (MarcoFalke)
19:10:27 <MarcoFalke> Pull request is here #16409
19:10:29 <gribble> https://github.com/bitcoin/bitcoin/issues/16409 | Remove mempool expiry, treat txs as replaceable instead by MarcoFalke · Pull Request #16409 · bitcoin/bitcoin · GitHub
19:10:35 <MarcoFalke> And already received some feedback
19:10:42 <luke-jr> Sounds like it should be two PRs…
19:10:55 <wumpus> luke-jr: not sure separating them out is a key to get more interest imo
19:11:15 <luke-jr> expiry helps remove unconfirming txs from RAM etc
19:11:19 <MarcoFalke> luke-jr: Yeah, if removing expiry is so controversial, then yes
19:11:37 <wumpus> anything that touches policy is probably somewhat controversial
19:11:52 <sipa> another approach is to make the marginal-cost-for-replacement go to 0 after some time
19:11:57 <sdaftuar> I think expiry performs an important job, albeit not perfectly, which is handling diverse node policies in a semi-reasonable way
19:12:08 <luke-jr> but what if the sender doesn't replace?
19:12:11 <MarcoFalke> luke-jr: YOu can not expect to use that ram anyway, since you set the max usage to well the max usage
19:12:11 <sipa> yeah, i'd need to think more about the impact there
19:12:38 <luke-jr> MarcoFalke: it might not always be maxxed out
19:12:47 <jnewbery> sipa: that doesn't solve the case of txs being pinned
19:12:56 <sipa> jnewbery: ah yes
19:14:01 <MarcoFalke> sipa: I don't see this as an issue in the network as of today (it is mostly theoretical)
19:14:23 <luke-jr> MarcoFalke: it's an issue because people are designing around it
19:14:28 <sdaftuar> the issue of diverse node policies is an issue
19:14:32 <midnightmagic> I was going to say, expiry is super useful..
19:14:43 <sdaftuar> if we deploy this now, then in the future when we make a policy change, we'll have a problem to think about
19:14:51 <sdaftuar> which we won't have a good way to solve
19:15:42 <MarcoFalke> sdaftuar: Agree that it might be premature to remove expiry right now, but in the long term it could at least be limited to high-fee txs
19:15:48 <wumpus> what is the primary motivation to remove expiry?
19:16:16 <sipa> sdaftuar: when a policy change happens, there is always a shutdown+restart involved, no?
19:16:25 <sipa> sdaftuar: so there is a load from mempool.dat in between?
19:16:29 <sdaftuar> the issue is around what happens to old nodes
19:16:34 <MarcoFalke> If miners are running the default settings, they might throw away income. If they are not, then nodes have a false sense of what can be replaced
19:16:46 <sdaftuar> which will not have the new rule, but will also not exprie things that are not getting mined
19:16:52 <luke-jr> not just old nodes. even updated nodes don't have a common policy
19:17:00 <sdaftuar> luke-jr: i agree with that too
19:17:15 <MarcoFalke> So the rbf pinning issue jnewbery mentions is not solved if miners are keeping the tx and you can replace it in your own mempool
19:17:32 <wumpus> so dropping expiry would mean that some transactions could get stuck in the mempool literally forever
19:17:38 <sdaftuar> wumpus: exactly
19:17:57 <sipa> i feel 2 weeks is sufficiently long that it isn't actually affecting anything that will likely confirm
19:18:01 <sipa> do we have numbers otherwise?
19:18:11 <wumpus> even an expiry of, say, 2 month would be better than none at all in that regard
19:18:16 <sipa> like at what rate do transactions get expired from common mempools?
19:18:23 <sipa> and do any of them still confirm after?
19:18:57 <sipa> or maybe even better, how often do transaction expire from the mempool, and then re-enter it?
19:18:58 <wumpus> would be useful to have that information
19:19:10 <kallewoof> sipa: I can get those numbers
19:19:22 <sipa> that'd be great to have in this discussion
19:19:25 <luke-jr> What problem is this trying to solve?
19:19:27 <kallewoof> sipa: They're not very large, from what I have seen
19:20:06 <MarcoFalke> kallewoof: The number of expiries or the number that are mined after expiry?
19:20:11 <sdaftuar> i think this needs to be thought about from the perspective of older software as well
19:20:30 <sdaftuar> if you only analyze this from the perspective of the latest version of Bitcoin Core, for instance, you only get a one-sided view
19:20:31 <kallewoof> MarcoFalke: I can fine tune it with some tweakery. Will not be done today, but can do tomorrow
19:20:34 <sipa> but it doesn't have to be large; say if 5 transactions expire per day, but 4.5 of those then re-enter the mempool again, that's evidence that due to rebroadcasting the expiration is effectively ineffective
19:20:43 * sipa likes "effectively ineffective"
19:21:02 <wumpus> hehe
19:21:23 <kallewoof> MarcoFalke: To clarify, I can fine tune it to count txs that do get mined after being purged from mempool at least once.
19:22:12 <sdaftuar> can you also do that from the perspective of an 0.15 node?
19:22:25 <kallewoof> sdaftuar: no
19:22:42 <Raystonn> It ineffectively affects the mempool, which results in not much net effect.
19:22:44 <kallewoof> sdaftuar: well, actually yes, but not without your help.
19:23:01 <sdaftuar> kallewoof: my data is probably insufficient too, unfortunately
19:23:24 <sdaftuar> i am not really sure i guess
19:23:26 <kallewoof> sdaftuar: i thought it recorded everything
19:24:01 <kallewoof> sdaftuar: besides, having the simulation mode alone means all you have to do is fake the time and throw the txs at the right moment and the node should do the expiration on its own
19:24:58 <sdaftuar> i don't know offhand how representative its outbound peers are
19:25:25 <sdaftuar> as an example, i was surprised today to discover that my 0.12 node accepted and then expired some transactions on a particular day in May
19:25:48 <sdaftuar> it looks like on May 5, it received some very low-fee transactions, which were accepted to the mempool as "priority" transactions
19:26:01 <sdaftuar> i was somewhat shocked that its peers (it only has the 8 outbound) would relay such things to it
19:26:09 <sdaftuar> but i guess it has some peer diversity
19:27:01 <MarcoFalke> sdaftuar: Those are probably valid txs, so I wouldn't call that "shocking"
19:27:40 <sdaftuar> MarcoFalke: i was shocked that the policy diversity on the network was so strong that 8 random peers would include some that aren't enforcing the minrelayfee we have had in place since 0.15
19:27:45 <luke-jr> mempool still had a priority exception in 0.15? O.o
19:27:55 <MarcoFalke> Ok, fine.
19:27:57 <sdaftuar> i think 0.15 was when we first got rid of it
19:28:02 <kallewoof> I looked at our email convo and it looks like I am waiting for you to give me a snapshot. :)
19:28:16 <sdaftuar> kallewoof: oops, thanks for the reminder :)
19:28:18 <MarcoFalke> I think this is too controversial right now, so I will let it sit for a while
19:28:25 <MarcoFalke> and revisit later
19:28:31 <MarcoFalke> Can we chat about 0.18.1?
19:28:43 <jnewbery> MarcoFalke: after some digging, it turns out those transactions were actually from 2017. Someone had dug them up and rebroadcast them
19:29:00 <MarcoFalke> sdaftuar and me had a topic on 0.18.1
19:29:20 <wumpus> #topic 0.18.1
19:29:22 <MarcoFalke> jnewbery: Which reads to me like a reason to remove expiry, but anyway
19:29:23 <Raystonn> A hodler found and opened his wallet again.
19:30:04 <sdaftuar> i wanted to discuss the impact of the -blocksonly change
19:30:08 <MarcoFalke> (Think how shocking it would be if you see a tx confirm 2 years after you "cancelled" it)
19:30:15 <wumpus> apparently, the maxtxfee fix is harder to backport than expected
19:30:40 <MarcoFalke> wumpus: I will take a look as well, but if it is really too hard I am fine with 0.18.2
19:30:55 <Raystonn> The only way to guarantee a transaction will not confirm once it has broadcast is to spend the relevant UTXOs in another.
19:30:59 <wumpus> oh, looks like promag  did already open a backport PR: #16414
19:31:01 <gribble> https://github.com/bitcoin/bitcoin/issues/16414 | 0.18: wallet: Fix -maxtxfee check by moving it to CWallet::CreateTransaction by promag · Pull Request #16414 · bitcoin/bitcoin · GitHub
19:31:05 <wumpus> hadn't seen this
19:31:14 <MarcoFalke> https://github.com/bitcoin/bitcoin/milestone/41
19:31:17 <wumpus> in any case as this is a non-clean backport it does need review and testing
19:31:34 <wumpus> more than had it been a clean backport, at least…
19:31:39 <MarcoFalke> Three people wrote tests for it
19:32:05 <MarcoFalke> So #16412 will get in as well?
19:32:07 <gribble> https://github.com/bitcoin/bitcoin/issues/16412 | net: Make poll in InterruptibleRecv only filter for POLLIN events. by tecnovert · Pull Request #16412 · bitcoin/bitcoin · GitHub
19:32:31 <wumpus> MarcoFalke: it should, it's a small and obvious change, that fixes a real problem, and is easy to backport
19:32:47 <MarcoFalke> #15911 doesn't make progess, so it'll have to wait
19:32:51 <gribble> https://github.com/bitcoin/bitcoin/issues/15911 | Use wallet RBF default for walletcreatefundedpsbt by Sjors · Pull Request #15911 · bitcoin/bitcoin · GitHub
19:33:06 <wumpus> yes
19:33:25 <wumpus> it's not ready for master yet, let alone backporting, so let's move it to 0.18.2
19:34:29 <wumpus> done
19:35:08 <wumpus> what about #15706?
19:35:10 <gribble> https://github.com/bitcoin/bitcoin/issues/15706 | build: Check QT library version by lucayepa · Pull Request #15706 · bitcoin/bitcoin · GitHub
19:35:28 <fanquake> I think that wait
19:35:32 <wumpus> it's been waiting for author for a while
19:35:33 <MarcoFalke> agree
19:35:37 <wumpus> even though my comment is trivial
19:35:46 <fanquake> *can
19:35:53 <wumpus> ok, moving
19:36:20 <wumpus> ok let's go to sdaftuar's topic
19:36:35 <promag> hi
19:36:44 <wumpus> -blocksonly is now a non-hidden option in 0.18.1
19:36:56 <sdaftuar> so historically, i believe we have not made the -blocksonly option more widely known because we don't have good protections in place for a network where many listening nodes are not relaying transactions, i think
19:37:15 <sdaftuar> now that we're doing it, i think we should make sure there are not unintended side effects
19:37:24 <MarcoFalke> sdaftuar: It is mentioned on bitcoin.org
19:37:36 <MarcoFalke> And has been for years
19:37:46 <sdaftuar> for instance, if we see a rise in -blocksonly listening nodes, then right now we have no protections in place for ensuring connectivity to transaction-relaying peers
19:37:54 <jonasschnelli> fee-estimations are not possible with -blocksonly? right?
19:38:10 <sdaftuar> MarcoFalke: i was not aware!  anyway, perhaps that doesn't matter much, i think we should still be concerned about making our software more robust
19:38:21 <MarcoFalke> agree
19:38:22 <jnewbery> jonasschnelli: correct, but I don't see how that's relevant
19:38:24 <kallewoof> jonasschnelli: fee estimations use blocks only, so it should be possible
19:38:27 <wumpus> I don't personally think unhiding the option in help is going to make that much of a difference, it's been well-known
19:38:33 <warren> maybe an informational tx relaying service bit?
19:38:48 <jnewbery> kallewoof: fee estimation requires a mempool
19:38:50 <sdaftuar> wumpus: fair point, perhaps our choice here is not relevant
19:38:54 <luke-jr> do they dset NODE_NETWORK?
19:38:59 <warren> or informational not-tx-relaying service bit?
19:39:03 <sdaftuar> luke-jr: yes
19:39:18 <kallewoof> jnewbery: weird. i thought it only used blocks. maybe it changed since last i looked at it.
19:39:41 <sdaftuar> i think we could make our p2p code smarter to ensure we have "enough" tx-relaying outbounds, but this work needs to be done, and i think we should prioritize it
19:39:57 <luke-jr> warren: I don't think we gain anything by making it a negative bit
19:40:35 <MarcoFalke> sdaftuar: Could you create a brainstorming issue for that?
19:40:59 <sdaftuar> also, old software has no protections, i don't know what to do about that. presumably we're still a while away from there being a large fraction of listening nodes being blocksonly, so we have some time
19:41:09 <sdaftuar> MarcoFalke: sure
19:41:20 <wumpus> is blocksonly really that popular?
19:41:26 <jonasschnelli> jnewbery: I think its relevant because it reduces the usability of blocksonly&wallet significant. I's already an expert features and thouse who are know the blocksonly feature.
19:41:28 <sdaftuar> wumpus: i forgot to mention why i brought this up
19:41:42 <gleb> I'll try to simulate how "half of nodes turning blocksonly" or something like that might affect tx relay latency or compact blocks relay latency, let me know if you have other metrics in mind
19:41:43 <sdaftuar> i encountered a random blocksonly listening node when testing something recently, and i was surprised
19:42:11 <wumpus> sdaftuar: that's very anocdotal but yeah :)
19:42:13 <MarcoFalke> Could a short-term fix be to disconnect a random outbound if all outbounds are blocksonly?
19:42:15 <sdaftuar> wumpus: so i don't know how common it is now, but when i realized we were unhiding the option and recommending it in our docs, i figured its popularity could rise
19:42:25 <jnewbery> kallewoof: https://bitcointechtalk.com/an-introduction-to-bitcoin-core-fee-estimation-27920880ad0 :)
19:42:34 <warren> I had been running a blocksonly node for years
19:42:34 <wumpus> if you find one by accident then it's probably more common
19:43:02 <jonasschnelli> MarcoFalke: wouldn't that require a new service flag (or a split of NODE_NETWORK / RELAY)?
19:43:08 <jonasschnelli> (to work reliable)
19:43:30 <warren> not "reliable" but at least you could be told all your peers aren't relaying
19:43:32 <sdaftuar> jonasschnelli: right now, nodes running with -blocksonly achieve that by setting the fRelayTxes=false in the VERSION message
19:43:35 <sdaftuar> so we can detect it on connection
19:44:08 <wumpus> right, you can detect it for connected nodes through the version message
19:44:36 <jonasschnelli> I see. So we could just disconnect fRelayTxes=false up to a certain threshold
19:45:21 <wumpus> for outbound only I hope
19:45:53 <sdaftuar> i wonder if the dnsseeds people run are tuned to look for this?
19:45:58 * sdaftuar stares at all of you
19:46:10 * jonasschnelli checking....
19:46:17 <sipa> i doubt it
19:46:54 <jonasschnelli> it's not checking the fRelayTxes
19:47:27 <sdaftuar> matt seems skeptical this is a good idea, anyway i just wanted bring up the topic so others can think about it
19:47:35 <wumpus> I'm not sure it should filter out all nodes that are blocksonly
19:47:51 <cfields> sdaftuar: why would it? Seeds are primarily hit during IBD.
19:48:23 <sdaftuar> cfields: i think in practice we hit the seeds a lot now, actually, but i think that's also a bug
19:48:33 <wumpus> the main task of the P2P network is relaying blocks, after all, sure there need to be nodes that relay transactions, but I think it's a bad idea to just ignore nodes that don't
19:48:45 <wumpus> cfields: right
19:49:00 <sdaftuar> wumpus: yeah i think we can tolerate some level of blocksonly peers, but not all of our outbounds
19:49:01 <wumpus> for IBD it definitely doesn't matter
19:49:04 <sdaftuar> i'm not sure where to draw the line
19:49:06 <wumpus> sdaftuar: agreed
19:49:27 <wumpus> (it ignores transactions during IBD, after all)
19:50:05 <cfields> sdaftuar: agree that it's something that should be handled, though.
19:50:12 <MarcoFalke> Heh, so it would be helpful to connect to blocksonly in IBD
19:50:12 <cfields> s/handled/tolerated/
19:50:58 <kallewoof> 50/50 for tx relaying nodes, anything-goes for blocksonly nodes?
19:51:18 <luke-jr> MarcoFalke: well, except some people use blocksonly as a kind of "super low bandwidth mode" :P
19:51:38 <wumpus> well if you enable pruning too...
19:51:45 <warren> perhaps many using blocksonly would be happy with erlay
19:51:57 <MarcoFalke> I hope they don't accept incoming then
19:52:32 <warren> hope isn't a defense
19:52:49 <MarcoFalke> At least o
19:53:31 <MarcoFalke> https://github.com/bitcoin/bitcoin/blob/master/doc/reduce-traffic.md
19:53:47 <MarcoFalke> Listen is mentioned as second step, blocksonly as 4th
19:54:20 * MarcoFalke getting tea ...
19:54:47 <warren> I dunno about other people but I had blocksonly=1 on one node listening even before it was in a stable release.
19:55:11 <gleb> warren: You should tell sdaftuar your ip, perhaps you were the node he connected to :)
19:55:30 <warren> I'd be surprised if I was the only
19:56:13 * sdaftuar is done with this topic if we want to move on
19:56:52 <MarcoFalke> #action create brainstorming issue to not forget about it
19:56:52 <tryphe> looking for a relabel, and some brainstorming around file handle counts, if possible: https://github.com/bitcoin/bitcoin/pull/16003 :)
19:57:36 <fanquake> Don’t think there are any more proposed topics.
19:57:45 <fanquake> End meeting?
19:58:00 <MarcoFalke> tryphe: What label do you want?
19:58:02 <wumpus> #endmeeting