inamerrata

General background: Bitcoin is a consensus system — it works because there are a set of rules on how Bitcoin transactions work, and everyone agrees on what they are. Changing those rules is called “forking” — when some people want to change the rules in a non-backwards compatible way while others don’t, that results in a new altcoin that follows the changed rules (eg BCH), while Bitcoin’s rules stay the same; and when everyone agrees to change the rules in a backwards compatible way, we have what’s called a soft fork. Most of the interesting development in Bitcoin doesn’t require changes to the consensus rules, but some changes do. In essence, these sorts changes touch the fundamentals of Bitcoin, and thus warrant extra care and attention.

Specific background: The proposed taproot soft fork is something we’ve been working on for quite a while now, and the underlying code changes got merged into the bitcoin codebase a bit over a week ago, just in time for the 0.21 feature freeze. Those changes allow the new taproot rules to be used for testing purposes on the regtest chain, and also on the new signet chain, but do not change how things work on the real, live, Bitcoin network. The idea there is to allow people to check that the major upgrade to 0.21 works as expected and is safe to widely deploy, and only after that’s done worry about the soft fork. Exactly how to activate the soft fork is something of an open question though — while we’ve done a number of them in the past, the last one ended up a bit of a debacle. Back in July, we started discussing activation methods more seriously, and came up with some ideas.

At the time, I wanted to get a better idea of what people thought of the fundamental constraints, so I tried writing up a survey and sent an email to a bunch of smart dev-type people inviting them to fill it in if they were interested:

We seem to be getting to the point where people are making memes about activation methods [0] [1] [2], but I think we’re also still at the point where pretty smart people still have big differences of opinion over technical issues [3].

I feel like we’ve made some progress on ##taproot-activation, but talking with harding after he did his wiki summary of the state of things, I didn’t feel like that was quite getting at the heart of the differences. So I’ve had a go at writing up a survey for (what I think are) the underlying questions that are driving the differences between the proposals. There’s only 10 real questions there, but I’ve added a whole bunch of text that (hopefully) neutrally explains most of the tradeoffs between the choices, hopefully without introducing too much of my own bias. I’m hoping it covers all the choices people are currently favouring, even if they’re “comically moronic”, and, ideally at least, will give some clue as to the tradeoffs people are considering/ignoring that’s leading them to that preference. Ideally the results might indicate where there’s already widespread agreement, what might be worth talking through more, and what productive ways there might be of dealing with any remaining disagreements…

If there’s some important issues / responses the survey doesn’t cater for, that would be good to know. And, obviously, if you’re happy to fill in the survey, that would be awesome

My thought is, assuming the response isn’t “this is a stupid, counter-productive idea”, to post the url at the next weekly core dev irc meeting for a broader but still cluey audience, and post to bitcoin-dev and ##taproot-activation afterwards, and then do something about collating and publishing the results, which might hopefully help promote intelligent discussion vs meme wars…

I’ve bcc’ed people so they don’t get included in replies if they’re not interested; but fwiw the list is […]. . Random collection of people who have participated in recent discussions, might have varying strong opinions on some of the topics, and/or who did bunches of work and who I’d be embarrassed to exclude.

Steve Lee, A. Jonas, and Mike Schmidt helped with drafting and will hopefully help with/do all the work of collating responses; Dave Harding, Russell O’Connor both offered helpful comments that assisted significantly with early drafting. Any remaining stupid counter-productivity is mine of course.

(I’m hoping this survey will help result in a better idea of what to do about activation which will then inform what we actually do. But either way it’s certainly not a “vote by sms now, and whichever answers get the most votes will be your new american idol, uh, taproot activation method” thing, or even a “nope, everyone else voted X, your opinion is unimportant”. Hopefully that didn’t need to be said.)

I sent the survey to about 20 people and got 13 responses (including my own). I figure not identifying who responded or tying responses with people is probably best, since that avoids tying anyone to their opinion from a month or three ago, and thus maybe makes it easier for people to adjust their views to new information and eventually come to an agreement.

If you’re interested in the details around this topic, I think the survey’s worth a read, and I’ve left it open in case anyone wants to fill in their own answers.

The results turned out harder to collate than I expected — mostly because google’s CSV export isn’t that great when you have “choose as many as you like” questions that each have full sentences for the answers, but also because there were fewer obvious patterns than I expected. But anyway.

Results for the first set of questions, about activation via enforcement by a supermajority of hashpower, ended up being:

• What do you consider a reasonable threshold for activation by hashpower supermajority?
  • Eight people selected 90%-95%, 85%-95%, 90% or 95%
  • Four people selected 60%/70%/75% as the lower bound and 95% as the upper
  • One person selected just 75%
• If everything goes well, how long will it take miners to upgrade and enable signalling for activation by hashpower supermajority?
  • Six people chose “up to 12 months
  • Five people chose “up to 3 months”
  • One person chose “up to 6 months”
  • One person didn’t answer
• How long should it be at minimum between software release and activation actually taking effect?
  • Five people chose “6 retarget periods” (3 months)
  • Four people chose “4 retarget periods” (2 months)
  • Two people chose “2 retarget periods” (1 month)
  • One person didn’t answer
  • One person gave a free form answer: “Unpopular opinion: between 3 and 6 months. Need to give time for users to update too. Otherwise miners can do play dirty (I suppose but I haven’t thought deeply about this). “

For the “flag day activation” section, the answers were:

• What concerns do you think should be taken into account in choosing a flag day?
  • Eleven people chose “plenty of people will enforce the rules, after the flag day, though maybe not the flag day itself”
  • Eleven people chose “sufficient number of people enforcing the flag day that ignoring it will be economically unviable”
  • Seven people chose “almost every node will enforce the flag day”
  • Five people chose “not introducing precedents that will cause problems”
  • Four people chose “soon enough to keep development momentum”
• How long away should the flag day be?
  • Seven people found 12 or 18 months acceptable. Of those, six found 12, 18 or 24 months acceptable, and two of them also considered 36 or 48 months acceptable.
  • One person found only 6 or 12 months acceptable.
  • One person found only 36 or 48 months acceptable.
  • Two people only indicated 12 months.
  • One person only indicated 18 months.
  • One person chose “never”.
• When should we decide on the flag day?
  • Nine people chose answers that depend on uptake (seven wanted to see how users upgrade; six wanted to see how miners behave; five wanted to be sure a flag day is actually needed)
  • Four people chose “before the first activation attempt”, though two of those also wanted to see how users upgrade, and one also selected the “never” option (not the same person that chose “never” for the previous question)
• How should disagreement on a choice of flag day be resolved?
  • Six people indicated “whatever the BIP authors and core maintainers agree on is fine”
  • Four people indicated “only do a flag day if there’s clear consensus” (no overlap with the previous six)
  • Four people chose “Pick my answer (or a later one)”; one of those also chose “Pick the average answer”
  • There were a bunch of free form answers as well: “Pick a reasonable answer”, “6 months or 1 year only, unless there’s a clear reason more time is required (e.g., fixing timestamp overflow bugs in far future). Anything in between 6 months and 1 year is bikeshedding, anything less than 6 months is too fast, and anything further than 1 year is too far out”, “Pick the Nth percentile wait, where N is pretty high. I’m fine waiting longer, I just want the flag day locked in”, “rough consensus and running code”, “A thought I had during the segwit2x debacle is that I don’t think there is consensus for playing games of chicken with consensus. I think taproot is a good idea, but I don’t think chain splits are, and I think we should take our time to be careful about deploying consensus changes in a way that is not likely to produce a chain split. No one has any reason to think that taproot won’t activate, so let’s not rashly move forward in a way that could provoke a chain split due to errors or oversights.”
• How will we know there is community support for a flag day by default?
  • Ten people chose “enough time for reasonable objections to be reported, but none have been”
  • Nine people chose “uptake of software supporting hashpower supermajority activation”
  • Eight people chose “we see manual signalling” (everyone chose at least one of these three responses, except for one person who only entered a free form response)
  • Five people chose “uptake of opt-in activation”
  • Four people chose “we see price information”
  • Four people chose “we already do”
  • One person chose “we never will” (along with other options)
  • There were also a couple of free form additions: “Every softfork is a user-activated softfork”, “when anyone on reddit/reading coindesk would understand that there are no objections, and understand the care that went into design.”, “Know it when we see it, but should only be used if necessary”
• How should users opt-in to flag day activation?
  • Seven people chose “never opt-in, should be the default for everyone once community support is established”
  • Five people chose “upgrading to a new (optional) version of bitcoin core” — eleven people chose at least one of these two options
  • Four people chose “setting a configuration flag”
  • Four people chose “an alternative forked client”
  • Six people chose “editing the source and recompiling”, however all of those people also chose at least one other option
  • The only free form comment was: “Configs can be set wrong accidentally and is hard to test, a bit harder to run wrong binary for a long time. (speaking against config flag option)”
• Signalling a flag-day activation?
  
  • Six people chose “mandatory signalling only when bringing activation forward”
  • Five people chose “always require signalling prior to activation” (nine people chose one of these options)
  • One person chose “never mandate signalling”
  • Two people gave no answer, and one just gave the free form response: “No opinion at this time”
  • Remaining free form comments were: “I think forced signaling flag days are really only interesting for two phase deployments where the first phase doesn’t know about the flag day but hasn’t timed out, and where the flag day is far enough out that disruption from it can be minimized (e.g. miners can get told to at least adjust their versions)”, “We want mandatory signalling to bring to ensure activation on nodes that do not enforce the flag day. The “proposed update to BIP 8 [3]” is a very good solution to this.”

I don’t think the “opinion weighting” answers ended up being very interesting:

• How informed are your opinions?
  • Six people chose “based on years of experience with multiple activations”
  • Two people chose “in depth study of bitcoin activation”
  • Four people chose “knowledge about other aspects of bitcoin and reading the questions”
  • One person chose “you wanted my opinion, you got it. caveat emptor”.
• How confident are you about your opinions?
  • Eight people chose “right balance of tradeoffs”
  • Three people chose “not very”
  • One person chose “anything else will be a disaster”

Overall free form comments were:

• Your choices for how sure I am are pretty rough. I mean, there probably isn’t anyone with more activation experience than me (though several equal) but no one has activated anything in the current network, no one has activated taproot. etc. A sentiment I hoped to be able to express was support for nested activations like harding’s start now and improve later. Forced activation specifics are likely to be complicated and painful to decide and that decision would be greatly simplified by initial robust deployment. … plus I think there are good odds that forced activation will be unnecessary (esp if its clear that we will use it if needed)– so why serialize getting this stuff activated on figuring out forced activation details? Better to do whatever thing has good odds of getting it activated fast assuming miners cooperate then worry about more dramatic things if they don’t. Without the initial attempt everyone is just guessing– guessing on uptake– guessing on miner behaviour– etc. Plus people who want more aggressive and less aggressive approaches differ a lot based just on how pessimistic they are about miners, a question that will be resolved by seeing what miners do. The primary counter argument to this approach is that if we don’t plan for a flagday in advance there is a risk that the moment miners drag their feet at all, the pitchforks will come out and the least reasonable people will immediately move forward with a 30 day flagday or whatever. I think that this can be avoided by the author of the parameters making a clear statement of intent. That if users adopt but miners holdback the intention will be to flag day and we’ll start discussing the details of that in 3 months… or something like that.
• I can’t answer about how “correct” my opinions are… My feelings about activation methods are strongest when it comes to the narrative around them, and least strong when it comes to the specifics (provided that they are reasonably in line with a good narrative). I think we could take almost any activation method and tell a story about it that is terrible — miner-activation means that miners dictate the rules; user-activation means that developers dictate the rules, etc. In my view the most important thing here is to have as strong a sense as reasonably possible that no one is opposed to the consensus change and that activation is very likely to not cause a chain split (at the time of activation or down the road). How we get there is a matter of debate and discussion, but if we can agree that those two principles are paramount and other issues are secondary, then I think I’d be on board with any number of proposals that are crafted around such a narrative.
• To summarize my current thinking:
  • deploy bip8(false) in Bitcoin Core
  • If it becomes clear that the miners use their veto to prevent activation let users coordinate on a flag day. In order to opt-in to flag day activation (bip8(true)) users should create their own fork of Bitcoin Core. For this to work properly, it would be ideal to use Anthony Town’s suggested changes to BIP 8. If the users fail to cooperate and the softfork doesn’t activate, then that’s fine too, but maybe the softfork wasn’t useful enough then. We can propose another softfork that hopefully gets more user support (sigagg, etc.).
  • Thanks for making this! Looking forward to see what comes out of it.
• I think that we’ve micro-managed soft-forking patterns as the biggest threat that bitcoin faces and worthy of all the attention and fuss… There are bigger problems and challenges facing bitcoin centralization that we can work on (e.g., proliferation of storing funds in custodial wallets), but that feel more outside our control, so we focus on something that is in our control. I think any reasonable choice that allows us to ship small soft-forks when recommended by devs and users is the right tradeoff when we’re already suffering from other centralization vectors.
• Perhaps not a disaster for Taproot, but some deviation could set very harmful precedents. Looking at upgrade history, I worry we ought to set a minimum 1 year after software release before starttime, but the question only asked about a *minimum*.
• Waiting too long means hats come out of closets and we get something much less organised and safe. Let’s get a flag day set far enough in the future and move on with our lives

Hopefully the length of that summary when there’s only 13 responses serves as a good explanation why I didn’t summarise this earlier, or try getting more responses first…

One thing I’ve been thinking more and more is that the exact activation method isn’t really what’s important. I think that the whole “BIP 9 allows an activation attempt to fail” has been somewhat misleading — while it’s technically true, the fact that the activation could succeed at all is more important, and that possibility implies that we have to be absolutely confident that a deployment is definitely worth activating before we risk activation. And if we are absolutely confident it’s worthwhile, then so is everyone else, and we will eventually activated it one way or another, and the details of exactly how it happens are just that: details. And while details matter, it’s much more important to make sure the idea is sound, and to do that first — so I think it’s actually more of a big deal that we’ve in addition to all the review and unit tests and guides and explanations we’ve now also got taproot activated on signet which should make third-party development feasible, and in some sense allow an extra layer of testing.

But anyway, as far as activation parameters go, your takeaways from the above might be different, but I think mine are:

• Activation threshold should stay at 95% (or at most be reduced to 90%)
• We don’t really know how quickly miners will react; so hope for a quick response within a few months, but plan for it taking up to a year, even if everything goes well
• Setting the startheight to be about a month or two worth of blocks away is probably about right (along with a retarget period each of STARTED and LOCKED_IN this gets at least two or three months of deployment time before activation is possible)
• Almost everyone is open to the idea of a flag day in some circumstances
• If there’s a flag day, we should expect it to be a year or two away (though it might turn out sooner than that, or later)
• There probably isn’t support for setting a flag day initially (only 4/13 support choosing a day early enough; only (a different) 4/13 think we already know there’s sufficient community support; 9/13 want to see adoption of hashpower-based activation to establish there’s consensus for a flag day)
• Almost everyone wants to see as many nodes as possible enforce the rules after activation
• Most people seem to be willing to accept bringing a flag day forward by mandatory signalling
• There’s not a lot of support for opting-in to flag day activation by setting a configuration option

So I think to me that means:

• Initial activation parameters should be included in a minor update release (eg, version 0.21.1 or 0.21.2) and be something like:
  • lockinontimeout = false
  • startheight = release height + 3 retarget periods, rounded up (to get a two or three month delay before activation is possible)
  • timeoutheight = startheight + 209,664 blocks (4 years worth — in case the 12-24 months estimates turn out to be too low)
  • threshold = 95% (no point changing it)
• We then see what happens — if miners activate it within the first three or 12 months: great. If they don’t, we’ll have more data, and use that to refine the deployment strategy. For example, if miners have been having legitimate problems deploying the new software, we’ll help them fix that; if not, and there’s plenty of uptake of the new software and other support, we’ll run some numbers on the rate at which users are upgrading, and pick a date for a flag day based on that.
• When we work out what flag day is best supported by the data (suppose for the sake of example that it’s startheight + 18 months, to be roughly in line with what people expected per the above), then we’d update the deployment parameters to:
  • lockinontimeout = true
  • startheight = unchanged
  • timeoutheight = startheight + 78,624 blocks (18 months’ worth)
  • threshold = unchanged
• The updated activation parameters should be backported for each major version (eg, if startheight is March 2020 and timeoutheight is in September 2021, that might be 0.21.5, 0.22.3, and 0.23.1, and already included in master by the time 0.24.0 is branched off)

This is more or less the “gently discourage apathy” approach though with a longer initial timeout.

Note that with 13 version bits reasonably available for use (BIP320 reserves the remainder, and miners are actively using them), a four year timeout still allows for a new soft fork every four months on average without having to overlap version bits or come up with a new signalling method; which seems likely to be more than sufficient.

Compared to “modern soft fork activation“, I think the main differences are that it plans for an earlier flag day (though only if that’s actually supportable via adoption data), does not include a config parameter for updating to flag day activation but instead requires upgrading to a new minor release (unavoidable given the flag day isn’t decided in advance and manually setting the flag day would be too easy to get wrong, which risks breaking consensus), and requires mandatory signalling if the flag day occurs.

If you want to maximise the number of nodes that will enforce the rules should a flag day occur, but also only choose the flag day after an initial activation attempt is already widely deployed, then you have no choice but to make signalling mandatory when the flag day occurs. I think it’s a good idea to do a little more work to minimise the costs that mandatory signalling might impose on miners, so have proposed some updates to BIP 8 to that effect — one to not require signalling during LOCKED_IN, and one to reduce signalling during MUST_SIGNAL from 100% of blocks down to the threshold figure; I think the latter also is potentially somewhat protective against miner gamesmanship, as noted in the link. That’s still not zero-impact on miners in the way the “modern soft fork activation” approach is, but I think it’s near enough.

Apart from that, I think the current BIP 8 spec/code should more or less work for the above aleady.