19:00:35 <wumpus> #startmeeting
19:00:35 <lightningbot> Meeting started Thu May 17 19:00:35 2018 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:35 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:00:37 <jonasschnelli> hi
19:00:40 <sipa> hi
19:00:41 <promag> hi
19:00:43 <jamesob> howdy
19:01:11 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
19:01:20 <kanzure> hi.
19:01:48 <wumpus> proposed topics: 0.16.1 I guess
19:01:50 <jimpo> hi
19:01:51 <cfields> hi
19:02:20 <wumpus> #topic High priority for review
19:02:31 <sipa> i'd like to add #13142
19:02:33 <gribble> https://github.com/bitcoin/bitcoin/issues/13142 | Separate IsMine from solvability by sipa · Pull Request #13142 · bitcoin/bitcoin · GitHub
19:03:14 <wumpus> we merged #10740 this week, #12254 #10757 #13011 #13097 #12979 #12634 are left
19:03:16 <jnewbery> hello
19:03:16 <jonasschnelli> I'd like to add #12196
19:03:19 <gribble> https://github.com/bitcoin/bitcoin/issues/10740 | [wallet] `loadwallet` RPC - load wallet at runtime by jnewbery · Pull Request #10740 · bitcoin/bitcoin · GitHub
19:03:23 <gribble> https://github.com/bitcoin/bitcoin/issues/12254 | BIP 158: Compact Block Filters for Light Clients by jimpo · Pull Request #12254 · bitcoin/bitcoin · GitHub
19:03:30 <gribble> https://github.com/bitcoin/bitcoin/issues/10757 | RPC: Introduce getblockstats to plot things by jtimon · Pull Request #10757 · bitcoin/bitcoin · GitHub
19:03:31 <gribble> https://github.com/bitcoin/bitcoin/issues/13011 | Cache witness hash in CTransaction by MarcoFalke · Pull Request #13011 · bitcoin/bitcoin · GitHub
19:03:33 <gribble> https://github.com/bitcoin/bitcoin/issues/13097 | ui: Support wallets loaded dynamically by promag · Pull Request #13097 · bitcoin/bitcoin · GitHub
19:03:36 <gribble> https://github.com/bitcoin/bitcoin/issues/12979 | Split validationinterface into parallel validation/mempool interfaces by TheBlueMatt · Pull Request #12979 · bitcoin/bitcoin · GitHub
19:03:36 <wumpus> there's quite a lot of things on the list yet, should we also remove something?
19:03:37 <gribble> https://github.com/bitcoin/bitcoin/issues/12634 | [refactor] Make TransactionWithinChainLimit more flexible by kallewoof · Pull Request #12634 · bitcoin/bitcoin · GitHub
19:03:40 <gribble> https://github.com/bitcoin/bitcoin/issues/12196 | Add scantxoutset RPC method by jonasschnelli · Pull Request #12196 · bitcoin/bitcoin · GitHub
19:04:01 <BlueMatt> #13011 looks merge-ableish
19:04:04 <gribble> https://github.com/bitcoin/bitcoin/issues/13011 | Cache witness hash in CTransaction by MarcoFalke · Pull Request #13011 · bitcoin/bitcoin · GitHub
19:04:10 <jtimon> hi
19:04:28 <BlueMatt> I dunno if #12254 should stay on there, there's now discussion of the bip on the ml so its gonna be some time yet, I think
19:04:30 <BlueMatt> jimpo: thoughts?
19:04:31 <gribble> https://github.com/bitcoin/bitcoin/issues/12254 | BIP 158: Compact Block Filters for Light Clients by jimpo · Pull Request #12254 · bitcoin/bitcoin · GitHub
19:04:40 <jonasschnelli> What is the maxlen for high-prio-list?
19:04:48 <BlueMatt> 1 per regular contributor :p
19:04:55 <wumpus> ok added #12196 #13142
19:04:58 <gribble> https://github.com/bitcoin/bitcoin/issues/12196 | Add scantxoutset RPC method by jonasschnelli · Pull Request #12196 · bitcoin/bitcoin · GitHub
19:05:00 <gribble> https://github.com/bitcoin/bitcoin/issues/13142 | Separate IsMine from solvability by sipa · Pull Request #13142 · bitcoin/bitcoin · GitHub
19:05:03 <jonasschnelli> Thanks wumpus
19:05:09 <jonasschnelli> Agree with BlueMatt about 12254
19:05:17 <wumpus> yes, what BlueMatt says, though PRs that are not actively updated should be removed
19:05:47 <wumpus> agree, removed 12254
19:06:04 <jtimon> I was expecting https://github.com/bitcoin/bitcoin/pull/10757 to be merged soonish and thus go out of the list
19:06:16 <BlueMatt> topic: 0.16.1
19:06:17 <promag> and I guess 13097 can be merged after jonasschnelli review
19:06:18 <wumpus> something that is still being discussed on the mailing list certainly doesn't belong in the blocker slist
19:06:36 <jonasschnelli> Yes. 13097 is in review here...
19:06:42 <jimpo> Can I get #13243 then so progress continues? :-)
19:06:44 <gribble> https://github.com/bitcoin/bitcoin/issues/13243 | Make reusable base class for auxiliary indices by jimpo · Pull Request #13243 · bitcoin/bitcoin · GitHub
19:06:48 <MarcoFalke> #12979 needs a rebase
19:06:50 <sipa> sgtm
19:06:51 <gribble> https://github.com/bitcoin/bitcoin/issues/12979 | Split validationinterface into parallel validation/mempool interfaces by TheBlueMatt · Pull Request #12979 · bitcoin/bitcoin · GitHub
19:06:59 <BlueMatt> MarcoFalke: yup, doing now
19:07:11 <BlueMatt> i was waiting on sdaftuar's review so I could take nits at the same time
19:07:36 <jonasschnelli> unicorn for 10757
19:07:41 <sipa> great.
19:07:46 <BlueMatt> topic: replacing github
19:07:50 <jonasschnelli> heh
19:07:51 <promag> wumpus: fyi 13063 on high priority after 13097 merge
19:08:12 <BlueMatt> topic: replacing github (not entirely unserious)
19:08:17 <jonasschnelli> don't make queues for highprio list. :)
19:08:27 <wumpus> promag: you already have one on the list!
19:08:56 <promag> wumpus: right, after 13097 merge
19:09:05 <wumpus> due to the length of the list I'm going to have to enforce one PR per person, sorry
19:09:29 <wumpus> #topic 0.16.1
19:09:44 <BlueMatt> huh? we always enforce one per person (well, one nomination per person, you can nominate someone elses')
19:09:55 <BlueMatt> I think we just need to finish backports and tag for 0.16.1rc1, no?
19:10:00 <wumpus> mostly #13253
19:10:01 <gribble> https://github.com/bitcoin/bitcoin/issues/13253 | [0.16] Further Backports by fanquake · Pull Request #13253 · bitcoin/bitcoin · GitHub
19:10:24 <wumpus> BlueMatt: we had multiple theuni PRs on there at some point :)
19:10:36 <BlueMatt> well those got removed, and cfields confirmed that was ok
19:11:27 <wumpus> there's also three issues marked 0.16.1: #13110 #12646 #12337
19:11:28 <gribble> https://github.com/bitcoin/bitcoin/issues/13110 | 0.16.0 bitcoin-qt: "Assertion `copyFrom failed" during launch · Issue #13110 · bitcoin/bitcoin · GitHub
19:11:29 <gribble> https://github.com/bitcoin/bitcoin/issues/12646 | Assertion failure during rescan · Issue #12646 · bitcoin/bitcoin · GitHub
19:11:31 <gribble> https://github.com/bitcoin/bitcoin/issues/12337 | 0.16 Shutdown assertion · Issue #12337 · bitcoin/bitcoin · GitHub
19:11:45 <wumpus> not sure whether they are critical or can just be bumped to 0.16.2 or so
19:11:57 <sipa> we have one issue marked 0.15.2 which i don't understand
19:12:39 <wumpus> I don't know either, but at least that's not a blocker for 0.16.1
19:12:40 <BlueMatt> jonasschnelli: was the last to comment on 12337 " I'll try to find a solution for this..."
19:12:43 <MarcoFalke> wumpus: The assertion is a regression if I am not mistaken
19:12:51 <MarcoFalke> 13110
19:13:04 <MarcoFalke> wasn't 12337 fixed?
19:13:10 <jonasschnelli> Will look into 12337...
19:13:24 <wumpus> I proposed a fix in 13110 and it apparently worked
19:13:41 <cfields> yes
19:14:08 <jonasschnelli> wumpus: can you PR https://github.com/bitcoin/bitcoin/issues/13110#issuecomment-385708583?
19:14:22 <wumpus> jonasschnelli: sure
19:15:51 <wumpus> so that leaves 12646
19:16:59 <jonasschnelli> maybe jnewbery can look into 12646?
19:17:10 <wumpus> anyone want to look int othat? or can we just bump it forward if it's not so importent
19:17:43 <jonasschnelli> I think its okay to bump this forward (as long as we track it)
19:17:57 <wumpus> yes I don't mean closing it
19:18:09 <jnewbery> Yes, I can look at 12646 (next week)
19:19:02 <wumpus> moved to 0.16.2
19:19:21 <wumpus> other proposed topics?
19:19:28 <BlueMatt> trashing github
19:19:36 <wumpus> #topic Trashing github
19:19:45 <BlueMatt> so it hasnt been working for like 3 weeks now
19:20:02 <BlueMatt> and I'd kinda like to have something self-hosted with better review tools anyway, which I know a lot of people wanted
19:20:12 <jonasschnelli> Should we give it more time?... I'm pretty sure they are aware of it
19:20:13 <sipa> there was some suggestiin (was it on twitter) to use gitlab
19:20:29 <BlueMatt> so I figured we should do a "do people think its actually a good idea to switch to something self-hosted" semi-poll
19:20:33 <wumpus> gitlab seems ok
19:20:34 <BlueMatt> or we could switch to gitlab
19:20:40 <jonasschnelli> BlueMatt: what alternatives would you propose?
19:20:43 <jtimon> I like gitlab
19:20:45 <sdaftuar> seems to me like it's way harder to get it right hosting ourselves
19:20:47 <BlueMatt> though gitlab seems to have no better review tools than github
19:20:55 <wumpus> self-hosted I don't know, who is going to babysit this, monitor it and apply security patches etc?
19:20:57 <sipa> it would  e really cool if all pr comment history was in git too
19:20:57 <BlueMatt> sdaftuar: oh? I mean I kinda disagree
19:21:07 <sdaftuar> dude we can't even keep the computers in our office running
19:21:09 <jamesob> self-hosted is very risky and potentially time-consumptive IMO
19:21:10 <BlueMatt> sipa: does gitlab do that?
19:21:17 <BlueMatt> sdaftuar: bitcoincore.org does just fine....
19:21:18 <sipa> BlueMatt: i have no clue
19:21:25 <wumpus> if no one is, it's going to become worse, not better, at least Github has a dedicated paid team
19:21:35 <sdaftuar> definitely agree with wumpus
19:21:37 <cfields> general nack, self-hosting issues aside, Github's network effect is too strong imo.
19:21:39 <sipa> blockstream uses gitlab internally, which seems to work fine (pribably due to people maintaining it)
19:21:54 <BlueMatt> i mean we could do self-hosted gitlab
19:22:18 <MarcoFalke> what advantage does that give, BlueMatt?
19:22:37 <jtimon> I assume the goal is less unicorns?
19:22:40 <jnewbery> cfields: +1
19:22:52 <wumpus> someone from github promised to look into the unicorn issue, maybe we should give them some more time
19:22:53 <MarcoFalke> gitlab also does hostign for free
19:22:59 <jimpo> Cursory internet search turned up reviewable.io, which is like a hosted layer on top of GitHub
19:23:01 <sipa> jnewbery, cfields: what do you suggest instead? waiting until github fixes it?
19:23:06 <jimpo> free for public repos
19:23:08 <BlueMatt> MarcoFalke: we can do our own security additions like putting the pr and comment history in git
19:23:13 <cfields> I can't be the only one who gets irrationally frustrated when the code I want to mess with is on BitBucket..
19:23:14 <BlueMatt> and have stuff that verifies it
19:23:33 <sipa> cfields: we'd mirror on github of course
19:23:39 <jnewbery> In the absence of something better, we should continue nagging them
19:23:54 <wumpus> cfields: yes, only players like freedesktop can really afford to host on separate infrastructure, for smaller projects the lack of network effect (and having to register separately) is bad
19:24:04 <cfields> sipa: all things considered, yes, I'd say waiting it out makes the most sense.
19:24:17 <BlueMatt> jnewbery: I think we *do* have ideas for better things
19:24:22 <cfields> I think I'm in the minority there, though :)
19:24:22 <jonasschnelli> Moving away from GitHub seems meh,... especially for a self-hostes solutions... looks after centralizing development
19:24:25 <jimpo> Who has reached out to GitHub and through what channel so far?
19:24:26 <BlueMatt> the self-hosting question is more a "will it be maintained" question
19:24:29 <BlueMatt> not "will it be better"
19:24:33 <jtimon> I'm ok with both people working on a gitlab instance and people nagging github devs
19:24:38 <sipa> cfields: i'moretty unconfortable with the fact that network effect is making us stick with a specific provider, even in the oresence of obvious issies
19:25:03 <sipa> of course, it's not like we could migrate quickly anyway
19:25:06 <wumpus> 'will it be maintained' is really important though to not end up blaming each other
19:25:06 <jamesob> how much effort will, e.g., DoS protection be for something self-hosted?
19:25:17 <wumpus> at least now we can blame github people :)
19:25:20 <sipa> haha
19:25:21 <jtimon> we could perhaps save money on travis workers by using gitlab-CI too?
19:25:45 <wumpus> jamesob: exactly...
19:25:48 <BlueMatt> jamesob: dos protection is 6$/month
19:25:50 <BlueMatt> and works perfectly
19:25:52 <cfields> sipa: it's worth considering that the 0.16.1 issues might've never been reported if not for Github's issues
19:26:11 <jnewbery> BlueMatt: Yes, we have ideas, but that's different from something that's actually running. I don't have any interest in maintaining a self-hosted solution, and I don't think it's worth anyone else's time doing it either
19:26:17 <sipa> cfields: that's a good point
19:26:32 <wumpus> we could still use github for *issues*
19:26:39 <wumpus> gitlab would be for patches and review
19:26:49 <jtimon> jnewbery: yeah, I'm personally not interested in maintaining it either
19:26:56 <wumpus> doesn't necessarily need to be the same place
19:27:24 <sipa> let's move back to sourceforge
19:27:26 <wumpus> yes tbh I don't think we should change the issue reporting URL
19:27:26 <BlueMatt> I'm not a fan of using issues and patches/review being in separate places
19:27:36 <wumpus> I've been spamming that to so many people over the years
19:27:37 <jamesob> BlueMatt: I don't think CloudFlare works with git protocol, so you need to reveal underlying IPs: https://stackoverflow.com/questions/31817004/git-push-not-working-after-using-cloudflare-reverse-proxy
19:27:39 <jonasschnelli> if the unicorns is the showstopper, then better mirror github PR with comments > 20 via API with comment through API function
19:27:46 <jtimon> or just everything on gitlab, but since we have the github mirror, we will see issues created there by people who missed that the project moved
19:27:49 <wumpus> I don't really want to move it anywhere else. THe unicorns are only an issue for code review.
19:27:55 <BlueMatt> jamesob: bitcoincore.org does not use cloudflare (and costs 6$/month), cloudflare sucks ass
19:27:55 <jnewbery> I also think that the network effect thing is important. What percentage of new contributors/issue reports would we lose if we weren't on github?
19:28:11 <BlueMatt> jamesob: (and that's for redundant providers)
19:28:41 <BlueMatt> jonasschnelli: I'd actually be much happier with github if we had a client-side api-based cli-only github interface
19:28:51 <BlueMatt> (that verified eg pgp signatures on comments)
19:28:52 <wumpus> there is a github cli interface
19:28:54 <jtimon> wumpus: well the main point of using github is for code review, no?
19:28:57 <kanzure> email seems to work for long reviews (diffs)
19:29:09 <jonasschnelli> BlueMatt: that seems easyer then installing a gitlab solution on a custom server
19:29:21 <jonasschnelli> *easier
19:29:28 <BlueMatt> jonasschnelli: its the difference between building a whole app and installing one
19:29:30 <BlueMatt> so...no, not really
19:29:31 <jimpo> jnewberry: Who has reached out to GitHub and through what channel so far?
19:29:33 <moneyball> I am happy to follow-up with GitHub to try and accelerate a fix. Can someone provide me background info on the existing communication we have with GitHub?
19:29:48 <jnewbery> I've contacted Github support. I don't know who else has
19:29:59 <jtimon> I think someone reached to them on twitter too
19:30:01 <wumpus> I have contacted them through tthe contact site, and was told by support that many others have
19:30:05 <moneyball> Is there an open issue # that I can reference?
19:30:09 <jnewbery> jtimon: that was me
19:30:14 <jimpo> I can try asking someone I know that used to work there supporting open source projects
19:30:19 <jnewbery> moneyball: I'll forward you the email thread
19:30:23 <BlueMatt> moneyball: a few folks have emailed support@github, which historically has always gotten a response, some other projects were posting responses they got where they were saying "we dont actually know what change we made that triggered these issues, hold on"
19:30:26 <BlueMatt> buts its been like 3 weeks ago
19:31:02 <moneyball> jnewbery ok great i'll use that as context and follow-up
19:31:16 <moneyball> jimpo we can coordinate our efforts if you'd like
19:31:37 <jimpo> yeah, we can chat about it outside the meeting
19:31:56 <BlueMatt> ok, so it sounds like consensus is "stick with the broken shit we have now" :/
19:32:02 <BlueMatt> or at least no consensus on moving to something else
19:32:14 <wumpus> feel fre to set up something better and convince us to use it
19:32:26 <sipa> yeah i believe it will require someone to set up a demo
19:32:40 <wumpus> if not, this is just empty talk, we *have* nothing better
19:32:54 <wumpus> any other topics?
19:32:59 <MarcoFalke> And a plan to transition all open patches to the new review system?
19:33:01 <sipa> right; the question is whether we should look into alternatives
19:33:10 <sipa> not so much whether we should or shouldn't move
19:33:33 <wumpus> right
19:33:38 <cfields> iirc some alternatives support oath login via github
19:33:47 <BlueMatt> yea, it seems like lack of consensus to move even if we find something good
19:33:53 <cfields> that would go a long way towards shutting me up
19:33:59 <BlueMatt> which was mostly why I brought it up
19:34:13 <wumpus> I'm open to being convinced to use something else, if it's really better
19:34:38 <jimpo> cfields: I agree a code review tool with GitHub integration (where main repo is still hosted there) is ideal
19:35:13 <jnewbery> BlueMatt: If there was something else better running now AND there was a way to migrate all state AND we wouldn't lose contributors by switching AND we have someone committed to maintaining it, then I'd be open to it. Without that, I think it's a non-starter.
19:35:28 <jonasschnelli> Something like https://github.com/piotrmurach/github_cli seems a better start to deal with the unicorns
19:35:54 <wumpus> jonasschnelli: yes, I plan to look into github cli commands as well, there's a few (also "hub" IIRC)
19:36:07 <MarcoFalke> Agree with jnewbery. I am open to switching, but not without solving the transition issues first.
19:36:31 <wumpus> we'd end up with parallel infrastructure for a while anyway
19:36:34 <jtimon> well BlueMatt I think it would be easier to get consensus to move to something else if somebody had it working with CI and all
19:36:44 <jnewbery> topic request: separate wallet from node (#10973)
19:36:49 <gribble> https://github.com/bitcoin/bitcoin/issues/10973 | Refactor: separate wallet from node by ryanofsky · Pull Request #10973 · bitcoin/bitcoin · GitHub
19:37:04 <jonasschnelli> Moving away from Github just because of load issues for three weeks seems insane...
19:37:08 <BlueMatt> jtimon: it sounds like definite "no", so I'm not gonna spend time looking into it
19:37:23 <jtimon> in the meantime, prs with many comments get the unicorn and we have to try again until it loads
19:37:27 <wumpus> #topic separate wallet from node (jnewbery)
19:37:28 <BlueMatt> jonasschnelli: there are way more reasons people dont like github
19:37:33 <sipa> jonasschnelli: not being able to move away from github just because of network effect seems scary...
19:37:43 <jamesob> the upside is that this is an additional incentive to keep PRs small :)
19:37:46 <jtimon> BlueMatt: fair enough
19:37:52 <cfields> jonasschnelli: I agree. But I think the frustration comes from the helplessness that it's brought to light.
19:37:53 <wumpus> this discussion is starting to repeat itself
19:38:04 <jonasschnelli> sipa: that is a point we should take into consideration,.. but stop focusing on load issues
19:38:13 <sipa> ok, next topic it seems
19:38:38 <BlueMatt> oh, I have 2 more topics....
19:38:53 <wumpus> I don't think there's realistically any chance of anything replacing github until someone sets up a feasible alternative and shows us that it is better
19:38:55 <sipa> jnewbery: your topic
19:38:57 <jonasschnelli> (#topic separate wallet from node (jnewbery))
19:39:06 <BlueMatt> sipa: well I'm gonna look into having a cli app that checks signatures off github api comments/etc
19:39:09 <jnewbery> #10973 is a big PR, but I think it's very worthwhile
19:39:12 <BlueMatt> cause I think that would alleviate a lot of it
19:39:14 <gribble> https://github.com/bitcoin/bitcoin/issues/10973 | Refactor: separate wallet from node by ryanofsky · Pull Request #10973 · bitcoin/bitcoin · GitHub
19:39:25 <jnewbery> jamesob and I have both reviewed now, but it requires continual rebase
19:39:52 <jnewbery> There are a lot of PRs competing for high priority for review, but I think it'd be great to make some progress on this one
19:40:03 <promag> place in high priority this week?
19:40:07 <jamesob> I'll start a round of manual testing if we can get another reviewer or two
19:40:10 <jimpo> can it be broken up at all?
19:40:14 <jnewbery> so, next steps would be: concecpt ACK/NACKs
19:40:25 <BlueMatt> sounds like a high-priority-for-review nomination?
19:40:31 <jnewbery> and if people think it's too much to review in one go, ryanofsky is happy to split
19:40:37 <jimpo> 1500+ lines is too big IMO
19:40:41 <wumpus> oh no, not more high priority for review
19:40:48 <jonasschnelli> jnewbery: this is orthogonal of using light client mode to decouple the wallet from the node? right,... it's more architectural?
19:41:07 <jnewbery> jimpo: it's mostly very mechanical changes, but yes it's a large +-loc
19:41:16 <wumpus> is it blocking anything? is it importnat for 0.17?
19:41:25 <jnewbery> there are only a couple of commits that require deep review
19:41:52 <sipa> jnewbery: thanks for bringing it up; big PRs are sometimes unnecessarily scary to dig into
19:42:04 <jnewbery> wumpus: it blocks (but doesn't necessarily have to lead to) process separation
19:42:12 <jonasschnelli> But I guess it's hard/impossible to break it into smaller PRs
19:42:16 <jamesob> jonasschnelli: it makes explicit the bitcoind interface that the wallet relies on, which is in itself useful but also if we want to do any kind of process separation
19:42:19 <jnewbery> jonasschnelli: you're correct
19:42:20 <wumpus> process separation is not something we'll have for 0.17 anyway
19:42:55 <jnewbery> so at this stage it's more of a concept review beg from me, and a poll on whether russ should spend time splitting it up
19:43:14 <jimpo> I'll give it a pass by tomorrow
19:43:38 <jnewbery> The reason I raised it is that because of the frequent rebases, reviews go stale very quickly, and it's now had two ACKs
19:43:41 <jnewbery> thanks jimpo
19:43:50 <ryanofsky> i actually don't think it's hard to split up, first 6 commits seem to group together nicely, with rest of changes more independent
19:44:27 <jnewbery> that's all from me. If 2 or 3 regular reviewers are happy to concept review, I think that's good progress
19:44:52 <jonasschnelli> If we assume the long term goal is process separation (where the wallet will turn into a pure transaction-filtering-thinkg), isn't it, that the coin-selection & signing elements in this interface will get throw away later?
19:45:03 <BlueMatt> topic: unverified-block-message
19:45:15 <BlueMatt> topic: queue drain lock assertions to avoid deadlocks
19:45:42 <wumpus> #topic unverified-block-message (BlueMatt)
19:45:48 <ryanofsky> jonasschnelli, there aren't any coin-selection or signing things in node/wallet interface in that pr
19:46:24 <jonasschnelli> maybe I should review the PR first...
19:46:54 <sipa> BlueMatt: your topic
19:47:01 <BlueMatt> so sdaftuar pointed out an old issue that we never really addressed where if you relay someone a script-invalid block they may announce it to their peers via compact blocks high-bandwidth-mode and then if you for some reason fall back to downloading it via getdata due to short id collision or so (we dont think there is a way to game it), then you'll end up disconnecting that peer for never responding to your request
19:47:25 <ryanofsky> jonasschnelli, yeah, you can just take a look at https://github.com/ryanofsky/bitcoin/blob/pr/wipc-sep/src/interfaces/chain.h to see the interface (first link in PR description)
19:47:43 <jtimon> re 10973 I agree I would preffer a few smaller ones, specially if there's commits that needs deeper review
19:47:43 <BlueMatt> we only came up with two real potential solutions: a "no, I'm not gonna send you that block" message (ie a not-batshit-insane reject message) or a "here is a block that may be invalid, but is valid pow+merkle root ala compact blocks requirement" message
19:47:51 <BlueMatt> or, I guess the second one is a getdata type
19:48:01 <sipa> BlueMatt: we have "notfound" also
19:48:13 <BlueMatt> sipa: isnt that a reject type?
19:48:16 <sipa> no
19:48:42 <wumpus> NetMsgType::NOTFOUND
19:48:46 <jnewbery> *5 chaincoders furiously grep for notfound*
19:48:48 <sipa> it's just a "i can't give you these INVs"
19:48:51 <sdaftuar> oh wow
19:48:52 <BlueMatt> sipa: ah, but we dont use it for blocks
19:48:54 <BlueMatt> only txn
19:48:57 <sipa> ah
19:49:04 <BlueMatt> still, easier would be a "here is a block that may be invalid" as that would remove the ABC in ProcessGetBlockData
19:49:09 <wumpus> the client ignores it just the same though
19:49:19 <cfields> BlueMatt: so, lots of NOTFOUNDs :)
19:49:45 <sipa> BlueMatt: we should have had that before compact blocks, i guess
19:49:56 <BlueMatt> sipa: should have had what?
19:50:05 <BlueMatt> sdaftuar: points out that it can be wholly independant, just a new getdata type
19:50:05 <sipa> a possibly invalid block relay
19:50:17 <sdaftuar> i think we could just add a new BLOKC response type
19:50:21 <sdaftuar> BLOCK_COULDBEBAD
19:50:26 <BlueMatt> well or both or whatever
19:50:40 <sipa> 0xDEADB10C
19:50:42 <sdaftuar> where if someone requests a block but you don't know if its valid, or you know it's invalid, you return a different message containing the block to indicate that
19:50:47 <wumpus> hehe
19:51:07 <BlueMatt> yea, so old nodes would ignore it, and you'd just be sending a new message type
19:51:09 <sdaftuar> currently we would let the peer time us out instead
19:51:09 <sipa> sdaftuar: but only if you know they support such a message
19:51:17 <BlueMatt> or not
19:51:17 <sdaftuar> meh, sure, but not even needed imo
19:51:19 <BlueMatt> doesnt really matter
19:51:28 <BlueMatt> I mean they're about to disconnect you either way
19:51:32 <sipa> fair
19:51:44 <sipa> that makes compatibility really easy
19:52:04 <sipa> i guess... suggest something and write a bip?
19:52:35 <BlueMatt> oh, wait, no, you also want a new getdata type
19:52:43 <BlueMatt> cause otherwise you still need the ActivateBestChain in ProcessGetBlockData
19:52:52 <BlueMatt> which would require negotiation
19:53:02 <sdaftuar> yeah ok
19:53:11 <BlueMatt> so either that, or we start using NOTFOUNDs, I guess
19:53:25 <BlueMatt> I'm not sure what I prefer, so.....thoughts?
19:53:31 <sdaftuar> i think notfounds are worse because of the case where the block might not have been validated either way
19:53:35 <sdaftuar> it complicates download logic
19:53:51 <wumpus> if there is no specific reason to re-use NOTFOUND, a new message is much better imo
19:53:53 <BlueMatt> yea, its not really "notfound" its "I may find this soon"
19:54:08 <BlueMatt> or, really, "I dont currently find this"
19:54:13 <sipa> i don't think we should do protocol design in this meeting
19:54:29 <BlueMatt> well, there's a few options, so...good to ask
19:54:46 <wumpus> not the design but discussing it as a concern is valid
19:54:51 <wumpus> agree BlueMatt
19:55:01 <BlueMatt> obviously requires BIP and whatever else
19:55:05 <wumpus> yes
19:55:55 <wumpus> #topic queue drain lock assertions to avoid deadlocks (BlueMatt)
19:56:09 <BlueMatt> this one is less interesting, i realize now I should just open a pr and people will see it
19:56:14 <BlueMatt> its kinda knotty to describe
19:56:25 <BlueMatt> but, essentially, if you call ABC within a validationinterface callback you're hosed
19:56:31 <wumpus> ok, well only 4 minutes t ogo anyhow
19:56:35 <BlueMatt> which sucks, but I dont think we have a way to fix it
19:56:44 <BlueMatt> so current approach is to document it and DEBUG_LOCKORDER-assert
19:56:55 <BlueMatt> we can relax the requirement a bit with skeees' proposed validation-in-its-own-thread stuff
19:57:02 <BlueMatt> but there will still be some call restrictions
19:58:18 <BlueMatt> ok endtopic
19:58:35 <wumpus> #endmeeting