19:01:03 <wumpus> #startmeeting
19:01:03 <lightningbot> Meeting started Thu Aug  8 19:01:03 2019 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:03 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:01:07 <moneyball> hi
19:01:08 <kanzure> hi
19:01:09 <jnewbery> hi
19:01:10 <jonasschnelli> hi
19:01:10 <MarcoFalke> hi
19:01:13 <achow101> hi
19:01:13 <ariard> hi
19:01:15 <sipa> hi
19:01:16 <phantomcircuit> hi
19:01:33 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball kvaciral
19:01:40 <wumpus> hi
19:01:42 <fanquake> Hi
19:01:46 <real_or_random> hi
19:01:58 <meshcollider> hi
19:02:04 <MarcoFalke> I'd rather stick with some GitHub specific stuff that works than to ditch GitHub specific stuff and make running the tests harder for developers
19:02:13 <wumpus> any last-minute proposed topics?
19:02:22 <aj> hey
19:02:25 <moneyball> https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a
19:02:26 <wumpus> there's one by MarcoFalke: Who wants to volunteer as co-maintainer of the Bitcoin Core flatpak
19:02:29 <dongcarl> wumpus: could you remove my proposed?
19:02:47 <wumpus> dongcarl: moneyball already did
19:02:50 <MarcoFalke> We can always keep the bitcoin-builds as a backup fallback ci
19:02:53 <phantomcircuit> MarcoFalke, it really depends on how powerful their build servers are, for a relatively small price a very powerful server could be acquired and used for builds
19:03:02 <MarcoFalke> #proposedmeetingtopic GitHub ci
19:03:42 <wumpus> #topic High priority for review
19:03:44 <MarcoFalke> phantomcircuit: Not every developer has a credit card, even if they can afford the price of the ci
19:03:46 <wumpus> https://github.com/bitcoin/bitcoin/projects/8
19:03:56 <achow101> I don't see why we can't just use both. would it not be possible to make bitcoinbuilds just read the same config file as github's?
19:04:02 <wumpus> 6 blockers, 6 chasing concept ACKs
19:04:08 <wumpus> why are we talking about credit cards ?
19:04:08 <MarcoFalke> Agree with achow101
19:04:08 <jonasschnelli> (lets take the CI discussion to #bitcoin-builds plz)
19:04:21 <MarcoFalke> wumpus: Sorry, will shut up now about ci
19:04:48 <wumpus> I think the BIP9 bury PR is very near merge-ready
19:04:52 <wumpus> #16060
19:04:56 <gribble> https://github.com/bitcoin/bitcoin/issues/16060 | Bury bip9 deployments by jnewbery · Pull Request #16060 · bitcoin/bitcoin · GitHub
19:05:18 <wumpus> anything to add/remove/replace from the project?
19:05:22 <MarcoFalke> It is still in the process of review and some fixups keep getting found
19:05:40 <MarcoFalke> (re 16060)
19:05:50 <wumpus> MarcoFalke: mostly small nits on nits about RPC behavior, from what I've seen
19:05:55 <aj> #8994 can be removed from chasing concept ack?
19:05:58 <gribble> https://github.com/bitcoin/bitcoin/issues/8994 | Testchains: Introduce custom chain whose constructor... by jtimon · Pull Request #8994 · bitcoin/bitcoin · GitHub
19:06:10 <phantomcircuit> MarcoFalke, no what im saying is that we could collectively buy a server that could do the builds, in general things where you're buying "packaged" cpu time from third parties end up being very expensive
19:06:17 <jnewbery> wumpus: there was at least a bug in the test case which I'm fixing now
19:06:25 <jnewbery> I would appreciate rereview after I push that fix
19:06:47 <wumpus> jnewbery: sure
19:07:02 <MarcoFalke> jnewbery: Will take another look in the next couple of days
19:07:35 <MarcoFalke> Also, I plan to review sdaftuar's pr (it has been on my list for a while, but I haven't gotten to it yet)
19:08:05 <wumpus> thanks
19:08:40 <fanquake> wumpus: first topic?
19:08:53 <wumpus> we're done with high priority for review?
19:09:10 * jonasschnelli waves at #16202 (quick review, on the path to p2p encryption)
19:09:12 <gribble> https://github.com/bitcoin/bitcoin/issues/16202 | Refactor network message deserialization by jonasschnelli · Pull Request #16202 · bitcoin/bitcoin · GitHub
19:10:02 <wumpus> #topic Who wants to volunteer as co-maintainer of the Bitcoin Core flatpak (MarcoFalke)
19:10:14 <achow101> who currently maintains it?
19:10:21 <jonasschnelli> can someone explain flatpak?
19:10:26 <MarcoFalke> It isn't created yet
19:10:36 <MarcoFalke> jonasschnelli: It is like a snap package
19:10:42 <wumpus> it's a distribution agnostic binary package distribution format
19:10:44 <MarcoFalke> Different name, though
19:10:44 <wumpus> for linux
19:11:45 <jonasschnelli> Okay. Sounds acceptable.
19:11:47 <wumpus> it's especially good for GUI applications, it has some features like sandboxing and desktop UI integration which simply dropping static binaries has not
19:12:21 <MarcoFalke> I will be a repository hosted under the flatpak org, so not in our org. Which is why I am asking for co-maintainers.
19:12:24 <wumpus> see #16550
19:12:25 <gribble> https://github.com/bitcoin/bitcoin/issues/16550 | Distribute via flatpak · Issue #16550 · bitcoin/bitcoin · GitHub
19:12:54 <wumpus> i'd love to help but can't promise to have any time and motivation for any extra work at the moment, sorry
19:13:06 <MarcoFalke> Sure. Maybe jonasschnelli?
19:13:10 <MarcoFalke> Or fanquake?
19:13:20 <sipa> it seems flatpak is also usable independent of a central repo
19:13:34 <sipa> while snap really needs canonical's infrastructure?
19:13:48 <MarcoFalke> jup, snap needs the Ubuntu infrastructure
19:13:55 <fanquake> MarcoFalke: what’s required to maintain? Just uploading releases and managing issues?
19:14:04 <sipa> flatpak has flathub, but it's not required
19:14:17 <MarcoFalke> fanquake: Only requirement is that you are a less agressive jaywalker
19:14:23 <sipa> (just reading and learning)
19:14:28 <fanquake> 👀
19:14:40 <jonasschnelli> Sorry... already at my timelimit
19:14:42 <sipa> MarcoFalke: one must know how to spell aggressive correctly, however
19:15:09 <MarcoFalke> right
19:15:14 <fanquake> Can’t promise anything 🏃‍♂️
19:15:19 <MarcoFalke> Is there an autocorrect for IRC?
19:16:15 <MarcoFalke> fanquake: Cool, I'll sign you up.
19:16:28 <wumpus> hehe,switch to matrix, it has message editing
19:17:02 <fanquake> MarcoFalke: heh, I was worried that just by asking a question that would happen
19:17:44 <fanquake> wumpus: Next topic then?
19:18:05 <wumpus> #topic GitHub ci (MarcoFalke)
19:18:22 <MarcoFalke> fanquake: I expect it will look like this: https://github.com/bitcoin-core/packaging/pulls?q=is%3Apr+author%3AMarcoFalke+is%3Aclosed
19:18:31 <MarcoFalke> I.e. bump version every 3 months
19:18:39 <MarcoFalke> So nothing too fancy
19:18:51 <MarcoFalke> ok, next topic
19:18:53 <fanquake> MarcoFalke: ok 👍
19:19:26 <MarcoFalke> I think everyone knows by now that travis isn't the most stable ci solution
19:19:33 <wumpus> PSA: first deadline (soft translation string freeze) for 0.19.0 is in less than a month, 2019-09-01
19:20:14 <MarcoFalke> So in exploring alternatives, the GitHub ci might be a good medium-term replacement
19:20:14 <wumpus> MarcoFalke: definitely, it's not a good fit for how we're using it at least
19:20:41 <moneyball> MarcoFalke: is the GitHub CI public info?
19:20:47 <MarcoFalke> Both, travis and the GitHub ci, will be GitHub-centered. That should be fine, imo.
19:20:49 <achow101> https://github.com/features/actions
19:20:53 <moneyball> ah, i see it is
19:20:56 <achow101> they just announced it today, it's in beta
19:20:57 <wumpus> I've never heard of github ci
19:20:57 <MarcoFalke> moneyball: Yes, according to achow101
19:21:12 <moneyball> when Pieter, Cory, Matt, and I visited GitHub, they demo'd this
19:21:23 <wumpus> but if it can take over what travis does and does it beter, without too much work from our side, that'd be very useful
19:21:48 <moneyball> cfields has strong views on this
19:21:50 <MarcoFalke> wumpus: Jup, agree
19:22:06 <achow101> moneyball: any opinions about it from the demo you saw?
19:22:07 <MarcoFalke> moneyball: cfields: Elaborate, pls
19:22:08 <wumpus> cfields can't be here today afaik
19:22:19 <jnewbery> moneyball: can you summarise the strong feelings?
19:22:23 <fanquake> Also agree.
19:22:39 <achow101> strong feelings in a good or bad way?
19:22:50 <sipa> bad.
19:22:59 <moneyball> Concerns about increasing dependence on GitHub/Microsoft
19:23:06 <jonasschnelli> Yes
19:23:47 <MarcoFalke> Again, I don't think it gets worse. Also, we can use jonasschnelli's ci as a fallback (running in the background 24/7)
19:23:51 <fanquake> jonasschnelli: do you want to update on the state of your CI? As an alternative?
19:23:56 <moneyball> Which I agree is a valid point. My response was "What kind of productivity improvement over Travis would we need to see to want to move in that direction?" but I didn't really get a good answer. So, I'd pose that question here.
19:23:56 <jonasschnelli> Update on the self hosted custom open source CI (bitcoinbuilds.org): it does now (since a week) successfully builds all PRs, mostly quicker than travis (though less envs). Integration with Github is doable and easy. But first lets see how it runs for at least 2 months.
19:24:21 <jonasschnelli> Plan is to use the Github checks API when we think it has been proven to be reliable (2 month)
19:24:29 <wumpus> moneyball: "no false positives"
19:24:34 <moneyball> Another point made was concern that GitHub 3rd party API might suffer / degrade over time making 3rd party tools like Travis even worse (relative to GitHub CI)
19:24:42 <wumpus> (or at least, neglible amount of, the problem with travis is false positives)
19:24:53 <sipa> one point is that it seems to provide far more entry points than a normal CI does (it does much more automation of workflows, not just CI), and starting to depend on those seems like it would make it harder to move away (if ever needed)
19:25:10 <jonasschnelli> good point
19:25:17 <MarcoFalke> Yeah, good point.
19:25:26 <wumpus> I wouldn't want to rely on it for automation of workflows
19:25:33 <fanquake> Heh. We’ve already got plenty of GH “turned off” as well.
19:25:40 <moneyball> I'm not sure if their announcement today means anyway can experiment with it not, but if not, then we have an invitation to experiment with it, if anyone is interested.
19:25:40 <wumpus> just taking over what travis does now
19:25:41 <jonasschnelli> They tend to mix auto-fix (code format correction and stuff) with CI which is indeed not only good
19:25:46 <sipa> as long as it's a drop-in replacement of one CI for another, i agree there isn't much impact
19:25:47 <MarcoFalke> We should only use it for the purpose of ci builds
19:25:54 <wumpus> auto-fix ? no way
19:26:08 <MarcoFalke> " You’re in the queue for the beta—we’ll let you know as soon as you have access. "
19:26:12 <wumpus> that sounds realllllly scary
19:26:14 <wumpus> damn
19:26:37 <sipa> that's just an example of something you can configure
19:26:43 <sipa> not a default thing
19:26:47 <MarcoFalke> yes, all of the other stuff is just scary
19:26:49 <wumpus> so now you CI doesn't only have false positives, it can introduce bugs and backdoors into your code !
19:26:51 <jonasschnelli> wumpus: https://developer.github.com/apps/quickstart-guides/creating-ci-tests-with-the-checks-api/#step-26-automatically-fixing-rubocop-errors
19:27:34 <MarcoFalke> Like self-driving cars for software development
19:27:41 <wumpus> yea, lt's disable it, as fanquake says we have plenty of gh features disabled, what is one more … though it's something to watch
19:27:43 <wumpus> yes, exactly
19:28:17 <luke-jr> jonasschnelli: we've "tested" other CIs live - no reason not to do it with our own
19:29:03 <fanquake> So is the resolution here to just wait until we can look at GH CI properly? Maybe test it in production alongside Travis?
19:29:15 <wumpus> that sounds good to me
19:29:22 <jonasschnelli> ack
19:29:24 <achow101> ack
19:29:25 <luke-jr> Does GH CI work with non-GH repos?
19:29:30 <MarcoFalke> no
19:29:38 <achow101> luke-jr: i highly doubt it
19:29:38 <wumpus> luke-jr: only if you mirror them to github, I guess
19:29:39 <sipa> luke-jr: embrace, extend, ...
19:29:43 <luke-jr> I'm with cfields on that then…
19:29:56 <MarcoFalke> luke-jr: Does travis work with non-GH repos?
19:30:08 <luke-jr> I mean, if it's the only thing working, fine temporarily I guess, but I'd rather get away from GH dependency, not more entrenched
19:30:17 <wumpus> luke-jr: definitely agree on that
19:30:18 <luke-jr> MarcoFalke: not sure
19:30:31 <achow101> I really thing we should just do both GH CI and bitcoinbuilds and just have them use the same config file
19:30:41 <jonasschnelli> It looks like GitHub is taking over Git the same way gmail took over email... be aware
19:30:42 <moneyball> does anyone want to test the GitHub CI now? let me know and I will introduce
19:30:44 <luke-jr> achow101: +1
19:30:52 <luke-jr> I wonder how long until there's some kind of standard for CIs
19:30:56 <jonasschnelli> Nice centralized development functionality comes at a price
19:31:01 <achow101> so we aren't entirely dependent on Github, but for individual developers, they can still run CI checks
19:31:08 <sipa> achow101: i like that idea; it also forces us to keep the configuration to be compatible with non-GH features
19:31:13 <wumpus> luke-jr: seems no, travis cannot use anything else than github, unless you have the enterprise version
19:31:17 <luke-jr> moneyball: does it require permission to use?
19:31:20 <aj> jonasschnelli: could test in production, but have it report "success" or "neutral" rather than "success" or "failure" maybe?
19:31:25 <sipa> luke-jr: during the beta, yes
19:31:37 <luke-jr> achow101: "make cichecks" would be nice :D
19:31:56 <achow101> luke-jr: can't run on all of the platforms on one machine
19:32:06 <luke-jr> achow101: why not? :\
19:32:11 <dongcarl> qemu
19:32:17 <jonasschnelli> aj: Yes. That would make sense.
19:33:16 <dongcarl> so, perhaps _someone_ should try out the GitHub CI, and see if it's better than Travis, as a stop gap
19:33:20 <wumpus> dongcarl: from my testing a few years ago, qemu-user works pretty well for the tests, at least if you stick to platforms with the same endianness
19:33:41 <luke-jr> it sounds like bitcoinbuilds is further along than GitHub CI?
19:33:53 <MarcoFalke> Could someone share a writeup of how to run the functional tests in qemu, pls?
19:34:15 <luke-jr> MarcoFalke: write a shell script that runs the bitcoind bin, and export BITCOIND=/path/to/script
19:34:26 <achow101> moneyball: you said we have an invitation to experiment with github CI?
19:34:51 <moneyball> Yes
19:34:52 <achow101> can we skip the queue?
19:35:06 <moneyball> Presumably since they offered to let us use it a month ago
19:35:30 <dongcarl> Is there a zero-effort Travis yml -> GH CI transition?
19:35:37 <sipa> i doubt that
19:35:40 <MarcoFalke> dongcarl: no
19:35:44 <dongcarl> :-/
19:36:03 <dongcarl> Why not stick to Travis till bitcoinbuilds is ready?
19:36:16 <MarcoFalke> But the travis.yml mostly specifies the env, so it shouldn't be too hard either
19:36:41 <achow101> dongcarl: because travis false positives too much
19:36:41 <luke-jr> yeah, I imagine at least some of the scripts should be portable
19:36:51 <MarcoFalke> I'd still like to run a ci with free minutes along of bitcoinbuilds
19:37:00 <jonasschnelli> Me 2
19:37:16 <luke-jr> "free minutes along of bitcoinbuilds"?
19:37:20 <achow101> moneyball: can you ask them let some of us into the beta so we can try it out on our own repos first?
19:37:22 <MarcoFalke> fee cpu time
19:37:24 <jonasschnelli> I think centralized development tools are fine as long as there is an alternative running in parallel
19:37:31 <MarcoFalke> *free
19:37:33 <luke-jr> ah
19:37:41 <luke-jr> I have lots of free ppc64le CPU time I think
19:37:49 <jonasschnelli> The problem is lock-in, suddenly they charge for it or make it incompatible with your needs, ...and you'r screwed
19:37:50 <dongcarl> jonasschnelli: Agreed.
19:37:58 <moneyball> achow101: yes would you like access?
19:38:10 <achow101> sure
19:38:10 <phantomcircuit> achow101, travis is mostly hitting fp on resource exhaustion issues
19:38:12 <luke-jr> moneyball: they can't just enable the entire repo +forks? :/
19:38:26 <moneyball> i don't know
19:38:28 <sipa> yeah, let's ask them how this works
19:38:36 <moneyball> i will clarify with them
19:38:47 <achow101> just enable it for anyone in the bitcoin and bitcoin-core orgs :)
19:38:53 <MarcoFalke> Let's not enable the repo for now and keep testing in forks only
19:38:58 <jonasschnelli> I'd also hope we consider some ethos and not further extend our reliance on github
19:40:16 <wumpus> jonasschnelli: I agree, it's just, if we don't rely on anything further then switching from travis to github for CI doesn't make a difference, they're both centralized and githyb only
19:40:40 <fanquake> wumpus: any other topics?
19:40:49 <wumpus> nope
19:41:02 <jonasschnelli> wumpus: There is maybe no technical difference,... but the signal to the community/world is different
19:41:41 <MarcoFalke> Ok, that is a point I buy. They might say the GitHub ci is "approved by Bitcoin Core"
19:41:52 <wumpus> Could someone share a writeup of how to run the functional tests in qemu <- I'll try if I get around to it, I think the most difficult part is setting up the faux-root filesystem for the target, though on some distros you can install that as package
19:42:03 <wumpus> MarcoFalke: yes, that's scary
19:42:21 <jonasschnelli> "Bitcoin Core implemente GitHub CI after talks with their CEO,... a week after they acted in massive censorship"
19:42:40 <jonasschnelli> (I know I overstretch it)
19:43:11 <fanquake> Then we’d all have to jump on Medium and explain our actions in an emoji filled post mortem.
19:43:58 <luke-jr> might be a harder story to spin if we have our own CI at the same time
19:44:11 <achow101> we could deploy both github CI and bitcoinbuilds simultaneously?
19:44:11 <wumpus> haha way to go, fly ahead of the rumors in the meeting 😅
19:44:24 <luke-jr> "We're migrating to our own CI, but helping GitHub test theirs too"
19:44:27 <MarcoFalke> I'll experiment with the ci a bit in the coming days to see if it fits our use case. We can discuss more after everyone had a look
19:44:41 <jonasschnelli> thanks MarcoFalke
19:44:46 <achow101> +1
19:45:25 <wumpus> yes, let's try that, maybe it's worse a match for our CI needs than travis is, then it's not even worth discussing further
19:45:46 <wumpus> #endmeeting