19:00:54 <wumpus> #startmeeting
19:00:54 <lightningbot> Meeting started Thu Jan 24 19:00:54 2019 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:54 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:00:55 <provoostenator> hi
19:01:14 <moneyball> topic proposed by jnewbery: Chaincode summer residency: looking for (remote) mentors and recommendations for residents
19:01:20 <sipa> suggested topic: globals and initialization order
19:01:50 <promag_> hi
19:01:51 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb
19:01:54 <instagibbs> hi
19:01:55 <meshcollider> hi
19:01:58 <jamesob> yo
19:02:03 <gwillen> \o
19:02:06 <marcinja> hi
19:02:29 <wumpus> #topic High priority for review
19:03:09 <wumpus> #11082 #14491 #14711 #14897 #15153 #15226
19:03:13 <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
19:03:16 <gribble> https://github.com/bitcoin/bitcoin/issues/14491 | Allow descriptor imports with importmulti by MeshCollider · Pull Request #14491 · bitcoin/bitcoin · GitHub
19:03:21 <gribble> https://github.com/bitcoin/bitcoin/issues/14711 | Remove uses of chainActive and mapBlockIndex in wallet code by ryanofsky · Pull Request #14711 · bitcoin/bitcoin · GitHub
19:03:23 <gribble> https://github.com/bitcoin/bitcoin/issues/14897 | randomize GETDATA(tx) request order and introduce bias toward outbound by naumenkogs · Pull Request #14897 · bitcoin/bitcoin · GitHub
19:03:25 <gribble> https://github.com/bitcoin/bitcoin/issues/15153 | gui: Add Open Wallet menu by promag · Pull Request #15153 · bitcoin/bitcoin · GitHub
19:03:27 <gribble> https://github.com/bitcoin/bitcoin/issues/15226 | Allow creating blank (empty) wallets (alternative) by achow101 · Pull Request #15226 · bitcoin/bitcoin · GitHub
19:03:55 <wumpus> anything to be added or removed?
19:04:20 <achow101> can I have #15225 on hi prio?
19:04:23 <gribble> https://github.com/bitcoin/bitcoin/issues/15225 | GUI: Change the receive button to respond to keypool state changing by achow101 · Pull Request #15225 · bitcoin/bitcoin · GitHub
19:04:27 <promag> regarding open wallet menu - there are concerns regarding blocking GUI - is this something to avoid or can it be improved in 0.18.1?
19:04:30 <jamesob> #15118
19:04:33 <gribble> https://github.com/bitcoin/bitcoin/issues/15118 | Refactor block file logic by jimpo · Pull Request #15118 · bitcoin/bitcoin · GitHub
19:04:37 <achow101> it's actually a blocker for 15226
19:05:07 <wumpus> 14711 should be almost mergeable
19:05:13 <jnewbery> +1 for #15118. It blocks #14121
19:05:15 <gribble> https://github.com/bitcoin/bitcoin/issues/15118 | Refactor block file logic by jimpo · Pull Request #15118 · bitcoin/bitcoin · GitHub
19:05:19 <gribble> https://github.com/bitcoin/bitcoin/issues/14121 | Index for BIP 157 block filters by jimpo · Pull Request #14121 · bitcoin/bitcoin · GitHub
19:05:22 <wumpus> though promag keeps commenting so I'm not sure xD
19:05:39 <gwillen> I am hoping for movement on #13932 but I think it needs love from achow101 more than review at present
19:05:42 <gribble> https://github.com/bitcoin/bitcoin/issues/13932 | Additional utility RPCs for PSBT by achow101 · Pull Request #13932 · bitcoin/bitcoin · GitHub
19:05:57 <achow101> gwillen: oh yeah, I forgot to update that
19:06:06 <promag> wumpus: :(
19:06:09 <gwillen> achow101: lmk if there's any way I can help!
19:07:16 <wumpus> promag: I don't mean that negatively! if there's issues there's issues no matter *when* you find them, just mean it postpones the merge if there's new review comments
19:07:58 <wumpus> ok added #15225  #15118
19:08:00 <gribble> https://github.com/bitcoin/bitcoin/issues/15225 | GUI: Change the receive button to respond to keypool state changing by achow101 · Pull Request #15225 · bitcoin/bitcoin · GitHub
19:08:02 <gribble> https://github.com/bitcoin/bitcoin/issues/15118 | Refactor block file logic by jimpo · Pull Request #15118 · bitcoin/bitcoin · GitHub
19:08:05 <jamesob> wumpus: thanks
19:08:09 <wumpus> let's also try to remove (merge) a few this weke
19:08:36 <phantomcircuit> hi
19:09:02 <wumpus> 8 is kind of a lot to have on the list, and closer to the release the focus shifts to the milestone instead of this list
19:10:00 <sdaftuar> #15141 is ready for review again btw
19:10:03 <wumpus> so 2019-02-01 is string freeze, 2019-02-15 is feature freeze for 0.18
19:10:04 <gribble> https://github.com/bitcoin/bitcoin/issues/15141 | Rewrite DoS interface between validation and net_processing by sdaftuar · Pull Request #15141 · bitcoin/bitcoin · GitHub
19:10:04 <sdaftuar> oops
19:10:18 <sdaftuar> oh, wait i did type that correcly
19:10:26 <sipa> sdaftuar: cool will review
19:10:27 <jonasschnelli> hi
19:11:07 <wumpus> #topic Globals and initialization order (sipa)
19:11:21 <sipa> hi!
19:11:56 <sipa> we currently have a bit of a mix in the codebase for dealing with initialization order of globals
19:12:29 <sipa> some things are explicitly initialized using init functions, called from main/test startup
19:12:42 <sipa> some things are initialized just using global initializers
19:13:08 <sipa> and some things are using once/init-on-first-use-block-scoped-statics
19:13:32 <sipa> and mixing them is pretty fragile
19:14:16 <wumpus> I prefer explicit initializer functions unless it's simply setting a value, at least it's perfectly clear what the order is in which things get initialized
19:14:37 <wumpus> which is very important if things depend on each other
19:14:49 <wumpus> also things running before main() is quite annoying for debugging
19:14:52 <sipa> the problem is that with explicit initializer functions, things don't work when called from global initializer
19:16:08 <wumpus> (and things that take significant time to run certainly shouldn't be called from global initializers, as they'll delay showing even, say, the help message, which doesn't need any initialization at all)
19:16:32 <sipa> and i think we actually had a long-standing problem with the RNG, which was used possibly before being initialized (since #14955 it uses an init-on-first use construction, which should always be fine)
19:16:36 <gribble> https://github.com/bitcoin/bitcoin/issues/14955 | Switch all RNG code to the built-in PRNG by sipa · Pull Request #14955 · bitcoin/bitcoin · GitHub
19:17:18 <wumpus> anyhow that's my opinion on it, I'm aware the codebase is quite crazy in that regard, we've had initialation order issues since the first release...
19:17:44 <promag> sipa: what's your preference?
19:17:49 <sipa> so i'm getting more and more in favor of using this init-on-first-use construction in more places
19:17:57 <wumpus> meh
19:18:00 <sipa> since it's compatible with everything
19:18:12 <wumpus> it's hard to reason about
19:18:35 <jimpo> I also like explicit initialization of non-trivial things
19:18:36 <wumpus> accidentally using something before something else will suddenly change the initialization order
19:18:38 <wumpus> instead of just fail
19:18:46 <wumpus> we've seen that with logging, for example
19:18:58 <sipa> that's because logging using a global initialized
19:19:01 <provoostenator> I'm in favor of having fewer globals :-) But otherwise haven't really developed a preference in C++
19:19:28 <promag> I prefer explicit order initialization
19:19:49 <wumpus> initializing on first use also doesn't really regard initialization dependencies
19:19:49 <luke-jr> wumpus: in a bad way? the only time I can think of order mattering is when we're pre-config-files-loaded
19:20:09 <sipa> wumpus: how so?
19:20:10 <wumpus> like "we need to have the datadirectory set before writing to it"
19:20:13 <luke-jr> (thinking RNG specifically)
19:20:38 <sipa> oh, i'm not really talking about application level things
19:21:09 <sipa> more things like RNG, logging objects (not actually setting the logfile, which happens later), libsecp, ...
19:21:21 <sipa> syncronization debugging state
19:21:31 <wumpus> if it's really only setting a value to a data structure I agree it's different, but if there's extensive work that might depend on other or OS things, it gets hairy fast
19:21:51 <wumpus> only allocating a data structure on first use is fine...
19:22:03 <gmaxwell> It's also important to avoid undefined behavior over and above just avoiding doing the wrong thing.
19:22:24 <sipa> basically the RNG right now can't use the SHA module, because the RNG is invoked from global constructors (and it works fine with it), and SHA needs explicit initialzation
19:22:46 <wumpus> but I think we agree that global constructors are bad
19:22:49 <wumpus> that's one thing :)
19:23:04 <sipa> so i think there are two solutions to that... avoid global constructors everywhere, or make everything work fine on first use
19:23:55 <wumpus> (I here mean global constructors as 'runs before main', not the static initializers that run on first use)
19:24:03 <sipa> wumpus: right
19:24:31 <sipa> wumpus: so what's your opinion on solving this particular issue?
19:24:35 <gmaxwell> Or make SHA module's autodetect get resolved by the linker, using the GCC extension that does that. :P
19:25:20 <gmaxwell> (doesn't address the general question about dependencies between global constructors)
19:25:23 <wumpus> sipa: so move to initialization on first use or explicit initialization, whatever makes sense in the case, move away from global initializers that do anything more significant than assigning a constant value
19:25:27 <sipa> we can get rid of all globals whose construction needs randomness, but making the SHA256 code autodetect on first use seems a simpler change
19:25:53 <wumpus> I really like how you got rid of CInit, for example
19:26:04 <gmaxwell> The downside of autodetect on first use is that it would make every call to sha256 slightly slower. :(
19:26:46 <gmaxwell> One way to compensate for that would be make sure that there is a batch sha256 function that does many of them and only does the detection once, and be sure to use it where possible.
19:27:12 <sipa> gmaxwell: hmm?
19:27:38 <sipa> i benchmarked it as a 1.8ns slowdown here to use an on-first-use construction
19:27:49 <gmaxwell> sipa: I assume 'autodetect on first use' means "Check a synchronized variable every time the function is run".
19:27:55 <sipa> gmaxwell: nope!
19:28:24 <sipa> it actually compiles to both a synchronized and unsynchronized variable, and in the initialized case, only the latter is checked
19:28:33 <gmaxwell> Okay, 1.8ns doesn't sound that terrible. What the runtime of the function on that host?
19:28:36 <wumpus> nice
19:28:55 <gmaxwell> sipa: how does it know its initialized?
19:29:26 <wumpus> though this is a compiler implementation specific thing, clang and gcc might not do it the same way?
19:29:37 <sipa> gmaxwell: it's just a boolean; when the boolean is false, it means it could be initialized or not (to be checked using synchronized primitives), if it is true, it is guaranteed to be initialized
19:30:22 <sipa> from cppreference.com:If multiple threads attempt to initialize the same static local variable concurrently, the initialization occurs exactly once (similar behavior can be obtained for arbitrary functions with std::call_once).
19:30:26 <sipa> Note: usual implementations of this feature use variants of the double-checked locking pattern, which reduces runtime overhead for already-initialized local statics to a single non-atomic boolean comparison.
19:30:41 <wumpus> good to know
19:31:02 <gmaxwell> in any case, I think that seems an obviously better way to handle it. Residual performance concerns could be handled by my above batching suggestion (which would be a win regardless because of function call overheads).  MOST places where we'd have this concern wouldn't be a big performance bottleneck like sha256 is.
19:31:19 <wumpus> exactly
19:31:25 <wumpus> maybe sha256 is just special here
19:31:41 <wumpus> and we can at least decide for the general case
19:31:45 <sipa> agree
19:32:02 <sipa> enough on this topic, i think
19:32:05 <wumpus> ok!
19:32:23 <gmaxwell> well I could see the same problem showing up for crc32 and being worse, because 1.8ns would be like a 2x slowdown for it. :P  but otherwise I can't come up with much where a 1.8ns slowdown would matter.
19:32:39 <wumpus> #topic Chaincode summer residency (jnewbery)
19:32:40 <luke-jr> could have the calls be pointers that get changed on initialisation (I thought we already did?)
19:32:56 <jnewbery> hi
19:33:06 <wumpus> luke-jr: that means the overhead of an indirect function call every time, that's worse than checking a boolean
19:33:19 <jnewbery> we're hosting the next iteration of the Chaincode residency this summer
19:33:34 <jnewbery> it'll be in our new office in Midtown Manhattan
19:33:41 <jnewbery> details are here: https://residency.chaincode.com/
19:34:00 <wumpus> great!
19:34:05 <jnewbery> we'll take care of sourcing residents, paying travel/accommodation/stipend and hosting them in our office
19:34:20 <sipa> nice!
19:34:35 <provoostenator> nice!
19:34:48 <jnewbery> it's 2-3 weeks of seminars followed by 2 months of project work. We're expecting the residents to make really meaningful contributions to Bitcoin Core and other Bitcoin/Lightning projects while they're here
19:35:05 <jnewbery> I bring this up here because we need help for a couple of things:
19:35:51 <jnewbery> 1. We (Chaincode) will be doing the heavy lifting for the seminar series and hosting, but we need (remote) mentors for the 2 month project period.
19:36:03 <kanzure> what are the responsibilities of the mentors
19:36:31 <jnewbery> each resident will be paired with a mentor. We're looking for 1 hour per week video calls with the resident to help guide them in their project
19:37:13 <jnewbery> obviously chaincoders and their peers will be on hand for incidental questions during the week, and the mentor will be providing overall guidance helping with the project
19:37:28 <meshcollider> Will the peers be chosen based on areas of knowledge
19:37:43 <instagibbs> jnewbery, what's the action item here? ping you if interested in mentoring?
19:38:01 <jamesob> oh don't worry instagibbs, we've already signed you up
19:38:06 <jamesob> ;)
19:38:08 <jnewbery> We'll try to pair residents with mentors who have overlapping interests obviously
19:38:16 <jnewbery> instagibbs - I'll be reaching out individually
19:38:40 <instagibbs> hah
19:38:50 <instagibbs> ok, so check e-mails DMs
19:39:07 <jnewbery> 2. We're looking for recommendations for residents. If you know anyone who wants to immerse themselves in Bitcoin/Lightning over summer and is excited about making a real contribution to the project, please send them our way
19:39:23 <jnewbery> Adam Jonas is taking the lead on organizing this one
19:39:41 <jnewbery> So you can ping him or me if you have any questions
19:40:15 <jnewbery> that's it!
19:40:37 <wumpus> ok! thanks for organizing this
19:40:46 <jnewbery> We're really excited about this one. The longer format means we're expecting to have a lot of great contributions from the residents
19:41:09 <wumpus> hope so!
19:41:43 <wumpus> any other topics?
19:42:19 <jnewbery> one reminder: I'd encourage people to use moneyball's #proposedmeetingtopic to propose meeting topics during the week!
19:42:33 <gmaxwell> Could I nag for review on #14929 ?  it's getting forced to rebase faster than its being reviewed...
19:42:36 <gribble> https://github.com/bitcoin/bitcoin/issues/14929 | net: Allow connections from misbehavior banned peers by gmaxwell · Pull Request #14929 · bitcoin/bitcoin · GitHub
19:43:06 <sdaftuar> gmaxwell: yep i'm actually in the middle of re-reviewing so i can re-ack
19:43:18 <wumpus> #action review #14929
19:43:19 <sipa> add to high priority?
19:43:21 <gribble> https://github.com/bitcoin/bitcoin/issues/14929 | net: Allow connections from misbehavior banned peers by gmaxwell · Pull Request #14929 · bitcoin/bitcoin · GitHub
19:43:35 <wumpus> ok
19:44:32 <gmaxwell> Thanks.
19:46:21 <wumpus> that... concludes the meeting, I guess
19:47:19 <wumpus> #endmeeting