19:00:38 <wumpus> #startmeeting
19:00:38 <lightningbot> Meeting started Thu Jun  9 19:00:38 2016 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:38 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:01:01 <gmaxwell> phantomcircuit: sipa: morcos: sdaftuar: btcdrak: jonasschnelli: luke-jr:
19:01:22 <wumpus> first at PSA: the feature freeze for 0.13 is next week. Make sure that whatever features need to be merged are merged before that time. If there are any pulls that require special attention, or are ready, let me know.
19:01:25 <gmaxwell> petertodd: MarcoFalke:
19:01:39 <wumpus> #link 0.13 release schedule: https://github.com/bitcoin/bitcoin/issues/7679
19:01:52 <MarcoFalke> any topic suggestions today?
19:02:23 <gmaxwell> We can talk some about ongoing compact block testings, I have a few things to report.
19:02:42 <wumpus> last meeting there was talk of release lifecycles documentation, btcdrak and David Harding have been working on that page here: https://github.com/bitcoin-core/bitcoincore.org/pull/179 https://github.com/btcdrak/bitcoincore.org/pull/2  this needs review
19:02:53 <cfields_> wumpus: I have 2 p2p refactor PRs that i'd _very_ much like to have in 0.13. I'm not sure how you're considering those in terms of freezing
19:02:54 <gmaxwell> instagibbs: nickler: NicolasDorier: CodeShark:
19:03:14 <CodeShark> yo
19:03:23 <MarcoFalke> cfields_: I think p2p refactor can go in after the feature freeze?
19:03:29 <gmaxwell> We apparently can no longer compile on hosts with only 2GB ram with defaults.
19:03:39 <wumpus> other TODOs from last week: review and merge #8126 (std::shared_ptr based CTransaction storage in mempool) - that was done, #7935 (Versionbits: GBT support) - also done
19:03:46 <MarcoFalke> I mean it is not a new feature ;)
19:04:04 <gmaxwell> well it was more like 1.5GB ram before.
19:04:35 <wumpus> others have not yet finished: #7598 (Refactor CreateNewBlock to be a method of the BlockAssembler class)
19:04:37 <jonasschnelli> gmaxwell: I compiled on a 2GG AARCH this week successfully.
19:04:41 <jonasschnelli> *GB
19:04:46 <gmaxwell> We have docs that say 1.5GB, they're gonna be like the blocksize on bitcoin.org :)
19:04:49 <wumpus> #7600  Mining: Select transactions using feerate-with-ancestors
19:04:55 <wumpus> depends on what else is running on the machine
19:05:34 <gmaxwell> I've been going through #7598/#7600.
19:05:38 <wumpus> #topic compile-time memory usage
19:05:45 <wumpus> what can *concretely* be done here?
19:05:58 <jonasschnelli> would kick out boost help?
19:05:59 <luke-jr> -O0
19:06:01 <wumpus> is it something worrying?
19:06:08 <cfields_> has anyone measured to see if there are particular objects that are especially guilty?
19:06:11 <CodeShark> what's eating up all the RAM?
19:06:14 <wumpus> yes, we have cfields_
19:06:20 <cfields_> ie. main.cpp/net.cpp ?
19:06:21 <luke-jr> CodeShark: ld/GCC doesn't free memory
19:06:22 <wumpus> especialy some autogenerated c++ files
19:06:33 <wumpus> I made some tables back in the issue about this
19:06:37 <gmaxwell> main.cpp, matt has a patch that moves all the mempool stuff out of it taht apparently gets it back to 1.5GB.
19:06:50 <luke-jr> CFLAGS="-O0 -g0 --param ggc-min-expand=0 --param ggc-min-heapsize=32768"
19:06:54 <wumpus> #link https://github.com/bitcoin/bitcoin/issues/7471
19:06:54 <gmaxwell> I dunno why he hasn't PRed it, I asked him to.
19:07:00 <cfields_> wumpus: thanks
19:07:10 <wumpus> eeh that's the wrong one
19:07:22 <gmaxwell> wumpus: unthanks
19:07:30 <wumpus> well it is about the same subject
19:07:33 <wumpus> #link https://github.com/bitcoin/bitcoin/issues/6658
19:07:52 <wumpus> lots of people have posted about it, but there doesn't seem to be a clear solution
19:08:00 <jonasschnelli> main.cpp -> 1248524bytes ... ^^
19:08:37 <wumpus> reducing the number of included headers works, I think
19:08:41 <sipa> present
19:08:46 <cfields_> I have PRs which break up net.h/netbase.h, i'd be curious to see if those make a significant difference
19:09:00 <gmaxwell> in any case, something to be aware of and nudge a bit at... some refactorings to move code around would help.
19:09:13 <wumpus> also building with clang helps
19:09:21 <wumpus> it uses a lot less memory at the same compile settings, usually
19:09:25 <gmaxwell> and be independantly good for reasons unrelated to peak memory usage.
19:10:05 <cfields_> i'd assume that mem usage correlates solidly with compile time
19:10:55 <CodeShark> not so sure - lots of small files might mean the bottleneck is disk access
19:11:29 <sipa> CodeShark: come on
19:11:32 <CodeShark> in any case, it would be good to bring down the peak mem usage
19:11:32 <wumpus> the bottleneck in compilation is hardly ever disk access, at least *reading* disk access
19:11:37 <sipa> reading in 100 files?
19:11:43 <sipa> sequentially
19:12:08 <BakSAj> would be cool, if btc full nodes could continue to be runnable on Rasberry Pi ... with 1GB RAM
19:12:37 <jeremyrubin> BakSAj: runnable is not compileable on?
19:12:40 <gmaxwell> not sure there is much else to say here.  I only brought it up for general awareness issues, since I think it's likely a death by 1000 cuts that can be improved in a multitude of ways.
19:12:54 <wumpus> seek/read access for source files is only a problem for really huge projects, and then especially when the source is hosted on some horrible network file system (like clearcase), in any case bitcoin doesn't even come close
19:13:03 <cfields_> heh, disk is negligible. It's easy to see where time is spent with -ftime-report.
19:13:17 <wumpus> but like always: measure before you start talking about bottlenecks
19:13:35 <jonasschnelli> I think adding cross compile depends options for ARM and AARCH64 would also reduce the "memory problem" (at least the amount of complains): https://github.com/bitcoin/bitcoin/issues/8162
19:13:58 <BakSAj> jeremyrubin: preferably both compileable and operatable
19:13:59 <wumpus> BakSAj: for small embedded systems you should use cross-compilation
19:14:03 <cfields_> jonasschnelli: i'm halfway through the changes needed there.
19:14:13 <jeremyrubin> i have had machines take a bit of time on autogen.sh fyi
19:14:18 <jonasschnelli> cfields_: nice. Focus on Qt5.6 first. :)
19:14:34 <wumpus> you can cross compile on ARM using depends, we just don't distribute ARM binaries
19:14:37 <cfields_> jonasschnelli: actually, arm/aarch64 already work fine with depends. Just have to use NO_QT=1 manually.
19:14:40 <wumpus> s/on/to
19:14:42 <gmaxwell> I'm skeptical that the intersection of rpi users that complain about compile issues and people who will cross compile is the emptyset. But cross compiling is good.
19:14:44 <wumpus> cfields_: yes, it works fine
19:14:51 <luke-jr> for comparison, webkit-based stuff typically uses up to 12 GB RAM with debug symbols, and much much less without..
19:14:56 <gmaxwell> er isn't the empty set, you get what I mean.
19:15:07 <cfields_> luke-jr: oh, good point...
19:15:13 <wumpus> it's *very easy* tocross compile for ARM
19:15:14 <jonasschnelli> I think NO_QT=1 for ARM/AARCH64 could be a start (even for "official binaries").
19:15:20 <wumpus> with the depends system
19:15:24 <wumpus> jonasschnelli: yes
19:15:28 <cfields_> the gitian-debug PR turns on debug symbols, so gitian mem requirement is bumped after that.
19:15:37 <gmaxwell> wumpus: a lot of people using rpi2 like systems do not have another linux host.
19:15:42 <luke-jr> wumpus: that builds static binaries, which is wasteful on RAM
19:15:44 <jonasschnelli> Also ARM is used more and more for GUI systems.
19:15:50 <jeremyrubin> can autogen.sh be made faster?
19:16:01 <wumpus> jeremyrubin: no
19:16:15 <wumpus> not by us, at least
19:16:15 <BakSAj> ok, thanks for explaining.. personally i had no trouble compiling 0.12.1 on rpi 3, was afraid that minimum requirements will raise with future releases
19:16:32 <jonasschnelli> luke-jr: if you want to run bitcoind on a RiP (or similar) static builds are fine. Mostly you don't have tons of other tools that could share libraries installed.
19:16:32 <BakSAj> since suprisingly many nodes run on rpi
19:16:44 <jeremyrubin> wumpus: maybe the one thing that is fixed by a faster disk
19:16:50 <luke-jr> jonasschnelli: I'm thinking more of Bitcoin-Qt
19:16:59 <cfields_> jonasschnelli: as qt's gui plugin situation improves, we may be able to move back to the shared-qt builds
19:17:05 <wumpus> I'm sure minimum requirements will raise with future releases, that's just the way things are, we'll try to raise them not too much though
19:17:11 <MarcoFalke> jonasschnelli: I think we already have notes on how to corss compile to arm?
19:17:13 <jonasschnelli> Agree. Static linking qt is not ideal. But lets don't roll this up again.
19:17:14 <MarcoFalke> https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md#arm-cross-compilation
19:17:23 <btcdrak> oh meeting
19:17:35 <jonasschnelli> MarcoFalke: notes, yes. But it should be included in our release builds (gitian)
19:17:42 <MarcoFalke> jup, agree
19:17:46 <cfields_> jonasschnelli: sorry, i meant that directly in the context of shipping arm+gui binaries
19:17:56 <luke-jr> jonasschnelli: for now, people just compile natively to avoid static, so suggesting cross-compile isn't a real option
19:18:06 <wumpus> jeremyrubin: I'm not sure. I have no idea what autogen.sh would be spending time on. But it seems more a GNU problem thatn a bitcoin core problem :)
19:18:09 * gmaxwell looks forward to arm (+gui) binaries in the sometime future.
19:18:23 <wumpus> jeremyrubin: I'm surprised it's autogen.sh taking a lot of time not configure, which has this huge list of scripts to execute for probing
19:18:48 * luke-jr fully intends to use an ARM system with Core(Knots) as his hot wallet in some months.
19:18:50 <cfields_> jeremyrubin: you can use a quicker shell for autogen. IIRC dash vs. bash shaves a few seconds off
19:19:15 <wumpus> well arm non-GUI binaries would already be a great step forward, one step at a tme
19:19:15 <luke-jr> autogen.sh isn't even part of building; it's a developer tool
19:19:17 <cfields_> wumpus: ah, as a feature-freeze request: ok to plan on arm bins (without gui) for 0.13 ?
19:19:27 <luke-jr> if you're running autogen.sh, that means you're running from git, and you shouldn't do that
19:19:30 <cfields_> i can try to have that done today
19:19:30 <jonasschnelli> cfields_: ack, +1
19:19:33 <wumpus> I think arm gui would be prett much a per-distro afair
19:19:36 <wumpus> cfields_: sure!
19:19:37 <luke-jr> cfields_: +1
19:19:39 <jeremyrubin> luke-jr: are we making build faster for developers or for users?
19:19:45 <gmaxwell> +1
19:19:47 <cfields_> ok
19:20:06 <luke-jr> jeremyrubin: I think the concern is "ability to build" rather than "speed to build"
19:20:11 <gmaxwell> jeremyrubin: users shouldn't need to run autogen-- if they get the source tarballs we have, it should already be autogenned.
19:20:26 <cfields_> ^^
19:20:36 <BakSAj> cool, rpi fans will love you :-)
19:20:44 <wumpus> but the people actually doing a lot of builds are developers, only they would care about a few more/less seconds in the build scripting
19:20:57 <BakSAj> next step - run full node on cell phone :-)
19:21:01 <jeremyrubin> wumpus: ++
19:21:11 <luke-jr> BakSAj: I believe a number of people have done this.
19:21:18 <wumpus> BakSAj: people are doing that actually, that's one of the motivations for the ARM binaries
19:21:28 <BakSAj> lol ok
19:21:28 <luke-jr> next step is therefore to support SPV mode when bandwidth is expensive ;)
19:21:30 <gmaxwell> BakSAj: abcore, it works fine.
19:21:39 <jonasschnelli> luke-jr: +1
19:21:41 <luke-jr> but that's post-0.13 IMO
19:21:45 <wumpus> absolutely
19:22:22 <wumpus> in any case it's too late to start on anything new for 0.13, for that we have to consider which of the current pulls can go in
19:22:47 <luke-jr> can we get in [8-bit] key generation type?
19:23:32 <jonasschnelli> 32bit!
19:23:41 <jonasschnelli> You can provide a migration patch for Knots
19:23:52 <jonasschnelli> Isn't that trivial?
19:23:53 <BakSAj> will 0.13 contain just segwit code or actual softfork also? tnx
19:24:03 <jonasschnelli> SW SF can be 0.13.1
19:24:17 <jonasschnelli> SW are mostly not coupled with major releases
19:24:21 <jeremyrubin> I think that 0.13.1 will be worse for upgrade times
19:24:24 <wumpus> SW should be released in a minor release
19:24:26 <jeremyrubin> does anyone have data on that
19:24:29 <gmaxwell> BakSAj: major releases do not contain network consensus changes.
19:24:31 <sdaftuar> do we think segwit is going in to 0.13?
19:24:42 <sdaftuar> (without activation scheduled)
19:24:52 <jeremyrubin> wumpus: isn't it a major change?
19:24:52 <petertodd> sdaftuar: you mean, 0.13.0, or 0.13.x>0?
19:24:53 <luke-jr> jonasschnelli: migration is not very practical if 32-bit uses the same version number in Core as 8-bit in Knots already is
19:24:57 <sdaftuar> 0.13.0
19:24:59 <btcdrak> sdaftuar: yes, sipa wanted to merge it soon to master
19:25:01 <CodeShark> what happened to doing it in 0.12.x?
19:25:02 <luke-jr> jonasschnelli: maybe this needs more off-meeting discussion then
19:25:08 <btcdrak> (without mainnet defs)
19:25:08 <jonasschnelli> luke-jr: agree
19:25:15 <sdaftuar> seems like there are still open issues, and no ACKs
19:25:16 <wumpus> jeremyrubin: well from what I've heard minor releases are usually more popular, especialy .1, as some people don't trust .0  :)
19:25:26 <gmaxwell> CodeShark: nothing, there are confused questions.
19:25:27 <sdaftuar> so i don't see how it's going to be merged in the next week
19:25:48 <btcdrak> sdaftuar: why?
19:25:48 <luke-jr> jeremyrubin: segwit is a major change to Bitcoin - not Bitcoin Core.
19:25:51 * jonasschnelli thinks sipa is allowed to merge without ACK
19:25:52 <gmaxwell> jeremyrubin: we have a long thought out published spec on this, please don't divert the meeting to debating it. I can direct you to the information after the meeting.
19:26:07 <jeremyrubin> wumpus: interesting... I do tend to not upgrade any of my software major versions for 6 months. diversion over
19:26:27 <sipa> well merging segwit without fork enabled is not in contradiction with "not doing a consensus change in a major release"
19:26:36 <jonasschnelli> agree
19:26:41 <CodeShark> right
19:26:47 <wumpus> sure
19:26:49 <luke-jr> no objections to merging segwit code without activation
19:26:50 <jonasschnelli> Also, getting ACK for SW is extremly hard. Nobody wants to take the risk.
19:26:51 <gmaxwell> sure, there are code motion logistics that favor merging it.
19:26:51 <sdaftuar> to be clear i'm not talking about any kind of release policy, just code-readiness / review
19:26:54 <btcdrak> jeremyrubin: see out lifecycle docs https://github.com/bitcoin-core/bitcoincore.org/pull/179
19:27:07 <btcdrak> s/out/our/
19:27:57 <wumpus> so is SW ready for merge (into master/0.13)?
19:28:06 <sdaftuar> it has no ACKs, and some open issues to be resolved
19:28:31 <wumpus> ok
19:28:33 <jonasschnelli> major open issue? Or more nitish stuff?
19:28:35 <sdaftuar> minor
19:28:49 <wumpus> if it is not critical it can also be fixed in a later pull
19:28:51 <sdaftuar> but bugs, not style nits
19:29:09 <wumpus> oh known bugs should be addressed in the pull itself
19:29:21 <sipa> i think everything will be addressed in my next batch of patches
19:29:29 <btcdrak> sipa: great!
19:29:42 <gmaxwell> should people be acking the reviwew PR or the rebase/reorg?
19:29:42 <luke-jr> sipa: does that include expanding 2nd push to 75 bytes max? or is that still an open thing?
19:30:19 <sipa> luke-jr: this is the place to ask, and i would say no, there is no point
19:30:27 <sipa> but perhaps others have another opinion
19:30:32 <btcdrak> luke-jr: I didnt understand where 75 came from.
19:30:42 <sipa> btcdrak: up to 75 is easy
19:30:43 <luke-jr> btcdrak: largest size that wouldn't require additional testing
19:31:18 <gmaxwell> has to do with the opcode types changing for different sizes of push.
19:31:47 <sipa> so, opinions?
19:31:51 <btcdrak> 32->40->75 seems like a big jump
19:32:06 <gmaxwell> btcdrak: from the code perspective they're all the same.
19:32:19 <luke-jr> my opinion is there is no point limiting it (beyond the impl/test cost of >75), and such limits could very well prevent future softforks
19:32:49 <luke-jr> more tolerant enables softforks, so should be preferred over useless limits
19:33:08 <gmaxwell> Luke-jr's argument has merit in my opinion-- it can be reduced later, but I don't have a strongly held view. I'm not aware of a DOS attack risk created by not having the stricter limit earlier.
19:33:44 <gmaxwell> (of course, IsStandardness should be strictly limited)
19:33:44 <luke-jr> to expand the limit later requires a hardfork
19:34:00 <luke-jr> yes, node policy should reject any unknown witnesses period
19:34:23 <CodeShark> ok, I think luke-jr has a strong argument
19:34:27 <btcdrak> that makes sense
19:34:47 <sipa> there should be no need for more than 256-bit hash + some versioning metadata
19:35:14 <sipa> and setting it to more gives it the impression that there is
19:35:16 <petertodd> sipa: or, to be precise if there is that means Bitcoin is more broken than that
19:35:24 <sipa> petertodd: exactly
19:35:33 <jeremyrubin> luke-jr: in general I agree with keeping flexible, but do you have an example for sipa of why you'd want it?
19:35:36 <gmaxwell> The biggest harm I see is that allowing a larger size here does limit the ability to make utxo entries limited in size in the future, potentially. But it could be done later.  It also enabled policy bypass to abuse the utxo set for data storage, though it's not much of an issue there.
19:35:42 <luke-jr> sipa: it doesn't need to give that impression. I don't think we need to predict the future too much here.
19:36:23 <gmaxwell> luke-jr: for example, if you were to argue that we might someday need 512 bit hashes, I'd agree-- but then I'd point out that in that case there would need to be a hardfork to change all the other things.
19:36:26 <sipa> i'd rather not rely on isstandardness when reasoning about longer term future
19:37:01 <petertodd> in a MR implementation I did, it turned out to be very advantageous if the things in the MMR were fixed side forperformance
19:37:13 <luke-jr> jeremyrubin: any case where we would need indicators in the UTXO set itself; but I don't have a concrete example at this time
19:37:19 <gmaxwell> Also, not allowing it in SW doesn't preclude it in the future, you'd just need to use a different version type signaling in that case.
19:37:35 <luke-jr> for example, we could have added the maturity stuff in the 2nd push if we didn't have nSequence
19:37:43 <gmaxwell> Yes, I really wish UTXO entries were fixed size.
19:37:53 <sdaftuar> sipa: isn't there a strong deterrent against abuse, because your funds are anyone-can-spend to older nodes?
19:37:59 <luke-jr> gmaxwell: you'd need a new commitment entirely
19:38:13 <luke-jr> gmaxwell: in addition to the current one
19:38:24 <sipa> sdaftuar: there is no rule preventing 0-value outputs
19:38:34 <_anthony_> just use the private key of a payment address to store the 256 bits
19:38:37 <sdaftuar> ah, good point
19:38:41 <sipa> (if you ignore relay polify)
19:39:09 <luke-jr> abuse is already possible. this doesn't make it worse. if in the future we make it better, we can limit this at the same time
19:39:28 <gmaxwell> if one assumes a fixed size utxo entry, luke's suggestion basically doubles the utxo set size.
19:39:32 <petertodd> sipa: though, for that specific case I find it ahrd to think of a abuse use-case that'd care about that, given you could screw up the usse-case by spending those outputs
19:39:51 <luke-jr> gmaxwell: we can't assume that today, and if we softfork an assumption tomorrow, we can limit this then also
19:40:44 <gmaxwell> We've probably spent more time discussing it now than the decision is worth.
19:41:14 <wumpus> ok, next topic?
19:41:24 <wumpus> #topic compact block testing
19:41:27 <luke-jr> so we use 75 for now, and discuss reducing it later?
19:41:33 <gmaxwell> (and that time could be better spent reviewing/testing more corner cases... lets continue discussion elsewhere I guess)
19:42:34 <btcdrak> so compact blocks...
19:42:41 <gmaxwell> OK. So there are some number of nodes running compactblocks on the public network.. I have 12 peers at the moment, matt has another half dozen in the new relay network that I'm not connected to.
19:42:52 <gmaxwell> Things seem to be working well there, instagibbs has posted some charts.
19:42:56 <wumpus> I've been running a compact blocks node for a few days, no crashes to report :)
19:43:16 <instagibbs> yes i love charts http://imgur.com/iq2lRGl
19:43:26 <gmaxwell> I've been conducting some new tests with a network of nodes with a modified version of compact blocks that reduces the hash size to 16 bits in order to test corner cases around collisions.
19:43:27 <instagibbs> blue stuff is in kB fwiw
19:43:30 <wumpus> lots of succesfully reconstructed blocks
19:43:31 <luke-jr> (ugh, Travis is apparently "detecting abuse" on the Bitcoin code itself, so every clone will be affected?)
19:43:36 <btcdrak> Two large mining pools have also been running them, connected to their pool nodes for block source, one is behind the GFW
19:43:40 <instagibbs> blue dot == 0 fetched txns
19:44:20 <gmaxwell> I found a few bugs, which matt has fixed but not pushed to the PR yet.  Bugs were things like if the cmptblk message was rubbish, it would wait for the peer to timeout before requesting the block normally.
19:44:48 <instagibbs> I intended to review the PR then got ill. Still planning to review.
19:44:57 <gmaxwell> I think this particular testing technique of modifying the code to make rare cases common is pretty effective and will result in good testing of most of those corner cases.
19:45:05 <wumpus> luke-jr: (offtopic) that started happening with the parallel testing I think
19:45:08 <sipa> gmaxwell: agree
19:45:19 <MarcoFalke> luke-jr: Shoot them an email
19:45:39 <gmaxwell> The compact block code is now rebased on top of the sharedptr work, so it's now a fair bit simpler.
19:45:54 <luke-jr> MarcoFalke: I have. My concern is more than just whitelisting individual repos though. (Let's continue discussion after the meeting)
19:45:54 <instagibbs> gmaxwell, matt's rebase is on that now?
19:45:59 <instagibbs> err pr is rebased*
19:46:01 <gmaxwell> instagibbs: yes.
19:46:12 <gmaxwell> matt's PR is on master as of last night.
19:46:20 <sipa> yes, forget my branch
19:46:34 <CodeShark> what PR#?
19:46:48 <instagibbs> #8086
19:47:07 <wumpus> #link https://github.com/bitcoin/bitcoin/pull/8068
19:47:15 <cfields_> has there been discussion of a servicebit for compact blocks? Now that we have the dns seed prefixes, that would allow for very quick discovery
19:47:23 <gmaxwell> Based on the issues I found, probably the interaction with block fetching logic needs more review.
19:47:37 <btcdrak> cfields_: if it deploys in 0.13 it wont be necessary
19:47:43 <gmaxwell> cfields_: IMO I don't see a need to preferrentially peer. I expect support to become sufficiently ubiquitious fast enough.
19:47:46 <wumpus> #action forget sipa's compact blocks branch and use thebluematt's PR
19:48:10 <gmaxwell> it's not something that anyone has a reason to not support, except for just not having implemented it.
19:48:13 <btcdrak> hrm, action point is to forget :)
19:48:16 <sipa> cfields_: the argument brought up before was tgat service bits should be used for critical
19:48:28 <sipa> for critically required services
19:48:41 <gmaxwell> like your node won't work right if you don't have peers with the right services.
19:48:46 <wumpus> btcdrak: yeah for people testing the code to use the other branch
19:48:49 <luke-jr> makes sense
19:48:55 <sipa> and the only time when yoi critically need a compact block peer is as a miner, who should be curating their connections anyway
19:48:59 <jeremyrubin> in #8086 where is the salt generated btw?
19:49:09 <cfields_> hmm, fair enough
19:49:35 <wumpus> and miners can look at the protocol version to see if their peer supports compact blocks?
19:49:48 <gmaxwell> jeremyrubin:
19:49:49 <gmaxwell> +CBlockHeaderAndShortTxIDs::CBlockHeaderAndShortTxIDs(const CBlock& block) :
19:49:52 <gmaxwell> +        nonce(GetRand(std::numeric_limits<uint64_t>::max())),
19:50:12 <luke-jr> wumpus: I don't think we can assume a specific protocol version supports it
19:50:27 <luke-jr> if we have a future version with better compact blocks, we may want to drop support for the current one
19:50:28 <jeremyrubin> thanks
19:50:29 <Lightsword> I think using service bits is a good idea, mainually curtailing connections is very time consuming and raisies the barrier to entry for mining
19:50:32 <gmaxwell> wumpus: you can do the handshake.
19:50:36 <sipa> wumpus: no, miners should connect to a known peer that supports it
19:50:41 <luke-jr> Lightsword: neither are likely to be necessary
19:51:09 <wumpus> gmaxwell: right
19:51:22 <gmaxwell> Please, service bits are basically forever and we only have 32 of them, I expect the window between some and nearly all use of this to only be a few months to a year long.
19:51:24 <sipa> wumpus: because just supporting compact blocks is not enough, they also need to have good uptime and reliability  latency, bandwodth, ...
19:51:29 <sipa> gmaxwell: we have 64
19:51:41 <gmaxwell> Same difference. (really? hmph!)
19:51:46 <jeremyrubin> I would suggest either writing the entropy to a file once or having it settable in a config file
19:51:48 <wumpus> we should have a concept of temporary service bits, like for the versionbits
19:52:05 <sipa> jeremyrubin: that's a good idea but orthogonal
19:52:11 <luke-jr> as long as nobody relies on service bits, they can be temporary
19:52:16 <btcdrak> we dont need preferential peering for compact blocks. It wont take long for wide network support.
19:52:17 <luke-jr> ie, use them as hints
19:52:23 <cfields_> don't we have a range designated for playground?
19:52:27 <luke-jr> yes
19:52:31 <jeremyrubin> sipa: (yes, sorry, just reviewing it now)
19:52:34 <Lightsword> a service bit to indicate a secondary service bit field needs to be used?
19:52:47 <luke-jr> one of which is currently getting full-RBF temporary usage
19:52:50 <wumpus> Lightsword: that would completely make it unuseful for preferential peering
19:52:50 <gmaxwell> jeremyrubin: uh. I'm not sure what you're talking about there... the nonces are per block and should not be predictable.
19:53:12 <wumpus> Lightsword: (as neither addr messages nor the DNS seeds would be aware of the secondary mechanism)
19:53:14 <gmaxwell> statically configuring it would be broken.
19:53:29 <wumpus> why would you want to fix the entropy statically?
19:53:33 <instagibbs> gmaxwell, perhaps setting cmpctblock as a tie-breaker for keeping connection?
19:53:41 <gmaxwell> Okay, in any case, I think thats all I've got there.
19:53:50 <btcdrak> ding ding, we have 7 mins remaining
19:53:52 <instagibbs> well, I guess "he sent me blocks fast" is/will be one, same thing
19:53:54 <cfields_> static entropy is much easier to test :p
19:53:54 <gmaxwell> instagibbs: sounds like a fine additional ranker in the connection management stuff.
19:54:05 <wumpus> instagibbs: +1
19:54:06 <sipa> indeed
19:54:13 <jeremyrubin> cfields_: yep
19:54:16 <gmaxwell> instagibbs: though yea, the 'most recent blocks' probably mostly covers it.
19:54:35 <BakSAj> which version are compact blocks planned for?
19:54:40 <sipa> related to that: please review gmaxwell's patch for adding fast blkck and tx relayers for not evicted
19:54:41 <jeremyrubin> gmaxwell: it doesn't harm security so long as it's kept secret from peers
19:54:50 <btcdrak> BakSAj: 0.13.0
19:54:51 <instagibbs> sipa, which number
19:54:59 <sipa> instagibbs: sec
19:55:04 <jeremyrubin> gmaxwell: nvm -- forgot you have to send it?
19:55:07 <gmaxwell> jeremyrubin: the nonce used for compact blocks must be sent to peers or they can't recover the block.
19:55:12 <petertodd> wumpus: we do have temporary service bits
19:55:19 <BakSAj> btcdrak: thanks!
19:55:33 <sdaftuar> gmaxwell: thoughts on #7598/#7600? you said above that you'd started review
19:55:40 <Lightsword> isn’t it likely we’re going to overhaul the p2p protocol by the time we run out of service bits?
19:55:45 <sdaftuar> i still think it should be a priority to get those PRs merged for 0.13.0...
19:55:47 <instagibbs> I don't think connecting to cmpctblock peers will be hard unless we get sybil'd by AWS forks
19:55:57 <gmaxwell> sdaftuar: I like them and will ACK soon, once I come up with a useful way to test.
19:56:08 <sipa> sdaftuar: me too, i started revieweing but got caught up on other things
19:56:09 <gmaxwell> sdaftuar: I agree.
19:56:24 <sipa> Lightsword: maybe
19:56:41 <sipa> Lightsword: it's often hard to predict how long protocols live
19:56:47 <gmaxwell> I think important big PRs I'd really like to have in 0.13 are SW, Compact blocks, CFPF related, and BIP32.
19:56:51 <sdaftuar> gmaxwell: ok, let me know if you want help with the sim environment i shared with you, i think that makes it easy
19:56:54 <instagibbs> sipa, https://github.com/bitcoin/bitcoin/pull/8084
19:56:59 <gmaxwell> There are a bunch of small things (including all of mine)
19:57:16 <sipa> instagibbs: that one, thanks
19:57:45 <cfields_> off-topic: quickly, before I forget. I'll be headed out of town on Friday and only reachable for emergencies for ~10 days. If anyone needs anything from me before I go, speak up now :)
19:57:58 <sipa> cfields_: for how long?
19:58:03 <sipa> a month?
19:58:04 <Lightsword> maybe we should just have a service bit for flagging fast relay nodes/miners in general for preferential peering rather than making it flag compact blocks specifically
19:58:17 <wumpus> only features have to be in before the feature freeze, anything that can be interpreted as bug fixes or anti-DoS measures doesn't have the deadline of next week
19:58:20 <wumpus> also SW is special
19:58:30 <sipa> Lightsword: we should also have an evil bit that abusive nodes should set
19:58:45 <gmaxwell> Lightsword: "the I am a DOS attack master node, please connect to me" flag?
19:58:48 <Chris_St1> brilliant
19:58:51 <btcdrak> sipa: ^.^
19:58:56 <wumpus> as we discussed above it's a consensus change so it can't be enabled in a major release first
19:59:04 <cfields_> sipa: for ~10 days. I'll be gone for a month total, but working for the last few weeks.
19:59:10 <sipa> i see
19:59:23 <wumpus> he announced that well in advance
19:59:30 <gmaxwell> lynch him!
19:59:33 <luke-jr> cfields_: review of https://github.com/bitcoin/bitcoin/pull/5872 ? :D
19:59:36 <gmaxwell> oh in advance, okay.
19:59:38 <gmaxwell> :P
19:59:47 <BakSAj> :-)
19:59:54 <cfields_> luke-jr: added to the list, thanks
20:00:01 <sipa> *ding dong*
20:00:03 <instagibbs> wumpus, all-but-SF SW would be nice
20:00:05 <wumpus> #endmeeting