12016-03-24T00:00:45 <gmaxwell> oh hmph. made some pgoress but stuck again. :(
22016-03-24T00:01:17 <gmaxwell> now this looks like local corruption from running out of space. 2016-03-24 00:00:19 ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=-1, nPos=0)
32016-03-24T00:01:50 <sipax> ugh!
42016-03-24T00:02:04 <sipax> that means the block data wasn't written but the index entry for it was
52016-03-24T00:02:20 <sipax> remind me to investigate the write order during flush in such a case
62016-03-24T00:02:29 * sipax goes into standby mode
72016-03-24T00:03:17 <gmaxwell> well it may have been out of space and the write could have failed. Our out of space handling is not so great always.
82016-03-24T00:03:37 <sipax> it should not go write a database entry for a file it failed to create
92016-03-24T00:03:46 <sipax> is it a pruned node?
102016-03-24T00:03:50 <gmaxwell> no.
112016-03-24T00:03:56 <sipax> in that case, certainly not
122016-03-24T00:04:44 <gmaxwell> what the heck, even after logging that a bunch of times it's making progress again.
132016-03-24T00:06:01 <gmaxwell> okay, actually the readblockfromdisk is being triggered by getblock rpcs being called by p2pool
142016-03-24T00:06:40 <gmaxwell> so this might just be an artifact of running getblock on a block we have headers for but no data.
152016-03-24T00:07:32 <sipax> gmaxwell: no, it shouldn't
162016-03-24T00:07:45 <sipax> getblock RPC checks whether the BLOCK_HAVE_DATA flag is present for that block index entry
172016-03-24T00:14:33 <gmaxwell> well my log is full of
182016-03-24T00:14:33 <gmaxwell> 2016-03-23 23:58:18 Received a POST request for / from 127.0.0.1:51001
192016-03-24T00:14:34 <gmaxwell> 2016-03-23 23:58:18 ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=-1, nPos=0)
202016-03-24T00:14:36 <gmaxwell> 2016-03-23 23:58:18 ThreadRPCServer method=getblock
212016-03-24T00:32:55 <gmaxwell> and now that it's caught up, no more..
222016-03-24T00:34:32 *** afk11 has quit IRC
232016-03-24T00:37:17 *** afk11 has joined #bitcoin-core-dev
242016-03-24T01:09:49 *** murch has quit IRC
252016-03-24T01:11:01 *** Ylbam has quit IRC
262016-03-24T01:12:32 *** fengling has joined #bitcoin-core-dev
272016-03-24T01:56:42 *** supasonic has quit IRC
282016-03-24T01:57:10 *** supasonic has joined #bitcoin-core-dev
292016-03-24T02:09:02 *** Alopex has quit IRC
302016-03-24T02:10:07 *** Alopex has joined #bitcoin-core-dev
312016-03-24T02:19:06 <morcos> gmaxwell: sipax: looks like thats just the error that happens, the RPC only checks the BLOCK_HAVE_DATA flag if you're pruning. Ha!
322016-03-24T02:20:23 *** baldur has quit IRC
332016-03-24T02:20:41 *** baldur has joined #bitcoin-core-dev
342016-03-24T02:21:10 *** afk11 has quit IRC
352016-03-24T02:24:04 *** zooko has joined #bitcoin-core-dev
362016-03-24T02:28:57 *** afk11 has joined #bitcoin-core-dev
372016-03-24T02:52:51 *** Chris_Stewart_5 has quit IRC
382016-03-24T02:53:59 *** AdrianG_ is now known as AdrianG
392016-03-24T02:54:29 *** AdrianG is now known as Guest82537
402016-03-24T02:54:43 *** Guest82537 is now known as AdrianG_
412016-03-24T03:01:02 *** AdrianG_ has quit IRC
422016-03-24T03:01:02 *** AdrianG_ has joined #bitcoin-core-dev
432016-03-24T03:01:16 *** AdrianG_ is now known as Aleph0
442016-03-24T03:08:34 *** afk11 has quit IRC
452016-03-24T03:16:08 *** PRab has quit IRC
462016-03-24T03:17:08 *** PRab has joined #bitcoin-core-dev
472016-03-24T03:22:01 *** afk11 has joined #bitcoin-core-dev
482016-03-24T03:26:56 *** zooko has quit IRC
492016-03-24T03:29:10 *** xiangfu has joined #bitcoin-core-dev
502016-03-24T03:31:54 *** molz has joined #bitcoin-core-dev
512016-03-24T03:34:05 *** mrkent has quit IRC
522016-03-24T03:34:51 *** moli has quit IRC
532016-03-24T03:37:10 *** frankenmint has quit IRC
542016-03-24T03:39:36 *** achow101 has quit IRC
552016-03-24T03:41:35 *** supasonic has quit IRC
562016-03-24T03:54:25 *** supasonic has joined #bitcoin-core-dev
572016-03-24T03:56:47 *** mrkent has joined #bitcoin-core-dev
582016-03-24T04:49:01 *** Alopex has quit IRC
592016-03-24T04:50:06 *** Alopex has joined #bitcoin-core-dev
602016-03-24T04:50:58 *** frankenmint has joined #bitcoin-core-dev
612016-03-24T04:55:17 *** BCB has quit IRC
622016-03-24T05:09:24 *** mrkent has quit IRC
632016-03-24T05:11:46 *** mrkent has joined #bitcoin-core-dev
642016-03-24T05:21:57 *** Don_John has quit IRC
652016-03-24T05:23:49 *** mrkent has quit IRC
662016-03-24T05:24:02 *** mrkent has joined #bitcoin-core-dev
672016-03-24T05:44:15 *** schmidty has quit IRC
682016-03-24T05:48:01 *** Alopex has quit IRC
692016-03-24T05:49:06 *** Alopex has joined #bitcoin-core-dev
702016-03-24T05:51:15 *** supasonic has quit IRC
712016-03-24T05:52:25 *** d_t has joined #bitcoin-core-dev
722016-03-24T05:55:02 *** d_t has quit IRC
732016-03-24T05:56:25 *** mrkent has quit IRC
742016-03-24T05:59:36 *** fengling has quit IRC
752016-03-24T06:00:03 *** xiangfu has quit IRC
762016-03-24T06:00:04 *** dermoth has quit IRC
772016-03-24T06:00:48 *** dermoth has joined #bitcoin-core-dev
782016-03-24T06:02:04 *** xiangfu has joined #bitcoin-core-dev
792016-03-24T06:04:01 *** Alopex has quit IRC
802016-03-24T06:05:06 *** Alopex has joined #bitcoin-core-dev
812016-03-24T06:07:05 *** mrkent has joined #bitcoin-core-dev
822016-03-24T06:08:23 *** fengling has joined #bitcoin-core-dev
832016-03-24T06:17:34 *** molly has joined #bitcoin-core-dev
842016-03-24T06:20:58 *** molz has quit IRC
852016-03-24T06:45:23 <jonasschnelli> encryption: would it be stupid to use the identity keypair (used for Auth / MITM protection) in a ECDH scheme to encrypt a dh key-exchange for further encryption?
862016-03-24T06:45:47 <jonasschnelli> I think we should not use the "static" identity key for encryption.
872016-03-24T06:46:16 <jonasschnelli> It would expose the shared ecdh secred (AES256key) to known-plaintext-attacks.
882016-03-24T06:46:59 <jonasschnelli> But not sure if using the identity key for a quick ECDH keyexchange (known-plaintext-attacks are pretty impossible) would make sense.
892016-03-24T06:51:35 <gmaxwell> It would be stupid beyond all comprehension.
902016-03-24T06:52:14 <gmaxwell> The encryption should be ephemerally keyed. There is no need for the key to outlive the session.
912016-03-24T06:57:35 *** Ylbam has joined #bitcoin-core-dev
922016-03-24T06:57:40 <gmaxwell> What I suggested earlier-- which is a pretty standard and well studied architecture is to use randomly generated keys to establish a secure channel (with authenticated encryption like AES-GCM or chacha20-poly1305); then inside if-- if some identity is in use-- you use that identity to authenticate the secure session you just established. E.g. by signing the hash of the shared session key.
932016-03-24T07:00:44 <sipax> gmaxwell: that is MitMable?
942016-03-24T07:01:02 <gmaxwell> ... No.
952016-03-24T07:01:36 <sipax> oh, no!
962016-03-24T07:01:37 <jonasschnelli> gmaxwell: how would you pass the random generated symmetric cipher key to "the other side"? ECDH channel with the identity key?
972016-03-24T07:03:06 <gmaxwell> Please please stop trying to use ecdh with long lived keys. This is almost always a severe design mistake.
982016-03-24T07:04:21 <gmaxwell> Each side sends a new, randomly generated public key to the other side. The session keys (there will be more than one, likely four-- auth and encrypt in each direction) are the output of ECDH with those keys, fed through a KDF.
992016-03-24T07:06:16 <jonasschnelli> gmaxwell: Okay. How would you protect from MITM. By auth with the identity key AFTER the encryption channel is setup?
1002016-03-24T07:08:03 <gmaxwell> if identification is in use, inside the encrypted channel you send a signature of the session id (a hash of the ephemerial key from ecdh which also commits to all the channel parameters) using the relevant identity.
1012016-03-24T07:09:20 <sipax> what do you mean by channel parameters?
1022016-03-24T07:09:47 <jonasschnelli> gmaxwell: how would you protect from attacking(DOSing) the encryption request (key generation!) if the auth is after the enc?
1032016-03-24T07:10:02 <jonasschnelli> Also fingerprinting
1042016-03-24T07:10:28 <jonasschnelli> (the later is probably fine if the 4 key are generated randomly)
1052016-03-24T07:10:31 <gmaxwell> key generation is very fast, faster then verifying your digital signatures you propose for authentication.
1062016-03-24T07:11:01 <gmaxwell> Fingerprinting what?
1072016-03-24T07:11:02 * jonasschnelli always though key generation is much more CPU intense then ecdsa verify.
1082016-03-24T07:11:12 <sipax> jonasschnelli: if you always use freshly generated ecdh keys, there is no fongerprinting
1092016-03-24T07:11:14 <jonasschnelli> gmaxwell: fingerprinting the node.
1102016-03-24T07:11:44 <jonasschnelli> Well,.. right. We could ban a node after a failed encryption request.
1112016-03-24T07:12:11 <sipax> i don't think it can fail
1122016-03-24T07:12:24 <jonasschnelli> sipax: I mean the identity check afterwards
1132016-03-24T07:13:04 <sipax> jonasschnelli: it's much cheaper for them to go ask us a bunch of random blocks, with far higher costs
1142016-03-24T07:13:09 <sipax> if they want to dos
1152016-03-24T07:13:11 <gmaxwell> jonasschnelli: no, it's fundimentally easier to generate a new keypair than to verify a ecdh signature. They're pretty close in timings, because we use a constant time implementation for the former.
1162016-03-24T07:13:14 <jonasschnelli> But this would basically mean, every peer can request encrypted sessions... (which i'm not sure if its good or bad).
1172016-03-24T07:13:44 <gmaxwell> they should, and they should all be encrypted so as not to distingush the ones using authentication.
1182016-03-24T07:14:10 <gmaxwell> and this can be made faster than the existing transport (or at least not considerably worse).
1192016-03-24T07:14:38 <sipax> jonasschnelli: for ECDSA, verify > sign >>> keygen
1202016-03-24T07:14:59 <jonasschnelli> sipax: I see. Thanks!
1212016-03-24T07:15:17 <sipax> jonasschnelli: computing the pubkey for a generated private key is similar to signing
1222016-03-24T07:15:24 <gmaxwell> well if by keygen you mean generate a pubkey too, then sign == keygen, pretty much.
1232016-03-24T07:15:48 <gmaxwell> and this is faster than verify.
1242016-03-24T07:16:01 <sipax> same order of magnitude in any case
1252016-03-24T07:16:05 <jonasschnelli> gmaxwell: So. The peer would pass one pubkey for the dh secret calculation and the (static) identity pubkey (static) for auth to the other peer. Right?
1262016-03-24T07:16:52 <jonasschnelli> (doesn't the current[and most] ec keygen algos do a dummy sign/verify to ensure validity?)
1272016-03-24T07:17:05 <sipax> jonasschnelli: if by auth you mean the signing of the session key, indeed
1282016-03-24T07:17:12 <sipax> jonasschnelli: bitcoin does
1292016-03-24T07:17:14 <gmaxwell> the only software ever created that does that is bitcoin core.
1302016-03-24T07:17:19 <gmaxwell> as far as I can tell.
1312016-03-24T07:17:34 <gmaxwell> and it's only sensible to do that for payment addresses, it wouldn't be done here.
1322016-03-24T07:18:15 <gmaxwell> first an encrypted, authenticated channel would be brought up using a ephemeral pubkey sent in each direction. Then _inside_ that channel, the participants could identify themselves by signing the session-id.
1332016-03-24T07:18:24 <jonasschnelli> only signing the session key would not allow the remote peer to identify which identity is signing. Wouldn't it require at least the hash of the identity pubkey?
1342016-03-24T07:19:45 <gmaxwell> Techincally the signature and message its signing is enough, you can just get the pubkey from that. Though it could send the pubkey its using.
1352016-03-24T07:20:14 <jonasschnelli> Ah. Right pubkey recover.
1362016-03-24T07:20:20 <gmaxwell> but there are better protocols for identifying than just sending a signature-- ones that do not leak the identity of at least one participant.
1372016-03-24T07:20:24 *** mrkent has quit IRC
1382016-03-24T07:20:35 <jonasschnelli> 0-knowlage?
1392016-03-24T07:21:32 *** mrkent has joined #bitcoin-core-dev
1402016-03-24T07:21:49 *** PaulCape_ has joined #bitcoin-core-dev
1412016-03-24T07:22:53 <jonasschnelli> What about allowing auth without enc? We probably presume encryption when authenticating.
1422016-03-24T07:23:12 * gmaxwell looks through logs.
1432016-03-24T07:23:14 <gmaxwell> 13:14 < gmaxwell> There are protocols that avoid the fingerprinting; e.g. (the server protectng mode of) https://www.ietf.org/proceedings/54/I-D/draft-ietf-ipsec-jfk-04.txt
1442016-03-24T07:23:39 <gmaxwell> jonasschnelli: I think there is no utility to identifying without encryption.
1452016-03-24T07:24:21 *** PaulCapestany has quit IRC
1462016-03-24T07:24:29 <jonasschnelli> Agree. The current auth proposal would also require signing (auth) each message (which is resource hungry).
1472016-03-24T07:24:42 <gmaxwell> thats unacceptably slow.
1482016-03-24T07:24:45 <jonasschnelli> encryption before authentication allows to keep a auth-session.
1492016-03-24T07:25:34 <sipax> jonasschnelli: note that authentication is used in two meanings here; there is identification (done by signing the session key) and MAC (which you still need, but ks covered by using an authenticated mode of encryption such as AES-GCM)
1502016-03-24T07:25:40 <gmaxwell> (and signining indivigual messages provides non-repudiation, which is usually not what you want; except in specific instances where you do want it.)
1512016-03-24T07:25:42 *** sipax is now known as sipa
1522016-03-24T07:26:14 <jonasschnelli> I see.
1532016-03-24T07:26:38 <sipa> jonasschnelli: so there is "auth" on each message, but it's done with a symmetric shared session key, rather than the identity key
1542016-03-24T07:26:48 <jonasschnelli> And I guess it would be acceptable if the remote peer can fingerprint the requesting peer because it reveals its identity after the encryption channel is established.
1552016-03-24T07:27:29 <gmaxwell> jonasschnelli: it's better than the other way around at least. preferably no one could fingerprint anyone; but it's not clear how possible that is.
1562016-03-24T07:27:32 <jonasschnelli> sipa: Right. But we just need to make sure the symmetric shared session key was shared between the right parties and not Mitmdled.
1572016-03-24T07:27:42 <sipa> jonasschnelli: i think you probably want a per-IP identity, or perhaps just randomly generated identities by default, and allow a peer to ask "hey, you're identity X, right? prove it"
1582016-03-24T07:27:52 <gmaxwell> Other than the "connect to my own server" case it's not clear to me exactly how the identification would be used.
1592016-03-24T07:28:03 <jonasschnelli> sipa: But that would not work for DHCP nodes (SPV)?
1602016-03-24T07:29:03 <sipa> jonasschnelli: so perhaps a separate private non-random configurable host key, which is only used if requested by the peer
1612016-03-24T07:29:29 <sipa> that means no TOFU, though
1622016-03-24T07:29:32 <gmaxwell> we don't want to create cases where nodes can be tracked easily, we've worked to remove identifying information from the protocol... having a protocol where there is mutual auth, say between your own 'server' and 'client' or between your own hosts, having the requester leak something the allows them to be correlated, but only inside an encrypted channel which they think is to their trusted pe
1632016-03-24T07:29:38 <gmaxwell> er, would be minimally harmful.
1642016-03-24T07:30:40 <gmaxwell> sipa: I don't know what use tofu would have... and what happens when a node goes away (say, gets deleted) and a new one is at that address? now you need an expiration mechenism.
1652016-03-24T07:31:03 <jonasschnelli> gmaxwell: at least you could setup a honeypot-like peer that waits until other peers connect, request encryption and identify themself. You could than combine this data with other fingerprint possibilities (mempool and stuff like that).
1662016-03-24T07:32:13 <gmaxwell> jonasschnelli: the party initating authentication would be the only one to lose privacy, so a honey pot would not work; unless it also successfully mitmed hosts that clients were attempting to connect to.
1672016-03-24T07:32:33 <jonasschnelli> But somehow I think either you have a MITM attack possibility or you reveal ones identity. Maybe some zero knowledge prove method could be used here (not familiar with these)
1682016-03-24T07:32:52 <gmaxwell> There are probably multiple usecases which would identify the channel in different ways; but the same secure channel mechenism would work.
1692016-03-24T07:33:58 <gmaxwell> jonasschnelli: I linked a likely sutable identification protocol above.
1702016-03-24T07:34:20 * jonasschnelli is reading the JFK proposal. Thanks to gmaxwell
1712016-03-24T07:35:28 <gmaxwell> I am not suggesting we specifically use that, as much as pointing out that it's possible to do better... but even a very simple protocol does what we want, I think.
1722016-03-24T07:35:42 <sipa> jonasschnelli: you could also avoid the problem, and by default not have identification, and only have a "prove to me you're X before continuing" message inside the encryptee channel
1732016-03-24T07:36:37 <sipa> perhaps even using symmetric crypto, so it only works with a preshared symmetric key
1742016-03-24T07:36:44 <jonasschnelli> sipa: Yes. Agreed. Auth would only be necessary or useful if the both peers know each other (preshared key, same owner, SPV<->node stuff).
1752016-03-24T07:37:31 <gmaxwell> identification, when you say auth it easily gets confused with the authentication part of encryption which is necessary and cannot be omitted when encryption is used. :)
1762016-03-24T07:37:32 *** PaulCapestany has joined #bitcoin-core-dev
1772016-03-24T07:37:48 <jonasschnelli> A preshared preshared symmetric key would also work. Right. IMO its just easier to share two pubkeys instead of one "symmetric key".
1782016-03-24T07:38:59 <jonasschnelli> gmaxwell: Can you explain the differences between the identity authentication and the encryption authentication? Why would the later be required if both parties stay anonyme to each other?
1792016-03-24T07:39:37 *** PaulCape_ has quit IRC
1802016-03-24T07:39:55 <jonasschnelli> Once you exchanged encryption ephemeral pubkeys, MITM should no longer be possible?
1812016-03-24T07:40:23 <gmaxwell> jonasschnelli: any use of encryption _must_ come with symmetric-key authentication, or otherwise the encryption will be mallible and an attacker can corrupt the channel in ways that leak information about the content.
1822016-03-24T07:41:35 <gmaxwell> e.g. they flip a bit, and instead of instantly hanging up, the application is fed some gibberish, and then based on exactly how it responseds/doesn't respond the attacker learns something about the state of the connection and the prior encrypted data.
1832016-03-24T07:42:26 <gmaxwell> so the encryption must comes with a message-authentication-code keyed at the same time as the encryption, so any corruption is detected and the connection terminated without revealing anything, ideally before even decrypting the data.
1842016-03-24T07:42:55 <jonasschnelli> gmaxwell: sharing the encryption ephemeral pubkeys, calculate symmetric secret and hash the complete communication (starting with both pubkey in deterministic order), compare the hashes after decrypt..
1852016-03-24T07:43:20 <gmaxwell> jonasschnelli: that isn't sufficient.
1862016-03-24T07:43:50 <sipa> jonasschnelli: encryption provides confidentiality, not integrity
1872016-03-24T07:44:31 <jonasschnelli> hmm....
1882016-03-24T07:44:55 * jonasschnelli is reading...
1892016-03-24T07:45:05 <Luke-Jr> jonasschnelli: it might or might not make sense to coordinate with the new payment protocol encryption stuff
1902016-03-24T07:45:20 <Luke-Jr> in BIP 75
1912016-03-24T07:47:22 <jonasschnelli> Luke-Jr: thanks. Will check. Agree on the p2p 16x num range.
1922016-03-24T07:49:18 <sipa> jonasschnelli, gmaxwell: the paper my const time aes is based on actually discusses aes-gcm (and easily has 4x the performance of the cbc case, as it can do 4 parallel encryption operations)
1932016-03-24T07:50:27 <jonasschnelli> gmaxwell sipa: hmm... (having a hard time to understand this correctly): so you would require to generate two keypairs at the same time, one for the encryption sym. cipher key, one for the MAC key. Then you sould sha256_hmac the complete message traffic on both sides and compare to validate the integrity?
1942016-03-24T07:50:52 <sipa> jonasschnelli: no, you use aes-gcm
1952016-03-24T07:51:03 <sipa> which does encryption and mac simultanelusly
1962016-03-24T07:51:47 <jonasschnelli> sipa: okay. I see. But the overall concept would be correct (just to understand the basics).
1972016-03-24T07:51:56 <jonasschnelli> s/./?
1982016-03-24T07:52:10 <sipa> though, conceptually, yes, you can use aes for encryption, and then do hmac-sha256 on the encrypted data for integrity
1992016-03-24T07:52:56 <gmaxwell> There should be two "keys" if using authenticated encryption (like AES-GCM, chacha20-poly, or things coming out of CAESAR) one key in each direction; or four keys if constructing autenticated encryption from a cipher and hmac.
2002016-03-24T07:53:15 <jonasschnelli> What I have problems to understand is how attacks would be possible if you would take the encryption ephemeral publey calculated sym. key for the MAC...
2012016-03-24T07:53:49 <sipa> jonasschnelli: let's say you use aes-ctr
2022016-03-24T07:53:57 <sipa> you know ctr?
2032016-03-24T07:54:13 * jonasschnelli reading... got the essence. Yes.
2042016-03-24T07:54:18 <jonasschnelli> streaming.
2052016-03-24T07:55:26 <sipa> someone may not be able to decrypt without having the key, but if they can guess what the contents is of a particular message, they can modify it by xoring it with both what they think the decrypted data is, and xoring it with what they want the decrypted data to be
2062016-03-24T07:56:33 <sipa> there are more complex bad things you can do with other encryption schemes
2072016-03-24T07:56:52 <jonasschnelli> sipa: Got it! Thanks!
2082016-03-24T07:57:18 <sipa> but ctr is a nice example, because it perfectly covers confidentially, but provides almost nothing else
2092016-03-24T07:57:36 <sipa> so it clearly shows the distinction
2102016-03-24T07:58:35 * jonasschnelli is reading the details of AES-GCM
2112016-03-24T07:59:39 <jonasschnelli> sipa: But IIRC, your AES PR is CBC. So GCM would require another implementation while our codebase already supports SHA2 HMAC?
2122016-03-24T07:59:57 <sipa> jonasschnelli: GCM is easy :)
2132016-03-24T08:00:19 <jonasschnelli> sipa: for you or for all other lazy programmers.. :)
2142016-03-24T08:00:35 <gmaxwell> sha2 is slow.
2152016-03-24T08:01:05 <jonasschnelli> And I guess we could drop the message headers 4byte sha256 checksum once encrypted and MACed
2162016-03-24T08:01:49 <gmaxwell> (AES-GCM is also not astonishingly fast without hardware AES, but should be faster than sha2)
2172016-03-24T08:02:09 <gmaxwell> jonasschnelli: yes, and the result, with sutiable primitive selection would be _faster_ than the existing protocol, even though it was encrypted.
2182016-03-24T08:02:25 <jonasschnelli> hah. Thats an argument.
2192016-03-24T08:02:32 <gmaxwell> (faster in terms of cpu usage, it would likely have some more bandwidth overhead, but not a ton)
2202016-03-24T08:03:15 <jonasschnelli> But I'm mostly thinking "in implementations". So,... I could write a first implementation that drops the 4byte header sha and uses SHA2 HMAC for the MAC.. just to have an implementation basepoint.
2212016-03-24T08:03:37 <jonasschnelli> (in addition to the whole encryption keysharing stuff)
2222016-03-24T08:03:47 *** mrkent has quit IRC
2232016-03-24T08:04:58 <jonasschnelli> I could be the guy that works on the code frontend while I require analysis and conceptual support from gmaxwell/sipa and others. This could lead to a pratical p2p encryption IMO.
2242016-03-24T08:06:26 <sipa> gmaxwell: my current const aes needs 270 cycles/byte; making it do 4 in parallel is trivial and would make it need 72; researchers claim they can do ~10 using simd
2252016-03-24T08:06:37 <sipa> and ~3 with aes-ni
2262016-03-24T08:06:39 *** mrkent has joined #bitcoin-core-dev
2272016-03-24T08:06:41 <gmaxwell> sipa: I think with a.. right.
2282016-03-24T08:07:13 <gmaxwell> See also the motivationes behind the CAESAR competition.
2292016-03-24T08:11:16 *** asdfdsas has quit IRC
2302016-03-24T08:13:20 <gmaxwell> sipa: lol one of the CAESAR entries is based on keccak and is named "keyak". That totally isn't going to cause any confusion...
2312016-03-24T08:16:15 <GitHub117> [bitcoin] laanwj closed pull request #7694: Rename AcceptBlock/AcceptBlockHeader to StoreBlock/StoreBlockHeader (master...2016-03-15-naming) https://github.com/bitcoin/bitcoin/pull/7694
2322016-03-24T08:16:45 <sipa> gmaxwell: i was just reading the keyak v2 paper
2332016-03-24T08:18:03 <btcdrak> I've asked a cryptographer to look at #7689
2342016-03-24T08:22:59 <jonasschnelli> Could we ask apoelstra?
2352016-03-24T08:23:41 <jonasschnelli> I'm not sure if a standalone AES library (move sipas PR to a custom library) would make sense.
2362016-03-24T08:24:59 <jonasschnelli> Adding more non-consensus stuff there will very likely lead to a openssl clone?!
2372016-03-24T08:25:05 <jonasschnelli> And adding SHA512 but not SHA256?
2382016-03-24T08:25:27 *** cjcj has joined #bitcoin-core-dev
2392016-03-24T08:25:40 <wumpus> jonasschnelli: correct
2402016-03-24T08:26:06 <wumpus> I don't think petertodd's concern is so much about the library, but more about 'how much review doees this get'
2412016-03-24T08:26:22 *** mrkent has quit IRC
2422016-03-24T08:28:19 <jonasschnelli> wumpus: after thinking about it. maybe a simple C bases AES library could have a reason to exists.
2432016-03-24T08:28:22 <sipa> gmaxwell: what is the fastest authenticated encryption scheme implementation that you know? (assume no hardware support, but constant time)?
2442016-03-24T08:28:52 <wumpus> jonasschnelli: sure, it could, but this is specifically a constant-time AES for wallet encryption
2452016-03-24T08:29:20 *** Arnavion has quit IRC
2462016-03-24T08:29:38 <jonasschnelli> wumpus: Right. If the p2p encryption will be implemented once, it could be extended there (AES CGM).
2472016-03-24T08:29:46 <gmaxwell> sipa: people who want that are using chacha20-poly1305. AES-GCM with hardware support is apparently somewhat faster (and much less power hungry); but without it the chacha20-poly1305 stuff is faster.
2482016-03-24T08:29:47 <wumpus> it's e.g. not the fastest way to implement AES, because we specifically don't need that. There may not be many projects that want to tuse it at all, and there's a long tradition of copy/pasting crypto code anyhow....
2492016-03-24T08:30:31 <gmaxwell> people who care about fast AES use hardware or use these parallel implementations which cannot be used for CBC mode, which compatiblity with the wallet encryption needs.
2502016-03-24T08:31:11 <wumpus> gmaxwell: yes, exactly
2512016-03-24T08:31:48 <wumpus> for the AES code I really have only one concern: is it correct
2522016-03-24T08:32:10 <wumpus> for the random code I'm more scared because of the platform dependence and other uglyness involved
2532016-03-24T08:32:45 <sipa> wumpus: agree
2542016-03-24T08:32:57 <gmaxwell> currently we don't care at all about fast AES. We should somewhat care that it doesn't have stupid sidechannels. Later, if we care about fast AES for p2p encryption, it would be AES-GCM and end up with a CBC incompatible implementation in any case.
2552016-03-24T08:32:57 *** Arnavion has joined #bitcoin-core-dev
2562016-03-24T08:33:27 *** AtashiCon has quit IRC
2572016-03-24T08:33:41 <wumpus> yes, for P2P we could use a different implementation of AES, or not AES at all (chacha20 or so :p)
2582016-03-24T08:34:27 <gmaxwell> right or whatever CAESER selects, depending on what that happens.
2592016-03-24T08:34:38 <wumpus> yes
2602016-03-24T08:35:25 <sipa> gmaxwell: that's 2 years out
2612016-03-24T08:35:46 <gmaxwell> (most of the CAESER candidates are AES based in any case, mostly addressing security complaints about GCM, and expecting hardware support to address the obnoxiousness of constant time AES)
2622016-03-24T08:36:11 * wumpus really likes chacha20, it's so small, efficient, and seemingly simple
2632016-03-24T08:36:51 *** AtashiCon has joined #bitcoin-core-dev
2642016-03-24T08:37:56 <gmaxwell> it has a mixed following, it's apparently enough faster on ARM and without AES-NI that many in the web world have considered it urgent to support... but apparently a lot of the datacenter folks are less than enthused since it ends up much more power hungry than hardware AES.
2652016-03-24T08:39:23 <wumpus> yes, right, it's specifically efficient in sw, if there is a hw implementation of AES it'll win out
2662016-03-24T08:39:39 <jonasschnelli> What MAC AAD would make sense for the p2p encryption? hash of the pubkey&pubkey? IP/Port?
2672016-03-24T08:40:44 <wumpus> let's just wait for hw chacha20 *ducks*
2682016-03-24T08:41:03 <gmaxwell> wumpus: well, its not like they have anything better to do with the transistors...
2692016-03-24T08:41:35 <wumpus> gmaxwell: agree, the dark silicon problem
2702016-03-24T08:42:59 <wumpus> then again - for bitcoin nodes we need to worry about consumer hw, not so much big datacenters
2712016-03-24T08:43:39 <wumpus> I think it's safe to say there won't be so many bitcoin nodes in datacenters to rival the number of https implementations, let alone be a power usage concern
2722016-03-24T08:43:53 <gmaxwell> well a lot of 'consumer hardware' has hardware aes now too, just not phones so much. :)
2732016-03-24T08:44:00 <gmaxwell> but indeed.
2742016-03-24T08:44:41 <wumpus> power usage on the other hand is mostly a phone concern. But yeah.
2752016-03-24T08:46:32 <gmaxwell> in any case both, given good implementations, should be faster the current 'checksum'
2762016-03-24T08:48:16 <sipa> jonasschnelli: the additional data is sent in every message
2772016-03-24T08:51:15 <sipa> jonasschnelli: you would include the message length there, for example
2782016-03-24T08:51:21 <jonasschnelli> sipa: hmm.. so hash would be too expansive?
2792016-03-24T08:51:39 <sipa> hash of what?
2802016-03-24T08:52:06 <jonasschnelli> hash of the encryption pubkeys or could also be a hash of the plaintext message.
2812016-03-24T08:54:29 <sipa> what are you trying to do?
2822016-03-24T08:55:02 <sipa> the additional data is a feature provided by authenticated encryption schemes; you don't need to use it if you have nk need for it
2832016-03-24T09:02:24 *** davec has quit IRC
2842016-03-24T09:03:12 *** davec has joined #bitcoin-core-dev
2852016-03-24T09:04:09 <jonasschnelli> sipa: So do I got this right? You could use something like the current protocol version "70012" as AAD. And during decryption, you feed the TAG (processed AAD) together with the AAD into the decryption method and get a true or false?
2862016-03-24T09:04:18 <jonasschnelli> And thanks for you time to explain that to me. :)
2872016-03-24T09:06:01 <gmaxwell> you could-- generally it's used for additional per message data that must be sent cleartext for protocol reasons, but which you'd still like included in the authentication.
2882016-03-24T09:06:58 *** xiangfu has quit IRC
2892016-03-24T09:07:37 <jonasschnelli> gmaxwell: Okay. Right. Message length would be ideal then.
2902016-03-24T09:09:59 <gmaxwell> that would be the kind of thing; even that may not be necessary, if the length was wrong, the auth would fail in any case-- wrong amount of data in it.
2912016-03-24T09:23:01 *** Alopex has quit IRC
2922016-03-24T09:24:06 *** Alopex has joined #bitcoin-core-dev
2932016-03-24T09:32:24 *** Arnavion has quit IRC
2942016-03-24T09:35:32 *** MarcoFalke has joined #bitcoin-core-dev
2952016-03-24T09:36:58 *** Arnavion has joined #bitcoin-core-dev
2962016-03-24T09:37:19 *** Arnavion has quit IRC
2972016-03-24T09:38:25 *** Arnavion has joined #bitcoin-core-dev
2982016-03-24T09:38:34 *** molz has joined #bitcoin-core-dev
2992016-03-24T09:41:41 *** molly has quit IRC
3002016-03-24T09:46:27 *** cdecker has quit IRC
3012016-03-24T09:55:02 *** p15 has joined #bitcoin-core-dev
3022016-03-24T10:11:32 *** AaronvanW has joined #bitcoin-core-dev
3032016-03-24T10:15:57 <sipa> jonasschnelli: i guess you can include the network magic to the aad
3042016-03-24T10:16:44 <jonasschnelli> sipa: okay. Would that be much different then the encrypted message length?
3052016-03-24T10:18:56 <sipa> jonasschnelli: i mean both... but there is no strong reason
3062016-03-24T10:19:08 <jonasschnelli> okay.
3072016-03-24T10:19:37 <sipa> essentially it guarantees that the length/magic came from someone with the session key
3082016-03-24T10:20:39 <jonasschnelli> ok
3092016-03-24T10:22:57 *** p15 has quit IRC
3102016-03-24T10:36:40 <sipa> aes-gcm uses 96-bit IVs, but they only require being unique for a given key
3112016-03-24T10:36:49 <sipa> and the key is establishes through ecdh
3122016-03-24T10:37:23 <sipa> so they can just be the message number, and don't need to be transmitted
3132016-03-24T11:14:44 *** MarcoFalke has quit IRC
3142016-03-24T11:18:48 <GitHub140> [bitcoin] laanwj pushed 1 new commit to 0.12: https://github.com/bitcoin/bitcoin/commit/597494f5a90c041945006b8f3eff8f7e482e0f2f
3152016-03-24T11:18:48 <GitHub140> bitcoin/0.12 597494f Jonas Schnelli: Remove openssl info from init/log and from Qt debug window...
3162016-03-24T11:20:01 <GitHub121> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/e5c35119e967...3ba07bdf7da4
3172016-03-24T11:20:02 <GitHub121> bitcoin/master 146746b Alice Wonder: All files related to my RPM spec file project in one commit
3182016-03-24T11:20:02 <GitHub121> bitcoin/master 0e4b50a Alice Wonder: Description of RPM directory
3192016-03-24T11:20:03 <GitHub121> bitcoin/master 3ba07bd Wladimir J. van der Laan: Merge #7609: All files related to my RPM spec file project in one commit...
3202016-03-24T11:20:08 <GitHub36> [bitcoin] laanwj closed pull request #7609: All files related to my RPM spec file project in one commit (master...master) https://github.com/bitcoin/bitcoin/pull/7609
3212016-03-24T11:21:31 <GitHub199> [bitcoin] laanwj closed pull request #7650: Cache CBlockIndex::GetMedianTimePast (master...enhancement/cache-getmediantimepast) https://github.com/bitcoin/bitcoin/pull/7650
3222016-03-24T11:28:48 <GitHub62> [bitcoin] laanwj closed pull request #6973: Compress Blocks before sending (master...compress) https://github.com/bitcoin/bitcoin/pull/6973
3232016-03-24T11:32:34 <sipa> jonasschnelli: i now want to go implement gcm or poly1305...
3242016-03-24T11:32:56 * sipa resists, and works on segwit instead
3252016-03-24T11:50:56 *** fengling has quit IRC
3262016-03-24T11:58:47 <jonasschnelli> sipa: hah. Yes. Work on segwit. The p2p encryption is not urgent.
3272016-03-24T11:59:21 *** fengling has joined #bitcoin-core-dev
3282016-03-24T12:02:26 <jonasschnelli> sipa: regarding GCM IV: «[...] For a fixed value of the key, each IV value must be distinct, but need not have equal lengths [...]»
3292016-03-24T12:03:11 <jonasschnelli> 96 bit is ideal though.
3302016-03-24T12:09:15 <sipa> jonasschnelli: you could get 64 bits of the iv out of ecdh, and then add a 32-bit message counter, but that shouldn't be needed
3312016-03-24T12:15:38 *** laurentmt has joined #bitcoin-core-dev
3322016-03-24T12:28:35 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3332016-03-24T12:38:37 <Chris_Stewart_5> is there a BIP66 post mortem some where about the hard fork?
3342016-03-24T12:39:17 <sipa> BIP66 was not a hard fork... there were some miners that mined invalid blocks, and kept going
3352016-03-24T12:39:48 *** mihi has joined #bitcoin-core-dev
3362016-03-24T12:40:32 <Chris_Stewart_5> sipa: Essentially the miners not enforcing this function right? https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L98
3372016-03-24T12:52:59 <jonasschnelli> sipa: ecdh produces a 256bit secret? right? How would you take 64bits for the iv if you need to 256bits for the symmetric key?
3382016-03-24T13:24:09 *** achow101 has joined #bitcoin-core-dev
3392016-03-24T13:42:58 <jonasschnelli> sipa gmaxwell: I'm happy if you are interested in reviewing the authentication and the encryption BIP (https://github.com/jonasschnelli/bips/blob/7e8a9f6e6a06cec1e7842452c85a9dec3730771b/bip-undef-1.mediawiki https://github.com/jonasschnelli/bips/blob/7e8a9f6e6a06cec1e7842452c85a9dec3730771b/bip-undef-0.mediawiki)
3402016-03-24T13:43:45 <jonasschnelli> Don't review the language/orthography, will overhaul that once the technical details are clear.
3412016-03-24T13:44:27 * jonasschnelli can't tell you how hard it is to talk and write english if its a complete foreign language
3422016-03-24T13:56:29 *** supasonic has joined #bitcoin-core-dev
3432016-03-24T13:58:59 *** mihi has quit IRC
3442016-03-24T14:00:13 <GitHub55> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/3ba07bdf7da4...b88e0b0c610a
3452016-03-24T14:00:13 <GitHub55> bitcoin/master d6cc6a1 João Barbosa: Use CCoinControl selection in CWallet::FundTransaction
3462016-03-24T14:00:14 <GitHub55> bitcoin/master b88e0b0 Wladimir J. van der Laan: Merge #7506: Use CCoinControl selection in CWallet::FundTransaction...
3472016-03-24T14:00:18 <GitHub17> [bitcoin] laanwj closed pull request #7506: Use CCoinControl selection in CWallet::FundTransaction (master...enhancement/use-coin-control-selection) https://github.com/bitcoin/bitcoin/pull/7506
3482016-03-24T14:18:11 <sipa> jonasschnelli: if you want to use a single negotiation to intialize encryption for both directions, you need to make sure that the IVs in bith directions are different, or you break the assumption that an attacker cannot see two different messages with the same key and iv
3492016-03-24T14:20:27 <sipa> jonasschnelli: my suggestion here would be to have an encinit ("if you want, you can start sending encrypted messages to me and this is my key") and an encack ("yeah, every message after this one will be encrypted, and this is my key"), which you both send in both directions
3502016-03-24T14:21:24 <sipa> jonasschnelli: that way, both directions have independent ECDH negotiations, and you also have no problems with synchronization
3512016-03-24T14:22:18 <sipa> jonasschnelli: an alternative is thinking hard about synchronization, and making sure that the initiator and accepter of the encryption connection use separate IVs
3522016-03-24T14:22:45 <sipa> jonasschnelli: reusing the key as IV is not very useful, i think
3532016-03-24T14:24:50 <sipa> jonasschnelli: you probably want to include some text about how this interacts with version/verack (does encinit have to be the first message entirely? can it be done at any time?
3542016-03-24T14:31:31 <sipa> jonasschnelli: your encrypted message protocol spec could make the authentication tag explicit (it's 16 bytes for aes-gcm, though we could decide to truncate it)
3552016-03-24T14:33:09 *** davec has quit IRC
3562016-03-24T14:38:16 *** fengling has quit IRC
3572016-03-24T14:40:49 <jonasschnelli> sipa: thanks for the review! Will "process" your points and come back to you with feedback.
3582016-03-24T14:44:11 *** lysobit has joined #bitcoin-core-dev
3592016-03-24T14:44:41 *** musalbas has joined #bitcoin-core-dev
3602016-03-24T14:54:26 *** BCBot_ has joined #bitcoin-core-dev
3612016-03-24T15:02:43 *** davec has joined #bitcoin-core-dev
3622016-03-24T15:05:32 *** fengling has joined #bitcoin-core-dev
3632016-03-24T15:08:10 *** goregrin1 has joined #bitcoin-core-dev
3642016-03-24T15:08:34 *** gmaxwell_ has joined #bitcoin-core-dev
3652016-03-24T15:08:58 *** gmaxwell_ is now known as Guest18913
3662016-03-24T15:11:43 *** CyrusV` has joined #bitcoin-core-dev
3672016-03-24T15:13:16 *** arubi has quit IRC
3682016-03-24T15:13:16 *** ebfull has quit IRC
3692016-03-24T15:13:17 *** gmaxwell has quit IRC
3702016-03-24T15:13:17 *** Cory has quit IRC
3712016-03-24T15:13:18 *** goregrind has quit IRC
3722016-03-24T15:13:18 *** CyrusV has quit IRC
3732016-03-24T15:13:19 *** windsok has quit IRC
3742016-03-24T15:29:39 *** ebfull has joined #bitcoin-core-dev
3752016-03-24T15:29:39 *** Cory has joined #bitcoin-core-dev
3762016-03-24T15:29:39 *** windsok has joined #bitcoin-core-dev
3772016-03-24T15:30:21 *** Cory has quit IRC
3782016-03-24T15:31:36 *** Guyver2 has joined #bitcoin-core-dev
3792016-03-24T15:33:38 *** Cory has joined #bitcoin-core-dev
3802016-03-24T15:36:28 *** arubi has joined #bitcoin-core-dev
3812016-03-24T15:38:39 *** CyrusV` is now known as CyrusV
3822016-03-24T16:03:57 *** arowser has quit IRC
3832016-03-24T16:04:16 *** arowser has joined #bitcoin-core-dev
3842016-03-24T16:51:24 *** Aleph0 has quit IRC
3852016-03-24T16:54:07 *** Guest18913 has quit IRC
3862016-03-24T16:54:08 *** Guest18913 has joined #bitcoin-core-dev
3872016-03-24T16:54:18 *** Guest18913 is now known as gmaxwell
3882016-03-24T17:01:53 *** Don_John has joined #bitcoin-core-dev
3892016-03-24T17:03:51 *** Don_John has quit IRC
3902016-03-24T17:04:21 *** Don_John has joined #bitcoin-core-dev
3912016-03-24T17:09:53 *** laurentmt has quit IRC
3922016-03-24T17:27:46 <sipa> jonasschnelli: this is interesting: https://github.com/jhcloos/openssh-chacha-poly1305/blob/master/PROTOCOL.chacha20poly1305
3932016-03-24T17:28:10 <sipa> openssh's chacha20-poly1305 aead at a high level
3942016-03-24T17:28:30 <sipa> they encrypt the packet sizes too, but with a separate key
3952016-03-24T17:29:00 <sipa> and then authenticate using the second key which is used for authentication and encryption of the data
3962016-03-24T17:29:28 <sipa> which means that an attacker can't even see the message sizes, except by traffic analysis
3972016-03-24T17:30:02 <btcdrak> Clever
3982016-03-24T17:48:53 <gmaxwell> a little sad to require 4 byte lenghts.
3992016-03-24T17:50:38 *** jgarzik has joined #bitcoin-core-dev
4002016-03-24T17:52:40 *** jcorgan has joined #bitcoin-core-dev
4012016-03-24T17:58:52 *** moli has joined #bitcoin-core-dev
4022016-03-24T17:59:30 <gmaxwell> sipa: for identified links, we could do a nice thing with rekeying to make them strong against ECC breaks.
4032016-03-24T18:00:50 *** molz has quit IRC
4042016-03-24T18:01:17 <sipa> cfields: if we'd want to use something like that, i guess we want an api that allows a client to tell the comnection manager "let me know when you have X bytes"
4052016-03-24T18:01:50 <sipa> cfields: which is oerhaps a cleaner abstraction than hardcoding a particular stream protocol?
4062016-03-24T18:04:00 *** t800 has joined #bitcoin-core-dev
4072016-03-24T18:04:27 <sipa> jonasschnelli: another suggestion: i think we should drop the network magic... resynchronization after garbage hasn't been supported in a long time, it wastes (a tiny bit of) bandwidth, and makes the encrypted stream look decidedly identifiable (though if encrypted connections bootstrap as unencrypted ones, that isn't solvable)
4082016-03-24T18:05:05 <gmaxwell> Alice connects to Bob, Alice sends, Nonce_a, and alice ephemeral pubky (AEP), bob sends nonce_b and BEP. ECDH runs, then a KDF that takes the two nonces, and ecdh output, and spits out a session id and two keys (one for each direction. Then Alice proposes, "I think you have identity X=H(bob pubkey, session_id)", if this is true, bob responds with "I am rekeying with IDX" and takes his cur
4092016-03-24T18:05:11 <gmaxwell> rent encryption key and changes it to H(bob pubkey|enckey). When alice sees that message she updates her keys for the stream. Then also does the same herselve (sends a message, updates key). Then bob sends a signature with his pubkey. Then this authentication procedure can just be repeated every so many bytes in order to rotate the stream cipher keys.
4102016-03-24T18:05:45 <gmaxwell> If the 'public keys' are kept private, the link has at least symmetric keyed integrity and confidentality, even with an ECC break.
4112016-03-24T18:08:06 *** kx has joined #bitcoin-core-dev
4122016-03-24T18:14:45 *** kx has quit IRC
4132016-03-24T18:17:10 *** kangx has joined #bitcoin-core-dev
4142016-03-24T18:23:29 *** treehug88 has joined #bitcoin-core-dev
4152016-03-24T18:25:38 *** MarcoFalke has joined #bitcoin-core-dev
4162016-03-24T18:27:12 *** frankenmint has quit IRC
4172016-03-24T18:28:29 <wumpus> <sipa> which means that an attacker can't even see the message sizes, except by traffic analysis <- I think that would be quite essential for bitcoin, as so much can already be identified from the packet sizes
4182016-03-24T18:28:56 <wumpus> certain blocks, certain transactions, etc
4192016-03-24T18:30:39 *** belcher has joined #bitcoin-core-dev
4202016-03-24T18:31:25 <gmaxwell> I had assumed that the encrypted transport would encapsulate one or more length coded messages inside it... so mostly you'd just see the 'send units'. Doing this would alow avoiding that extra layering, however... and thus decrease overhead.
4212016-03-24T18:32:31 <gmaxwell> though perhaps we should make the encoding support padding, so that messages can be padded up to particular size multiplies, to reduce the traffic analysis sidechannel.
4222016-03-24T18:34:09 <CodeShark> is there a meeting today?
4232016-03-24T18:34:42 <gmaxwell> yes, in about 20 minutes.
4242016-03-24T18:35:01 <gmaxwell> You've suffered DST. :)
4252016-03-24T18:36:04 <btcdrak> we dont suffer it over this side of the Atlantic until Sunday
4262016-03-24T18:37:59 *** moli has quit IRC
4272016-03-24T18:38:30 <btcdrak> gmaxwell: is it maybe worth adding a version byte to the encryption scheme to allow upgrades in the future?
4282016-03-24T18:38:31 <jonasschnelli> gmaxwell: +1 for padding to reduce sidechannel identification.
4292016-03-24T18:39:03 <jonasschnelli> sipa: do you mean dropping the network magic for all post-encryption-init messages?
4302016-03-24T18:39:27 <sipa> also +1 to allowing multiple messages in a packet
4312016-03-24T18:39:49 <jonasschnelli> You mean one header with a message count int followed by n payloads?
4322016-03-24T18:39:51 <sipa> jonasschnelli: yes, it serves no purpose other than detecting accidental connection to the wrong network
4332016-03-24T18:40:05 *** moli has joined #bitcoin-core-dev
4342016-03-24T18:40:07 <sipa> jonasschnelli: or just a concatenation of payloads even
4352016-03-24T18:40:36 <jonasschnelli> Okay. size + payload + size + payload, etc. That makes sense.
4362016-03-24T18:41:44 <sipa> jonasschnelli: inside the aead
4372016-03-24T18:42:03 <gmaxwell> that cuts down on the authentication overhead.
4382016-03-24T18:42:08 <jonasschnelli> sipa: But I still try to understand your AES GCM 96bit IV solution. How would you create distinctable IV without transmitting in in the message?
4392016-03-24T18:43:15 <sipa> jonasschnelli: use 0000 + counter for one side, use 1111 + counter for the other?
4402016-03-24T18:43:39 <jonasschnelli> sipa: Are predictable IVs not a problem?
4412016-03-24T18:43:45 <sipa> jonasschnelli: no, just no reuse
4422016-03-24T18:43:49 <sipa> ivs are nkt secret
4432016-03-24T18:43:52 <sipa> *not
4442016-03-24T18:44:12 <jonasschnelli> Okay. I see.
4452016-03-24T18:44:23 <gmaxwell> sipa: AEAD packet size does imply memory usage implications though.
4462016-03-24T18:45:16 <gmaxwell> but its maximum could just be the same as the largest message.
4472016-03-24T18:45:20 <sipa> right
4482016-03-24T18:45:44 <jonasschnelli> So. Changes i'll work on a) ecdh exchange for both direction, two sym. keys, fix synchro. b) use chacha20 poly1305 instead of AES GCM c) optimize message structure (drop magic, allow multiple payloads in the AEAD part)
4492016-03-24T18:46:17 <sipa> jonasschnelli: no need to fix synchronization if both directions are independent
4502016-03-24T18:46:58 <jonasschnelli> yes. I mean fix syncho by using two keypair, one per direction.
4512016-03-24T18:47:04 *** jcorgan has quit IRC
4522016-03-24T18:47:59 <gmaxwell> great.
4532016-03-24T18:47:59 <jonasschnelli> Has chacha20-poly1305 been proven to be stable? Reasonable amount of cryptanalysis? Or do we threat stuff out of DJB's kitchen as stable anyways? :)
4542016-03-24T18:48:05 *** gevs has quit IRC
4552016-03-24T18:48:18 <gmaxwell> jonasschnelli: it's part of TLS and SSH.
4562016-03-24T18:48:28 <gmaxwell> I think it's sutiable for our purposes.
4572016-03-24T18:48:31 <wumpus> it's geting a lot of testing and review in ssh at least
4582016-03-24T18:49:00 <jonasschnelli> Okay. And how complex is an implementation? Or would this lead to a revival of openssl in core?
4592016-03-24T18:49:12 <gmaxwell> it's very simple.
4602016-03-24T18:49:26 <wumpus> their implementation is very short, that's one thing that is so cool about it
4612016-03-24T18:49:29 <jonasschnelli> what speaks again unsing openssl (p2p layer enc is non consensus)?
4622016-03-24T18:49:47 <jonasschnelli> *against
4632016-03-24T18:50:07 <gmaxwell> we're not taking 400,000+ lines of unreviewable, frequently CVEed code for this; I'd rather not have the functionality.
4642016-03-24T18:50:11 <wumpus> attack surface, making it impossible to get rid of openssl dependency
4652016-03-24T18:50:44 <wumpus> and you can already do it using stunnel
4662016-03-24T18:51:22 <gmaxwell> I hope that this works so that even if we don't care about the confidentiality improvement, it's just simply faster than the existing transport.
4672016-03-24T18:52:06 <jonasschnelli> Right. I think dropping the sha256 per message could be an improvement in performance.
4682016-03-24T18:52:24 <jonasschnelli> combining multiple messages into one encrypted message also
4692016-03-24T18:53:03 <jonasschnelli> The question is, how to "bundle" messages (example: bundle invs). When releasing them, etc. I mean for the implementation.
4702016-03-24T18:53:13 <gmaxwell> https://www.imperialviolet.org/2014/02/27/tlssymmetriccrypto.html has some benchmarks vs AES-GCM fwiw.
4712016-03-24T18:53:17 <sipa> jonasschnelli: we already do that
4722016-03-24T18:53:38 <sipa> jonasschnelli: SendMessages is a function in main that computes a block of messages to send at once
4732016-03-24T18:53:44 <jonasschnelli> Ah. But they get sent individually (one inv per msg)?
4742016-03-24T18:54:02 <sipa> jonasschnelli: maybe you should look into how our current protocol works :)
4752016-03-24T18:54:08 <jonasschnelli> haha. True.
4762016-03-24T18:54:23 <sipa> no, an inv messages can contain up to 5000 invs
4772016-03-24T18:55:00 <jonasschnelli> Okay. So where would we benefit from bundling messages in one encrypted message?
4782016-03-24T18:55:20 <wumpus> it makes sense to make it possible, it benefits privacy
4792016-03-24T18:55:58 *** achow101 has quit IRC
4802016-03-24T18:55:59 <sipa> well, if you take as baseline an implementation that encrypts message sizes, there is no privacy benefit to bundling
4812016-03-24T18:56:04 <wumpus> even if your first implementation doesn't actually use it, imo the protocol should support it
4822016-03-24T18:56:16 <gmaxwell> phantomcircuit had a patch to add corking and uncoarking in the message handling loop, I presume something similar would be done for message batching.
4832016-03-24T18:56:18 <jonasschnelli> agree
4842016-03-24T18:56:20 *** achow101 has joined #bitcoin-core-dev
4852016-03-24T18:56:37 <wumpus> ok, yes, no privacy benefit to bundling makes it less important to me
4862016-03-24T18:56:39 <sipa> as they would be sent back-to-back, so traffic analysis would not let you discover independent messages
4872016-03-24T18:56:51 <gmaxwell> e.g. we'd cork both the crypto and tcp, do all the message handling for a peer, then uncork both.
4882016-03-24T18:57:32 <sipa> however, bundling does give a small bandwidth improvement, as you'd share a authentication tag acorss muktiple messages
4892016-03-24T18:57:50 <wumpus> well, let's not micro-optimize before we have anything
4902016-03-24T18:57:56 <jonasschnelli> tag is 12bytes. Right.
4912016-03-24T18:58:05 <gmaxwell> not necessarily all that small, the auth is relatively large to our average message size.
4922016-03-24T18:58:38 <wumpus> let's first make sure it is secure, then wonder about bandwidth optimization
4932016-03-24T18:59:11 <jonasschnelli> Right. I'll try to update the Bips with what we have discusses and ask for another review.
4942016-03-24T18:59:20 <jonasschnelli> *discussed
4952016-03-24T18:59:23 <wumpus> trying to tackle everything at once is going to make this go nowhere, I'm afraid
4962016-03-24T18:59:31 <gmaxwell> the bundling also makes padding easy, you just define a new message type "pad" that is just discarded on the other end. :)
4972016-03-24T18:59:37 <sipa> jo 16 if you want 128 bit security
4982016-03-24T18:59:40 *** molz has joined #bitcoin-core-dev
4992016-03-24T18:59:44 <gmaxwell> yea, iteration is fine.
5002016-03-24T18:59:46 <jonasschnelli> should i say now: "perfect is the enemy of good". Yes? No. *duck*
5012016-03-24T19:00:04 <MarcoFalke> meeting time?
5022016-03-24T19:00:10 <gmaxwell> Meeting time.
5032016-03-24T19:00:10 <wumpus> jonasschnelli: more like: keep your goal focused. Your goal is security and privacy, first. Make no dumb mistakes there ;)
5042016-03-24T19:00:24 <jonasschnelli> RIght!
5052016-03-24T19:00:25 <wumpus> #startmeeting
5062016-03-24T19:00:25 <lightningbot> Meeting started Thu Mar 24 19:00:25 2016 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
5072016-03-24T19:00:25 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
5082016-03-24T19:00:47 <wumpus> jonasschnelli: scope creep is a very bitcoin-y thing
5092016-03-24T19:00:52 <wumpus> ok, topic proposals?
5102016-03-24T19:01:05 <MarcoFalke> python 3 and rpc tests?
5112016-03-24T19:01:14 <wumpus> from last meeitng we have ACTION: merge #7575 (wumpus, 19:04:04) ... that was done
5122016-03-24T19:01:19 <btcdrak> softfork #7648 status
5132016-03-24T19:01:31 <wumpus> next was ACTION: review #7648 after #7575 is merged (wumpus, 19:09:39)
5142016-03-24T19:01:32 <Luke-Jr> topic proposal: v0.11.3 and v0.10.5
5152016-03-24T19:01:34 *** moli has quit IRC
5162016-03-24T19:01:42 <wumpus> how is that going?
5172016-03-24T19:01:53 <wumpus> #topic softfork #7648 status
5182016-03-24T19:02:08 <wumpus> MarcoFalke: will get to python3 at the end, I think the softfork stuff is most important right now
5192016-03-24T19:02:12 <btcdrak> #7648 has received some tested ACKs.
5202016-03-24T19:02:37 <MarcoFalke> sure
5212016-03-24T19:03:08 <wumpus> okay, good
5222016-03-24T19:03:46 <wumpus> I think it needs more review still
5232016-03-24T19:04:15 <btcdrak> time to crack the whip.
5242016-03-24T19:04:24 <wumpus> for a softfork the bar is somewhat higher than for random pull #1234
5252016-03-24T19:04:39 <wumpus> yes
5262016-03-24T19:05:03 <jonasschnelli> 7648 looks good and will also review it in details soon (during weekend)
5272016-03-24T19:05:08 <wumpus> I mean I've seen some acks but very little comment on the actual code, that could be a sign everything is perfect, or people need to look better
5282016-03-24T19:05:29 <wumpus> thanks jonasschnelli
5292016-03-24T19:05:40 <btcdrak> MarcFalke: can you look as well please?
5302016-03-24T19:05:42 <wumpus> #action review more at https://github.com/bitcoin/bitcoin/pull/7648
5312016-03-24T19:05:47 <btcdrak> and CodeShark
5322016-03-24T19:05:52 <wumpus> I'll take a more in-depth look at it as well
5332016-03-24T19:05:55 <btcdrak> ty
5342016-03-24T19:05:57 <MarcoFalke> sure
5352016-03-24T19:06:08 <gmaxwell> has anyone looked to see if there are MTP violations still being accepted on the network?
5362016-03-24T19:06:39 <gmaxwell> I wouldn't consider them a hard blocker to 113 deployment, but if any are, we should make a concerted effort to extinguish them first.
5372016-03-24T19:06:59 <btcdrak> gmaxwell: there should be 100% coverage of 113 since the upgrade to 0.11.2
5382016-03-24T19:07:16 <gmaxwell> btcdrak: there may be miners with order or modified code, however.
5392016-03-24T19:07:28 <gmaxwell> er, s/order/older/
5402016-03-24T19:07:51 <sipa> s/may be/are/
5412016-03-24T19:08:25 <gmaxwell> if there are any, we should try to find them and get them to stop mining MTP violations.
5422016-03-24T19:09:34 <wumpus> yes
5432016-03-24T19:09:39 <gmaxwell> I can work on this.
5442016-03-24T19:09:49 <cfields> sipa: missed you earlier. ack on allowing for chunked decrypt/decode.
5452016-03-24T19:10:02 <btcdrak> cfields: can you put 7648 review on your tasklist too please?
5462016-03-24T19:10:08 <wumpus> #action hunt down miners that mine MTP violations
5472016-03-24T19:10:35 <cfields> btcdrak: yes
5482016-03-24T19:11:34 <wumpus> ok
5492016-03-24T19:11:54 <wumpus> next topic I guess
5502016-03-24T19:12:01 <wumpus> #topic proposal: v0.11.3 and v0.10.5
5512016-03-24T19:12:29 <sipa> we discussed what would go in last week; did anythkng chang
5522016-03-24T19:12:30 <sipa> ?
5532016-03-24T19:13:03 <wumpus> I don't know, luke-jr requested the topic
5542016-03-24T19:13:10 <btcdrak> well we didnt discuss 0.10.5, we discussed 0.12.1
5552016-03-24T19:13:12 <Luke-Jr> I'm going through https://github.com/bitcoin/bitcoin/pull/7047 again, to check everything included is in 0.12 and make it mergable again
5562016-03-24T19:14:00 <wumpus> that's way too much for a softfork release at least
5572016-03-24T19:14:01 <Luke-Jr> what isn't clear to me is what the plan is for v0.11.3 being released - I assume wumpus is doing that at some point? - and whether I'm taking over for v0.10.5, or if wumpus wanted to do one final 0.10 before passing it off
5582016-03-24T19:14:27 <wumpus> for a normal minor release it'd make more sense
5592016-03-24T19:14:52 <wumpus> yes I'll do the 0.11.x release
5602016-03-24T19:14:58 <MarcoFalke> 7047 also includes a lot of "comment-fixed". I am not sure if they are useful in our branch
5612016-03-24T19:15:01 <gmaxwell> whats deployment of 0.11.2 vs 0.12 look like right now?
5622016-03-24T19:15:06 <MarcoFalke> I'd rather only have bugfixes
5632016-03-24T19:15:08 <wumpus> I'll leave 0.10.x to you
5642016-03-24T19:15:08 <Luke-Jr> are we doing a softfork release in the next month?
5652016-03-24T19:15:21 <sipa> Luke-Jr: i say yes
5662016-03-24T19:15:32 <Luke-Jr> ok, so then we should let these things wait until after that
5672016-03-24T19:15:38 <wumpus> I think we should do a softfork release asap
5682016-03-24T19:15:50 <Luke-Jr> or maybe I should go through it and make sure there's nothing important
5692016-03-24T19:15:52 <wumpus> let's just get those stupid things reviewed and merged
5702016-03-24T19:15:57 <jonasschnelli> As soon as #7648 has been reviewed more and merged
5712016-03-24T19:16:10 <wumpus> jonasschnelli: and backports reviewed and merged ofc
5722016-03-24T19:16:21 <wumpus> but yes 7648 is the first step
5732016-03-24T19:16:31 <jonasschnelli> RIght
5742016-03-24T19:16:32 <Luke-Jr> (also, depending on ease of backporting the softforks, I may decide to just give up on 0.10 support; tbd)
5752016-03-24T19:16:42 <btcdrak> /Satoshi:0.12.0/ 1713
5762016-03-24T19:16:43 <btcdrak> /Satoshi:0.11.2/ 1387
5772016-03-24T19:16:50 <jonasschnelli> 0.10 can be given up IMO
5782016-03-24T19:16:55 * jonasschnelli is checking his seeder data
5792016-03-24T19:17:01 <gmaxwell> Luke-Jr: I think absent someone requesting otherwise (and perhaps funding your effort) you probably should.
5802016-03-24T19:17:08 <Luke-Jr> jonasschnelli: it's currently Gentoo stable, so at the very least I will need to bump that
5812016-03-24T19:17:21 <btcdrak> The problem with 0.10 is we all need to put review time into the backports and that's difficult when we have trouble even reviewing backports for 0.11
5822016-03-24T19:17:37 <petertodd> btcdrak: +1
5832016-03-24T19:17:56 <Luke-Jr> btcdrak: most backports are trivial, but for softforks I agree if they're not a clean backport
5842016-03-24T19:18:04 <wumpus> right
5852016-03-24T19:18:12 <btcdrak> I think if there is a 0.10.5 release it should be extremely minimal... and frankly, backporting BIP68 wont be fun or easy.
5862016-03-24T19:18:14 <sipa> bip68/112/113 are quite a bit of code
5872016-03-24T19:18:28 <wumpus> let's spend our energy on moving forward
5882016-03-24T19:18:33 <jonasschnelli> 0.10 should be last priority. If someone offers to do that: fine.
5892016-03-24T19:18:41 <gmaxwell> plus, a far backport which is done by one person and used by few more likely won't be materially more stable than a more recent release.
5902016-03-24T19:18:44 <Luke-Jr> ok, so for 0.10 I'll see the complexity in the softfork backports, and if it's anything more than absolutely trivial, I'll just decide to drop it
5912016-03-24T19:18:47 <btcdrak> Luke-Jr: even BIP68 backport to 0.11 was tricky, not a clean cherry-pick.
5922016-03-24T19:18:58 <Luke-Jr> btcdrak: well, I'd be backporting to 0.10 *from* 0.11
5932016-03-24T19:19:19 <jonasschnelli> at dnsseed.dump | grep " 100.00% 100.00%" | grep "/Satoshi:0.10." | wc -l ---> 266
5942016-03-24T19:19:25 <sipa> we can still do critical security updates for 0.10 without choosing to backport sotfork
5952016-03-24T19:19:33 <wumpus> sure
5962016-03-24T19:19:45 <jonasschnelli> 6.3% 0.10.x nodes
5972016-03-24T19:19:49 <btcdrak> so on this topic, people need to also look at backport #7543
5982016-03-24T19:19:58 <Luke-Jr> and for 0.11, sounds like the plan is to do a softfork-only release before any non-essential bugfix backports
5992016-03-24T19:20:08 <CodeShark> +1 jonasschnelli
6002016-03-24T19:20:09 <petertodd> sipa: and people who need a v0.10.x node for software compatibility reasons can generally run it behind a newer node
6012016-03-24T19:20:18 <sipa> petertodd: also true
6022016-03-24T19:20:31 <wumpus> #action people need to also look at backport #7543: [0.12] Backport BIP9, BIP68 and BIP112 with softfork
6032016-03-24T19:20:36 <sipa> petertodd: specifically, behind a 0.11 or 0.12 pruned one :)
6042016-03-24T19:20:45 <petertodd> sipa: yes! that too
6052016-03-24T19:20:57 <Luke-Jr> #action 0.10: attempt to backport softforks, and if non-trivial, discontinue support
6062016-03-24T19:21:12 <Luke-Jr> #action 0.11: no non-essential fix backports until softfork release
6072016-03-24T19:21:26 <Luke-Jr> #action bump Gentoo stable to 0.11 (from 0.10)
6082016-03-24T19:21:38 <sipa> Luke-Jr: support is already discontinued according to policy, only critical biffixes
6092016-03-24T19:21:42 <sipa> *bugfixes
6102016-03-24T19:21:51 <CodeShark> I'd say discontinue support for 0.10 unless someone wishes to provide resources for it
6112016-03-24T19:21:59 <wumpus> well if luke-jr wants to try to backport the softfork that's up to him
6122016-03-24T19:21:59 <Luke-Jr> sipa: policy only dictates guaranteed support, not guaranteed to end
6132016-03-24T19:22:22 <sipa> wumpus: sure, but i don't think that's very relevant here
6142016-03-24T19:22:27 <wumpus> no need to even discuss it here :p
6152016-03-24T19:22:28 <wumpus> right
6162016-03-24T19:22:44 <btcdrak> #action 0.11 review #7716 Backport BIP9 and softfork for BIP's 68,112,113
6172016-03-24T19:22:58 <Luke-Jr> any more topics?
6182016-03-24T19:23:32 <sipa> as soon as we have 0.12.1, i will rebase segwit on top, so segnet can benefit from its bip9/68/112/113 support
6192016-03-24T19:23:37 <cfields> can give a quick update on the net refactor
6202016-03-24T19:23:50 <wumpus> I'd say spend the rest of the meeting time reviewing #7648 :p
6212016-03-24T19:23:53 <wumpus> sure cfields!
6222016-03-24T19:24:00 <wumpus> #topic cfields net refactor
6232016-03-24T19:24:34 <sipa> please, stick to the net refactor, not the gross one
6242016-03-24T19:24:56 <cfields> i finally have a working full-featured branch. it's a mess code-wise, but i think it should be approachable in the next day or two after some code movement. I expect to be looking for concept review next week
6252016-03-24T19:24:58 <gmaxwell> cfields: start taking, I can't take the wait.
6262016-03-24T19:25:02 <wumpus> lol
6272016-03-24T19:25:23 <wumpus> cfields: I've been testing it on a node btw for last week, seems to be stable and working
6282016-03-24T19:25:48 <wumpus> haven't looked at details of performance etc but no crashes or insane log entries
6292016-03-24T19:26:02 <btcdrak> sipa: you can cherry-pick from #7543 in that case
6302016-03-24T19:26:10 <cfields> wumpus: that's great to hear. lots has changed since then, but the core is the same
6312016-03-24T19:26:35 <cfields> i think i have an idea of how to handle code review, but i'd like to get some concept acks first...
6322016-03-24T19:26:50 <wumpus> you got my concept ack at the boston meeting
6332016-03-24T19:26:56 <sipa> and mine
6342016-03-24T19:27:09 <cfields> there's a huge mix of code movement, a new lib, abstraction, etc. I think i have a plan for getting it slotted in
6352016-03-24T19:27:28 <wumpus> great
6362016-03-24T19:27:29 <cfields> ok. i can start PRing chunks into core as i go, then
6372016-03-24T19:27:44 <Luke-Jr> cfields: practical to add block streaming? ;)
6382016-03-24T19:28:02 *** frankenmint has joined #bitcoin-core-dev
6392016-03-24T19:28:02 <cfields> first step is getting the net stuff instanced. that's a lot of movement without much functional difference
6402016-03-24T19:28:02 <gmaxwell> do not creep cfield's scope. :P
6412016-03-24T19:28:06 <wumpus> please, let's replace the current things first, then add additional features
6422016-03-24T19:28:08 <jonasschnelli> hah
6432016-03-24T19:28:09 <wumpus> example gmaxwell
6442016-03-24T19:28:12 <wumpus> exactly*
6452016-03-24T19:28:13 <cfields> i think that can start going in parallel
6462016-03-24T19:28:37 <cfields> gmaxwell: heh, i've adjusted scope so many times now that i'm certainly not budging again :p
6472016-03-24T19:28:44 <jonasschnelli> The net refactor will be very hard to review already. Lets keep it as simple as possible.
6482016-03-24T19:28:51 <wumpus> that's good, stand your ground , people here are awful :)
6492016-03-24T19:28:57 <cfields> turns out the absolute minimum scope for net refactor is already enormous
6502016-03-24T19:29:07 <jonasschnelli> Indeed.
6512016-03-24T19:29:29 <wumpus> yes it'll already be challenging to get this done before 0.13 I think, but we have to try
6522016-03-24T19:29:31 <Luke-Jr> gmaxwell: indeed, just hope we don't need to refactor it again after
6532016-03-24T19:29:41 <sipa> Luke-Jr: we absolutely will
6542016-03-24T19:29:48 <cfields> wumpus: yes, it's much later than I was hoping for. If it slips past 0.13, I'll understand.
6552016-03-24T19:29:51 <sipa> but at different layers :)
6562016-03-24T19:30:04 <wumpus> right, this is only the bottom layer
6572016-03-24T19:30:19 <cfields> sipa: well the hope is that it's been split up into chunks that can be changed in the future much easier
6582016-03-24T19:30:37 <sipa> cfields: yes, that's what abstractions are for
6592016-03-24T19:31:27 <jonasschnelli> topic: Should we shorty touch how we should proceed with sipas CT AES implementation? Extract to library? Yes/No?
6602016-03-24T19:31:40 <cfields> there are a few hacks that i'm not going to mess with for now. For ex, even though it's all instanced, there's still a g_connman that's used liberally as opposed to the clean way of passing the instance around
6612016-03-24T19:31:42 <wumpus> meh
6622016-03-24T19:31:55 <cfields> i'll start working on a list of todos so that some of the decisions are more clear
6632016-03-24T19:32:24 <petertodd> jonasschnelli: extract is a great idea I think - don't have to be fancy, just enough to continue to set a better standard than "roll our own"
6642016-03-24T19:32:36 <sipa> cfields: just PR it! i want to see the code :)
6652016-03-24T19:32:47 *** frankenmint has quit IRC
6662016-03-24T19:32:49 <jonasschnelli> sipa: code: https://github.com/theuni/bitcoin/tree/net-refactor8
6672016-03-24T19:32:52 <cfields> sipa: i'm still frantically coding it :)
6682016-03-24T19:32:58 <sipa> ah.
6692016-03-24T19:33:07 <cfields> jonasschnelli: nooooooo. everyone shield your eyes from that :)
6702016-03-24T19:33:07 <wumpus> the code is actually buildable and works
6712016-03-24T19:33:14 <sipa> now i have a moral obligation to actually go look, right?
6722016-03-24T19:33:28 <wumpus> well test at least :)
6732016-03-24T19:33:33 <btcdrak> cfields: love that last commit message :)
6742016-03-24T19:33:35 <cfields> sipa: with the understanding that it's basically my /tmp :)
6752016-03-24T19:33:35 <jonasschnelli> cfields: We all know how code during implementation can look. No shame for it!
6762016-03-24T19:34:09 <wumpus> yes :)
6772016-03-24T19:34:31 <jonasschnelli> petertodd: I'm also for extracting.
6782016-03-24T19:34:54 <jonasschnelli> Simple autotools setup, add as subtree place on bitcoin/libctaes (or similar)
6792016-03-24T19:34:57 <wumpus> #topic CT AES library
6802016-03-24T19:35:12 <Luke-Jr> what is "CT" for here?
6812016-03-24T19:35:14 <wumpus> I don't think extracting it into a library has to affect our use of it
6822016-03-24T19:35:14 <petertodd> jonasschnelli yup, subtree is fine
6832016-03-24T19:35:17 <wumpus> Constant Time
6842016-03-24T19:35:18 <jonasschnelli> constant time
6852016-03-24T19:35:24 <Luke-Jr> ah
6862016-03-24T19:35:45 <sipa> i feel like it's more a code snippet than a library
6872016-03-24T19:35:46 <gmaxwell> constant time.
6882016-03-24T19:35:53 <wumpus> yes, it's more like a snippet
6892016-03-24T19:35:56 <gmaxwell> "Petertodd raised verification concerns for the internal AES implementation. I could suggest some strategies on that." is what I wrote earlier. :P
6902016-03-24T19:35:59 <wumpus> crypto has a rich tradition of copy/paste
6912016-03-24T19:36:11 <cfields> fwiw, I plan on doing the same with libbtcnet
6922016-03-24T19:36:13 <jonasschnelli> wumpus: I think placing stuff in libraries follows a modular approach which is desirable for Bitcoin IMO
6932016-03-24T19:36:24 <Luke-Jr> +1 for libctaes
6942016-03-24T19:36:29 *** jcorgan has joined #bitcoin-core-dev
6952016-03-24T19:36:34 <cfields> i have a tiny/crappy makefile that i use to build it externally, but I plan to just copy/paste it into Core and keep it in sync
6962016-03-24T19:36:35 <petertodd> sipa: well, to be clear, you did write that code snippet from scratch yourself right?
6972016-03-24T19:36:35 <wumpus> you could stil include it in a library, but please don't compilicate the build
6982016-03-24T19:36:40 <Luke-Jr> cfields: libbcnet I hope; what does the BTC unit size have to do with networking? :P
6992016-03-24T19:36:49 <gmaxwell> in any case, I'm not convinced that the standard test vectors prove the code correct. Beyond soliciting outside review, which btcdrak has done. I was going to suggest trying some mutation testing to see if some additional better test vectors can be constructed.
7002016-03-24T19:36:54 <jonasschnelli> sipa: Its a snipped now, ... but we all know how snippets evolve over time.
7012016-03-24T19:37:00 <cfields> Luke-Jr: when i googled "libp2p" some etherium thing came up :)
7022016-03-24T19:37:01 <sipa> petertodd: yes, though the logic gates for the sbox come from a paper
7032016-03-24T19:37:14 <wumpus> I think we should separate concerns here: getting the code into a library, and getting the damn code reviewed
7042016-03-24T19:37:18 <wumpus> the first is easy, just work
7052016-03-24T19:37:18 <Luke-Jr> cfields: oh great, let's just integrate that! :P
7062016-03-24T19:37:37 <petertodd> sipa: right, so it's independent work, which means it's crypto we wrote, which means no matter how small it is, we should at least make a token effort to try to let independent use and review :)
7072016-03-24T19:37:38 <Luke-Jr> wumpus: I don't feel competent to review crypto code :<
7082016-03-24T19:37:49 <wumpus> Luke-Jr: me neither.
7092016-03-24T19:38:01 <wumpus> Luke-Jr: so we'll have to find someone that can, btcdrak had some ideas.
7102016-03-24T19:38:04 <gmaxwell> You're all competent to review if the code is constant time or not.
7112016-03-24T19:38:16 <sipa> or that it has test coverage
7122016-03-24T19:38:16 <jonasschnelli> What about apoelstra?
7132016-03-24T19:38:32 <gmaxwell> And probably no one is competent to review if it's correct, fortunately AES is a permutation, which does make confidence building via testing easier.
7142016-03-24T19:38:40 <wumpus> but not if it's correct AES, or has some weird edge cases, mathatical inconsistencies, or other wacky side channels
7152016-03-24T19:38:55 <wumpus> right
7162016-03-24T19:39:02 <sipa> and notice it has no tables or branches, so it would be exceedingly hard to make it broken in a very small number of cases
7172016-03-24T19:39:04 <jonasschnelli> IMO that all speak for splitting it off into a own library.
7182016-03-24T19:39:04 <petertodd> gmaxwell: the problem here, is I don't know enough about the problem to know if I'm actually competent to review that
7192016-03-24T19:39:07 <gmaxwell> (by no one, I mean no human. :) )
7202016-03-24T19:39:35 <btcdrak> petertodd: I asked isis to take a look on a paid basis.
7212016-03-24T19:39:41 <petertodd> btcdrak: good!
7222016-03-24T19:39:48 <gmaxwell> petertodd: the requirement is that there should be no leakage of the secret state (key or data) in the timing of the program on any hardware we support.
7232016-03-24T19:39:53 <btcdrak> that's @isislovecruft from Tor for anyone that doesnt know.
7242016-03-24T19:40:01 * Luke-Jr volunteers to do the splitting-off-into-library work if sipa doesn't want to
7252016-03-24T19:40:18 <sipa> anyway, i'll turn it into a .c/.h; if someone else wants to actually do the work for autotools etc, go ahead.
7262016-03-24T19:40:19 <jonasschnelli> Luke-Jr: I already have started that. :)
7272016-03-24T19:40:28 <wumpus> Luke-Jr: this will again trigger the subtree-or-external discussion, I can't take that another time
7282016-03-24T19:40:45 <wumpus> Luke-Jr: and linux distributions messing it up etc
7292016-03-24T19:40:51 <gmaxwell> it will not be external.
7302016-03-24T19:40:53 <Luke-Jr> wumpus: seems that discussion should be project-global, not per-lib
7312016-03-24T19:40:56 <jonasschnelli> sipa: I have checked to code and it looks pretty Cish.
7322016-03-24T19:41:05 <Luke-Jr> subtree-with-configure-option seems to work
7332016-03-24T19:41:12 <sipa> imho, it can be a separate repository without being a library
7342016-03-24T19:41:22 <sipa> just copy it into your project if you neeed it
7352016-03-24T19:41:27 <wumpus> right.
7362016-03-24T19:41:38 <sipa> it's a single file and has no intention to grow beyond that
7372016-03-24T19:41:56 <gmaxwell> nor any utility to grow beyond that.
7382016-03-24T19:41:59 <jonasschnelli> meh. Adding a build setup should be trivial and would hurt nobody?
7392016-03-24T19:42:02 <cfields> sipa: is there enough non-standard stuff to necessitate a buildsystem? or would a simple makefile do?
7402016-03-24T19:42:11 <wumpus> the API will never change, the implementation will hopefully also never change (excluding critical problems)
7412016-03-24T19:42:22 <sipa> cfields: it will be a single C89 file with no dependencies beyond stdint
7422016-03-24T19:42:36 <Luke-Jr> stdint isn't C89, though?
7432016-03-24T19:42:36 <gmaxwell> wumpus: the only 'change' I could imagine for it is wrapping it with something that uses AESNI when available instead.
7442016-03-24T19:42:48 <sipa> Luke-Jr: it indeed isn't, which is why i mention it :)
7452016-03-24T19:42:50 <cfields> sipa: ah great
7462016-03-24T19:42:56 <gmaxwell> well C89+stdint, or C99...
7472016-03-24T19:43:19 <gmaxwell> stdint is almost universal even in terrible embedded complilers.
7482016-03-24T19:43:28 <jonasschnelli> Are there no plans to extend the "lib" with more stuff? Like for a possible p2p encryption?
7492016-03-24T19:43:36 <wumpus> and if not it's easy enough to define the types yourself
7502016-03-24T19:43:53 <Luke-Jr> jonasschnelli: well, then the name libctaes would be wrong
7512016-03-24T19:43:57 <sipa> jonasschnelli: i'd do that on another layer
7522016-03-24T19:43:58 <jonasschnelli> Right.
7532016-03-24T19:44:03 <wumpus> no, jonasschnelli, that should be something else yet again
7542016-03-24T19:44:05 <sipa> jonasschnelli: it would still rely on aes primitives
7552016-03-24T19:44:27 <gmaxwell> This pressure to constantly create "libraries" to dump things in is streesful for me. A well constructed library is a serious amount of work in and of itself.
7562016-03-24T19:44:29 <wumpus> ctaes would be constant time aes :-) I don't think you want/need that for the p2p
7572016-03-24T19:44:31 <petertodd> gmaxwell: I may have to use a climbing analogy to get my point accross here :) sure, I can easily read a book on knots and belay systems and feel like I _should_ know everything I need to know to do that stuff safely, but the consequences of failure are high enough - and the field unusual enough - that I'm going to insist I check things out with others generally accepted as experts first - who cares how irrational ...
7582016-03-24T19:44:32 <jonasschnelli> Okay. Then we might agree in a subtree with just a .c/.h instead of a autotools setup.
7592016-03-24T19:44:37 <petertodd> ... that feels. I think general standards for low-level crypto implementations tend in that direction.
7602016-03-24T19:45:05 <wumpus> jonasschnelli: yes
7612016-03-24T19:45:06 <petertodd> jonasschnelli: yes, just a .c/.h is fine! like I said, bare minimum to set a precedent that we're trying to invite outside review
7622016-03-24T19:45:08 <Luke-Jr> subtree with just .c/.h until some future time when someone decides to lib-ify it seems ok
7632016-03-24T19:45:11 <wumpus> jonasschnelli: that's very common for crypto
7642016-03-24T19:45:18 <sipa> and maybe for performance reasons, it makes sense to use a more complicated faster implementation, and maybe also warrants more maintainance afterwards if it's going to be adopted by other prijects for the purpose of p2p encryption
7652016-03-24T19:45:31 <jonasschnelli> okay.
7662016-03-24T19:45:43 <gmaxwell> petertodd: considering that basically all AES-CBC software (not AESNI) out there is gratitiously timing vulnerable, and we were going to take such a patch ourselves before; I feel you're compent enough to review and increase confidence, if not alone to achieve perfection.
7672016-03-24T19:45:47 <Luke-Jr> sipa: seems likely other wallets may use it
7682016-03-24T19:45:47 <jonasschnelli> agree on bitcoin/libctaes?
7692016-03-24T19:45:48 <wumpus> and just copy/subtree the thing into your project and include it into your build system, all the dependency micro management isnt' worth it for everything
7702016-03-24T19:46:05 <sipa> Luke-Jr: they absolutely may!
7712016-03-24T19:46:16 <jonasschnelli> Luke-Jr: right. I have two projects where i will use it immediately.
7722016-03-24T19:46:21 <wumpus> jonasschnelli: bitcoin-core/libctaes it would be then
7732016-03-24T19:46:27 <gmaxwell> as I noted before, this implementation is not super fast. We have no need to make it super fast. It may be of idependant interest to others because it is quite small.
7742016-03-24T19:46:29 <sipa> Luke-Jr: but for wallets, it has no performance requirements
7752016-03-24T19:46:36 <jonasschnelli> wumpus: ah. Right. The moving.
7762016-03-24T19:46:56 <sipa> Luke-Jr: which is why it's easy, and needs no dependencies on assemblers or instruction sets, or simd extensions, or ...
7772016-03-24T19:47:13 * Luke-Jr not sure libctaes makes sense as bitcoin-core as opposed to just bitcoin, but meh
7782016-03-24T19:47:38 <petertodd> gmaxwell: yeah, I probably am - I also learned how to do climbing/caving ropework by buying a book on it, some equipment, and going out to find some trees to practice on. Doesn't mean it was a good idea. :) just having a separate repo with a bare .c/.h goes a long way to making our intentions clear and having good precedents, even if pragmatically we need a solution right now.
7792016-03-24T19:47:38 <wumpus> jonasschnelli: which reminds me, I think I still need to add you to that project
7802016-03-24T19:47:49 <jonasschnelli> bitcoin/ should be consensus critical stuff
7812016-03-24T19:47:52 <wumpus> jonasschnelli: we're not actually doing anything with it at the moment
7822016-03-24T19:47:59 <wumpus> jonasschnelli: right
7832016-03-24T19:48:09 <jonasschnelli> +1 ;-)
7842016-03-24T19:48:20 <cfields> topic has strayed a bit :)
7852016-03-24T19:48:35 <wumpus> yes
7862016-03-24T19:48:55 <sipa> petertodd: ok, agreed
7872016-03-24T19:49:09 <petertodd> sipa: thanks!
7882016-03-24T19:49:28 <wumpus> #topic convert python RPC tests to python 3
7892016-03-24T19:49:29 <gmaxwell> I am disappointed that all of this time is being wasted over faffing about libraries, for code that has very narrow independant interest and likely against the wishes of its author. .. and no one commented on further verification strategy.
7902016-03-24T19:49:49 * gmaxwell sees topic is over.
7912016-03-24T19:50:00 <MarcoFalke> I think it's cleaner to just switch to py3
7922016-03-24T19:50:04 <petertodd> wumpus: concept ack!
7932016-03-24T19:50:26 <MarcoFalke> but I'd like to hear any drawbacks first.
7942016-03-24T19:50:30 <wumpus> gmaxwell: I agree, I think this will do just as well as a code snippet, but I don't think discussing this further is constructive. The people that want a library so bad can work on it I guess :)
7952016-03-24T19:50:41 <Luke-Jr> py3 vs py2, I can think of no drawbacks to exclusively py3
7962016-03-24T19:50:44 <wumpus> as you may have read in https://github.com/bitcoin/bitcoin/issues/7717, the next release of ubuntu, 16.04, xenial something will no longer ship python 2
7972016-03-24T19:50:47 <btcdrak> MarcoFalke: agreed
7982016-03-24T19:51:01 <jonasschnelli> Is the reason for the py3 switch the deprecating of py2 or the missing of it in some of the distributions?
7992016-03-24T19:51:09 <wumpus> I think this makes a nice moment to decide on a switch
8002016-03-24T19:51:16 <MarcoFalke> ok, so I will close the "refactor py2 code" pull and just go py3 only
8012016-03-24T19:51:22 <wumpus> we're modernizing the build environment anyway by starting to rely on c++11
8022016-03-24T19:51:31 <wumpus> MarcoFalke: it's the first step right
8032016-03-24T19:51:43 <jonasschnelli> ACK on py3... the RPC tests are not something we need "full portability".
8042016-03-24T19:51:56 <wumpus> for the build system I have py2+py3 compatibility ready
8052016-03-24T19:51:56 <Luke-Jr> actually, we should retain support py2 at least initially
8062016-03-24T19:51:59 *** jcorgan has quit IRC
8072016-03-24T19:52:00 <Luke-Jr> since we may need to backport
8082016-03-24T19:52:08 <wumpus> for the RPC tests I don't think that is necessary
8092016-03-24T19:52:09 <petertodd> I'm not sure there exist any build environments that we need to support that are py2 only
8102016-03-24T19:52:23 <wumpus> agree petertodd
8112016-03-24T19:52:28 <jonasschnelli> agree with petertodd
8122016-03-24T19:52:30 <cfields> agreed
8132016-03-24T19:52:32 <sipa> agree, the backport trees could also just switch to py3
8142016-03-24T19:52:42 <Luke-Jr> if it's not much effort, moving to py2+py3, and then to py3 before 0.13 would be best
8152016-03-24T19:52:52 <wumpus> Luke-Jr: that's a lot more work
8162016-03-24T19:52:58 <Luke-Jr> if it's a lot more work, forget it
8172016-03-24T19:53:05 <wumpus> I did that for scripts used by the build system itself
8182016-03-24T19:53:10 <wumpus> but the test framework is huge
8192016-03-24T19:53:21 <wumpus> and it's a lot of work for people to make sure it works for both python versions
8202016-03-24T19:53:25 <MarcoFalke> considering that the current rpc test are full of bugs due to missing maintanance and review, doing a py2+3 coverage is impossible
8212016-03-24T19:53:25 <wumpus> for every change
8222016-03-24T19:53:37 <sipa> i think we should just outright switch to py3
8232016-03-24T19:53:39 <jonasschnelli> Yes. No backward comp. for test script please.
8242016-03-24T19:53:42 <petertodd> fwiw, I'm somewhat in favor of making my own python-bitcoinlib py3-only, simply because I personally never use it on py2, which makes me worried I'm missing bugs
8252016-03-24T19:53:42 <wumpus> exactly, practical concerns kind of rule this out
8262016-03-24T19:54:16 <btcdrak> I think we should switch to py3 outright also
8272016-03-24T19:54:27 <wumpus> that's clear, then
8282016-03-24T19:54:31 <jonasschnelli> And kudos to MarcoFalke for working on this!
8292016-03-24T19:54:36 <wumpus> yes!
8302016-03-24T19:54:55 <btcdrak> +1
8312016-03-24T19:55:32 <MarcoFalke> You can pipe the files through 2to3 and then "only" check the diff.
8322016-03-24T19:55:32 <wumpus> ok, that concludes the topic I guess, I'm happy we can finally agree on something :)
8332016-03-24T19:55:39 <cfields> yes, it's much easier for us than most because it's all internal and we really have no downstreams to cater to
8342016-03-24T19:55:41 <wumpus> more topics?
8352016-03-24T19:55:58 <wumpus> 5 minutes to go
8362016-03-24T19:56:26 <gmaxwell> Lets end early.
8372016-03-24T19:56:29 <MarcoFalke> #action Close https://github.com/bitcoin/bitcoin/pull/7722 and switch to py3
8382016-03-24T19:56:33 <wumpus> great
8392016-03-24T19:56:39 <wumpus> #endmeeting
8402016-03-24T19:56:39 <lightningbot> Meeting ended Thu Mar 24 19:56:39 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
8412016-03-24T19:56:39 <lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.html
8422016-03-24T19:56:39 <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.txt
8432016-03-24T19:56:39 <lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.log.html
8442016-03-24T19:57:03 <GitHub184> [bitcoin] btcdrak opened pull request #7741: [0.12] Mark p2p alert system as deprecated (0.12...alert0) https://github.com/bitcoin/bitcoin/pull/7741
8452016-03-24T19:57:21 <jonasschnelli> sipa: how would you support test if the ctaeslib is just a c./.h?
8462016-03-24T19:57:29 <jonasschnelli> Add a test.c and a Makefile?
8472016-03-24T19:57:41 <gmaxwell> with a define.
8482016-03-24T19:57:47 <gmaxwell> #ifdef CTAES_TESTS int main()...
8492016-03-24T19:57:58 <sipa> i preferna test.c
8502016-03-24T19:58:12 <sipa> as it's perfectly externally testable
8512016-03-24T19:58:24 <Luke-Jr> in this case, it may be wise to have a function for tests that software can call at startup
8522016-03-24T19:58:35 <gmaxwell> sipa: k.
8532016-03-24T19:58:52 <Luke-Jr> ie, refuse to start if the AES code fails its own testws
8542016-03-24T19:59:17 <gmaxwell> well a test harness for verification may be very different than built in selftests.
8552016-03-24T19:59:24 <Luke-Jr> sure
8562016-03-24T19:59:48 <petertodd> test harness for the constant-time functionality is probably going to be a lot less portable than the code itself for instance...
8572016-03-24T20:00:44 <gmaxwell> well for example, a verification test harness would likely contain another whole implementation of AES. :)
8582016-03-24T20:00:48 <cfields> sipa: just pushed the code-movement to net-refactor8 branch. Still lots of placeholders, todos, and hacks in there, but it now looks overall how i want it to
8592016-03-24T20:02:13 *** mrkent has joined #bitcoin-core-dev
8602016-03-24T20:03:34 <sipa> gmaxwell: hmm... about tests; the s-box implementation is the most obscure/unreviewable/largest part of the code... but it's trivial to.test exhaustivelg against another implementation, as it only deals with 32-bit inputs
8612016-03-24T20:05:09 <sipa> ah, nvm
8622016-03-24T20:05:14 <gmaxwell> sipa: right. an exhaustive sbox test, e.g. with forward sbox pulled out of another implemetation. and checking forward and reverse. And then I thought it might be interesting to see if the existing vectors are enough to catch any simple or or two byte mutations in the current implementation. And if not, we could add vectors until they are.
8632016-03-24T20:05:29 <gmaxwell> why cant the sbox be exaustively tested?
8642016-03-24T20:06:23 <sipa> gmaxwell: not as a black box; its inputs are 8 16-bit integers (and.it does 16 sbox applications in parallel)... you can't exhaustive test 128 bits (i hope)
8652016-03-24T20:07:08 <gmaxwell> right. well it could test each lane one at a time.
8662016-03-24T20:07:14 <sipa> if you can inspect the code and see that it's "obviously" doing the same thing for every bit position, it suffices to test just the 2^8 inputs
8672016-03-24T20:08:07 <sipa> petertodd: how is the bip9 warning code broken?
8682016-03-24T20:08:13 <sipa> (re: github pr)
8692016-03-24T20:08:43 <petertodd> sipa: oh, I spoke to soon on that - but it was more confusing than it should have been, which I'll fix next week when I get back after the long weekend
8702016-03-24T20:08:48 <sipa> petertodd: the warning logic uses bit positions; the trigger code uses DeplymentPos
8712016-03-24T20:08:58 <sipa> (i agree DeploymentId is better)
8722016-03-24T20:09:24 <petertodd> sipa: yeah, simple enough pull-req, and I've got a few more improvements re: adding comments that I'll add to it too
8732016-03-24T20:09:33 <sipa> great!
8742016-03-24T20:10:13 <petertodd> oh, one thing I forgot to ask: if I understand it correctly, right now changes to the "meta-version" - the top three bits - do *not* trigger a "network may have upgraded warning"
8752016-03-24T20:10:36 <sipa> petertodd: they do if it's permanent
8762016-03-24T20:10:53 <petertodd> sipa: ah, alright, I'll go look at that more then and verify
8772016-03-24T20:10:57 *** gevs has joined #bitcoin-core-dev
8782016-03-24T20:10:58 *** gevs has joined #bitcoin-core-dev
8792016-03-24T20:11:05 <sipa> there is the 50% of last blocks habe unexpected version warning still
8802016-03-24T20:11:14 <petertodd> sipa: ah, ok, good
8812016-03-24T20:11:31 <sipa> in addition, there is warning logic for each bip9 bit position, which works retroactively
8822016-03-24T20:11:31 *** hybridsole has quit IRC
8832016-03-24T20:11:57 <petertodd> sipa: anyway, probably best I don't ask too many more questions to keep my mind uncontaminated with what you guys think the code should be doing :)
8842016-03-24T20:12:23 <sipa> petertodd: good strategy
8852016-03-24T20:12:50 <petertodd> bbl - have a good long weekend!
8862016-03-24T20:13:57 <cfields> wumpus: https://github.com/travis-ci/travis-build/pull/680
8872016-03-24T20:14:03 <cfields> one step closer, i hope
8882016-03-24T20:16:33 <wumpus> cfields: wow! finally :)
8892016-03-24T20:16:47 *** hybridsole has joined #bitcoin-core-dev
8902016-03-24T20:17:26 <cfields> wumpus: i have some work on docker that may not have been in vain, though. It looks like a very possible gitian replacement
8912016-03-24T20:18:31 <warren> cfields: for your own deterministic toolchain?
8922016-03-24T20:18:35 <wumpus> cfields: interesting
8932016-03-24T20:19:23 <cfields> (after all, Gitian basically is an early/simplified version of what Docker has come to be)
8942016-03-24T20:20:03 <wumpus> in a way, yes. At least docker is very actively supported/maintained.
8952016-03-24T20:20:33 <warren> docker still puts an OS into a chroot
8962016-03-24T20:20:42 <cfields> warren: kinda. it just builds from a pre-determined sandbox and caches each step. as a bonus, each step can be shared as well
8972016-03-24T20:21:20 <warren> IMHO it's worthwhile to switch away from gitian if we are at the stage of being fully in control of our own deterministic toolchain
8982016-03-24T20:21:32 <cfields> warren: well you'll always need a reference environment
8992016-03-24T20:21:44 *** d_t has joined #bitcoin-core-dev
9002016-03-24T20:22:05 <warren> cfields: the reference environment could be deterministic as well
9012016-03-24T20:22:09 <wumpus> that's a long way, you don't just need a reference toolchain, but there's also other tools that have an effect on the result
9022016-03-24T20:22:24 <cfields> in this case, docker basically acts as that reference environment, containing only our reference toolchain and the bare minimum other tools
9032016-03-24T20:22:41 <cfields> seems to me, docker is the perfect tool for us in this case
9042016-03-24T20:22:48 <wumpus> warren: we need a deterministically buildable OS, isn't debian working on that :)
9052016-03-24T20:23:03 <cfields> warren: sure. and you have to run it somewhere :)
9062016-03-24T20:23:14 <warren> ANY OS > build stage A that probably isn't deterministic -> use that to build stage B
9072016-03-24T20:24:31 <cfields> warren: regardless of the determinism of the base, you have to have some tool for actually building the thing. Short of handing out a rootfs.tar.gz and praying that it will run on some baremetal, ofc
9082016-03-24T20:25:01 <wumpus> anyhow this is driftng off topic
9092016-03-24T20:25:10 <wumpus> main thing I'd like is to have c++11 in travis
9102016-03-24T20:25:18 <cfields> warren: btw, I believe I've seen some early deterministic Debian base images on dockerhub
9112016-03-24T20:25:54 <wumpus> cfields: so deterministic debian doesn't only exist in theory anymore? that's good to hear there's progress
9122016-03-24T20:25:56 <cfields> wumpus: yes, i'm committed to making that happen one way or another. The net code depends on it, so I'm kinda forced to :)
9132016-03-24T20:26:19 <wumpus> cfields: it sounds like a project akin to building the pyramids
9142016-03-24T20:27:01 <cfields> hmm, trying to re-find what i saw
9152016-03-24T20:27:02 <wumpus> even getting the bitcoin dependencies deterministic was a hell of a lot of work, just imagine for a full OS
9162016-03-24T20:27:21 <wumpus> although it gets easier, with better tooling, some things only had to be done onece
9172016-03-24T20:28:55 *** frankenmint has joined #bitcoin-core-dev
9182016-03-24T20:29:06 <cfields> wumpus: hmm, i might've made up the Debian part. I saw someone working on a deterministic base for Travis, my brain might've just lumped 'em together.
9192016-03-24T20:29:16 <cfields> s/Travis/Docker/
9202016-03-24T20:30:10 <wumpus> right - any deterministic base wouldb e fine, it doesn't need to be debian
9212016-03-24T20:32:56 <wumpus> docker is based on lxc isn't it? that means it at least doesn't need deterministic kernel, init etc
9222016-03-24T20:33:33 <cfields> based on cgroups, I don't believe it uses lxc itself
9232016-03-24T20:34:02 *** frankenmint has quit IRC
9242016-03-24T20:39:12 *** t800 has quit IRC
9252016-03-24T20:41:49 <wumpus> right, the same underlying mechanism
9262016-03-24T20:43:11 <kanzure> in addition to deterministic debian please also consider deterministic nixos things
9272016-03-24T20:43:58 <kanzure> fwiw i recommend not using docker's build tool. it's a mess.
9282016-03-24T20:44:28 <kanzure> using docker for running containers or something is probably fine (although coreos/rkt things should be considered, since they seem to be compatible with docker containers and have a specification and words written)
9292016-03-24T20:45:05 <kanzure> "docker build" is a build tool crammed into a container management system. a more realistic build tool is something like bazel ( http://bazel.io/ ), which can spit out finished container images.
9302016-03-24T20:45:25 *** t800 has joined #bitcoin-core-dev
9312016-03-24T20:45:54 <GitHub178> [bitcoin] jonasschnelli opened pull request #7742: [Wallet][RPC] add missing abandon status documentation (master...2016/03/ab_doc) https://github.com/bitcoin/bitcoin/pull/7742
9322016-03-24T20:46:25 *** musalbas has quit IRC
9332016-03-24T20:46:40 <kanzure> (with "docker build" you lose out on things like build caching unless you always build only on one machine. this negatively impacts things like distributed continuous integration build runners.)
9342016-03-24T20:46:41 <cfields> kanzure: docker would only be used as a way to launch our own build process inside of a pre-determined environment
9352016-03-24T20:46:49 *** lysobit has quit IRC
9362016-03-24T20:47:16 *** Thireus has quit IRC
9372016-03-24T20:47:30 *** treehug88 has quit IRC
9382016-03-24T20:48:31 *** t800 has joined #bitcoin-core-dev
9392016-03-24T20:48:38 *** lysobit has joined #bitcoin-core-dev
9402016-03-24T20:48:56 <warren> kanzure: coreos supports docker, not the other way around
9412016-03-24T20:49:10 *** t800 has quit IRC
9422016-03-24T20:49:12 <kanzure> i don't think i said otherwise.
9432016-03-24T20:49:49 *** musalbas has joined #bitcoin-core-dev
9442016-03-24T20:50:39 <kanzure> cfields: right but even the pre-determined environment has a build toolchain. and the default way when using docker is to use "docker build". which is what my above rant is about.
9452016-03-24T20:51:27 <kanzure> oh i guess that is inconsistent with me mentioning rkt. my mention of rkt is unrelated to my complaints about "docker build".
9462016-03-24T20:51:47 <kanzure> (rkt doesn't do container building)
9472016-03-24T20:53:12 <cfields> kanzure: I'll read up on those things
9482016-03-24T20:53:33 *** lysobit has quit IRC
9492016-03-24T20:53:51 <kanzure> ok feel free to pester me. i have been reading docker source code and such.
9502016-03-24T20:54:01 *** musalbas has quit IRC
9512016-03-24T20:54:07 <cfields> great, thanks :)
9522016-03-24T20:54:20 *** musalbas has joined #bitcoin-core-dev
9532016-03-24T20:55:42 *** Thireus has joined #bitcoin-core-dev
9542016-03-24T20:56:26 *** laurentmt has joined #bitcoin-core-dev
9552016-03-24T20:56:31 *** laurentmt has quit IRC
9562016-03-24T20:56:50 *** lysobit has joined #bitcoin-core-dev
9572016-03-24T20:57:28 *** droark has quit IRC
9582016-03-24T21:07:37 *** Guest27862 has quit IRC
9592016-03-24T21:08:46 *** ChillazZ has joined #bitcoin-core-dev
9602016-03-24T21:13:28 *** droark has joined #bitcoin-core-dev
9612016-03-24T21:19:50 <GitHub139> [bitcoin] luke-jr closed pull request #7047: [WIP] Backports for 0.11.3 (updated to 93ca5a3) (0.11...backports-for-0.11.3) https://github.com/bitcoin/bitcoin/pull/7047
9622016-03-24T21:22:16 *** fengling has quit IRC
9632016-03-24T21:27:23 <GitHub149> [bitcoin] luke-jr opened pull request #7743: [0.11] Important backports for 0.11.3 (updated to v0.12.0) (0.11...backports-for-0.11.3) https://github.com/bitcoin/bitcoin/pull/7743
9642016-03-24T21:29:44 *** frankenmint has joined #bitcoin-core-dev
9652016-03-24T21:31:32 *** Hybrids0le has joined #bitcoin-core-dev
9662016-03-24T21:31:32 *** hybridsole has quit IRC
9672016-03-24T21:32:45 *** CodeShark has quit IRC
9682016-03-24T21:33:02 *** NicolasDorier has quit IRC
9692016-03-24T21:33:03 *** CodeShark has joined #bitcoin-core-dev
9702016-03-24T21:33:56 *** NicolasDorier has joined #bitcoin-core-dev
9712016-03-24T21:33:57 *** Hybrids0le has quit IRC
9722016-03-24T21:34:01 *** droark has quit IRC
9732016-03-24T21:34:25 *** lysobit has quit IRC
9742016-03-24T21:34:26 *** musalbas has quit IRC
9752016-03-24T21:34:49 *** lesderid has quit IRC
9762016-03-24T21:36:17 *** frankenmint has quit IRC
9772016-03-24T21:41:00 *** musalbas has joined #bitcoin-core-dev
9782016-03-24T21:41:24 *** lysobit has joined #bitcoin-core-dev
9792016-03-24T21:41:34 *** lesderid has joined #bitcoin-core-dev
9802016-03-24T21:44:43 *** d_t has quit IRC
9812016-03-24T21:44:44 *** ebfull has quit IRC
9822016-03-24T21:44:44 *** windsok has quit IRC
9832016-03-24T21:50:10 *** fengling has joined #bitcoin-core-dev
9842016-03-24T22:00:40 *** d_t has joined #bitcoin-core-dev
9852016-03-24T22:00:40 *** ebfull has joined #bitcoin-core-dev
9862016-03-24T22:00:40 *** windsok has joined #bitcoin-core-dev
9872016-03-24T22:03:42 *** ibrightly has quit IRC
9882016-03-24T22:05:05 *** ibrightly has joined #bitcoin-core-dev
9892016-03-24T22:07:46 *** justanotheruser has quit IRC
9902016-03-24T22:18:01 *** supasonic has quit IRC
9912016-03-24T22:19:01 *** supasonic has joined #bitcoin-core-dev
9922016-03-24T22:32:46 *** frankenmint has joined #bitcoin-core-dev
9932016-03-24T22:33:16 *** fengling has quit IRC
9942016-03-24T22:33:32 *** Guyver2 has quit IRC
9952016-03-24T22:38:19 *** frankenmint has quit IRC
9962016-03-24T22:43:01 *** Hybrids0le has joined #bitcoin-core-dev
9972016-03-24T22:44:57 *** Hybrids0le has quit IRC
9982016-03-24T22:55:39 *** shesek has joined #bitcoin-core-dev
9992016-03-24T22:59:44 *** wallet42 has joined #bitcoin-core-dev
10002016-03-24T23:02:29 *** mrkent_ has joined #bitcoin-core-dev
10012016-03-24T23:04:35 *** mrkent has quit IRC
10022016-03-24T23:06:30 *** hybridsole has joined #bitcoin-core-dev
10032016-03-24T23:10:06 *** fengling has joined #bitcoin-core-dev
10042016-03-24T23:20:29 *** justanotheruser has joined #bitcoin-core-dev
10052016-03-24T23:23:26 *** musalbas has quit IRC
10062016-03-24T23:23:35 *** musalbas- has joined #bitcoin-core-dev
10072016-03-24T23:23:46 *** musalbas- is now known as musalbas
10082016-03-24T23:27:49 *** PRab has quit IRC
10092016-03-24T23:50:10 *** justanotheruser has quit IRC
10102016-03-24T23:52:19 *** droark has joined #bitcoin-core-dev