20:02:38 #startmeeting 20:02:38 Meeting started Mon Jun 10 20:02:38 2019 UTC. The chair is cdecker. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:02:38 Useful Commands: #action #agreed #help #info #idea #link #topic. 20:03:06 hi 20:03:28 Ping rusty BlueMatt roasbeef bitconner sstone araspitzu niftynei kanzure Chris_Stewart_5 20:03:52 Will be in and out, only have my phone. Suspect roasbeef is still in transit 20:03:59 hi everyone! 20:04:06 hi guys! 20:04:27 #link https://github.com/lightningnetwork/lightning-rfc/projects/1 20:04:34 Hi everyone :-) 20:04:55 conner: still travelling from coredev? 20:05:38 #topic Resuming video-chat spec meetings in parallel to the IRC meetings 20:05:40 cdecker, something like that, don’t leave till mañana 20:05:53 Before we begin I wanted to bring this up 20:07:09 During CoreDev, we were talking about how abysmal the bandwidth for actual discussions is on IRC, so we thought it might be nice to resume the video calls, that could be life-streamed in order not to exclude people, and still use the IRC channel as a sort of log and for people not on the call to pitch in 20:07:23 NACK 20:07:35 Just wanted to put that out there and see what the general feeling was 20:08:22 I think it would be more productive than IRC 20:08:23 I prefer video for chit-chat and throwing ideas around, but I like IRC for the actual logging decisions. 20:09:19 Ok, reasonable. However our actual decisionmaking became really slow once we moved to IRC, since we get hung up in misunderstandings and discussions 20:09:28 I think our hangout meetings were significantly more efficient 20:09:41 then maybe we should use video outside of the IRC meetings more often to make progress before the IRC logging decisions are taken 20:09:44 If we had people actually contribute, and discuss topics on the list / tracker, yes, we could just nod off stuff on IRC 20:10:36 Anyway, don't want to bog down the meeting with this discussion at the start, just thought I'd throw it out there, and we can discuss outside the meeting 20:10:42 Let's get to ACK stuff ^^ 20:10:45 Yeah, maybe a pre-meeting, or post? 20:10:57 Sgtm. I wouldn’t mind going back to video if we also log to irc during 20:11:06 Tho the transparency of irc is nice for others 20:11:22 #topic bolt04: Variable `hop_payload` for the sphinx onion #619 20:11:24 #link https://github.com/lightningnetwork/lightning-rfc/pull/619 20:12:20 This is the 3rd attempt at getting larger payloads into the spec, with #593 we had fixed size frames, with #603 we had explicit enumeration of frame counts, and with #619 we now have everything variable 20:12:41 Did we figure out what the max path length with this proposal would be? 20:12:52 After implementing it I quite like #619, it breaks with the 20 hop limit, but that's ok I guess 20:13:06 after implementation I find it very flexible too 20:13:08 Did some mental maths and came up with 23-24 20:13:18 yeah I think it was around 25 20:13:23 Which I think is fine 20:13:39 Think it depends on whether the final node can drop the hmac 20:13:40 conner: so with legacy payload we had 12 bytes of padding, that means we now can salvage those (240 bytes in total), divide that by 53 and we get 4 additional hops 20:14:04 Assuming (wrongly) we'd still use the legacy payload 20:14:24 this proposal completely gets rid of any kind of padding so it feels like we're close to optimal in terms of what data we're able to put in the onion 20:14:36 The new format allows us to better compress some numbers (cltv and msatoshi) but add the TLV overhead 20:14:58 Notice that I actually have TLV types defined in the PR (stolen from #603) 20:15:37 Damn, and I just noticed I have typos in the PR (thanks t-bast for the review) 20:15:46 I dislike including hmac in the length, which would seem the easiest way to drop the final zero HMAC. 20:16:11 rusty: this proposal doesn't include the HMAC in the length 20:16:11 I like the general direction of this, but idk how many people have been able to look at it since Friday 20:16:25 it's varint(len), len x bytes, 32 bytes hmac 20:16:25 cdecker: yes, but t-bast I think suggested it. 20:16:36 Oh, ok 20:16:37 yes but I said it wasn't a good idea afterwards :) 20:16:44 t-bast: OK :) 20:16:47 in the same comment I believe, in an edit 20:16:51 I think it's not worth adding the HMAC to the payload itself 20:16:57 yeah me neither 20:17:17 btw, with this construction we can use the 32 bytes at the end as well 20:17:45 At one point we discussed being able to drop the hmac for the final hop. Is that included in this or are there any attractive ways to do so? 20:17:46 t-bast: OK. Re: your TLV formatting suggestion, they are not a varint, so that's confusing. They're explicitly 'integer' from a previous spec which defined that to be "an integer field of 0 to 8 bytes in length". 20:17:53 If we do so, the last "HMAC" which is never used hangs over the end of the packet and will be shifted in as part of the fillers, which is totally ok 20:18:18 conner: see my comment above 20:18:27 cdecker: what's our termination condition then? 20:18:28 rusty: hum ok I think I got it 20:18:39 We add an explicit flag in the onion :-) 20:18:52 cdecker, gotcha makes sense 20:19:09 t-bast: BTW niftynei had a script PR which parsed TLVs, which will need to coordinate with exact format. 20:19:12 We could signal the final hop via a short_chan_id of 0 20:19:16 It's a TLV value with 1-byte type and 1-byte length of 0, meaning we now signal in 2 bytes that the node is the final node 20:19:42 conner: I prefer explicit signal in this case, it's 8 bytes shorter 20:19:44 rusty: got it, I understand what you meant. I'll have a look at niftynei's PR 20:20:35 Ok, seems I need to go back and fix some typos, but the general idea is ok? 20:20:43 cdecker: I like that 20:20:54 (We could still accept and merge after typo fixes) 20:20:59 And that would also mean “I haz no hmac”? 20:21:05 cdecker / conner: Agreed skipping that final hmac is attractive, and a terminator TLV seems like the most attractive. 20:21:05 Right 20:21:15 :tada: 20:21:30 ACK on the general idea 20:21:37 Nice :-) 20:21:51 t-bast: were you able to reproduce the onion test vector on your end? 20:22:02 yes I tested it on friday and it worked like a charm 20:22:06 cdecker: Ack. Unf we need to add terminator, otehrwise I would say include it now and fix up typos later. Oh well :( 20:22:08 Awesome 20:22:51 Ok, so vote on accepting #619, me fixing typos and adding the terminator, and then merging? @all 20:22:58 Would be good to run these ideas by roasbeef as well before pulling the trigger 20:23:32 Right, will wait for an ACK on the PR (would like to get these merged, and not postponing 2 more weeks) 20:23:39 agreed :) 20:23:42 Ack. 20:24:06 (Also I noticed that the code cleans up nicely with this variable payload stuff ^^) 20:24:10 action item on conner to tie roasbeef to his chair and make him look at the PR this week? 20:24:38 Y’all seem trigger happy today lol 20:24:44 haha 20:24:46 #action cdecker to fix typos in PR #619 and add the route termination TLV value to spec 20:24:49 But yes we’ll look at it 20:25:00 #action conner to ask roasbeef for an ACK on the PR 20:25:20 #agreed cdecker good to merge #619 once fixes are in and roasbeef ACKd 20:25:24 Sounds good? 20:25:42 sgtm 20:25:51 Yep will ping him 20:26:10 ok, I also closed #593 to reduce clutter 20:26:13 Shall we move to #607, which I must say is the MOST FLAWLESS PR EVER. 20:26:32 Sure 20:26:37 #topic BOLT01: TLV proposal #607 20:26:38 TLV? never heard about those 20:26:44 #link https://github.com/lightningnetwork/lightning-rfc/pull/607 20:27:21 I want to thank conner for sitting down with me and thrashing this out. We ended up with something we're happy is flexible and yet tightly specified. 20:27:52 Shall I ask the dreaded question? 20:27:58 Yeah pretty happy with how this turned out! 20:28:06 What about the "it's ok to be odd" rule? 20:28:13 heh 20:28:21 cdecker: yeah, we kept it. It's redundant, but it's nice to have a backup. 20:28:25 t-bast had a good q on if we should move it to it’s own bolt 20:28:45 +1 on giving it its own bolt 20:28:50 Rationale that there we could also define common encodings that are used, and which other bolts can reference 20:29:07 +1 for its own bolt to group "known" specified TLV types as well 20:29:11 If a bolt needs a more nuanced encoding, it can be done inline 20:29:50 That's kind of waht bolt1 is... 20:30:19 t-bast: I'm not sure, though, since each type only makes sense in context. Grouping them together loses that. 20:30:47 I think the idea is that we shouldn’t have to define a uint32 encoding everywhere 20:30:54 Not sure we have common types except for the "0-8 byte integer" type. 20:30:59 Or perhaps a compressed uint32 encoding 20:31:18 rusty: there are some common types that will end up being redefined all the time, like a node_id, an amount_msat, etc, don't you think? 20:31:18 The separation of bolts is rather chaotic tbh, the message format is in bolt 02, while 01 is the initial handshake and other control messages 20:31:56 t-bast: that's more of a type, rather than an instance in a field though isn't it? 20:31:59 cdecker: ? Message format is in 01... 20:32:20 Oh, right, got confused with the framing in bolt 02 20:32:50 I did have an old PR which replaced the '8:' etc in the spec with type names, which were defined in 01. Might drag that out, but it's a format thing, rather than something code. 20:32:51 Still, 01 now has top-level message format, some messages and the sub-message level TLV field format 20:33:22 Anyhow, we can reorg later 20:33:34 Let's talk about the TLV proposal as is 20:33:51 Agreed, reeorg is a formatting change we can have someone propose and implement later. 20:34:11 Personally, I prefer having standalone bolts that clearly delineate what is encapsulated, rather than needing to search through multiple. Yes, TLV format, any qualms? 20:34:16 sounds good, I can draft a more detailed explanation (with examples) to explain why I think it might make sense to group some types together and we'll see if people agree with that 20:34:28 TLV format sgtm 20:34:33 Nice touch with the monotonically increasing type, but how about having multiple fields of the same type? 20:34:51 we said it could be encompassed by the type itself 20:35:07 My only proviso is that it goes a little meta with the varbytes comment. We don't have a good place in the spec for "advice to spec writers" though. Again, maybe a new bolt... 20:35:16 Ok, so we allow nested arrays for example 20:35:23 cdecker: exactly 20:35:36 ok 20:36:05 cdecker: disallowed. We discussed it, but all existing examples were kind of destroyed by the order being required by the formatting (eg. bolt11 has dup 'f' fields, but order matters). 20:36:08 rusty, yeah I was looking at that too. Indeed the suggestion should be more general: don’t encode a single variable length field within a record 20:36:56 lndbot: you mean no *redundant* length prefixes inside a TLV, right? 20:37:08 rusty: yes 20:38:07 Yeah, we def. need a BOLT-BOLT.md Advice To Young Authors for this kind of stuff. 20:38:18 But, again, formatting... Content is good. 20:38:43 Ok, seems rusty and conner like the proposal, how about t-bast and sstone? 20:39:19 I like it 20:39:25 I approved it on github a few days ago already 20:39:29 I like it too. Was just thinking that when you define an "array" subtype you'll have to specify ordering rules too 20:39:40 Ok, so everybody agrees on accepting PR #607? 20:40:06 sstone: true, our work is never done :) 20:40:16 sstone, is that not true either way? 20:40:20 What does "done" mean, rusty? 20:40:32 ;-) 20:40:39 conner: yes 20:40:50 cdecker: very meta. In this case I meant we will have to consider that when we define our first such field. 20:41:34 Okay, so soft ack on this? 20:41:47 Seems like there isn’t anything substantial to discuss? 20:41:48 SGTM 20:41:53 sgtm 20:41:57 Why soft? I think it's good to go. 20:42:01 sgtm 20:42:03 I think this is actually more than soft 20:42:12 #agreed Merge #607 20:42:18 Congrats conner ^^ 20:42:25 congrats! 20:42:28 * rusty applauds 20:42:30 Woo!!! 20:42:48 I can push some slight touch ups before or after, either way is fine with me :) 20:43:22 Any preference for next issue to discuss? 20:43:24 Let's merge as is for the record, then add typo fixes? 20:43:35 cdecker: range queries? 20:43:37 Thank you to everyone for the feedback and discussion :D 20:43:45 t-bast: number? 20:43:51 #557 20:44:00 https://github.com/lightningnetwork/lightning-rfc/pull/557 20:44:12 #topic BOLT7: extend channel range queries with optional fields #557 20:44:19 #link https://github.com/lightningnetwork/lightning-rfc/pull/557 20:44:20 yes please since we agree on TLV it is now implementable too :) 20:45:39 This is even older than my multi-frame PR :-) 20:45:43 Sorry, I've been slack. I *will* implement the new TLV version before next meeting, sorry. 20:45:49 I think the only nitpick left was the choice if CRC for the checksum (Adler32 vs CRC32) ? 20:46:08 proposal looks pretty good to me, still some typos but not critical 20:46:11 (Somewhere just out of reach the c-lightning 0.7.1-rc1 is screaming at me for saying that) 20:47:01 sstone: crc has the advantage that it *will* catch single bit changes, such as a single flag. 20:47:12 So most people seem to be fine with the proposal 20:47:26 Do we have 2 independent implementations though? 20:47:40 sstone: but OTOH I'm pretty sure that we can trivially ensure lack of collisions w/ Adler by tweaking timestamps. 20:48:07 Let's not be "clever" in even more places 20:48:09 Idt there are two impls 20:48:22 rusty: I don;t mind switching to CRC32 20:48:25 Ok, so I think we can't really merge, but we can collect ACKs 20:49:07 sstone: I already implemented Adler so I don't really mind. We use crc32 in our codebase as well, so . 20:50:00 CRC32 sgtm 20:50:21 CRC32 it is then 20:50:23 * cdecker abstains from having an opinion in this case 20:50:45 gcc has a crc32 primitive, *sometimes* it seems. OK. 20:50:57 So, with CRC32, everybody is fine with the proposal? 20:51:02 sgtm 20:51:05 Yes, ack! 20:51:11 Conner? 20:51:28 Can probably get an impl of timestamp querying in the near future 20:51:48 But yeah, o/w happy with the proposal 20:52:04 BTW, c-lightning 0.7.1 will no longer ask all nodes for the gossip firehose, assuming my PR gets through review :) 20:52:24 #agreed Merge #557 once we have a second implementation 20:52:28 Nice ^^ 20:52:33 Making progress 20:52:37 awesome 20:52:42 rusty: that would be awesome 20:53:04 Might even make sense to disallow historical gossip filters... 20:53:48 Ok, moving on 20:53:54 A small proposal we wanted to discuss is #596. We've got many people asking for the ability of opening bigger channels so it might be the right time to discuss this? 20:54:05 Was seriously tempted to make gossip-extensions a required feature, but there's really v. little work to support the old ones. 20:54:18 #topic Single-option large channel proposal #596 20:54:24 #link https://github.com/lightningnetwork/lightning-rfc/pull/596 20:54:50 Great, thanks t-bast, was about to ask for the next most urgent proposal 20:54:57 ;) 20:55:59 maybe we need to add some signaling on the biggest channel size we accept (not in the PR)? I don't know if there's a field already indicating that somewhere. 20:56:01 Looks like the only issue is that some higher bits were chosen, rather than starting from 0 20:56:57 I think the bits choice was made to play nicely with the feature bit unification (which we could discuss as well) 20:57:00 Dunno, if we need yet more signalling (explicit size limits) 20:57:19 The 2^24 limit was chosen mainly to avoid having people put their life-savings in there 20:57:37 I don't think an upper limit should be forced by the fundee 20:57:53 If it has a lower preference it can always start rejecting HTLCs 20:58:05 yeah that makes sense 20:58:07 cdecker: agreed 20:58:18 This takes us back to "what's happening with features?" I think. Weird he put it in as "wumborama" which is a channel option, instead of i_wumbo_you_wumbo which is the node option. 20:58:21 I’m fine with the feature bits not intersecting so we can keep the option to unify feature bits later 20:59:04 Oh, you're right, it should be a node feature, not a channel feature 20:59:10 cdecker: people want a node option to say "I support large channels!". We may not need a channel feature, sure. 20:59:21 rusty: doesn't that proposal replace previous wumbo bits (I might be lacking some context on the previous wumbo stuff)? 21:00:01 in the PR it looks like only a global (node) flag is defined so it should be ok, isn't it? 21:00:10 t-bast: but there were two sets of wumbo bits: 14/15 for "this node can wumbo with you" and 16/17 for "this channel has wumbo" 21:00:23 https://github.com/lightningnetwork/lightning-rfc/issues/605 21:01:03 Also shouldn't we communicate this in the `init` message, not the `channel_open`, or am I misreading this? 21:01:04 Those numbers were only a suggestion anyway, but this seems to be 14/15? 21:01:28 +1 for local and global signaling 21:01:36 yeah I might be misreading but it seems like nothing is needed in the `channel_open` is it? 21:02:12 to use rusty's issue naming only bits 14/15 should be needed right? 21:02:42 t-bast: as long as we decide to reflect local (aka "node" features) into node_announcement, yes. 21:02:48 Yep, I think we need a feature bit defined, and have that in the `init` and the `node_announcement`, to signal to peers on connection and to signal potential peers in gossip 21:03:10 Plus private nodes wouldn’t be able to make big channels if we only signaled global 21:03:14 rusty: I thought that would be the case, is there a reason *not* to do it? 21:04:42 t-bast: we don't today. But it became clear that people want to announce what features their node supports without requiring a connect, so hence https://github.com/lightningnetwork/lightning-rfc/pull/571 says that we should reflect node features in node_announcement. 21:05:11 (Previously the thinking was that if you're not a direct peer, you don't need to know this stuff) 21:05:39 ok, thanks for the explanation 21:06:14 https://github.com/lightningnetwork/lightning-rfc/pull/571/commits/14ced3b6c33961f1adc3699994c21a4abc4958f8 21:06:39 I think we should talk through 571, TBH. I know we're over-time already though :( 21:06:53 They're quite nicely separate commits: https://github.com/lightningnetwork/lightning-rfc/pull/571/commits 21:07:04 Ok, seems we are indeed out of time 21:07:08 I think that we agree on the need for large channels, but the discussion revolves more around 571 atm 21:07:17 Shall we defer #571 and move discussion to ML and tracker? 21:07:39 maybe update feature bit unification and large channels proposal before next meeting and discuss them on github? 21:07:41 cdecker: reluctantly agree. I'll stay on for a bit to give an overview and answer qs though 21:07:54 t-bast: ack 21:07:55 Ok, any objections to deferring? 21:08:01 nope, sgtm 21:08:19 no 21:08:21 we already made some nice progress today 21:08:26 Sorry about that, would have liked to get it through 21:08:57 #agreed Defer PR #571 to the ML and tracker, to be decided during the next meeting 21:09:00 defer sgtm 21:09:12 #topic Chit-chat and announcements 21:09:36 Ok, let's finish up with announcements, anybody brought something for show-and-tell? 21:09:39 :-) 21:10:34 I brought a trampoline proposal that will be ripe when variable-length onion is merged :D 21:10:38 c-lightning will soon release 0.7.1, containing performance improvements, bug fixes and extending some of the plugin API 21:10:48 Yep, trampoline is very interesting as well 21:10:49 0.7.1-rc1 this week. For sure... Main improvement people care about (other than bugfixes) is that if our gossip is current, we only ask 2 peers for the last 24 hours, 8 peers for current gossip, and any other peers for no gossip at all. 21:11:13 And Stepan has sent an e-mail for spontaneous payments based on the onion shared secret 21:11:27 If we find our gossip isn't current, we ask two random peers for complete gossip though, so not perfect. 21:11:42 Talking to conner last week, we think it might work nicely, but needs tagged hashes so we don't lose flexibility for other uses 21:12:00 nice gossip improvements on c-lightning! we also briefly discussed exploring minisketch during coredev, hopefully in a few weeks/months we'll have more to share 21:12:15 I have something, if it's ok to interject from a non-contributing dev. 21:12:31 Sure Rob 21:12:41 A researcher working at University of Edinburgh’s Blockchain Technology Lab (funded by IOHK, which contributes to Cardano) is about to publish a paper entitled “A Composable Security Treatment of the Lightning Network”. It should be completed and submitted by the end of the week. 21:13:02 Nice, looking forward to it 21:13:05 that's great! 21:13:30 * rusty misread that as "Compostable"... I hope not! 21:13:45 Seems we all have more than enough on our plates for the next days, let's get to it :-) 21:13:46 you should definitely send something to the mailing list once the paper is out, a lot of people will be interested 21:13:53 #endmeeting