20:11:44 <roasbeef> #startmeeting
20:11:44 <lightningbot> Meeting started Mon Aug 19 20:11:44 2019 UTC.  The chair is roasbeef. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:11:44 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
20:11:58 <roasbeef> will go down le list in order
20:12:25 <t-bast> Is there a reason why 608 wasn't merged? We had an action item to merge it last time.
20:12:29 <roasbeef> #topic #658 and #643 (mpp and amp drafts)
20:12:58 <roasbeef> say no more t-bast
20:13:12 <t-bast> neat, the merging machine is on
20:13:16 <roasbeef> ok in these two, conner recently put up the draft of the secret sharing spontaenous variant
20:13:21 <cdecker> #643 needs a rebase now that #619 was merged
20:13:26 <roasbeef> thinking that there's a goodo bit over overlap between this an the invoice based on
20:13:29 <cdecker> Shall I quickly do that?
20:13:48 <roasbeef> for example, they may share some TLV keys (the strem_id) field for example
20:14:02 <roasbeef> cdecker: rebase it?
20:14:13 <cdecker> roasbeef: yes :-)
20:14:33 <t-bast> I think the main difference is that #658 adds the spontaneous part: both are atomic (in a way, one has atomicity controlled by the sender, the other by the recipient)
20:14:57 <t-bast> I would see #658 as an addition to #643
20:15:04 <roasbeef> heh "atomic", but let's not get caught up on wording
20:15:27 <t-bast> (by the way there are many open comments on both PRs, interesting discussions there)
20:15:28 <cdecker> Sender controlled atomicity is a red herring if you ask me
20:15:55 <t-bast> depends on the context, in #658 it kinda makes sense
20:16:01 <roasbeef> we may even just merge them into a single PR possibly, since there's so much overlap
20:16:08 <roasbeef> both needs a feature bit global, invoice tags, etc
20:16:38 <t-bast> roasbeef: that would be a good idea to make it easy to digest as a whole, but implementation can really be done in several independent steps imo
20:17:04 <t-bast> rusty's MPP doesn't need a global feature bit as it's invoice-driven (the feature bit is inside the invoice)
20:17:58 <t-bast> unless we want to allow receiving invoice-less payments through rusty's proposal, but for invoice-less payments I think #658 makes more sense
20:18:01 <cdecker> I don't think that putting both proposals in a single PR is a good idea (I for one am not sold on the idea of #658)
20:18:37 <t-bast> Can you explain what you don't like in #658?
20:18:42 <rusty> I think #658 is eventually obsoleted by Schnorr and bolt 11 offers, but that's a long enough lifetime to be useful IMHO.
20:19:44 <roasbeef> obsolted by schnorr, maybe in 2 years
20:19:48 <bitconner> both are obsoleted by schnorr
20:19:52 <roasbeef> even then you can apply the same derivation to make it work with schnorr
20:20:23 <roasbeef> tons of devs are begging for the spontaneous payment feature,so would be curious what your qualms are decker
20:20:35 <roasbeef> there's even a cut out for re-occuring payments now
20:20:52 <roasbeef> and many exchange favor that model over invoices, since if re-used they can be dangerous
20:20:55 <rusty> roasbeef: well, bolt11 offers are a superior way of doing this and getting a receipt.
20:21:01 <cdecker> How is base-amp obsoleted by schnorr?
20:21:29 <cdecker> The idea is still sound: have the recipient trigger the settlement of all parts, and communicate the total amount in the onion
20:21:29 <roasbeef> ofc there are trade offs rusty, in some cases the "receipt" is less critical, and a slighlty weaker proof of payment is possilbe
20:21:35 <t-bast> rusty: could you share what you mean by Bolt11 offers? I haven't heard of that before
20:21:36 <bitconner> you can have parial payment decorrelation and pop
20:21:40 <roasbeef> in the end it falls back to disputes with the sender/receiver which are all extra protocol
20:22:03 <roasbeef> i think we means the feature bits in bolt 11? (another PR)
20:22:12 <BlueMatt> right, but non-base amp makes sense for spontaneous payments, base amp keeps the same properties people are used to today
20:22:29 <BlueMatt> I dunno why we have to decide between one or the other, combine them as much as possible and realize they're both useful
20:22:36 <bitconner> ^^
20:23:13 <t-bast> I agree with BlueMatt there, I think base-amp is simple enough to implement and spec, and can be enriched with the spontaneous part afterwards with bitconner's proposal
20:23:18 <rusty> t-bast: https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-November/001602.html
20:23:24 <t-bast> thx
20:23:25 <BlueMatt> ux messaging is hard, but imo makes sense to use base amp where possible if you have an invoice and original amp if you want spontaneous payment
20:23:41 <BlueMatt> obviously with schnorr once we get there
20:23:48 <cdecker> BlueMatt: we can have spontaneous payments for free if we use the shared secret from the onion, no need for anything more complex than the hash functions we already use
20:24:20 <cdecker> And the shared secret preimage trick works great in conjunction with base-amp
20:24:22 <BlueMatt> right, fair, but might as well combine in a useful way with amp, is my point
20:24:34 <BlueMatt> ie if you want amp over spontaneous payments, might as well do it this way
20:25:59 <bitconner> cdecker: tho cool, i'm not sure it's worth adding yet another payment mechanism when one is a generalization of the other
20:26:08 <t-bast> rusty, cdecker, is your concern mostly because you'd like spontaneous payments to have a proof-of-payment, which Bolt11 offers can provide and #658 can't provide?
20:27:05 <cdecker> t-bast: spontaneous payments do not have a proof by definition (until we get schnorr)
20:27:30 <cdecker> My objection is that #658 doesn't add much if we have two simpler mechanisms already
20:27:56 <t-bast> cdecker: but the proposal linked by rusty does provide a pop - it's not fully spontaneous though, but probably spontaneous enough?
20:28:21 <rusty> t-bast: can't speak for cdecker, but I feel strongly that in a cash-like system without an intermediary, receipts have proven themselves throughout history to be a vital protection.  It's more work to spec up, but it's where we want to end up IMHO.
20:28:34 <bitconner> cdecker: how does schnorr enable pop for spontaneous payments?
20:28:39 <t-bast> rusty: got it, thanks for clarifying
20:28:49 <BlueMatt> cdecker: ah, wait, am I misunderstanding (sorry, I havent been as active recently, so feel free to tell me to fuck off), but is there a way to do spontaneous payments over BAMP?
20:28:50 <rusty> I'm not sure if the layering violation of using the onion to hold the spontaneous secrets is worthwhile.
20:29:06 <t-bast> Maybe it would be worth drafting a formal proposal using the onion shared secret preimage trick to compare to #658?
20:29:33 <BlueMatt> cause spontaneous payments over BAMP seems like a nice thing to me
20:29:34 <roasbeef> BlueMatt: +1 on combine they serve diff needs
20:30:13 <roasbeef> cdecker: xor is pretty basic, even less complex that hash functions ;), the shared secret approach is nice, but possible consumders i've talked to for this feature find it lacking and want more structured information at the other end
20:30:27 <cdecker> BlueMatt: yes, there is a way to do spont payments over base-amp (sender encodes the preimage in the shared secret for the onion, and then we deterministically generate from there)
20:30:35 <BlueMatt> ah, right
20:30:39 <BlueMatt> welllllll
20:31:03 <roasbeef> better to have a unified way to do spont payments that 3 methods, that case is just amp with a payment shard size of 1
20:31:36 <cdecker> Unless you try to generate preimages in a certain format (common prefix or something), which I'd strongly discourage, there is absolutely no downside to using the shared secret there
20:31:42 <roasbeef> but feels liek we're getting off track here, what do people think of combining them (PR wise) since ther's so much over lap
20:31:51 <cdecker> NACK
20:31:56 <roasbeef> you're limited in what secret you can pick cdecker, since it depend son their pubkey
20:32:21 <roasbeef> "no down side", it leaks layers, and doesn't provide any additional structure
20:32:35 <cdecker> But you should be picking it randomly anyway, and it is the result of an ECDH that you executed with a random secret you generated yourself
20:32:50 <roasbeef> sure they shoudl be random, but also cuould be derived off soem other sub-protocol
20:33:05 <cdecker> There is no additional structure, but I'll give you the layering violation :-)
20:33:08 <roasbeef> that case is a sub-set of the greater amp scheme, since it's a single payment vs many others
20:33:11 <rusty> TBH I share roasbeef's dislike of the layering violation.
20:33:35 <t-bast> +1 on the layering violation, it feels a bit hackish
20:33:38 <roasbeef> it's a cool trick, but we have a way to encompass that use case, and provide additional utility with the greater amp proposal
20:33:45 * cdecker reluctantly agrees that the layering violation is bad
20:35:49 <t-bast> I'm afraid that sharing the PR would slow us down. It seems like #658 can easily build on top of #643, wdyt?
20:36:20 <roasbeef> are there major blockers with either of them?
20:36:21 <cdecker> Agreed
20:36:40 <rusty> I rather like the XOR proposal.  It's nice because it genuinely decorrelates the separate paths.  It also uses our shiny new onion format FTW.  I agree with bitconner that there's a large amount of overlap, so #643 should be generalized (make sure terminology is OK) so it's easy to add #658.
20:37:12 <t-bast> No major blocker from my point of view, a lot of interesting discussion have already started on the PRs themselves, we should keep them active on Github.
20:38:23 <bitconner> agree on keeping both as separate proposals, i tried to base 658 on 643 as far as requirements so that feedback can easily be shared between the two
20:38:35 <bitconner> :)
20:39:01 <roasbeef> ok, so move on there and keep the dicsussion in github
20:39:03 <roasbeef> ?
20:39:10 <bitconner> sgtm
20:39:12 <t-bast> So next steps: rebase #643 on master, make progress on comments and rebase #658 on #643?
20:39:52 <roasbeef> #action rebase #643 on master, continue discussion of #658 and #643 on github, to bubble up in the meeting as needed
20:40:02 <t-bast> ACK
20:40:27 <rusty> (Assume there's an implied "rusty" in that rebase... will do :)
20:40:31 <roasbeef> #topic 657
20:40:33 <t-bast> heh
20:40:49 <roasbeef> (channel_update after exchange funding_locked)
20:41:24 <t-bast> there are a few distinct PRs that bubble up issues with channel establishment messages (657, 620 and 625)
20:41:29 <t-bast> it would be nice to clear these up
20:42:54 <roasbeef> so seems to be a question of synchronization here
20:42:56 <bitconner> for 620 i think the general consensus is actually to exclude announcement_signatures rather than include them as BOLT 7 msgs
20:43:04 <rusty> Yeah, I think there are a number of things gated by funding_locked *exchange* ie. both sides say they've seen it.
20:43:06 <bitconner> can update the pr if people agree
20:43:12 <roasbeef> related question is that should impls just be able to handle things slightly out of order for robustness purposes
20:43:40 <BlueMatt> I think if the spec says order must be X, you dont need to handle it
20:43:45 <BlueMatt> if spec doesnt say that, then you do?
20:43:48 <BlueMatt> thats the point of a spec :p
20:44:04 <rusty> Ack #657 BTW
20:44:04 <BlueMatt> and, generally, gating things on funding_locked is nice cause thats an explicit indication, as rusty says, that you've seen it
20:44:07 <BlueMatt> and that you're ready to go
20:44:40 <roasbeef> sure impls can be very strict, but then they may poorly handle edge cases where resumption is possible
20:44:50 <rusty> #620 is exactly wrong.  announcement_signatures is *not* a bolt 7 message, and having it there was a mistake.
20:45:08 <roasbeef> the thing here again is timing, if they other preson doesn't know of the channel yet (say slow processing) they can't process the update
20:45:23 <roasbeef> but you can say, shuoldn't they be expecting that channel given that the funding flow finished?
20:45:34 <rusty> roasbeef: yeah, so you wait until they say they're ready, and problems vanish.
20:45:40 <bitconner> rusty: that doesn't rectify the fact that it *is* there :)
20:45:49 <BlueMatt> concept ack 657, tho I'd like some wording clarifications
20:46:12 <cdecker> ACK #657 :-)
20:46:16 <rusty> bitconner: indeed! But saying that it's "independent of particular channels" is just ridiculousness.
20:46:40 <rusty> I will propose removing it from bolt 7.  I think the desire was to reduce the amount of stuff in bolt 2.
20:46:44 <bitconner> maybe moving it out of bolt 7 is the real fix haha
20:46:59 <BlueMatt> ack moving it out of bolt 7 :)
20:47:06 <bitconner> okie doke
20:47:11 <t-bast> ack
20:47:30 <bitconner> #action connder to fine new home for announcement_signatures
20:48:30 <roasbeef> rusty: why wouldn't they be ready? the've had 3 confs
20:48:43 <roasbeef> this feels like more of an implementation issue
20:48:46 <rusty> It's not a "gossip" message, by any stretch.
20:49:02 <rusty> roasbeef: there's no agreed short_channel_id until it's hit the depth specified.
20:49:15 <bitconner> 657 also introduces more ordering requirements on retransmission, which makes the section touched in 620 incorrect
20:49:32 <rusty> Until we both acknowledge that has been reached, messaged with a scid are meaningless.
20:49:48 <BlueMatt> roasbeef: because speed of light is not infinite
20:50:32 <bitconner> so on its own idt 657 is complete
20:51:30 <roasbeef> #action #657 to have wording clarified on PR
20:51:36 <roasbeef> lol sorry if i'm slow, in another meeting
20:51:51 <roasbeef> #topic 656
20:51:55 <roasbeef> (bolt 11 feature bits)
20:51:58 <roasbeef> related to the amp stuff
20:52:01 <bitconner> ack
20:52:11 <roasbeef> basically to have bits on the invoice to signal what type of payments the receiver accepts
20:52:36 <roasbeef> which can then possilby gate additional tags, such as the stream_id in the current amp draft
20:52:58 <cdecker> ACK
20:52:59 <roasbeef> comments on the PR are pretty minor, so maybe we leave it to there unless there're burning questions? (also to get to other stuff)
20:53:17 <rusty> Yeah, future expandability FTW.  I mean, optional fields are good, but if they only need a single bit it's overkill, and there's no other way to say "if yuo don't udnerstand this, you're screwed".
20:53:47 <t-bast> ACK 656
20:54:21 <t-bast> Can we spend some time on 557?
20:54:29 <roasbeef> #action general acceptance of #656, to hash out on PR
20:54:36 <rusty> (The best thing about IRC meetings is I can cook breakfast at the same time and nobody knows!)
20:54:47 <roasbeef> #topic #577 (range queries)
20:55:02 <t-bast> #link https://github.com/lightningnetwork/lightning-rfc/pull/557
20:55:38 <bitconner> last i looked 557 was pretty close, haven't validated the test vectors myself tho
20:55:42 <t-bast> c-lightning supports this with the experimental flag and it inter-operates with eclair correctly (sstone correct me if I'm wrong)
20:55:58 <t-bast> bitconner do you have a draft implementation?
20:56:05 <bitconner> we do not atm
20:56:15 <t-bast> ok
20:56:33 <t-bast> what's your opinion on the PR as it stands though?
20:57:04 <rusty> We need confirmation of eclair interop, IMHO.  There were questions over final format (compression for checksums or not)
20:57:13 <sstone> yes I did basic tests against CL on master and eclair and CL can talk to each other and understand their queries
20:57:34 <t-bast> I think we agreed to drop compression for checksums last time
20:57:35 <sstone> rosabeef: we ditched compression for checksums
20:57:58 <bitconner> and drop the encoding type iirc
20:58:06 <t-bast> exactly
20:58:17 <t-bast> I think the PR has been updated accordingly and implementations too, right?
20:58:53 <sstone> yes PR, eclair and CL and up to date afaik
20:58:54 <bitconner> pr looks good to me, in the future we should also consider an analogous method for querying node_announcments, but that can be separate
20:59:06 <rusty> sstone: OK, I am happy for this to be included then!
20:59:22 <bitconner> definitely a big improvement :)
20:59:29 <rusty> ... in theory since our release is delayed, I could remove the experimental gate on release...
20:59:30 <BlueMatt> rusty: where are we on #513?
20:59:47 <BlueMatt> or, can we talk about that next
21:00:25 <t-bast> Do we think we're ready to drop the experimental gate for CL and Eclair release and merge 557?
21:00:25 <sstone> rusty: when is your release planned ? I can do more comprehensive tests now (was traveling last week)
21:01:15 <rusty> sstone: we're a few days overdue already, was hoping for final tomorrow.  I think it's too late, but next release in 8 weeks for sure.
21:01:40 <roasbeef> (queue'ing up #513 as next topic)
21:02:04 <t-bast> ok let's be safe and keep it as experimental for now then, but shall we merge 557 once more tests have been done between Eclair and CL?
21:02:11 <roasbeef> main thing of #577 now seems to need for itnerop tests?
21:02:44 <bitconner> i think so, yes
21:03:16 <rusty> sstone: I haven't completed my protocol tests for 557, which I can do this week too.
21:03:19 <rusty> t-bast: ack
21:03:26 <roasbeef> #info CL and eclair working on interop, CL working on further tests then can start
21:03:48 <t-bast> ack
21:03:49 <roasbeef> #action 577 mostly reeady to go, gated on interop tests between CL and eclair, lnd to follow later (but not blocking)
21:03:56 <bitconner> ack
21:04:09 <t-bast> just for clarification, when roasbeef states 577 he does mean 557 ;)
21:04:11 <sstone> roasbeef, conner: do you plan to eventually implement it ? I think it's a good basis for INV-based gossip (but am biased obviously)
21:04:40 <roasbeef> lol yeah 557 sorry
21:05:03 <roasbeef> would say we care more about the inv stuff, not sure there's a really strong over lap though depending on which way we go, as there's that other inv PR as well
21:05:16 <roasbeef> y'all know how I feel about channel updates n stuff, so I won't rehash that lol
21:05:26 <niftynei> #action s/577/557/
21:05:34 <roasbeef> #topic #642 (static remote key)
21:05:48 <roasbeef> ok i've mostly implemented this on the lnd side, but only outstanding comment is to have a funding flag
21:06:03 <roasbeef> so impls can initaite one or the other, even if they have the feature bit set
21:06:13 <roasbeef> i think the impact of this PR depends on how impls do their key derivation
21:06:38 <rusty> roasbeef: I've implemented too.  Hmm, I assumed if we both supported it, we both wanted it?
21:06:55 <roasbeef> one other thing that dawned on me while implementing this and the tlv onion PR is the blow up in test scenarios (with the way our tests are at least)
21:07:25 <roasbeef> due to the growing compat matrix
21:07:37 <roasbeef> but if I have optional, and someone connects to me, I need to know that they want the legacy
21:08:02 <roasbeef> also if people want to mix it for w/e reason, and also for older impls that may not have the newer stuff but still need basic compat on channel funding
21:08:17 <cdecker> roasbeef: I'm afraid that combinatorial explosion will always be there if we add new features
21:08:33 <rusty> roasbeef: if someone doesn't offer it and it's optional for me, I know not to use it.  I'm hoping we can quickly (~1yr) move off the old style.
21:08:40 <roasbeef> yeh, it just became a bit more real to me lol cdecker
21:09:11 <cdecker> Hehe, yeah, in CL we're fighting the same issue as well, with more platforms that we deploy to
21:09:24 <rusty> and the resulting Travis timeouts!
21:09:32 <roasbeef> rusty: so then old nodes just can't make channels as soon as the network starts to switch over?
21:09:38 <roasbeef> the diff is a hard change vs a more gradual one
21:10:02 <bitconner> old nodes would use the original format, no?
21:10:17 <bitconner> since they wouldn't advertise the feature bit
21:10:19 <roasbeef> i mean if a new node has the optional feature bit, and an old nodes want to open a chanenl, what then?
21:10:26 <rusty> roasbeef: if everyone makes it a compulsory option (even) they won't even be able to connect any more!
21:10:33 <roasbeef> if you have the funding flag, the old node sends the flag and then they know
21:10:38 <bitconner> they'd use the old format iiuc
21:10:48 <rusty> roasbeef: if you don't both offer it, it's old-style.
21:11:01 <rusty> If both offer it (odd or even) it's on.
21:11:05 <roasbeef> so new nodes won't open channel w/ old nodes?
21:11:13 <rusty> roasbeef: eventually, sure.
21:11:26 <bitconner> or only if the new node requires
21:11:27 <t-bast> during the transition new nodes should only optionally support it
21:11:33 <rusty> (This is true of any even feature of course)
21:12:08 <t-bast> for X releases just make the default be optional
21:12:28 <roasbeef> ok i guess it's a diff of explicit vs implicit negotiation
21:12:44 <roasbeef> explicit being to send the flag, implicit being to do logical and of feature bit existence
21:13:18 <roasbeef> but for interop, it seems useful to have a new node open an old channel
21:13:41 <roasbeef> as we'll still want to test this legacy format for some period of time, for eaxmple in cdecker's tests
21:13:57 <t-bast> yes, but that's just a matter of changing configuration files for some tests
21:14:01 <roasbeef> as an eaxmple for lnd, we now have a build tag for legacy features that lets you toggle them on, while we default to the newer stuff internally
21:14:07 <roasbeef> well that depends lol
21:14:19 <t-bast> oh you don't enable it by environment flags or config?
21:14:29 <t-bast> it's really set at build time?
21:14:37 <roasbeef> what? legacy stuff? it depends
21:14:42 <roasbeef> there's a lot of ways to do it
21:14:47 <rusty> roasbeef: sure, and my protocol tests test this (https://github.com/lightningnetwork/lightning-rfc/pull/655) (well, I havent pushed the static remotekey tests yet)
21:14:48 <roasbeef> but the other comment on there was about the existing DLP stuff
21:14:56 <roasbeef> and if we can "remove" fields on not
21:14:58 <roasbeef> or not
21:15:12 <roasbeef> as is we can't since this message was created pre-tlv, so you need to have som eplace holder there if we want to extend it in the future
21:15:28 <bitconner> imo it's better to leave blank than remove
21:16:37 <cdecker> Ultimately the cdecker/lightning-integration tests will need to be replaced by Rusty's more targetted tests, since we already have a combinatorial nightmare
21:17:11 <roasbeef> just glanced at rusty's thing (hadn't seen it before), but can we use json rather than some custom grammar?
21:17:29 <rusty> roasbeef: No, I tried that and it's unwritable :(
21:17:38 <roasbeef> less parsing to do
21:17:44 <roasbeef> but this isn't easily machine readable
21:18:03 <roasbeef> you can still have comments/commentary as well
21:18:25 <roasbeef> but this is anotehr topic, ok so maybe we don't need that funding flag and we can make things implicit
21:18:44 <roasbeef> the remainder we can hash out on the PR?
21:18:54 <roasbeef> lnd's impl should be ready this week, so we can start testing
21:19:54 <bitconner> hash out on pr sgtm
21:20:08 <rusty> roasbeef: ack!
21:20:20 <t-bast> sgtm
21:20:20 <roasbeef> #action stick with only feature bit on #642, next steps are interop
21:20:28 <roasbeef> #topic 627
21:20:33 <roasbeef> (onion falire code)
21:20:51 <roasbeef> rn for lnd we send invalid onion key iirc, but would be nice to give the sender more context
21:21:05 <t-bast> ok good to know
21:21:15 <roasbeef> I guess there's a question of if there's some other probing vector here
21:21:17 <t-bast> I changed eclair to just send PERM right now
21:21:55 <rusty> I'd like to see an offset field in there, to provide a hint as to what in the onion we disliked...
21:22:08 <roasbeef> or even the type?
21:22:14 <roasbeef> (tlv type)
21:22:17 <t-bast> rusty: but that doesn't always make sense
21:22:19 <roasbeef> assuming it awas a parsing failure
21:22:33 <roasbeef> maybe we should reduce the scope and add another type if needed?
21:22:45 <roasbeef> the use case I have in mind is "i can't parase this type" or "i don't know of this type"
21:22:47 <t-bast> that doesn't cover the case where we're missing data (or we put as offset the length of the tlv stream)?
21:23:10 <t-bast> I think this is more general, we also have to cover the case "you didn't include all the types I need to process this"
21:23:18 <roasbeef> two paths again here: more specific information for debugging, or more vague information to just signal that we're unable to proceed
21:23:46 <rusty> t-bast: missing data, yes I think so.  I think we;ve enjoyed the debugging hints when we've added them to msgs, OTOH FFS TLV isn't that hard :)
21:24:01 <t-bast> rusty: heh
21:24:33 <t-bast> so should I update this PR to list more explicitly all the cases covered by this and add an offset field? anything else you'd like to see in there?
21:24:59 <bitconner> on a tlv-related note: be sure to add a soft cap when decoding varbytes, o/w someone can trivially cause you to oom
21:24:59 <t-bast> I don't think it introduces a probing vector, but if you do find one I'm interested
21:25:16 <bitconner> thanks to eugene for fuzzing our tlv library!
21:25:22 <t-bast> bitconner: yep ;)
21:26:09 <roasbeef> ok so we're leaning towards keeping the error general then?
21:26:17 <roasbeef> or minimally add a type and some length offset?
21:26:37 <t-bast> I think type + length offset would be useful
21:26:56 <t-bast> and can be removed in X releases once we're confident implementations interoperate flawlessly
21:27:23 <bitconner> tbh i'd rather add it later than try to remove it down the road
21:27:41 <rusty> t-bast: we'll want it in as we add new TLV fields anyway.
21:27:52 <roasbeef> #action possibly add type+length offset to #627, need of some sort is clear
21:28:15 <rusty> t-bast: offset is offset within onion or within specific TLV...
21:28:17 <roasbeef> do we want to conclude? or can ppl hang around?
21:28:31 <t-bast> rusty: I'll brainstorm this and propose something on the PR
21:28:44 <bitconner> thoughts on https://github.com/lightningnetwork/lightning-rfc/pull/660?
21:28:47 <bitconner> pretty easy 1
21:29:03 <t-bast> could we quickly merge 601? It's been there for ages and is a quite simple clarification (with approvals already)
21:29:37 * cdecker waves, needs to run off :-)
21:30:09 <t-bast> 660: I put some comments in there, I think that it either needs to be a simple tlv "property bag" that we define for that specific use-case, or we should have a more general discussion on what feature flags / tlv types devs can use for experimentations
21:30:43 <bitconner> 601 lgtm
21:30:45 <t-bast> something like what z-man proposed with "I can haz extension", extended to cover "I can haz tlv"
21:31:16 <t-bast> bitconner: thx, can I merge it then (we already had rusty's ack)
21:32:12 <bitconner> t-bast: i think those are slightly different tho
21:32:12 <roasbeef> oh yeh the question of experimetns is coming up again
21:32:28 <roasbeef> basically how can devs experiment with new funding or channel stuff, that doesn't deviate too far from the current protocol
21:33:04 <t-bast> yeah I think this is a more general discussion and #660 is either too specific or not general enough
21:33:17 <t-bast> let me rephrase that
21:33:41 <bitconner> discussion leading up on tlv stuff was that we'd allow people to experiment with their own tlv records, tho we never defined which ones would be reserved for official spec use
21:33:53 <t-bast> #660 should either be very specific and solve only the mentioned use-case of an opaque property bag for onion tlv, or touch on the general subject of experimenting with new tlv types without impacting the network
21:34:51 <roasbeef> t-bast: wells there's two dimensions: the onion and general message extensions
21:34:53 <t-bast> if we start the general discussion of reserving tlv type ranges, I think it would make sense to group it with "I can haz extension"
21:35:13 <roasbeef> i didn't get that extension thign
21:35:16 <t-bast> I think both should use the same kind of mechanism / ranges (or at least we should consider it)
21:35:38 <roasbeef> i guess the q is where should the signaling be done, in the protocl or outsside to reserve ranges
21:35:56 <lndbot> <joost.jager> my main q around 660 is whether a single opaque tlv field for user data would have any down sides compared to multiple custom fields that can be picked from a range.
21:36:08 <roasbeef> it restricts devs imo
21:36:11 <roasbeef> as now there's on slot
21:36:16 <roasbeef> also not like we're short on types with 64 bits
21:36:17 <t-bast> that's honestly an interesting question because it's not only technical, it also touches on coordination in an open p2p network
21:36:19 <roasbeef> one*
21:37:03 <bitconner> i dislike the idea of cramming all extensions into a single tlv record
21:37:10 <bitconner> we have 2^64 options
21:37:12 <t-bast> definitely
21:37:59 <roasbeef> it could be as light weight as resreve 16 or so bits, then have ppl shout out on the ML what they're using
21:38:05 <lndbot> <joost.jager> yes, people can even put their own tlv into the single tlv fields, but it isn't necessary.
21:38:06 <roasbeef> then if stronger coordination is needed, we'll see the pain points
21:38:09 <t-bast> but how do we allow people to pick values (tlv types of message types) that they can "safely" use to experiment by forking an existing implementation without negative impact on the network - that's what we need a solution to
21:38:10 <roasbeef> (speaking of onion tlv only)
21:38:23 <roasbeef> what negative impact would you see t-bast ?
21:38:29 <roasbeef> assuming it's mostly at sender/receiver
21:38:37 <roasbeef> and intermediate nodes can use that error to say "i don't understand this"
21:38:54 <t-bast> just two dev teams inadvertently using the same values for two different things and thus breaking each other's feature
21:38:57 <bitconner> t-bast: read from /dev/random :)
21:39:21 <t-bast> bitconner: that's a solution :)
21:39:48 <bitconner> t-bast: i think leaving things open for experimentation will always result in possible collisions to some extent
21:39:53 <t-bast> bitconner: and then if you want to promote this to a spec-ed thing you re-assign and update nodes?
21:39:57 <niftynei> +1 for a range. we could put up wiki page on the rfc repo for 'claiming' extension numbers
21:40:14 <bitconner> worst case most likely is just a parsing failure
21:40:22 <t-bast> a wiki page or a github issue where anyone could grab available values could be a start
21:40:35 <niftynei> ideally some page that's pretty widely editable
21:40:46 <bitconner> i hereby reserve all unreserved types
21:40:47 <t-bast> but append-only ideally
21:40:50 <roasbeef> would like to do it in a way that involved as little coordination as possible
21:40:52 <niftynei> having a singular document rather than a comment set (issue) is preferable imo
21:41:01 <roasbeef> for example, the colored coins people are attepmting something more aggressive
21:41:05 <roasbeef> basically a sub-protocol
21:41:13 <roasbeef> to what extend do we all need to be involved in that?
21:41:30 <niftynei> yeah a globally editable page (wiki) with a changelog history seems ideal
21:41:43 <bitconner> my main concern with a doc would be the opportunity for squatting
21:41:48 <t-bast> if they really launch something and depend on the values they chose, at some point this needs to be spec-ed somewhere to avoid collision - I'm not sure how we should be involved in that
21:41:53 <roasbeef> bitconner: you snooze you lose
21:42:02 <niftynei> having them all in one place rather than spread out over the mailing list is nice for discoverability
21:42:15 <roasbeef> t-bast: yeh but imo would rather allow ppl to experiment w/o having to always go thru the spec process, and the ML is enogh for that in the short term imo
21:42:20 <bitconner> tlv.casino
21:42:23 <niftynei> :D
21:42:30 <niftynei> high rollers only
21:42:34 <roasbeef> (again there's two optics in flight tlv in the onion and arbitrary tlv extensions on all messages)
21:42:37 <roasbeef> topics*
21:42:39 <roasbeef> kek
21:42:40 <t-bast> roasbeef: I completely agree that the spec process isn't what should be used, but I'd like to brainstorm what we can use :)
21:42:43 <roasbeef> biiiiig monay
21:42:49 <lndbot> <joost.jager> the main intention of the pr was to at least define and defend (with constraints in the implementation) an official range. a second question is how to coordinate the experiments range.
21:43:12 <niftynei> i'm +1 for a range designation
21:43:31 <t-bast> agreed
21:43:32 <t-bast> but
21:43:41 <t-bast> I don't understand why you only did that for onion tlv
21:43:58 <t-bast> imo it makes sense to define a tlv range open for experimentation for all tlv in all messages
21:44:19 <bitconner> i'm in favor of formalizing the range, i think 2^32 is plenty for our use case, but yes should apply to all tlv types
21:44:33 <t-bast> bitconner: sgtm
21:45:34 <lndbot> <joost.jager> pr was written in line with my current focus. we could generalize that, but does it need to be a single pr?
21:45:50 <bitconner> as far as coordination, seems like it should roughly translate to the problem of choosing default ports, but with much lower collision rate
21:46:15 <bitconner> self-coordination works for the most part :)
21:46:20 <t-bast> joost: I think it will end up being a small PR even if we say this range applies to all tlvs, and it's not too costly to do the general one from the start
21:46:39 <bitconner> possibly the change should be in bolt 1 then?
21:46:51 <t-bast> we can probably start without specifying coordination, and suggesting using randomness to avoid collisions
21:46:57 <lndbot> <joost.jager> for custom tlv in the onion, it may also be useful to distinguish between key/values that are just passed on without any interpretation to higher level applications and types that are used internally in forked versions of node software
21:47:02 <t-bast> I think it would make sense in Bolt 1 yes
21:47:38 <t-bast> joost: I think that's a different issue: this is solved by assigning a tlv type in the onion namespace to act as a property bag
21:47:58 <lndbot> <joost.jager> the range can be defined in a generic way, but at least for the onion there are still some specific directions on how to deal with it as a reader and writer
21:48:09 <bitconner> joost: does that mean even/odd rule wouldn't apply to "big" types?
21:48:10 <lndbot> <joost.jager> t-bast: is there consensus on that already?
21:48:25 <niftynei> should we only open up odd types in a range?
21:48:52 <t-bast> joost: not yet I think, that's just my 2 cents ;)
21:48:56 <lndbot> <joost.jager> i am not sure what the best decision is there. is assumed we only use the odd types
21:49:47 <bitconner> if the fields are being passed to the application layer, then likely the software won't ever know the encoding
21:50:03 <t-bast> it depends, if you want to experiment with something that forks away from the main network you can use even experimental tlv types for that - but that's probably not a common use-case
21:50:26 <bitconner> we could petition the gods of math to make all numbers > 2^32 odd, but may be a bigger change
21:51:04 <t-bast> I think we should simply explain that users should pick odd numbers if they want to interact with the main network, and that they may pick even number but in that case they'll likely fork the network
21:51:31 <t-bast> and they'll be in a very small fork :)
21:51:49 <bitconner> sgtm, add range + recommend odd
21:52:23 <lndbot> <joost.jager> ok. and a defined `user_data` field, we'll drop that then?
21:52:46 <t-bast> I think that's a separate PR imo
21:53:01 <lndbot> <joost.jager> if we have the custom range, why would I use the user data field then still?
21:53:11 <roasbeef> you wouldn't lol
21:53:18 <t-bast> if you don't need it anymore once you have the range, perfect then!
21:53:24 <t-bast> we can forget about it ;)
21:53:39 <lndbot> <joost.jager> Ok, clear then :slightly_smiling_face:
21:53:56 <t-bast> I gotta go, thanks guys see you next time!
21:54:15 * bitconner waves to t-bast
21:55:02 <lndbot> <joost.jager> leaving too, bye!
21:55:55 <bitconner> time to conclude?
21:55:57 <bitconner> #endmeeting
21:57:53 <BlueMatt> rusty: where are we on #513?
21:58:18 <BlueMatt> specifically, looks like core may be able to land all the required bits for the next release
21:58:28 <BlueMatt> and its...uhhh...kinda critical to the lightning security story
22:03:24 <roasbeef> BlueMatt: that's been split into two pr s
22:03:33 <roasbeef> one that does the key derivation change, and the other that'll do the fee related stuff
22:04:05 <roasbeef> even once they land, it'll be some time for relaying nodes to update, so we can go w/ a staged version: add the hooks, keep in update_fee (to set a base line)
22:06:58 <BlueMatt> roasbeef: huh? what do you mean keep in update_fee?
22:07:11 <BlueMatt> roasbeef: its a per-node decision? I mean sure people have to update but it only effects your own channels with your peers
22:07:17 <BlueMatt> not across the network
22:07:48 <BlueMatt> roasbeef: it may be useful to comment on github what the status of things is, so that people can see it :p
23:00:08 <roasbeef> BlueMatt: i mean an intermediate stage where that's still used to handle fees, and we aren't solely depending on the CPFP mechanism
23:00:20 <roasbeef> oh that other PR is up, we discussed it a bit earlier
23:02:51 <roasbeef> #endmeeting