12018-09-09T00:24:35  *** miknotauro has joined #bitcoin-core-dev
  22018-09-09T00:29:00  *** echonaut has quit IRC
  32018-09-09T00:29:07  *** echonaut10 has joined #bitcoin-core-dev
  42018-09-09T01:36:02  *** drexl has quit IRC
  52018-09-09T01:38:58  *** polydin has joined #bitcoin-core-dev
  62018-09-09T02:19:51  *** saransh_ has joined #bitcoin-core-dev
  72018-09-09T02:31:39  *** saransh_ has quit IRC
  82018-09-09T02:53:48  *** ken2812221_ has joined #bitcoin-core-dev
  92018-09-09T02:54:47  *** ken2812221 has quit IRC
 102018-09-09T03:06:31  *** unholymachine has quit IRC
 112018-09-09T03:07:27  *** unholymachine has joined #bitcoin-core-dev
 122018-09-09T03:16:15  *** polydin has quit IRC
 132018-09-09T03:23:19  *** Krellan has quit IRC
 142018-09-09T03:24:17  *** Krellan has joined #bitcoin-core-dev
 152018-09-09T04:12:40  *** RubenSomsen has quit IRC
 162018-09-09T04:22:07  *** polydin has joined #bitcoin-core-dev
 172018-09-09T04:43:15  *** jhfrontz has joined #bitcoin-core-dev
 182018-09-09T04:50:09  *** adiabat has joined #bitcoin-core-dev
 192018-09-09T05:02:08  *** jhfrontz has quit IRC
 202018-09-09T05:05:27  *** jhfrontz has joined #bitcoin-core-dev
 212018-09-09T05:29:13  *** Krellan has quit IRC
 222018-09-09T05:33:40  *** Krellan has joined #bitcoin-core-dev
 232018-09-09T05:38:32  *** jhfrontz has quit IRC
 242018-09-09T05:40:56  *** pkx1 has joined #bitcoin-core-dev
 252018-09-09T05:41:25  *** jhfrontz has joined #bitcoin-core-dev
 262018-09-09T05:45:20  *** pkx1 has quit IRC
 272018-09-09T05:45:54  *** pkx1 has joined #bitcoin-core-dev
 282018-09-09T05:51:01  *** TheHoliestRoger has quit IRC
 292018-09-09T05:53:43  *** pkx1 has quit IRC
 302018-09-09T05:53:50  *** TheHoliestRoger has joined #bitcoin-core-dev
 312018-09-09T05:54:22  *** hebasto has joined #bitcoin-core-dev
 322018-09-09T05:59:19  *** TheHoliestRoger has quit IRC
 332018-09-09T06:00:52  *** jhfrontz has quit IRC
 342018-09-09T06:01:04  *** jhfrontz has joined #bitcoin-core-dev
 352018-09-09T06:02:07  *** TheHoliestRoger has joined #bitcoin-core-dev
 362018-09-09T06:17:07  *** plankers has joined #bitcoin-core-dev
 372018-09-09T06:18:06  *** jhfrontz has quit IRC
 382018-09-09T06:27:57  *** hebasto has quit IRC
 392018-09-09T06:33:43  *** pkx1 has joined #bitcoin-core-dev
 402018-09-09T07:09:49  *** pkx1 has quit IRC
 412018-09-09T07:19:32  *** phwalkr has joined #bitcoin-core-dev
 422018-09-09T07:28:13  *** phwalkr has quit IRC
 432018-09-09T07:35:18  *** Krellan has quit IRC
 442018-09-09T07:36:22  *** Krellan has joined #bitcoin-core-dev
 452018-09-09T07:43:32  *** plankers has quit IRC
 462018-09-09T07:52:27  *** echonaut has joined #bitcoin-core-dev
 472018-09-09T07:53:52  *** echonaut10 has quit IRC
 482018-09-09T08:16:12  <wumpus> there is a feature for manual pruning, isn't  there?
 492018-09-09T08:16:43  <wumpus> I remember something like that was merged at some point, a command line flag to prevent automatic pruning as well as a RPC to prune
 502018-09-09T08:17:47  <wumpus> client software could use this to prune only what they no longer require
 512018-09-09T08:21:53  <wumpus> (I don't think any lightning daemons do this, at the moment, but would be useful)
 522018-09-09T08:37:53  <CubicEarth> wumpus: yeah, sipa was also saying there is manual prune option
 532018-09-09T08:39:53  <CubicEarth> so that should be able to be used by lightning nodes to ensure the only blocks pruned are ones they don't need anymore
 542018-09-09T08:42:01  *** hebasto has joined #bitcoin-core-dev
 552018-09-09T08:42:57  <gmaxwell> the design of manual prune is kinda not ideal if you have multiple potential things that need data. :(
 562018-09-09T08:44:47  <CubicEarth> if there was a DoNotPruneAbove, it could be set by different things, with the lowest one controlling
 572018-09-09T08:45:30  <gmaxwell> indeed but then you'd need interface elements to remove zombie DNPAs.
 582018-09-09T08:45:55  <CubicEarth> Yeah :)
 592018-09-09T08:46:09  <CubicEarth> If too many things want the data, not pruning at all might be favorable
 602018-09-09T08:50:43  <gmaxwell> it might be more useful if there were just a way of setting it up so you won't process blocks unless your lightning is running, or you've removed the binding.
 612018-09-09T08:51:00  <CubicEarth> In either case though, if blocks are to be retained until the pruning command is given by another service, it seems useful to have bitcoin have an option to not download more than some MBs or GBs worth of blocks
 622018-09-09T08:51:12  <CubicEarth> ahead
 632018-09-09T08:53:23  <CubicEarth> gmaxwell: are we saying the same thing?
 642018-09-09T08:59:21  <wumpus> it's not ideal, but nothing is even using the current functionality as far as I know, so I'm kind of reluctant to discuss adding even more
 652018-09-09T09:00:12  <wumpus> once it works for the 1 bitcoind - 1 lightningd case, it'd be more fruitful to discuss extending it to other cases
 662018-09-09T09:01:16  <wumpus> instead of adding a lot of logic for something that no one will be using in practice
 672018-09-09T09:02:43  <wumpus> FWIW "DNPA" could be implemented by an external process using the current RPC interface
 682018-09-09T09:02:57  <wumpus> have it manage the list of requirements and manage manual pruning
 692018-09-09T09:03:01  <wumpus> accordingly
 702018-09-09T09:05:01  <CubicEarth> Yeah. DNPA is mostly the same as manual pruning it seems.
 712018-09-09T09:05:13  <CubicEarth> And that makes sense about not adding complexity.
 722018-09-09T09:05:55  <CubicEarth> Maybe it would be simple to add a DoNotValidateBeyond ?
 732018-09-09T09:06:21  <wumpus> what I mean is that manual pruning provides the required low-level interface, DNPA would be one strategy to manage it at a higher level with multiple data consumers, but likely not the only way (at some point you might want to kill off consumers if they're too slow, for example, depending on disk space etc)
 742018-09-09T09:08:15  <CubicEarth> I understood as such ^^ even if my reply suggested otherwise :)
 752018-09-09T09:08:23  <wumpus> something like that, though validation is not the problem in this case, block download is - you'd want to tell it to pause downloading new blocks
 762018-09-09T09:08:58  <wumpus> I think there's a command that more or less does that, pause networking, but might be GUI-only I don't remember exactly
 772018-09-09T09:10:04  *** SopaXorzTaker has joined #bitcoin-core-dev
 782018-09-09T09:10:55  <wumpus> "setnetworkactive"
 792018-09-09T09:11:04  *** Krellan has quit IRC
 802018-09-09T09:12:35  <wumpus> it can continue validating the blocks it already has, that won't fill up disk space
 812018-09-09T09:13:13  <wumpus> with manual pruning, it's not the validation that triggers pruning anyhow
 822018-09-09T09:15:09  <CubicEarth> hmmm, but block are not downloaded sequentially anymore, right?
 832018-09-09T09:16:08  <CubicEarth> or non-sequentially within a sliding window?
 842018-09-09T09:16:13  <wumpus> I don't see how that is relevant to this, stopping block download is stopping block download, whether they're downloaded sequentially or not, if you want it to stop filling up disk you want to pause block download
 852018-09-09T09:17:49  <wumpus> the blocks it downloads can be non-sequential in a consensus sense, but on disk, they're all simply to the block files in a first come first serve basis
 862018-09-09T09:17:56  <wumpus> +appended
 872018-09-09T09:19:13  <CubicEarth> I was thinking it wouldn't be as helpful to have downloaded and storing a bunch of blocks, and yet be missing an earlier one, at the point networking was shut off
 882018-09-09T09:19:36  <wumpus> why not?
 892018-09-09T09:20:02  <CubicEarth> validation couldn't proceed
 902018-09-09T09:20:03  <wumpus> that stops filling the disk; once it's turned on again, it will download the rest and connect them
 912018-09-09T09:20:09  <wumpus> of course validation cannot proceed
 922018-09-09T09:20:20  <wumpus> you're essentially shutting down your node without really shutting it down
 932018-09-09T09:20:53  <wumpus> so it will respond to RPC queries to get the blocks and other data already there
 942018-09-09T09:21:01  <wumpus> and to pruning commands
 952018-09-09T09:21:44  <wumpus> once the clients have caught up, and a pruning command has been issues, the networking can be waken up again, but maybe I'm missing something I don't know...
 962018-09-09T09:22:44  <wumpus> and validation will continue as far as it can with the blocks that it had at the time of network shutdown, not further
 972018-09-09T09:24:30  <CubicEarth> I need to think about it more (not from the lightning perspective ... I am no lighting expert). What you are describing might be totally workable. I was just thinking if blocks are not downloaded sequentially, and networking was turned off, there will be storage being taken up with blocks that can't be validated due to gaps. I wasn't thinking practically so much. Just in theory, it would be nice to only be storing
 982018-09-09T09:24:30  <CubicEarth> what could be validated
 992018-09-09T09:25:09  <CubicEarth> Yes, those gaps would be filled as soon as networking was turned back on, and it would proceed.
1002018-09-09T09:25:46  <wumpus> yes I think it makes sense to get a clear idea of what you practically need, otherwise you'll by definition drag unrelated concerns into it
1012018-09-09T09:25:59  <wumpus> exactly
1022018-09-09T09:29:37  <wumpus> from the perspective of the lightning clients they will see the block number stop increasing when validation is blocked, which will negatively affect their usefulness as lightning nodes, channel status updates will no longer be detected
1032018-09-09T09:31:55  <CubicEarth> the block download window is 1024, btw
1042018-09-09T09:32:52  <gmaxwell> CubicEarth: no, above I meant a something much simpler. A setting in bitcoind where it just won't run unless your lightning daemon is connected.
1052018-09-09T09:33:55  <gmaxwell> but as wumpus was saying, they aren't even bothering to use the existing functionality.
1062018-09-09T09:37:30  <CubicEarth> those lazy lighting devs!
1072018-09-09T09:39:51  <gmaxwell> more like they only recently graduated to not requiring -txindex...
1082018-09-09T09:41:55  <jl2012> I have a question about this: https://github.com/bitcoin/bitcoin/blob/master/src/secp256k1/src/scalar_8x32_impl.h#L78
1092018-09-09T09:42:34  <jl2012> if no becomes true in line 78 to 81, yes must be false
1102018-09-09T09:42:46  <jl2012> why don't it return the result earlier?
1112018-09-09T09:44:09  <gmaxwell> Because there is no need to, it might well even be slower to do so.
1122018-09-09T09:47:25  <gmaxwell> it would also make the function non-constant-time if it terminates early.
1132018-09-09T09:47:47  <jl2012> oh, I think this is the real reason?
1142018-09-09T09:48:52  <gmaxwell> if that were the only reason we'd have two versions.
1152018-09-09T09:49:13  <gmaxwell> But as I said, I wouldn't be surprised if branching out earlier was slower.
1162018-09-09T09:52:51  <jl2012> thanks!
1172018-09-09T09:59:28  <wumpus> CubicEarth: please, don't call anyone lazy just because they don't focus on your specific issue, everyone is more like overworked
1182018-09-09T10:00:36  <wumpus> btw, wow, lightiningd's API to get the last log messages for a specific node is very nice
1192018-09-09T10:01:50  <wumpus> even works to get log messages at a lower logging level (say, debug) than are written to disk
1202018-09-09T10:02:29  <gmaxwell> I had aspirations of doing all our logging into a ringbuffer... with disk just taking a feed off it.
1212018-09-09T10:02:49  <gmaxwell> which would allow us to do things like no logs normally, but dump the ring on crash.
1222018-09-09T10:02:59  <wumpus> that's what they do, I think, well I don't know if it's implemented as a ringbuffer but it acts that way on the outside
1232018-09-09T10:03:54  <wumpus> but yes that would be neat to have
1242018-09-09T10:04:03  *** rex4539 has quit IRC
1252018-09-09T10:04:10  <wumpus> at least for log messages that don't cost a lot of overhead to format unnecessarily
1262018-09-09T10:04:23  <CubicEarth> wumpus: you are right. I meant it not seriously, but things don't always come across as intended on irc. Nothing but respect for lightning devs from me!
1272018-09-09T10:04:37  <wumpus> CubicEarth: okay, sorry for misreading you then
1282018-09-09T10:05:32  <gmaxwell> wumpus: yea, I wondered about that but IIRC other than the format strings themselves there are only ~two places in the code where we totally skip some processing according the the log flags.
1292018-09-09T10:05:57  <gmaxwell> and I seem to recall thinking that those could probably just go away.
1302018-09-09T10:06:24  <wumpus> gmaxwell: agree, my concern i not relevant to many places and there we already skip the processin explicitly, so the ring-buffer thing would just work
1312018-09-09T10:06:37  <wumpus> (except for those messages, obviously)
1322018-09-09T10:07:45  <gmaxwell> I think in general, for most users, the logging provides no real value, just wastes disk space even becomes an attack vector (e.g. via spam up the disk attacks). So as a result we've made default logging pretty conservative, but then useful stuff isn't there when something goes wrong.  perhaps more interesting is how any of this would interact with concurrency.
1332018-09-09T10:12:47  <wumpus> it'd have to synchronize around writes to the ringbuffer, instead of around disk writes like now
1342018-09-09T10:13:54  <wumpus> there is a PR that moves part of logging to a thread I suddenly remember, #13200, should look into it in more detail
1352018-09-09T10:13:56  <gribble> https://github.com/bitcoin/bitcoin/issues/13200 | Process logs in a separate thread by jamesob · Pull Request #13200 · bitcoin/bitcoin · GitHub
1362018-09-09T10:14:10  <gmaxwell> right but if the ringbuffer is getting more logs into it because it logs more things than disk, that might be annoying. Of course we could have a seperate buffer per thread(group) and only take locks for all of them to merge output for display.
1372018-09-09T10:14:41  *** Krellan has joined #bitcoin-core-dev
1382018-09-09T10:15:14  <wumpus> yes, fair enough
1392018-09-09T10:42:16  *** rex4539 has joined #bitcoin-core-dev
1402018-09-09T11:09:57  *** SopaXorzTaker has quit IRC
1412018-09-09T11:24:36  *** SopaXorzTaker has joined #bitcoin-core-dev
1422018-09-09T11:32:24  *** drexl has joined #bitcoin-core-dev
1432018-09-09T12:17:10  *** timothy has quit IRC
1442018-09-09T12:19:45  *** Krellan has quit IRC
1452018-09-09T12:20:36  *** Krellan has joined #bitcoin-core-dev
1462018-09-09T12:54:06  *** owowo has quit IRC
1472018-09-09T13:15:56  *** polydin has quit IRC
1482018-09-09T13:21:57  *** drexl has quit IRC
1492018-09-09T13:53:37  *** owowo has joined #bitcoin-core-dev
1502018-09-09T13:57:11  *** jcorgan has joined #bitcoin-core-dev
1512018-09-09T14:04:16  *** Guyver2 has joined #bitcoin-core-dev
1522018-09-09T14:23:24  *** belcher_ has joined #bitcoin-core-dev
1532018-09-09T14:56:57  *** emilengler has joined #bitcoin-core-dev
1542018-09-09T15:25:47  *** nullptr| has quit IRC
1552018-09-09T15:31:04  *** nullptr| has joined #bitcoin-core-dev
1562018-09-09T15:32:31  *** lnostdal has quit IRC
1572018-09-09T15:34:03  *** lnostdal has joined #bitcoin-core-dev
1582018-09-09T15:40:21  *** jarthur has joined #bitcoin-core-dev
1592018-09-09T15:43:17  *** grubles has quit IRC
1602018-09-09T15:47:29  *** miknotauro has quit IRC
1612018-09-09T15:56:41  *** Chris_Stewart_5 has joined #bitcoin-core-dev
1622018-09-09T15:59:39  *** Cheeseo has joined #bitcoin-core-dev
1632018-09-09T16:03:51  *** emilengler has quit IRC
1642018-09-09T16:07:06  *** jhfrontz has joined #bitcoin-core-dev
1652018-09-09T16:07:44  *** hebasto has quit IRC
1662018-09-09T16:15:56  *** jhfrontz has quit IRC
1672018-09-09T16:16:43  *** Chris_Stewart_5 has quit IRC
1682018-09-09T16:49:59  *** promag has joined #bitcoin-core-dev
1692018-09-09T16:50:07  *** jhfrontz has joined #bitcoin-core-dev
1702018-09-09T16:50:40  *** Cheeseo has quit IRC
1712018-09-09T16:52:59  *** jhfrontz has quit IRC
1722018-09-09T16:56:01  *** jhfrontz has joined #bitcoin-core-dev
1732018-09-09T16:58:27  *** Guyver2 has quit IRC
1742018-09-09T17:07:28  *** itaseski has joined #bitcoin-core-dev
1752018-09-09T17:11:32  *** jhfrontz has quit IRC
1762018-09-09T17:31:56  *** grubles has joined #bitcoin-core-dev
1772018-09-09T17:46:50  *** AaronvanW has joined #bitcoin-core-dev
1782018-09-09T17:48:16  *** jarthur has quit IRC
1792018-09-09T17:51:31  *** DougieBot5000 has joined #bitcoin-core-dev
1802018-09-09T17:51:43  *** DougieBot5000_ has quit IRC
1812018-09-09T17:56:53  *** promag has quit IRC
1822018-09-09T17:57:27  *** promag has joined #bitcoin-core-dev
1832018-09-09T17:59:34  *** ExtraCrispy has quit IRC
1842018-09-09T18:01:45  *** promag has quit IRC
1852018-09-09T18:10:58  *** owowo has quit IRC
1862018-09-09T18:11:39  *** DougieBot5000 has quit IRC
1872018-09-09T18:16:27  *** DougieBot5000 has joined #bitcoin-core-dev
1882018-09-09T18:29:04  *** owowo has joined #bitcoin-core-dev
1892018-09-09T18:49:55  *** SopaXorzTaker has quit IRC
1902018-09-09T20:17:09  *** promag has joined #bitcoin-core-dev
1912018-09-09T20:29:09  *** Linrono has joined #bitcoin-core-dev
1922018-09-09T20:30:09  *** promag has quit IRC
1932018-09-09T21:28:32  <phantomcircuit> it seems kind of awkward to me to use the new naming convention in the same (small) function as the old one
1942018-09-09T21:30:47  *** promag has joined #bitcoin-core-dev
1952018-09-09T21:31:18  *** Linrono has quit IRC
1962018-09-09T21:36:24  <sipa> phantomcircuit: if it's a small function, just change it all
1972018-09-09T21:38:46  <phantomcircuit> sipa, im looking at https://github.com/bitcoin/bitcoin/blob/718843aed5e60e42ebbe822457511b4fbbf318eb/src/net.cpp#L1225
1982018-09-09T21:39:01  <phantomcircuit> which is mostly moveonly, changing it would change that
1992018-09-09T21:39:09  <phantomcircuit> but the nMicroTime variable is new
2002018-09-09T21:46:35  *** DigiByteDev has joined #bitcoin-core-dev
2012018-09-09T22:04:39  *** drexl has joined #bitcoin-core-dev
2022018-09-09T22:17:54  *** spinza has quit IRC
2032018-09-09T22:26:25  *** spinza has joined #bitcoin-core-dev
2042018-09-09T22:35:55  *** Zenton has quit IRC
2052018-09-09T22:43:11  *** AaronvanW has quit IRC
2062018-09-09T22:43:43  *** AaronvanW has joined #bitcoin-core-dev
2072018-09-09T22:48:09  *** miknotauro has joined #bitcoin-core-dev
2082018-09-09T23:13:42  *** Chris_Stewart_5 has joined #bitcoin-core-dev
2092018-09-09T23:15:42  *** belcher_ has quit IRC
2102018-09-09T23:19:06  *** profmac has quit IRC
2112018-09-09T23:33:24  *** profmac has joined #bitcoin-core-dev
2122018-09-09T23:34:11  *** Aaronvan_ has joined #bitcoin-core-dev
2132018-09-09T23:37:08  *** AaronvanW has quit IRC
2142018-09-09T23:38:54  *** profmac has quit IRC
2152018-09-09T23:41:08  *** Aaronvan_ has quit IRC
2162018-09-09T23:41:43  *** AaronvanW has joined #bitcoin-core-dev
2172018-09-09T23:43:51  *** Chris_Stewart_5 has quit IRC
2182018-09-09T23:45:47  *** AaronvanW has quit IRC
2192018-09-09T23:48:51  *** promag has quit IRC
2202018-09-09T23:49:25  *** promag has joined #bitcoin-core-dev
2212018-09-09T23:53:27  *** promag has quit IRC