12018-08-01T00:04:26 *** masonicboom has joined #bitcoin-core-dev
22018-08-01T00:04:38 *** Krellan has joined #bitcoin-core-dev
32018-08-01T00:08:23 <BlueMatt> gmaxwell: the fd_set 1024 limit stuff shouldn't cause that kind of problem, ldb shouldnt use almost any fds on 64 bit and should use a low-ish count on 32 bit, no?
42018-08-01T00:08:37 <BlueMatt> I thought that's where we landed (as mmap doesnt need to use an fd)
52018-08-01T00:08:43 <sipa> ossifrage: are you on a 32-bit system?
62018-08-01T00:08:54 <BlueMatt> fwiw I think I have a branch with a rdwr lock implemented somewhere, though I never ended up using it
72018-08-01T00:11:10 <sipa> can you implement a shared lock without support from underlying primitives?
82018-08-01T00:11:27 *** nodweber has quit IRC
92018-08-01T00:11:42 <sipa> seems yes, after short googling
102018-08-01T00:11:42 <BlueMatt> sipa: sure? with atomics and condvars, ofc
112018-08-01T00:12:17 *** Krellan has quit IRC
122018-08-01T00:13:17 <sipa> yeah
132018-08-01T00:13:55 <sipa> with just locks and condition variables you can implement any synchronization primitive, i believe
142018-08-01T00:13:58 * sipa forgot
152018-08-01T00:14:25 <BlueMatt> well, or s/atomics//
162018-08-01T00:17:22 <BlueMatt> my old commit, no idea if its right but... https://github.com/TheBlueMatt/bitcoin/commit/8c5dc5e0ae19120fc79a6e4f7a56d60bbbacf348
172018-08-01T00:24:01 *** d9b4bef9 has quit IRC
182018-08-01T00:24:59 *** grubles has quit IRC
192018-08-01T00:25:15 *** d9b4bef9 has joined #bitcoin-core-dev
202018-08-01T00:26:09 *** Chris_Stewart_5 has joined #bitcoin-core-dev
212018-08-01T00:31:02 *** Chris_Stewart_5 has quit IRC
222018-08-01T00:38:48 *** michaelsdunn1 has joined #bitcoin-core-dev
232018-08-01T00:41:42 <ossifrage> sipa, no x86_64
242018-08-01T00:41:57 <sipa> ossifrage: what OS?
252018-08-01T00:42:19 <ossifrage> linux 4.16.5-300
262018-08-01T00:43:02 <gmaxwell> ossifrage: why did you say above that ldb was using lots of FDs?
272018-08-01T00:43:25 <ossifrage> That was from the output of lsof and some counting
282018-08-01T00:44:21 <BlueMatt> that seems...surprising, given its supposed to do mmap and then close the fd
292018-08-01T00:44:25 <sipa> i have a number of chainstate files open by bitcoind as well - most are mmaped, but not all
302018-08-01T00:44:27 <BlueMatt> so it may still show up in lsof but not use an fd?
312018-08-01T00:44:46 <ossifrage> I was counting file descriptors not maps
322018-08-01T00:45:28 <sipa> i have 30 chainstate files open with an FD, and 999 with mmap
332018-08-01T00:45:29 <ossifrage> The node has been up for >30 days if that makes any difference
342018-08-01T00:45:46 <gmaxwell> come on, why can't we just take the not-many-line-change to use poll? I know libevent future ra ra ra... but we have held off this simple fix for years. :(
352018-08-01T00:46:03 <sipa> gmaxwell: we totally should
362018-08-01T00:46:08 <ossifrage> I'm willing to test :-)
372018-08-01T00:46:22 <BlueMatt> its super trivial to write
382018-08-01T00:46:29 <BlueMatt> would take me longer to dig it up than someone rewrite it
392018-08-01T00:46:33 <gmaxwell> I could dig up an old copy, but I know that phantomcircuit and BlueMatt run nodes with it.
402018-08-01T00:46:45 <sipa> but i've also basically never encountered anyone actually seeing the "FD above 1024" error resulting in actually closed connections
412018-08-01T00:46:54 <sipa> so i wonder what is special about ossifrage's setup
422018-08-01T00:47:15 <sipa> or maybe just nobody ever paid attention to it
432018-08-01T00:47:19 <gmaxwell> more fragmentation of databases? You do see a bunch of files open.
442018-08-01T00:47:27 <gmaxwell> It also can be caused by RPC connections using up FDs.
452018-08-01T00:47:28 <BlueMatt> I dont bother running mine with it anymore, and regularly have 500+ connections
462018-08-01T00:47:39 <BlueMatt> at least that's usually when one asshat makes 100 connections, but usually that asshat is me
472018-08-01T00:47:55 <sipa> having hundreds of files would certainly explain fd<1024 shortage
482018-08-01T00:48:18 <BlueMatt> and I have a ton of scripts that poll rpc regularly, but not constant, sooo
492018-08-01T00:48:19 <ossifrage> I have txindex and sadly my bitcoin data is on a btrfs filesystem (a mistake I won't make again)
502018-08-01T00:48:32 <BlueMatt> both of those are also true on my seednodes
512018-08-01T00:48:35 <gmaxwell> I think we have had other complaints about fd shortage... but I think we were chalking them up to rpc.
522018-08-01T00:48:48 <BlueMatt> I *know* there are issues with rpc
532018-08-01T00:48:52 <ossifrage> The only reason I noticed a problem was I was dropping a ton of connection due to select()
542018-08-01T00:49:27 <gmaxwell> ossifrage: and the problem remained after restarting the node?
552018-08-01T00:49:46 <ossifrage> gmaxwell, I haven't restarted the node
562018-08-01T00:49:53 <gmaxwell> I wonder if this is a high uptime + txindex + only guy in that config who is watching the logs problem?
572018-08-01T00:50:29 <gmaxwell> it would be nice to better understand why leveldb is leaving the files open... but ... switching to poll eliminates all these problems.
582018-08-01T00:50:43 <ossifrage> both txindex and chainstate are gobbling up file descriptors (let me count the maps)
592018-08-01T00:51:13 *** promag has quit IRC
602018-08-01T00:51:15 <BlueMatt> ossifrage: are you sure its using an fd, or just mmap, cause mmap *shouldnt* at least for ldb
612018-08-01T00:51:56 <sipa> BlueMatt: i confirm that lsof shows both fd-ful opened files and mmapped ones
622018-08-01T00:52:08 <BlueMatt> that...sucks
632018-08-01T00:52:13 <ossifrage> 685 maps of chainstate/*.ldb and 268 maps of txindex (odd)
642018-08-01T00:52:15 <sipa> example of an fd one:
652018-08-01T00:52:16 <sipa> bitcoind 11155 pw mem REG 252,1 2173885 783231 /home/pw/.bitcoin/chainstate/864555.ldb
662018-08-01T00:52:26 <sipa> and another:
672018-08-01T00:52:27 <sipa> bitcoind 11155 pw 40r REG 252,1 2173957 779300 /home/pw/.bitcoin/chainstate/864998.ldb
682018-08-01T00:52:41 <sipa> (sorry, swapped them; the first is mmap'ed, the other is with fd)
692018-08-01T00:52:48 <ossifrage> the 40r is a fd map and the mem line is a mmapped one
702018-08-01T00:52:58 <ossifrage> (fd open not fd map)
712018-08-01T00:53:12 <BlueMatt> hmm, I wonder if ldb has tunables for that shit?
722018-08-01T00:53:14 <sipa> though i only have 30 open files
732018-08-01T00:53:20 <sipa> with fd
742018-08-01T00:53:33 <ossifrage> There is a tunable to use 1024 fds, but is that per database?
752018-08-01T00:53:38 <bitcoin-git> [bitcoin] MarcoFalke pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/0fb9c87815d1...e83d82a85c53
762018-08-01T00:53:39 <bitcoin-git> bitcoin/master 9994d01 Jesse Cohen: Add Unit Test for SingleThreadedSchedulerClient...
772018-08-01T00:53:39 <bitcoin-git> bitcoin/master b296b42 Jesse Cohen: Update documentation for SingleThreadedSchedulerClient() to specify the memory model
782018-08-01T00:53:40 <bitcoin-git> bitcoin/master cbeaa91 Jesse Cohen: Update ValidationInterface() documentation to explicitly specify threading and memory model
792018-08-01T00:53:47 <sipa> ossifrage: yes
802018-08-01T00:53:52 <BlueMatt> MarcoFalke: wat
812018-08-01T00:54:15 <bitcoin-git> [bitcoin] MarcoFalke closed pull request #13247: Add tests to SingleThreadedSchedulerClient() and document the memory model (master...scheduler-tests) https://github.com/bitcoin/bitcoin/pull/13247
822018-08-01T00:54:37 <BlueMatt> oh, it did get rewritten to be r-a, nvm
832018-08-01T00:54:54 *** grubles has joined #bitcoin-core-dev
842018-08-01T00:55:05 <BlueMatt> no, it wasnt
852018-08-01T00:55:06 <BlueMatt> https://github.com/bitcoin/bitcoin/compare/0fb9c87815d1...e83d82a85c53#diff-ff632a82ae2fed5941a9dae2c725ad9bR113
862018-08-01T00:55:15 <BlueMatt> MarcoFalke: plz2fix comment broken ^
872018-08-01T00:55:20 <gmaxwell> sipa: also even if the issue didn't exist on x86_64 (though it seems it does), we'd still have it on 32 bit.
882018-08-01T00:55:54 <sipa> how do i find the uptime of by bitcoind?
892018-08-01T00:56:06 <gmaxwell> there is a starttime in node info or something?
902018-08-01T00:56:27 <ken2812221> uptime?
912018-08-01T00:56:33 <ossifrage> Oh, in my case it was bitcoin-qt not bitcoind
922018-08-01T00:56:33 <ken2812221> rpc
932018-08-01T00:56:49 <sipa> ah, i tried getnodeinfo and getuptime
942018-08-01T00:56:55 <sipa> seems it's just 'uptime'
952018-08-01T00:57:21 <sipa> 10 days, apparently
962018-08-01T00:57:59 <ossifrage> 32 days, currently with ~160 connections (which seems to be the most I can get, and I think it has been shrinking as more fds are used)
972018-08-01T00:58:52 <ossifrage> I was going to build a new version, is it useful to reduce the max fd open count for leveldb?
982018-08-01T00:59:04 <sipa> it may impact performance
992018-08-01T00:59:49 <sipa> gmaxwell: on 32-bit systems we limit the max open files to 64
1002018-08-01T01:01:04 <ossifrage> I've mmapped >10k pgm files on this box at one point, not sure why leveldb wouldn't just mmap all of the ldb files?
1012018-08-01T01:03:32 <gmaxwell> ossifrage: oh you've increased your max connections over default.
1022018-08-01T01:03:48 <gmaxwell> so that might be one reason you're seeing this and we are not getting other reports.
1032018-08-01T01:06:35 <ossifrage> gmaxwell, yeah I have a gbit connection, figured I might as well get the most out of the blood money I pay verizon
1042018-08-01T01:07:12 <ossifrage> But if leveldb where to use 1024 fds, there would be nothing left for sockets
1052018-08-01T01:07:18 <gmaxwell> interesting to me that you're actually ending up with that many peers.
1062018-08-01T01:07:43 <ossifrage> gmaxwell, it takes a while, but the connection count slowly increases over time
1072018-08-01T01:08:40 <ossifrage> Before I had a UPS on my ONT, I'd change IPs ever power failure and then it would take a long time before the connection count would go back up (with the new address)
1082018-08-01T01:09:11 <sipa> i have 148 connections
1092018-08-01T01:09:20 <gmaxwell> cool.
1102018-08-01T01:09:48 <gmaxwell> some months back, on my long running nodes I was unable to break 125. must be more inbound right now.
1112018-08-01T01:09:53 <ossifrage> The txindex has been useful a few (rare) times, but just turning that off would delay the problem
1122018-08-01T01:10:00 <gmaxwell> (well, or spies, mine blocked spies really agressively)
1132018-08-01T01:10:15 <ossifrage> that is ~160 connections (with your block list)
1142018-08-01T01:10:28 <gmaxwell> yea, though I haven't updated the blocklist for a while
1152018-08-01T01:10:33 <sipa> ossifrage: txindex on or off wouldn't affect the behaviour w.r.t the chainstate ldb files
1162018-08-01T01:10:36 <gmaxwell> (right now I hav no nodes with public inbound)
1172018-08-01T01:10:48 <gmaxwell> sipa: yes but he is also losing a bunch of fds to txindex.
1182018-08-01T01:10:48 <sipa> and your number for those is on itself pretty high
1192018-08-01T01:10:52 <ossifrage> sipa, yeah but it was the chainstate + txindex that pushed the fd count >1024
1202018-08-01T01:10:59 *** michaelsdunn1 has quit IRC
1212018-08-01T01:11:29 <gmaxwell> Poll is also good because its faster.
1222018-08-01T01:11:54 <ossifrage> I'd happily test a patch :-)
1232018-08-01T01:12:02 <sipa> do we require Vista or later?
1242018-08-01T01:12:11 <sipa> because windows introduced a poll equivalent in Vista
1252018-08-01T01:12:41 <achow101> yes, xp support was removed a few versions ago
1262018-08-01T01:12:41 <gmaxwell> I think we don't formally support older versions but they still work.
1272018-08-01T01:12:44 <gmaxwell> ?
1282018-08-01T01:13:20 <sipa> xp certainly doesn't work anymore
1292018-08-01T01:13:23 <gmaxwell> Also at least in theory we might work on some other platform that doesn't have poll. We could try switching to only poll and see if someone complains.
1302018-08-01T01:13:39 <achow101> "Microsoft ended support for Windows XP on April 8th, 2014, No attempt is made to prevent installing or running the software on Windows XP, you can still do so at your own risk but be aware that there are known instabilities and issues. Please do not report issues about Windows XP to the issue tracker." <-- from 0.14 release notes
1312018-08-01T01:14:06 <ossifrage> I use XP to do my taxes, amazingly the tax software works on it
1322018-08-01T01:16:08 <gmaxwell> hm, I thought poll was faster than select, https://monkey.org/~provos/libevent/libevent-benchmark.jpg then again, maybe I don't understand that graph, because how did they manage 25000 FDs with select?
1332018-08-01T01:17:13 <gmaxwell> sipa: in windows select doesn't have the 1024 fd limit thing
1342018-08-01T01:17:25 <gmaxwell> it's implemented as a linked list or something.
1352018-08-01T01:19:20 <luke-jr> array of fd numbers IIRC
1362018-08-01T01:19:26 <luke-jr> and it does have a limit
1372018-08-01T01:19:37 <luke-jr> just not for the fd numbers themselves
1382018-08-01T01:19:55 <luke-jr> defaults to 64
1392018-08-01T01:20:48 *** nodweber has joined #bitcoin-core-dev
1402018-08-01T01:21:08 <ossifrage> I thought on linux you could call select() with larger fdsets and it would work, but the libc fd_set is a fixed size?
1412018-08-01T01:21:35 <ossifrage> But it is not exactly efficient, especially with sparce sets
1422018-08-01T01:21:44 <luke-jr> you're supposed to be able to #define FD_SETSIZE before including stuff, to get more, but last I checked that was broken in glibc
1432018-08-01T01:22:17 <ossifrage> I've used epoll() for so long, using select() just makes me sad
1442018-08-01T01:22:53 *** Krellan has joined #bitcoin-core-dev
1452018-08-01T01:25:06 <gmaxwell> ossifrage: indeed, but unfortunately BSDs and linux solved "life beyond poll" differently.
1462018-08-01T01:25:55 <ossifrage> gmaxwell, yeah that was never a concern for the stuff I was writing
1472018-08-01T01:26:23 <ossifrage> Sure it's portable, you can port it to any linux you like
1482018-08-01T01:26:30 <gmaxwell> we manage few enough connections that poll is fine anyways.
1492018-08-01T01:28:01 <phantomcircuit> gmaxwell, think you're looking at that graph wrong
1502018-08-01T01:28:08 <phantomcircuit> smaller is better
1512018-08-01T01:28:26 <phantomcircuit> or did you mean that poll() is the same as select() ?
1522018-08-01T01:30:15 <phantomcircuit> select and poll do basically the same thing just with a much better api for poll
1532018-08-01T01:30:26 <phantomcircuit> both pass the entire list to the kernel in every call
1542018-08-01T01:33:02 <ossifrage> epoll() was a big win for high connection count, low traffic
1552018-08-01T01:37:24 <Dave18> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
1562018-08-01T01:37:27 <Dave18> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
1572018-08-01T01:37:31 <Dave18> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
1582018-08-01T01:37:34 <Dave18> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
1592018-08-01T01:39:09 <ken2812221> spam again
1602018-08-01T01:42:00 <phantomcircuit> gmaxwell, epoll and kqueue are virtually identical
1612018-08-01T01:42:03 <phantomcircuit> it's really silly
1622018-08-01T01:46:27 *** michaelsdunn1 has joined #bitcoin-core-dev
1632018-08-01T01:56:16 *** ChanServ sets mode: +o sipa
1642018-08-01T01:56:22 *** sipa sets mode: +n
1652018-08-01T01:56:25 *** sipa sets mode: -o sipa
1662018-08-01T01:56:46 *** nodweber has quit IRC
1672018-08-01T01:57:09 *** Emcy has quit IRC
1682018-08-01T01:58:44 <midnightmagic> I think there's something even more different in NetBSD too.. kevent()? kfilter_register()? I forget now.
1692018-08-01T01:58:51 <gmaxwell> phantomcircuit: yes, I was saying I thought poll was somewhat faster, but apparently not.
1702018-08-01T01:58:56 <gmaxwell> phantomcircuit: go PR that poll patch.
1712018-08-01T01:59:10 <midnightmagic> There are faster things than poll if you use whatever they provide natively.
1722018-08-01T02:00:10 <gmaxwell> yes, but faster is not generally our issue, max connections = 100, or at most a few hundred.
1732018-08-01T02:02:39 <midnightmagic> (also no limitations a la select()'s irritating problems) -- and with a usage of the native event'ing things get very scaleable. But.. not like anyone but somoene like me is going to run a larger-scale system with NetBSD anyway.
1742018-08-01T02:02:59 *** elkalamar has quit IRC
1752018-08-01T02:04:28 <phantomcircuit> midnightmagic, the issue is that epoll and kqeueue and whatever windows uses are all platform specific
1762018-08-01T02:04:38 <phantomcircuit> there's some work being done to move to libevent but that's not done
1772018-08-01T02:04:45 <phantomcircuit> the poll() thing is pretty trivial iirc
1782018-08-01T02:04:59 <phantomcircuit> 80% solution for 10% the effort
1792018-08-01T02:05:31 <gmaxwell> phantomcircuit: open the PR!
1802018-08-01T02:05:38 <gmaxwell> I know you have had a patch.
1812018-08-01T02:06:00 <phantomcircuit> it's like 3 years old now but should be trivial to rewrite
1822018-08-01T02:08:28 <midnightmagic> using things like kevent natively isn't hard, it just needs to be clean and people on those platforms will look after it.
1832018-08-01T02:09:56 <gmaxwell> true but not obviously of any real value.
1842018-08-01T02:13:19 *** nodweber has joined #bitcoin-core-dev
1852018-08-01T02:13:54 <phantomcircuit> midnightmagic, under thousands of fds it doesn't much matter
1862018-08-01T02:15:23 <gmaxwell> phantomcircuit: PR PR PR
1872018-08-01T02:17:40 <phantomcircuit> gmaxwell, yeah yeah
1882018-08-01T02:17:57 *** nodweber has quit IRC
1892018-08-01T02:21:16 <phantomcircuit> gmaxwell, i remember
1902018-08-01T02:21:22 <phantomcircuit> windows is WSAPoll not poll
1912018-08-01T02:21:27 <phantomcircuit> and all the types are insane
1922018-08-01T02:21:40 <phantomcircuit> like it's virtually identical
1932018-08-01T02:21:41 <phantomcircuit> but not
1942018-08-01T02:30:45 <phantomcircuit> but i guess that codes already full of hacks for that anyways
1952018-08-01T02:34:09 *** nodweber has joined #bitcoin-core-dev
1962018-08-01T02:39:06 *** nodweber has quit IRC
1972018-08-01T02:55:06 *** nodweber has joined #bitcoin-core-dev
1982018-08-01T02:58:48 *** michaelsdunn1 has quit IRC
1992018-08-01T03:00:08 *** nodweber has quit IRC
2002018-08-01T03:03:19 *** AaronvanW has quit IRC
2012018-08-01T03:12:48 <gmaxwell> phantomcircuit: windows can keep using select, it doesn't hae the same limit.
2022018-08-01T03:15:03 <sipa> yes, but its fdset implementation is a linked list with horrendous performance
2032018-08-01T03:15:43 <gmaxwell> so? I mean, we only need it to support max-connections. and it already does.
2042018-08-01T03:15:57 *** nodweber has joined #bitcoin-core-dev
2052018-08-01T03:16:01 <gmaxwell> or is it so bad that its noticably slow even for 125 connections?
2062018-08-01T03:16:20 <sipa> probably not
2072018-08-01T03:21:02 *** nodweber has quit IRC
2082018-08-01T03:21:03 <luke-jr> last time I checked, the only better alternative Windows had required significantly re-architecturing most programs to use it
2092018-08-01T03:21:20 <luke-jr> something along the lines of async IO, rather than write-ready notification/checking
2102018-08-01T03:36:52 *** nodweber has joined #bitcoin-core-dev
2112018-08-01T03:41:27 *** nodweber has quit IRC
2122018-08-01T03:57:49 *** nodweber has joined #bitcoin-core-dev
2132018-08-01T04:02:24 *** masonicboom has quit IRC
2142018-08-01T04:02:50 *** nodweber has quit IRC
2152018-08-01T04:15:59 *** ossifrage has quit IRC
2162018-08-01T04:17:36 *** booyah has joined #bitcoin-core-dev
2172018-08-01T04:18:41 *** nodweber has joined #bitcoin-core-dev
2182018-08-01T04:22:42 *** masonicboom has joined #bitcoin-core-dev
2192018-08-01T04:23:34 *** nodweber has quit IRC
2202018-08-01T04:25:30 *** Krellan has quit IRC
2212018-08-01T04:26:02 *** d9b4bef9 has quit IRC
2222018-08-01T04:27:07 *** d9b4bef9 has joined #bitcoin-core-dev
2232018-08-01T04:28:36 *** masonicboom has quit IRC
2242018-08-01T04:31:15 *** masonicboom has joined #bitcoin-core-dev
2252018-08-01T04:36:40 *** masonicboom has quit IRC
2262018-08-01T04:39:07 *** masonicboom has joined #bitcoin-core-dev
2272018-08-01T04:39:30 *** nodweber has joined #bitcoin-core-dev
2282018-08-01T04:44:05 *** nodweber has quit IRC
2292018-08-01T04:45:52 *** masonicboom has quit IRC
2302018-08-01T04:47:56 *** masonicboom has joined #bitcoin-core-dev
2312018-08-01T05:00:25 *** nodweber has joined #bitcoin-core-dev
2322018-08-01T05:01:21 *** grubles_ has joined #bitcoin-core-dev
2332018-08-01T05:02:10 *** grubles has quit IRC
2342018-08-01T05:05:41 *** nodweber has quit IRC
2352018-08-01T05:11:21 *** Amuza has joined #bitcoin-core-dev
2362018-08-01T05:14:34 *** bsm117532 has quit IRC
2372018-08-01T05:19:42 *** Amuza has quit IRC
2382018-08-01T05:21:23 *** nodweber has joined #bitcoin-core-dev
2392018-08-01T05:26:34 *** nodweber has quit IRC
2402018-08-01T05:35:14 *** rex4539 has quit IRC
2412018-08-01T05:40:47 *** Emcy has joined #bitcoin-core-dev
2422018-08-01T05:42:16 *** nodweber has joined #bitcoin-core-dev
2432018-08-01T05:45:32 *** grubles_ has quit IRC
2442018-08-01T05:46:57 *** nodweber has quit IRC
2452018-08-01T06:03:18 *** nodweber has joined #bitcoin-core-dev
2462018-08-01T06:08:10 *** nodweber has quit IRC
2472018-08-01T06:12:15 *** Krellan has joined #bitcoin-core-dev
2482018-08-01T06:23:56 *** murrayn has quit IRC
2492018-08-01T06:24:04 *** nodweber has joined #bitcoin-core-dev
2502018-08-01T06:29:10 *** nodweber has quit IRC
2512018-08-01T06:45:02 *** nodweber has joined #bitcoin-core-dev
2522018-08-01T06:49:37 *** nodweber has quit IRC
2532018-08-01T06:54:46 *** ossifrage has joined #bitcoin-core-dev
2542018-08-01T06:58:34 *** jtimon has quit IRC
2552018-08-01T07:05:50 *** nodweber has joined #bitcoin-core-dev
2562018-08-01T07:07:00 <ossifrage> My computer shat itself, but I found the leveldb mmap limit and bumped it from 1000 to 4096, hopefully that will address my problem
2572018-08-01T07:10:25 *** nodweber has quit IRC
2582018-08-01T07:17:03 *** Krellan has quit IRC
2592018-08-01T07:17:54 *** Krellan has joined #bitcoin-core-dev
2602018-08-01T07:26:45 *** nodweber has joined #bitcoin-core-dev
2612018-08-01T07:31:16 <wumpus> luke-jr: yes let's definitely not do that, last thing we want to maintain is complex specificially for windows rearchitected network code in the repository
2622018-08-01T07:31:29 *** nodweber has quit IRC
2632018-08-01T07:34:00 *** reardencode has quit IRC
2642018-08-01T07:35:22 *** elkalamar has joined #bitcoin-core-dev
2652018-08-01T07:35:27 *** setpill has joined #bitcoin-core-dev
2662018-08-01T07:47:37 *** nodweber has joined #bitcoin-core-dev
2672018-08-01T07:49:57 *** elkalamar has quit IRC
2682018-08-01T07:50:10 *** elkalamar has joined #bitcoin-core-dev
2692018-08-01T07:52:27 *** nodweber has quit IRC
2702018-08-01T08:02:19 *** grubles has joined #bitcoin-core-dev
2712018-08-01T08:04:05 *** Cory has quit IRC
2722018-08-01T08:08:37 *** nodweber has joined #bitcoin-core-dev
2732018-08-01T08:09:42 *** Pasha has joined #bitcoin-core-dev
2742018-08-01T08:12:53 *** Pasha is now known as Cory
2752018-08-01T08:12:57 *** nodweber has quit IRC
2762018-08-01T08:15:49 *** timothy has joined #bitcoin-core-dev
2772018-08-01T08:17:12 *** ula has joined #bitcoin-core-dev
2782018-08-01T08:25:02 *** Krellan has quit IRC
2792018-08-01T08:25:22 *** promag has joined #bitcoin-core-dev
2802018-08-01T08:26:06 <kallewoof> Are there cases where the rev file for a previous blkXXX file are modified? Is this something that happens often? I assume it only happens at reorgs, in which case it should be very seldom except at transition to XXX+1
2812018-08-01T08:26:18 <kallewoof> s/are modified/is modified/
2822018-08-01T08:29:37 *** nodweber has joined #bitcoin-core-dev
2832018-08-01T08:30:43 *** ryankung_ has joined #bitcoin-core-dev
2842018-08-01T08:31:15 *** Victorsueca has quit IRC
2852018-08-01T08:31:38 *** ryankung has joined #bitcoin-core-dev
2862018-08-01T08:31:58 *** ryankung_ has quit IRC
2872018-08-01T08:32:27 *** Victorsueca has joined #bitcoin-core-dev
2882018-08-01T08:33:43 *** ryankung has quit IRC
2892018-08-01T08:34:22 *** nodweber has quit IRC
2902018-08-01T08:41:03 *** Krellan has joined #bitcoin-core-dev
2912018-08-01T08:46:27 *** elkalamar has quit IRC
2922018-08-01T08:50:22 *** nodweber has joined #bitcoin-core-dev
2932018-08-01T08:54:55 <wumpus> kallewoof: no, that never happens
2942018-08-01T08:55:15 <wumpus> kallewoof: only the last blkXXXXX file is written too, other files are only potentially deleted (pruning)
2952018-08-01T08:55:20 *** nodweber has quit IRC
2962018-08-01T08:55:28 <wumpus> in case of a reorg old blocks will not actually be overwritten
2972018-08-01T08:55:35 <kallewoof> wumpus: really? what if a 2-block reorg happens right after a new blk file was created containing a single block?
2982018-08-01T08:55:45 <kallewoof> the rev file was for reorgs, i thought
2992018-08-01T08:56:19 <wumpus> same for rev files, as far as I know, the data for rev-ing the old blocks will stay there
3002018-08-01T08:56:34 <wumpus> it just won't be referenced by the active chain anymore
3012018-08-01T09:00:33 <kallewoof> wumpus: Huh, okay. Well, that's good news for masterdatadir PR then
3022018-08-01T09:00:59 *** SopaXorzTaker has joined #bitcoin-core-dev
3032018-08-01T09:01:04 <wumpus> yes
3042018-08-01T09:05:47 <wumpus> that principle works, I've been using it for a long time with an external script
3052018-08-01T09:06:29 <wumpus> https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/fastcopy-chaindata.py
3062018-08-01T09:11:14 *** nodweber has joined #bitcoin-core-dev
3072018-08-01T09:15:57 *** nodweber has quit IRC
3082018-08-01T09:27:19 *** SopaXT has joined #bitcoin-core-dev
3092018-08-01T09:27:42 *** farmerwampum_ has joined #bitcoin-core-dev
3102018-08-01T09:27:57 *** jimpo_ has joined #bitcoin-core-dev
3112018-08-01T09:28:02 *** achow101 has quit IRC
3122018-08-01T09:28:02 *** farmerwampum has quit IRC
3132018-08-01T09:28:02 *** [b__b] has quit IRC
3142018-08-01T09:28:03 *** jimpo has quit IRC
3152018-08-01T09:28:10 *** farmerwampum_ is now known as farmerwampum
3162018-08-01T09:28:39 *** [b__b] has joined #bitcoin-core-dev
3172018-08-01T09:29:05 *** marcinja has quit IRC
3182018-08-01T09:29:23 <kallewoof> wumpus: Wait, ldb files are readonly too? Right now I am copying the chainstate over (~4 gb)
3192018-08-01T09:29:37 *** SopaXorzTaker has quit IRC
3202018-08-01T09:30:24 *** CubicEarth has quit IRC
3212018-08-01T09:30:27 *** rev_strangehope has quit IRC
3222018-08-01T09:30:58 <kallewoof> wumpus: Though I can't really use the same approach there... wonder if it would be useful to check for hard linking capabilities and using them if found...
3232018-08-01T09:31:08 *** achow101 has joined #bitcoin-core-dev
3242018-08-01T09:32:09 *** nodweber has joined #bitcoin-core-dev
3252018-08-01T09:33:41 *** CubicEarth has joined #bitcoin-core-dev
3262018-08-01T09:34:23 *** rev_strangehope has joined #bitcoin-core-dev
3272018-08-01T09:36:46 *** SopaXT is now known as SopaXorzTaker
3282018-08-01T09:36:57 *** nodweber has quit IRC
3292018-08-01T09:37:33 *** JackH has joined #bitcoin-core-dev
3302018-08-01T09:38:31 *** Guyver2 has joined #bitcoin-core-dev
3312018-08-01T09:41:44 <kallewoof> So, about lint-locale-dependence.sh, which by the way has a list of violations about as long as the linter itself, complains about a bunch of functions because they are locale dependent. But there is no alternative (fix). If you need e.g. std::strtoull() you need to add to the list of violations in the linter. Is this even useful at all, when there are no non-locale dependent alternatives you can switch to?
3322018-08-01T09:43:16 <kallewoof> s/strtoull/stoull/g
3332018-08-01T09:53:04 *** nodweber has joined #bitcoin-core-dev
3342018-08-01T09:53:10 *** JackH has quit IRC
3352018-08-01T09:56:39 *** JackH has joined #bitcoin-core-dev
3362018-08-01T09:57:53 *** nodweber has quit IRC
3372018-08-01T10:13:55 *** nodweber has joined #bitcoin-core-dev
3382018-08-01T10:18:59 *** nodweber has quit IRC
3392018-08-01T10:22:58 *** nodweber has joined #bitcoin-core-dev
3402018-08-01T10:37:27 *** AaronvanW has joined #bitcoin-core-dev
3412018-08-01T10:41:22 *** Aaronvan_ has joined #bitcoin-core-dev
3422018-08-01T10:45:04 *** AaronvanW has quit IRC
3432018-08-01T10:53:22 *** Krellan has quit IRC
3442018-08-01T10:56:24 *** Krellan has joined #bitcoin-core-dev
3452018-08-01T11:02:35 *** Emcy has quit IRC
3462018-08-01T11:05:40 *** Krellan has quit IRC
3472018-08-01T11:11:10 *** Krellan has joined #bitcoin-core-dev
3482018-08-01T11:15:55 *** Krellan has quit IRC
3492018-08-01T11:17:29 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3502018-08-01T11:26:02 *** Krellan has joined #bitcoin-core-dev
3512018-08-01T11:55:26 *** HoMM has joined #bitcoin-core-dev
3522018-08-01T12:00:53 *** promag has quit IRC
3532018-08-01T12:19:41 *** rafalcpp has quit IRC
3542018-08-01T12:20:31 *** rafalcpp has joined #bitcoin-core-dev
3552018-08-01T12:21:56 *** qu4ku has joined #bitcoin-core-dev
3562018-08-01T12:40:00 *** grubles_ has joined #bitcoin-core-dev
3572018-08-01T12:42:31 *** berndj-blackout has joined #bitcoin-core-dev
3582018-08-01T12:42:36 *** gribble has quit IRC
3592018-08-01T12:45:33 *** berndj has quit IRC
3602018-08-01T12:45:33 *** berndj-blackout is now known as berndj
3612018-08-01T12:47:30 *** stevenroose has joined #bitcoin-core-dev
3622018-08-01T12:47:49 *** gribble has joined #bitcoin-core-dev
3632018-08-01T12:48:57 *** grubles has quit IRC
3642018-08-01T12:48:57 *** no_input_found has quit IRC
3652018-08-01T12:48:58 *** ExtraCrispy has quit IRC
3662018-08-01T12:48:58 *** BGL has quit IRC
3672018-08-01T12:48:58 *** kakobrekla has quit IRC
3682018-08-01T12:48:58 *** marcoagner has quit IRC
3692018-08-01T12:48:58 *** z323 has quit IRC
3702018-08-01T12:48:58 *** stevenroose_ has quit IRC
3712018-08-01T12:48:59 *** dgenr8 has quit IRC
3722018-08-01T12:48:59 *** Madars has quit IRC
3732018-08-01T12:48:59 *** sdaftuar has quit IRC
3742018-08-01T12:50:07 *** no_input_found has joined #bitcoin-core-dev
3752018-08-01T12:50:25 *** no_input_found has quit IRC
3762018-08-01T12:50:52 *** no_input_found has joined #bitcoin-core-dev
3772018-08-01T12:51:28 *** infernix has quit IRC
3782018-08-01T12:52:06 *** jtimon has joined #bitcoin-core-dev
3792018-08-01T12:53:25 *** kakobrekla has joined #bitcoin-core-dev
3802018-08-01T12:55:12 *** jtimon has quit IRC
3812018-08-01T12:56:57 *** nodweber has quit IRC
3822018-08-01T12:57:25 *** dgenr8 has joined #bitcoin-core-dev
3832018-08-01T12:57:51 *** z323 has joined #bitcoin-core-dev
3842018-08-01T13:00:25 *** Guyver2 has quit IRC
3852018-08-01T13:10:29 *** promag has joined #bitcoin-core-dev
3862018-08-01T13:16:13 *** infernix has joined #bitcoin-core-dev
3872018-08-01T13:27:13 *** qu4ku has quit IRC
3882018-08-01T13:29:12 *** Krellan has quit IRC
3892018-08-01T13:31:40 *** Krellan has joined #bitcoin-core-dev
3902018-08-01T13:36:32 *** Victorsueca has quit IRC
3912018-08-01T13:37:41 *** Victorsueca has joined #bitcoin-core-dev
3922018-08-01T13:39:46 *** promag has quit IRC
3932018-08-01T13:43:52 *** aLK-[i] has joined #bitcoin-core-dev
3942018-08-01T14:07:16 *** qu4ku has joined #bitcoin-core-dev
3952018-08-01T14:11:38 *** Krellan has quit IRC
3962018-08-01T14:16:53 *** Krellan has joined #bitcoin-core-dev
3972018-08-01T14:31:42 *** Krellan has quit IRC
3982018-08-01T14:33:38 *** Aaronvan_ is now known as AaronvanW
3992018-08-01T14:36:51 *** Krellan has joined #bitcoin-core-dev
4002018-08-01T14:39:54 *** qu4ku has quit IRC
4012018-08-01T14:51:14 *** michaelsdunn1 has joined #bitcoin-core-dev
4022018-08-01T14:51:32 *** Chris_Stewart_5 has quit IRC
4032018-08-01T15:05:45 *** Chris_Stewart_5 has joined #bitcoin-core-dev
4042018-08-01T15:11:50 *** Krellan has quit IRC
4052018-08-01T15:12:25 *** Krellan has joined #bitcoin-core-dev
4062018-08-01T15:21:44 *** Krellan has quit IRC
4072018-08-01T15:27:00 *** Krellan has joined #bitcoin-core-dev
4082018-08-01T15:28:34 *** booyah has quit IRC
4092018-08-01T15:29:14 *** nodweber has joined #bitcoin-core-dev
4102018-08-01T15:30:14 *** marcinja_ has joined #bitcoin-core-dev
4112018-08-01T15:32:22 *** masonicboom has quit IRC
4122018-08-01T15:33:03 *** JackH has quit IRC
4132018-08-01T15:39:58 *** nodweber has quit IRC
4142018-08-01T15:43:05 *** nodweber has joined #bitcoin-core-dev
4152018-08-01T15:43:22 *** booyah has joined #bitcoin-core-dev
4162018-08-01T16:01:10 *** Emcy has joined #bitcoin-core-dev
4172018-08-01T16:09:32 *** JackH has joined #bitcoin-core-dev
4182018-08-01T16:13:20 *** grubles_ is now known as grubles
4192018-08-01T16:16:48 *** JackH has quit IRC
4202018-08-01T16:22:40 <skeees> BlueMatt: sorry missed that one, updated in https://github.com/bitcoin/bitcoin/pull/13835
4212018-08-01T16:37:36 <provoostenator> Potentially trivial to review RPC doc improvements: #13676, #13662
4222018-08-01T16:37:38 <gribble> https://github.com/bitcoin/bitcoin/issues/13676 | Explain that mempool memory is added to -dbcache during IBD by Sjors · Pull Request #13676 · bitcoin/bitcoin · GitHubAsset 1Asset 1
4232018-08-01T16:37:39 <gribble> https://github.com/bitcoin/bitcoin/issues/13662 | Explain when reindex-chainstate can be used instead of reindex by Sjors · Pull Request #13662 · bitcoin/bitcoin · GitHubAsset 1Asset 1
4242018-08-01T16:39:37 *** intcat has quit IRC
4252018-08-01T16:40:42 *** masonicboom has joined #bitcoin-core-dev
4262018-08-01T16:41:33 *** intcat has joined #bitcoin-core-dev
4272018-08-01T16:43:47 *** intcat has quit IRC
4282018-08-01T16:45:38 *** intcat has joined #bitcoin-core-dev
4292018-08-01T16:45:56 *** booyah has quit IRC
4302018-08-01T16:50:31 *** booyah has joined #bitcoin-core-dev
4312018-08-01T16:56:59 *** instagibbs has quit IRC
4322018-08-01T17:00:08 *** masonicboom has quit IRC
4332018-08-01T17:19:28 *** Orion3k has quit IRC
4342018-08-01T17:25:51 *** setpill has quit IRC
4352018-08-01T17:28:16 *** promag has joined #bitcoin-core-dev
4362018-08-01T17:31:32 *** Krellan has quit IRC
4372018-08-01T17:38:00 *** Krellan has joined #bitcoin-core-dev
4382018-08-01T17:52:53 *** nodweber has quit IRC
4392018-08-01T17:58:49 *** atroxes has quit IRC
4402018-08-01T17:59:21 *** timothy has quit IRC
4412018-08-01T17:59:57 *** nmnkgl has quit IRC
4422018-08-01T18:00:03 *** atroxes has joined #bitcoin-core-dev
4432018-08-01T18:01:04 *** luke-jr has quit IRC
4442018-08-01T18:02:41 *** Chris_Stewart_5 has quit IRC
4452018-08-01T18:05:24 *** luke-jr has joined #bitcoin-core-dev
4462018-08-01T18:09:52 <sipa> do we want to make the bot join in order to message here?
4472018-08-01T18:10:05 <sipa> join/leave spam would be somewhat annoying, but not as bad as spam
4482018-08-01T18:12:33 *** Krellan has quit IRC
4492018-08-01T18:13:08 *** Krellan has joined #bitcoin-core-dev
4502018-08-01T18:18:05 *** nmnkgl has joined #bitcoin-core-dev
4512018-08-01T18:20:17 <achow101> most clients can hide join and leave messages, so I think that's fine
4522018-08-01T18:21:04 *** promag has quit IRC
4532018-08-01T18:22:15 <midnightmagic> Does the bot have a freenode account?
4542018-08-01T18:22:38 *** Krellan has quit IRC
4552018-08-01T18:22:39 <midnightmagic> If so, then +q $~a allows people to still join and watch, and worst case we get join/parts from the spammer bots.
4562018-08-01T18:23:10 *** Krellan has joined #bitcoin-core-dev
4572018-08-01T18:42:43 *** ula has quit IRC
4582018-08-01T18:51:51 *** HoMM has quit IRC
4592018-08-01T19:02:47 <booyah> sipa: bot must join/part becasue that is how github works?
4602018-08-01T19:03:19 <booyah> possible solution: create #botx channel, have bot join say and part there. Setup a message relaying bot (tiny python script) to relay msgs from there to here, and the relay bot will be always joined
4612018-08-01T19:04:08 <sipa> booyah: we added +n to this channel (which requires joining in order to speak) to combat spam
4622018-08-01T19:12:24 <midnightmagic> the second bot would be present here as well and just speak it, I think is what he means.
4632018-08-01T19:12:42 <sipa> yeah, i understand the suggestion - i don't have much of an opinion on it :)
4642018-08-01T19:13:09 <sipa> i was just explaining it's not because how github works but because we have +n on
4652018-08-01T19:13:16 <midnightmagic> ah
4662018-08-01T19:13:51 <sipa> oh i see; i guess booyah understood that, but by "because that is how github works" booyah means as opposed to have it be continuously present
4672018-08-01T19:13:54 <sipa> right
4682018-08-01T19:14:59 <booyah> (yeah just afair github bot anyway was always joining and parting)
4692018-08-01T19:15:27 *** SopaXorzTaker has quit IRC
4702018-08-01T19:15:34 <midnightmagic> \o
4712018-08-01T19:17:02 <sipa> actually, #bitcoin-commits already exists
4722018-08-01T19:17:13 <sipa> we could have a bot mirror from there
4732018-08-01T19:22:22 <booyah> sipa: https://github.com/str4d/RelayBot
4742018-08-01T19:24:08 <booyah> I hope it works between 2 chans on same server
4752018-08-01T19:24:35 <sipa> i've just turned on join/leave
4762018-08-01T20:07:24 *** booyah has quit IRC
4772018-08-01T20:20:46 *** promag has joined #bitcoin-core-dev
4782018-08-01T20:25:53 *** promag has quit IRC
4792018-08-01T20:43:25 *** Victorsueca has quit IRC
4802018-08-01T20:44:43 *** Victorsueca has joined #bitcoin-core-dev
4812018-08-01T21:06:12 *** no_input_found has quit IRC
4822018-08-01T21:06:31 *** no_input_found has joined #bitcoin-core-dev
4832018-08-01T21:39:19 *** Victorsueca has quit IRC
4842018-08-01T21:40:43 *** Victorsueca has joined #bitcoin-core-dev
4852018-08-01T21:58:56 *** booyah has joined #bitcoin-core-dev
4862018-08-01T22:09:01 *** d9b4bef9 has quit IRC
4872018-08-01T22:10:17 *** d9b4bef9 has joined #bitcoin-core-dev
4882018-08-01T22:13:39 *** promag has joined #bitcoin-core-dev
4892018-08-01T22:17:47 *** Dizzle has joined #bitcoin-core-dev
4902018-08-01T22:18:09 *** promag has quit IRC
4912018-08-01T22:24:58 *** michaelsdunn1 has quit IRC
4922018-08-01T22:25:32 *** michaelsdunn1 has joined #bitcoin-core-dev
4932018-08-01T22:30:19 *** michaelsdunn1 has quit IRC
4942018-08-01T22:36:26 *** AaronvanW has quit IRC
4952018-08-01T22:37:03 *** AaronvanW has joined #bitcoin-core-dev
4962018-08-01T22:37:47 *** Krellan has quit IRC
4972018-08-01T22:38:22 *** Krellan has joined #bitcoin-core-dev
4982018-08-01T22:44:42 *** vicenteH has quit IRC
4992018-08-01T22:45:13 *** vicenteH has joined #bitcoin-core-dev
5002018-08-01T23:05:56 *** n00bington has joined #bitcoin-core-dev
5012018-08-01T23:06:42 <n00bington> so I'm looking at this page on the wiki
5022018-08-01T23:06:49 <n00bington> https://en.bitcoin.it/wiki/Secp256k1
5032018-08-01T23:07:00 <n00bington> where in the source code are those parameters being implemented?
5042018-08-01T23:07:14 <n00bington> doing some security research for my compsec class
5052018-08-01T23:07:20 <achow101> n00bington: somewhere in src/secp256k1
5062018-08-01T23:07:27 <n00bington> achow101, thanks
5072018-08-01T23:07:49 <achow101> there's a library that implements all of that stuff: https://github.com/bitcoin-core/secp256k1
5082018-08-01T23:07:58 <achow101> that lib is put in src/secp256k1
5092018-08-01T23:08:14 <n00bington> cool lemme see if i can find it
5102018-08-01T23:08:16 <n00bington> thanks
5112018-08-01T23:11:16 <sipa> n00bington: it's spread out
5122018-08-01T23:11:32 <n00bington> sipa, what do you mean?
5132018-08-01T23:11:34 <sipa> the library implementing the elliptic curve things is in bhttps://github.com/bitcoin-core/secp256k1
5142018-08-01T23:11:41 <n00bington> right
5152018-08-01T23:11:48 <sipa> there's also a separate IRC channel about it, #secp256k1
5162018-08-01T23:11:52 <n00bington> oh
5172018-08-01T23:11:53 <n00bington> thanks
5182018-08-01T23:13:19 *** Dizzle has quit IRC
5192018-08-01T23:15:00 *** Cory has quit IRC
5202018-08-01T23:20:53 *** n00bington has quit IRC
5212018-08-01T23:26:04 *** Cory has joined #bitcoin-core-dev
5222018-08-01T23:45:57 *** Cory has quit IRC
5232018-08-01T23:46:09 *** e4xit has quit IRC
5242018-08-01T23:46:57 *** masonicboom has joined #bitcoin-core-dev
5252018-08-01T23:51:08 *** masonicboom has quit IRC
5262018-08-01T23:52:19 *** Cory has joined #bitcoin-core-dev
5272018-08-01T23:53:56 *** AaronvanW has quit IRC
5282018-08-01T23:54:33 *** AaronvanW has joined #bitcoin-core-dev
5292018-08-01T23:58:35 *** AaronvanW has quit IRC