12017-12-05T00:00:18 <BlueMatt> achow101: give 11824 a spin?
22017-12-05T00:00:22 <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #11824: Block ActivateBestChain to empty validationinterface queue (master...2017-12-11822-debug) https://github.com/bitcoin/bitcoin/pull/11824
32017-12-05T00:00:22 <BlueMatt> seems to be working for me
42017-12-05T00:00:47 <achow101> pulling now
52017-12-05T00:01:17 <achow101> why's it so big?
62017-12-05T00:02:23 *** dpg has joined #bitcoin-core-dev
72017-12-05T00:02:41 *** jb55 has quit IRC
82017-12-05T00:04:37 *** intcat has quit IRC
92017-12-05T00:05:03 <BlueMatt> cause you cant call the callbacks themselves with cs_main held...I mean the first commit is just a nice-to-have that makes the queue drain faster, the third commit adds DEBUG_LOCKORDER annotations and removes spurious locks in tests, and the last two should probably be squashed
102017-12-05T00:05:04 <achow101> BlueMatt: build error ---> validation.cpp:2480:18: error: âpromiseâ is not a member of âstdâ
112017-12-05T00:05:06 <BlueMatt> its not as bad as it looks
122017-12-05T00:05:37 <BlueMatt> achow101: oops, try now, forgot to commit that line
132017-12-05T00:05:52 *** intcat has joined #bitcoin-core-dev
142017-12-05T00:07:36 <achow101> what's the queue depth set to?
152017-12-05T00:08:10 <BlueMatt> max is 10
162017-12-05T00:08:14 <BlueMatt> feel free to play with it
172017-12-05T00:09:34 <achow101> well it compiled and it's running
182017-12-05T00:10:00 <sipa> ship it.
192017-12-05T00:10:57 <BlueMatt> someone should tag 11822 as 0.16
202017-12-05T00:11:16 <achow101> I'll let you know how it goes, so far so good.
212017-12-05T00:11:47 <BlueMatt> I mean it definitely fixes the obvious infite-queue issue, but there could be other gremlins hiding
222017-12-05T00:11:50 <BlueMatt> I doubt it
232017-12-05T00:12:07 <BlueMatt> my bigger worry with that patch is future lockorder violations, but not a ton to be done about it, honestly
242017-12-05T00:12:11 <sipa> tagged
252017-12-05T00:15:42 <cfields> BlueMatt: i think it'd make for a smoother transition if a fence was inserted after each block's callbacks had been queued, rather than stalling at an arbitrary depth
262017-12-05T00:16:04 <cfields> that way we could still work with the same assumptions as before, while we work out any remaining out-of-sync issues
272017-12-05T00:16:30 <BlueMatt> cfields: ugh, we started the fence-removal process two releases ago, I'd rather not add it back if we can avoid it :(
282017-12-05T00:16:44 *** intcat has quit IRC
292017-12-05T00:18:19 <cfields> BlueMatt: the PR is the same thing, I'm just suggesting using a block-callback-depth (depth == 1 initially) rather than an individual callback depth.
302017-12-05T00:18:27 <BlueMatt> hmm, that patch may not be sufficient :/ achow101 you see it staying low?
312017-12-05T00:18:57 <achow101> BlueMatt: it's reasonable given my current dbcache of 8000
322017-12-05T00:19:19 *** intcat has joined #bitcoin-core-dev
332017-12-05T00:19:56 <BlueMatt> cfields: well if you want to do that we should move the blocking to the end and not the beginning and set it to 1 right before release/on the backport branch, not on master, please
342017-12-05T00:20:08 <BlueMatt> cfields: 90% of the reason to merge this stuff sooner rather than later is to get testing cycles on it
352017-12-05T00:20:19 <BlueMatt> otherwise its never gonna happen
362017-12-05T00:21:41 <achow101> BlueMatt: I'd like this to be merged so I can run my fucking node :p
372017-12-05T00:22:08 <BlueMatt> achow101: lol, you dont carry patches on your node?
382017-12-05T00:23:25 *** jb55 has joined #bitcoin-core-dev
392017-12-05T00:23:28 <achow101> BlueMatt: my node is also what I test and develop on and I would like to be able to test the other stuff I'm working on and PR it without extra patches
402017-12-05T00:23:30 *** wxss has quit IRC
412017-12-05T00:24:11 *** microapple has quit IRC
422017-12-05T00:24:33 <BlueMatt> heh, well feel free to review
432017-12-05T00:26:32 <cfields> BlueMatt: to be clear, i was suggesting that we let it run a full block ahead (depth of single block's callbacks). I wasn't suggesting that we use an actual depth of 1.
442017-12-05T00:26:36 <BlueMatt> well res size seems to have stabilized on my reindex-chainstate node
452017-12-05T00:26:55 <BlueMatt> cfields: I dont know how much that'd do aside from hide bugs?
462017-12-05T00:27:20 <BlueMatt> if we want to neuter it cause we're worried, we should make ProcessNewBlock/ActivateBestChain block until callbacks finish for the latest action
472017-12-05T00:27:38 <BlueMatt> letting it run ahead but heavily limiting how much it can run ahead just sounds like it'll hide more bugs than anything, no?
482017-12-05T00:29:40 <cfields> well yes, but it'd ensure that we're only introducing one class of bugs at a time
492017-12-05T00:33:16 <achow101> for each extra item in the queue, how much extra memory does that use
502017-12-05T00:34:27 *** microapple has joined #bitcoin-core-dev
512017-12-05T00:35:27 <sipa> a few MB, i expect
522017-12-05T00:36:54 *** jb55 has quit IRC
532017-12-05T00:36:57 *** ensign has quit IRC
542017-12-05T00:37:29 <BlueMatt> achow101: (or not reindex)
552017-12-05T00:37:52 <BlueMatt> achow101: its (during IBD, mostly just) a shared_ptr<const CBlock>
562017-12-05T00:38:05 <achow101> BlueMatt: I'm not sure if the problem I saw was limited to reindex only though
572017-12-05T00:38:06 <BlueMatt> cfields: I'm not sure which class of bugs is resolved by your suggestion, tbh?
582017-12-05T00:38:18 <BlueMatt> achow101: it should almost entirely be
592017-12-05T00:38:34 <BlueMatt> your wallet shouldnt take 10 minutes to process a new block....
602017-12-05T00:38:44 <achow101> BlueMatt: the node crashed at some point before I had to reindex, and I think that was because of an OOM to (had to do the whole reboot computer stuff as I did when it crashed during reindex)
612017-12-05T00:38:48 <BlueMatt> cfields: blocking before return may fix some stuff, but....
622017-12-05T00:39:03 <achow101> that crash caused me to need to reindex it now
632017-12-05T00:39:13 <BlueMatt> achow101: thats.....weird
642017-12-05T00:39:58 <achow101> it's possible that that was unrelated, but I'm not sure
652017-12-05T00:39:59 *** ensign has joined #bitcoin-core-dev
662017-12-05T00:41:11 <BlueMatt> was that like right after a restart where you needed to catch up a few days blocks?
672017-12-05T00:41:12 <BlueMatt> though even that shouldnt use all that much memory....
682017-12-05T00:41:25 *** dcousens has joined #bitcoin-core-dev
692017-12-05T00:41:31 <BlueMatt> i mean this has been on master a while...
702017-12-05T00:42:01 <achow101> no, it had been up for a few days at that point, and I had it running for several weeks with a build of master prior to that
712017-12-05T00:42:19 <achow101> although it is possible that the build I was running before did not have your PR merged
722017-12-05T00:44:48 <achow101> I can grep through my log file to see what was running before I started on this build. although it may take a while since that log file is 42 GB because I forgot to turn of debug=net
732017-12-05T00:45:10 <BlueMatt> hmm, I mean I did a read-through of the queue logic and didnt see any obvious races
742017-12-05T00:45:42 <BlueMatt> ls -lh ~/.bitcoin/debug.log
752017-12-05T00:45:42 <BlueMatt> -rw------- 1 matt matt 6.5T Dec 4 19:45 /home/matt/.bitcoin/debug.log
762017-12-05T00:45:59 <BlueMatt> errr, s/races/leaks/
772017-12-05T00:46:28 <achow101> BlueMatt: do you have your node patched to not shrink the debug.log at each restart?
782017-12-05T00:46:32 *** DvdKhl has quit IRC
792017-12-05T00:46:44 <BlueMatt> i believe it does not shrink by default if -debug? or something like that
802017-12-05T00:46:53 *** microapple has quit IRC
812017-12-05T00:47:26 * BlueMatt -> out, if bug is still there, do a heap profile
822017-12-05T00:47:45 *** microapple has joined #bitcoin-core-dev
832017-12-05T00:52:17 *** microapple has quit IRC
842017-12-05T00:53:09 *** jb55 has joined #bitcoin-core-dev
852017-12-05T00:55:45 *** dabura667 has joined #bitcoin-core-dev
862017-12-05T00:59:13 *** jb55 has quit IRC
872017-12-05T01:03:09 <jamesob> going to test #11824 with -reindex
882017-12-05T01:03:10 <gribble> https://github.com/bitcoin/bitcoin/issues/11824 | Block ActivateBestChain to empty validationinterface queue by TheBlueMatt · Pull Request #11824 · bitcoin/bitcoin · GitHub
892017-12-05T01:05:46 *** Aaronvan_ has joined #bitcoin-core-dev
902017-12-05T01:06:02 *** Aaronvan_ has quit IRC
912017-12-05T01:09:21 *** AaronvanW has quit IRC
922017-12-05T01:18:35 *** Ylbam has quit IRC
932017-12-05T01:19:51 *** Deacyde has joined #bitcoin-core-dev
942017-12-05T01:22:39 *** intcat has quit IRC
952017-12-05T01:23:32 *** intcat has joined #bitcoin-core-dev
962017-12-05T01:24:58 *** Randolf has joined #bitcoin-core-dev
972017-12-05T01:40:54 *** microapple has joined #bitcoin-core-dev
982017-12-05T01:56:41 *** cdecker has quit IRC
992017-12-05T01:57:07 *** cdecker has joined #bitcoin-core-dev
1002017-12-05T01:57:20 *** lifeofguenter has quit IRC
1012017-12-05T01:57:33 *** arubi has quit IRC
1022017-12-05T01:59:21 *** lifeofguenter_ has joined #bitcoin-core-dev
1032017-12-05T01:59:51 *** arubi has joined #bitcoin-core-dev
1042017-12-05T02:01:29 *** microapple has quit IRC
1052017-12-05T02:03:34 *** rhavar has quit IRC
1062017-12-05T02:09:41 *** intcat has quit IRC
1072017-12-05T02:10:34 *** intcat has joined #bitcoin-core-dev
1082017-12-05T02:14:29 *** Murch has quit IRC
1092017-12-05T02:14:56 *** intcat has quit IRC
1102017-12-05T02:15:02 <promag> please give feedback to #11826, ty
1112017-12-05T02:15:04 <gribble> https://github.com/bitcoin/bitcoin/issues/11826 | RFC: Activity feature · Issue #11826 · bitcoin/bitcoin · GitHub
1122017-12-05T02:15:45 *** intcat has joined #bitcoin-core-dev
1132017-12-05T02:19:50 *** microapple has joined #bitcoin-core-dev
1142017-12-05T02:21:59 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1152017-12-05T02:26:35 *** microapple has quit IRC
1162017-12-05T02:31:20 *** Randolf has quit IRC
1172017-12-05T02:32:35 *** shesek has quit IRC
1182017-12-05T02:42:01 *** Emcy has joined #bitcoin-core-dev
1192017-12-05T02:44:17 *** Emcy_ has quit IRC
1202017-12-05T02:47:29 *** jamesob has quit IRC
1212017-12-05T02:48:42 *** shesek has joined #bitcoin-core-dev
1222017-12-05T02:49:18 *** Emcy has quit IRC
1232017-12-05T02:52:25 *** promag has quit IRC
1242017-12-05T03:18:50 *** intcat has quit IRC
1252017-12-05T03:20:58 *** intcat has joined #bitcoin-core-dev
1262017-12-05T03:23:12 *** Emcy has joined #bitcoin-core-dev
1272017-12-05T03:23:30 *** Chris_Stewart_5 has quit IRC
1282017-12-05T03:26:58 *** jb55 has joined #bitcoin-core-dev
1292017-12-05T03:29:15 *** microapple has joined #bitcoin-core-dev
1302017-12-05T03:53:05 *** promag has joined #bitcoin-core-dev
1312017-12-05T04:02:26 *** jtimon has quit IRC
1322017-12-05T04:08:21 *** dgenr8 has quit IRC
1332017-12-05T04:09:06 *** dgenr8 has joined #bitcoin-core-dev
1342017-12-05T04:10:44 *** intcat has quit IRC
1352017-12-05T04:11:38 *** intcat has joined #bitcoin-core-dev
1362017-12-05T04:27:09 *** Victorsueca has quit IRC
1372017-12-05T04:28:24 *** Victorsueca has joined #bitcoin-core-dev
1382017-12-05T04:38:41 *** microapple has quit IRC
1392017-12-05T04:44:15 <bitcoin-git> [bitcoin] kallewoof closed pull request #11489: [wallet] sendtoaddress style argument (master...201709_segwitwallet2_sendtoaddress) https://github.com/bitcoin/bitcoin/pull/11489
1402017-12-05T04:44:15 *** Randolf has joined #bitcoin-core-dev
1412017-12-05T04:46:20 *** brianhoffman has quit IRC
1422017-12-05T04:46:39 *** brianhoffman has joined #bitcoin-core-dev
1432017-12-05T04:48:01 *** bule has joined #bitcoin-core-dev
1442017-12-05T04:48:56 *** bule2 has quit IRC
1452017-12-05T04:59:54 *** roidster has joined #bitcoin-core-dev
1462017-12-05T05:00:16 *** roidster is now known as Guest70224
1472017-12-05T05:00:42 *** Guest70224 is now known as roidster
1482017-12-05T05:07:40 *** intcat has quit IRC
1492017-12-05T05:08:35 *** intcat has joined #bitcoin-core-dev
1502017-12-05T05:11:34 *** musalbas has joined #bitcoin-core-dev
1512017-12-05T05:16:24 *** bule has quit IRC
1522017-12-05T05:16:48 *** bule has joined #bitcoin-core-dev
1532017-12-05T05:22:49 <kallewoof> I've been asked about address index on top of bitcoin core (to replace the unmaintained fork that exists somewhere else). What are people's opinions on this? I know some people prefer a minimal code base, but having an address index option seems like it would alleviate a lot of problems in a lot of places.
1542017-12-05T05:27:15 <gmaxwell> a history address index is unsustainable; anyone who uses it is eventually going to be forced to use trust a third party service.
1552017-12-05T05:27:45 <kallewoof> Why?
1562017-12-05T05:28:00 <kallewoof> Too much space, or..?
1572017-12-05T05:28:05 <gmaxwell> I don't see any problems with having hooks to support such things, but directly incorporating an index of history wouldn't just be a waste of our review and maintaince resources, it would be actively harmful to the ecosystem.
1582017-12-05T05:28:28 <gmaxwell> Yes too much space, too much seek load, and incompatible with pruning; or any kind of pruned sync.
1592017-12-05T05:29:02 * kallewoof nods
1602017-12-05T05:29:17 <gmaxwell> we've also already seen that behavior where people using such indexes constructed via abe or other tools that created them, failed back to blockchain.info and similar.
1612017-12-05T05:29:57 <gmaxwell> They're also not required except for unusual cases (like making an explorer, which also requires spent-position indexes). An index of unspent outputs would be far more reasonable.
1622017-12-05T05:30:43 <kallewoof> Yeah, this would be used by explorers, I would expect. I think some (lots?) of them use the unmaintained fork I mentioned earlier.
1632017-12-05T05:30:59 <gmaxwell> I sure hope not.
1642017-12-05T05:31:16 <kallewoof> Rumor says it is so. Part of why I was asked to look into it.
1652017-12-05T05:31:23 <gmaxwell> I know with absolute certanty that that old index patch could lose data, I just don't know _how_.
1662017-12-05T05:31:41 *** intcat has quit IRC
1672017-12-05T05:32:04 <kallewoof> I think having hooks in bitcoin core that let such a system exist separately is a good plan. It would only rely on API breakage, which bitcoin core is very careful about.
1682017-12-05T05:32:28 *** intcat has joined #bitcoin-core-dev
1692017-12-05T05:32:32 <gmaxwell> As in I had a node with it where it was faliing to actual spendable outputs connected to an address, and I tested that it was incorrect quite throughly. I assumed the bad data was some gotcha with reorgs or likewise, but no one investigated.
1702017-12-05T05:32:41 <gmaxwell> Yes, hooks for such things seem fine to me.
1712017-12-05T05:32:59 <kallewoof> Ah.. yeah, reorgs is definitely a challenge.
1722017-12-05T05:35:34 <sipa> if i had infinite time i'd write a separate efficient indexing service, which just speaks the p2p protocol
1732017-12-05T05:36:18 <sipa> which you then can connect to bitcoind
1742017-12-05T05:38:00 <kallewoof> sipa: I guess that would have the obvious benefit of not actually needing a node at all if it could just sync from the network?
1752017-12-05T05:38:38 <sipa> right
1762017-12-05T05:39:25 <sipa> it would be nice to reuse some of the headers sync logic in core, but if you assume you're connected to a trusted peer anyway, i don't think it needs to be allthat complicated
1772017-12-05T05:39:29 <sipa> just deal with reorgs
1782017-12-05T05:40:09 *** xRavenheart has joined #bitcoin-core-dev
1792017-12-05T05:40:53 <gmaxwell> from an arch perspective you'd even want it to just keep a SQL database in sync, unfortunately the performance would be horrible except on gold plated system. (massive nvem raid 1 arrays and gobs of ram or whatever)
1802017-12-05T05:42:06 <kallewoof> SQL db sounds like it would be kinda slow yeah.
1812017-12-05T05:42:39 <gmaxwell> thats how abe worked.
1822017-12-05T05:42:58 <kallewoof> I was thinking of using redis, but I think I'm gonna have to do profiling to find a good solution.
1832017-12-05T05:43:07 <gmaxwell> last I heard of anyone successfully running it, it required well over 1TB state and would take a month to sync or something.
1842017-12-05T05:43:18 <kallewoof> Wow..
1852017-12-05T05:43:54 <sipa> i think it's been unusably slow for the better part of 5 years now
1862017-12-05T05:44:10 <gmaxwell> this was years ago, it would presumably be some multiple of that now.
1872017-12-05T05:49:19 <kallewoof> It would be address -> tx list, tx -> address list, block -> tx list.. tx'es could be compressed as block height + index (so ~8b). Not sure what else neat stuff you could do. sipa's work on tx format should be inspiring.
1882017-12-05T05:50:12 <esotericnonsense> blockheight + index doesn't work well at all, what about reorgs
1892017-12-05T05:51:29 <kallewoof> Maybe keep track of last 1000 entries and undo all based on shared fork point height?
1902017-12-05T05:51:30 <gmaxwell> works fine, so long as you're only tracking a single chain at a time.
1912017-12-05T05:51:36 <gmaxwell> you have to back out the index and reapply.
1922017-12-05T05:51:48 <kallewoof> Could just look at block again and undo, yeah.
1932017-12-05T05:53:10 <esotericnonsense> eh I suppose that's right. SQL is a bit abstracted for me i'm not sure how it stores repeated data internally. f.ex. if you have a ton of tables with foreign keys do they store the keys repeatedly or use a more space efficient method.
1942017-12-05T05:55:45 <esotericnonsense> for example on chain you might have a 256bit txid used hundreds of times if it has a bunch of outputs but you can probably just store that as a 32bit(?) incrementing integer on every tx you've seen since genesis.
1952017-12-05T05:55:47 <kallewoof> I really doubt SQL would work well for this, but I could be wrong.
1962017-12-05T05:56:45 <Randolf> esotericnonsense: It depends on the SQL server software being used. As far as I know, whatever you choose to store in the columns is what's stored there.
1972017-12-05T05:57:18 <Randolf> esotericnonsense: Usually it's up to the DBA to choose to set up smaller IDs that reference more data in a separate row.
1982017-12-05T05:58:40 <kallewoof> We've used up 6.5% of a uint32 (278236762/4294967295), but it could work for awhile at least.
1992017-12-05T05:58:51 <Randolf> esotericnonsense: For Bitcoin stuff, I'd create custom datatypes that save the data in binary format.
2002017-12-05T05:59:21 <esotericnonsense> basically what i'm thinking is that these indexes can easily multiply the blockchain size by five and possibly more depending on how you do it if you store the full data.
2012017-12-05T05:59:48 <Randolf> It's possible.
2022017-12-05T06:00:29 <esotericnonsense> address -> txlist, block -> txlist, and tx (unless you don't store tx but pull it out of a stored blockchain) is 3x txid
2032017-12-05T06:00:40 <Randolf> An index normally refers to an internal reference that's usually smaller than the column data it's indexing.
2042017-12-05T06:01:23 <Randolf> Can arrays help? In PostgreSQL a column type can be an array.
2052017-12-05T06:01:31 <Randolf> These can also be indexed efficiently.
2062017-12-05T06:02:35 <Randolf> While you'd still be storing TX data, you could cut down on the number of rows.
2072017-12-05T06:08:05 <gmaxwell> this sort of speculation isn't so useful.
2082017-12-05T06:09:06 <gmaxwell> IIRC the abe schema wasn't too offensively inefficienct, but there are overheads and billions of records add up... but nothing is really learned from this discussion. just "I think this would be not efficienct" and "I think it would be, maybe" :P
2092017-12-05T06:09:30 <esotericnonsense> yeah agree.
2102017-12-05T06:09:38 <kallewoof> I'll just try a bunch of stuff I guess.
2112017-12-05T06:10:51 <esotericnonsense> it sounds to me like what might be useful is to have a seperate service that only maintains the set of things core doesn't
2122017-12-05T06:11:22 <esotericnonsense> so basically addrindex but as an external thing. all it needs is addr->[txids]. if you want the transactions just ask core rpc with txindex on
2132017-12-05T06:12:23 <esotericnonsense> that'd only need the core rpc api to remain (relatively) static and not have to be maintained as a thing on an ongoing basis if internals shuffle around.
2142017-12-05T06:12:43 <kallewoof> If txs are stored as height+index you don't even need txindex, at least if #10275 is merged.
2152017-12-05T06:12:47 <gribble> https://github.com/bitcoin/bitcoin/issues/10275 | [rpc] Allow fetching tx directly from specified block in getrawtransaction by kallewoof · Pull Request #10275 · bitcoin/bitcoin · GitHub
2162017-12-05T06:13:36 <kallewoof> And yeah, we agreed this should be an external thing that either uses hooks in bitcoin or implements the p2p protocol and talks to the network directly.
2172017-12-05T06:15:22 <esotericnonsense> i guess i was commenting on "address -> tx list, tx -> address list, block -> tx list" and how i don't think you need the latter two because bitcoind does it
2182017-12-05T06:15:45 * kallewoof nods
2192017-12-05T06:16:09 <kallewoof> I wasn't assuming the person running had a local synced node. Maybe I should.
2202017-12-05T06:17:00 <esotericnonsense> eh? addrindex implies that no? (well that or that you have rpc access anyway).
2212017-12-05T06:18:07 <kallewoof> sipa had the idea of an efficient indexing service which speaks the p2p protocol.
2222017-12-05T06:19:12 <gmaxwell> that assumption saves you basically 1x copy of the blockchain at most, but then means you can't run the indexer on the seperate hardware from the node or fail over between multiple nodes, etc.
2232017-12-05T06:19:28 <gmaxwell> or upgrade the node independantly from the indexer
2242017-12-05T06:21:46 *** roidster has quit IRC
2252017-12-05T06:22:24 <kallewoof> Makes sense
2262017-12-05T06:22:35 <kallewoof> (to not assume local node)
2272017-12-05T06:23:36 <esotericnonsense> i'm confused. you can run the indexer on seperate hardware just fine. failover works in the same way anyway (you can have multiple bitcoinds if you want)
2282017-12-05T06:24:48 <esotericnonsense> height+index storing of transactions becomes complicated in that case sure.
2292017-12-05T06:25:19 *** intcat has quit IRC
2302017-12-05T06:25:40 <kallewoof> Ultimately it doesn't matter to the indexer if you have a local node or not. It won't use it directly, it will use the p2p network. The only time it needs the RPC is when it wants to convert e.g. block height+index into a tx, but it might just as well give you the height+index pairs and let you deal yourself.
2312017-12-05T06:26:12 *** intcat has joined #bitcoin-core-dev
2322017-12-05T06:26:57 *** intcat has quit IRC
2332017-12-05T06:29:15 <gmaxwell> also if the node the indexer is behind is pruned then the overhead of a redundant copy mostly goes away.
2342017-12-05T06:30:34 *** intcat has joined #bitcoin-core-dev
2352017-12-05T06:33:52 *** indistylo has joined #bitcoin-core-dev
2362017-12-05T06:38:16 <kallewoof> If the node is pruned, the indexer will not be able to use it for any lookups of block height <= pruned height
2372017-12-05T06:38:42 * esotericnonsense is in a state of befuddlement
2382017-12-05T06:39:11 <esotericnonsense> if it has p2p logic (basically it's a node) then what is the seperate bitcoind node being used for at all
2392017-12-05T06:39:16 *** dcousens has quit IRC
2402017-12-05T06:39:41 <esotericnonsense> trusted verifier i guess?
2412017-12-05T06:39:47 *** dcousens has joined #bitcoin-core-dev
2422017-12-05T06:39:59 <kallewoof> The indexer doesn't store anything but the indexing parts, so it needs a node to resolve the 8 byte (block height + tx index) into actual transactions. It could store the entire block chain while at it, but that seems a bit redundant.
2432017-12-05T06:40:25 <sipa> kallewoof: that seems inefficient
2442017-12-05T06:40:46 <sipa> relying on a node for storage won't gove the kind of performance a blockexplorer needs
2452017-12-05T06:41:30 <kallewoof> I thought you were suggesting a p2p indexer that doesn't depend on a local node. Why not use RPC then?
2462017-12-05T06:42:41 <sipa> rpc is slow
2472017-12-05T06:42:43 <kallewoof> Or are you saying store the entire transaction in the indexer db?
2482017-12-05T06:42:47 <sipa> yes
2492017-12-05T06:42:51 <kallewoof> Ah
2502017-12-05T07:07:00 *** DvdKhl has joined #bitcoin-core-dev
2512017-12-05T07:14:51 *** intcat has quit IRC
2522017-12-05T07:15:13 *** jb55 has quit IRC
2532017-12-05T07:15:45 *** intcat has joined #bitcoin-core-dev
2542017-12-05T07:17:21 *** promag has quit IRC
2552017-12-05T07:17:35 *** promag has joined #bitcoin-core-dev
2562017-12-05T07:17:36 *** intcat has quit IRC
2572017-12-05T07:19:27 *** promag has quit IRC
2582017-12-05T07:20:17 *** Ylbam has joined #bitcoin-core-dev
2592017-12-05T07:22:26 *** intcat has joined #bitcoin-core-dev
2602017-12-05T07:33:38 *** shesek has quit IRC
2612017-12-05T07:39:44 *** xRavenheart has quit IRC
2622017-12-05T07:41:09 *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
2632017-12-05T07:41:09 *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
2642017-12-05T07:44:38 *** promag has joined #bitcoin-core-dev
2652017-12-05T07:53:06 *** unholymachine has quit IRC
2662017-12-05T07:54:37 *** unholymachine has joined #bitcoin-core-dev
2672017-12-05T07:57:23 *** promag has quit IRC
2682017-12-05T08:01:42 *** sanjeev has joined #bitcoin-core-dev
2692017-12-05T08:04:48 *** Deacyde has quit IRC
2702017-12-05T08:05:17 *** Deacyde has joined #bitcoin-core-dev
2712017-12-05T08:07:45 *** promag has joined #bitcoin-core-dev
2722017-12-05T08:10:37 *** Deacyde has quit IRC
2732017-12-05T08:10:43 *** Deacyde has joined #bitcoin-core-dev
2742017-12-05T08:19:24 *** promag has quit IRC
2752017-12-05T08:20:30 *** meshcollider has quit IRC
2762017-12-05T08:22:53 *** intcat has quit IRC
2772017-12-05T08:23:59 *** intcat has joined #bitcoin-core-dev
2782017-12-05T08:25:06 *** JDRacer has joined #bitcoin-core-dev
2792017-12-05T08:25:30 *** laurentmt has joined #bitcoin-core-dev
2802017-12-05T08:25:30 *** JDRacer has quit IRC
2812017-12-05T08:34:52 *** bule has quit IRC
2822017-12-05T08:39:44 *** rhavar has joined #bitcoin-core-dev
2832017-12-05T08:49:23 *** Deacyde has quit IRC
2842017-12-05T08:49:56 *** Deacyde has joined #bitcoin-core-dev
2852017-12-05T09:04:10 *** indistylo has quit IRC
2862017-12-05T09:10:42 *** DvdKhl has quit IRC
2872017-12-05T09:13:37 *** Randolf has quit IRC
2882017-12-05T09:20:39 *** zshlyk has joined #bitcoin-core-dev
2892017-12-05T09:21:26 <kallewoof> Hum. I could avoid a lot of reinventing wheels by simply requiring libbitcoin_[x].a in the indexer to link p2p stuff in. Potential minefield, but would be super easy to keep up to date with bitcoin core if it worked.
2902017-12-05T09:22:05 *** timothy has joined #bitcoin-core-dev
2912017-12-05T09:23:20 <kallewoof> Rewriting a minimal p2p codebase could be useful for lots of cases, but would need to be maintained to a greater degree.
2922017-12-05T09:23:56 *** intcat has quit IRC
2932017-12-05T09:24:48 <gmaxwell> also less important if its out of date... p2p protocol intentionally does not change frequently.
2942017-12-05T09:25:16 <gmaxwell> an indexer written against 0.2.10 would likely still work (though wouldn't index p2sh/segwit, but that may not matter if you're not using those things)
2952017-12-05T09:25:51 <kallewoof> That's a good point.
2962017-12-05T09:27:08 *** zshlyk has quit IRC
2972017-12-05T09:27:19 <kallewoof> I guess I'll try linking directly and see how far that gets me. I suspect there will be a lot of issues where the bitcoin code expects a fully fledged bitcoin instance running.
2982017-12-05T09:28:09 <kallewoof> If not, not regularly maintained would not be a critical issue as you noted.
2992017-12-05T09:31:15 *** Babozor_ has quit IRC
3002017-12-05T09:32:07 *** zshlyk has joined #bitcoin-core-dev
3012017-12-05T09:36:29 *** promag has joined #bitcoin-core-dev
3022017-12-05T09:45:54 *** tiagotrs has joined #bitcoin-core-dev
3032017-12-05T09:48:07 *** AaronvanW has joined #bitcoin-core-dev
3042017-12-05T09:53:10 *** Ylbam has quit IRC
3052017-12-05T09:53:45 *** zshlyk has quit IRC
3062017-12-05T09:53:56 *** rafalcpp has quit IRC
3072017-12-05T09:54:38 *** zshlyk has joined #bitcoin-core-dev
3082017-12-05T09:54:50 *** Guyver2 has joined #bitcoin-core-dev
3092017-12-05T09:57:33 *** Victorsueca has quit IRC
3102017-12-05T09:58:44 *** Victorsueca has joined #bitcoin-core-dev
3112017-12-05T10:00:35 *** rafalcpp has joined #bitcoin-core-dev
3122017-12-05T10:03:42 *** zshlyk has quit IRC
3132017-12-05T10:05:10 *** zshlyk has joined #bitcoin-core-dev
3142017-12-05T10:06:10 *** meshcollider has joined #bitcoin-core-dev
3152017-12-05T10:06:29 *** AaronvanW has quit IRC
3162017-12-05T10:07:45 *** zshlyk has quit IRC
3172017-12-05T10:08:38 *** zshlyk has joined #bitcoin-core-dev
3182017-12-05T10:28:43 *** belcher has joined #bitcoin-core-dev
3192017-12-05T10:55:29 *** esotericnonsense has quit IRC
3202017-12-05T11:04:46 *** zshlyk has quit IRC
3212017-12-05T11:05:15 *** Babozor has joined #bitcoin-core-dev
3222017-12-05T11:05:52 *** zshlyk has joined #bitcoin-core-dev
3232017-12-05T11:23:22 *** Victorsueca has quit IRC
3242017-12-05T11:23:27 *** Victor_sueca has joined #bitcoin-core-dev
3252017-12-05T11:26:25 *** tiagotrs has quit IRC
3262017-12-05T11:28:55 <bitcoin-git> [bitcoin] MeshCollider opened pull request #11829: Test datadir specified in conf file exists (master...201712_datadir_crash) https://github.com/bitcoin/bitcoin/pull/11829
3272017-12-05T11:30:20 *** luihuidui has joined #bitcoin-core-dev
3282017-12-05T11:30:25 <luihuidui> Hi core devs
3292017-12-05T11:32:57 *** zshlyk has quit IRC
3302017-12-05T11:33:40 *** zshlyk has joined #bitcoin-core-dev
3312017-12-05T11:51:25 *** BashCo_ has joined #bitcoin-core-dev
3322017-12-05T11:52:33 *** BashCo has quit IRC
3332017-12-05T11:54:56 *** hellxcode has quit IRC
3342017-12-05T11:57:13 *** Victor_sueca is now known as Victorsueca
3352017-12-05T12:06:43 *** zshlyk has quit IRC
3362017-12-05T12:07:33 *** zshlyk has joined #bitcoin-core-dev
3372017-12-05T12:09:01 *** tiagotrs has joined #bitcoin-core-dev
3382017-12-05T12:10:22 *** tiagotrs has quit IRC
3392017-12-05T12:10:22 *** tiagotrs has joined #bitcoin-core-dev
3402017-12-05T12:15:58 *** meshcollider has quit IRC
3412017-12-05T12:16:57 *** hellxcode has joined #bitcoin-core-dev
3422017-12-05T12:17:00 *** dabura667 has quit IRC
3432017-12-05T12:19:27 *** luihuidui has quit IRC
3442017-12-05T12:24:22 *** SopaXorzTaker has joined #bitcoin-core-dev
3452017-12-05T12:35:35 *** microapple has joined #bitcoin-core-dev
3462017-12-05T12:37:14 *** harrymm has joined #bitcoin-core-dev
3472017-12-05T12:41:00 *** checksauce has joined #bitcoin-core-dev
3482017-12-05T12:42:59 *** checksauce has quit IRC
3492017-12-05T12:53:45 *** zshlyk has quit IRC
3502017-12-05T12:54:38 *** zshlyk has joined #bitcoin-core-dev
3512017-12-05T12:56:42 *** hellxcode has quit IRC
3522017-12-05T12:57:34 *** JackH has joined #bitcoin-core-dev
3532017-12-05T13:00:01 *** promag has quit IRC
3542017-12-05T13:05:02 *** zshlyk has quit IRC
3552017-12-05T13:05:55 *** zshlyk has joined #bitcoin-core-dev
3562017-12-05T13:31:50 *** zshlyk has quit IRC
3572017-12-05T13:32:47 *** zshlyk has joined #bitcoin-core-dev
3582017-12-05T13:38:04 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3592017-12-05T13:57:07 *** microapple has quit IRC
3602017-12-05T14:02:37 *** tiagotrs has quit IRC
3612017-12-05T14:02:44 *** mypopydev has joined #bitcoin-core-dev
3622017-12-05T14:04:59 *** harrymm has quit IRC
3632017-12-05T14:08:11 *** promag has joined #bitcoin-core-dev
3642017-12-05T14:12:06 *** esotericnonsense has joined #bitcoin-core-dev
3652017-12-05T14:12:32 *** Chris_Stewart_5 has quit IRC
3662017-12-05T14:15:44 *** zshlyk has quit IRC
3672017-12-05T14:16:38 *** tiagotrs has joined #bitcoin-core-dev
3682017-12-05T14:16:40 *** zshlyk has joined #bitcoin-core-dev
3692017-12-05T14:19:01 *** harrymm has joined #bitcoin-core-dev
3702017-12-05T14:23:53 <promag> ping jnewbery
3712017-12-05T14:24:27 <promag> is it possible to test .conf file?
3722017-12-05T14:24:52 <promag> ref #11829
3732017-12-05T14:24:54 <gribble> https://github.com/bitcoin/bitcoin/issues/11829 | Test datadir specified in conf file exists by MeshCollider · Pull Request #11829 · bitcoin/bitcoin · GitHub
3742017-12-05T14:25:14 *** esotericnonsense has quit IRC
3752017-12-05T14:27:52 *** zshlyk has quit IRC
3762017-12-05T14:28:46 *** zshlyk has joined #bitcoin-core-dev
3772017-12-05T14:29:24 <aj> promag: do stuff like https://github.com/bitcoin/bitcoin/pull/10996/files#diff-7814a1916d2f12f31509d33370931974 ?
3782017-12-05T14:29:59 <promag> aj: \m/
3792017-12-05T14:35:04 *** roidster has joined #bitcoin-core-dev
3802017-12-05T14:35:27 *** roidster is now known as Guest24280
3812017-12-05T14:37:33 *** Guest24280 is now known as [_smitty]
3822017-12-05T14:40:30 *** Victorsueca has quit IRC
3832017-12-05T14:41:44 *** Victorsueca has joined #bitcoin-core-dev
3842017-12-05T14:41:48 *** ghost43 has quit IRC
3852017-12-05T14:42:54 *** mypopydev has left #bitcoin-core-dev
3862017-12-05T14:44:35 *** [_smitty] is now known as roidster
3872017-12-05T14:45:57 *** ekrok_ has joined #bitcoin-core-dev
3882017-12-05T14:46:58 *** ghost43 has joined #bitcoin-core-dev
3892017-12-05T14:48:31 *** ekrok has quit IRC
3902017-12-05T14:51:41 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3912017-12-05T14:54:03 <bitcoin-git> [bitcoin] hkjn opened pull request #11830: rpcuser.py: Use 'python' not 'python2' (master...rpcuser-py) https://github.com/bitcoin/bitcoin/pull/11830
3922017-12-05T14:54:57 *** sanjeev has quit IRC
3932017-12-05T14:57:11 *** Chris_Stewart_5 has quit IRC
3942017-12-05T15:10:58 *** jtimon has joined #bitcoin-core-dev
3952017-12-05T15:23:56 *** microapple has joined #bitcoin-core-dev
3962017-12-05T15:27:44 *** zshlyk has quit IRC
3972017-12-05T15:29:01 *** zshlyk has joined #bitcoin-core-dev
3982017-12-05T15:30:11 *** ghost43 has quit IRC
3992017-12-05T15:30:36 *** ghost43 has joined #bitcoin-core-dev
4002017-12-05T15:31:02 *** tiagotrs has quit IRC
4012017-12-05T15:32:25 *** tiagotrs has joined #bitcoin-core-dev
4022017-12-05T15:35:36 *** ghost43 has quit IRC
4032017-12-05T15:35:51 *** ghost43 has joined #bitcoin-core-dev
4042017-12-05T15:36:48 *** zshlyk has quit IRC
4052017-12-05T15:37:35 *** zshlyk has joined #bitcoin-core-dev
4062017-12-05T15:41:16 <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #11831: Always return true if AppInitMain got to the end (master...2017-12-startup-exit-return-code-race) https://github.com/bitcoin/bitcoin/pull/11831
4072017-12-05T15:44:12 *** dermoth has quit IRC
4082017-12-05T15:44:42 *** dermoth has joined #bitcoin-core-dev
4092017-12-05T15:45:18 *** sanjeev has joined #bitcoin-core-dev
4102017-12-05T15:53:57 *** Chris_Stewart_5 has joined #bitcoin-core-dev
4112017-12-05T15:56:35 <promag> BlueMatt: I didn't realise main thread was sitting there in WaitForShutdown
4122017-12-05T15:56:57 <BlueMatt> hmm?
4132017-12-05T15:57:08 <BlueMatt> ohoh, yea, we have a global thread that sits there and does nothing :(
4142017-12-05T15:57:52 <promag> actually it does.. are we there yet? are we there yet? are we there yet? ..
4152017-12-05T15:58:27 <BlueMatt> heh
4162017-12-05T15:58:29 <BlueMatt> yea...that
4172017-12-05T15:59:28 <promag> so zapwallettxes causes the process to exit with non-zero
4182017-12-05T15:59:55 <BlueMatt> Ive never seen that failure before, but I think its the race with ReacceptWalletTransactions
4192017-12-05T16:00:03 <BlueMatt> if nothing else that pr should be an improvement
4202017-12-05T16:00:08 <BlueMatt> its crazy to have that race
4212017-12-05T16:11:55 *** ExtraCrispy has joined #bitcoin-core-dev
4222017-12-05T16:12:49 *** tiagotrs has quit IRC
4232017-12-05T16:13:45 *** zshlyk has quit IRC
4242017-12-05T16:14:37 *** zshlyk has joined #bitcoin-core-dev
4252017-12-05T16:16:08 *** AaronvanW has joined #bitcoin-core-dev
4262017-12-05T16:18:15 *** Aaronvan_ has joined #bitcoin-core-dev
4272017-12-05T16:20:16 *** ExtraCrispy has quit IRC
4282017-12-05T16:20:20 *** wraithm has joined #bitcoin-core-dev
4292017-12-05T16:20:45 *** AaronvanW has quit IRC
4302017-12-05T16:25:52 *** DvdKhl has joined #bitcoin-core-dev
4312017-12-05T16:27:00 *** tiagotrs has joined #bitcoin-core-dev
4322017-12-05T16:34:35 *** tiagotrs has quit IRC
4332017-12-05T16:40:01 *** JackH has quit IRC
4342017-12-05T16:41:10 *** promag has quit IRC
4352017-12-05T16:41:22 *** wxss has joined #bitcoin-core-dev
4362017-12-05T16:48:44 *** zshlyk has quit IRC
4372017-12-05T16:49:31 *** zshlyk has joined #bitcoin-core-dev
4382017-12-05T16:50:03 *** microapple has quit IRC
4392017-12-05T16:53:41 *** JackH has joined #bitcoin-core-dev
4402017-12-05T16:53:46 *** earlz is now known as WowSuchEarlz
4412017-12-05T16:58:38 *** microapple has joined #bitcoin-core-dev
4422017-12-05T17:11:08 *** zshlyk has quit IRC
4432017-12-05T17:12:01 *** zshlyk has joined #bitcoin-core-dev
4442017-12-05T17:18:41 *** Aaronvan_ is now known as AaronvanW
4452017-12-05T17:32:14 *** zshlyk has quit IRC
4462017-12-05T17:33:06 *** zshlyk has joined #bitcoin-core-dev
4472017-12-05T17:33:17 *** spudowiar has joined #bitcoin-core-dev
4482017-12-05T17:33:43 <sipa> off topic for at least 5 years: c++17 is now officia
4492017-12-05T17:37:58 *** jamesob has joined #bitcoin-core-dev
4502017-12-05T17:41:08 *** jb55 has joined #bitcoin-core-dev
4512017-12-05T17:42:32 *** Murch has joined #bitcoin-core-dev
4522017-12-05T17:42:48 *** BGL has quit IRC
4532017-12-05T17:43:27 *** twistedline has quit IRC
4542017-12-05T17:47:04 *** twistedline has joined #bitcoin-core-dev
4552017-12-05T17:47:17 *** goatpig has joined #bitcoin-core-dev
4562017-12-05T17:47:44 <spudowiar> sipa: With these new C++ standards I'll soon playing fast and loose with the word "know" when I say that I know C++
4572017-12-05T17:47:48 <spudowiar> *soon be
4582017-12-05T17:49:20 *** Ylbam has joined #bitcoin-core-dev
4592017-12-05T17:50:15 <sipa> haha
4602017-12-05T17:58:31 *** WowSuchEarlz is now known as eWowSuchEarlz
4612017-12-05T17:58:49 *** eWowSuchEarlz is now known as WowSuchEarlz
4622017-12-05T18:00:01 *** timothy has quit IRC
4632017-12-05T18:05:12 *** ghost43 has quit IRC
4642017-12-05T18:05:25 *** ghost43 has joined #bitcoin-core-dev
4652017-12-05T18:07:59 *** microapple has quit IRC
4662017-12-05T18:10:12 *** zshlyk has quit IRC
4672017-12-05T18:11:02 *** zshlyk has joined #bitcoin-core-dev
4682017-12-05T18:14:32 *** rafalcpp has quit IRC
4692017-12-05T18:15:20 *** ekrok_ has quit IRC
4702017-12-05T18:15:32 *** Victorsueca has quit IRC
4712017-12-05T18:16:45 *** Victorsueca has joined #bitcoin-core-dev
4722017-12-05T18:18:28 <phantomcircuit> BlueMatt, wait doesn't WaitForShutdown wait in the kernel?
4732017-12-05T18:19:17 *** ekrok has joined #bitcoin-core-dev
4742017-12-05T18:20:03 <BlueMatt> phantomcircuit: because signal handlers cant do "things"
4752017-12-05T18:20:26 <BlueMatt> so eg we cant do a cv notify in the SIGTERM signal handler
4762017-12-05T18:20:32 <BlueMatt> which is what we'd prefer to do
4772017-12-05T18:21:04 <BlueMatt> so *something* has to check for fShutdownRequested regularly
4782017-12-05T18:22:00 *** goatpig has quit IRC
4792017-12-05T18:22:46 *** microapple has joined #bitcoin-core-dev
4802017-12-05T18:24:45 *** zshlyk has quit IRC
4812017-12-05T18:25:36 *** zshlyk has joined #bitcoin-core-dev
4822017-12-05T18:38:37 *** SopaXorzTaker has quit IRC
4832017-12-05T18:38:48 *** goatpig has joined #bitcoin-core-dev
4842017-12-05T18:39:42 *** owowo has quit IRC
4852017-12-05T18:43:18 *** BGL has joined #bitcoin-core-dev
4862017-12-05T18:45:26 *** owowo has joined #bitcoin-core-dev
4872017-12-05T18:57:44 *** microapple has quit IRC
4882017-12-05T18:58:09 *** microapple has joined #bitcoin-core-dev
4892017-12-05T18:58:25 *** microapple has quit IRC
4902017-12-05T19:04:40 *** sanjeev has quit IRC
4912017-12-05T19:07:45 *** zshlyk has quit IRC
4922017-12-05T19:08:50 *** yoctopede has joined #bitcoin-core-dev
4932017-12-05T19:12:36 *** ekrok has quit IRC
4942017-12-05T19:14:23 <sipa> BlueMatt: did i do something wrong when merging #10773? All travis master builds since report 'HEAD was not signed with a trusted key!'
4952017-12-05T19:14:25 <gribble> https://github.com/bitcoin/bitcoin/issues/10773 | Shell script cleanups by practicalswift · Pull Request #10773 · bitcoin/bitcoin · GitHub
4962017-12-05T19:14:39 <BlueMatt> almost certainly, then
4972017-12-05T19:14:40 <BlueMatt> :p
4982017-12-05T19:14:52 <BlueMatt> sipa: this is why you're supposed to use the verify-commits pre-push-hook :(
4992017-12-05T19:15:11 *** roidster has quit IRC
5002017-12-05T19:15:54 <sipa> well where is the error?
5012017-12-05T19:16:06 <sipa> it's signed with the same key as all my previous merges
5022017-12-05T19:17:14 *** Randolf has joined #bitcoin-core-dev
5032017-12-05T19:18:27 <sipa> BlueMatt: also, please document the pre-push things in developer-notes.md or so
5042017-12-05T19:18:52 <BlueMatt> sipa: well the pre-push-hook is only for the committers...and I could have sworn I've yelled at all of y'all about it
5052017-12-05T19:19:19 <sipa> then create a committers.md or so
5062017-12-05T19:20:23 <BlueMatt> cp contrib/verify-commits/pre-push-hook.sh .git/hooks/pre-push
5072017-12-05T19:20:43 <BlueMatt> or, if you prefer, echo 'cp contrib/verify-commits/pre-push-hook.sh .git/hooks/pre-push' > committers.md
5082017-12-05T19:21:10 <sipa> please document it somewhere more permanently than IRC :)
5092017-12-05T19:21:32 <sipa> haha
5102017-12-05T19:21:53 <BlueMatt> as for travis, cant reproduce, so need to dig more, hold on
5112017-12-05T19:24:34 <sipa> thanks!
5122017-12-05T19:25:32 *** Babozor has quit IRC
5132017-12-05T19:25:47 *** yoctopede has quit IRC
5142017-12-05T19:26:37 *** yoctopede has joined #bitcoin-core-dev
5152017-12-05T19:29:05 *** Dizzle has joined #bitcoin-core-dev
5162017-12-05T19:29:36 *** laurentmt has quit IRC
5172017-12-05T19:55:28 *** helloworld has joined #bitcoin-core-dev
5182017-12-05T19:58:01 <aj> jnewbery: ping?
5192017-12-05T19:58:12 <BlueMatt> sipa: did you extend your key's (or subkey's) expiry period?
5202017-12-05T19:58:42 <BlueMatt> sipa: for some reason travis keeps getting a copy where the A636E97631F767E0 subkey expires on oct 19, 2017
5212017-12-05T19:58:42 <sipa> BlueMatt: no
5222017-12-05T19:58:47 <sipa> heh
5232017-12-05T19:59:07 <sipa> i can't remember changing that, at least
5242017-12-05T19:59:27 *** helloworld has quit IRC
5252017-12-05T19:59:27 *** Victorsueca has quit IRC
5262017-12-05T20:00:39 *** Victorsueca has joined #bitcoin-core-dev
5272017-12-05T20:09:55 *** yoctopede has quit IRC
5282017-12-05T20:10:42 *** yoctopede has joined #bitcoin-core-dev
5292017-12-05T20:13:23 *** yoctopede has quit IRC
5302017-12-05T20:14:02 *** yoctopede has joined #bitcoin-core-dev
5312017-12-05T20:17:30 <BlueMatt> sipa: lol, did you know POSIX echo is poorly-defined?
5322017-12-05T20:17:44 <sipa> ?
5332017-12-05T20:18:01 <BlueMatt> sec
5342017-12-05T20:18:58 *** JackH has quit IRC
5352017-12-05T20:19:35 <aj> BlueMatt: "echo -e hi" ?
5362017-12-05T20:19:57 <BlueMatt> aj: -e is implied for (some) \ things in dash, and you cant turn it off
5372017-12-05T20:21:21 <aj> BlueMatt: yeah. use printf or printf "%s" "what you actually want to print" ...
5382017-12-05T20:21:28 <spudowiar> BlueMatt: That is why I always use python -c "print(*__import__(\"sys\").argv[1:])" in my shell scripts
5392017-12-05T20:21:29 <BlueMatt> yes
5402017-12-05T20:21:38 <BlueMatt> spudowiar: lolwat
5412017-12-05T20:21:55 *** JackH has joined #bitcoin-core-dev
5422017-12-05T20:22:41 <spudowiar> BlueMatt: :)
5432017-12-05T20:22:57 *** jb55 has quit IRC
5442017-12-05T20:23:13 <spudowiar> I wonder if we should discuss #11833 here as it's turned into a conversation that might be easier to have on IRC :P
5452017-12-05T20:23:14 <gribble> https://github.com/bitcoin/bitcoin/issues/11833 | [Net] WebSocket support · Issue #11833 · bitcoin/bitcoin · GitHub
5462017-12-05T20:23:26 <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #11834: [verify-commits] Fix gpg.sh's echoing for commits with '\n' (master...2017-12-verify-commits-fix) https://github.com/bitcoin/bitcoin/pull/11834
5472017-12-05T20:27:06 <BlueMatt> spudowiar: lots of folks seem like the "bitcoin has a p2p network, lets shove more features in it so that *everything* can use it!" approach, cause its easy, but it also doesnt scale....ultimately the bitcoin p2p network is full of garbage evil nodes anyway, its not magic...
5482017-12-05T20:27:26 <bitcoin-git> [bitcoin] practicalswift opened pull request #11835: Add Travis check for unused Python imports (master...lint-python) https://github.com/bitcoin/bitcoin/pull/11835
5492017-12-05T20:28:43 <spudowiar> BlueMatt: Awww, but I was trying to run bitcoind in the browser though :P
5502017-12-05T20:29:30 <BlueMatt> spudowiar: please build second network of servers which provide blockchain data, it'd be useful to have redundancy for bitcoind to sync from them, even
5512017-12-05T20:29:46 <BlueMatt> (or...whats the wallet for that again? electrum?)
5522017-12-05T20:30:23 <spudowiar> huh?
5532017-12-05T20:32:06 <spudowiar> Does --with-sysroot= work with Bitcoin Core?
5542017-12-05T20:32:14 <achow101> spudowiar: why the hell would you want to run bitcoind in the browser
5552017-12-05T20:32:23 <spudowiar> achow101: Because I can, I guess?
5562017-12-05T20:32:30 <spudowiar> I'm a weird guy
5572017-12-05T20:33:05 <sipa> do you mean... compile with emscripten?
5582017-12-05T20:34:03 <spudowiar> sipa: Yeah, I'm using Emscripten and I have Bitcoin-Qt mostly working (except networking is disabled)
5592017-12-05T20:34:07 <spudowiar> And some graphics are acting wierd
5602017-12-05T20:34:10 <spudowiar> *weird
5612017-12-05T20:34:23 <instagibbs> neveraskedifheshould.gif
5622017-12-05T20:34:46 <sipa> spudowiar: that's pretty cool :)
5632017-12-05T20:39:05 *** goatpig has quit IRC
5642017-12-05T20:43:02 <jonasschnelli> BlueMatt: https://github.com/bitcoin/bitcoin/pull/11740/commits/f9cc6dcc65a9cd9ce72c7fbb6bde9d2efb71a8f7#r154686982
5652017-12-05T20:43:09 <jonasschnelli> can you elaborate? Can't follow
5662017-12-05T20:43:38 *** goatpig has joined #bitcoin-core-dev
5672017-12-05T20:44:16 <BlueMatt> jonasschnelli: the buffer comment?
5682017-12-05T20:44:20 <jonasschnelli> Yeah
5692017-12-05T20:44:46 <jonasschnelli> So accepting blocks up to NODE_NETWORK_LIMITED_MIN_BLOCKS(288)+2?`
5702017-12-05T20:45:08 <BlueMatt> yea, that
5712017-12-05T20:45:19 *** spudowiar has quit IRC
5722017-12-05T20:47:08 <jonasschnelli> BlueMatt: current impl.: if you tip is 1000 and someone requests height 713, you will disconnect because (1000-713 == 289)
5732017-12-05T20:47:23 <jonasschnelli> But you want disconnect 712 (==288 == within the treshold)
5742017-12-05T20:47:35 <jonasschnelli> BlueMatt: I don't see whats wrong with it
5752017-12-05T20:48:21 <jonasschnelli> BlueMatt: We already have a buffer of 144 (we have 144 additional blocks for the reorg buffer)
5762017-12-05T20:48:44 <jonasschnelli> (goal is 144 [one day], +144 reorg buffer = 288)
5772017-12-05T20:48:45 <BlueMatt> the service bit seems to imply that you can/will serve the top 288 blocks, but if you request a block, and then they get a new block before they respond to you, you get disconnected
5782017-12-05T20:48:49 <BlueMatt> even though you didnt violate the bip
5792017-12-05T20:49:42 <jonasschnelli> I guess if a client detects that he will request at the very bottom of NODE_NETWORK_LIMITED_MIN_BLOCKS he needs to expect a disconnect
5802017-12-05T20:50:01 <BlueMatt> I'd call the current implementation a violation of the BIP
5812017-12-05T20:50:04 <jonasschnelli> BlueMatt: what you propose would essentially increase the 144 block buffer leading to a similar event just two blocks higher?
5822017-12-05T20:50:07 <BlueMatt> "If signaled, the peer MUST be capable of serving at least the last 288 blocks "
5832017-12-05T20:50:14 <BlueMatt> the current implementation is clearly not capable of doign so
5842017-12-05T20:50:37 <jonasschnelli> BlueMatt: hmm? why? I can serve the last 288 blocks?
5852017-12-05T20:50:40 <BlueMatt> if I request block at 288 deep, I do not expect to get disconnected for it, even if there is a race and you get another block at that same time
5862017-12-05T20:50:59 <jonasschnelli> hmm..
5872017-12-05T20:51:17 <BlueMatt> while we're on the bip (cause I just pulled it up): "A safety buffer of additional 144 blocks to handle chain reorganizations SHOULD be taken into account when connecting to a peer signaling the NODE_NETWORK_LIMITED service bit." <-- I have absolutely no idea what in the hell that means
5882017-12-05T20:51:26 <jonasschnelli> https://github.com/bitcoin/bips/blob/master/bip-0159.mediawiki#counter-measures-for-peer-fingerprinting
5892017-12-05T20:51:41 <jonasschnelli> Okay... your right. Got your point!
5902017-12-05T20:52:05 <BlueMatt> I mean you're not *really* serving deeper, maybe one or two, but its certainly not a fingerprinting attack if everyone does it
5912017-12-05T20:52:05 <jonasschnelli> Though...
5922017-12-05T20:52:28 <jonasschnelli> But what if we use two blocks and other implementation would use a single block as additional buffer?
5932017-12-05T20:52:41 <jonasschnelli> But... meh,.. doesn't matter
5942017-12-05T20:52:47 <BlueMatt> then they'd already be fingerprintable because they have a different subver :p
5952017-12-05T20:52:48 <jonasschnelli> let me just add two blocks
5962017-12-05T20:52:53 <jonasschnelli> BlueMatt: indeed
5972017-12-05T20:53:10 <BlueMatt> I'm still confused as to what you meant by that sentence in the BIP
5982017-12-05T20:53:18 <BlueMatt> "A safety buffer of additional 144 blocks to handle chain reorganizations SHOULD be taken into account when connecting to a peer signaling the NODE_NETWORK_LIMITED service bit."
5992017-12-05T20:53:23 <BlueMatt> I have no idea what that means
6002017-12-05T20:53:45 <gmaxwell> 144? I thought that was supposted to advise 12.
6012017-12-05T20:53:46 <sipa> i think it means you should only connect out to a LIMITED peer if you don't think you need more than 144 blocks (rather than 288)
6022017-12-05T20:53:47 <jonasschnelli> BlueMatt: assume you are a light client and what to fetch a block within the last 144
6032017-12-05T20:54:01 <jonasschnelli> what sipa said
6042017-12-05T20:54:25 <BlueMatt> sipa: elaborate? I'm still lost...
6052017-12-05T20:54:31 <BlueMatt> gmaxwell: wat?
6062017-12-05T20:54:39 <gmaxwell> oh jesus people.
6072017-12-05T20:54:57 <sipa> BlueMatt: i think it means that you should only signal LIMITED if you keep the last 288 blocks, but only fetch from a peer if you think you don't need more than 144
6082017-12-05T20:55:16 <BlueMatt> sipa: yea, sorry, that half I got, the *why* part I'm still missing...
6092017-12-05T20:55:25 <sipa> i don't know!
6102017-12-05T20:55:25 <BlueMatt> gmaxwell: what'd we blow up?
6112017-12-05T20:56:05 <jonasschnelli> BlueMatt: It's a kindly reminder for a reorg buffer...
6122017-12-05T20:56:30 <jonasschnelli> BlueMatt: if you have a light client app, you need tip-150, don't connect to LIMITED
6132017-12-05T20:56:37 <gmaxwell> BlueMatt: please read the prior discussions.
6142017-12-05T20:56:44 <jonasschnelli> BlueMatt: but do if you need block TIP-144
6152017-12-05T20:57:03 <jonasschnelli> (it's pure informative)
6162017-12-05T20:57:12 <sipa> it shouldn't be just informative
6172017-12-05T20:57:18 <jonasschnelli> Technically, it's always 288
6182017-12-05T20:57:25 <gmaxwell> gah
6192017-12-05T20:57:52 <sipa> gmaxwell: i vaguely remember discussions about shorter windows, but i wouldn't know where to find them
6202017-12-05T20:57:55 <gmaxwell> look, the BIP requires you keep your last 288. Thats it, the rest is advice.
6212017-12-05T20:58:02 <jonasschnelli> "A safety buffer of additional 144 blocks to handle chain reorganizations SHOULD be taken into account when connecting to a peer signaling the NODE_NETWORK_LIMITED service bit."
6222017-12-05T20:58:10 <jonasschnelli> Is purely advice... you can't set a MUST there IMO
6232017-12-05T20:58:23 <sipa> it can be a SHOULD
6242017-12-05T20:58:53 <gmaxwell> If a client _guesses_ it's going to need blocks 285 ago, then a LIMITED peer is quite possibly useless to it: one, it's estimate of how many it will need will not be accurate (there could have been more blocks) and even it if is, they could be on a fork from where the peer is.
6252017-12-05T20:59:22 <BlueMatt> yes, that much I get, I'm just really confused as to where the N/2 came from there
6262017-12-05T20:59:35 <BlueMatt> why does the buffer need to be 2x what you want?
6272017-12-05T20:59:57 <gmaxwell> The actual correct behavior is complicated; if you're estimating your need based on nothing but time you should be pretty conservative (n/2 for example). If you have headers and know what you need, that doesn't exist.
6282017-12-05T21:00:27 <BlueMatt> ok, it just seemed to be weird to be so specific...
6292017-12-05T21:02:36 <gmaxwell> I think the 144 number comes more out of "if we want to be able to reliably serve 144, then our target will need to be higher, e.g. 288)
6302017-12-05T21:02:49 *** yoctopede has quit IRC
6312017-12-05T21:03:42 *** yoctopede has joined #bitcoin-core-dev
6322017-12-05T21:03:52 <BlueMatt> ok, fair
6332017-12-05T21:06:33 <BlueMatt> it just reads funny, was my point :p
6342017-12-05T21:06:57 <BlueMatt> jonasschnelli: wanna just squash and I'll ack
6352017-12-05T21:07:19 <jonasschnelli> BlueMatt: Yeah. Just squashing (had to rebase anyways)
6362017-12-05T21:07:23 <jonasschnelli> Give me a min
6372017-12-05T21:11:46 <BlueMatt> suresure
6382017-12-05T21:21:36 <jonasschnelli> BlueMatt: done
6392017-12-05T21:26:00 <BlueMatt> lol, how's this for an idea: try to load libsensors and if it loads write cpu temperateure to debug.log on a regular basis
6402017-12-05T21:26:44 *** yoctopede has quit IRC
6412017-12-05T21:27:36 *** yoctopede has joined #bitcoin-core-dev
6422017-12-05T21:29:10 *** dermoth has quit IRC
6432017-12-05T21:32:38 *** jb55 has joined #bitcoin-core-dev
6442017-12-05T21:33:10 <bitcoin-git> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/91eeaa03354b...5bea05bc1d17
6452017-12-05T21:33:10 <bitcoin-git> bitcoin/master a38686c Matt Corallo: [verify-commits] Fix gpg.sh's echoing for commits with '\n'
6462017-12-05T21:33:10 <bitcoin-git> bitcoin/master 5bea05b Pieter Wuille: Merge #11834: [verify-commits] Fix gpg.sh's echoing for commits with '\n'...
6472017-12-05T21:33:20 *** wraithm has quit IRC
6482017-12-05T21:33:45 <bitcoin-git> [bitcoin] sipa closed pull request #11834: [verify-commits] Fix gpg.sh's echoing for commits with '\n' (master...2017-12-verify-commits-fix) https://github.com/bitcoin/bitcoin/pull/11834
6492017-12-05T21:33:47 *** Murch has quit IRC
6502017-12-05T21:34:02 <gmaxwell> BlueMatt: oh no, moar dependencies.
6512017-12-05T21:34:13 <jimpo> Is there a preferred implementation of union/variant types? I know about boost::variant, but I've heard Core is trying to use boost less.
6522017-12-05T21:34:22 <BlueMatt> gmaxwell: well you could do a if-you-cant-find-dep-dont-print-temp
6532017-12-05T21:34:48 <BlueMatt> gmaxwell: but it'd be so nice in debugging peoples' rejecting-chain issues to see debug.log entries that say "WARNING: My hardware is overheating garbage, dont trust output"
6542017-12-05T21:34:50 <sipa> jimpo: std::variant in c++17 ? :)
6552017-12-05T21:35:06 <jimpo> Oh, is C++17 allowed/enabled?
6562017-12-05T21:35:11 <BlueMatt> lol no
6572017-12-05T21:35:16 <jimpo> derp
6582017-12-05T21:35:16 <sipa> no
6592017-12-05T21:35:28 <sipa> it was just officially published... today
6602017-12-05T21:35:31 <BlueMatt> is it even in debian-testing?
6612017-12-05T21:35:42 <BlueMatt> i think clang supports it
6622017-12-05T21:35:45 <sipa> needs GCC 7
6632017-12-05T21:36:23 <sipa> and Clang 5
6642017-12-05T21:36:30 <jimpo> Ah, that explains why std::variant came up in searches today and I didn't see it a month ago
6652017-12-05T21:37:02 <sipa> to be fair, the last month or so was just formally for ISO to publish it
6662017-12-05T21:37:11 <sipa> it was exactly known what the spec would be
6672017-12-05T21:37:17 <jimpo> So is a ghetto struct with an enum + union preferred to boost::variant?
6682017-12-05T21:38:27 <sipa> i think you can use boost::variant, as we're already using it anyway
6692017-12-05T21:38:32 <jimpo> k
6702017-12-05T21:38:48 <BlueMatt> if you can keep it in a cpp, use boost::variant, if its in an infection header....ugh, but maybe ok
6712017-12-05T21:51:58 *** rabidus has quit IRC
6722017-12-05T21:54:55 *** dqx has joined #bitcoin-core-dev
6732017-12-05T21:56:35 *** Chris_Stewart_5 has quit IRC
6742017-12-05T22:00:35 *** rabidus has joined #bitcoin-core-dev
6752017-12-05T22:07:08 *** Murch has joined #bitcoin-core-dev
6762017-12-05T22:07:33 *** Victorsueca has quit IRC
6772017-12-05T22:08:39 *** Murch has quit IRC
6782017-12-05T22:08:45 *** Victorsueca has joined #bitcoin-core-dev
6792017-12-05T22:13:33 *** Murch has joined #bitcoin-core-dev
6802017-12-05T22:14:46 *** yoctopede has quit IRC
6812017-12-05T22:15:22 *** Murch has quit IRC
6822017-12-05T22:16:53 *** Murch has joined #bitcoin-core-dev
6832017-12-05T22:16:54 *** yoctopede has joined #bitcoin-core-dev
6842017-12-05T22:17:51 *** rhavar has quit IRC
6852017-12-05T22:27:35 *** DvdKhl has quit IRC
6862017-12-05T22:31:08 *** arubi has quit IRC
6872017-12-05T22:35:43 *** Guyver2 has quit IRC
6882017-12-05T22:35:51 *** belcher_ has joined #bitcoin-core-dev
6892017-12-05T22:35:52 *** arubi has joined #bitcoin-core-dev
6902017-12-05T22:36:23 *** belcher_ has quit IRC
6912017-12-05T22:37:32 <luke-jr> jimpo: boost is fine outside of libconsensus I think
6922017-12-05T22:38:54 <GAit> patches to get core to build with the ndk (working on android shell with no deps or glibc/LD_LIBRARY_PATH hacks) if anyone is interested can be reviewed at https://github.com/greenaddress/bitcoin_ndk - warning, just like abcore, these patches are also very alpha and currently I don't think there's anything worth making an upstream PR for. and fwiw later ndk releases seem to fail.
6932017-12-05T22:39:20 *** belcher has quit IRC
6942017-12-05T22:40:12 <GAit> mostly deals with setting up the toolchain and patching some different headers or things missing in android (ifaddrs for example)
6952017-12-05T22:55:40 *** arubi has quit IRC
6962017-12-05T22:56:26 *** arubi has joined #bitcoin-core-dev
6972017-12-05T22:56:42 *** roadcrap has quit IRC
6982017-12-05T22:57:06 *** dqx has quit IRC
6992017-12-05T23:01:45 *** Cogito_Ergo_Sum has quit IRC
7002017-12-05T23:02:06 *** Randolf has quit IRC
7012017-12-05T23:07:30 *** meshcollider has joined #bitcoin-core-dev
7022017-12-05T23:07:55 *** yoctopede has quit IRC
7032017-12-05T23:08:46 *** yoctopede has joined #bitcoin-core-dev
7042017-12-05T23:09:51 *** shesek has joined #bitcoin-core-dev
7052017-12-05T23:09:51 *** shesek has joined #bitcoin-core-dev
7062017-12-05T23:10:44 <TD-Linux> BlueMatt, I think a built-in memtest would be a better use of time :)
7072017-12-05T23:11:08 <BlueMatt> TD-Linux: that is almost certainly true
7082017-12-05T23:16:47 *** yoctopede has quit IRC
7092017-12-05T23:17:38 *** yoctopede has joined #bitcoin-core-dev
7102017-12-05T23:25:11 *** ghost43 has quit IRC
7112017-12-05T23:25:45 *** ghost43 has joined #bitcoin-core-dev
7122017-12-05T23:28:24 <phantomcircuit> iirc there are user space memtest utilities but i dont see how they can actually be effective unless they use all available memory
7132017-12-05T23:30:21 <phantomcircuit> BlueMatt, ^
7142017-12-05T23:31:18 <BlueMatt> I think, or at least have a suspicion, that half our user-memory-corruption errors are due to cpu overheating and writing garbage
7152017-12-05T23:31:57 <BlueMatt> but that isnt based on much and may be very bogus
7162017-12-05T23:32:19 <phantomcircuit> hmm
7172017-12-05T23:32:31 <phantomcircuit> indeed that would be caught with a pretty simple test
7182017-12-05T23:32:47 <phantomcircuit> just load some test vectors and check siphash results
7192017-12-05T23:33:36 <BlueMatt> yea, I mean its a shame we have assumevalid now...otherwise I'd suggest just busywaiting one or two of the script check threads with something like that
7202017-12-05T23:36:27 *** Giszmo has quit IRC
7212017-12-05T23:37:50 *** jb55 has quit IRC
7222017-12-05T23:39:47 *** Randolf has joined #bitcoin-core-dev
7232017-12-05T23:46:24 *** treeuntor has joined #bitcoin-core-dev
7242017-12-05T23:51:12 *** Giszmo has joined #bitcoin-core-dev
7252017-12-05T23:52:33 *** gonzazoid has joined #bitcoin-core-dev
7262017-12-05T23:53:31 *** Evel-Knievel has quit IRC
7272017-12-05T23:53:54 *** str4d has joined #bitcoin-core-dev
7282017-12-05T23:54:29 *** shesek has quit IRC
7292017-12-05T23:56:02 <gmaxwell> we could make block validation fails sleep for a second then try again... and log&crash if successful the second time.
7302017-12-05T23:56:13 *** jb55 has joined #bitcoin-core-dev
7312017-12-05T23:58:24 *** shesek has joined #bitcoin-core-dev
7322017-12-05T23:58:24 *** shesek has joined #bitcoin-core-dev
7332017-12-05T23:59:20 *** gonzazoid has quit IRC