12016-09-14T00:04:16 *** zxzzt has joined #bitcoin-core-dev
22016-09-14T00:08:43 *** fengling has joined #bitcoin-core-dev
32016-09-14T00:13:41 *** fengling has quit IRC
42016-09-14T00:33:31 *** droark has quit IRC
52016-09-14T00:37:42 *** fengling has joined #bitcoin-core-dev
62016-09-14T00:39:14 *** justanotheruser has quit IRC
72016-09-14T00:39:34 *** justanotheruser has joined #bitcoin-core-dev
82016-09-14T00:40:45 *** justanotheruser has quit IRC
92016-09-14T00:41:05 *** justanotheruser has joined #bitcoin-core-dev
102016-09-14T00:42:40 *** fengling has quit IRC
112016-09-14T00:46:54 *** Ylbam has quit IRC
122016-09-14T00:52:45 *** fengling has joined #bitcoin-core-dev
132016-09-14T00:57:28 *** fengling has quit IRC
142016-09-14T00:59:37 *** Giszmo has quit IRC
152016-09-14T01:04:38 *** justanotheruser is now known as hughmungus
162016-09-14T01:04:49 *** hughmungus is now known as justanotheruser
172016-09-14T02:04:38 *** fengling has joined #bitcoin-core-dev
182016-09-14T02:07:24 *** btcdrak has quit IRC
192016-09-14T02:49:05 *** fengling has quit IRC
202016-09-14T03:06:33 *** fengling has joined #bitcoin-core-dev
212016-09-14T03:20:06 *** Alopex has quit IRC
222016-09-14T03:21:11 *** Alopex has joined #bitcoin-core-dev
232016-09-14T03:22:49 *** netsin has joined #bitcoin-core-dev
242016-09-14T03:31:24 *** netsin has joined #bitcoin-core-dev
252016-09-14T03:35:12 *** Alopex has quit IRC
262016-09-14T03:36:17 *** Alopex has joined #bitcoin-core-dev
272016-09-14T03:47:28 *** netsin has quit IRC
282016-09-14T03:53:11 *** Alopex has quit IRC
292016-09-14T03:54:17 *** Alopex has joined #bitcoin-core-dev
302016-09-14T04:00:33 *** netsin has joined #bitcoin-core-dev
312016-09-14T04:07:02 *** Alopex has quit IRC
322016-09-14T04:08:07 *** Alopex has joined #bitcoin-core-dev
332016-09-14T04:21:15 *** aj_ is now known as aj
342016-09-14T04:23:38 *** netsin has joined #bitcoin-core-dev
352016-09-14T04:30:12 *** netsin has quit IRC
362016-09-14T04:31:31 *** Ismar has joined #bitcoin-core-dev
372016-09-14T04:36:42 *** Ismar has quit IRC
382016-09-14T04:49:07 *** Alopex has quit IRC
392016-09-14T04:50:12 *** Alopex has joined #bitcoin-core-dev
402016-09-14T04:51:31 *** justanotheruser has quit IRC
412016-09-14T05:00:25 *** Squidicc has quit IRC
422016-09-14T05:00:34 *** Squidicuz has joined #bitcoin-core-dev
432016-09-14T05:02:17 *** justanotheruser has joined #bitcoin-core-dev
442016-09-14T05:03:01 *** Alopex has quit IRC
452016-09-14T05:04:07 *** Alopex has joined #bitcoin-core-dev
462016-09-14T05:05:06 *** Squidicuz has quit IRC
472016-09-14T05:09:40 *** btcdrak has joined #bitcoin-core-dev
482016-09-14T05:13:00 *** dcousens has left #bitcoin-core-dev
492016-09-14T05:13:05 *** dcousens has joined #bitcoin-core-dev
502016-09-14T05:47:31 *** kadoban has quit IRC
512016-09-14T05:56:16 *** owowo has quit IRC
522016-09-14T06:01:32 *** Squidicuz has joined #bitcoin-core-dev
532016-09-14T06:13:02 *** Alopex has quit IRC
542016-09-14T06:14:07 *** Alopex has joined #bitcoin-core-dev
552016-09-14T06:25:25 *** Madars_ is now known as Madars
562016-09-14T06:32:20 *** assder has joined #bitcoin-core-dev
572016-09-14T06:32:59 <GitHub186> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/fa7caf6d9116...57b34599b2de
582016-09-14T06:32:59 <GitHub186> bitcoin/master 1b6bcdd Jonas Schnelli: Remove maxuploadtargets recommended minimum
592016-09-14T06:33:00 <GitHub186> bitcoin/master 57b3459 Jonas Schnelli: Merge #8712: Remove maxuploadtargets recommended minimum...
602016-09-14T06:33:09 <GitHub42> [bitcoin] jonasschnelli closed pull request #8712: Remove maxuploadtargets recommended minimum (master...2016/09/rem_maxupt_min) https://github.com/bitcoin/bitcoin/pull/8712
612016-09-14T06:43:22 *** owowo has joined #bitcoin-core-dev
622016-09-14T06:48:49 <jonasschnelli> I guess the Travis master issue is due to a random flickering in walletbackup.py
632016-09-14T06:50:42 <jonasschnelli> Would be nice if anyone could review/ACK/NACK https://github.com/bitcoin/bitcoin/pull/7783 (nested RPC commands).
642016-09-14T06:51:38 <dcousens> jonasschnelli: how would you extend it to the RPC?
652016-09-14T06:52:10 <jonasschnelli> dcousens: there are no plans to extend it to the RPC layer... the PR once had support for RPC but we decided to keep it QT only for now.
662016-09-14T06:52:40 <jonasschnelli> Moving the function RPCExecuteCommandLine(std::string &strResult, const std::string &strCommand) to a core class would be trivial.
672016-09-14T06:53:12 <jonasschnelli> We could either have a special RPC command that result in parsing and executing multiple nested commands...but I don't think this would be clever.
682016-09-14T06:53:25 <jonasschnelli> The Qt version is "client-side".
692016-09-14T06:53:40 <sipa> i'd have liked that, but it seems i was alone with that :)
702016-09-14T06:53:41 <jonasschnelli> A better approach for the RPC layer would be to add it into bitcoin-cli
712016-09-14T06:54:13 <jonasschnelli> Having it server-side i just fear some uncontrollable resource usage.
722016-09-14T06:54:32 <jonasschnelli> I think its better to start with it client-side QT only, and if it turns to be useful, add it to bitcoin-cli.
732016-09-14T06:54:33 <sipa> rpc already has uncontrollable resource usage
742016-09-14T06:54:40 <dcousens> Personally I just always used bash, but I never use the QT UI so *shrug*
752016-09-14T06:54:42 <sipa> fair enough :)
762016-09-14T06:55:16 <jonasschnelli> dcousens: Adding in in Qt has less of exposures... a better test-bed .:)
772016-09-14T06:55:26 <jonasschnelli> *it
782016-09-14T06:55:59 <dcousens> As for the RPC extension, I think a special RPC command, say a mechanism to chain calls - makes sense, would feel like the idea by wumpus, but won't break all our RPC implementations
792016-09-14T06:56:32 <dcousens> s/break/require changing
802016-09-14T06:57:16 <sipa> well if we turn it into an RPC thing, we'd arguably want more discussion about what the "query language" will look like
812016-09-14T06:57:17 <dcousens> as for resource usage, RPC is already meant to be a "safe space", haha
822016-09-14T06:57:33 <sipa> as it may be harder to make incompatible changes latee
832016-09-14T06:58:01 <dcousens> sipa: absolutely, the amount of time to get right may not be worth the extra line/rpc call for the few times this matters :P
842016-09-14T06:58:53 <jonasschnelli> The query language is pretty homebrew and non-standard..
852016-09-14T06:59:11 <jonasschnelli> Also,... once we have it in RPC, people are going to see it as part of the API...
862016-09-14T06:59:17 <jonasschnelli> changes at this point will probably more "difficult".
872016-09-14T06:59:35 <jonasschnelli> Lets find out through Qt how it could work in RPC. :)
882016-09-14T06:59:37 <dcousens> ultimately any bulk usage of the RPC ends up seeing the piped network latency being completely insignificant... and chaining the commands won't lower the JSON bulk
892016-09-14T06:59:59 <dcousens> unless their doing it over an open network I suppose
902016-09-14T07:00:03 <sipa> jonasschnelli: yes, agree there
912016-09-14T07:00:15 <wumpus> what did I break?
922016-09-14T07:00:39 <jonasschnelli> ;-)
932016-09-14T07:01:29 <sipa> wumpus: ?
942016-09-14T07:01:48 <wumpus> oh that, yes that's a simple extension to JSON batching that would allow chaining of return values
952016-09-14T07:03:13 <wumpus> don't know whether to propose it or not, I'm fairly sure their reply will be 'if you need this level of sophistication don't use JSON-RPC you noob'
962016-09-14T07:03:35 <sipa> who is they?
972016-09-14T07:03:59 <wumpus> the people who made the JSON RPC standard
982016-09-14T07:04:02 <dcousens> I think if the RTT time is more significant than the encoding of the data itself, and you want to optimize RPC, its very likely you have control over the node anyway, just write a small HTTP wrapper that does that over localhost for you/
992016-09-14T07:05:11 <dcousens> or hell... write a RPC command that does what you want
1002016-09-14T07:05:28 <sipa> hah, yes
1012016-09-14T07:05:34 <sipa> that's my approach usually :)
1022016-09-14T07:05:41 * sipa zZzZ
1032016-09-14T07:05:42 <dcousens> as is mine :S
1042016-09-14T07:05:59 <wumpus> the point is that we don't want the API clogged full of 'chain A and B' commands
1052016-09-14T07:06:07 <wumpus> but for a local patch sure
1062016-09-14T07:06:30 <wumpus> in the end I think no one really has any convincing use-cases for this
1072016-09-14T07:06:48 <dcousens> yup, we're all just lazy cause it means 1 more line
1082016-09-14T07:07:04 <jonasschnelli> I think the RPC layer is for app to app communication
1092016-09-14T07:07:15 <wumpus> I've had approximately zero replies on me JSON-RPC extension proposal, though I think it makes a lot of sense *if* people are seriously considering this
1102016-09-14T07:07:18 <jonasschnelli> The Qt console is supposed to be a human command line interface
1112016-09-14T07:07:53 *** OdicforceSounds has joined #bitcoin-core-dev
1122016-09-14T07:07:55 <wumpus> yes, the RPC layer is for app to app communication, it is an API
1132016-09-14T07:07:57 <jonasschnelli> The convenience of nested commands are probably on the client side
1142016-09-14T07:08:02 <dcousens> wumpus: I'm against the idea, just wasn't sure if the Qt change (#7783) was relevant
1152016-09-14T07:08:16 <wumpus> dcousens: huh?
1162016-09-14T07:08:28 <wumpus> no, qt has nothing to do with it
1172016-09-14T07:08:39 <wumpus> that' just a user convenience
1182016-09-14T07:08:47 <dcousens> indeed, hence ACK :)
1192016-09-14T07:09:14 <wumpus> I mean with bitcoin-cli you have full access to bash' scripting/variable munging/etc capabilities, in the Qt console you don't, so this adds a bit of shell power to it
1202016-09-14T07:09:21 <sipa> wumpus: oh, i had no idea that that chaining value idea was your proposal
1212016-09-14T07:09:48 *** rubensayshi has joined #bitcoin-core-dev
1222016-09-14T07:10:10 <gmaxwell> or the rpc console could just link in a lua/python/js engine, all of which are embedable.
1232016-09-14T07:10:14 <wumpus> this is a completely orthogonal concern from being able to chain values at the server side to avoid latency
1242016-09-14T07:10:22 <wumpus> gmaxwell: :-(
1252016-09-14T07:10:51 <gmaxwell> heh
1262016-09-14T07:10:54 <wumpus> no interpreters in bitcoin please
1272016-09-14T07:11:04 <wumpus> (apart from the one we really need)
1282016-09-14T07:11:06 <dcousens> no more*
1292016-09-14T07:11:32 <wumpus> having a powerful interpreter in the target executable makes exploitation so much easier
1302016-09-14T07:12:02 <dcousens> maybe the CLI could just have an interactive mode with a LUA shell or something?
1312016-09-14T07:12:04 <gmaxwell> well it could be a seperate interface, e.g. fancy-console.
1322016-09-14T07:12:14 *** OdicforceSounds has quit IRC
1332016-09-14T07:12:15 <dcousens> that way its not in the core as such
1342016-09-14T07:12:19 <wumpus> dcousens: yes, that has been proposed, we could jut have a python-based interactive CLI
1352016-09-14T07:12:24 <gmaxwell> I've only heard of five or six instances where the Js enable ethereum wallet stuff was used to trick people into sending away all their funds. :P
1362016-09-14T07:12:26 <dcousens> bitcoin-cli*, that is
1372016-09-14T07:12:47 <wumpus> dcousens: it could be based on ncurses, ipython, etc, that idea is old as the hills, yet no one wrote one :)
1382016-09-14T07:12:55 <gmaxwell> but yes, not having it in the main binary would be obvious. I dunno, I think the utility is all that great.
1392016-09-14T07:12:56 <wumpus> I thnk the current tools are jut ok
1402016-09-14T07:12:57 <dcousens> wumpus: but jonasschnelli wrote this :P
1412016-09-14T07:13:13 <wumpus> gmaxwell: right, it just doesn't matter that much, people like talking about this but it's no itch to scratch
1422016-09-14T07:13:19 <wumpus> dcousens: yes, he did, and it works
1432016-09-14T07:13:21 <wumpus> it's awesome
1442016-09-14T07:13:24 <dcousens> haha,indeed
1452016-09-14T07:13:41 <wumpus> let's not try to shed-paint it to death
1462016-09-14T07:14:01 <gmaxwell> the standard python bitcoin json library that people get linked to is not at all reliable (gets disconnected and then throws exceptions, and has to be wrapped to be usable) and I don't see anyone even complaining about it.
1472016-09-14T07:14:31 <wumpus> s/json/json-rpc and you're right
1482016-09-14T07:15:05 <dcousens> I've found it much easier to just always use a well-tested rpc library than use something "bitcoin tailored"
1492016-09-14T07:15:23 <wumpus> which one are you using for your projects?
1502016-09-14T07:15:25 <gmaxwell> json-rpc, yes.
1512016-09-14T07:15:50 <wumpus> are there any RPC mechanisms that don't suck?
1522016-09-14T07:16:34 <dcousens> wumpus: myself? as most of my work is in JS, I ended up using my own (haha) simply because all the batch interfaces sucked with error handling for well tested ones
1532016-09-14T07:16:40 <wumpus> this presentation talks about people using MQTT for bitcoin wallet control, among other things: https://media.defcon.org/DEF%20CON%2024/DEF%20CON%2024%20presentations/DEFCON-24-Lucas-Lundgren-Light-Weight%20Protocol-Critical-Implications.pdf of course, without authentication or encryption and on the public internet :)
1542016-09-14T07:17:11 <wumpus> tl.dr: don't do that
1552016-09-14T07:17:17 <luke-jr> [06:53:12] <jonasschnelli> We could either have a special RPC command that result in parsing and executing multiple nested commandsâ¦but I don't think this would be clever. <-- this would be useful if the RPC client was over a netwrok, to avoid round-trips and such
1562016-09-14T07:17:29 <dcousens> but, my point was, even my own implementation has nothing to do bitcoin as such, its just an RPC implementation
1572016-09-14T07:17:43 <jonasschnelli> Luke-Jr: Yes. The server-side nesting could save bandwidth...
1582016-09-14T07:18:17 <luke-jr> FWIW, I both dislike and like the idea at the same time. >_<
1592016-09-14T07:18:22 <gmaxwell> wumpus: What did we do in the embeded python json-rpc used in the tests to make it reliable? (or did we not and the test just run fast enough that it isn't an issue?)
1602016-09-14T07:18:28 <wumpus> server-side promise pipelining aves some bandwidth, but it mostly saves *latency*
1612016-09-14T07:18:45 <jonasschnelli> Example: with RPC nesting, you could get the first outputs value of the second transactions of the bestblock with just one command ---> bandwith: a couple of bytes:
1622016-09-14T07:19:05 <jonasschnelli> s/:/.
1632016-09-14T07:19:16 <luke-jr> it's easier to support server-side expressions when bitcoind is split from the consensus stuff someday
1642016-09-14T07:19:22 <wumpus> gmaxwell: which issue do you exactly mean?
1652016-09-14T07:19:42 <luke-jr> ("support" in a human manner, not technical)
1662016-09-14T07:20:34 <wumpus> gmaxwell: we did add some code to AuthServiceProxy._request to handle connection loss
1672016-09-14T07:21:16 <luke-jr> I think there's some RPC mechanism which does expressions already BTW
1682016-09-14T07:21:38 <wumpus> luke-jr: yes, it's old as the hills, "promise pipelining" is usually what it's called
1692016-09-14T07:21:47 <wumpus> luke-jr: capnproto does it for example
1702016-09-14T07:22:04 <wumpus> I think even CORBA could do it in the 90's, but don't look for that, it'll turn you crazy
1712016-09-14T07:22:36 <wumpus> XCB (X11's RPC protocol) can do it too
1722016-09-14T07:22:41 <gmaxwell> wumpus: varrious forms of connection loss; bitcoind either times out the persistant connection then any other interaction with the proxy object throws an exception, or I believe any case where bitcoind returns an error. I don't have a detailed list because I've always just soloved it by creating a wrapper that catches the exception and reconnects.
1732016-09-14T07:23:28 <wumpus> gmaxwell: the high-level problem is that AuthServiceProxy uses python's http mechanism in an unconventional way: it opens a connection to a HTTP server, and instead of immediately queuing a bunch of requests, it keeps the connection open
1742016-09-14T07:23:55 <wumpus> gmaxwell: which in principle is fine, web browsers also do that, but it needs code to handle the case where the remote server hung up on you
1752016-09-14T07:24:14 <wumpus> gmaxwell: stock AuthServiceProxy has no code for that, our version (in the tests) does
1762016-09-14T07:24:28 <luke-jr> it used to :x
1772016-09-14T07:25:29 <wumpus> no, it assumes a unwordly friendly RPC server that never disconnects. Which used to be the case, approximately, before the switch to evhttpd, which introduced (like any other HTTP server in the world) connection timeouts
1782016-09-14T07:25:53 <wumpus> (but network problems can *also* result in the http connection being closed, you just can't make the assumption it will always be there in the real world)
1792016-09-14T07:26:17 <dcousens> wumpus: fwiw, I occassionally sync an address index from genesis just to see how fast it goes, and its never the throughput/bandwidth that slows it up, its the disk thrashing, even in just a query-of txindex case
1802016-09-14T07:26:42 <dcousens> s/address index/script index
1812016-09-14T07:27:01 <gmaxwell> wumpus: yea, it was unreliable before-- just more obvious now.
1822016-09-14T07:27:10 <wumpus> dcousens: thanks for actually benchmarking to find bottlenecks instead of making assumptions :) ("it must be RPC overhead")
1832016-09-14T07:27:28 *** Cheeseo has quit IRC
1842016-09-14T07:27:45 <luke-jr> oh, I see what it was: the older version that I thought worked simply never reused the connection :|
1852016-09-14T07:28:07 <wumpus> luke-jr: that's also a valid way to do it, if you don't have too many requests or theyr're sparse
1862016-09-14T07:28:30 <wumpus> it's how people usually use python's http: just open a connection per requests. The whole keep-alive thing is an optimization, not required
1872016-09-14T07:28:42 <wumpus> it brings a lot of state and complexity
1882016-09-14T07:29:32 <luke-jr> yet another unmaintained jgarzik project: https://github.com/jgarzik/python-bitcoinrpc/pull/49
1892016-09-14T07:29:42 <gmaxwell> though without it, I imagine performance is poor (well even with it, performance is poor)
1902016-09-14T07:30:09 <gmaxwell> it stinks mostly because everywhere recommends it, and it falls flat on its face... and I'm not sure what to recommend instead.
1912016-09-14T07:30:54 <wumpus> well "performance is poor" is relative. keep-alive only helps with repeated requests. In many cases of using RPC you don't really care about performance of repeated reuqests at all
1922016-09-14T07:31:43 <wumpus> many people just use the RPC through curl, or bitcoin-cli, which involves executing an external process per connection per command, and they don't complain either
1932016-09-14T07:32:33 <dcousens> wumpus: heh, I'm using 16 concurrent RPC connections w/ 300 batched in each
1942016-09-14T07:32:38 <wumpus> I think the problem is recommending using a broken-ish) library when the underlying mechanism is so trivial
1952016-09-14T07:33:31 <dcousens> needless to say I found where my custom RPC commands were missing LOCKs pretty quick
1962016-09-14T07:33:37 <gmaxwell> right, but "walk through the last N blocks to gather data" taking longer in python than it takes me to modify bitcoind and recompile feels a little silly.
1972016-09-14T07:34:00 <wumpus> yes, if you're one of the few people that needs to do really dense requesting, there are various optimizations that help
1982016-09-14T07:34:07 <wumpus> but that's not what JSON RPC was designed for in the first place
1992016-09-14T07:34:31 <wumpus> it's a high-overhead protocol that was designed to be simple to use, not for low-latency high-throughput application
2002016-09-14T07:34:32 <dcousens> wumpus: my point was, the RPC isn't whats slow, so no point changing it
2012016-09-14T07:35:01 <wumpus> something like capnproto is designed for the latter
2022016-09-14T07:35:28 <wumpus> howeer it's much harder to use, bind, and have basic setup, which is why companie generally offer JSON RPC or REST APIs
2032016-09-14T07:35:43 <dcousens> lest we forget SOAP
2042016-09-14T07:35:50 <wumpus> heh :)
2052016-09-14T07:36:17 <gmaxwell> author of capnproto didn't seem to regard sending uninitilized memory in struct padding to remote peers as a serious security problem when I spoke to him a year or two ago.
2062016-09-14T07:37:00 <wumpus> yes, security is another thing that is usually compromised on when latency is the most important thing
2072016-09-14T07:37:19 <gmaxwell> in any case, after seeing that the python stuff was slow I tested with curl and it didn't appear to be the bitcoind side that was slow.
2082016-09-14T07:37:25 <wumpus> it's a difficult choice, I think JSON-RPC was a fairly good choice as these things go
2092016-09-14T07:37:41 <gmaxwell> I agree.
2102016-09-14T07:38:55 * jonasschnelli thinks we should fork to use CORBA as our bitcoin scripting language and SOAP as http API :}
2112016-09-14T07:39:04 <luke-jr> ._.
2122016-09-14T07:39:04 *** dermoth has quit IRC
2132016-09-14T07:39:13 <luke-jr> jonasschnelli: you forgot /s
2142016-09-14T07:39:38 <jonasschnelli> Ah... I forgot to migrate the Qt client to a chrome html/css extension. :)
2152016-09-14T07:39:46 <wumpus> the network and JSON processing on the bitcoind side is fast: the most common bottleneck would be the locking, which has nothing to do with the RPC mechanism. Or reading/deserializing blocks. Or...
2162016-09-14T07:40:03 <wumpus> jonasschnelli: hehe, maybe if bitcoin was designed 10 years earlier
2172016-09-14T07:40:09 <dcousens> jonasschnelli: I wonder when a ECDSA library will be written with pure CSS
2182016-09-14T07:40:18 <jonasschnelli> lol
2192016-09-14T07:40:32 <wumpus> dcousens: at least you can write a ECDSA library with just x86 mov instructions
2202016-09-14T07:40:34 <luke-jr> >_<
2212016-09-14T07:40:47 <luke-jr> https://github.com/xoreaxeaxeax/movfuscator
2222016-09-14T07:40:56 <wumpus> yea
2232016-09-14T07:41:10 <luke-jr> no need to write a new one, just build libsecp256k1..
2242016-09-14T07:41:50 <jonasschnelli> How large will it be?
2252016-09-14T07:41:59 <wumpus> moon-sized
2262016-09-14T07:42:41 <jonasschnelli> Would be fun open the moon-sized library in IDA-PRO
2272016-09-14T07:42:41 <luke-jr> dunno, never tried it
2282016-09-14T07:43:40 <wumpus> yes I think that's the only reason why anyone would do something like that, throw off people trying to analyze it 'this can't be real x86 code'
2292016-09-14T07:44:23 <luke-jr> s/people/software/
2302016-09-14T07:44:28 <luke-jr> eg virus scanners
2312016-09-14T07:44:39 <wumpus> well not just software, software is much easier to throw off
2322016-09-14T07:45:00 <luke-jr> is it? âº
2332016-09-14T07:45:13 <dcousens> jonasschnelli: its possible to de-movuscate,isn't it? in which case eh?
2342016-09-14T07:45:31 <luke-jr> dcousens: IIRC this one intentionally makes it hard
2352016-09-14T07:46:31 *** PatBoy has quit IRC
2362016-09-14T07:48:31 <wumpus> then again if the virus is 1000MB it kind of inhibits its own spread
2372016-09-14T07:49:21 *** PatBoy has joined #bitcoin-core-dev
2382016-09-14T07:54:00 *** MarcoFalke has joined #bitcoin-core-dev
2392016-09-14T08:00:51 *** laurentmt has joined #bitcoin-core-dev
2402016-09-14T08:02:29 *** laurentmt has quit IRC
2412016-09-14T08:03:55 *** dermoth has joined #bitcoin-core-dev
2422016-09-14T08:11:37 *** dermoth has quit IRC
2432016-09-14T08:13:20 *** dermoth has joined #bitcoin-core-dev
2442016-09-14T08:15:20 *** dermoth has quit IRC
2452016-09-14T08:16:47 <GitHub179> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/57b34599b2de...881d7eaf29f7
2462016-09-14T08:16:48 <GitHub179> bitcoin/master 36fa01f Cory Fields: net: only delete CConnman if it's been created...
2472016-09-14T08:16:48 <GitHub179> bitcoin/master 881d7ea Wladimir J. van der Laan: Merge #8715: net: only delete CConnman if it's been created...
2482016-09-14T08:17:02 <GitHub0> [bitcoin] laanwj closed pull request #8715: net: only delete CConnman if it's been created (master...fix-connman-shutdown) https://github.com/bitcoin/bitcoin/pull/8715
2492016-09-14T08:18:15 *** davec has quit IRC
2502016-09-14T08:30:10 *** davec has joined #bitcoin-core-dev
2512016-09-14T08:50:34 *** dermoth has joined #bitcoin-core-dev
2522016-09-14T08:51:25 *** dermoth has quit IRC
2532016-09-14T08:56:12 *** AaronvanW has joined #bitcoin-core-dev
2542016-09-14T08:57:13 *** Ylbam has joined #bitcoin-core-dev
2552016-09-14T09:01:52 *** dermoth has joined #bitcoin-core-dev
2562016-09-14T09:02:43 <jonasschnelli> Is there a reason why the last key in out HD scheme (and in BIP32) is non-hardened? Its m/0'/0'/0 for the first key and not m/0'/0'/0'
2572016-09-14T09:02:58 <jonasschnelli> Or does it just don't matter at the last level
2582016-09-14T09:24:25 <gmaxwell> because the wallet itself isn't intended to iterate at that level. The part below is subkeys.
2592016-09-14T09:24:49 <gmaxwell> So every key the wallet gives out is strong against unzipping, but it could support repeated payment arrangements without address reuse.
2602016-09-14T09:24:52 <gmaxwell> IIRC.
2612016-09-14T09:28:27 *** MarcoFalke has left #bitcoin-core-dev
2622016-09-14T09:34:46 *** netzin has joined #bitcoin-core-dev
2632016-09-14T09:36:09 *** netzin has quit IRC
2642016-09-14T09:40:35 *** MarcoFalke has joined #bitcoin-core-dev
2652016-09-14T09:56:03 <MarcoFalke> wumpus: Can you elaborate? (random.random() *is* deterministic)
2662016-09-14T09:56:13 <MarcoFalke> I think the seed is the current time, so we may not know that
2672016-09-14T09:56:22 <MarcoFalke> But knowing the seed is not required.
2682016-09-14T09:56:24 <wumpus> MarcoFalke: I like test that simply test the same thing every time
2692016-09-14T09:56:32 <wumpus> there is no need to randomize here
2702016-09-14T09:56:43 <MarcoFalke> So just set it to usehd=0
2712016-09-14T09:56:52 <wumpus> just make every odd bitcoind a legacy wallet and every even one a HD one, or so
2722016-09-14T09:57:55 <wumpus> I just don't see the point of random() here, there's a fair chance that it ends up testing all-hd or all-non-hd
2732016-09-14T10:01:36 <MarcoFalke> Which is a legit case to test
2742016-09-14T10:01:50 <MarcoFalke> But I can remove it, so we don't need the import random
2752016-09-14T10:01:53 <wumpus> sure, but not randomly
2762016-09-14T10:02:17 <wumpus> I like deterministic tests, esp in travis
2772016-09-14T10:02:31 <MarcoFalke> But you want them to be constant?
2782016-09-14T10:02:36 <wumpus> yes
2792016-09-14T10:03:01 <MarcoFalke> Have you seen the print(extra_args) to make it reproducible in case of failure?
2802016-09-14T10:03:05 <wumpus> otherwise there's a larger chance it will fail due to reasons unrelated to the code changes
2812016-09-14T10:03:12 <wumpus> which confuses people
2822016-09-14T10:03:32 *** rubensayshi has quit IRC
2832016-09-14T10:04:03 <wumpus> tests run in travis should preferably be constant and deterministic. I'm ok with a random fuzzing jackpot testing method for running locally, but the tests running for every pull should be the same tests every time
2842016-09-14T10:07:03 <wumpus> if you want to test with a combination of legacy and HD wallets in a test that's good, but just hardcode that
2852016-09-14T10:09:20 <MarcoFalke> fine
2862016-09-14T10:09:44 <MarcoFalke> Then we should write some new tests which do "parameter fuzzing" some time
2872016-09-14T10:10:05 <MarcoFalke> Sadly this depends somewhat on rewrinting the arg parser
2882016-09-14T10:10:23 <wumpus> sorry that I feel so strongly about htis, but I've seen too many random travis failures in the last months :)
2892016-09-14T10:15:27 *** fengling has joined #bitcoin-core-dev
2902016-09-14T10:37:58 *** laurentmt has joined #bitcoin-core-dev
2912016-09-14T10:38:07 *** laurentmt has quit IRC
2922016-09-14T10:55:38 *** Samdney has left #bitcoin-core-dev
2932016-09-14T10:58:38 *** mol has joined #bitcoin-core-dev
2942016-09-14T10:59:40 <wumpus> cfields_: oh shit oh shit oh shit https://github.com/bitcoin/bitcoin/pull/8653#issuecomment-246973266
2952016-09-14T11:00:21 <wumpus> bitcoin core win64 cross-build is in a state of crap with ubuntu 16.04
2962016-09-14T11:01:04 <wumpus> I thought I did this before, and didn't have any of these issues, but I must hae imagined it
2972016-09-14T11:02:09 *** molz has quit IRC
2982016-09-14T11:06:12 <GitHub17> [bitcoin] andersoyvind opened pull request #8720: Minor change in section name (master...patch-1) https://github.com/bitcoin/bitcoin/pull/8720
2992016-09-14T11:39:18 *** JackH has joined #bitcoin-core-dev
3002016-09-14T11:44:17 *** cryptapus has joined #bitcoin-core-dev
3012016-09-14T11:44:17 *** cryptapus has joined #bitcoin-core-dev
3022016-09-14T12:13:41 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3032016-09-14T12:29:10 *** Chris_Stewart_5 has quit IRC
3042016-09-14T12:30:40 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3052016-09-14T12:35:45 *** cryptapus has quit IRC
3062016-09-14T13:00:41 *** Alina-malina has joined #bitcoin-core-dev
3072016-09-14T13:12:28 <GitHub88> [bitcoin] laanwj opened pull request #8722: bitcoin-cli: More detailed error reporting (master...2016_09_cli_http_error) https://github.com/bitcoin/bitcoin/pull/8722
3082016-09-14T13:22:41 <GitHub10> [bitcoin] jonasschnelli opened pull request #8723: [Wallet] Add support for flexible BIP32/HD keypath-scheme (master...2016/09/hd_flex) https://github.com/bitcoin/bitcoin/pull/8723
3092016-09-14T13:22:53 * jonasschnelli thinks he should review Luke-Jr multiwallet PR
3102016-09-14T13:23:22 <wumpus> jonasschnelli: whatevery you do don't look at #8653
3112016-09-14T13:24:41 <jonasschnelli> wumpus: hehe.. I did already and started disable --hardening wherever I can to avoid "possible startup problems". :P
3122016-09-14T13:25:24 <wumpus> my new favourite number is "zu"
3132016-09-14T13:26:11 <jonasschnelli> :-)
3142016-09-14T13:26:18 <wumpus> I think it's that posix mode of gcc that completely throws everything off track
3152016-09-14T13:26:35 <wumpus> (on windows)
3162016-09-14T13:27:17 <wumpus> I'm going to locally revert the usage of c++11 threads, compile with the -win32 compiler then see what happens, I think everything will be fine
3172016-09-14T13:27:48 *** Cheeseo has joined #bitcoin-core-dev
3182016-09-14T13:27:48 *** Cheeseo has joined #bitcoin-core-dev
3192016-09-14T13:29:10 *** cryptapus has joined #bitcoin-core-dev
3202016-09-14T13:29:11 *** cryptapus has joined #bitcoin-core-dev
3212016-09-14T13:32:17 <jonasschnelli> wumpus: Not sure if this helps.. but in another project I'm using mingw.thread.h (https://github.com/digitalbitbox/dbb-app/blob/master/src/mingw/mingw.thread.h)
3222016-09-14T13:32:41 *** Chris_Stewart_5 has quit IRC
3232016-09-14T13:33:20 <jonasschnelli> Then just include it with a #ifdef WIN32
3242016-09-14T13:33:24 <jonasschnelli> Probably a huge hack
3252016-09-14T13:34:17 *** Giszmo has joined #bitcoin-core-dev
3262016-09-14T13:36:06 *** Guyver2 has joined #bitcoin-core-dev
3272016-09-14T13:49:49 <wumpus> jonasschnelli: that makes sense; but I don't get it, why isn't it part of mingw-w64 itself?
3282016-09-14T13:50:35 <wumpus> well it's less of a hack than fully fledged POSIX emulation mode on windows
3292016-09-14T13:54:16 <wumpus> it's supposed to just support it, and on trusty it doe
3302016-09-14T13:54:28 <wumpus> (it=std::mutex and friends)
3312016-09-14T13:56:15 <jonasschnelli> wumpus: Yes. I don't know why its not part of mingw itself... I just added them and it worked fine, though, I'm not using stuff like the --enable-hardening we do at Core
3322016-09-14T13:56:53 <wumpus> I doubt this has anything to do with hardening, I think with hardening it just detects the crazy memory corruption that happens sooner
3332016-09-14T13:57:48 <GitHub93> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/881d7eaf29f7...a82e5d8220bb
3342016-09-14T13:57:48 <GitHub93> bitcoin/master 1111ddb MarcoFalke: gitignore: Remove unused lines
3352016-09-14T13:57:49 <GitHub93> bitcoin/master a82e5d8 MarcoFalke: Merge #8714: [qa] gitignore: Remove unused lines...
3362016-09-14T13:58:03 <GitHub7> [bitcoin] MarcoFalke closed pull request #8714: [qa] gitignore: Remove unused lines (master...Mf1609-qaUnused) https://github.com/bitcoin/bitcoin/pull/8714
3372016-09-14T13:58:43 <wumpus> also there's a mutex header in ./lib/gcc/x86_64-w64-mingw32/5.3-win32/include/c++/mutex
3382016-09-14T13:58:56 <wumpus> maybe we need to #define something
3392016-09-14T13:58:57 *** AaronvanW has quit IRC
3402016-09-14T13:59:33 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3412016-09-14T14:00:33 <wumpus> well at least that's one more 'trivial' I've removed from a pull title
3422016-09-14T14:02:43 <jonasschnelli> heh.. yes. That happens quick..
3432016-09-14T14:05:32 <MarcoFalke> Hmm. Github allows us to write to branches that have a pull request open against bitcoin.
3442016-09-14T14:05:34 <MarcoFalke> c.f. https://github.com/bitcoin/bitcoin/pull/8720/commits
3452016-09-14T14:05:48 <MarcoFalke> Might come in handy but I'd prefer to disable it
3462016-09-14T14:05:57 <MarcoFalke> (if possible)
3472016-09-14T14:08:07 *** AaronvanW has joined #bitcoin-core-dev
3482016-09-14T14:08:20 *** Chris_St1 has joined #bitcoin-core-dev
3492016-09-14T14:09:12 <jonasschnelli> MarcoFalke: so you did amend commit force push to 8720?
3502016-09-14T14:09:24 <MarcoFalke> Jup: https://github.com/blog/2247-improving-collaboration-with-forks
3512016-09-14T14:09:34 <MarcoFalke> Apparently not possible to disable
3522016-09-14T14:09:39 <jonasschnelli> Heh... that's handy but evil.
3532016-09-14T14:10:47 *** AaronvanW_ has joined #bitcoin-core-dev
3542016-09-14T14:10:58 *** Chris_Stewart_5 has quit IRC
3552016-09-14T14:14:10 *** AaronvanW has quit IRC
3562016-09-14T14:15:04 *** AaronvanW__ has joined #bitcoin-core-dev
3572016-09-14T14:18:15 *** AaronvanW_ has quit IRC
3582016-09-14T14:20:41 <wumpus> that can be really useful
3592016-09-14T14:21:03 <wumpus> was of course already possible to do that locally, but ok
3602016-09-14T14:21:11 <jonasschnelli> Yes. Thinking of all there cases where a nit holds back a PR
3612016-09-14T14:21:52 <jonasschnelli> I think its much better to have a more public way then locally alter during the merge
3622016-09-14T14:22:09 <wumpus> before the merge*
3632016-09-14T14:24:17 <wumpus> if you do a local merge you can simply add your own commits before doing it, that's pretty public
3642016-09-14T14:24:43 <jonasschnelli> It's public, but only in the commit list and not on the PR
3652016-09-14T14:25:16 <wumpus> right. Just wanted to be clear I didn't mean anything as sneaky as squashing it into the merge commit, or into the author's commits
3662016-09-14T14:25:24 <MarcoFalke> Also quite fiddly trying to emulate github-merge.py
3672016-09-14T14:25:41 <MarcoFalke> I think it is fine to use the new feature when we announce in in the pr that is affected
3682016-09-14T14:25:43 <wumpus> yes I've always intended to add a local mode to github-merge.py
3692016-09-14T14:25:57 <wumpus> (I have a local version of github-merge but it's really messy)
3702016-09-14T14:33:01 <GitHub197> [bitcoin] MarcoFalke opened pull request #8724: [qa] walletbackup: Sync blocks inside the loop (master...Mf1609-qaWalletBackupFix) https://github.com/bitcoin/bitcoin/pull/8724
3712016-09-14T14:39:35 *** Chris_St1 has quit IRC
3722016-09-14T14:55:07 <jonasschnelli> windows_hell, nice branch name
3732016-09-14T14:56:10 * jonasschnelli wishes we could include a ubuntu vm and the linux binaries for the windows package
3742016-09-14T14:56:39 *** Chris_St1 has joined #bitcoin-core-dev
3752016-09-14T14:57:19 <wumpus> well there seems to be a shitload of issues, for now the best recommendation would be to just use trusty. The "zu" issue doesn't exist with the gitian executables does it?
3762016-09-14T14:58:43 <wumpus> let's first try to figure out why the tests crash...
3772016-09-14T15:01:36 *** AaronvanW__ is now known as AaronvanW
3782016-09-14T15:01:37 <jonasschnelli> Yes. We should probably flag certain distros not compatible with out cross compile process or so
3792016-09-14T15:08:30 *** Chris_St1 has quit IRC
3802016-09-14T15:16:17 <wumpus> it only seemed like bitcoin-qt was working: I had disabled wallet support, apparently it works then
3812016-09-14T15:16:33 <wumpus> windows is truly a mess on gcc 5.3
3822016-09-14T15:16:58 <wumpus> the test crash before it even get to the BasicTestingSetup constructor
3832016-09-14T15:17:06 <wumpus> at least I can reproduce it on wine....
3842016-09-14T15:18:39 <achow101> Is this windows problem the same thing as I was having a while back?
3852016-09-14T15:21:48 *** Chris_St1 has joined #bitcoin-core-dev
3862016-09-14T15:31:25 *** Chris_St1 has quit IRC
3872016-09-14T15:34:14 *** Chris_St1 has joined #bitcoin-core-dev
3882016-09-14T15:41:28 <wumpus> there are at least three seperate issues here
3892016-09-14T15:42:18 <achow101> the thing with mutex and not compiling
3902016-09-14T15:42:22 <wumpus> 1) the std::mutex/thread c++11 compilation issue 2) something trange with libevent (the 'zu' issue) 3) something with hardening
3912016-09-14T15:43:07 <wumpus> some of these may partially overlap in cause or effect
3922016-09-14T15:43:12 <wumpus> but it's a mess
3932016-09-14T15:45:49 <achow101> Yeah, it looks like the same problem, it happened on ubuntu 15.10 (wily)
3942016-09-14T15:47:01 <achow101> This is the discussion if it's of any help: https://botbot.me/freenode/bitcoin-core-dev/2016-08-08/?msg=70975840&page=1
3952016-09-14T15:48:12 <wumpus> right - it's still an issue on 16.04
3962016-09-14T15:50:29 *** dermoth has quit IRC
3972016-09-14T15:51:50 *** achow101 has quit IRC
3982016-09-14T15:52:11 *** achow101 has joined #bitcoin-core-dev
3992016-09-14T15:57:18 *** spudowiar has joined #bitcoin-core-dev
4002016-09-14T16:00:21 *** Guyver2 has quit IRC
4012016-09-14T16:01:49 *** MarcoFalke has left #bitcoin-core-dev
4022016-09-14T16:07:29 *** Chris_St1 has quit IRC
4032016-09-14T16:11:15 <wumpus> ok, phew, at least verified that the libevent `zu` issue doesn't exist in the released 0.13.0 executables
4042016-09-14T16:15:24 <sipa> good
4052016-09-14T16:16:01 <wumpus> well travis RPC tests pass on windows, so it'd be surprising, but I don't understand why it suddenly starts happening with a certain gcc version
4062016-09-14T16:16:51 <wumpus> the libevent version didn't change since 0.13.0, neither did our use of it
4072016-09-14T16:20:51 *** spudowiar has quit IRC
4082016-09-14T16:23:25 *** Chris_St1 has joined #bitcoin-core-dev
4092016-09-14T16:28:17 *** aalex has quit IRC
4102016-09-14T16:32:12 *** aalex has joined #bitcoin-core-dev
4112016-09-14T16:32:58 <wumpus> achow101: which patches? I've also been testing w/ Ubuntu 16.04 and windows 10
4122016-09-14T16:37:21 *** neha has quit IRC
4132016-09-14T16:39:18 *** neha has joined #bitcoin-core-dev
4142016-09-14T16:53:03 <GitHub12> [bitcoin] rebroad opened pull request #8725: Split debug for estimatefee into {estimatefee,2} (master...MoreGranularDebug2) https://github.com/bitcoin/bitcoin/pull/8725
4152016-09-14T16:53:21 *** spudowiar has joined #bitcoin-core-dev
4162016-09-14T16:56:33 <GitHub174> [bitcoin] rebroad opened pull request #8726: Move a bunch of fairly verbose debug messages from mempool to mempool2 (master...MoreGranularDebug3) https://github.com/bitcoin/bitcoin/pull/8726
4172016-09-14T16:59:29 *** Guyver2 has joined #bitcoin-core-dev
4182016-09-14T17:01:45 <GitHub66> [bitcoin] rebroad opened pull request #8727: Move logic for TX INVs together (master...MoreGranularDebug4) https://github.com/bitcoin/bitcoin/pull/8727
4192016-09-14T17:05:23 <GitHub85> [bitcoin] rebroad opened pull request #8728: More granular debug5 (master...MoreGranularDebug5) https://github.com/bitcoin/bitcoin/pull/8728
4202016-09-14T17:18:44 <GitHub29> [bitcoin] rebroad opened pull request #8729: More granular debug6 (master...MoreGranularDebug6) https://github.com/bitcoin/bitcoin/pull/8729
4212016-09-14T17:31:56 *** dermoth has joined #bitcoin-core-dev
4222016-09-14T17:32:38 *** rebroad has joined #bitcoin-core-dev
4232016-09-14T17:34:16 <GitHub64> [bitcoin] laanwj opened pull request #8730: depends: Add libevent compatibility patch for windows (master...2016_09_libevent_windows_gcc_531) https://github.com/bitcoin/bitcoin/pull/8730
4242016-09-14T17:35:39 <wumpus> sheesh how many debug pulls do we need...
4252016-09-14T17:36:53 <paveljanik> 8(
4262016-09-14T17:37:11 <paveljanik> 6+1. The last one combines all prev 8)
4272016-09-14T17:37:48 <wumpus> we really should add a suggestion not to spam pull requests
4282016-09-14T17:39:02 <wumpus> especially if they're all related
4292016-09-14T17:48:22 <paveljanik> well, I know how he feels. Sometimes you know it is much easier to make it in steps. But doing steps separately and then all in one PR is nonsense...
4302016-09-14T17:49:31 <paveljanik> let's close the steps PR and comment only the final one.
4312016-09-14T17:49:49 *** Chris_St1 has quit IRC
4322016-09-14T17:50:23 <paveljanik> rebroad, is it ok for you?
4332016-09-14T17:56:43 <rebroad> paveljanik, hi
4342016-09-14T17:57:37 <rebroad> paveljanik, I find the whole process quite confusing... in my experience my smaller pull requests get merged but the larger ones don't... so this way I'm trying to hedge my bets and provide small ones and a larger one to work out which is preferred.
4352016-09-14T17:58:03 <rebroad> paveljanik, not sure what you mean by "the final one"
4362016-09-14T17:58:24 <paveljanik> the final one - the one containing all (almost) the previous
4372016-09-14T17:58:27 <rebroad> do you mean the one with all the commits in? and close the others?
4382016-09-14T17:58:32 <cfields_> wumpus: ok, diving into the win32 mess. I'd written it off as user error, but looks like I need to familiarize myself with what's going on now.
4392016-09-14T17:58:54 <paveljanik> 8729
4402016-09-14T17:58:55 <rebroad> paveljanik, I'm happy to do this, but I'm not sure if this is would the other devs would prefer also
4412016-09-14T17:59:32 <paveljanik> well, some changes are wrong, yes, but they can tell you in one PR.
4422016-09-14T17:59:54 <rebroad> paveljanik, wrong?
4432016-09-14T18:00:09 <paveljanik> yes
4442016-09-14T18:00:20 <paveljanik> lets wait for comments
4452016-09-14T18:00:41 <paveljanik> I have already added approx. 5 comments.
4462016-09-14T18:00:52 <rebroad> paveljanik, ah, thank you
4472016-09-14T18:01:00 <GitHub154> [bitcoin] rebroad opened pull request #8731: Debug headers received ("block" for new block announcement, "block2" ⦠(master...DebugHeadersReceived) https://github.com/bitcoin/bitcoin/pull/8731
4482016-09-14T18:01:32 <rebroad> paveljanik, is there a way to easily go to my pull requests that have had comments left? i.e. is there an inbox for these comments somewhere on github?
4492016-09-14T18:02:43 <paveljanik> rebroad, yes, please read https://help.github.com/articles/about-discussions-in-issues-and-pull-requests/
4502016-09-14T18:02:58 <rebroad> thanks
4512016-09-14T18:03:04 <paveljanik> and especially "Further reading"
4522016-09-14T18:04:32 <rebroad> paveljanik, what did you mean by "why?" in #8728?
4532016-09-14T18:07:04 <paveljanik> expanded there...
4542016-09-14T18:08:28 <rebroad> paveljanik, #8727 you asked what other inv.type I can see.. not sure why you are asking this
4552016-09-14T18:09:06 <rebroad> I just moved the code that was doing the same thing as the code above into the same if statement.. which is needed to avoid messier code with the later commits also
4562016-09-14T18:11:42 <wumpus> this is starting to be annoying, please stop pushing debug pull requests
4572016-09-14T18:11:54 <rebroad> paveljanik, #8725 have added a description now... I guess the "why" is a reasonable question given it was absent.
4582016-09-14T18:12:17 <rebroad> wumpus, what is your issue with debug pull requests?
4592016-09-14T18:12:17 <wumpus> most debugging information is only intersting for yourself during a debugging session, there is no reason to merge them upstream
4602016-09-14T18:12:39 <paveljanik> there are many of them and useless - wumpus wrote it above...
4612016-09-14T18:12:39 <wumpus> a) there are way too many already b) it's mostly ego-commits, not intersting to other people
4622016-09-14T18:13:28 <rebroad> wumpus, from the pull requests I have seen I would say that various developers have found a number of my debug pull requests useful, and they facilitate better understanding of what is happening. It would be unreasonable to expect everyone to have to make their own changed to obtain useful debug information.
4632016-09-14T18:13:49 <wumpus> try to be focused when you submit pulls upsteam. Every one of us likely has tons of patches adding personal debugging information to solve specific problems, but it's not in general useful for others and you're wasting review time
4642016-09-14T18:14:46 <wumpus> it wouldn't be a problem if this was one or two pulls, but you keep pushing this on
4652016-09-14T18:15:02 <rebroad> wumpus, i just spend 3 days having to rebase all these debug commits due to changes that were made recently.. I am getting very very fed up with the amount of time wasted rebasing when I could be spending that time adding functionality to bitcoin. this is why this needs to get merged!
4662016-09-14T18:15:08 <wumpus> do you really expect people to review "more granular debug6"?
4672016-09-14T18:15:20 <rebroad> i have a life, which I'd like to spend living rather than rebasing
4682016-09-14T18:15:40 <wumpus> that's not a reason to merge it though, to merge it it needs to actually be considered useful, the project doesn't exist to save you work
4692016-09-14T18:15:44 <wumpus> we all have a life
4702016-09-14T18:15:59 <wumpus> it's kind of insulting to suggest you're the only one
4712016-09-14T18:16:01 <paveljanik> so why do you spend your time with useless stuff instead of The functionality?
4722016-09-14T18:16:05 <wumpus> and you can just burden us with your shit
4732016-09-14T18:16:12 <rebroad> wumpus, it's not just saving me work. I wouldn't write it if I thought it was only useful for me
4742016-09-14T18:16:34 <rebroad> i agree that some of the commits are trivial...
4752016-09-14T18:16:42 <rebroad> wumpus, what do you mean by "ego-commits"?
4762016-09-14T18:16:42 <wumpus> well some of them may be useful to others
4772016-09-14T18:16:47 <wumpus> but it's the sheer volume
4782016-09-14T18:17:08 <wumpus> everyone tries to make time to write and review patches that add functionality and fix bugs
4792016-09-14T18:17:10 <wumpus> not just debugging spam
4802016-09-14T18:17:48 <wumpus> <paveljanik> so why do you spend your time with useless stuff instead of The functionality? <- very good question
4812016-09-14T18:18:07 <wumpus> we have 402 issues and 140 pull requests open at this moment
4822016-09-14T18:18:26 <paveljanik> yup, pick one of the issues and try to solve it!
4832016-09-14T18:18:39 <wumpus> lots of actual problems that need to be solved, that are experienced by users
4842016-09-14T18:18:56 <rebroad> wumpus, you say sheer volume, but I'm not sure what you mean. do you mean pull reqs, or lines of code?
4852016-09-14T18:19:00 <wumpus> and yes, some of them may be bullshit, but let's try not to add too much more
4862016-09-14T18:19:09 <wumpus> rebroad: number of pull requests inthis case
4872016-09-14T18:19:35 <wumpus> (same could hold for lines of changed code, in some cases)
4882016-09-14T18:20:26 <rebroad> wumpus, ok, I will close the smaller ones as paveljanik suggests
4892016-09-14T18:20:41 <wumpus> yes just merge some together if they are related to the same concern
4902016-09-14T18:20:56 <wumpus> and at least make some effort to name/describe them
4912016-09-14T18:21:02 <wumpus> 'more granular debug5' really sucks as name
4922016-09-14T18:21:22 <GitHub9> [bitcoin] rebroad closed pull request #8484: More granular debug (master...MoreGranularDebug) https://github.com/bitcoin/bitcoin/pull/8484
4932016-09-14T18:21:26 <rebroad> wumpus, you don't mince your words, do you :)
4942016-09-14T18:21:39 <wumpus> rebroad: sorry this has been annoying me for a while
4952016-09-14T18:22:01 <rebroad> spending 3 days rebasing has been annoying me too ...
4962016-09-14T18:22:07 <wumpus> well then don
4972016-09-14T18:22:08 *** Guyver2_ has joined #bitcoin-core-dev
4982016-09-14T18:22:08 <wumpus> 't
4992016-09-14T18:22:37 <rebroad> if i need to understand the code and how it functions I do... unless these commits get merged
5002016-09-14T18:22:57 <rebroad> ok, I am oversimplifying a little
5012016-09-14T18:23:06 <rebroad> but it all helps, IMHO
5022016-09-14T18:23:08 <wumpus> you can understand code without logging everything, or by adding temporary logging
5032016-09-14T18:23:17 <wumpus> or by stepping through with a debugger
5042016-09-14T18:23:27 <wumpus> many ways taht don't involve adding logging statements to the upstream code
5052016-09-14T18:23:41 <rebroad> part of me needs to disagree with you as otherwise I've spent a lot of time for very little reason :-s
5062016-09-14T18:24:16 <wumpus> I'm not saying it's all useless
5072016-09-14T18:24:41 *** Guyver2 has quit IRC
5082016-09-14T18:24:42 *** Guyver2_ is now known as Guyver2
5092016-09-14T18:24:43 <rebroad> there are some more commits yet to come that might help to reveal the usefulness of the so far seemingly useless ones
5102016-09-14T18:24:45 <wumpus> just try to have some discipline, spend time explaining to others why something is useful, instead of just adding pulls with very little description
5112016-09-14T18:26:25 <rebroad> but yes, the mempool2 and estimatefee2 ones were a bit trivial... estimatefee2 more so... but still useful to an extent... I'd still like to know what you mean by ego-commits... i do wonder how much ego is behind my struggle to get things merged.
5122016-09-14T18:26:51 <wumpus> and maybe explain the overall plan, people get lost if they only see details instead of where something is going
5132016-09-14T18:26:58 <rebroad> wumpus, you are right.. i do need to be less lazy when it comes to explaining
5142016-09-14T18:27:35 <rebroad> wumpus, thank you for your feedback and patience
5152016-09-14T18:27:38 <wumpus> an ego-commit is something that is useful to yourself, for example some feature that you need, or some piece of information that you specifically need, but without regard whether it's useful for other users
5162016-09-14T18:28:13 <rebroad> wumpus, I would find it hard to believe that some features are useful to only one person
5172016-09-14T18:28:14 <wumpus> and then to stop having to rebase it yourself you try to push it upstream
5182016-09-14T18:29:05 <wumpus> well it tends to happen if something is only a perceived need, someone is really held up about something
5192016-09-14T18:29:21 <rebroad> there seems to be a general attitude among many developers that "debug" commits are a waste of time.. I don't quite understand why that is the popular viewpoint
5202016-09-14T18:29:35 <wumpus> maybe it's useful to one other person :) but all code needs to be maintained, so there has to be some threashold
5212016-09-14T18:29:49 <wumpus> well debugging is something kind of personal to developers
5222016-09-14T18:29:55 <wumpus> what information do you need
5232016-09-14T18:29:57 <wumpus> for what you're doing
5242016-09-14T18:30:10 <wumpus> for the problem you're trying to find
5252016-09-14T18:30:24 <rebroad> a lot of people, non-developers especially often ask for more feedback on what their full-node is doing... they want to see what it is doing...
5262016-09-14T18:30:38 <rebroad> so in a sense, this debug info might benefit non-developers more than developers
5272016-09-14T18:30:40 <wumpus> e.g. to find a wine issue I"m now instrumenting some code to log every executed instruction
5282016-09-14T18:30:51 <wumpus> that's not something anyone would ever want upstream :-)
5292016-09-14T18:31:03 <rebroad> i could code something prettier like a matrix style green scrolly thing.. but that would move away from actual useful information then!
5302016-09-14T18:31:30 <wumpus> if I became really obsessed about this, and wanted this a default part of bitcoin, and became angry if other developers disagreed, that would be an ego commit
5312016-09-14T18:32:14 <wumpus> yes on a high level I agree with you, it'd be nice to have some more insight into what a node is doing
5322016-09-14T18:32:28 <wumpus> I kind of like statoshi's approach
5332016-09-14T18:33:07 <wumpus> but I'm not sure e.g. splitting up debug into a zillion categories, or adding lots of messages, is the way to go
5342016-09-14T18:33:29 <wumpus> haha if you'd add that to the GUI it may well be accepted
5352016-09-14T18:33:42 <rebroad> if it's granular enough, then no one should need to add adhoc debug message as you are suggesting... there'd be a debug message already there that they can activate just by editing bitcoin.conf
5362016-09-14T18:33:44 <wumpus> jonasschnelli did a screensaver-like thing once that showed statistics
5372016-09-14T18:33:49 <rebroad> and they'd not need to recompile etc
5382016-09-14T18:33:57 <rebroad> which wastes a lot of cpy, energy, etc...
5392016-09-14T18:33:59 <rebroad> cpu
5402016-09-14T18:34:00 <wumpus> but gave up on it unfortuantely
5412016-09-14T18:34:12 <jonasschnelli> https://github.com/bitcoin/bitcoin/pull/5896
5422016-09-14T18:34:27 <rebroad> :)
5432016-09-14T18:34:28 <jonasschnelli> I haven't gave it up... I just came with a more stable solution for a first start
5442016-09-14T18:34:30 <jonasschnelli> The mempool stats
5452016-09-14T18:34:40 <rebroad> I love the mempool graphs..
5462016-09-14T18:34:41 <jonasschnelli> Once this will be merged, more can be added
5472016-09-14T18:34:56 <jonasschnelli> Nodes / Tx / Mempool just don't fit in one screem
5482016-09-14T18:34:57 <rebroad> although it would be great if I could point to the graph and it would tell me the X-Y co-ords of the point closest to the pointer
5492016-09-14T18:34:59 <kanzure> not clear to me if rebroad has used gdb and code stepping
5502016-09-14T18:35:01 <wumpus> well i think there's always someone that wants to add an ad-hoc debug messgae, I don't think it's possible to be complete in that regard
5512016-09-14T18:35:44 <kanzure> if there needs to be metrics for user display then probably a more general frmaewokr would be useful, rather than individual debugger additions
5522016-09-14T18:35:46 <rebroad> kanzure, i have used gdb.. but i think it's of limited use, in my experience and not as clear as some good debug messages, imho
5532016-09-14T18:36:01 <wumpus> the linux kernel has something called 'tracepoints' which allows inserting print or other hooks at certain points to print information on the fly
5542016-09-14T18:36:06 <wumpus> I think a similar thing exists for user space
5552016-09-14T18:36:53 <kanzure> wumpus: probably there's a very noisy compiler option for this...
5562016-09-14T18:36:56 <wumpus> most annoying about gdb is <value optimized out>
5572016-09-14T18:37:45 <GitHub167> [bitcoin] rebroad closed pull request #8728: More granular debug5 (master...MoreGranularDebug5) https://github.com/bitcoin/bitcoin/pull/8728
5582016-09-14T18:38:04 <kanzure> -finstrument-functions
5592016-09-14T18:38:06 <kanzure> hehehe
5602016-09-14T18:39:16 <wumpus> another one is systemtap probe points
5612016-09-14T18:39:53 <GitHub28> [bitcoin] rebroad closed pull request #8725: Split debug for estimatefee into {estimatefee,2} (master...MoreGranularDebug2) https://github.com/bitcoin/bitcoin/pull/8725
5622016-09-14T18:40:08 <GitHub62> [bitcoin] rebroad closed pull request #8726: Move a bunch of fairly verbose debug messages from mempool to mempool2 (master...MoreGranularDebug3) https://github.com/bitcoin/bitcoin/pull/8726
5632016-09-14T18:40:22 <kanzure> cool.
5642016-09-14T18:40:28 <GitHub25> [bitcoin] rebroad closed pull request #8727: Move logic for TX INVs together (master...MoreGranularDebug4) https://github.com/bitcoin/bitcoin/pull/8727
5652016-09-14T18:41:02 <wumpus> or gdb's "The dynamic printf command dprintf combines a breakpoint with formatted printing of your program's data to give you the effect of inserting printf calls into your program on-the-fly, without having to recompile it."
5662016-09-14T18:41:26 <rebroad> someone on github mentioned that everytime i change the pull title, or close/open that 1200 emails get sent... is this true?
5672016-09-14T18:41:49 <achow101> rebroad: yes.
5682016-09-14T18:41:51 <wumpus> well I think 1200 is kind of overstating it
5692016-09-14T18:42:03 <rebroad> if so.... this sounds a little verbose... i agree with paveljanik's comment that things can be concise to one person while verbose to another
5702016-09-14T18:42:23 <rebroad> which is why I felt more granular was the answer..
5712016-09-14T18:42:27 <kanzure> it's the same thing with mailing lists too. your email will be sent to many thousands of people.
5722016-09-14T18:42:27 <wumpus> but at least the maintainers and people that have posted to an issue get a notification
5732016-09-14T18:42:43 <achow101> wumpus: I don't think it is actually overstating it. Anyone who watches the repo will get an email and there are 1206 watchers
5742016-09-14T18:42:47 <kanzure> and if everyone is reading their email then a 5 minute email ~= economic catastrophe of time spent reading
5752016-09-14T18:43:29 <wumpus> achow101: I'm not sure watchers get a mail for everything that happens in a repostiory, but you could be right
5762016-09-14T18:43:47 <kanzure> wumpus: are you a watcher?
5772016-09-14T18:44:00 <achow101> I'm watching and I get mails for every comment and commit to every pr or issue
5782016-09-14T18:44:20 <kanzure> i sort of assumed all the regulars were using the watchers feature.
5792016-09-14T18:44:42 <kanzure> ah maybe it's something about org members on github. nevermind.
5802016-09-14T18:44:46 <wumpus> kanzure: I'm repo owner, so I get a lot of mail from github, but I didn't realize watchers get all that too
5812016-09-14T18:44:51 <rebroad> paveljanik, was it you who sent me a github article earlier on here? i am struggling to find the link you quoted earlier :-s
5822016-09-14T18:45:12 <paveljanik> rebroad, see topic, it contains a link to log of this channel...
5832016-09-14T18:45:13 <kanzure> you want https://help.github.com/articles/about-discussions-in-issues-and-pull-requests/
5842016-09-14T18:45:19 <wumpus> I don't amange to read most of it though
5852016-09-14T18:45:21 *** Guyver2_ has joined #bitcoin-core-dev
5862016-09-14T18:45:59 <wumpus> still looking for a way to filter when someone @'s me from the rest of the github spam
5872016-09-14T18:46:41 *** Guyver2 has quit IRC
5882016-09-14T18:46:45 *** Guyver2_ is now known as Guyver2
5892016-09-14T18:46:49 <kanzure> wumpus: i think github is supposed to do that on its homepage, but IIRC nobody ever views the github homepage :)
5902016-09-14T18:46:56 <achow101> Does anyone think that the pull request review feature on github might be useful?
5912016-09-14T18:47:13 <wumpus> really those should end up in my personal mailbox instead of with the bulk
5922016-09-14T18:47:31 <wumpus> achow101: haven't looked at it yet
5932016-09-14T18:47:43 *** Soligor has joined #bitcoin-core-dev
5942016-09-14T18:47:45 *** achow101_ has joined #bitcoin-core-dev
5952016-09-14T18:48:05 <achow101> part of it looks like basically preventing PRs from merging without ACKs and ACKs are built in
5962016-09-14T18:48:30 <paveljanik> wumpus, github mails contain this weird CC: ... Comment <comment@noreply.github.com> or Mention or ...
5972016-09-14T18:48:34 <wumpus> ok that sounds fairly useful
5982016-09-14T18:48:37 <kanzure> achow101: are those ACKs tied to specific commit ids?
5992016-09-14T18:48:58 <achow101> Dunno, haven't fully checked it out yet. I just read over the help article they linked
6002016-09-14T18:49:21 <wumpus> paveljanik: that could be key
6012016-09-14T18:53:28 *** Chris_St1 has joined #bitcoin-core-dev
6022016-09-14T18:53:31 *** achow101_ has quit IRC
6032016-09-14T18:55:07 <wumpus> paveljanik: yes, searching for to:mention@noreply.github.com seems to do what I want
6042016-09-14T18:55:58 *** MarcoFalke has joined #bitcoin-core-dev
6052016-09-14T19:00:09 *** Chris_St1 has quit IRC
6062016-09-14T19:00:11 <GitHub131> [bitcoin] rebroad opened pull request #8733: More granular debug [WIP] (master...MoreGranularDebug.wip) https://github.com/bitcoin/bitcoin/pull/8733
6072016-09-14T19:00:28 *** Chris_Stewart_5 has joined #bitcoin-core-dev
6082016-09-14T19:00:44 <rebroad> this is the version with 22 commit.... hopefuly if anyone can be bothered to look they can get a clearer idea of where it was all going..
6092016-09-14T19:00:46 *** achow101_ has joined #bitcoin-core-dev
6102016-09-14T19:03:17 <wumpus> much better
6112016-09-14T19:13:10 <GitHub147> [bitcoin] MarcoFalke closed pull request #8731: Debug headers received ("block" for new block announcement, "block2" ⦠(master...DebugHeadersReceived) https://github.com/bitcoin/bitcoin/pull/8731
6122016-09-14T19:19:50 *** spudowiar has quit IRC
6132016-09-14T19:21:19 <btcdrak> wow, what is this new "review" thing on Github
6142016-09-14T19:21:25 *** BunBun has joined #bitcoin-core-dev
6152016-09-14T19:22:16 <BunBun> quit
6162016-09-14T19:22:19 *** BunBun has quit IRC
6172016-09-14T19:27:43 *** achow101_ has quit IRC
6182016-09-14T19:30:27 *** Chris_Stewart_5 has quit IRC
6192016-09-14T19:31:42 *** assder has quit IRC
6202016-09-14T19:40:09 <GitHub198> [bitcoin] rebroad closed pull request #8596: [WIP] Feeler code and debugging. (master...FeelerFixes) https://github.com/bitcoin/bitcoin/pull/8596
6212016-09-14T19:45:32 *** Chris_Stewart_5 has joined #bitcoin-core-dev
6222016-09-14T19:51:06 *** timothy has quit IRC
6232016-09-14T19:51:09 *** drizztbsd has joined #bitcoin-core-dev
6242016-09-14T19:51:42 *** drizztbsd is now known as timothy
6252016-09-14T20:00:47 *** achow101_ has joined #bitcoin-core-dev
6262016-09-14T20:00:58 *** achow101_ has quit IRC
6272016-09-14T20:07:46 *** spudowiar has joined #bitcoin-core-dev
6282016-09-14T20:23:31 *** aalex has quit IRC
6292016-09-14T20:25:24 <cfields_> wumpus: ok, i've finally had enough of the system package crap that we can't control. I'm adding native toolchains to depends, along with a BOOTSTRAP option, false by default. Bootstrapping can be done in gitian and uploaded, so that they're simply fetched 99% of the time.
6302016-09-14T20:29:37 <sipa> cfields_: so gcc being part of the build for all platforms?
6312016-09-14T20:30:12 <cfields_> sipa: that's what I'm considering, yes
6322016-09-14T20:31:24 <sipa> awesome.
6332016-09-14T20:32:07 <cfields_> I'm sure it'll spiral downward with arguments about where to stop, but gcc/binutils seems like a reasonable start.
6342016-09-14T20:33:11 *** aalex has joined #bitcoin-core-dev
6352016-09-14T20:33:35 <sipa> cfields_: do you know nixos?
6362016-09-14T20:34:02 <cfields_> sipa: not really
6372016-09-14T20:34:48 <cfields_> sipa: looks interesting. Is it intended to be deterministic across builders?
6382016-09-14T20:35:24 <sipa> it is not
6392016-09-14T20:35:53 <sipa> but it uses hashes of build instructions to identify packages
6402016-09-14T20:36:19 <cfields_> ah, nice. sounds familiar :)
6412016-09-14T20:36:33 <MarcoFalke> Isn't this what docker does?
6422016-09-14T20:37:46 <sipa> docker builds from source?
6432016-09-14T20:40:32 <cfields_> docker just does whatever you tell it to. but yes, i'd assume it uses the hash of the recipe for checkpointing
6442016-09-14T20:50:39 <luke-jr> btcdrak: paveljanik: where do you see the new review thingâ
6452016-09-14T20:50:57 <btcdrak> luke-jr: look on the diff tab
6462016-09-14T20:51:34 <luke-jr> cfields_: we need to support building outside gitian anyway, so not sure I see the point
6472016-09-14T20:51:35 <btcdrak> luke-jr: "the files changed" tab.
6482016-09-14T20:52:19 <luke-jr> huh
6492016-09-14T20:52:22 <luke-jr> interesting
6502016-09-14T20:58:16 <MarcoFalke> luke-jr: I think the goal is to solve the cross compile issues with depends.
6512016-09-14T21:00:40 <MarcoFalke> So if you fetch the (gitian-built) toolchain, you can just build outside gitian like you do today
6522016-09-14T21:22:42 *** spudowiar has quit IRC
6532016-09-14T21:35:10 *** Guyver2 has quit IRC
6542016-09-14T22:29:20 *** laurentmt has joined #bitcoin-core-dev
6552016-09-14T22:40:38 *** laurentmt has quit IRC
6562016-09-14T22:55:52 *** achow101_ has joined #bitcoin-core-dev
6572016-09-14T23:02:42 *** wjx has joined #bitcoin-core-dev
6582016-09-14T23:21:28 *** MarcoFalke has quit IRC
6592016-09-14T23:21:55 *** davec has quit IRC
6602016-09-14T23:22:25 *** davec has joined #bitcoin-core-dev
6612016-09-14T23:49:18 *** afk11 has joined #bitcoin-core-dev
6622016-09-14T23:49:18 *** afk11 has quit IRC
6632016-09-14T23:49:18 *** afk11 has joined #bitcoin-core-dev
6642016-09-14T23:59:53 *** achow101_ has quit IRC