12016-06-02T00:13:49  *** kadoban has quit IRC
  22016-06-02T00:32:09  *** TomMc has quit IRC
  32016-06-02T00:50:47  *** belcher has quit IRC
  42016-06-02T00:51:15  *** Ylbam has quit IRC
  52016-06-02T00:52:46  *** kadoban has joined #bitcoin-core-dev
  62016-06-02T00:56:17  *** frankenmint has quit IRC
  72016-06-02T01:12:14  *** justanotheruser has quit IRC
  82016-06-02T01:12:37  *** justanotheruser has joined #bitcoin-core-dev
  92016-06-02T01:41:01  *** Alopex has quit IRC
 102016-06-02T01:42:06  *** Alopex has joined #bitcoin-core-dev
 112016-06-02T01:54:13  *** xiangfu has joined #bitcoin-core-dev
 122016-06-02T02:17:45  *** moli has joined #bitcoin-core-dev
 132016-06-02T02:20:43  *** fengling has joined #bitcoin-core-dev
 142016-06-02T02:23:00  *** Chris_Stewart_5 has quit IRC
 152016-06-02T02:25:08  *** fengling has quit IRC
 162016-06-02T02:29:41  *** fengling has joined #bitcoin-core-dev
 172016-06-02T02:52:01  *** Alopex has quit IRC
 182016-06-02T02:53:06  *** Alopex has joined #bitcoin-core-dev
 192016-06-02T03:17:47  *** jcorgan has quit IRC
 202016-06-02T03:21:02  *** Alopex has quit IRC
 212016-06-02T03:22:07  *** Alopex has joined #bitcoin-core-dev
 222016-06-02T03:25:29  *** fengling has quit IRC
 232016-06-02T03:37:26  *** fengling has joined #bitcoin-core-dev
 242016-06-02T03:58:17  *** jtimon has quit IRC
 252016-06-02T04:22:29  *** gribble has quit IRC
 262016-06-02T04:23:12  *** grassass has joined #bitcoin-core-dev
 272016-06-02T04:23:27  *** molz has joined #bitcoin-core-dev
 282016-06-02T04:26:41  *** moli has quit IRC
 292016-06-02T04:31:51  *** gribble has joined #bitcoin-core-dev
 302016-06-02T04:34:52  *** kadoban has quit IRC
 312016-06-02T04:39:44  *** fengling has quit IRC
 322016-06-02T04:44:44  *** ghtdak has quit IRC
 332016-06-02T04:46:01  *** Alopex has quit IRC
 342016-06-02T04:47:06  *** Alopex has joined #bitcoin-core-dev
 352016-06-02T04:47:53  *** ghtdak has joined #bitcoin-core-dev
 362016-06-02T04:58:59  *** fengling has joined #bitcoin-core-dev
 372016-06-02T05:44:30  <gmaxwell> Our current addnode behavior is more than a little obnoxious.  We won't bring up an addnode if it would leave us with more than the maximum outbound connections, even if none of our existing connections are addnoded.  This means that if your network burps you can end up disconnecting all your addnoded peers, and then fill them back in with random peers.
 382016-06-02T05:45:25  <gmaxwell> Even sitting around manually disconnectnode-ing outbounds that weren't my addnodes took quite a while before it managed to try the addnode before connecting something else.
 392016-06-02T05:50:16  *** imacomput has joined #bitcoin-core-dev
 402016-06-02T05:50:48  <gmaxwell> I think the behavior should be, we should track if peers are addnodeed.  If we have unconnected addnodes and less han MAX connected addnodes, and less than MAX+1 total connected. ... try adding an addnode.
 412016-06-02T05:51:48  <btcdrak> gmaxwell: I have the same complaint. I just never articulated it.
 422016-06-02T05:52:06  <gmaxwell> I think we introduced this misbehavior around 0.9.x or something.
 432016-06-02T05:52:18  <gmaxwell> a long time ago addnode would just go over the connection limit.
 442016-06-02T05:52:56  <gmaxwell> ... and then seperately, every N seconds,  if we have >MAX outbound connections, pick one to evict. (where it would avoid evicting whitelisted and addnode peers, then ones that have recently sent us blocks, then txn, the prefer to keep long uptime).
 452016-06-02T05:53:39  <gmaxwell> The same eviction process could then be used to rotate the outbound connections. (just randomly try to connect to too many, and then let eviction do its thing.)
 462016-06-02T05:55:50  *** imacomput has quit IRC
 472016-06-02T06:02:00  *** calibre720 has joined #bitcoin-core-dev
 482016-06-02T06:03:26  <sipa> gmaxwell: indeed, and we already have eviction code
 492016-06-02T06:04:11  <sipa> just adding a bool to select for incoming or outgoing would be a large part already
 502016-06-02T06:08:12  <gmaxwell> the current eviction code only evicts incoming. I think our strategy for outgoing should be somewhat different... in particular, we have far fewer outgoing.
 512016-06-02T06:08:44  <sipa> for rotation we'd need better logic
 522016-06-02T06:08:59  <sipa> but for preferring addnode over random outgoing maybe not so much
 532016-06-02T06:09:14  <midnightmagic> +1 for addnode prioritization :-)
 542016-06-02T06:10:47  <gmaxwell> yes. I think sort by whitelist/addnode/most-recent-to-send-block/dice would be fine for evicting with the addnode case.
 552016-06-02T06:11:39  <gmaxwell> kinda odd that it would let you partition it by kicking off all your useful peers in favor of some dumb addnodes you plugged in.. but ::shrugs::
 562016-06-02T06:14:13  *** baldur has quit IRC
 572016-06-02T06:17:40  <sipa> yes, if you had to be whitelisted in order to get an addnode, we could just have addnode bypass the connection limit
 582016-06-02T06:18:03  <sipa> maybe something for post host authentication
 592016-06-02T06:20:25  <gmaxwell> right.
 602016-06-02T06:20:28  <gmaxwell> well baby steps
 612016-06-02T06:25:27  <midnightmagic> .w 2
 622016-06-02T06:26:50  <gmaxwell> sipa: could just be a feature of the p2p auth stuff, once we have that. If mutual auth is successful it bypasses limits.
 632016-06-02T06:27:09  <gmaxwell> (or at least a non-file descriptor related limit)
 642016-06-02T06:28:14  *** randy-waterhouse has joined #bitcoin-core-dev
 652016-06-02T06:30:45  <gmaxwell> sipa: another coner case to consider: say you set 8 addnode peers. They all ban you.
 662016-06-02T06:34:29  <midnightmagic> can I ask for a ptr to where the p2p auth is being worked on?
 672016-06-02T06:35:50  <sipa> midnightmagic: blocked by first having encryption in; see bip 151
 682016-06-02T06:37:16  <midnightmagic> ok
 692016-06-02T06:38:11  <gmaxwell> I guess that banning example suggests having some kind of addnode backoff... and making sure the auto evict runs slow enough after the connect that it won't kick a good peer to make room for a connection that is banned.
 702016-06-02T06:40:55  *** Giszmo has quit IRC
 712016-06-02T06:42:40  *** BashCo has quit IRC
 722016-06-02T06:43:38  <midnightmagic> I see a note of the mitm mitigation not being addressed and identity authentication: was this discussed somewhere that I missed?
 732016-06-02T06:44:19  <midnightmagic> (I would be happy with "sometime a few weeks ago in -wizards" or like, I don't mind doing my own legwork but would appreciate a general pointer)
 742016-06-02T06:49:17  <sipa> to address mitm, you need authentication
 752016-06-02T06:50:18  <sipa> jonas started on an authentication bip as well in parallel with bip 151
 762016-06-02T06:50:30  <sipa> but that was postponed to first get encryption done
 772016-06-02T06:52:14  <jonasschnelli> sipa, midnightmagic: there is some stuff from greg: https://people.xiph.org/~greg/auth1.txt
 782016-06-02T06:52:32  <jonasschnelli> Also the auth bip should probably be done before starting with the bip151 implementation
 792016-06-02T06:52:54  <midnightmagic> ok
 802016-06-02T07:03:26  <gmaxwell> I think it's fine to implement before auth is done, so long as whomever is implementing doesn't mind taking some small risk of disruption if changes to the auth design change the encryption.
 812016-06-02T07:05:05  <midnightmagic> gmaxwell: any thought to using proof-of-storage as ephemeral key ratchets? :-)
 822016-06-02T07:05:30  <midnightmagic> er. that was meant generally, not just to gmax. :)
 832016-06-02T07:06:28  <midnightmagic> and, would a group sig in log or constant space be possible for client group-membership proof non-interactively?
 842016-06-02T07:06:47  *** molly has joined #bitcoin-core-dev
 852016-06-02T07:09:30  *** molz has quit IRC
 862016-06-02T07:10:55  <gmaxwell> I don't think group sigs fit well in this context, unfortunately.  The server can tell if you are peer X by basically giving you a dummy keyring where all the keys are junk except yours.
 872016-06-02T07:11:18  <gmaxwell> you can't tell that it did this to you if it was successful in deanoning you
 882016-06-02T07:11:56  <gmaxwell> (the log size group sigs also take linear time to sign/verify... we're not actually super bandwidth constrained in this application-- so I think a log sized one doesn't buy us much)
 892016-06-02T07:12:13  <gmaxwell> The constant sized ones use pairing crypto, so then there is a bunch of extra crypto to bring in.
 902016-06-02T07:12:47  <midnightmagic> thank you
 912016-06-02T07:14:54  <gmaxwell> the chaumtoken auth was the best private auth I could come up with so far.
 922016-06-02T07:19:34  *** Ylbam has joined #bitcoin-core-dev
 932016-06-02T07:30:16  <jouke> I'm running a node on ipv4 and onion. I have a lot of peers that have the same "addr" IP, and my addresslocal says: "127.0.0.1". Are those connected trough TOR but are advertising that IP? The subver of those peers are different as well.
 942016-06-02T07:30:37  *** randy-waterhouse has quit IRC
 952016-06-02T07:35:17  <gmaxwell> if someone connects into you via onion, their addr will be 127.0.0.1:<something>  and the addrlocal should be your onion address.
 962016-06-02T07:35:27  <gmaxwell> though they can send whatever they want in addrlocal.
 972016-06-02T07:38:14  <jouke> Does the addrlocal in the gepeerinfo field specify on which IP address that node is connected to
 982016-06-02T07:38:42  <gmaxwell> No it is whatever address the remote party claims they think is your address.
 992016-06-02T07:38:48  <jouke> Oh right
1002016-06-02T07:38:59  <jouke> Thanks.
1012016-06-02T07:39:50  <jouke> And the addr field is the address where my nodes comunicates to?
1022016-06-02T07:40:06  <gmaxwell> yes, which is 127.0.0.1 for inbound onion peers.
1032016-06-02T07:41:31  <jouke> I have 57 peers with the same addr IP :-/
1042016-06-02T07:42:02  <gmaxwell> presumably 52.51.something.
1052016-06-02T07:42:07  <jouke> No
1062016-06-02T07:42:12  <jouke> Those I have as well
1072016-06-02T07:42:16  <jouke> On other nodes
1082016-06-02T07:42:39  <jouke> It is is actualy a /32 IP
1092016-06-02T07:43:04  <midnightmagic> that sounds super annoying.
1102016-06-02T07:43:10  <gmaxwell> What do you mean by a "/32 IP"  ... all IPs are /32... because they're hosts not networks. :P
1112016-06-02T07:43:33  <jouke> The bytessent/bytesrecv ratio isn't as skewed as with those 52.51
1122016-06-02T07:43:58  <jouke> gmaxwell: just one and the same IP, not multiple IP's from the same subnet.
1132016-06-02T07:44:16  <gmaxwell> I strongly recommend reporting abusive hosts to their providers, fwiw.
1142016-06-02T07:44:55  <jouke> Hetzner...
1152016-06-02T07:44:58  <midnightmagic> maybe it's a tor exit node or something
1162016-06-02T07:45:19  <gmaxwell> midnightmagic: well thats easily checked. :) (indeed, don't report tor exits)
1172016-06-02T07:46:13  <jouke> It's not in here: https://check.torproject.org/exit-addresses
1182016-06-02T07:46:57  <midnightmagic> Ah. Hetzner. Why am I not surprised. :-/
1192016-06-02T07:49:52  <jouke> I was thinking of having a script that would look at the upload/download ratio and ban nodes with a bad ratio. But wouldn't have worked in this case.
1202016-06-02T07:50:58  <gmaxwell> you can do varrious adhoc things personally, but most don't work broader than that.. too easily avoided. the 52.x sybils could start sending junk traffic... and just waste more of your bandwidth.
1212016-06-02T07:51:27  <gmaxwell> They case no harm except privacy loss, and the solution to that is to fix things so they can't hurt privacy.
1222016-06-02T07:52:56  <gmaxwell> though if they're irritating you, http://0bin.net/paste/EH8fTyfotJSU3Udw#TCHJfA7qDt5odgG+gW4dG9aUEYkJQo6oOTmsuXBAJke  works.
1232016-06-02T07:53:49  *** pmienk has quit IRC
1242016-06-02T07:55:06  <gmaxwell> ah seems that misses 52.51.180.197 and 52.51.204.60
1252016-06-02T07:56:37  <GitHub50> [bitcoin] jonasschnelli opened pull request #8135: [OSX] fix make deploy when compiling on OSX (master...2016/06/makedeploy) https://github.com/bitcoin/bitcoin/pull/8135
1262016-06-02T08:05:44  *** pmienk has joined #bitcoin-core-dev
1272016-06-02T08:06:53  <jouke> gmaxwell: heh, thanks :)
1282016-06-02T08:10:27  *** jannes has joined #bitcoin-core-dev
1292016-06-02T08:29:19  *** Guyver2 has joined #bitcoin-core-dev
1302016-06-02T08:31:32  *** randy-waterhouse has joined #bitcoin-core-dev
1312016-06-02T08:32:40  *** molz has joined #bitcoin-core-dev
1322016-06-02T08:35:32  *** molly has quit IRC
1332016-06-02T08:38:45  *** jyap has quit IRC
1342016-06-02T08:39:09  *** jyap has joined #bitcoin-core-dev
1352016-06-02T08:39:09  *** jyap has joined #bitcoin-core-dev
1362016-06-02T08:45:41  <GitHub104> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/715e9fd7454f...58725ba89dfc
1372016-06-02T08:45:41  <GitHub104> bitcoin/master 2692e1b fanquake: [Doc] Simplify OS X build notes...
1382016-06-02T08:45:42  <GitHub104> bitcoin/master 58725ba Jonas Schnelli: Merge #8029: [Doc] Simplify OS X build notes...
1392016-06-02T08:45:45  <GitHub56> [bitcoin] jonasschnelli closed pull request #8029: [Doc] Simplify OS X build notes (master...osx-build-notes) https://github.com/bitcoin/bitcoin/pull/8029
1402016-06-02T09:11:59  <GitHub85> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/58725ba89dfc...ee1533e262e7
1412016-06-02T09:11:59  <GitHub85> bitcoin/master 16698cb UdjinM6: PR #7772 is not enough to fix the issue with QCompleter, use event filter instead of `connect`
1422016-06-02T09:12:00  <GitHub85> bitcoin/master ee1533e Jonas Schnelli: Merge #8129: Fix RPC console auto completer...
1432016-06-02T09:12:09  <GitHub132> [bitcoin] jonasschnelli closed pull request #8129: Fix RPC console auto completer (master...fixRPCAutoCompleter_bitcoin) https://github.com/bitcoin/bitcoin/pull/8129
1442016-06-02T09:13:24  *** BashCo has joined #bitcoin-core-dev
1452016-06-02T09:19:39  *** paveljanik has joined #bitcoin-core-dev
1462016-06-02T09:19:39  *** paveljanik has joined #bitcoin-core-dev
1472016-06-02T09:19:52  <iniana> What can I do to debug the painfully slow sync of my core node on a linux machine? Currently at height 410k and it progresses around 1 block every 3 seconds on average.
1482016-06-02T09:27:38  <jonasschnelli> iniana: what processor? how many RAM? disk stype?
1492016-06-02T09:28:04  <jonasschnelli> 1 block every 3 seconds is very likely disk speed or CPU.
1502016-06-02T09:28:34  <jonasschnelli> If you have enough memory, you should try to pass -dbache=4000 (4GB)
1512016-06-02T09:28:47  <jonasschnelli> Even passing -dbcache=2000 can help
1522016-06-02T09:28:55  <jonasschnelli> (default is 300)
1532016-06-02T09:33:09  <iniana> jonasschnelli: blockchain is stored on hdd, dbcache was set to 2000. I have a feeling it is because it is run within a Qubes VM. Is there anyway to measure disk I/O?
1542016-06-02T09:34:08  <jonasschnelli> iniana: I guess you can use iostat on linux
1552016-06-02T09:34:41  <jonasschnelli> iniana: did you enable multicore on your VM? But yes,.. running in a VM can slow down the whole thing.
1562016-06-02T09:35:01  <jonasschnelli> Best fix: don't stress it and wait a couple of days. :)
1572016-06-02T09:36:35  <iniana> jonasschnelli: yes, set to 8 VCPUs, 8GB max memory, 4GB initial memory. Am at height 410k, so should only have to wait a couple of hours :)
1582016-06-02T09:36:54  <jonasschnelli> iniana: yes. What CPU?
1592016-06-02T09:37:39  <jonasschnelli> On my Xeon/SSD I could sync a node in a VirtualBox VM within 3-4h.
1602016-06-02T09:37:44  <iniana> core i7, 2yo, not sure on exact specs
1612016-06-02T09:38:03  <jonasschnelli> iniana: try run Core with --debug=bench
1622016-06-02T09:38:22  <jonasschnelli> You will see if its the disk latency or the ecdsa
1632016-06-02T09:39:26  <iniana> I managed to sync a pruned node on a whonix VM (tor-only) a lot faster than what I am experiencing now. Only difference is that was on SSD. But I have used the same HDD before and it was a lot faster then. Some combo of VM and HDD doesn't mix I guess.
1642016-06-02T09:39:40  <iniana> Ah great! Will try that debug flag
1652016-06-02T09:42:06  <phantomcircuit> iniana, does qubes pass the cpu flags?
1662016-06-02T09:42:20  <phantomcircuit> also what version are you running?
1672016-06-02T09:43:25  <iniana> phantomcircuit: What are the cpu flags? Am running Qubes 3.1, Core 0.12.1
1682016-06-02T09:47:32  <iniana> The following line seems to be the one that takes longest to generate: - Connect 647 transactions: 597.49ms (0.923ms/tx, 0.509ms/txin) [102.29s]
1692016-06-02T09:48:16  <iniana> also - Verify 4484 txins: 2407.61ms (0.537ms/txin) [104.76s]
1702016-06-02T09:48:37  *** cryptapus has joined #bitcoin-core-dev
1712016-06-02T10:01:42  <wumpus> looks like most time is spent on verification
1722016-06-02T10:01:50  <wumpus> how many cores are allocated to that VM?
1732016-06-02T10:02:36  <wumpus> bitcoin core makes heavy use of paralellism for verification
1742016-06-02T10:04:23  <wumpus> ok 8 VCPUs should be fine
1752016-06-02T10:05:18  <sipa> u
1762016-06-02T10:07:25  <iniana> Hmm.. There was a sequence of a couple of hundred blocks which were processed very quickly (10/s maybe) after which it resumed with 1 block per 3s again.
1772016-06-02T10:13:40  <sipa> are you reindexing or downloading?
1782016-06-02T10:13:53  <iniana> downloading
1792016-06-02T10:14:15  <sipa> that will explain the speed differences
1802016-06-02T10:14:31  <sipa> we download multiple blocks at once
1812016-06-02T10:14:51  <sipa> but they don't arrivs in the right order
1822016-06-02T10:17:20  <iniana> Are transactions that can't be verified yet during sync added to the mempool or someplace else?
1832016-06-02T10:17:20  <sipa> so validation only progresses when the first missing one arrices
1842016-06-02T10:17:52  <sipa> you mean transactions in blocks we download? no
1852016-06-02T10:18:11  <sipa> the mempool only contains validated transactions
1862016-06-02T10:18:40  <iniana> ah ok
1872016-06-02T10:18:55  *** tErik_mc has joined #bitcoin-core-dev
1882016-06-02T10:18:57  <sipa> say you're synced up to block 10000
1892016-06-02T10:19:13  <sipa> and you downloading the 100 blocks that follow
1902016-06-02T10:19:30  <sipa> and 10001 though 10099 have arrived
1912016-06-02T10:19:36  <sipa> but 10000 has not
1922016-06-02T10:19:46  <sipa> then 10000 arrvives
1932016-06-02T10:20:03  <sipa> and suddenly you can validate 100 blocks at once
1942016-06-02T10:25:24  <iniana> Still weird that it is progressing this slowly though. I mean, it was the same speed when reindexing a previously synced data directory I had copied over. It had all the data and did no downloading. How come there was that sequence of blocks that could be progressed so quickly?
1952016-06-02T10:27:12  <iniana> When I restarted bitcoin core recently the verification of the previous 288 blocks was also that slow.
1962016-06-02T10:29:58  <sipa> during reindex you also saw some sequence of blocks that wete fast?
1972016-06-02T10:30:19  <sipa> not all blocks have the same number of transactions
1982016-06-02T10:30:43  <sipa> the initial 288 being slow is due to the cache mot being warmed up
1992016-06-02T10:31:00  <sipa> the database entries still have to be loaded from disk
2002016-06-02T10:31:21  <iniana> no, during reindex I never saw a fast sequence, though I could have missed it.
2012016-06-02T10:32:16  <phantomcircuit> sipa, the p2p block fetching logic needs to be hardened against people being annoying
2022016-06-02T10:32:25  <phantomcircuit> there's some very very very slow nodes
2032016-06-02T10:32:29  <iniana> Sure, not all blocks are the same size, but from a couple of hundred blocks at height 411k, I bet most of them had a lot of transactions.
2042016-06-02T10:32:38  <phantomcircuit> but worse there's some that are ridiculously bursty
2052016-06-02T10:32:49  <phantomcircuit> they'll be fast for some blocks and then take 10 seconds to respond to another
2062016-06-02T10:55:43  *** Thireus1 has joined #bitcoin-core-dev
2072016-06-02T10:56:41  *** jtimon has joined #bitcoin-core-dev
2082016-06-02T10:56:53  *** Thireus1 has quit IRC
2092016-06-02T10:57:09  *** Thireus1 has joined #bitcoin-core-dev
2102016-06-02T11:02:06  <sipa> iniana: you have high dbcache set in both?
2112016-06-02T11:03:00  <iniana> sipa: yes, 2000
2122016-06-02T11:05:00  *** calibre720 has quit IRC
2132016-06-02T11:05:52  <sipa> whenever the dbcache fills up, it needs tp be written to disk, and afterwards processing is slower for a while
2142016-06-02T11:06:18  <sipa> you can see when this happens when the cache size in the UodateTip lines falls back to 0
2152016-06-02T11:06:33  <sipa> but that causes sudden slowness, not sudden fastness
2162016-06-02T11:08:24  *** fengling has quit IRC
2172016-06-02T11:10:55  <iniana> Cache currently at 1258MiB, and it is consistently slow.
2182016-06-02T11:11:14  <sipa> are you running with debug=bench ?
2192016-06-02T11:11:20  <iniana> yes
2202016-06-02T11:14:09  *** xiangfu has quit IRC
2212016-06-02T11:14:32  *** calibre720 has joined #bitcoin-core-dev
2222016-06-02T11:15:00  <iniana> sipa: Here is a snippet from my log http://pastebin.com/6jWv29uj
2232016-06-02T11:16:50  <sipa> all your time is lost on loading database entries
2242016-06-02T11:17:12  <sipa> either those are blocks that spend a lot of very old inouts that are not cached
2252016-06-02T11:17:18  <sipa> or your disk is very slow
2262016-06-02T11:17:31  <sipa> *old inputs
2272016-06-02T11:20:29  <iniana> iostat reports 7.8M read and 2.1M write for that particular block deivce. Doesn't feel like it is the bottleneck.
2282016-06-02T11:22:06  <sipa> they're all very small reads
2292016-06-02T11:23:24  <sipa> what kind of disk?
2302016-06-02T11:23:31  <iniana> sipa: ok. I guess this is the performance I have to live with until I upgrade to an SSD?
2312016-06-02T11:24:13  <sipa> i have a spinning disk, and numbers are far better than what you're seeing
2322016-06-02T11:24:21  <sipa> 18s to load all the inputs is crazy
2332016-06-02T11:24:49  <iniana> yes, I have used the same disk on a non-virtual machine with much better results.
2342016-06-02T11:25:05  <sipa> ah, maybe it's due to the VM setup
2352016-06-02T11:25:29  <wumpus> yes VM can make some difference
2362016-06-02T11:25:49  <wumpus> there are some curious interactions between fdatasync and VMs for example
2372016-06-02T11:26:09  <wumpus> probably because the VM 'disk' is one file in the host OS
2382016-06-02T11:26:20  <sipa> wumpus: this is slowness in reads, not writes
2392016-06-02T11:26:25  <sipa> so sync is not the issue
2402016-06-02T11:26:32  <wumpus> reads shouldn't be affected by VM
2412016-06-02T11:26:39  <wumpus> not much, at least
2422016-06-02T11:27:19  <wumpus> unless it is some weird compressed image format
2432016-06-02T11:27:45  <iniana> I'm using Qubes (xen) if that helps
2442016-06-02T11:29:08  <wumpus> what can make a difference in i/o speed is to make sure that you're using a paravirtualized disk, and not e.g. emulating IDE
2452016-06-02T11:30:01  <phantomcircuit> iniana, 200+ms to load a block from disk is terrrible
2462016-06-02T11:32:25  <iniana> Could be the way I attach the block device to my VM (qvm-block), will explore this further.
2472016-06-02T11:34:00  <wumpus> yes qemu allows configuring a lot of things related to emulated devices
2482016-06-02T11:35:03  <wumpus> I'd say it is only worth it if you intend to do a lot of syncing, I mean once your node syncs the i/o and CPU usage drops by lots and this all matters very little
2492016-06-02T11:36:13  *** airmace312 has joined #bitcoin-core-dev
2502016-06-02T11:36:16  * airmace312 is selling coins if you are intrested in buying please private msg me and we can negotiate the rate so if intrested please msg me for a good deal thanks
2512016-06-02T11:36:28  *** ChanServ sets mode: +o wumpus
2522016-06-02T11:36:52  *** wumpus sets mode: +b *!*@103-224-130-8.flip.co.nz
2532016-06-02T11:37:02  *** airmace312 was kicked by wumpus (no selling/buying here)
2542016-06-02T11:37:18  *** ChanServ sets mode: -o wumpus
2552016-06-02T11:37:36  <iniana> wumpus: I guess, but it would be annoying if when I have to turn off the machine only to have to wait for hours until it is synced up again when I turn it on. Plus its fun getting to know how stuff works :)
2562016-06-02T11:38:34  <wumpus> sure, as a learning experience it makes sense
2572016-06-02T11:48:14  <sipa> iniana: at your sync speed, it takes around 1-2 hours to sync a week of blockchain data
2582016-06-02T11:48:18  <sipa> that's terrible
2592016-06-02T11:48:28  <sipa> some people sync the whole chain in 4 hours
2602016-06-02T11:48:47  <iniana> sipa: yes, its very frustrating
2612016-06-02T11:48:49  <sipa> but i don't have any advice but checking your vm disk setup
2622016-06-02T11:49:32  *** randy-waterhouse has quit IRC
2632016-06-02T11:55:28  <phantomcircuit> iniana, it's xen based
2642016-06-02T11:57:01  <iniana> phantomcircuit: yes
2652016-06-02T11:57:14  <phantomcircuit> i would say
2662016-06-02T11:57:19  <phantomcircuit> "good luck" fixing that issue
2672016-06-02T11:57:20  <phantomcircuit> :/
2682016-06-02T11:57:46  <iniana> You say I shouldn't waste my time?
2692016-06-02T11:58:21  <phantomcircuit> i'd say go ask the qubes people
2702016-06-02T11:58:32  <GitHub8> [bitcoin] jonasschnelli opened pull request #8136: Log/report in 10% steps during VerifyDB (master...2016/06/init_checkblocks) https://github.com/bitcoin/bitcoin/pull/8136
2712016-06-02T12:00:07  <iniana> How would attaching the disk as a samba or nfs share instead of via xen work out? Has anyone tried loading blockchain data from a remote server that way?
2722016-06-02T12:02:17  *** fengling has joined #bitcoin-core-dev
2732016-06-02T12:03:44  <sipa> iniana: no, you need mmap support
2742016-06-02T12:03:53  <sipa> network filesystems don't offer that
2752016-06-02T12:04:24  <phantomcircuit> sipa, do we still require mmap?
2762016-06-02T12:04:30  <sipa> leveldb does
2772016-06-02T12:04:48  <phantomcircuit> hmm
2782016-06-02T12:04:59  <sipa> i think
2792016-06-02T12:05:06  <kinlo> isn't requiring mmap a good thing from a performance point of view?
2802016-06-02T12:05:13  <sipa> yes
2812016-06-02T12:05:17  <phantomcircuit> "meh"
2822016-06-02T12:05:27  <sipa> i think you can disable it
2832016-06-02T12:05:43  <sipa> i think we may not use mmap on 32-bit
2842016-06-02T12:07:04  *** fengling has quit IRC
2852016-06-02T12:07:22  <wumpus> loading the blockchain data from a network filesystem should work, just not having the database directories there (if it doesn't support mmap)
2862016-06-02T12:07:50  <wumpus> I've succesfully used a network block device to store both though
2872016-06-02T12:07:59  *** Chris_Stewart_5 has joined #bitcoin-core-dev
2882016-06-02T12:08:29  <wumpus> haven't tried samba nor nfs
2892016-06-02T12:09:36  <jonasschnelli> wumpus: is there a startup argument to set the blockfiles directory?
2902016-06-02T12:09:37  <wumpus> I've also shared synced node data between windows and linux locally, linux' ntfs implementation surprisingly does work good enough for leveldb
2912016-06-02T12:10:19  <jonasschnelli> Or do you need some ln -s?
2922016-06-02T12:10:21  <wumpus> no, but you can put stuff all over the place with symbolic linnks
2932016-06-02T12:10:43  <jonasschnelli> Right...
2942016-06-02T12:11:09  <wumpus> being able to specify different directories as arguments would be mildly useful, although for people that don't know what they're doing it's a foot shooting cannon
2952016-06-02T12:13:15  <wumpus> (e.g. we only aquire a lock on the data directory, not on the subdirectories individually, so if you can specify them elsewhere someone may do crazy things like point two bitcoinds at the same place)
2962016-06-02T12:21:07  *** cryptapus_ has joined #bitcoin-core-dev
2972016-06-02T12:21:07  *** cryptapus has quit IRC
2982016-06-02T12:22:24  *** cryptapus_ is now known as cryptapus
2992016-06-02T12:28:14  *** MrHodl has joined #bitcoin-core-dev
3002016-06-02T12:32:29  <jonasschnelli> hmm.. travis is strange today:
3012016-06-02T12:32:30  <jonasschnelli> W: Failed to fetch https://apt.dockerproject.org/repo/dists/ubuntu-trusty/main/binary-amd64/Packages  Hash Sum mismatch
3022016-06-02T12:34:24  <wumpus> scary
3032016-06-02T12:36:06  <wumpus> happens every time?
3042016-06-02T12:43:53  <sipa> try pressing more buttond
3052016-06-02T12:46:17  <wumpus> these are not even our own downloads, but travis pre-installing packages that fails
3062016-06-02T12:46:37  <wumpus> so less scary, probably just something with the repository that is misconfigured
3072016-06-02T12:48:36  *** achow101 has joined #bitcoin-core-dev
3082016-06-02T12:48:38  <wumpus> docketprojects's package repository that is, not our repository
3092016-06-02T12:49:52  *** Guyver2 has quit IRC
3102016-06-02T12:57:45  *** BashCo has quit IRC
3112016-06-02T13:03:47  <GitHub10> [bitcoin] pstratem opened pull request #8137: Improve CWallet API with new AccountMove function. (master...2016-06-02-cwallet-accountmove) https://github.com/bitcoin/bitcoin/pull/8137
3122016-06-02T13:03:48  *** fengling has joined #bitcoin-core-dev
3132016-06-02T13:06:27  *** Chris_Stewart_5 has quit IRC
3142016-06-02T13:08:44  *** fengling has quit IRC
3152016-06-02T13:12:00  <wumpus> pushing more buttons doesn't seem to help
3162016-06-02T13:28:22  *** MarcoFalke has joined #bitcoin-core-dev
3172016-06-02T13:36:05  *** cryptapus_ has joined #bitcoin-core-dev
3182016-06-02T13:38:52  *** cryptapus has quit IRC
3192016-06-02T13:40:33  *** cryptapus_ has quit IRC
3202016-06-02T13:43:44  *** cryptapus_ has joined #bitcoin-core-dev
3212016-06-02T13:46:03  *** jannes has quit IRC
3222016-06-02T13:46:09  *** jannes_ has joined #bitcoin-core-dev
3232016-06-02T13:49:27  *** zooko has joined #bitcoin-core-dev
3242016-06-02T13:58:51  <MarcoFalke> sipa, but this would cause an include main.h in wallet.h?
3252016-06-02T13:59:24  <sipa> MarcoFalke: why in wallet.h?
3262016-06-02T13:59:36  <sipa> the constant would just disappear from the .h fr
3272016-06-02T13:59:52  <sipa> wallet.cpp would call main to query the current relay fee
3282016-06-02T14:00:39  <MarcoFalke> Oh, you mean current relay fee and not minimum relay fee.
3292016-06-02T14:01:04  <sipa> even the minimum relay fee is configurable, so indeed :)
3302016-06-02T14:01:26  <sipa> or even the minimum confirmable feerate
3312016-06-02T14:01:39  <sipa> as we should never create outputs that are uneconomical to spend
3322016-06-02T14:01:51  <sipa> and the fee estimate is the best guess for that
3332016-06-02T14:04:50  *** Giszmo has joined #bitcoin-core-dev
3342016-06-02T14:05:12  *** robs has quit IRC
3352016-06-02T14:05:41  *** fengling has joined #bitcoin-core-dev
3362016-06-02T14:06:08  *** Thireus1 has quit IRC
3372016-06-02T14:10:10  <phantomcircuit> sipa, im seeing pull tester failures on master
3382016-06-02T14:10:22  <phantomcircuit> can you (or someone else) check?
3392016-06-02T14:10:24  *** fengling has quit IRC
3402016-06-02T14:10:25  <phantomcircuit> wumpus, ^
3412016-06-02T14:11:01  <wumpus> still the apt-get issue?
3422016-06-02T14:11:05  <wumpus> nothing we can do there
3432016-06-02T14:12:07  <wumpus> docker needs to wake up on this https://github.com/docker/docker/issues/23203
3442016-06-02T14:13:09  <sipa> MarcoFalke: also, unrelated... i don't think we should care much about header file dependencies (apart from making compilation time and memory larger, they don't hurt modularity)... what matters is dependencies between (.h,.c) pairs on eachother, and wallet.cpp already depends on main.cpp
3452016-06-02T14:13:16  <phantomcircuit> wumpus, no im testing locally
3462016-06-02T14:13:49  <MarcoFalke> ok
3472016-06-02T14:14:37  <sipa> the reasoning being: you already can't use the functionality from wallet.{cpp,h} without having main.{cpp,h} available, so there already is a dependency
3482016-06-02T14:16:04  *** Thireus1 has joined #bitcoin-core-dev
3492016-06-02T14:16:41  *** Thireus1 has quit IRC
3502016-06-02T14:21:59  *** TomMc has joined #bitcoin-core-dev
3512016-06-02T14:22:31  <jonasschnelli> MarcoFalke: sipa: what about introducing a nodemodel.{cpp/h} that could act as a wallet/node API?
3522016-06-02T14:22:52  <sipa> how is that different from clientmodel?
3532016-06-02T14:23:00  <jonasschnelli> clientmodel is GUI
3542016-06-02T14:23:06  <jonasschnelli> nodemodel would be core/wallet
3552016-06-02T14:23:12  <sipa> i don't understand
3562016-06-02T14:23:27  <sipa> clientmodel is the abstraction of the client/node that the GUI uses to communicate with core
3572016-06-02T14:23:31  <jonasschnelli> Main reason for this could be to have all node interaction in a single file (better readability, better source for later decoupling)
3582016-06-02T14:23:48  <jonasschnelli> Yes. It could also be clientmodel...
3592016-06-02T14:23:48  <sipa> ah, you mean a lower level
3602016-06-02T14:23:51  <jonasschnelli> yes.
3612016-06-02T14:24:04  <sipa> what sort of code would move there?
3622016-06-02T14:24:14  <jonasschnelli> Fee estimation, relay fee, etc.
3632016-06-02T14:24:19  <jonasschnelli> broadcast
3642016-06-02T14:25:36  <sipa> i wouldn't put wallet-related things there (as those shouldn't be lumped together long term)
3652016-06-02T14:26:00  <sipa> but if there are pieces of code shared between gui and rpc, it makes sense to move them to a common abstraction layer below
3662016-06-02T14:28:58  <wumpus> wallet.cpp can depend on main.cpp, but not the other way around
3672016-06-02T14:30:38  <wumpus> I'm not sure adding all node interaction in a single file is a step forward; we'tre trying to modularize things beyond that, e.g. net is different from blockchain handling
3682016-06-02T14:31:41  <jonasschnelli> Yes. I agree, move it to a single file would only increase code readability, ... and not sure if this is worth a PR.
3692016-06-02T14:32:48  <sipa> wumpus: agree
3702016-06-02T14:33:33  <sipa> i think we're perhaps better off spending time to move code from RPC calls (especially the complicated code inside RPCs that grabs cs_main) to somewhere in the core code instead, so that the RPC is just a single call
3712016-06-02T14:33:34  <wumpus> clientmodel has way too many responsibilities
3722016-06-02T14:34:00  <wumpus> (but it's fine for the GUI, and was a big step forward from satoshi's stuff)
3732016-06-02T14:34:40  <wumpus> yes, that makes sense, ideally only main functions would lock cs_main
3742016-06-02T14:34:49  <wumpus> on the other hand at this point I'd not like moving things into main
3752016-06-02T14:34:57  <wumpus> main has the same issue
3762016-06-02T14:35:03  <wumpus> but after net refactors etc, sure
3772016-06-02T14:36:43  <wumpus> I don't think RPC functions necessarily need to be just a single call, what matters is that operations that need to happen under cs_main lock and act on main data structures would be better off in main, but these could be the minimal possible encapsulated operations
3782016-06-02T14:36:54  <sipa> wumpus: +1
3792016-06-02T14:37:18  <sipa> anything that grabs a lock should be moved to the module that manages that lock
3802016-06-02T14:37:39  <sipa> (in practice, that often means introducing callback functions etc... though)
3812016-06-02T14:37:45  *** cryptapus_ is now known as cryptapus
3822016-06-02T14:37:50  <wumpus> aren't we happy with c++11 lambdas then
3832016-06-02T14:38:07  <sipa> i should learn to use those :)
3842016-06-02T14:38:11  <sipa> but yes
3852016-06-02T14:38:39  <wumpus> cfields taught me how to use them in Zurich
3862016-06-02T14:39:11  <wumpus> I'm impressed, the functionality is so flexible it seems almost un-c++
3872016-06-02T14:40:42  <wumpus> (e.g. how code in a lambda can use variables from the enclosing function is more remniscent of dynamic languages)
3882016-06-02T14:41:22  <wumpus> phantomcircuit: if you run it locally you should have an error message?
3892016-06-02T14:41:33  <gmaxwell> GCC nested functions can do that too.
3902016-06-02T14:44:08  <phantomcircuit> wumpus, i have lots of error messages
3912016-06-02T14:44:13  <wumpus> didn't know that! there's not really precedent, c++ nested classes cannot access members from enclosing classes, like in java static inner classes
3922016-06-02T14:44:21  <phantomcircuit> im checking again after a git clean -fdx
3932016-06-02T14:44:31  <wumpus> phantomcircuit: so the autotests fails?
3942016-06-02T14:44:48  <wumpus> or the qa tests?
3952016-06-02T14:44:49  <phantomcircuit> wumpus, rpc pull tests fail
3962016-06-02T14:44:54  <phantomcircuit> make check tests pass
3972016-06-02T14:45:19  <MarcoFalke> which rpc-test?
3982016-06-02T14:45:29  <phantomcircuit> a bunch of the wallet related ones
3992016-06-02T14:45:40  <phantomcircuit> i'll clarify in a minute when im done recompiling
4002016-06-02T14:45:58  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4012016-06-02T14:52:09  *** Guyver2 has joined #bitcoin-core-dev
4022016-06-02T14:54:08  <phantomcircuit> and now everything passes
4032016-06-02T14:54:09  <phantomcircuit> >.>
4042016-06-02T14:54:11  <phantomcircuit> nvm
4052016-06-02T14:58:37  *** moli has joined #bitcoin-core-dev
4062016-06-02T15:01:04  <GitHub38> [bitcoin] jonasschnelli opened pull request #8138: Add maximal amount-of-transactions limit to checkblock/CVerifyDB (master...2016/06/verify_db) https://github.com/bitcoin/bitcoin/pull/8138
4072016-06-02T15:01:09  *** molz has quit IRC
4082016-06-02T15:05:52  *** jl2012 has quit IRC
4092016-06-02T15:06:12  *** jl2012 has joined #bitcoin-core-dev
4102016-06-02T15:06:13  *** zooko has quit IRC
4112016-06-02T15:07:04  *** eragmus has quit IRC
4122016-06-02T15:07:13  *** fengling has joined #bitcoin-core-dev
4132016-06-02T15:09:26  *** eragmus has joined #bitcoin-core-dev
4142016-06-02T15:12:04  *** fengling has quit IRC
4152016-06-02T15:15:09  *** slackircbridge has quit IRC
4162016-06-02T15:15:21  *** slackircbridge has joined #bitcoin-core-dev
4172016-06-02T15:19:13  <sipa> ping for review: https://github.com/bitcoin/bitcoin/pull/7948 (RPC: augment getblockchaininfo bip9_softforks data)
4182016-06-02T15:20:03  <sipa> ping for review: https://github.com/bitcoin/bitcoin/pull/7967 ([RPC] add feerate option to fundrawtransaction)
4192016-06-02T15:23:43  *** cryptapus has quit IRC
4202016-06-02T15:23:51  *** Chris_Stewart_5 has quit IRC
4212016-06-02T15:50:25  *** BashCo has joined #bitcoin-core-dev
4222016-06-02T15:51:32  *** cryptapus has joined #bitcoin-core-dev
4232016-06-02T15:56:17  <sipa> phantomcircuit: are you seeing this:
4242016-06-02T15:56:25  <sipa> https://www.zerobin.net/?340fa0188dd35804#2Hkj8AQ8nQtI3LWQpGNrAHGnmRVeCiqVvm9BkG2+HpM=
4252016-06-02T16:00:53  <sipa> MarcoFalke: ^
4262016-06-02T16:04:22  <MarcoFalke> sipa: any zombie bitcoinds? ps -A |grep bitcoind
4272016-06-02T16:04:27  <sipa> nope
4282016-06-02T16:04:40  <MarcoFalke> reproducible?
4292016-06-02T16:04:48  <sipa> retried the test walletbackup.py itself and got a different error
4302016-06-02T16:05:05  <sipa> now trying rpc-tests.py again, and it seems to run ok
4312016-06-02T16:05:19  <sipa> at least it doesn't instantly fail like before
4322016-06-02T16:05:51  <MarcoFalke> Have you been running ./rpc-tests.py in parallel?
4332016-06-02T16:06:06  <MarcoFalke> (This doesn't work)
4342016-06-02T16:06:13  <sipa> no
4352016-06-02T16:06:28  <sipa> if you mean multiple instances of rpc-tests.py simultaneously, no
4362016-06-02T16:06:32  <MarcoFalke> jup
4372016-06-02T16:07:16  *** erasmospunk has joined #bitcoin-core-dev
4382016-06-02T16:08:43  *** fengling has joined #bitcoin-core-dev
4392016-06-02T16:13:24  *** fengling has quit IRC
4402016-06-02T16:28:20  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4412016-06-02T16:37:16  *** erasmospunk has quit IRC
4422016-06-02T16:37:50  *** BashCo has quit IRC
4432016-06-02T16:45:04  *** Thireus1 has joined #bitcoin-core-dev
4442016-06-02T16:53:52  *** Chris_Stewart_5 has quit IRC
4452016-06-02T16:57:21  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4462016-06-02T17:04:07  *** kadoban has joined #bitcoin-core-dev
4472016-06-02T17:06:35  *** erasmospunk has joined #bitcoin-core-dev
4482016-06-02T17:10:25  *** fengling has joined #bitcoin-core-dev
4492016-06-02T17:14:57  <GitHub121> [bitcoin] sipa pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/ee1533e262e7...ec45cc5e2766
4502016-06-02T17:14:58  <GitHub121> bitcoin/master 84c13e7 Wladimir J. van der Laan: chain: Add assertion in case of missing records in index db
4512016-06-02T17:14:58  <GitHub121> bitcoin/master 6030625 Wladimir J. van der Laan: test: Add more thorough test for dbwrapper iterators...
4522016-06-02T17:14:59  <GitHub121> bitcoin/master 269a440 Matt Corallo: Add test for dbwrapper iterators with same-prefix keys.
4532016-06-02T17:15:02  <GitHub78> [bitcoin] sipa closed pull request #7992: Extend #7956 with one more test. (master...16-5-7956-update) https://github.com/bitcoin/bitcoin/pull/7992
4542016-06-02T17:15:04  *** fengling has quit IRC
4552016-06-02T17:15:14  <GitHub30> [bitcoin] sipa closed pull request #8051: Seemingly fix walletbackup.py failure (master...fixwalletbackup) https://github.com/bitcoin/bitcoin/pull/8051
4562016-06-02T17:17:35  <GitHub20> [bitcoin] sipa opened pull request #8139: Fix interrupted HTTP RPC connection workaround for Python 3.5+ (master...fixwalletbackup) https://github.com/bitcoin/bitcoin/pull/8139
4572016-06-02T17:25:30  *** MarcoFalke has left #bitcoin-core-dev
4582016-06-02T17:26:25  *** Chris_Stewart_5 has quit IRC
4592016-06-02T17:31:06  *** molz has joined #bitcoin-core-dev
4602016-06-02T17:34:01  *** moli has quit IRC
4612016-06-02T17:41:15  *** Thireus1 has quit IRC
4622016-06-02T17:42:48  *** Chris_Stewart_5 has joined #bitcoin-core-dev
4632016-06-02T17:57:42  *** calibre720 has quit IRC
4642016-06-02T18:11:47  *** fengling has joined #bitcoin-core-dev
4652016-06-02T18:16:24  *** fengling has quit IRC
4662016-06-02T18:32:12  *** erasmospunk has quit IRC
4672016-06-02T18:39:07  *** Thireus1 has joined #bitcoin-core-dev
4682016-06-02T18:42:30  *** frankenmint has joined #bitcoin-core-dev
4692016-06-02T18:43:49  *** frankenm_ has joined #bitcoin-core-dev
4702016-06-02T18:44:26  *** frankenm_ is now known as frankenmint_
4712016-06-02T18:45:47  *** frankenm_ has joined #bitcoin-core-dev
4722016-06-02T18:46:09  *** erasmospunk has joined #bitcoin-core-dev
4732016-06-02T18:46:10  *** slackircbridge has quit IRC
4742016-06-02T18:46:27  *** slackircbridge has joined #bitcoin-core-dev
4752016-06-02T18:47:01  *** franken__ has joined #bitcoin-core-dev
4762016-06-02T18:47:14  *** laurentmt has joined #bitcoin-core-dev
4772016-06-02T18:47:27  *** frankenmint has quit IRC
4782016-06-02T18:47:54  *** Thireus1 has quit IRC
4792016-06-02T18:48:45  *** frankenmint_ has quit IRC
4802016-06-02T18:49:14  *** TheFactory7 has joined #bitcoin-core-dev
4812016-06-02T18:50:28  *** frankenm_ has quit IRC
4822016-06-02T18:50:33  *** GAit has joined #bitcoin-core-dev
4832016-06-02T18:50:39  *** frankenm_ has joined #bitcoin-core-dev
4842016-06-02T18:51:05  *** frankenm_ is now known as frankenmint_
4852016-06-02T18:51:47  *** franken__ has quit IRC
4862016-06-02T18:55:46  <GitHub100> [bitcoin] mrbandrews opened pull request #8141: Continuing port of java comparison tool (master...ba-comptool) https://github.com/bitcoin/bitcoin/pull/8141
4872016-06-02T18:56:01  *** moli has joined #bitcoin-core-dev
4882016-06-02T18:56:22  *** jannes_ has quit IRC
4892016-06-02T18:57:13  *** Hoshea has joined #bitcoin-core-dev
4902016-06-02T18:58:07  *** molz has quit IRC
4912016-06-02T18:58:46  *** BakSAj has joined #bitcoin-core-dev
4922016-06-02T18:58:52  <BakSAj> hi
4932016-06-02T18:59:17  <wumpus> hi
4942016-06-02T18:59:54  <BakSAj> bitcoin core live meeting now?
4952016-06-02T18:59:59  <wumpus> yes
4962016-06-02T19:00:04  <BakSAj> good :-)
4972016-06-02T19:00:10  <sipa> yes
4982016-06-02T19:00:13  <BakSAj> any outlook on merging segwit to master?
4992016-06-02T19:00:23  <wumpus> #startmeeting
5002016-06-02T19:00:23  <lightningbot> Meeting started Thu Jun  2 19:00:23 2016 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
5012016-06-02T19:00:23  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
5022016-06-02T19:00:35  <luke-jr> aww, too fast
5032016-06-02T19:00:59  <luke-jr> I was going to propose a topic very unseriously explicitly outside the meeting. :P now it's time to be serious
5042016-06-02T19:01:09  <petertodd> hi
5052016-06-02T19:01:34  <wumpus> I guess segwit is the recurring topic, any other topic proposals?
5062016-06-02T19:01:34  <gmaxwell> jtimon: Cory: morcos: sdaftuar: btcdrak: phantomcircuit: jonasschnelli:
5072016-06-02T19:01:42  <jonasschnelli> Here!
5082016-06-02T19:01:47  <sipa> cfields_!
5092016-06-02T19:02:02  <gmaxwell> I can give some updates on compactblock testing, since it seems that Matt isn't around (or if he shows up, I expect he can)
5102016-06-02T19:02:42  <wumpus> great
5112016-06-02T19:03:17  <wumpus> #topic segwit review
5122016-06-02T19:03:22  *** davec has quit IRC
5132016-06-02T19:03:49  <wumpus> let's start with that then
5142016-06-02T19:04:01  <sipa> My plan right now is making the BIP9/GBT changes, removing segnet, removing the positive witness flag, and then creating a parallel rebase
5152016-06-02T19:04:12  <sipa> with has a clean history but leads to the same tree
5162016-06-02T19:04:19  <sipa> if that is something wanted at this point
5172016-06-02T19:04:20  <wumpus> 111 comments on PR #7910, that must be a record
5182016-06-02T19:04:32  <luke-jr> "positive witness flag"?
5192016-06-02T19:05:27  <sipa> luke-jr: originally, there was a flag to the serialize code to say "serialize witnesses!"; at some point we realized that the opposite was better, as almost always you want to serialize witnesses, and there are just a few exceptions
5202016-06-02T19:05:48  <sipa> so i introduced both temporarily, and required that exactly one of both was set
5212016-06-02T19:05:51  <luke-jr> oh, I thought having it explicit was a good idea
5222016-06-02T19:06:23  <sipa> luke-jr: one reason for the opposite is that a failure where you miss the positive flag will not be detected usually, but failure to set a negative flag will
5232016-06-02T19:06:30  <sipa> luke-jr: i thought the same thing initially
5242016-06-02T19:06:59  <instagibbs> sipa, are you removing new wallet functionality as well?
5252016-06-02T19:07:07  <sipa> instagibbs: ?
5262016-06-02T19:07:09  <luke-jr> sipa: what downsides are there to the current explicit flag?
5272016-06-02T19:07:25  <sipa> 21:06:22 < sipa> luke-jr: one reason for the opposite is that a failure where you miss the positive flag will not be detected usually, but failure to set a negative flag will
5282016-06-02T19:07:37  <instagibbs> I assume you don't want users to have segwit addresses pre-rollout
5292016-06-02T19:07:41  <petertodd> luke-jr: indeed, I might have very well written it with a separate CWitnessTx class :)
5302016-06-02T19:07:53  <luke-jr> sipa: I don't understand that answer. :<
5312016-06-02T19:07:55  <instagibbs> (unless that would still be a testing branch)
5322016-06-02T19:08:31  <sipa> luke-jr: if we use a positive flag only, and we miss setting that flag somewhere, it won't easily be detected, as wherever that data goes, it will likely also accept tx without witness
5332016-06-02T19:08:45  <luke-jr> right, I'm talking about where we have both positive and negative flags..
5342016-06-02T19:08:51  <sipa> ah, that is also an option
5352016-06-02T19:08:55  *** davec has joined #bitcoin-core-dev
5362016-06-02T19:09:05  <sipa> it's just more code changes shattered all over the place
5372016-06-02T19:09:43  <luke-jr> less likely to have it accidentally wrong, though
5382016-06-02T19:09:49  <gmaxwell> With segwit in place there really isn't much further concern about getting the wrong flags, it should only have been an issue with the migration where support had to be added in many places (like RPCs) that might have less test coverage.
5392016-06-02T19:10:11  <gmaxwell> If you get it wrong in the future, the thing you changed simply won't work right. And hopefully you'll be testing the thing you just changed.
5402016-06-02T19:10:20  <gmaxwell> I don't have a strong or strongly held opinion however.
5412016-06-02T19:10:50  <luke-jr> we have a lot of outstanding PRs that may need to be updated that might not conflict
5422016-06-02T19:10:50  <jtimon> ack on cleaning history while removing the testnet
5432016-06-02T19:11:02  <wumpus> instagibbs: well in principle, master is attesting branch
5442016-06-02T19:11:35  <sipa> there are a few things we need in first, though
5452016-06-02T19:11:37  <petertodd> jtimon: you mean segnet?
5462016-06-02T19:11:39  <wumpus> jtimon: you mean removing the segnet? don't make him remove testnet too :)
5472016-06-02T19:11:44  <instagibbs> oh a testing, not attesting
5482016-06-02T19:12:03  <jtimon> petertodd: I believe sipa meant removing segwit's testnet
5492016-06-02T19:12:13  <gmaxwell> sipa: in any case, what do you think will let you get the code merged sooner?
5502016-06-02T19:12:13  <sipa> #7749 and #8083
5512016-06-02T19:12:27  <wumpus> but yes ACK on cleaning up the history before merge
5522016-06-02T19:12:37  <gmaxwell> I like cleaning history for sure.
5532016-06-02T19:12:43  <gmaxwell> Future me thanks you.
5542016-06-02T19:12:57  *** fengling has joined #bitcoin-core-dev
5552016-06-02T19:13:03  <sipa> well it definitely has to happen before merge
5562016-06-02T19:13:05  <jonasschnelli> As said, we can ACK the sha256sum of the diff (if someone cares about integrity of an ACK)
5572016-06-02T19:13:08  <sipa> the question whether the time is now for that
5582016-06-02T19:13:17  <luke-jr> (GBT VB as well)
5592016-06-02T19:13:21  <sipa> jonasschnelli: git internally has tree hashes
5602016-06-02T19:13:35  <jonasschnelli> sipa: Nice. That should work.
5612016-06-02T19:14:13  <sipa> So please review #7749 and #8083
5622016-06-02T19:14:26  <luke-jr> did we get anywhere with the fundrawtransaction issue?
5632016-06-02T19:14:40  <sipa> luke-jr: i can't remember the conclusion there
5642016-06-02T19:14:49  <jonasschnelli> Re 8083: im finalizing the seeder part (nits and filterbitmap-whitelist)
5652016-06-02T19:14:49  <wumpus> q: is everything in the segwit PR meant to go in at once, or will it be split up in to, say, a pull with consensus/network changes, following up with wallet, and GUI changes
5662016-06-02T19:15:03  <wumpus> #action review #7749 and #8083
5672016-06-02T19:15:07  <luke-jr> sipa: IIRC, 1) it's a problem; 2) we won't change consensus code to fix it
5682016-06-02T19:15:26  <luke-jr> I don't know of any actual resolution to the problem :/
5692016-06-02T19:15:41  <gmaxwell> wumpus: If sipa does the "rewrite history to result in the same code thing" then if it is is PR split they will need to go in one right after another to preserve the "same code" property.
5702016-06-02T19:15:56  <gmaxwell> (I think)
5712016-06-02T19:16:02  <sipa> wumpus: that's a good question; the history is broken into sections already
5722016-06-02T19:16:06  <sipa> so we can decide later
5732016-06-02T19:16:21  <sipa> though... maybe not
5742016-06-02T19:16:26  <gmaxwell> I suppose sipa should show "same code" at one point, then split, and the remaining parts could then change.
5752016-06-02T19:16:38  <sipa> the unit/rpc/p2p tests rely on the wallet code
5762016-06-02T19:16:39  <wumpus> gmaxwell: I don't have a strong opinion either way, if this change is sufficiently atomic it makes sense to do it at once, but see e.g. instagibbs's comment about wallet addresses
5772016-06-02T19:17:01  *** laurentmt has quit IRC
5782016-06-02T19:17:12  <wumpus> maybe hide it behind an option in the beginning?
5792016-06-02T19:17:23  <gmaxwell> My recommendation for the wallet parts was to just hide the changed functionality there behind a hidden configuration option that the tests could use.
5802016-06-02T19:17:28  <jtimon> well, the wallet code can be maybe be introduced disabled for users with a constant to be removed later or something?
5812016-06-02T19:17:29  <sipa> maybe we can disable addwitnessaddress before the softfork
5822016-06-02T19:17:32  <wumpus> gmaxwell: right
5832016-06-02T19:17:37  <sipa> that would make sense
5842016-06-02T19:17:44  *** fengling has quit IRC
5852016-06-02T19:17:47  <wumpus> jtimon: hey we're all saying the same :)
5862016-06-02T19:17:51  <jtimon> wumpus: yes
5872016-06-02T19:17:52  <sipa> yay, agreement
5882016-06-02T19:17:57  <sipa> i have another question
5892016-06-02T19:18:03  <gmaxwell> sipa: what is the meaning of life?
5902016-06-02T19:18:06  <sipa> 42
5912016-06-02T19:18:06  <jonasschnelli> :)
5922016-06-02T19:18:14  <gmaxwell> thats an answer, not a question!
5932016-06-02T19:18:32  <luke-jr> he has both the answer and a question
5942016-06-02T19:18:34  <gmaxwell> We're going to need to build a bigger computer...
5952016-06-02T19:18:36  <sipa> jl2012 brought up the issue that our witness program definition is limited to 16 versions, and that is not easy to extend without introducing a new witness storage
5962016-06-02T19:18:57  <sipa> there is an easy solution, by allowing witness programs to be slightly larger
5972016-06-02T19:19:01  <sipa> but this is a hard forking change
5982016-06-02T19:19:11  <luke-jr> sipa: it is? I thought the version could be any length?
5992016-06-02T19:19:14  <sipa> which, if done unconditionally, could hardfork testnet
6002016-06-02T19:19:22  <sipa> luke-jr: nope, has to be OP_0 through OP_16
6012016-06-02T19:19:26  <luke-jr> :/
6022016-06-02T19:19:32  <luke-jr> why limit it?
6032016-06-02T19:19:42  <jtimon> then would be something to move to the later hardfork, no?
6042016-06-02T19:20:01  <sipa> jtimon: i don't like relying on that
6052016-06-02T19:20:08  <gmaxwell> Oh ... how'd that happen? in any case, you can simply use 0..16 and then use another byte after that one.
6062016-06-02T19:20:16  <luke-jr> jtimon: we cannot assume there will ever be a hardfork.
6072016-06-02T19:20:18  <sipa> gmaxwell: max 32 bytes can follow
6082016-06-02T19:20:30  <gmaxwell> okay thats broken.
6092016-06-02T19:20:43  * luke-jr doesn't see the purpose of restricting the witness-triggering sPK like that
6102016-06-02T19:20:45  <jtimon> well, the plan was to deploy segwit as a softfork
6112016-06-02T19:21:08  <sipa> jtimon: changing it is a HF with respect to the current SW code
6122016-06-02T19:21:08  <gmaxwell> My assumption was that it would be arbritary and extended by just adding more bytes when the space ran out.
6132016-06-02T19:21:13  <sipa> jtimon: not with respect to bitcoin
6142016-06-02T19:21:34  <sipa> jtimon: so we can change it before merge while leaving SW a SF
6152016-06-02T19:21:36  <jtimon> sipa: oh, I see, it would still be a SF to bitcoin. ok
6162016-06-02T19:21:54  <sipa> gmaxwell, luke-jr: the reason was that constant size utxos are nice
6172016-06-02T19:21:55  <gmaxwell> sipa: testnet segwit rules can be changed so activiation doesn't begin until later.
6182016-06-02T19:22:06  <gmaxwell> So at worst that would be a reindex for testnet nodes, no?
6192016-06-02T19:22:11  <sipa> gmaxwell: ... segwit is already activated on testnet
6202016-06-02T19:22:16  <gmaxwell> yes, so?
6212016-06-02T19:22:16  <luke-jr> sipa: they're already non-constant size.
6222016-06-02T19:22:22  <sipa> luke-jr: ?
6232016-06-02T19:22:23  <gmaxwell> sipa: move it forward, reindex.
6242016-06-02T19:22:24  <jtimon> testnet4 ?
6252016-06-02T19:22:27  <gmaxwell> luke-jr: he means in the future.
6262016-06-02T19:22:36  <gmaxwell> sipa: in any case, 16 is unacceptably too few.
6272016-06-02T19:22:40  <sipa> agree
6282016-06-02T19:23:26  <sipa> gmaxwell: hmm, ok
6292016-06-02T19:23:31  <instagibbs> The bip doesn't read like it would be a HF, but maybe I'm being obtuse
6302016-06-02T19:23:37  <gmaxwell> I don't think constant size is as interesting as constant bound. so if you wanted to say that the version was limited to N bytes for some small N that would be fine. 4294967296 versions should be enough for anyone.
6312016-06-02T19:23:38  <luke-jr> gmaxwell: in the future, if we softfork out the current long sPKs, we can softfork a limit on witness sPK length as well
6322016-06-02T19:23:52  *** frankenmint_ is now known as frankenmint
6332016-06-02T19:23:58  <sipa> fair enough, will do
6342016-06-02T19:24:09  <sipa> limit to 36 or 40 bytes or so
6352016-06-02T19:24:10  <jtimon> instagibbs: it would be a HF only for testnet3 which has already deployed "old segwit"
6362016-06-02T19:24:31  <instagibbs> jtimon, no I mean, anything where version is 1+ has no consensus meaning yet
6372016-06-02T19:24:39  <gmaxwell> sipa: cool (also, testnet behavior has never been in a merged much less released version, I don't mind breaking it)
6382016-06-02T19:24:41  <instagibbs> when it's not simply 0
6392016-06-02T19:25:15  <sipa> instagibbs: the problem is that something of the form "1 + 33 bytes" is not treated as a witness program, and is not allowed to have any witness data
6402016-06-02T19:25:32  <sipa> we can extend the definition of a witness program, but it would require a new witness structure
6412016-06-02T19:25:48  <sipa> as old nodes won't relay witness data with something they don't treat as a witness output
6422016-06-02T19:25:59  <sipa> (and that rule is necessary to prevent spam)
6432016-06-02T19:26:26  *** laurentmt has joined #bitcoin-core-dev
6442016-06-02T19:26:27  <sipa> anyway, ok, will just change the rule
6452016-06-02T19:26:36  <sipa> other topics?
6462016-06-02T19:26:42  <wumpus> yes
6472016-06-02T19:26:45  <luke-jr> old nodes won't relay anything with witness data they can't verify anyway..
6482016-06-02T19:26:48  <sipa> (or more segwit review related points)
6492016-06-02T19:26:56  <wumpus> #topic compactblock testing
6502016-06-02T19:27:00  <sipa> luke-jr: old nodes in this case being witness v0 nodes
6512016-06-02T19:27:05  <instagibbs> luke-jr, I'm not convinced, but I think fixing now is better anyways so whatever
6522016-06-02T19:27:06  <luke-jr> sipa: yes, I also mean these
6532016-06-02T19:27:11  <wumpus> @gmaxwell
6542016-06-02T19:27:13  <luke-jr> IMO make any length limit a relay policy only.
6552016-06-02T19:27:21  <sipa> we'll discuss further in #segwit-dev
6562016-06-02T19:27:24  <luke-jr> k
6572016-06-02T19:27:27  <instagibbs> ack
6582016-06-02T19:28:00  <jtimon> compackblock
6592016-06-02T19:28:23  <gmaxwell> Okay. So matt has built a parallel relay network that uses compact blocks and the UDP network block coding stuff.
6602016-06-02T19:28:24  <sipa> i rebased BlueMatt's compactblock patch on top of the shared_ptr mempool change, and gmaxwell and i have been succesfully running it across the internet
6612016-06-02T19:28:38  <sipa> ^ and that's more interesting
6622016-06-02T19:28:54  <gmaxwell> ^ a number of people-- including some large miners-- are running both Sipa's rebase,  and Matt's PR without the rebase on public nodes.
6632016-06-02T19:29:16  <gmaxwell> Which are also connecting to matt's nodes.
6642016-06-02T19:29:31  <wumpus> good to hear
6652016-06-02T19:29:33  <gmaxwell> (I got bored with the simulated topology lab testing, this is potentially more interesting)
6662016-06-02T19:29:39  <sipa> 2016-06-02 18:36:13.412286 Successfully reconstructed block 0000000000000000014a6a924544dd097d538314281bebd95c89e50726e7d2ef with 1 txn prefilled, 2708 txn from mempool and 0 txn requested
6672016-06-02T19:29:43  <sipa> 2016-06-02 18:37:46.728092 Successfully reconstructed block 000000000000000001943282946e9579fd84def95df577ebb8bcda3b8d9f4d06 with 1 txn prefilled, 1529 txn from mempool and 0 txn requested
6682016-06-02T19:29:47  <sipa> 2016-06-02 18:43:32.713909 Successfully reconstructed block 000000000000000000499e1485726cd87003e7a6400118f8c061171748665612 with 1 txn prefilled, 1102 txn from mempool and 3 txn requested
6692016-06-02T19:29:52  <wumpus> yes, always good to test with real world data as well
6702016-06-02T19:29:55  *** GAit has quit IRC
6712016-06-02T19:30:03  <gmaxwell> I don't know that there is much to report, it's working as expected.  :)   Sipa's rebase on the sharedptr is pretty nice.
6722016-06-02T19:30:23  <instagibbs> gmaxwell, the rebase includes the predicted transactions?
6732016-06-02T19:30:29  <BakSAj> nice
6742016-06-02T19:30:31  <gmaxwell> As it also eliminates transaction duplication from the relay pool, and eliminates a fair bit of transaction copying.
6752016-06-02T19:30:31  <wumpus> is this branch available somewhere?
6762016-06-02T19:30:33  <sipa> instagibbs: it only sends the coinbase directly
6772016-06-02T19:30:38  <instagibbs> sipa, ah k
6782016-06-02T19:30:52  <gmaxwell> instagibbs: no, right now I don't believe anyone is running something with fancy prediction.  I think we'll leave that out in the initial PR. Easily added later.
6792016-06-02T19:30:52  <sipa> wumpus: https://github.com/sipa/bitcoin/commits/compactblocks
6802016-06-02T19:31:39  <wumpus> #link sipa's rebase of compactblocks on top of PR #8126: https://github.com/sipa/bitcoin/commits/compactblocks
6812016-06-02T19:31:54  <wumpus> #action review PR #8126
6822016-06-02T19:32:01  <gmaxwell> if other developers here are interested in running nodes connected to these nodes, lemme know and I'll give you addresses to connect to.
6832016-06-02T19:32:17  <wumpus> I'm interested
6842016-06-02T19:32:28  <gmaxwell> I should take an action to setup a couple on published addresses too, for people to connect to without asking. :)
6852016-06-02T19:32:59  <wumpus> yes, that always works best :)
6862016-06-02T19:33:15  <wumpus> any other topics?
6872016-06-02T19:33:19  <luke-jr> is sipa's rebase different enough that we ought to switch to reviewing that instead?
6882016-06-02T19:33:20  <gmaxwell> Yes, though they may get DDOS attacked, which is harmless but would waste time sorting out the issue. :)
6892016-06-02T19:33:41  <sipa> luke-jr: it's less code than the original :)
6902016-06-02T19:33:55  <wumpus> gmaxwell: you mean thoroughly stress-tested :)
6912016-06-02T19:34:03  * gmaxwell stabs
6922016-06-02T19:34:16  <gmaxwell> Does anyone know the current CFPF status?
6932016-06-02T19:34:34  <wumpus> #topic current CPFP status
6942016-06-02T19:34:43  <sipa> gmaxwell: review #7598
6952016-06-02T19:34:51  <luke-jr> afaik no show-stoppers found, but more review needed; there's a dep PR to get in first though
6962016-06-02T19:34:52  <wumpus> #action  review #7598
6972016-06-02T19:35:08  <sipa> it's a blocker/dependency for CPFP (#7600)
6982016-06-02T19:36:18  *** BashCo has joined #bitcoin-core-dev
6992016-06-02T19:37:33  <gmaxwell> I've been struggling a bit with too many PRs outstanding at once that I want to test.
7002016-06-02T19:37:35  <wumpus> #link CPFP is that like 'strong AI should be here in less than 20 years'
7012016-06-02T19:37:41  <wumpus> EH
7022016-06-02T19:37:47  <luke-jr> :<
7032016-06-02T19:37:48  <wumpus> #link https://github.com/bitcoin/bitcoin/pull/7600
7042016-06-02T19:38:01  <wumpus> (sorry, copy paste messup)
7052016-06-02T19:38:04  <gmaxwell> Merging them is a pain.  (thanks to sipa for merging a lot of things recently!)
7062016-06-02T19:38:48  <sipa> i've been going through all PRs... there are so many decent-but-not-quite-finished ones in the queue...
7072016-06-02T19:38:50  <wumpus> I've lost overview a bit
7082016-06-02T19:38:58  <wumpus> any PRs that should be close to be able to be merged?
7092016-06-02T19:39:17  <wumpus> sipa: yes, I've noticed
7102016-06-02T19:39:21  <jonasschnelli> IMO https://github.com/bitcoin/bitcoin/pull/7957 can be merged (save, tool only)
7112016-06-02T19:39:29  <gmaxwell> Mostly my minor difficulty there just comes from many things touching the same parts, and a lot of that was actually my fault. (e.g. opening three PRs at once that all conflicted with each other. :) )
7122016-06-02T19:39:51  <luke-jr> I'd like to see key generation type merged so there's no risk of other wallet upgrades conflicting it since these wallets are in the wild
7132016-06-02T19:39:56  <sipa> 17:19:13 < sipa> ping for review: https://github.com/bitcoin/bitcoin/pull/7948 (RPC: augment getblockchaininfo bip9_softforks data)
7142016-06-02T19:39:59  <sipa> 17:20:03 < sipa> ping for review: https://github.com/bitcoin/bitcoin/pull/7967 ([RPC] add feerate option to fundrawtransaction)
7152016-06-02T19:40:23  <wumpus> jonasschnelli: agree, but that one is probably not blocking anything
7162016-06-02T19:40:37  <sipa> also 7997
7172016-06-02T19:41:07  <jonasschnelli> I'm requesting permission to merge [docs] and [tools] PRs
7182016-06-02T19:41:20  <wumpus> jonasschnelli: sure
7192016-06-02T19:42:09  <gmaxwell> sounds fine, I know we sometimes don't get enough review on those, esp docs. Please feel empowered to nag people to review things.
7202016-06-02T19:42:11  <jonasschnelli> Okay. Will try to focus on trivial docs PRs, so wumpus and sipa can take care about the corish ones.
7212016-06-02T19:42:24  <luke-jr> https://github.com/bitcoin/bitcoin/pull/7935 is ready IMO
7222016-06-02T19:42:46  <wumpus> the problem with doc pulls is usually that they get review comments but the author disappears for large spans of time
7232016-06-02T19:42:47  <sipa> luke-jr: agree
7242016-06-02T19:43:10  <sipa> luke-jr: will do final review and reack
7252016-06-02T19:43:28  <cfields_> luke-jr: that looked good to me last time I checked. I'll re-review as well.
7262016-06-02T19:44:52  <sipa> also #7825 is good
7272016-06-02T19:45:26  <sipa> and #7942
7282016-06-02T19:46:23  <jonasschnelli> Also have a look at https://github.com/bitcoin/bitcoin/pull/7946 (could speed up things a little bit)
7292016-06-02T19:46:26  <wumpus> #7942 has an unaddressed comment by sipa
7302016-06-02T19:46:56  <sipa> tiny nit :)
7312016-06-02T19:47:19  *** skang404 has joined #bitcoin-core-dev
7322016-06-02T19:47:21  <jonasschnelli> nit is nit!
7332016-06-02T19:47:38  <wumpus> that's not always clear to me whether it should block merging
7342016-06-02T19:47:46  <wumpus> (I usually at least wait for the author to respond)
7352016-06-02T19:48:08  <sipa> the author is active, he probably just missed it
7362016-06-02T19:48:16  <sipa> jonasschnelli just pinged
7372016-06-02T19:48:21  <wumpus> ok good
7382016-06-02T19:48:24  <luke-jr> oh, topic: 0.11.next
7392016-06-02T19:49:16  <jonasschnelli> luke-jr, I guess we already discussed the 0.11 maintenance?
7402016-06-02T19:49:16  <wumpus> ok
7412016-06-02T19:49:21  <wumpus> #topic 0.11.next
7422016-06-02T19:49:53  <luke-jr> jonasschnelli: ?
7432016-06-02T19:50:12  <sipa> 0.11 goes to critical fixes only when 0.13 is released, right?
7442016-06-02T19:50:13  <jtimon> luke-jr: 0.11.next is supposed to include csv but not segwit, right?
7452016-06-02T19:50:33  <jonasschnelli> I had in mind we we BP to 0.12, 0.13 and only security stuff to 0.11?
7462016-06-02T19:50:37  <luke-jr> jtimon: unless it gets delayed until segwit is merged, I guess
7472016-06-02T19:50:52  <wumpus> is there any urgent reason to do a 0.11 release?
7482016-06-02T19:50:53  <luke-jr> sipa: unless someone decides to maintain it longer
7492016-06-02T19:51:00  <luke-jr> wumpus: CSV support, at least
7502016-06-02T19:51:05  <wumpus> right
7512016-06-02T19:51:20  <jtimon> wumpus: is there any reason not to do it while things are backported and tested?
7522016-06-02T19:51:24  <gmaxwell> looking at the network I'm not seeing any evidence of need to maintain 0.11 extensively, also we called for people running older versions in operations and got crickets in response, AFAIK.
7532016-06-02T19:51:33  *** belcher has joined #bitcoin-core-dev
7542016-06-02T19:51:35  * jonasschnelli checking the seeder
7552016-06-02T19:51:36  <wumpus> jtimon: developer overhead
7562016-06-02T19:51:49  <sipa> jtimon: is it tested?
7572016-06-02T19:51:50  <jtimon> wumpus: well, that's luke-jr's problem, isn't it?
7582016-06-02T19:52:33  <luke-jr> gmaxwell: we did? I don't recall seeing the "call for", and I know for a fact that Eligius relies on 0.11 for now
7592016-06-02T19:52:58  <jonasschnelli> 88  0.11  nodes
7602016-06-02T19:53:06  <jtimon> sipa: I don't know, I'm not reviewing or testing myself, but if luke-jr gets review and testing...
7612016-06-02T19:53:19  <sipa> luke-jr: perhaps the time is better spent in upgrading eligius then :)
7622016-06-02T19:53:28  <luke-jr> jtimon: which is unlikely, hence the history of git/rc only stable stuff :P
7632016-06-02T19:53:37  <luke-jr> sipa: probably.
7642016-06-02T19:53:49  <gmaxwell> esp since master will hopefully have CPFP soon.
7652016-06-02T19:53:51  <luke-jr> jonasschnelli: what?
7662016-06-02T19:53:56  <wumpus> yes, interest in older major versions is extrememly low
7672016-06-02T19:54:09  <btcdrak> there is a backport PR for 0.11 for CSV etc. but we sort of semi decided back then it was not urgent and much more risky.
7682016-06-02T19:54:14  <luke-jr> I see 1768 0.11 nodes.
7692016-06-02T19:54:28  <btcdrak> the BIP68 backport in particular is complex
7702016-06-02T19:54:40  <jtimon> my only point is that we shouldn't use the "we only promise to maintain the last 2 versions" as an artificial limitation beyond review and testing. If people are interested in working on that...
7712016-06-02T19:54:41  <gmaxwell> well in particular, interest in _updates_ for old versions is low...
7722016-06-02T19:54:55  <wumpus> gmaxwell: yes that's what I mean...
7732016-06-02T19:54:59  <luke-jr> we should remove the promise if we're not going to uphold it.
7742016-06-02T19:55:12  <wumpus> jtimon: the problem is that people are not interested
7752016-06-02T19:55:17  <gmaxwell> what promise?
7762016-06-02T19:55:26  <jtimon> well "promise"
7772016-06-02T19:55:35  <gmaxwell> also, not "not going"-- not able.. without people to test you really can't provide good releases.
7782016-06-02T19:55:48  <jtimon> wumpus: by people you mean users or reviewers/testers?
7792016-06-02T19:55:50  <gmaxwell> not doing someone a service to put out a barely tested update.
7802016-06-02T19:55:57  <wumpus> I mean if luke-jr really wants to handle a release by himself I'm not going to protest
7812016-06-02T19:56:05  <gmaxwell> ^ agreed.
7822016-06-02T19:56:05  <luke-jr> https://bitcoincore.org/en/lifecycle/ mentions something; whether promise or able or not, it should be updated if we can't do it
7832016-06-02T19:56:10  <btcdrak> the CSV backport PR was https://github.com/bitcoin/bitcoin/pull/7716
7842016-06-02T19:56:22  <btcdrak> we did pretty much decide not to merge it.
7852016-06-02T19:56:23  <luke-jr> wumpus: I can't get testing/reviews by myself.
7862016-06-02T19:56:28  <jonasschnelli> luke-jr: sorry,.. wrong file: we have 743 0.11x nodes and 1786 0.12.x nodes... so yes. CSV for 0.11 makes sense.
7872016-06-02T19:56:41  <luke-jr> I can maintain stable branches, but releases seem unlikely to work out at this point
7882016-06-02T19:56:54  <wumpus> but there's so much happening on master right now, and 0.13 release is near, I can't promise I will be able to spend any time on it (except gitian building / uploading executables)
7892016-06-02T19:57:23  <jonasschnelli> Yes. 0.11 is certainly not something for wumpus (would be a waste of his time)
7902016-06-02T19:57:57  <gmaxwell> Without people interested in using it, we can't get much platform qualification which is a lot of the difference between a branch and a release.
7912016-06-02T19:57:59  <wumpus> so if you want to do this: please create your own 0.11 branch, tag, do the release notes etc
7922016-06-02T19:58:00  <sipa> jtimon: i have 1093 'good' 0.11 nodes
7932016-06-02T19:58:09  <sipa> eh, jonasschnelli:
7942016-06-02T19:58:33  <jonasschnelli> good is not good enough...
7952016-06-02T19:58:34  <jonasschnelli> cat dnsseed.dump | grep " 100.00% 100.00% 100.00% " | grep "Satoshi:0.11." | wc -l
7962016-06-02T19:59:05  <sipa> anyway, besided the point
7972016-06-02T19:59:14  <sipa> we can't do releases without people interested in running and testing
7982016-06-02T19:59:17  <wumpus> yes, meeting time over
7992016-06-02T19:59:21  <sipa> oh, that too
8002016-06-02T19:59:24  <wumpus> #endmeeting
8012016-06-02T19:59:24  <lightningbot> Meeting ended Thu Jun  2 19:59:24 2016 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
8022016-06-02T19:59:24  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-06-02-19.00.html
8032016-06-02T19:59:24  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-06-02-19.00.txt
8042016-06-02T19:59:24  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-06-02-19.00.log.html
8052016-06-02T19:59:35  <jtimon> maybe we can merge it into the 0.11 branch without doing a release?
8062016-06-02T20:00:01  <wumpus> jtimon: sure. luke-jr can run his own 0.11 branch, I could just take that over
8072016-06-02T20:00:18  *** cryptapus has quit IRC
8082016-06-02T20:00:18  <gmaxwell> could be done, I suppose. But thing is: soft forks are more or less safe to run without, but a broken update may be less safe.
8092016-06-02T20:00:27  <gmaxwell> not that I think there is a lot of risk there.
8102016-06-02T20:00:37  <jtimon> and if it ever gets tested, release, otherwise there's never a 0.11.next
8112016-06-02T20:00:42  <wumpus> well if no one runs it, it doesn't create much risk either :)
8122016-06-02T20:00:56  <gmaxwell> Good point.
8132016-06-02T20:01:48  <luke-jr> wumpus: should I continue to maintain stable branches in a separate repo (the old one was on Gitorious which is dead, so a new location needs deciding if so), or would it make sense to do it on the main repo now (would require push access for me, at least in the stable branches)?
8142016-06-02T20:01:58  <luke-jr> [20:00:18] <gmaxwell> could be done, I suppose. But thing is: soft forks are more or less safe to run without, but a broken update may be less safe. <-- my main concern with backporting of the recent softforks
8152016-06-02T20:02:43  <wumpus> luke-jr: well you don't strictly need push access; I could just pull the 0.11 branch from your repo when you say so
8162016-06-02T20:02:43  <jtimon> can't we just merge PRs in the 0.11 branch in the main repo?
8172016-06-02T20:02:47  <btcdrak> luke-jr: morcos specific concern was the safety of BIP68 backport
8182016-06-02T20:02:54  <luke-jr> wumpus: that'd work too I guess
8192016-06-02T20:02:59  <btcdrak> the backports were done in #7716
8202016-06-02T20:03:05  <jtimon> or that, I'll shut up, you people coordinate
8212016-06-02T20:03:52  <btcdrak> The codebase is significantly different between 0.11 and 0.12 with regards to BIP68
8222016-06-02T20:03:58  <luke-jr> it probably would make sense to have a separate repo for stable in general, though, so we can't accidentally confuse PRs against the wrong branch
8232016-06-02T20:05:20  <wumpus> at least the github merge script now automatically gets the branch to merge against from github
8242016-06-02T20:05:39  <gmaxwell> luke-jr: is there any thing I could help do to get eligius off of 0.11 and to 0.12.1  (and maybe master with CPFP?)
8252016-06-02T20:07:20  <luke-jr> gmaxwell: my current target is Knots 0.12.1 including CPFP there, so the big step is backporting CPFP in a way that can be turned on/off (which AIUI, the CPFP dep makes easier); I don't think there's a good way to divide this work, however
8262016-06-02T20:08:31  <gmaxwell> luke-jr: ugh.
8272016-06-02T20:08:59  <gmaxwell> I don't see what benefit you get from 0.12.1 with such a large and invasive backport.
8282016-06-02T20:09:36  <luke-jr> as opposed to everything else in master? :x
8292016-06-02T20:10:48  <gmaxwell> which has a lot more eyes on it, and in the mining relevant codepaths, the changes for CPFP are probably the bulk of the changes.
8302016-06-02T20:11:08  <gmaxwell> also, look ahead a bit, and you'd have to forward port that backport onto segwit.
8312016-06-02T20:13:35  <luke-jr> hmm
8322016-06-02T20:14:27  *** fengling has joined #bitcoin-core-dev
8332016-06-02T20:15:06  <BakSAj> i understand the need for backporting in enterprise software, where upgrades might get messy, but what is the exact point in btc where process is quite simple...?
8342016-06-02T20:15:09  <luke-jr> not sure it'd be less work to forward-port Knots to master either, though
8352016-06-02T20:15:47  *** laurentmt has quit IRC
8362016-06-02T20:15:54  <luke-jr> BakSAj: consensus systems carry even more risk in upgrades, than enterprise
8372016-06-02T20:16:01  <gmaxwell> BakSAj: there are many large businesses that use Bitcoin too, and some that have complex customizations.
8382016-06-02T20:16:05  <sipa> luke-jr: what other kinds of changes does Knots have?
8392016-06-02T20:16:43  <gmaxwell> BakSAj: unfortunately, though we know these deployments exist-- we seldom hear from people involved with them.
8402016-06-02T20:18:43  <luke-jr> sipa: http://bitcoinknots.org/ has the full list, but relevant to Eligius.. *skims over list* #7149, #7533, and spamfilter; so maybe easier than backporting CPFP
8412016-06-02T20:18:48  <BakSAj> i wonder if they do patching
8422016-06-02T20:19:08  <BakSAj> otherwise its just a waste of dev that can go to master version
8432016-06-02T20:19:24  *** fengling has quit IRC
8442016-06-02T20:19:32  <luke-jr> would be nice to get #7149 reviewed and merged finally.
8452016-06-02T20:20:58  <btcdrak> This was the summary regarding the backports, for the record https://bitcoincore.org/en/meetings/2016/03/31/#softfork-backports
8462016-06-02T20:28:00  <BakSAj> ok, thanks for clarification
8472016-06-02T20:29:20  <BakSAj> i hope 0.12.1 will activate soon, so you guys can move on with segwit and get lightning finally
8482016-06-02T20:29:54  <BakSAj> anybody happen to know when e.g. f2pool moves to 0.12.1 ?
8492016-06-02T20:34:53  *** rubensayshi has quit IRC
8502016-06-02T20:35:40  *** face has quit IRC
8512016-06-02T20:36:06  *** erasmospunk has quit IRC
8522016-06-02T20:37:12  *** skang404 has quit IRC
8532016-06-02T20:37:47  *** skang404 has joined #bitcoin-core-dev
8542016-06-02T20:41:12  *** Hoshea has quit IRC
8552016-06-02T20:43:35  *** dgenr8 has joined #bitcoin-core-dev
8562016-06-02T20:45:39  *** BakSAj has quit IRC
8572016-06-02T21:15:49  *** fengling has joined #bitcoin-core-dev
8582016-06-02T21:20:44  *** fengling has quit IRC
8592016-06-02T21:40:36  *** AaronvanW has quit IRC
8602016-06-02T22:10:48  *** GAit has joined #bitcoin-core-dev
8612016-06-02T22:17:20  *** fengling has joined #bitcoin-core-dev
8622016-06-02T22:21:44  *** fengling has quit IRC
8632016-06-02T22:45:45  <GitHub51> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/ec45cc5e2766...f972b04d63eb
8642016-06-02T22:45:45  <GitHub51> bitcoin/master 0bf6f30 Pedro Branco: Prevent multiple calls to ExtractDestination
8652016-06-02T22:45:45  <GitHub51> bitcoin/master f972b04 Pieter Wuille: Merge #7825: Prevent multiple calls to ExtractDestination...
8662016-06-02T22:45:49  <GitHub30> [bitcoin] sipa closed pull request #7825: Prevent multiple calls to ExtractDestination (master...enhancement/prevent-multiple-calls-extractdestination) https://github.com/bitcoin/bitcoin/pull/7825
8672016-06-02T22:53:48  *** spudowiar is now known as spudowiar-banned
8682016-06-02T22:54:49  *** spudowiar-banned has quit IRC
8692016-06-02T22:58:24  *** spudowiar has joined #bitcoin-core-dev
8702016-06-02T23:01:23  *** Guyver2 has quit IRC
8712016-06-02T23:06:52  *** GAit has quit IRC
8722016-06-02T23:18:33  *** fengling has joined #bitcoin-core-dev
8732016-06-02T23:18:43  *** spudowiar is now known as spudowiar-ban
8742016-06-02T23:18:54  *** randy-waterhouse has joined #bitcoin-core-dev
8752016-06-02T23:20:50  *** spudowiar-ban is now known as spudowiar
8762016-06-02T23:21:40  *** kadoban is now known as kadoban_1ufYVpS
8772016-06-02T23:23:24  *** fengling has quit IRC
8782016-06-02T23:27:14  <GitHub138> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f972b04d63eb...a82f03393a32
8792016-06-02T23:27:14  <GitHub138> bitcoin/master 9805f4a Kaz Wesley: mapNextTx: use pointer as key, simplify value...
8802016-06-02T23:27:15  <GitHub138> bitcoin/master a82f033 Pieter Wuille: Merge #7997: replace mapNextTx with slimmer setSpends...
8812016-06-02T23:27:19  <GitHub69> [bitcoin] sipa closed pull request #7997: replace mapNextTx with slimmer setSpends (master...setSpends) https://github.com/bitcoin/bitcoin/pull/7997
8822016-06-02T23:36:32  *** kadoban_1ufYVpS is now known as kadoban
8832016-06-02T23:39:39  *** randy-waterhouse has quit IRC
8842016-06-02T23:54:04  *** AaronvanW has joined #bitcoin-core-dev