12015-11-16T00:19:28  *** d4de has joined #bitcoin-core-dev
  22015-11-16T00:27:00  <dcousens> the *.dat files, they are assumed to be in order yeah?  I know the blocks in them may be out of order, but the files themselves are in order yeah?
  32015-11-16T00:28:51  <sipa> what does order for files mean, if the blocks are not in order?
  42015-11-16T00:28:59  <dcousens> sequential*
  52015-11-16T00:29:25  <dcousens> in that, data from blk001.dat "flows" into blk002.dat
  62015-11-16T00:29:31  <sipa> yes
  72015-11-16T00:29:42  <sipa> :)
  82015-11-16T00:29:53  <sipa> though that has been broken in the past
  92015-11-16T00:30:01  <dcousens> sipa: oh?
 102015-11-16T00:30:26  <dcousens> (having troubles with a parser, it makes it 370k blocks in before blowing up with the magic header being wrong)
 112015-11-16T00:30:28  <sipa> it would at startup rescan through all block files the first time a block was written, and if there was space in one, it would go there
 122015-11-16T00:30:31  <dcousens> (running a re-index now)
 132015-11-16T00:33:11  <dcousens> sipa: is there a separate index for the blocks in the .dat files?
 142015-11-16T00:33:31  <sipa> that's called the block index :p
 152015-11-16T00:33:38  <sipa> $DATADIR/blocks/indx
 162015-11-16T00:33:39  <sipa> $DATADIR/blocks/index
 172015-11-16T00:34:05  <dcousens> haha
 182015-11-16T00:34:24  <dcousens> Sure, is that referencing file pointers?
 192015-11-16T00:34:34  <dcousens> like, file+offset pointers
 202015-11-16T00:34:58  <sipa> indeed
 212015-11-16T00:35:04  <sipa> file number, byte offset
 222015-11-16T00:35:23  <dcousens> and even then, the blocks would still be back to back right, there shouldn't be empty space out of order data etc?
 232015-11-16T00:35:43  <dcousens> like,  I wouldn't expect a block to flow from 001.dat to 003.dat,
 242015-11-16T00:35:49  <dcousens> (that is, skipping 002.dat)
 252015-11-16T00:35:52  <sipa> that's certainly possible
 262015-11-16T00:35:56  <sipa> there are really no guarantees
 272015-11-16T00:36:24  <sipa> we try to put blocks in order
 282015-11-16T00:36:33  <sipa> to the extent possible
 292015-11-16T00:36:45  <sipa> but only by limiting how much out-of-order blocks we fetch
 302015-11-16T00:36:45  <dcousens> sipa: wouldn't that mean the block index would have to maintain multiple file number/offsets?
 312015-11-16T00:36:50  <sipa> how so?
 322015-11-16T00:36:52  <dcousens> per block
 332015-11-16T00:37:03  <sipa> oh! a block is always in only exactly one file
 342015-11-16T00:37:05  <sipa> continuously
 352015-11-16T00:37:20  <dcousens> with a possibility to flow into the next file, right?
 362015-11-16T00:37:30  <sipa> how do you mean 'flow into' ?
 372015-11-16T00:37:38  <dcousens> half in blk001.dat, half in blk002.dat
 382015-11-16T00:37:40  <sipa> no
 392015-11-16T00:37:49  <sipa> always continuously in one file
 402015-11-16T00:37:58  <sipa> if file N can't fit it, it goes into file N+!
 412015-11-16T00:38:09  <sipa> which is why the block files aren't exactly the same size :)
 422015-11-16T00:40:47  <dcousens> ok! easy
 432015-11-16T00:41:06  <dcousens> but the block files themselves could contain complete garbage data mixed with actual data
 442015-11-16T00:41:09  <dcousens> conceptually
 452015-11-16T00:41:30  <sipa> yes
 462015-11-16T00:41:48  <sipa> though that too is something we try to prevent :)
 472015-11-16T00:42:41  <dcousens> well, crap. Haha, I'll let you know how I go, but, considered I got 360k blocks in... might be easier to just trash the ?garbage? blocks and see how it goes after
 482015-11-16T00:43:02  <sipa> what is the problem?
 492015-11-16T00:43:24  <dcousens> "(having troubles with a parser, it makes it ~365k blocks in before blowing up with the magic header being wrong)"
 502015-11-16T00:44:12  <sipa> well scan for the first valid header :)
 512015-11-16T00:44:13  <dcousens> but the parser makes the assumption the data is contigous with no gaps (aka, no garbage)
 522015-11-16T00:44:40  <dcousens> sipa: might do, just wasn't aware that was necessary
 532015-11-16T00:45:02  <sipa> i think it's very unlikely to happen in 0.11 or later
 542015-11-16T00:45:09  <sipa> would require block index corruption
 552015-11-16T00:45:17  <dcousens> is a fresh build and IBD from master
 562015-11-16T00:45:25  <sipa> weird
 572015-11-16T00:45:30  <sipa> maybe i'm wrong
 582015-11-16T00:45:41  <sipa> maybe your block.data is just actually corrupted? :)
 592015-11-16T00:45:45  <dcousens> maybe I am, like I said, trying a re-index now
 602015-11-16T00:45:53  <dcousens> see how bitcoind goes, if it gets past it
 612015-11-16T00:45:55  <dcousens> well
 622015-11-16T00:46:21  <dcousens> i know exactly where its blowing up, so I can hex dump and have a looksy :)
 632015-11-16T00:46:35  <dcousens> thankfully the parser only takes 23s to run
 642015-11-16T00:47:45  <sipa> that's impressive
 652015-11-16T00:47:52  <sipa> wait
 662015-11-16T00:48:03  <dcousens> no script validation, obviously
 672015-11-16T00:48:05  <sipa> you parse 2 Gbyte per second?
 682015-11-16T00:48:47  <dcousens> that sounds a bit majestic doesn't it
 692015-11-16T00:49:00  <dcousens> and not right...
 702015-11-16T00:49:26  <dcousens> but, its a full parse.  I've got a 100MB buffer and the disk IO is going hard
 712015-11-16T00:49:30  <dcousens> its on AWS
 722015-11-16T00:50:08  <dcousens> so reading 100MB at a time, lazy loads the block in terms of what I want out of it (in this case, for a script index)
 732015-11-16T00:50:09  <sipa> well how else do you parse 50 Gbyte of block data in 23s?
 742015-11-16T00:50:29  <sipa> oh, you don't actually read the block data, just the headers?
 752015-11-16T00:50:42  <dcousens> sipa: technically I do though, like, I'm not doing an ftell...
 762015-11-16T00:51:01  <sipa> not following
 772015-11-16T00:51:02  <dcousens> in terms of IO...
 782015-11-16T00:51:30  <dcousens> sipa: 2GB seems a bit fast for what IO should be, AFAIK the AWS SSD's are only about 600mb/s
 792015-11-16T00:51:59  <dcousens> remembering, this is 365k blocks
 802015-11-16T00:52:05  <sipa> ah, a bit less
 812015-11-16T00:52:18  <dcousens> assuming 0.5mb for the remaining 20k?
 822015-11-16T00:52:22  <dcousens> thats like, 10GB less
 832015-11-16T00:52:42  <dcousens> still... 1.7GB/s ?
 842015-11-16T01:13:16  <GitHub18> [bitcoin] CodeShark opened pull request #7023: Fixed integer comparison warning. (master...signed_int_comparison_fix) https://github.com/bitcoin/bitcoin/pull/7023
 852015-11-16T01:28:42  *** challisto has joined #bitcoin-core-dev
 862015-11-16T01:34:17  *** lightningbot has joined #bitcoin-core-dev
 872015-11-16T01:34:27  *** luke-jr_ has joined #bitcoin-core-dev
 882015-11-16T01:34:45  *** Luke-Jr has quit IRC
 892015-11-16T01:37:15  *** cfields has joined #bitcoin-core-dev
 902015-11-16T01:38:02  *** jonasschnelli has joined #bitcoin-core-dev
 912015-11-16T01:44:26  *** Ylbam has quit IRC
 922015-11-16T01:54:33  *** luke-jr_ is now known as Luke-Jr
 932015-11-16T02:19:06  *** d_t has joined #bitcoin-core-dev
 942015-11-16T02:25:00  *** CodeShark_ has quit IRC
 952015-11-16T02:36:01  <phantomcircuit> jgarzik, did you ever benchmark sqlite against leveldb with the WAL settings?
 962015-11-16T02:36:37  <jgarzik> phantomcircuit, yes and no - was in the 'obviously too slow' category
 972015-11-16T02:36:43  <jgarzik> sqlite4 storage engine looks interesting
 982015-11-16T02:37:30  <phantomcircuit> jgarzik, that's what i guessed
 992015-11-16T02:37:56  <phantomcircuit> the python data node i wrote in ~2010 had a super optimized sqlite3 db and even then it was comically slow
1002015-11-16T02:38:10  <phantomcircuit> even with a proper db like postgres it was pretty horrendously slow
1012015-11-16T03:20:32  <dcousens> jgarzik: even with prepared statements etc?
1022015-11-16T03:20:41  <jgarzik> dcousens, yes
1032015-11-16T03:20:41  *** droark has joined #bitcoin-core-dev
1042015-11-16T03:29:27  *** d_t has quit IRC
1052015-11-16T03:39:05  *** zooko has joined #bitcoin-core-dev
1062015-11-16T03:40:58  *** d_t has joined #bitcoin-core-dev
1072015-11-16T03:54:53  *** arowser has joined #bitcoin-core-dev
1082015-11-16T04:01:35  *** go1111111 has quit IRC
1092015-11-16T04:03:20  <gmaxwell> it's not really that sqllite is "slow", it's pretty fast for what it does; it's just that dedicated transactional key value stores are really quite fast.
1102015-11-16T04:05:15  <gmaxwell> for me bitcoin core sync runs at around 8000tx/s (with secp256k1), even though every tx is involving a non-cachable missing record check for bip30 (in addition to all the cached stuff)... thats really quite remarkable IMO.
1112015-11-16T04:15:22  *** go1111111 has joined #bitcoin-core-dev
1122015-11-16T04:20:15  *** zooko has quit IRC
1132015-11-16T04:22:30  *** challisto has quit IRC
1142015-11-16T04:44:57  *** Anduck has quit IRC
1152015-11-16T04:45:10  *** Anduck has joined #bitcoin-core-dev
1162015-11-16T04:49:40  *** d_t has quit IRC
1172015-11-16T05:18:04  *** zarathustra has quit IRC
1182015-11-16T05:39:20  *** Anduck has quit IRC
1192015-11-16T05:46:57  *** Anduck has joined #bitcoin-core-dev
1202015-11-16T05:48:05  *** d_t has joined #bitcoin-core-dev
1212015-11-16T06:41:30  *** Thireus1 has quit IRC
1222015-11-16T06:41:50  *** Thireus has joined #bitcoin-core-dev
1232015-11-16T06:45:33  *** Ylbam has joined #bitcoin-core-dev
1242015-11-16T07:01:18  *** guest234234 has joined #bitcoin-core-dev
1252015-11-16T07:03:12  *** Guest23423 has joined #bitcoin-core-dev
1262015-11-16T07:06:11  *** guest234234 has quit IRC
1272015-11-16T07:11:21  *** gavinand1esen has joined #bitcoin-core-dev
1282015-11-16T07:11:24  *** btcdrak_ has joined #bitcoin-core-dev
1292015-11-16T07:12:15  *** Anduck has quit IRC
1302015-11-16T07:13:41  *** Anduck has joined #bitcoin-core-dev
1312015-11-16T07:14:14  *** CodeShark_ has joined #bitcoin-core-dev
1322015-11-16T07:15:00  *** morcos_ has joined #bitcoin-core-dev
1332015-11-16T07:15:42  *** teward- has joined #bitcoin-core-dev
1342015-11-16T07:15:48  *** btcdrak has quit IRC
1352015-11-16T07:15:49  *** teward has quit IRC
1362015-11-16T07:15:49  *** CodeShark has quit IRC
1372015-11-16T07:15:49  *** morcos has quit IRC
1382015-11-16T07:15:50  *** gavinandresen has quit IRC
1392015-11-16T07:15:50  *** teward- is now known as teward
1402015-11-16T07:15:55  *** CodeShark_ is now known as CodeShark
1412015-11-16T07:16:04  *** Thireus has quit IRC
1422015-11-16T07:16:21  *** btcdrak_ is now known as btcdrak
1432015-11-16T07:16:22  *** Thireus has joined #bitcoin-core-dev
1442015-11-16T07:28:37  *** Thireus has quit IRC
1452015-11-16T07:30:06  *** Thireus has joined #bitcoin-core-dev
1462015-11-16T07:32:50  *** s1w has quit IRC
1472015-11-16T07:38:40  *** s1w has joined #bitcoin-core-dev
1482015-11-16T07:38:56  *** jonasschnelli has quit IRC
1492015-11-16T07:38:57  *** jonasschnelli has joined #bitcoin-core-dev
1502015-11-16T07:39:04  *** s1w is now known as Guest16267
1512015-11-16T07:40:05  <GitHub184> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/b632145edeb3...814697c5569c
1522015-11-16T07:40:06  <GitHub184> bitcoin/master 773ae46 Jonas Schnelli: [Qt] add shortcurts for debug-/console-window
1532015-11-16T07:40:06  <GitHub184> bitcoin/master 814697c Jonas Schnelli: Merge pull request #7000...
1542015-11-16T07:40:10  <GitHub19> [bitcoin] jonasschnelli closed pull request #7000: [Qt] add shortcurts for debug-/console-window (master...2015/11/qt_shortcuts) https://github.com/bitcoin/bitcoin/pull/7000
1552015-11-16T07:40:44  *** Thireus has quit IRC
1562015-11-16T07:41:04  *** Thireus has joined #bitcoin-core-dev
1572015-11-16T07:42:03  * btcdrak claps. jonas' first merge :)
1582015-11-16T07:43:14  <jonasschnelli> hah. Thanks btcdrak
1592015-11-16T07:47:03  *** Thireus has quit IRC
1602015-11-16T07:47:25  *** Thireus has joined #bitcoin-core-dev
1612015-11-16T07:49:57  *** Thireus has quit IRC
1622015-11-16T07:50:40  *** Thireus has joined #bitcoin-core-dev
1632015-11-16T08:09:27  <gmaxwell> BlueMatt: did he mess up the signing? :)
1642015-11-16T08:17:56  <phantomcircuit> jonasschnelli, *cough*
1652015-11-16T08:18:14  <phantomcircuit> oh nvm ignore me
1662015-11-16T08:18:37  <jonasschnelli> gmaxwell: hmm... i testes the verifiy-commit stuff before. Should work if one has a recent master where he verifies commit from.
1672015-11-16T08:19:12  <jonasschnelli> master needs to have https://github.com/bitcoin/bitcoin/pull/7004
1682015-11-16T08:19:25  <jonasschnelli> then my first merge should be valid. :) (hopefully)
1692015-11-16T08:19:41  <phantomcircuit> jonasschnelli, it's properly signed with 32EE 5C4C 3FA1 5CCA DB46  ABE5 29D4 BCB6 416F 53EC
1702015-11-16T08:20:27  <jonasschnelli> ID 416F53EC, 32EE 5C4C 3FA1 5CCA DB46  ABE5 29D4 BCB6 416F 53EC
1712015-11-16T08:20:52  <jonasschnelli> and this matched that: https://github.com/bitcoin/bitcoin/pull/7004/files
1722015-11-16T08:21:31  * jonasschnelli fingers where sweat during his first merge and double/tripple checked everything...
1732015-11-16T08:22:56  <gmaxwell> heh.
1742015-11-16T08:25:53  <wumpus> hah
1752015-11-16T08:28:16  <wumpus> jonasschnelli: looks good to me (w/ --show-signature)
1762015-11-16T08:29:08  <wumpus> (git log --merges --show-signature)
1772015-11-16T08:29:29  <gmaxwell> jonasschnelli: Congrats!
1782015-11-16T08:31:09  <wumpus> jonasschnelli: my plan was to walk you through at least one merge, but really needed some time out this weekend - seems you figured it out!
1792015-11-16T08:33:17  <jonasschnelli> I'm using the `github-merge.sh` script (1:1) in two other projects (https://github.com/libbtc/libbtc and https://github.com/digitalbitbox/mcu), so I'm a little bit familiar with it (also with signed commits)
1802015-11-16T08:36:34  <wumpus> great :D
1812015-11-16T08:38:50  <GitHub194> [bitcoin] gmaxwell pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/814697c5569c...6876a78b863e
1822015-11-16T08:38:50  <GitHub194> bitcoin/master 9bd3f03 Peter Todd: Clarify 'fee' field in fundrawtransaction help text...
1832015-11-16T08:38:51  <GitHub194> bitcoin/master 6876a78 Gregory Maxwell: Merge pull request #6991...
1842015-11-16T08:38:57  <GitHub101> [bitcoin] gmaxwell closed pull request #6991: Minor: Clarify 'fee' field in fundrawtransaction help text (master...clarify-fundrawtransaction-help) https://github.com/bitcoin/bitcoin/pull/6991
1852015-11-16T08:40:29  *** jtimon has joined #bitcoin-core-dev
1862015-11-16T08:50:49  <GitHub34> [bitcoin] laanwj closed pull request #7021: add bip65 tests to rpc-tests.sh -extended (in 0.11 branch) (0.11...11rpcfixups) https://github.com/bitcoin/bitcoin/pull/7021
1872015-11-16T08:50:53  <GitHub193> [bitcoin] laanwj pushed 2 new commits to 0.11: https://github.com/bitcoin/bitcoin/compare/7e278929df53...595c8d6301cf
1882015-11-16T08:50:54  <GitHub193> bitcoin/0.11 9730051 Alex Morcos: add bip65 tests to rpc-tests.sh -extended
1892015-11-16T08:50:54  <GitHub193> bitcoin/0.11 595c8d6 Wladimir J. van der Laan: Merge pull request #7021...
1902015-11-16T08:52:50  *** droark has quit IRC
1912015-11-16T08:55:06  *** d_t has quit IRC
1922015-11-16T09:13:03  *** Guyver2 has joined #bitcoin-core-dev
1932015-11-16T09:13:30  <gmaxwell> bleh, so it looks like my gbt latency measurements going up coincide not with mempool flooding attacks but with updating to master. Now I need to figure out what hurt the performance so much.
1942015-11-16T09:19:56  *** Thireus has quit IRC
1952015-11-16T09:21:00  *** Thireus has joined #bitcoin-core-dev
1962015-11-16T09:24:39  *** dcousens has quit IRC
1972015-11-16T09:30:06  *** rubensayshi has joined #bitcoin-core-dev
1982015-11-16T09:34:02  <wumpus> bleh indeed
1992015-11-16T09:34:21  <gmaxwell> https://people.xiph.org/~greg/p2pool-gbt.png < did you see my graph? :(
2002015-11-16T09:34:30  <wumpus> on the positive side, updating to master is easy to reproduce
2012015-11-16T09:34:45  <gmaxwell> well I need to figure out what I was running before! :)
2022015-11-16T09:34:51  <wumpus> git reflog?
2032015-11-16T09:35:14  <gmaxwell> yea, I'll try (tomorrow, late here)
2042015-11-16T09:35:21  <wumpus> no problem, there's always tons of metadata :p
2052015-11-16T09:37:11  <wumpus> so it's going to be either a) the switch to libevent's http server, there may be some foul interaction with longpoll b) one of the mempool changes
2062015-11-16T09:38:07  <wumpus> there's also c) something other, contending the cs_main lock  .. may even be related to the pruning-related changes. Ok, conclusion: no clue.
2072015-11-16T09:38:45  <gmaxwell> your thinking is already much more constructive than mine, I was stuck at "oh no" :)
2082015-11-16T09:40:00  <gmaxwell> the GBT latency doesn't LP. it's p2pool timing how long it takes it to make a gbt call on its own. The delay is constant, so I doubt contention unless its really hot.
2092015-11-16T09:40:12  <gmaxwell> if it's libevent it might just be some silly tunable.
2102015-11-16T09:40:32  <wumpus> well it may even be just some silly mistake
2112015-11-16T09:40:52  <wumpus> e.g. creating an event but not triggering it
2122015-11-16T09:47:18  <gmaxwell> hm. I don't see the time when I moved back to test 0.11.2rc on this graph. Interesting.
2132015-11-16T09:48:32  *** Thireus has quit IRC
2142015-11-16T09:48:53  *** Thireus has joined #bitcoin-core-dev
2152015-11-16T09:49:50  *** Thireus has joined #bitcoin-core-dev
2162015-11-16T09:50:45  <wumpus> btw using just seccomp-bpf it's not possible to limit the scope of opening files, seccomp scripts cannot dereference syscall argument pointers, it's limited to checks based on the numbers themselves  - e.g. disallow PROT_EXEC on mprotect
2172015-11-16T09:50:59  *** paveljanik has joined #bitcoin-core-dev
2182015-11-16T09:50:59  *** paveljanik has joined #bitcoin-core-dev
2192015-11-16T09:51:06  <wumpus> so if you allow open, it can open any file on the file system
2202015-11-16T09:51:28  <gmaxwell> oh crud, thats .. lame!
2212015-11-16T09:51:43  <gmaxwell> well of course selinux and such can control what files you can open.
2222015-11-16T09:52:09  <wumpus> it's possible to get around it with a dedicated process that traps some syscalls and then does the syscall for the restricted thread, but that's a lot more complexity
2232015-11-16T09:52:53  <wumpus> all those require privileged operations or changes to the OS
2242015-11-16T09:53:24  <wumpus> seccomp is IIRC the only one that can run unprivileged, chrome has a set-uid helper to do some jailing operations, but I don't like that
2252015-11-16T09:54:12  <wumpus> something may be possible with namespaces too: https://code.google.com/p/chromium/issues/detail?id=312380
2262015-11-16T09:54:15  <gmaxwell> Right, selinux requires privledge setup to establish the rules on the system (though after that the tags/contexts can be applied from userspace); I don't know how the apparmor stuff works.
2272015-11-16T09:54:30  <wumpus> apparmor works the same, it is a global config file per application
2282015-11-16T09:55:19  <gmaxwell> Hm I would have expected the namespaces thing to have some of the problems chroot has that makes you need privs to use it.
2292015-11-16T09:55:57  <gmaxwell> oh well even if we can't limit file scope at least that easily done with selinux/apparmor, and we could limit other things.
2302015-11-16T09:56:03  <wumpus> the advantage is that it can be set up externally, on the other hand that relies on specific files for the OS
2312015-11-16T10:00:55  <wumpus> nice about seccomp is that it can be set on a per-thread level. Although that doesn't help as much as long as signing and key storage happens within the main process :-)
2322015-11-16T10:01:40  <gmaxwell> oh I don't think I realized seccomp could be thread scoped.
2332015-11-16T10:02:17  <wumpus> the idea is to set up a global seccomp filter as soon as possible before any threads are created
2342015-11-16T10:02:22  <gmaxwell> though use of callbacks and event signaling would then make you have to be careful about what thread things actually excuted in.
2352015-11-16T10:02:27  <wumpus> but in a thread it can be restricted further
2362015-11-16T10:02:54  <wumpus> the security mesaure that amounst to is block all the exits but what people want to steal isn't outside, it's in the same memory space
2372015-11-16T10:02:59  <gmaxwell> e.g. I'd say the network thread could lose all file IO. ... except I'm not sure that it can.
2382015-11-16T10:03:33  <gmaxwell> sure if you lose control of control flow you've lost, but if the attacker can just call some libc functions, taking away file IO and such can be pretty limiting.
2392015-11-16T10:03:35  *** Anduck has quit IRC
2402015-11-16T10:04:10  *** Anduck has joined #bitcoin-core-dev
2412015-11-16T10:04:18  <wumpus> so I think we need seccomp lockdown of the main process (at least the one that does P2P networking) + key storage and signing in a different process and memory space
2422015-11-16T10:04:54  <wumpus> P2P networking would never have to see the passphrase or decryption key, for example
2432015-11-16T10:05:11  *** CodeShark_ has joined #bitcoin-core-dev
2442015-11-16T10:05:37  *** CodeShark_ has quit IRC
2452015-11-16T10:06:06  <wumpus> "sure if you lose control of control flow you've lost" the whole idea is to mitigate that, that if you have control flow, you can only keep up doing what the thread was doing anyway because nothing else is allowed :-)
2462015-11-16T10:06:09  <GitHub45> [bitcoin] jonasschnelli opened pull request #7025: [Qt] refactor and optimize proxy settings behavior (master...2015/11/qt_settingsvalidation) https://github.com/bitcoin/bitcoin/pull/7025
2472015-11-16T10:06:38  <gmaxwell> yea, I want to do signing in another process in any case for a multitude of reasons. Including being able to keep passphrases out of ram... being able to use HSMs, etc.
2482015-11-16T10:06:49  <wumpus> right
2492015-11-16T10:07:24  <wumpus> and my point is that I think that's a more useful step, given concrete thread models, than seccomp right now
2502015-11-16T10:07:48  <gmaxwell> annoyingly though, if compromising the threads that are doing network deseralization can be escaped to take over the user account, the likely they can compromise the signing process too.
2512015-11-16T10:08:00  <gmaxwell> Ah. Fair enough.
2522015-11-16T10:08:07  <wumpus> e.g. the UPnP vulnerability would have allowed getting control of a thread that has network access, no amount of lockdown couldh ave avoided it from leaking secrets from the process memory
2532015-11-16T10:08:31  <wumpus> *apart* from having the secrets in a seperate process in the first place
2542015-11-16T10:08:31  <gmaxwell> UPNP could also be easily moved into an external program, FWIW.
2552015-11-16T10:08:53  <wumpus> that's not my point - UPnP is just an example. The worst case scenario would be an attack through P2P
2562015-11-16T10:08:53  <gmaxwell> but thats much less interesting than seperating the secrets from the attack surface, as upnp is just one vector.
2572015-11-16T10:09:33  <wumpus> right, better to protect what has to be protected instead
2582015-11-16T10:09:44  <gmaxwell> I do think if we decide we'd like to turn upnp back on, it should go in another process and _that_ one we could seccomp to have no access to open(). :)
2592015-11-16T10:10:01  <gmaxwell> (even if we are otherwise protecting the signing)
2602015-11-16T10:11:46  <wumpus> sure
2612015-11-16T10:12:20  <wumpus> "given concrete thread models" eh THREAT models
2622015-11-16T10:22:07  *** Thireus has quit IRC
2632015-11-16T10:22:25  *** Thireus has joined #bitcoin-core-dev
2642015-11-16T10:22:55  *** Thireus has joined #bitcoin-core-dev
2652015-11-16T10:24:03  *** Thireus has quit IRC
2662015-11-16T10:24:25  *** Thireus has joined #bitcoin-core-dev
2672015-11-16T10:28:23  *** Guest23423 has quit IRC
2682015-11-16T10:37:06  *** guest234234 has joined #bitcoin-core-dev
2692015-11-16T10:43:35  *** randy-waterhouse has quit IRC
2702015-11-16T10:48:39  <wumpus> it's possible to use unprivileged user namespaces, but only on linux kernel 3.18+
2712015-11-16T10:49:05  <btcdrak> the message "UpdateTip: 1 of last 100 blocks above version 4" is not correct. It's above 3.
2722015-11-16T10:51:31  <wumpus> BTW: if you get "/home/user/bitcoin/src/key.cpp:204: undefined reference to `secp256k1_ecdsa_sign_recoverable'"  errors after updating to master you need to clean your git tree  (since #6983)
2732015-11-16T11:02:19  *** dcousens has joined #bitcoin-core-dev
2742015-11-16T11:03:32  <GitHub134> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/6876a78b863e...dafefb79244d
2752015-11-16T11:03:32  <GitHub134> bitcoin/master aee22bf Gregory Maxwell: Avoid a compile error on hosts with libevent too old for EVENT_LOG_WARN....
2762015-11-16T11:03:33  <GitHub134> bitcoin/master dafefb7 Wladimir J. van der Laan: Merge pull request #7016...
2772015-11-16T11:03:37  <GitHub72> [bitcoin] laanwj closed pull request #7016: Avoid a compile error on hosts with libevent too old for EVENT_LOG_WARN. (master...without_EVENT_LOG_WARN) https://github.com/bitcoin/bitcoin/pull/7016
2782015-11-16T11:03:47  <midnightmagic> yay!
2792015-11-16T11:23:03  <wumpus> user namespaces create a 'fantasy world' for a process in which it is root, but root is mapped (by the namespace) to the user own uid - it does have capabilities such as chroot *within* this user namespace, but it doesn't let it use its elevated privileges outside that. This gives me a headache.
2802015-11-16T11:23:35  <wumpus> "Over the years, there have been a lot of features that have been added to the Linux kernel that have been made available only to privileged users because of their potential to confuse set-user-ID-root applications.  In general, it becomes safe to allow the root user in a user namespace to use those features because it is impossible, while in a  user  namespace, to gain more privilege than the root user of a user namespace has"
2812015-11-16T11:26:35  *** Anduck has quit IRC
2822015-11-16T11:28:13  *** Anduck has joined #bitcoin-core-dev
2832015-11-16T11:44:35  *** Anduck has quit IRC
2842015-11-16T11:45:43  <wumpus> not really promising regarding stable API that the example program in "man user_namespaces" is already broken in kernel 4.2.0
2852015-11-16T11:48:43  <wumpus> ah there was a security problem, and the kernel as well as man page was updated, but didn't have that version: http://man7.org/linux/man-pages/man7/user_namespaces.7.html#EXAMPLE
2862015-11-16T11:50:13  *** Anduck has joined #bitcoin-core-dev
2872015-11-16T12:02:15  *** Anduck has quit IRC
2882015-11-16T12:02:33  *** Anduck has joined #bitcoin-core-dev
2892015-11-16T12:03:32  *** paveljanik has quit IRC
2902015-11-16T12:04:23  <GitHub43> [bitcoin] MarcoFalke opened pull request #7026: [contrib] Update versionprefix to "bitcoin-core" in verify.sh (master...MarcoFalke-2015-verify) https://github.com/bitcoin/bitcoin/pull/7026
2912015-11-16T12:25:17  <GitHub159> [bitcoin] MarcoFalke opened pull request #7027: [qa] rpc-tests: Properly use test framework (master...MarcoFalke-2015-rpcTestCleanup) https://github.com/bitcoin/bitcoin/pull/7027
2922015-11-16T12:25:27  <GitHub113> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/dafefb79244d...e54ebbf60097
2932015-11-16T12:25:28  <GitHub113> bitcoin/master 6e18268 Pieter Wuille: Switch to libsecp256k1-based validation for ECDSA
2942015-11-16T12:25:28  <GitHub113> bitcoin/master e54ebbf Wladimir J. van der Laan: Merge pull request #6954...
2952015-11-16T12:25:32  <GitHub119> [bitcoin] laanwj closed pull request #6954: Switch to libsecp256k1-based ECDSA validation (master...secp256k1) https://github.com/bitcoin/bitcoin/pull/6954
2962015-11-16T12:25:43  *** MarcoFalke has joined #bitcoin-core-dev
2972015-11-16T12:28:07  * jonasschnelli applaud to the merge of  #6954
2982015-11-16T12:28:12  *** dcousens has quit IRC
2992015-11-16T12:32:21  <jonasschnelli> wumpus: sorry for 6900, i could not create a trusty base image. Seems not to work on my debian7 machine. I tried to change the gitian vm builder... but looks like a VitrualBox mkfs issue.
3002015-11-16T12:32:39  <jonasschnelli> The only gitian host i have is debian...
3012015-11-16T12:32:47  * jonasschnelli needs more nativ linux machines...
3022015-11-16T12:43:50  *** mm_1 has quit IRC
3032015-11-16T12:44:27  *** mm_1 has joined #bitcoin-core-dev
3042015-11-16T12:48:21  *** ParadoxSpiral has joined #bitcoin-core-dev
3052015-11-16T12:58:30  <phantomcircuit> wumpus, neat
3062015-11-16T13:01:29  <sipa> yay :)
3072015-11-16T13:02:45  <phantomcircuit> sipa, how long have you been working on libsecp256k1?
3082015-11-16T13:03:18  <sipa> i started somewhere in 2013, i think
3092015-11-16T13:03:20  *** MarcoFalke has quit IRC
3102015-11-16T13:03:33  *** MarcoFalke has joined #bitcoin-core-dev
3112015-11-16T13:11:34  <wumpus> jonasschnelli: doesn't strictly need to be native - w/ lxc you can also use VMs, I think it's even possible to do nested kvm on recent CPUs
3122015-11-16T13:14:51  <btcdrak> sipa: wow, that is like a life's work
3132015-11-16T13:24:29  <jl2012> Sorry if this is a stupid question. Is there any contingency plan if libsecp256k1 turns out to be a softfork or hardfork?
3142015-11-16T13:30:55  <instagibbs> or both? ;)
3152015-11-16T13:35:12  <sipa> if there is any difference with openssl, it is a hard fork
3162015-11-16T13:35:37  <sipa> because you can negate the outcome of checksig
3172015-11-16T13:35:59  <sipa> so any strictening of rules can be turned into widening, in theory
3182015-11-16T13:37:26  <sipa> though at this point i think it's fair to say that due the amount of testing and formal validation that went into libsecp256k1, that is more likely to be a bug in openssl then
3192015-11-16T13:38:56  <sipa> also, it may not be exploitable... the BN_sqr bug we found in OpenSSL (thanks to libsecp256k1's unit tests that compared with OpenSSL) was technically a hard fork for bitcoin when it got foxed, but an unexploitable one as far as we know
3202015-11-16T13:39:06  <sipa> *fixed
3212015-11-16T13:39:59  <sipa> so i guess it would depend on what sort of consensus change it would be
3222015-11-16T13:42:14  *** d_t has joined #bitcoin-core-dev
3232015-11-16T13:42:51  *** d_t has joined #bitcoin-core-dev
3242015-11-16T13:43:01  <sipa> and if it's before 0.12 is released, we can always just revert it :)
3252015-11-16T13:45:48  *** MarcoFalke has quit IRC
3262015-11-16T13:53:07  *** zmanian_ has quit IRC
3272015-11-16T13:54:43  *** zmanian_ has joined #bitcoin-core-dev
3282015-11-16T13:54:49  <instagibbs> sipa, is there a summary of the review/testing process for it?
3292015-11-16T14:00:13  *** MarcoFalke has joined #bitcoin-core-dev
3302015-11-16T14:00:27  *** davec has quit IRC
3312015-11-16T14:02:57  *** d_t has quit IRC
3322015-11-16T14:06:26  *** Guest95455 has quit IRC
3332015-11-16T14:07:31  <sipa> instagibbs: not really, but i agree we need that
3342015-11-16T14:07:39  *** pigeons has joined #bitcoin-core-dev
3352015-11-16T14:08:02  *** pigeons is now known as Guest89319
3362015-11-16T14:08:51  <sipa> instagibbs: several types of tests are not yet merged (though they pass fine), including sage scripts to algebraically verify the group formulas, and an alterrnative build that results in a very small size curve so we can do high-level exhaustive testing
3372015-11-16T14:08:54  *** d_t has joined #bitcoin-core-dev
3382015-11-16T14:09:38  <sipa> those should go in before 0.12, as well as some document describing all the various methods
3392015-11-16T14:13:02  *** d_t has quit IRC
3402015-11-16T14:15:23  <jonasschnelli> wumpus: get a linker error when building #6900 (trusty gitian). Check: https://bitcoin.jonasschnelli.ch/pulls/6900/build-linux.log (very bottom)
3412015-11-16T14:41:58  <GitHub162> [bitcoin] MarcoFalke opened pull request #7028: [doc] qa: Move README.md and update -help (master...MarcoFalke-2015-qaReadme) https://github.com/bitcoin/bitcoin/pull/7028
3422015-11-16T14:54:42  *** grieve has quit IRC
3432015-11-16T14:58:10  <GitHub151> [bitcoin] sdaftuar opened pull request #7029: Remove unmaintained example test script_test.py (master...script-test-cleanup) https://github.com/bitcoin/bitcoin/pull/7029
3442015-11-16T14:58:12  <jgarzik> Great to see libsecp256k1 go in
3452015-11-16T14:58:51  <btcdrak> and openssl to be shown the door
3462015-11-16T14:59:22  *** guest234234 has quit IRC
3472015-11-16T15:08:15  <GitHub155> [bitcoin] MarcoFalke opened pull request #7030: [contrib] Delete test-patches and move testgen to devtools (master...MarcoFalke-2015-contribC) https://github.com/bitcoin/bitcoin/pull/7030
3482015-11-16T15:12:26  *** davec has joined #bitcoin-core-dev
3492015-11-16T15:19:16  *** zooko has joined #bitcoin-core-dev
3502015-11-16T15:23:03  *** fkhan has quit IRC
3512015-11-16T15:33:13  *** skyraider has joined #bitcoin-core-dev
3522015-11-16T15:33:35  *** Thireus has quit IRC
3532015-11-16T15:34:22  *** Thireus has joined #bitcoin-core-dev
3542015-11-16T15:36:26  *** fkhan has joined #bitcoin-core-dev
3552015-11-16T15:39:46  *** bsm1175321 has joined #bitcoin-core-dev
3562015-11-16T15:44:57  *** treehug88 has joined #bitcoin-core-dev
3572015-11-16T15:47:20  <wumpus> jonasschnelli: don't really understand
3582015-11-16T15:47:34  <jonasschnelli> wumpus: The 6900 issues?
3592015-11-16T15:47:38  <wumpus> yes
3602015-11-16T15:47:49  <jonasschnelli> seems something with boost
3612015-11-16T15:48:13  <wumpus> I thought the point of gitian was that the results are the same for everyone
3622015-11-16T15:48:38  <wumpus> you're also using kvm? or lxc?
3632015-11-16T15:48:51  <jonasschnelli> ha... right.. maybe i do it slightly different..
3642015-11-16T15:48:57  <jonasschnelli> wumpus: kvm,... not i try to build 0340ffb
3652015-11-16T15:49:10  <jonasschnelli> not/now
3662015-11-16T15:54:27  <wumpus> the non-working boost sleep implementation error usually means it cannot link against boost at all
3672015-11-16T15:54:54  <wumpus> unfortunately it's not really easy to get the config.log in the general case from the gitian builder
3682015-11-16T15:56:31  *** Thireus has quit IRC
3692015-11-16T15:56:52  *** Thireus has joined #bitcoin-core-dev
3702015-11-16T15:58:09  <wumpus> linux seems still to be building properly here - although it has the dependencies cached, could try removing the cache and starting over from the beginning but that's going to take a long time
3712015-11-16T16:13:48  <jonasschnelli> linux is still building here...
3722015-11-16T16:20:40  <jonasschnelli> wumpus : still have the linking issue on linux (https://bitcoin.jonasschnelli.ch/pulls/6900/build-linux.log)
3732015-11-16T16:21:01  <wumpus> can you try removing the cache? maybe something went wrong there
3742015-11-16T16:22:26  *** MarcoFalke has quit IRC
3752015-11-16T16:22:35  *** Thireus has quit IRC
3762015-11-16T16:22:50  *** treehug88 has quit IRC
3772015-11-16T16:22:51  *** Thireus has joined #bitcoin-core-dev
3782015-11-16T16:27:24  <jonasschnelli> wumpus: remove cache = rm -Rf gitian-builder/cache/common/*?
3792015-11-16T16:28:07  <wumpus> not common
3802015-11-16T16:28:18  <wumpus> I think only bitcoin-linux-0.12
3812015-11-16T16:28:43  * jonasschnelli ran rm -Rf gitian-builder/cache/bitcoin-linux-0.12/  ... does build again now
3822015-11-16T16:29:09  <wumpus> I think the problem in your case is that it was using the previous cache (built on precise) on trusty
3832015-11-16T16:31:56  *** zooko has quit IRC
3842015-11-16T16:32:14  *** Thireus has quit IRC
3852015-11-16T16:32:14  <jonasschnelli> right. That would make sense.
3862015-11-16T16:32:28  *** zooko has joined #bitcoin-core-dev
3872015-11-16T16:32:36  *** Thireus has joined #bitcoin-core-dev
3882015-11-16T16:34:43  <wumpus> going to try following gitian-building.md on a new debian8 vm
3892015-11-16T16:36:26  *** Thireus has quit IRC
3902015-11-16T16:36:46  *** Thireus has joined #bitcoin-core-dev
3912015-11-16T16:46:53  *** Thireus has quit IRC
3922015-11-16T16:47:11  *** Thireus has joined #bitcoin-core-dev
3932015-11-16T16:52:35  *** treehug88 has joined #bitcoin-core-dev
3942015-11-16T16:58:23  *** zooko has quit IRC
3952015-11-16T17:00:40  *** Thireus has quit IRC
3962015-11-16T17:01:01  *** Thireus has joined #bitcoin-core-dev
3972015-11-16T17:02:02  *** Thireus has joined #bitcoin-core-dev
3982015-11-16T17:05:00  *** Thireus has quit IRC
3992015-11-16T17:05:21  *** Thireus has joined #bitcoin-core-dev
4002015-11-16T17:22:48  <jonasschnelli> wumpus: seems to work now: https://bitcoin.jonasschnelli.ch/pulls/6900/
4012015-11-16T17:43:23  *** d_t has joined #bitcoin-core-dev
4022015-11-16T17:43:29  <morcos_> sipa: Congrats and thank you and everybody who helped with libsecp256k1.  Great work!
4032015-11-16T17:44:52  <sipa> morcos_: thanks :)
4042015-11-16T17:46:13  *** morcos_ is now known as morcos
4052015-11-16T17:56:01  *** rubensayshi has quit IRC
4062015-11-16T18:03:15  *** Anduck has quit IRC
4072015-11-16T18:04:02  *** Anduck has joined #bitcoin-core-dev
4082015-11-16T18:28:01  *** Guest89319 is now known as pigeons
4092015-11-16T18:30:33  <morcos> jonasschnelli: when you have a chance, i'd like to get your thoughts on #6134.  i want to make sure i understood your comments.
4102015-11-16T18:55:59  *** PRab_ has joined #bitcoin-core-dev
4112015-11-16T18:56:59  *** PRab has quit IRC
4122015-11-16T18:56:59  *** PRab_ is now known as PRab
4132015-11-16T19:12:24  *** Naphex has quit IRC
4142015-11-16T19:13:39  *** PRab_ has joined #bitcoin-core-dev
4152015-11-16T19:14:50  *** PRab__ has joined #bitcoin-core-dev
4162015-11-16T19:16:59  *** PRab has quit IRC
4172015-11-16T19:18:40  *** PRab_ has quit IRC
4182015-11-16T19:18:46  *** PRab has joined #bitcoin-core-dev
4192015-11-16T19:19:39  *** PRab__ has quit IRC
4202015-11-16T19:31:01  <jonasschnelli> wumpus: https://bitcoin.jonasschnelli.ch/pulls/6900/build-win.log MinGW still has troubles with boost #6900
4212015-11-16T19:31:24  <jonasschnelli> ah. wait. I forgot to clean the cache for windows. Nm.
4222015-11-16T19:31:31  <jonasschnelli> morcos: sure.
4232015-11-16T19:31:36  <jonasschnelli> Let me open the PR first
4242015-11-16T19:34:23  <phantomcircuit> 2015-11-16 19:34:00 PartitionCheck : Found 24 blocks in the last 4 hours
4252015-11-16T19:34:23  <phantomcircuit> 2015-11-16 19:34:00 PartitionCheck : likelihood: 0.0811515
4262015-11-16T19:34:39  <phantomcircuit> that seems like a small window
4272015-11-16T19:37:26  <jonasschnelli> morcos: I have to admit, that i haven't fully reviewed #6134, my focus was the GUI part.
4282015-11-16T19:37:54  <morcos> jonasschnelli: i just wanted to make sure i understood your comment about never returning -1
4292015-11-16T19:38:12  <jonasschnelli> I think for an user it's strange that he might get the same fee for targeting conf. withing the next 25blocks and the next 18blocks.
4302015-11-16T19:38:35  <jonasschnelli> morcos: I was calling estimatefee instead of estimateaproxfee
4312015-11-16T19:38:37  <morcos> part of the point of the PR was to make it so you got -1 a lot less often.  but i didn't change what estimatefee returns, i just made a new function estimateapproxfee which is smarter
4322015-11-16T19:38:45  <morcos> perhaps i should call it estimatesmartfee instead
4332015-11-16T19:39:27  <morcos> that one will still return -1, but only at the very startup when there is not enough data period (and you don't have an old estimates file, although thats maybe dangerous for its own reasons)
4342015-11-16T19:39:30  <jonasschnelli> I think the name is okay. I just wasn't really concentrated while reviewing and mixed up the extimateaproxfee and estimatefee
4352015-11-16T19:39:41  <morcos> ok
4362015-11-16T19:39:52  <morcos> as for same fee for different confirmations
4372015-11-16T19:39:55  <jonasschnelli> UI: why can't you target for 1 block?
4382015-11-16T19:40:06  <jonasschnelli> (conf. withing next block)
4392015-11-16T19:40:14  <jonasschnelli> *within
4402015-11-16T19:40:30  <morcos> i'm not really sure there is anything to be done about that.  its just as accurate an answer as you can get from the data
4412015-11-16T19:40:42  <morcos> how do you mean, you can target for 1 block
4422015-11-16T19:40:58  <jonasschnelli> The slider i got was from block 25 to block 2.
4432015-11-16T19:41:13  <jonasschnelli> So, as enduser i would whish i could select 1 block.
4442015-11-16T19:41:16  <morcos> ok, so that is the UI change i wanted fee back from
4452015-11-16T19:41:28  <morcos> if you look at the right end of the slider
4462015-11-16T19:41:35  <morcos> position 1 and position 2, both say 2 blocks
4472015-11-16T19:42:08  <jonasschnelli> Right. I saw that. Is that because 1 block is just very hard (impossible to estimate)?
4482015-11-16T19:42:28  <morcos> so you are actually trying to get an estiamte for 1 block, but estimatefee returns -1.   so estimateapproxfee looks at 2 blocks, gives you an answer there, and lets you know that the answer is for 2, so you're not expecting to get confirmed in 1 with that fee
4492015-11-16T19:42:29  <jonasschnelli> * ... hard (impossible) to estimate?
4502015-11-16T19:42:53  <morcos> yes, thats correct, there is not enough data to indicate that any fee will have very high likelihood of being confirmed in 1 block
4512015-11-16T19:43:29  <morcos> this is primarily because i require 95% (used to be 85%) of txs with that fee or higher to be confirmed within the desired number of blocks, and often miners only recalculate their block every minute
4522015-11-16T19:43:49  <morcos> so that puts a ceiling of about 90% on the chance you will be confirmed in the next blcok
4532015-11-16T19:44:53  <morcos> my prior version of the change had the slider snap back to position 2, but that seemed like i did not implement that very well.  so the simpler change is this, but i'm happy for suggestions on how to improve it from a UI standpoint
4542015-11-16T19:45:29  <morcos> what i'm trying to communicate is we can't give you an answer for 1 block, the best i can do is give you an answer for 2 blocks and that answer is X
4552015-11-16T19:45:51  <jonasschnelli> hm... hmm.. somehow it feels that the user must be able to select 1 block... i know, it's not possible to estimate... :)
4562015-11-16T19:46:49  <jonasschnelli> we could also go the way other wallets do: "confirmation speed" : "slow" = ~8b | "avg" ~4b | "fast" ~2b.
4572015-11-16T19:47:19  <jonasschnelli> we just use the words "slow", "average", "fest"
4582015-11-16T19:47:26  <jonasschnelli> *"fast"
4592015-11-16T19:47:35  <morcos> i do think the ability to choose any # is overkill
4602015-11-16T19:47:44  <morcos> but maybe thats too big a change now for 0.12?
4612015-11-16T19:48:20  <jonasschnelli> I mean, it's nitpicking. The change loos good to me (need to test more in detail).
4622015-11-16T19:48:25  <jonasschnelli> *looks
4632015-11-16T19:48:39  <jonasschnelli> what do you think by aggregating block/fees.
4642015-11-16T19:48:51  <morcos> ok...  what do you think about changing estimateapproxfee to estimatesmartfee.  i think i like that name better
4652015-11-16T19:49:14  <morcos> better holds to how i think that function will evolve in the future
4662015-11-16T19:49:18  <jonasschnelli> yeah. Smart* sounds always good. I also like estimatesmartfee better.
4672015-11-16T19:49:22  <morcos> aggregating block/fees?
4682015-11-16T19:49:51  <jonasschnelli> the thing is, if you move the slider, you might get the same fee between block 25 and block 18 (was the case in my test),...
4692015-11-16T19:50:11  <jonasschnelli> we could just stop at block 18 (as min value, to very left).
4702015-11-16T19:50:23  <morcos> hmm.. the problem is its not really possible to know any 2 targets would give you the same number until you ask
4712015-11-16T19:51:08  <jonasschnelli> we could run from 2-25, then aggregate values, invalidate (this cache) after a new block comes in (or similar)
4722015-11-16T19:51:26  <jonasschnelli> but, yeah. sounds after some lines of code.
4732015-11-16T19:51:36  <jonasschnelli> could also be done later.
4742015-11-16T19:51:44  <jonasschnelli> I mean the PR should not be hold back because of the UI!
4752015-11-16T19:51:46  <morcos> yeah i think after we get 0.12.  we should talk about how to improve it
4762015-11-16T19:52:19  <morcos> maybe do something like only store results for 1,2,4,8,16  (and then possibly add longer time horizons: 32,64,128,256?)
4772015-11-16T19:52:28  <morcos> and then thats the only place the slider stops
4782015-11-16T19:52:39  <morcos> perhaps they will sometimes give the same value, but a lot less often
4792015-11-16T19:52:42  <jonasschnelli> yes. This could make sense.
4802015-11-16T19:53:23  <jonasschnelli> IMO we should keep the UI as it is for #6134 and focus on get it in.
4812015-11-16T19:53:38  <jonasschnelli> What is the reason to have two estimatefee rpc calls. Keep the API?
4822015-11-16T19:53:46  <morcos> ok agreed.  especially as i am out of town for 10 days starting on friday!
4832015-11-16T19:53:59  <morcos> yeah i think fee estimation still has some evolving to do
4842015-11-16T19:54:00  <jonasschnelli> Take your laptop with you. :)
4852015-11-16T19:54:17  <morcos> so i'd like to have estimatesmartfee be the new API, that will evolve with the functionality
4862015-11-16T19:54:38  <morcos> but for the meantime keep the old API consistent until we have a sense of where the smart version will end up
4872015-11-16T19:55:25  <jonasschnelli> will it only be estimation? Or would a RPC call "smartfee estiamte" make more sense?
4882015-11-16T19:55:48  <morcos> i didn't understand
4892015-11-16T19:55:49  <jonasschnelli> *estimate
4902015-11-16T19:56:43  <jonasschnelli> I'm just thinking loud. Maybe other smartfee interaction would be possible in future, setting some parameters or similar.
4912015-11-16T19:56:45  <morcos> oh, like you could configure state within the estimator, like smartfee timehorizon 1008
4922015-11-16T19:56:56  <jonasschnelli> Then we could collect everything in the smartfee rpc call.
4932015-11-16T19:57:33  <morcos> well i think maybe i should just call it smartfee and smartpriority ?
4942015-11-16T19:57:35  <jonasschnelli> something like "smartfee estimate" and "smartfee set timehorizon 100"
4952015-11-16T19:57:55  *** tripleslash has quit IRC
4962015-11-16T19:58:18  <morcos> i'm not sure we'll want to set state like that, i think it would be safer to only be able to configure parameters on startup
4972015-11-16T19:58:29  *** AtashiCon has quit IRC
4982015-11-16T19:58:33  *** Arnavion3 has joined #bitcoin-core-dev
4992015-11-16T19:58:37  *** Arnavion3 is now known as AtashiCon
5002015-11-16T19:58:38  *** gribble has quit IRC
5012015-11-16T19:58:44  <jonasschnelli> Hm.. no. better keep it "estimatesmartfee".
5022015-11-16T19:59:11  <morcos> ok...  estimatesmartfee it is.  i'll rebase at some point with just the name change...
5032015-11-16T19:59:15  <jonasschnelli> We could still have something like "estiamtesmartfee settimehorzon X"
5042015-11-16T19:59:35  <jonasschnelli> I'll try to do a full test after your rebase.
5052015-11-16T19:59:43  <morcos> ok
5062015-11-16T19:59:47  <jonasschnelli> Keep the UI stuff as it is. We can polish that later.
5072015-11-16T20:01:26  <morcos> yikes, so many commits, i'm just going to make one final commit which changes the names i think...
5082015-11-16T20:01:45  <jonasschnelli> Ok.
5092015-11-16T20:02:08  <jonasschnelli> morcos: one thing: I think we sould remove ""\nWARNING: This interface is unstable and may disappear or change!\n""
5102015-11-16T20:02:38  <morcos> oh why?  i wanted to have that in there so people didn't write code depending on estimatesmartfee
5112015-11-16T20:02:51  <morcos> i'm keeping the estimatefee interface the same
5122015-11-16T20:03:08  <morcos> and for the most part they can design their own smart algorithm by using estimatefee directly
5132015-11-16T20:03:32  <morcos> but i don't want to tie down the interface to the built-in smart algorithm now
5142015-11-16T20:03:42  <morcos> i almost didn't expose it as rpc calls, but its nice for testing
5152015-11-16T20:03:48  <jonasschnelli> Well, i think if we pack that into a release, we need to consider that people write code depending on that... new features in a 0. version are not meant to habe stable API IMO
5162015-11-16T20:04:08  <jonasschnelli> *have
5172015-11-16T20:04:21  <morcos> well thats the point of the warning, so they don't write code or they're aware.  i think that was wumpus' recommendation
5182015-11-16T20:04:25  <jonasschnelli> But no strong opinion on that.
5192015-11-16T20:04:39  <jonasschnelli> Okay. Then lets keep this.
5202015-11-16T20:04:43  <morcos> ok thx
5212015-11-16T20:05:10  *** gribble has joined #bitcoin-core-dev
5222015-11-16T20:06:05  <jonasschnelli> morcos: nice code/PR by the way. You guys always come directly with a rpc test and a bunch of unittests!
5232015-11-16T20:06:22  <morcos> ha, sdaftuar makes me!
5242015-11-16T20:06:37  <jonasschnelli> hah!
5252015-11-16T20:06:42  <jonasschnelli> Good boy.
5262015-11-16T20:07:08  *** tripleslash has joined #bitcoin-core-dev
5272015-11-16T20:42:49  *** belcher has joined #bitcoin-core-dev
5282015-11-16T20:43:08  *** belcher is now known as Guest49481
5292015-11-16T20:43:34  *** Guest49481 has quit IRC
5302015-11-16T20:45:15  *** belcher has joined #bitcoin-core-dev
5312015-11-16T21:00:25  *** d_t has quit IRC
5322015-11-16T21:04:55  *** Anduck has quit IRC
5332015-11-16T21:05:19  *** Anduck has joined #bitcoin-core-dev
5342015-11-16T21:17:23  *** d_t has joined #bitcoin-core-dev
5352015-11-16T21:32:18  *** randy-waterhouse has joined #bitcoin-core-dev
5362015-11-16T21:37:26  <gmaxwell> https://www.reddit.com/r/Bitcoin/comments/3t04fp/psa_please_upgrade_to_bitcoin_core_0112_to/cx2b117 < anyone know whats up with someone saying they can't upgrade bitcoin core without upgrading OSX?
5372015-11-16T21:45:24  *** Guyver2 has quit IRC
5382015-11-16T22:11:59  *** paveljanik has joined #bitcoin-core-dev
5392015-11-16T22:11:59  *** paveljanik has joined #bitcoin-core-dev
5402015-11-16T22:12:43  <treehug88> gmaxwell : he doesn't mention which OS version he's currently on. I had no trouble upgrading to 0.11.2 on 10.10.5 (Yosemite)
5412015-11-16T22:18:18  *** treehug88 has quit IRC
5422015-11-16T22:19:37  *** paveljanik has quit IRC
5432015-11-16T22:32:42  *** ParadoxSpiral has quit IRC
5442015-11-16T22:52:26  *** jgarzik has quit IRC
5452015-11-16T22:54:15  *** davec has quit IRC
5462015-11-16T22:54:16  *** bsm117532 has quit IRC
5472015-11-16T22:54:17  *** gmaxwell has quit IRC
5482015-11-16T22:54:44  *** bsm1175321 has quit IRC
5492015-11-16T22:54:45  *** berndj has quit IRC
5502015-11-16T22:54:45  *** isis has quit IRC
5512015-11-16T22:55:31  *** gmaxwell has joined #bitcoin-core-dev
5522015-11-16T22:55:55  *** gmaxwell is now known as Guest42878
5532015-11-16T22:55:58  *** davec has joined #bitcoin-core-dev
5542015-11-16T22:56:20  *** Guest42878 has quit IRC
5552015-11-16T22:56:20  *** Guest42878 has joined #bitcoin-core-dev
5562015-11-16T22:56:30  *** Guest42878 is now known as gmaxwell
5572015-11-16T22:59:28  *** berndj has joined #bitcoin-core-dev
5582015-11-16T23:00:46  *** gribble has quit IRC
5592015-11-16T23:03:25  *** isis has joined #bitcoin-core-dev
5602015-11-16T23:05:13  *** wump has joined #bitcoin-core-dev
5612015-11-16T23:06:50  *** wumpus has quit IRC
5622015-11-16T23:07:13  *** bsm117532 has joined #bitcoin-core-dev
5632015-11-16T23:09:03  *** zooko has joined #bitcoin-core-dev
5642015-11-16T23:09:32  *** bsm1175321 has joined #bitcoin-core-dev
5652015-11-16T23:09:38  *** guest234234 has joined #bitcoin-core-dev
5662015-11-16T23:10:08  *** gribble has joined #bitcoin-core-dev
5672015-11-16T23:13:59  *** guest234234 has quit IRC
5682015-11-16T23:27:34  *** zarathustra has joined #bitcoin-core-dev
5692015-11-16T23:27:34  *** zarathustra has joined #bitcoin-core-dev
5702015-11-16T23:48:35  *** dcousens has joined #bitcoin-core-dev
5712015-11-16T23:52:19  *** skyraider has quit IRC
5722015-11-16T23:53:14  <dcousens> sipa: so, :S
5732015-11-16T23:53:29  <dcousens> I have about 10mib garbage at the end of my leading .dat atm
5742015-11-16T23:53:43  <dcousens> and one of my .dat files in the middle has 23mb garbage at the end
5752015-11-16T23:54:04  <dcousens> other than that, all blocks accounted for
5762015-11-16T23:54:28  <dcousens> bitcoind's rescan went without a hitch