12015-11-12T00:00:07 *** evoskuil has joined #bitcoin-core-dev
22015-11-12T00:01:57 *** [1]evoskuil has joined #bitcoin-core-dev
32015-11-12T00:04:26 *** evoskuil has quit IRC
42015-11-12T00:04:26 *** [1]evoskuil is now known as evoskuil
52015-11-12T00:12:18 *** skyraider has quit IRC
62015-11-12T01:26:38 *** gribble has quit IRC
72015-11-12T01:32:23 *** zooko has joined #bitcoin-core-dev
82015-11-12T01:40:18 *** gribble has joined #bitcoin-core-dev
92015-11-12T01:42:51 *** zooko has quit IRC
102015-11-12T01:44:25 *** Ylbam has quit IRC
112015-11-12T01:54:50 *** randy-waterhouse has quit IRC
122015-11-12T02:00:06 *** evoskuil has quit IRC
132015-11-12T02:00:39 *** evoskuil has joined #bitcoin-core-dev
142015-11-12T02:00:46 *** trippysalmon has joined #bitcoin-core-dev
152015-11-12T02:25:41 *** trippysalmon has quit IRC
162015-11-12T02:26:47 <GitHub16> [bitcoin] pstratem opened pull request #6993: Add -blocksonly option (master...blocksonly) https://github.com/bitcoin/bitcoin/pull/6993
172015-11-12T02:30:46 *** evoskuil has quit IRC
182015-11-12T02:30:47 <gmaxwell> phantomcircuit: does that actually work?!
192015-11-12T02:30:59 <gmaxwell> add docs.
202015-11-12T02:31:51 *** evoskuil has joined #bitcoin-core-dev
212015-11-12T02:49:49 *** jgarzik has joined #bitcoin-core-dev
222015-11-12T02:55:36 <phantomcircuit> gmaxwell, yeah it does
232015-11-12T02:57:23 <phantomcircuit> i had completely forgotten about the fRelay flag
242015-11-12T03:08:49 <dcousens> phantomcircuit: its cool tbh haha
252015-11-12T03:21:11 *** jgarzik has quit IRC
262015-11-12T03:23:15 *** jgarzik has joined #bitcoin-core-dev
272015-11-12T03:33:37 *** dcousens has quit IRC
282015-11-12T03:50:08 *** fanquake has quit IRC
292015-11-12T03:50:24 *** randy-waterhouse has joined #bitcoin-core-dev
302015-11-12T04:19:02 *** CodeShark has joined #bitcoin-core-dev
312015-11-12T04:19:25 *** Arnavion has quit IRC
322015-11-12T04:19:41 *** [1]evoskuil has joined #bitcoin-core-dev
332015-11-12T04:19:43 *** Arnavion has joined #bitcoin-core-dev
342015-11-12T04:21:46 *** evoskuil has quit IRC
352015-11-12T04:21:46 *** [1]evoskuil is now known as evoskuil
362015-11-12T04:27:17 *** dcousens has joined #bitcoin-core-dev
372015-11-12T04:31:30 *** gribble has quit IRC
382015-11-12T04:44:01 *** gribble has joined #bitcoin-core-dev
392015-11-12T04:56:04 *** jtimon has quit IRC
402015-11-12T05:14:03 *** Thireus has quit IRC
412015-11-12T05:18:09 *** NLNico has joined #bitcoin-core-dev
422015-11-12T05:38:18 *** dcousens has left #bitcoin-core-dev
432015-11-12T05:48:13 *** dcousens has joined #bitcoin-core-dev
442015-11-12T05:49:07 *** d_t has joined #bitcoin-core-dev
452015-11-12T05:49:21 <dcousens> conceptually, phantomcircuit I feel like you could separate mempool from bitcoind with -blocksonly :P
462015-11-12T05:51:09 <dcousens> phantomcircuit: also, would it be worth just dropping the transactions before they even get to AcceptToMempool?
472015-11-12T05:51:32 <dcousens> or would advertising you don't care about txs be a BIP requiring protocol change?
482015-11-12T05:55:28 <dcousens> gmaxwell: getdata being dropping the transactions before ATM? :P
492015-11-12T05:56:16 <gmaxwell> jinx I guess. :)
502015-11-12T05:56:42 <dcousens> haha
512015-11-12T06:00:00 <gmaxwell> but yea, I think he should change it to not getdata any announcements instead from non-whitebound peers, and leave the mempool alone.
522015-11-12T06:00:28 <dcousens> gmaxwell: agreed, most likely the mempool would always be empty
532015-11-12T06:00:31 <dcousens> but that is intended
542015-11-12T06:00:43 <dcousens> (obviously)
552015-11-12T06:01:20 <gmaxwell> Right but then you could still use this node to e.g. relay transactions created by an offline wallet behind it and such.
562015-11-12T06:01:47 <dcousens> gmaxwell: or as I was thinking, have multiple relay nodes with decent bandwidth, but low memory, backed by a [lower] bandwidth blockchain node
572015-11-12T06:02:10 <dcousens> assuming you had a service big enough where that is warranted anyway
582015-11-12T06:02:45 <dcousens> hmmm, though, its the mempool that takes all the memory anyway haha
592015-11-12T06:03:15 <dcousens> well, it'll be interesting to play with for sure
602015-11-12T06:03:43 <phantomcircuit> dcousens, im dropping them in AcceptToMempool to prevent leaking which transactions are yours
612015-11-12T06:03:52 <phantomcircuit> im not sure i did a thorough job of that though
622015-11-12T06:04:40 <gmaxwell> phantomcircuit: nah, don't do that. Turning off wallet broadcast (already an option) is how thats achieved.
632015-11-12T06:05:01 <dcousens> gmaxwell: what about if it was submitted via RPC?
642015-11-12T06:05:30 <gmaxwell> Then we want to relay it.
652015-11-12T06:05:58 <dcousens> gmaxwell: sure, but, that seems counter to the point of preventing leakage
662015-11-12T06:06:02 <gmaxwell> blocksonly shouldn't refuse to relay transactions when you've specifically asked it to so. :)
672015-11-12T06:06:18 <gmaxwell> dcousens: if you don't want it to broadcast a transaction then don't ask it to!
682015-11-12T06:06:34 <dcousens> gmaxwell: sure, but, you could say the same about using it as a wallet
692015-11-12T06:06:39 <phantomcircuit> gmaxwell, well then it's a one line patch
702015-11-12T06:07:30 <gmaxwell> phantomcircuit: I think not, you need to avoid getdataing if they ignore your flags.
712015-11-12T06:07:49 <dcousens> gmaxwell: I agree with you, just pointing out the logic didn't work with what phantomcircuit logic said "preventing leaking [of] transactions which are yours"
722015-11-12T06:08:02 *** CodeShark_ has joined #bitcoin-core-dev
732015-11-12T06:08:15 <gmaxwell> dcousens: his current behavior will not relay a sendrawtransaction.
742015-11-12T06:08:27 *** CodeShark_ has quit IRC
752015-11-12T06:08:32 <gmaxwell> I thought you were asking what would happen in my proposed change.
762015-11-12T06:08:37 <dcousens> dw, I was
772015-11-12T06:08:49 *** CodeShark has quit IRC
782015-11-12T06:09:01 *** CodeShark_ has joined #bitcoin-core-dev
792015-11-12T06:09:05 <dcousens> unimportant, semantics are clear either way ;)
802015-11-12T06:09:11 *** CodeShark_ has quit IRC
812015-11-12T06:09:27 *** CodeShark has joined #bitcoin-core-dev
822015-11-12T06:10:20 <gmaxwell> phantomcircuit: so I think behavior should be when set, you will set the flag (to suppress inv announcements), and ignore getdatas. But only from non-white peers. And then anything sendraw/wallet will still work like normal, and if you don't want that for privacy reasons..then disable wallet broadcast and don't use sendraw. :)
832015-11-12T06:10:57 <gmaxwell> I imagine post 0.12 we'll also make this behavior one of the things we do in response to running into bandwidth limits.
842015-11-12T06:11:09 <gmaxwell> (e.g. if you get too close to your limit, go into blocks only mode.)
852015-11-12T06:11:40 *** Thireus has joined #bitcoin-core-dev
862015-11-12T06:12:12 <phantomcircuit> gmaxwell, that reminds me, thoughts on having a global settings object? (something that ya know.. actually acquired a lock)
872015-11-12T06:13:29 <gmaxwell> Big lock protecting bag-of-assorted things isn't generally great.
882015-11-12T06:14:24 <gmaxwell> in any case, not needed here, auto-blocks only I think is out of scope for 0.12. It's a pretty big behavior change, and we'd have to do something about the transaction privacy loss. (hopefully fix that via a seperate private announcement mechenism)
892015-11-12T06:15:22 <dcousens> gmaxwell: by auto, do you mean default?
902015-11-12T06:15:32 <dcousens> oh, you mean auto bandwidth response
912015-11-12T06:15:33 <dcousens> nvm
922015-11-12T06:17:35 <phantomcircuit> gmaxwell, could equally be a separate lock on each of the settings
932015-11-12T06:18:18 <gmaxwell> phantomcircuit: right for settings that should chang at runtime the setting should be read in init and set in a datastructure with its own lock for further changes.
942015-11-12T06:20:12 <phantomcircuit> gmaxwell, well and most of them can just be atomic ints
952015-11-12T06:21:41 <gmaxwell> We are not yet using C++ new enough to have standard atomic types.
962015-11-12T06:21:50 <gmaxwell> And plain integers ARE NOT ATOMIC.
972015-11-12T06:23:30 <jgarzik> without help
982015-11-12T06:24:06 <gmaxwell> sure, you can use locks. Or custom asm or what not.
992015-11-12T06:25:04 <jgarzik> gcc & clang provide atomic intrinsics to give C/C++ level access to cmpxchg, ints with lock prefixes and more.
1002015-11-12T06:25:16 <jgarzik> not C++ standard but it is available on all our platforms
1012015-11-12T06:25:51 <phantomcircuit> gmaxwell, something something boost atomic
1022015-11-12T06:26:33 <phantomcircuit> (im opposed to using boost in places where replacing it would be a giant headache, this is not one of them)
1032015-11-12T06:26:44 <gmaxwell> Well we'll upgrade to C++11 probably sooner rather than later and pick up language standard atomics.
1042015-11-12T06:27:40 <jgarzik> phantomcircuit, nod, http://www.boost.org/doc/libs/1_59_0/doc/html/atomic.html
1052015-11-12T06:27:50 <gmaxwell> /usually/ one shouldn't be directly using atomics in any case except in performance crticial algorithims or when building lower level constructs to be used from other things.
1062015-11-12T06:28:06 <gmaxwell> If the boost stuff is drop in for c++11 too than thats not so bad.
1072015-11-12T06:28:21 <jgarzik> that's the idea
1082015-11-12T06:29:10 <gmaxwell> I think we're already in a mode of only using boost features that aren't in C++11 if we have reason good enough to justify reimplementing them internally in any case. (e.g. mempool indexing)
1092015-11-12T06:32:34 <jgarzik> https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html
1102015-11-12T06:52:33 <phantomcircuit> gmaxwell, hmm so im thinking blocksonly should be undocumented as there are actually very few people who should be running such a thing
1112015-11-12T06:53:03 <phantomcircuit> the privacy loss form running with blocksonly and walletbroadcast=1 is potentially huge
1122015-11-12T06:54:01 *** Ylbam has joined #bitcoin-core-dev
1132015-11-12T06:55:21 <dcousens> phantomcircuit: that seems wrong
1142015-11-12T06:55:51 <dcousens> maybe a compile flag to allow it
1152015-11-12T06:55:52 <dcousens> or something
1162015-11-12T06:56:07 <wumpus> just disable walletbroadcast by default if it's enabled?
1172015-11-12T06:56:09 <dcousens> but not undocumented
1182015-11-12T06:56:16 <dcousens> or that ^
1192015-11-12T06:56:20 <wumpus> and emit a warning to the log
1202015-11-12T06:57:00 <wumpus> PRIVACYWARNING: wallet broadcasting has been disabled, because any transaction you send while not relaying transactions will stand out like a sore thumb
1212015-11-12T06:57:28 <phantomcircuit> wumpus, maybe... that's just what we need a more complicated tree of which settings are compatible :P
1222015-11-12T06:57:39 <wumpus> if you are sure you want to enable this, use -walletbroadcast
1232015-11-12T06:57:53 <gmaxwell> phantomcircuit: yea, you can imply walletbroadcast off for that case. We have handling like this for many options related to privacy.
1242015-11-12T06:57:55 <wumpus> you call that complicated?
1252015-11-12T06:57:55 <sipa> phantomcircuit: wallet broadcasting and whitelisting are already inherently a privacy leak
1262015-11-12T06:58:17 <gmaxwell> though I think you are GREATLY overestimating the improvement relay has on privacy.
1272015-11-12T06:58:20 <dcousens> wumpus: he just doesn't want to make it more than 1 or 2 lines
1282015-11-12T06:58:45 <gmaxwell> I think either way (defaulting walletbroadcast off or on) is okay with me, though I would leave it defaulting on.
1292015-11-12T06:59:06 <phantomcircuit> i uh dcousens *shhhh*
1302015-11-12T06:59:10 <wumpus> 'it is a already bad' is no excuse for allowing things to get silently even worse, hence the warning.
1312015-11-12T07:00:06 <wumpus> that's the problem with privacy everywhere, people think well we have no privacy already anyway, so why not do this and this also, it's not like it could get any worse. Yes, it can get worse.
1322015-11-12T07:00:45 <sipa> wumpus: agree, but to someone already listening on many nodes in the network, if they're connected to the original broadcaster too, they have a fairly good chance of finding you
1332015-11-12T07:00:48 <gmaxwell> wumpus: I don't know if it makes it worse or not, it might-- so fair enough.
1342015-11-12T07:01:17 <sipa> not relaying transactions otherwise just reduces the noticed; it doesn't fix the fact that information is being leaked
1352015-11-12T07:01:18 <wumpus> gmaxwell: well if you never send transactions then suddenlly send one, that makes it extremely clear it's yours, doesn't it?
1362015-11-12T07:01:23 <gmaxwell> sipa: research so far suggests that its currently about 100% effective if you transact more than once.
1372015-11-12T07:01:49 <gmaxwell> wumpus: well with whitebind, perhaps not! (also if we start turning this on dynamically in the future e.g. when low on bandwidth) perhaps not. But fair enough.
1382015-11-12T07:01:54 <sipa> s/noticed/noise/ i do not comprehend how i made that typo
1392015-11-12T07:02:04 <gmaxwell> phantomcircuit: so go softset off wallet broadcast and log, see the other examples in init.cpp.
1402015-11-12T07:02:23 <wumpus> I just don't like a setting that can silently make your privacy worse
1412015-11-12T07:02:37 <gmaxwell> wumpus: yup easily avoided, I agree.
1422015-11-12T07:02:40 <sipa> i'm fine with default off
1432015-11-12T07:02:41 <wumpus> the other option would be to indeed not document the option
1442015-11-12T07:03:15 <wumpus> but I think a warning + disable wallet broadcast by default is good enough
1452015-11-12T07:03:20 <sipa> what about whitebind?
1462015-11-12T07:03:26 <wumpus> what's with whitebind?
1472015-11-12T07:03:46 <wumpus> you mean - do relay transactions for whitelisters?
1482015-11-12T07:03:47 <sipa> whitelisted peers that relay transactions
1492015-11-12T07:04:01 <gmaxwell> I think it'll be a nice bandwidth reduction feature for watching wallets. It's a >2x reduction in bandwidth (probably more like 3x typically) and also saves you from a number of dos attacks. :)
1502015-11-12T07:04:36 <gmaxwell> I think it should relay for whitelisted peers. So you can use a node configured like this as a gateway to get your transactions to the outside world.
1512015-11-12T07:04:40 <wumpus> gmaxwell: it is a great feature, but the catch is that you have to make sure you're silent yourself, as there is no noise to hide in. That's acceptable, but needs to be transparent.
1522015-11-12T07:05:09 <sipa> how is whitelisting different from wallet broadcasting yourself?
1532015-11-12T07:06:13 <gmaxwell> because you choose and configure whitelists, it's like setting broadcast=1
1542015-11-12T07:06:16 <sipa> even if it doesn't disclose the ip of the actual sender anymore, they do know an ip address that the sender trusts
1552015-11-12T07:06:32 <wumpus> right it's something you have to explicitly enable
1562015-11-12T07:06:46 <gmaxwell> That could be a seperate setting. I've wanted that before, to control the forced relay for whitehosts.
1572015-11-12T07:06:47 <wumpus> ... which could come with a warning, of course
1582015-11-12T07:07:04 <sipa> wumpus: then how is it different from walletbroadcast?
1592015-11-12T07:07:23 <wumpus> sipa: my proposal is that walletbroadcast would be disabled by default if block-only
1602015-11-12T07:07:30 <wumpus> whitelist already *is* disabled by default
1612015-11-12T07:07:39 <sipa> hmm, ok
1622015-11-12T07:07:44 <wumpus> walletbroadcast is enabled by default
1632015-11-12T07:07:53 <sipa> ok, that is a fair point
1642015-11-12T07:08:15 <gmaxwell> Whitelist is a bot too overloaded. We might want to add a whiterelay that controls the forced relay for whitelisted hosts, which defaults on, but is soft-off with blocksonly
1652015-11-12T07:08:20 <wumpus> there are probably more ways that whitelisted peers can screw up your privacy :D
1662015-11-12T07:08:23 <gmaxwell> s/bot/bit/
1672015-11-12T07:08:27 <wumpus> they're sort of trusted
1682015-11-12T07:08:33 <gmaxwell> yea, we already bypass mempool policy
1692015-11-12T07:08:53 <sipa> wumpus: they are very trusted... but you yourself are also trusted
1702015-11-12T07:09:41 <sipa> if you do a wallet transaction, you expect it to be relayed; if a whitelisted peer relays a transaction, you also expect it to be relayed
1712015-11-12T07:10:02 <wumpus> the wallet may also rebroadcast old transactions
1722015-11-12T07:10:05 <wumpus> that's the scary part
1732015-11-12T07:10:18 <sipa> whitelisted peers will also rebroadcast old transactions
1742015-11-12T07:10:20 <wumpus> even transactions sent to you
1752015-11-12T07:10:29 <wumpus> ok -I feel this is not constructive anymore
1762015-11-12T07:10:48 <wumpus> don't introduce features that may silently reduce user's privacy even more, that's all I ask
1772015-11-12T07:11:04 <sipa> ok
1782015-11-12T07:11:26 <wumpus> if there is warning, or a knob that has to be explicitly enabled, I'm ok with it
1792015-11-12T07:12:26 <gmaxwell> yea, seems simple enough, softset off walletbroadcast when this is set.
1802015-11-12T07:12:47 <wumpus> right
1812015-11-12T07:12:58 <wumpus> and yes, whitelisted peers have to be careful, we should document that
1822015-11-12T07:13:19 <gmaxwell> yea, we should document that already because they are already deanonymizing.
1832015-11-12T07:14:07 <gmaxwell> (not even the primary reason I really don't like the whitepeer relay rules--)
1842015-11-12T07:14:07 <wumpus> and yeah we could have an option to not relay transactions from whitelisted peers...
1852015-11-12T07:14:15 *** [b__b] has quit IRC
1862015-11-12T07:14:35 *** [b__b] has joined #bitcoin-core-dev
1872015-11-12T07:14:36 *** Apocalyptic has quit IRC
1882015-11-12T07:14:39 <gmaxwell> (I mostly don't like them because I whitebinded the relaynetwork client and p2pool for obvious reasons, and then found they were blowing past my relay policy :-/ )
1892015-11-12T07:14:47 <sipa> gmaxwell: i think the rebroadcasting is fundamentally the problem, because nobody but the owner of the transaction will rebroadcast
1902015-11-12T07:15:11 <sipa> an alternative is peers syncing the mempool with each other...
1912015-11-12T07:15:42 <gmaxwell> sipa: It's fairly busted without that, in fact.
1922015-11-12T07:15:48 <sipa> so transaction owners don't need to rebroadcast randomly
1932015-11-12T07:15:53 <wumpus> policy 192.168.1.* { allow relay; bypass relaypolicy; } policy bind :1234 { ... } *ducks*
1942015-11-12T07:16:29 <sipa> it is... in theory you can end up with a cordon of nodes that already know your transaction between you and miners
1952015-11-12T07:16:53 <gmaxwell> Right now on any given publically accessible node >=15 peers are spynodes that connect to everyone. They also use addr behavior to infer things about the non-listening peers behind them; and research shows high success rates for even single transaction origins. If you make a couple transactions in the current network you should assume your origin WILL be idenfied.
1962015-11-12T07:17:14 <gmaxwell> To avoid this we need an actually private broadcast mechenism.
1972015-11-12T07:17:24 <wumpus> yes
1982015-11-12T07:17:37 *** Apocalyptic has joined #bitcoin-core-dev
1992015-11-12T07:18:14 <gmaxwell> and once we have that, it would work just as well for rebroadcast. :)
2002015-11-12T07:18:28 <wumpus> we need a private transaction mechanism badly, but we already knew that I think :)
2012015-11-12T07:18:45 <gmaxwell> and also would remove some of these discussions since then the p2p protocol would not have any privacy implications for full nodes (beyond running a node at all. :) )
2022015-11-12T07:19:16 <wumpus> right, it would, but it doesn't affect our decisions right now because it's entirely hypothetical :)
2032015-11-12T07:19:51 <gmaxwell> Just another benefit. It's really nice to not have to consider privacy implications for things; especially when the current state is bad enough that it can be hard to tell if you're making it worse. :)
2042015-11-12T07:20:04 <gmaxwell> E.g. one of the problems with peer rotation, ... in privacy.
2052015-11-12T07:20:07 <gmaxwell> er is privacy.
2062015-11-12T07:20:31 <sipa> we just need separate networks for block relay and tx relay :)
2072015-11-12T07:20:47 <gmaxwell> One of the problems with pruning ... is privacy (makes nodes more fingerprintable; so you can track nodes across tor and then correlate transactions they originate)
2082015-11-12T07:21:16 <sipa> well... i guess the same arguments apply to mining actually, it's just much less data to analyse, but ideally you also don't want to be able to pinpoint block creators
2092015-11-12T07:21:18 <gmaxwell> yadda yadda. All becomes more tidy if these things are seperated. In one sense you can see that privacy and partition resistance are actually in conflict.
2102015-11-12T07:21:43 <gmaxwell> For privacy you want to tell your transactions to very few parties, hopefully not attackers. For partition resistance you want to talk to everyone. :)
2112015-11-12T07:22:27 <gmaxwell> sipa: yea, though different problems; mining is very latency sensntive; tx broadcast is not.
2122015-11-12T07:22:50 <GitHub183> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/cbf9609c71c6...5fcc14ee0562
2132015-11-12T07:22:50 <GitHub183> bitcoin/master b5cbd39 James O'Beirne: Add basic coverage reporting for RPC tests...
2142015-11-12T07:22:50 <sipa> and partitioning for tx relay is less of a problem
2152015-11-12T07:22:51 <GitHub183> bitcoin/master 5fcc14e Wladimir J. van der Laan: Merge pull request #6804...
2162015-11-12T07:22:55 <GitHub198> [bitcoin] laanwj closed pull request #6804: [tests] Add basic coverage reporting for RPC tests (master...rpc_coverage) https://github.com/bitcoin/bitcoin/pull/6804
2172015-11-12T07:23:30 <wumpus> even dropping off your transactions to a random node over Tor is better than publicly broadcasting your transaction from your own node
2182015-11-12T07:29:39 <gmaxwell> yea, but requires having a tor client. :(
2192015-11-12T07:30:23 <wumpus> it does, having a tor client seems more and more a requirement for any kind of privacy unfortunately
2202015-11-12T07:31:25 <gmaxwell> it's just a lot of code and exposure, hate having reason to mix things like that with other things that have critical security requirements.
2212015-11-12T07:31:31 <wumpus> a tor bitcoin bundle would be pretty nice
2222015-11-12T07:31:58 <wumpus> well at least it gives plenty of noise to hide in
2232015-11-12T07:32:57 <wumpus> if you were to say, set up an onion routing network just for transactions, even partaking in it would still leak a lot information about you. Also, if you write it yourself doesn't make it any less 'code and exposure'
2242015-11-12T07:33:32 <gmaxwell> It's true, for tx broadcast though, one could send a transaction through a high latency mixmaster like system entered via normal internet. And the daemon could just send an empty message an interior hop one ever couple minutes to hide the traffic. Doing the first hop over tor is superior, but I think not strictly needed.
2252015-11-12T07:34:11 <wumpus> I agree that such a solution could theoretically be better, the problem as we've seen for years is that no one is implementing such a thing, and tor already exists
2262015-11-12T07:34:30 <gmaxwell> wumpus: the client could be quite simple relative to tor (which at the very begining immediately needs a full SSL stack)
2272015-11-12T07:34:36 <gmaxwell> Indeed.
2282015-11-12T07:34:49 <gmaxwell> Actually someone contacted me recently about implementing such a thing, but I don't know if they will.
2292015-11-12T07:35:18 <gmaxwell> (well they said they will, and asked if I'd be willing/able to review; but you know most things don't actually happen :) )
2302015-11-12T07:36:20 <gmaxwell> wumpus: e.g. sending client for such a thing could encrypt a message, connect to a TCP socket, and send it over. Never reading a byte off the socket even. Attack surface would be incompariable to tor. :)
2312015-11-12T07:38:26 <wumpus> the reason that Tor uses SSL, by the way, is that that masquerading as SSL is one of the few feasible ways to hide an encrypted connection in plain sight
2322015-11-12T07:39:20 <gmaxwell> Indeed, for good reason. I don't think you're going to manage to hide that you're sending a transaction at all unless you use tor... but you well could hide which transaction you were sending. Esp if you don't mind broadcast taking a minute.
2332015-11-12T07:39:21 <wumpus> if you start with your own such system you'd very soon run against the same issue
2342015-11-12T07:41:49 <gmaxwell> (in particular it's completely reasonable to have high cover traffic, if you don't mind latency... just send a 100k message into the routing network every N minutes, containing some random assortment of transactions from the public network, and any of your own). If you're not running this over tor, you'll stand out as a user of the software but not more than that.
2352015-11-12T07:41:54 <gmaxwell> )
2362015-11-12T07:42:12 <wumpus> well you could still hide that you're sending a transaction by hiding in noise, e.g. also using the system to automatically send other people's transactions
2372015-11-12T07:42:48 *** Thireus has quit IRC
2382015-11-12T07:43:01 <wumpus> just like the relaying system we have now but encrypted, and high-latency, and with padding so that you can't look at the envelope and identify by the size, etc etc
2392015-11-12T07:43:44 <gmaxwell> exactly. which is also good in that if someone finds out how to distinguish transactions exiting the system they might seek to punish their authors, but if you're sending random other transactions into the system, what exits will not just be from witting users.
2402015-11-12T07:45:03 <kanzure> we are still missing out on a high-latency mixnet of some kind
2412015-11-12T07:45:20 <wumpus> but anyhow - just like 5 years ago, I'd be pleasantly surprised if someone build something like that
2422015-11-12T07:45:41 <wumpus> exactly kanzure
2432015-11-12T07:45:49 <gmaxwell> prior to bitcoin they were hard(er) because of flooding attacks, but we have a one-two punch here: we have highly valuable tiny messages AND mechenisms to stop flooding attacks.
2442015-11-12T07:52:34 <wumpus> lol "On January 23, 2006, the Russian FSB accused Britain of using wireless dead drops concealed inside hollowed-out rocks to collect espionage information from agents in Russia. According to the Russian authorities, the agent delivering information would approach the rock and transmit data wirelessly into it from a hand-held device, and later his British handlers would pick up the stored data by similar means." If
2452015-11-12T07:52:35 <wumpus> everything fails, we could always use fake rocks :-)
2462015-11-12T07:53:57 <gmaxwell> "some of us already are"
2472015-11-12T07:54:27 <randy-waterhouse> yes, btcoin has been accused of using "fake rocks" many times already
2482015-11-12T07:55:03 <wumpus> randy-waterhouse: any links?
2492015-11-12T07:55:17 * phantomcircuit looks at scrollback
2502015-11-12T07:55:36 *** MarcoFalke has joined #bitcoin-core-dev
2512015-11-12T07:56:29 <randy-waterhouse> wumpus: tongue-in-cheek reference to "fake gold" accusations
2522015-11-12T07:56:43 <MarcoFalke> wumpus, using `-proxy` I get "2015-11-12 07:43:06 ERROR: Proxy error: connection not allowed
2532015-11-12T07:56:43 <MarcoFalke> " for .onion and "2015-11-12 07:43:56 ERROR: Proxy error: host unreachable
2542015-11-12T07:56:44 <MarcoFalke> " for the seeds
2552015-11-12T07:57:01 <MarcoFalke> Are those implying censorship?
2562015-11-12T07:57:03 <wumpus> randy-waterhouse: ahh old pyrite, or fool's gold :-)
2572015-11-12T07:57:25 <randy-waterhouse> heh, right
2582015-11-12T07:58:21 <wumpus> MarcoFalke: it may just mean what it says - the proxy cannot reach those hosts
2592015-11-12T07:58:52 <wumpus> MarcoFalke: sometimes hosts are truly unreachable. I think you need to look at tor's logging to see if it can reach the first hop to troubleshoot this
2602015-11-12T07:59:34 <wumpus> but at least you're one step further- you get errors from the proxy, instead of the level before of connecting to the proxy
2612015-11-12T08:00:44 <wumpus> it may just have a hard time finding a working .onion peer here try this one: nns4r54x3lfbrkq5.onion
2622015-11-12T08:13:25 <MarcoFalke> It only connects to nkf5e6b7pl4jfd4a.onion (TheBlueMatt)
2632015-11-12T08:14:44 <wumpus> ok, in that case it's at least not censorship, if it can connect to one onion it can in principle connect to all without your ISP knowing anything
2642015-11-12T08:18:15 *** Thireus has joined #bitcoin-core-dev
2652015-11-12T08:26:04 *** NLNico has quit IRC
2662015-11-12T08:26:11 <MarcoFalke> I don't have to set `-proxy=127...`?
2672015-11-12T08:26:24 <wumpus> you do have to set a proxy
2682015-11-12T08:26:35 <wumpus> otherwise it won't know how to connect to onions
2692015-11-12T08:26:47 <MarcoFalke> Right now I am doing `-onlynet=onion -torpassword=`
2702015-11-12T08:26:58 <MarcoFalke> and I can `-connect=` to it
2712015-11-12T08:27:01 <MarcoFalke> but not sync
2722015-11-12T08:27:18 <wumpus> it isn't able to derive the Tor proxy from the control port. Maybe this is possible, but it is not implemented.
2732015-11-12T08:28:07 <MarcoFalke> I am doing `-connect=moheur3skn7jbca4.onion -proxy=127.0.0.1:9050 -testnet` locally and get the connection
2742015-11-12T08:29:45 <MarcoFalke> But I need the proxy for outgoing connections, I see.
2752015-11-12T08:29:52 <wumpus> ohh testnet
2762015-11-12T08:30:15 <wumpus> probably very, very few onion testnet peers
2772015-11-12T08:30:36 <MarcoFalke> Still, why wouldn't they sync each others blocks if connected?
2782015-11-12T08:32:01 <MarcoFalke> Could you try `-connect=moheur3skn7jbca4.onion -proxy=127.0.0.1:9050 -testnet` on your machine?
2792015-11-12T08:32:27 <wumpus> sure
2802015-11-12T08:32:38 <wumpus> with an empty datadir?
2812015-11-12T08:33:03 <MarcoFalke> yes, I have 2400 blocks
2822015-11-12T08:33:17 <MarcoFalke> *47762
2832015-11-12T08:34:13 <wumpus> 2015-11-12 08:34:02 SOCKS5 connecting moheur3skn7jbca4.onion / 2015-11-12 08:34:08 SOCKS5 connected moheur3skn7jbca4.onion/ 2015-11-12 08:34:08 receive version message: /Satoshi:0.11.99/: version 70011, blocks=47762, us=0.0.0.0:0, peer=1
2842015-11-12T08:34:27 <wumpus> and nothing after that
2852015-11-12T08:34:53 <wumpus> could mean two things a) I'm not sending getheaders b) peer is ignoring my getheaders
2862015-11-12T08:35:27 <wumpus> as for (b) I know this issue all too well: https://github.com/bitcoin/bitcoin/issues/6971
2872015-11-12T08:36:25 <wumpus> but this required P2P-level debugging, it's not tor's fault :)
2882015-11-12T08:36:39 *** gribble has quit IRC
2892015-11-12T08:37:08 <MarcoFalke> I don't run master. Still running `f1d548d squashme: mention torcontrol in doc/tor.md
2902015-11-12T08:37:09 <MarcoFalke> ` with no merge
2912015-11-12T08:37:41 <MarcoFalke> I guess thats an tested ACK on 6639, then?
2922015-11-12T08:37:46 <wumpus> that doesn't matter - if it's the *other* peer that's ignoring you, nothing you can do
2932015-11-12T08:38:54 <MarcoFalke> We are both in IBD?
2942015-11-12T08:39:03 <wumpus> using debug=net I can see that I'm sending getheaders, but he's not responding
2952015-11-12T08:39:26 <wumpus> or he's forked off the chain completely
2962015-11-12T08:39:45 <wumpus> anything that would make the tip older than 24 hours old will preent him from sending headers
2972015-11-12T08:40:14 <wumpus> I remember there were some issues on testnet
2982015-11-12T08:41:02 <MarcoFalke> block 47762 is Jan 2013
2992015-11-12T08:41:50 <wumpus> but he seems stuck there. blocks=... doesn't go up after reconnecting :)
3002015-11-12T08:42:06 <wumpus> so we'll have to find a good testnet onion peer to check this
3012015-11-12T08:42:16 <MarcoFalke> one sec...
3022015-11-12T08:42:55 <MarcoFalke> going to sync and then `-connect=` again
3032015-11-12T08:43:31 <wumpus> I can spin up one, after catching up (or I'll just comment out that check in #6971, so anyone-can-getheaders)
3042015-11-12T08:44:14 <wumpus> ok one min
3052015-11-12T08:45:03 *** JackH has joined #bitcoin-core-dev
3062015-11-12T08:47:26 *** gribble has joined #bitcoin-core-dev
3072015-11-12T08:48:30 *** BashCo has quit IRC
3082015-11-12T08:50:03 <wumpus> MarcoFalke: tor: Got service ID vbzzvljohcyzlfzg, advertizing service vbzzvljohcyzlfzg.onion:8333 you should be able to sync from that
3092015-11-12T08:50:24 <wumpus> eh wait that's not testnet
3102015-11-12T08:51:00 <wumpus> it is: ytiwtue6no6c3cit.onion:18333
3112015-11-12T08:51:33 <wumpus> may take some seconds for the new service id to propagate over the tor network
3122015-11-12T08:53:07 <MarcoFalke> Could you make that `-connect=moheur3skn7jbca4.onion -proxy=127.0.0.1:9050 -testnet` and then sync me?
3132015-11-12T08:53:27 <MarcoFalke> because I am still running with no `-proxy` to prevent outgoing connections
3142015-11-12T08:55:39 <wumpus> ok
3152015-11-12T08:57:03 <wumpus> I can connect, but I cannot sync from you, because your node is in IBD
3162015-11-12T08:58:23 <wumpus> /Satoshi:0.11.99(hi_wumpus)/ hehe
3172015-11-12T09:00:01 <wumpus> anyhow yes that's a tested ACK on #6639 - it's clear that creating the hidden service and makng it connectible works
3182015-11-12T09:06:10 *** molly has quit IRC
3192015-11-12T09:09:19 <MarcoFalke> Ok, then. Will do some more testing and comment on the pull. ;)
3202015-11-12T09:09:45 *** BashCo has joined #bitcoin-core-dev
3212015-11-12T09:12:12 *** MarcoFalke has left #bitcoin-core-dev
3222015-11-12T10:22:58 *** rubensayshi has joined #bitcoin-core-dev
3232015-11-12T10:25:43 *** zarathustra has joined #bitcoin-core-dev
3242015-11-12T10:34:41 *** d_t has quit IRC
3252015-11-12T10:38:42 <phantomcircuit> gmaxwell, so im (soft) disabling walletbroadcast but what am i doing about whitelisted peers
3262015-11-12T10:38:56 <phantomcircuit> it seems that there's an argument to be made both ways
3272015-11-12T10:40:29 <gmaxwell> phantomcircuit: relay for them; and add an option to control their relay special casing (e.g. so you can turn off the behavior that irritates me too. :) ) and then you can soft set that off too when blocksonly is on.
3282015-11-12T10:41:11 <sipa> that seems neat
3292015-11-12T10:41:59 <phantomcircuit> -relaywhitelisted ?
3302015-11-12T10:43:29 *** pmienk has quit IRC
3312015-11-12T10:43:41 <gmaxwell> whitelistalwaysrelay might be more descriptive.
3322015-11-12T10:44:07 <gmaxwell> since turning it off should turn off the current behavior where we will accept txn from a whitelisted peer that we wouldn't otherwise.
3332015-11-12T10:44:42 <sipa> right, normal relay for whitelisted peers would keep working even without the special case behaviour
3342015-11-12T10:45:09 <phantomcircuit> default to true or false?
3352015-11-12T10:45:23 <phantomcircuit> i guess default to true unless blocksonly=1
3362015-11-12T10:45:29 <gmaxwell> default to true (current behavior) and soft-off it in blocksonly mode.
3372015-11-12T10:46:16 <gmaxwell> then people who want to relay for nodes behind them while in blocksonly can turn it on.
3382015-11-12T10:46:19 *** pmienk has joined #bitcoin-core-dev
3392015-11-12T10:56:04 <phantomcircuit> gmaxwell, bleh it's now a hideously huge 13 loc patch
3402015-11-12T10:56:32 <phantomcircuit> :P
3412015-11-12T10:57:30 <gmaxwell> obviously you have to find some unrelated code to remove at the same time... (heh no not really)
3422015-11-12T10:59:34 *** paveljanik has joined #bitcoin-core-dev
3432015-11-12T11:00:08 <wumpus> phantomcircuit: we don't make it easy to make a small patch, do we :)
3442015-11-12T11:01:27 <sipa> we need patches that are C++-neutral
3452015-11-12T11:02:34 <phantomcircuit> wumpus, lol @ constant
3462015-11-12T11:02:38 <phantomcircuit> BUT BUT
3472015-11-12T11:03:18 <phantomcircuit> wumpus, main.h or net.h?
3482015-11-12T11:03:48 <wumpus> phantomcircuit: depends on where it's used
3492015-11-12T11:03:52 <wumpus> I thought main.h
3502015-11-12T11:04:21 *** randy-waterhouse has quit IRC
3512015-11-12T11:05:51 <sipa> phantomcircuit: your patch will go into history books as the one with the higher discussion per lines of code patch ever that got merged
3522015-11-12T11:06:26 <phantomcircuit> lol
3532015-11-12T11:07:54 <paveljanik> if it will be merged at all! :-)
3542015-11-12T11:26:41 <gmaxwell> sipa: pfft.
3552015-11-12T11:27:39 <gmaxwell> https://github.com/bitcoin/bitcoin/commit/b196b685c9089b74fd4ff3d9a28ea847ab36179b
3562015-11-12T11:36:35 <phantomcircuit> oy the mix of logic between net.cpp and main.cpp is super annoying
3572015-11-12T11:36:57 <sipa> phantomcircuit: the certainties of life :)
3582015-11-12T11:37:13 <phantomcircuit> net.cpp:454:89: error: âDEFAULT_BLOCKSONLYâ was not declared in this scope
3592015-11-12T11:37:20 <phantomcircuit> well yes because it's defined in main.h
3602015-11-12T11:37:23 <phantomcircuit> >.>
3612015-11-12T11:37:58 <sipa> put it in the node state in main
3622015-11-12T11:38:33 <phantomcircuit> sipa, put it in CNode ?
3632015-11-12T11:38:37 <sipa> no
3642015-11-12T11:38:42 <sipa> in CNodeState
3652015-11-12T11:39:09 <sipa> CNode is the network layer's kitchen sink
3662015-11-12T11:39:19 <sipa> (with a ton of state from processing)
3672015-11-12T11:39:25 <sipa> CNodeState is just processing
3682015-11-12T11:40:20 <phantomcircuit> sipa, is that accessible from CNode::Cnode ?
3692015-11-12T11:40:25 <sipa> no
3702015-11-12T11:40:27 <sipa> that's the point
3712015-11-12T11:40:31 <sipa> none of this should be in net
3722015-11-12T11:45:23 <gmaxwell> http://blog.nordeus.com/dev-ops/power-failure-testing-with-ssds.htm
3732015-11-12T11:52:16 <wumpus> <phantomcircuit> oy the mix of logic between net.cpp and main.cpp is super annoying <- oh, you're only now discovering that? :-)
3742015-11-12T11:52:32 <phantomcircuit> sipa, ok then i have to move PushVerison out of Cnode::CNode :/
3752015-11-12T11:52:52 <phantomcircuit> (it actually shouldn't be there anyways so i guess net win... horray)
3762015-11-12T11:59:20 <wumpus> but yes, the split between net.cpp and main.cpp is not optimal, it left too many P2P related code in main
3772015-11-12T11:59:39 <GitHub139> [bitcoin] sipa opened pull request #6996: Add preciousblock RPC (master...preciousblock) https://github.com/bitcoin/bitcoin/pull/6996
3782015-11-12T12:00:18 <sipa> wumpus: and too much processing in net
3792015-11-12T12:00:54 <sipa> vRecvGetData has no business in CNode, for example
3802015-11-12T12:01:02 <wumpus> right
3812015-11-12T12:01:22 <sipa> how do i run the python tests from the command line?
3822015-11-12T12:01:49 <wumpus> cd qa/pulltester; ./rpc-tests.py
3832015-11-12T12:02:57 <sipa> ha, ok!
3842015-11-12T12:04:41 <wumpus> and if you want to run a seperate test you can execute it manually from qa/rpc-tests
3852015-11-12T12:05:23 <sipa> oh lol, i can't remember it being so easy
3862015-11-12T12:05:46 *** jtimon has joined #bitcoin-core-dev
3872015-11-12T12:06:43 <wumpus> whoa does anyone ever read README.md? :-) it still mentions the pull-tester
3882015-11-12T12:06:47 <GitHub93> [bitcoin] sturles opened pull request #6997: Don't use hard-coded AllowFree, as this is usually far too low. (master...no-default-free-priority) https://github.com/bitcoin/bitcoin/pull/6997
3892015-11-12T12:09:56 <GitHub55> [bitcoin] laanwj opened pull request #6998: doc: Remove mention of pulltester from README.md (master...2015_11_readme_pull_pulltester) https://github.com/bitcoin/bitcoin/pull/6998
3902015-11-12T12:13:46 <wumpus> I also think the "Manual Quality Assurance (QA) Testing" is very much out of date
3912015-11-12T12:15:53 <GitHub86> [bitcoin] jonasschnelli opened pull request #6999: add (max)uploadtarget infos to getnetinfos RPC help (master...2015/11/maxuploadtarget_rpc_help) https://github.com/bitcoin/bitcoin/pull/6999
3922015-11-12T12:15:56 <GitHub29> [bitcoin] jonasschnelli opened pull request #7000: [Qt] add shortcurts for debug-/console-window (master...2015/11/qt_shortcuts) https://github.com/bitcoin/bitcoin/pull/7000
3932015-11-12T12:15:58 <jonasschnelli> Yeah. PR #7000!
3942015-11-12T12:16:28 <phantomcircuit> sipa, where else would vRecvGetData be?
3952015-11-12T12:16:38 <sipa> phantomcircuit: CNodeState
3962015-11-12T12:16:45 <phantomcircuit> ah
3972015-11-12T12:17:32 <phantomcircuit> sipa, is the distinction you're thinking between the socket and the protocol things?
3982015-11-12T12:17:48 <phantomcircuit> if so maybe CBitcoinNode : CNode
3992015-11-12T12:18:37 <sipa> phantomcircuit: the idea (well, my idea... i don't know how many people share it) is that CNode should hold pointers to various module-specific state objects
4002015-11-12T12:18:52 <sipa> phantomcircuit: and that when invoking code from one module, it just gets its own state objected handed to it
4012015-11-12T12:19:43 <sipa> so ping handling can be entirely separate from block synchronization for example, and use completely independent locks, and even be totally unaware of the other's existance
4022015-11-12T12:23:39 <wumpus> sounds like a sensible way to handle a protocol with different (mostly) independent concerns
4032015-11-12T12:28:42 <phantomcircuit> yes except for the part where we decided to maintain the order of responses
4042015-11-12T12:29:35 <wumpus> uhm, you mean that responses come in the same order as commands that trigger them?
4052015-11-12T12:30:03 <wumpus> that's a pretty sane assumption that holds for most network protocols AFAIK
4062015-11-12T12:30:39 <sipa> phantomcircuit: that's not incompatible
4072015-11-12T12:31:08 <sipa> phantomcircuit: the ping module could be processing a message from node A, while the block sync logic is processing a message from node B
4082015-11-12T12:32:51 <phantomcircuit> ah
4092015-11-12T12:33:23 <phantomcircuit> wumpus, there's afaict no really good reason to enforce ordering of the responses actually
4102015-11-12T12:33:54 <phantomcircuit> the only thing that relies on it in anyway is the initial block download thing
4112015-11-12T12:34:02 <phantomcircuit> and im not sure it really does anymore
4122015-11-12T12:34:06 <sipa> it does
4132015-11-12T12:34:09 <sipa> normal block sync does too
4142015-11-12T12:34:20 <sipa> bip37-based downloading does too
4152015-11-12T12:34:27 <wumpus> phantomcircuit: I don't see a reason to make it different though
4162015-11-12T12:34:36 <sipa> and a ton of test infrastructure relies on it
4172015-11-12T12:36:34 <wumpus> is there anything specific that could be done better if it was not the case?
4182015-11-12T12:59:56 *** go1111111 has quit IRC
4192015-11-12T13:02:10 *** go1111111 has joined #bitcoin-core-dev
4202015-11-12T13:09:51 *** fkhan has quit IRC
4212015-11-12T13:24:56 *** fkhan has joined #bitcoin-core-dev
4222015-11-12T13:37:39 <GitHub129> [bitcoin] sipa pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/5fcc14ee0562...54e8bfec8387
4232015-11-12T13:37:40 <GitHub129> bitcoin/master 06d81ad Alex Morcos: Skip BIP30 check after BIP34 activation
4242015-11-12T13:37:40 <GitHub129> bitcoin/master 33c90cf Alex Morcos: Make skipping BIP30 check chain agnostic
4252015-11-12T13:37:41 <GitHub129> bitcoin/master 54e8bfe Pieter Wuille: Merge pull request #6931...
4262015-11-12T13:37:42 <GitHub166> [bitcoin] sipa closed pull request #6931: Skip BIP 30 verification where not necessary (master...skipBIP30) https://github.com/bitcoin/bitcoin/pull/6931
4272015-11-12T13:45:40 *** jgarzik has quit IRC
4282015-11-12T13:49:38 *** fkhan has quit IRC
4292015-11-12T13:51:56 *** molly has joined #bitcoin-core-dev
4302015-11-12T14:01:09 *** jtimon has quit IRC
4312015-11-12T14:04:46 <phantomcircuit> wumpus, yes we could improve the throughput of getdata requests if they were out of order
4322015-11-12T14:05:06 <phantomcircuit> as it is there's a large latency hit if the block requested isn't in cache
4332015-11-12T14:05:06 <wumpus> how?
4342015-11-12T14:06:05 <phantomcircuit> wumpus, parallel fetch from disk
4352015-11-12T14:06:41 <wumpus> getdata is already enough of a i/o DoS attack as it is, thank you
4362015-11-12T14:06:49 <sipa> i don't think we should go write our own optimizer for disk fetches
4372015-11-12T14:06:53 <phantomcircuit> hell we could even rewrite the requests so that they hit disk sequentially
4382015-11-12T14:07:17 <sipa> phantomcircuit: you'd also need a reorganizer that rewrites the block files to be sequential
4392015-11-12T14:07:25 <sipa> please don't go there
4402015-11-12T14:07:29 <wumpus> TBH this doesn't sound very reasonable nor useful, please profile first before you think about optimizing anything
4412015-11-12T14:08:12 <phantomcircuit> sipa, parallel fetch isn't anywhere near as complex as writing our own optimizer.... but it might need one if it's not to completely break hdds
4422015-11-12T14:08:23 <wumpus> optimizing the disk component of getdata doesn't rank anywhere in the 'concerns that slow down bitcoin core in practice' I'm sure
4432015-11-12T14:08:57 *** BashCo has quit IRC
4442015-11-12T14:10:04 <sipa> phantomcircuit: so if that's really a concern at a time when the protocol too strongly relies on the ordering to meaningfully change it, we can always add a pargetdata which allows answers in any order. fixed.
4452015-11-12T14:10:45 <wumpus> we already do parallel getdata on different nodes, that's even better paralleism thanks to sipa :)
4462015-11-12T14:13:18 <phantomcircuit> wumpus, it basically only effects people like me syncing over gigabit lans from systems with the entire thing in page cache
4472015-11-12T14:13:31 <wumpus> ... right
4482015-11-12T14:13:59 <phantomcircuit> increasing the fetch depth to 100 mostly fixes it
4492015-11-12T14:23:58 *** jgarzik has joined #bitcoin-core-dev
4502015-11-12T14:23:58 *** jgarzik has joined #bitcoin-core-dev
4512015-11-12T14:24:49 *** Guyver2 has joined #bitcoin-core-dev
4522015-11-12T14:28:56 <morcos> sipa: ha, i was chasing my own tail on that insecure_rand. Ok so should I fix the extra 30 bits of randomness or take our chances on only 32. I vote take our chances.
4532015-11-12T14:29:46 <sipa> morcos: will the test fail if it actually is identical?
4542015-11-12T14:30:48 <morcos> i was going to consider testing that. i was thinking that it might allow you to spend a duplicate coinbase which has already been spent
4552015-11-12T14:30:51 *** fkhan has joined #bitcoin-core-dev
4562015-11-12T14:31:36 <morcos> but maybe even thats ok, unless you randomly spend the second copy in exactly the same way. in which case it would break
4572015-11-12T14:31:39 <morcos> i think?
4582015-11-12T14:32:14 <sipa> morcos: hmm, your cache will be very slow
4592015-11-12T14:32:16 <sipa> eh, small
4602015-11-12T14:32:23 *** Guyver2 has quit IRC
4612015-11-12T14:32:29 <morcos> it is slow, but what do you mean small?
4622015-11-12T14:33:10 <sipa> eh, for some reason i was assuming that normal transactions decreased the number of entries... but they create a new output too of course
4632015-11-12T14:33:23 <morcos> it about triples the time of that test from 300ms to 1sec
4642015-11-12T14:33:24 *** Guyver2 has joined #bitcoin-core-dev
4652015-11-12T14:34:09 <sipa> i guess you grow to around 4000 entries
4662015-11-12T14:34:28 <morcos> yeah so on a typical test you generate around 4000 coinbases about 40 of which are duplicate and you end up towards the end of the test, failing on about 40 of them (if you have the broken fix)
4672015-11-12T14:34:38 <morcos> that should be right
4682015-11-12T14:35:08 <morcos> sorry around 400 of which are duplicate
4692015-11-12T14:37:09 <morcos> b/c you only fail on the duplicate if it happens to get spent before its flushed. so i didn't check for coverage of that, since its a bit tricky, but that seemed a high enough failure rate that slight tweaks to the test shouldn't affect it.
4702015-11-12T14:38:21 <sipa> you have around 1.7% chance to hit a duplicate during a test run :)
4712015-11-12T14:38:56 <sipa> oh, wait, that's assuming there are only coinbases
4722015-11-12T14:41:30 <sipa> around 0.2% chance per run; i think you need 64-bit amounts :)
4732015-11-12T14:41:59 <morcos> how did you get 1.7%? does that sound really high? do you have to have the same random 32 bit number in 4000 tries (or 40k for the calculation you did)
4742015-11-12T14:42:28 <sipa> i calculated 1 - multiply(1 - x/2^32, x=1..4000)
4752015-11-12T14:42:33 <sipa> that's 0.2%
4762015-11-12T14:42:53 <morcos> oh silly me
4772015-11-12T14:43:06 <morcos> yes i calculated that wrong
4782015-11-12T14:44:27 <morcos> ok i'll make it a 64-bit number with the cast.
4792015-11-12T14:44:50 <sipa> of course, you could just use the iteration counter instead :)
4802015-11-12T14:45:31 <morcos> good even easier
4812015-11-12T14:45:52 <morcos> in a car now.. i'll push it in a bit
4822015-11-12T14:58:23 <morcos> ok lots of traffic, squashed the fix in.
4832015-11-12T14:59:06 <instagibbs> Hopefully not while driving :)
4842015-11-12T14:59:40 <morcos> heh, no...
4852015-11-12T15:00:04 <jgarzik> heh
4862015-11-12T15:00:47 <sipa> Hopefully not while trafficking
4872015-11-12T15:03:06 <jgarzik> with Uber & self driving cars, non stop hacking is possible :) Ah, the future.
4882015-11-12T15:03:33 <sipa> Where is my hoverboard?
4892015-11-12T15:23:51 *** ParadoxSpiral has joined #bitcoin-core-dev
4902015-11-12T15:25:29 <jgarzik> sipa, http://hendohover.com
4912015-11-12T15:26:34 <sipa> jgarzik: ha, yes, i've heard about that
4922015-11-12T15:27:17 <wumpus> hah
4932015-11-12T15:28:24 *** treehug88 has joined #bitcoin-core-dev
4942015-11-12T15:29:09 *** Thireus has quit IRC
4952015-11-12T15:48:53 *** zooko has joined #bitcoin-core-dev
4962015-11-12T15:49:52 *** jtimon has joined #bitcoin-core-dev
4972015-11-12T16:06:19 *** zooko has quit IRC
4982015-11-12T16:32:52 *** zooko has joined #bitcoin-core-dev
4992015-11-12T16:38:14 <GitHub77> [bitcoin] laanwj pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/54e8bfec8387...eb6172a8ca7e
5002015-11-12T16:38:15 <GitHub77> bitcoin/master 830e3f3 Pieter Wuille: Make sigcache faster and more efficient
5012015-11-12T16:38:15 <GitHub77> bitcoin/master 0b9e9dc Pieter Wuille: Evict sigcache entries that are seen in a block
5022015-11-12T16:38:16 <GitHub77> bitcoin/master 69d373f Pieter Wuille: Don't wipe the sigcache in TestBlockValidity
5032015-11-12T16:38:21 <GitHub101> [bitcoin] laanwj closed pull request #6918: Make sigcache faster, more efficient, larger (master...smallsigcache) https://github.com/bitcoin/bitcoin/pull/6918
5042015-11-12T16:48:17 *** rubensayshi has quit IRC
5052015-11-12T16:52:57 *** d_t has joined #bitcoin-core-dev
5062015-11-12T16:56:36 *** Thireus has joined #bitcoin-core-dev
5072015-11-12T16:58:17 *** d_t has quit IRC
5082015-11-12T17:37:23 *** MarcoFalke has joined #bitcoin-core-dev
5092015-11-12T17:45:57 *** skyraider has joined #bitcoin-core-dev
5102015-11-12T18:33:46 <GitHub51> [bitcoin] laanwj pushed 6 new commits to master: https://github.com/bitcoin/bitcoin/compare/eb6172a8ca7e...bd629d77edbe
5112015-11-12T18:33:46 <GitHub33> [bitcoin] laanwj closed pull request #6639: net: Automatically create hidden service, listen on Tor (master...2015_08_tor_hs_v2) https://github.com/bitcoin/bitcoin/pull/6639
5122015-11-12T18:33:47 <GitHub51> bitcoin/master 8f4e67f Wladimir J. van der Laan: net: Automatically create hidden service, listen on Tor...
5132015-11-12T18:33:47 <GitHub51> bitcoin/master 2f796e5 Peter Todd: Better error message if Tor version too old
5142015-11-12T18:33:48 <GitHub51> bitcoin/master 09c1ae1 Wladimir J. van der Laan: torcontrol improvements and fixes...
5152015-11-12T18:44:30 *** morcos has quit IRC
5162015-11-12T18:45:41 *** morcos has joined #bitcoin-core-dev
5172015-11-12T18:59:00 *** d_t has joined #bitcoin-core-dev
5182015-11-12T18:59:41 *** d_t has joined #bitcoin-core-dev
5192015-11-12T19:00:20 *** d_t has joined #bitcoin-core-dev
5202015-11-12T19:00:58 *** d_t has joined #bitcoin-core-dev
5212015-11-12T19:02:01 <jcorgan> wow, i missed this PR entirely. what a great idea, glad it is merged.
5222015-11-12T19:12:13 <MarcoFalke> yes, good to see this in 0.12!
5232015-11-12T19:35:13 <GitHub56> [bitcoin] sdaftuar closed pull request #6992: [WIP] [Mempool] Add space for priority transactions (master...add-priority-to-mempool) https://github.com/bitcoin/bitcoin/pull/6992
5242015-11-12T19:55:52 *** randy-waterhouse has joined #bitcoin-core-dev
5252015-11-12T19:57:28 <Luke-Jr> morcos: ok, so back to your stuff? :P
5262015-11-12T19:58:22 <gmaxwell> Luke-Jr: on priority. the exact current behavior requires walking all inputs for all transactioin in the mempool on every CNB to update priorities. With large mempools this is an enormous amount of time, and we cannot continue doing this.
5272015-11-12T19:58:27 <Luke-Jr> morcos: can you make it so the accurate priority gets used with a policy option miners can enable? that makes current policy possible, right?
5282015-11-12T19:58:34 <gmaxwell> There are multiple alternatives but most result in lots of software complexity, which we also probably don't want.
5292015-11-12T19:58:46 <morcos> Hmm.. I think that would be possible
5302015-11-12T19:58:47 <Luke-Jr> gmaxwell: it can be disabled by default
5312015-11-12T19:58:59 <morcos> The original version of my pull did the expensive calculation
5322015-11-12T19:59:14 <morcos> it'll make the code slightly uglier casing that out
5332015-11-12T19:59:21 <gmaxwell> Luke-Jr: I think miners would be stupid to enable something that makes CNB 10x + slower, and we'd be stupid to offer it since slow CNB means more 'spv mining' and other bad effects.
5342015-11-12T19:59:32 <morcos> but yes, i agree with that concern
5352015-11-12T19:59:44 <sdaftuar> what's the motivation to keep the current version of priority rather than starting priority as the metric?
5362015-11-12T19:59:51 <Luke-Jr> gmaxwell: CNB is not time-critical with Eloipool, and doesn't imply SPV mining
5372015-11-12T19:59:53 <gmaxwell> I wouldn't _oppose_ an option, default off, except on the basis of carrying around code, and lack of testing (but I assume you'll test)
5382015-11-12T19:59:56 <morcos> Luke-Jr: it may be something you can just easily patch in yourself
5392015-11-12T20:00:07 <petertodd> sdaftuar: I think for most uses of priority, updating every time shouldn't make much of a difference - we're talking about old inputs after all
5402015-11-12T20:00:21 <petertodd> sdaftuar: whats the difference betweek 1 month and 1 month + 1 hour?
5412015-11-12T20:00:31 <Luke-Jr> I suggest we have it in Core for 0.12, and try to optimise (or remove) by 0.13
5422015-11-12T20:00:45 <gmaxwell> sdaftuar: I think starting priority is a pretty poor approximation, though on the basis that I wouldn't be too opposed to ditching priority entirely, using a poor approximation isn't so bad.
5432015-11-12T20:00:56 <sdaftuar> it just seems pretty arbitrary to me to decide to age inputs rather than use the age when relayed to you
5442015-11-12T20:01:04 <sdaftuar> so given two arbitrary calculations, might as well do the faster one
5452015-11-12T20:01:19 <gmaxwell> sdaftuar: age inputs has a property that the tx shouldn't hang around forever, but will actually bubble up and get mined at some point.
5462015-11-12T20:01:24 <Luke-Jr> sdaftuar: well, aging inputs properly may in some cases be necessary for txns to confirm ever
5472015-11-12T20:01:44 <sdaftuar> well we're about to deploy RBF and expiry
5482015-11-12T20:01:46 <petertodd> the only time this matters is very high value outputs which we're expecting to accumulate priority quickly, and why incentivise people to put everything in on etxout? can have bad privacy implications
5492015-11-12T20:02:02 <Luke-Jr> sdaftuar: as optional policies
5502015-11-12T20:02:18 *** skyraider has quit IRC
5512015-11-12T20:02:22 <phantomcircuit> Luke-Jr, then write a background thread to update priority
5522015-11-12T20:02:27 <petertodd> phantomcircuit: +1
5532015-11-12T20:02:31 <morcos> Luke-Jr: I'll see if I can whip up a dependent PR that adds back in the accurate calculation with an option, that could be included.
5542015-11-12T20:02:43 <Luke-Jr> phantomcircuit: that would be one way to optimise it, but I hope to have something better for 0.13 anyway
5552015-11-12T20:03:13 <gmaxwell> I do agree if the functionality was retained update in the background would be prudent.
5562015-11-12T20:03:15 <Luke-Jr> morcos: well, I don't want to merge the existing PR without that included, so might as well put it in the same PR..
5572015-11-12T20:03:16 <morcos> If you want to properly calculate priority, you need to do it dynamicalyl as in 6357 otherwise you destroy your UTXO cache
5582015-11-12T20:03:26 <morcos> 6357 is strictly better than that
5592015-11-12T20:03:40 <morcos> Luke-Jr: yes but that may not be the prevailing view.
5602015-11-12T20:03:58 <morcos> Well lets argue about it when we see what it looks like anyway
5612015-11-12T20:04:04 <phantomcircuit> Luke-Jr, the priority calculation as it existed before mempool limiting cannot be reasonably calculated in realtime, the only way to do it is as a background thread
5622015-11-12T20:04:11 <wumpus> * [new tag] v0.11.2 -> v0.11.2
5632015-11-12T20:04:23 <petertodd> wumpus: +1
5642015-11-12T20:04:24 <morcos> First I have to fix the existing pull, it uses the wrong calculation now anyway
5652015-11-12T20:04:30 <petertodd> wumpus: or really, -rc
5662015-11-12T20:04:39 <Luke-Jr> petertodd: you missed the rc
5672015-11-12T20:04:50 <morcos> phantomcircuit: no, background thread is bad b/c of cache. see 6357
5682015-11-12T20:04:54 <petertodd> Luke-Jr: lol, I mean, we're taking the rc off of v0.11.2rc1 :)
5692015-11-12T20:05:08 <Luke-Jr> o
5702015-11-12T20:05:27 <phantomcircuit> morcos, a background thread is equally as bad as any other way of doing it in that regard
5712015-11-12T20:06:00 <morcos> 6357 does not destroy your cache, and is way less computational complexity
5722015-11-12T20:06:26 <gmaxwell> as far as this priority discussion, I think people are being far too dismissive about priority. In particular, this idea of fee competition vs dos attacks is a situation where the defender has no advantage on an attacker at all... they spend the same. Having a tie breaker that rewards coins that haven't recently moved is, I think, an elegent way to create a gap so that attackers don't merely nee
5732015-11-12T20:06:32 <gmaxwell> d to pay market rate but slightly above. And impact to miner income can be made negligible.
5742015-11-12T20:07:09 <morcos> gmaxwell: that argument doesn't make sense to me
5752015-11-12T20:07:24 <morcos> i think the HOV/fast lane arguement makes sense
5762015-11-12T20:07:32 <morcos> but to have priority boost fee doesn't make sense
5772015-11-12T20:08:04 <petertodd> gmaxwell: why do we assume the defender always has more old coins than the attacker?
5782015-11-12T20:08:07 <morcos> so it boosts it by delta. then attackers just have to pay market rate + delta, who cares, unless delta is large, in which case then you have prioirty txs crowding out fee paying txs and hurting miners profits
5792015-11-12T20:08:10 <gmaxwell> morcos: right now the attacker need only bid episilon more than market rate to replace the honest users at market rate in the mempool.
5802015-11-12T20:08:52 <gmaxwell> petertodd: it's a hurestic, and it maps to the actual attack pattern in that by their nature an attack requires making lots of transactions. Honest use may or may not have that property; but it remains a distinguisher, if not a super strong one.
5812015-11-12T20:08:59 <morcos> its impossible to pick the conversion so that it accomplishes exactly your goal.. unless you reserve only a certain amount of space for priority, which is what the existing code does
5822015-11-12T20:09:01 <petertodd> e.g. IIRC in lightning you can be in a situation where you need to get a recently mined tx spent, and the attacker may be trying to flood mempools to keep you from doing that
5832015-11-12T20:09:21 <morcos> the existing code makes sense for that purpose, i agree. but combining to just one bucket filled on a combined metric does not
5842015-11-12T20:09:34 <Luke-Jr> petertodd: I'm sure there are lightning-optimised policies that are possible?
5852015-11-12T20:09:35 <petertodd> morcos: and *that* can be easily done with two separate mempools, each separately limited, which keeps the codebase simple
5862015-11-12T20:09:54 <petertodd> Luke-Jr: better for privacy if you can't distingish lightning txs from othre txs
5872015-11-12T20:10:06 <Luke-Jr> â¦
5882015-11-12T20:10:12 <gmaxwell> morcos: Okay, you're right there. I was thinking that even having the attacker have to pay the extra delta is a benefit, but you're right that its probably negligible.
5892015-11-12T20:10:13 <Luke-Jr> but you can
5902015-11-12T20:11:13 <phantomcircuit> morcos, 6357 is less computationally complex, but significantly more complex code
5912015-11-12T20:11:19 <wumpus> and another: * [new tag] v0.10.4 -> v0.10.4
5922015-11-12T20:11:27 <morcos> phantomcircuit: ok that i will agree with!
5932015-11-12T20:11:34 <gmaxwell> The two pools approach can also resolve the repricing cost issue, I think, if transactions cannot move back from the feerate pool to the priority pool, and if the priority pool is small.
5942015-11-12T20:11:53 <morcos> sdaftuar's pull was 2 pools
5952015-11-12T20:12:04 <morcos> maintained in the same structure
5962015-11-12T20:12:11 <phantomcircuit> morcos, and it's a bunch of caching of values which is genuinely hard to get right
5972015-11-12T20:12:15 <gmaxwell> (as you could limit repricing to the priority pool if the above assumptions are kept)
5982015-11-12T20:13:04 <gmaxwell> maybe I just need to come around to bluematt's thinking that priority is not worth maintaining...
5992015-11-12T20:13:27 <dgenr8> different-sized mempools mean that attacker doesn't have a perfectly defined function to erase a tx with a given fee
6002015-11-12T20:13:35 <morcos> gmaxwell: i mostly agree, i wanted to keep priority, but it is a lot of complication, and maybe its better to realize that if we can come up with some incentive compatible metric like priority
6012015-11-12T20:13:41 <morcos> then we can just add that back in in a clean way
6022015-11-12T20:13:54 <morcos> we dont' have to permanently shut the door on a metric like this
6032015-11-12T20:14:10 <dgenr8> also, attacker has to erase 299M of transactions to affect the lowest-feerate tx that's going into the next block
6042015-11-12T20:14:26 <dgenr8> at default mempool setting
6052015-11-12T20:14:33 <sipa> dgenr8: highest
6062015-11-12T20:15:55 <dgenr8> 300M to affect highest, no?
6072015-11-12T20:16:06 <gmaxwell> morcos: yes, if block costing is changed, then it becomes incentive compatible and works better; though I am really really not fond of multi-metric block costs.
6082015-11-12T20:16:34 <morcos> but block costs are inherently multi-metric with sig-op limit
6092015-11-12T20:16:58 <morcos> you could buy extra block size with older age coins even
6102015-11-12T20:17:06 <phantomcircuit> morcos, the sigop/byte limit is so high that no legitimate user is going to hit it
6112015-11-12T20:17:10 <Luke-Jr> removing policy actually increases maintenance costs
6122015-11-12T20:17:15 <sipa> dgenr8: oh, i see what you're saying; in theory, yes, but in practice chains of transactions are harder to kick out and tend to linger, making it more spread out
6132015-11-12T20:17:15 <gmaxwell> morcos: yes, and I want that to go away (it's busted anyways. :) ) fortunately the sigop limit is high enough that under ordinary use it's never hit.
6142015-11-12T20:17:44 <phantomcircuit> you basically need to have a transaction that consists of OP_DUP OP_DUP OP_CHECKSIG repeated 100+ times to hit the limit
6152015-11-12T20:17:44 <morcos> gmaxwell: ha, but you're the one tryign to worry about hitting it now b/c of the flood
6162015-11-12T20:18:02 <morcos> if you dont' have multi metrics, then you have conversion factors between metrics which are hard to get right
6172015-11-12T20:18:11 <gmaxwell> yea I know, I know, short term effects because of people specifically trying to exploit it.
6182015-11-12T20:18:21 <gmaxwell> morcos: I don't know if for incentive reasons it matters if they're "right"
6192015-11-12T20:18:45 <gmaxwell> in that I think mostly the sign matters. "oh I should prefer to make transactions that do X because it'll cost less"
6202015-11-12T20:18:57 <gmaxwell> or at least only matters to within an order of magnitude.
6212015-11-12T20:18:58 <morcos> gmaxwell: fair enough, but i'd say heuristics for optimizing multi metrics are also good enough for mining
6222015-11-12T20:19:20 *** d_t has quit IRC
6232015-11-12T20:19:32 <gmaxwell> morcos: but will anyone ever implement them? Multiple limits also complicate the implementation of fraud proofs, making it less likely that we'll ever have them.
6242015-11-12T20:20:07 <morcos> sure i will! but fraud proofs, hmm... because you ahve to be able to prove any one of those things?
6252015-11-12T20:20:53 <morcos> i don't understand enough about how fraud proofs work
6262015-11-12T20:21:01 <gmaxwell> yes, and efficiently proving a global limit requires commiting to a tree over its computation.
6272015-11-12T20:22:30 * michagogo has his gitian-build script already running
6282015-11-12T20:23:01 <gmaxwell> E.g. you commit to the value of of each input and output, and hashtree it, and in the interior nodes include the sum fees of the children. Now you can make an efficient proof that a block creates inflation. This can be done for any limit, e.g. sigops, whatever, but the interior nodes in the hashtree have to track each limit.
6292015-11-12T20:24:41 *** d_t has joined #bitcoin-core-dev
6302015-11-12T20:24:42 <gmaxwell> morcos: I worry less that you won't implement it, and more that miners won't bother running code that almost never fires. "Morcos solved it" is a stable solution only if we assume people are running your code, which they might not for smarter or dumber reasons.
6312015-11-12T20:26:03 *** skyraider has joined #bitcoin-core-dev
6322015-11-12T20:26:11 <gmaxwell> morcos: also I think seperate limits are somewhat backwards in the sense that what we really seek to limit is "cost"; and turning cost into seperate limits is something we do because we don't know how to weigh for cost, but we do know what demand looks like, and demand uses a little of each dimension.
6332015-11-12T20:27:49 <gmaxwell> in any case the issue with fraud proofs is mostly just making things even more complex reduces the odds that they get correctly implemented. Mempool hurestics are one thing-- they're not consensus critical, but limits are.
6342015-11-12T20:27:51 <wangchun> Luke-Jr: We have priority size set to 0 for maybe one year. But I have blockminsize set to 250 KB.
6352015-11-12T20:28:04 <sipa> wangchun: good to know!
6362015-11-12T20:28:19 <michagogo> Also, I found out I'm a bit of an idiot. The issue I was having where as soon as I upgraded the LXC base, I had to keep it updated or the build would fail?
6372015-11-12T20:28:22 *** d4de has joined #bitcoin-core-dev
6382015-11-12T20:28:36 <Luke-Jr> wangchun: let's chat in #bitcoin since there is conversation here already
6392015-11-12T20:28:51 <michagogo> The fix is just to on-target -u root rm /var/cache/gitian/initial-upgrade
6402015-11-12T20:29:13 <michagogo> (on the target, before copying it back over the base)
6412015-11-12T20:29:29 <michagogo> (That's just a flag file that the upgrade script touches)
6422015-11-12T20:30:01 <kanzure> fraud proofs require each unique validation rule has separate fraud proof implementation to check https://bitcointalk.org/index.php?topic=1103281.msg11743498#msg11743498
6432015-11-12T20:31:15 <morcos> gmaxwell: ok, i'm not against weighted cost, i just worry about getting the weights right.. especially if those are consensus rules!
6442015-11-12T20:31:27 <jgarzik> giving #6771 (chain limits) one last test run
6452015-11-12T20:31:49 <gmaxwell> kanzure: that link is not correct in the details fwiw.
6462015-11-12T20:31:58 <michagogo> Hm, weird lines from configure:
6472015-11-12T20:31:59 <kanzure> quick overview of wrongness?
6482015-11-12T20:32:04 <gmaxwell> (like saying utxo commitments are required, they're not)
6492015-11-12T20:32:09 <michagogo> Checking for QT... no
6502015-11-12T20:32:10 <michagogo> Checking for QT... yes
6512015-11-12T20:32:12 <kanzure> okay, i'll keep that in mind.
6522015-11-12T20:32:19 <michagogo> One right after the other
6532015-11-12T20:32:37 <sipa> gmaxwell: until a few days ago i would have claimed that utxo commitments are required too :)
6542015-11-12T20:33:02 <gmaxwell> sipa: pshaw, I've known they were for like ... months now. You need to spend more time in my head. :)
6552015-11-12T20:33:26 <sipa> *walks back slowly*
6562015-11-12T20:33:45 <kanzure> why few days ago?
6572015-11-12T20:34:03 <sipa> kanzure: because that's when gmaxwell told me how to do it without commitments in a PM :)
6582015-11-12T20:34:09 <kanzure> PMs don't count
6592015-11-12T20:34:12 <kanzure> that's cheating
6602015-11-12T20:34:20 <gmaxwell> it's been discussed in #bitcoin-wizards.
6612015-11-12T20:34:23 <sipa> Oh.
6622015-11-12T20:34:26 <kanzure> ah okay
6632015-11-12T20:34:41 <kanzure> great
6642015-11-12T20:35:05 <gmaxwell> in any case, you do them without a utxo commitment by instead commiting to the input block and offset that the input came from.
6652015-11-12T20:35:35 <gmaxwell> then fraud can be proven by simply showing whatever is at that location, if its not the thing its supposted to be.
6662015-11-12T20:38:00 <gmaxwell> I mean, I'm probably at fault for making people think that its the only way to get a fraud proof for spending non-existing coins, since in the very first writeup I suggest that https://en.bitcoin.it/wiki/User:Gmaxwell/features#Proofs
6672015-11-12T20:38:18 <gmaxwell> but always things like that should be considered necessary, back then I didn't give it more than a moment's thought.
6682015-11-12T20:38:28 <gmaxwell> er considered sufficient not necessary.
6692015-11-12T20:40:56 <kanzure> ok, just posted follow-up to that thread linking to this explanation.
6702015-11-12T20:42:34 <gmaxwell> (and obviously, spends of already spent coins proven by showing the other spend)
6712015-11-12T20:48:56 <phantomcircuit> morcos, i dont really see the point of using a complex cost metric today as the principle limiting factor is bandwidth (and presumably will remain that way for many years)
6722015-11-12T20:49:50 <phantomcircuit> if i was building a cost function is would be weighted to size so heavily as to be redundant
6732015-11-12T20:50:53 <jgarzik> morcos, currently waiting on travis for #6771, then go for launch
6742015-11-12T20:51:13 <gmaxwell> phantomcircuit: uh, even with libsecp256k1 dsa signature validation run at something like only 9mbit/sec/core (in terms of transaction size).
6752015-11-12T20:51:19 <morcos> phantomcircuit: i thought you were arguing otherwise? but yes personally i think the biggest costs are bandwidth and utxo bloat. my idea was the metric should be something simple like .5 * total_bytes + 1 * output_bytes, but then maybe discounting provably unspendable output bytes
6762015-11-12T20:52:08 <morcos> jgarzik: did it have a problem or you just rerun it for completeness
6772015-11-12T20:52:53 <jgarzik> morcos, the latter - I always travis in jgarzik/bitcoin on the merged tree before pushing
6782015-11-12T20:53:01 <jgarzik> travis results get stale rapidly
6792015-11-12T20:54:14 <gmaxwell> phantomcircuit: and those figures are without multisig, it's worse with multisig. (like 2 of 3 is like 6mb/s/core)
6802015-11-12T20:55:38 <morcos> gmaxwell: but you're saying that number is too low?
6812015-11-12T20:56:31 <morcos> oh, i guess for a degenerate block that has all new sigs...
6822015-11-12T20:56:51 <morcos> so maybe there is a standard cost metric, and then separate limits like sig op is now for the degenerate cases
6832015-11-12T20:59:10 *** grieve has joined #bitcoin-core-dev
6842015-11-12T21:00:46 *** Guyver2 has quit IRC
6852015-11-12T21:26:36 <phantomcircuit> gmaxwell, hmm that's slower than i was thinking
6862015-11-12T21:26:51 <phantomcircuit> why was i thinking it was faster?
6872015-11-12T21:26:58 * phantomcircuit doesn't know
6882015-11-12T21:28:30 <gmaxwell> phantomcircuit: thats the only reason I mentioned it.
6892015-11-12T21:28:57 <gmaxwell> Not that I thought it was super important here, but just because you were clearly thinking it was faster than that; you're probably thinking in terms of IBD with signature checking disabled.
6902015-11-12T21:33:09 <phantomcircuit> gmaxwell, hmm maybe
6912015-11-12T21:33:21 <phantomcircuit> gmaxwell, i know that we cant do >50mbps for that
6922015-11-12T21:33:50 <phantomcircuit> possibly with some of the recent performance improvements we can improve on that significantly though
6932015-11-12T21:39:39 <michagogo> CXX test/test_test_bitcoin-Checkpoints_tests.o
6942015-11-12T21:39:46 <michagogo> That capital C seems out of place
6952015-11-12T21:40:15 <michagogo> So does _DoS just below
6962015-11-12T21:40:55 <michagogo> -DoS*
6972015-11-12T21:41:08 <michagogo> (Contrast to -pow which is kept lowercase)
6982015-11-12T21:41:56 <michagogo> Okay, 0.11.2 built, PR imminent
6992015-11-12T21:42:10 * michagogo starts the script to build 0.10.;
7002015-11-12T21:42:13 <michagogo> .4*
7012015-11-12T21:43:38 <GitHub107> [bitcoin] jgarzik pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/bd629d77edbe...38ed190eefcc
7022015-11-12T21:43:38 <GitHub107> bitcoin/master 971a4e6 Alex Morcos: Lower default policy limits...
7032015-11-12T21:43:39 <GitHub107> bitcoin/master 38ed190 Jeff Garzik: Merge #6771 from branch 'lowerLimits' of git://github.com/morcos/bitcoin
7042015-11-12T21:43:44 <GitHub188> [bitcoin] jgarzik closed pull request #6771: Policy: Lower default limits for tx chains (master...lowerLimits) https://github.com/bitcoin/bitcoin/pull/6771
7052015-11-12T21:43:47 <jgarzik> morcos, email req :)
7062015-11-12T21:45:03 <morcos> jgarzik: sent
7072015-11-12T21:45:53 <morcos> I think I have 2 backed up in moderation queue now
7082015-11-12T21:46:23 <michagogo> Hm, I wonder if my script will break if there's an outstanding PR
7092015-11-12T21:47:47 <michagogo> Actually, probably not. The Ruby oneliner that makes the PR is the last line in the script, so even if it fails and exits non-zero, the script is over anyway and there's nothing to abort
7102015-11-12T21:48:12 <petertodd> gmaxwell: "utxo commitment by instead commiting to the input block and offset" <- that falls into the category of !@#$ why didn't I think of that :)
7112015-11-12T21:49:42 <michagogo> (set -ex)
7122015-11-12T21:54:46 *** ParadoxSpiral has quit IRC
7132015-11-12T21:56:20 <CodeShark> petertodd: that was actually my stumbling block to not fully getting some of the potential advantages of MMR trees
7142015-11-12T21:57:27 *** treehug88 has quit IRC
7152015-11-12T21:57:58 *** jgarzik has quit IRC
7162015-11-12T21:58:16 <CodeShark> in retrospect, it makes a lot more sense to index outputs that way
7172015-11-12T21:59:43 <CodeShark> at least for existence/nonexistence proofs - obviously for spending we need to reference the transaction in some way
7182015-11-12T22:20:08 *** MarcoFalke has quit IRC
7192015-11-12T22:37:14 *** aburan28 has joined #bitcoin-core-dev
7202015-11-12T23:52:09 *** murr4y has quit IRC
7212015-11-12T23:59:15 *** murr4y has joined #bitcoin-core-dev