12016-12-09T00:00:20  *** koha has quit IRC
  22016-12-09T00:07:29  <CubicEarth> I had a repeat of my node stalling when trying to sync a couple of days ago. Recent version of Ubuntu, 13.1, and basically nothing else on the machine. It was about 6 days behind, and promptly caught up to about 40 hours remaining.  Then it just stopped. CPU was idle, only relaying traffic was seemingly being passed. After about 10 minutes I restarted the node, and syncing was completed within just a coup
  32016-12-09T00:07:29  <CubicEarth> le minutes of he restart.
  42016-12-09T00:09:01  <CubicEarth> I did try booting peers, but that didn't help anything.
  52016-12-09T00:09:01  <sipa> this is a known issue
  62016-12-09T00:09:16  <sipa> it is resolved automatically if you wait for the next block
  72016-12-09T00:09:37  <CubicEarth> I'm glad it's know :)
  82016-12-09T00:10:30  *** MarcoFalke has left #bitcoin-core-dev
  92016-12-09T00:11:36  *** jtimon has quit IRC
 102016-12-09T00:54:42  *** TomMc has quit IRC
 112016-12-09T01:09:27  *** TomMc has joined #bitcoin-core-dev
 122016-12-09T01:10:04  <bitcoin-git> [bitcoin] sipa opened pull request #9303: Update comments in ctaes (master...ctaes) https://github.com/bitcoin/bitcoin/pull/9303
 132016-12-09T01:17:50  <gmaxwell> CubicEarth: it's due to nodes (usually spy nodes) that break the protocol and don't respond to headers requests interacting poorly with the sync logic.
 142016-12-09T01:25:20  <CubicEarth> I was wondering...  Seems like my node ought to be a little bit more selfish, a little bit more aggressive, in requesting the data it needs. At least I wish it was. I'm guessing what you are describing is due to the fact Core codes nodes to be good network citizens, and non-standard nodes can disrupt that?
 152016-12-09T01:26:26  *** alpalp has joined #bitcoin-core-dev
 162016-12-09T01:29:48  <CubicEarth> Once payment channels and LN become a reality, I foresee the P2P layer integrating lots of micro-fees, charging for serving block data, for relaying TX, etc.
 172016-12-09T01:34:08  *** alpalp has quit IRC
 182016-12-09T01:36:13  *** alpalp has joined #bitcoin-core-dev
 192016-12-09T01:36:13  *** alpalp has joined #bitcoin-core-dev
 202016-12-09T01:36:38  <gmaxwell> CubicEarth: no, it's not due to that, pretty much the opposite.
 212016-12-09T01:37:05  <gmaxwell> if it requests redundant data, then it might knock itself off the network if otherwise it only has enough bandwidth available to only fetch one copy.
 222016-12-09T01:39:58  <CubicEarth> So my node requests a piece of data, what happens?  A peer says "yes, I have it, I'll give it to you" and then never does? And my node sites there, waiting, because if it asked another peer for data they could both end up providing it?
 232016-12-09T01:42:09  <gmaxwell> CubicEarth: right. It will eventually give up-- but for this particular request its triggered by a new block showing up on the network.
 242016-12-09T01:43:22  <gmaxwell> Smarter would be dynamic timeouts, -- the tricky thing is that care has to be taken to avoid unstable algorithins that can suffer congestion collapse. E.g. you have a limited bandwith network with 5 nodes on it... and then they fall behind and start agressively rerequesting and never recover.
 252016-12-09T01:43:40  <gmaxwell> It's not _that_ hard, but ... so many other things going on...
 262016-12-09T01:50:36  <CubicEarth> Other priorities for sure. It's nice that it respects bandwidth limits currently.  It makes sense for it to be conservative by default, but perhaps there should / could be a setting where you tell how much bandwidth you would like it make use of, not just an "upper limit" for inbound connections, but "please use this much to make things happen faster"
 272016-12-09T01:51:29  <CubicEarth> Onto the back burner...
 282016-12-09T01:52:44  <gmaxwell> well it's not that it respects, it just has no idea what they are, so it assumes it's operating with very little, since thats consderative.
 292016-12-09T01:53:45  <gmaxwell> RE manual setting, I think very few users will use that correctly--  we should have settings, but they're way less important than better default behavior.
 302016-12-09T01:57:45  *** abpa has quit IRC
 312016-12-09T02:00:00  <CubicEarth> The funny thing is the software is already 'aware' in the sense that it can generate a graph of network activity, but I get that it's probably not 'hardened code'.
 322016-12-09T02:01:57  <gmaxwell> Past performance doesn't indicate future results. Assuming that it does is how you get schemes that suffer congestion collapse. :P
 332016-12-09T02:03:02  <CubicEarth> gmaxwell: so a node dosent DDoS itself?
 342016-12-09T02:07:22  *** Ylbam has quit IRC
 352016-12-09T02:08:11  <gmaxwell> CubicEarth: e.g. you have 5 nodes on a 1 mbit connection. They each observe 1mbit available.. but then they all try to use it at once and there will be far less than 1mbit available.
 362016-12-09T02:09:11  <CubicEarth> got it
 372016-12-09T02:22:43  *** Giszmo has joined #bitcoin-core-dev
 382016-12-09T03:14:18  *** fengling has quit IRC
 392016-12-09T03:19:14  *** afk11 has quit IRC
 402016-12-09T03:19:29  *** afk11 has joined #bitcoin-core-dev
 412016-12-09T03:19:29  *** afk11 has quit IRC
 422016-12-09T03:19:29  *** afk11 has joined #bitcoin-core-dev
 432016-12-09T03:22:12  *** fengling has joined #bitcoin-core-dev
 442016-12-09T03:25:32  *** murch has joined #bitcoin-core-dev
 452016-12-09T03:26:14  *** murch_ has quit IRC
 462016-12-09T03:28:23  *** afk11 has quit IRC
 472016-12-09T03:31:22  *** abpa has joined #bitcoin-core-dev
 482016-12-09T03:33:31  *** afk11 has joined #bitcoin-core-dev
 492016-12-09T03:33:32  *** afk11 has quit IRC
 502016-12-09T03:33:32  *** afk11 has joined #bitcoin-core-dev
 512016-12-09T03:35:53  *** fengling has quit IRC
 522016-12-09T03:39:57  <bitcoin-git> [bitcoin] droark opened pull request #9304: Allow linearization scripts to support little endian hashes (master...master) https://github.com/bitcoin/bitcoin/pull/9304
 532016-12-09T03:47:11  *** Giszmo has quit IRC
 542016-12-09T04:05:41  *** fengling has joined #bitcoin-core-dev
 552016-12-09T04:22:18  <bitcoin-git> [bitcoin] kallewoof opened pull request #9305: Refactor: Removed begin/end_ptr functions. (master...remove-begin-end_ptr-usage) https://github.com/bitcoin/bitcoin/pull/9305
 562016-12-09T04:22:50  *** alpalp has quit IRC
 572016-12-09T04:24:39  *** alpalp has joined #bitcoin-core-dev
 582016-12-09T04:25:13  *** fengling has quit IRC
 592016-12-09T04:32:01  *** abpa has quit IRC
 602016-12-09T04:34:12  *** alpalp has quit IRC
 612016-12-09T04:37:56  *** afk11 has quit IRC
 622016-12-09T04:43:32  *** afk11 has joined #bitcoin-core-dev
 632016-12-09T04:43:32  *** afk11 has quit IRC
 642016-12-09T04:43:32  *** afk11 has joined #bitcoin-core-dev
 652016-12-09T04:53:42  *** TomMc has quit IRC
 662016-12-09T05:06:39  *** Squidicuz has quit IRC
 672016-12-09T05:08:59  *** Squidicuz has joined #bitcoin-core-dev
 682016-12-09T05:10:37  *** afk11 has quit IRC
 692016-12-09T05:14:17  *** afk11 has joined #bitcoin-core-dev
 702016-12-09T05:14:18  *** afk11 has quit IRC
 712016-12-09T05:14:18  *** afk11 has joined #bitcoin-core-dev
 722016-12-09T05:28:01  *** Alopex has quit IRC
 732016-12-09T05:29:06  *** Alopex has joined #bitcoin-core-dev
 742016-12-09T07:00:17  *** dermoth has quit IRC
 752016-12-09T07:00:58  *** dermoth has joined #bitcoin-core-dev
 762016-12-09T07:09:26  *** Alopex has quit IRC
 772016-12-09T07:10:31  *** Alopex has joined #bitcoin-core-dev
 782016-12-09T07:13:38  *** SKESAFIROS_ZEF2P has joined #bitcoin-core-dev
 792016-12-09T07:21:05  *** Murh has joined #bitcoin-core-dev
 802016-12-09T07:22:24  *** Murh has left #bitcoin-core-dev
 812016-12-09T07:23:15  *** SKESAFIROS_ZEF2P has quit IRC
 822016-12-09T07:24:13  *** MurhS0xFF has joined #bitcoin-core-dev
 832016-12-09T07:24:42  *** MurhS2x1 has joined #bitcoin-core-dev
 842016-12-09T07:25:12  *** MurhS0xFF has quit IRC
 852016-12-09T07:25:12  *** MurhS2x1 has quit IRC
 862016-12-09T07:31:03  *** fengling has joined #bitcoin-core-dev
 872016-12-09T07:59:08  *** BashCo has quit IRC
 882016-12-09T08:28:35  *** BashCo has joined #bitcoin-core-dev
 892016-12-09T08:29:20  *** BashCo_ has joined #bitcoin-core-dev
 902016-12-09T08:33:20  *** BashCo has quit IRC
 912016-12-09T08:41:48  *** JackH has joined #bitcoin-core-dev
 922016-12-09T09:09:15  *** Ylbam has joined #bitcoin-core-dev
 932016-12-09T09:14:13  *** laurentmt has joined #bitcoin-core-dev
 942016-12-09T09:15:00  *** laurentmt has quit IRC
 952016-12-09T09:18:16  *** jannes has joined #bitcoin-core-dev
 962016-12-09T09:18:41  *** MarcoFalke has joined #bitcoin-core-dev
 972016-12-09T09:23:21  <bitcoin-git> [bitcoin] MarcoFalke pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/86017842d6ef...72bf1b3d0962
 982016-12-09T09:23:22  <bitcoin-git> bitcoin/master 8501bed Pieter Wuille: Squashed 'src/crypto/ctaes/' changes from cd3c3ac..003a4ac...
 992016-12-09T09:23:23  <bitcoin-git> bitcoin/master 760765d Pieter Wuille: Update ctaes
1002016-12-09T09:23:23  <bitcoin-git> bitcoin/master 72bf1b3 MarcoFalke: Merge #9303: Update comments in ctaes...
1012016-12-09T09:23:41  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9303: Update comments in ctaes (master...ctaes) https://github.com/bitcoin/bitcoin/pull/9303
1022016-12-09T09:36:04  *** luke-jr has quit IRC
1032016-12-09T09:37:55  *** AaronvanW has quit IRC
1042016-12-09T09:39:18  *** AaronvanW has joined #bitcoin-core-dev
1052016-12-09T09:46:44  *** LeMiner2 has joined #bitcoin-core-dev
1062016-12-09T09:46:44  *** LeMiner has quit IRC
1072016-12-09T09:46:44  *** LeMiner2 is now known as LeMiner
1082016-12-09T09:53:02  *** Guyver2 has joined #bitcoin-core-dev
1092016-12-09T10:01:06  *** MarcoFalke has left #bitcoin-core-dev
1102016-12-09T10:08:46  *** chris2000 has joined #bitcoin-core-dev
1112016-12-09T10:17:32  *** harrymm has quit IRC
1122016-12-09T10:20:48  *** harrymm has joined #bitcoin-core-dev
1132016-12-09T10:30:41  *** VeryWellAged has joined #bitcoin-core-dev
1142016-12-09T11:04:58  *** MurhS0xFF has joined #bitcoin-core-dev
1152016-12-09T11:17:57  *** MurhS0xFF has quit IRC
1162016-12-09T12:34:02  *** Giszmo has joined #bitcoin-core-dev
1172016-12-09T12:37:51  *** LeMiner2 has joined #bitcoin-core-dev
1182016-12-09T12:40:19  *** LeMiner has quit IRC
1192016-12-09T12:40:20  *** LeMiner2 is now known as LeMiner
1202016-12-09T12:50:10  *** jtimon has joined #bitcoin-core-dev
1212016-12-09T12:59:09  *** jtimon has quit IRC
1222016-12-09T13:03:31  *** michagogo has quit IRC
1232016-12-09T13:04:17  *** michagogo has joined #bitcoin-core-dev
1242016-12-09T13:13:23  *** jtimon has joined #bitcoin-core-dev
1252016-12-09T13:23:23  <BlueMatt> hmmmmmm....I'm pretty sure CheckForkWarningConditionsOnNewFork is completely useless atm...
1262016-12-09T13:24:12  <BlueMatt> it looks like it was written assuming pindexNewForkTip would be a CBlockIndex* to the highest block on a new fork (which I think is the case in the original code, and I'm not just saying it because I originally wrote it)
1272016-12-09T13:24:43  <BlueMatt> but in the current code its given the last block which ActivateBestChainStep wanted to connect (but failed because either it, or a previous block) was invalid
1282016-12-09T13:25:30  <BlueMatt> and that last block is always either current chain + 1 unless its a reorg
1292016-12-09T13:37:16  <BlueMatt> ehh, excuse me...CheckForkWarningConditionsOnNewFork is called with a larger vector which isnt exactly the things ABCS tries, but what it might've tried if it didnt want to give up cs_main earlier
1302016-12-09T13:37:31  <BlueMatt> still, i think due to headers first this stuff is horribly broken
1312016-12-09T13:42:15  *** laurentmt has joined #bitcoin-core-dev
1322016-12-09T13:43:39  *** laurentmt has quit IRC
1332016-12-09T14:06:47  *** CubicEarth has quit IRC
1342016-12-09T14:07:31  *** Chris_Stewart_5 has quit IRC
1352016-12-09T14:16:13  *** cryptapus has joined #bitcoin-core-dev
1362016-12-09T14:18:19  *** TomMc has joined #bitcoin-core-dev
1372016-12-09T14:29:31  *** Chris_Stewart_5 has joined #bitcoin-core-dev
1382016-12-09T14:37:03  <gmaxwell> nah it does work.
1392016-12-09T14:42:27  <BlueMatt> you sure? that code doesnt look sane to me now
1402016-12-09T14:42:42  <BlueMatt> (do we have any tests for it? couldnt find any)
1412016-12-09T14:43:01  <BlueMatt> and jl2012 was saying he tried to trigger it by sending invalid blocks with valid headers and couldnt
1422016-12-09T14:52:03  * BlueMatt -> out
1432016-12-09T15:07:15  *** CubicEarth has joined #bitcoin-core-dev
1442016-12-09T15:10:15  <bitcoin-git> [bitcoin] ryanofsky opened pull request #9306: Make CCoinsViewCache::Cursor() return latest data (master...pr/coins-cursor) https://github.com/bitcoin/bitcoin/pull/9306
1452016-12-09T15:12:23  *** CubicEarth has quit IRC
1462016-12-09T15:33:48  <bitcoin-git> [bitcoin] ryanofsky opened pull request #9307: [trivial] Remove undefined FetchCoins method declaration (master...pr/coins-delfetch) https://github.com/bitcoin/bitcoin/pull/9307
1472016-12-09T15:36:19  <bitcoin-git> [bitcoin] ryanofsky opened pull request #9308: [test] Add CCoinsViewCache Access/Modify/Write tests (master...pr/coins-test) https://github.com/bitcoin/bitcoin/pull/9308
1482016-12-09T15:37:42  *** aalex has joined #bitcoin-core-dev
1492016-12-09T15:52:45  *** Samdney has joined #bitcoin-core-dev
1502016-12-09T15:54:27  *** Teroxice has joined #bitcoin-core-dev
1512016-12-09T15:54:37  <Teroxice> Hello
1522016-12-09T15:54:52  <Teroxice> I build an ATM software and right now its sending money from a centralized bitcore server with just one wallet.  I will offer my software to the market but I would like that every client has their own independent wallet in my centralized bitcore server.  That is why I'm asking if it is posible to have more than one wallet on the same bitcore server.  Anyone knows?
1532016-12-09T15:55:19  <achow101> Teroxice: bitcore or bitcoin core? There is a difference.
1542016-12-09T15:55:36  <achow101> also, wrong channel
1552016-12-09T15:55:57  <Teroxice> bitcoin core
1562016-12-09T15:56:20  <instagibbs> Teroxice, try #bitcoin
1572016-12-09T15:56:20  <achow101> bitcoin core does not have multiwallet support
1582016-12-09T16:00:53  *** laurentmt has joined #bitcoin-core-dev
1592016-12-09T16:01:05  *** laurentmt has quit IRC
1602016-12-09T16:02:44  *** aalex__ has joined #bitcoin-core-dev
1612016-12-09T16:06:30  *** aalex has quit IRC
1622016-12-09T16:06:38  *** aalex_ has quit IRC
1632016-12-09T16:06:57  *** aalex has joined #bitcoin-core-dev
1642016-12-09T16:07:24  *** aalex__ has quit IRC
1652016-12-09T16:07:42  *** aalex__ has joined #bitcoin-core-dev
1662016-12-09T16:10:59  *** aalex has quit IRC
1672016-12-09T16:11:10  *** aalex has joined #bitcoin-core-dev
1682016-12-09T16:23:39  *** Teroxice has left #bitcoin-core-dev
1692016-12-09T16:42:30  *** BashCo_ has quit IRC
1702016-12-09T16:43:07  *** BashCo has joined #bitcoin-core-dev
1712016-12-09T16:43:27  *** aalex__ has quit IRC
1722016-12-09T16:44:24  *** Samdney has quit IRC
1732016-12-09T16:46:58  <bitcoin-git> [bitcoin] morcos opened pull request #9309: Spurious RPC test failure (master...rarerpcfail) https://github.com/bitcoin/bitcoin/pull/9309
1742016-12-09T16:48:04  *** BashCo has quit IRC
1752016-12-09T16:53:41  *** aalex_ has joined #bitcoin-core-dev
1762016-12-09T16:57:30  *** aalex has quit IRC
1772016-12-09T17:04:46  *** BashCo has joined #bitcoin-core-dev
1782016-12-09T17:10:38  *** CubicEarth has joined #bitcoin-core-dev
1792016-12-09T17:15:38  *** abpa has joined #bitcoin-core-dev
1802016-12-09T17:27:33  *** TomMc has quit IRC
1812016-12-09T17:32:07  *** Samdney has joined #bitcoin-core-dev
1822016-12-09T17:40:03  *** TomMc has joined #bitcoin-core-dev
1832016-12-09T18:09:28  <bitcoin-git> [bitcoin] ryanofsky opened pull request #9310: Assert FRESH validity in CCoinsViewCache::BatchWrite (on top of #9308) (master...pr/coins-batch-assert) https://github.com/bitcoin/bitcoin/pull/9310
1842016-12-09T18:24:54  *** laurentmt has joined #bitcoin-core-dev
1852016-12-09T18:25:40  *** laurentmt has quit IRC
1862016-12-09T18:32:06  *** TomMc has quit IRC
1872016-12-09T18:40:24  *** jtimon has quit IRC
1882016-12-09T18:44:25  *** TomMc has joined #bitcoin-core-dev
1892016-12-09T18:46:01  *** dermoth has quit IRC
1902016-12-09T18:47:13  *** dermoth has joined #bitcoin-core-dev
1912016-12-09T18:53:47  <bitcoin-git> [bitcoin] morcos opened pull request #9311: Flush wallet after abandontransaction (master...flushwalletonabandon) https://github.com/bitcoin/bitcoin/pull/9311
1922016-12-09T19:03:42  <morcos> gmaxwell: maybe easier to explain what's happening in #9167 here
1932016-12-09T19:03:44  <gribble> https://github.com/bitcoin/bitcoin/issues/9167 | IsAllFromMe by morcos · Pull Request #9167 · bitcoin/bitcoin · GitHub
1942016-12-09T19:04:09  <morcos> if you ran that same command on the old code your grep would find fee twice
1952016-12-09T19:04:34  <morcos> it prints an overall fee for the transaction and it prints a fee for each accounting entry
1962016-12-09T19:05:23  <morcos> it was already the case that if it didn't think the tx was from you (meaning none of it was from you) it would leave off the overall fee for the tx
1972016-12-09T19:05:50  <gmaxwell> ah, I see. what the heck is the fee for the accounting entries for?
1982016-12-09T19:05:52  <morcos> i changed that logic to be whether all of it was from you
1992016-12-09T19:06:01  <morcos> don't get me started on that
2002016-12-09T19:06:32  <morcos> getbalance("*") uses those random incorrect negative fees to offset other errors in tracking balances
2012016-12-09T19:06:38  <morcos> so it was not possible to fix those
2022016-12-09T19:06:58  <morcos> i suppose we could not print them, but that seems like an api change
2032016-12-09T19:07:34  <gmaxwell> We should probably be telling the wallet about the actual values of all the inputs... we know them (we're a full node, after all!). then the fees it displays can be correct.
2042016-12-09T19:08:01  <gmaxwell> but thats a bigger change.
2052016-12-09T19:08:17  <morcos> yeah i don't think we should try to make any changes of that behavior until we remove accounts
2062016-12-09T19:08:21  <morcos> then we can clean up a lot
2072016-12-09T19:08:55  <morcos> which reminds me i need to be participating in wumpus's labels discussion
2082016-12-09T19:09:02  <sdaftuar> gmaxwell: +1 on telling the wallet!
2092016-12-09T19:09:08  <sdaftuar> er, about al the fees
2102016-12-09T19:09:12  <sdaftuar> i think it's nuts the way that works now
2112016-12-09T19:09:50  <gmaxwell> we've let the accounts stuff deadlock us for a long time, I believe that this was also the reason we didn't fix the absurd handling wrt fees when it was first noticed. :(
2122016-12-09T19:10:31  <sdaftuar> i think my proposal would just be to pass fee information through SyncTransaction().  we have it during ATMP, and we could cache it while validating a block
2132016-12-09T19:11:37  <sdaftuar> that doesn't go as far as all input values, but i think it'd be a simple improvement
2142016-12-09T19:12:36  <gmaxwell> it would be.  full input values are needed if we want to create detailed corrective accounting entries for txn with inputs which aren't fromme.
2152016-12-09T19:13:32  <gmaxwell> e.g. txid 1234  spend 1000 of our coins, spent 4001 coins from {these sources}, paid 5000 coins, and 1 coin fee.
2162016-12-09T19:13:52  <gmaxwell> if {these sources} is just "from outside this wallet" then we don't need per input amounts.
2172016-12-09T19:14:03  <gmaxwell> we just need the fee.
2182016-12-09T19:17:23  <sdaftuar> well i definitely agree that with the goal of per-input amounts being passed through!  maybe that's not so hard to implement either, actually...
2192016-12-09T19:18:11  <sdaftuar> we can probably come up with some reasonable data structure and pass that through to the wallet as well
2202016-12-09T19:19:07  <gmaxwell> sipa: do we have a philosophical opposition to decoderawtransaction using the UTXO set, telling you if each of the inputs is unspent,  and if they all are-- displaying the fee?
2212016-12-09T19:19:55  <sipa> gmaxwell: i think that should be a separate rpc call
2222016-12-09T19:20:16  <gmaxwell> sipa: what would it be called?
2232016-12-09T19:20:18  <sipa> gmaxwell: decoderawtransaction is purely a utility function now, and i think it should stay that way
2242016-12-09T19:20:26  <sipa> analyserawtransaction ?
2252016-12-09T19:20:42  <gmaxwell> please no more words with different en_gb/en_us spelling!
2262016-12-09T19:20:42  <gmaxwell> :P
2272016-12-09T19:21:17  <sipa> rawtransactionanalysis
2282016-12-09T19:21:19  <sipa> :p
2292016-12-09T19:21:33  <sdaftuar> could we add memory-only per-input values to CTransaction(), so that they get filled in and passed through to the wallet in SyncTransaction()?
2302016-12-09T19:22:05  <sipa> evaluaterawtransaction
2312016-12-09T19:22:48  <sipa> sdaftuar: bleh... what if they aren't known? the consensus logic (which uses CTransaction) shouldn't need such values
2322016-12-09T19:23:05  <sdaftuar> right, consensus wouldn't use it.  but it would be convenient to fill in for downstream consumers
2332016-12-09T19:23:21  <gmaxwell> well consensus certantly does eventually need to know the input values! :)
2342016-12-09T19:23:25  <sdaftuar> we could do outside of CTransaction() of course
2352016-12-09T19:23:37  <sipa> gmaxwell: but CTransaction is by design now immutable
2362016-12-09T19:23:42  <sdaftuar> the witness is not?
2372016-12-09T19:23:56  <sipa> sdaftuar: it will be
2382016-12-09T19:24:03  <sdaftuar> ah! didn't realize that.
2392016-12-09T19:24:11  <sipa> (also, CTransactionRef is a ref to a _const_ CTransaction, which includes the witness)
2402016-12-09T19:24:44  <sdaftuar> ok so i guess stuffing data into that just won't work
2412016-12-09T19:24:50  <sipa> but we could have a wrapper around CTransaction that adds some metadata, which is used by ATMP and wallet code
2422016-12-09T19:25:03  <sipa> or just pass along a separate object that contains that metadata
2432016-12-09T19:25:21  <morcos> but speakign of that idea... lets imagine inputs were part of CTxMempoolEntry, then maybe you don't need a UTXO cache anymore
2442016-12-09T19:25:46  <sipa> morcos: though it makes the mempool's correctness now consensus critical
2452016-12-09T19:25:55  <sdaftuar> nack
2462016-12-09T19:25:58  <morcos> thats what we're arguign about
2472016-12-09T19:26:02  *** molz is now known as moli
2482016-12-09T19:26:37  <gmaxwell> eventually the fast that txn are already verified in the mempool will have to be exploited for performance reasons.  :(
2492016-12-09T19:27:23  *** Lightsword has quit IRC
2502016-12-09T19:27:28  <sdaftuar> i would like that to be the last change that goes in before i stop working on the codebase :)
2512016-12-09T19:27:37  <gmaxwell> hah. yea. :(
2522016-12-09T19:27:44  <gmaxwell> or we'll just end up with miners using Joe-Marketers-Recklessly-Optimized-Fork that "validates blocks 5x faster"...
2532016-12-09T19:27:48  *** Lightsword has joined #bitcoin-core-dev
2542016-12-09T19:28:08  <gmaxwell> and then all the care in not being reckless didn't matter because relevant parties aren't using it.
2552016-12-09T19:31:10  <sipa> morcos: even if we had that, we'd need to apply utxo changes to the chainstate... which is perhaps harder if we haven't previously looked up the entry (because it's missing from intermediate cache layers, we don't know if it's fresh...)
2562016-12-09T19:33:06  <sipa> i'd prefer something weak-block based to have pre-evaluated sets of transactions to apply to the chainstate
2572016-12-09T19:34:50  <sdaftuar> sipa: so assuming the set that gets mined is identical to something you were expecting, you can have very fast validation?
2582016-12-09T19:35:00  <sipa> sdaftuar: yup
2592016-12-09T19:35:21  <sipa> you can basically have the utxo set diff cached
2602016-12-09T19:36:16  <gmaxwell> (not just that but you can have the block template for the next block you'd mine after it cached)
2612016-12-09T19:38:45  *** laurentmt has joined #bitcoin-core-dev
2622016-12-09T19:40:38  *** TomMc has quit IRC
2632016-12-09T19:48:51  <bitcoin-git> [bitcoin] morcos opened pull request #9312: Increase mempool expiry time to 2 weeks (master...longerexpiry) https://github.com/bitcoin/bitcoin/pull/9312
2642016-12-09T19:49:18  *** laurentmt has quit IRC
2652016-12-09T19:54:20  *** luke-jr has joined #bitcoin-core-dev
2662016-12-09T19:57:20  *** TomMc has joined #bitcoin-core-dev
2672016-12-09T20:23:28  *** Guyver2 has quit IRC
2682016-12-09T20:34:40  <gmaxwell> #9295 is an obvious simply bugfix that really would like to be merged (and backported)
2692016-12-09T20:34:41  <gribble> https://github.com/bitcoin/bitcoin/issues/9295 | An error has occurred and has been logged. Please contact this bot's administrator for more information.
2702016-12-09T20:44:16  <morcos> gmaxwell: the whole requesting parents of orphans functionality...  i didn't realize it basically doesn't work when your peer is a core node.  were you aware of that?
2712016-12-09T20:44:44  <morcos> b/c you won't let a peer getdata a tx that's not in your relay map
2722016-12-09T20:44:50  <gmaxwell> morcos: it works so long as the parents are still in the relay pool, or if they're an older version.
2732016-12-09T20:45:02  <gmaxwell> (which will answer out of the mempool)
2742016-12-09T20:45:14  <gmaxwell> Yes, I knew that when I did it.
2752016-12-09T20:45:17  *** aalex__ has joined #bitcoin-core-dev
2762016-12-09T20:45:37  <gmaxwell> In particular it's helpful for older versions that still do the trickling.. they're the source of most orphans I see, and they also answer out of the mempool.
2772016-12-09T20:45:54  <morcos> ok.. i just noticed it on a new node i started up
2782016-12-09T20:46:37  <gmaxwell> I'm glad someone else noticed.
2792016-12-09T20:47:34  <morcos> i'm about to PR a small fix to fee filter your minrelaytxfee if your limitfreerelay is 0..  will save some unnecessary free tx requesting/rejecting
2802016-12-09T20:47:53  <gmaxwell> thank god.
2812016-12-09T20:48:52  <gmaxwell> Re the minrelayfee decay,  did much thought or testing go into it?  ISTM it looks like it goes too low. e.g. continues dropping at a relatively high speed even once its past a level where your mempool will fill again, given enough time.
2822016-12-09T20:49:48  *** aalex_ has quit IRC
2832016-12-09T20:52:02  <bitcoin-git> [bitcoin] morcos opened pull request #9313: If we don't allow free txs, always send a fee filter (master...minminfee) https://github.com/bitcoin/bitcoin/pull/9313
2842016-12-09T20:53:14  <morcos> gmaxwell: much thought went into it, there is a bit of a discussion on #6722 on my reasoning behind the half-life of 12 hours...   But its certainly possible that the tradeoff has changed a bit.
2852016-12-09T20:53:17  <gribble> https://github.com/bitcoin/bitcoin/issues/6722 | Limit mempool by throwing away the cheapest txn and setting min relay fee to it by TheBlueMatt · Pull Request #6722 · bitcoin/bitcoin · GitHub
2862016-12-09T20:54:21  <morcos> The idea behind the min fee was to protect the limited resource of your memory, so it wasn't meant to be smart enough to know that certain fees are really never going to be worth it..
2872016-12-09T20:57:24  <morcos> Actually rereading the justification now, I think if anything we'd move it the other way..   Packages are limited to much smaller than the 2.5MB envisioned in that argument...  So the amount of "free relay" is really small.
2882016-12-09T20:57:56  <gmaxwell> morcos: I guess what trips it up is that it decays at a constant rate but the supply of transactions is not uniform (or even 1/rate).
2892016-12-09T20:58:22  <morcos> what exactly is the behavior you are seeing that you think is not good
2902016-12-09T20:59:56  <gmaxwell> that it drops it back down to nothing and then one of these clowns that relays old transactions connects, fills me back to 300mb... then 72 hours later, these expire, and it slides back down....
2912016-12-09T21:00:57  <gmaxwell> At least in my mind what it should be trying to do is find a value that results in the mempool being close to full-- but it ends up far lower than that; maybe I'm thinking of the wrong goal.
2922016-12-09T21:01:08  <morcos> yeah but even if it didn't go down at all, he could do the exact same thing at 2 sat/byte.  They still wouldn't be mined within 82 hours
2932016-12-09T21:01:14  <morcos> 72
2942016-12-09T21:01:49  <sdaftuar> i think modeling the arrival rate of transactions is hard
2952016-12-09T21:02:47  <gmaxwell> I think what I'm talking about is as much a question of the distribution of feerates in the supply of unconfirmed transactions, as it is arrival. (more so, because at the 2sat byte level, they're never getting mined.)
2962016-12-09T21:03:03  <morcos> i think if you look at tx supply..  there is a backlog of between 1MB - 100MB of txs that pay > 1 sat/byte   and there is a backlog of 100-1000MB additional of txs that pay 1 sat/byte   it just happens to be the distribution of txs now i think
2972016-12-09T21:03:15  <morcos> gmaxwell: ah, but thats wrong
2982016-12-09T21:04:22  <morcos> for txs that pay between 1.5-2 sat/byte  95% of them are mined within 1000 blocks and 75% of them are mined within 256 blocks
2992016-12-09T21:04:24  <morcos> crazy right
3002016-12-09T21:05:05  <gmaxwell> bleh. okay. Crazy.
3012016-12-09T21:06:47  *** aalex_ has joined #bitcoin-core-dev
3022016-12-09T21:06:55  <morcos> it's just there are sooo many in the 1-1.5 range that only about 10% of them get mined within 1000 blocks
3032016-12-09T21:10:51  *** aalex__ has quit IRC
3042016-12-09T21:30:04  *** aalex__ has joined #bitcoin-core-dev
3052016-12-09T21:33:23  *** aalex_ has quit IRC
3062016-12-09T21:56:51  <luke-jr> gmaxwell: hmm, I wonder if we should be rescanning for conflicts then (#9290)
3072016-12-09T21:56:53  <gribble> https://github.com/bitcoin/bitcoin/issues/9290 | Make RelayWalletTransaction attempt to AcceptToMemoryPool. by gmaxwell · Pull Request #9290 · bitcoin/bitcoin · GitHub
3082016-12-09T21:57:15  <gmaxwell> luke-jr: make rescan take less than N hours? :-/
3092016-12-09T21:57:26  <luke-jr> hours now? :o
3102016-12-09T21:57:40  <gmaxwell> I think it takes 5 on my laptop.
3112016-12-09T21:57:54  <gmaxwell> on a really fast machine it's not so terrible.
3122016-12-09T21:58:10  <luke-jr> I guess ideally we should be waiting until the blockchain is synced, checking if we have unconfirmed txns, checking a wallet flag, and rescanning at runtime
3132016-12-09T21:58:18  <luke-jr> sounds complex though :/
3142016-12-09T21:59:06  <luke-jr> (oh, and then we could simply not broadcast unconfirmed txns until it finishes the rescan)
3152016-12-09T21:59:13  <gmaxwell> the 'getting real fee info' will be another reason to rescan the chain for all wallets.
3162016-12-09T21:59:48  <gmaxwell> so it would be worth giving some thought to adding an extra kinda of versioning to the wallet (metadata-rescanned-since).. and making rescan faster...
3172016-12-09T21:59:57  <luke-jr> yeah
3182016-12-09T22:00:21  <luke-jr> "rescan depth" or something
3192016-12-09T22:00:25  <sdaftuar> luke-jr: it's in general not really possible to always know that a transaction is conflicted.  i think we should keep that in mind before doing anything expensive...
3202016-12-09T22:00:28  <gmaxwell> well they won't broadcast if conflicted-- they'll fail for mempool add. so the only harm is wasting a little time trying to look up utxos that aren't there.
3212016-12-09T22:00:55  <luke-jr> to be happy with downgrading+upgrading, we'd need a timestamp per each depth, but that seems unnecessarily over-compatible
3222016-12-09T22:01:03  <dcousens> gmaxwell: rescan is for private keys no?
3232016-12-09T22:01:13  <dcousens> or is it UTXO?
3242016-12-09T22:01:14  <luke-jr> gmaxwell: oh, right. that's not too bad compared to hours rescanning
3252016-12-09T22:01:55  <luke-jr> sdaftuar: ah, parent conflicts
3262016-12-09T22:02:24  <gmaxwell> Yes, though we won't spend non-wallet inputs (ignoring coinjoins because stupid) until they are six confirms old.
3272016-12-09T22:02:40  <gmaxwell> So it's really hard to end up in a case where there is an undetectable conflict in your wallet.
3282016-12-09T22:02:52  <luke-jr> gmaxwell: if someone else sends you a payment using coins later conflicted?
3292016-12-09T22:03:07  <dcousens> why is rescan so slow? is it because it tries to match all possible scripts or?
3302016-12-09T22:03:22  <gmaxwell> luke-jr: you won't spend that payment until it is 6 confirmed.
3312016-12-09T22:03:34  <gmaxwell> dcousens: I think much (most?) of the time is spent hashing the blocks.
3322016-12-09T22:04:12  <luke-jr> gmaxwell: but you'll try to mempool-add the receive, no?
3332016-12-09T22:04:16  <gmaxwell> luke-jr: okay, sure nevermind me.. that payment itself will be an undetectable conflict, I was only thinking about IsFromMe transactions.
3342016-12-09T22:04:55  <dcousens> gmaxwell: why is it hashing blocks? (i'll look into the code ooi)
3352016-12-09T22:05:09  *** Giszmo has quit IRC
3362016-12-09T22:05:19  <gmaxwell> dcousens: side-effect of deseralization.
3372016-12-09T22:06:56  *** Giszmo has joined #bitcoin-core-dev
3382016-12-09T22:07:31  <dcousens> gmaxwell: still weir,t I use similar enough deserialization code (lib/consensus) in my own parser and it does a full parse purely bottlenecked at IO, so 3-4 minutes on an SSD, script checking shouldn't be much more on that i'd of though...
3392016-12-09T22:07:41  <dcousens> but I guess I'd have to look into the code to find out why
3402016-12-09T22:09:04  <gmaxwell> At least my vague recollection of profiling it before was that the time was all in the heap allocator and sha256.
3412016-12-09T22:12:46  <dcousens> gmaxwell: hmmm, by hashing do you mean the merkle tree generation?
3422016-12-09T22:13:02  <dcousens> well, merkle root calculation***
3432016-12-09T22:13:19  <gmaxwell> I assume, I don't know what else sha256 would be used for while scanning blocks.
3442016-12-09T22:14:42  <sipa> the checksum?
3452016-12-09T22:15:02  <sipa> blocks on disk have a checksum, i think?
3462016-12-09T22:15:17  <sipa> or am i confusing it with peers.day
3472016-12-09T22:15:25  <dcousens> sipa: they have a magic byte
3482016-12-09T22:15:30  <dcousens> no checksum afaik
3492016-12-09T22:15:50  <gmaxwell> fun random graph https://people.xiph.org/~greg/temp/blk-sizes-windowed.png
3502016-12-09T22:16:14  <dcousens> a checksum would probably work wonders in bypassing that
3512016-12-09T22:16:34  <dcousens> esp. given the situation of all the zero padding in the files
3522016-12-09T22:41:21  *** CubicEarth has quit IRC
3532016-12-09T22:41:54  *** CubicEarth has joined #bitcoin-core-dev
3542016-12-09T22:43:05  *** aalex_ has joined #bitcoin-core-dev
3552016-12-09T22:43:50  *** Samdney has quit IRC
3562016-12-09T22:44:10  *** abpa has quit IRC
3572016-12-09T22:44:35  *** MykelSIlver has joined #bitcoin-core-dev
3582016-12-09T22:46:43  *** aalex__ has quit IRC
3592016-12-09T22:53:14  *** abpa has joined #bitcoin-core-dev
3602016-12-09T22:58:55  *** btcdrak has quit IRC
3612016-12-09T23:00:30  *** AaronvanW has quit IRC
3622016-12-09T23:06:57  *** AaronvanW has joined #bitcoin-core-dev
3632016-12-09T23:06:57  *** AaronvanW has joined #bitcoin-core-dev
3642016-12-09T23:11:12  <dcousens> RE: 9312, 2 week expiration time,  won't that further prevent parties from attempting broadcast a conflict of a stuck transaction 2.1 days after the first broadcast, and having a high chance of the other transaction being "mostly" out of the network by then?
3652016-12-09T23:11:17  <dcousens> of course, that is what RBF is for, but
3662016-12-09T23:13:37  <gmaxwell> dcousens: not really. (1) right now those conflicts are pretty successful immediately; due to nodes restarting, full-rbf miners, etc.  (2) there are people who go around connecting to everyone constantly rebroadcasting old transactions... so they're already defeating that timeout.
3672016-12-09T23:13:55  <dcousens> gmaxwell: in practice it works though?
3682016-12-09T23:14:10  <dcousens> oh wait, misread what you wrote
3692016-12-09T23:14:25  <gmaxwell> in practice it works even in 1 hour, so it works-- but not because of the 72 hour timeout.
3702016-12-09T23:14:31  <dcousens> yeah
3712016-12-09T23:16:44  <gmaxwell> we could also consider adding a new datastructure, a blacklist: if something hits the expire time, that txid becomes blacklisted for n-months.  That would actually make things much easier to replace with a double spend after two weeks.
3722016-12-09T23:17:34  <dcousens> gmaxwell: is there a way to "push out" a transaction from the mempool using the RPC? aka, "oops, forget the last one, use this instead"
3732016-12-09T23:17:50  <dcousens> aka, ignore conflicts, or drop conflicts
3742016-12-09T23:18:13  <dcousens> just thinking about you could by-pass that timeout without wiping your local mempool
3752016-12-09T23:18:35  <dcousens> obviously that wouldn't help others
3762016-12-09T23:18:49  <dcousens> but, as you say, others are quite the dynamic
3772016-12-09T23:22:24  <gmaxwell> dcousens: kinda useless to do something locally, it won't do it to anyone else...
3782016-12-09T23:25:13  <dcousens> gmaxwell: depending on their expiration timeout, mempool size & fee filter, full-RBF, etc, could go either way no?
3792016-12-09T23:26:34  <gmaxwell> dcousens: you can send things that aren't in your mempool, the whitelisting stuff does that already.
3802016-12-09T23:27:20  <dcousens> gmaxwell: true
3812016-12-09T23:35:49  *** CubicEarth has quit IRC
3822016-12-09T23:54:46  *** cryptapus is now known as cryptapus_afk