12018-10-25T00:00:31 *** sipa has quit IRC
22018-10-25T00:00:56 *** sipa has joined #bitcoin-core-dev
32018-10-25T00:16:25 *** proletesseract has quit IRC
42018-10-25T00:16:50 *** bralyclo_ has joined #bitcoin-core-dev
52018-10-25T00:17:07 *** proletesseract has joined #bitcoin-core-dev
62018-10-25T00:17:25 *** proletesseract has quit IRC
72018-10-25T00:17:57 *** grubles has joined #bitcoin-core-dev
82018-10-25T00:18:42 *** bralycl__ has joined #bitcoin-core-dev
92018-10-25T00:19:33 *** bralycl__ has quit IRC
102018-10-25T00:19:47 *** bralyclow has quit IRC
112018-10-25T00:20:50 *** bralyclow has joined #bitcoin-core-dev
122018-10-25T00:22:03 *** bralyclo_ has quit IRC
132018-10-25T00:22:14 *** bralyclo_ has joined #bitcoin-core-dev
142018-10-25T00:25:33 *** bralyclow has quit IRC
152018-10-25T00:29:08 *** Tralfaz has quit IRC
162018-10-25T00:34:49 *** Mutter has joined #bitcoin-core-dev
172018-10-25T00:37:47 *** proletesseract has joined #bitcoin-core-dev
182018-10-25T00:40:20 *** prole has joined #bitcoin-core-dev
192018-10-25T00:42:26 *** dqx has quit IRC
202018-10-25T00:42:40 *** proletesseract has quit IRC
212018-10-25T00:42:48 *** dqx has joined #bitcoin-core-dev
222018-10-25T00:43:26 *** bitcoin-git has joined #bitcoin-core-dev
232018-10-25T00:43:26 <bitcoin-git> [bitcoin] jameshilliard opened pull request #14564: Adjust configure so that only bip70 is disabled when protobuf is missing instead of the GUI (master...bip70-disable-check) https://github.com/bitcoin/bitcoin/pull/14564
242018-10-25T00:43:26 *** bitcoin-git has left #bitcoin-core-dev
252018-10-25T00:45:04 *** prole has quit IRC
262018-10-25T00:54:45 *** Tralfaz has joined #bitcoin-core-dev
272018-10-25T00:57:40 *** dqx has quit IRC
282018-10-25T00:58:33 *** dqx has joined #bitcoin-core-dev
292018-10-25T01:00:28 *** dqx has quit IRC
302018-10-25T01:01:50 *** dqx has joined #bitcoin-core-dev
312018-10-25T01:04:39 *** Murch has quit IRC
322018-10-25T01:20:02 *** michaelsdunn1 has joined #bitcoin-core-dev
332018-10-25T01:24:01 *** michaelsdunn1 has quit IRC
342018-10-25T01:31:20 *** esotericnonsense has quit IRC
352018-10-25T01:31:28 *** esotericnonsense has joined #bitcoin-core-dev
362018-10-25T01:32:26 *** lnostdal has joined #bitcoin-core-dev
372018-10-25T01:37:20 *** justanotheruser has quit IRC
382018-10-25T01:40:40 *** bitcoin-git has joined #bitcoin-core-dev
392018-10-25T01:40:40 <bitcoin-git> [bitcoin] sipa opened pull request #14565: Overhaul importmulti logic (master...201810_refactor_importmulti) https://github.com/bitcoin/bitcoin/pull/14565
402018-10-25T01:40:40 *** bitcoin-git has left #bitcoin-core-dev
412018-10-25T01:44:06 <sipa> achow101: see #14565... i tried implementing importmulti by recursing into all scripts and pattern matching, and it seems more concise than the special signer logic approach
422018-10-25T01:44:08 <gribble> https://github.com/bitcoin/bitcoin/issues/14565 | Overhaul importmulti logic by sipa · Pull Request #14565 · bitcoin/bitcoin · GitHub
432018-10-25T01:55:45 *** Krellan has quit IRC
442018-10-25T01:56:30 *** Krellan has joined #bitcoin-core-dev
452018-10-25T01:56:55 *** Aaronvan_ has quit IRC
462018-10-25T02:04:49 *** bitcoin-git has joined #bitcoin-core-dev
472018-10-25T02:04:50 <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14566: 0.17: qa backports (0.17...Mf1810-qaBackports) https://github.com/bitcoin/bitcoin/pull/14566
482018-10-25T02:04:50 *** bitcoin-git has left #bitcoin-core-dev
492018-10-25T02:20:24 *** Bullit has quit IRC
502018-10-25T02:22:45 *** luke-jr has quit IRC
512018-10-25T02:23:01 *** luke-jr has joined #bitcoin-core-dev
522018-10-25T02:24:18 *** luke-jr has quit IRC
532018-10-25T02:25:35 *** luke-jr has joined #bitcoin-core-dev
542018-10-25T02:43:40 *** dqx has joined #bitcoin-core-dev
552018-10-25T03:00:22 *** luke-jr has quit IRC
562018-10-25T03:00:32 *** luke-jr has joined #bitcoin-core-dev
572018-10-25T03:00:52 *** justanotheruser has joined #bitcoin-core-dev
582018-10-25T03:20:24 *** Chris_Stewart_5 has quit IRC
592018-10-25T03:48:59 *** mr_paz_ has quit IRC
602018-10-25T03:50:53 *** schnerchi has joined #bitcoin-core-dev
612018-10-25T03:52:32 *** dqx has quit IRC
622018-10-25T03:53:44 *** schnerch_ has quit IRC
632018-10-25T04:01:53 *** satwo has joined #bitcoin-core-dev
642018-10-25T04:19:58 <meshcollider> sipa: I really like your importmulti cleanup
652018-10-25T04:45:19 *** murrayn has quit IRC
662018-10-25T04:45:34 *** bitcoin-git has joined #bitcoin-core-dev
672018-10-25T04:45:34 <bitcoin-git> [bitcoin] kallewoof closed pull request #14507: net: avoid being disconnected from pruned nodes when syncing up (master...net-pruned-limit-requests) https://github.com/bitcoin/bitcoin/pull/14507
682018-10-25T04:45:34 *** bitcoin-git has left #bitcoin-core-dev
692018-10-25T04:47:43 *** jarthur has joined #bitcoin-core-dev
702018-10-25T05:01:33 *** klot has joined #bitcoin-core-dev
712018-10-25T05:02:29 *** klot has quit IRC
722018-10-25T05:02:56 *** klot has joined #bitcoin-core-dev
732018-10-25T05:03:59 *** klot has quit IRC
742018-10-25T05:04:27 *** klot has joined #bitcoin-core-dev
752018-10-25T05:07:42 *** murrayn has joined #bitcoin-core-dev
762018-10-25T05:07:42 *** murrayn has joined #bitcoin-core-dev
772018-10-25T05:47:51 *** bitconner has quit IRC
782018-10-25T05:52:44 *** lnostdal has quit IRC
792018-10-25T06:29:06 *** promag has joined #bitcoin-core-dev
802018-10-25T06:31:44 *** jarthur has quit IRC
812018-10-25T06:32:34 *** ap4lmtree- has quit IRC
822018-10-25T06:33:01 *** ap4lmtree- has joined #bitcoin-core-dev
832018-10-25T06:36:23 *** promag has quit IRC
842018-10-25T06:52:48 *** morcos has quit IRC
852018-10-25T06:53:03 *** morcos has joined #bitcoin-core-dev
862018-10-25T06:59:54 *** Murch has joined #bitcoin-core-dev
872018-10-25T07:03:44 *** setpill has joined #bitcoin-core-dev
882018-10-25T07:03:49 *** ap4lmtree- has quit IRC
892018-10-25T07:03:52 *** promag has joined #bitcoin-core-dev
902018-10-25T07:04:03 *** Murch has quit IRC
912018-10-25T07:04:17 *** ap4lmtree- has joined #bitcoin-core-dev
922018-10-25T07:19:06 <sipa> meshcollider: cool
932018-10-25T07:20:38 *** promag has quit IRC
942018-10-25T07:24:44 *** timothy has joined #bitcoin-core-dev
952018-10-25T07:32:33 <meshcollider> sipa: is there a way to ToString() on a specific index of a ranged descriptor to get a concrete derivation path? Or does ToString() always only return the ranged one
962018-10-25T07:32:49 <meshcollider> looks like the latter
972018-10-25T07:38:15 <meshcollider> and if not, would it be sensible for me to add it? Or is there a better approach im missing
982018-10-25T07:44:15 *** bitconner has joined #bitcoin-core-dev
992018-10-25T07:46:53 *** Hayro has joined #bitcoin-core-dev
1002018-10-25T07:48:51 *** bitconner has quit IRC
1012018-10-25T07:48:53 *** lnostdal has joined #bitcoin-core-dev
1022018-10-25T07:50:46 <Hayro> hello
1032018-10-25T07:52:02 *** Hayro has left #bitcoin-core-dev
1042018-10-25T07:56:39 <sipa> meshcollider: i thought about adding a way to 'specialize' a ranged descriptor to just a specific indix
1052018-10-25T07:57:11 <sipa> but then i realized that that would actually be duplicating #14477
1062018-10-25T07:57:14 <gribble> https://github.com/bitcoin/bitcoin/issues/14477 | Add ability to convert solvability info to descriptor by sipa · Pull Request #14477 · bitcoin/bitcoin · GitHub
1072018-10-25T07:57:17 <meshcollider> ah so you could call something like desc.Specialize(1).ToString()
1082018-10-25T07:58:02 <sipa> meshcollider: instead, you can expand a descriptor at a certain position, and then run InferDescriptor on the result
1092018-10-25T07:58:30 <sipa> and you'll get exactly the same as you'd get from such a Specialize
1102018-10-25T07:58:58 <sipa> i have a branch that uses that trick to add descriptors to scantxoutset's output
1112018-10-25T07:59:11 <meshcollider> oh, so like
1122018-10-25T07:59:11 <meshcollider> desc.Expand(1, sp, scripts, out);
1132018-10-25T07:59:11 <meshcollider> whatever = InferDescriptor(scripts[0], sp);
1142018-10-25T07:59:35 <meshcollider> ok ill take a look
1152018-10-25T07:59:40 <meshcollider> no PR for that yet?
1162018-10-25T08:00:05 <meshcollider> or are you waiting for 14477 to go in
1172018-10-25T08:02:16 <sipa> yeah, i was waiting for 14477, but it's a really small patch
1182018-10-25T08:02:21 <sipa> i can just add it i guess
1192018-10-25T08:04:59 *** luke-jr has quit IRC
1202018-10-25T08:05:08 *** luke-jr has joined #bitcoin-core-dev
1212018-10-25T08:14:31 *** BGL has quit IRC
1222018-10-25T08:16:11 *** jungly_ has joined #bitcoin-core-dev
1232018-10-25T08:19:26 *** Krellan has quit IRC
1242018-10-25T08:20:53 *** Krellan has joined #bitcoin-core-dev
1252018-10-25T08:43:31 *** proletesseract has joined #bitcoin-core-dev
1262018-10-25T08:44:29 *** chubao_ has joined #bitcoin-core-dev
1272018-10-25T08:44:58 *** drexl has quit IRC
1282018-10-25T09:01:55 *** anon777 has joined #bitcoin-core-dev
1292018-10-25T09:07:43 *** shesek has quit IRC
1302018-10-25T09:12:38 *** anon777 has quit IRC
1312018-10-25T09:14:07 *** owowo has quit IRC
1322018-10-25T09:39:58 *** owowo has joined #bitcoin-core-dev
1332018-10-25T09:40:00 *** BGL has joined #bitcoin-core-dev
1342018-10-25T09:46:57 *** ExtraCrispy has joined #bitcoin-core-dev
1352018-10-25T09:51:37 *** spinza has quit IRC
1362018-10-25T10:04:16 *** promag has joined #bitcoin-core-dev
1372018-10-25T10:12:32 *** dgenr8 has quit IRC
1382018-10-25T10:13:43 *** dgenr8 has joined #bitcoin-core-dev
1392018-10-25T10:16:04 *** shesek has joined #bitcoin-core-dev
1402018-10-25T10:19:31 *** spinza has joined #bitcoin-core-dev
1412018-10-25T10:21:18 *** AaronvanW has joined #bitcoin-core-dev
1422018-10-25T10:39:10 *** shesek has quit IRC
1432018-10-25T10:40:51 *** shesek has joined #bitcoin-core-dev
1442018-10-25T10:40:51 *** shesek has joined #bitcoin-core-dev
1452018-10-25T10:45:57 *** michaelfolkson has joined #bitcoin-core-dev
1462018-10-25T10:51:36 *** spinza has quit IRC
1472018-10-25T10:58:07 *** shesek has quit IRC
1482018-10-25T10:58:41 *** shesek has joined #bitcoin-core-dev
1492018-10-25T10:58:41 *** shesek has joined #bitcoin-core-dev
1502018-10-25T11:04:57 *** spinza has joined #bitcoin-core-dev
1512018-10-25T11:10:02 *** shesek has quit IRC
1522018-10-25T11:10:33 *** shesek has joined #bitcoin-core-dev
1532018-10-25T11:10:33 *** shesek has joined #bitcoin-core-dev
1542018-10-25T11:13:38 *** ap4lmtree- has quit IRC
1552018-10-25T11:14:03 *** ap4lmtree- has joined #bitcoin-core-dev
1562018-10-25T11:31:54 *** ap4lmtree- has quit IRC
1572018-10-25T11:33:45 <ken2812221> Gitian build for Windows is fail on master branch, I can confirm this with WSL. https://bitcoin.jonasschnelli.ch/build/858
1582018-10-25T11:44:05 <ken2812221> The problem seems to be 14451, revert this commit and the build works fine
1592018-10-25T12:01:38 *** klot has quit IRC
1602018-10-25T12:15:55 *** bitcoin-git has joined #bitcoin-core-dev
1612018-10-25T12:15:55 <bitcoin-git> [bitcoin] ken2812221 opened pull request #14568: build: Fix Qt link order for Windows build (master...win-qt-fix) https://github.com/bitcoin/bitcoin/pull/14568
1622018-10-25T12:15:55 *** bitcoin-git has left #bitcoin-core-dev
1632018-10-25T12:16:28 *** Victor_sueca has quit IRC
1642018-10-25T12:17:36 *** Victor_sueca has joined #bitcoin-core-dev
1652018-10-25T12:25:59 *** proletesseract has quit IRC
1662018-10-25T12:26:36 *** proletesseract has joined #bitcoin-core-dev
1672018-10-25T12:28:13 *** proletesseract has quit IRC
1682018-10-25T12:32:22 *** Krellan has quit IRC
1692018-10-25T12:33:43 *** Krellan has joined #bitcoin-core-dev
1702018-10-25T12:34:47 *** bitcoin-git has joined #bitcoin-core-dev
1712018-10-25T12:34:47 <bitcoin-git> [bitcoin] ken2812221 opened pull request #14569: travis: Print characters per 9 min to avoid timeout (master...travis-avoid-timeout) https://github.com/bitcoin/bitcoin/pull/14569
1722018-10-25T12:34:47 *** bitcoin-git has left #bitcoin-core-dev
1732018-10-25T12:35:26 *** IGHOR has quit IRC
1742018-10-25T12:36:48 *** IGHOR has joined #bitcoin-core-dev
1752018-10-25T12:47:52 *** emilengler has joined #bitcoin-core-dev
1762018-10-25T12:48:06 *** emilengler has left #bitcoin-core-dev
1772018-10-25T12:53:11 <promag> wumpus or MarcoFalke, please see #14561
1782018-10-25T12:53:14 <gribble> https://github.com/bitcoin/bitcoin/issues/14561 | Remove fs::relative call and fix listwalletdir tests by promag · Pull Request #14561 · bitcoin/bitcoin · GitHub
1792018-10-25T13:01:31 *** hebasto_ has quit IRC
1802018-10-25T13:07:00 <wumpus> I'm getting really lost in all the wallet directory stuff to be honest
1812018-10-25T13:15:07 <wumpus> didn't I look at another PR for exactly this shortly ago
1822018-10-25T13:15:31 <promag> it was closed and replaced with this
1832018-10-25T13:16:08 <promag> the other used path accessors and fs::equivalent which touches the filesystem
1842018-10-25T13:16:22 <promag> this one only drops the base string from the path string
1852018-10-25T13:17:46 <promag> and since we are recursively iterating walletdir there won't be errors, hence the assert() instead
1862018-10-25T13:19:16 *** bitcoin-git has joined #bitcoin-core-dev
1872018-10-25T13:19:16 <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14571: [tests] Test that nodes respond to getdata with notfound (master...Mf1810-qaNotfound) https://github.com/bitcoin/bitcoin/pull/14571
1882018-10-25T13:19:16 *** bitcoin-git has left #bitcoin-core-dev
1892018-10-25T13:20:03 *** michaelfolkson has quit IRC
1902018-10-25T13:20:16 <wumpus> well, sure, in the context you could say that it never happens, but if you define an utility function I think you need to handle errors properly and not assert
1912018-10-25T13:21:05 <wumpus> or at the least add a comment and explain that the function will crash your program if the input isn't exactly as expected
1922018-10-25T13:21:27 <wumpus> otherwise, before you know it, someone will use it in network code or whatever and you have a DoS
1932018-10-25T13:22:19 *** merland has joined #bitcoin-core-dev
1942018-10-25T13:22:23 <wumpus> documenting assumptions is very important
1952018-10-25T13:25:21 *** shesek has quit IRC
1962018-10-25T13:26:17 *** shesek has joined #bitcoin-core-dev
1972018-10-25T13:26:17 *** shesek has joined #bitcoin-core-dev
1982018-10-25T13:28:19 <wumpus> and I think in general handling errors at the callsite (the decision can always be to crash) is better than crashing inside a helper function
1992018-10-25T13:29:20 * wumpus really appraciates rust's Option<> and Result<> types it's so refreshing after seeing years of broken error handling hacks in C-ish languages
2002018-10-25T13:32:13 *** proletesseract has joined #bitcoin-core-dev
2012018-10-25T13:32:16 *** setpill has quit IRC
2022018-10-25T13:32:41 *** proletesseract has quit IRC
2032018-10-25T13:32:46 *** michaelfolkson has joined #bitcoin-core-dev
2042018-10-25T13:37:12 <promag> wumpus: I can just inline the expression
2052018-10-25T13:37:32 <wumpus> yes
2062018-10-25T13:37:43 <wumpus> still, add a comment please
2072018-10-25T13:38:27 <wumpus> why the assert isn't randomly going to crash the program for some user at some point
2082018-10-25T13:43:23 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2092018-10-25T13:46:03 *** bitcoin-git has joined #bitcoin-core-dev
2102018-10-25T13:46:04 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/613fc95ee4ea...754a00d55f30
2112018-10-25T13:46:05 <bitcoin-git> bitcoin/master 43719e0 Jonas Schnelli: [macOS] Remove DS_Store WindowBounds bytes object
2122018-10-25T13:46:05 <bitcoin-git> bitcoin/master 754a00d Wladimir J. van der Laan: Merge #14416: Fix OSX dmg issue (10.12 to 10.14)...
2132018-10-25T13:46:06 *** bitcoin-git has left #bitcoin-core-dev
2142018-10-25T13:47:38 *** bitcoin-git has joined #bitcoin-core-dev
2152018-10-25T13:47:39 <bitcoin-git> [bitcoin] laanwj closed pull request #14416: Fix OSX dmg issue (10.12 to 10.14) (master...2018/10/osx_dmg) https://github.com/bitcoin/bitcoin/pull/14416
2162018-10-25T13:47:39 *** bitcoin-git has left #bitcoin-core-dev
2172018-10-25T13:49:21 <promag> wumpus: let me know if https://github.com/bitcoin/bitcoin/pull/14561/files lgty
2182018-10-25T13:50:06 *** bitcoin-git has joined #bitcoin-core-dev
2192018-10-25T13:50:07 <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.17: https://github.com/bitcoin/bitcoin/commit/eb2cc84a31fb923b2b25b979682904cb81edec7e
2202018-10-25T13:50:07 <bitcoin-git> bitcoin/0.17 eb2cc84 Jonas Schnelli: [macOS] Remove DS_Store WindowBounds bytes object...
2212018-10-25T13:50:07 *** bitcoin-git has left #bitcoin-core-dev
2222018-10-25T13:51:03 <wumpus> promag: yes lgtm now!
2232018-10-25T13:51:16 <promag> can I squash?
2242018-10-25T13:51:58 <promag> btw, what is preventing from bumping boost? old lts?
2252018-10-25T13:54:32 <wumpus> very simply: there is no good reason to
2262018-10-25T13:54:36 *** Chris_Stewart_5 has quit IRC
2272018-10-25T13:54:44 <wumpus> nothing is *preventing* it but that's not the point, a change needs to be driven by a good reason
2282018-10-25T13:55:11 <wumpus> if we can support old boost versions why not? why give users/developers unnecessary woes?
2292018-10-25T13:55:17 *** shesek has quit IRC
2302018-10-25T13:55:51 <wumpus> ideally we can go with this version until boost dependency can be removed completely
2312018-10-25T13:56:04 <wumpus> we don't really *want* to use anything new from boost
2322018-10-25T13:56:34 *** shesek has joined #bitcoin-core-dev
2332018-10-25T13:57:13 <luke-jr> +1
2342018-10-25T13:57:39 <luke-jr> frankly, I think we bumped univalue too prematurely (and ended up with an unnecessary/unreasonable fork in bitcoin/univalue as a result)
2352018-10-25T13:58:39 <promag> got it +1
2362018-10-25T13:59:14 <wumpus> at least univalue was already part of our own repository, we've more or less took up maintenance
2372018-10-25T13:59:35 <luke-jr> except it wasn't
2382018-10-25T13:59:58 <luke-jr> unless you mean the embedded copy, which should really be removed
2392018-10-25T14:00:33 <luke-jr> there's no justification for forking or embedding univalue, unlike leveldb
2402018-10-25T14:10:13 <wumpus> I really, really don't feel like having this discussion, sorry
2412018-10-25T14:10:23 <luke-jr> probably not worth the time, hence the status quo
2422018-10-25T14:11:55 <wumpus> travis failing on both 0.17 and master, ahhh
2432018-10-25T14:12:40 <luke-jr> :/
2442018-10-25T14:13:24 <promag> "Remote end closed connection without response"
2452018-10-25T14:14:39 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2462018-10-25T14:15:28 *** michaelfolkson has quit IRC
2472018-10-25T14:17:02 <wumpus> promag: is that one of the travis failures?
2482018-10-25T14:17:11 *** Tralfaz has quit IRC
2492018-10-25T14:17:12 <promag> yes, the last
2502018-10-25T14:17:42 <promag> https://travis-ci.org/bitcoin/bitcoin/jobs/446182728#L2801
2512018-10-25T14:19:13 *** drexl has joined #bitcoin-core-dev
2522018-10-25T14:19:16 *** shesek has quit IRC
2532018-10-25T14:19:50 *** shesek has joined #bitcoin-core-dev
2542018-10-25T14:19:50 *** shesek has joined #bitcoin-core-dev
2552018-10-25T14:19:51 <wumpus> the 0.17 issue is a linter issue?!? https://travis-ci.org/bitcoin/bitcoin/jobs/446184576
2562018-10-25T14:20:04 <wumpus> how can there suddenly be so many linter problems
2572018-10-25T14:21:41 <promag> wumpus: new version?
2582018-10-25T14:21:48 *** andytosh1 has quit IRC
2592018-10-25T14:21:49 *** andytosh1 has joined #bitcoin-core-dev
2602018-10-25T14:21:54 *** andytosh1 is now known as andytoshi
2612018-10-25T14:22:24 <promag> does 0.17 locks flake8?
2622018-10-25T14:24:18 <promag> no it doesn't
2632018-10-25T14:25:29 <promag> https://github.com/bitcoin/bitcoin/blob/0.17/.travis.yml#L151 should be `travis_retry pip install flake8==3.5.0`
2642018-10-25T14:26:27 <karelb> not sure if it belongs here - when I read this https://bitcoinops.org/en/newsletters/2018/10/23/ and the issues around remote RPCs, I realized this might be a problem if you run a browser on the same PC
2652018-10-25T14:26:32 <karelb> since browsers now have localhost as a trusted origin, so you can connect to HTTP (without SSL) from any website
2662018-10-25T14:26:37 <karelb> Evil website can try to guess bitcoind credentials and you have the same problems, that the article describes
2672018-10-25T14:27:01 <sdaftuar> is there any reason to run a linter on an old branch?
2682018-10-25T14:28:02 *** shesek has quit IRC
2692018-10-25T14:28:21 *** shesek has joined #bitcoin-core-dev
2702018-10-25T14:28:21 *** shesek has joined #bitcoin-core-dev
2712018-10-25T14:31:07 <promag> sdaftuar: keep consistency on backports?
2722018-10-25T14:31:57 <wumpus> karelb: it's a drawback of using a tcp port for RPC I suppose, let alone http
2732018-10-25T14:33:00 <karelb> wumpus: maybe httpserver.cpp should check an origin header and not allow cross-origin browser requests? (or whatever header browsers add)
2742018-10-25T14:33:45 *** shesek has quit IRC
2752018-10-25T14:33:47 <wumpus> that might be a good precaution
2762018-10-25T14:33:49 <karelb> I haven't actually tried that, maybe it won't work
2772018-10-25T14:34:00 <wumpus> but only if you're sure this is actually a threat
2782018-10-25T14:34:10 <karelb> (I mean I havenmaking a request from w
2792018-10-25T14:34:28 *** shesek has joined #bitcoin-core-dev
2802018-10-25T14:34:28 <karelb> *I haven't tried making a request from a website
2812018-10-25T14:35:15 <karelb> it's a similar threat as connecting from a different IP, no?
2822018-10-25T14:36:15 <karelb> *IF* it actually work :)
2832018-10-25T14:36:34 <wumpus> so are you sure browsers allow making json-rpc requests to localhost? this didn't use to be the case at least
2842018-10-25T14:38:43 <karelb> wumpus: no, I am not sure, I just don't see why it would't work. Since it is just a http GET request.
2852018-10-25T14:38:58 <wumpus> it requires a *post* request
2862018-10-25T14:39:08 <promag> I think it's possible
2872018-10-25T14:39:43 <wumpus> submitting a JSON-RPC command through get is impossibl
2882018-10-25T14:39:55 *** shesek has quit IRC
2892018-10-25T14:40:41 *** shesek has joined #bitcoin-core-dev
2902018-10-25T14:41:19 <karelb> wumpus: what recently changed is that browsers allow connection to http from https websites, if the url is "localhost" (or "localost:port"). It is special-cased. Normally, you cannot make a request to http from https website. It's quite new (1 year, 2 years-ish)
2912018-10-25T14:41:19 *** Krellan has quit IRC
2922018-10-25T14:41:29 <queip> karelb: browsers just trust 127.0.0.1? so any JS on any site could talk to say https://127.0.0.1:7657 or such? that would be a critical vulnerability to many services that have localhost admin panels (with no password) no?
2932018-10-25T14:41:30 <promag> this should be possible: fetch(url, { method: 'post', body: ... })
2942018-10-25T14:42:12 *** Krellan has joined #bitcoin-core-dev
2952018-10-25T14:42:13 <queip> karelb: is this really how it works? this means i2p, and many web-panel based servers are compromised now
2962018-10-25T14:42:50 <karelb> well the service might restrict origin, or cross requests in general
2972018-10-25T14:43:23 <promag> afak with the right cors headers it is possible
2982018-10-25T14:43:26 <karelb> browsers behave well, they send the origin in some header
2992018-10-25T14:43:32 <queip> I bet most do not... seems like idiotic move by browsers?
3002018-10-25T14:43:54 *** shesek has quit IRC
3012018-10-25T14:43:56 *** michaelsdunn1 has joined #bitcoin-core-dev
3022018-10-25T14:43:56 *** michaelsdunn1 has quit IRC
3032018-10-25T14:43:56 *** michaelsdunn1 has joined #bitcoin-core-dev
3042018-10-25T14:44:25 *** shesek has joined #bitcoin-core-dev
3052018-10-25T14:44:25 *** shesek has joined #bitcoin-core-dev
3062018-10-25T14:45:29 <karelb> see https://www.chromestatus.com/feature/6269417340010496
3072018-10-25T14:47:48 *** Chris_Stewart_5 has quit IRC
3082018-10-25T14:48:01 <karelb> but it's solving the "inverse" problem - if a website can trust what it is fetching
3092018-10-25T14:48:43 <karelb> anyway I will just try to hack something to test it, better than talk :D
3102018-10-25T14:53:13 <queip> I have an issue with the github merge script, used in another (sort of private temp, but opensource overall) project... In one case (out of hundreds times using it with bo problem) now git diff HEAD~ in the merge shell shows nothing
3112018-10-25T14:53:16 *** shesek has quit IRC
3122018-10-25T14:53:39 *** shesek has joined #bitcoin-core-dev
3132018-10-25T14:53:47 <queip> any idea what it might be? or, someone wants to look at the git with me? it might be some bug in the tool (or this time out of 1000+ somehow we're using it wrong)
3142018-10-25T14:54:28 *** tintin has joined #bitcoin-core-dev
3152018-10-25T14:54:58 <instagibbs> wumpus, pyflake major version update in a minor flake8 update :shrug:
3162018-10-25T14:55:39 <wumpus> instagibbs: :shrug: typical
3172018-10-25T14:56:19 <instagibbs> we filed a complaint on their gitlab to at least have release notes
3182018-10-25T14:57:59 *** jarthur has joined #bitcoin-core-dev
3192018-10-25T15:00:59 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3202018-10-25T15:01:28 <queip> wumpus: wouldn't you be interested by any chance to look at this git problem, which miiight be a problem in github merge script? I can't figure out why diff HEAD~ is empty... if no one is interested will just merge it, so might be not able to reproduce that later
3212018-10-25T15:02:12 <karelb> OK I panicked too soon; browser does a CORS request and it fails, so it won't continue to connect further.
3222018-10-25T15:02:30 <karelb> `Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://localhost:18444/. (Reason: CORS request did not succeed).`
3232018-10-25T15:02:49 <wumpus> queip: if git diff HEAD~ shows nothing, you have an empty commit
3242018-10-25T15:03:04 <karelb> and the website cannot even distinguish whether it's because the server is not running or whether it is not existent. So all it OK.
3252018-10-25T15:03:17 <wumpus> karelb: good to know; thank you for trying
3262018-10-25T15:04:23 <queip> wumpus: git log -p (int githubmerge.py shell) does indeed show various commits, that are not present on the PR-target branch normally, this is not a "doing nothing" MR. or did you ment that ONE of commits there might be empty and this triggers such reaction
3272018-10-25T15:04:45 <karelb> (maybe it would be a good feature to allow that, to allow websites that interact with full node :D but that is beyond the scope)
3282018-10-25T15:04:59 <queip> karelb: :]
3292018-10-25T15:05:37 <wumpus> karelb: that's going to be very firmly rejected, I expect
3302018-10-25T15:05:58 *** bitcoin-git has joined #bitcoin-core-dev
3312018-10-25T15:05:58 <bitcoin-git> [bitcoin] promag opened pull request #14573: qt: Add Wallet and Window menus (master...2018-10-overhaul-menus) https://github.com/bitcoin/bitcoin/pull/14573
3322018-10-25T15:05:58 *** bitcoin-git has left #bitcoin-core-dev
3332018-10-25T15:07:08 <wumpus> queip: which repository/PR is this?
3342018-10-25T15:07:49 <karelb> you *could* whitelist, as a user, the websites that can connect to the full node. and the user would still need to give the website his name/password/cookie explicitly. so it would not compromise security. :P *but* I see what you mean :D
3352018-10-25T15:08:47 <wumpus> yes, but no way
3362018-10-25T15:10:33 <karelb> I see. :))
3372018-10-25T15:11:58 *** jarthur has quit IRC
3382018-10-25T15:12:25 <karelb> well, you can always write the web app in Electron, that ignore the CORS requests and can grab the cookie directly from the filesystem. Yum, Electron. OK, really out of scope now. :)
3392018-10-25T15:14:46 *** tintin has quit IRC
3402018-10-25T15:16:22 <wumpus> this is getting too scary
3412018-10-25T15:17:12 *** Chris_Stewart_5 has quit IRC
3422018-10-25T15:17:59 <karelb> javascript world is scary
3432018-10-25T15:18:09 <wumpus> don't let it bleed into here
3442018-10-25T15:22:19 *** merland has quit IRC
3452018-10-25T15:26:28 *** shesek has quit IRC
3462018-10-25T15:26:55 *** shesek has joined #bitcoin-core-dev
3472018-10-25T15:26:55 *** shesek has joined #bitcoin-core-dev
3482018-10-25T15:27:13 *** crazyprodigy has joined #bitcoin-core-dev
3492018-10-25T15:27:16 *** shesek has quit IRC
3502018-10-25T15:27:32 <Sentineo> hn
3512018-10-25T15:27:59 <Sentineo> but someoone might try to give you a browser which does not check CORS and connect to your bitcoin through rpc?
3522018-10-25T15:28:11 <Sentineo> hopefully will not happen :)
3532018-10-25T15:28:19 *** shesek has joined #bitcoin-core-dev
3542018-10-25T15:32:29 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3552018-10-25T15:33:08 <karelb> once he already gave you his own binary on your pc, he can do worse...
3562018-10-25T15:33:24 *** crazyprodigy is now known as nerdy_panda
3572018-10-25T15:33:48 *** ExtraCrispy has quit IRC
3582018-10-25T15:35:21 *** shesek has quit IRC
3592018-10-25T15:36:05 *** shesek has joined #bitcoin-core-dev
3602018-10-25T15:36:05 *** shesek has joined #bitcoin-core-dev
3612018-10-25T15:39:29 *** nerdy_panda has left #bitcoin-core-dev
3622018-10-25T15:40:32 *** rex4539 has joined #bitcoin-core-dev
3632018-10-25T15:48:55 *** shesek has quit IRC
3642018-10-25T15:49:36 *** shesek has joined #bitcoin-core-dev
3652018-10-25T15:50:00 <wumpus> if someone 'gives you a browser' that is already the trojan horse scenario
3662018-10-25T15:51:01 *** rh0nj has quit IRC
3672018-10-25T15:52:07 *** rh0nj has joined #bitcoin-core-dev
3682018-10-25T15:52:28 <wumpus> I think what discourages people from even trying browser-based attacks is that bitcoin-qt by default has the RPC server disabled, so most unknowning users won't be affected, the people that enable RPC will generally be more technical and hopefully be careful what they allow
3692018-10-25T15:54:52 *** shesek has quit IRC
3702018-10-25T15:55:27 *** shesek has joined #bitcoin-core-dev
3712018-10-25T15:55:27 *** shesek has joined #bitcoin-core-dev
3722018-10-25T15:58:09 *** jarthur has joined #bitcoin-core-dev
3732018-10-25T16:04:52 *** ExtraCrispy has joined #bitcoin-core-dev
3742018-10-25T16:05:22 <harding> wumpus: someone in #bitcoin the other day said that RPC was enabled by default on Windows in bitcoin-qt. That surprised me, as I didn't even know bitcoin-qt could expose RPC (I thought you needed to use bitcoind for that). I didn't think to mention it here because I didn't realize that not running RPC with bitcoin-qt was supposed to be a security feature.
3752018-10-25T16:05:51 *** shesek has quit IRC
3762018-10-25T16:06:14 <luke-jr> harding: RPC is intentionally enabled by default *on localhost*, and disabling it wouldn't provide any real security improvement I can think of
3772018-10-25T16:06:20 <achow101> harding: it's supposed to only be enabled in qt if you have -server=1
3782018-10-25T16:06:40 <wumpus> harding: setting server=1 in bitcoin.conf should be the only way to enable it with bitcoin-qt
3792018-10-25T16:06:46 <luke-jr> I thought that changed with RPC cookies?
3802018-10-25T16:06:59 <wumpus> no? not that I know, let's check
3812018-10-25T16:07:11 <gmaxwell> I think we discussed that but didn't do it.
3822018-10-25T16:07:13 <achow101> i don't think so
3832018-10-25T16:07:44 <gmaxwell> wumpus: re 'gives you a brower' more like "this site only works in IE 6" (which has broken cross site requrest handing...)
3842018-10-25T16:07:50 *** ppisati has joined #bitcoin-core-dev
3852018-10-25T16:09:20 <gmaxwell> ethereum stuff has been raided over and over again with browser based attacks.
3862018-10-25T16:10:24 *** shesek has joined #bitcoin-core-dev
3872018-10-25T16:11:00 <harding> Looked up the conversation on #bitcoin, the user didn't explicitly say that he was using the default config, so he could've had -server=1.
3882018-10-25T16:11:27 <harding> I did test on Linux, and bitcoin-qt doesn't do RPC there by default.
3892018-10-25T16:11:55 <gmaxwell> I don't think we really should be counting on the default config to protect people... lots of people copy and paste configs.
3902018-10-25T16:12:33 <gmaxwell> echeveria crawled the internet and found something like 3000 bitcoin rpc ports listening, which would be a substantial percentage of all p2p listening nodes.
3912018-10-25T16:12:39 <wumpus> I'm not *counting* on anything, I was just mentioning that it's not a problem with the default confi
3922018-10-25T16:12:45 <harding> gmaxwell: I crawled too and only found 1,100.
3932018-10-25T16:13:08 <wumpus> it shouldn't be a problem with RPC listening on localhost either, it's just defense in depth I suppose..
3942018-10-25T16:13:17 <wumpus> if you don't need a RPC server it's better to disable it
3952018-10-25T16:13:29 <wumpus> even if there is no way you can imagine it can be exploited
3962018-10-25T16:13:32 *** shesek has quit IRC
3972018-10-25T16:14:57 *** shesek has joined #bitcoin-core-dev
3982018-10-25T16:15:26 <wumpus> in any case I checked: bitcoin-qt still doesn't enable RPC server by default
3992018-10-25T16:15:39 <harding> wumpus: thanks. Sorry for the false alarm.
4002018-10-25T16:15:44 <gmaxwell> harding: so 10% instead of 30%. :P
4012018-10-25T16:15:54 <harding> gmaxwell: yeah, it was 13% of my sample.
4022018-10-25T16:15:57 *** shesek has quit IRC
4032018-10-25T16:17:58 *** jungly_ has quit IRC
4042018-10-25T16:18:48 *** shesek has joined #bitcoin-core-dev
4052018-10-25T16:19:11 <phantomcircuit> harding, that is significant
4062018-10-25T16:19:29 <harding> phantomcircuit: I agree.
4072018-10-25T16:21:15 <harding> phantomcircuit: I wish I knew the cause. The only instructions I've seen for enabling it were from a particular mobile wallet, but I have a hard time believing there are over 1,000 mobile wallet users who also run a full node and also opened RPC to use them together.
4082018-10-25T16:22:02 <queip> wumpus: the problem is in PR https://github.com/userghmrt/testrepo/pull/3 . Although on unpatched githubmerge.py it's needed to replace/steart def tree_sha512sum() function with return "0" (because repo uses symlinks) . I confirmed, there in github on that issue 3 git dif HEAD~ is empty evne though on GH page "Files changed: 1"
4092018-10-25T16:22:52 <queip> btw you have invite to edit that test repo if you need to test
4102018-10-25T16:23:09 <gmaxwell> harding: I think there are a lot of example config files floating around.
4112018-10-25T16:23:41 <phantomcircuit> harding, the cause is pretty clearly people being told they need a specific config
4122018-10-25T16:23:54 <gmaxwell> I know from past expirence that joe-user when faced with discovering they need a config file, go and paste some example. When I've asked users for their configs, they have in the past frequently given me ones copy and pasted from the wiki.
4132018-10-25T16:24:11 <harding> gmaxwell, phantomcircuit: that seems likely.
4142018-10-25T16:24:28 <queip> btw, we've patched github merge to support symlinks, submodules (incluees them in tree hash) and gitlab, if anyone wants at some point
4152018-10-25T16:25:22 <harding> I don't know what the solution for that is, though. Better documentation and sample configs provided from an official source?
4162018-10-25T16:28:20 <gmaxwell> harding: probably making bitcoin make its own template config file when one doesn't exist would help.
4172018-10-25T16:28:27 <harding> I guess when .bitcoin is created, a default bitcoin.conf could be created with some normal-to-change options and basic comment documentation. I think you'd want to keep it short, rather than providing every possible option, so that people aren't tempted to delete the whole thing and replace it with a random config from the Wiki.
4182018-10-25T16:29:31 <gmaxwell> I had assumed that problem had gotten less bad because of cookies -- you don't need to make a config file to make bitcoind usable at all... it may be that your 1k rpc listeners have a lot of nodes that came up before cookies existed.
4192018-10-25T16:31:14 <harding> gmaxwell: interesting thought, and something that seems not to hard to check---for an open port 8332, get the node version for port 8333.
4202018-10-25T16:32:55 <gmaxwell> I mean they could have installed 0.11 or whatever and since upgraded but already have a config.
4212018-10-25T16:33:44 <harding> gmaxwell: oh.
4222018-10-25T16:33:55 <gmaxwell> So this would be another advantage in adding an additonal config option that needs to be set to listen to the internet.
4232018-10-25T16:34:04 <harding> Yeah.
4242018-10-25T16:35:29 <luke-jr> my PR kindof does that
4252018-10-25T16:35:37 <luke-jr> (they have to specify rpcbind explicitly)
4262018-10-25T16:37:50 <wumpus> solution: delete all RPC binding functionality, switch to UNIX sockets
4272018-10-25T16:38:12 <luke-jr> that doesn't work on Windows
4282018-10-25T16:38:14 <wumpus> like c-lightning
4292018-10-25T16:38:23 <wumpus> windows has local sockets as well (called differently) IIRC
4302018-10-25T16:38:46 <gmaxwell> wumpus: would be nice, but perhaps too hard to get random software stacks to speak domain sockets.
4312018-10-25T16:38:47 <wumpus> heck, windows 10 even has actual UNIX sockets
4322018-10-25T16:39:15 <gmaxwell> luke-jr: Did you consider doing something more explicit? e.g. making an option called enable-insecurely-exposing-rpc-to-network=1?
4332018-10-25T16:39:19 <wumpus> gmaxwell: I know right! should have done that from the start like c-lightning :-(
4342018-10-25T16:39:34 <wumpus> so many things are imposslbe to change now
4352018-10-25T16:39:48 <wumpus> queip: ok I'll have a look
4362018-10-25T16:39:49 <luke-jr> gmaxwell: no. it's not necessarily insecure, if it's a private LAN
4372018-10-25T16:40:04 <luke-jr> wumpus: can normal Windows software bind to UNIX sockets?
4382018-10-25T16:40:18 <gmaxwell> luke-jr: enable-potentially-inscure-rpc-network-exposure? the point being so you can't just copy and paste without getting a warning.
4392018-10-25T16:40:28 <wumpus> luke-jr: yes I think so
4402018-10-25T16:41:01 <gmaxwell> wumpus: even there, my concern isn't so much intertia-- if it was inertia I think we could just do it and include some shim utility... but like, can nodejs applications ever speak to domain sockets?
4412018-10-25T16:41:15 <luke-jr> gmaxwell: might be worth considering. I expect we'll find other problematic copy-and-paste configs though..
4422018-10-25T16:41:22 <wumpus> we don't even have UNIX support for RPC yet, let alone could set it as only option :(
4432018-10-25T16:41:28 *** OzPac has joined #bitcoin-core-dev
4442018-10-25T16:41:47 <gmaxwell> I think it's okay if for good reason we introduce some incompatiblity in a major version, esp if we give people a long time to switch first... but it wouldn't be good if there is no easy way to become compatible with it. :)
4452018-10-25T16:42:06 <gmaxwell> wumpus: indeed. well we certantly could do that.
4462018-10-25T16:42:06 <wumpus> I'm sure nodejs can use any system API, it's a full environment for server software
4472018-10-25T16:42:20 <gmaxwell> luke-jr: the worst is rpcpassword, but hopefully cookies reduced that.
4482018-10-25T16:42:28 <wumpus> it might be in javascript but it's not *that* much of a joke
4492018-10-25T16:42:46 <queip> would it be possible to tell users what is the problem, other than generic 403 message? like, tell them the option to add but warn them of consequences. or users will complain that "it stopped working after update"
4502018-10-25T16:42:47 *** Chris_Stewart_5 has quit IRC
4512018-10-25T16:43:12 <wumpus> you can send any message with the 403 status, even have a full error html document
4522018-10-25T16:43:14 *** shesek has quit IRC
4532018-10-25T16:43:26 <queip> actually it will not even listen on it to reply 403, just no connection by luke's default?
4542018-10-25T16:43:34 <gmaxwell> Brendan Eich would say never underestimate javascript... which I always took to mean never underestimate the ability of something javascript to be a joke.
4552018-10-25T16:43:42 <wumpus> though I don't think *the latter* is a good idea because software will try to parse text/html replies as json
4562018-10-25T16:43:52 <wumpus> gmaxwell: +1
4572018-10-25T16:44:33 <wumpus> a lot of software uses local pipes for RPC mechanism
4582018-10-25T16:44:39 <wumpus> including databases, which nodejs people love
4592018-10-25T16:45:20 <gmaxwell> queip: sure, our normal practice for depricating RPCs is to make them return errors and have an option to reenable them for one version. So the normal sequence we use where possible is (1) support the new thing (2) announce the old thing is going away [time passes] (3) disable by default the old thing but with a switch to override [time passes] (4) take out the old thing.
4602018-10-25T16:46:09 <gmaxwell> So I think we could change this assuming things actually could talk to it. I don't know that we need to (like adding increasingly agressive warnings against enabling rpc, shifting more stuff onto domain sockets, etc may be enough...
4612018-10-25T16:47:44 <wumpus> FWIW ssh, and I think stunnel, allow forwarding a UNIX domain socket, so it doesn't even make it impossible to expose the RPC over the network, just requires actual setup (and thinking about security)
4622018-10-25T16:48:38 <gmaxwell> if stunnel can do it then that even gives us a backwards compt method if we ever drop TCP support.
4632018-10-25T16:49:51 *** Chris_Stewart_5 has joined #bitcoin-core-dev
4642018-10-25T16:49:51 <wumpus> queip: yes, can you temporarily give me acces to the repo? I can work around it, as I don't actually need to push anything to test, but it's easier
4652018-10-25T16:51:57 <wumpus> just checked: yes, stunnel supports a UNIX socket as destination, but only on UNIX; nginx also supports forwarding requests to UNIX socket based http servers
4662018-10-25T16:51:58 *** Krellan has quit IRC
4672018-10-25T16:52:29 <wumpus> queip: oh nm I have an invite
4682018-10-25T16:52:55 <jarthur> wumpus: heard any word from libevent crew on your pre-existing-fd PR?
4692018-10-25T16:53:00 *** Krellan has joined #bitcoin-core-dev
4702018-10-25T16:55:33 <wumpus> jarthur: they wanted a different solution at the time; which was above my head, certainly, don't know if that ever made any progress
4712018-10-25T16:56:22 <wumpus> the only functionality I needed was to inject an existing fd into the http client, same as they allow for the server binding
4722018-10-25T16:56:57 <esotericnonsense> hm
4732018-10-25T16:57:06 <esotericnonsense> i've just jumped in here and stuck some chat in #bitcoin
4742018-10-25T16:57:23 <esotericnonsense> yes, CORS should (on behaving browsers) prevent requests to localhost and also prevent requests on different ports
4752018-10-25T16:58:14 <esotericnonsense> so _even if_ you have some service running on localhost:10080 say, and you're at http://localhost:10080, you shouldn't hit it (that is if CORS is still enabled for http, i don't really use bare http sites, but I'd assume why not)
4762018-10-25T16:58:58 <esotericnonsense> a bad browser can just ignore CORS but then that's just equivalent to having an insecure system, you're running vulnerable code
4772018-10-25T17:01:02 <esotericnonsense> personally i'd probably ask why bitcoin-qt even has the ability to enable RPC
4782018-10-25T17:01:11 <esotericnonsense> it feels like a 'nice to have' footgun
4792018-10-25T17:01:41 <esotericnonsense> is anyone actually seriously running the GUI client as their backend
4802018-10-25T17:01:48 <wumpus> what, if you want to enable RPC in bitcoin-qt you should be able to
4812018-10-25T17:02:30 <esotericnonsense> what's the use case?
4822018-10-25T17:02:35 <wumpus> I don't believe in making things hard to do because there are a few people that do stupid stuff with it
4832018-10-25T17:02:36 <gmaxwell> esotericnonsense: it enables you to do things like use joinmarket.
4842018-10-25T17:02:43 <luke-jr> esotericnonsense: doing stuff from the commandline.. or third party apps
4852018-10-25T17:03:09 <luke-jr> eg, I think many people probably use it for their taxes
4862018-10-25T17:03:09 <wumpus> punishing power users for the mistakes of others
4872018-10-25T17:03:16 <gmaxwell> If there were litterally no usecase we could come up with then I would probably agree with esotericnonsense's point, but there are many. :P
4882018-10-25T17:03:25 <wumpus> I have various scripts that interface to bitcoin and I use them with the GUI too.
4892018-10-25T17:03:51 <esotericnonsense> i suppose this is just part of my general 'why is the gui still linked in' grumble but then I stopped working on it so can't complain :P
4902018-10-25T17:03:56 <wumpus> sigh...
4912018-10-25T17:04:23 <wumpus> I'll just shut up, I end up disagreeing with everyone on everything anyway
4922018-10-25T17:04:24 <luke-jr> if GUI had to access over RPC, then RPC would become mandatory for GUI users ;)
4932018-10-25T17:04:31 <esotericnonsense> (not that it would help especially I guess since someone copying a random config could just as easily copy a random config in to their bitcoind instance)
4942018-10-25T17:05:25 <wumpus> if it's any guide, even professional stock trading software has a way to enable a RPC interface in their GUI; there's simply users that want to control a program programmatically even if it's a GUI program
4952018-10-25T17:05:28 <sipa> wumpus: please don't shut up :)
4962018-10-25T17:05:45 <wumpus> in some operating systems there is hardly anything *but* GUI programs
4972018-10-25T17:06:06 <esotericnonsense> luke-jr: sure but it wouldn't have to be tcp. i don't know enough about cross platform sockets to comment properly though. the default could be that the bitcoind and qt processes are started with a user that sets permissions on the socket
4982018-10-25T17:06:18 <esotericnonsense> but that's just unix, i've no idea how this works on win
4992018-10-25T17:06:18 <jarthur> I run the GUI client as my backend all the time, though am probably a "power user". Same with Trader Workstation, as wumpus alludes.
5002018-10-25T17:06:20 <wumpus> you might have completely the wrong idea of what people use a GUI, it's not only clueless people
5012018-10-25T17:06:33 <esotericnonsense> wumpus: I don't think it's the case that only clueless people use a GUI
5022018-10-25T17:06:39 <esotericnonsense> at all, sorry if it seemed that way
5032018-10-25T17:06:51 * harding loves the GUI, but also loves CLIs for almost everything else
5042018-10-25T17:07:12 <esotericnonsense> more that it seemed using the GUI as the GUI, and the backend as the backend makes sense, but of course you currently can't do that, so I'm just ranting, basically :P
5052018-10-25T17:08:09 <esotericnonsense> if this is actually a problem, could there be a warning sign or something in the corner of the gui that says 'rpc is enabled, did you know?'? (maybe it already exists)
5062018-10-25T17:09:05 *** Squidicc has joined #bitcoin-core-dev
5072018-10-25T17:09:26 *** dqx has joined #bitcoin-core-dev
5082018-10-25T17:10:07 <wumpus> esotericnonsense: I think that's a good suggestion, no that doesn't exist, feel free to make an issue!
5092018-10-25T17:10:07 *** Squidicuz has quit IRC
5102018-10-25T17:10:19 <wumpus> could certainly have an icon for that
5112018-10-25T17:10:20 *** Squidicc is now known as squidicuz
5122018-10-25T17:10:28 <luke-jr> IMO sensible defaults + education is the solution for things like this
5132018-10-25T17:11:07 <esotericnonsense> i've just sort of popped in and out on this but i've seen the masses of open RPC ports mentioned a bit, do we have any idea why this is the case?
5142018-10-25T17:11:21 * esotericnonsense should make a PR for an alert RPC
5152018-10-25T17:11:53 <harding> esotericnonsense: the guess above was that people are copy/pasting configs that do things they don't necessarily need.
5162018-10-25T17:11:54 <esotericnonsense> connect to all of them and say 'oi, you should probably not do this, especially with password:password' :P
5172018-10-25T17:14:46 <wumpus> 'alert' RPC hehe
5182018-10-25T17:16:20 <wumpus> but I think a good point is that having RPC listening is currently effectively hidden to the user, it's also not configurable from the GUI afaik, only by editing the conf
5192018-10-25T17:17:07 <luke-jr> maybe RPC should accept an 'alert' RPC without password :P
5202018-10-25T17:17:15 <esotericnonsense> luke-jr: that was my thought
5212018-10-25T17:17:29 <wumpus> uhm no
5222018-10-25T17:17:31 <esotericnonsense> i mean if you wanted to get really clever with this
5232018-10-25T17:17:50 <esotericnonsense> you could build in to the p2p network that clients attempt to connect to each others' RPC and issue the 'kill RPC' command if it's publicly routable
5242018-10-25T17:17:51 <esotericnonsense> lol
5252018-10-25T17:18:04 <esotericnonsense> now i'm just having too much fun ;P
5262018-10-25T17:20:54 <Sentineo> :)
5272018-10-25T17:21:02 <Sentineo> they would have to signal the killme bit :)
5282018-10-25T17:22:14 *** dqx has quit IRC
5292018-10-25T17:34:00 <wumpus> an alert RPC, simple as that: https://github.com/laanwj/bitcoin/commit/ace137ff25ab4c7c2a521abe9ba2af0d8af32ec2 (could theoretically make the style flags configurable to send errors etc as well but anyhow xD)
5302018-10-25T17:52:07 *** Victor_sueca has quit IRC
5312018-10-25T17:53:15 *** Victorsueca has joined #bitcoin-core-dev
5322018-10-25T17:56:22 <esotericnonsense> ! :D
5332018-10-25T17:56:22 <gribble> Error: ":D" is not a valid command.
5342018-10-25T17:57:13 <esotericnonsense> i don't have a build env set up at the moment but I am certainly enjoying the lack of any control on this
5352018-10-25T17:57:32 <esotericnonsense> running RPC alert in a tight loop is basically RPC kill ;P
5362018-10-25T17:57:50 <esotericnonsense> (unless this message box is going to replace itself, I assume it just spawns new ones forever)
5372018-10-25T18:00:02 *** Chris_Stewart_5 has quit IRC
5382018-10-25T18:00:56 <wumpus> so just idly wondering: did anyone that tried scanning for open RPC ports, check if the REST interface was enabled?
5392018-10-25T18:01:52 *** spinza has quit IRC
5402018-10-25T18:02:46 <wumpus> esotericnonsense: every open dialog holds up a RPC thread, so you won't be able to open more than four in the default config
5412018-10-25T18:02:56 <esotericnonsense> neat
5422018-10-25T18:03:08 <esotericnonsense> wait, so it is still RPC kill\
5432018-10-25T18:03:11 <esotericnonsense> it's just not node kill
5442018-10-25T18:03:30 <esotericnonsense> if you're AFK and someone gets in to RPC and hits you with four alerts, all the threads are taken. :P
5452018-10-25T18:03:52 <esotericnonsense> that's a neat side effect actually.
5462018-10-25T18:04:12 <wumpus> of course, you can still open a new one immediately after the user closed the old one
5472018-10-25T18:04:17 <wumpus> right
5482018-10-25T18:04:25 <midnightmagic> seems like maybe a bad idea
5492018-10-25T18:05:02 <luke-jr> arbitrary messages could be dangerous
5502018-10-25T18:05:16 <midnightmagic> Any weird QT display interpretation logic there might be an issue with..?
5512018-10-25T18:05:52 <gmaxwell> EMERGENCY: INSTALL UPGRADE FROM http://haxorsserver.com/badsoftware.exe RIGHT AWAY.
5522018-10-25T18:06:19 <luke-jr> yeah, exactly
5532018-10-25T18:06:33 <esotericnonsense> well if the alert command is behind rpc auth
5542018-10-25T18:06:47 <luke-jr> if it's sufficiently secure, it's useless for notifying people with the port exposed
5552018-10-25T18:07:01 <esotericnonsense> yeah
5562018-10-25T18:07:09 <esotericnonsense> well, not quite.
5572018-10-25T18:07:36 <esotericnonsense> if bad passwords are brute forced then it might be useful.
5582018-10-25T18:07:40 <wumpus> of course it's dangerous, then again, so are many other RPC commmands, this would be useful to communicate from say, a backend script to the UI⦠but I don't htink it's actually a good idea just wanted to show how easy it is to do
5592018-10-25T18:08:05 <gmaxwell> hm. so one thing that could be done in the GUI is the first time the rpc is connected after you start, don't allow the connection until the user confirms a dialog.
5602018-10-25T18:08:06 <esotericnonsense> (i'd consider RPC access as probably RCE anyway though I suppose it would require a determined attacker)
5612018-10-25T18:08:35 <gmaxwell> esotericnonsense: part of the reason we don't want the RPC internet exposed is because we don't want YET ANOTHER vector for unauthenticated RCEs.
5622018-10-25T18:08:35 <wumpus> you could do the same for REST, with a pre-programmed message, if you want it to be available with less security
5632018-10-25T18:08:55 <esotericnonsense> gmaxwell: yes, exactly, I think i'm being misinterpreted, sorry
5642018-10-25T18:09:15 * midnightmagic secretly merges alert rpc
5652018-10-25T18:09:27 <wumpus> midnightmagic: it reminds me of the fun when windows had this built in
5662018-10-25T18:09:29 <esotericnonsense> what I mean is that authenticated alert being used to send "install this badness.exe" seems irrelevant if authenticated rpc means you own the box anyway
5672018-10-25T18:09:42 <esotericnonsense> unauthenticated alert is bad sure
5682018-10-25T18:09:51 <wumpus> midnightmagic: you could make *any* computer pop up an alert xD
5692018-10-25T18:10:00 <esotericnonsense> that said, unauth alert could just give a predefined message
5702018-10-25T18:10:10 <midnightmagic> wumpus: death on flaxen wings :-)
5712018-10-25T18:10:20 <gmaxwell> E.g. "A remote control connection is being attempted to your wallet. If you did not initiate this action rejected it. [Allow remote control during this session.] [Reject this attempt.] [Disable remote access].
5722018-10-25T18:10:38 <gmaxwell> "
5732018-10-25T18:10:59 <aj> gmaxwell: needs a "[ ] Always accept these requests" checkbox...
5742018-10-25T18:11:04 <wumpus> yes, so, this is even more dangerous for the wallet
5752018-10-25T18:11:15 <wumpus> which is another reason why having the wall... ok never mind
5762018-10-25T18:12:12 <gmaxwell> aj: why?
5772018-10-25T18:12:34 <wumpus> I kind of like how monero doesn't have a remote API, or daemon for the wallet at all, it's just a command line tool that connects to the node
5782018-10-25T18:12:53 <aj> gmaxwell: sorry, being sarcastic because i hate permission dialogs
5792018-10-25T18:13:06 <gmaxwell> all the examples I gave above about rpc being needed were wallet examples.
5802018-10-25T18:13:28 <esotericnonsense> aj: it would be Always accept these requests by default
5812018-10-25T18:13:44 <esotericnonsense> > gmaxwell | hm. so one thing that could be done in the GUI is the first time the rpc is connected after you start
5822018-10-25T18:13:55 <wumpus> I don't think popping up a dialog for for the first RPC request is a bad idea btw
5832018-10-25T18:14:10 <esotericnonsense> i suppose once per application start is distinct from once per... ever
5842018-10-25T18:14:12 <wumpus> that's, kind of, how those things usually work
5852018-10-25T18:14:16 <gmaxwell> It's one of the things you can do with the GUI... you can get user interaction.
5862018-10-25T18:14:21 <wumpus> yes
5872018-10-25T18:14:31 <esotericnonsense> once per IP address (ever) seems reasonable
5882018-10-25T18:14:48 <gmaxwell> esotericnonsense: meh, ever requires remebering it, and also ends up being less secure.
5892018-10-25T18:14:51 <wumpus> nah first make the RPC localhost-only
5902018-10-25T18:15:06 <gmaxwell> if it's a nussance, there could be a config override.
5912018-10-25T18:15:07 <wumpus> that's like low-hanging fruit
5922018-10-25T18:15:21 <wumpus> after that, you could add interaction
5932018-10-25T18:15:44 <gmaxwell> the non-interaction improvements are needed to protect bitcoind in any cas.e
5942018-10-25T18:15:51 <wumpus> but *remote* RPC is just a stupid idea
5952018-10-25T18:16:08 <gmaxwell> I wouldn't be surprised if most of those rpc listners are bitcoind... after all, bitcoind needs some use of the rpc regardless.
5962018-10-25T18:16:21 <wumpus> yess I know how stupid that sounds as RPC stands for Remote Procedure Call
5972018-10-25T18:16:43 <wumpus> right
5982018-10-25T18:16:51 *** spinza has joined #bitcoin-core-dev
5992018-10-25T18:16:56 <esotericnonsense> yeah i mean personally i don't see why it should be able to listen outside of localhost (or just, socket preferably)
6002018-10-25T18:17:10 <esotericnonsense> even though it would probably break All The Things
6012018-10-25T18:17:44 <wumpus> it's kind of a no-brainer, I'm just afraid of hordes of users complaining about Breaking Things
6022018-10-25T18:17:50 <jarthur> Any time I've done remote RPC (whether with SSH tunnel, or plain old terrible direct connection), it was because one server had enough hard drive space and the other did not.
6032018-10-25T18:18:06 <jarthur> or I was testing something
6042018-10-25T18:18:15 <wumpus> I've actually used remote connections for valid reasons (but yes, always over SSH tunnels)
6052018-10-25T18:18:36 *** Krellan has quit IRC
6062018-10-25T18:18:43 <esotericnonsense> about the only valid config I can think of right now apart from _really_ obtuse cases like a raspberry pi connected directly to another box without a switch
6072018-10-25T18:18:49 <esotericnonsense> would be rpcallowips with wireguard
6082018-10-25T18:18:59 <esotericnonsense> so that it's actually authenticated remote rpc
6092018-10-25T18:19:07 <wumpus> there's also the multi-VM scenario where network connections are inherently secure because the network is virtual
6102018-10-25T18:19:11 <gmaxwell> the problem with disabling non-local is things like private networks between hosts, which are perfectly fine assuming its setup correctly.
6112018-10-25T18:19:16 <wumpus> but yeah...
6122018-10-25T18:19:25 <gmaxwell> and indeed VMs.
6132018-10-25T18:19:33 <gmaxwell> We could set the TTL on those connections to 1. :P
6142018-10-25T18:19:44 <esotericnonsense> i mean localhost in general bugs me.
6152018-10-25T18:19:59 <echeveria> just disallow binding to 0/0.
6162018-10-25T18:20:05 <esotericnonsense> that it's a tcp socket is already 'everyone on this box'. the remote within "trusted" LAN situation is essentially the same
6172018-10-25T18:20:06 <wumpus> with docker it's incredibly common to have one application per container and have them communicate over point-to-point virtual networking, I'm sort of afraid of making those use-cases impossibl
6182018-10-25T18:20:09 <echeveria> require binds to be explicit.
6192018-10-25T18:20:47 <gmaxwell> esotericnonsense: yes, localhost is also a problem, but short of domain sockets we can't really do better than localhost + auth.
6202018-10-25T18:20:52 <esotericnonsense> wumpus: the docker case is reasonable yeah, as with VMs. a proper ethernet bridge.
6212018-10-25T18:21:08 <echeveria> this prevents copy-paste configs from working, massively reduces the ability for bad configurations, but still allows for usability over VPN tunnels, etc.
6222018-10-25T18:21:16 <gmaxwell> echeveria: interesting point, normally you don't want to beceause addresses change, but all the "okay" examples I gave above, they don't.
6232018-10-25T18:21:18 <esotericnonsense> the case in which you have boxA and boxB on the bridge and _noone else_, nice,.
6242018-10-25T18:21:25 <wumpus> yes luke-jr's PR makes sense, to at at least make binding explicit
6252018-10-25T18:21:45 <esotericnonsense> binding to an IP explicitly breaks docker. I think. unless you script it.
6262018-10-25T18:21:49 <gmaxwell> luke's patch also changes it to how I already thought it worked.
6272018-10-25T18:21:56 <esotericnonsense> no idea what IP a container will get.
6282018-10-25T18:22:20 <jarthur> link to luke's PR?
6292018-10-25T18:23:09 *** Murch has joined #bitcoin-core-dev
6302018-10-25T18:23:12 <harding> #14532
6312018-10-25T18:23:14 <gribble> https://github.com/bitcoin/bitcoin/issues/14532 | Never bind INADDR_ANY by default, and warn when doing so explicitly by luke-jr · Pull Request #14532 · bitcoin/bitcoin · GitHub
6322018-10-25T18:23:22 <jarthur> ty
6332018-10-25T18:23:29 <esotericnonsense> i suppose localhost + cookie, if the cookie has the correct permissions set, is similar to a permissioned domain socket
6342018-10-25T18:23:51 <esotericnonsense> the issue is really the strength of the auth mechanism and basically whether the RPC is safe pre-auth
6352018-10-25T18:24:08 <esotericnonsense> 'open port' doesn't really matter if that _actually works_
6362018-10-25T18:24:10 *** Chris_Stewart_5 has joined #bitcoin-core-dev
6372018-10-25T18:24:47 <esotericnonsense> (as in, the situation post 14532 seems reasonable)
6382018-10-25T18:25:02 <echeveria> esotericnonsense: the RPC server is unlikely unsafe pre-auth, it's at least a denial of service risk.
6392018-10-25T18:25:16 <jarthur> gmaxwell: huh, it's funny, that's how I thought it worked already too
6402018-10-25T18:25:34 *** OzPac has quit IRC
6412018-10-25T18:26:27 <jarthur> esotericnonsense: one thing to consider with permissioned domain sockets is Linux abstract unix domain sockets have similar "openness" as TCP loopback.
6422018-10-25T18:27:00 <esotericnonsense> jarthur: eh? it's not possible to listen on a specific user, right?
6432018-10-25T18:27:01 <jarthur> With loopback, you at least have some decent control via netfilter/iptables/ufw on Linux
6442018-10-25T18:27:14 <esotericnonsense> (for TCP loopback)
6452018-10-25T18:27:20 <esotericnonsense> the domain socket can be rw only for the user
6462018-10-25T18:27:25 *** dqx has joined #bitcoin-core-dev
6472018-10-25T18:27:45 <esotericnonsense> e.g. if you had bitcoind, bitcoin-qt and bitcoin-qt had perms on the socket but other than bitcoin-qt only root did
6482018-10-25T18:29:04 <wumpus> !action merge #14532 I guess
6492018-10-25T18:29:04 * gribble merge #14532 I guess
6502018-10-25T18:29:10 <jarthur> esotericnonsense: it can't if it's a Linux abstract unix domain socket
6512018-10-25T18:30:01 *** Murch has quit IRC
6522018-10-25T18:30:09 <esotericnonsense> oh sorry. missed 'abstract'.
6532018-10-25T18:30:39 <booyah> are people actually losing money / being hacked, as result of many listening on internet?
6542018-10-25T18:31:51 <luke-jr> Typically exploits are fixed even if nobody has been compromised with them yet
6552018-10-25T18:32:32 <booyah> sure, not suggesting it's not something to be fixed, just wondering what is the situation
6562018-10-25T18:34:32 <gmaxwell> not that we're aware of, but I suspect most of us are uncomfortable with all this rpc exposure.
6572018-10-25T18:34:47 <gmaxwell> the additional attack surface is a liability we don't want to deal with.
6582018-10-25T18:35:03 <wumpus> how would you even know if people are losing money through this
6592018-10-25T18:35:14 <jarthur> It was real embarrassing for us on the Electrum project when the localhost auto-bind we were going to get around to at some point wasn't gotten around to until it hit the media. :)
6602018-10-25T18:35:24 <booyah> if someone would reported that happening, wumpus. ofc most would not
6612018-10-25T18:35:48 <esotericnonsense> well the fact that it's now known increases the probability of exploit too, right.
6622018-10-25T18:35:52 <wumpus> also agree with luke-jr, it's better to prevent in this case
6632018-10-25T18:36:05 <gmaxwell> booyah: yea, problem is that people don't know or don't report. I'm somewhat doubtful it's causing much loss right now but it means that any rpc/rest bug is potentially much more serious.
6642018-10-25T18:36:18 <luke-jr> jarthur: what port does Electrum bind?
6652018-10-25T18:36:27 <gmaxwell> at least given what we currently believe: that it's being enabled due to accident/confusion.
6662018-10-25T18:36:32 <luke-jr> I wonder if these 8332s aren't even bitcoin itself?
6672018-10-25T18:36:49 <esotericnonsense> luke-jr: is it possible to get a bitcoin-like response back with invalid auth
6682018-10-25T18:36:58 <esotericnonsense> could just hit a few (or a lot) of them and check
6692018-10-25T18:37:14 * esotericnonsense checks his node
6702018-10-25T18:37:16 <luke-jr> has anyone done so?
6712018-10-25T18:37:19 <booyah> do we have the IP list? anyone proceessed it, are theses opened e.g. mostly in some specific network, or OS? (maybe there's a product or something that does that)
6722018-10-25T18:37:29 <wumpus> maybe it's a honeypot
6732018-10-25T18:37:34 <harding> luke-jr: the list of IPs I scan cam from bitnodes, which I think currently filters out Bitcoin Cash nodes. Obviously so were probably spy nodes and the like.
6742018-10-25T18:37:40 <harding> s/so/some/
6752018-10-25T18:38:36 <luke-jr> harding: well, I mean perhaps some other software is listening in 8332 while Bitcoin isn't
6762018-10-25T18:38:41 <wumpus> or maybe they found an exploit in bitcoin-cli and are waiting for you to use it on them :-)
6772018-10-25T18:38:41 <esotericnonsense> lol. I forgot, it's obvious
6782018-10-25T18:38:43 <esotericnonsense> curl localhost:8332
6792018-10-25T18:38:45 <esotericnonsense> JSONRPC server handles only POST requests
6802018-10-25T18:39:00 <gmaxwell> wumpus: lol
6812018-10-25T18:39:07 *** ap4lmtree has joined #bitcoin-core-dev
6822018-10-25T18:39:07 <esotericnonsense> so yeah if you have the list it would probably take a few seconds to hit them all and see if they're not bitcoin. :P
6832018-10-25T18:39:31 <jarthur> luke-jr: electrum's RPC port is randomly selected at startup if you don't configure one. Still didn't take long for scans to pick them up though.
6842018-10-25T18:40:31 <wumpus> yeh randomizing ports helps against attackers looking for targets, not against anyone that already selected you
6852018-10-25T18:40:50 <esotericnonsense> luke-jr: what might be interesting
6862018-10-25T18:41:04 <esotericnonsense> is scanning all IPv4 for 8332 (i.e. hosts that don't listen on 8333)
6872018-10-25T18:41:06 <wumpus> e.g. you can't currently scan the whole IPv4 range for all ports
6882018-10-25T18:41:13 <wumpus> right
6892018-10-25T18:41:22 <esotericnonsense> you can disable listening whilst leaving rpc up right? :P
6902018-10-25T18:41:29 <wumpus> oh sure
6912018-10-25T18:41:43 <esotericnonsense> the odd person might have even fat fingered 8332 instead of 8333 in their dnat :P
6922018-10-25T18:42:34 <luke-jr> that sounds more likely
6932018-10-25T18:42:34 <gmaxwell> you can scan all ports on all hosts that visit your website/irc channel/etc. however.
6942018-10-25T18:43:30 <wumpus> yes, I guess the threat model for bitcoin is somewhat different than say, ssh
6952018-10-25T18:43:45 <booyah> but there are people where actually something respons to TCP 8332 right? so not just opened firewalls / forwarded ports
6962018-10-25T18:44:22 <esotericnonsense> wumpus: heh you've reminded me that I have a weird memory-like error in ssh at the moment... my linux-fu is insufficient to debug it
6972018-10-25T18:44:38 <esotericnonsense> ssh host cat /dev/zero > /dev/null gives me an EFAULT in strace, read into a bad memory address
6982018-10-25T18:44:59 <esotericnonsense> anyway, this is not bitcoin core dev, sorry :P
6992018-10-25T18:47:33 <wumpus> I'd think it's more likely for the crash to be the result of some over-protective security feature in ssh than a security bug, but yeah
7002018-10-25T18:48:31 <luke-jr> I don't see how
7012018-10-25T18:48:44 <luke-jr> cat /dev/zero shouldn't have any security implications
7022018-10-25T18:48:58 *** hebasto has joined #bitcoin-core-dev
7032018-10-25T18:49:08 <esotericnonsense> the system call that fails is a read from the TCP socket (checked fd)
7042018-10-25T18:49:55 <esotericnonsense> the second parameter, the memory address (https://linux.die.net/man/2/read) is not valid
7052018-10-25T18:50:34 <gmaxwell> Is there a socket option to set the TTL on tcp connections? If there is we should perhaps make the rpc use TTL=1 unless overridden by a config option.
7062018-10-25T18:50:51 <esotericnonsense> gmaxwell: ooh!!
7072018-10-25T18:50:54 <esotericnonsense> that is really, really nice.
7082018-10-25T18:51:00 <esotericnonsense> if it's possible.
7092018-10-25T18:51:15 <luke-jr> good idea
7102018-10-25T18:51:59 <echeveria> esotericnonsense: it's fairly trivial to scan 0/0 for a specific port now. there's port scanners that run directly on the NIC which can saturate multi gigabit links.
7112018-10-25T18:52:19 <gmaxwell> that nicely covers the 'lan/vpn good / internet bad'.
7122018-10-25T18:52:19 <echeveria> that said, I'd expect all ports listening on 8332 are listening on 8333.
7132018-10-25T18:52:22 <esotericnonsense> echeveria: yeah I'm aware of that, hence me thinking it might be interesting to see what's on 8332 "globally".
7142018-10-25T18:52:25 <wumpus> if there is one I've never heard of it, you can usually set it on a OS level but not per socket IIR
7152018-10-25T18:52:35 <esotericnonsense> echeveria: probably yeah.
7162018-10-25T18:52:45 <wumpus> it's a fascinating idea though
7172018-10-25T18:54:27 <aj> ip(7) IP_TTL sounds plausible?
7182018-10-25T18:54:30 <esotericnonsense> man 7 ip; IP_TTL (since Linux 1.0)
7192018-10-25T18:54:32 <esotericnonsense> yes
7202018-10-25T18:54:48 <esotericnonsense> you want it to be multiplatform though, :P
7212018-10-25T18:56:38 <aj> https://docs.microsoft.com/en-us/windows/desktop/winsock/ipproto-ip-socket-options and https://www.freebsd.org/cgi/man.cgi?query=ip&sektion=4&manpath=FreeBSD+9.0-RELEASE make it sound pretty multiplatform
7222018-10-25T18:56:54 <gmaxwell> well, linux only would be better than not having it, since so many of our users are on linux.
7232018-10-25T18:57:52 *** dqx has quit IRC
7242018-10-25T18:58:14 *** dqx has joined #bitcoin-core-dev
7252018-10-25T18:59:56 *** clarkmoody has joined #bitcoin-core-dev
7262018-10-25T19:01:48 <achow101> meeting?
7272018-10-25T19:02:03 <wumpus> #startmeeting
7282018-10-25T19:02:03 <lightningbot> Meeting started Thu Oct 25 19:02:03 2018 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
7292018-10-25T19:02:03 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
7302018-10-25T19:02:09 <promag> howdy
7312018-10-25T19:02:15 <achow101> hi
7322018-10-25T19:02:34 <gleb> hi
7332018-10-25T19:02:49 <wumpus> hey
7342018-10-25T19:02:59 *** dqx_ has joined #bitcoin-core-dev
7352018-10-25T19:03:33 <jonasschnelli> Hi
7362018-10-25T19:03:33 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
7372018-10-25T19:03:34 <sipa> hi
7382018-10-25T19:03:48 <luke-jr> ..
7392018-10-25T19:03:59 <midnightmagic> \o
7402018-10-25T19:04:27 *** dqx has quit IRC
7412018-10-25T19:05:04 <wumpus> topics?\
7422018-10-25T19:05:37 <jonasschnelli> topic proposal: 0.17.0.1 or 0.17.1
7432018-10-25T19:05:39 <sipa> what is the status with the linter issues on travis?
7442018-10-25T19:06:07 <phantomcircuit> hello
7452018-10-25T19:06:59 <wumpus> #topic 0.17.0.1 or 0.17.1
7462018-10-25T19:07:33 *** dqx_ has quit IRC
7472018-10-25T19:07:34 <jonasschnelli> I think we should release 0.17.0.1 (osx only) to mitigate the non opening DMG issue with 0.17 (https://github.com/bitcoin/bitcoin/pull/14416)
7482018-10-25T19:08:47 <jonasschnelli> We could release just 0.17.0 + 14416 as 0.17.0.1 macOS only (not release the current 0.17 branch)
7492018-10-25T19:09:04 <luke-jr> #14416
7502018-10-25T19:09:06 <gribble> https://github.com/bitcoin/bitcoin/issues/14416 | Fix OSX dmg issue (10.12 to 10.14) by jonasschnelli · Pull Request #14416 · bitcoin/bitcoin · GitHub
7512018-10-25T19:09:14 *** harrymm has quit IRC
7522018-10-25T19:09:33 <jonasschnelli> The current DMG provided in bitcoincore.org/bin does not open on macOS
7532018-10-25T19:09:34 <wumpus> agree, would make sense to make a 0.17.0.1 for macosx only
7542018-10-25T19:09:43 <luke-jr> jonasschnelli: the only other fix currently in the branch is so minor, it wouldn't make sense to make a new branch for 0.17.0.1
7552018-10-25T19:10:12 *** bitconner has joined #bitcoin-core-dev
7562018-10-25T19:10:18 <jonasschnelli> We can also directly move to 0.17.1,...
7572018-10-25T19:10:23 <sipa> that seems in line with versioning we've used before, using the 4th number for platform specific fixes
7582018-10-25T19:10:25 <luke-jr> doesn't MarcoFalke have a bunch of fixes backported to 0.17 though? might make more sense to just move on 0.17.1
7592018-10-25T19:10:41 <luke-jr> sipa: sure, I'm just saying we can tag it on 0.17 branch
7602018-10-25T19:11:02 <sipa> some of those may be nontrivial; i saw some issue with his backports PR having a merge conflict?
7612018-10-25T19:11:03 <cfields> luke-jr: that would mean that Mac users couldn't drop back to 0.17 if 0.17.1 was buggy.
7622018-10-25T19:11:09 <cfields> +1 for 0.17.0.1
7632018-10-25T19:11:16 <luke-jr> cfields: 0.17.1 should only be fixes on top of 0.17.0 anyway
7642018-10-25T19:11:39 <wumpus> sipa: yep
7652018-10-25T19:12:52 <promag> we could pick the simple backport fixes (including macos fix) and let the remaining for 0.17.2?
7662018-10-25T19:12:59 <wumpus> I dont' think we have enough for 0.17.1 yet
7672018-10-25T19:13:13 <wumpus> if a lot of people are experiencing issues with 0.17.0 on MacOSX we should do 0.17.0.1 soon
7682018-10-25T19:13:16 <wumpus> like, tomorrow
7692018-10-25T19:13:36 <sipa> agree
7702018-10-25T19:13:41 <jonasschnelli> ack
7712018-10-25T19:13:50 <cfields> sgtm
7722018-10-25T19:13:57 <jonasschnelli> I think it's not an urgent thing,.. but it may fraighten off users since it can cause a finder crash
7732018-10-25T19:13:57 * luke-jr shrugs
7742018-10-25T19:14:23 <cfields> jonasschnelli: now this is interesting!
7752018-10-25T19:14:40 <jonasschnelli> Someone told me Apple is aware...
7762018-10-25T19:14:53 <cfields> heh, ok
7772018-10-25T19:15:51 <jonasschnelli> A finder crash smells just really bad and AFAIK there has been some exploitable bugs in that area in the past.
7782018-10-25T19:16:00 *** reallll has joined #bitcoin-core-dev
7792018-10-25T19:16:23 <wumpus> it was funny hwo this is caused by a python unicode versus bytes issue
7802018-10-25T19:16:51 <jonasschnelli> Indeed!
7812018-10-25T19:17:15 <jonasschnelli> I just don't get why it was a non-issue when compiling with trusty.. I though we had the same python version.
7822018-10-25T19:17:19 <jonasschnelli> *thought
7832018-10-25T19:17:41 <jonasschnelli> however,.. lets just do a 0.17.0.1 macOS asap
7842018-10-25T19:18:27 <wumpus> it's great that you managed to isolate it
7852018-10-25T19:18:39 <jonasschnelli> Took me around 30 gitian builds. :)
7862018-10-25T19:18:49 *** belcher_ has quit IRC
7872018-10-25T19:19:35 <cfields> yes, thanks for that. I always check that dmgs open before tagging the detached sigs, but I've stayed on 10.11 to catch back-compat issues, so I guess I avoided it :\
7882018-10-25T19:20:25 <jonasschnelli> Yes. I also take it on me,... I haven't done that. But non of us somehow did test the DMG during all the RCs...
7892018-10-25T19:20:27 <jonasschnelli> which is a bit strange
7902018-10-25T19:20:52 <sipa> that's a bad sign
7912018-10-25T19:21:44 <wumpus> not many people testing on macosx?
7922018-10-25T19:21:48 <cfields> indeed. Maybe we could start adding "Tested ACKs" for the gitian sigs PRs
7932018-10-25T19:22:03 <cfields> to at least verify that someone has started up the release
7942018-10-25T19:22:26 <wumpus> I can only test the linux release myself
7952018-10-25T19:22:46 <wumpus> (and I test compiles on FreeBSD and OpenBSD! but that's irrelevant to gitian)
7962018-10-25T19:22:51 <jonasschnelli> At least fanquake, sjors and other gitian builders use macOS regularly... including myself. But meh,.. I don't know why I haven't detected it
7972018-10-25T19:23:03 <luke-jr> testing RCs is IMO the job of users, not builders or coders
7982018-10-25T19:23:13 <jonasschnelli> Both I'd say
7992018-10-25T19:23:13 <booyah> I can get you mac os X (old macbook) or bsd test (install+run) if you want
8002018-10-25T19:23:56 <cfields> I'll at least start ACKing the platforms that I've verified startup for the sake of posterity.
8012018-10-25T19:24:02 <sipa> luke-jr: agree, but no reason why someone can't be both - and regardless of what you call them, not enough people testing rcs is concerning
8022018-10-25T19:24:38 <luke-jr> sipa: right, my point is that PRs isn't a good place for it
8032018-10-25T19:24:47 <luke-jr> users don't make PRs
8042018-10-25T19:25:07 <sipa> that's reasonable... though how else do you get feedback?
8052018-10-25T19:26:37 <luke-jr> short of writing a website that collects it, and asking with the RC announcement to post there.. coming up blank
8062018-10-25T19:26:53 <wumpus> usually we ask people to open issues on github for problems
8072018-10-25T19:26:54 <luke-jr> maybe bitcointalk can offer a forum specifically for testing reports or something
8082018-10-25T19:26:58 <wumpus> I'm nto sure a different website is needed
8092018-10-25T19:27:02 <luke-jr> true
8102018-10-25T19:27:09 <luke-jr> what's missing is *positive* feedback
8112018-10-25T19:27:10 <wumpus> reddit etc have too much noise
8122018-10-25T19:27:15 <wumpus> yes, true
8132018-10-25T19:27:47 <luke-jr> and apparently the users testing to give it
8142018-10-25T19:28:35 *** Krellan has joined #bitcoin-core-dev
8152018-10-25T19:29:38 <luke-jr> maybe the -core-dev ML needs to get more attention from users so they learn about and try RCs?
8162018-10-25T19:30:30 <wumpus> I.. dont' think you can really get users into a ML these days
8172018-10-25T19:30:39 <sipa> haha
8182018-10-25T19:30:45 <luke-jr> how do users learn about stuff now?
8192018-10-25T19:30:53 <sipa> maybe we need a facebook group *ducks*
8202018-10-25T19:30:56 <luke-jr> >_<
8212018-10-25T19:31:16 <wumpus> :-)
8222018-10-25T19:31:17 <cfields> luke-jr: real answer: software force-updates itself.
8232018-10-25T19:31:28 <cfields> :(
8242018-10-25T19:31:29 <wumpus> twitter is really popular yes
8252018-10-25T19:31:42 <luke-jr> how about I make a Twitter account for posting experimental releases of Core, Knots, and other reputable Bitcoin projects? (Electrum, etc?)
8262018-10-25T19:32:21 <luke-jr> or is there some kind of shared Twitter account thing so more than just I can post?
8272018-10-25T19:33:02 <luke-jr> cfields: not to testing versions..
8282018-10-25T19:33:03 <sipa> we can tweet from the bitcoincoreorg account
8292018-10-25T19:33:17 <luke-jr> sipa: not everyone wants to see experimental releases
8302018-10-25T19:33:57 *** Emcy has quit IRC
8312018-10-25T19:34:54 <aj> luke-jr: could have a flag to say "auto update to current release version" and another to say "follow experimental release candidate stream"
8322018-10-25T19:35:18 *** h00t_ has joined #bitcoin-core-dev
8332018-10-25T19:35:47 <achow101> no auto update pls
8342018-10-25T19:35:50 <warren> luke-jr: have a different twitter account for those interested in following RC's
8352018-10-25T19:36:06 <luke-jr> warren: that's what I said :P
8362018-10-25T19:36:07 <wumpus> right, at least at the moemnt we don't tweet experimental releases from bitcoincore twitter
8372018-10-25T19:36:16 <sipa> yeah
8382018-10-25T19:36:24 <wumpus> I don't even post them to my personal account ,maybe I should
8392018-10-25T19:36:42 <luke-jr> wumpus: that might be enough
8402018-10-25T19:36:51 <warren> Thought you migrated permanently from Twitter. =)
8412018-10-25T19:37:20 <luke-jr> XD
8422018-10-25T19:37:32 <sipa> remind me, i can tweet from my personal account too
8432018-10-25T19:38:15 <wumpus> warren: well I'm more comfortable posting development-related stuff on mastodon, but for noticifcations to reach as many people as possible I'd certainly use twitter
8442018-10-25T19:38:25 <warren> (What is the protocol for topic proposals? wait until this topic is done?)
8452018-10-25T19:38:33 <wumpus> warren: no, just propose
8462018-10-25T19:38:57 <wumpus> ideally you propose topic subjects at the beginning of the meeting
8472018-10-25T19:39:06 <wumpus> then we can cycle through them
8482018-10-25T19:39:12 <wumpus> but it's fine we're done with this now
8492018-10-25T19:39:36 <sipa> i wanted to bring up the linter issues
8502018-10-25T19:40:37 <warren> topic proposal: Interested in opinions regarding the risk of bringing back Fortuna. Along with deprecation of BIP70, we are on the path toward eventual removal of the openssl dependency.
8512018-10-25T19:41:23 <sipa> we really don't need fortuna or a high-performance built-in randomness pool - we don't need randomness frequently
8522018-10-25T19:41:38 <sipa> what we do need is a good way to seed entropy from the environment
8532018-10-25T19:41:43 <rex4539> I think it is quite unlikely that "normal" people will ever run an RC because they are afraid of losing funds because of bugs. Others like me, on the other hand, only run beta software. If something is "stable" I don't want it. I want experimental, unstable software full of bugs. More fun. I believe the reason for missing the DMG bug is that everyone here is building from master and doesn't actually run the public builds.
8542018-10-25T19:41:43 <rex4539> Perhaps there should be a pre-release QA checklist for basic functionality on all supported platforms.
8552018-10-25T19:42:00 <wumpus> #topic Fortuna
8562018-10-25T19:42:18 <wumpus> âor other randomness
8572018-10-25T19:42:20 <warren> well, Fortuna or the lesser goal of seeding entropy from the environment
8582018-10-25T19:42:20 <sipa> and seeding entropy from the environment is annoying as it's a platform specific business
8592018-10-25T19:42:39 <sipa> but we have a built-in randomness pool now - it's not fast, but it's more than good enough for what we need
8602018-10-25T19:42:44 *** reallll is now known as belcher_
8612018-10-25T19:42:50 <sipa> it's just seeding through OpenSSL mostly
8622018-10-25T19:43:07 <warren> I am encouraged that #14451 happened, deprecating BIP70 (huge attack surface, nobody uses it etc.) This means we will eventually be able to remove the openssl dependency. Except for that part.
8632018-10-25T19:43:09 <gribble> https://github.com/bitcoin/bitcoin/issues/14451 | Add BIP70 deprecation warning and allow building GUI without BIP70 support by jameshilliard · Pull Request #14451 · bitcoin/bitcoin · GitHub
8642018-10-25T19:43:38 <sipa> and whenever we need "strong randomness" we mix data from openssl and our own pool
8652018-10-25T19:44:50 <sipa> i think it's sufficient to have a c++ file with a bunch of entropy gathering things in it, without turning it into a C API or whatever
8662018-10-25T19:45:39 <jonasschnelli> #10299
8672018-10-25T19:45:41 <gribble> https://github.com/bitcoin/bitcoin/issues/10299 | Remove OpenSSL by sipa · Pull Request #10299 · bitcoin/bitcoin · GitHub
8682018-10-25T19:45:48 <cfields> sipa: you mean something that is essentially a standalone lib, but with no effort made to actually give it an external api?
8692018-10-25T19:46:17 <warren> #5885 had a previous attempt to replace the openssl PRNG. Reading those old comments remains interesting today.
8702018-10-25T19:46:20 <gribble> https://github.com/bitcoin/bitcoin/issues/5885 | [WIP] Replace OpenSSL PRNG with built-in Fortuna implementation by sipa · Pull Request #5885 · bitcoin/bitcoin · GitHub
8712018-10-25T19:46:32 <sipa> warren: nah, i think that's total overkill now
8722018-10-25T19:47:18 <jonasschnelli> Just read /dev/urandom? *duck*
8732018-10-25T19:47:23 <warren> external API is risky as you need to worry about about fork safety and conditions you can't predict
8742018-10-25T19:47:40 <gmaxwell> jonasschnelli: and then not get randomness when FD exhaustion means you can't open it.
8752018-10-25T19:47:43 <sipa> jonasschnelli: we already do that
8762018-10-25T19:48:00 <sipa> warren: not if it just gathers some entropy from the environment (and not a full RNG library)
8772018-10-25T19:48:19 <warren> Do we already use the newer syscall that blocks if the kernel prng is not seeded?
8782018-10-25T19:48:24 <sipa> yes
8792018-10-25T19:48:40 <kanzure> how do the proofs of randomness/entropy incorporation work
8802018-10-25T19:48:40 <jonasschnelli> gmaxwell: don't we just need a single seed during startup then ChaCha20 PRNG from. there? But i'm not familiar with the details..
8812018-10-25T19:48:43 <wumpus> yes we use the syscall where available
8822018-10-25T19:48:46 <warren> I know a lot less about the other Unixes.
8832018-10-25T19:48:50 <wumpus> on Linux and various BSDs
8842018-10-25T19:48:59 <sipa> jonasschnelli: no, that's FastRandomContext
8852018-10-25T19:49:14 <sipa> and it's only used to generate randomness that doesn't need independently seeding
8862018-10-25T19:49:22 <jonasschnelli> I see
8872018-10-25T19:49:36 <gmaxwell> jonasschnelli: only if everything works right, users never use virtual machines with snapshooting, and we don't care about being totally broken in the face of OS bugs like netbsd and freebsd had in the last couple years.
8882018-10-25T19:49:41 <sipa> for generating private keys etc, we gather new entropy every time
8892018-10-25T19:50:05 <warren> ----> <sipa> i think it's sufficient to have a c++ file with a bunch of entropy gathering things in it, without turning it into a C API or whatever <--- This would be good enough and people would feel it is worth the risk of change to be able to eventually remove the openssl dependency?
8902018-10-25T19:50:10 <gmaxwell> (and don't mind a later process memory leak potentially revealing the keys we previously generated, etc)
8912018-10-25T19:51:23 <gmaxwell> Until BIP70 is gone we're stuck with openssl regardless. we lost urgency on discontinuing using openssl as a randomness input after bitpay started requiring BIP70 to make payments.
8922018-10-25T19:51:40 <gmaxwell> So long as we have openssl for other things it's a harmless addition to random inputs.
8932018-10-25T19:52:10 <cfields> gmaxwell: IIRC that's only supported in -qt
8942018-10-25T19:52:45 <sipa> ah, for "non-strong" randomness (GetRandBytes, as opposed to GetStrongRandBytes) we use just OpenSSL, would people be ok with switching that to our own randomness pool?
8952018-10-25T19:53:04 <gmaxwell> we do? well that should be switched.
8962018-10-25T19:53:10 <jonasschnelli> Agree
8972018-10-25T19:53:43 <sipa> GetStrongRand uses all sources available, and mixes them into our own pool
8982018-10-25T19:53:47 <sipa> including OpenSSL
8992018-10-25T19:55:55 <wumpus> yes
9002018-10-25T19:56:45 <wumpus> switching over non-strong randomness is a no-brainer
9012018-10-25T19:57:15 <jarthur> topic proposal: Unix socket support for RPC API
9022018-10-25T19:57:48 <wumpus> eh <3 minutes left
9032018-10-25T19:57:57 <jarthur> No prob. Next time.
9042018-10-25T19:58:06 <wumpus> I agree though
9052018-10-25T19:58:09 <wumpus> FWIW
9062018-10-25T19:58:38 <wumpus> need that yesterday!
9072018-10-25T19:58:51 <aj> missed out on the linter stuff?
9082018-10-25T19:59:16 <sipa> guess we'll do that another time
9092018-10-25T19:59:22 <sipa> or hope it resolves itself
9102018-10-25T19:59:36 <wumpus> linters are great fun for the whole family, you can watch them fail in real time in so many different ways
9112018-10-25T19:59:38 *** opdenkamp has quit IRC
9122018-10-25T20:00:27 <wumpus> #endmeeting
9132018-10-25T20:00:27 <lightningbot> Meeting ended Thu Oct 25 20:00:27 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
9142018-10-25T20:00:27 <lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-10-25-19.02.html
9152018-10-25T20:00:27 <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-10-25-19.02.txt
9162018-10-25T20:00:27 <lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-10-25-19.02.log.html
9172018-10-25T20:00:29 <cfields> Quick takeback: I wasn't entirely ready yet, and jumped the gun on coming back to dev after some time off a few weeks ago. Sorry if I've slowed anything down since then as a result, and for the annoying false start. Please don't let anything block on waiting for me in the near future.
9182018-10-25T20:00:52 <cfields> That's probably implied by now, just wanted to be explicit about it.
9192018-10-25T20:01:18 <wumpus> cfields: thanks for anything you've done, and no pressure for anything else
9202018-10-25T20:02:06 <cfields> wumpus: I'll for sure take care of whatever's needed from me for 0.17.0.1
9212018-10-25T20:02:20 <cfields> though I suppose that's nothing, if it's mac only :)
9222018-10-25T20:03:20 <wumpus> usually we build the release for all platforms I think, even though there's no effective chagne for others
9232018-10-25T20:03:56 <wumpus> but e.g. the website doesn't support hosting one version for one platform and another for another
9242018-10-25T20:04:33 <cfields> Makes sense, avoids having to customize all of the build scripts and stuff
9252018-10-25T20:04:56 <cfields> ok, will handle that as usual then.
9262018-10-25T20:05:40 <wumpus> do we really need release notes for 0.17.0.1?
9272018-10-25T20:09:10 <cfields> no opinion
9282018-10-25T20:11:51 *** clarkmoody has quit IRC
9292018-10-25T20:14:19 *** harrymm has joined #bitcoin-core-dev
9302018-10-25T20:15:57 *** opdenkamp has joined #bitcoin-core-dev
9312018-10-25T20:16:30 <wumpus> nah just have to mention the macosx issue I guess
9322018-10-25T20:18:19 <sipa> yeah
9332018-10-25T20:23:58 <jnewbery> cfields: take care of yourself first. Everyone here is very grateful for the enormous contributions you've made.
9342018-10-25T20:26:13 <jnewbery> sorry I missed the meeting. On the topic of testing RCs, the optech newsletter has action items each week, which included 'allocate time to test Bitcoin Core RC' for the weeks that we had RCs (eg https://bitcoinops.org/en/newsletters/2018/09/11/). If there are more specific instructions, let us know and we can help spread them.
9352018-10-25T20:26:25 <wumpus> jnewbery: +1
9362018-10-25T20:31:43 <gmaxwell> kallewoof: Are you or anyone else working on getting the "spend all inputs to an address at once if it isn't more expensive" thing going again?
9372018-10-25T20:31:50 <sipa> I dont't Optech members to be the ones who would find issues with the OSX installer, though
9382018-10-25T20:31:57 <gmaxwell> kallewoof: there was recently another round of massive dusting attacks.
9392018-10-25T20:36:11 <jarthur> While some folks are still here, I'm curious if any of you would be against an incremental introduction of unix domain sockets for the RPC API. e.g. release with just server support first, no bitcoin-cli support and no official work around abstract sockets. Server support may be exhumable from wumpus PR #9919.
9402018-10-25T20:36:14 <gribble> https://github.com/bitcoin/bitcoin/issues/9919 | UNIX sockets support for RPC by laanwj · Pull Request #9919 · bitcoin/bitcoin · GitHub
9412018-10-25T20:37:25 <jarthur> bitcoin-cli support is what was held up by proposed upstream libevent work
9422018-10-25T20:38:20 <gmaxwell> jonasschnelli: it's hard to test extensively without bitcoin-cli support, I'd worry about it bitrotting.
9432018-10-25T20:38:23 <gmaxwell> er jarthur
9442018-10-25T20:38:32 <gmaxwell> otherwise getting it in sounds fine to me.
9452018-10-25T20:39:07 <jarthur> Yea, there's only so much we'd be testing from the python functional tests.
9462018-10-25T20:40:18 *** bitconner has quit IRC
9472018-10-25T20:40:54 <sipa> gmaxwell: if the python test framework uses it, i'd be less concerned
9482018-10-25T20:41:50 <gmaxwell> I think thats clearly a requirement.
9492018-10-25T20:42:27 <gmaxwell> Still, its not like our tests are comprehensive enough that it's automatically okay.
9502018-10-25T20:42:39 *** bitconner has joined #bitcoin-core-dev
9512018-10-25T20:43:51 <jnewbery> sipa: perhaps not, but lots of people have subscribed to our newsletter who aren't members. We have about 1500 subs, who I expect are mostly quite technical.
9522018-10-25T20:44:12 <sipa> jnewbery: true, and i'm a big fan of that work, to be clear
9532018-10-25T20:44:51 <sipa> gmaxwell: any type of issue you're worried about in particular w.r.t. unix socket support but excluding bitcoin-cli initially?
9542018-10-25T20:45:06 <jnewbery> I didn't interpret your comment to mean that you weren't :)
9552018-10-25T20:45:39 <gmaxwell> sipa: e.g. will it crash under concurrent request load, etc. thats something we'd probably find out really fast with bitcoin-cli switched to it.
9562018-10-25T20:45:44 <jarthur> On the bright side, having the existing py func tests use a unix socket pretty doable. wumpus had it as an option in his PR.
9572018-10-25T20:46:50 <jnewbery> anyway, I don't think it can harm. If there's any advice on how people can better test RCs and provide feedback, I think we'd be happy to push it out to our subscribers.
9582018-10-25T20:47:07 *** bitconner has quit IRC
9592018-10-25T20:47:10 <jarthur> My reason for proposing server-only is to make the change small enough to not get sidelined again. I know I don't have enough time to work with libevent team on getting preexisting client connection support rolled in.
9602018-10-25T20:57:15 *** Chris_Stewart_5 has quit IRC
9612018-10-25T20:57:43 *** bitcoin-git has joined #bitcoin-core-dev
9622018-10-25T20:57:43 <bitcoin-git> [bitcoin] laanwj opened pull request #14576: Release 0.17.0.1 (0.17...2018_10_release_0.17.0.1) https://github.com/bitcoin/bitcoin/pull/14576
9632018-10-25T20:57:43 *** bitcoin-git has left #bitcoin-core-dev
9642018-10-25T20:58:37 <wumpus> jarthur: yes, I had the functional tests use a UNIX socket, next step would have been to have them interface with P2P over UNIX socket as well but never got that far
9652018-10-25T20:58:51 <wumpus> *for RPC
9662018-10-25T20:59:59 <wumpus> everyone was too much focused on the cli client support and I'm not confident that I can get the libevent people to adapt anything in that regard so I kind of gave up
9672018-10-25T21:00:35 <wumpus> feel free to pick it up though !
9682018-10-25T21:01:03 <jarthur> wumpus: thanks. I don't know the client code very well. Is there an easy alternative where we can just have libevent create the socket?
9692018-10-25T21:01:11 *** h00t_ has quit IRC
9702018-10-25T21:02:26 <wumpus> heh you wish
9712018-10-25T21:03:03 <wumpus> I wouldn't have gone though all that trouble if that existed, right :/
9722018-10-25T21:03:21 <jarthur> figured
9732018-10-25T21:03:25 <gmaxwell> I think it's fine to go without the -cli... just means a somewhat greater amount of testing should be done.
9742018-10-25T21:06:45 *** bitconner has joined #bitcoin-core-dev
9752018-10-25T21:10:08 <wumpus> jarthur: I think the only way to do that would be to implement the entire http protocol (e.g. port over libevhttp), but that's a pretty bad place to be in
9762018-10-25T21:11:39 <jarthur> Is it because we do some custom stuff libevhttp doesn't do?
9772018-10-25T21:11:42 *** dtr has joined #bitcoin-core-dev
9782018-10-25T21:12:37 <wumpus> there's more reasons why libevent's http server isn't really that great for what we're doing (such as the work pool) but there's nothing else that we *can't* do with it
9792018-10-25T21:13:15 <wumpus> httpserver.cpp is basically one complex of working-around thread limitations of their http server
9802018-10-25T21:13:44 <wumpus> (but that's not relevant to UNIX socket support at least :)
9812018-10-25T21:16:58 <wumpus> and there's a more advanced web server that runs on libevent which is commonly recommended: https://github.com/criticalstack/libevhtp ... I don't think that'd work with the client part though, and it's yet another dependency
9822018-10-25T21:17:25 <wumpus> I mean, help with the client part, and no idea if they do support UNIX sockets
9832018-10-25T21:17:51 <wumpus> I'm...so disasppointed
9842018-10-25T21:18:36 <wumpus> (but it's 10000% better than boost asio so...)
9852018-10-25T21:18:40 <sipa> haha
9862018-10-25T21:20:48 <sipa> gmaxwell: the current way GetStrongRandBytes works is by computing SHA512(state || hw_entropy || os_entropy || openssl_entropy), and then using one half of the output as new state, and one half as random output
9872018-10-25T21:21:43 <sipa> for non-strong we can probably replace that with SHA512(state || high-accuracy-timer || some_other_cheap_entropy) or so?
9882018-10-25T21:22:10 <wumpus> their server isn't the problem, it allows injecting custom FDs for UNIX sockets fine! it's the client, we'd need another http *client* that supports http over UNIX sockets
9892018-10-25T21:23:10 <sipa> wumpus: you may hate this... but given that bitcoin-cli is specific to bitcoind, it probably doesn't need to actually implement full HTTP; just whatever subset is needed for bitcoind
9902018-10-25T21:23:47 <wumpus> sipa: yes, that's a good point!
9912018-10-25T21:24:22 <wumpus> hadn't really thought of it that way
9922018-10-25T21:24:39 <sipa> and if there's an adequate HTTP implementation that does what we need, that certainly feels better than something NIHed
9932018-10-25T21:24:49 *** bralyclow2 has joined #bitcoin-core-dev
9942018-10-25T21:25:03 <sipa> but if there's concerns... i don't expect that the subset of HTTP we need is all that hard to write
9952018-10-25T21:25:31 *** bralyclow2 has quit IRC
9962018-10-25T21:25:45 <wumpus> right
9972018-10-25T21:26:09 <wumpus> I missed the part where it only needs to work with our own server :)
9982018-10-25T21:26:16 <jarthur> wumpus: ahh, was the reason we needed the fd-passing ability because their client didn't even have support for starting a unix socket fd to begin with?
9992018-10-25T21:26:33 <wumpus> I guess that makes it quite simple
10002018-10-25T21:27:48 <wumpus> jarthur: yes
10012018-10-25T21:28:25 <wumpus> jarthur: at the time, at least, if they added it since then that'd be incredible
10022018-10-25T21:30:40 *** CubicEarth_ has quit IRC
10032018-10-25T21:31:18 <wumpus> (but I doubt it, they've always limited support to the intersection of various OSes, so having a specific unix API is probably even more out of the question)
10042018-10-25T21:32:20 <jarthur> Yeah, just checked, they haven't
10052018-10-25T21:32:51 *** CubicEarth has joined #bitcoin-core-dev
10062018-10-25T21:33:27 *** timothy has quit IRC
10072018-10-25T21:33:52 <wumpus> too bad
10082018-10-25T21:35:23 <wumpus> it's kind of interesting, I mean Tor uses libevent and they certainly support UNIX sockets for many things, for which they use fd injection at the base layer, they just don't happen to use the http server so that's not important
10092018-10-25T21:35:34 <wumpus> (nor http client)
10102018-10-25T21:36:18 <wumpus> this is just a case of whyyy do you have to make it so difficult for me
10112018-10-25T21:36:34 *** shesek has joined #bitcoin-core-dev
10122018-10-25T21:36:34 *** shesek has joined #bitcoin-core-dev
10132018-10-25T21:36:38 *** promag has quit IRC
10142018-10-25T21:38:44 *** promag has joined #bitcoin-core-dev
10152018-10-25T21:39:40 <wumpus> (why does bitcoin use http? I do not know, I guess JSON-RPC is marginally better than rolling yet another custom protocol, though the line-based JSON RPC of c-lightning is *pretty neat*)
10162018-10-25T21:40:56 <wumpus> why bother wrapping your JSON RPC requests in all this http stuff when you can just do request\nreply\n
10172018-10-25T21:43:00 <jarthur> Perhaps to enable local browser-based experiences?
10182018-10-25T21:43:13 <jarthur> Though there are WebSocket transport options for JSON-RPC now
10192018-10-25T21:43:16 <wumpus> yea exploitability++
10202018-10-25T21:43:20 <jarthur> :)
10212018-10-25T21:44:57 <promag> lol
10222018-10-25T21:47:07 *** promag has quit IRC
10232018-10-25T21:59:47 <phantomcircuit> wumpus, line based json rpc? so stratum?
10242018-10-25T22:01:21 *** bitcoin-git has joined #bitcoin-core-dev
10252018-10-25T22:01:21 <bitcoin-git> [bitcoin] hebasto opened pull request #14577: qt: Cleanup `textInteractionFlags` for `QLabel` (master...20181025-textInteractionFlags) https://github.com/bitcoin/bitcoin/pull/14577
10262018-10-25T22:01:21 *** bitcoin-git has left #bitcoin-core-dev
10272018-10-25T22:01:58 *** Krellan has quit IRC
10282018-10-25T22:02:49 *** Krellan has joined #bitcoin-core-dev
10292018-10-25T22:04:25 *** Emcy has joined #bitcoin-core-dev
10302018-10-25T22:05:42 <wumpus> phantomcircuit: maybe? I've never used stratum, might be similar/the same as c-lightning's protocol
10312018-10-25T22:08:06 *** promag has joined #bitcoin-core-dev
10322018-10-25T22:09:41 *** shesek has quit IRC
10332018-10-25T22:09:58 <queip> <luke-jr> how do users learn about stuff now? <--- make a twitch stream by skimply-dressed lady on configuring RPC ;)
10342018-10-25T22:10:15 *** shesek has joined #bitcoin-core-dev
10352018-10-25T22:10:20 <esotericnonsense> obviously it's so that you can stick your bitcoin rpc behind nginx and put it on https://myexchange.com/api
10362018-10-25T22:10:34 <esotericnonsense> with no auth
10372018-10-25T22:10:45 <esotericnonsense> the RPC port won't be exposed as 8332 so it's fine
10382018-10-25T22:10:56 <wumpus> which, in some sort of way, makes sense if wallets aren't involved
10392018-10-25T22:11:11 * esotericnonsense cries
10402018-10-25T22:11:19 <esotericnonsense> if it's sufficiently sandboxed... maybe...
10412018-10-25T22:11:38 <wumpus> in some ways this is really between a rock and a hard place, most of the information in the API is as public as it gets
10422018-10-25T22:11:59 <esotericnonsense> i don't think that's true
10432018-10-25T22:12:10 <wumpus> you can put it behind a caching API and it's still fine, this was the idea behind the REST API
10442018-10-25T22:12:11 <esotericnonsense> well, I mean, define "most", I guess
10452018-10-25T22:12:11 *** bitconner has quit IRC
10462018-10-25T22:12:37 <esotericnonsense> I think someone who looks at my node monitor for long enough would be able to figure out, for example, geographically where it's located
10472018-10-25T22:12:44 <wumpus> I mean things like 'what peers am I connected to' can be more or less sensitive?
10482018-10-25T22:13:20 <wumpus> I certainly don't mean exposing any information by default, but if someone decides to do it for a public node...
10492018-10-25T22:13:37 <esotericnonsense> yes
10502018-10-25T22:13:38 <esotericnonsense> sure
10512018-10-25T22:13:45 <wumpus> which is what you implied with the nginx thing
10522018-10-25T22:14:09 <jarthur> wumpus: re stratum, it's actually implemented on top of JSON-RPC. A lot of the stratum implementations in crypto projects transport JSON-RPC over TCP with line breaks.
10532018-10-25T22:14:11 <esotericnonsense> if the actual interface itself is hardened and/or it's running in a jail/VM/whatever with resource constraints (ideally both) I don't see an issue
10542018-10-25T22:14:43 <esotericnonsense> i would just assume there are obvious buffer overflows in the rpc interface though
10552018-10-25T22:15:05 <esotericnonsense> without really thinking about it too much just because I'm not sure anyone cares (because it shouldn't be public so why care)
10562018-10-25T22:15:32 <esotericnonsense> just little things, like what happens if someone sends an rpc request with a string parameter that is 400MB
10572018-10-25T22:15:34 <esotericnonsense> lol
10582018-10-25T22:16:11 <wumpus> well then it will use 400MB, it's not really interesting
10592018-10-25T22:16:13 <esotericnonsense> if not RCE-type overflows then probably a DoS if the server dies
10602018-10-25T22:16:21 *** shesek has quit IRC
10612018-10-25T22:16:43 <esotericnonsense> actually, lol
10622018-10-25T22:16:45 <wumpus> you can do that to most websites if you really want
10632018-10-25T22:16:48 <esotericnonsense> doesn't rpc just have a shutdown command
10642018-10-25T22:16:52 <esotericnonsense> :D
10652018-10-25T22:17:40 *** shesek has joined #bitcoin-core-dev
10662018-10-25T22:17:40 *** shesek has joined #bitcoin-core-dev
10672018-10-25T22:19:28 <esotericnonsense> WRT bitcoin-cli and http/domain sockets/etc
10682018-10-25T22:21:24 *** bitconner has joined #bitcoin-core-dev
10692018-10-25T22:21:30 <esotericnonsense> i'm not suggesting a rewrite, but if it were easier that way, is it necessary to restrict to use of C++? I mean it barely does anything anyway
10702018-10-25T22:21:41 *** hebasto has quit IRC
10712018-10-25T22:22:10 <sipa> ?
10722018-10-25T22:22:18 <wumpus> replace c++ by, waht?
10732018-10-25T22:22:21 <esotericnonsense> there's a few config args, the simulated getinfo, help message, and the rest of it is pretty much standard
10742018-10-25T22:22:38 <esotericnonsense> anything you needed to to find a non NIH http solution
10752018-10-25T22:22:50 <wumpus> I think if you'd rewrite the entire thing in rust overnight a lot of people would be happy xD
10762018-10-25T22:22:51 <esotericnonsense> python, say (no idea if python does http over domain sockets)
10772018-10-25T22:23:03 <esotericnonsense> what bitcoin-cli?
10782018-10-25T22:23:17 <wumpus> noo
10792018-10-25T22:23:24 *** ajtowns[m] has joined #bitcoin-core-dev
10802018-10-25T22:23:34 <esotericnonsense> all of bitcoin? :D:D
10812018-10-25T22:23:40 <gmaxwell> I think esotericnonsense is suggesting bitcoin-cli get replaced with a python tool?
10822018-10-25T22:23:57 <queip> that seems easy... but exactly what for>
10832018-10-25T22:24:14 <wumpus> that's trivial
10842018-10-25T22:24:18 <esotericnonsense> gmaxwell: i'm not suggesting it but rather responding to wumpus'/sipa message above about bitcoin-cli not needing to be a full http client and inventing some thing
10852018-10-25T22:24:55 <sipa> afaik the origin bitcoin-cli code was just some printf statements into a socket
10862018-10-25T22:25:07 <gmaxwell> I don't think thats absurd on its face, but: it would probably be a massive slowdown for those users who do processing using bitcoin-cli in scripts, and would suffer the standard "throwaway and rewrite software issues" -- the complexity of existing code is effectively all the accrewed knoweldge built from years and years of use... and that it probably gets right lots of behaviors none of us
10872018-10-25T22:25:07 <gmaxwell> ever explicitly realized were ever requirements.
10882018-10-25T22:25:14 <wumpus> rewriting bitcoin-cli in python, certainly using the code already in the test framework, would be trivial, but why?
10892018-10-25T22:25:35 <esotericnonsense> hm
10902018-10-25T22:25:49 <esotericnonsense> i guess I don't hit the right endpoints for the rpc client to be the bottleneck, heh
10912018-10-25T22:25:57 <wumpus> I mean it means that all users need to have python installed
10922018-10-25T22:26:07 *** bitconner has quit IRC
10932018-10-25T22:26:12 <wumpus> which is the case on linux, and maybe on macosx? but certainly not on windows
10942018-10-25T22:26:20 <gmaxwell> esotericnonsense: I mean just forking _python_ for each request is probably going to be a lot slower.
10952018-10-25T22:26:35 <sipa> gmaxwell: generally i agree with that... but i also think bitcoin-cli is very simple :)
10962018-10-25T22:26:47 <gmaxwell> yea, if we wanted to rewrite anything bitcoin-cli would be it! :)
10972018-10-25T22:26:50 <jarthur> esotericnonsense: on Python, aiohttp supports unix sockets out of the box. The popular synchronous libs (httplib, requests) need supplemental libs to enable unix socket usage currently.
10982018-10-25T22:26:51 <wumpus> it woult make zero difference for bitcoin-cli wrt performance
10992018-10-25T22:26:51 *** spinza has quit IRC
11002018-10-25T22:26:54 <esotericnonsense> i feel like
11012018-10-25T22:27:04 <esotericnonsense> someone using bitcoin-cli in scripts ... argh\
11022018-10-25T22:27:11 <esotericnonsense> supporting that feels awful
11032018-10-25T22:27:23 <sipa> my preference is just replacing the libevent http code in bitcoin-cli with hand-written HTTP though :)
11042018-10-25T22:27:27 <gmaxwell> esotericnonsense: you anti-unix heretic.
11052018-10-25T22:27:29 <wumpus> starting python is very fast, certainly if it's already cached in memory, and all work it does is *trivial*
11062018-10-25T22:27:30 <esotericnonsense> I mean, with interpretation of output, and so on, the whole point is that json rpc is standard
11072018-10-25T22:27:47 <wumpus> python is slow for heavily multi-threaded work but some comand like this?
11082018-10-25T22:27:50 <gmaxwell> wumpus: k. I was just guessing, I have no idea if it actually would be meaningfully slower.
11092018-10-25T22:28:08 * esotericnonsense uses python in production as a http client and gets pretty high concurrent workload
11102018-10-25T22:28:14 <esotericnonsense> er, server
11112018-10-25T22:28:19 <wumpus> gmaxwell: I mean it compiled everything to byte-code and the interpreter is very small
11122018-10-25T22:28:22 <esotericnonsense> it is single threaded though yes
11132018-10-25T22:28:36 <esotericnonsense> the overhead of starting an instance does exist, i just wonder about the cases in which that actually matters
11142018-10-25T22:28:42 <wumpus> I just don't think it's a relevant concern in this case
11152018-10-25T22:28:57 <wumpus> please work on something that actually affects users
11162018-10-25T22:29:06 <esotericnonsense> yes
11172018-10-25T22:29:14 <esotericnonsense> this is silly, sorry for raising it :P
11182018-10-25T22:29:51 <esotericnonsense> jarthur: missed your message, I'm using aiohttp actually, hadn't ever tried it with sockets though.
11192018-10-25T22:29:52 <gmaxwell> I don't think it was bad to raise, we might end up back on it if later bitcoin-cli is blocking disabling tcp rpc by default, and libevent still can't handle using domain sockets for the -cli case.
11202018-10-25T22:29:54 <esotericnonsense> might have to have a play.
11212018-10-25T22:31:14 <wumpus> let's write bitcoin-cli in rust
11222018-10-25T22:31:39 <gmaxwell> K.
11232018-10-25T22:31:46 * esotericnonsense is on page 3 of the rust book after finally getting around to it this morning.
11242018-10-25T22:31:50 <jarthur> CC andytoshi
11252018-10-25T22:32:02 <gmaxwell> Maybe the rust-bitcoin people have already done it.
11262018-10-25T22:32:04 <jarthur> esotericnonsense: works pretty well https://aiohttp.readthedocs.io/en/latest/client_advanced.html#unix-domain-sockets. I wrote the unix socket support for aiohttp server. Performance-wise it's pretty great on both sides.
11272018-10-25T22:32:15 <esotericnonsense> it took me a while to figure out how to add 1 to an integer because x++ doesn't work and that threw me for ages. :D
11282018-10-25T22:32:33 <wumpus> there is a rust bitcoinrpc library, but no command-line thing
11292018-10-25T22:32:44 <esotericnonsense> jarthur: oh! that's awesome! I don't know if you've had your fingers in any other parts of it but I've been really impressed with aiohttp
11302018-10-25T22:33:39 <esotericnonsense> I replaced my old flask backend for my node monitor with aiohttp a week or so ago and it's orders of magnitude faster, (though that is primarily due to using asyncio and not blocking on calls to bitcoind)
11312018-10-25T22:33:55 <jarthur> esotericnonsense: thanks! I'm a small-time contributor to it. I'm a big fan, though I tire of all the nested context managers involved in using the client :)
11322018-10-25T22:33:57 <esotericnonsense> so cheating, really :P
11332018-10-25T22:34:03 <wumpus> which makes sense, why go all the way to compile a rust crate for something then use it from friggin bash xD
11342018-10-25T22:34:40 *** Bullit has joined #bitcoin-core-dev
11352018-10-25T22:34:54 <esotericnonsense> just write bitcoin-cli in bash.
11362018-10-25T22:34:56 <esotericnonsense> and powershell.
11372018-10-25T22:34:58 <esotericnonsense> done.
11382018-10-25T22:35:13 <esotericnonsense> or make windows users use WSL or something. :>
11392018-10-25T22:35:21 <wumpus> ahhh shut up or I'm going to rewrite it in risc-v assembly
11402018-10-25T22:35:46 *** spinza has joined #bitcoin-core-dev
11412018-10-25T22:36:10 <sipa> what's wrong with Visual Basic?
11422018-10-25T22:36:17 <wumpus> xD
11432018-10-25T22:36:52 <midnightmagic> +1 risc-v assembly
11442018-10-25T22:36:56 <wumpus> visual basic 6 was the height of programming language development, after that it's only been a race downward
11452018-10-25T22:37:08 <sipa> wumpus: that was the last one i used :p
11462018-10-25T22:37:13 <wumpus> me too
11472018-10-25T22:37:20 <sipa> after VB6 i learned Perl.
11482018-10-25T22:37:25 <kallewoof> gmaxwell: I will work on getting that in place. Though will it actually help in this case? Dusting attacks I mean.
11492018-10-25T22:37:43 <jarthur> esotericnonsense: biggest problem I have with working in Python and Core RPC so far is how slow the std lib's JSON parsing/creating is, just because it's written in Python itself. I need to see if electrumx will be cool with switching to the raw REST APIs for block retrieval.
11502018-10-25T22:38:02 <kallewoof> gmaxwell: It seems like a more helpful thing to push for is #13756
11512018-10-25T22:38:04 <gribble> https://github.com/bitcoin/bitcoin/issues/13756 | wallet: -avoidreuse feature for improved privacy by kallewoof · Pull Request #13756 · bitcoin/bitcoin · GitHub
11522018-10-25T22:38:24 <esotericnonsense> jarthur: aaargh. you just reminded me.
11532018-10-25T22:38:39 <esotericnonsense> i spent a while hacking together some string stuff to optimize a json call the other day.
11542018-10-25T22:38:53 <esotericnonsense> i forgot to uncomment 'import ujson as json'.
11552018-10-25T22:38:57 <esotericnonsense> -_-
11562018-10-25T22:39:03 <jarthur> ahh, yea, that'll help :D
11572018-10-25T22:39:22 <esotericnonsense> can't be bothered to switch it back now until I need to change it. lol.
11582018-10-25T22:40:13 *** TheCharlatan has quit IRC
11592018-10-25T22:40:14 <wumpus> midnightmagic: so for all other platforms we'll have to ship qemu to emulate it
11602018-10-25T22:40:37 <esotericnonsense> wumpus: no just write the bits of qemu you need to emulate bitcoin-cli on riscv.
11612018-10-25T22:40:43 <esotericnonsense> obviously.
11622018-10-25T22:40:48 <wumpus> I have a risc-v emulator in rust ?
11632018-10-25T22:40:58 * esotericnonsense goes to get a beer.
11642018-10-25T22:41:02 <wumpus> :')
11652018-10-25T22:43:37 <wumpus> sipa: I've also used perl for a little while, don't remember anything though
11662018-10-25T22:43:55 * esotericnonsense can't really remember his route
11672018-10-25T22:44:05 *** michaelsdunn1 has quit IRC
11682018-10-25T22:44:38 <esotericnonsense> i think qbasic, c, ???, python, then just small amounts of arbitrary languages?
11692018-10-25T22:46:33 *** jarthur has quit IRC
11702018-10-25T22:47:31 *** bitconner has joined #bitcoin-core-dev
11712018-10-25T22:53:08 <wumpus> something like MSX basic -> Z80 asm -> gwbasic/qbasic -> PASCAL -> C -> perl -> C++ -> python -> rust ... though I don't exactly remember anymore, and there's some languages like haskell that I tried but never got very far with
11722018-10-25T22:56:09 <sipa> gwbasic, qbasic, vb, perl, c, java, haskell, c++, python
11732018-10-25T22:56:22 <wumpus> I regret everything after PASCAL xD
11742018-10-25T22:57:14 <aj> GWBasic, C64 Basic, AMOS Basic, qbasic, C, [Pascal], Smalltalk, Ada, Perl, C++, Prolog, Java, Python...
11752018-10-25T22:57:32 <aj> oh, actually would've been quickbasic not qbasic
11762018-10-25T22:58:42 <sipa> oh, there's a small amount PHP somewhere
11772018-10-25T23:00:51 <jcorgan> after Forth it all went downhill
11782018-10-25T23:02:54 <wumpus> oh I forgot about PHP as well, that was between perl and C++ I guess
11792018-10-25T23:04:32 <sipa> between java and haskell for me
11802018-10-25T23:04:52 <wumpus> aj: Ada is neat, I think it's a nice evolution of pascal, too bad it's mostly only used for us military stuff
11812018-10-25T23:06:12 *** so has quit IRC
11822018-10-25T23:08:15 <aj> wumpus: the precondition/postcondition stuff was pretty nifty. i remember it being a gratuitious pain to program with though, for what felt like syntactic reasons. might've been weird manual array management that would be completely laughable these days, don't remember
11832018-10-25T23:08:44 <aj> wumpus: i was never a fan of pascal though *shrug*
11842018-10-25T23:09:29 <esotericnonsense> i think I used PHP enough to think 'f this'
11852018-10-25T23:09:33 <wumpus> aj: I *loved* pascal, it was only because C seemed to be the wave of the future that I learned it and moved to it
11862018-10-25T23:10:28 <wumpus> that's a loong time ago though, I don't think I could do much with it these days
11872018-10-25T23:10:52 *** so has joined #bitcoin-core-dev
11882018-10-25T23:11:07 <wumpus> but things like the module system, bounded arrays, actually enforcede enumerations, it seemed to be ahead of C in many things
11892018-10-25T23:12:32 <wumpus> C was pretty nice as a low level, platform independent assembler replacement, but then they started to actually enforce things like undefined behavior, and platform-dependent behavior became important, and heck all of today's pain
11902018-10-25T23:12:50 <aj> wumpus: i guess bounded arrays, enforced enumerations are all strong-typing things where you'd go to haskell (or maybe rust?) these days
11912018-10-25T23:13:26 <wumpus> aj: it's why I like rust I think, I also *like* Haskell but cannot do anything practical with it
11922018-10-25T23:13:54 <sipa> i absolutely loved haskell as a language, and in many ways wished mainstream languages were more similar to it
11932018-10-25T23:14:31 <sipa> except in practice the type of programs i like to write are ones where performance and predictable resource usage matter... exactly what it abstracts away
11942018-10-25T23:14:36 <jcorgan> the older i get, the more languages i learn, and the fewer i use
11952018-10-25T23:14:53 <jcorgan> i'm pretty much down to python and c++ these days
11962018-10-25T23:15:00 <aj> sipa: haskell, but with performance specified as part of the type ;)
11972018-10-25T23:15:03 <sipa> not to say that you can't write performant code in haskell, but it feels like you need to break the otherwise awesome abstraction it provides in order to do so
11982018-10-25T23:15:18 *** Krellan has quit IRC
11992018-10-25T23:15:18 <sipa> i need to learn rust
12002018-10-25T23:15:27 <jcorgan> the last language to really interest me was Go, but that was self-limiting
12012018-10-25T23:15:36 <esotericnonsense> I like go
12022018-10-25T23:15:57 <esotericnonsense> i usually just run into a wall of sort of... OK, this is an interesting language, but does anything I do actually matter enough to optimise for language
12032018-10-25T23:16:08 <esotericnonsense> the answer is no, so I just end up reaching for python
12042018-10-25T23:16:31 <jcorgan> i liked Go's built-in concurrency and message passing
12052018-10-25T23:16:42 * sipa discovers the -j option to test_runner.py
12062018-10-25T23:16:44 <wumpus> sipa: +1 on Haskell, it's nice and pure but also I really like to understand how things map to the mechanical reality how things run on CPUs, and that's hard to grasp for me
12072018-10-25T23:16:51 * esotericnonsense has opened the rust tutorial again.
12082018-10-25T23:16:58 <gmaxwell> kallewoof: the dusting party was paying people that had existing outputs.
12092018-10-25T23:17:02 <esotericnonsense> I killed my vfio VM and can't be arsed to reboot the whole box to get it to work. heh
12102018-10-25T23:17:15 <gmaxwell> kallewoof: so if we automatically would spend that dust it would help and make the attacks less interesting.
12112018-10-25T23:17:32 <sipa> (maybe the language discussion is getting a bit off topic here; sorry for continuing it myself)
12122018-10-25T23:18:11 <gmaxwell> kallewoof: the forced avoidreuse is interesting too by my opinion is that off by default behavior is almost irrelevant from a privacy perspective. We should still do it, as part of a prinicipled commitment to privacy, but in practice if it's not on by default it's mostly not helping people.
12132018-10-25T23:18:31 <wumpus> yes, sorry, I don't really want to say much about language, except taht I increasingly dislike c++ and like rust
12142018-10-25T23:18:34 <gmaxwell> I think that probably it'll only be aggregatable signatures that really get us to where we need to be on that.
12152018-10-25T23:18:44 <kallewoof> gmaxwell: If I dust your bc1qgmax you will never use that "group" unless you turn on -avoidpartialspends, because using it will always give a higher fee.
12162018-10-25T23:19:07 <kallewoof> gmaxwell: with the "do if it gives same fee" feature, I mean.
12172018-10-25T23:19:19 <gmaxwell> kallewoof: not necessarily, because the alternative spend could have just as many but different inputs.
12182018-10-25T23:19:46 <kallewoof> But if it's dust, will that ever happen?
12192018-10-25T23:20:26 <gmaxwell> kallewoof: sure, you can miss by any amount, if your payment plus fees ends up being 5 satoshi short, then you're going to need another input and a dust one might be perfectly reasonsable.
12202018-10-25T23:21:11 <kallewoof> Sounds unusual but sure! :) I'll work on getting that back in. I am not sure the approach I had was reasonable though (doing the entire coin selection twice and comparing), esp for bigger wallets
12212018-10-25T23:23:36 <gmaxwell> kallewoof: why do you think it's unusual? I expect the amount missed in any coin selection to be a small value with a peak near zero.
12222018-10-25T23:24:17 <gmaxwell> And importantly, this really can be always on for everyone with effectively no compromise (beyond the development effort)
12232018-10-25T23:24:28 *** OzPac has joined #bitcoin-core-dev
12242018-10-25T23:24:49 <gmaxwell> It also can be generalized to a form where you'll pay extra for it by some configurable small amount, but not an unbounded amount extra... which is I suspect what joe-user probably wants.
12252018-10-25T23:24:51 *** OzPac has quit IRC
12262018-10-25T23:25:08 <kallewoof> Oh, that sounds pretty useful, yeah.
12272018-10-25T23:25:42 <kallewoof> The only compromise is that coin selection takes 2x the time. For big wallets I hear rumors that this may be a big deal actually
12282018-10-25T23:25:53 <kallewoof> I should probably make a big wallet and see for myself.
12292018-10-25T23:26:00 <gmaxwell> I don't think it would on those wallets.
12302018-10-25T23:26:24 <gmaxwell> Also if you hear rumors like that you should invite people to report them, there is a lot of rumor that turns out to be spurrious.
12312018-10-25T23:26:53 <kallewoof> I think it was rumors that came from some exchange. I don't know if they're comfortable revealing too much about their set up.
12322018-10-25T23:26:56 <gmaxwell> Considering that we used to sign in the coinselection loop and are now well over 100x faster for many input spends, I suspect we have headroom. :)
12332018-10-25T23:27:07 <kallewoof> Should be easy to test though. Just make a wallet with a ton of UTXOs
12342018-10-25T23:27:17 <kallewoof> Oh.. wow.
12352018-10-25T23:27:56 <gmaxwell> Right but the rumors aren't useful reports-- because they can't distinguish 'slow' being from things like signing in-loop for many input spends vs basic behavior. If businesses want to get behaior out of us, they need to behave like professionals.
12362018-10-25T23:28:52 <gmaxwell> kallewoof: also we could just have an option to turn it off if we're concerned... and the few parties that care could flip it.
12372018-10-25T23:30:53 <wumpus> yes, the problem has always been that the big users neither contribute actually useful reports, nor developers that optimize what is important for them
12382018-10-25T23:30:54 <kallewoof> gmaxwell: sdaftuar spoke against the feature in the original PR here: https://github.com/bitcoin/bitcoin/pull/12257#discussion_r204168100 ("I'm not sure that -avoidpartialspends works very well on wallets that have substantial address reuse, in particular I think you can devolve into cases where you'd produce giant transactions that take forever to sign and would never pass policy (or consensus) limits, in extreme cases.")
12392018-10-25T23:32:05 <kallewoof> The concern was partially mitigated by putting a cap on the UTXOs in a single group so maybe sdaftuar is cool with the idea now.
12402018-10-25T23:32:46 <wumpus> oh it's slow if you have tens of thousands of utxos, yea... that's awesome, no one that actually partakes in development uses it with that
12412018-10-25T23:33:07 *** Krellan has joined #bitcoin-core-dev
12422018-10-25T23:33:20 <gmaxwell> wumpus: it shouldn't be.
12432018-10-25T23:33:54 <wumpus> of course it shouldn't
12442018-10-25T23:33:59 <gmaxwell> kallewoof: re coinselection speed, another thing that used to make it slow is having a lot of unconfirmed outputs... we got 1000x fold speedups due to fixing it so that IsMine didn't have quasi-factorial complexity.
12452018-10-25T23:34:36 * kallewoof looks up quasi-factorial
12462018-10-25T23:34:55 <echeveria> from my experience people making a lot of these sort of claims donât actually use the software at all, so keep that in mind too. Iâve had people sit and insist to me that the bitcoin wallet has all sorts of stupid behaviour I full well know it doesnât.
12472018-10-25T23:35:01 <gmaxwell> I think we need to draw a line in the sand. If these large commercial players can't behave professionally enough to actually report their concerns (much less contribute effort to fix them and/or add test cases) we should ignore them... not fall into some rumor based development.
12482018-10-25T23:35:05 <wumpus> but I couldn't much be concerned about companies that make millions from using the software but contribute nothing back
12492018-10-25T23:35:12 <gmaxwell> Right.
12502018-10-25T23:35:32 <kallewoof> Makes sense to me
12512018-10-25T23:36:19 <echeveria> some of this comes from parties like blockchain.info who have frequently gone around conferences repeating stories of how they tried so hard to contribute to bitcoin core and were rejected for their efforts.
12522018-10-25T23:36:50 <gmaxwell> (I very much want their input for sure, but if they don't provide it.. they don't provide it. Unfortunately, my expirence with many bitcoin companies is that they have executives that have never actually used software in a commercial context before, much less open source... and they treat it like the android play store... an app doesn't do what you want, you swap out for another one).
12532018-10-25T23:37:14 <kallewoof> heh
12542018-10-25T23:37:32 <echeveria> now if thatâs true or not I donât know, but Iâve not seen that sort of behaviour. thereâs some contributes in the source tree from people I wouldnât want to be in the same room with- because theyâre evaluated on their merit rather than where they are from.
12552018-10-25T23:37:40 <gmaxwell> hopefully optech will do a useful job in getting more input, but otherwise we should just do the best we can. :)
12562018-10-25T23:38:48 <gmaxwell> echeveria: I personally liked the one company going around telling everyone that bitcoin core constantly crashed, and then got shut up by a developer offering them a sizable bet that they couldn't report any crash...
12572018-10-25T23:39:30 <sipa> in particular about coin selection, one question that came out of optech was making coin selection code more accessible... which sounds to me like they're not using the built-in wallet, and thus won't ever observe the coin selection logic
12582018-10-25T23:39:36 <sipa> (that may be an overgeneralization)
12592018-10-25T23:49:22 *** lnostdal has quit IRC
12602018-10-25T23:56:46 *** drexl has quit IRC