1 2018-10-25T00:00:31  *** sipa has quit IRC
   2 2018-10-25T00:00:56  *** sipa has joined #bitcoin-core-dev
   3 2018-10-25T00:16:25  *** proletesseract has quit IRC
   4 2018-10-25T00:16:50  *** bralyclo_ has joined #bitcoin-core-dev
   5 2018-10-25T00:17:07  *** proletesseract has joined #bitcoin-core-dev
   6 2018-10-25T00:17:25  *** proletesseract has quit IRC
   7 2018-10-25T00:17:57  *** grubles has joined #bitcoin-core-dev
   8 2018-10-25T00:18:42  *** bralycl__ has joined #bitcoin-core-dev
   9 2018-10-25T00:19:33  *** bralycl__ has quit IRC
  10 2018-10-25T00:19:47  *** bralyclow has quit IRC
  11 2018-10-25T00:20:50  *** bralyclow has joined #bitcoin-core-dev
  12 2018-10-25T00:22:03  *** bralyclo_ has quit IRC
  13 2018-10-25T00:22:14  *** bralyclo_ has joined #bitcoin-core-dev
  14 2018-10-25T00:25:33  *** bralyclow has quit IRC
  15 2018-10-25T00:29:08  *** Tralfaz has quit IRC
  16 2018-10-25T00:34:49  *** Mutter has joined #bitcoin-core-dev
  17 2018-10-25T00:37:47  *** proletesseract has joined #bitcoin-core-dev
  18 2018-10-25T00:40:20  *** prole has joined #bitcoin-core-dev
  19 2018-10-25T00:42:26  *** dqx has quit IRC
  20 2018-10-25T00:42:40  *** proletesseract has quit IRC
  21 2018-10-25T00:42:48  *** dqx has joined #bitcoin-core-dev
  22 2018-10-25T00:43:26  *** bitcoin-git has joined #bitcoin-core-dev
  23 2018-10-25T00:43:26  <bitcoin-git> [bitcoin] jameshilliard opened pull request #14564: Adjust configure so that only bip70 is disabled when protobuf is missing instead of the GUI (master...bip70-disable-check) https://github.com/bitcoin/bitcoin/pull/14564
  24 2018-10-25T00:43:26  *** bitcoin-git has left #bitcoin-core-dev
  25 2018-10-25T00:45:04  *** prole has quit IRC
  26 2018-10-25T00:54:45  *** Tralfaz has joined #bitcoin-core-dev
  27 2018-10-25T00:57:40  *** dqx has quit IRC
  28 2018-10-25T00:58:33  *** dqx has joined #bitcoin-core-dev
  29 2018-10-25T01:00:28  *** dqx has quit IRC
  30 2018-10-25T01:01:50  *** dqx has joined #bitcoin-core-dev
  31 2018-10-25T01:04:39  *** Murch has quit IRC
  32 2018-10-25T01:20:02  *** michaelsdunn1 has joined #bitcoin-core-dev
  33 2018-10-25T01:24:01  *** michaelsdunn1 has quit IRC
  34 2018-10-25T01:31:20  *** esotericnonsense has quit IRC
  35 2018-10-25T01:31:28  *** esotericnonsense has joined #bitcoin-core-dev
  36 2018-10-25T01:32:26  *** lnostdal has joined #bitcoin-core-dev
  37 2018-10-25T01:37:20  *** justanotheruser has quit IRC
  38 2018-10-25T01:40:40  *** bitcoin-git has joined #bitcoin-core-dev
  39 2018-10-25T01:40:40  <bitcoin-git> [bitcoin] sipa opened pull request #14565: Overhaul importmulti logic (master...201810_refactor_importmulti) https://github.com/bitcoin/bitcoin/pull/14565
  40 2018-10-25T01:40:40  *** bitcoin-git has left #bitcoin-core-dev
  41 2018-10-25T01:44:06  <sipa> achow101: see #14565... i tried implementing importmulti by recursing into all scripts and pattern matching, and it seems more concise than the special signer logic approach
  42 2018-10-25T01:44:08  <gribble> https://github.com/bitcoin/bitcoin/issues/14565 | Overhaul importmulti logic by sipa · Pull Request #14565 · bitcoin/bitcoin · GitHub
  43 2018-10-25T01:55:45  *** Krellan has quit IRC
  44 2018-10-25T01:56:30  *** Krellan has joined #bitcoin-core-dev
  45 2018-10-25T01:56:55  *** Aaronvan_ has quit IRC
  46 2018-10-25T02:04:49  *** bitcoin-git has joined #bitcoin-core-dev
  47 2018-10-25T02:04:50  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14566: 0.17: qa backports (0.17...Mf1810-qaBackports) https://github.com/bitcoin/bitcoin/pull/14566
  48 2018-10-25T02:04:50  *** bitcoin-git has left #bitcoin-core-dev
  49 2018-10-25T02:20:24  *** Bullit has quit IRC
  50 2018-10-25T02:22:45  *** luke-jr has quit IRC
  51 2018-10-25T02:23:01  *** luke-jr has joined #bitcoin-core-dev
  52 2018-10-25T02:24:18  *** luke-jr has quit IRC
  53 2018-10-25T02:25:35  *** luke-jr has joined #bitcoin-core-dev
  54 2018-10-25T02:43:40  *** dqx has joined #bitcoin-core-dev
  55 2018-10-25T03:00:22  *** luke-jr has quit IRC
  56 2018-10-25T03:00:32  *** luke-jr has joined #bitcoin-core-dev
  57 2018-10-25T03:00:52  *** justanotheruser has joined #bitcoin-core-dev
  58 2018-10-25T03:20:24  *** Chris_Stewart_5 has quit IRC
  59 2018-10-25T03:48:59  *** mr_paz_ has quit IRC
  60 2018-10-25T03:50:53  *** schnerchi has joined #bitcoin-core-dev
  61 2018-10-25T03:52:32  *** dqx has quit IRC
  62 2018-10-25T03:53:44  *** schnerch_ has quit IRC
  63 2018-10-25T04:01:53  *** satwo has joined #bitcoin-core-dev
  64 2018-10-25T04:19:58  <meshcollider> sipa: I really like your importmulti cleanup
  65 2018-10-25T04:45:19  *** murrayn has quit IRC
  66 2018-10-25T04:45:34  *** bitcoin-git has joined #bitcoin-core-dev
  67 2018-10-25T04:45:34  <bitcoin-git> [bitcoin] kallewoof closed pull request #14507: net: avoid being disconnected from pruned nodes when syncing up (master...net-pruned-limit-requests) https://github.com/bitcoin/bitcoin/pull/14507
  68 2018-10-25T04:45:34  *** bitcoin-git has left #bitcoin-core-dev
  69 2018-10-25T04:47:43  *** jarthur has joined #bitcoin-core-dev
  70 2018-10-25T05:01:33  *** klot has joined #bitcoin-core-dev
  71 2018-10-25T05:02:29  *** klot has quit IRC
  72 2018-10-25T05:02:56  *** klot has joined #bitcoin-core-dev
  73 2018-10-25T05:03:59  *** klot has quit IRC
  74 2018-10-25T05:04:27  *** klot has joined #bitcoin-core-dev
  75 2018-10-25T05:07:42  *** murrayn has joined #bitcoin-core-dev
  76 2018-10-25T05:07:42  *** murrayn has joined #bitcoin-core-dev
  77 2018-10-25T05:47:51  *** bitconner has quit IRC
  78 2018-10-25T05:52:44  *** lnostdal has quit IRC
  79 2018-10-25T06:29:06  *** promag has joined #bitcoin-core-dev
  80 2018-10-25T06:31:44  *** jarthur has quit IRC
  81 2018-10-25T06:32:34  *** ap4lmtree- has quit IRC
  82 2018-10-25T06:33:01  *** ap4lmtree- has joined #bitcoin-core-dev
  83 2018-10-25T06:36:23  *** promag has quit IRC
  84 2018-10-25T06:52:48  *** morcos has quit IRC
  85 2018-10-25T06:53:03  *** morcos has joined #bitcoin-core-dev
  86 2018-10-25T06:59:54  *** Murch has joined #bitcoin-core-dev
  87 2018-10-25T07:03:44  *** setpill has joined #bitcoin-core-dev
  88 2018-10-25T07:03:49  *** ap4lmtree- has quit IRC
  89 2018-10-25T07:03:52  *** promag has joined #bitcoin-core-dev
  90 2018-10-25T07:04:03  *** Murch has quit IRC
  91 2018-10-25T07:04:17  *** ap4lmtree- has joined #bitcoin-core-dev
  92 2018-10-25T07:19:06  <sipa> meshcollider: cool
  93 2018-10-25T07:20:38  *** promag has quit IRC
  94 2018-10-25T07:24:44  *** timothy has joined #bitcoin-core-dev
  95 2018-10-25T07:32:33  <meshcollider> sipa: is there a way to ToString() on a specific index of a ranged descriptor to get a concrete derivation path? Or does ToString() always only return the ranged one
  96 2018-10-25T07:32:49  <meshcollider> looks like the latter
  97 2018-10-25T07:38:15  <meshcollider> and if not, would it be sensible for me to add it? Or is there a better approach im missing
  98 2018-10-25T07:44:15  *** bitconner has joined #bitcoin-core-dev
  99 2018-10-25T07:46:53  *** Hayro has joined #bitcoin-core-dev
 100 2018-10-25T07:48:51  *** bitconner has quit IRC
 101 2018-10-25T07:48:53  *** lnostdal has joined #bitcoin-core-dev
 102 2018-10-25T07:50:46  <Hayro> hello
 103 2018-10-25T07:52:02  *** Hayro has left #bitcoin-core-dev
 104 2018-10-25T07:56:39  <sipa> meshcollider: i thought about adding a way to 'specialize' a ranged descriptor to just a specific indix
 105 2018-10-25T07:57:11  <sipa> but then i realized that that would actually be duplicating #14477
 106 2018-10-25T07:57:14  <gribble> https://github.com/bitcoin/bitcoin/issues/14477 | Add ability to convert solvability info to descriptor by sipa · Pull Request #14477 · bitcoin/bitcoin · GitHub
 107 2018-10-25T07:57:17  <meshcollider> ah so you could call something like desc.Specialize(1).ToString()
 108 2018-10-25T07:58:02  <sipa> meshcollider: instead, you can expand a descriptor at a certain position, and then run InferDescriptor on the result
 109 2018-10-25T07:58:30  <sipa> and you'll get exactly the same as you'd get from such a Specialize
 110 2018-10-25T07:58:58  <sipa> i have a branch that uses that trick to add descriptors to scantxoutset's output
 111 2018-10-25T07:59:11  <meshcollider> oh, so like
 112 2018-10-25T07:59:11  <meshcollider> desc.Expand(1, sp, scripts, out);
 113 2018-10-25T07:59:11  <meshcollider> whatever = InferDescriptor(scripts[0], sp);
 114 2018-10-25T07:59:35  <meshcollider> ok ill take a look
 115 2018-10-25T07:59:40  <meshcollider> no PR for that yet?
 116 2018-10-25T08:00:05  <meshcollider> or are you waiting for 14477 to go in
 117 2018-10-25T08:02:16  <sipa> yeah, i was waiting for 14477, but it's a really small patch
 118 2018-10-25T08:02:21  <sipa> i can just add it i guess
 119 2018-10-25T08:04:59  *** luke-jr has quit IRC
 120 2018-10-25T08:05:08  *** luke-jr has joined #bitcoin-core-dev
 121 2018-10-25T08:14:31  *** BGL has quit IRC
 122 2018-10-25T08:16:11  *** jungly_ has joined #bitcoin-core-dev
 123 2018-10-25T08:19:26  *** Krellan has quit IRC
 124 2018-10-25T08:20:53  *** Krellan has joined #bitcoin-core-dev
 125 2018-10-25T08:43:31  *** proletesseract has joined #bitcoin-core-dev
 126 2018-10-25T08:44:29  *** chubao_ has joined #bitcoin-core-dev
 127 2018-10-25T08:44:58  *** drexl has quit IRC
 128 2018-10-25T09:01:55  *** anon777 has joined #bitcoin-core-dev
 129 2018-10-25T09:07:43  *** shesek has quit IRC
 130 2018-10-25T09:12:38  *** anon777 has quit IRC
 131 2018-10-25T09:14:07  *** owowo has quit IRC
 132 2018-10-25T09:39:58  *** owowo has joined #bitcoin-core-dev
 133 2018-10-25T09:40:00  *** BGL has joined #bitcoin-core-dev
 134 2018-10-25T09:46:57  *** ExtraCrispy has joined #bitcoin-core-dev
 135 2018-10-25T09:51:37  *** spinza has quit IRC
 136 2018-10-25T10:04:16  *** promag has joined #bitcoin-core-dev
 137 2018-10-25T10:12:32  *** dgenr8 has quit IRC
 138 2018-10-25T10:13:43  *** dgenr8 has joined #bitcoin-core-dev
 139 2018-10-25T10:16:04  *** shesek has joined #bitcoin-core-dev
 140 2018-10-25T10:19:31  *** spinza has joined #bitcoin-core-dev
 141 2018-10-25T10:21:18  *** AaronvanW has joined #bitcoin-core-dev
 142 2018-10-25T10:39:10  *** shesek has quit IRC
 143 2018-10-25T10:40:51  *** shesek has joined #bitcoin-core-dev
 144 2018-10-25T10:40:51  *** shesek has joined #bitcoin-core-dev
 145 2018-10-25T10:45:57  *** michaelfolkson has joined #bitcoin-core-dev
 146 2018-10-25T10:51:36  *** spinza has quit IRC
 147 2018-10-25T10:58:07  *** shesek has quit IRC
 148 2018-10-25T10:58:41  *** shesek has joined #bitcoin-core-dev
 149 2018-10-25T10:58:41  *** shesek has joined #bitcoin-core-dev
 150 2018-10-25T11:04:57  *** spinza has joined #bitcoin-core-dev
 151 2018-10-25T11:10:02  *** shesek has quit IRC
 152 2018-10-25T11:10:33  *** shesek has joined #bitcoin-core-dev
 153 2018-10-25T11:10:33  *** shesek has joined #bitcoin-core-dev
 154 2018-10-25T11:13:38  *** ap4lmtree- has quit IRC
 155 2018-10-25T11:14:03  *** ap4lmtree- has joined #bitcoin-core-dev
 156 2018-10-25T11:31:54  *** ap4lmtree- has quit IRC
 157 2018-10-25T11:33:45  <ken2812221> Gitian build for Windows is fail on master branch, I can confirm this with WSL. https://bitcoin.jonasschnelli.ch/build/858
 158 2018-10-25T11:44:05  <ken2812221> The problem seems to be 14451, revert this commit and the build works fine
 159 2018-10-25T12:01:38  *** klot has quit IRC
 160 2018-10-25T12:15:55  *** bitcoin-git has joined #bitcoin-core-dev
 161 2018-10-25T12:15:55  <bitcoin-git> [bitcoin] ken2812221 opened pull request #14568: build: Fix Qt link order for Windows build (master...win-qt-fix) https://github.com/bitcoin/bitcoin/pull/14568
 162 2018-10-25T12:15:55  *** bitcoin-git has left #bitcoin-core-dev
 163 2018-10-25T12:16:28  *** Victor_sueca has quit IRC
 164 2018-10-25T12:17:36  *** Victor_sueca has joined #bitcoin-core-dev
 165 2018-10-25T12:25:59  *** proletesseract has quit IRC
 166 2018-10-25T12:26:36  *** proletesseract has joined #bitcoin-core-dev
 167 2018-10-25T12:28:13  *** proletesseract has quit IRC
 168 2018-10-25T12:32:22  *** Krellan has quit IRC
 169 2018-10-25T12:33:43  *** Krellan has joined #bitcoin-core-dev
 170 2018-10-25T12:34:47  *** bitcoin-git has joined #bitcoin-core-dev
 171 2018-10-25T12:34:47  <bitcoin-git> [bitcoin] ken2812221 opened pull request #14569: travis: Print characters per 9 min to avoid timeout (master...travis-avoid-timeout) https://github.com/bitcoin/bitcoin/pull/14569
 172 2018-10-25T12:34:47  *** bitcoin-git has left #bitcoin-core-dev
 173 2018-10-25T12:35:26  *** IGHOR has quit IRC
 174 2018-10-25T12:36:48  *** IGHOR has joined #bitcoin-core-dev
 175 2018-10-25T12:47:52  *** emilengler has joined #bitcoin-core-dev
 176 2018-10-25T12:48:06  *** emilengler has left #bitcoin-core-dev
 177 2018-10-25T12:53:11  <promag> wumpus or MarcoFalke, please see #14561
 178 2018-10-25T12:53:14  <gribble> https://github.com/bitcoin/bitcoin/issues/14561 | Remove fs::relative call and fix listwalletdir tests by promag · Pull Request #14561 · bitcoin/bitcoin · GitHub
 179 2018-10-25T13:01:31  *** hebasto_ has quit IRC
 180 2018-10-25T13:07:00  <wumpus> I'm getting really lost in all the wallet directory stuff to be honest
 181 2018-10-25T13:15:07  <wumpus> didn't I look at another PR for exactly this shortly ago
 182 2018-10-25T13:15:31  <promag> it was closed and replaced with this
 183 2018-10-25T13:16:08  <promag> the other used path accessors and fs::equivalent which touches the filesystem
 184 2018-10-25T13:16:22  <promag> this one only drops the base string from the path string
 185 2018-10-25T13:17:46  <promag> and since we are recursively iterating walletdir there won't be errors, hence the assert() instead
 186 2018-10-25T13:19:16  *** bitcoin-git has joined #bitcoin-core-dev
 187 2018-10-25T13:19:16  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14571: [tests] Test that nodes respond to getdata with notfound (master...Mf1810-qaNotfound) https://github.com/bitcoin/bitcoin/pull/14571
 188 2018-10-25T13:19:16  *** bitcoin-git has left #bitcoin-core-dev
 189 2018-10-25T13:20:03  *** michaelfolkson has quit IRC
 190 2018-10-25T13:20:16  <wumpus> well, sure, in the context you could say that it never happens, but if you define an utility function I think you need to handle errors properly and not assert
 191 2018-10-25T13:21:05  <wumpus> or at the least add a comment and explain that the function will crash your program if the input isn't exactly as expected
 192 2018-10-25T13:21:27  <wumpus> otherwise, before you know it, someone will use it in network code or whatever and you have a DoS
 193 2018-10-25T13:22:19  *** merland has joined #bitcoin-core-dev
 194 2018-10-25T13:22:23  <wumpus> documenting assumptions is very important
 195 2018-10-25T13:25:21  *** shesek has quit IRC
 196 2018-10-25T13:26:17  *** shesek has joined #bitcoin-core-dev
 197 2018-10-25T13:26:17  *** shesek has joined #bitcoin-core-dev
 198 2018-10-25T13:28:19  <wumpus> and I think in general handling errors at the callsite (the decision can always be to crash) is better than crashing inside a helper function
 199 2018-10-25T13:29:20  * wumpus really appraciates rust's Option<> and Result<> types it's so refreshing after seeing years of broken error handling hacks in C-ish languages
 200 2018-10-25T13:32:13  *** proletesseract has joined #bitcoin-core-dev
 201 2018-10-25T13:32:16  *** setpill has quit IRC
 202 2018-10-25T13:32:41  *** proletesseract has quit IRC
 203 2018-10-25T13:32:46  *** michaelfolkson has joined #bitcoin-core-dev
 204 2018-10-25T13:37:12  <promag> wumpus: I can just inline the expression
 205 2018-10-25T13:37:32  <wumpus> yes
 206 2018-10-25T13:37:43  <wumpus> still, add a comment please
 207 2018-10-25T13:38:27  <wumpus> why the assert isn't randomly going to crash the program for some user at some point
 208 2018-10-25T13:43:23  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 209 2018-10-25T13:46:03  *** bitcoin-git has joined #bitcoin-core-dev
 210 2018-10-25T13:46:04  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/613fc95ee4ea...754a00d55f30
 211 2018-10-25T13:46:05  <bitcoin-git> bitcoin/master 43719e0 Jonas Schnelli: [macOS] Remove DS_Store WindowBounds bytes object
 212 2018-10-25T13:46:05  <bitcoin-git> bitcoin/master 754a00d Wladimir J. van der Laan: Merge #14416: Fix OSX dmg issue (10.12 to 10.14)...
 213 2018-10-25T13:46:06  *** bitcoin-git has left #bitcoin-core-dev
 214 2018-10-25T13:47:38  *** bitcoin-git has joined #bitcoin-core-dev
 215 2018-10-25T13:47:39  <bitcoin-git> [bitcoin] laanwj closed pull request #14416: Fix OSX dmg issue (10.12 to 10.14) (master...2018/10/osx_dmg) https://github.com/bitcoin/bitcoin/pull/14416
 216 2018-10-25T13:47:39  *** bitcoin-git has left #bitcoin-core-dev
 217 2018-10-25T13:49:21  <promag> wumpus: let me know if https://github.com/bitcoin/bitcoin/pull/14561/files lgty
 218 2018-10-25T13:50:06  *** bitcoin-git has joined #bitcoin-core-dev
 219 2018-10-25T13:50:07  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.17: https://github.com/bitcoin/bitcoin/commit/eb2cc84a31fb923b2b25b979682904cb81edec7e
 220 2018-10-25T13:50:07  <bitcoin-git> bitcoin/0.17 eb2cc84 Jonas Schnelli: [macOS] Remove DS_Store WindowBounds bytes object...
 221 2018-10-25T13:50:07  *** bitcoin-git has left #bitcoin-core-dev
 222 2018-10-25T13:51:03  <wumpus> promag: yes lgtm now!
 223 2018-10-25T13:51:16  <promag> can I squash?
 224 2018-10-25T13:51:58  <promag> btw, what is preventing from bumping boost? old lts?
 225 2018-10-25T13:54:32  <wumpus> very simply: there is no good reason to
 226 2018-10-25T13:54:36  *** Chris_Stewart_5 has quit IRC
 227 2018-10-25T13:54:44  <wumpus> nothing is *preventing* it but that's not the point, a change needs to be driven by a good reason
 228 2018-10-25T13:55:11  <wumpus> if we can support old boost versions why not? why give users/developers unnecessary woes?
 229 2018-10-25T13:55:17  *** shesek has quit IRC
 230 2018-10-25T13:55:51  <wumpus> ideally we can go with this version until boost dependency can be removed completely
 231 2018-10-25T13:56:04  <wumpus> we don't really *want* to use anything new from boost
 232 2018-10-25T13:56:34  *** shesek has joined #bitcoin-core-dev
 233 2018-10-25T13:57:13  <luke-jr> +1
 234 2018-10-25T13:57:39  <luke-jr> frankly, I think we bumped univalue too prematurely (and ended up with an unnecessary/unreasonable fork in bitcoin/univalue as a result)
 235 2018-10-25T13:58:39  <promag> got it +1
 236 2018-10-25T13:59:14  <wumpus> at least univalue was already part of our own repository, we've more or less took up maintenance
 237 2018-10-25T13:59:35  <luke-jr> except it wasn't
 238 2018-10-25T13:59:58  <luke-jr> unless you mean the embedded copy, which should really be removed
 239 2018-10-25T14:00:33  <luke-jr> there's no justification for forking or embedding univalue, unlike leveldb
 240 2018-10-25T14:10:13  <wumpus> I really, really don't feel like having this discussion, sorry
 241 2018-10-25T14:10:23  <luke-jr> probably not worth the time, hence the status quo
 242 2018-10-25T14:11:55  <wumpus> travis failing on both 0.17 and master, ahhh
 243 2018-10-25T14:12:40  <luke-jr> :/
 244 2018-10-25T14:13:24  <promag> "Remote end closed connection without response"
 245 2018-10-25T14:14:39  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 246 2018-10-25T14:15:28  *** michaelfolkson has quit IRC
 247 2018-10-25T14:17:02  <wumpus> promag: is that one of the travis failures?
 248 2018-10-25T14:17:11  *** Tralfaz has quit IRC
 249 2018-10-25T14:17:12  <promag> yes, the last
 250 2018-10-25T14:17:42  <promag> https://travis-ci.org/bitcoin/bitcoin/jobs/446182728#L2801
 251 2018-10-25T14:19:13  *** drexl has joined #bitcoin-core-dev
 252 2018-10-25T14:19:16  *** shesek has quit IRC
 253 2018-10-25T14:19:50  *** shesek has joined #bitcoin-core-dev
 254 2018-10-25T14:19:50  *** shesek has joined #bitcoin-core-dev
 255 2018-10-25T14:19:51  <wumpus> the 0.17 issue is a linter issue?!? https://travis-ci.org/bitcoin/bitcoin/jobs/446184576
 256 2018-10-25T14:20:04  <wumpus> how can there suddenly be so many linter problems
 257 2018-10-25T14:21:41  <promag> wumpus: new version?
 258 2018-10-25T14:21:48  *** andytosh1 has quit IRC
 259 2018-10-25T14:21:49  *** andytosh1 has joined #bitcoin-core-dev
 260 2018-10-25T14:21:54  *** andytosh1 is now known as andytoshi
 261 2018-10-25T14:22:24  <promag> does 0.17 locks flake8?
 262 2018-10-25T14:24:18  <promag> no it doesn't
 263 2018-10-25T14:25:29  <promag> https://github.com/bitcoin/bitcoin/blob/0.17/.travis.yml#L151 should be `travis_retry pip install flake8==3.5.0`
 264 2018-10-25T14:26:27  <karelb> not sure if it belongs here - when I read this https://bitcoinops.org/en/newsletters/2018/10/23/ and the issues around remote RPCs, I realized this might be a problem if you run a browser on the same PC
 265 2018-10-25T14:26:32  <karelb> since browsers now have localhost as a trusted origin, so you can connect to HTTP (without SSL) from any website
 266 2018-10-25T14:26:37  <karelb> Evil website can try to guess bitcoind credentials and you have the same problems, that the article describes
 267 2018-10-25T14:27:01  <sdaftuar> is there any reason to run a linter on an old branch?
 268 2018-10-25T14:28:02  *** shesek has quit IRC
 269 2018-10-25T14:28:21  *** shesek has joined #bitcoin-core-dev
 270 2018-10-25T14:28:21  *** shesek has joined #bitcoin-core-dev
 271 2018-10-25T14:31:07  <promag> sdaftuar: keep consistency on backports?
 272 2018-10-25T14:31:57  <wumpus> karelb: it's a drawback of using a tcp port for RPC I suppose, let alone http
 273 2018-10-25T14:33:00  <karelb> wumpus: maybe httpserver.cpp should check an origin header and not allow cross-origin browser requests? (or whatever header browsers add)
 274 2018-10-25T14:33:45  *** shesek has quit IRC
 275 2018-10-25T14:33:47  <wumpus> that might be a good precaution
 276 2018-10-25T14:33:49  <karelb> I haven't actually tried that, maybe it won't work
 277 2018-10-25T14:34:00  <wumpus> but only if you're sure this is actually a threat
 278 2018-10-25T14:34:10  <karelb> (I mean I havenmaking a request from w
 279 2018-10-25T14:34:28  *** shesek has joined #bitcoin-core-dev
 280 2018-10-25T14:34:28  <karelb> *I haven't tried making a request from a website
 281 2018-10-25T14:35:15  <karelb> it's a similar threat as connecting from a different IP, no?
 282 2018-10-25T14:36:15  <karelb> *IF* it actually work :)
 283 2018-10-25T14:36:34  <wumpus> so are you sure browsers allow making json-rpc requests to localhost? this didn't use to be the case at least
 284 2018-10-25T14:38:43  <karelb> wumpus: no, I am not sure, I just don't see why it would't work. Since it is just a http GET request.
 285 2018-10-25T14:38:58  <wumpus> it requires a *post* request
 286 2018-10-25T14:39:08  <promag> I think it's possible
 287 2018-10-25T14:39:43  <wumpus> submitting a JSON-RPC command through get is impossibl
 288 2018-10-25T14:39:55  *** shesek has quit IRC
 289 2018-10-25T14:40:41  *** shesek has joined #bitcoin-core-dev
 290 2018-10-25T14:41:19  <karelb> wumpus: what recently changed is that browsers allow connection to http from https websites, if the url is "localhost" (or "localost:port"). It is special-cased. Normally, you cannot make a request to http from https website. It's quite new (1 year, 2 years-ish)
 291 2018-10-25T14:41:19  *** Krellan has quit IRC
 292 2018-10-25T14:41:29  <queip> karelb: browsers just trust 127.0.0.1? so any JS on any site could talk to say https://127.0.0.1:7657 or such? that would be a critical vulnerability to many services that have localhost admin panels (with no password) no?
 293 2018-10-25T14:41:30  <promag> this should be possible: fetch(url, { method: 'post', body: ... })
 294 2018-10-25T14:42:12  *** Krellan has joined #bitcoin-core-dev
 295 2018-10-25T14:42:13  <queip> karelb: is this really how it works? this means i2p, and many web-panel based servers are compromised now
 296 2018-10-25T14:42:50  <karelb> well the service might restrict origin, or cross requests in general
 297 2018-10-25T14:43:23  <promag> afak with the right cors headers it is possible
 298 2018-10-25T14:43:26  <karelb> browsers behave well, they send the origin in some header
 299 2018-10-25T14:43:32  <queip> I bet most do not... seems like idiotic move by browsers?
 300 2018-10-25T14:43:54  *** shesek has quit IRC
 301 2018-10-25T14:43:56  *** michaelsdunn1 has joined #bitcoin-core-dev
 302 2018-10-25T14:43:56  *** michaelsdunn1 has quit IRC
 303 2018-10-25T14:43:56  *** michaelsdunn1 has joined #bitcoin-core-dev
 304 2018-10-25T14:44:25  *** shesek has joined #bitcoin-core-dev
 305 2018-10-25T14:44:25  *** shesek has joined #bitcoin-core-dev
 306 2018-10-25T14:45:29  <karelb> see https://www.chromestatus.com/feature/6269417340010496
 307 2018-10-25T14:47:48  *** Chris_Stewart_5 has quit IRC
 308 2018-10-25T14:48:01  <karelb> but it's solving the "inverse" problem - if a website can trust what it is fetching
 309 2018-10-25T14:48:43  <karelb> anyway I will just try to hack something to test it, better than talk :D
 310 2018-10-25T14:53:13  <queip> I have an issue with the github merge script, used in another (sort of private temp, but opensource overall) project...    In one case (out of hundreds times using it with bo problem) now  git diff HEAD~  in the merge shell shows nothing
 311 2018-10-25T14:53:16  *** shesek has quit IRC
 312 2018-10-25T14:53:39  *** shesek has joined #bitcoin-core-dev
 313 2018-10-25T14:53:47  <queip> any idea what it might be?  or, someone wants to look at the git with me?  it might be some bug in the tool (or this time out of 1000+ somehow we're using it wrong)
 314 2018-10-25T14:54:28  *** tintin has joined #bitcoin-core-dev
 315 2018-10-25T14:54:58  <instagibbs> wumpus, pyflake major version update in a minor flake8 update :shrug:
 316 2018-10-25T14:55:39  <wumpus> instagibbs: :shrug: typical
 317 2018-10-25T14:56:19  <instagibbs> we filed a complaint on their gitlab to at least have release notes
 318 2018-10-25T14:57:59  *** jarthur has joined #bitcoin-core-dev
 319 2018-10-25T15:00:59  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 320 2018-10-25T15:01:28  <queip> wumpus: wouldn't you be interested by any chance to look at this git problem, which miiight be a problem in github merge script? I can't figure out why  diff HEAD~ is empty... if no one is interested will just merge it, so might be not able to reproduce that later
 321 2018-10-25T15:02:12  <karelb> OK I panicked too soon; browser does a CORS request and it fails, so it won't continue to connect further.
 322 2018-10-25T15:02:30  <karelb> `Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://localhost:18444/. (Reason: CORS request did not succeed).`
 323 2018-10-25T15:02:49  <wumpus> queip: if git diff HEAD~ shows nothing, you have an empty commit
 324 2018-10-25T15:03:04  <karelb> and the website cannot even distinguish whether it's because the server is not running or whether it is not existent. So all it OK.
 325 2018-10-25T15:03:17  <wumpus> karelb: good to know; thank you for trying
 326 2018-10-25T15:04:23  <queip> wumpus: git log -p (int githubmerge.py shell) does indeed show various commits, that are not present on the PR-target branch normally, this is not a "doing nothing" MR.   or did you ment that ONE of commits there might be empty and this triggers such reaction
 327 2018-10-25T15:04:45  <karelb> (maybe it would be a good feature to allow that, to allow websites that interact with full node :D but that is beyond the scope)
 328 2018-10-25T15:04:59  <queip> karelb: :]
 329 2018-10-25T15:05:37  <wumpus> karelb: that's going to be very firmly rejected, I expect
 330 2018-10-25T15:05:58  *** bitcoin-git has joined #bitcoin-core-dev
 331 2018-10-25T15:05:58  <bitcoin-git> [bitcoin] promag opened pull request #14573: qt: Add Wallet and Window menus (master...2018-10-overhaul-menus) https://github.com/bitcoin/bitcoin/pull/14573
 332 2018-10-25T15:05:58  *** bitcoin-git has left #bitcoin-core-dev
 333 2018-10-25T15:07:08  <wumpus> queip: which repository/PR is this?
 334 2018-10-25T15:07:49  <karelb> you *could* whitelist, as a user, the websites that can connect to the full node. and the user would still need to give the website his name/password/cookie explicitly. so it would not compromise security. :P *but* I see what you mean :D
 335 2018-10-25T15:08:47  <wumpus> yes, but no way
 336 2018-10-25T15:10:33  <karelb> I see. :))
 337 2018-10-25T15:11:58  *** jarthur has quit IRC
 338 2018-10-25T15:12:25  <karelb> well, you can always write the web app in Electron, that ignore the CORS requests and can grab the cookie directly from the filesystem. Yum, Electron. OK, really out of scope now. :)
 339 2018-10-25T15:14:46  *** tintin has quit IRC
 340 2018-10-25T15:16:22  <wumpus> this is getting too scary
 341 2018-10-25T15:17:12  *** Chris_Stewart_5 has quit IRC
 342 2018-10-25T15:17:59  <karelb> javascript world is scary
 343 2018-10-25T15:18:09  <wumpus> don't let it bleed into here
 344 2018-10-25T15:22:19  *** merland has quit IRC
 345 2018-10-25T15:26:28  *** shesek has quit IRC
 346 2018-10-25T15:26:55  *** shesek has joined #bitcoin-core-dev
 347 2018-10-25T15:26:55  *** shesek has joined #bitcoin-core-dev
 348 2018-10-25T15:27:13  *** crazyprodigy has joined #bitcoin-core-dev
 349 2018-10-25T15:27:16  *** shesek has quit IRC
 350 2018-10-25T15:27:32  <Sentineo> hn
 351 2018-10-25T15:27:59  <Sentineo> but someoone might try to give you a browser which does not check CORS and connect to your bitcoin through rpc?
 352 2018-10-25T15:28:11  <Sentineo> hopefully will not happen :)
 353 2018-10-25T15:28:19  *** shesek has joined #bitcoin-core-dev
 354 2018-10-25T15:32:29  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 355 2018-10-25T15:33:08  <karelb> once he already gave you his own binary on your pc, he can do worse...
 356 2018-10-25T15:33:24  *** crazyprodigy is now known as nerdy_panda
 357 2018-10-25T15:33:48  *** ExtraCrispy has quit IRC
 358 2018-10-25T15:35:21  *** shesek has quit IRC
 359 2018-10-25T15:36:05  *** shesek has joined #bitcoin-core-dev
 360 2018-10-25T15:36:05  *** shesek has joined #bitcoin-core-dev
 361 2018-10-25T15:39:29  *** nerdy_panda has left #bitcoin-core-dev
 362 2018-10-25T15:40:32  *** rex4539 has joined #bitcoin-core-dev
 363 2018-10-25T15:48:55  *** shesek has quit IRC
 364 2018-10-25T15:49:36  *** shesek has joined #bitcoin-core-dev
 365 2018-10-25T15:50:00  <wumpus> if someone 'gives you a browser' that is already the trojan horse scenario
 366 2018-10-25T15:51:01  *** rh0nj has quit IRC
 367 2018-10-25T15:52:07  *** rh0nj has joined #bitcoin-core-dev
 368 2018-10-25T15:52:28  <wumpus> I think what discourages people from even trying browser-based attacks is that bitcoin-qt by default has the RPC server disabled, so most unknowning users won't be affected, the people that enable RPC will generally be more technical and hopefully be careful what they allow
 369 2018-10-25T15:54:52  *** shesek has quit IRC
 370 2018-10-25T15:55:27  *** shesek has joined #bitcoin-core-dev
 371 2018-10-25T15:55:27  *** shesek has joined #bitcoin-core-dev
 372 2018-10-25T15:58:09  *** jarthur has joined #bitcoin-core-dev
 373 2018-10-25T16:04:52  *** ExtraCrispy has joined #bitcoin-core-dev
 374 2018-10-25T16:05:22  <harding> wumpus: someone in #bitcoin the other day said that RPC was enabled by default on Windows in bitcoin-qt.  That surprised me, as I didn't even know bitcoin-qt could expose RPC (I thought you needed to use bitcoind for that).  I didn't think to mention it here because I didn't realize that not running RPC with bitcoin-qt was supposed to be a security feature.
 375 2018-10-25T16:05:51  *** shesek has quit IRC
 376 2018-10-25T16:06:14  <luke-jr> harding: RPC is intentionally enabled by default *on localhost*, and disabling it wouldn't provide any real security improvement I can think of
 377 2018-10-25T16:06:20  <achow101> harding: it's supposed to only be enabled in qt if you have -server=1
 378 2018-10-25T16:06:40  <wumpus> harding: setting server=1 in bitcoin.conf should be the only way to enable it with bitcoin-qt
 379 2018-10-25T16:06:46  <luke-jr> I thought that changed with RPC cookies?
 380 2018-10-25T16:06:59  <wumpus> no? not that I know, let's check
 381 2018-10-25T16:07:11  <gmaxwell> I think we discussed that but didn't do it.
 382 2018-10-25T16:07:13  <achow101> i don't think so
 383 2018-10-25T16:07:44  <gmaxwell> wumpus: re 'gives you a brower'  more like "this site only works in IE 6"  (which has broken cross site requrest handing...)
 384 2018-10-25T16:07:50  *** ppisati has joined #bitcoin-core-dev
 385 2018-10-25T16:09:20  <gmaxwell> ethereum stuff has been raided over and over again with browser based attacks.
 386 2018-10-25T16:10:24  *** shesek has joined #bitcoin-core-dev
 387 2018-10-25T16:11:00  <harding> Looked up the conversation on #bitcoin, the user didn't explicitly say that he was using the default config, so he could've had -server=1.
 388 2018-10-25T16:11:27  <harding> I did test on Linux, and bitcoin-qt doesn't do RPC there by default.
 389 2018-10-25T16:11:55  <gmaxwell> I don't think we really should be counting on the default config to protect people... lots of people copy and paste configs.
 390 2018-10-25T16:12:33  <gmaxwell> echeveria crawled the internet and found something like 3000 bitcoin rpc ports listening, which would be a substantial percentage of all p2p listening nodes.
 391 2018-10-25T16:12:39  <wumpus> I'm not *counting* on anything, I was just mentioning that it's not a problem with the default confi
 392 2018-10-25T16:12:45  <harding> gmaxwell: I crawled too and only found 1,100.
 393 2018-10-25T16:13:08  <wumpus> it shouldn't be a problem with RPC listening on localhost either, it's just defense in depth I suppose..
 394 2018-10-25T16:13:17  <wumpus> if you don't need a RPC server it's better to disable it
 395 2018-10-25T16:13:29  <wumpus> even if there is no way you can imagine it can be exploited
 396 2018-10-25T16:13:32  *** shesek has quit IRC
 397 2018-10-25T16:14:57  *** shesek has joined #bitcoin-core-dev
 398 2018-10-25T16:15:26  <wumpus> in any case I checked: bitcoin-qt still doesn't enable RPC server by default
 399 2018-10-25T16:15:39  <harding> wumpus: thanks.  Sorry for the false alarm.
 400 2018-10-25T16:15:44  <gmaxwell> harding: so 10% instead of 30%. :P
 401 2018-10-25T16:15:54  <harding> gmaxwell: yeah, it was 13% of my sample.
 402 2018-10-25T16:15:57  *** shesek has quit IRC
 403 2018-10-25T16:17:58  *** jungly_ has quit IRC
 404 2018-10-25T16:18:48  *** shesek has joined #bitcoin-core-dev
 405 2018-10-25T16:19:11  <phantomcircuit> harding, that is significant
 406 2018-10-25T16:19:29  <harding> phantomcircuit: I agree.
 407 2018-10-25T16:21:15  <harding> phantomcircuit: I wish I knew the cause.  The only instructions I've seen for enabling it were from a particular mobile wallet, but I have a hard time believing there are over 1,000 mobile wallet users who also run a full node and also opened RPC to use them together.
 408 2018-10-25T16:22:02  <queip> wumpus: the problem is in PR https://github.com/userghmrt/testrepo/pull/3 . Although on unpatched githubmerge.py it's needed to replace/steart  def tree_sha512sum() function with return "0" (because repo uses symlinks)  .  I confirmed, there in github on that issue 3  git dif HEAD~ is empty evne though on GH page "Files changed: 1"
 409 2018-10-25T16:22:52  <queip> btw you have invite to edit that test repo if you need to test
 410 2018-10-25T16:23:09  <gmaxwell> harding: I think there are a lot of example config files floating around.
 411 2018-10-25T16:23:41  <phantomcircuit> harding, the cause is pretty clearly people being told they need a specific config
 412 2018-10-25T16:23:54  <gmaxwell> I know from past expirence that joe-user when faced with discovering they need a config file, go and paste some example. When I've asked users for their configs, they have in the past frequently given me ones copy and pasted from the wiki.
 413 2018-10-25T16:24:11  <harding> gmaxwell, phantomcircuit: that seems likely.
 414 2018-10-25T16:24:28  <queip> btw, we've patched github merge to support symlinks, submodules (incluees them in tree hash) and gitlab, if anyone wants at some point
 415 2018-10-25T16:25:22  <harding> I don't know what the solution for that is, though.  Better documentation and sample configs provided from an official source?
 416 2018-10-25T16:28:20  <gmaxwell> harding: probably making bitcoin make its own template config file when one doesn't exist would help.
 417 2018-10-25T16:28:27  <harding> I guess when .bitcoin is created, a default bitcoin.conf could be created with some normal-to-change options and basic comment documentation.  I think you'd want to keep it short, rather than providing every possible option, so that people aren't tempted to delete the whole thing and replace it with a random config from the Wiki.
 418 2018-10-25T16:29:31  <gmaxwell> I had assumed that problem had gotten less bad because of cookies -- you don't need to make a config file to make bitcoind usable at all... it may be that your 1k rpc listeners have a lot of nodes that came up before cookies existed.
 419 2018-10-25T16:31:14  <harding> gmaxwell: interesting thought, and something that seems not to hard to check---for an open port 8332, get the node version for port 8333.
 420 2018-10-25T16:32:55  <gmaxwell> I mean they could have installed 0.11 or whatever and since upgraded but already have a config.
 421 2018-10-25T16:33:44  <harding> gmaxwell: oh.
 422 2018-10-25T16:33:55  <gmaxwell> So this would be another advantage in adding an additonal config option that needs to be set to listen to the internet.
 423 2018-10-25T16:34:04  <harding> Yeah.
 424 2018-10-25T16:35:29  <luke-jr> my PR kindof does that
 425 2018-10-25T16:35:37  <luke-jr> (they have to specify rpcbind explicitly)
 426 2018-10-25T16:37:50  <wumpus> solution: delete all RPC binding functionality, switch to UNIX sockets
 427 2018-10-25T16:38:12  <luke-jr> that doesn't work on Windows
 428 2018-10-25T16:38:14  <wumpus> like c-lightning
 429 2018-10-25T16:38:23  <wumpus> windows has local sockets as well (called differently) IIRC
 430 2018-10-25T16:38:46  <gmaxwell> wumpus: would be nice, but perhaps too hard to get random software stacks to speak domain sockets.
 431 2018-10-25T16:38:47  <wumpus> heck, windows 10 even has actual UNIX sockets
 432 2018-10-25T16:39:15  <gmaxwell> luke-jr: Did you consider doing something more explicit?  e.g. making an option called enable-insecurely-exposing-rpc-to-network=1?
 433 2018-10-25T16:39:19  <wumpus> gmaxwell: I know right! should have done that from the start like c-lightning :-(
 434 2018-10-25T16:39:34  <wumpus> so many things are imposslbe to change now
 435 2018-10-25T16:39:48  <wumpus> queip: ok I'll have a look
 436 2018-10-25T16:39:49  <luke-jr> gmaxwell: no. it's not necessarily insecure, if it's a private LAN
 437 2018-10-25T16:40:04  <luke-jr> wumpus: can normal Windows software bind to UNIX sockets?
 438 2018-10-25T16:40:18  <gmaxwell> luke-jr: enable-potentially-inscure-rpc-network-exposure?  the point being so you can't just copy and paste without getting a warning.
 439 2018-10-25T16:40:28  <wumpus> luke-jr: yes I think so
 440 2018-10-25T16:41:01  <gmaxwell> wumpus: even there, my concern isn't so much intertia-- if it was inertia I think we could just do it and include some shim utility... but like, can nodejs applications ever speak to domain sockets?
 441 2018-10-25T16:41:15  <luke-jr> gmaxwell: might be worth considering. I expect we'll find other problematic copy-and-paste configs though..
 442 2018-10-25T16:41:22  <wumpus> we don't even have UNIX support for RPC yet, let alone could set it as only option :(
 443 2018-10-25T16:41:28  *** OzPac has joined #bitcoin-core-dev
 444 2018-10-25T16:41:47  <gmaxwell> I think it's okay if for good reason we introduce some incompatiblity in a major version, esp if we give people a long time to switch first...  but it wouldn't be good if there is no easy way to become compatible with it. :)
 445 2018-10-25T16:42:06  <gmaxwell> wumpus: indeed. well we certantly could do that.
 446 2018-10-25T16:42:06  <wumpus> I'm sure nodejs can use any system API, it's a full environment for server software
 447 2018-10-25T16:42:20  <gmaxwell> luke-jr: the worst is rpcpassword, but hopefully cookies reduced that.
 448 2018-10-25T16:42:28  <wumpus> it might be in javascript but it's not *that* much of a joke
 449 2018-10-25T16:42:46  <queip> would it be possible to tell users what is the problem, other than generic 403 message?  like, tell them the option to add but warn them of consequences.  or users will complain that "it stopped working after update"
 450 2018-10-25T16:42:47  *** Chris_Stewart_5 has quit IRC
 451 2018-10-25T16:43:12  <wumpus> you can send any message with the 403 status, even have a full error html document
 452 2018-10-25T16:43:14  *** shesek has quit IRC
 453 2018-10-25T16:43:26  <queip> actually it will not even listen on it to reply 403, just no connection by luke's default?
 454 2018-10-25T16:43:34  <gmaxwell> Brendan Eich would say never underestimate javascript... which I always took to mean never underestimate the ability of something javascript to be a joke.
 455 2018-10-25T16:43:42  <wumpus> though I don't think *the latter* is a good idea because software will try to parse text/html replies as json
 456 2018-10-25T16:43:52  <wumpus> gmaxwell: +1
 457 2018-10-25T16:44:33  <wumpus> a lot of software uses local pipes for RPC mechanism
 458 2018-10-25T16:44:39  <wumpus> including databases, which nodejs people love
 459 2018-10-25T16:45:20  <gmaxwell> queip: sure, our normal practice for depricating RPCs is to make them return errors and have an option to reenable them for one version.  So the normal sequence we use where possible is (1) support the new thing (2) announce the old thing is going away [time passes] (3) disable by default the old thing but with a switch to override [time passes] (4) take out the old thing.
 460 2018-10-25T16:46:09  <gmaxwell> So I think we could change this assuming things actually could talk to it. I  don't know that we need to (like adding increasingly agressive warnings against enabling rpc, shifting more stuff onto domain sockets, etc may be enough...
 461 2018-10-25T16:47:44  <wumpus> FWIW ssh, and I think stunnel, allow forwarding a UNIX domain socket, so it doesn't even make it impossible to expose the RPC over the network, just requires actual setup (and thinking about security)
 462 2018-10-25T16:48:38  <gmaxwell> if stunnel can do it then that even gives us a backwards compt method if we ever drop TCP support.
 463 2018-10-25T16:49:51  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 464 2018-10-25T16:49:51  <wumpus> queip: yes, can you temporarily give me acces to the repo? I can work around it, as I don't actually need to push anything to test, but it's easier
 465 2018-10-25T16:51:57  <wumpus> just checked: yes, stunnel supports a UNIX socket as destination, but only on UNIX; nginx also supports forwarding requests to UNIX socket based http servers
 466 2018-10-25T16:51:58  *** Krellan has quit IRC
 467 2018-10-25T16:52:29  <wumpus> queip: oh nm I have an invite
 468 2018-10-25T16:52:55  <jarthur> wumpus: heard any word from libevent crew on your pre-existing-fd PR?
 469 2018-10-25T16:53:00  *** Krellan has joined #bitcoin-core-dev
 470 2018-10-25T16:55:33  <wumpus> jarthur: they wanted a different solution at the time; which was above my head, certainly, don't know if that ever made any progress
 471 2018-10-25T16:56:22  <wumpus> the only functionality I needed was to inject an existing fd into the http client, same as they allow for the server binding
 472 2018-10-25T16:56:57  <esotericnonsense> hm
 473 2018-10-25T16:57:06  <esotericnonsense> i've just jumped in here and stuck some chat in #bitcoin
 474 2018-10-25T16:57:23  <esotericnonsense> yes, CORS should (on behaving browsers) prevent requests to localhost and also prevent requests on different ports
 475 2018-10-25T16:58:14  <esotericnonsense> so _even if_ you have some service running on localhost:10080 say, and you're at http://localhost:10080, you shouldn't hit it (that is if CORS is still enabled for http, i don't really use bare http sites, but I'd assume why not)
 476 2018-10-25T16:58:58  <esotericnonsense> a bad browser can just ignore CORS but then that's just equivalent to having an insecure system, you're running vulnerable code
 477 2018-10-25T17:01:02  <esotericnonsense> personally i'd probably ask why bitcoin-qt even has the ability to enable RPC
 478 2018-10-25T17:01:11  <esotericnonsense> it feels like a 'nice to have' footgun
 479 2018-10-25T17:01:41  <esotericnonsense> is anyone actually seriously running the GUI client as their backend
 480 2018-10-25T17:01:48  <wumpus> what, if you want to enable RPC in bitcoin-qt you should be able to
 481 2018-10-25T17:02:30  <esotericnonsense> what's the use case?
 482 2018-10-25T17:02:35  <wumpus> I don't believe in making things hard to do because there are a few people that do stupid stuff with it
 483 2018-10-25T17:02:36  <gmaxwell> esotericnonsense: it enables you to do things like use joinmarket.
 484 2018-10-25T17:02:43  <luke-jr> esotericnonsense: doing stuff from the commandline.. or third party apps
 485 2018-10-25T17:03:09  <luke-jr> eg, I think many people probably use it for their taxes
 486 2018-10-25T17:03:09  <wumpus> punishing power users for the mistakes of others
 487 2018-10-25T17:03:16  <gmaxwell> If there were litterally no usecase we could come up with then I would probably agree with esotericnonsense's point, but there are many. :P
 488 2018-10-25T17:03:25  <wumpus> I have various scripts that interface to bitcoin and I use them with the GUI too.
 489 2018-10-25T17:03:51  <esotericnonsense> i suppose this is just part of my general 'why is the gui still linked in' grumble but then I stopped working on it so can't complain :P
 490 2018-10-25T17:03:56  <wumpus> sigh...
 491 2018-10-25T17:04:23  <wumpus> I'll just shut up, I end up disagreeing with everyone on everything anyway
 492 2018-10-25T17:04:24  <luke-jr> if GUI had to access over RPC, then RPC would become mandatory for GUI users ;)
 493 2018-10-25T17:04:31  <esotericnonsense> (not that it would help especially I guess since someone copying a random config could just as easily copy a random config in to their bitcoind instance)
 494 2018-10-25T17:05:25  <wumpus> if it's any guide, even professional stock trading software has a way to enable a RPC interface in their GUI; there's simply users that want to control a program programmatically even if it's a GUI program
 495 2018-10-25T17:05:28  <sipa> wumpus: please don't shut up :)
 496 2018-10-25T17:05:45  <wumpus> in some operating systems there is hardly anything *but* GUI programs
 497 2018-10-25T17:06:06  <esotericnonsense> luke-jr: sure but it wouldn't have to be tcp. i don't know enough about cross platform sockets to comment properly though. the default could be that the bitcoind and qt processes are started with a user that sets permissions on the socket
 498 2018-10-25T17:06:18  <esotericnonsense> but that's just unix, i've no idea how this works on win
 499 2018-10-25T17:06:18  <jarthur> I run the GUI client as my backend all the time, though am probably a "power user". Same with Trader Workstation, as wumpus alludes.
 500 2018-10-25T17:06:20  <wumpus> you might have completely the wrong idea of what people use a GUI, it's not only clueless people
 501 2018-10-25T17:06:33  <esotericnonsense> wumpus: I don't think it's the case that only clueless people use a GUI
 502 2018-10-25T17:06:39  <esotericnonsense> at all, sorry if it seemed that way
 503 2018-10-25T17:06:51  * harding loves the GUI, but also loves CLIs for almost everything else
 504 2018-10-25T17:07:12  <esotericnonsense> more that it seemed using the GUI as the GUI, and the backend as the backend makes sense, but of course you currently can't do that, so I'm just ranting, basically :P
 505 2018-10-25T17:08:09  <esotericnonsense> if this is actually a problem, could there be a warning sign or something in the corner of the gui that says 'rpc is enabled, did you know?'? (maybe it already exists)
 506 2018-10-25T17:09:05  *** Squidicc has joined #bitcoin-core-dev
 507 2018-10-25T17:09:26  *** dqx has joined #bitcoin-core-dev
 508 2018-10-25T17:10:07  <wumpus> esotericnonsense: I think that's a good suggestion, no that doesn't exist, feel free to make an issue!
 509 2018-10-25T17:10:07  *** Squidicuz has quit IRC
 510 2018-10-25T17:10:19  <wumpus> could certainly have an icon for that
 511 2018-10-25T17:10:20  *** Squidicc is now known as squidicuz
 512 2018-10-25T17:10:28  <luke-jr> IMO sensible defaults + education is the solution for things like this
 513 2018-10-25T17:11:07  <esotericnonsense> i've just sort of popped in and out on this but i've seen the masses of open RPC ports mentioned a bit, do we have any idea why this is the case?
 514 2018-10-25T17:11:21  * esotericnonsense should make a PR for an alert RPC
 515 2018-10-25T17:11:53  <harding> esotericnonsense: the guess above was that people are copy/pasting configs that do things they don't necessarily need.
 516 2018-10-25T17:11:54  <esotericnonsense> connect to all of them and say 'oi, you should probably not do this, especially with password:password' :P
 517 2018-10-25T17:14:46  <wumpus> 'alert' RPC hehe
 518 2018-10-25T17:16:20  <wumpus> but I think a good point is that having RPC listening is currently effectively hidden to the user, it's also not configurable from the GUI afaik, only by editing the conf
 519 2018-10-25T17:17:07  <luke-jr> maybe RPC should accept an 'alert' RPC without password :P
 520 2018-10-25T17:17:15  <esotericnonsense> luke-jr: that was my thought
 521 2018-10-25T17:17:29  <wumpus> uhm no
 522 2018-10-25T17:17:31  <esotericnonsense> i mean if you wanted to get really clever with this
 523 2018-10-25T17:17:50  <esotericnonsense> you could build in to the p2p network that clients attempt to connect to each others' RPC and issue the 'kill RPC' command if it's publicly routable
 524 2018-10-25T17:17:51  <esotericnonsense> lol
 525 2018-10-25T17:18:04  <esotericnonsense> now i'm just having too much fun ;P
 526 2018-10-25T17:20:54  <Sentineo> :)
 527 2018-10-25T17:21:02  <Sentineo> they would have to signal the killme bit :)
 528 2018-10-25T17:22:14  *** dqx has quit IRC
 529 2018-10-25T17:34:00  <wumpus> an alert RPC, simple as that: https://github.com/laanwj/bitcoin/commit/ace137ff25ab4c7c2a521abe9ba2af0d8af32ec2  (could theoretically make the style flags configurable to send errors etc as well but anyhow xD)
 530 2018-10-25T17:52:07  *** Victor_sueca has quit IRC
 531 2018-10-25T17:53:15  *** Victorsueca has joined #bitcoin-core-dev
 532 2018-10-25T17:56:22  <esotericnonsense> ! :D
 533 2018-10-25T17:56:22  <gribble> Error: ":D" is not a valid command.
 534 2018-10-25T17:57:13  <esotericnonsense> i don't have a build env set up at the moment but I am certainly enjoying the lack of any control on this
 535 2018-10-25T17:57:32  <esotericnonsense> running RPC alert in a tight loop is basically RPC kill ;P
 536 2018-10-25T17:57:50  <esotericnonsense> (unless this message box is going to replace itself, I assume it just spawns new ones forever)
 537 2018-10-25T18:00:02  *** Chris_Stewart_5 has quit IRC
 538 2018-10-25T18:00:56  <wumpus> so just idly wondering: did anyone that tried scanning for open RPC ports, check if the REST interface was enabled?
 539 2018-10-25T18:01:52  *** spinza has quit IRC
 540 2018-10-25T18:02:46  <wumpus> esotericnonsense: every open dialog holds up a RPC thread, so you won't be able to open more than four in the default config
 541 2018-10-25T18:02:56  <esotericnonsense> neat
 542 2018-10-25T18:03:08  <esotericnonsense> wait, so it is still RPC kill\
 543 2018-10-25T18:03:11  <esotericnonsense> it's just not node kill
 544 2018-10-25T18:03:30  <esotericnonsense> if you're AFK and someone gets in to RPC and hits you with four alerts, all the threads are taken. :P
 545 2018-10-25T18:03:52  <esotericnonsense> that's a neat side effect actually.
 546 2018-10-25T18:04:12  <wumpus> of course, you can still open a new one immediately after the user closed the old one
 547 2018-10-25T18:04:17  <wumpus> right
 548 2018-10-25T18:04:25  <midnightmagic> seems like maybe a bad idea
 549 2018-10-25T18:05:02  <luke-jr> arbitrary messages could be dangerous
 550 2018-10-25T18:05:16  <midnightmagic> Any weird QT display interpretation logic there might be an issue with..?
 551 2018-10-25T18:05:52  <gmaxwell> EMERGENCY: INSTALL UPGRADE FROM http://haxorsserver.com/badsoftware.exe RIGHT AWAY.
 552 2018-10-25T18:06:19  <luke-jr> yeah, exactly
 553 2018-10-25T18:06:33  <esotericnonsense> well if the alert command is behind rpc auth
 554 2018-10-25T18:06:47  <luke-jr> if it's sufficiently secure, it's useless for notifying people with the port exposed
 555 2018-10-25T18:07:01  <esotericnonsense> yeah
 556 2018-10-25T18:07:09  <esotericnonsense> well, not quite.
 557 2018-10-25T18:07:36  <esotericnonsense> if bad passwords are brute forced then it might be useful.
 558 2018-10-25T18:07:40  <wumpus> of course it's dangerous, then again, so are many other RPC commmands, this would be useful to communicate from say, a backend script to the UI… but I don't htink it's actually a good idea just wanted to show how easy it is to do
 559 2018-10-25T18:08:05  <gmaxwell> hm. so one thing that could be done in the GUI is the first time the rpc is connected after you start, don't allow the connection until the user confirms a dialog.
 560 2018-10-25T18:08:06  <esotericnonsense> (i'd consider RPC access as probably RCE anyway though I suppose it would require a determined attacker)
 561 2018-10-25T18:08:35  <gmaxwell> esotericnonsense: part of the reason we don't want the RPC internet exposed is because we don't want YET ANOTHER vector for unauthenticated RCEs.
 562 2018-10-25T18:08:35  <wumpus> you could do the same for REST, with a pre-programmed message, if you want it to be available with less security
 563 2018-10-25T18:08:55  <esotericnonsense> gmaxwell: yes, exactly, I think i'm being misinterpreted, sorry
 564 2018-10-25T18:09:15  * midnightmagic secretly merges alert rpc
 565 2018-10-25T18:09:27  <wumpus> midnightmagic: it reminds me of the fun when windows had this built in
 566 2018-10-25T18:09:29  <esotericnonsense> what I mean is that authenticated alert being used to send "install this badness.exe" seems irrelevant if authenticated rpc means you own the box anyway
 567 2018-10-25T18:09:42  <esotericnonsense> unauthenticated alert is bad sure
 568 2018-10-25T18:09:51  <wumpus> midnightmagic: you could make *any* computer pop up an alert xD
 569 2018-10-25T18:10:00  <esotericnonsense> that said, unauth alert could just give a predefined message
 570 2018-10-25T18:10:10  <midnightmagic> wumpus: death on flaxen wings :-)
 571 2018-10-25T18:10:20  <gmaxwell> E.g. "A remote control connection is being attempted to your wallet. If you did not initiate this action rejected it.  [Allow remote control during this session.] [Reject this attempt.] [Disable remote access].
 572 2018-10-25T18:10:38  <gmaxwell> "
 573 2018-10-25T18:10:59  <aj> gmaxwell: needs a "[ ] Always accept these requests" checkbox...
 574 2018-10-25T18:11:04  <wumpus> yes, so, this is even more dangerous for the wallet
 575 2018-10-25T18:11:15  <wumpus> which is another reason why having the wall... ok never mind
 576 2018-10-25T18:12:12  <gmaxwell> aj: why?
 577 2018-10-25T18:12:34  <wumpus> I kind of like how monero doesn't have a remote API, or daemon for the wallet at all, it's just a command line tool that connects to the node
 578 2018-10-25T18:12:53  <aj> gmaxwell: sorry, being sarcastic because i hate permission dialogs
 579 2018-10-25T18:13:06  <gmaxwell> all the examples I gave above about rpc being needed were wallet examples.
 580 2018-10-25T18:13:28  <esotericnonsense> aj: it would be Always accept these requests by default
 581 2018-10-25T18:13:44  <esotericnonsense> > gmaxwell | hm. so one thing that could be done in the GUI is the first time the rpc is connected after you start
 582 2018-10-25T18:13:55  <wumpus> I don't think popping up a dialog for for the first RPC request is a bad idea btw
 583 2018-10-25T18:14:10  <esotericnonsense> i suppose once per application start is distinct from once per... ever
 584 2018-10-25T18:14:12  <wumpus> that's, kind of, how those things usually work
 585 2018-10-25T18:14:16  <gmaxwell> It's one of the things you can do with the GUI... you can get user interaction.
 586 2018-10-25T18:14:21  <wumpus> yes
 587 2018-10-25T18:14:31  <esotericnonsense> once per IP address (ever) seems reasonable
 588 2018-10-25T18:14:48  <gmaxwell> esotericnonsense: meh, ever requires remebering it, and also ends up being less secure.
 589 2018-10-25T18:14:51  <wumpus> nah first make the RPC localhost-only
 590 2018-10-25T18:15:06  <gmaxwell> if it's a nussance, there could be a config override.
 591 2018-10-25T18:15:07  <wumpus> that's like low-hanging fruit
 592 2018-10-25T18:15:21  <wumpus> after that, you could add interaction
 593 2018-10-25T18:15:44  <gmaxwell> the non-interaction improvements are needed to protect bitcoind in any cas.e
 594 2018-10-25T18:15:51  <wumpus> but *remote* RPC is just a stupid idea
 595 2018-10-25T18:16:08  <gmaxwell> I wouldn't be surprised if most of those rpc listners are bitcoind... after all, bitcoind needs some use of the rpc regardless.
 596 2018-10-25T18:16:21  <wumpus> yess I know how stupid that sounds as RPC stands for Remote Procedure Call
 597 2018-10-25T18:16:43  <wumpus> right
 598 2018-10-25T18:16:51  *** spinza has joined #bitcoin-core-dev
 599 2018-10-25T18:16:56  <esotericnonsense> yeah i mean personally i don't see why it should be able to listen outside of localhost (or just, socket preferably)
 600 2018-10-25T18:17:10  <esotericnonsense> even though it would probably break All The Things
 601 2018-10-25T18:17:44  <wumpus> it's kind of a no-brainer, I'm just afraid of hordes of users complaining about Breaking Things
 602 2018-10-25T18:17:50  <jarthur> Any time I've done remote RPC (whether with SSH tunnel, or plain old terrible direct connection), it was because one server had enough hard drive space and the other did not.
 603 2018-10-25T18:18:06  <jarthur> or I was testing something
 604 2018-10-25T18:18:15  <wumpus> I've actually used remote connections for valid reasons (but yes, always over SSH tunnels)
 605 2018-10-25T18:18:36  *** Krellan has quit IRC
 606 2018-10-25T18:18:43  <esotericnonsense> about the only valid config I can think of right now apart from _really_ obtuse cases like a raspberry pi connected directly to another box without a switch
 607 2018-10-25T18:18:49  <esotericnonsense> would be rpcallowips with wireguard
 608 2018-10-25T18:18:59  <esotericnonsense> so that it's actually authenticated remote rpc
 609 2018-10-25T18:19:07  <wumpus> there's also the multi-VM scenario where network connections are inherently secure because the network is virtual
 610 2018-10-25T18:19:11  <gmaxwell> the problem with disabling non-local is things like private networks between hosts, which are perfectly fine assuming its setup correctly.
 611 2018-10-25T18:19:16  <wumpus> but yeah...
 612 2018-10-25T18:19:25  <gmaxwell> and indeed VMs.
 613 2018-10-25T18:19:33  <gmaxwell> We could set the TTL on those connections to 1. :P
 614 2018-10-25T18:19:44  <esotericnonsense> i mean localhost in general bugs me.
 615 2018-10-25T18:19:59  <echeveria> just disallow binding to 0/0.
 616 2018-10-25T18:20:05  <esotericnonsense> that it's a tcp socket is already 'everyone on this box'. the remote within "trusted" LAN situation is essentially the same
 617 2018-10-25T18:20:06  <wumpus> with docker it's incredibly common to have one application per container and have them communicate over point-to-point virtual networking, I'm sort of afraid of making those use-cases impossibl
 618 2018-10-25T18:20:09  <echeveria> require binds to be explicit.
 619 2018-10-25T18:20:47  <gmaxwell> esotericnonsense: yes, localhost is also a problem, but short of domain sockets we can't really do better than localhost + auth.
 620 2018-10-25T18:20:52  <esotericnonsense> wumpus: the docker case is reasonable yeah, as with VMs. a proper ethernet bridge.
 621 2018-10-25T18:21:08  <echeveria> this prevents copy-paste configs from working, massively reduces the ability for bad configurations, but still allows for usability over VPN tunnels, etc.
 622 2018-10-25T18:21:16  <gmaxwell> echeveria: interesting point, normally you don't want to beceause addresses change, but all the "okay" examples I gave above, they don't.
 623 2018-10-25T18:21:18  <esotericnonsense> the case in which you have boxA and boxB on the bridge and _noone else_, nice,.
 624 2018-10-25T18:21:25  <wumpus> yes luke-jr's PR makes sense, to at at least make binding explicit
 625 2018-10-25T18:21:45  <esotericnonsense> binding to an IP explicitly breaks docker. I think. unless you script it.
 626 2018-10-25T18:21:49  <gmaxwell> luke's patch also changes it to how I already thought it worked.
 627 2018-10-25T18:21:56  <esotericnonsense> no idea what IP a container will get.
 628 2018-10-25T18:22:20  <jarthur> link to luke's PR?
 629 2018-10-25T18:23:09  *** Murch has joined #bitcoin-core-dev
 630 2018-10-25T18:23:12  <harding> #14532
 631 2018-10-25T18:23:14  <gribble> https://github.com/bitcoin/bitcoin/issues/14532 | Never bind INADDR_ANY by default, and warn when doing so explicitly by luke-jr · Pull Request #14532 · bitcoin/bitcoin · GitHub
 632 2018-10-25T18:23:22  <jarthur> ty
 633 2018-10-25T18:23:29  <esotericnonsense> i suppose localhost + cookie, if the cookie has the correct permissions set, is similar to a permissioned domain socket
 634 2018-10-25T18:23:51  <esotericnonsense> the issue is really the strength of the auth mechanism and basically whether the RPC is safe pre-auth
 635 2018-10-25T18:24:08  <esotericnonsense> 'open port' doesn't really matter if that _actually works_
 636 2018-10-25T18:24:10  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 637 2018-10-25T18:24:47  <esotericnonsense> (as in, the situation post 14532 seems reasonable)
 638 2018-10-25T18:25:02  <echeveria> esotericnonsense: the RPC server is unlikely unsafe pre-auth, it's at least a denial of service risk.
 639 2018-10-25T18:25:16  <jarthur> gmaxwell: huh, it's funny, that's how I thought it worked already too
 640 2018-10-25T18:25:34  *** OzPac has quit IRC
 641 2018-10-25T18:26:27  <jarthur> esotericnonsense: one thing to consider with permissioned domain sockets is Linux abstract unix domain sockets have similar "openness" as TCP loopback.
 642 2018-10-25T18:27:00  <esotericnonsense> jarthur: eh? it's not possible to listen on a specific user, right?
 643 2018-10-25T18:27:01  <jarthur> With loopback, you at least have some decent control via netfilter/iptables/ufw on Linux
 644 2018-10-25T18:27:14  <esotericnonsense> (for TCP loopback)
 645 2018-10-25T18:27:20  <esotericnonsense> the domain socket can be rw only for the user
 646 2018-10-25T18:27:25  *** dqx has joined #bitcoin-core-dev
 647 2018-10-25T18:27:45  <esotericnonsense> e.g. if you had bitcoind, bitcoin-qt and bitcoin-qt had perms on the socket but other than bitcoin-qt only root did
 648 2018-10-25T18:29:04  <wumpus> !action merge #14532 I guess
 649 2018-10-25T18:29:04  * gribble merge #14532 I guess
 650 2018-10-25T18:29:10  <jarthur> esotericnonsense: it can't if it's a Linux abstract unix domain socket
 651 2018-10-25T18:30:01  *** Murch has quit IRC
 652 2018-10-25T18:30:09  <esotericnonsense> oh sorry. missed 'abstract'.
 653 2018-10-25T18:30:39  <booyah> are people actually losing money / being hacked, as result of many listening on internet?
 654 2018-10-25T18:31:51  <luke-jr> Typically exploits are fixed even if nobody has been compromised with them yet
 655 2018-10-25T18:32:32  <booyah> sure, not suggesting it's not something to be fixed, just wondering what is the situation
 656 2018-10-25T18:34:32  <gmaxwell> not that we're aware of, but I suspect most of us are uncomfortable with all this rpc exposure.
 657 2018-10-25T18:34:47  <gmaxwell> the additional attack surface is a liability we don't want to deal with.
 658 2018-10-25T18:35:03  <wumpus> how would you even know if people are losing money through this
 659 2018-10-25T18:35:14  <jarthur> It was real embarrassing for us on the Electrum project when the localhost auto-bind we were going to get around to at some point wasn't gotten around to until it hit the media. :)
 660 2018-10-25T18:35:24  <booyah> if someone would reported that happening, wumpus.  ofc most would not
 661 2018-10-25T18:35:48  <esotericnonsense> well the fact that it's now known increases the probability of exploit too, right.
 662 2018-10-25T18:35:52  <wumpus> also agree with luke-jr, it's better to prevent in this case
 663 2018-10-25T18:36:05  <gmaxwell> booyah: yea, problem is that people don't know or don't report. I'm somewhat doubtful it's causing much loss right now but it means that any rpc/rest bug is potentially much more serious.
 664 2018-10-25T18:36:18  <luke-jr> jarthur: what port does Electrum bind?
 665 2018-10-25T18:36:27  <gmaxwell> at least given what we currently believe: that it's being enabled due to accident/confusion.
 666 2018-10-25T18:36:32  <luke-jr> I wonder if these 8332s aren't even bitcoin itself?
 667 2018-10-25T18:36:49  <esotericnonsense> luke-jr: is it possible to get a bitcoin-like response back with invalid auth
 668 2018-10-25T18:36:58  <esotericnonsense> could just hit a few (or a lot) of them and check
 669 2018-10-25T18:37:14  * esotericnonsense checks his node
 670 2018-10-25T18:37:16  <luke-jr> has anyone done so?
 671 2018-10-25T18:37:19  <booyah> do we have the IP list?  anyone proceessed it, are theses opened e.g. mostly in some specific network, or OS?  (maybe there's a product or something that does that)
 672 2018-10-25T18:37:29  <wumpus> maybe it's a honeypot
 673 2018-10-25T18:37:34  <harding> luke-jr: the list of IPs I scan cam from bitnodes, which I think currently filters out Bitcoin Cash nodes.  Obviously so were probably spy nodes and the like.
 674 2018-10-25T18:37:40  <harding> s/so/some/
 675 2018-10-25T18:38:36  <luke-jr> harding: well, I mean perhaps some other software is listening in 8332 while Bitcoin isn't
 676 2018-10-25T18:38:41  <wumpus> or maybe they found an exploit in bitcoin-cli and are waiting for you to use it on them :-)
 677 2018-10-25T18:38:41  <esotericnonsense> lol. I forgot, it's obvious
 678 2018-10-25T18:38:43  <esotericnonsense> curl localhost:8332
 679 2018-10-25T18:38:45  <esotericnonsense> JSONRPC server handles only POST requests
 680 2018-10-25T18:39:00  <gmaxwell> wumpus: lol
 681 2018-10-25T18:39:07  *** ap4lmtree has joined #bitcoin-core-dev
 682 2018-10-25T18:39:07  <esotericnonsense> so yeah if you have the list it would probably take a few seconds to hit them all and see if they're not bitcoin. :P
 683 2018-10-25T18:39:31  <jarthur> luke-jr: electrum's RPC port is randomly selected at startup if you don't configure one. Still didn't take long for scans to pick them up though.
 684 2018-10-25T18:40:31  <wumpus> yeh randomizing ports helps against attackers looking for targets, not against anyone that already selected you
 685 2018-10-25T18:40:50  <esotericnonsense> luke-jr: what might be interesting
 686 2018-10-25T18:41:04  <esotericnonsense> is scanning all IPv4 for 8332 (i.e. hosts that don't listen on 8333)
 687 2018-10-25T18:41:06  <wumpus> e.g. you can't currently scan the whole IPv4 range for all ports
 688 2018-10-25T18:41:13  <wumpus> right
 689 2018-10-25T18:41:22  <esotericnonsense> you can disable listening whilst leaving rpc up right? :P
 690 2018-10-25T18:41:29  <wumpus> oh sure
 691 2018-10-25T18:41:43  <esotericnonsense> the odd person might have even fat fingered 8332 instead of 8333 in their dnat :P
 692 2018-10-25T18:42:34  <luke-jr> that sounds more likely
 693 2018-10-25T18:42:34  <gmaxwell> you can scan all ports on all hosts that visit your website/irc channel/etc. however.
 694 2018-10-25T18:43:30  <wumpus> yes, I guess the threat model for bitcoin is somewhat different than say, ssh
 695 2018-10-25T18:43:45  <booyah> but there are people where actually something respons to TCP 8332 right?  so not just opened firewalls / forwarded ports
 696 2018-10-25T18:44:22  <esotericnonsense> wumpus: heh you've reminded me that I have a weird memory-like error in ssh at the moment... my linux-fu is insufficient to debug it
 697 2018-10-25T18:44:38  <esotericnonsense> ssh host cat /dev/zero > /dev/null gives me an EFAULT in strace, read into a bad memory address
 698 2018-10-25T18:44:59  <esotericnonsense> anyway, this is not bitcoin core dev, sorry :P
 699 2018-10-25T18:47:33  <wumpus> I'd think it's more likely for the crash to be the result of some over-protective security feature in ssh than a security bug, but yeah
 700 2018-10-25T18:48:31  <luke-jr> I don't see how
 701 2018-10-25T18:48:44  <luke-jr> cat /dev/zero shouldn't have any security implications
 702 2018-10-25T18:48:58  *** hebasto has joined #bitcoin-core-dev
 703 2018-10-25T18:49:08  <esotericnonsense> the system call that fails is a read from the TCP socket (checked fd)
 704 2018-10-25T18:49:55  <esotericnonsense> the second parameter, the memory address (https://linux.die.net/man/2/read) is not valid
 705 2018-10-25T18:50:34  <gmaxwell> Is there a socket option to set the TTL on tcp connections?  If there is we should perhaps make the rpc use TTL=1 unless overridden by a config option.
 706 2018-10-25T18:50:51  <esotericnonsense> gmaxwell: ooh!!
 707 2018-10-25T18:50:54  <esotericnonsense> that is really, really nice.
 708 2018-10-25T18:51:00  <esotericnonsense> if it's possible.
 709 2018-10-25T18:51:15  <luke-jr> good idea
 710 2018-10-25T18:51:59  <echeveria> esotericnonsense: it's fairly trivial to scan 0/0 for a specific port now. there's port scanners that run directly on the NIC which can saturate multi gigabit links.
 711 2018-10-25T18:52:19  <gmaxwell> that nicely covers the 'lan/vpn good / internet bad'.
 712 2018-10-25T18:52:19  <echeveria> that said, I'd expect all ports listening on 8332 are listening on 8333.
 713 2018-10-25T18:52:22  <esotericnonsense> echeveria: yeah I'm aware of that, hence me thinking it might be interesting to see what's on 8332 "globally".
 714 2018-10-25T18:52:25  <wumpus> if there is one I've never heard of it, you can usually set it on a OS level but not per socket IIR
 715 2018-10-25T18:52:35  <esotericnonsense> echeveria: probably yeah.
 716 2018-10-25T18:52:45  <wumpus> it's a fascinating idea though
 717 2018-10-25T18:54:27  <aj> ip(7) IP_TTL sounds plausible?
 718 2018-10-25T18:54:30  <esotericnonsense> man 7 ip; IP_TTL (since Linux 1.0)
 719 2018-10-25T18:54:32  <esotericnonsense> yes
 720 2018-10-25T18:54:48  <esotericnonsense> you want it to be multiplatform though, :P
 721 2018-10-25T18:56:38  <aj> https://docs.microsoft.com/en-us/windows/desktop/winsock/ipproto-ip-socket-options and https://www.freebsd.org/cgi/man.cgi?query=ip&sektion=4&manpath=FreeBSD+9.0-RELEASE make it sound pretty multiplatform
 722 2018-10-25T18:56:54  <gmaxwell> well, linux only would be better than not having it, since so many of our users are on linux.
 723 2018-10-25T18:57:52  *** dqx has quit IRC
 724 2018-10-25T18:58:14  *** dqx has joined #bitcoin-core-dev
 725 2018-10-25T18:59:56  *** clarkmoody has joined #bitcoin-core-dev
 726 2018-10-25T19:01:48  <achow101> meeting?
 727 2018-10-25T19:02:03  <wumpus> #startmeeting
 728 2018-10-25T19:02:03  <lightningbot> Meeting started Thu Oct 25 19:02:03 2018 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
 729 2018-10-25T19:02:03  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
 730 2018-10-25T19:02:09  <promag> howdy
 731 2018-10-25T19:02:15  <achow101> hi
 732 2018-10-25T19:02:34  <gleb> hi
 733 2018-10-25T19:02:49  <wumpus> hey
 734 2018-10-25T19:02:59  *** dqx_ has joined #bitcoin-core-dev
 735 2018-10-25T19:03:33  <jonasschnelli> Hi
 736 2018-10-25T19:03:33  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
 737 2018-10-25T19:03:34  <sipa> hi
 738 2018-10-25T19:03:48  <luke-jr> ..
 739 2018-10-25T19:03:59  <midnightmagic> \o
 740 2018-10-25T19:04:27  *** dqx has quit IRC
 741 2018-10-25T19:05:04  <wumpus> topics?\
 742 2018-10-25T19:05:37  <jonasschnelli> topic proposal: 0.17.0.1 or 0.17.1
 743 2018-10-25T19:05:39  <sipa> what is the status with the linter issues on travis?
 744 2018-10-25T19:06:07  <phantomcircuit> hello
 745 2018-10-25T19:06:59  <wumpus> #topic 0.17.0.1 or 0.17.1
 746 2018-10-25T19:07:33  *** dqx_ has quit IRC
 747 2018-10-25T19:07:34  <jonasschnelli> I think we should release 0.17.0.1 (osx only) to mitigate the non opening DMG issue with 0.17 (https://github.com/bitcoin/bitcoin/pull/14416)
 748 2018-10-25T19:08:47  <jonasschnelli> We could release just 0.17.0 + 14416 as 0.17.0.1 macOS only (not release the current 0.17 branch)
 749 2018-10-25T19:09:04  <luke-jr> #14416
 750 2018-10-25T19:09:06  <gribble> https://github.com/bitcoin/bitcoin/issues/14416 | Fix OSX dmg issue (10.12 to 10.14) by jonasschnelli · Pull Request #14416 · bitcoin/bitcoin · GitHub
 751 2018-10-25T19:09:14  *** harrymm has quit IRC
 752 2018-10-25T19:09:33  <jonasschnelli> The current DMG provided in bitcoincore.org/bin does not open on macOS
 753 2018-10-25T19:09:34  <wumpus> agree, would make sense to make a 0.17.0.1 for macosx only
 754 2018-10-25T19:09:43  <luke-jr> jonasschnelli: the only other fix currently in the branch is so minor, it wouldn't make sense to make a new branch for 0.17.0.1
 755 2018-10-25T19:10:12  *** bitconner has joined #bitcoin-core-dev
 756 2018-10-25T19:10:18  <jonasschnelli> We can also directly move to 0.17.1,...
 757 2018-10-25T19:10:23  <sipa> that seems in line with versioning we've used before, using the 4th number for platform specific fixes
 758 2018-10-25T19:10:25  <luke-jr> doesn't MarcoFalke have a bunch of fixes backported to 0.17 though? might make more sense to just move on 0.17.1
 759 2018-10-25T19:10:41  <luke-jr> sipa: sure, I'm just saying we can tag it on 0.17 branch
 760 2018-10-25T19:11:02  <sipa> some of those may be nontrivial; i saw some issue with his backports PR having a merge conflict?
 761 2018-10-25T19:11:03  <cfields> luke-jr: that would mean that Mac users couldn't drop back to 0.17 if 0.17.1 was buggy.
 762 2018-10-25T19:11:09  <cfields> +1 for 0.17.0.1
 763 2018-10-25T19:11:16  <luke-jr> cfields: 0.17.1 should only be fixes on top of 0.17.0 anyway
 764 2018-10-25T19:11:39  <wumpus> sipa: yep
 765 2018-10-25T19:12:52  <promag> we could pick the simple backport fixes (including macos fix) and let the remaining for 0.17.2?
 766 2018-10-25T19:12:59  <wumpus> I dont' think we have enough for 0.17.1 yet
 767 2018-10-25T19:13:13  <wumpus> if a lot of people are experiencing issues with 0.17.0 on MacOSX we should do 0.17.0.1 soon
 768 2018-10-25T19:13:16  <wumpus> like, tomorrow
 769 2018-10-25T19:13:36  <sipa> agree
 770 2018-10-25T19:13:41  <jonasschnelli> ack
 771 2018-10-25T19:13:50  <cfields> sgtm
 772 2018-10-25T19:13:57  <jonasschnelli> I think it's not an urgent thing,.. but it may fraighten off users since it can cause a finder crash
 773 2018-10-25T19:13:57  * luke-jr shrugs
 774 2018-10-25T19:14:23  <cfields> jonasschnelli: now this is interesting!
 775 2018-10-25T19:14:40  <jonasschnelli> Someone told me Apple is aware...
 776 2018-10-25T19:14:53  <cfields> heh, ok
 777 2018-10-25T19:15:51  <jonasschnelli> A finder crash smells just really bad and AFAIK there has been some exploitable bugs in that area in the past.
 778 2018-10-25T19:16:00  *** reallll has joined #bitcoin-core-dev
 779 2018-10-25T19:16:23  <wumpus> it was funny hwo this is caused by a python unicode versus bytes issue
 780 2018-10-25T19:16:51  <jonasschnelli> Indeed!
 781 2018-10-25T19:17:15  <jonasschnelli> I just don't get why it was a non-issue when compiling with trusty.. I though we had the same python version.
 782 2018-10-25T19:17:19  <jonasschnelli> *thought
 783 2018-10-25T19:17:41  <jonasschnelli> however,.. lets just do a 0.17.0.1 macOS asap
 784 2018-10-25T19:18:27  <wumpus> it's great that you managed to isolate it
 785 2018-10-25T19:18:39  <jonasschnelli> Took me around 30 gitian builds. :)
 786 2018-10-25T19:18:49  *** belcher_ has quit IRC
 787 2018-10-25T19:19:35  <cfields> yes, thanks for that. I always check that dmgs open before tagging the detached sigs, but I've stayed on 10.11 to catch back-compat issues, so I guess I avoided it :\
 788 2018-10-25T19:20:25  <jonasschnelli> Yes. I also take it on me,... I haven't done that. But non of us somehow did test the DMG during all the RCs...
 789 2018-10-25T19:20:27  <jonasschnelli> which is a bit strange
 790 2018-10-25T19:20:52  <sipa> that's a bad sign
 791 2018-10-25T19:21:44  <wumpus> not many people testing on macosx?
 792 2018-10-25T19:21:48  <cfields> indeed. Maybe we could start adding "Tested ACKs" for the gitian sigs PRs
 793 2018-10-25T19:22:03  <cfields> to at least verify that someone has started up the release
 794 2018-10-25T19:22:26  <wumpus> I can only test the linux release myself
 795 2018-10-25T19:22:46  <wumpus> (and I test compiles on FreeBSD and OpenBSD! but that's irrelevant to gitian)
 796 2018-10-25T19:22:51  <jonasschnelli> At least fanquake, sjors and other gitian builders use macOS regularly... including myself. But meh,.. I don't know why I haven't detected it
 797 2018-10-25T19:23:03  <luke-jr> testing RCs is IMO the job of users, not builders or coders
 798 2018-10-25T19:23:13  <jonasschnelli> Both I'd say
 799 2018-10-25T19:23:13  <booyah> I can get you mac os X (old macbook) or bsd  test (install+run) if you want
 800 2018-10-25T19:23:56  <cfields> I'll at least start ACKing the platforms that I've verified startup for the sake of posterity.
 801 2018-10-25T19:24:02  <sipa> luke-jr: agree, but no reason why someone can't be both - and regardless of what you call them, not enough people testing rcs is concerning
 802 2018-10-25T19:24:38  <luke-jr> sipa: right, my point is that PRs isn't a good place for it
 803 2018-10-25T19:24:47  <luke-jr> users don't make PRs
 804 2018-10-25T19:25:07  <sipa> that's reasonable... though how else do you get feedback?
 805 2018-10-25T19:26:37  <luke-jr> short of writing a website that collects it, and asking with the RC announcement to post there.. coming up blank
 806 2018-10-25T19:26:53  <wumpus> usually we ask people to open issues on github for problems
 807 2018-10-25T19:26:54  <luke-jr> maybe bitcointalk can offer a forum specifically for testing reports or something
 808 2018-10-25T19:26:58  <wumpus> I'm nto sure a different website is needed
 809 2018-10-25T19:27:02  <luke-jr> true
 810 2018-10-25T19:27:09  <luke-jr> what's missing is *positive* feedback
 811 2018-10-25T19:27:10  <wumpus> reddit etc have too much noise
 812 2018-10-25T19:27:15  <wumpus> yes, true
 813 2018-10-25T19:27:47  <luke-jr> and apparently the users testing to give it
 814 2018-10-25T19:28:35  *** Krellan has joined #bitcoin-core-dev
 815 2018-10-25T19:29:38  <luke-jr> maybe the -core-dev ML needs to get more attention from users so they learn about and try RCs?
 816 2018-10-25T19:30:30  <wumpus> I.. dont' think you can really get users into a ML these days
 817 2018-10-25T19:30:39  <sipa> haha
 818 2018-10-25T19:30:45  <luke-jr> how do users learn about stuff now?
 819 2018-10-25T19:30:53  <sipa> maybe we need a facebook group  *ducks*
 820 2018-10-25T19:30:56  <luke-jr> >_<
 821 2018-10-25T19:31:16  <wumpus> :-)
 822 2018-10-25T19:31:17  <cfields> luke-jr: real answer: software force-updates itself.
 823 2018-10-25T19:31:28  <cfields> :(
 824 2018-10-25T19:31:29  <wumpus> twitter is really popular yes
 825 2018-10-25T19:31:42  <luke-jr> how about I make a Twitter account for posting experimental releases of Core, Knots, and other reputable Bitcoin projects? (Electrum, etc?)
 826 2018-10-25T19:32:21  <luke-jr> or is there some kind of shared Twitter account thing so more than just I can post?
 827 2018-10-25T19:33:02  <luke-jr> cfields: not to testing versions..
 828 2018-10-25T19:33:03  <sipa> we can tweet from the bitcoincoreorg account
 829 2018-10-25T19:33:17  <luke-jr> sipa: not everyone wants to see experimental releases
 830 2018-10-25T19:33:57  *** Emcy has quit IRC
 831 2018-10-25T19:34:54  <aj> luke-jr: could have a flag to say "auto update to current release version" and another to say "follow experimental release candidate stream"
 832 2018-10-25T19:35:18  *** h00t_ has joined #bitcoin-core-dev
 833 2018-10-25T19:35:47  <achow101> no auto update pls
 834 2018-10-25T19:35:50  <warren> luke-jr: have a different twitter account for those interested in following RC's
 835 2018-10-25T19:36:06  <luke-jr> warren: that's what I said :P
 836 2018-10-25T19:36:07  <wumpus> right, at least at the moemnt we don't tweet experimental releases from bitcoincore twitter
 837 2018-10-25T19:36:16  <sipa> yeah
 838 2018-10-25T19:36:24  <wumpus> I don't even post them to my personal account ,maybe I should
 839 2018-10-25T19:36:42  <luke-jr> wumpus: that might be enough
 840 2018-10-25T19:36:51  <warren> Thought you migrated permanently from Twitter. =)
 841 2018-10-25T19:37:20  <luke-jr> XD
 842 2018-10-25T19:37:32  <sipa> remind me, i can tweet from my personal account too
 843 2018-10-25T19:38:15  <wumpus> warren: well I'm more comfortable posting development-related stuff on mastodon, but for noticifcations to reach as many people as possible I'd certainly use twitter
 844 2018-10-25T19:38:25  <warren> (What is the protocol for topic proposals? wait until this topic is done?)
 845 2018-10-25T19:38:33  <wumpus> warren: no, just propose
 846 2018-10-25T19:38:57  <wumpus> ideally you propose topic subjects at the beginning of the meeting
 847 2018-10-25T19:39:06  <wumpus> then we can cycle through them
 848 2018-10-25T19:39:12  <wumpus> but it's fine we're done with this now
 849 2018-10-25T19:39:36  <sipa> i wanted to bring up the linter issues
 850 2018-10-25T19:40:37  <warren> topic proposal: Interested in opinions regarding the risk of bringing back Fortuna. Along with deprecation of BIP70, we are on the path toward eventual removal of the openssl dependency.
 851 2018-10-25T19:41:23  <sipa> we really don't need fortuna or a high-performance built-in randomness pool - we don't need randomness frequently
 852 2018-10-25T19:41:38  <sipa> what we do need is a good way to seed entropy from the environment
 853 2018-10-25T19:41:43  <rex4539> I think it is quite unlikely that "normal" people will ever run an RC because they are afraid of losing funds because of bugs. Others like me, on the other hand, only run beta software. If something is "stable" I don't want it. I want experimental, unstable software full of bugs. More fun. I believe the reason for missing the DMG bug is that everyone here is building from master and doesn't actually run the public builds.
 854 2018-10-25T19:41:43  <rex4539>  Perhaps there should be a pre-release QA checklist for basic functionality on all supported platforms.
 855 2018-10-25T19:42:00  <wumpus> #topic Fortuna
 856 2018-10-25T19:42:18  <wumpus> —or other randomness
 857 2018-10-25T19:42:20  <warren> well, Fortuna or the lesser goal of seeding entropy from the environment
 858 2018-10-25T19:42:20  <sipa> and seeding entropy from the environment is annoying as it's a platform specific business
 859 2018-10-25T19:42:39  <sipa> but we have a built-in randomness pool now - it's not fast, but it's more than good enough for what we need
 860 2018-10-25T19:42:44  *** reallll is now known as belcher_
 861 2018-10-25T19:42:50  <sipa> it's just seeding through OpenSSL mostly
 862 2018-10-25T19:43:07  <warren> I am encouraged that #14451 happened, deprecating BIP70 (huge attack surface, nobody uses it etc.) This means we will eventually be able to remove the openssl dependency. Except for that part.
 863 2018-10-25T19:43:09  <gribble> https://github.com/bitcoin/bitcoin/issues/14451 | Add BIP70 deprecation warning and allow building GUI without BIP70 support by jameshilliard · Pull Request #14451 · bitcoin/bitcoin · GitHub
 864 2018-10-25T19:43:38  <sipa> and whenever we need "strong randomness" we mix data from openssl and our own pool
 865 2018-10-25T19:44:50  <sipa> i think it's sufficient to have a c++ file with a bunch of entropy gathering things in it, without turning it into a C API or whatever
 866 2018-10-25T19:45:39  <jonasschnelli> #10299
 867 2018-10-25T19:45:41  <gribble> https://github.com/bitcoin/bitcoin/issues/10299 | Remove OpenSSL by sipa · Pull Request #10299 · bitcoin/bitcoin · GitHub
 868 2018-10-25T19:45:48  <cfields> sipa: you mean something that is essentially a standalone lib, but with no effort made to actually give it an external api?
 869 2018-10-25T19:46:17  <warren> #5885 had a previous attempt to replace the openssl PRNG. Reading those old comments remains interesting today.
 870 2018-10-25T19:46:20  <gribble> https://github.com/bitcoin/bitcoin/issues/5885 | [WIP] Replace OpenSSL PRNG with built-in Fortuna implementation by sipa · Pull Request #5885 · bitcoin/bitcoin · GitHub
 871 2018-10-25T19:46:32  <sipa> warren: nah, i think that's total overkill now
 872 2018-10-25T19:47:18  <jonasschnelli> Just read /dev/urandom? *duck*
 873 2018-10-25T19:47:23  <warren> external API is risky as you need to worry about about fork safety and conditions you can't predict
 874 2018-10-25T19:47:40  <gmaxwell> jonasschnelli: and then not get randomness when FD exhaustion means you can't open it.
 875 2018-10-25T19:47:43  <sipa> jonasschnelli: we already do that
 876 2018-10-25T19:48:00  <sipa> warren: not if it just gathers some entropy from the environment (and not a full RNG library)
 877 2018-10-25T19:48:19  <warren> Do we already use the newer syscall that blocks if the kernel prng is not seeded?
 878 2018-10-25T19:48:24  <sipa> yes
 879 2018-10-25T19:48:40  <kanzure> how do the proofs of randomness/entropy incorporation work
 880 2018-10-25T19:48:40  <jonasschnelli> gmaxwell: don't we just need a single seed during startup then ChaCha20 PRNG from. there? But i'm not familiar with the details..
 881 2018-10-25T19:48:43  <wumpus> yes we use the syscall where available
 882 2018-10-25T19:48:46  <warren> I know a lot less about the other Unixes.
 883 2018-10-25T19:48:50  <wumpus> on Linux and various BSDs
 884 2018-10-25T19:48:59  <sipa> jonasschnelli: no, that's FastRandomContext
 885 2018-10-25T19:49:14  <sipa> and it's only used to generate randomness that doesn't need independently seeding
 886 2018-10-25T19:49:22  <jonasschnelli> I see
 887 2018-10-25T19:49:36  <gmaxwell> jonasschnelli: only if everything works right, users never use virtual machines with snapshooting, and we don't care about being totally broken in the face of OS bugs like netbsd and freebsd had in the last couple years.
 888 2018-10-25T19:49:41  <sipa> for generating private keys etc, we gather new entropy every time
 889 2018-10-25T19:50:05  <warren> ----> <sipa> i think it's sufficient to have a c++ file with a bunch of entropy gathering things in it, without turning it into a C API or whatever <--- This would be good enough and people would feel it is worth the risk of change to be able to eventually remove the openssl dependency?
 890 2018-10-25T19:50:10  <gmaxwell> (and don't mind a later process memory leak potentially revealing the keys we previously generated, etc)
 891 2018-10-25T19:51:23  <gmaxwell> Until BIP70 is gone we're stuck with openssl regardless. we lost urgency on discontinuing using openssl as a randomness input after bitpay started requiring BIP70 to make payments.
 892 2018-10-25T19:51:40  <gmaxwell> So long as we have openssl for other things it's a harmless addition to random inputs.
 893 2018-10-25T19:52:10  <cfields> gmaxwell: IIRC that's only supported in -qt
 894 2018-10-25T19:52:45  <sipa> ah, for "non-strong" randomness (GetRandBytes, as opposed to GetStrongRandBytes) we use just OpenSSL, would people be ok with switching that to our own randomness pool?
 895 2018-10-25T19:53:04  <gmaxwell> we do? well that should be switched.
 896 2018-10-25T19:53:10  <jonasschnelli> Agree
 897 2018-10-25T19:53:43  <sipa> GetStrongRand uses all sources available, and mixes them into our own pool
 898 2018-10-25T19:53:47  <sipa> including OpenSSL
 899 2018-10-25T19:55:55  <wumpus> yes
 900 2018-10-25T19:56:45  <wumpus> switching over non-strong randomness is a no-brainer
 901 2018-10-25T19:57:15  <jarthur> topic proposal: Unix socket support for RPC API
 902 2018-10-25T19:57:48  <wumpus> eh <3 minutes left
 903 2018-10-25T19:57:57  <jarthur> No prob. Next time.
 904 2018-10-25T19:58:06  <wumpus> I agree though
 905 2018-10-25T19:58:09  <wumpus> FWIW
 906 2018-10-25T19:58:38  <wumpus> need that yesterday!
 907 2018-10-25T19:58:51  <aj> missed out on the linter stuff?
 908 2018-10-25T19:59:16  <sipa> guess we'll do that another time
 909 2018-10-25T19:59:22  <sipa> or hope it resolves itself
 910 2018-10-25T19:59:36  <wumpus> linters are great fun for the whole family, you can watch them fail in real time in so many different ways
 911 2018-10-25T19:59:38  *** opdenkamp has quit IRC
 912 2018-10-25T20:00:27  <wumpus> #endmeeting
 913 2018-10-25T20:00:27  <lightningbot> Meeting ended Thu Oct 25 20:00:27 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
 914 2018-10-25T20:00:27  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-10-25-19.02.html
 915 2018-10-25T20:00:27  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-10-25-19.02.txt
 916 2018-10-25T20:00:27  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-10-25-19.02.log.html
 917 2018-10-25T20:00:29  <cfields> Quick takeback: I wasn't entirely ready yet, and jumped the gun on coming back to dev after some time off a few weeks ago. Sorry if I've slowed anything down since then as a result, and for the annoying false start. Please don't let anything block on waiting for me in the near future.
 918 2018-10-25T20:00:52  <cfields> That's probably implied by now, just wanted to be explicit about it.
 919 2018-10-25T20:01:18  <wumpus> cfields: thanks for anything you've done, and no pressure for anything else
 920 2018-10-25T20:02:06  <cfields> wumpus: I'll for sure take care of whatever's needed from me for 0.17.0.1
 921 2018-10-25T20:02:20  <cfields> though I suppose that's nothing, if it's mac only :)
 922 2018-10-25T20:03:20  <wumpus> usually we build the release for all platforms I think, even though there's no effective chagne for others
 923 2018-10-25T20:03:56  <wumpus> but e.g. the website doesn't support hosting one version for one platform and another for another
 924 2018-10-25T20:04:33  <cfields> Makes sense, avoids having to customize all of the build scripts and stuff
 925 2018-10-25T20:04:56  <cfields> ok, will handle that as usual then.
 926 2018-10-25T20:05:40  <wumpus> do we really need release notes for 0.17.0.1?
 927 2018-10-25T20:09:10  <cfields> no opinion
 928 2018-10-25T20:11:51  *** clarkmoody has quit IRC
 929 2018-10-25T20:14:19  *** harrymm has joined #bitcoin-core-dev
 930 2018-10-25T20:15:57  *** opdenkamp has joined #bitcoin-core-dev
 931 2018-10-25T20:16:30  <wumpus> nah just have to mention the macosx issue I guess
 932 2018-10-25T20:18:19  <sipa> yeah
 933 2018-10-25T20:23:58  <jnewbery> cfields: take care of yourself first. Everyone here is very grateful for the enormous contributions you've made.
 934 2018-10-25T20:26:13  <jnewbery> sorry I missed the meeting. On the topic of testing RCs, the optech newsletter has action items each week, which included 'allocate time to test Bitcoin Core RC' for the weeks that we had RCs (eg https://bitcoinops.org/en/newsletters/2018/09/11/). If there are more specific instructions, let us know and we can help spread them.
 935 2018-10-25T20:26:25  <wumpus> jnewbery: +1
 936 2018-10-25T20:31:43  <gmaxwell> kallewoof: Are you or anyone else working on getting the "spend all inputs to an address at once if it isn't more expensive" thing going again?
 937 2018-10-25T20:31:50  <sipa> I dont't Optech members to be the ones who would find issues with the OSX installer, though
 938 2018-10-25T20:31:57  <gmaxwell> kallewoof: there was recently another round of massive dusting attacks.
 939 2018-10-25T20:36:11  <jarthur> While some folks are still here, I'm curious if any of you would be against an incremental introduction of unix domain sockets for the RPC API. e.g. release with just server support first, no bitcoin-cli support and no official work around abstract sockets. Server support may be exhumable from wumpus PR #9919.
 940 2018-10-25T20:36:14  <gribble> https://github.com/bitcoin/bitcoin/issues/9919 | UNIX sockets support for RPC by laanwj · Pull Request #9919 · bitcoin/bitcoin · GitHub
 941 2018-10-25T20:37:25  <jarthur> bitcoin-cli support is what was held up by proposed upstream libevent work
 942 2018-10-25T20:38:20  <gmaxwell> jonasschnelli: it's hard to test extensively without bitcoin-cli support, I'd worry about it bitrotting.
 943 2018-10-25T20:38:23  <gmaxwell> er jarthur
 944 2018-10-25T20:38:32  <gmaxwell> otherwise getting it in sounds fine to me.
 945 2018-10-25T20:39:07  <jarthur> Yea, there's only so much we'd be testing from the python functional tests.
 946 2018-10-25T20:40:18  *** bitconner has quit IRC
 947 2018-10-25T20:40:54  <sipa> gmaxwell: if the python test framework uses it, i'd be less concerned
 948 2018-10-25T20:41:50  <gmaxwell> I think thats clearly a requirement.
 949 2018-10-25T20:42:27  <gmaxwell> Still, its not like our tests are comprehensive enough that it's automatically okay.
 950 2018-10-25T20:42:39  *** bitconner has joined #bitcoin-core-dev
 951 2018-10-25T20:43:51  <jnewbery> sipa: perhaps not, but lots of people have subscribed to our newsletter who aren't members. We have about 1500 subs, who I expect are mostly quite technical.
 952 2018-10-25T20:44:12  <sipa> jnewbery: true, and i'm a big fan of that work, to be clear
 953 2018-10-25T20:44:51  <sipa> gmaxwell: any type of issue you're worried about in particular w.r.t. unix socket support but excluding bitcoin-cli initially?
 954 2018-10-25T20:45:06  <jnewbery> I didn't interpret your comment to mean that you weren't :)
 955 2018-10-25T20:45:39  <gmaxwell> sipa: e.g. will it crash under concurrent request load, etc. thats something we'd probably find out really fast with bitcoin-cli switched to it.
 956 2018-10-25T20:45:44  <jarthur> On the bright side, having the existing py func tests use a unix socket pretty doable. wumpus had it as an option in his PR.
 957 2018-10-25T20:46:50  <jnewbery> anyway, I don't think it can harm. If there's any advice on how people can better test RCs and provide feedback, I think we'd be happy to push it out to our subscribers.
 958 2018-10-25T20:47:07  *** bitconner has quit IRC
 959 2018-10-25T20:47:10  <jarthur> My reason for proposing server-only is to make the change small enough to not get sidelined again. I know I don't have enough time to work with libevent team on getting preexisting client connection support rolled in.
 960 2018-10-25T20:57:15  *** Chris_Stewart_5 has quit IRC
 961 2018-10-25T20:57:43  *** bitcoin-git has joined #bitcoin-core-dev
 962 2018-10-25T20:57:43  <bitcoin-git> [bitcoin] laanwj opened pull request #14576: Release 0.17.0.1 (0.17...2018_10_release_0.17.0.1) https://github.com/bitcoin/bitcoin/pull/14576
 963 2018-10-25T20:57:43  *** bitcoin-git has left #bitcoin-core-dev
 964 2018-10-25T20:58:37  <wumpus> jarthur: yes, I had the functional tests use a UNIX socket, next step would have been to have them interface with P2P over UNIX socket as well but never got that far
 965 2018-10-25T20:58:51  <wumpus> *for RPC
 966 2018-10-25T20:59:59  <wumpus> everyone was too much focused on the cli client support and I'm not confident that I can get the libevent people to adapt anything in that regard so I kind of gave up
 967 2018-10-25T21:00:35  <wumpus> feel free to pick it up though !
 968 2018-10-25T21:01:03  <jarthur> wumpus: thanks. I don't know the client code very well. Is there an easy alternative where we can just have libevent create the socket?
 969 2018-10-25T21:01:11  *** h00t_ has quit IRC
 970 2018-10-25T21:02:26  <wumpus> heh you wish
 971 2018-10-25T21:03:03  <wumpus> I wouldn't have gone though all that trouble if that existed, right :/
 972 2018-10-25T21:03:21  <jarthur> figured
 973 2018-10-25T21:03:25  <gmaxwell> I think it's fine to go without the -cli... just means a somewhat greater amount of testing should be done.
 974 2018-10-25T21:06:45  *** bitconner has joined #bitcoin-core-dev
 975 2018-10-25T21:10:08  <wumpus> jarthur: I think the only way to do that would be to implement the entire http protocol (e.g. port over libevhttp), but that's a pretty bad place to be in
 976 2018-10-25T21:11:39  <jarthur> Is it because we do some custom stuff libevhttp doesn't do?
 977 2018-10-25T21:11:42  *** dtr has joined #bitcoin-core-dev
 978 2018-10-25T21:12:37  <wumpus> there's more reasons why libevent's http server isn't really that great for what we're doing (such as the work pool) but there's nothing else that we *can't* do with it
 979 2018-10-25T21:13:15  <wumpus> httpserver.cpp is basically one complex of working-around thread limitations of their http server
 980 2018-10-25T21:13:44  <wumpus> (but that's not relevant to UNIX socket support at least :)
 981 2018-10-25T21:16:58  <wumpus> and there's a more advanced web server that runs on libevent which is commonly recommended: https://github.com/criticalstack/libevhtp  ... I don't think that'd work with the client part though, and it's yet another dependency
 982 2018-10-25T21:17:25  <wumpus> I mean, help with the client part, and no idea if they do support UNIX sockets
 983 2018-10-25T21:17:51  <wumpus> I'm...so disasppointed
 984 2018-10-25T21:18:36  <wumpus> (but it's 10000% better than boost asio so...)
 985 2018-10-25T21:18:40  <sipa> haha
 986 2018-10-25T21:20:48  <sipa> gmaxwell: the current way GetStrongRandBytes works is by computing SHA512(state || hw_entropy || os_entropy || openssl_entropy), and then using one half of the output as new state, and one half as random output
 987 2018-10-25T21:21:43  <sipa> for non-strong we can probably replace that with SHA512(state || high-accuracy-timer || some_other_cheap_entropy) or so?
 988 2018-10-25T21:22:10  <wumpus> their server isn't the problem, it allows injecting custom FDs for UNIX sockets fine! it's the client, we'd need another http *client* that supports http over UNIX sockets
 989 2018-10-25T21:23:10  <sipa> wumpus: you may hate this... but given that bitcoin-cli is specific to bitcoind, it probably doesn't need to actually implement full HTTP; just whatever subset is needed for bitcoind
 990 2018-10-25T21:23:47  <wumpus> sipa: yes, that's a good point!
 991 2018-10-25T21:24:22  <wumpus> hadn't really thought of it that way
 992 2018-10-25T21:24:39  <sipa> and if there's an adequate HTTP implementation that does what we need, that certainly feels better than something NIHed
 993 2018-10-25T21:24:49  *** bralyclow2 has joined #bitcoin-core-dev
 994 2018-10-25T21:25:03  <sipa> but if there's concerns... i don't expect that the subset of HTTP we need is all that hard to write
 995 2018-10-25T21:25:31  *** bralyclow2 has quit IRC
 996 2018-10-25T21:25:45  <wumpus> right
 997 2018-10-25T21:26:09  <wumpus> I missed the part where it only needs to work with our own server :)
 998 2018-10-25T21:26:16  <jarthur> wumpus: ahh, was the reason we needed the fd-passing ability because their client didn't even have support for starting a unix socket fd to begin with?
 999 2018-10-25T21:26:33  <wumpus> I guess that makes it quite simple
1000 2018-10-25T21:27:48  <wumpus> jarthur: yes
1001 2018-10-25T21:28:25  <wumpus> jarthur: at the time, at least, if they added it since then that'd be incredible
1002 2018-10-25T21:30:40  *** CubicEarth_ has quit IRC
1003 2018-10-25T21:31:18  <wumpus> (but I doubt it, they've always limited support to the intersection of various OSes, so having a specific unix API is probably even more out of the question)
1004 2018-10-25T21:32:20  <jarthur> Yeah, just checked, they haven't
1005 2018-10-25T21:32:51  *** CubicEarth has joined #bitcoin-core-dev
1006 2018-10-25T21:33:27  *** timothy has quit IRC
1007 2018-10-25T21:33:52  <wumpus> too bad
1008 2018-10-25T21:35:23  <wumpus> it's kind of interesting, I mean Tor uses libevent and they certainly support UNIX sockets for many things, for which they use fd injection at the base layer, they just don't happen to use the http server so that's not important
1009 2018-10-25T21:35:34  <wumpus> (nor http client)
1010 2018-10-25T21:36:18  <wumpus> this is just a case of whyyy do you have to make it so difficult for me
1011 2018-10-25T21:36:34  *** shesek has joined #bitcoin-core-dev
1012 2018-10-25T21:36:34  *** shesek has joined #bitcoin-core-dev
1013 2018-10-25T21:36:38  *** promag has quit IRC
1014 2018-10-25T21:38:44  *** promag has joined #bitcoin-core-dev
1015 2018-10-25T21:39:40  <wumpus> (why does bitcoin use http? I do not know, I guess JSON-RPC is marginally better than rolling yet another custom protocol, though the line-based JSON RPC of c-lightning is *pretty neat*)
1016 2018-10-25T21:40:56  <wumpus> why bother wrapping your JSON RPC requests in all this http stuff when you can just do request\nreply\n
1017 2018-10-25T21:43:00  <jarthur> Perhaps to enable local browser-based experiences?
1018 2018-10-25T21:43:13  <jarthur> Though there are WebSocket transport options for JSON-RPC now
1019 2018-10-25T21:43:16  <wumpus> yea exploitability++
1020 2018-10-25T21:43:20  <jarthur> :)
1021 2018-10-25T21:44:57  <promag> lol
1022 2018-10-25T21:47:07  *** promag has quit IRC
1023 2018-10-25T21:59:47  <phantomcircuit> wumpus, line based json rpc? so stratum?
1024 2018-10-25T22:01:21  *** bitcoin-git has joined #bitcoin-core-dev
1025 2018-10-25T22:01:21  <bitcoin-git> [bitcoin] hebasto opened pull request #14577: qt: Cleanup `textInteractionFlags` for `QLabel` (master...20181025-textInteractionFlags) https://github.com/bitcoin/bitcoin/pull/14577
1026 2018-10-25T22:01:21  *** bitcoin-git has left #bitcoin-core-dev
1027 2018-10-25T22:01:58  *** Krellan has quit IRC
1028 2018-10-25T22:02:49  *** Krellan has joined #bitcoin-core-dev
1029 2018-10-25T22:04:25  *** Emcy has joined #bitcoin-core-dev
1030 2018-10-25T22:05:42  <wumpus> phantomcircuit: maybe? I've never used stratum, might be similar/the same as c-lightning's protocol
1031 2018-10-25T22:08:06  *** promag has joined #bitcoin-core-dev
1032 2018-10-25T22:09:41  *** shesek has quit IRC
1033 2018-10-25T22:09:58  <queip> <luke-jr> how do users learn about stuff now?   <--- make a twitch stream by skimply-dressed lady on configuring RPC ;)
1034 2018-10-25T22:10:15  *** shesek has joined #bitcoin-core-dev
1035 2018-10-25T22:10:20  <esotericnonsense> obviously it's so that you can stick your bitcoin rpc behind nginx and put it on https://myexchange.com/api
1036 2018-10-25T22:10:34  <esotericnonsense> with no auth
1037 2018-10-25T22:10:45  <esotericnonsense> the RPC port won't be exposed as 8332 so it's fine
1038 2018-10-25T22:10:56  <wumpus> which, in some sort of way, makes sense if wallets aren't involved
1039 2018-10-25T22:11:11  * esotericnonsense cries
1040 2018-10-25T22:11:19  <esotericnonsense> if it's sufficiently sandboxed... maybe...
1041 2018-10-25T22:11:38  <wumpus> in some ways this is really between a rock and a hard place, most of the information in the API is as public as it gets
1042 2018-10-25T22:11:59  <esotericnonsense> i don't think that's true
1043 2018-10-25T22:12:10  <wumpus> you can put it behind a caching API and it's still fine, this was the idea behind the REST API
1044 2018-10-25T22:12:11  <esotericnonsense> well, I mean, define "most", I guess
1045 2018-10-25T22:12:11  *** bitconner has quit IRC
1046 2018-10-25T22:12:37  <esotericnonsense> I think someone who looks at my node monitor for long enough would be able to figure out, for example, geographically where it's located
1047 2018-10-25T22:12:44  <wumpus> I mean things like 'what peers am I connected to' can be more or less sensitive?
1048 2018-10-25T22:13:20  <wumpus> I certainly don't mean exposing any information by default, but if someone decides to do it for a public node...
1049 2018-10-25T22:13:37  <esotericnonsense> yes
1050 2018-10-25T22:13:38  <esotericnonsense> sure
1051 2018-10-25T22:13:45  <wumpus> which is what you implied with the nginx thing
1052 2018-10-25T22:14:09  <jarthur> wumpus: re stratum, it's actually implemented on top of JSON-RPC. A lot of the stratum implementations in crypto projects transport JSON-RPC over TCP with line breaks.
1053 2018-10-25T22:14:11  <esotericnonsense> if the actual interface itself is hardened and/or it's running in a jail/VM/whatever with resource constraints (ideally both) I don't see an issue
1054 2018-10-25T22:14:43  <esotericnonsense> i would just assume there are obvious buffer overflows in the rpc interface though
1055 2018-10-25T22:15:05  <esotericnonsense> without really thinking about it too much just because I'm not sure anyone cares (because it shouldn't be public so why care)
1056 2018-10-25T22:15:32  <esotericnonsense> just little things, like what happens if someone sends an rpc request with a string parameter that is 400MB
1057 2018-10-25T22:15:34  <esotericnonsense> lol
1058 2018-10-25T22:16:11  <wumpus> well then it will use 400MB, it's not really interesting
1059 2018-10-25T22:16:13  <esotericnonsense> if not RCE-type overflows then probably a DoS if the server dies
1060 2018-10-25T22:16:21  *** shesek has quit IRC
1061 2018-10-25T22:16:43  <esotericnonsense> actually, lol
1062 2018-10-25T22:16:45  <wumpus> you can do that to most websites if you really want
1063 2018-10-25T22:16:48  <esotericnonsense> doesn't rpc just have a shutdown command
1064 2018-10-25T22:16:52  <esotericnonsense> :D
1065 2018-10-25T22:17:40  *** shesek has joined #bitcoin-core-dev
1066 2018-10-25T22:17:40  *** shesek has joined #bitcoin-core-dev
1067 2018-10-25T22:19:28  <esotericnonsense> WRT bitcoin-cli and http/domain sockets/etc
1068 2018-10-25T22:21:24  *** bitconner has joined #bitcoin-core-dev
1069 2018-10-25T22:21:30  <esotericnonsense> i'm not suggesting a rewrite, but if it were easier that way, is it necessary to restrict to use of C++? I mean it barely does anything anyway
1070 2018-10-25T22:21:41  *** hebasto has quit IRC
1071 2018-10-25T22:22:10  <sipa> ?
1072 2018-10-25T22:22:18  <wumpus> replace c++ by, waht?
1073 2018-10-25T22:22:21  <esotericnonsense> there's a few config args, the simulated getinfo, help message, and the rest of it is pretty much standard
1074 2018-10-25T22:22:38  <esotericnonsense> anything you needed to to find a non NIH http solution
1075 2018-10-25T22:22:50  <wumpus> I think if you'd rewrite the entire thing in rust overnight a lot of people would be happy xD
1076 2018-10-25T22:22:51  <esotericnonsense> python, say (no idea if python does http over domain sockets)
1077 2018-10-25T22:23:03  <esotericnonsense> what bitcoin-cli?
1078 2018-10-25T22:23:17  <wumpus> noo
1079 2018-10-25T22:23:24  *** ajtowns[m] has joined #bitcoin-core-dev
1080 2018-10-25T22:23:34  <esotericnonsense> all of bitcoin? :D:D
1081 2018-10-25T22:23:40  <gmaxwell> I think esotericnonsense is suggesting bitcoin-cli get replaced with a python tool?
1082 2018-10-25T22:23:57  <queip> that seems easy... but exactly what for>
1083 2018-10-25T22:24:14  <wumpus> that's trivial
1084 2018-10-25T22:24:18  <esotericnonsense> gmaxwell: i'm not suggesting it but rather responding to wumpus'/sipa message above about bitcoin-cli not needing to be a full http client and inventing some thing
1085 2018-10-25T22:24:55  <sipa> afaik the origin bitcoin-cli code was just some printf statements into a socket
1086 2018-10-25T22:25:07  <gmaxwell> I don't think thats absurd on its face, but: it would probably be a massive slowdown for those users who do processing using bitcoin-cli in scripts,  and would suffer the standard "throwaway and rewrite software issues" -- the complexity of existing code is effectively all the accrewed knoweldge built from years and years of use... and that it probably gets right lots of behaviors none of us
1087 2018-10-25T22:25:07  <gmaxwell> ever explicitly realized were ever requirements.
1088 2018-10-25T22:25:14  <wumpus> rewriting bitcoin-cli in python, certainly using the code already in the test framework, would be trivial, but why?
1089 2018-10-25T22:25:35  <esotericnonsense> hm
1090 2018-10-25T22:25:49  <esotericnonsense> i guess I don't hit the right endpoints for the rpc client to be the bottleneck, heh
1091 2018-10-25T22:25:57  <wumpus> I mean it means that all users need to have python installed
1092 2018-10-25T22:26:07  *** bitconner has quit IRC
1093 2018-10-25T22:26:12  <wumpus> which is the case on linux, and maybe on macosx? but certainly not on windows
1094 2018-10-25T22:26:20  <gmaxwell> esotericnonsense: I mean just forking _python_ for each request is probably going to be a lot slower.
1095 2018-10-25T22:26:35  <sipa> gmaxwell: generally i agree with that... but i also think bitcoin-cli is very simple :)
1096 2018-10-25T22:26:47  <gmaxwell> yea, if we wanted to rewrite anything bitcoin-cli would be it! :)
1097 2018-10-25T22:26:50  <jarthur> esotericnonsense: on Python, aiohttp supports unix sockets out of the box. The popular synchronous libs (httplib, requests) need supplemental libs to enable unix socket usage currently.
1098 2018-10-25T22:26:51  <wumpus> it woult make zero difference for bitcoin-cli wrt performance
1099 2018-10-25T22:26:51  *** spinza has quit IRC
1100 2018-10-25T22:26:54  <esotericnonsense> i feel like
1101 2018-10-25T22:27:04  <esotericnonsense> someone using bitcoin-cli in scripts ... argh\
1102 2018-10-25T22:27:11  <esotericnonsense> supporting that feels awful
1103 2018-10-25T22:27:23  <sipa> my preference is just replacing the libevent http code in bitcoin-cli with hand-written HTTP though :)
1104 2018-10-25T22:27:27  <gmaxwell> esotericnonsense: you anti-unix heretic.
1105 2018-10-25T22:27:29  <wumpus> starting python is very fast, certainly if it's already cached in memory, and all work it does is *trivial*
1106 2018-10-25T22:27:30  <esotericnonsense> I mean, with interpretation of output, and so on, the whole point is that json rpc is standard
1107 2018-10-25T22:27:47  <wumpus> python is slow for heavily multi-threaded work but some comand like this?
1108 2018-10-25T22:27:50  <gmaxwell> wumpus: k. I was just guessing, I have no idea if it actually would be meaningfully slower.
1109 2018-10-25T22:28:08  * esotericnonsense uses python in production as a http client and gets pretty high concurrent workload
1110 2018-10-25T22:28:14  <esotericnonsense> er, server
1111 2018-10-25T22:28:19  <wumpus> gmaxwell: I mean it compiled everything to byte-code and the interpreter is very small
1112 2018-10-25T22:28:22  <esotericnonsense> it is single threaded though yes
1113 2018-10-25T22:28:36  <esotericnonsense> the overhead of starting an instance does exist, i just wonder about the cases in which that actually matters
1114 2018-10-25T22:28:42  <wumpus> I just don't think it's a relevant concern in this case
1115 2018-10-25T22:28:57  <wumpus> please work on something that actually affects users
1116 2018-10-25T22:29:06  <esotericnonsense> yes
1117 2018-10-25T22:29:14  <esotericnonsense> this is silly, sorry for raising it :P
1118 2018-10-25T22:29:51  <esotericnonsense> jarthur: missed your message, I'm using aiohttp actually, hadn't ever tried it with sockets though.
1119 2018-10-25T22:29:52  <gmaxwell> I don't think it was bad to raise, we might end up back on it if later bitcoin-cli is blocking disabling tcp rpc by default, and libevent still can't handle using domain sockets for the -cli case.
1120 2018-10-25T22:29:54  <esotericnonsense> might have to have a play.
1121 2018-10-25T22:31:14  <wumpus> let's write bitcoin-cli in rust
1122 2018-10-25T22:31:39  <gmaxwell> K.
1123 2018-10-25T22:31:46  * esotericnonsense is on page 3 of the rust book after finally getting around to it this morning.
1124 2018-10-25T22:31:50  <jarthur> CC andytoshi
1125 2018-10-25T22:32:02  <gmaxwell> Maybe the rust-bitcoin people have already done it.
1126 2018-10-25T22:32:04  <jarthur> esotericnonsense: works pretty well https://aiohttp.readthedocs.io/en/latest/client_advanced.html#unix-domain-sockets. I wrote the unix socket support for aiohttp server. Performance-wise it's pretty great on both sides.
1127 2018-10-25T22:32:15  <esotericnonsense> it took me a while to figure out how to add 1 to an integer because x++ doesn't work and that threw me for ages. :D
1128 2018-10-25T22:32:33  <wumpus> there is a rust bitcoinrpc library, but no command-line thing
1129 2018-10-25T22:32:44  <esotericnonsense> jarthur: oh! that's awesome! I don't know if you've had your fingers in any other parts of it but I've been really impressed with aiohttp
1130 2018-10-25T22:33:39  <esotericnonsense> I replaced my old flask backend for my node monitor with aiohttp a week or so ago and it's orders of magnitude faster, (though that is primarily due to using asyncio and not blocking on calls to bitcoind)
1131 2018-10-25T22:33:55  <jarthur> esotericnonsense: thanks! I'm a small-time contributor to it. I'm a big fan, though I tire of all the nested context managers involved in using the client :)
1132 2018-10-25T22:33:57  <esotericnonsense> so cheating, really :P
1133 2018-10-25T22:34:03  <wumpus> which makes sense, why go all the way to compile a rust crate for something then use it from friggin bash xD
1134 2018-10-25T22:34:40  *** Bullit has joined #bitcoin-core-dev
1135 2018-10-25T22:34:54  <esotericnonsense> just write bitcoin-cli in bash.
1136 2018-10-25T22:34:56  <esotericnonsense> and powershell.
1137 2018-10-25T22:34:58  <esotericnonsense> done.
1138 2018-10-25T22:35:13  <esotericnonsense> or make windows users use WSL or something. :>
1139 2018-10-25T22:35:21  <wumpus> ahhh shut up or I'm going to rewrite it in risc-v assembly
1140 2018-10-25T22:35:46  *** spinza has joined #bitcoin-core-dev
1141 2018-10-25T22:36:10  <sipa> what's wrong with Visual Basic?
1142 2018-10-25T22:36:17  <wumpus> xD
1143 2018-10-25T22:36:52  <midnightmagic> +1 risc-v assembly
1144 2018-10-25T22:36:56  <wumpus> visual basic 6 was the height of programming language development, after that it's only been a race downward
1145 2018-10-25T22:37:08  <sipa> wumpus: that was the last one i used :p
1146 2018-10-25T22:37:13  <wumpus> me too
1147 2018-10-25T22:37:20  <sipa> after VB6 i learned Perl.
1148 2018-10-25T22:37:25  <kallewoof> gmaxwell: I will work on getting that in place. Though will it actually help in this case? Dusting attacks I mean.
1149 2018-10-25T22:37:43  <jarthur> esotericnonsense: biggest problem I have with working in Python and Core RPC so far is how slow the std lib's JSON parsing/creating is, just because it's written in Python itself. I need to see if electrumx will be cool with switching to the raw REST APIs for block retrieval.
1150 2018-10-25T22:38:02  <kallewoof> gmaxwell: It seems like a more helpful thing to push for is #13756
1151 2018-10-25T22:38:04  <gribble> https://github.com/bitcoin/bitcoin/issues/13756 | wallet: -avoidreuse feature for improved privacy by kallewoof · Pull Request #13756 · bitcoin/bitcoin · GitHub
1152 2018-10-25T22:38:24  <esotericnonsense> jarthur: aaargh. you just reminded me.
1153 2018-10-25T22:38:39  <esotericnonsense> i spent a while hacking together some string stuff to optimize a json call the other day.
1154 2018-10-25T22:38:53  <esotericnonsense> i forgot to uncomment 'import ujson as json'.
1155 2018-10-25T22:38:57  <esotericnonsense> -_-
1156 2018-10-25T22:39:03  <jarthur> ahh, yea, that'll help :D
1157 2018-10-25T22:39:22  <esotericnonsense> can't be bothered to switch it back now until I need to change it. lol.
1158 2018-10-25T22:40:13  *** TheCharlatan has quit IRC
1159 2018-10-25T22:40:14  <wumpus> midnightmagic: so for all other platforms we'll have to ship qemu to emulate it
1160 2018-10-25T22:40:37  <esotericnonsense> wumpus: no just write the bits of qemu you need to emulate bitcoin-cli on riscv.
1161 2018-10-25T22:40:43  <esotericnonsense> obviously.
1162 2018-10-25T22:40:48  <wumpus> I have a risc-v emulator in rust ?
1163 2018-10-25T22:40:58  * esotericnonsense goes to get a beer.
1164 2018-10-25T22:41:02  <wumpus> :')
1165 2018-10-25T22:43:37  <wumpus> sipa: I've also used perl for a little while, don't remember anything though
1166 2018-10-25T22:43:55  * esotericnonsense can't really remember his route
1167 2018-10-25T22:44:05  *** michaelsdunn1 has quit IRC
1168 2018-10-25T22:44:38  <esotericnonsense> i think qbasic, c, ???, python, then just small amounts of arbitrary languages?
1169 2018-10-25T22:46:33  *** jarthur has quit IRC
1170 2018-10-25T22:47:31  *** bitconner has joined #bitcoin-core-dev
1171 2018-10-25T22:53:08  <wumpus> something like MSX basic -> Z80 asm -> gwbasic/qbasic -> PASCAL -> C -> perl -> C++ -> python -> rust ... though I don't exactly remember anymore, and there's some languages like haskell that I tried but never got very far with
1172 2018-10-25T22:56:09  <sipa> gwbasic, qbasic, vb, perl, c, java, haskell, c++, python
1173 2018-10-25T22:56:22  <wumpus> I regret everything after PASCAL xD
1174 2018-10-25T22:57:14  <aj> GWBasic, C64 Basic, AMOS Basic, qbasic, C, [Pascal], Smalltalk, Ada, Perl, C++, Prolog, Java, Python...
1175 2018-10-25T22:57:32  <aj> oh, actually would've been quickbasic not qbasic
1176 2018-10-25T22:58:42  <sipa> oh, there's a small amount PHP somewhere
1177 2018-10-25T23:00:51  <jcorgan> after Forth it all went downhill
1178 2018-10-25T23:02:54  <wumpus> oh I forgot about PHP as well, that was between perl and C++ I guess
1179 2018-10-25T23:04:32  <sipa> between java and haskell for me
1180 2018-10-25T23:04:52  <wumpus> aj: Ada is neat, I think it's a nice evolution of pascal, too bad it's mostly only used for us military stuff
1181 2018-10-25T23:06:12  *** so has quit IRC
1182 2018-10-25T23:08:15  <aj> wumpus: the precondition/postcondition stuff was pretty nifty. i remember it being a gratuitious pain to program with though, for what felt like syntactic reasons. might've been weird manual array management that would be completely laughable these days, don't remember
1183 2018-10-25T23:08:44  <aj> wumpus: i was never a fan of pascal though *shrug*
1184 2018-10-25T23:09:29  <esotericnonsense> i think I used PHP enough to think 'f this'
1185 2018-10-25T23:09:33  <wumpus> aj: I *loved* pascal, it was only because C seemed to be the wave of the future that I learned it and moved to it
1186 2018-10-25T23:10:28  <wumpus> that's a loong time ago though, I don't think I could do much with it these days
1187 2018-10-25T23:10:52  *** so has joined #bitcoin-core-dev
1188 2018-10-25T23:11:07  <wumpus> but things like the module system, bounded arrays, actually enforcede enumerations, it seemed to be ahead of C in many things
1189 2018-10-25T23:12:32  <wumpus> C was pretty nice as a low level, platform independent assembler replacement, but then they started to actually enforce things like undefined behavior, and platform-dependent behavior became important, and heck all of today's pain
1190 2018-10-25T23:12:50  <aj> wumpus: i guess bounded arrays, enforced enumerations are all strong-typing things where you'd go to haskell (or maybe rust?) these days
1191 2018-10-25T23:13:26  <wumpus> aj: it's why I like rust I think, I also *like* Haskell but cannot do anything practical with it
1192 2018-10-25T23:13:54  <sipa> i absolutely loved haskell as a language, and in many ways wished mainstream languages were more similar to it
1193 2018-10-25T23:14:31  <sipa> except in practice the type of programs i like to write are ones where performance and predictable resource usage matter... exactly what it abstracts away
1194 2018-10-25T23:14:36  <jcorgan> the older i get, the more languages i learn, and the fewer i use
1195 2018-10-25T23:14:53  <jcorgan> i'm pretty much down to python and c++ these days
1196 2018-10-25T23:15:00  <aj> sipa: haskell, but with performance specified as part of the type ;)
1197 2018-10-25T23:15:03  <sipa> not to say that you can't write performant code in haskell, but it feels like you need to break the otherwise awesome abstraction it provides in order to do so
1198 2018-10-25T23:15:18  *** Krellan has quit IRC
1199 2018-10-25T23:15:18  <sipa> i need to learn rust
1200 2018-10-25T23:15:27  <jcorgan> the last language to really interest me was Go, but that was self-limiting
1201 2018-10-25T23:15:36  <esotericnonsense> I like go
1202 2018-10-25T23:15:57  <esotericnonsense> i usually just run into a wall of sort of... OK, this is an interesting language, but does anything I do actually matter enough to optimise for language
1203 2018-10-25T23:16:08  <esotericnonsense> the answer is no, so I just end up reaching for python
1204 2018-10-25T23:16:31  <jcorgan> i liked Go's built-in concurrency and message passing
1205 2018-10-25T23:16:42  * sipa discovers the -j option to test_runner.py
1206 2018-10-25T23:16:44  <wumpus> sipa: +1 on Haskell, it's nice and pure but also I really like to understand how things map to the mechanical reality how things run on CPUs, and that's hard to grasp for me
1207 2018-10-25T23:16:51  * esotericnonsense has opened the rust tutorial again.
1208 2018-10-25T23:16:58  <gmaxwell> kallewoof: the dusting party was paying people that had existing outputs.
1209 2018-10-25T23:17:02  <esotericnonsense> I killed my vfio VM and can't be arsed to reboot the whole box to get it to work. heh
1210 2018-10-25T23:17:15  <gmaxwell> kallewoof: so if we automatically would spend that dust it would help and make the attacks less interesting.
1211 2018-10-25T23:17:32  <sipa> (maybe the language discussion is getting a bit off topic here; sorry for continuing it myself)
1212 2018-10-25T23:18:11  <gmaxwell> kallewoof: the forced avoidreuse is interesting too by my opinion is that off by default behavior is almost irrelevant from a privacy perspective. We should still do it, as part of a prinicipled commitment to privacy, but in practice if it's not on by default it's mostly not helping people.
1213 2018-10-25T23:18:31  <wumpus> yes, sorry, I don't really want to say much about language, except taht I increasingly dislike c++ and like rust
1214 2018-10-25T23:18:34  <gmaxwell> I think that probably it'll only be aggregatable signatures that really get us to where we need to be on that.
1215 2018-10-25T23:18:44  <kallewoof> gmaxwell: If I dust your bc1qgmax you will never use that "group" unless you turn on -avoidpartialspends, because using it will always give a higher fee.
1216 2018-10-25T23:19:07  <kallewoof> gmaxwell: with the "do if it gives same fee" feature, I mean.
1217 2018-10-25T23:19:19  <gmaxwell> kallewoof: not necessarily, because the alternative spend could have just as many but different inputs.
1218 2018-10-25T23:19:46  <kallewoof> But if it's dust, will that ever happen?
1219 2018-10-25T23:20:26  <gmaxwell> kallewoof: sure, you can miss by any amount, if your payment plus fees ends up being 5 satoshi short, then you're going to need another input and a dust one might be perfectly reasonsable.
1220 2018-10-25T23:21:11  <kallewoof> Sounds unusual but sure! :) I'll work on getting that back in. I am not sure the approach I had was reasonable though (doing the entire coin selection twice and comparing), esp for bigger wallets
1221 2018-10-25T23:23:36  <gmaxwell> kallewoof: why do you think it's unusual?  I expect the amount missed in any coin selection to be a small value with a peak near zero.
1222 2018-10-25T23:24:17  <gmaxwell> And importantly, this really can be always on for everyone with effectively no compromise (beyond the development effort)
1223 2018-10-25T23:24:28  *** OzPac has joined #bitcoin-core-dev
1224 2018-10-25T23:24:49  <gmaxwell> It also can be generalized to a form where you'll pay extra for it by some configurable small amount, but not an unbounded amount extra... which is I suspect what joe-user probably wants.
1225 2018-10-25T23:24:51  *** OzPac has quit IRC
1226 2018-10-25T23:25:08  <kallewoof> Oh, that sounds pretty useful, yeah.
1227 2018-10-25T23:25:42  <kallewoof> The only compromise is that coin selection takes 2x the time. For big wallets I hear rumors that this may be a big deal actually
1228 2018-10-25T23:25:53  <kallewoof> I should probably make a big wallet and see for myself.
1229 2018-10-25T23:26:00  <gmaxwell> I don't think it would on those wallets.
1230 2018-10-25T23:26:24  <gmaxwell> Also if you hear rumors like that you should invite people to report them, there is a lot of rumor that turns out to be spurrious.
1231 2018-10-25T23:26:53  <kallewoof> I think it was rumors that came from some exchange. I don't know if they're comfortable revealing too much about their set up.
1232 2018-10-25T23:26:56  <gmaxwell> Considering that we used to sign in the coinselection loop and are now well over 100x faster for many input spends, I suspect we have headroom. :)
1233 2018-10-25T23:27:07  <kallewoof> Should be easy to test though. Just make a wallet with a ton of UTXOs
1234 2018-10-25T23:27:17  <kallewoof> Oh.. wow.
1235 2018-10-25T23:27:56  <gmaxwell> Right but the rumors aren't useful reports-- because they can't distinguish 'slow' being from things like signing in-loop for many input spends vs basic behavior.  If businesses want to get behaior out of us, they need to behave like professionals.
1236 2018-10-25T23:28:52  <gmaxwell> kallewoof: also we could just have an option to turn it off if we're concerned... and the few parties that care could flip it.
1237 2018-10-25T23:30:53  <wumpus> yes, the problem has always been that the big users neither contribute actually useful reports, nor developers that optimize what is important for them
1238 2018-10-25T23:30:54  <kallewoof> gmaxwell: sdaftuar spoke against the feature in the original PR here: https://github.com/bitcoin/bitcoin/pull/12257#discussion_r204168100 ("I'm not sure that -avoidpartialspends works very well on wallets that have substantial address reuse, in particular I think you can devolve into cases where you'd produce giant transactions that take forever to sign and would never pass policy (or consensus) limits, in extreme cases.")
1239 2018-10-25T23:32:05  <kallewoof> The concern was partially mitigated by putting a cap on the UTXOs in a single group so maybe sdaftuar is cool with the idea now.
1240 2018-10-25T23:32:46  <wumpus> oh it's slow if you have tens of thousands of utxos, yea... that's awesome, no one that actually partakes in development uses it with that
1241 2018-10-25T23:33:07  *** Krellan has joined #bitcoin-core-dev
1242 2018-10-25T23:33:20  <gmaxwell> wumpus: it shouldn't be.
1243 2018-10-25T23:33:54  <wumpus> of course it shouldn't
1244 2018-10-25T23:33:59  <gmaxwell> kallewoof: re coinselection speed, another thing that used to make it slow is having a lot of unconfirmed outputs... we got 1000x fold speedups due to fixing it so that IsMine didn't have quasi-factorial complexity.
1245 2018-10-25T23:34:36  * kallewoof looks up quasi-factorial
1246 2018-10-25T23:34:55  <echeveria> from my experience people making a lot of these sort of claims don’t actually use the software at all, so keep that in mind too. I’ve had people sit and insist to me that the bitcoin wallet has all sorts of stupid behaviour I full well know it doesn’t.
1247 2018-10-25T23:35:01  <gmaxwell> I think we need to draw a line in the sand. If these large commercial players can't behave professionally enough to actually report their concerns (much less contribute effort to fix them and/or add test cases) we should ignore them... not fall into some rumor based development.
1248 2018-10-25T23:35:05  <wumpus> but I couldn't much be concerned about companies that make millions from using the software but contribute nothing back
1249 2018-10-25T23:35:12  <gmaxwell> Right.
1250 2018-10-25T23:35:32  <kallewoof> Makes sense to me
1251 2018-10-25T23:36:19  <echeveria> some of this comes from parties like blockchain.info who have frequently gone around conferences repeating stories of how they tried so hard to contribute to bitcoin core and were rejected for their efforts.
1252 2018-10-25T23:36:50  <gmaxwell> (I very much want their input for sure, but if they don't provide it.. they don't provide it.  Unfortunately, my expirence with many bitcoin companies is that they have executives that have never actually used software in a commercial context before, much less open source... and they treat it like the android play store... an app doesn't do what you want, you swap out for another one).
1253 2018-10-25T23:37:14  <kallewoof> heh
1254 2018-10-25T23:37:32  <echeveria> now if that’s true or not I don’t know, but I’ve not seen that sort of behaviour. there’s some contributes in the source tree from people I wouldn’t want to be in the same room with- because they’re evaluated on their merit rather than where they are from.
1255 2018-10-25T23:37:40  <gmaxwell> hopefully optech will do a useful job in getting more input, but otherwise we should just do the best we can. :)
1256 2018-10-25T23:38:48  <gmaxwell> echeveria: I personally liked the one company going around telling everyone that bitcoin core constantly crashed, and then got shut up by a developer offering them a sizable bet that they couldn't report any crash...
1257 2018-10-25T23:39:30  <sipa> in particular about coin selection, one question that came out of optech was making coin selection code more accessible... which sounds to me like they're not using the built-in wallet, and thus won't ever observe the coin selection logic
1258 2018-10-25T23:39:36  <sipa> (that may be an overgeneralization)
1259 2018-10-25T23:49:22  *** lnostdal has quit IRC
1260 2018-10-25T23:56:46  *** drexl has quit IRC