1 2018-12-13T00:10:53  *** Victorsueca has quit IRC
   2 2018-12-13T00:26:04  *** Chris_Stewart_5 has quit IRC
   3 2018-12-13T00:32:15  *** smaho has joined #bitcoin-core-dev
   4 2018-12-13T00:35:46  *** smaho has quit IRC
   5 2018-12-13T00:41:49  *** _cryptodesktop_i has joined #bitcoin-core-dev
   6 2018-12-13T00:53:27  *** shesek has quit IRC
   7 2018-12-13T00:53:49  *** shesek has joined #bitcoin-core-dev
   8 2018-12-13T00:54:34  *** shesek has quit IRC
   9 2018-12-13T00:54:56  *** shesek has joined #bitcoin-core-dev
  10 2018-12-13T01:01:56  *** dviola has joined #bitcoin-core-dev
  11 2018-12-13T01:06:22  *** miknotauro has quit IRC
  12 2018-12-13T01:15:54  *** _cryptodesktop_i has quit IRC
  13 2018-12-13T01:22:56  *** millerti has quit IRC
  14 2018-12-13T01:38:31  *** promag has quit IRC
  15 2018-12-13T01:40:41  <gmaxwell> promag: maybe everywhere we should split min_conf  into min_conf_trusted and min_conf_untrusted.  Most wallet behaior defaults to 0,1.
  16 2018-12-13T01:41:55  <gmaxwell> (or more specifically, the coinselection code by default does something like, 6,6 and if that fails 1,1 and if that fails 0,1.)
  17 2018-12-13T01:46:01  *** bitcoinjunior has joined #bitcoin-core-dev
  18 2018-12-13T02:30:02  *** rh0nj has quit IRC
  19 2018-12-13T02:30:58  *** ap4lmtree has quit IRC
  20 2018-12-13T02:31:08  *** rh0nj has joined #bitcoin-core-dev
  21 2018-12-13T02:31:23  *** ap4lmtree has joined #bitcoin-core-dev
  22 2018-12-13T02:32:05  <phantomcircuit> gmaxwell, what?
  23 2018-12-13T02:32:15  <phantomcircuit> of min_conf for our own transactions nvm
  24 2018-12-13T02:34:28  *** DougieBot5000_ has joined #bitcoin-core-dev
  25 2018-12-13T02:37:55  *** DougieBot5000 has quit IRC
  26 2018-12-13T02:43:08  *** DougieBot5000_ is now known as DougieBot5000
  27 2018-12-13T03:15:06  *** ap4lmtree has quit IRC
  28 2018-12-13T03:15:35  *** ap4lmtree has joined #bitcoin-core-dev
  29 2018-12-13T03:15:48  *** jonasschnelli has quit IRC
  30 2018-12-13T03:17:47  *** AaronvanW has quit IRC
  31 2018-12-13T03:22:57  *** jonasschnelli has joined #bitcoin-core-dev
  32 2018-12-13T03:25:20  *** ap4lmtree has quit IRC
  33 2018-12-13T03:28:07  *** ap4lmtree has joined #bitcoin-core-dev
  34 2018-12-13T03:28:15  *** lopp has quit IRC
  35 2018-12-13T03:38:54  *** bitcoinjunior has quit IRC
  36 2018-12-13T03:53:06  *** Murch has quit IRC
  37 2018-12-13T03:58:48  *** ghost43 has quit IRC
  38 2018-12-13T03:59:49  *** ghost43 has joined #bitcoin-core-dev
  39 2018-12-13T04:08:32  *** miknotauro has joined #bitcoin-core-dev
  40 2018-12-13T04:20:40  *** Murch has joined #bitcoin-core-dev
  41 2018-12-13T04:31:40  *** spinza has quit IRC
  42 2018-12-13T04:42:32  *** adam3us has quit IRC
  43 2018-12-13T04:44:47  *** spinza has joined #bitcoin-core-dev
  44 2018-12-13T04:46:35  *** shesek has quit IRC
  45 2018-12-13T04:48:01  *** shesek has joined #bitcoin-core-dev
  46 2018-12-13T04:48:02  *** adam3us has joined #bitcoin-core-dev
  47 2018-12-13T05:04:39  *** EREVAN has joined #bitcoin-core-dev
  48 2018-12-13T05:04:47  *** EREVAN is now known as Guest24281
  49 2018-12-13T05:05:35  *** droark has joined #bitcoin-core-dev
  50 2018-12-13T05:18:49  *** ossifrage has quit IRC
  51 2018-12-13T05:23:09  <droark> Hi. I have a question. On mainnet, about how long does Core need to run before the fee estimation code is considered accurate?
  52 2018-12-13T05:23:51  <sipa> about twice as long as the delay you want to estimate for
  53 2018-12-13T05:26:55  *** bitcoinsushi has joined #bitcoin-core-dev
  54 2018-12-13T05:39:25  *** nelsonhb has joined #bitcoin-core-dev
  55 2018-12-13T05:51:07  <droark> Thank you.
  56 2018-12-13T05:55:46  *** midnightmagic has quit IRC
  57 2018-12-13T06:01:18  <droark> Also, I seem to recall there being some reason why the code can't provide an estimate for TX insertion within one block. I can't remember it, though. Does it have to do with statistical variance?
  58 2018-12-13T06:01:44  <sipa> yes, the next block may be 1 second away
  59 2018-12-13T06:01:58  <sipa> no fee can give you a 95% chance of being included in that
  60 2018-12-13T06:03:00  <droark> Thanks. One last question for now. To clarify, when you say twice as long as the delay, do you mean actual time or # of blocks? I assume the latter but it sounds like I might be wrong.
  61 2018-12-13T06:04:10  <sipa> #blocks
  62 2018-12-13T06:04:29  <droark> Thanks.
  63 2018-12-13T06:07:10  *** hebasto has joined #bitcoin-core-dev
  64 2018-12-13T06:12:07  *** midnightmagic has joined #bitcoin-core-dev
  65 2018-12-13T06:14:04  *** jonasschnelli has quit IRC
  66 2018-12-13T06:14:05  *** jonasschnelli has joined #bitcoin-core-dev
  67 2018-12-13T06:24:28  *** ghost43 has quit IRC
  68 2018-12-13T06:24:39  *** ghost43 has joined #bitcoin-core-dev
  69 2018-12-13T07:01:57  *** Cory has quit IRC
  70 2018-12-13T07:07:17  *** Pasha has joined #bitcoin-core-dev
  71 2018-12-13T07:10:19  *** CodeBlue1776 has quit IRC
  72 2018-12-13T07:10:28  *** Pasha is now known as Cory
  73 2018-12-13T07:11:22  *** CodeBlue1776 has joined #bitcoin-core-dev
  74 2018-12-13T07:38:33  *** shesek has quit IRC
  75 2018-12-13T07:38:53  *** shesek has joined #bitcoin-core-dev
  76 2018-12-13T07:38:53  *** shesek has joined #bitcoin-core-dev
  77 2018-12-13T07:49:11  *** dviola has quit IRC
  78 2018-12-13T07:51:34  *** shesek has quit IRC
  79 2018-12-13T07:51:58  *** BGL has quit IRC
  80 2018-12-13T07:54:42  *** shesek has joined #bitcoin-core-dev
  81 2018-12-13T07:55:43  *** bitcoinsushi has quit IRC
  82 2018-12-13T08:07:13  *** shesek has quit IRC
  83 2018-12-13T08:08:35  *** shesek has joined #bitcoin-core-dev
  84 2018-12-13T08:08:35  *** shesek has joined #bitcoin-core-dev
  85 2018-12-13T08:13:31  *** shesek has quit IRC
  86 2018-12-13T08:13:56  *** Murch has quit IRC
  87 2018-12-13T08:15:00  *** nelsonhb has quit IRC
  88 2018-12-13T08:49:40  *** booyah_ has joined #bitcoin-core-dev
  89 2018-12-13T08:49:46  *** booyah has quit IRC
  90 2018-12-13T08:57:09  *** TX1683 has quit IRC
  91 2018-12-13T08:59:49  *** shesek has joined #bitcoin-core-dev
  92 2018-12-13T08:59:49  *** shesek has joined #bitcoin-core-dev
  93 2018-12-13T09:03:01  *** TX1683 has joined #bitcoin-core-dev
  94 2018-12-13T09:08:30  *** owowo has joined #bitcoin-core-dev
  95 2018-12-13T09:10:21  *** TheRec has quit IRC
  96 2018-12-13T09:35:45  *** BGL has joined #bitcoin-core-dev
  97 2018-12-13T10:02:16  *** Guyver2 has joined #bitcoin-core-dev
  98 2018-12-13T10:04:34  *** Sentineo has quit IRC
  99 2018-12-13T10:07:57  *** TheRec has joined #bitcoin-core-dev
 100 2018-12-13T10:07:57  *** TheRec has joined #bitcoin-core-dev
 101 2018-12-13T10:27:39  *** promag has joined #bitcoin-core-dev
 102 2018-12-13T10:34:43  *** timothy has joined #bitcoin-core-dev
 103 2018-12-13T10:35:06  *** spinza has quit IRC
 104 2018-12-13T11:05:01  *** rh0nj has quit IRC
 105 2018-12-13T11:06:08  *** rh0nj has joined #bitcoin-core-dev
 106 2018-12-13T11:25:54  *** spinza has joined #bitcoin-core-dev
 107 2018-12-13T11:29:02  *** Sentineo has joined #bitcoin-core-dev
 108 2018-12-13T11:35:24  *** Guyver2_ has joined #bitcoin-core-dev
 109 2018-12-13T11:37:09  <wumpus> unknown options in the config file are ignored, unknown command line options give an error, I think that's a good middle ground
 110 2018-12-13T11:38:07  <wumpus> if you explicitly provide -wallet= on *the command line* with a -disablewallet build I think it's fair to error out, you're requesting something it cannot do
 111 2018-12-13T11:38:28  *** Guyver2 has quit IRC
 112 2018-12-13T11:38:30  <wumpus> while if there's still left-over wallet configuration in the configuration file, well that probably shouldn't trip up too bad
 113 2018-12-13T11:42:50  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 114 2018-12-13T11:47:08  <promag> wumpus: I tend to agree with that
 115 2018-12-13T11:51:25  *** shesek has quit IRC
 116 2018-12-13T11:54:00  *** shesek has joined #bitcoin-core-dev
 117 2018-12-13T11:54:00  *** shesek has joined #bitcoin-core-dev
 118 2018-12-13T12:16:15  *** fanquake has joined #bitcoin-core-dev
 119 2018-12-13T12:16:55  <fanquake> wumpus I think #14741 and #14319 can go in
 120 2018-12-13T12:16:56  <gribble> https://github.com/bitcoin/bitcoin/issues/14741 | doc: Indicate -rpcauth option password hashing alg by dongcarl · Pull Request #14741 · bitcoin/bitcoin · GitHub
 121 2018-12-13T12:16:58  <gribble> https://github.com/bitcoin/bitcoin/issues/14319 | doc: Fix PSBT howto and example parameters by priscoan · Pull Request #14319 · bitcoin/bitcoin · GitHub
 122 2018-12-13T12:17:12  <fanquake> Both trivialish doc changes
 123 2018-12-13T12:18:56  *** shesek has quit IRC
 124 2018-12-13T12:20:21  *** shesek has joined #bitcoin-core-dev
 125 2018-12-13T12:20:21  *** shesek has joined #bitcoin-core-dev
 126 2018-12-13T12:25:50  *** shesek has quit IRC
 127 2018-12-13T12:26:11  *** shesek has joined #bitcoin-core-dev
 128 2018-12-13T12:27:35  *** shesek has quit IRC
 129 2018-12-13T12:28:06  <wumpus> fanquake: looks like it!
 130 2018-12-13T12:28:16  *** shesek has joined #bitcoin-core-dev
 131 2018-12-13T12:29:54  *** Chris_Stewart_5 has quit IRC
 132 2018-12-13T12:43:16  *** midnightmagic has quit IRC
 133 2018-12-13T12:46:58  *** midnightmagic has joined #bitcoin-core-dev
 134 2018-12-13T12:52:51  *** midnightmagic has quit IRC
 135 2018-12-13T12:55:18  *** miknotauro has quit IRC
 136 2018-12-13T12:58:41  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 137 2018-12-13T13:00:48  *** midnightmagic has joined #bitcoin-core-dev
 138 2018-12-13T13:03:12  <fanquake> Anyone found any issues testing rc1?
 139 2018-12-13T13:03:33  <wumpus> haven't seen any issues reported ye
 140 2018-12-13T13:15:41  <wumpus> not sure we've given it enough time yet, though
 141 2018-12-13T13:19:13  <wumpus> though rc period for backport releases needs to be less than for major releases
 142 2018-12-13T13:20:19  *** promag has quit IRC
 143 2018-12-13T13:20:30  <fanquake> fair enough
 144 2018-12-13T13:20:54  <fanquake> was just wondering, because there's at least one more fix, plus the PSBT documentation I'm about to backport we could get into .1
 145 2018-12-13T13:21:08  <fanquake> Otherwise we'll just wait for .2
 146 2018-12-13T13:21:25  <wumpus> but could discuss that at the meeting, along with whether to drop vista support in 0.18.0
 147 2018-12-13T13:21:38  <wumpus> right
 148 2018-12-13T13:22:56  <fanquake> I was actually considering getting up for the meeting this morning. Had a few things to bring up
 149 2018-12-13T13:24:02  <fanquake> Also qt 5.12 for 0.18.0
 150 2018-12-13T13:26:05  *** Guyver2__ has joined #bitcoin-core-dev
 151 2018-12-13T13:28:52  *** Guyver2_ has quit IRC
 152 2018-12-13T13:30:52  <wumpus> right
 153 2018-12-13T13:31:03  <wumpus> ok that'd be awesome :)
 154 2018-12-13T13:32:08  <wumpus> re: qt, we probably want #14849 (5.9.7) first
 155 2018-12-13T13:32:10  <gribble> https://github.com/bitcoin/bitcoin/issues/14849 | depends: qt 5.9.7 by fanquake · Pull Request #14849 · bitcoin/bitcoin · GitHub
 156 2018-12-13T13:32:37  <wumpus> that looks ready
 157 2018-12-13T13:32:44  <fanquake> wumpus yep. I was going to add that as my "high priority" PR. Once that's in I'll backport to 0.17, then start looking at 5.12
 158 2018-12-13T13:36:29  <fanquake> Guess I'll add something else to HP
 159 2018-12-13T13:39:39  <wumpus> heh! it's good to have this in, I think, so that it (hopefully) gets some manual testing before release, qt relies mostly on manual testing afterall
 160 2018-12-13T13:41:23  <fanquake> hopefully nobody has been trying to use the (now disabled) lcd number functionality :p
 161 2018-12-13T13:43:00  *** Guyver2__ has quit IRC
 162 2018-12-13T13:44:38  <wumpus> hah just looked what that does and it really emulates a LCD display on the screen, no I don't think we plan on using that :p
 163 2018-12-13T13:44:54  <wumpus> always good to speed up qt compile by removing unnecessary modules
 164 2018-12-13T13:46:32  <wumpus> I'm happy how fast we got the boost compile already in the same way
 165 2018-12-13T13:47:08  <fanquake> Yes. A NO_QT=1 depends build is quite fast
 166 2018-12-13T13:47:29  <wumpus> yep even on slow hw
 167 2018-12-13T13:58:10  *** shesek has quit IRC
 168 2018-12-13T13:58:11  *** Chris_Stewart_5 has quit IRC
 169 2018-12-13T13:58:35  *** shesek has joined #bitcoin-core-dev
 170 2018-12-13T14:12:39  *** shesek has quit IRC
 171 2018-12-13T14:13:51  *** shesek has joined #bitcoin-core-dev
 172 2018-12-13T14:14:36  <wumpus> speaking of which, my RISC-V node still runs fine <3
 173 2018-12-13T14:18:06  *** shesek has quit IRC
 174 2018-12-13T14:19:41  *** shesek has joined #bitcoin-core-dev
 175 2018-12-13T14:21:04  *** shesek has joined #bitcoin-core-dev
 176 2018-12-13T14:21:17  <fanquake> nice
 177 2018-12-13T14:26:26  *** shesek has quit IRC
 178 2018-12-13T14:26:45  *** shesek has joined #bitcoin-core-dev
 179 2018-12-13T14:29:02  *** wxss has joined #bitcoin-core-dev
 180 2018-12-13T14:29:06  *** shesek has joined #bitcoin-core-dev
 181 2018-12-13T14:30:30  <fanquake> wumpus have you found a RISC-V board that will run Qt yet :o
 182 2018-12-13T14:33:16  <wumpus> fanquake: I'm fairly sure it would run qt (using remote X server) :-) but no, I don't have the extension board with PCI-E functionality so I can't actually connect a monitor
 183 2018-12-13T14:33:29  *** shesek has quit IRC
 184 2018-12-13T14:33:58  *** shesek has joined #bitcoin-core-dev
 185 2018-12-13T14:33:58  *** shesek has joined #bitcoin-core-dev
 186 2018-12-13T14:35:09  <wumpus> SiFive Unleashed + Extension board + AMD gfx card could run qt, but the former two are really hard to get hold of. I think that's going to improve though, more RISC-V cores are in the works, maybe next year...
 187 2018-12-13T14:35:36  *** shesek has quit IRC
 188 2018-12-13T14:35:54  *** shesek has joined #bitcoin-core-dev
 189 2018-12-13T14:39:54  <fanquake> wumpus cool. Will have to track something down to experiment with.
 190 2018-12-13T14:44:40  *** shesek has quit IRC
 191 2018-12-13T14:45:25  *** shesek has joined #bitcoin-core-dev
 192 2018-12-13T14:45:35  *** shesek has quit IRC
 193 2018-12-13T14:45:36  *** shesek has joined #bitcoin-core-dev
 194 2018-12-13T14:48:55  <wumpus> I really hope there will be more affordable boards that can run Linux, next
 195 2018-12-13T14:53:51  *** belcher has quit IRC
 196 2018-12-13T14:54:49  *** belcher has joined #bitcoin-core-dev
 197 2018-12-13T15:01:56  *** shesek has quit IRC
 198 2018-12-13T15:03:01  *** shesek has joined #bitcoin-core-dev
 199 2018-12-13T15:03:01  *** shesek has joined #bitcoin-core-dev
 200 2018-12-13T15:06:40  *** millerti has joined #bitcoin-core-dev
 201 2018-12-13T15:08:22  *** shesek has quit IRC
 202 2018-12-13T15:09:37  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 203 2018-12-13T15:10:54  *** shesek has joined #bitcoin-core-dev
 204 2018-12-13T15:10:54  *** shesek has joined #bitcoin-core-dev
 205 2018-12-13T15:13:05  *** shesek has quit IRC
 206 2018-12-13T15:13:45  *** shesek has joined #bitcoin-core-dev
 207 2018-12-13T15:18:56  *** shesek has joined #bitcoin-core-dev
 208 2018-12-13T15:25:41  *** shesek has quit IRC
 209 2018-12-13T15:28:10  *** shesek has joined #bitcoin-core-dev
 210 2018-12-13T15:28:10  *** shesek has joined #bitcoin-core-dev
 211 2018-12-13T15:29:40  <luke-jr> would be nice to get POWER gitian builds merged, considering it's a much better deal than RISC-V etc
 212 2018-12-13T15:30:00  *** jarthur has joined #bitcoin-core-dev
 213 2018-12-13T15:30:35  <luke-jr> shesek: please fix your connection problems sometime in the next 4 hours
 214 2018-12-13T15:34:07  *** michaelsdunn1 has quit IRC
 215 2018-12-13T15:34:54  <wumpus> luke-jr: what's blocking that?
 216 2018-12-13T15:34:58  *** shesek has quit IRC
 217 2018-12-13T15:36:59  *** shesek has joined #bitcoin-core-dev
 218 2018-12-13T15:37:08  <luke-jr> wumpus: no idea; there's a minor rebase needed, but it sat idle for some time before that
 219 2018-12-13T15:39:34  <luke-jr> #14066
 220 2018-12-13T15:39:37  <gribble> https://github.com/bitcoin/bitcoin/issues/14066 | gitian-linux: Build binaries for 64-bit POWER by luke-jr · Pull Request #14066 · bitcoin/bitcoin · GitHub
 221 2018-12-13T15:40:01  <wumpus> I guess we need some more testers
 222 2018-12-13T15:40:33  <wumpus> also: did you fix cfields's remarks yet?
 223 2018-12-13T15:41:19  <luke-jr> I answered them, at least
 224 2018-12-13T15:41:42  <wumpus> ok
 225 2018-12-13T15:43:54  *** davec has quit IRC
 226 2018-12-13T15:44:28  *** shesek has quit IRC
 227 2018-12-13T15:45:23  *** shesek has joined #bitcoin-core-dev
 228 2018-12-13T15:45:35  *** shesek has joined #bitcoin-core-dev
 229 2018-12-13T15:46:49  *** promag has joined #bitcoin-core-dev
 230 2018-12-13T15:51:55  *** davec has joined #bitcoin-core-dev
 231 2018-12-13T15:52:53  *** shesek has quit IRC
 232 2018-12-13T15:53:00  *** michaelsdunn1 has joined #bitcoin-core-dev
 233 2018-12-13T15:53:56  *** shesek has joined #bitcoin-core-dev
 234 2018-12-13T16:02:03  <luke-jr> kinda wish I knew what was going on here: https://bugzilla.mozilla.org/show_bug.cgi?id=1512162
 235 2018-12-13T16:02:54  *** shesek has quit IRC
 236 2018-12-13T16:07:16  *** jungly has joined #bitcoin-core-dev
 237 2018-12-13T16:08:14  <wumpus> that's a *weird* issue, why would that only happen on one platform
 238 2018-12-13T16:08:54  *** shesek has joined #bitcoin-core-dev
 239 2018-12-13T16:08:54  *** shesek has joined #bitcoin-core-dev
 240 2018-12-13T16:10:03  *** promag has quit IRC
 241 2018-12-13T16:11:57  *** fanquake has quit IRC
 242 2018-12-13T16:12:12  <wumpus> could be a compiler bug
 243 2018-12-13T16:17:50  <luke-jr> yes, it's not very clear if the compiler or Linux is screwing up here
 244 2018-12-13T16:18:04  <luke-jr> since the VDSO is provided at runtime, I would expect Linux
 245 2018-12-13T16:18:32  <luke-jr> but this VDSO code hasn't changed since 2010
 246 2018-12-13T16:21:04  *** shesek has quit IRC
 247 2018-12-13T16:24:25  *** shesek has joined #bitcoin-core-dev
 248 2018-12-13T16:26:24  <cjd> Could have been interesting to see the backtrace on the gettimeofday() to see if there is perhaps something weird about where the timeval is placed
 249 2018-12-13T16:26:25  *** shesek has joined #bitcoin-core-dev
 250 2018-12-13T16:26:25  *** shesek has joined #bitcoin-core-dev
 251 2018-12-13T16:27:24  <cjd> I could imagine something like timeval placed on a 32 bit boundry, linux thinks it's supposed to be on the 64, trashes the stack cookie
 252 2018-12-13T16:27:49  <cjd> but I only looked at it for 5 min so my 2c is probably overpriced
 253 2018-12-13T16:31:31  *** shesek has quit IRC
 254 2018-12-13T16:32:45  *** shesek has joined #bitcoin-core-dev
 255 2018-12-13T16:34:27  *** shesek has quit IRC
 256 2018-12-13T16:36:59  *** shesek has joined #bitcoin-core-dev
 257 2018-12-13T16:43:59  *** promag has joined #bitcoin-core-dev
 258 2018-12-13T16:47:42  *** shesek has quit IRC
 259 2018-12-13T16:49:18  *** shesek has joined #bitcoin-core-dev
 260 2018-12-13T16:52:22  *** shesek has quit IRC
 261 2018-12-13T16:52:30  *** Tralfaz has joined #bitcoin-core-dev
 262 2018-12-13T16:55:03  *** shesek has joined #bitcoin-core-dev
 263 2018-12-13T16:55:03  *** shesek has quit IRC
 264 2018-12-13T16:55:03  *** shesek has joined #bitcoin-core-dev
 265 2018-12-13T16:56:59  *** shesek has quit IRC
 266 2018-12-13T16:58:03  *** timothy has quit IRC
 267 2018-12-13T16:58:26  *** promag has quit IRC
 268 2018-12-13T17:02:19  *** shesek has joined #bitcoin-core-dev
 269 2018-12-13T17:02:19  *** shesek has joined #bitcoin-core-dev
 270 2018-12-13T17:04:55  *** miknotauro has joined #bitcoin-core-dev
 271 2018-12-13T17:13:10  *** miknotauro has quit IRC
 272 2018-12-13T17:31:46  *** wxss has quit IRC
 273 2018-12-13T17:39:43  *** jungly has quit IRC
 274 2018-12-13T17:40:19  *** promag has joined #bitcoin-core-dev
 275 2018-12-13T17:45:27  *** danra has joined #bitcoin-core-dev
 276 2018-12-13T17:48:17  *** booyah_ is now known as booyah
 277 2018-12-13T17:53:27  *** spinza has quit IRC
 278 2018-12-13T17:54:03  *** wxss has joined #bitcoin-core-dev
 279 2018-12-13T18:02:09  *** ChanServ sets mode: +o wumpus
 280 2018-12-13T18:02:21  *** ChanServ sets mode: -o wumpus
 281 2018-12-13T18:03:02  *** kmels has joined #bitcoin-core-dev
 282 2018-12-13T18:10:02  *** promag has quit IRC
 283 2018-12-13T18:16:43  *** Chris_Stewart_5 has quit IRC
 284 2018-12-13T18:17:44  *** Murch has joined #bitcoin-core-dev
 285 2018-12-13T18:20:43  *** Giszmo has quit IRC
 286 2018-12-13T18:25:54  *** miknotauro has joined #bitcoin-core-dev
 287 2018-12-13T18:34:57  *** bitcoinjunior has joined #bitcoin-core-dev
 288 2018-12-13T18:39:56  *** Giszmo has joined #bitcoin-core-dev
 289 2018-12-13T18:41:40  *** spinza has joined #bitcoin-core-dev
 290 2018-12-13T18:42:44  <wumpus> looks like travis is failing on all new PRs
 291 2018-12-13T18:43:34  <wumpus> some extrememly scary error
 292 2018-12-13T18:43:39  <wumpus> https://travis-ci.org/bitcoin/bitcoin/jobs/467611982
 293 2018-12-13T18:44:34  <wumpus> (this is for #14951 which is only a RPC tests change and definitely shouldn't cause like this)
 294 2018-12-13T18:44:36  <gribble> https://github.com/bitcoin/bitcoin/issues/14951 | Revert "tests: Support calling add_nodes more than once" by MarcoFalke · Pull Request #14951 · bitcoin/bitcoin · GitHub
 295 2018-12-13T18:48:49  <gmaxwell> it appears to be saying that two threads are using the same fastrandomcontext at once.
 296 2018-12-13T18:49:05  *** hebasto has quit IRC
 297 2018-12-13T18:49:48  <gmaxwell> I didn't know we were running tsan in travis.
 298 2018-12-13T18:49:53  <wumpus> oof likely caused by #14624 then
 299 2018-12-13T18:49:56  <gribble> https://github.com/bitcoin/bitcoin/issues/14624 | Some simple improvements to the RNG code by sipa · Pull Request #14624 · bitcoin/bitcoin · GitHub
 300 2018-12-13T18:50:03  <jonasschnelli> probably...
 301 2018-12-13T18:50:42  *** chenpo has joined #bitcoin-core-dev
 302 2018-12-13T18:50:56  <gmaxwell> wumpus: looking at src/test/validation_block_tests.cpp it's totally misusing the rng in the test.
 303 2018-12-13T18:52:38  <sipa> why did travis not detect this in the pr?
 304 2018-12-13T18:54:00  <gmaxwell> I don't even really get why this paticular test exists-- like it's just starting lots of threads that call ProcessNewBlock at once. But ProcessNewBlock pretty much instantly takes a lock.
 305 2018-12-13T18:54:12  <wumpus> yeh it's clearly sharing a FastRandomContext between multiple threads
 306 2018-12-13T18:54:40  <sipa> actually, it's good to know tsan catches this
 307 2018-12-13T18:54:45  *** brianhoffman_ has joined #bitcoin-core-dev
 308 2018-12-13T18:54:46  <wumpus> sipa: hmm, maybe because your PR was from before TSAN checking was added
 309 2018-12-13T18:54:58  *** brianhoffman has quit IRC
 310 2018-12-13T18:54:59  *** brianhoffman_ is now known as brianhoffman
 311 2018-12-13T18:55:03  <sipa> ah
 312 2018-12-13T18:55:18  <gmaxwell> if so, how come it wasn't caught when tsan checking was added?
 313 2018-12-13T18:55:27  <gmaxwell> simple fix, and just a test bug regardless.
 314 2018-12-13T18:55:29  *** brianhoffman has quit IRC
 315 2018-12-13T18:55:42  <wumpus> it doesn't re-run all the travis runs for allPRs on every merge
 316 2018-12-13T18:55:48  <gmaxwell> ah.
 317 2018-12-13T18:55:50  <wumpus> might be an old cache, or who knows
 318 2018-12-13T18:55:51  <gmaxwell> though I still don't really get the utility of that test.
 319 2018-12-13T18:56:19  <wumpus> it's pretty nice that this was aught and that you were able to decipher the error :)
 320 2018-12-13T18:57:23  <gmaxwell> for some reason the tsan output is a little weird, it's only giving one side's backtrace.
 321 2018-12-13T18:57:44  <gmaxwell> Not my first time decyphering tsan output. :P
 322 2018-12-13T18:57:47  *** hex17or has quit IRC
 323 2018-12-13T19:00:06  *** hex17or has joined #bitcoin-core-dev
 324 2018-12-13T19:00:08  <provoostenator> Meeting?
 325 2018-12-13T19:00:13  <wumpus> I guess it tests the locking in ProcessNewBlock in case that becomes less granular inthe future
 326 2018-12-13T19:00:20  <wumpus> #startmeeting
 327 2018-12-13T19:00:20  <lightningbot> Meeting started Thu Dec 13 19:00:20 2018 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
 328 2018-12-13T19:00:20  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
 329 2018-12-13T19:00:37  <moneyball> hi
 330 2018-12-13T19:00:56  <wumpus> proposed topic: drop Windows Vista support, make minimum supported Windows 7
 331 2018-12-13T19:01:03  <moneyball> just one topic proposed during the week - jamesob
 332 2018-12-13T19:01:06  <gmaxwell> wumpus: ping people.
 333 2018-12-13T19:01:35  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb
 334 2018-12-13T19:01:38  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 335 2018-12-13T19:01:42  <provoostenator> (ah well, good way to get more people to review that RNG PR post merge)
 336 2018-12-13T19:01:43  <achow101> hi
 337 2018-12-13T19:01:44  <meshcollider> hi
 338 2018-12-13T19:01:45  <jonasschnelli> hi
 339 2018-12-13T19:02:04  <luke-jr> hi
 340 2018-12-13T19:02:09  <chenpo> hi
 341 2018-12-13T19:02:41  <wumpus> #topic High priority for review
 342 2018-12-13T19:02:45  <provoostenator> Let's call the topic Asta la la Vista :-)
 343 2018-12-13T19:03:19  <wumpus> https://github.com/bitcoin/bitcoin/projects/8 three PRs left on the list right now: #14336 #14565 #14573
 344 2018-12-13T19:03:22  <gribble> https://github.com/bitcoin/bitcoin/issues/14336 | net: implement poll by pstratem · Pull Request #14336 · bitcoin/bitcoin · GitHub
 345 2018-12-13T19:03:24  <gribble> https://github.com/bitcoin/bitcoin/issues/14565 | Overhaul importmulti logic by sipa · Pull Request #14565 · bitcoin/bitcoin · GitHub
 346 2018-12-13T19:03:26  <gribble> https://github.com/bitcoin/bitcoin/issues/14573 | qt: Add Window menu by promag · Pull Request #14573 · bitcoin/bitcoin · GitHub
 347 2018-12-13T19:03:27  <wumpus> provoostenator: good idea
 348 2018-12-13T19:03:58  <sipa> hi
 349 2018-12-13T19:03:59  <wumpus> the poll one should be (almost?) ready to merge
 350 2018-12-13T19:04:09  <provoostenator> Nominating #11082
 351 2018-12-13T19:04:11  <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
 352 2018-12-13T19:04:41  <provoostenator> Needs rebase, but really could use more review before that.
 353 2018-12-13T19:04:48  <wumpus> provoostenator: already needs rebase now
 354 2018-12-13T19:04:58  <wumpus> (and has, for 24 days)
 355 2018-12-13T19:05:12  <jonasschnelli> provoostenator: Maybe take that over from luke-jr
 356 2018-12-13T19:05:23  <sipa> does it need concept discussion?
 357 2018-12-13T19:05:34  <jonasschnelli> probably...
 358 2018-12-13T19:05:38  <provoostenator> I will eventually, but I have 15 other thing on my list. I also think it needs concept discussion.
 359 2018-12-13T19:06:02  <luke-jr> it doesn't need taking over, although if someone wants to rebase it sooner, I can push that
 360 2018-12-13T19:06:13  <provoostenator> Though we've had IRC chats about it before and everyone seemed to think it was at least a reasonable approach, compared to alterantives.
 361 2018-12-13T19:06:18  <wumpus> if it needs concept discussion, it definitely doesn't belong on the high priority for review list
 362 2018-12-13T19:06:26  <wumpus> yes, I think it's a reasonable approach
 363 2018-12-13T19:06:54  <provoostenator> Alright, maybe add it after rebase?
 364 2018-12-13T19:06:55  <wumpus> I dread making init option parsing even more involved, but it's better than the alternatives
 365 2018-12-13T19:07:16  <provoostenator> Dynamic wallet loading also needs this to make it good UX.
 366 2018-12-13T19:07:17  <sipa> yeah, fair
 367 2018-12-13T19:07:25  <wumpus> this needs *very* good tests
 368 2018-12-13T19:07:27  <provoostenator> Otherwise you'd have to manually load each time you start.
 369 2018-12-13T19:07:44  <sipa> provoostenator: i was expecting that to go in the qt settings
 370 2018-12-13T19:07:50  <wumpus> especially for the qt case ast hat's even crazier
 371 2018-12-13T19:08:14  <wumpus> how many levels of overlapping options can you have :)
 372 2018-12-13T19:08:21  <provoostenator> It has a bunch of Boost tests, but can probably use some QT tests and functional tests.
 373 2018-12-13T19:08:38  <wumpus> though this is supposed to get rid of most of the qt settings I guess?
 374 2018-12-13T19:08:46  <gmaxwell> obviously we should store what wallets get loaded in wallet.dat...
 375 2018-12-13T19:08:50  <jonasschnelli> Overlapping options is already problematic on the QT layer...
 376 2018-12-13T19:08:54  <provoostenator> Well, fewer levels thanks to my followup #12833
 377 2018-12-13T19:08:54  <luke-jr> wumpus: in a subsequent PR
 378 2018-12-13T19:08:55  <sipa> gmaxwell: wahaha
 379 2018-12-13T19:08:57  <gribble> https://github.com/bitcoin/bitcoin/issues/12833 | [qt] move QSettings to bitcoin_rw.conf where possible by Sjors · Pull Request #12833 · bitcoin/bitcoin · GitHub
 380 2018-12-13T19:09:01  <wumpus> gmaxwell: yesss
 381 2018-12-13T19:09:17  <wumpus> luke-jr: right
 382 2018-12-13T19:09:35  <provoostenator> But I'm reluctant to add more settings, as that makes rebasing 12833 a pain.
 383 2018-12-13T19:09:54  <wumpus> luke-jr: so eventually it'll replace non-pure-user-interface settings in the qt settings, I mean?
 384 2018-12-13T19:10:01  <provoostenator> (and that one also needs tests, I haven't delved into how to write QT tests yet, can someone make me a Python framework?)
 385 2018-12-13T19:10:04  <luke-jr> wumpus: ideally
 386 2018-12-13T19:10:28  <provoostenator> Yeah the idea is to get rid of QTsettings, with the exception of the datadir.
 387 2018-12-13T19:10:34  <jonasschnelli> I guess the QT settings files will always be there for window sizes and such... but non critical settings for sure
 388 2018-12-13T19:10:39  <wumpus> qtsettings is fine for *ui* settings
 389 2018-12-13T19:10:45  <wumpus> such as the last position of the window
 390 2018-12-13T19:10:46  <provoostenator> And what jonasschnelli says.
 391 2018-12-13T19:10:48  <wumpus> and things like that
 392 2018-12-13T19:10:48  <jonasschnelli> indeed
 393 2018-12-13T19:10:57  <wumpus> but not for settings shared with the core
 394 2018-12-13T19:11:02  <provoostenator> But not stuff that's shared with bitcoind like prune=
 395 2018-12-13T19:11:03  <jonasschnelli> yes
 396 2018-12-13T19:11:07  <wumpus> such as dbcache, etc
 397 2018-12-13T19:11:10  <wumpus> right.
 398 2018-12-13T19:11:25  <jonasschnelli> That was just a bypass of not having a way to write to a config section
 399 2018-12-13T19:11:45  <luke-jr> ?
 400 2018-12-13T19:12:09  <jonasschnelli> We wanted dbcache, proxys in Qt but didn't had a way to write to the config, so we added another layer
 401 2018-12-13T19:12:15  <wumpus> yep
 402 2018-12-13T19:12:37  <jonasschnelli> Which is no longer a clever approach once we have rw_config
 403 2018-12-13T19:12:46  <sipa> i'd be more comfortable if the set of modifiable settings (the list of wallets?) were distinct from those that aren't
 404 2018-12-13T19:12:53  <wumpus> we definitely don't want to write to bitcoin.conf directly, but having a separate configuration file that's writable is fine
 405 2018-12-13T19:13:02  <sipa> and have the GUI (where you really want everything to be modifable) directly modify bitcoin.conf
 406 2018-12-13T19:13:07  <wumpus> noooooo
 407 2018-12-13T19:13:14  <wumpus> don't write to bitcoin.conf, never
 408 2018-12-13T19:13:18  <sipa> if you use the GUI, the GUI manages the settings
 409 2018-12-13T19:13:23  <wumpus> conf should be read only
 410 2018-12-13T19:13:24  <sipa> if you don't, the config file does
 411 2018-12-13T19:13:26  <wumpus> we've been over this, please
 412 2018-12-13T19:13:35  <sipa> right, or have a separate config entirely
 413 2018-12-13T19:13:48  <wumpus> let's not change the idea now
 414 2018-12-13T19:13:56  <sipa> i really dislike having a UI edit things at the same level as the user is supposed to
 415 2018-12-13T19:13:59  <sipa> okay.
 416 2018-12-13T19:14:00  * sipa shuts up
 417 2018-12-13T19:14:40  <wumpus> we can have this discussion for a few years then never decide to do anything
 418 2018-12-13T19:14:50  <provoostenator> There's also a bunch of settings that can go straight into the wallet payload, like RBF and address type (though not their defaults).
 419 2018-12-13T19:14:58  <provoostenator> (or maybe even their defaults)
 420 2018-12-13T19:15:25  <provoostenator> #13044
 421 2018-12-13T19:15:26  <gribble> https://github.com/bitcoin/bitcoin/issues/13044 | [RFC] Long term plan for wallet command-line args · Issue #13044 · bitcoin/bitcoin · GitHub
 422 2018-12-13T19:15:39  <jonasschnelli> maybe this also raises the question how to distinct global settings between wallets
 423 2018-12-13T19:15:45  <provoostenator> That also simplifies QT, because wallet settings can be updated without any of the QTSettings stuff.
 424 2018-12-13T19:16:01  <wumpus> I'm unconfortable with writing to bitcoin.conf directly, as it is specified with -conf, which might point to a read-only configuration file, it should not be assumed it is writable
 425 2018-12-13T19:16:32  <wumpus> putting settings in the wallet again?
 426 2018-12-13T19:16:37  <gmaxwell> ugh
 427 2018-12-13T19:16:41  <sipa> ugh
 428 2018-12-13T19:16:42  <wumpus> yea deja vu...
 429 2018-12-13T19:16:50  <gmaxwell> putting settings in the wallet I think is even worse now that we have multiwallet.
 430 2018-12-13T19:16:51  <jonasschnelli> wumpus: there are the wallet flags now...
 431 2018-12-13T19:16:55  <provoostenator> wumpus: yes, that was the idea. Wallets have meta data entries. We already put binary flags in the wallet.
 432 2018-12-13T19:16:57  <jonasschnelli> but yeah...not settings
 433 2018-12-13T19:17:07  <sipa> provoostenator: originally all settings were in the wallet, it was pretty bad :)
 434 2018-12-13T19:17:12  <luke-jr> it might make sense for some subset of settings
 435 2018-12-13T19:17:14  <wumpus> sure, wallet-specific metadata should be stored in the wallet
 436 2018-12-13T19:17:18  <wumpus> but not settings
 437 2018-12-13T19:17:19  <gmaxwell> so you load another wallet and then your software starts behaving differently, madness!
 438 2018-12-13T19:17:23  <provoostenator> Yes, obviously only wallet settings.
 439 2018-12-13T19:17:27  <jamesob> write to wallet.conf.qt in datadir, have that override -conf settings?
 440 2018-12-13T19:17:30  <gmaxwell> yea sure wallet specific metadata sure fine whatever.
 441 2018-12-13T19:17:43  <jamesob> *bitcoin.conf.qt
 442 2018-12-13T19:17:44  <wumpus> jamesob: that's exactly the idea behind the bitcoin_rw.conf
 443 2018-12-13T19:17:44  <jonasschnelli> I just though about address type per wallet,... how do we handle different address types with different wallets?
 444 2018-12-13T19:17:46  <luke-jr> eg, you might have a non-segwit wallet, and a segwit wallet
 445 2018-12-13T19:17:49  <provoostenator> See the list in the issue I linked to above.
 446 2018-12-13T19:17:50  <jonasschnelli> *thought
 447 2018-12-13T19:18:07  <jamesob> wumpus: my bad, getting to the meetin glate
 448 2018-12-13T19:18:07  <wumpus> jamesob: it contains writable settings which override what is in bitcoin.conf
 449 2018-12-13T19:18:34  <wumpus> jamesob: this means being able to edit settings through RPC as well as the GUI
 450 2018-12-13T19:18:47  <jamesob> oh cool
 451 2018-12-13T19:18:49  <provoostenator> Some of these make sense int he wallet: addresstype, changetype, discardfee, fallbackfee, keypool, mintxfee, paytxfee, txconfirmtarget, etc. For others it doesn't make sense.
 452 2018-12-13T19:19:07  <gmaxwell> I don't see how fee settings make much sense in a wallet.
 453 2018-12-13T19:19:10  <wumpus> I'm not sure
 454 2018-12-13T19:19:17  <gmaxwell> Addresstype I could buy. maybe.
 455 2018-12-13T19:19:23  <provoostenator> gmaxwell: to make them behave consistently.
 456 2018-12-13T19:19:30  <jonasschnelli> Addresstype certenly,... fees, probably not
 457 2018-12-13T19:19:38  <sipa> addresstype will go away in the brave new world future anyway
 458 2018-12-13T19:19:43  <wumpus> yes
 459 2018-12-13T19:19:43  <gmaxwell> keypool no, it almost could go away.
 460 2018-12-13T19:19:44  <jonasschnelli> But I know a lot of people running a SW and a non-SW wallet in parallel
 461 2018-12-13T19:19:45  <provoostenator> But maybe fee preferences are more mempool weather dependent than wallet specific.
 462 2018-12-13T19:19:59  <gmaxwell> jonasschnelli: sounds brain damaged.
 463 2018-12-13T19:20:15  <gmaxwell> jonasschnelli: do you mean they just want to get keys with different address types?
 464 2018-12-13T19:20:18  <jonasschnelli> It may be,... but it's just how transition phases happens
 465 2018-12-13T19:20:42  <provoostenator> "keypool" might be replaced with a descriptor specific keypool, but I don't think that changes anything. Unless we stop using hardened derivation at the address level.
 466 2018-12-13T19:20:43  <wumpus> jamesob: please review #11082 :)
 467 2018-12-13T19:20:46  <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
 468 2018-12-13T19:20:53  <gmaxwell> in any case, changing wallets doesn't change what network you're on, so changing txfee settings doesn't really follow logically.
 469 2018-12-13T19:20:55  <sipa> provoostenator: yes keypool will go away as well
 470 2018-12-13T19:21:11  <wumpus> gmaxwell: agree
 471 2018-12-13T19:21:18  <luke-jr> gmaxwell: maybe if you're worried the wallets will get linked by behaviour?
 472 2018-12-13T19:21:21  <gmaxwell> provoostenator: keypool was configurable in the first place because it interacted with backup durability.
 473 2018-12-13T19:21:23  <jonasschnelli> gmaxwell: I guess they want a SW wallet that derives P2SH(P2WPKH) and and their old wallet that derives P2PKH... (and not mix them)
 474 2018-12-13T19:21:28  <jamesob> wumpus: roger that
 475 2018-12-13T19:21:33  <provoostenator> I do like the idea of giving config / rw_config wallet specific sections.
 476 2018-12-13T19:21:52  <wumpus> scoping settings on the wallet will be confusing for users, should imo be avoided if possible
 477 2018-12-13T19:22:14  *** riemann has joined #bitcoin-core-dev
 478 2018-12-13T19:22:16  <gmaxwell> provoostenator: so it's not clear to me that forward caching of keys would be configurable in the future, or at least that it would be anything but an advanced thing that users usually shouldn't mess with.
 479 2018-12-13T19:22:26  <wumpus> if anything it's very difficult to come up with a good user interface for that
 480 2018-12-13T19:22:33  <sipa> i think all of those can either be reasonably done at the node level, or turn into descriptor-specific things in the wallet (and thus not need config)
 481 2018-12-13T19:22:43  <provoostenator> gmaxwell: ah I see, making it "just work" could make sense
 482 2018-12-13T19:22:45  * luke-jr suggests ignoring wallet-specific settings for initial rwconf purposes <.<
 483 2018-12-13T19:22:52  <wumpus> luke-jr: yes, I agree
 484 2018-12-13T19:22:59  <gmaxwell> yea agreed.
 485 2018-12-13T19:23:01  <wumpus> luke-jr: one step at a time
 486 2018-12-13T19:23:06  <wumpus> we've been on this step for years
 487 2018-12-13T19:23:45  <wumpus> every time it's the same discussion, though I haven't heard the idea of settings in the wallet seriously proposed for a while :)
 488 2018-12-13T19:23:49  <gmaxwell> to the extent that there are wallet specific things they probably should be in the wallet so they move with the wallet.  Regardless, they don't need to be part of the rwconf discussion.
 489 2018-12-13T19:24:27  <provoostenator> Agree regarding getting rwconf out the door without adding anything to it.
 490 2018-12-13T19:24:43  <jonasschnelli> wumpus: since its only addresstype that may be relevant and will go away over time,... I take back the argument that settings per wallet are relevant
 491 2018-12-13T19:24:53  <wumpus> jonasschnelli: thank you
 492 2018-12-13T19:25:01  <gmaxwell> (I'm just doubtful that there actually are more than a couple wallet specific things, addresstypes seem the most realistic of the ones mentioned here)
 493 2018-12-13T19:25:34  <provoostenator> Indeed, that seems the most important thing to keep consistent per wallet for privacy reasons.
 494 2018-12-13T19:25:50  <jonasschnelli> boolean things like disable-privatekeys can be handles with the 64bit wallet flags
 495 2018-12-13T19:25:54  <jonasschnelli> *handled
 496 2018-12-13T19:26:00  <provoostenator> One day everyone will send to bech32 addresses, and then we'll do v1 SegWit to start the problem over again :-)
 497 2018-12-13T19:26:11  <sipa> jonasschnelli: even that we don't need post descriptors
 498 2018-12-13T19:26:14  <wumpus> at the least, settings that are part of the wallet *should not* be part of the global options sytem
 499 2018-12-13T19:26:23  <wumpus> otherwise it just becomes too many levels
 500 2018-12-13T19:26:26  <sipa> you'd just not a have a descriptor with private keys if you don't want private keys
 501 2018-12-13T19:26:37  <gmaxwell> provoostenator: no, because v1 segwit doesnt' change the address type, bech32 already supports it.
 502 2018-12-13T19:26:49  <gmaxwell> (shocking, we already anticipated that problem... :P )
 503 2018-12-13T19:27:03  *** timothy has joined #bitcoin-core-dev
 504 2018-12-13T19:27:05  <luke-jr> gmaxwell: but p2sh^2 could (independnetly of segwitv1)
 505 2018-12-13T19:27:22  <gmaxwell> provoostenator: I'm sure there will be some broken sites, but it should be much less of an issue.
 506 2018-12-13T19:27:26  <jonasschnelli> sipa: is that similar of saying "you don't need to import private keys if you don't want to have private keys"?
 507 2018-12-13T19:28:08  <jonasschnelli> disable-private key seems to be another layer,... a footgun protection. But seems like we are going OT
 508 2018-12-13T19:28:12  <sipa> jonasschnelli: it would turn no-private-keys into a flag for wallet creation perhaps, to determine what descriptors a new wallet is born with; but it wouldn't affect anything later
 509 2018-12-13T19:28:17  <provoostenator> gmaxwell: but you still don't want to e.g. consistently use or not use schnorr, so you still need to maintain a preference. Even if it's all bech32.
 510 2018-12-13T19:28:24  <provoostenator> *do want
 511 2018-12-13T19:28:33  <luke-jr> so back to rwconf.. <.<
 512 2018-12-13T19:28:34  <sipa> anyway, sure - we could keep it as a footgun protection, but it seems kind of pointless
 513 2018-12-13T19:28:40  <gmaxwell> provoostenator: yes but thats just a property of which keys/descriptors are in the wallet.
 514 2018-12-13T19:28:42  <sipa> yeah, getting offtopic, sorry
 515 2018-12-13T19:28:51  <provoostenator> gmaxwell: good point
 516 2018-12-13T19:28:51  <wumpus> I think it's time to move to the next topic
 517 2018-12-13T19:29:06  <provoostenator> Descriptors are very useful.
 518 2018-12-13T19:29:10  <meshcollider> sipa: I assume you'd still need a way to fail if you try and import private keys though
 519 2018-12-13T19:29:43  <provoostenator> That's the foodgun protection.
 520 2018-12-13T19:29:46  <provoostenator> footgun
 521 2018-12-13T19:30:21  <provoostenator> I see two topics proposed here https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a
 522 2018-12-13T19:30:29  <provoostenator> jamesob and moneyball
 523 2018-12-13T19:30:32  <wumpus> #topic Asta la la Vista
 524 2018-12-13T19:30:51  <jamesob> Terminator movies?
 525 2018-12-13T19:31:03  <provoostenator> Dropping Vista support
 526 2018-12-13T19:31:07  <wumpus> so the idea to move the minimum supported windows to W7 recently came up in #14922
 527 2018-12-13T19:31:09  <gribble> https://github.com/bitcoin/bitcoin/issues/14922 | [WIP] windows: Set _WIN32_WINNT to 0x0601 (Windows 7) by ken2812221 · Pull Request #14922 · bitcoin/bitcoin · GitHub
 528 2018-12-13T19:31:29  <sipa> vista extended support ended on april 11, 2017
 529 2018-12-13T19:31:36  <wumpus> apparently this was already changed in the release notes #12546
 530 2018-12-13T19:31:38  <gribble> https://github.com/bitcoin/bitcoin/issues/12546 | [docs] Minor improvements to Compatibility Notes by randolf · Pull Request #12546 · bitcoin/bitcoin · GitHub
 531 2018-12-13T19:31:46  <sipa> end of mainstream support was in 2009
 532 2018-12-13T19:31:51  <luke-jr> for Linux, we just support latest stable distros, so if we mirror that for Windows, we can move to Win10 :P
 533 2018-12-13T19:32:05  <sipa> it was also one of the least popular windows releases...
 534 2018-12-13T19:32:14  <wumpus> so I think dropping support for Vista is pretty non-controversial
 535 2018-12-13T19:32:15  <wumpus> yes
 536 2018-12-13T19:32:22  <wumpus> it wasn't even popular at the time
 537 2018-12-13T19:32:29  <achow101> +1
 538 2018-12-13T19:32:32  <jonasschnelli> ack
 539 2018-12-13T19:32:34  <meshcollider> Yep
 540 2018-12-13T19:32:41  <sipa> so i think there is no real to keep it
 541 2018-12-13T19:32:51  <provoostenator> I'm a Mac fanboy, so conflict of interest :-)
 542 2018-12-13T19:32:54  <sipa> i actually expect less opposition than the opposition we got from dropping XP support
 543 2018-12-13T19:33:08  <gmaxwell> People are still using XP, I think less so than vista.
 544 2018-12-13T19:33:13  <luke-jr> sipa: this may be the first time we actually break XP, note
 545 2018-12-13T19:33:15  <wumpus> luke-jr: W7 is still used quite a lot by people unwilling to move to W10
 546 2018-12-13T19:33:16  <gmaxwell> er more so than vista.
 547 2018-12-13T19:33:38  <gmaxwell> the bigger issue will be W10 which is kinda spyware land windows from what I'm told.
 548 2018-12-13T19:33:50  <wumpus> yes
 549 2018-12-13T19:33:51  <meshcollider> Most people consider W7 as a strict improvement over vista so there should be minimal resistance :)
 550 2018-12-13T19:33:52  <sipa> is there anything between w7 and w10?
 551 2018-12-13T19:33:59  <luke-jr> 8.1
 552 2018-12-13T19:34:08  <wumpus> there's a windows 8 but I don't think it ever caught on much
 553 2018-12-13T19:34:40  <sipa> ah yes
 554 2018-12-13T19:34:47  <sipa> anyway, ack dropping vista support
 555 2018-12-13T19:34:52  <wumpus> ok
 556 2018-12-13T19:35:02  <sipa> based on what i read; not a windows user myself
 557 2018-12-13T19:35:04  <gmaxwell> that PR will it make it actually not work when it does otherwise?
 558 2018-12-13T19:35:24  *** bitcoin-git has joined #bitcoin-core-dev
 559 2018-12-13T19:35:24  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14953: test: Make g_insecure_rand_ctx thread_local (master...Mf1812-testThreadLocal) https://github.com/bitcoin/bitcoin/pull/14953
 560 2018-12-13T19:35:24  *** bitcoin-git has left #bitcoin-core-dev
 561 2018-12-13T19:35:25  <MarcoFalke> Should fix the thread sanitizer issue ^
 562 2018-12-13T19:35:30  <gmaxwell> That isn't what we mean by supported on other platforms, on linux if you want to try running on really old stuff, you can try but you're own your own.
 563 2018-12-13T19:35:34  <wumpus> yes it sets the minimum API level so that it won't start in <W7, and makes it possible to use newer APIs
 564 2018-12-13T19:35:52  * luke-jr pokes MarcoFalke
 565 2018-12-13T19:36:13  <gmaxwell> ah it gates newer APIs.  Is there a reason to not announce we don't support vista, but only bump that flag when we actually go to use one of those apis?
 566 2018-12-13T19:36:34  <gmaxwell> (just blocking software where it otherwise works doesn't really feel in the spirit of open source)
 567 2018-12-13T19:36:47  <wumpus> you're pedaling back now?
 568 2018-12-13T19:36:58  <luke-jr> gmaxwell: IIRC some PR wants to use Win7 API
 569 2018-12-13T19:37:05  <gmaxwell> luke-jr: okay.
 570 2018-12-13T19:37:05  <wumpus> FWIW qt is also going to break vista support
 571 2018-12-13T19:37:16  <luke-jr> I did suggest making the change in *that* PR..;
 572 2018-12-13T19:37:24  <wumpus> this is for 0.18
 573 2018-12-13T19:37:31  <wumpus> which is still a few months away
 574 2018-12-13T19:37:38  <gmaxwell> wumpus: no wumpus, I'm not "pedaling back"   but the logic given above that we don't support older linux doesn't actually apply to actively breaking older windows.
 575 2018-12-13T19:37:50  <gmaxwell> If it's going to be broken by other changes in any case, thats fine then.
 576 2018-12-13T19:37:53  <luke-jr> eg, ionice_win bumps the Windows API version too, but wasn't merged yet
 577 2018-12-13T19:37:58  <wumpus> gmaxwell: fair enough but luke-jr was talking about supporting *only* windows 10
 578 2018-12-13T19:38:05  <wumpus> which is kind of extreme
 579 2018-12-13T19:38:18  <wumpus> W7 as bottom should be in line with all modern software
 580 2018-12-13T19:38:29  <luke-jr> wumpus: just mentioning it as a comparison to how we handle Linux distros ;)
 581 2018-12-13T19:38:35  <gmaxwell> from all I've heard running bitcoin on windows 10 sounds like it's probably a bad idea in general.
 582 2018-12-13T19:38:38  <meshcollider> From the PR, "#14881 is using inet_pton and it's only for Vista or later. So I create this PR just for that."
 583 2018-12-13T19:38:40  <gribble> https://github.com/bitcoin/bitcoin/issues/14881 | Tests: Contract testing for the procedure AddTimeData by mmachicao · Pull Request #14881 · bitcoin/bitcoin · GitHub
 584 2018-12-13T19:39:05  <gmaxwell> wumpus: okay, concern withdrawn. I just thought it would be sad to gratitiously break it, but since we have a reason to to do, good to go.
 585 2018-12-13T19:39:07  <luke-jr> gmaxwell: but only Windows 10 is actually supports anymore IIRC
 586 2018-12-13T19:39:16  <wumpus> luke-jr: yes, I know it was not seriously, but would be in line with what we do for wsay, MacOSX, but mac users like upgrading a lot more than windows users
 587 2018-12-13T19:39:41  <luke-jr> supported*
 588 2018-12-13T19:39:57  <wumpus> luke-jr: what does microsoft still officially support? only W10? are you sure?
 589 2018-12-13T19:40:03  * luke-jr double checks
 590 2018-12-13T19:40:16  <luke-jr> Win 8.1: Mainstream support ended on January 9, 2018
 591 2018-12-13T19:40:23  <luke-jr> apparently you can pay for support until 2023
 592 2018-12-13T19:40:42  <achow101> windows 7 will have security updates until jan 2020
 593 2018-12-13T19:40:56  <wumpus> great
 594 2018-12-13T19:41:06  <luke-jr> maybe I misunderstand extended support
 595 2018-12-13T19:41:18  <luke-jr> Win 7: Mainstream support ended on January 13, 2015. Extended support until January 14, 2020 (January 2023 for Professional and Enterprise, users will need to pay for security updates).[5][6]
 596 2018-12-13T19:41:39  <wumpus> so w7 is still more or less relevant
 597 2018-12-13T19:41:49  <achow101> yes
 598 2018-12-13T19:42:04  <achow101> and i'm pretty sure a lot of people still use it too
 599 2018-12-13T19:42:09  <wumpus> right
 600 2018-12-13T19:42:26  <wumpus> let's move to next topic
 601 2018-12-13T19:42:51  <wumpus> #topic "add address index" PRs (jamesob)
 602 2018-12-13T19:43:19  <jamesob> I've talked to a number of companies who're clamoring for an address index and we've got four attempts buuuut
 603 2018-12-13T19:43:49  <jamesob> sipa has potentially disabused me of the notion that an addr index is a legitimate approach to just about anything that isn't debugging or analysis
 604 2018-12-13T19:43:49  <jonasschnelli> jamesob: you mean unspent address index?
 605 2018-12-13T19:44:03  <jamesob> sure, or a more general script descriptor index
 606 2018-12-13T19:44:06  <jonasschnelli> I agree with sipa
 607 2018-12-13T19:44:22  <achow101> jamesob: for what are the companies trying to use an address index for?
 608 2018-12-13T19:44:32  <wumpus> an address index over the full block chain never was a good idea, one over the utxo set was considered but never made itin yet
 609 2018-12-13T19:44:41  <jamesob> in any case, I think we should have some sort of recommendation (and ideally a maintained piece of software) to recommend to the folks who want to do addr-indexy thing
 610 2018-12-13T19:44:44  <jamesob> s
 611 2018-12-13T19:45:04  <luke-jr> jamesob: "don't do that" :p
 612 2018-12-13T19:45:08  <gmaxwell> I think unspents indexes are fine, generally, but shouldn't be a priority for the project over other concerns.. if someone wants to come and do the work for them, and to do them well, great.
 613 2018-12-13T19:45:15  <jamesob> achow101: afaict mostly watching for activity on various addresses
 614 2018-12-13T19:45:23  <jonasschnelli> jamesob: you could do the address index externaly via p2p..... look at https://github.com/jonasschnelli/bitcoincore-indexd (experiment)
 615 2018-12-13T19:45:24  <provoostenator> Always happy to test and review those address index PRs. I think that on top of the new index scheme it's not too bad to maintain.
 616 2018-12-13T19:45:40  <luke-jr> what if we have a full TXO/script index, and only expose it in the GUI as a block explorer? (ie, no RPC to be abused)
 617 2018-12-13T19:45:42  <gmaxwell> jamesob: our general recommendation for watching is to import them into a watching wallet.
 618 2018-12-13T19:45:42  <wumpus> *watcing activiity* shouldn't require an index of everything over the whole block chain
 619 2018-12-13T19:45:44  <provoostenator> Or we could come up with a plugin system for indexes, though that's also a can of worms.
 620 2018-12-13T19:45:47  <jamesob> jonasschnelli: indeed, as well as projects like electrs
 621 2018-12-13T19:46:10  <wumpus> wasn't there some external project addressing this?
 622 2018-12-13T19:46:11  <jonasschnelli> I think we should not load a complete address index on the shoulders of Core
 623 2018-12-13T19:46:14  <gmaxwell> jamesob: and I think if there are limitations on importing for watching, we'd like to improve those.
 624 2018-12-13T19:46:18  <luke-jr> provoostenator: I thought we had that now
 625 2018-12-13T19:46:25  <sipa> gmaxwell: yup!
 626 2018-12-13T19:46:31  <wumpus> it's a *huge* database in any case
 627 2018-12-13T19:46:33  <jonasschnelli> But I agree, externally makes it a bit more complex
 628 2018-12-13T19:46:41  <jamesob> in the next few weeks I'm going to be getting in touch with companies to get a more concrete idea of the usecases, so I'll report back with what I find
 629 2018-12-13T19:46:59  <provoostenator> luke-jr: have what? People can compile their own client of course, but that's pretty high bar and can easily lead to rebase nightmares.
 630 2018-12-13T19:47:01  <wumpus> bitcoin core is not meant as a chain analysis platform
 631 2018-12-13T19:47:06  <luke-jr> wumpus: I did it in ~2 GB a year or so ago, IIRC
 632 2018-12-13T19:47:07  <wumpus> you can do forensics using your own tools
 633 2018-12-13T19:47:13  <jamesob> wumpus: very much agree
 634 2018-12-13T19:47:20  <gmaxwell> jonasschnelli: right the problem for externally, is that implementing blockchain consistency is non-trivial and in my expirence most people who want to build an index don't want to deal with that.
 635 2018-12-13T19:47:44  <gmaxwell> jamesob: more information on the actual usecases would be very helpful.
 636 2018-12-13T19:47:49  <wumpus> it's non trivial but not rocket science either
 637 2018-12-13T19:47:54  <jonasschnelli> Sadly people expect fast BIP44 recovery (incl. history). This seems to be the most prominent real-world usecase for an address index
 638 2018-12-13T19:48:07  <wumpus> it shouldn't be that anything non-trivial needs to end up[ in bitcoin core
 639 2018-12-13T19:48:12  <wumpus> we're not the world of non-trivial solutions
 640 2018-12-13T19:48:26  <gmaxwell> jonasschnelli: most (I believe most, many at least) electrum servers don't do history, so I'm not entirely clear on the including history part of your comment.
 641 2018-12-13T19:48:35  <provoostenator> Let's also not forget to update #14053 with concept ACK / NACK so people don't waste time on trying it if it's undesirable.
 642 2018-12-13T19:48:38  <gmaxwell> wumpus: heh.
 643 2018-12-13T19:48:39  <gribble> https://github.com/bitcoin/bitcoin/issues/14053 | Add address-based index (attempt 4?) by marcinja · Pull Request #14053 · bitcoin/bitcoin · GitHub
 644 2018-12-13T19:48:43  <jonasschnelli> gmaxwell: IMO they do (index everything)
 645 2018-12-13T19:49:04  <jonasschnelli> (not electrum personal server though)
 646 2018-12-13T19:49:04  <wumpus> I mean this is another thing that's been *years*
 647 2018-12-13T19:49:09  <gmaxwell> jonasschnelli: no, I know that for a fact-- many electrum servers don't index history.
 648 2018-12-13T19:49:11  <wumpus> really, no one has been able to build a solution to this?
 649 2018-12-13T19:49:14  <gmaxwell> (I just don't know if its most of them)
 650 2018-12-13T19:49:18  <wumpus> not one of all those companies that want it?
 651 2018-12-13T19:49:24  <jamesob> provoostenator: and its sibling #14035
 652 2018-12-13T19:49:25  <luke-jr> wumpus: people capable of it don't want it :p
 653 2018-12-13T19:49:26  <gribble> https://github.com/bitcoin/bitcoin/issues/14035 | Utxoscriptindex by mgrychow · Pull Request #14035 · bitcoin/bitcoin · GitHub
 654 2018-12-13T19:49:37  <sipa> wumpus: blockstream.info uses electrs, which seems to work very well
 655 2018-12-13T19:49:43  <gmaxwell> wumpus: I think the process of learning enough to do it well does a pretty good job of convincing you that you don't want it.
 656 2018-12-13T19:49:51  <wumpus> gmaxwell: heh
 657 2018-12-13T19:49:59  <provoostenator> jonasschnelli: BIP44 recovery can be handled once we have descriptor support for importmulti and slightly more sane behavior (or a replacement for) the keypool.
 658 2018-12-13T19:50:02  <wumpus> sipa: that's a good suggestion then!
 659 2018-12-13T19:50:31  <gmaxwell> provoostenator: history recovery requires a scan of the chain, or a phenominally expensive index.
 660 2018-12-13T19:50:31  <jonasschnelli> provoostenator: you can't recover the history without an address index or a complete rescan (which is not possible for pruned peers)
 661 2018-12-13T19:50:43  <wumpus> https://github.com/romanz/electrs
 662 2018-12-13T19:50:51  <provoostenator> Ok, rescan for pruned nodes isn't there yet.
 663 2018-12-13T19:50:53  <gmaxwell> jonasschnelli: note that an index is also not compatible with pruning. :P
 664 2018-12-13T19:50:56  <wumpus> alrady had it bookmarked apparently :)
 665 2018-12-13T19:50:59  <jonasschnelli> The only light here is the scantxoutset where you can recover within seconds but not the spent-history
 666 2018-12-13T19:51:19  <provoostenator> Though you could use the neutrino filters for that and then refetch the relevant blocks.
 667 2018-12-13T19:51:24  <jonasschnelli> gmaxwell: it could be,... if we just keep the pointers and load the data via p2p once requested
 668 2018-12-13T19:51:32  <gmaxwell> jonasschnelli: I think we'd get a lot more bang for our buck working on making 'wallet without history before x' (pruned wallet) a first class supported thing.
 669 2018-12-13T19:51:48  <jonasschnelli> gmaxwell: completely agree on this
 670 2018-12-13T19:51:54  <gmaxwell> jonasschnelli: transfering 200GB of data to do the input is not really reasonable...
 671 2018-12-13T19:52:24  <gmaxwell> (and then not saving it... this would be pretty harmful to the p2p network if people were actually patient enough to use it. :) )
 672 2018-12-13T19:52:28  <jonasschnelli> gmaxwell: I mean there is a PR that enabled txindex with pruning... and fetches the tx over p2p once requested outside the kept block area
 673 2018-12-13T19:52:39  <jonasschnelli> which is slow.... but for debugging proposes okay
 674 2018-12-13T19:52:43  <luke-jr> (this makes me think again of the concept of prune-to-slow-storage where you can plug in a USB drive when/if you need old data..)
 675 2018-12-13T19:52:50  <gmaxwell> jonasschnelli: oh, I didn't recall that PR.
 676 2018-12-13T19:53:06  <wumpus> luke-jr: prune-to-tape *hides*
 677 2018-12-13T19:53:11  <provoostenator> luke-jr: slow storage being the internet? :-)
 678 2018-12-13T19:53:17  <jonasschnelli> #13014
 679 2018-12-13T19:53:19  <gribble> https://github.com/bitcoin/bitcoin/issues/13014 | Allow txindex in prune mode by jonasschnelli · Pull Request #13014 · bitcoin/bitcoin · GitHub
 680 2018-12-13T19:53:30  <luke-jr> provoostenator: could be local
 681 2018-12-13T19:53:53  <gmaxwell> jonasschnelli: bip 157 filtering could be used similarly.
 682 2018-12-13T19:54:15  <gmaxwell> jonasschnelli: e.g. saved the bip157 filters locally, and scan against them.
 683 2018-12-13T19:54:15  <jonasschnelli> yes... less disk space, more time spent on filtering... but same idea
 684 2018-12-13T19:54:37  <luke-jr> hmm
 685 2018-12-13T19:54:55  <luke-jr> what data do they want returned from the index? maybe we don't need to retain/fetch the block
 686 2018-12-13T19:54:58  *** dviola has joined #bitcoin-core-dev
 687 2018-12-13T19:55:01  *** rh0nj has quit IRC
 688 2018-12-13T19:55:08  <luke-jr> eg, if the client would be happy with a list of block numbers or txids..
 689 2018-12-13T19:55:08  <jamesob> are any of the BIP157/158 PRs on the high prio list? if not, they should be
 690 2018-12-13T19:55:11  <gmaxwell> wumpus: sadly, freeking tape costs about as much as HDDs today...  LTO7 tapes (6TB) cost about $70.
 691 2018-12-13T19:55:28  <wumpus> gmaxwell: ouch
 692 2018-12-13T19:55:38  <provoostenator> So the wallet recovery flow would be: add descriptors, download BIP-157 filters (if you didn't keep them), process filters, fetch a few blocks. User can immediately use their wallet to receive and send.
 693 2018-12-13T19:55:48  <wumpus> gmaxwell: I guess they're more reliable though
 694 2018-12-13T19:55:49  <luke-jr> jamesob: making light wallets stronger has a negative impact on Bitcoin :/
 695 2018-12-13T19:56:08  *** rh0nj has joined #bitcoin-core-dev
 696 2018-12-13T19:56:23  <gmaxwell> provoostenator: I don't really think thats a viable long term workflow either...  rather if you couldn't afford the space to keep them you probably don't want the time/bandwidth to download them.
 697 2018-12-13T19:56:27  <sipa> luke-jr: BIP158 helps our local wallet too
 698 2018-12-13T19:56:28  <provoostenator> luke-jr: not making them possible sends most people to web wallets, not to full nodes.
 699 2018-12-13T19:56:35  <jamesob> luke-jr: better light wallets might help adoption
 700 2018-12-13T19:56:39  <gmaxwell> luke-jr: I'm only interested in it for the local wallets.
 701 2018-12-13T19:57:00  <luke-jr> jamesob: better to not have that adoption..
 702 2018-12-13T19:57:05  <luke-jr> sipa / gmaxwell: yes, that's a point
 703 2018-12-13T19:57:22  <provoostenator> gmaxwell: recovery should be a rare thing anyway, I assume it only happens after a disaster. In which case you'd probably download the whole chain again anyway.
 704 2018-12-13T19:57:32  <sipa> provoostenator: it should be.
 705 2018-12-13T19:57:34  <gmaxwell> jamesob: history has shown otherwise, bip158 doesn't make lite wallets fundimentally more usable than they are now.  They're still massively worse than server driven wallets like electrum or web wallets.
 706 2018-12-13T19:58:05  <gmaxwell> provoostenator: right but thats also a reason that fetching them when you don't have them isn't that interesting, IMO.
 707 2018-12-13T19:58:22  <jamesob> good point - but I guess if existing light wallets switched to bip157 it'd at least ease load on existing full nodes
 708 2018-12-13T19:58:41  <gmaxwell> jamesob: what light wallets?
 709 2018-12-13T19:58:55  <provoostenator> gmawell is that _because_ of bip158 or just because there aren't that many developers working on light (non web, non electrum) wallets? That could change over time.
 710 2018-12-13T19:58:59  <wumpus> 1 minute to go
 711 2018-12-13T19:59:04  <jamesob> anything using bloom-filter-based SPV
 712 2018-12-13T19:59:12  <gmaxwell> jamesob: at least connections I see there is virtually no acutal usage of p2p lite wallets anymore.
 713 2018-12-13T19:59:15  <wumpus> please wrap up the meeting
 714 2018-12-13T19:59:44  <jamesob> I'll report on any compelling usecases I find for addr index, but I suspect sipa et al are right that that's usually just the Wrong way
 715 2018-12-13T19:59:48  <gmaxwell> provoostenator: P2p lite wallets that scan the chain just end up with a very poor user expirence.
 716 2018-12-13T20:00:06  <jamesob> in the meantime I recommend we give concept ACK/NACK to outstanding PRs which are related
 717 2018-12-13T20:00:11  <gmaxwell> and that doesn't really have anything to do with bip158 vs bip37.
 718 2018-12-13T20:00:25  *** timothy has quit IRC
 719 2018-12-13T20:00:36  <gmaxwell> (in fact BIP158 is somewhat worse, but slightly less of a privacy disaster)
 720 2018-12-13T20:00:37  <wumpus> #endmeeting
 721 2018-12-13T20:00:37  <lightningbot> Meeting ended Thu Dec 13 20:00:37 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
 722 2018-12-13T20:00:37  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.html
 723 2018-12-13T20:00:37  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.txt
 724 2018-12-13T20:00:37  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.log.html
 725 2018-12-13T20:01:03  <provoostenator> gmaxwell: I was thinking mostly about Lightning wallets that rely on a lightweight wallet but don't have support for on chain (other than topping up channels). Those don't need to care about unconfirmed transactions.
 726 2018-12-13T20:01:10  <wumpus> jamesob: I think poeople have become tired of arguing, or even explicitly NACKing those things
 727 2018-12-13T20:01:11  <gmaxwell> (it's worse because it takes a client more time to scan the chain than with BIP37, as it has to get quite a bit more data)
 728 2018-12-13T20:01:51  <gmaxwell> wumpus: +1
 729 2018-12-13T20:02:18  <jamesob> wumpus: makes sense. I'll come up with a reply to both PRs that tries to explain why they're not a fit for core
 730 2018-12-13T20:02:30  <wumpus> at least I have, I mean, if you've been in this since 2011 or so, and have to keep telling people something is abad idea while it becomes ever a worse idea (due to block chain growth)
 731 2018-12-13T20:02:32  <jamesob> and maybe we can think about adding something to the developer guidelines about indexing
 732 2018-12-13T20:02:41  <gmaxwell> UTXO-iyx indexes I generally concept ack (at least if they're ones that aren't specific about 'addresses' but work for spk's generally), history ones, I have NAKed.
 733 2018-12-13T20:02:54  <wumpus> gmaxwell: yep
 734 2018-12-13T20:03:30  <sipa> provoostenator: yes, i think BIP157 may be useful for a new class of clients that may become popular
 735 2018-12-13T20:03:56  <jamesob> gmaxwell: so a scriptpubkey index on UTXOs is worthwhile? is that a generally useful index to have for use by core?
 736 2018-12-13T20:03:58  <gmaxwell> jamesob: we have a problem that there aren't good instructions on watchign without an index (and we likely have bugs and limitations that make it unattractive that haven't been adequately reported)..  And for the analysis usecases what people want is an analysis platform, which we're not going to provide but don't really have much to recommend.
 737 2018-12-13T20:04:29  <jamesob> gmaxwell: agree, I don't want to spend my time on the analysis usecases
 738 2018-12-13T20:05:02  <sipa> jamesob: i dislike UTXO index equally, but it's certainly less of a burden
 739 2018-12-13T20:05:05  <gmaxwell> jamesob: It's both useful (esp if we gain pruned wallet support-- but also scantxoutset speedup), but also it's not incompatiable with scalablity.
 740 2018-12-13T20:05:39  <jamesob> good point
 741 2018-12-13T20:05:40  <sipa> gmaxwell: assuming having a local UTXO set is forever
 742 2018-12-13T20:05:52  <gmaxwell> jamesob: it's my personal belief that within 10 years most users won't ever bother fetching the far back history (or only doing so as a background test and not keeping the results).
 743 2018-12-13T20:06:03  <gmaxwell> sipa: yes, well when otherwise becomes viable my thinking may change!
 744 2018-12-13T20:06:09  <sipa> (which at least for the foreseeable future it will be)
 745 2018-12-13T20:06:23  <jamesob> #utreexo :)
 746 2018-12-13T20:06:44  <gmaxwell> jamesob: and moreover, with the exception of ultra high end commercial installs, few users would have the resources or patience to fetch 15 years of bitcoin history.
 747 2018-12-13T20:07:19  <wumpus> doesn't need to be 'forever', it's useful *now*
 748 2018-12-13T20:07:23  <gmaxwell> jamesob: so any of indexes on that history are is trouble for the ecosystem, as the indexes are many times less viable resource wise than the history itself.
 749 2018-12-13T20:07:34  <jamesob> gmaxwell: so we'd expect "hobbyists" to run pruned nodes, or do you see some more sophisticated compaction of chain history?
 750 2018-12-13T20:07:57  <sipa> jamesob: providing bulk access to sequential data is *cheap*
 751 2018-12-13T20:08:14  <sipa> i expect nearly everyone to not bother with having the full chain
 752 2018-12-13T20:08:14  <gmaxwell> jamesob: My expirence is that commerical parties are often less tolerant of resource usage and more willing to accept "query blockchain.info" than hobbists.
 753 2018-12-13T20:08:17  <wumpus> UTXO db=sophisticaed compaction of chain history
 754 2018-12-13T20:08:35  <gmaxwell> jamesob: so I think you probably have that part backwards. :)
 755 2018-12-13T20:08:49  <jamesob> wumpus: right but you have to validate the construction of a utxo db...
 756 2018-12-13T20:09:13  <gmaxwell> jamesob: I expect that in the future nodes will download a historical UTXO set, verifying it against an assumevalid in their code, and then sync forward from that.
 757 2018-12-13T20:09:36  <wumpus> jamesob: oh sure if you really want to bypass validation
 758 2018-12-13T20:09:41  <sipa> (or in the even further future, in a utreexo or magical accumulator world, not even have that UTXO set...)
 759 2018-12-13T20:10:12  <gmaxwell> sipa: yes, but so far all those proposals are too slow to use or increase bandwidth 10x fold, which isn't a winning tradeoff.
 760 2018-12-13T20:10:13  <jamesob> gmaxwell wumpus: okay so that leads back into the discussion of some kind of utxo set commitment
 761 2018-12-13T20:10:15  <wumpus> jamesob: you can have my UTXO set if you want, no guarantees though :<
 762 2018-12-13T20:10:24  <jamesob> wumpus: ha generous
 763 2018-12-13T20:10:26  <gmaxwell> jamesob: no, it has nothing to do with a utxo set commitment in fact.
 764 2018-12-13T20:10:27  <sipa> gmaxwell: utreexo is at 2x bandwidth or so
 765 2018-12-13T20:11:06  <sipa> (according to simulation results i heard)
 766 2018-12-13T20:11:12  <jamesob> gmaxwell: you're talking about "assumevalid" in the existing sense and not an assumevalid on a utxo hash, right?
 767 2018-12-13T20:11:22  <provoostenator> What does "utreexo" stand for again?
 768 2018-12-13T20:11:29  <sipa> provoostenator: utxo + tree
 769 2018-12-13T20:11:36  <provoostenator> uTreeXO?
 770 2018-12-13T20:11:38  <jamesob> provoostenator: https://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-utxo-accumulators-and-utreexo/
 771 2018-12-13T20:11:43  <gmaxwell> jamesob: a utxo has, but "utxo commitment" has generally ment putting it in blocks which is irrelevant here.
 772 2018-12-13T20:12:14  <sipa> unfortunately it needs blocks that commit to the utxo *proofs* (not the root) for DoS resistance
 773 2018-12-13T20:13:37  <sipa> imo
 774 2018-12-13T20:13:38  <jamesob> gmaxwell: so what you're saying is that 15 years from now, you think that we'll have a hardcoded hash of some historical utxo snapshot (a la assumevalid) and users will start their chain validation from there?
 775 2018-12-13T20:13:51  <gmaxwell> jamesob: doing an assumevalid merely needs a hash, which we already have... though it would be better if nodes could record their hash for each block to make review easier.
 776 2018-12-13T20:14:20  <gmaxwell> jamesob: Yes, short of breakthroughs in ZKPs I don't think anything else will be viable.
 777 2018-12-13T20:15:38  <gmaxwell> We needed to constrain blocksize to a fraction of the current size to avoid time to initial sync constantly growing up even assuming computers/bandwidth/disk improving by 18% or whatever year over year. Since that isn't the case the validation time will continue to grow.
 778 2018-12-13T20:16:01  <jamesob> gmaxwell: makes sense
 779 2018-12-13T20:16:54  <gmaxwell> And it's already large enough that it pushes many companies to prefer hosted APIs... (even if they don't mind the initial sync time, the risk of having to take it again while the service is down is pretty unattractive).  Some end users are less tolerant of loading delays, some more.
 780 2018-12-13T20:17:40  <gmaxwell> And in any case, the same security argument for assumevalid-scripts would apply: if someone can get away with making really obvious bad changes to the code you're running, all bets are off.
 781 2018-12-13T20:18:20  <gmaxwell> Main gaps in implementation now are that it would be really good to make it easier to audit an assumevalid hash, which maybe implies adding an incrementally updatable hash.
 782 2018-12-13T20:18:32  <gmaxwell> And then just storing and fetching snapshots for the p2p protocol.
 783 2018-12-13T20:19:30  <jamesob> i.e. sipa's rolling MUHASH
 784 2018-12-13T20:20:05  <sipa> i'm leaning towards the ECMH approach now, as it's easier to cache for prevalidated txn etc
 785 2018-12-13T20:20:34  <sipa> it's more CPU, but in generally CPU time that can easily move to separate threads etc
 786 2018-12-13T20:21:00  <jamesob> (for anyone curious about rolling UTXO set hashes: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014337.html)
 787 2018-12-13T20:24:43  <luke-jr> will it upset anyone's reviewing if I rebase #11082 now?
 788 2018-12-13T20:24:45  <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
 789 2018-12-13T20:25:00  <gmaxwell> I don't disagree, however for the 'sync from a snapshot and everyone can validate the hash', we could do something more boring and just compute a conventional hash of the utxo set in the background every 25000 blocks.
 790 2018-12-13T20:25:38  <gmaxwell> There is no particular need to know the hash at every single block... sync logistics mean that it wouldn't be realistic to support syncing from an arbritary block anyways.
 791 2018-12-13T20:27:55  <gmaxwell> The other issue with using muash/ecmh with a sync is that it doesn't really lend itself to incremental validation of fetched chunks. So it would probably need to be used in addition to another hash in any case.
 792 2018-12-13T20:28:39  <gmaxwell> incremental validation of chunks-- I mean you're gonna download the 2gb utxo set from 8 peers... okay, great, you get the whole thing and the echm doesn't match. What now?  You don't even know what peer gave you the bad data.
 793 2018-12-13T20:28:49  <luke-jr> btw, it's not too late to reduce block sizes; if we get an organized dev-supported proposal following Lightning going production-ready, I think we can get enough support
 794 2018-12-13T20:29:14  <gmaxwell> luke-jr: it is too late because it already takes much larger to sync the chain than many people will tolerate.
 795 2018-12-13T20:29:30  <luke-jr> gmaxwell: we can catch up with time
 796 2018-12-13T20:29:44  <gmaxwell> Also, the harm from the significant reduction required would be considerable.
 797 2018-12-13T20:29:54  <luke-jr> not post-Lightning
 798 2018-12-13T20:30:14  <gmaxwell> luke-jr: yes post lightning too, since channel creations/closers still use capacity.
 799 2018-12-13T20:30:33  <luke-jr> less than we need
 800 2018-12-13T20:30:39  <gmaxwell> So you probably want a tree hash so that you can have each peer prove that the chunk they're sending you is consistent with the download you're expecting.
 801 2018-12-13T20:31:57  <sipa> luke-jr: i don't expect there to be a "lightning going production-ready"; there may or may not be a phase where it grows sufficiently mature, and there may or may not be a phase where large scale adoption happens
 802 2018-12-13T20:32:11  <sipa> don't put all eggs in one basket
 803 2018-12-13T20:32:31  <gmaxwell> luke-jr: I don't think it's at all obvious that the result is less than we need.
 804 2018-12-13T20:32:33  <luke-jr> one basket is better than none
 805 2018-12-13T20:32:57  <gmaxwell> Current blocksizes are perfectly survivable if we get out of this fetch the whole history on every install mode.
 806 2018-12-13T20:33:23  *** Murch has quit IRC
 807 2018-12-13T20:33:29  <luke-jr> I don't think changing the security model is an appropriate avenue for that.
 808 2018-12-13T20:34:01  <gmaxwell> I don't agree that its a meaninful change to the security model, but even if it were, it would be a good tradeoff.
 809 2018-12-13T20:34:28  <gmaxwell> having users and businesses not willing to run nodes because bringing one up takes days of syncing is a change to the security model.
 810 2018-12-13T20:34:31  <gmaxwell> unambigiously.
 811 2018-12-13T20:34:46  *** Murch has joined #bitcoin-core-dev
 812 2018-12-13T20:35:32  <gmaxwell> and we're already there, and no reduction will eliminate that. (you're right that assuming (cough) continued computer speedups it would eventually get better if blocks were significantly limited today, but even that would take a long time)
 813 2018-12-13T20:36:16  *** phwalkr has joined #bitcoin-core-dev
 814 2018-12-13T20:37:46  <gmaxwell> luke-jr: not to mention that the political viablity of a decrease is essentially 0, -- something you always knew, as that was also the reason that "the limit can be removed and restored if there are issues!" was a dumb idea on arrival.
 815 2018-12-13T20:39:06  <luke-jr> it's not zero. It just requires more education, less block space need (ie, Lightning), and ideally developer support.
 816 2018-12-13T20:39:54  *** shesek has quit IRC
 817 2018-12-13T20:40:28  *** shesek has joined #bitcoin-core-dev
 818 2018-12-13T20:40:42  <gwillen> given the massive political consequences of merely _not raising_ the size, I can hardly even imagine the idea of lowering it being taken seriously.
 819 2018-12-13T20:43:18  <luke-jr> gwillen: the trolls are off having fun with their altcoin now, no need to appease them anymore (in fact, they will probably encourage it because they think it's a bad idea)
 820 2018-12-13T20:44:08  <gmaxwell> The fundimental issue that many people can't wrap their heads around a limit having any purpose at all still remains. Also the issue of it being totally unclear how much is enough.
 821 2018-12-13T20:44:24  <luke-jr> gmaxwell: that's where education comes in
 822 2018-12-13T20:44:36  <gmaxwell> Education only goes so far.
 823 2018-12-13T20:45:07  *** gabridome has joined #bitcoin-core-dev
 824 2018-12-13T20:45:10  <gwillen> you've proposed reductions in size before, and I do not have the sense that they have ever gotten traction
 825 2018-12-13T20:45:17  <gmaxwell> in any case, the system inherently resists changes, so the deck is stacked against you... this is a good thing usually.
 826 2018-12-13T20:45:20  <gwillen> even among people opposed to increasing it
 827 2018-12-13T20:45:21  <luke-jr> gwillen: not really
 828 2018-12-13T20:45:45  <gmaxwell> luke-jr: I don't personally see a need or use to decrease it now.
 829 2018-12-13T20:46:01  <luke-jr> gmaxwell: because you've given up, it sounds like
 830 2018-12-13T20:46:17  <gmaxwell> Improvements to initial sync are required regardless! and I don't see them as regretable.
 831 2018-12-13T20:46:50  <luke-jr> real improvements to initial sync only work in combination with a block size decrease
 832 2018-12-13T20:46:51  <gmaxwell> and with them I'm not aware of an argument as to why the current size is particularly problematic,  we have _radically_ improved things like block propagation.
 833 2018-12-13T20:47:01  <gmaxwell> (for example)
 834 2018-12-13T20:47:56  <gmaxwell> and with the work on tx relay improvements we'll be able to cut relay overheads by maybe 40x.
 835 2018-12-13T20:48:10  *** Guyver2 has joined #bitcoin-core-dev
 836 2018-12-13T20:48:21  <luke-jr> that's unrelated to initial sync..
 837 2018-12-13T20:49:31  <gmaxwell> exactly, I am pointing out that the current size is okay except for initial sync. And initial sync needs to be safely shortcutted regardless of the ongoing future size. And once it is, initial sync would no longer be a huge issue.
 838 2018-12-13T20:50:53  <luke-jr> I don't agree. That's what I mean by just giving up.
 839 2018-12-13T20:51:18  *** bitcoin-git has joined #bitcoin-core-dev
 840 2018-12-13T20:51:19  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14954: WIP: build: Require python 3.5 (master...Mf1811-buildPython3) https://github.com/bitcoin/bitcoin/pull/14954
 841 2018-12-13T20:51:19  *** bitcoin-git has left #bitcoin-core-dev
 842 2018-12-13T20:51:40  *** Murch has quit IRC
 843 2018-12-13T20:51:42  <gmaxwell> I don't think thats giving up at all.
 844 2018-12-13T20:52:21  <gmaxwell> unless you think that downloading bitcoin node software written by other people and not doing the work of writing it yourself is giving up. :P
 845 2018-12-13T20:52:34  <luke-jr> code can be verified
 846 2018-12-13T20:53:51  <gmaxwell> with an extreme amount of work, and an AV can be verified, with considerably less cost/work than code.
 847 2018-12-13T20:53:55  *** rex4539 has joined #bitcoin-core-dev
 848 2018-12-13T20:54:07  <luke-jr> not if we give up on a full sync
 849 2018-12-13T20:55:18  <gmaxwell> I'm confused by that view.
 850 2018-12-13T20:56:09  <gmaxwell> set -assumevalid=0 and you'll not use it, you'll take a lot longer to sync.  It's still radically cheaper than doing pratically any code audit.  (especially since in the common case of someone bringing up a node when they already had one running, their review need only be to compare between their nodes)
 851 2018-12-13T20:56:17  *** rockhouse has quit IRC
 852 2018-12-13T20:56:18  *** victorSN has quit IRC
 853 2018-12-13T20:56:37  <luke-jr> gmaxwell: but we're talking about leaving in place an infinitely growing initial sync time
 854 2018-12-13T20:56:53  <luke-jr> right _now_ it's easier, but it won't be for long
 855 2018-12-13T20:57:07  *** phwalkr_ has joined #bitcoin-core-dev
 856 2018-12-13T20:58:01  <gmaxwell> The cost of an outsider finding a subtly introduced 'bug' in the code is phenomial, maybe the cost of a review that could reliably catch something like that is on the order of $100,000  ... and that cost goes up over time as wages for experts seem to go up and the codebase increases in size.
 857 2018-12-13T20:58:34  <gmaxwell> It would take a LONG time of growth before the cost of a non-AV sync matches the cost of review that would reliably find a subtle bugdoor.
 858 2018-12-13T20:58:42  *** Murch has joined #bitcoin-core-dev
 859 2018-12-13T20:59:11  <luke-jr> therefore everyone should just trust developers /s
 860 2018-12-13T20:59:29  <gmaxwell> I don't see how that follows.
 861 2018-12-13T21:00:38  *** phwalkr has quit IRC
 862 2018-12-13T21:00:45  <gmaxwell> Verification is costly, there isn't any way around that.
 863 2018-12-13T21:01:27  <luke-jr> it's something people can still do today, for the blockchain
 864 2018-12-13T21:05:37  *** phwalkr has joined #bitcoin-core-dev
 865 2018-12-13T21:08:07  *** phwalkr_ has quit IRC
 866 2018-12-13T21:09:42  <gmaxwell> luke-jr: and they will still be able to do it in 10 years at current growth rates. it will just continue to be burdensome enough that most people will choose not to do so, exactly as is the case for the rest of the code today.
 867 2018-12-13T21:10:43  <gmaxwell> Worrying that it'll take 40 hours of computer time instead of 8  seems penny-wise-and-pound foolish, when it'll take 40 hours of _human_ time to do even a light review of the code.
 868 2018-12-13T21:11:03  <gmaxwell> in terms of aggregate validation, political efforts would probably be better spent making implementations more verifyable.
 869 2018-12-13T21:20:56  <treyzania> inb4 bitcoin node in coq
 870 2018-12-13T21:21:25  <aj> fwiw, with the "do nothing" approach, assuming IBD speeds increase by 18% per year, and chain length grows by 200GB/year, IBD maxes out at around 2.6x current length in 5 years, and finally gets back to current time after ~18 years
 871 2018-12-13T21:23:55  *** jarthur has quit IRC
 872 2018-12-13T21:26:20  <gmaxwell> aj: thanks.
 873 2018-12-13T21:29:38  *** phwalkr has quit IRC
 874 2018-12-13T21:30:32  *** phwalkr has joined #bitcoin-core-dev
 875 2018-12-13T21:31:37  <aj> oh, except 100GB/year is more reasonable? 2MB * 1000 blks/week * 52 weeks/year?
 876 2018-12-13T21:32:27  <aj> that's max of 1.5x in 4 years, back to same time as now in 12 years
 877 2018-12-13T21:33:44  <phantomcircuit> gmaxwell, the only reason i can think you'd want an addrindex is if you have a pile of private keys and cant remember any metadata about them
 878 2018-12-13T21:34:27  <gmaxwell> phantomcircuit: even there, do you really want an addrindex or a utxo spk index? Also if it's actually a  pile, scantxout set will be faster.
 879 2018-12-13T21:35:13  <phantomcircuit> gmaxwell, for whatever accounting reason you need the history not just the current utxo
 880 2018-12-13T21:35:28  <aj> decreasing the weight limit to 2Mweight under those assumptions (so 50GB/year) keeps IBD time increase under 10% from now
 881 2018-12-13T21:36:34  <gmaxwell> phantomcircuit: indeed.  But for that case, you'd still probably be just as well off with bip157 filters on disk, and sequential scanning those.
 882 2018-12-13T21:45:41  *** bitcoin-git has joined #bitcoin-core-dev
 883 2018-12-13T21:45:42  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/9a4334443085...a5aea9654f36
 884 2018-12-13T21:45:42  <bitcoin-git> bitcoin/master faead93 MarcoFalke: test: Make g_insecure_rand_ctx thread_local
 885 2018-12-13T21:45:43  <bitcoin-git> bitcoin/master a5aea96 MarcoFalke: Merge #14953: test: Make g_insecure_rand_ctx thread_local...
 886 2018-12-13T21:45:43  *** bitcoin-git has left #bitcoin-core-dev
 887 2018-12-13T21:45:48  <phantomcircuit> gmaxwell, that wasn't an option when people were clamoring for an addrindex though
 888 2018-12-13T21:46:24  *** bitcoin-git has joined #bitcoin-core-dev
 889 2018-12-13T21:46:24  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #14953: test: Make g_insecure_rand_ctx thread_local (master...Mf1812-testThreadLocal) https://github.com/bitcoin/bitcoin/pull/14953
 890 2018-12-13T21:46:24  *** bitcoin-git has left #bitcoin-core-dev
 891 2018-12-13T21:46:37  <gmaxwell> well it's not an option yet. yea, without more inforation on what people are asking for I'm not sure if people would be satisified by disk-filter powered rescan.
 892 2018-12-13T21:47:22  <phantomcircuit> gmaxwell, im 99% certain what people want is an easy way to import historic payments to an address for accounting reasons
 893 2018-12-13T21:47:42  <phantomcircuit> and have that work with whatever custom wallet stuff they've written
 894 2018-12-13T21:47:48  <phantomcircuit> (ie no creation timestamp)
 895 2018-12-13T21:47:53  <gmaxwell> that absoluely is a reason, I doubt its the only one.
 896 2018-12-13T21:48:16  <phantomcircuit> im sure the other 1% is blockexplorers calling rpc directly
 897 2018-12-13T21:48:38  <gmaxwell> e.g. like I've talked to an exchange who was asking for an index because they thought it would make the daemon into a block explorer and they could use it to answer 'sent from' for blacklisting purposes.
 898 2018-12-13T21:48:54  <gmaxwell> But none of the addrindex proposals would have made that possible, regardless...
 899 2018-12-13T21:49:01  <gmaxwell> because they index outputs, not inputs.
 900 2018-12-13T21:52:06  *** fabianfabian has joined #bitcoin-core-dev
 901 2018-12-13T21:52:10  *** AaronvanW has joined #bitcoin-core-dev
 902 2018-12-13T21:53:56  <adiabat> sorry just got to this but a couple maybe relevant points:
 903 2018-12-13T21:54:23  <kanzure> late hi
 904 2018-12-13T21:54:35  <adiabat> I'm hoping the utreexo thing will help clients sync faster, even without any checkpoints.  If you're running on an HDD, disk I/O is the main delay for IBD
 905 2018-12-13T21:55:00  *** miknotauro has quit IRC
 906 2018-12-13T21:55:12  <adiabat> and with utreexo / other utxo accumulator, you can avoid touching the disk and stay in ram, so you are just cpu limited by signatures
 907 2018-12-13T21:55:36  <adiabat> (which fast computers with SSDs are now anyway, but lots of people run nodes with spinning drives)
 908 2018-12-13T21:56:14  *** chenpo has quit IRC
 909 2018-12-13T21:56:26  <adiabat> also separate point is that the need for a bridge node; these nodes can attach proofs to inputs for the nodes which hage pruned the utxos set away and only hold the accumulator
 910 2018-12-13T21:56:29  <treyzania> I wonder how long it would take to sync off of a bunch of RAIDed floppies
 911 2018-12-13T21:57:09  <adiabat> the bridge node needs a utxo index; not of addresses though.  But I'm not sure if it makes sense to build that into core or keep it as a separate server / node / software
 912 2018-12-13T21:57:45  <adiabat> so that might be a place to put address index software if people don't want it in bitcoind
 913 2018-12-13T21:58:06  <luke-jr> oh wow, I totally forgot about adiabat's thing after Tokyo >_<
 914 2018-12-13T21:58:47  <adiabat> yeah I need to put stuff up publicly.  I've been working on it but it takes longer than I would expect, like evertyhing else :)
 915 2018-12-13T21:58:54  <instagibbs> (btw he quoted 15% overhead during IBD in other channel)
 916 2018-12-13T21:59:40  <adiabat> instagibbs: yeah, overhead depends on how much ram you dedicate to caching the proofs
 917 2018-12-13T22:00:12  <adiabat> with no caching I have 250GB of proofs, so not great
 918 2018-12-13T22:00:49  <adiabat> but it goes down quickly with increased caching.  Also I have a maybe controversial way to use shorter hashes.
 919 2018-12-13T22:01:21  <luke-jr> did kanzure transcribe the discussions in Tokyo? I feel like I should review them now :/
 920 2018-12-13T22:01:24  <adiabat> in that you can argue that you're only needing 2nd preimage resistance, not collision resistance.  But I'm not 100% set on that, even though I'm pretty convinced
 921 2018-12-13T22:02:20  <adiabat> yeah kanzure's transcript is here: http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-utxo-accumulators-and-utreexo/
 922 2018-12-13T22:03:53  <luke-jr> thanks
 923 2018-12-13T22:05:03  <kanzure> :)
 924 2018-12-13T22:05:08  <kanzure> you beat me to it
 925 2018-12-13T22:05:16  <sipa> a rare event.
 926 2018-12-13T22:08:42  *** Tralfaz has quit IRC
 927 2018-12-13T22:11:12  <gmaxwell> adiabat: you can sync entirely in ram today, it just takes 8GB of cache... so you can measure the performance impact of that vs defaults.
 928 2018-12-13T22:12:00  <gmaxwell> adiabat: it's non-trivial, but I am not expecting that it would be better for sync time to double bandwidth usage but avoid the in memory state.
 929 2018-12-13T22:12:21  <sipa> gmaxwell: with a couple hundred MB of cache it seems the overhead is just 15%
 930 2018-12-13T22:13:01  <sipa> if that's true, it starts to sound appealing for far more deployments
 931 2018-12-13T22:13:07  <gmaxwell> well currently.
 932 2018-12-13T22:14:42  <gmaxwell> I'm confused about that, it seems improbable to me. 15% would mean only on the order of _one_ extra hash per input.
 933 2018-12-13T22:15:10  * sipa looks at adiabat 
 934 2018-12-13T22:15:28  <gmaxwell> 2x I believed. :)
 935 2018-12-13T22:15:51  <treyzania> it works *really* great for embedded devices which might have a few hundred MiB memory and flash storage
 936 2018-12-13T22:15:53  *** chenpo has joined #bitcoin-core-dev
 937 2018-12-13T22:16:06  <treyzania> or like rpis
 938 2018-12-13T22:16:08  *** phwalkr has quit IRC
 939 2018-12-13T22:16:49  <adiabat> gmaxwell: it's because so many utxos are short-lived, and this is with relying on "hints" from the archive node you IBD from
 940 2018-12-13T22:17:17  <adiabat> the archive node knows what's happening next and can give TTL values for all the utxos created in a block
 941 2018-12-13T22:17:19  <gmaxwell> Oh I see the few hundred mb cache is basically a cache of recent outputs, plus the top of a tree for non-recent outputs (or similar)?
 942 2018-12-13T22:17:31  <adiabat> better than recent -- upcoming
 943 2018-12-13T22:17:56  <sipa> adiabat: but that doesn't apply for chain tip updates, only historical sync
 944 2018-12-13T22:17:57  <adiabat> that part is trusted, so they could lie and say long-lived utxos will be consumed soon
 945 2018-12-13T22:18:02  <adiabat> right, this is only for IBD
 946 2018-12-13T22:18:18  <sipa> i think the affect on bandwidth to keep in sync is more important
 947 2018-12-13T22:18:23  <adiabat> these optimizations don't help once you're synced up; I've mostly been focusing on IBD so far
 948 2018-12-13T22:19:06  <adiabat> there are other ideas for in-sync, but I haven't worked on that much; with IBD I have a nice data set to work off of to get performance numbers
 949 2018-12-13T22:20:36  <adiabat> archive nodes indicating which utxos will be consumed soon could potentially be useful for IBD with the current levelDB as well I guess
 950 2018-12-13T22:21:11  <adiabat> worst case if the node you're downloading from lies to you, your cache is full of the wrong things and you have to go to disk more and hang up on the lying node
 951 2018-12-13T22:22:37  <gmaxwell> I don't think this is currently an interesting route of optimizations, from an engineering perspective.   Right now the existing software doesn't implement a simple fifo for management of the cache, in part due to the complexity of doing so.  Fifoing inputs would probably have 98% of the locality gain (a question I guess your research could answer), but still less complexity than cache
 952 2018-12-13T22:22:38  <gmaxwell> management hints.
 953 2018-12-13T22:23:28  <adiabat> yeah, fair, it makes more sense with the accumulator proofs in that you prevent having to download a lot of data
 954 2018-12-13T22:23:46  <gmaxwell> Right sounds more useful there.
 955 2018-12-13T22:25:34  <gmaxwell> But even there in IBD you're essentially trading off (say) 200gb * .15 = 30GB more data transfered, in order to avoid having to store 2GB durign sync.  I think under most relaistic models of the relative costs of storage and bandwidth, this tradeoff isn't that amazing.  Having the option of making it would be nice, but I would expect relatively few systems to actually be better off with that
 956 2018-12-13T22:25:35  <gmaxwell> choice.
 957 2018-12-13T22:28:09  <adiabat> for powerful computers it doesn't really help, but for computers with HDDs / space constraints it can help
 958 2018-12-13T22:28:28  <sipa> or with I/O-starved VPSes
 959 2018-12-13T22:28:43  <sipa> and perhaps it's also just faster
 960 2018-12-13T22:28:46  <adiabat> but yeah if you have a fast computer and slow internet, it makes things worse
 961 2018-12-13T22:28:48  <sipa> (to be seen)
 962 2018-12-13T22:29:31  <sipa> and it also makes assumevalid-utxo models trivial
 963 2018-12-13T22:29:40  <sipa> (no snapshot to transfer...)
 964 2018-12-13T22:29:56  <adiabat> well a bech32-encodable snapshot
 965 2018-12-13T22:30:09  <adiabat> hm actually no, it's a few hundred bytes so not quite... :)
 966 2018-12-13T22:30:09  *** shesek has quit IRC
 967 2018-12-13T22:30:54  *** shesek has joined #bitcoin-core-dev
 968 2018-12-13T22:32:34  <gmaxwell> if you're assuming that the node knows the top of the tree, in order to avoid an absurd bandwidth blowup, then you'll have to transfer that.
 969 2018-12-13T22:33:27  <sipa> that just gets relayed along with the first transactions that need it
 970 2018-12-13T22:33:30  <adiabat> you could tranfer the top along with the root, but as soon as you see a few proofs in the mempool, you'll get the top
 971 2018-12-13T22:33:49  <adiabat> right, what sipa said
 972 2018-12-13T22:33:56  <sipa> it's of course still bandwidth; just saying it doesn't need a special store-and-transfer-a-snapshot protocol
 973 2018-12-13T22:34:01  <gmaxwell> Okay, but thats just hiding the cost.
 974 2018-12-13T22:34:38  <sipa> by trivial i meant complexity, not bandwidth - but even there you're probably right that it's complexity we're paying anyway in a more complex block/tx sync protocl
 975 2018-12-13T22:34:53  <gmaxwell> sipa: it does need a snapshot, because you cannot run from the tip without wreaking security, you'll need the state as of the point you know is valid.
 976 2018-12-13T22:35:20  <adiabat> yeah, network bandwidth is the big cost to this, CPU cost is low
 977 2018-12-13T22:35:23  <gmaxwell> (I almost agreed with you for a moment there though! :P )
 978 2018-12-13T22:35:44  <sipa> gmaxwell: ok, sure - but it's a few hundred bytes
 979 2018-12-13T22:38:20  *** Guyver2 has quit IRC
 980 2018-12-13T22:38:22  *** michaelsdunn1 has quit IRC
 981 2018-12-13T22:38:47  <gmaxwell> really? there are 68m utxo or so, 1000 bytes would allow for 32 hashes.   for 68m hashes we need a tree of depth 29.  or 24 if 5 levels are saved from the top, so that results in 768 bytes per input of membership proof.
 982 2018-12-13T22:39:16  <gmaxwell> so I think not a few hundreds bytes... certantly a lot smaller than the whole utxo set though... assuming you don't want to be able to offer proofs yourself.
 983 2018-12-13T22:40:29  *** sipa has quit IRC
 984 2018-12-13T22:41:44  *** sipa has joined #bitcoin-core-dev
 985 2018-12-13T22:41:59  <sipa> what did i miss?
 986 2018-12-13T22:42:05  <luke-jr> pizza
 987 2018-12-13T22:42:10  <gmaxwell> dunno if you missed anything.
 988 2018-12-13T22:42:14  <sipa> that's okay
 989 2018-12-13T22:42:17  <gmaxwell> did you see
 990 2018-12-13T22:42:18  <gmaxwell> 14:38:47 < gmaxwell> really? there are 68m utxo or so, 1000 bytes would allow for 32 hashes.   for 68m hashes we need a tree of depth 29.  or 24 if 5
 991 2018-12-13T22:42:18  <gmaxwell>                      levels are saved from the top, so that results in 768 bytes per input of membership proof.
 992 2018-12-13T22:42:18  <gmaxwell> 14:39:16 < gmaxwell> so I think not a few hundreds bytes... certantly a lot smaller than the whole utxo set though... assuming you don't want to be
 993 2018-12-13T22:42:18  <gmaxwell>                      able to offer proofs yourself.
 994 2018-12-13T22:42:21  <gmaxwell> ?
 995 2018-12-13T22:43:11  <adiabat> I'll clean up the code a bit, but the code I have is not a single tree; it's a forest of perfect binary trees
 996 2018-12-13T22:43:21  <sipa> gmaxwell: the "state" of the utxo set is just the roots of each of the trees in the forest
 997 2018-12-13T22:43:22  <adiabat> so the worst case proof yeah is like 28 high
 998 2018-12-13T22:43:37  <sipa> it's distinct from the cache of elements in the trees (which just gets transferred along with the blocks)
 999 2018-12-13T22:43:55  <adiabat> but you can try to put most of the old utxos on the left in the big tree, and most proofs that you need to transfer are in the smaller trees with high turnover
1000 2018-12-13T22:44:52  <gmaxwell> sipa ... adding 28 hashes per input is a non-starter for general usage. There are some applications, vps in a datacenter where internal bandwidth is free, where it might be attractive.  but for most systems _bandwidth_ is the most limited resource.
1001 2018-12-13T22:45:18  <sipa> gmaxwell: of course
1002 2018-12-13T22:45:37  <adiabat> gmaxwell: 28 hashes per input is too much, agreed.  There's ways to bring that number down significantly
1003 2018-12-13T22:45:43  <sipa> gmaxwell: i think you're missing some of the ideas
1004 2018-12-13T22:46:08  <adiabat> (which I haven't written up, so not you're fault :)
1005 2018-12-13T22:46:37  <gmaxwell> adiabat: it can be brought down by caching more of the top...
1006 2018-12-13T22:46:37  <sipa> but in general i think it's a semantics discussion about what you call state and what you call cache
1007 2018-12-13T22:46:41  <adiabat> but a big part is exploiting the power-law dureation of utxos
1008 2018-12-13T22:46:44  <sipa> gmaxwell: adiabat is well aware
1009 2018-12-13T22:47:17  <gmaxwell> adiabat: That doesn't come about as a result of any physical law or even economic incentive in the current system. :(
1010 2018-12-13T22:47:48  <adiabat> true, a lot of is is a heuristic, so who knows.  if you have uniform random access across the utxo set, it gets worse.
1011 2018-12-13T22:48:55  <adiabat> fitting it to the current 200GB data set we have though, it's quite consistent that many utxos persist for only a few blocks
1012 2018-12-13T22:49:26  <gmaxwell> sipa: please we cannot continue to fail to do something about sync times in favor of conjecture about future ideas with uncertian performance.  Like here "It's 2x" but only under assumptions about spending patterns, that is _really_ misleading and I think it undermines productive discussion.
1013 2018-12-13T22:49:27  <adiabat> (many get created & destroyed in the same block which means they never even touch the accumulator)
1014 2018-12-13T22:50:08  *** wxss has quit IRC
1015 2018-12-13T22:50:34  <sipa> gmaxwell: for the sake of discussion, please just treat the caching we're talking about as what you're calling keeping the top levels of the tree as state
1016 2018-12-13T22:50:40  <adiabat> gmaxwell: I certainly don't mean to over-sell this; it's not a silver bullet, but I think it can help in some cases
1017 2018-12-13T22:50:59  <gmaxwell> adiabat: Sure.
1018 2018-12-13T22:51:38  <sipa> gmaxwell: fwiw, our current performance for sync with dbcache less than the entire UTXO set is also dependent on spending patterns
1019 2018-12-13T22:52:18  <gmaxwell> sipa: sure. (though less than it would be if the dbcache's managment weren't kinda stupid wrt locality)
1020 2018-12-13T22:53:04  <gmaxwell> But "oh look spending patterns changed and with no impact in fees, bandwidth needed by bitcoin nodes just went up 5x" isn't the same as saying dbcache works better with more locality.
1021 2018-12-13T22:53:27  <sipa> yeah, agree
1022 2018-12-13T22:54:07  <sipa> we need a separate average case and worst case analysis w.r.t. bandwidth impact
1023 2018-12-13T22:55:13  <gmaxwell> I am frustrated because these kind of far off researchy things seem to keep getting pulled out as a reason to not do something more near term about the initial synctime problem.
1024 2018-12-13T22:56:16  <sipa> i don't think so
1025 2018-12-13T22:56:32  <gmaxwell> Well, that is how I saw the discussion today.
1026 2018-12-13T22:57:06  <sipa> as far as i'm concerned, the potential prospect of utreexo or similar techniques are only a reason to not support a utxo-address index for me
1027 2018-12-13T22:57:58  <gmaxwell> That makes sense. I wasn't connecting it to that discussion.
1028 2018-12-13T22:58:47  *** Tralfaz has joined #bitcoin-core-dev
1029 2018-12-13T22:59:12  <gmaxwell> We can debate on the necessity of users getting historical data except for research purposes, no such debate exists for utxo though... so some scanning mechenism must be provided even in a utxotree world. ... Ideally one that preserves user privacy. But there can be ways of doing that.
1030 2018-12-13T22:59:13  <adiabat> fwiw I think it's towards the practical / implementable side of research-land.  I certainly don't want to say "this solves everything, don't worry about sync times, I got this" though.
1031 2018-12-13T23:00:15  *** Tralfaz has quit IRC
1032 2018-12-13T23:01:10  <gmaxwell> adiabat: Towards perhaps, but I think you overestimate that.  The gap between what we can imagine and build for testing and what we can safely put into production is pretty big.   Right now Bitcoind's dbcache grows up to some limit then is flushed completely, rather than acting as a fifo queue that exploits locality.  "Make it fifo" sounds trivial, but actually getting it done, and Right, and
1033 2018-12-13T23:01:10  <gmaxwell> being confident that its right is not.
1034 2018-12-13T23:01:23  <sipa> adiabat: well, we'll see - i think you may be underestimating the amount of work needed to deploy archive nodes, and the practically implementing a dos-resistant stateful sync protocol for parts of the utxo tree
1035 2018-12-13T23:02:03  <sipa> adiabat: even after you've characterized all the memory and cpu requirements of a close to fully written up spec
1036 2018-12-13T23:02:12  <adiabat> agreed, it's always harder than it seems.  Also, I don't mean to put it into bitcoind, I'm going to start with more of a server / client model
1037 2018-12-13T23:02:54  <adiabat> I guess in comparison to the class group accumulator it's more implementable though
1038 2018-12-13T23:03:00  <sipa> wahaha
1039 2018-12-13T23:03:14  *** Tralfaz has joined #bitcoin-core-dev
1040 2018-12-13T23:03:29  <sipa> yes
1041 2018-12-13T23:03:29  <gmaxwell> sipa: in any case, I was connecting this discussion back more to the assume valid discusion instead of the indexing discussion.  So to me it sounded like an argument to do nothing because utxotree will solve everything... and I'm dubious that the bandwidth overheads will make it a _general_ solution (obviously a big win in some cases), and I'm dubious as to how fast it can be made available.
1042 2018-12-13T23:03:50  <gmaxwell> adiabat: yea, funny I dunno about that! so like a class group accumulator is an isolated number theory programming project.
1043 2018-12-13T23:04:07  <adiabat> oh yeah definitely don't count on anything I'm doing time-wise, takes forever :)
1044 2018-12-13T23:04:15  <gmaxwell> so just from a software eng I think CGA might be more implementable, but it's just too slow to make useful regardless.
1045 2018-12-13T23:04:21  <sipa> gmaxwell: my current estimate for updating an archival node with a new block is about a year of CPU time
1046 2018-12-13T23:04:30  <sipa> gmaxwell: with class group based accumulators
1047 2018-12-13T23:04:40  <gmaxwell> sipa: lol yes, thats my 'too slow to make useful'
1048 2018-12-13T23:04:53  <sipa> oh of course
1049 2018-12-13T23:05:02  <gmaxwell> but like, _ignoring that_ (lol), in terms of actually switching bitcoin to use it, I expect it would be easier than utxotree.
1050 2018-12-13T23:05:26  <sipa> also ignoring the novel cryptographic assumption? :)
1051 2018-12-13T23:05:34  <adiabat> huh, yeah the proofs are tiny, everything's constant sized, etc.  The utreexo accumulator stuff does have a whole bunch of weird logic.
1052 2018-12-13T23:05:37  <gmaxwell> Even including the cost of having to write a "fast" classgroup acc library (assuming fast doesn't mean anything more than well optimized)
1053 2018-12-13T23:06:43  <sipa> but sure, everything constant size makes a lot of logistical problems go away
1054 2018-12-13T23:07:07  <gmaxwell> including need for protocols that make roundtrips... the bane of everyone's existance.
1055 2018-12-13T23:08:28  *** Giszmo has quit IRC
1056 2018-12-13T23:11:50  <gmaxwell> sipa: in any case, going back to the indexing question... for history indexing part of my concern there is that for some use cases there is a "just don't do that" or "look in the utxo instead" option.  Like "We need a history index to look up from addresses!"  :P   for the utxo set that is much less the case.
1057 2018-12-13T23:11:51  <gmaxwell> so if you imagine that in the future everything is using utxotree, we'll still need to provide some way to look up scriptpubkeys... maybe it's PIR queries to nodes what have an N of M shared copy of the whole database or whatever... but something will have to be provided.
1058 2018-12-13T23:11:54  <gmaxwell> as rescan is already pretty unrealistic and contantly getting worse.
1059 2018-12-13T23:12:29  <sipa> gmaxwell: i'm not sure what use cases really need access to the full UTXO set
1060 2018-12-13T23:12:40  *** promag has joined #bitcoin-core-dev
1061 2018-12-13T23:12:41  <sipa> of course there is a convenience in having it
1062 2018-12-13T23:12:51  <sipa> but there is convenience in having an address index for the whole chain too
1063 2018-12-13T23:13:05  <gmaxwell> sipa: I need to take my wallet offline for a long time or recover a backup and want to spend my funds.
1064 2018-12-13T23:13:36  <gmaxwell> To actually _use_ bitcoin I need that data... vs history is needed for analysis/auditing/etc purposes.
1065 2018-12-13T23:14:06  *** Chris_Stewart_5 has quit IRC
1066 2018-12-13T23:14:11  <gmaxwell> (and for those analysis you probably don't just want an address index in history you want all sorts of other indexes)
1067 2018-12-13T23:15:12  <sipa> gmaxwell: assuming the UTXO set was equally expensive to look things up as in the blockchain (it's obviously not, but just as a hypothetical), would you still say that?
1068 2018-12-13T23:15:37  <gmaxwell> yes, because you can't spend your bitcoins without the utxo data!
1069 2018-12-13T23:16:07  <sipa> sure, but we need access to the chain anyway in recovery scenarios
1070 2018-12-13T23:16:09  <gmaxwell> (I mean if they were equal, I suppose you could just use the blockchain instead. but the utxo part is special because it's actually needed to make any use of all of your coins)
1071 2018-12-13T23:16:39  <gmaxwell> sipa: we don't, the fact that we recover tx history is an artifact of how the software was implemented.
1072 2018-12-13T23:16:53  <gmaxwell> Considering we can't recover tx medatadata, its not like we're actually recovering the real history.
1073 2018-12-13T23:17:06  <gmaxwell> The history can well be useful, for sure, but it is not needed to make use of your bitcoins.
1074 2018-12-13T23:17:19  <gmaxwell> (and other wallets, e.g. electrum, can run without history)
1075 2018-12-13T23:17:25  *** lopp has joined #bitcoin-core-dev
1076 2018-12-13T23:17:27  <sipa> gmaxwell: for a business knowing history may be far more important than the ability to spend coins
1077 2018-12-13T23:18:08  <sipa> my point is that what you're talking about is pretty much a recovery scenario that should be rare; and the only reason you consider is more reasonable than seeing whole history is because it's currently (and presumably later) much cheaper
1078 2018-12-13T23:18:08  <gmaxwell> sipa: I disagree because the thing we can recover from the blockchain is not history, it's partial history-- it lacks metadata which is actually the important information.
1079 2018-12-13T23:18:24  <gmaxwell> moreover, that data is there-- go use a block explorer, there is no need a bitcoin wallet or node need to provide it.
1080 2018-12-13T23:18:29  <sipa> also true; generally you also need information you'd have backed up separately
1081 2018-12-13T23:18:51  * gmaxwell bbl
1082 2018-12-13T23:22:32  *** Giszmo has joined #bitcoin-core-dev
1083 2018-12-13T23:23:09  <lopp> question regarding maintaining code integrity during the development process: is there any way that an adversarial maintainer could merge an altered PR commit? just trying to fully visualize the "chain of code custody" between the steps of peer review and actually being merged
1084 2018-12-13T23:23:58  *** chenpo has quit IRC
1085 2018-12-13T23:25:20  *** fabianfabian has quit IRC
1086 2018-12-13T23:25:45  *** hex17or has quit IRC
1087 2018-12-13T23:38:22  *** Giszmo has quit IRC
1088 2018-12-13T23:42:49  *** shesek has quit IRC
1089 2018-12-13T23:59:19  *** Giszmo has joined #bitcoin-core-dev