12018-12-13T00:10:53  *** Victorsueca has quit IRC
   22018-12-13T00:26:04  *** Chris_Stewart_5 has quit IRC
   32018-12-13T00:32:15  *** smaho has joined #bitcoin-core-dev
   42018-12-13T00:35:46  *** smaho has quit IRC
   52018-12-13T00:41:49  *** _cryptodesktop_i has joined #bitcoin-core-dev
   62018-12-13T00:53:27  *** shesek has quit IRC
   72018-12-13T00:53:49  *** shesek has joined #bitcoin-core-dev
   82018-12-13T00:54:34  *** shesek has quit IRC
   92018-12-13T00:54:56  *** shesek has joined #bitcoin-core-dev
  102018-12-13T01:01:56  *** dviola has joined #bitcoin-core-dev
  112018-12-13T01:06:22  *** miknotauro has quit IRC
  122018-12-13T01:15:54  *** _cryptodesktop_i has quit IRC
  132018-12-13T01:22:56  *** millerti has quit IRC
  142018-12-13T01:38:31  *** promag has quit IRC
  152018-12-13T01:40:41  <gmaxwell> promag: maybe everywhere we should split min_conf  into min_conf_trusted and min_conf_untrusted.  Most wallet behaior defaults to 0,1.
  162018-12-13T01:41:55  <gmaxwell> (or more specifically, the coinselection code by default does something like, 6,6 and if that fails 1,1 and if that fails 0,1.)
  172018-12-13T01:46:01  *** bitcoinjunior has joined #bitcoin-core-dev
  182018-12-13T02:30:02  *** rh0nj has quit IRC
  192018-12-13T02:30:58  *** ap4lmtree has quit IRC
  202018-12-13T02:31:08  *** rh0nj has joined #bitcoin-core-dev
  212018-12-13T02:31:23  *** ap4lmtree has joined #bitcoin-core-dev
  222018-12-13T02:32:05  <phantomcircuit> gmaxwell, what?
  232018-12-13T02:32:15  <phantomcircuit> of min_conf for our own transactions nvm
  242018-12-13T02:34:28  *** DougieBot5000_ has joined #bitcoin-core-dev
  252018-12-13T02:37:55  *** DougieBot5000 has quit IRC
  262018-12-13T02:43:08  *** DougieBot5000_ is now known as DougieBot5000
  272018-12-13T03:15:06  *** ap4lmtree has quit IRC
  282018-12-13T03:15:35  *** ap4lmtree has joined #bitcoin-core-dev
  292018-12-13T03:15:48  *** jonasschnelli has quit IRC
  302018-12-13T03:17:47  *** AaronvanW has quit IRC
  312018-12-13T03:22:57  *** jonasschnelli has joined #bitcoin-core-dev
  322018-12-13T03:25:20  *** ap4lmtree has quit IRC
  332018-12-13T03:28:07  *** ap4lmtree has joined #bitcoin-core-dev
  342018-12-13T03:28:15  *** lopp has quit IRC
  352018-12-13T03:38:54  *** bitcoinjunior has quit IRC
  362018-12-13T03:53:06  *** Murch has quit IRC
  372018-12-13T03:58:48  *** ghost43 has quit IRC
  382018-12-13T03:59:49  *** ghost43 has joined #bitcoin-core-dev
  392018-12-13T04:08:32  *** miknotauro has joined #bitcoin-core-dev
  402018-12-13T04:20:40  *** Murch has joined #bitcoin-core-dev
  412018-12-13T04:31:40  *** spinza has quit IRC
  422018-12-13T04:42:32  *** adam3us has quit IRC
  432018-12-13T04:44:47  *** spinza has joined #bitcoin-core-dev
  442018-12-13T04:46:35  *** shesek has quit IRC
  452018-12-13T04:48:01  *** shesek has joined #bitcoin-core-dev
  462018-12-13T04:48:02  *** adam3us has joined #bitcoin-core-dev
  472018-12-13T05:04:39  *** EREVAN has joined #bitcoin-core-dev
  482018-12-13T05:04:47  *** EREVAN is now known as Guest24281
  492018-12-13T05:05:35  *** droark has joined #bitcoin-core-dev
  502018-12-13T05:18:49  *** ossifrage has quit IRC
  512018-12-13T05:23:09  <droark> Hi. I have a question. On mainnet, about how long does Core need to run before the fee estimation code is considered accurate?
  522018-12-13T05:23:51  <sipa> about twice as long as the delay you want to estimate for
  532018-12-13T05:26:55  *** bitcoinsushi has joined #bitcoin-core-dev
  542018-12-13T05:39:25  *** nelsonhb has joined #bitcoin-core-dev
  552018-12-13T05:51:07  <droark> Thank you.
  562018-12-13T05:55:46  *** midnightmagic has quit IRC
  572018-12-13T06:01:18  <droark> Also, I seem to recall there being some reason why the code can't provide an estimate for TX insertion within one block. I can't remember it, though. Does it have to do with statistical variance?
  582018-12-13T06:01:44  <sipa> yes, the next block may be 1 second away
  592018-12-13T06:01:58  <sipa> no fee can give you a 95% chance of being included in that
  602018-12-13T06:03:00  <droark> Thanks. One last question for now. To clarify, when you say twice as long as the delay, do you mean actual time or # of blocks? I assume the latter but it sounds like I might be wrong.
  612018-12-13T06:04:10  <sipa> #blocks
  622018-12-13T06:04:29  <droark> Thanks.
  632018-12-13T06:07:10  *** hebasto has joined #bitcoin-core-dev
  642018-12-13T06:12:07  *** midnightmagic has joined #bitcoin-core-dev
  652018-12-13T06:14:04  *** jonasschnelli has quit IRC
  662018-12-13T06:14:05  *** jonasschnelli has joined #bitcoin-core-dev
  672018-12-13T06:24:28  *** ghost43 has quit IRC
  682018-12-13T06:24:39  *** ghost43 has joined #bitcoin-core-dev
  692018-12-13T07:01:57  *** Cory has quit IRC
  702018-12-13T07:07:17  *** Pasha has joined #bitcoin-core-dev
  712018-12-13T07:10:19  *** CodeBlue1776 has quit IRC
  722018-12-13T07:10:28  *** Pasha is now known as Cory
  732018-12-13T07:11:22  *** CodeBlue1776 has joined #bitcoin-core-dev
  742018-12-13T07:38:33  *** shesek has quit IRC
  752018-12-13T07:38:53  *** shesek has joined #bitcoin-core-dev
  762018-12-13T07:38:53  *** shesek has joined #bitcoin-core-dev
  772018-12-13T07:49:11  *** dviola has quit IRC
  782018-12-13T07:51:34  *** shesek has quit IRC
  792018-12-13T07:51:58  *** BGL has quit IRC
  802018-12-13T07:54:42  *** shesek has joined #bitcoin-core-dev
  812018-12-13T07:55:43  *** bitcoinsushi has quit IRC
  822018-12-13T08:07:13  *** shesek has quit IRC
  832018-12-13T08:08:35  *** shesek has joined #bitcoin-core-dev
  842018-12-13T08:08:35  *** shesek has joined #bitcoin-core-dev
  852018-12-13T08:13:31  *** shesek has quit IRC
  862018-12-13T08:13:56  *** Murch has quit IRC
  872018-12-13T08:15:00  *** nelsonhb has quit IRC
  882018-12-13T08:49:40  *** booyah_ has joined #bitcoin-core-dev
  892018-12-13T08:49:46  *** booyah has quit IRC
  902018-12-13T08:57:09  *** TX1683 has quit IRC
  912018-12-13T08:59:49  *** shesek has joined #bitcoin-core-dev
  922018-12-13T08:59:49  *** shesek has joined #bitcoin-core-dev
  932018-12-13T09:03:01  *** TX1683 has joined #bitcoin-core-dev
  942018-12-13T09:08:30  *** owowo has joined #bitcoin-core-dev
  952018-12-13T09:10:21  *** TheRec has quit IRC
  962018-12-13T09:35:45  *** BGL has joined #bitcoin-core-dev
  972018-12-13T10:02:16  *** Guyver2 has joined #bitcoin-core-dev
  982018-12-13T10:04:34  *** Sentineo has quit IRC
  992018-12-13T10:07:57  *** TheRec has joined #bitcoin-core-dev
 1002018-12-13T10:07:57  *** TheRec has joined #bitcoin-core-dev
 1012018-12-13T10:27:39  *** promag has joined #bitcoin-core-dev
 1022018-12-13T10:34:43  *** timothy has joined #bitcoin-core-dev
 1032018-12-13T10:35:06  *** spinza has quit IRC
 1042018-12-13T11:05:01  *** rh0nj has quit IRC
 1052018-12-13T11:06:08  *** rh0nj has joined #bitcoin-core-dev
 1062018-12-13T11:25:54  *** spinza has joined #bitcoin-core-dev
 1072018-12-13T11:29:02  *** Sentineo has joined #bitcoin-core-dev
 1082018-12-13T11:35:24  *** Guyver2_ has joined #bitcoin-core-dev
 1092018-12-13T11:37:09  <wumpus> unknown options in the config file are ignored, unknown command line options give an error, I think that's a good middle ground
 1102018-12-13T11:38:07  <wumpus> if you explicitly provide -wallet= on *the command line* with a -disablewallet build I think it's fair to error out, you're requesting something it cannot do
 1112018-12-13T11:38:28  *** Guyver2 has quit IRC
 1122018-12-13T11:38:30  <wumpus> while if there's still left-over wallet configuration in the configuration file, well that probably shouldn't trip up too bad
 1132018-12-13T11:42:50  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 1142018-12-13T11:47:08  <promag> wumpus: I tend to agree with that
 1152018-12-13T11:51:25  *** shesek has quit IRC
 1162018-12-13T11:54:00  *** shesek has joined #bitcoin-core-dev
 1172018-12-13T11:54:00  *** shesek has joined #bitcoin-core-dev
 1182018-12-13T12:16:15  *** fanquake has joined #bitcoin-core-dev
 1192018-12-13T12:16:55  <fanquake> wumpus I think #14741 and #14319 can go in
 1202018-12-13T12:16:56  <gribble> https://github.com/bitcoin/bitcoin/issues/14741 | doc: Indicate -rpcauth option password hashing alg by dongcarl · Pull Request #14741 · bitcoin/bitcoin · GitHub
 1212018-12-13T12:16:58  <gribble> https://github.com/bitcoin/bitcoin/issues/14319 | doc: Fix PSBT howto and example parameters by priscoan · Pull Request #14319 · bitcoin/bitcoin · GitHub
 1222018-12-13T12:17:12  <fanquake> Both trivialish doc changes
 1232018-12-13T12:18:56  *** shesek has quit IRC
 1242018-12-13T12:20:21  *** shesek has joined #bitcoin-core-dev
 1252018-12-13T12:20:21  *** shesek has joined #bitcoin-core-dev
 1262018-12-13T12:25:50  *** shesek has quit IRC
 1272018-12-13T12:26:11  *** shesek has joined #bitcoin-core-dev
 1282018-12-13T12:27:35  *** shesek has quit IRC
 1292018-12-13T12:28:06  <wumpus> fanquake: looks like it!
 1302018-12-13T12:28:16  *** shesek has joined #bitcoin-core-dev
 1312018-12-13T12:29:54  *** Chris_Stewart_5 has quit IRC
 1322018-12-13T12:43:16  *** midnightmagic has quit IRC
 1332018-12-13T12:46:58  *** midnightmagic has joined #bitcoin-core-dev
 1342018-12-13T12:52:51  *** midnightmagic has quit IRC
 1352018-12-13T12:55:18  *** miknotauro has quit IRC
 1362018-12-13T12:58:41  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 1372018-12-13T13:00:48  *** midnightmagic has joined #bitcoin-core-dev
 1382018-12-13T13:03:12  <fanquake> Anyone found any issues testing rc1?
 1392018-12-13T13:03:33  <wumpus> haven't seen any issues reported ye
 1402018-12-13T13:15:41  <wumpus> not sure we've given it enough time yet, though
 1412018-12-13T13:19:13  <wumpus> though rc period for backport releases needs to be less than for major releases
 1422018-12-13T13:20:19  *** promag has quit IRC
 1432018-12-13T13:20:30  <fanquake> fair enough
 1442018-12-13T13:20:54  <fanquake> was just wondering, because there's at least one more fix, plus the PSBT documentation I'm about to backport we could get into .1
 1452018-12-13T13:21:08  <fanquake> Otherwise we'll just wait for .2
 1462018-12-13T13:21:25  <wumpus> but could discuss that at the meeting, along with whether to drop vista support in 0.18.0
 1472018-12-13T13:21:38  <wumpus> right
 1482018-12-13T13:22:56  <fanquake> I was actually considering getting up for the meeting this morning. Had a few things to bring up
 1492018-12-13T13:24:02  <fanquake> Also qt 5.12 for 0.18.0
 1502018-12-13T13:26:05  *** Guyver2__ has joined #bitcoin-core-dev
 1512018-12-13T13:28:52  *** Guyver2_ has quit IRC
 1522018-12-13T13:30:52  <wumpus> right
 1532018-12-13T13:31:03  <wumpus> ok that'd be awesome :)
 1542018-12-13T13:32:08  <wumpus> re: qt, we probably want #14849 (5.9.7) first
 1552018-12-13T13:32:10  <gribble> https://github.com/bitcoin/bitcoin/issues/14849 | depends: qt 5.9.7 by fanquake · Pull Request #14849 · bitcoin/bitcoin · GitHub
 1562018-12-13T13:32:37  <wumpus> that looks ready
 1572018-12-13T13:32:44  <fanquake> wumpus yep. I was going to add that as my "high priority" PR. Once that's in I'll backport to 0.17, then start looking at 5.12
 1582018-12-13T13:36:29  <fanquake> Guess I'll add something else to HP
 1592018-12-13T13:39:39  <wumpus> heh! it's good to have this in, I think, so that it (hopefully) gets some manual testing before release, qt relies mostly on manual testing afterall
 1602018-12-13T13:41:23  <fanquake> hopefully nobody has been trying to use the (now disabled) lcd number functionality :p
 1612018-12-13T13:43:00  *** Guyver2__ has quit IRC
 1622018-12-13T13:44:38  <wumpus> hah just looked what that does and it really emulates a LCD display on the screen, no I don't think we plan on using that :p
 1632018-12-13T13:44:54  <wumpus> always good to speed up qt compile by removing unnecessary modules
 1642018-12-13T13:46:32  <wumpus> I'm happy how fast we got the boost compile already in the same way
 1652018-12-13T13:47:08  <fanquake> Yes. A NO_QT=1 depends build is quite fast
 1662018-12-13T13:47:29  <wumpus> yep even on slow hw
 1672018-12-13T13:58:10  *** shesek has quit IRC
 1682018-12-13T13:58:11  *** Chris_Stewart_5 has quit IRC
 1692018-12-13T13:58:35  *** shesek has joined #bitcoin-core-dev
 1702018-12-13T14:12:39  *** shesek has quit IRC
 1712018-12-13T14:13:51  *** shesek has joined #bitcoin-core-dev
 1722018-12-13T14:14:36  <wumpus> speaking of which, my RISC-V node still runs fine <3
 1732018-12-13T14:18:06  *** shesek has quit IRC
 1742018-12-13T14:19:41  *** shesek has joined #bitcoin-core-dev
 1752018-12-13T14:21:04  *** shesek has joined #bitcoin-core-dev
 1762018-12-13T14:21:17  <fanquake> nice
 1772018-12-13T14:26:26  *** shesek has quit IRC
 1782018-12-13T14:26:45  *** shesek has joined #bitcoin-core-dev
 1792018-12-13T14:29:02  *** wxss has joined #bitcoin-core-dev
 1802018-12-13T14:29:06  *** shesek has joined #bitcoin-core-dev
 1812018-12-13T14:30:30  <fanquake> wumpus have you found a RISC-V board that will run Qt yet :o
 1822018-12-13T14:33:16  <wumpus> fanquake: I'm fairly sure it would run qt (using remote X server) :-) but no, I don't have the extension board with PCI-E functionality so I can't actually connect a monitor
 1832018-12-13T14:33:29  *** shesek has quit IRC
 1842018-12-13T14:33:58  *** shesek has joined #bitcoin-core-dev
 1852018-12-13T14:33:58  *** shesek has joined #bitcoin-core-dev
 1862018-12-13T14:35:09  <wumpus> SiFive Unleashed + Extension board + AMD gfx card could run qt, but the former two are really hard to get hold of. I think that's going to improve though, more RISC-V cores are in the works, maybe next year...
 1872018-12-13T14:35:36  *** shesek has quit IRC
 1882018-12-13T14:35:54  *** shesek has joined #bitcoin-core-dev
 1892018-12-13T14:39:54  <fanquake> wumpus cool. Will have to track something down to experiment with.
 1902018-12-13T14:44:40  *** shesek has quit IRC
 1912018-12-13T14:45:25  *** shesek has joined #bitcoin-core-dev
 1922018-12-13T14:45:35  *** shesek has quit IRC
 1932018-12-13T14:45:36  *** shesek has joined #bitcoin-core-dev
 1942018-12-13T14:48:55  <wumpus> I really hope there will be more affordable boards that can run Linux, next
 1952018-12-13T14:53:51  *** belcher has quit IRC
 1962018-12-13T14:54:49  *** belcher has joined #bitcoin-core-dev
 1972018-12-13T15:01:56  *** shesek has quit IRC
 1982018-12-13T15:03:01  *** shesek has joined #bitcoin-core-dev
 1992018-12-13T15:03:01  *** shesek has joined #bitcoin-core-dev
 2002018-12-13T15:06:40  *** millerti has joined #bitcoin-core-dev
 2012018-12-13T15:08:22  *** shesek has quit IRC
 2022018-12-13T15:09:37  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 2032018-12-13T15:10:54  *** shesek has joined #bitcoin-core-dev
 2042018-12-13T15:10:54  *** shesek has joined #bitcoin-core-dev
 2052018-12-13T15:13:05  *** shesek has quit IRC
 2062018-12-13T15:13:45  *** shesek has joined #bitcoin-core-dev
 2072018-12-13T15:18:56  *** shesek has joined #bitcoin-core-dev
 2082018-12-13T15:25:41  *** shesek has quit IRC
 2092018-12-13T15:28:10  *** shesek has joined #bitcoin-core-dev
 2102018-12-13T15:28:10  *** shesek has joined #bitcoin-core-dev
 2112018-12-13T15:29:40  <luke-jr> would be nice to get POWER gitian builds merged, considering it's a much better deal than RISC-V etc
 2122018-12-13T15:30:00  *** jarthur has joined #bitcoin-core-dev
 2132018-12-13T15:30:35  <luke-jr> shesek: please fix your connection problems sometime in the next 4 hours
 2142018-12-13T15:34:07  *** michaelsdunn1 has quit IRC
 2152018-12-13T15:34:54  <wumpus> luke-jr: what's blocking that?
 2162018-12-13T15:34:58  *** shesek has quit IRC
 2172018-12-13T15:36:59  *** shesek has joined #bitcoin-core-dev
 2182018-12-13T15:37:08  <luke-jr> wumpus: no idea; there's a minor rebase needed, but it sat idle for some time before that
 2192018-12-13T15:39:34  <luke-jr> #14066
 2202018-12-13T15:39:37  <gribble> https://github.com/bitcoin/bitcoin/issues/14066 | gitian-linux: Build binaries for 64-bit POWER by luke-jr · Pull Request #14066 · bitcoin/bitcoin · GitHub
 2212018-12-13T15:40:01  <wumpus> I guess we need some more testers
 2222018-12-13T15:40:33  <wumpus> also: did you fix cfields's remarks yet?
 2232018-12-13T15:41:19  <luke-jr> I answered them, at least
 2242018-12-13T15:41:42  <wumpus> ok
 2252018-12-13T15:43:54  *** davec has quit IRC
 2262018-12-13T15:44:28  *** shesek has quit IRC
 2272018-12-13T15:45:23  *** shesek has joined #bitcoin-core-dev
 2282018-12-13T15:45:35  *** shesek has joined #bitcoin-core-dev
 2292018-12-13T15:46:49  *** promag has joined #bitcoin-core-dev
 2302018-12-13T15:51:55  *** davec has joined #bitcoin-core-dev
 2312018-12-13T15:52:53  *** shesek has quit IRC
 2322018-12-13T15:53:00  *** michaelsdunn1 has joined #bitcoin-core-dev
 2332018-12-13T15:53:56  *** shesek has joined #bitcoin-core-dev
 2342018-12-13T16:02:03  <luke-jr> kinda wish I knew what was going on here: https://bugzilla.mozilla.org/show_bug.cgi?id=1512162
 2352018-12-13T16:02:54  *** shesek has quit IRC
 2362018-12-13T16:07:16  *** jungly has joined #bitcoin-core-dev
 2372018-12-13T16:08:14  <wumpus> that's a *weird* issue, why would that only happen on one platform
 2382018-12-13T16:08:54  *** shesek has joined #bitcoin-core-dev
 2392018-12-13T16:08:54  *** shesek has joined #bitcoin-core-dev
 2402018-12-13T16:10:03  *** promag has quit IRC
 2412018-12-13T16:11:57  *** fanquake has quit IRC
 2422018-12-13T16:12:12  <wumpus> could be a compiler bug
 2432018-12-13T16:17:50  <luke-jr> yes, it's not very clear if the compiler or Linux is screwing up here
 2442018-12-13T16:18:04  <luke-jr> since the VDSO is provided at runtime, I would expect Linux
 2452018-12-13T16:18:32  <luke-jr> but this VDSO code hasn't changed since 2010
 2462018-12-13T16:21:04  *** shesek has quit IRC
 2472018-12-13T16:24:25  *** shesek has joined #bitcoin-core-dev
 2482018-12-13T16:26:24  <cjd> Could have been interesting to see the backtrace on the gettimeofday() to see if there is perhaps something weird about where the timeval is placed
 2492018-12-13T16:26:25  *** shesek has joined #bitcoin-core-dev
 2502018-12-13T16:26:25  *** shesek has joined #bitcoin-core-dev
 2512018-12-13T16:27:24  <cjd> I could imagine something like timeval placed on a 32 bit boundry, linux thinks it's supposed to be on the 64, trashes the stack cookie
 2522018-12-13T16:27:49  <cjd> but I only looked at it for 5 min so my 2c is probably overpriced
 2532018-12-13T16:31:31  *** shesek has quit IRC
 2542018-12-13T16:32:45  *** shesek has joined #bitcoin-core-dev
 2552018-12-13T16:34:27  *** shesek has quit IRC
 2562018-12-13T16:36:59  *** shesek has joined #bitcoin-core-dev
 2572018-12-13T16:43:59  *** promag has joined #bitcoin-core-dev
 2582018-12-13T16:47:42  *** shesek has quit IRC
 2592018-12-13T16:49:18  *** shesek has joined #bitcoin-core-dev
 2602018-12-13T16:52:22  *** shesek has quit IRC
 2612018-12-13T16:52:30  *** Tralfaz has joined #bitcoin-core-dev
 2622018-12-13T16:55:03  *** shesek has joined #bitcoin-core-dev
 2632018-12-13T16:55:03  *** shesek has quit IRC
 2642018-12-13T16:55:03  *** shesek has joined #bitcoin-core-dev
 2652018-12-13T16:56:59  *** shesek has quit IRC
 2662018-12-13T16:58:03  *** timothy has quit IRC
 2672018-12-13T16:58:26  *** promag has quit IRC
 2682018-12-13T17:02:19  *** shesek has joined #bitcoin-core-dev
 2692018-12-13T17:02:19  *** shesek has joined #bitcoin-core-dev
 2702018-12-13T17:04:55  *** miknotauro has joined #bitcoin-core-dev
 2712018-12-13T17:13:10  *** miknotauro has quit IRC
 2722018-12-13T17:31:46  *** wxss has quit IRC
 2732018-12-13T17:39:43  *** jungly has quit IRC
 2742018-12-13T17:40:19  *** promag has joined #bitcoin-core-dev
 2752018-12-13T17:45:27  *** danra has joined #bitcoin-core-dev
 2762018-12-13T17:48:17  *** booyah_ is now known as booyah
 2772018-12-13T17:53:27  *** spinza has quit IRC
 2782018-12-13T17:54:03  *** wxss has joined #bitcoin-core-dev
 2792018-12-13T18:02:09  *** ChanServ sets mode: +o wumpus
 2802018-12-13T18:02:21  *** ChanServ sets mode: -o wumpus
 2812018-12-13T18:03:02  *** kmels has joined #bitcoin-core-dev
 2822018-12-13T18:10:02  *** promag has quit IRC
 2832018-12-13T18:16:43  *** Chris_Stewart_5 has quit IRC
 2842018-12-13T18:17:44  *** Murch has joined #bitcoin-core-dev
 2852018-12-13T18:20:43  *** Giszmo has quit IRC
 2862018-12-13T18:25:54  *** miknotauro has joined #bitcoin-core-dev
 2872018-12-13T18:34:57  *** bitcoinjunior has joined #bitcoin-core-dev
 2882018-12-13T18:39:56  *** Giszmo has joined #bitcoin-core-dev
 2892018-12-13T18:41:40  *** spinza has joined #bitcoin-core-dev
 2902018-12-13T18:42:44  <wumpus> looks like travis is failing on all new PRs
 2912018-12-13T18:43:34  <wumpus> some extrememly scary error
 2922018-12-13T18:43:39  <wumpus> https://travis-ci.org/bitcoin/bitcoin/jobs/467611982
 2932018-12-13T18:44:34  <wumpus> (this is for #14951 which is only a RPC tests change and definitely shouldn't cause like this)
 2942018-12-13T18:44:36  <gribble> https://github.com/bitcoin/bitcoin/issues/14951 | Revert "tests: Support calling add_nodes more than once" by MarcoFalke · Pull Request #14951 · bitcoin/bitcoin · GitHub
 2952018-12-13T18:48:49  <gmaxwell> it appears to be saying that two threads are using the same fastrandomcontext at once.
 2962018-12-13T18:49:05  *** hebasto has quit IRC
 2972018-12-13T18:49:48  <gmaxwell> I didn't know we were running tsan in travis.
 2982018-12-13T18:49:53  <wumpus> oof likely caused by #14624 then
 2992018-12-13T18:49:56  <gribble> https://github.com/bitcoin/bitcoin/issues/14624 | Some simple improvements to the RNG code by sipa · Pull Request #14624 · bitcoin/bitcoin · GitHub
 3002018-12-13T18:50:03  <jonasschnelli> probably...
 3012018-12-13T18:50:42  *** chenpo has joined #bitcoin-core-dev
 3022018-12-13T18:50:56  <gmaxwell> wumpus: looking at src/test/validation_block_tests.cpp it's totally misusing the rng in the test.
 3032018-12-13T18:52:38  <sipa> why did travis not detect this in the pr?
 3042018-12-13T18:54:00  <gmaxwell> I don't even really get why this paticular test exists-- like it's just starting lots of threads that call ProcessNewBlock at once. But ProcessNewBlock pretty much instantly takes a lock.
 3052018-12-13T18:54:12  <wumpus> yeh it's clearly sharing a FastRandomContext between multiple threads
 3062018-12-13T18:54:40  <sipa> actually, it's good to know tsan catches this
 3072018-12-13T18:54:45  *** brianhoffman_ has joined #bitcoin-core-dev
 3082018-12-13T18:54:46  <wumpus> sipa: hmm, maybe because your PR was from before TSAN checking was added
 3092018-12-13T18:54:58  *** brianhoffman has quit IRC
 3102018-12-13T18:54:59  *** brianhoffman_ is now known as brianhoffman
 3112018-12-13T18:55:03  <sipa> ah
 3122018-12-13T18:55:18  <gmaxwell> if so, how come it wasn't caught when tsan checking was added?
 3132018-12-13T18:55:27  <gmaxwell> simple fix, and just a test bug regardless.
 3142018-12-13T18:55:29  *** brianhoffman has quit IRC
 3152018-12-13T18:55:42  <wumpus> it doesn't re-run all the travis runs for allPRs on every merge
 3162018-12-13T18:55:48  <gmaxwell> ah.
 3172018-12-13T18:55:50  <wumpus> might be an old cache, or who knows
 3182018-12-13T18:55:51  <gmaxwell> though I still don't really get the utility of that test.
 3192018-12-13T18:56:19  <wumpus> it's pretty nice that this was aught and that you were able to decipher the error :)
 3202018-12-13T18:57:23  <gmaxwell> for some reason the tsan output is a little weird, it's only giving one side's backtrace.
 3212018-12-13T18:57:44  <gmaxwell> Not my first time decyphering tsan output. :P
 3222018-12-13T18:57:47  *** hex17or has quit IRC
 3232018-12-13T19:00:06  *** hex17or has joined #bitcoin-core-dev
 3242018-12-13T19:00:08  <provoostenator> Meeting?
 3252018-12-13T19:00:13  <wumpus> I guess it tests the locking in ProcessNewBlock in case that becomes less granular inthe future
 3262018-12-13T19:00:20  <wumpus> #startmeeting
 3272018-12-13T19:00:20  <lightningbot> Meeting started Thu Dec 13 19:00:20 2018 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
 3282018-12-13T19:00:20  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
 3292018-12-13T19:00:37  <moneyball> hi
 3302018-12-13T19:00:56  <wumpus> proposed topic: drop Windows Vista support, make minimum supported Windows 7
 3312018-12-13T19:01:03  <moneyball> just one topic proposed during the week - jamesob
 3322018-12-13T19:01:06  <gmaxwell> wumpus: ping people.
 3332018-12-13T19:01:35  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb
 3342018-12-13T19:01:38  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 3352018-12-13T19:01:42  <provoostenator> (ah well, good way to get more people to review that RNG PR post merge)
 3362018-12-13T19:01:43  <achow101> hi
 3372018-12-13T19:01:44  <meshcollider> hi
 3382018-12-13T19:01:45  <jonasschnelli> hi
 3392018-12-13T19:02:04  <luke-jr> hi
 3402018-12-13T19:02:09  <chenpo> hi
 3412018-12-13T19:02:41  <wumpus> #topic High priority for review
 3422018-12-13T19:02:45  <provoostenator> Let's call the topic Asta la la Vista :-)
 3432018-12-13T19:03:19  <wumpus> https://github.com/bitcoin/bitcoin/projects/8 three PRs left on the list right now: #14336 #14565 #14573
 3442018-12-13T19:03:22  <gribble> https://github.com/bitcoin/bitcoin/issues/14336 | net: implement poll by pstratem · Pull Request #14336 · bitcoin/bitcoin · GitHub
 3452018-12-13T19:03:24  <gribble> https://github.com/bitcoin/bitcoin/issues/14565 | Overhaul importmulti logic by sipa · Pull Request #14565 · bitcoin/bitcoin · GitHub
 3462018-12-13T19:03:26  <gribble> https://github.com/bitcoin/bitcoin/issues/14573 | qt: Add Window menu by promag · Pull Request #14573 · bitcoin/bitcoin · GitHub
 3472018-12-13T19:03:27  <wumpus> provoostenator: good idea
 3482018-12-13T19:03:58  <sipa> hi
 3492018-12-13T19:03:59  <wumpus> the poll one should be (almost?) ready to merge
 3502018-12-13T19:04:09  <provoostenator> Nominating #11082
 3512018-12-13T19:04:11  <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
 3522018-12-13T19:04:41  <provoostenator> Needs rebase, but really could use more review before that.
 3532018-12-13T19:04:48  <wumpus> provoostenator: already needs rebase now
 3542018-12-13T19:04:58  <wumpus> (and has, for 24 days)
 3552018-12-13T19:05:12  <jonasschnelli> provoostenator: Maybe take that over from luke-jr
 3562018-12-13T19:05:23  <sipa> does it need concept discussion?
 3572018-12-13T19:05:34  <jonasschnelli> probably...
 3582018-12-13T19:05:38  <provoostenator> I will eventually, but I have 15 other thing on my list. I also think it needs concept discussion.
 3592018-12-13T19:06:02  <luke-jr> it doesn't need taking over, although if someone wants to rebase it sooner, I can push that
 3602018-12-13T19:06:13  <provoostenator> Though we've had IRC chats about it before and everyone seemed to think it was at least a reasonable approach, compared to alterantives.
 3612018-12-13T19:06:18  <wumpus> if it needs concept discussion, it definitely doesn't belong on the high priority for review list
 3622018-12-13T19:06:26  <wumpus> yes, I think it's a reasonable approach
 3632018-12-13T19:06:54  <provoostenator> Alright, maybe add it after rebase?
 3642018-12-13T19:06:55  <wumpus> I dread making init option parsing even more involved, but it's better than the alternatives
 3652018-12-13T19:07:16  <provoostenator> Dynamic wallet loading also needs this to make it good UX.
 3662018-12-13T19:07:17  <sipa> yeah, fair
 3672018-12-13T19:07:25  <wumpus> this needs *very* good tests
 3682018-12-13T19:07:27  <provoostenator> Otherwise you'd have to manually load each time you start.
 3692018-12-13T19:07:44  <sipa> provoostenator: i was expecting that to go in the qt settings
 3702018-12-13T19:07:50  <wumpus> especially for the qt case ast hat's even crazier
 3712018-12-13T19:08:14  <wumpus> how many levels of overlapping options can you have :)
 3722018-12-13T19:08:21  <provoostenator> It has a bunch of Boost tests, but can probably use some QT tests and functional tests.
 3732018-12-13T19:08:38  <wumpus> though this is supposed to get rid of most of the qt settings I guess?
 3742018-12-13T19:08:46  <gmaxwell> obviously we should store what wallets get loaded in wallet.dat...
 3752018-12-13T19:08:50  <jonasschnelli> Overlapping options is already problematic on the QT layer...
 3762018-12-13T19:08:54  <provoostenator> Well, fewer levels thanks to my followup #12833
 3772018-12-13T19:08:54  <luke-jr> wumpus: in a subsequent PR
 3782018-12-13T19:08:55  <sipa> gmaxwell: wahaha
 3792018-12-13T19:08:57  <gribble> https://github.com/bitcoin/bitcoin/issues/12833 | [qt] move QSettings to bitcoin_rw.conf where possible by Sjors · Pull Request #12833 · bitcoin/bitcoin · GitHub
 3802018-12-13T19:09:01  <wumpus> gmaxwell: yesss
 3812018-12-13T19:09:17  <wumpus> luke-jr: right
 3822018-12-13T19:09:35  <provoostenator> But I'm reluctant to add more settings, as that makes rebasing 12833 a pain.
 3832018-12-13T19:09:54  <wumpus> luke-jr: so eventually it'll replace non-pure-user-interface settings in the qt settings, I mean?
 3842018-12-13T19:10:01  <provoostenator> (and that one also needs tests, I haven't delved into how to write QT tests yet, can someone make me a Python framework?)
 3852018-12-13T19:10:04  <luke-jr> wumpus: ideally
 3862018-12-13T19:10:28  <provoostenator> Yeah the idea is to get rid of QTsettings, with the exception of the datadir.
 3872018-12-13T19:10:34  <jonasschnelli> I guess the QT settings files will always be there for window sizes and such... but non critical settings for sure
 3882018-12-13T19:10:39  <wumpus> qtsettings is fine for *ui* settings
 3892018-12-13T19:10:45  <wumpus> such as the last position of the window
 3902018-12-13T19:10:46  <provoostenator> And what jonasschnelli says.
 3912018-12-13T19:10:48  <wumpus> and things like that
 3922018-12-13T19:10:48  <jonasschnelli> indeed
 3932018-12-13T19:10:57  <wumpus> but not for settings shared with the core
 3942018-12-13T19:11:02  <provoostenator> But not stuff that's shared with bitcoind like prune=
 3952018-12-13T19:11:03  <jonasschnelli> yes
 3962018-12-13T19:11:07  <wumpus> such as dbcache, etc
 3972018-12-13T19:11:10  <wumpus> right.
 3982018-12-13T19:11:25  <jonasschnelli> That was just a bypass of not having a way to write to a config section
 3992018-12-13T19:11:45  <luke-jr> ?
 4002018-12-13T19:12:09  <jonasschnelli> We wanted dbcache, proxys in Qt but didn't had a way to write to the config, so we added another layer
 4012018-12-13T19:12:15  <wumpus> yep
 4022018-12-13T19:12:37  <jonasschnelli> Which is no longer a clever approach once we have rw_config
 4032018-12-13T19:12:46  <sipa> i'd be more comfortable if the set of modifiable settings (the list of wallets?) were distinct from those that aren't
 4042018-12-13T19:12:53  <wumpus> we definitely don't want to write to bitcoin.conf directly, but having a separate configuration file that's writable is fine
 4052018-12-13T19:13:02  <sipa> and have the GUI (where you really want everything to be modifable) directly modify bitcoin.conf
 4062018-12-13T19:13:07  <wumpus> noooooo
 4072018-12-13T19:13:14  <wumpus> don't write to bitcoin.conf, never
 4082018-12-13T19:13:18  <sipa> if you use the GUI, the GUI manages the settings
 4092018-12-13T19:13:23  <wumpus> conf should be read only
 4102018-12-13T19:13:24  <sipa> if you don't, the config file does
 4112018-12-13T19:13:26  <wumpus> we've been over this, please
 4122018-12-13T19:13:35  <sipa> right, or have a separate config entirely
 4132018-12-13T19:13:48  <wumpus> let's not change the idea now
 4142018-12-13T19:13:56  <sipa> i really dislike having a UI edit things at the same level as the user is supposed to
 4152018-12-13T19:13:59  <sipa> okay.
 4162018-12-13T19:14:00  * sipa shuts up
 4172018-12-13T19:14:40  <wumpus> we can have this discussion for a few years then never decide to do anything
 4182018-12-13T19:14:50  <provoostenator> There's also a bunch of settings that can go straight into the wallet payload, like RBF and address type (though not their defaults).
 4192018-12-13T19:14:58  <provoostenator> (or maybe even their defaults)
 4202018-12-13T19:15:25  <provoostenator> #13044
 4212018-12-13T19:15:26  <gribble> https://github.com/bitcoin/bitcoin/issues/13044 | [RFC] Long term plan for wallet command-line args · Issue #13044 · bitcoin/bitcoin · GitHub
 4222018-12-13T19:15:39  <jonasschnelli> maybe this also raises the question how to distinct global settings between wallets
 4232018-12-13T19:15:45  <provoostenator> That also simplifies QT, because wallet settings can be updated without any of the QTSettings stuff.
 4242018-12-13T19:16:01  <wumpus> I'm unconfortable with writing to bitcoin.conf directly, as it is specified with -conf, which might point to a read-only configuration file, it should not be assumed it is writable
 4252018-12-13T19:16:32  <wumpus> putting settings in the wallet again?
 4262018-12-13T19:16:37  <gmaxwell> ugh
 4272018-12-13T19:16:41  <sipa> ugh
 4282018-12-13T19:16:42  <wumpus> yea deja vu...
 4292018-12-13T19:16:50  <gmaxwell> putting settings in the wallet I think is even worse now that we have multiwallet.
 4302018-12-13T19:16:51  <jonasschnelli> wumpus: there are the wallet flags now...
 4312018-12-13T19:16:55  <provoostenator> wumpus: yes, that was the idea. Wallets have meta data entries. We already put binary flags in the wallet.
 4322018-12-13T19:16:57  <jonasschnelli> but yeah...not settings
 4332018-12-13T19:17:07  <sipa> provoostenator: originally all settings were in the wallet, it was pretty bad :)
 4342018-12-13T19:17:12  <luke-jr> it might make sense for some subset of settings
 4352018-12-13T19:17:14  <wumpus> sure, wallet-specific metadata should be stored in the wallet
 4362018-12-13T19:17:18  <wumpus> but not settings
 4372018-12-13T19:17:19  <gmaxwell> so you load another wallet and then your software starts behaving differently, madness!
 4382018-12-13T19:17:23  <provoostenator> Yes, obviously only wallet settings.
 4392018-12-13T19:17:27  <jamesob> write to wallet.conf.qt in datadir, have that override -conf settings?
 4402018-12-13T19:17:30  <gmaxwell> yea sure wallet specific metadata sure fine whatever.
 4412018-12-13T19:17:43  <jamesob> *bitcoin.conf.qt
 4422018-12-13T19:17:44  <wumpus> jamesob: that's exactly the idea behind the bitcoin_rw.conf
 4432018-12-13T19:17:44  <jonasschnelli> I just though about address type per wallet,... how do we handle different address types with different wallets?
 4442018-12-13T19:17:46  <luke-jr> eg, you might have a non-segwit wallet, and a segwit wallet
 4452018-12-13T19:17:49  <provoostenator> See the list in the issue I linked to above.
 4462018-12-13T19:17:50  <jonasschnelli> *thought
 4472018-12-13T19:18:07  <jamesob> wumpus: my bad, getting to the meetin glate
 4482018-12-13T19:18:07  <wumpus> jamesob: it contains writable settings which override what is in bitcoin.conf
 4492018-12-13T19:18:34  <wumpus> jamesob: this means being able to edit settings through RPC as well as the GUI
 4502018-12-13T19:18:47  <jamesob> oh cool
 4512018-12-13T19:18:49  <provoostenator> Some of these make sense int he wallet: addresstype, changetype, discardfee, fallbackfee, keypool, mintxfee, paytxfee, txconfirmtarget, etc. For others it doesn't make sense.
 4522018-12-13T19:19:07  <gmaxwell> I don't see how fee settings make much sense in a wallet.
 4532018-12-13T19:19:10  <wumpus> I'm not sure
 4542018-12-13T19:19:17  <gmaxwell> Addresstype I could buy. maybe.
 4552018-12-13T19:19:23  <provoostenator> gmaxwell: to make them behave consistently.
 4562018-12-13T19:19:30  <jonasschnelli> Addresstype certenly,... fees, probably not
 4572018-12-13T19:19:38  <sipa> addresstype will go away in the brave new world future anyway
 4582018-12-13T19:19:43  <wumpus> yes
 4592018-12-13T19:19:43  <gmaxwell> keypool no, it almost could go away.
 4602018-12-13T19:19:44  <jonasschnelli> But I know a lot of people running a SW and a non-SW wallet in parallel
 4612018-12-13T19:19:45  <provoostenator> But maybe fee preferences are more mempool weather dependent than wallet specific.
 4622018-12-13T19:19:59  <gmaxwell> jonasschnelli: sounds brain damaged.
 4632018-12-13T19:20:15  <gmaxwell> jonasschnelli: do you mean they just want to get keys with different address types?
 4642018-12-13T19:20:18  <jonasschnelli> It may be,... but it's just how transition phases happens
 4652018-12-13T19:20:42  <provoostenator> "keypool" might be replaced with a descriptor specific keypool, but I don't think that changes anything. Unless we stop using hardened derivation at the address level.
 4662018-12-13T19:20:43  <wumpus> jamesob: please review #11082 :)
 4672018-12-13T19:20:46  <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
 4682018-12-13T19:20:53  <gmaxwell> in any case, changing wallets doesn't change what network you're on, so changing txfee settings doesn't really follow logically.
 4692018-12-13T19:20:55  <sipa> provoostenator: yes keypool will go away as well
 4702018-12-13T19:21:11  <wumpus> gmaxwell: agree
 4712018-12-13T19:21:18  <luke-jr> gmaxwell: maybe if you're worried the wallets will get linked by behaviour?
 4722018-12-13T19:21:21  <gmaxwell> provoostenator: keypool was configurable in the first place because it interacted with backup durability.
 4732018-12-13T19:21:23  <jonasschnelli> gmaxwell: I guess they want a SW wallet that derives P2SH(P2WPKH) and and their old wallet that derives P2PKH... (and not mix them)
 4742018-12-13T19:21:28  <jamesob> wumpus: roger that
 4752018-12-13T19:21:33  <provoostenator> I do like the idea of giving config / rw_config wallet specific sections.
 4762018-12-13T19:21:52  <wumpus> scoping settings on the wallet will be confusing for users, should imo be avoided if possible
 4772018-12-13T19:22:14  *** riemann has joined #bitcoin-core-dev
 4782018-12-13T19:22:16  <gmaxwell> provoostenator: so it's not clear to me that forward caching of keys would be configurable in the future, or at least that it would be anything but an advanced thing that users usually shouldn't mess with.
 4792018-12-13T19:22:26  <wumpus> if anything it's very difficult to come up with a good user interface for that
 4802018-12-13T19:22:33  <sipa> i think all of those can either be reasonably done at the node level, or turn into descriptor-specific things in the wallet (and thus not need config)
 4812018-12-13T19:22:43  <provoostenator> gmaxwell: ah I see, making it "just work" could make sense
 4822018-12-13T19:22:45  * luke-jr suggests ignoring wallet-specific settings for initial rwconf purposes <.<
 4832018-12-13T19:22:52  <wumpus> luke-jr: yes, I agree
 4842018-12-13T19:22:59  <gmaxwell> yea agreed.
 4852018-12-13T19:23:01  <wumpus> luke-jr: one step at a time
 4862018-12-13T19:23:06  <wumpus> we've been on this step for years
 4872018-12-13T19:23:45  <wumpus> every time it's the same discussion, though I haven't heard the idea of settings in the wallet seriously proposed for a while :)
 4882018-12-13T19:23:49  <gmaxwell> to the extent that there are wallet specific things they probably should be in the wallet so they move with the wallet.  Regardless, they don't need to be part of the rwconf discussion.
 4892018-12-13T19:24:27  <provoostenator> Agree regarding getting rwconf out the door without adding anything to it.
 4902018-12-13T19:24:43  <jonasschnelli> wumpus: since its only addresstype that may be relevant and will go away over time,... I take back the argument that settings per wallet are relevant
 4912018-12-13T19:24:53  <wumpus> jonasschnelli: thank you
 4922018-12-13T19:25:01  <gmaxwell> (I'm just doubtful that there actually are more than a couple wallet specific things, addresstypes seem the most realistic of the ones mentioned here)
 4932018-12-13T19:25:34  <provoostenator> Indeed, that seems the most important thing to keep consistent per wallet for privacy reasons.
 4942018-12-13T19:25:50  <jonasschnelli> boolean things like disable-privatekeys can be handles with the 64bit wallet flags
 4952018-12-13T19:25:54  <jonasschnelli> *handled
 4962018-12-13T19:26:00  <provoostenator> One day everyone will send to bech32 addresses, and then we'll do v1 SegWit to start the problem over again :-)
 4972018-12-13T19:26:11  <sipa> jonasschnelli: even that we don't need post descriptors
 4982018-12-13T19:26:14  <wumpus> at the least, settings that are part of the wallet *should not* be part of the global options sytem
 4992018-12-13T19:26:23  <wumpus> otherwise it just becomes too many levels
 5002018-12-13T19:26:26  <sipa> you'd just not a have a descriptor with private keys if you don't want private keys
 5012018-12-13T19:26:37  <gmaxwell> provoostenator: no, because v1 segwit doesnt' change the address type, bech32 already supports it.
 5022018-12-13T19:26:49  <gmaxwell> (shocking, we already anticipated that problem... :P )
 5032018-12-13T19:27:03  *** timothy has joined #bitcoin-core-dev
 5042018-12-13T19:27:05  <luke-jr> gmaxwell: but p2sh^2 could (independnetly of segwitv1)
 5052018-12-13T19:27:22  <gmaxwell> provoostenator: I'm sure there will be some broken sites, but it should be much less of an issue.
 5062018-12-13T19:27:26  <jonasschnelli> sipa: is that similar of saying "you don't need to import private keys if you don't want to have private keys"?
 5072018-12-13T19:28:08  <jonasschnelli> disable-private key seems to be another layer,... a footgun protection. But seems like we are going OT
 5082018-12-13T19:28:12  <sipa> jonasschnelli: it would turn no-private-keys into a flag for wallet creation perhaps, to determine what descriptors a new wallet is born with; but it wouldn't affect anything later
 5092018-12-13T19:28:17  <provoostenator> gmaxwell: but you still don't want to e.g. consistently use or not use schnorr, so you still need to maintain a preference. Even if it's all bech32.
 5102018-12-13T19:28:24  <provoostenator> *do want
 5112018-12-13T19:28:33  <luke-jr> so back to rwconf.. <.<
 5122018-12-13T19:28:34  <sipa> anyway, sure - we could keep it as a footgun protection, but it seems kind of pointless
 5132018-12-13T19:28:40  <gmaxwell> provoostenator: yes but thats just a property of which keys/descriptors are in the wallet.
 5142018-12-13T19:28:42  <sipa> yeah, getting offtopic, sorry
 5152018-12-13T19:28:51  <provoostenator> gmaxwell: good point
 5162018-12-13T19:28:51  <wumpus> I think it's time to move to the next topic
 5172018-12-13T19:29:06  <provoostenator> Descriptors are very useful.
 5182018-12-13T19:29:10  <meshcollider> sipa: I assume you'd still need a way to fail if you try and import private keys though
 5192018-12-13T19:29:43  <provoostenator> That's the foodgun protection.
 5202018-12-13T19:29:46  <provoostenator> footgun
 5212018-12-13T19:30:21  <provoostenator> I see two topics proposed here https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a
 5222018-12-13T19:30:29  <provoostenator> jamesob and moneyball
 5232018-12-13T19:30:32  <wumpus> #topic Asta la la Vista
 5242018-12-13T19:30:51  <jamesob> Terminator movies?
 5252018-12-13T19:31:03  <provoostenator> Dropping Vista support
 5262018-12-13T19:31:07  <wumpus> so the idea to move the minimum supported windows to W7 recently came up in #14922
 5272018-12-13T19:31:09  <gribble> https://github.com/bitcoin/bitcoin/issues/14922 | [WIP] windows: Set _WIN32_WINNT to 0x0601 (Windows 7) by ken2812221 · Pull Request #14922 · bitcoin/bitcoin · GitHub
 5282018-12-13T19:31:29  <sipa> vista extended support ended on april 11, 2017
 5292018-12-13T19:31:36  <wumpus> apparently this was already changed in the release notes #12546
 5302018-12-13T19:31:38  <gribble> https://github.com/bitcoin/bitcoin/issues/12546 | [docs] Minor improvements to Compatibility Notes by randolf · Pull Request #12546 · bitcoin/bitcoin · GitHub
 5312018-12-13T19:31:46  <sipa> end of mainstream support was in 2009
 5322018-12-13T19:31:51  <luke-jr> for Linux, we just support latest stable distros, so if we mirror that for Windows, we can move to Win10 :P
 5332018-12-13T19:32:05  <sipa> it was also one of the least popular windows releases...
 5342018-12-13T19:32:14  <wumpus> so I think dropping support for Vista is pretty non-controversial
 5352018-12-13T19:32:15  <wumpus> yes
 5362018-12-13T19:32:22  <wumpus> it wasn't even popular at the time
 5372018-12-13T19:32:29  <achow101> +1
 5382018-12-13T19:32:32  <jonasschnelli> ack
 5392018-12-13T19:32:34  <meshcollider> Yep
 5402018-12-13T19:32:41  <sipa> so i think there is no real to keep it
 5412018-12-13T19:32:51  <provoostenator> I'm a Mac fanboy, so conflict of interest :-)
 5422018-12-13T19:32:54  <sipa> i actually expect less opposition than the opposition we got from dropping XP support
 5432018-12-13T19:33:08  <gmaxwell> People are still using XP, I think less so than vista.
 5442018-12-13T19:33:13  <luke-jr> sipa: this may be the first time we actually break XP, note
 5452018-12-13T19:33:15  <wumpus> luke-jr: W7 is still used quite a lot by people unwilling to move to W10
 5462018-12-13T19:33:16  <gmaxwell> er more so than vista.
 5472018-12-13T19:33:38  <gmaxwell> the bigger issue will be W10 which is kinda spyware land windows from what I'm told.
 5482018-12-13T19:33:50  <wumpus> yes
 5492018-12-13T19:33:51  <meshcollider> Most people consider W7 as a strict improvement over vista so there should be minimal resistance :)
 5502018-12-13T19:33:52  <sipa> is there anything between w7 and w10?
 5512018-12-13T19:33:59  <luke-jr> 8.1
 5522018-12-13T19:34:08  <wumpus> there's a windows 8 but I don't think it ever caught on much
 5532018-12-13T19:34:40  <sipa> ah yes
 5542018-12-13T19:34:47  <sipa> anyway, ack dropping vista support
 5552018-12-13T19:34:52  <wumpus> ok
 5562018-12-13T19:35:02  <sipa> based on what i read; not a windows user myself
 5572018-12-13T19:35:04  <gmaxwell> that PR will it make it actually not work when it does otherwise?
 5582018-12-13T19:35:24  *** bitcoin-git has joined #bitcoin-core-dev
 5592018-12-13T19:35:24  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14953: test: Make g_insecure_rand_ctx thread_local (master...Mf1812-testThreadLocal) https://github.com/bitcoin/bitcoin/pull/14953
 5602018-12-13T19:35:24  *** bitcoin-git has left #bitcoin-core-dev
 5612018-12-13T19:35:25  <MarcoFalke> Should fix the thread sanitizer issue ^
 5622018-12-13T19:35:30  <gmaxwell> That isn't what we mean by supported on other platforms, on linux if you want to try running on really old stuff, you can try but you're own your own.
 5632018-12-13T19:35:34  <wumpus> yes it sets the minimum API level so that it won't start in <W7, and makes it possible to use newer APIs
 5642018-12-13T19:35:52  * luke-jr pokes MarcoFalke
 5652018-12-13T19:36:13  <gmaxwell> ah it gates newer APIs.  Is there a reason to not announce we don't support vista, but only bump that flag when we actually go to use one of those apis?
 5662018-12-13T19:36:34  <gmaxwell> (just blocking software where it otherwise works doesn't really feel in the spirit of open source)
 5672018-12-13T19:36:47  <wumpus> you're pedaling back now?
 5682018-12-13T19:36:58  <luke-jr> gmaxwell: IIRC some PR wants to use Win7 API
 5692018-12-13T19:37:05  <gmaxwell> luke-jr: okay.
 5702018-12-13T19:37:05  <wumpus> FWIW qt is also going to break vista support
 5712018-12-13T19:37:16  <luke-jr> I did suggest making the change in *that* PR..;
 5722018-12-13T19:37:24  <wumpus> this is for 0.18
 5732018-12-13T19:37:31  <wumpus> which is still a few months away
 5742018-12-13T19:37:38  <gmaxwell> wumpus: no wumpus, I'm not "pedaling back"   but the logic given above that we don't support older linux doesn't actually apply to actively breaking older windows.
 5752018-12-13T19:37:50  <gmaxwell> If it's going to be broken by other changes in any case, thats fine then.
 5762018-12-13T19:37:53  <luke-jr> eg, ionice_win bumps the Windows API version too, but wasn't merged yet
 5772018-12-13T19:37:58  <wumpus> gmaxwell: fair enough but luke-jr was talking about supporting *only* windows 10
 5782018-12-13T19:38:05  <wumpus> which is kind of extreme
 5792018-12-13T19:38:18  <wumpus> W7 as bottom should be in line with all modern software
 5802018-12-13T19:38:29  <luke-jr> wumpus: just mentioning it as a comparison to how we handle Linux distros ;)
 5812018-12-13T19:38:35  <gmaxwell> from all I've heard running bitcoin on windows 10 sounds like it's probably a bad idea in general.
 5822018-12-13T19:38:38  <meshcollider> From the PR, "#14881 is using inet_pton and it's only for Vista or later. So I create this PR just for that."
 5832018-12-13T19:38:40  <gribble> https://github.com/bitcoin/bitcoin/issues/14881 | Tests: Contract testing for the procedure AddTimeData by mmachicao · Pull Request #14881 · bitcoin/bitcoin · GitHub
 5842018-12-13T19:39:05  <gmaxwell> wumpus: okay, concern withdrawn. I just thought it would be sad to gratitiously break it, but since we have a reason to to do, good to go.
 5852018-12-13T19:39:07  <luke-jr> gmaxwell: but only Windows 10 is actually supports anymore IIRC
 5862018-12-13T19:39:16  <wumpus> luke-jr: yes, I know it was not seriously, but would be in line with what we do for wsay, MacOSX, but mac users like upgrading a lot more than windows users
 5872018-12-13T19:39:41  <luke-jr> supported*
 5882018-12-13T19:39:57  <wumpus> luke-jr: what does microsoft still officially support? only W10? are you sure?
 5892018-12-13T19:40:03  * luke-jr double checks
 5902018-12-13T19:40:16  <luke-jr> Win 8.1: Mainstream support ended on January 9, 2018
 5912018-12-13T19:40:23  <luke-jr> apparently you can pay for support until 2023
 5922018-12-13T19:40:42  <achow101> windows 7 will have security updates until jan 2020
 5932018-12-13T19:40:56  <wumpus> great
 5942018-12-13T19:41:06  <luke-jr> maybe I misunderstand extended support
 5952018-12-13T19:41:18  <luke-jr> Win 7: Mainstream support ended on January 13, 2015. Extended support until January 14, 2020 (January 2023 for Professional and Enterprise, users will need to pay for security updates).[5][6]
 5962018-12-13T19:41:39  <wumpus> so w7 is still more or less relevant
 5972018-12-13T19:41:49  <achow101> yes
 5982018-12-13T19:42:04  <achow101> and i'm pretty sure a lot of people still use it too
 5992018-12-13T19:42:09  <wumpus> right
 6002018-12-13T19:42:26  <wumpus> let's move to next topic
 6012018-12-13T19:42:51  <wumpus> #topic "add address index" PRs (jamesob)
 6022018-12-13T19:43:19  <jamesob> I've talked to a number of companies who're clamoring for an address index and we've got four attempts buuuut
 6032018-12-13T19:43:49  <jamesob> sipa has potentially disabused me of the notion that an addr index is a legitimate approach to just about anything that isn't debugging or analysis
 6042018-12-13T19:43:49  <jonasschnelli> jamesob: you mean unspent address index?
 6052018-12-13T19:44:03  <jamesob> sure, or a more general script descriptor index
 6062018-12-13T19:44:06  <jonasschnelli> I agree with sipa
 6072018-12-13T19:44:22  <achow101> jamesob: for what are the companies trying to use an address index for?
 6082018-12-13T19:44:32  <wumpus> an address index over the full block chain never was a good idea, one over the utxo set was considered but never made itin yet
 6092018-12-13T19:44:41  <jamesob> in any case, I think we should have some sort of recommendation (and ideally a maintained piece of software) to recommend to the folks who want to do addr-indexy thing
 6102018-12-13T19:44:44  <jamesob> s
 6112018-12-13T19:45:04  <luke-jr> jamesob: "don't do that" :p
 6122018-12-13T19:45:08  <gmaxwell> I think unspents indexes are fine, generally, but shouldn't be a priority for the project over other concerns.. if someone wants to come and do the work for them, and to do them well, great.
 6132018-12-13T19:45:15  <jamesob> achow101: afaict mostly watching for activity on various addresses
 6142018-12-13T19:45:23  <jonasschnelli> jamesob: you could do the address index externaly via p2p..... look at https://github.com/jonasschnelli/bitcoincore-indexd (experiment)
 6152018-12-13T19:45:24  <provoostenator> Always happy to test and review those address index PRs. I think that on top of the new index scheme it's not too bad to maintain.
 6162018-12-13T19:45:40  <luke-jr> what if we have a full TXO/script index, and only expose it in the GUI as a block explorer? (ie, no RPC to be abused)
 6172018-12-13T19:45:42  <gmaxwell> jamesob: our general recommendation for watching is to import them into a watching wallet.
 6182018-12-13T19:45:42  <wumpus> *watcing activiity* shouldn't require an index of everything over the whole block chain
 6192018-12-13T19:45:44  <provoostenator> Or we could come up with a plugin system for indexes, though that's also a can of worms.
 6202018-12-13T19:45:47  <jamesob> jonasschnelli: indeed, as well as projects like electrs
 6212018-12-13T19:46:10  <wumpus> wasn't there some external project addressing this?
 6222018-12-13T19:46:11  <jonasschnelli> I think we should not load a complete address index on the shoulders of Core
 6232018-12-13T19:46:14  <gmaxwell> jamesob: and I think if there are limitations on importing for watching, we'd like to improve those.
 6242018-12-13T19:46:18  <luke-jr> provoostenator: I thought we had that now
 6252018-12-13T19:46:25  <sipa> gmaxwell: yup!
 6262018-12-13T19:46:31  <wumpus> it's a *huge* database in any case
 6272018-12-13T19:46:33  <jonasschnelli> But I agree, externally makes it a bit more complex
 6282018-12-13T19:46:41  <jamesob> in the next few weeks I'm going to be getting in touch with companies to get a more concrete idea of the usecases, so I'll report back with what I find
 6292018-12-13T19:46:59  <provoostenator> luke-jr: have what? People can compile their own client of course, but that's pretty high bar and can easily lead to rebase nightmares.
 6302018-12-13T19:47:01  <wumpus> bitcoin core is not meant as a chain analysis platform
 6312018-12-13T19:47:06  <luke-jr> wumpus: I did it in ~2 GB a year or so ago, IIRC
 6322018-12-13T19:47:07  <wumpus> you can do forensics using your own tools
 6332018-12-13T19:47:13  <jamesob> wumpus: very much agree
 6342018-12-13T19:47:20  <gmaxwell> jonasschnelli: right the problem for externally, is that implementing blockchain consistency is non-trivial and in my expirence most people who want to build an index don't want to deal with that.
 6352018-12-13T19:47:44  <gmaxwell> jamesob: more information on the actual usecases would be very helpful.
 6362018-12-13T19:47:49  <wumpus> it's non trivial but not rocket science either
 6372018-12-13T19:47:54  <jonasschnelli> Sadly people expect fast BIP44 recovery (incl. history). This seems to be the most prominent real-world usecase for an address index
 6382018-12-13T19:48:07  <wumpus> it shouldn't be that anything non-trivial needs to end up[ in bitcoin core
 6392018-12-13T19:48:12  <wumpus> we're not the world of non-trivial solutions
 6402018-12-13T19:48:26  <gmaxwell> jonasschnelli: most (I believe most, many at least) electrum servers don't do history, so I'm not entirely clear on the including history part of your comment.
 6412018-12-13T19:48:35  <provoostenator> Let's also not forget to update #14053 with concept ACK / NACK so people don't waste time on trying it if it's undesirable.
 6422018-12-13T19:48:38  <gmaxwell> wumpus: heh.
 6432018-12-13T19:48:39  <gribble> https://github.com/bitcoin/bitcoin/issues/14053 | Add address-based index (attempt 4?) by marcinja · Pull Request #14053 · bitcoin/bitcoin · GitHub
 6442018-12-13T19:48:43  <jonasschnelli> gmaxwell: IMO they do (index everything)
 6452018-12-13T19:49:04  <jonasschnelli> (not electrum personal server though)
 6462018-12-13T19:49:04  <wumpus> I mean this is another thing that's been *years*
 6472018-12-13T19:49:09  <gmaxwell> jonasschnelli: no, I know that for a fact-- many electrum servers don't index history.
 6482018-12-13T19:49:11  <wumpus> really, no one has been able to build a solution to this?
 6492018-12-13T19:49:14  <gmaxwell> (I just don't know if its most of them)
 6502018-12-13T19:49:18  <wumpus> not one of all those companies that want it?
 6512018-12-13T19:49:24  <jamesob> provoostenator: and its sibling #14035
 6522018-12-13T19:49:25  <luke-jr> wumpus: people capable of it don't want it :p
 6532018-12-13T19:49:26  <gribble> https://github.com/bitcoin/bitcoin/issues/14035 | Utxoscriptindex by mgrychow · Pull Request #14035 · bitcoin/bitcoin · GitHub
 6542018-12-13T19:49:37  <sipa> wumpus: blockstream.info uses electrs, which seems to work very well
 6552018-12-13T19:49:43  <gmaxwell> wumpus: I think the process of learning enough to do it well does a pretty good job of convincing you that you don't want it.
 6562018-12-13T19:49:51  <wumpus> gmaxwell: heh
 6572018-12-13T19:49:59  <provoostenator> jonasschnelli: BIP44 recovery can be handled once we have descriptor support for importmulti and slightly more sane behavior (or a replacement for) the keypool.
 6582018-12-13T19:50:02  <wumpus> sipa: that's a good suggestion then!
 6592018-12-13T19:50:31  <gmaxwell> provoostenator: history recovery requires a scan of the chain, or a phenominally expensive index.
 6602018-12-13T19:50:31  <jonasschnelli> provoostenator: you can't recover the history without an address index or a complete rescan (which is not possible for pruned peers)
 6612018-12-13T19:50:43  <wumpus> https://github.com/romanz/electrs
 6622018-12-13T19:50:51  <provoostenator> Ok, rescan for pruned nodes isn't there yet.
 6632018-12-13T19:50:53  <gmaxwell> jonasschnelli: note that an index is also not compatible with pruning. :P
 6642018-12-13T19:50:56  <wumpus> alrady had it bookmarked apparently :)
 6652018-12-13T19:50:59  <jonasschnelli> The only light here is the scantxoutset where you can recover within seconds but not the spent-history
 6662018-12-13T19:51:19  <provoostenator> Though you could use the neutrino filters for that and then refetch the relevant blocks.
 6672018-12-13T19:51:24  <jonasschnelli> gmaxwell: it could be,... if we just keep the pointers and load the data via p2p once requested
 6682018-12-13T19:51:32  <gmaxwell> jonasschnelli: I think we'd get a lot more bang for our buck working on making 'wallet without history before x' (pruned wallet) a first class supported thing.
 6692018-12-13T19:51:48  <jonasschnelli> gmaxwell: completely agree on this
 6702018-12-13T19:51:54  <gmaxwell> jonasschnelli: transfering 200GB of data to do the input is not really reasonable...
 6712018-12-13T19:52:24  <gmaxwell> (and then not saving it... this would be pretty harmful to the p2p network if people were actually patient enough to use it. :) )
 6722018-12-13T19:52:28  <jonasschnelli> gmaxwell: I mean there is a PR that enabled txindex with pruning... and fetches the tx over p2p once requested outside the kept block area
 6732018-12-13T19:52:39  <jonasschnelli> which is slow.... but for debugging proposes okay
 6742018-12-13T19:52:43  <luke-jr> (this makes me think again of the concept of prune-to-slow-storage where you can plug in a USB drive when/if you need old data..)
 6752018-12-13T19:52:50  <gmaxwell> jonasschnelli: oh, I didn't recall that PR.
 6762018-12-13T19:53:06  <wumpus> luke-jr: prune-to-tape *hides*
 6772018-12-13T19:53:11  <provoostenator> luke-jr: slow storage being the internet? :-)
 6782018-12-13T19:53:17  <jonasschnelli> #13014
 6792018-12-13T19:53:19  <gribble> https://github.com/bitcoin/bitcoin/issues/13014 | Allow txindex in prune mode by jonasschnelli · Pull Request #13014 · bitcoin/bitcoin · GitHub
 6802018-12-13T19:53:30  <luke-jr> provoostenator: could be local
 6812018-12-13T19:53:53  <gmaxwell> jonasschnelli: bip 157 filtering could be used similarly.
 6822018-12-13T19:54:15  <gmaxwell> jonasschnelli: e.g. saved the bip157 filters locally, and scan against them.
 6832018-12-13T19:54:15  <jonasschnelli> yes... less disk space, more time spent on filtering... but same idea
 6842018-12-13T19:54:37  <luke-jr> hmm
 6852018-12-13T19:54:55  <luke-jr> what data do they want returned from the index? maybe we don't need to retain/fetch the block
 6862018-12-13T19:54:58  *** dviola has joined #bitcoin-core-dev
 6872018-12-13T19:55:01  *** rh0nj has quit IRC
 6882018-12-13T19:55:08  <luke-jr> eg, if the client would be happy with a list of block numbers or txids..
 6892018-12-13T19:55:08  <jamesob> are any of the BIP157/158 PRs on the high prio list? if not, they should be
 6902018-12-13T19:55:11  <gmaxwell> wumpus: sadly, freeking tape costs about as much as HDDs today...  LTO7 tapes (6TB) cost about $70.
 6912018-12-13T19:55:28  <wumpus> gmaxwell: ouch
 6922018-12-13T19:55:38  <provoostenator> So the wallet recovery flow would be: add descriptors, download BIP-157 filters (if you didn't keep them), process filters, fetch a few blocks. User can immediately use their wallet to receive and send.
 6932018-12-13T19:55:48  <wumpus> gmaxwell: I guess they're more reliable though
 6942018-12-13T19:55:49  <luke-jr> jamesob: making light wallets stronger has a negative impact on Bitcoin :/
 6952018-12-13T19:56:08  *** rh0nj has joined #bitcoin-core-dev
 6962018-12-13T19:56:23  <gmaxwell> provoostenator: I don't really think thats a viable long term workflow either...  rather if you couldn't afford the space to keep them you probably don't want the time/bandwidth to download them.
 6972018-12-13T19:56:27  <sipa> luke-jr: BIP158 helps our local wallet too
 6982018-12-13T19:56:28  <provoostenator> luke-jr: not making them possible sends most people to web wallets, not to full nodes.
 6992018-12-13T19:56:35  <jamesob> luke-jr: better light wallets might help adoption
 7002018-12-13T19:56:39  <gmaxwell> luke-jr: I'm only interested in it for the local wallets.
 7012018-12-13T19:57:00  <luke-jr> jamesob: better to not have that adoption..
 7022018-12-13T19:57:05  <luke-jr> sipa / gmaxwell: yes, that's a point
 7032018-12-13T19:57:22  <provoostenator> gmaxwell: recovery should be a rare thing anyway, I assume it only happens after a disaster. In which case you'd probably download the whole chain again anyway.
 7042018-12-13T19:57:32  <sipa> provoostenator: it should be.
 7052018-12-13T19:57:34  <gmaxwell> jamesob: history has shown otherwise, bip158 doesn't make lite wallets fundimentally more usable than they are now.  They're still massively worse than server driven wallets like electrum or web wallets.
 7062018-12-13T19:58:05  <gmaxwell> provoostenator: right but thats also a reason that fetching them when you don't have them isn't that interesting, IMO.
 7072018-12-13T19:58:22  <jamesob> good point - but I guess if existing light wallets switched to bip157 it'd at least ease load on existing full nodes
 7082018-12-13T19:58:41  <gmaxwell> jamesob: what light wallets?
 7092018-12-13T19:58:55  <provoostenator> gmawell is that _because_ of bip158 or just because there aren't that many developers working on light (non web, non electrum) wallets? That could change over time.
 7102018-12-13T19:58:59  <wumpus> 1 minute to go
 7112018-12-13T19:59:04  <jamesob> anything using bloom-filter-based SPV
 7122018-12-13T19:59:12  <gmaxwell> jamesob: at least connections I see there is virtually no acutal usage of p2p lite wallets anymore.
 7132018-12-13T19:59:15  <wumpus> please wrap up the meeting
 7142018-12-13T19:59:44  <jamesob> I'll report on any compelling usecases I find for addr index, but I suspect sipa et al are right that that's usually just the Wrong way
 7152018-12-13T19:59:48  <gmaxwell> provoostenator: P2p lite wallets that scan the chain just end up with a very poor user expirence.
 7162018-12-13T20:00:06  <jamesob> in the meantime I recommend we give concept ACK/NACK to outstanding PRs which are related
 7172018-12-13T20:00:11  <gmaxwell> and that doesn't really have anything to do with bip158 vs bip37.
 7182018-12-13T20:00:25  *** timothy has quit IRC
 7192018-12-13T20:00:36  <gmaxwell> (in fact BIP158 is somewhat worse, but slightly less of a privacy disaster)
 7202018-12-13T20:00:37  <wumpus> #endmeeting
 7212018-12-13T20:00:37  <lightningbot> Meeting ended Thu Dec 13 20:00:37 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
 7222018-12-13T20:00:37  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.html
 7232018-12-13T20:00:37  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.txt
 7242018-12-13T20:00:37  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.log.html
 7252018-12-13T20:01:03  <provoostenator> gmaxwell: I was thinking mostly about Lightning wallets that rely on a lightweight wallet but don't have support for on chain (other than topping up channels). Those don't need to care about unconfirmed transactions.
 7262018-12-13T20:01:10  <wumpus> jamesob: I think poeople have become tired of arguing, or even explicitly NACKing those things
 7272018-12-13T20:01:11  <gmaxwell> (it's worse because it takes a client more time to scan the chain than with BIP37, as it has to get quite a bit more data)
 7282018-12-13T20:01:51  <gmaxwell> wumpus: +1
 7292018-12-13T20:02:18  <jamesob> wumpus: makes sense. I'll come up with a reply to both PRs that tries to explain why they're not a fit for core
 7302018-12-13T20:02:30  <wumpus> at least I have, I mean, if you've been in this since 2011 or so, and have to keep telling people something is abad idea while it becomes ever a worse idea (due to block chain growth)
 7312018-12-13T20:02:32  <jamesob> and maybe we can think about adding something to the developer guidelines about indexing
 7322018-12-13T20:02:41  <gmaxwell> UTXO-iyx indexes I generally concept ack (at least if they're ones that aren't specific about 'addresses' but work for spk's generally), history ones, I have NAKed.
 7332018-12-13T20:02:54  <wumpus> gmaxwell: yep
 7342018-12-13T20:03:30  <sipa> provoostenator: yes, i think BIP157 may be useful for a new class of clients that may become popular
 7352018-12-13T20:03:56  <jamesob> gmaxwell: so a scriptpubkey index on UTXOs is worthwhile? is that a generally useful index to have for use by core?
 7362018-12-13T20:03:58  <gmaxwell> jamesob: we have a problem that there aren't good instructions on watchign without an index (and we likely have bugs and limitations that make it unattractive that haven't been adequately reported)..  And for the analysis usecases what people want is an analysis platform, which we're not going to provide but don't really have much to recommend.
 7372018-12-13T20:04:29  <jamesob> gmaxwell: agree, I don't want to spend my time on the analysis usecases
 7382018-12-13T20:05:02  <sipa> jamesob: i dislike UTXO index equally, but it's certainly less of a burden
 7392018-12-13T20:05:05  <gmaxwell> jamesob: It's both useful (esp if we gain pruned wallet support-- but also scantxoutset speedup), but also it's not incompatiable with scalablity.
 7402018-12-13T20:05:39  <jamesob> good point
 7412018-12-13T20:05:40  <sipa> gmaxwell: assuming having a local UTXO set is forever
 7422018-12-13T20:05:52  <gmaxwell> jamesob: it's my personal belief that within 10 years most users won't ever bother fetching the far back history (or only doing so as a background test and not keeping the results).
 7432018-12-13T20:06:03  <gmaxwell> sipa: yes, well when otherwise becomes viable my thinking may change!
 7442018-12-13T20:06:09  <sipa> (which at least for the foreseeable future it will be)
 7452018-12-13T20:06:23  <jamesob> #utreexo :)
 7462018-12-13T20:06:44  <gmaxwell> jamesob: and moreover, with the exception of ultra high end commercial installs, few users would have the resources or patience to fetch 15 years of bitcoin history.
 7472018-12-13T20:07:19  <wumpus> doesn't need to be 'forever', it's useful *now*
 7482018-12-13T20:07:23  <gmaxwell> jamesob: so any of indexes on that history are is trouble for the ecosystem, as the indexes are many times less viable resource wise than the history itself.
 7492018-12-13T20:07:34  <jamesob> gmaxwell: so we'd expect "hobbyists" to run pruned nodes, or do you see some more sophisticated compaction of chain history?
 7502018-12-13T20:07:57  <sipa> jamesob: providing bulk access to sequential data is *cheap*
 7512018-12-13T20:08:14  <sipa> i expect nearly everyone to not bother with having the full chain
 7522018-12-13T20:08:14  <gmaxwell> jamesob: My expirence is that commerical parties are often less tolerant of resource usage and more willing to accept "query blockchain.info" than hobbists.
 7532018-12-13T20:08:17  <wumpus> UTXO db=sophisticaed compaction of chain history
 7542018-12-13T20:08:35  <gmaxwell> jamesob: so I think you probably have that part backwards. :)
 7552018-12-13T20:08:49  <jamesob> wumpus: right but you have to validate the construction of a utxo db...
 7562018-12-13T20:09:13  <gmaxwell> jamesob: I expect that in the future nodes will download a historical UTXO set, verifying it against an assumevalid in their code, and then sync forward from that.
 7572018-12-13T20:09:36  <wumpus> jamesob: oh sure if you really want to bypass validation
 7582018-12-13T20:09:41  <sipa> (or in the even further future, in a utreexo or magical accumulator world, not even have that UTXO set...)
 7592018-12-13T20:10:12  <gmaxwell> sipa: yes, but so far all those proposals are too slow to use or increase bandwidth 10x fold, which isn't a winning tradeoff.
 7602018-12-13T20:10:13  <jamesob> gmaxwell wumpus: okay so that leads back into the discussion of some kind of utxo set commitment
 7612018-12-13T20:10:15  <wumpus> jamesob: you can have my UTXO set if you want, no guarantees though :<
 7622018-12-13T20:10:24  <jamesob> wumpus: ha generous
 7632018-12-13T20:10:26  <gmaxwell> jamesob: no, it has nothing to do with a utxo set commitment in fact.
 7642018-12-13T20:10:27  <sipa> gmaxwell: utreexo is at 2x bandwidth or so
 7652018-12-13T20:11:06  <sipa> (according to simulation results i heard)
 7662018-12-13T20:11:12  <jamesob> gmaxwell: you're talking about "assumevalid" in the existing sense and not an assumevalid on a utxo hash, right?
 7672018-12-13T20:11:22  <provoostenator> What does "utreexo" stand for again?
 7682018-12-13T20:11:29  <sipa> provoostenator: utxo + tree
 7692018-12-13T20:11:36  <provoostenator> uTreeXO?
 7702018-12-13T20:11:38  <jamesob> provoostenator: https://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-utxo-accumulators-and-utreexo/
 7712018-12-13T20:11:43  <gmaxwell> jamesob: a utxo has, but "utxo commitment" has generally ment putting it in blocks which is irrelevant here.
 7722018-12-13T20:12:14  <sipa> unfortunately it needs blocks that commit to the utxo *proofs* (not the root) for DoS resistance
 7732018-12-13T20:13:37  <sipa> imo
 7742018-12-13T20:13:38  <jamesob> gmaxwell: so what you're saying is that 15 years from now, you think that we'll have a hardcoded hash of some historical utxo snapshot (a la assumevalid) and users will start their chain validation from there?
 7752018-12-13T20:13:51  <gmaxwell> jamesob: doing an assumevalid merely needs a hash, which we already have... though it would be better if nodes could record their hash for each block to make review easier.
 7762018-12-13T20:14:20  <gmaxwell> jamesob: Yes, short of breakthroughs in ZKPs I don't think anything else will be viable.
 7772018-12-13T20:15:38  <gmaxwell> We needed to constrain blocksize to a fraction of the current size to avoid time to initial sync constantly growing up even assuming computers/bandwidth/disk improving by 18% or whatever year over year. Since that isn't the case the validation time will continue to grow.
 7782018-12-13T20:16:01  <jamesob> gmaxwell: makes sense
 7792018-12-13T20:16:54  <gmaxwell> And it's already large enough that it pushes many companies to prefer hosted APIs... (even if they don't mind the initial sync time, the risk of having to take it again while the service is down is pretty unattractive).  Some end users are less tolerant of loading delays, some more.
 7802018-12-13T20:17:40  <gmaxwell> And in any case, the same security argument for assumevalid-scripts would apply: if someone can get away with making really obvious bad changes to the code you're running, all bets are off.
 7812018-12-13T20:18:20  <gmaxwell> Main gaps in implementation now are that it would be really good to make it easier to audit an assumevalid hash, which maybe implies adding an incrementally updatable hash.
 7822018-12-13T20:18:32  <gmaxwell> And then just storing and fetching snapshots for the p2p protocol.
 7832018-12-13T20:19:30  <jamesob> i.e. sipa's rolling MUHASH
 7842018-12-13T20:20:05  <sipa> i'm leaning towards the ECMH approach now, as it's easier to cache for prevalidated txn etc
 7852018-12-13T20:20:34  <sipa> it's more CPU, but in generally CPU time that can easily move to separate threads etc
 7862018-12-13T20:21:00  <jamesob> (for anyone curious about rolling UTXO set hashes: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014337.html)
 7872018-12-13T20:24:43  <luke-jr> will it upset anyone's reviewing if I rebase #11082 now?
 7882018-12-13T20:24:45  <gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
 7892018-12-13T20:25:00  <gmaxwell> I don't disagree, however for the 'sync from a snapshot and everyone can validate the hash', we could do something more boring and just compute a conventional hash of the utxo set in the background every 25000 blocks.
 7902018-12-13T20:25:38  <gmaxwell> There is no particular need to know the hash at every single block... sync logistics mean that it wouldn't be realistic to support syncing from an arbritary block anyways.
 7912018-12-13T20:27:55  <gmaxwell> The other issue with using muash/ecmh with a sync is that it doesn't really lend itself to incremental validation of fetched chunks. So it would probably need to be used in addition to another hash in any case.
 7922018-12-13T20:28:39  <gmaxwell> incremental validation of chunks-- I mean you're gonna download the 2gb utxo set from 8 peers... okay, great, you get the whole thing and the echm doesn't match. What now?  You don't even know what peer gave you the bad data.
 7932018-12-13T20:28:49  <luke-jr> btw, it's not too late to reduce block sizes; if we get an organized dev-supported proposal following Lightning going production-ready, I think we can get enough support
 7942018-12-13T20:29:14  <gmaxwell> luke-jr: it is too late because it already takes much larger to sync the chain than many people will tolerate.
 7952018-12-13T20:29:30  <luke-jr> gmaxwell: we can catch up with time
 7962018-12-13T20:29:44  <gmaxwell> Also, the harm from the significant reduction required would be considerable.
 7972018-12-13T20:29:54  <luke-jr> not post-Lightning
 7982018-12-13T20:30:14  <gmaxwell> luke-jr: yes post lightning too, since channel creations/closers still use capacity.
 7992018-12-13T20:30:33  <luke-jr> less than we need
 8002018-12-13T20:30:39  <gmaxwell> So you probably want a tree hash so that you can have each peer prove that the chunk they're sending you is consistent with the download you're expecting.
 8012018-12-13T20:31:57  <sipa> luke-jr: i don't expect there to be a "lightning going production-ready"; there may or may not be a phase where it grows sufficiently mature, and there may or may not be a phase where large scale adoption happens
 8022018-12-13T20:32:11  <sipa> don't put all eggs in one basket
 8032018-12-13T20:32:31  <gmaxwell> luke-jr: I don't think it's at all obvious that the result is less than we need.
 8042018-12-13T20:32:33  <luke-jr> one basket is better than none
 8052018-12-13T20:32:57  <gmaxwell> Current blocksizes are perfectly survivable if we get out of this fetch the whole history on every install mode.
 8062018-12-13T20:33:23  *** Murch has quit IRC
 8072018-12-13T20:33:29  <luke-jr> I don't think changing the security model is an appropriate avenue for that.
 8082018-12-13T20:34:01  <gmaxwell> I don't agree that its a meaninful change to the security model, but even if it were, it would be a good tradeoff.
 8092018-12-13T20:34:28  <gmaxwell> having users and businesses not willing to run nodes because bringing one up takes days of syncing is a change to the security model.
 8102018-12-13T20:34:31  <gmaxwell> unambigiously.
 8112018-12-13T20:34:46  *** Murch has joined #bitcoin-core-dev
 8122018-12-13T20:35:32  <gmaxwell> and we're already there, and no reduction will eliminate that. (you're right that assuming (cough) continued computer speedups it would eventually get better if blocks were significantly limited today, but even that would take a long time)
 8132018-12-13T20:36:16  *** phwalkr has joined #bitcoin-core-dev
 8142018-12-13T20:37:46  <gmaxwell> luke-jr: not to mention that the political viablity of a decrease is essentially 0, -- something you always knew, as that was also the reason that "the limit can be removed and restored if there are issues!" was a dumb idea on arrival.
 8152018-12-13T20:39:06  <luke-jr> it's not zero. It just requires more education, less block space need (ie, Lightning), and ideally developer support.
 8162018-12-13T20:39:54  *** shesek has quit IRC
 8172018-12-13T20:40:28  *** shesek has joined #bitcoin-core-dev
 8182018-12-13T20:40:42  <gwillen> given the massive political consequences of merely _not raising_ the size, I can hardly even imagine the idea of lowering it being taken seriously.
 8192018-12-13T20:43:18  <luke-jr> gwillen: the trolls are off having fun with their altcoin now, no need to appease them anymore (in fact, they will probably encourage it because they think it's a bad idea)
 8202018-12-13T20:44:08  <gmaxwell> The fundimental issue that many people can't wrap their heads around a limit having any purpose at all still remains. Also the issue of it being totally unclear how much is enough.
 8212018-12-13T20:44:24  <luke-jr> gmaxwell: that's where education comes in
 8222018-12-13T20:44:36  <gmaxwell> Education only goes so far.
 8232018-12-13T20:45:07  *** gabridome has joined #bitcoin-core-dev
 8242018-12-13T20:45:10  <gwillen> you've proposed reductions in size before, and I do not have the sense that they have ever gotten traction
 8252018-12-13T20:45:17  <gmaxwell> in any case, the system inherently resists changes, so the deck is stacked against you... this is a good thing usually.
 8262018-12-13T20:45:20  <gwillen> even among people opposed to increasing it
 8272018-12-13T20:45:21  <luke-jr> gwillen: not really
 8282018-12-13T20:45:45  <gmaxwell> luke-jr: I don't personally see a need or use to decrease it now.
 8292018-12-13T20:46:01  <luke-jr> gmaxwell: because you've given up, it sounds like
 8302018-12-13T20:46:17  <gmaxwell> Improvements to initial sync are required regardless! and I don't see them as regretable.
 8312018-12-13T20:46:50  <luke-jr> real improvements to initial sync only work in combination with a block size decrease
 8322018-12-13T20:46:51  <gmaxwell> and with them I'm not aware of an argument as to why the current size is particularly problematic,  we have _radically_ improved things like block propagation.
 8332018-12-13T20:47:01  <gmaxwell> (for example)
 8342018-12-13T20:47:56  <gmaxwell> and with the work on tx relay improvements we'll be able to cut relay overheads by maybe 40x.
 8352018-12-13T20:48:10  *** Guyver2 has joined #bitcoin-core-dev
 8362018-12-13T20:48:21  <luke-jr> that's unrelated to initial sync..
 8372018-12-13T20:49:31  <gmaxwell> exactly, I am pointing out that the current size is okay except for initial sync. And initial sync needs to be safely shortcutted regardless of the ongoing future size. And once it is, initial sync would no longer be a huge issue.
 8382018-12-13T20:50:53  <luke-jr> I don't agree. That's what I mean by just giving up.
 8392018-12-13T20:51:18  *** bitcoin-git has joined #bitcoin-core-dev
 8402018-12-13T20:51:19  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #14954: WIP: build: Require python 3.5 (master...Mf1811-buildPython3) https://github.com/bitcoin/bitcoin/pull/14954
 8412018-12-13T20:51:19  *** bitcoin-git has left #bitcoin-core-dev
 8422018-12-13T20:51:40  *** Murch has quit IRC
 8432018-12-13T20:51:42  <gmaxwell> I don't think thats giving up at all.
 8442018-12-13T20:52:21  <gmaxwell> unless you think that downloading bitcoin node software written by other people and not doing the work of writing it yourself is giving up. :P
 8452018-12-13T20:52:34  <luke-jr> code can be verified
 8462018-12-13T20:53:51  <gmaxwell> with an extreme amount of work, and an AV can be verified, with considerably less cost/work than code.
 8472018-12-13T20:53:55  *** rex4539 has joined #bitcoin-core-dev
 8482018-12-13T20:54:07  <luke-jr> not if we give up on a full sync
 8492018-12-13T20:55:18  <gmaxwell> I'm confused by that view.
 8502018-12-13T20:56:09  <gmaxwell> set -assumevalid=0 and you'll not use it, you'll take a lot longer to sync.  It's still radically cheaper than doing pratically any code audit.  (especially since in the common case of someone bringing up a node when they already had one running, their review need only be to compare between their nodes)
 8512018-12-13T20:56:17  *** rockhouse has quit IRC
 8522018-12-13T20:56:18  *** victorSN has quit IRC
 8532018-12-13T20:56:37  <luke-jr> gmaxwell: but we're talking about leaving in place an infinitely growing initial sync time
 8542018-12-13T20:56:53  <luke-jr> right _now_ it's easier, but it won't be for long
 8552018-12-13T20:57:07  *** phwalkr_ has joined #bitcoin-core-dev
 8562018-12-13T20:58:01  <gmaxwell> The cost of an outsider finding a subtly introduced 'bug' in the code is phenomial, maybe the cost of a review that could reliably catch something like that is on the order of $100,000  ... and that cost goes up over time as wages for experts seem to go up and the codebase increases in size.
 8572018-12-13T20:58:34  <gmaxwell> It would take a LONG time of growth before the cost of a non-AV sync matches the cost of review that would reliably find a subtle bugdoor.
 8582018-12-13T20:58:42  *** Murch has joined #bitcoin-core-dev
 8592018-12-13T20:59:11  <luke-jr> therefore everyone should just trust developers /s
 8602018-12-13T20:59:29  <gmaxwell> I don't see how that follows.
 8612018-12-13T21:00:38  *** phwalkr has quit IRC
 8622018-12-13T21:00:45  <gmaxwell> Verification is costly, there isn't any way around that.
 8632018-12-13T21:01:27  <luke-jr> it's something people can still do today, for the blockchain
 8642018-12-13T21:05:37  *** phwalkr has joined #bitcoin-core-dev
 8652018-12-13T21:08:07  *** phwalkr_ has quit IRC
 8662018-12-13T21:09:42  <gmaxwell> luke-jr: and they will still be able to do it in 10 years at current growth rates. it will just continue to be burdensome enough that most people will choose not to do so, exactly as is the case for the rest of the code today.
 8672018-12-13T21:10:43  <gmaxwell> Worrying that it'll take 40 hours of computer time instead of 8  seems penny-wise-and-pound foolish, when it'll take 40 hours of _human_ time to do even a light review of the code.
 8682018-12-13T21:11:03  <gmaxwell> in terms of aggregate validation, political efforts would probably be better spent making implementations more verifyable.
 8692018-12-13T21:20:56  <treyzania> inb4 bitcoin node in coq
 8702018-12-13T21:21:25  <aj> fwiw, with the "do nothing" approach, assuming IBD speeds increase by 18% per year, and chain length grows by 200GB/year, IBD maxes out at around 2.6x current length in 5 years, and finally gets back to current time after ~18 years
 8712018-12-13T21:23:55  *** jarthur has quit IRC
 8722018-12-13T21:26:20  <gmaxwell> aj: thanks.
 8732018-12-13T21:29:38  *** phwalkr has quit IRC
 8742018-12-13T21:30:32  *** phwalkr has joined #bitcoin-core-dev
 8752018-12-13T21:31:37  <aj> oh, except 100GB/year is more reasonable? 2MB * 1000 blks/week * 52 weeks/year?
 8762018-12-13T21:32:27  <aj> that's max of 1.5x in 4 years, back to same time as now in 12 years
 8772018-12-13T21:33:44  <phantomcircuit> gmaxwell, the only reason i can think you'd want an addrindex is if you have a pile of private keys and cant remember any metadata about them
 8782018-12-13T21:34:27  <gmaxwell> phantomcircuit: even there, do you really want an addrindex or a utxo spk index? Also if it's actually a  pile, scantxout set will be faster.
 8792018-12-13T21:35:13  <phantomcircuit> gmaxwell, for whatever accounting reason you need the history not just the current utxo
 8802018-12-13T21:35:28  <aj> decreasing the weight limit to 2Mweight under those assumptions (so 50GB/year) keeps IBD time increase under 10% from now
 8812018-12-13T21:36:34  <gmaxwell> phantomcircuit: indeed.  But for that case, you'd still probably be just as well off with bip157 filters on disk, and sequential scanning those.
 8822018-12-13T21:45:41  *** bitcoin-git has joined #bitcoin-core-dev
 8832018-12-13T21:45:42  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/9a4334443085...a5aea9654f36
 8842018-12-13T21:45:42  <bitcoin-git> bitcoin/master faead93 MarcoFalke: test: Make g_insecure_rand_ctx thread_local
 8852018-12-13T21:45:43  <bitcoin-git> bitcoin/master a5aea96 MarcoFalke: Merge #14953: test: Make g_insecure_rand_ctx thread_local...
 8862018-12-13T21:45:43  *** bitcoin-git has left #bitcoin-core-dev
 8872018-12-13T21:45:48  <phantomcircuit> gmaxwell, that wasn't an option when people were clamoring for an addrindex though
 8882018-12-13T21:46:24  *** bitcoin-git has joined #bitcoin-core-dev
 8892018-12-13T21:46:24  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #14953: test: Make g_insecure_rand_ctx thread_local (master...Mf1812-testThreadLocal) https://github.com/bitcoin/bitcoin/pull/14953
 8902018-12-13T21:46:24  *** bitcoin-git has left #bitcoin-core-dev
 8912018-12-13T21:46:37  <gmaxwell> well it's not an option yet. yea, without more inforation on what people are asking for I'm not sure if people would be satisified by disk-filter powered rescan.
 8922018-12-13T21:47:22  <phantomcircuit> gmaxwell, im 99% certain what people want is an easy way to import historic payments to an address for accounting reasons
 8932018-12-13T21:47:42  <phantomcircuit> and have that work with whatever custom wallet stuff they've written
 8942018-12-13T21:47:48  <phantomcircuit> (ie no creation timestamp)
 8952018-12-13T21:47:53  <gmaxwell> that absoluely is a reason, I doubt its the only one.
 8962018-12-13T21:48:16  <phantomcircuit> im sure the other 1% is blockexplorers calling rpc directly
 8972018-12-13T21:48:38  <gmaxwell> e.g. like I've talked to an exchange who was asking for an index because they thought it would make the daemon into a block explorer and they could use it to answer 'sent from' for blacklisting purposes.
 8982018-12-13T21:48:54  <gmaxwell> But none of the addrindex proposals would have made that possible, regardless...
 8992018-12-13T21:49:01  <gmaxwell> because they index outputs, not inputs.
 9002018-12-13T21:52:06  *** fabianfabian has joined #bitcoin-core-dev
 9012018-12-13T21:52:10  *** AaronvanW has joined #bitcoin-core-dev
 9022018-12-13T21:53:56  <adiabat> sorry just got to this but a couple maybe relevant points:
 9032018-12-13T21:54:23  <kanzure> late hi
 9042018-12-13T21:54:35  <adiabat> I'm hoping the utreexo thing will help clients sync faster, even without any checkpoints.  If you're running on an HDD, disk I/O is the main delay for IBD
 9052018-12-13T21:55:00  *** miknotauro has quit IRC
 9062018-12-13T21:55:12  <adiabat> and with utreexo / other utxo accumulator, you can avoid touching the disk and stay in ram, so you are just cpu limited by signatures
 9072018-12-13T21:55:36  <adiabat> (which fast computers with SSDs are now anyway, but lots of people run nodes with spinning drives)
 9082018-12-13T21:56:14  *** chenpo has quit IRC
 9092018-12-13T21:56:26  <adiabat> also separate point is that the need for a bridge node; these nodes can attach proofs to inputs for the nodes which hage pruned the utxos set away and only hold the accumulator
 9102018-12-13T21:56:29  <treyzania> I wonder how long it would take to sync off of a bunch of RAIDed floppies
 9112018-12-13T21:57:09  <adiabat> the bridge node needs a utxo index; not of addresses though.  But I'm not sure if it makes sense to build that into core or keep it as a separate server / node / software
 9122018-12-13T21:57:45  <adiabat> so that might be a place to put address index software if people don't want it in bitcoind
 9132018-12-13T21:58:06  <luke-jr> oh wow, I totally forgot about adiabat's thing after Tokyo >_<
 9142018-12-13T21:58:47  <adiabat> yeah I need to put stuff up publicly.  I've been working on it but it takes longer than I would expect, like evertyhing else :)
 9152018-12-13T21:58:54  <instagibbs> (btw he quoted 15% overhead during IBD in other channel)
 9162018-12-13T21:59:40  <adiabat> instagibbs: yeah, overhead depends on how much ram you dedicate to caching the proofs
 9172018-12-13T22:00:12  <adiabat> with no caching I have 250GB of proofs, so not great
 9182018-12-13T22:00:49  <adiabat> but it goes down quickly with increased caching.  Also I have a maybe controversial way to use shorter hashes.
 9192018-12-13T22:01:21  <luke-jr> did kanzure transcribe the discussions in Tokyo? I feel like I should review them now :/
 9202018-12-13T22:01:24  <adiabat> in that you can argue that you're only needing 2nd preimage resistance, not collision resistance.  But I'm not 100% set on that, even though I'm pretty convinced
 9212018-12-13T22:02:20  <adiabat> yeah kanzure's transcript is here: http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-utxo-accumulators-and-utreexo/
 9222018-12-13T22:03:53  <luke-jr> thanks
 9232018-12-13T22:05:03  <kanzure> :)
 9242018-12-13T22:05:08  <kanzure> you beat me to it
 9252018-12-13T22:05:16  <sipa> a rare event.
 9262018-12-13T22:08:42  *** Tralfaz has quit IRC
 9272018-12-13T22:11:12  <gmaxwell> adiabat: you can sync entirely in ram today, it just takes 8GB of cache... so you can measure the performance impact of that vs defaults.
 9282018-12-13T22:12:00  <gmaxwell> adiabat: it's non-trivial, but I am not expecting that it would be better for sync time to double bandwidth usage but avoid the in memory state.
 9292018-12-13T22:12:21  <sipa> gmaxwell: with a couple hundred MB of cache it seems the overhead is just 15%
 9302018-12-13T22:13:01  <sipa> if that's true, it starts to sound appealing for far more deployments
 9312018-12-13T22:13:07  <gmaxwell> well currently.
 9322018-12-13T22:14:42  <gmaxwell> I'm confused about that, it seems improbable to me. 15% would mean only on the order of _one_ extra hash per input.
 9332018-12-13T22:15:10  * sipa looks at adiabat 
 9342018-12-13T22:15:28  <gmaxwell> 2x I believed. :)
 9352018-12-13T22:15:51  <treyzania> it works *really* great for embedded devices which might have a few hundred MiB memory and flash storage
 9362018-12-13T22:15:53  *** chenpo has joined #bitcoin-core-dev
 9372018-12-13T22:16:06  <treyzania> or like rpis
 9382018-12-13T22:16:08  *** phwalkr has quit IRC
 9392018-12-13T22:16:49  <adiabat> gmaxwell: it's because so many utxos are short-lived, and this is with relying on "hints" from the archive node you IBD from
 9402018-12-13T22:17:17  <adiabat> the archive node knows what's happening next and can give TTL values for all the utxos created in a block
 9412018-12-13T22:17:19  <gmaxwell> Oh I see the few hundred mb cache is basically a cache of recent outputs, plus the top of a tree for non-recent outputs (or similar)?
 9422018-12-13T22:17:31  <adiabat> better than recent -- upcoming
 9432018-12-13T22:17:56  <sipa> adiabat: but that doesn't apply for chain tip updates, only historical sync
 9442018-12-13T22:17:57  <adiabat> that part is trusted, so they could lie and say long-lived utxos will be consumed soon
 9452018-12-13T22:18:02  <adiabat> right, this is only for IBD
 9462018-12-13T22:18:18  <sipa> i think the affect on bandwidth to keep in sync is more important
 9472018-12-13T22:18:23  <adiabat> these optimizations don't help once you're synced up; I've mostly been focusing on IBD so far
 9482018-12-13T22:19:06  <adiabat> there are other ideas for in-sync, but I haven't worked on that much; with IBD I have a nice data set to work off of to get performance numbers
 9492018-12-13T22:20:36  <adiabat> archive nodes indicating which utxos will be consumed soon could potentially be useful for IBD with the current levelDB as well I guess
 9502018-12-13T22:21:11  <adiabat> worst case if the node you're downloading from lies to you, your cache is full of the wrong things and you have to go to disk more and hang up on the lying node
 9512018-12-13T22:22:37  <gmaxwell> I don't think this is currently an interesting route of optimizations, from an engineering perspective.   Right now the existing software doesn't implement a simple fifo for management of the cache, in part due to the complexity of doing so.  Fifoing inputs would probably have 98% of the locality gain (a question I guess your research could answer), but still less complexity than cache
 9522018-12-13T22:22:38  <gmaxwell> management hints.
 9532018-12-13T22:23:28  <adiabat> yeah, fair, it makes more sense with the accumulator proofs in that you prevent having to download a lot of data
 9542018-12-13T22:23:46  <gmaxwell> Right sounds more useful there.
 9552018-12-13T22:25:34  <gmaxwell> But even there in IBD you're essentially trading off (say) 200gb * .15 = 30GB more data transfered, in order to avoid having to store 2GB durign sync.  I think under most relaistic models of the relative costs of storage and bandwidth, this tradeoff isn't that amazing.  Having the option of making it would be nice, but I would expect relatively few systems to actually be better off with that
 9562018-12-13T22:25:35  <gmaxwell> choice.
 9572018-12-13T22:28:09  <adiabat> for powerful computers it doesn't really help, but for computers with HDDs / space constraints it can help
 9582018-12-13T22:28:28  <sipa> or with I/O-starved VPSes
 9592018-12-13T22:28:43  <sipa> and perhaps it's also just faster
 9602018-12-13T22:28:46  <adiabat> but yeah if you have a fast computer and slow internet, it makes things worse
 9612018-12-13T22:28:48  <sipa> (to be seen)
 9622018-12-13T22:29:31  <sipa> and it also makes assumevalid-utxo models trivial
 9632018-12-13T22:29:40  <sipa> (no snapshot to transfer...)
 9642018-12-13T22:29:56  <adiabat> well a bech32-encodable snapshot
 9652018-12-13T22:30:09  <adiabat> hm actually no, it's a few hundred bytes so not quite... :)
 9662018-12-13T22:30:09  *** shesek has quit IRC
 9672018-12-13T22:30:54  *** shesek has joined #bitcoin-core-dev
 9682018-12-13T22:32:34  <gmaxwell> if you're assuming that the node knows the top of the tree, in order to avoid an absurd bandwidth blowup, then you'll have to transfer that.
 9692018-12-13T22:33:27  <sipa> that just gets relayed along with the first transactions that need it
 9702018-12-13T22:33:30  <adiabat> you could tranfer the top along with the root, but as soon as you see a few proofs in the mempool, you'll get the top
 9712018-12-13T22:33:49  <adiabat> right, what sipa said
 9722018-12-13T22:33:56  <sipa> it's of course still bandwidth; just saying it doesn't need a special store-and-transfer-a-snapshot protocol
 9732018-12-13T22:34:01  <gmaxwell> Okay, but thats just hiding the cost.
 9742018-12-13T22:34:38  <sipa> by trivial i meant complexity, not bandwidth - but even there you're probably right that it's complexity we're paying anyway in a more complex block/tx sync protocl
 9752018-12-13T22:34:53  <gmaxwell> sipa: it does need a snapshot, because you cannot run from the tip without wreaking security, you'll need the state as of the point you know is valid.
 9762018-12-13T22:35:20  <adiabat> yeah, network bandwidth is the big cost to this, CPU cost is low
 9772018-12-13T22:35:23  <gmaxwell> (I almost agreed with you for a moment there though! :P )
 9782018-12-13T22:35:44  <sipa> gmaxwell: ok, sure - but it's a few hundred bytes
 9792018-12-13T22:38:20  *** Guyver2 has quit IRC
 9802018-12-13T22:38:22  *** michaelsdunn1 has quit IRC
 9812018-12-13T22:38:47  <gmaxwell> really? there are 68m utxo or so, 1000 bytes would allow for 32 hashes.   for 68m hashes we need a tree of depth 29.  or 24 if 5 levels are saved from the top, so that results in 768 bytes per input of membership proof.
 9822018-12-13T22:39:16  <gmaxwell> so I think not a few hundreds bytes... certantly a lot smaller than the whole utxo set though... assuming you don't want to be able to offer proofs yourself.
 9832018-12-13T22:40:29  *** sipa has quit IRC
 9842018-12-13T22:41:44  *** sipa has joined #bitcoin-core-dev
 9852018-12-13T22:41:59  <sipa> what did i miss?
 9862018-12-13T22:42:05  <luke-jr> pizza
 9872018-12-13T22:42:10  <gmaxwell> dunno if you missed anything.
 9882018-12-13T22:42:14  <sipa> that's okay
 9892018-12-13T22:42:17  <gmaxwell> did you see
 9902018-12-13T22:42:18  <gmaxwell> 14:38:47 < gmaxwell> really? there are 68m utxo or so, 1000 bytes would allow for 32 hashes.   for 68m hashes we need a tree of depth 29.  or 24 if 5
 9912018-12-13T22:42:18  <gmaxwell>                      levels are saved from the top, so that results in 768 bytes per input of membership proof.
 9922018-12-13T22:42:18  <gmaxwell> 14:39:16 < gmaxwell> so I think not a few hundreds bytes... certantly a lot smaller than the whole utxo set though... assuming you don't want to be
 9932018-12-13T22:42:18  <gmaxwell>                      able to offer proofs yourself.
 9942018-12-13T22:42:21  <gmaxwell> ?
 9952018-12-13T22:43:11  <adiabat> I'll clean up the code a bit, but the code I have is not a single tree; it's a forest of perfect binary trees
 9962018-12-13T22:43:21  <sipa> gmaxwell: the "state" of the utxo set is just the roots of each of the trees in the forest
 9972018-12-13T22:43:22  <adiabat> so the worst case proof yeah is like 28 high
 9982018-12-13T22:43:37  <sipa> it's distinct from the cache of elements in the trees (which just gets transferred along with the blocks)
 9992018-12-13T22:43:55  <adiabat> but you can try to put most of the old utxos on the left in the big tree, and most proofs that you need to transfer are in the smaller trees with high turnover
10002018-12-13T22:44:52  <gmaxwell> sipa ... adding 28 hashes per input is a non-starter for general usage. There are some applications, vps in a datacenter where internal bandwidth is free, where it might be attractive.  but for most systems _bandwidth_ is the most limited resource.
10012018-12-13T22:45:18  <sipa> gmaxwell: of course
10022018-12-13T22:45:37  <adiabat> gmaxwell: 28 hashes per input is too much, agreed.  There's ways to bring that number down significantly
10032018-12-13T22:45:43  <sipa> gmaxwell: i think you're missing some of the ideas
10042018-12-13T22:46:08  <adiabat> (which I haven't written up, so not you're fault :)
10052018-12-13T22:46:37  <gmaxwell> adiabat: it can be brought down by caching more of the top...
10062018-12-13T22:46:37  <sipa> but in general i think it's a semantics discussion about what you call state and what you call cache
10072018-12-13T22:46:41  <adiabat> but a big part is exploiting the power-law dureation of utxos
10082018-12-13T22:46:44  <sipa> gmaxwell: adiabat is well aware
10092018-12-13T22:47:17  <gmaxwell> adiabat: That doesn't come about as a result of any physical law or even economic incentive in the current system. :(
10102018-12-13T22:47:48  <adiabat> true, a lot of is is a heuristic, so who knows.  if you have uniform random access across the utxo set, it gets worse.
10112018-12-13T22:48:55  <adiabat> fitting it to the current 200GB data set we have though, it's quite consistent that many utxos persist for only a few blocks
10122018-12-13T22:49:26  <gmaxwell> sipa: please we cannot continue to fail to do something about sync times in favor of conjecture about future ideas with uncertian performance.  Like here "It's 2x" but only under assumptions about spending patterns, that is _really_ misleading and I think it undermines productive discussion.
10132018-12-13T22:49:27  <adiabat> (many get created & destroyed in the same block which means they never even touch the accumulator)
10142018-12-13T22:50:08  *** wxss has quit IRC
10152018-12-13T22:50:34  <sipa> gmaxwell: for the sake of discussion, please just treat the caching we're talking about as what you're calling keeping the top levels of the tree as state
10162018-12-13T22:50:40  <adiabat> gmaxwell: I certainly don't mean to over-sell this; it's not a silver bullet, but I think it can help in some cases
10172018-12-13T22:50:59  <gmaxwell> adiabat: Sure.
10182018-12-13T22:51:38  <sipa> gmaxwell: fwiw, our current performance for sync with dbcache less than the entire UTXO set is also dependent on spending patterns
10192018-12-13T22:52:18  <gmaxwell> sipa: sure. (though less than it would be if the dbcache's managment weren't kinda stupid wrt locality)
10202018-12-13T22:53:04  <gmaxwell> But "oh look spending patterns changed and with no impact in fees, bandwidth needed by bitcoin nodes just went up 5x" isn't the same as saying dbcache works better with more locality.
10212018-12-13T22:53:27  <sipa> yeah, agree
10222018-12-13T22:54:07  <sipa> we need a separate average case and worst case analysis w.r.t. bandwidth impact
10232018-12-13T22:55:13  <gmaxwell> I am frustrated because these kind of far off researchy things seem to keep getting pulled out as a reason to not do something more near term about the initial synctime problem.
10242018-12-13T22:56:16  <sipa> i don't think so
10252018-12-13T22:56:32  <gmaxwell> Well, that is how I saw the discussion today.
10262018-12-13T22:57:06  <sipa> as far as i'm concerned, the potential prospect of utreexo or similar techniques are only a reason to not support a utxo-address index for me
10272018-12-13T22:57:58  <gmaxwell> That makes sense. I wasn't connecting it to that discussion.
10282018-12-13T22:58:47  *** Tralfaz has joined #bitcoin-core-dev
10292018-12-13T22:59:12  <gmaxwell> We can debate on the necessity of users getting historical data except for research purposes, no such debate exists for utxo though... so some scanning mechenism must be provided even in a utxotree world. ... Ideally one that preserves user privacy. But there can be ways of doing that.
10302018-12-13T22:59:13  <adiabat> fwiw I think it's towards the practical / implementable side of research-land.  I certainly don't want to say "this solves everything, don't worry about sync times, I got this" though.
10312018-12-13T23:00:15  *** Tralfaz has quit IRC
10322018-12-13T23:01:10  <gmaxwell> adiabat: Towards perhaps, but I think you overestimate that.  The gap between what we can imagine and build for testing and what we can safely put into production is pretty big.   Right now Bitcoind's dbcache grows up to some limit then is flushed completely, rather than acting as a fifo queue that exploits locality.  "Make it fifo" sounds trivial, but actually getting it done, and Right, and
10332018-12-13T23:01:10  <gmaxwell> being confident that its right is not.
10342018-12-13T23:01:23  <sipa> adiabat: well, we'll see - i think you may be underestimating the amount of work needed to deploy archive nodes, and the practically implementing a dos-resistant stateful sync protocol for parts of the utxo tree
10352018-12-13T23:02:03  <sipa> adiabat: even after you've characterized all the memory and cpu requirements of a close to fully written up spec
10362018-12-13T23:02:12  <adiabat> agreed, it's always harder than it seems.  Also, I don't mean to put it into bitcoind, I'm going to start with more of a server / client model
10372018-12-13T23:02:54  <adiabat> I guess in comparison to the class group accumulator it's more implementable though
10382018-12-13T23:03:00  <sipa> wahaha
10392018-12-13T23:03:14  *** Tralfaz has joined #bitcoin-core-dev
10402018-12-13T23:03:29  <sipa> yes
10412018-12-13T23:03:29  <gmaxwell> sipa: in any case, I was connecting this discussion back more to the assume valid discusion instead of the indexing discussion.  So to me it sounded like an argument to do nothing because utxotree will solve everything... and I'm dubious that the bandwidth overheads will make it a _general_ solution (obviously a big win in some cases), and I'm dubious as to how fast it can be made available.
10422018-12-13T23:03:50  <gmaxwell> adiabat: yea, funny I dunno about that! so like a class group accumulator is an isolated number theory programming project.
10432018-12-13T23:04:07  <adiabat> oh yeah definitely don't count on anything I'm doing time-wise, takes forever :)
10442018-12-13T23:04:15  <gmaxwell> so just from a software eng I think CGA might be more implementable, but it's just too slow to make useful regardless.
10452018-12-13T23:04:21  <sipa> gmaxwell: my current estimate for updating an archival node with a new block is about a year of CPU time
10462018-12-13T23:04:30  <sipa> gmaxwell: with class group based accumulators
10472018-12-13T23:04:40  <gmaxwell> sipa: lol yes, thats my 'too slow to make useful'
10482018-12-13T23:04:53  <sipa> oh of course
10492018-12-13T23:05:02  <gmaxwell> but like, _ignoring that_ (lol), in terms of actually switching bitcoin to use it, I expect it would be easier than utxotree.
10502018-12-13T23:05:26  <sipa> also ignoring the novel cryptographic assumption? :)
10512018-12-13T23:05:34  <adiabat> huh, yeah the proofs are tiny, everything's constant sized, etc.  The utreexo accumulator stuff does have a whole bunch of weird logic.
10522018-12-13T23:05:37  <gmaxwell> Even including the cost of having to write a "fast" classgroup acc library (assuming fast doesn't mean anything more than well optimized)
10532018-12-13T23:06:43  <sipa> but sure, everything constant size makes a lot of logistical problems go away
10542018-12-13T23:07:07  <gmaxwell> including need for protocols that make roundtrips... the bane of everyone's existance.
10552018-12-13T23:08:28  *** Giszmo has quit IRC
10562018-12-13T23:11:50  <gmaxwell> sipa: in any case, going back to the indexing question... for history indexing part of my concern there is that for some use cases there is a "just don't do that" or "look in the utxo instead" option.  Like "We need a history index to look up from addresses!"  :P   for the utxo set that is much less the case.
10572018-12-13T23:11:51  <gmaxwell> so if you imagine that in the future everything is using utxotree, we'll still need to provide some way to look up scriptpubkeys... maybe it's PIR queries to nodes what have an N of M shared copy of the whole database or whatever... but something will have to be provided.
10582018-12-13T23:11:54  <gmaxwell> as rescan is already pretty unrealistic and contantly getting worse.
10592018-12-13T23:12:29  <sipa> gmaxwell: i'm not sure what use cases really need access to the full UTXO set
10602018-12-13T23:12:40  *** promag has joined #bitcoin-core-dev
10612018-12-13T23:12:41  <sipa> of course there is a convenience in having it
10622018-12-13T23:12:51  <sipa> but there is convenience in having an address index for the whole chain too
10632018-12-13T23:13:05  <gmaxwell> sipa: I need to take my wallet offline for a long time or recover a backup and want to spend my funds.
10642018-12-13T23:13:36  <gmaxwell> To actually _use_ bitcoin I need that data... vs history is needed for analysis/auditing/etc purposes.
10652018-12-13T23:14:06  *** Chris_Stewart_5 has quit IRC
10662018-12-13T23:14:11  <gmaxwell> (and for those analysis you probably don't just want an address index in history you want all sorts of other indexes)
10672018-12-13T23:15:12  <sipa> gmaxwell: assuming the UTXO set was equally expensive to look things up as in the blockchain (it's obviously not, but just as a hypothetical), would you still say that?
10682018-12-13T23:15:37  <gmaxwell> yes, because you can't spend your bitcoins without the utxo data!
10692018-12-13T23:16:07  <sipa> sure, but we need access to the chain anyway in recovery scenarios
10702018-12-13T23:16:09  <gmaxwell> (I mean if they were equal, I suppose you could just use the blockchain instead. but the utxo part is special because it's actually needed to make any use of all of your coins)
10712018-12-13T23:16:39  <gmaxwell> sipa: we don't, the fact that we recover tx history is an artifact of how the software was implemented.
10722018-12-13T23:16:53  <gmaxwell> Considering we can't recover tx medatadata, its not like we're actually recovering the real history.
10732018-12-13T23:17:06  <gmaxwell> The history can well be useful, for sure, but it is not needed to make use of your bitcoins.
10742018-12-13T23:17:19  <gmaxwell> (and other wallets, e.g. electrum, can run without history)
10752018-12-13T23:17:25  *** lopp has joined #bitcoin-core-dev
10762018-12-13T23:17:27  <sipa> gmaxwell: for a business knowing history may be far more important than the ability to spend coins
10772018-12-13T23:18:08  <sipa> my point is that what you're talking about is pretty much a recovery scenario that should be rare; and the only reason you consider is more reasonable than seeing whole history is because it's currently (and presumably later) much cheaper
10782018-12-13T23:18:08  <gmaxwell> sipa: I disagree because the thing we can recover from the blockchain is not history, it's partial history-- it lacks metadata which is actually the important information.
10792018-12-13T23:18:24  <gmaxwell> moreover, that data is there-- go use a block explorer, there is no need a bitcoin wallet or node need to provide it.
10802018-12-13T23:18:29  <sipa> also true; generally you also need information you'd have backed up separately
10812018-12-13T23:18:51  * gmaxwell bbl
10822018-12-13T23:22:32  *** Giszmo has joined #bitcoin-core-dev
10832018-12-13T23:23:09  <lopp> question regarding maintaining code integrity during the development process: is there any way that an adversarial maintainer could merge an altered PR commit? just trying to fully visualize the "chain of code custody" between the steps of peer review and actually being merged
10842018-12-13T23:23:58  *** chenpo has quit IRC
10852018-12-13T23:25:20  *** fabianfabian has quit IRC
10862018-12-13T23:25:45  *** hex17or has quit IRC
10872018-12-13T23:38:22  *** Giszmo has quit IRC
10882018-12-13T23:42:49  *** shesek has quit IRC
10892018-12-13T23:59:19  *** Giszmo has joined #bitcoin-core-dev