1 2017-05-04T00:04:29  *** tw2006 has joined #bitcoin-core-dev
  2 2017-05-04T00:09:04  *** tw2006 has quit IRC
  3 2017-05-04T00:20:32  *** btcdrak has joined #bitcoin-core-dev
  4 2017-05-04T00:22:35  *** Ylbam has quit IRC
  5 2017-05-04T00:28:50  *** tw2006 has joined #bitcoin-core-dev
  6 2017-05-04T00:50:28  *** anthonyjd has joined #bitcoin-core-dev
  7 2017-05-04T00:51:10  *** kadoban has quit IRC
  8 2017-05-04T00:51:39  *** kadoban has joined #bitcoin-core-dev
  9 2017-05-04T00:57:44  *** kadoban has quit IRC
 10 2017-05-04T00:58:46  *** kadoban has joined #bitcoin-core-dev
 11 2017-05-04T01:05:21  *** PaulCape_ has joined #bitcoin-core-dev
 12 2017-05-04T01:05:33  *** PaulCapestany has quit IRC
 13 2017-05-04T01:06:17  *** kadoban has quit IRC
 14 2017-05-04T01:06:25  <cfields> sipa: you're going to hate me for this...
 15 2017-05-04T01:07:31  <bitcoin-git> [bitcoin] theuni opened pull request #10335: back-compat: add fallback getentropy implementation (master...getentropy-back-compat) https://github.com/bitcoin/bitcoin/pull/10335
 16 2017-05-04T01:12:01  <sipa> cfields: we don't use getentropy on linux
 17 2017-05-04T01:12:22  <warren> cfields: btw, Ubuntu 12.04 Precise is hitting EOL, will gitian switch to a newer base soon?
 18 2017-05-04T01:12:46  <sipa> cfields: or we shouldn't, i guess
 19 2017-05-04T01:13:26  <sipa> if getrandom isn't available on linux, i think we should fallback to using /dev/urandom directly rather than through a fallback for a wrapper for a BSD function :)
 20 2017-05-04T01:16:00  <cfields> sipa: atm it's being used in linux, best i can tell
 21 2017-05-04T01:16:19  <cfields> (i get a runtime link error, so it must be)
 22 2017-05-04T01:17:21  <cfields> warren: gitian uses Trusty currently
 23 2017-05-04T01:17:36  <warren> oops
 24 2017-05-04T01:18:59  *** Alina-malina has quit IRC
 25 2017-05-04T01:19:05  <sipa> cfields: yes, we compile in code to use getrandom whenever configure detects it
 26 2017-05-04T01:19:57  <cfields> sipa: sorry, i was referring to getentropy
 27 2017-05-04T01:20:05  <sipa> eh, i mean getentropy
 28 2017-05-04T01:20:30  <sipa> cfields: the getentropy implementation in glibc on Linux just uses SYS_getrandom, just like we already do directly
 29 2017-05-04T01:21:19  <sipa> so a fallback that just returns 0 should be fine
 30 2017-05-04T01:21:23  <sipa> we check the return status
 31 2017-05-04T01:21:58  *** kadoban has joined #bitcoin-core-dev
 32 2017-05-04T01:22:21  <sipa> ah, no
 33 2017-05-04T01:22:22  <sipa> hmm
 34 2017-05-04T01:23:25  *** Alina-malina has joined #bitcoin-core-dev
 35 2017-05-04T01:27:22  <cfields> sipa: looks like getentropy needs to check for ENOSYS, and do GetDevURandom as a fallback. Then the weak getentropy just returns ENOSYS.
 36 2017-05-04T01:27:27  <cfields> I think that works?
 37 2017-05-04T01:28:06  <sipa> cfields: yup
 38 2017-05-04T01:28:25  <cfields> ok, will do.
 39 2017-05-04T01:28:25  <sipa> cfields: i think our GetOSEntropy call should always compile in the GetDevURandom fallback
 40 2017-05-04T01:28:40  <sipa> (so move it out of the #ifdef ... #elif .. #endif switch)
 41 2017-05-04T01:28:54  <cfields> yep, agreed
 42 2017-05-04T01:29:01  <sipa> then the getrandom/getentropy calls can fail without problems
 43 2017-05-04T01:29:26  <sipa> and agreed, a simple fallback returning ENOSYS is perfect then
 44 2017-05-04T01:57:12  *** tw2006 has quit IRC
 45 2017-05-04T01:58:21  *** tw2006 has joined #bitcoin-core-dev
 46 2017-05-04T02:00:12  *** dermoth has quit IRC
 47 2017-05-04T02:01:02  *** dermoth has joined #bitcoin-core-dev
 48 2017-05-04T02:02:05  *** juscamarena_ has joined #bitcoin-core-dev
 49 2017-05-04T02:06:07  *** harrymm1 has joined #bitcoin-core-dev
 50 2017-05-04T02:08:32  *** harrymm has quit IRC
 51 2017-05-04T02:30:10  *** tripleslash has quit IRC
 52 2017-05-04T02:35:40  *** molz_ has quit IRC
 53 2017-05-04T02:38:18  *** moli_ has joined #bitcoin-core-dev
 54 2017-05-04T03:02:59  *** QBcrusher__ has joined #bitcoin-core-dev
 55 2017-05-04T03:05:58  *** vicenteH` has joined #bitcoin-core-dev
 56 2017-05-04T03:08:19  *** xHire has joined #bitcoin-core-dev
 57 2017-05-04T03:09:44  *** Dyaheon- has joined #bitcoin-core-dev
 58 2017-05-04T03:10:06  *** kcud_dab has joined #bitcoin-core-dev
 59 2017-05-04T03:11:46  *** vicenteH has quit IRC
 60 2017-05-04T03:11:46  *** xhire_ has quit IRC
 61 2017-05-04T03:11:46  *** Dyaheon has quit IRC
 62 2017-05-04T03:11:47  *** fao3 has quit IRC
 63 2017-05-04T03:11:47  *** justanotheruser has quit IRC
 64 2017-05-04T03:11:47  *** Naphex has quit IRC
 65 2017-05-04T03:11:47  *** instagibbs has quit IRC
 66 2017-05-04T03:11:47  *** Madars has quit IRC
 67 2017-05-04T03:11:47  *** profall has quit IRC
 68 2017-05-04T03:11:47  *** bad_duck has quit IRC
 69 2017-05-04T03:11:47  *** QBcrusher_ has quit IRC
 70 2017-05-04T03:11:47  *** chjj has quit IRC
 71 2017-05-04T03:12:46  *** instagibbs has joined #bitcoin-core-dev
 72 2017-05-04T03:14:20  *** tw2006 has quit IRC
 73 2017-05-04T03:18:30  *** chjj has joined #bitcoin-core-dev
 74 2017-05-04T03:19:11  *** Madars has joined #bitcoin-core-dev
 75 2017-05-04T03:19:19  *** fao3 has joined #bitcoin-core-dev
 76 2017-05-04T03:22:11  *** justanotheruser has joined #bitcoin-core-dev
 77 2017-05-04T03:32:36  <cfields> <sipa> cfields: i think our GetOSEntropy call should always compile in the GetDevURandom fallback
 78 2017-05-04T03:32:42  <cfields> sipa: looking again, I'm not sure what you meant by that
 79 2017-05-04T03:33:02  <cfields> we always compile GetDevURandom, except for win32
 80 2017-05-04T03:34:38  <sipa> cfields: i mean the GetDevURandom call at the bottom of GetOSEntropy
 81 2017-05-04T03:35:01  <sipa> which is only compiled if getrandom and getentropy are not availablr
 82 2017-05-04T03:35:04  <sipa> that should change
 83 2017-05-04T03:35:51  <cfields> sipa: so you want early returns for any better function, and fallback if one isn't hit by the end?
 84 2017-05-04T03:36:08  <sipa> right
 85 2017-05-04T03:36:28  <cfields> makes sense, will do
 86 2017-05-04T03:37:18  <sipa> getrandom currently has a fallback already
 87 2017-05-04T03:37:21  <sipa> but not the rest
 88 2017-05-04T03:43:45  <Lightsword> so I should remove the fallback here as well? https://github.com/bitcoin/bitcoin/pull/10301/files#diff-35f8a407f8c21cda300a45f50b6e9c74R172
 89 2017-05-04T03:50:49  *** talmai has joined #bitcoin-core-dev
 90 2017-05-04T04:15:14  *** tw2006 has joined #bitcoin-core-dev
 91 2017-05-04T04:19:47  *** tw2006 has quit IRC
 92 2017-05-04T04:32:12  *** aantonop has joined #bitcoin-core-dev
 93 2017-05-04T04:43:03  *** d_t has joined #bitcoin-core-dev
 94 2017-05-04T05:06:22  *** fao3 has quit IRC
 95 2017-05-04T05:08:17  *** fao3 has joined #bitcoin-core-dev
 96 2017-05-04T05:11:10  *** Giszmo has quit IRC
 97 2017-05-04T05:13:05  *** profall has joined #bitcoin-core-dev
 98 2017-05-04T05:14:10  *** Dyaheon- has quit IRC
 99 2017-05-04T05:14:28  *** Dyaheon has joined #bitcoin-core-dev
100 2017-05-04T05:16:10  *** tw2006 has joined #bitcoin-core-dev
101 2017-05-04T05:20:40  *** tw2006 has quit IRC
102 2017-05-04T05:24:25  *** fao3 has quit IRC
103 2017-05-04T05:24:51  *** fao3 has joined #bitcoin-core-dev
104 2017-05-04T05:31:56  *** PaulCape_ has quit IRC
105 2017-05-04T05:35:57  *** PaulCapestany has joined #bitcoin-core-dev
106 2017-05-04T05:39:43  *** aantonop has quit IRC
107 2017-05-04T05:47:32  *** tripleslash has joined #bitcoin-core-dev
108 2017-05-04T05:51:28  *** mol has joined #bitcoin-core-dev
109 2017-05-04T05:53:48  *** moli_ has quit IRC
110 2017-05-04T05:59:51  *** talmai has quit IRC
111 2017-05-04T06:00:12  *** RubenSomsen has joined #bitcoin-core-dev
112 2017-05-04T06:00:12  *** dermoth has quit IRC
113 2017-05-04T06:00:48  *** dermoth has joined #bitcoin-core-dev
114 2017-05-04T06:07:41  *** Ylbam has joined #bitcoin-core-dev
115 2017-05-04T06:17:10  *** tw2006 has joined #bitcoin-core-dev
116 2017-05-04T06:21:56  *** tw2006 has quit IRC
117 2017-05-04T06:25:13  *** afk11 has quit IRC
118 2017-05-04T06:26:05  *** afk11 has joined #bitcoin-core-dev
119 2017-05-04T06:28:13  *** Alina-malina has quit IRC
120 2017-05-04T06:28:14  *** Alina-malina has joined #bitcoin-core-dev
121 2017-05-04T06:35:27  <jonasschnelli> Grml... if we enforce a min HD recover gap limit (currently 20 keypool keys), all tests are broken (mosts tests run with a keypool of size 1). I guess I have to add a -hdignoregaplimit and set it to true by default for the tests.
122 2017-05-04T06:35:50  *** harrymm1 has quit IRC
123 2017-05-04T06:37:22  *** harrymm has joined #bitcoin-core-dev
124 2017-05-04T06:40:20  *** harrymm has quit IRC
125 2017-05-04T06:48:12  *** BashCo has quit IRC
126 2017-05-04T06:54:23  *** JackH has quit IRC
127 2017-05-04T07:12:39  *** d_t has quit IRC
128 2017-05-04T07:12:49  *** BashCo has joined #bitcoin-core-dev
129 2017-05-04T07:18:02  *** tw2006 has joined #bitcoin-core-dev
130 2017-05-04T07:22:28  *** tw2006 has quit IRC
131 2017-05-04T07:28:52  <jonasschnelli> wumpus: Re multiple tx-row-entries after a Qt bump: https://github.com/bitcoin/bitcoin/pull/9697#issuecomment-298339022
132 2017-05-04T07:29:12  <jonasschnelli> At the moment, list transactions has the same behavior, though you can see conflicts there.
133 2017-05-04T07:29:17  <jonasschnelli> Not sure how we should "solve" that in Qt...
134 2017-05-04T07:29:37  <jonasschnelli> I think I once had a PR that marked conflicted tx with a special color (maybe orange or something)
135 2017-05-04T07:30:00  <jonasschnelli> Or should we hide them completely (only the own bump conflicts)
136 2017-05-04T07:30:02  <jonasschnelli> ?
137 2017-05-04T07:36:18  *** jtimon has quit IRC
138 2017-05-04T07:36:39  *** mol has quit IRC
139 2017-05-04T07:36:47  *** moli_ has joined #bitcoin-core-dev
140 2017-05-04T07:42:47  *** aantonop has joined #bitcoin-core-dev
141 2017-05-04T07:47:58  *** aantonop has quit IRC
142 2017-05-04T07:48:55  *** justanotheruser has quit IRC
143 2017-05-04T07:52:40  <wumpus> jonasschnelli: the problem is that the transactions are only marked after a restart IMO
144 2017-05-04T07:53:27  <wumpus> not so much the marking itself
145 2017-05-04T07:54:08  <jonasschnelli> wumpus: Hmm.. you mean the opacity change, right?
146 2017-05-04T07:55:06  <jonasschnelli> Yes. The opacity is only changes after restart,... though, that's solvable.
147 2017-05-04T07:55:37  <wumpus> jonasschnelli: or how the amounts are shown (e.g. [] around it)
148 2017-05-04T07:55:38  <jonasschnelli> In a follow up PR we could introduce a new icon like this: https://github.com/bitcoin/bitcoin/pull/7826#issuecomment-220644102
149 2017-05-04T07:55:53  <jonasschnelli> Ah.. yes. the [] is also part of it. Will fix that now
150 2017-05-04T07:58:33  <wumpus> yeah :)
151 2017-05-04T08:00:37  *** Dyaheon has quit IRC
152 2017-05-04T08:02:39  *** Dyaheon has joined #bitcoin-core-dev
153 2017-05-04T08:18:15  <wumpus> I don't understand why we'd need *yet another* workaround for getentropy on linux
154 2017-05-04T08:18:44  <wumpus> man if it's really that difficult let's just use /dev/urandom and give this up
155 2017-05-04T08:19:02  *** tw2006 has joined #bitcoin-core-dev
156 2017-05-04T08:22:08  <jonasschnelli> wumpus: Agree. If /dev/urandom is not reliable, your OS is not and thus, getentropy doesn't give much more value... or do I misinterpret this?
157 2017-05-04T08:22:42  <wumpus> well the idea is that it can be run in a jail/container without /dev
158 2017-05-04T08:23:25  <jonasschnelli> I see...
159 2017-05-04T08:23:37  <wumpus> it's not so much that the syscall is more reliable (though that may be the case) but that it's always available (if the OS happens to have it avaiable...)
160 2017-05-04T08:23:37  *** tw2006 has quit IRC
161 2017-05-04T08:24:14  <wumpus> I just don't see the point in a getentropy() implementation that reads /dev/iurandom anyway
162 2017-05-04T08:24:25  <wumpus> we already have a function for that
163 2017-05-04T08:24:33  <wumpus> this is getting way more complicated than I imaginedc
164 2017-05-04T08:37:45  *** JackH has joined #bitcoin-core-dev
165 2017-05-04T08:38:48  *** harrymm has joined #bitcoin-core-dev
166 2017-05-04T08:44:42  *** mturquette has quit IRC
167 2017-05-04T08:44:59  *** mturquette has joined #bitcoin-core-dev
168 2017-05-04T08:46:41  *** jannes has joined #bitcoin-core-dev
169 2017-05-04T08:52:00  *** timothy has joined #bitcoin-core-dev
170 2017-05-04T09:11:27  *** elkalamar_ has joined #bitcoin-core-dev
171 2017-05-04T09:11:33  *** elkalamar has quit IRC
172 2017-05-04T09:11:41  *** twistedline_ has quit IRC
173 2017-05-04T09:13:23  *** Guyver2 has joined #bitcoin-core-dev
174 2017-05-04T09:16:47  *** twistedline has joined #bitcoin-core-dev
175 2017-05-04T09:19:58  *** tw2006 has joined #bitcoin-core-dev
176 2017-05-04T09:24:49  *** tw2006 has quit IRC
177 2017-05-04T09:45:24  *** e4xit has quit IRC
178 2017-05-04T10:07:12  *** e4xit has joined #bitcoin-core-dev
179 2017-05-04T10:14:56  *** nejon has quit IRC
180 2017-05-04T10:15:10  *** nejon has joined #bitcoin-core-dev
181 2017-05-04T10:20:51  *** tw2006 has joined #bitcoin-core-dev
182 2017-05-04T10:25:31  *** tw2006 has quit IRC
183 2017-05-04T10:31:28  *** chjj has quit IRC
184 2017-05-04T10:38:24  *** face has quit IRC
185 2017-05-04T10:44:32  *** chjj has joined #bitcoin-core-dev
186 2017-05-04T11:10:34  *** tw2006 has joined #bitcoin-core-dev
187 2017-05-04T11:18:24  <bitcoin-git> [bitcoin] snvakula opened pull request #10336: Get actual path for EUID instead of HOME dir (master...contrib) https://github.com/bitcoin/bitcoin/pull/10336
188 2017-05-04T11:27:44  *** jouke has quit IRC
189 2017-05-04T11:55:58  *** tw2006 has quit IRC
190 2017-05-04T11:56:35  *** tw2006 has joined #bitcoin-core-dev
191 2017-05-04T12:10:22  *** tw2006 has quit IRC
192 2017-05-04T12:11:04  *** tw2006 has joined #bitcoin-core-dev
193 2017-05-04T12:14:38  * jonasschnelli wishes a special IDE editing mode where editing a line directly amend edits & rebase and older commit
194 2017-05-04T12:23:35  *** harrymm1 has joined #bitcoin-core-dev
195 2017-05-04T12:24:30  *** harrymm1 has joined #bitcoin-core-dev
196 2017-05-04T12:25:39  *** harrymm has quit IRC
197 2017-05-04T12:35:29  *** laurentmt has joined #bitcoin-core-dev
198 2017-05-04T12:58:54  *** n1ce has joined #bitcoin-core-dev
199 2017-05-04T12:59:44  *** n1ce has quit IRC
200 2017-05-04T13:00:34  *** n1ce has joined #bitcoin-core-dev
201 2017-05-04T13:06:22  *** marcoagner has quit IRC
202 2017-05-04T13:19:38  *** marcoagner has joined #bitcoin-core-dev
203 2017-05-04T13:21:01  <jonasschnelli> morcos: ping re mempool stats: https://github.com/bitcoin/bitcoin/pull/8501#issuecomment-270714233
204 2017-05-04T13:22:35  *** chjj has quit IRC
205 2017-05-04T13:23:20  *** davec has quit IRC
206 2017-05-04T13:25:22  <jonasschnelli> morcos: Would you say it makes sense to keep a singe samples vector and add fixed time-limits for sample frequency and purge out in-between values? Example: we add a sample whenever the mempool has changed while respecting a min delta of 2s, == no extra thread required.
207 2017-05-04T13:25:30  *** davec has joined #bitcoin-core-dev
208 2017-05-04T13:26:07  <jonasschnelli> Then, during the cleanup (every 100 samples or so), we purge out in-between samples to ensure a min-frequency of, say, delta 30s for samples older then 2h (TBD)
209 2017-05-04T13:26:29  <jonasschnelli> same for older then 24h (maybe ensure a min-delta of 2min), etc.
210 2017-05-04T13:27:13  <jonasschnelli> The question then is, if we should interpolate fix time steps during cleanup/purge. Because the samples may be distributed "randomly" over time.
211 2017-05-04T13:30:11  *** Sosumi has joined #bitcoin-core-dev
212 2017-05-04T13:34:38  *** moli_ has quit IRC
213 2017-05-04T13:34:54  *** harrymm1 has quit IRC
214 2017-05-04T13:35:53  *** chjj has joined #bitcoin-core-dev
215 2017-05-04T13:35:53  *** harrymm has joined #bitcoin-core-dev
216 2017-05-04T13:36:41  *** harrymm has joined #bitcoin-core-dev
217 2017-05-04T13:39:09  <morcos> jonasschnelli: that seems considerably messier to me than keeping several samples vectors (one for each time scale we might care about) and only recording a data point in the appropriate vector if its time to. i don't think that requires any extra thread either (althgough its not clear that it wouldn't be better in a separate thread)
218 2017-05-04T13:39:21  <morcos> periodic cleanup just seems weird
219 2017-05-04T13:40:31  <jonasschnelli> morcos: Separate vectors would require move samples from one to another vector... though not sure if this bad or good. :)
220 2017-05-04T13:40:32  <morcos> i think the lack of precision in recording the data point at exactly the right time is just something i would ignore witht he exception of if we miss a sample period entirely we should do some sort of filling in
221 2017-05-04T13:40:42  <morcos> i don't understand that
222 2017-05-04T13:41:28  <morcos> every 2 seconds record in the short vector..  on the 60th second add the same sample point to the short vector and the medium vector at the same time.. on the hour add to all 3
223 2017-05-04T13:41:42  <morcos> there is some very small data duplication, but no moving, cleaning or reorganizing
224 2017-05-04T13:41:48  <jonasschnelli> Oh. Yes. I overcomplicated.
225 2017-05-04T13:42:02  <jonasschnelli> Yes. Some duplication.. but I guess that okay.
226 2017-05-04T13:42:11  <morcos> very little
227 2017-05-04T13:42:38  <jonasschnelli> Yeah. Use shared pointers *duck*
228 2017-05-04T13:43:04  <jonasschnelli> About the periodic cleanup:
229 2017-05-04T13:43:26  *** goksinen has joined #bitcoin-core-dev
230 2017-05-04T13:43:34  <jonasschnelli> I though calculating the dynamic allocation on each added sample seems a bit to expansive,.. that's why I added the periodic check
231 2017-05-04T13:43:57  <jonasschnelli> But if we have fixed timespans with a thread, then this can be solved simpler.
232 2017-05-04T13:44:30  <jonasschnelli> Though I wanted to avoid grabbing the cs_mempool to calculate the dynamic mempool size, etc.
233 2017-05-04T13:45:00  <jonasschnelli> Stats could then have an more significan impact on the mempool performance then with "passive collecting"
234 2017-05-04T13:47:18  *** davec has quit IRC
235 2017-05-04T13:50:18  *** chjj has quit IRC
236 2017-05-04T13:50:34  *** d_t has joined #bitcoin-core-dev
237 2017-05-04T13:50:51  <sipa> wumpus: presumably his configure did not detect getrandom (old kernel headers?), but did detect getentropy?
238 2017-05-04T13:52:32  <morcos> jonasschnelli: you could cache the latest value for things that were being calculated anyway, and then only record it on the periodic sample
239 2017-05-04T13:53:56  <morcos> but i'm not sure what you mean about the dynamic allocation for the added sample..  i'm assuming that at least for the smaller time scales you're going to have a limited time span for which you keep samples..  like just a recent buffer of the last couple hours at 2 sec sample periods or something..  so it seems like it should not really require much allocation
240 2017-05-04T13:54:40  <morcos> anyway.. i'm happy to revisit it in a week or so..  but i'm mostly out of the office this week.. so just trying to keep head above water
241 2017-05-04T13:55:08  *** d_t has quit IRC
242 2017-05-04T13:56:40  <jonasschnelli> What I meant is the check against the set max memory size for the stats.... say someone has set 10MB as max memory for stats then you have to know when you overshoot it. Not sure if vector-size * sizeof(sample) and cutoff the unnecessary samples is very expansive... I though doing it every 100-added sample makes sense.
243 2017-05-04T13:57:13  <jonasschnelli> morcos: Yes. No hurry... I'll ping you once I have more...
244 2017-05-04T13:57:30  *** davec has joined #bitcoin-core-dev
245 2017-05-04T13:58:56  <morcos> jonasschnelli: yeah i was imagining you initially just calculate how many samples you're going to get to keep given their memory max  and then have a ring buffer, but you coudl do it many ways.. personally i think its kind of an over configuration option.  i'd have a -stats=0 option to save the memory, but then otherwise everyone gets whatever the default memory usage we think makes sense.  not everything has to be con
246 2017-05-04T13:59:37  <jonasschnelli> Yes. fair enough... thanks for your inputs
247 2017-05-04T14:03:20  *** chjj has joined #bitcoin-core-dev
248 2017-05-04T14:06:15  <sipa> wumpus: we don't need another wrapper around urandom though; only a weak alias to a function that always returns ENOSYS, and a change in GetOSEntropy that falls back to GetDevURandom in case getentropy fails
249 2017-05-04T14:06:43  <sipa> wumpus: or, alternatively, just never use getentropy on Linux
250 2017-05-04T14:13:45  *** goksinen has quit IRC
251 2017-05-04T14:14:22  *** goksinen has joined #bitcoin-core-dev
252 2017-05-04T14:16:40  *** moli_ has joined #bitcoin-core-dev
253 2017-05-04T14:18:24  *** RubenSomsen has quit IRC
254 2017-05-04T14:23:03  *** Giszmo has joined #bitcoin-core-dev
255 2017-05-04T14:28:40  *** BashCo has quit IRC
256 2017-05-04T14:44:56  *** RubenSomsen has joined #bitcoin-core-dev
257 2017-05-04T14:45:04  *** str4d has quit IRC
258 2017-05-04T14:49:21  *** resi has joined #bitcoin-core-dev
259 2017-05-04T14:56:17  *** talmai has joined #bitcoin-core-dev
260 2017-05-04T14:59:21  <wumpus> sipa: not using getentropy on linux would make sense, after all there is a specific syscall handling for thatt
261 2017-05-04T14:59:50  * wumpus is probably not going to make today's meeting, can anyone else chair please?
262 2017-05-04T15:06:50  *** d_t has joined #bitcoin-core-dev
263 2017-05-04T15:09:20  <BlueMatt> jonasschnelli: how is #10238 intended to interact with #10235? It looks like it was designed to be complementary, but which performance improvements are you envisioning using 10238 for?
264 2017-05-04T15:09:22  <gribble> https://github.com/bitcoin/bitcoin/issues/10238 | Change setKeyPool to hold flexible entries by jonasschnelli · Pull Request #10238 · bitcoin/bitcoin · GitHub
265 2017-05-04T15:09:23  <gribble> https://github.com/bitcoin/bitcoin/issues/10235 | Track keypool entries as internal vs external in memory by TheBlueMatt · Pull Request #10235 · bitcoin/bitcoin · GitHub
266 2017-05-04T15:10:06  *** BashCo has joined #bitcoin-core-dev
267 2017-05-04T15:10:11  *** elkalamar_ has quit IRC
268 2017-05-04T15:12:54  *** talmai has quit IRC
269 2017-05-04T15:18:10  <sipa> wumpus: will do
270 2017-05-04T15:21:56  *** laurentmt has quit IRC
271 2017-05-04T15:45:17  *** jtimon has joined #bitcoin-core-dev
272 2017-05-04T15:48:36  *** abpa has joined #bitcoin-core-dev
273 2017-05-04T15:49:55  <jonasschnelli> BlueMatt: I wrote 10238 to ensure we have a fast way to lookup CKeyIds... without this, we have to read each keypool key from the database on each SyncTransaction
274 2017-05-04T15:50:05  <jonasschnelli> (with HD restore)
275 2017-05-04T15:50:31  <jonasschnelli> I shouldn't conceptually conflict with your 10235
276 2017-05-04T15:50:50  <BlueMatt> ahh, ok, i was missing the motivation part, thanks
277 2017-05-04T15:51:28  <jonasschnelli> I though the description was good... need to check...
278 2017-05-04T15:51:34  <BlueMatt> oh, maybe i missed that
279 2017-05-04T15:51:45  <jonasschnelli> Currently we only keep the keypool index in memory.. the rest is database AFAIK
280 2017-05-04T15:53:24  <morcos> wumpus: jonasschnelli: does anyone care about the QT option in Custom fee to pay "total at least"?
281 2017-05-04T15:54:26  <morcos> I've been looking at some of these recently reported issues with transations paying too high a fee, and they seem likely to be a result of coin selection logic
282 2017-05-04T15:54:52  <morcos> I think it might make for a cleaner redesign to make all fee calcluations in terms of fee rate and then take that into account when selecting inputs the first time around
283 2017-05-04T15:55:17  <jonasschnelli> morcos: not sure... I'm not sure if we want this feature...
284 2017-05-04T15:55:21  <morcos> The "total at least" option only applies to pre-selected coins anyway, but it seems like it would make for cleaner code to just dump that
285 2017-05-04T15:55:48  <morcos> I mean why does anyone care about a minimum amount of fee, only the rate matters, its not like there is a dust issue with fees
286 2017-05-04T15:56:53  <morcos> Another thing that might be nice to clean up, is the fact that setting Custom Fee in the GUI changes the paytxfee used by the command line, that just seems like a mistake to me, but i suppose changing that now might be a surprise to some poeple
287 2017-05-04T16:02:48  *** Giszmo has quit IRC
288 2017-05-04T16:09:21  *** goksinen has quit IRC
289 2017-05-04T16:15:13  *** goksinen has joined #bitcoin-core-dev
290 2017-05-04T16:20:39  *** laurentmt has joined #bitcoin-core-dev
291 2017-05-04T16:22:29  *** Giszmo has joined #bitcoin-core-dev
292 2017-05-04T16:24:00  <gmaxwell> The total at least stuff makes no sense, it should go. Feerate at least, sure.  Total no more than, sure.
293 2017-05-04T16:31:13  *** resi has left #bitcoin-core-dev
294 2017-05-04T16:35:40  *** nejon has quit IRC
295 2017-05-04T16:35:40  *** mturquette has quit IRC
296 2017-05-04T16:35:48  <morcos> gmaxwell:hmm... total no more than... you mean just the existing maxtxfee?  shoot..  i had forgotten about that
297 2017-05-04T16:36:48  <morcos> i wanted to reduce each coin by the fee needed to spend it before doing coin selection, and then we could be a bit smarter about which ones we added..
298 2017-05-04T16:37:36  <morcos> i suppose it would make some sense to have the -maxtxfee failsafe, but maybe that is at a different place in the code
299 2017-05-04T16:37:58  <gmaxwell> Yes, I think thats at a different place in the code.
300 2017-05-04T16:38:49  <morcos> no its not now, but i could move it to a different place.. like repeat coin selection if your coin selection violated maxtxfee
301 2017-05-04T16:38:59  <morcos> i want to also get rid of the knapsack solver
302 2017-05-04T16:39:53  <morcos> if you have enough inputs that are lower than the target to reach target + CENT, just start randomly adding them until you are > target+CENT, and then you're done
303 2017-05-04T16:40:44  <morcos> the idea that you are trying to get exactly target + CENT is stupid.   target + 3.5 CENTS is just as good or whatever
304 2017-05-04T16:41:07  <gmaxwell> getting a result with no change is also attractive, but I think that should be a seperate algorithim run first.
305 2017-05-04T16:41:45  <morcos> gmaxwell: hmm.. i think thats rare enough that i'm not sure its worth trying to hard for
306 2017-05-04T16:42:24  <gmaxwell> I don't think it should be, at least if we're reasonable about how much fee we'll throw away.
307 2017-05-04T16:42:48  <morcos> its the way we do that now that is kind of causing the super high fees i think
308 2017-05-04T16:42:49  <BlueMatt> so apparently the coin selection dialog's "received by" address is, essentially, complete gibberish
309 2017-05-04T16:43:03  <morcos> but that is fixable while still trying to do that, for sure
310 2017-05-04T16:43:04  *** mturquette has joined #bitcoin-core-dev
311 2017-05-04T16:43:20  <BlueMatt> and its "tree" (which I thought was supposed to group things ala listaddressgroupings) does not at all do that, and is equally useless
312 2017-05-04T16:45:24  *** RubenSomsen has quit IRC
313 2017-05-04T16:45:44  *** nejon has joined #bitcoin-core-dev
314 2017-05-04T16:48:10  <gmaxwell> morcos: I've tried a bit and have not managed to make it cause high fees since the changes we made to fix the fees in later passes. How confident are we that there is still an issue?
315 2017-05-04T16:49:13  <morcos> i think its pretty plausible
316 2017-05-04T16:49:50  <morcos> if nTotalLower is < target + CENT, then you have a decent chance of the problem happening
317 2017-05-04T16:50:14  <morcos> once nTotalLower < target + CENT, the knapsack algorithm tries to find you exactly target
318 2017-05-04T16:50:23  <morcos> which leads to decent chance that your change is dust
319 2017-05-04T16:50:28  <morcos> so your change gets eliminated
320 2017-05-04T16:51:04  <morcos> then when you callculate the fee and you need to go re-find inputs which meet target_2  (= target + fee)
321 2017-05-04T16:51:30  <morcos> you end up choosing fewer different inputs, have a lower fee, and now no change to dump the excess fee on
322 2017-05-04T16:52:49  <sipa> why does CENT even still occur in the algorithm?
323 2017-05-04T16:53:04  <morcos> actually, i used to hate that, now i think its the best part of the algorithm!
324 2017-05-04T16:53:26  <morcos> its the goal size for change, it shouldn't be an exact target, but a minimum as i mentioned above
325 2017-05-04T16:53:53  <morcos> but it's better to have a target like that, orders of magnitude higher than dust, rather than just randomly creating change ouputs that might barely pass dust
326 2017-05-04T16:54:47  <morcos> what's silly is that if you can't get CENT, then you aim for 0.
327 2017-05-04T16:54:48  *** Giszmo has quit IRC
328 2017-05-04T16:55:25  *** Giszmo has joined #bitcoin-core-dev
329 2017-05-04T16:55:29  <morcos> but i think the key improvement is to look at the net value of inputs in coin selection given the desired fee rate
330 2017-05-04T16:56:20  *** timothy has quit IRC
331 2017-05-04T16:56:37  <morcos> i suppose i could have written this all up first (i still will) but my idea was you'd have some sort of economic threshold such as only use inputs whose net_input_value > total_input_value/2 unless your tx confirm target > 48, then you use all inputs whos net_input_value > 0
332 2017-05-04T16:56:53  <morcos> the current algorithm throws in stupid inputs whose net_input_value is 0
333 2017-05-04T16:56:58  <morcos> i mean negative
334 2017-05-04T16:59:10  <morcos> btw.. i'm probably out for the meeting today.. so if anyone has any fee estimation questions, feel free to ask before..   i'm still working through comments on PR
335 2017-05-04T17:04:28  *** Giszmo has quit IRC
336 2017-05-04T17:05:25  *** anthonyjd has quit IRC
337 2017-05-04T17:05:32  *** anthonyjd has joined #bitcoin-core-dev
338 2017-05-04T17:14:22  <gmaxwell> morcos: I'd rather two two total solutions, e.g. a change and no-change solution, and then use post selection.
339 2017-05-04T17:15:53  <morcos> What information would you use from the change solution to make that decision?  It seems to me either you find a good no-change solution or you don't?
340 2017-05-04T17:19:05  <bitcoin-git> [bitcoin] sipa opened pull request #10338: Maintain state across GetStrongRandBytes calls (master...stateful_rng) https://github.com/bitcoin/bitcoin/pull/10338
341 2017-05-04T17:21:45  *** Giszmo has joined #bitcoin-core-dev
342 2017-05-04T17:21:57  <gmaxwell> both may be 'less than perfect', e.g.  no-change might give away more than we'd like, but change may involve higher fees, so it's no better.
343 2017-05-04T17:22:24  <gmaxwell> e.g. because the extra output increases the size, and then an extra input is needed... which increases the size further.
344 2017-05-04T17:22:56  <gmaxwell> and so you'd prefer the no change even though it gave away more than you might normally like, because the other option was worse.
345 2017-05-04T17:23:01  <morcos> hmm.. so my way of thinking about it, which is i thought your logic, that a solution that is 10x as big and costs 10x as much is not worse
346 2017-05-04T17:23:10  <morcos> granted, the change output vs no change is strictly worse
347 2017-05-04T17:23:29  <morcos> but that in my mind is a bit how you would go about figuring out whether you have an acceptable no change solution
348 2017-05-04T17:23:38  <gmaxwell> it's true that you'll pay the fees eventually.
349 2017-05-04T17:24:35  <gmaxwell> but then there is another angle is that our strategy should be changing if we think the feerate is 'high' for the moment, vs 'low'... in the latter we should optimize for larger transactions.
350 2017-05-04T17:24:50  <morcos> yeah, that last part is important, but i think a complicated optimization
351 2017-05-04T17:24:57  <morcos> maybe not ready for that yet
352 2017-05-04T17:25:21  <morcos> i'm including a tiny bit of that in not being willing to spend most of inputs unless its a low confirm target
353 2017-05-04T17:25:51  <morcos> but then there get to be all kinds of complicated questiosn, what kinds of fee rates does this user usually use and compare to current market conditions and size of current transaction
354 2017-05-04T17:26:20  <gmaxwell> it's a fair point that you don't want to tie up all your inputs on a rare 25-confirm-target payment.
355 2017-05-04T17:26:21  <morcos> that was all kind of a next level in my mind...  first level was meant to try to eliminate some stupid edge cases but not deviate too much from existing logic
356 2017-05-04T17:26:40  <morcos> yeah..  so many ways to so grade tx selection
357 2017-05-04T17:26:41  <gmaxwell> Fair enough.
358 2017-05-04T17:28:26  <instagibbs> morcos, I think earlier you basically described murch's algorithm, first it does some quick search for "exact match", meaning sum of effective amounts of the inputs that don't create a change output, and then if that fails backs off to random selection
359 2017-05-04T17:29:35  <morcos> instagibbs: ah yes, ok i'll go reread what he said...   i wanted to keep random selection from the lower than target value inputs if possible..  and do them on net value... that seems to me the important improvements
360 2017-05-04T17:30:17  <instagibbs> sure, that seems to mesh. He said he got 50x more exact matches than Core. Which means it might be worth it.
361 2017-05-04T17:30:33  <instagibbs> (on a real world scrubbed dataset)
362 2017-05-04T17:31:40  <instagibbs> I still think 10333 is complementary. Anytime we screw up somehow hitting the feerate we want, we can try and salvage. Hopefully will be much less important at least.
363 2017-05-04T17:32:08  <gmaxwell> and exploiting no change is important for reducing the utxo set size; I think if payment amounts come from a unimodal distribution the utxo count will grow expoentially under any algorithim that minimizes input counts and always creates change.
364 2017-05-04T17:33:12  <morcos> instagibbs: yes, i haven't read the code yet, but i'm not opposed to the idea...   but i think if we consider net values for inputs, then there is no such thing as screwing up the fee.
365 2017-05-04T17:33:53  <sipa> gmaxwell: but aiming for no change probably requires that every wallet permanently has a set of utxos of various sizes
366 2017-05-04T17:34:02  <sipa> gmaxwell: which on itself may not be ideal
367 2017-05-04T17:34:40  <gmaxwell> well then there is the extra change output, which I'd like to work out how to do, at least when the change would be large we should split the outputs.
368 2017-05-04T17:36:10  <instagibbs> morcos, yeah I can't convince myself otherwise, but having a safety net feels better. It's not a lot of logic. Treating the cause is clearly fixing coin selection.
369 2017-05-04T17:43:07  <gmaxwell> as far as the safty net, I think it's fine if it's just a dumb check at the end that aborts the transfer if it fails.
370 2017-05-04T17:43:32  <gmaxwell> people get spazzy about automatic fees because they don't know what it could pay.
371 2017-05-04T17:43:39  <gmaxwell> omg what if it pays all my monies!
372 2017-05-04T17:46:33  *** Giszmo has quit IRC
373 2017-05-04T18:05:05  *** Giszmo has joined #bitcoin-core-dev
374 2017-05-04T18:10:36  *** elkalamar has joined #bitcoin-core-dev
375 2017-05-04T18:27:20  *** d_t has joined #bitcoin-core-dev
376 2017-05-04T18:27:31  <sipa> cfields: a bit ironic... if you build with new glibc on an old kernel, but later run on a new kernel... using getentropy actually improves things
377 2017-05-04T18:27:36  *** laurentmt has quit IRC
378 2017-05-04T18:33:26  <cfields> sipa: ironic how?
379 2017-05-04T18:34:59  <sipa> cfields: in that it would seem that getentropy is never useful on linux
380 2017-05-04T18:35:14  <sipa> s/ironic/surprising/
381 2017-05-04T18:35:18  <cfields> ah
382 2017-05-04T18:37:45  *** talmai has joined #bitcoin-core-dev
383 2017-05-04T18:38:18  <cfields> well, it basically just forces the syscall to be tried on all systems, when built on something newish. but yea, the path isn't exactly obvious in all cases :)
384 2017-05-04T18:38:59  *** talmai has quit IRC
385 2017-05-04T18:39:01  <sipa> we could of course just hardcode the SYS_getrandom constant :p
386 2017-05-04T18:39:05  * sipa ducks
387 2017-05-04T18:39:12  <cfields> I suppose we could add a weak getrandom() that does the syscall and avoid the loops
388 2017-05-04T18:39:15  <cfields> haha
389 2017-05-04T18:39:27  *** d_t has quit IRC
390 2017-05-04T18:45:27  <cfields> sipa: i find the logic harder to follow in your version :\
391 2017-05-04T18:46:58  <cfields> i'm not sure if we want to cascade like that?
392 2017-05-04T18:53:17  <sipa> ok, just a suggestion
393 2017-05-04T18:57:43  *** PaulCapestany has quit IRC
394 2017-05-04T19:00:12  *** praxeology has joined #bitcoin-core-dev
395 2017-05-04T19:00:48  <sipa> yow
396 2017-05-04T19:00:56  <sipa> meeting?
397 2017-05-04T19:01:07  <sdaftuar> did anyone volunteer to chair?
398 2017-05-04T19:01:13  <sipa> #startmeeting
399 2017-05-04T19:01:13  <lightningbot> Meeting started Thu May  4 19:01:13 2017 UTC.  The chair is sipa. Information about MeetBot at http://wiki.debian.org/MeetBot.
400 2017-05-04T19:01:13  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
401 2017-05-04T19:01:23  <sipa> topics?
402 2017-05-04T19:01:36  <BlueMatt> oh, wumpus is out?
403 2017-05-04T19:01:41  <sipa> he is
404 2017-05-04T19:01:45  <instagibbs> hi
405 2017-05-04T19:01:51  <BlueMatt> #10337
406 2017-05-04T19:01:52  <gribble> https://github.com/bitcoin/bitcoin/issues/10337 | Coin Control Dialog is not (very) useful for manual privacy protection · Issue #10337 · bitcoin/bitcoin · GitHub
407 2017-05-04T19:01:56  <BlueMatt> :(
408 2017-05-04T19:02:06  <jonasschnelli> Proposed low prio topic: HD restore update / questions
409 2017-05-04T19:02:11  <sipa> #topic #10337
410 2017-05-04T19:02:12  <gribble> https://github.com/bitcoin/bitcoin/issues/10337 | Coin Control Dialog is not (very) useful for manual privacy protection · Issue #10337 · bitcoin/bitcoin · GitHub
411 2017-05-04T19:02:22  <sipa> gmaxwell: ping people?
412 2017-05-04T19:02:30  <luke-jr> BlueMatt: I agree change shouldn't be grouped as it is, but I don't understand how "received by address" is wrong
413 2017-05-04T19:02:38  <BlueMatt> turns out the coin control dialog is almost entirely useless
414 2017-05-04T19:03:01  <jonasschnelli> BlueMatt: entierly useless? I disagree
415 2017-05-04T19:03:08  <BlueMatt> the "received by address" thing works fine for coins you just received, but if it is a change output it walks back the first input until it finds a non-change output
416 2017-05-04T19:03:11  <luke-jr> BlueMatt: seems plenty useful to me
417 2017-05-04T19:03:20  <BlueMatt> so it, essentially, picks an address at random if the output is change
418 2017-05-04T19:03:24  <luke-jr> BlueMatt: I see different "received by address" for change too
419 2017-05-04T19:03:34  <BlueMatt> and groups by it, which obviously isnt what you want for privacy
420 2017-05-04T19:03:45  <BlueMatt> luke-jr: for addresses marked as change in the wallet? no
421 2017-05-04T19:04:02  *** elkalamar has quit IRC
422 2017-05-04T19:04:05  <luke-jr> BlueMatt: yes..
423 2017-05-04T19:04:08  <BlueMatt> for random addresses of yours it should work, but not for addresses via getrawchangeaddress
424 2017-05-04T19:04:23  <luke-jr> I don't use that RPC, but it works for normal change
425 2017-05-04T19:04:26  <jtimon> hi
426 2017-05-04T19:04:28  <BlueMatt> (or, ofc, normal change)
427 2017-05-04T19:05:04  <BlueMatt> https://github.com/bitcoin/bitcoin/blob/master/src/qt/walletmodel.cpp#L620
428 2017-05-04T19:05:35  <sipa> it sounds to me like it is doing a mix of "receive by address" and "linked grouping"
429 2017-05-04T19:05:44  <sipa> both are perhaps useful
430 2017-05-04T19:05:56  <BlueMatt> more importantly, really, is that I've repeatedly seen the tree mode of the coin picker dialog as the same as listaddressgroupings, which it is clearly not
431 2017-05-04T19:06:06  <BlueMatt> sipa: well the change-output-results-in-random-grouping thing is kinda strange
432 2017-05-04T19:06:20  <sipa> right, it shouldn't walk for receive address
433 2017-05-04T19:06:35  <luke-jr> BlueMatt: I have nfc what that code does, but it *looks* right in the end window :/
434 2017-05-04T19:06:39  <sipa> and it should alwaya walk (to some deterministic representative) for grouping
435 2017-05-04T19:06:45  <sipa> *alwaya
436 2017-05-04T19:06:48  <sipa> *alwayS
437 2017-05-04T19:06:53  <BlueMatt> *always
438 2017-05-04T19:07:11  <BlueMatt> but, yea, anyway, I think this should really emulate the grouping rpc
439 2017-05-04T19:07:38  <BlueMatt> the "where did these coins come from" question is not really useful for anything but coins you just got, in which case they will already be ungrouped
440 2017-05-04T19:07:38  <sipa> a "received by address" is still useful i think, but it's not the same as grouping
441 2017-05-04T19:07:50  <kanzure> hi.
442 2017-05-04T19:07:51  <BlueMatt> yes, but if it is not received directly, it should be "Change"
443 2017-05-04T19:08:03  <sipa> BlueMatt: seems reasonable to me
444 2017-05-04T19:08:28  <BlueMatt> anyway, other topics?
445 2017-05-04T19:08:39  <gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
446 2017-05-04T19:08:47  <sipa> #topic HD restore update
447 2017-05-04T19:08:57  <jonasschnelli> #10240
448 2017-05-04T19:08:58  <gribble> https://github.com/bitcoin/bitcoin/issues/10240 | Add HD wallet auto-restore functionality by jonasschnelli · Pull Request #10240 · bitcoin/bitcoin · GitHub
449 2017-05-04T19:09:14  <jonasschnelli> I have worked out a solution that seems to work for encrypted and encrypted&pruned wallets.
450 2017-05-04T19:09:17  * BlueMatt wonders why we cant have a derivation key which is not encrypted in our wallet so taht we dont have to pause sync
451 2017-05-04T19:09:24  <jonasschnelli> It can halt the sync / validation progress.
452 2017-05-04T19:09:34  <jonasschnelli> But,.. I'm not sure what gap limit and default keypool size we should use
453 2017-05-04T19:09:35  <BlueMatt> (to generate new keys)
454 2017-05-04T19:09:38  <jonasschnelli> 100 and 20 seems very little
455 2017-05-04T19:10:10  <sipa> BlueMatt: that requires non-hardened derivation
456 2017-05-04T19:10:19  <jonasschnelli> BlueMatt: IMO encryption (private key wallets) with public key derivation should be avoided
457 2017-05-04T19:10:32  <BlueMatt> sipa: yes, is that a concern?
458 2017-05-04T19:10:41  <sipa> BlueMatt: yes, we have a dumpprivkey command
459 2017-05-04T19:10:54  <jonasschnelli> We should aim – longterm – for watchonly-hd (see NicolasDorier workd) and add a signing-agent ala gpg / ssh
460 2017-05-04T19:10:56  <sipa> leaking one private key means you leak your whole walley
461 2017-05-04T19:11:09  <BlueMatt> jonasschnelli: why? your list of funds is already public to the encrypted wallet holders? that wouldnt change?
462 2017-05-04T19:11:13  <jonasschnelli> But that is alredy the next topic. :)
463 2017-05-04T19:11:25  <luke-jr> sipa: dumpprivkey isn't supposed to be safe
464 2017-05-04T19:11:32  <sipa> luke-jr: i know
465 2017-05-04T19:11:36  <luke-jr> but we could make it fail for non-hardened keys?
466 2017-05-04T19:11:43  <sipa> luke-jr: but it breaks expectations
467 2017-05-04T19:11:52  <sipa> people have a mental model about how it works
468 2017-05-04T19:12:15  <BlueMatt> sipa: maybe I'm confused on the format of HD...seems you can build a list of derivation secrets which is based on a non-encrypted private key which is unexportable?
469 2017-05-04T19:12:19  <BlueMatt> sipa: and then that would not be the case
470 2017-05-04T19:12:21  <jonasschnelli> Yes. But I vaguely remember that we once said we don't want to mix private-key wallets with public key derivation... and this makes very much sense to me
471 2017-05-04T19:12:32  <BlueMatt> (dont know the format of HD, but we could do something else if its way better)
472 2017-05-04T19:12:47  <sipa> BlueMatt: if you have the parent public key + private child key, you can compute all private child keys
473 2017-05-04T19:12:57  <jonasschnelli> If we would do child pubkey derivation, keypools could be removed (at least for HD)
474 2017-05-04T19:13:05  <sipa> BlueMatt: that is an inevitable weakness of EC based derivation
475 2017-05-04T19:13:19  <jonasschnelli> What sipa said
476 2017-05-04T19:13:20  <sipa> BlueMatt: and it is reason why bip32 has hardened keys
477 2017-05-04T19:13:30  <sipa> which core uses
478 2017-05-04T19:13:31  <BlueMatt> sipa: even if your list of privkeys is based on adding a new random value to the previous privkey where the new random value is just a hashchain of a private secret?
479 2017-05-04T19:13:53  <sipa> BlueMatt: then you lose public derivation
480 2017-05-04T19:14:13  <sipa> (the ability to compute child pubkeys without knowing the parent privkey)
481 2017-05-04T19:14:25  <BlueMatt> lets take this offline
482 2017-05-04T19:14:29  <sipa> sounds good
483 2017-05-04T19:14:35  <jonasschnelli> back to the topic: what GAP limit should we enforce by default?
484 2017-05-04T19:14:45  <BlueMatt> 1000
485 2017-05-04T19:14:52  <BlueMatt> default keypool 10k
486 2017-05-04T19:14:54  <jonasschnelli> Yeah.. I like this
487 2017-05-04T19:15:01  <jonasschnelli> But only for encrypted wallets?
488 2017-05-04T19:15:09  <jonasschnelli> IMO we should (only encrypted)
489 2017-05-04T19:15:10  <sipa> but in general i believe that most cases where the public derivation is wanted, just use huge precomputed key lists
490 2017-05-04T19:15:14  <jonasschnelli> non encrypted can say with 100
491 2017-05-04T19:15:18  <sipa> jonasschnelli: meh
492 2017-05-04T19:15:26  <sipa> why bother differentiating?
493 2017-05-04T19:15:28  <gmaxwell> why only encryption? I don't think that makes sense.
494 2017-05-04T19:15:36  <jonasschnelli> True, the gap limit must be the same.. right
495 2017-05-04T19:15:41  <jonasschnelli> sry
496 2017-05-04T19:15:42  <gmaxwell> less complexity please, and keys are cheap.
497 2017-05-04T19:15:59  <jonasschnelli> Okay, ... I'll bump it to 1000
498 2017-05-04T19:16:10  <bitcoin-git> [bitcoin] jtimon opened pull request #10339: Optimization: Calculate block hash less times (master...b15-optimization-blockhash) https://github.com/bitcoin/bitcoin/pull/10339
499 2017-05-04T19:16:29  <jonasschnelli> Next question: the tests are mostly running with a keypool size of 1... so the gap limit stuff is only enforced for the new test in #10240
500 2017-05-04T19:16:30  <gribble> https://github.com/bitcoin/bitcoin/issues/10240 | Add HD wallet auto-restore functionality by jonasschnelli · Pull Request #10240 · bitcoin/bitcoin · GitHub
501 2017-05-04T19:16:34  <jonasschnelli> Is that a problem?
502 2017-05-04T19:16:36  <gmaxwell> jtimon: thanks for working on that.
503 2017-05-04T19:17:22  <jonasschnelli> (but maybe take the test question offline).
504 2017-05-04T19:17:37  <jonasschnelli> Well,.. keypool size is answered... all good. Thanks for reviews. :p
505 2017-05-04T19:17:51  <jtimon> gmaxwell: no problem, thank you for pointing it out the other day
506 2017-05-04T19:18:16  *** tripleslash has quit IRC
507 2017-05-04T19:19:09  <jonasschnelli> other topics?
508 2017-05-04T19:19:43  <BlueMatt> review, review, review, review, review :)
509 2017-05-04T19:19:53  <sipa> yes, let's go over priority reviews
510 2017-05-04T19:20:04  <jonasschnelli> https://github.com/bitcoin/bitcoin/projects/8
511 2017-05-04T19:20:05  <sipa> #topic review requests
512 2017-05-04T19:20:17  <Chris_Stewart_5> Can #9980 be merged? Might be some what controversial
513 2017-05-04T19:20:23  <gribble> https://github.com/bitcoin/bitcoin/issues/9980 | Fix mem access violation merkleblock by Christewart · Pull Request #9980 · bitcoin/bitcoin · GitHub
514 2017-05-04T19:20:35  <BlueMatt> Chris_Stewart_5: we're super backlogged on review right now :(
515 2017-05-04T19:20:40  <Chris_Stewart_5> I thought jnewberry did a good job with the comments
516 2017-05-04T19:20:53  *** tripleslash has joined #bitcoin-core-dev
517 2017-05-04T19:20:56  <jonasschnelli> Chris_Stewart_5: I can't see an tested or untested ACK there
518 2017-05-04T19:21:10  <BlueMatt> like, super backlogged-not-gonna-get-everything-for-0.15 backlogged :(
519 2017-05-04T19:21:30  <Chris_Stewart_5> That's fine :-). Thought I would bring it up since asking for topics
520 2017-05-04T19:21:39  <BlueMatt> we can add it to the review heap
521 2017-05-04T19:22:10  <BlueMatt> can we remove #8694 until it gets fixed+rebased?
522 2017-05-04T19:22:13  <gribble> https://github.com/bitcoin/bitcoin/issues/8694 | Basic multiwallet support by luke-jr · Pull Request #8694 · bitcoin/bitcoin · GitHub
523 2017-05-04T19:22:25  <BlueMatt> it seems to have been in a constant state of not-reviewable since it was added to the "high priority for review"
524 2017-05-04T19:22:26  <instagibbs> sorry, when it 0.15 feature freeze
525 2017-05-04T19:22:31  <jonasschnelli> luke-jr: plans to rebase it?
526 2017-05-04T19:22:50  <BlueMatt> instagibbs: 2017-07-16
527 2017-05-04T19:22:58  <instagibbs> eep, ok
528 2017-05-04T19:23:25  <jonasschnelli> I though MW was one of the high profiled features targets for 0.15..
529 2017-05-04T19:23:27  <jtimon> is there anything else that needs to be done for #9494 ?
530 2017-05-04T19:23:28  <luke-jr> I just did? :/
531 2017-05-04T19:23:28  <gribble> https://github.com/bitcoin/bitcoin/issues/9494 | Introduce an ArgsManager class encapsulating cs_args, mapArgs and mapMultiArgs by jtimon · Pull Request #9494 · bitcoin/bitcoin · GitHub
532 2017-05-04T19:23:29  <BlueMatt> sdaftuar: asks for #9208
533 2017-05-04T19:23:31  <gribble> https://github.com/bitcoin/bitcoin/issues/9208 | Improve DisconnectTip performance by sdaftuar · Pull Request #9208 · bitcoin/bitcoin · GitHub
534 2017-05-04T19:23:56  <luke-jr> not really sure how to address the mapMultiArgs thing
535 2017-05-04T19:24:03  <luke-jr> besides refactoring everything using it
536 2017-05-04T19:24:06  <jonasschnelli> I add 9208 to the review-prio-list
537 2017-05-04T19:24:09  <jtimon> on the list we have #8855 from me, which remains being simple to review
538 2017-05-04T19:24:11  <gribble> https://github.com/bitcoin/bitcoin/issues/8855 | Use a proper factory for creating chainparams by jtimon · Pull Request #8855 · bitcoin/bitcoin · GitHub
539 2017-05-04T19:24:27  <BlueMatt> jonasschnelli: you already have one
540 2017-05-04T19:24:39  <gmaxwell> I really would like to see us get per-txout + atomic merged sooner rather than later, so we can get more testing time on the code.
541 2017-05-04T19:24:39  <jonasschnelli> BlueMatt: It's sdaftuar. :P
542 2017-05-04T19:24:47  *** tripleslash has quit IRC
543 2017-05-04T19:24:52  <BlueMatt> ohoh
544 2017-05-04T19:24:54  <BlueMatt> yes, sorry
545 2017-05-04T19:24:55  <jtimon> luke-jr: refactoring everything that uses mapMultiArgs is what #9494 does
546 2017-05-04T19:24:57  <gribble> https://github.com/bitcoin/bitcoin/issues/9494 | Introduce an ArgsManager class encapsulating cs_args, mapArgs and mapMultiArgs by jtimon · Pull Request #9494 · bitcoin/bitcoin · GitHub
547 2017-05-04T19:24:57  <jonasschnelli> No worries... I protect the ratio. :)
548 2017-05-04T19:25:05  <BlueMatt> jonasschnelli: I read that as "add for me", not "added"
549 2017-05-04T19:25:05  <luke-jr> jtimon: k, I'll take a look
550 2017-05-04T19:25:09  <instagibbs> jtimon, will review
551 2017-05-04T19:25:18  <cfields> gmaxwell: agreed
552 2017-05-04T19:25:20  <jtimon> awesome, thanks
553 2017-05-04T19:25:58  <sipa> let's put 9494 on the list this week
554 2017-05-04T19:26:04  <BlueMatt> either way, #8694 probably needs deleted
555 2017-05-04T19:26:04  *** tripleslash has joined #bitcoin-core-dev
556 2017-05-04T19:26:06  <gribble> https://github.com/bitcoin/bitcoin/issues/8694 | Basic multiwallet support by luke-jr · Pull Request #8694 · bitcoin/bitcoin · GitHub
557 2017-05-04T19:26:11  <luke-jr> BlueMatt: why?
558 2017-05-04T19:26:12  <jonasschnelli> I guess soon we have to introduce a review/open-new-PR ratio (only allowed to open a PR is you have carefully reviewed other PRs)
559 2017-05-04T19:26:20  <BlueMatt> luke-jr: because its not reviewable?
560 2017-05-04T19:26:20  <luke-jr> oh, from the list only, ok
561 2017-05-04T19:26:25  <BlueMatt> yeayea
562 2017-05-04T19:26:34  <sipa> i want to keep 8694 as a priority for 0.15
563 2017-05-04T19:26:51  <jonasschnelli> #8855 is already in the list by jtimon
564 2017-05-04T19:26:52  <gribble> https://github.com/bitcoin/bitcoin/issues/8855 | Use a proper factory for creating chainparams by jtimon · Pull Request #8855 · bitcoin/bitcoin · GitHub
565 2017-05-04T19:26:52  <jtimon> sipa: there's already one from me on the list
566 2017-05-04T19:27:08  <BlueMatt> sipa: I'm just saying gotta remove it from the list because its not reviewable atm, even if we want it for 0.15
567 2017-05-04T19:27:23  <sipa> BlueMatt: agree
568 2017-05-04T19:27:44  <sipa> jtimon: let's focus on the args refactoring first... it seems that that will more easily go stale
569 2017-05-04T19:27:49  <luke-jr> 0.15 and priority-review are two diff lists for a reason; let's do jtimon's PR first
570 2017-05-04T19:28:09  <sipa> luke-jr: agree
571 2017-05-04T19:28:59  <sipa> any further topics?
572 2017-05-04T19:29:13  <gmaxwell> sipa: where are things with per-txo?
573 2017-05-04T19:29:31  <jonasschnelli> #10195
574 2017-05-04T19:29:33  <gribble> https://github.com/bitcoin/bitcoin/issues/10195 | Switch chainstate db and cache to per-txout model by sipa · Pull Request #10195 · bitcoin/bitcoin · GitHub
575 2017-05-04T19:29:37  <BlueMatt> gmaxwell: needs more review, could use side-by-side benchmarks incl: memory usage, disk usage, performance numbers
576 2017-05-04T19:29:50  <sipa> yes, i'm planning to do benchmarks
577 2017-05-04T19:30:19  <gmaxwell> BlueMatt: "much faster"
578 2017-05-04T19:30:28  <sipa> other todos are better upgrade code (with a fancy progress bar...), that doesn't leave gigabytes of uncompacted data in the chainstate
579 2017-05-04T19:30:41  <sipa> but i believe it is functionally complete and tested
580 2017-05-04T19:31:00  *** Sosumi has quit IRC
581 2017-05-04T19:31:45  <BlueMatt> alright, if there are no more topics I'd emplore people to keep reviewing the big 0.15 things, since it looks like we're gonna slip a few, which is sad
582 2017-05-04T19:31:46  <sipa> it seems to make the chainstate some 20% larger
583 2017-05-04T19:32:18  <sipa> i'll report numbers on the PR, no need to discuss here
584 2017-05-04T19:32:24  <sipa> BlueMatt: ack
585 2017-05-04T19:32:31  <sipa> #endmeeting
586 2017-05-04T19:32:31  <lightningbot> Meeting ended Thu May  4 19:32:31 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
587 2017-05-04T19:32:31  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-05-04-19.01.html
588 2017-05-04T19:32:31  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-05-04-19.01.txt
589 2017-05-04T19:32:31  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-05-04-19.01.log.html
590 2017-05-04T19:32:38  <sipa> #topic lunch
591 2017-05-04T19:32:54  <luke-jr> ._.
592 2017-05-04T19:33:08  <jonasschnelli> sipa: those 120% happens also for the in-memory dbcache?
593 2017-05-04T19:33:15  <jonasschnelli> *those +20%
594 2017-05-04T19:33:18  <sipa> jonasschnelli: in memory it's much more
595 2017-05-04T19:33:31  <sipa> but that's not a fair comparison
596 2017-05-04T19:33:46  <jtimon> of "preparation PRs" for 10195: 10249 10250 have been merged, 10248 has not, still 24 commits...
597 2017-05-04T19:33:58  <sipa> as it's much more granular, fewer cached outputs may have better performamce
598 2017-05-04T19:34:12  <sipa> #10248
599 2017-05-04T19:34:13  <gribble> https://github.com/bitcoin/bitcoin/issues/10248 | Introduce CHashVerifier to hash while deserializing by sipa · Pull Request #10248 · bitcoin/bitcoin · GitHub
600 2017-05-04T19:34:31  <sipa> jtimon: most are small, but yes, it's a big change
601 2017-05-04T19:35:08  <BlueMatt> also #10199 is pretty critical for 0.15
602 2017-05-04T19:35:10  <gribble> https://github.com/bitcoin/bitcoin/issues/10199 | Better fee estimates by morcos · Pull Request #10199 · bitcoin/bitcoin · GitHub
603 2017-05-04T19:35:16  <jtimon> yeah, I was just wondering if further preparations could be separated, to make review less scary, but in the end they will be the same commits
604 2017-05-04T19:35:22  * BlueMatt votes for 10199 as most-critical user-facing feature, maybe multiwallet too
605 2017-05-04T19:35:24  <cfields> I think #10189 is ready to go, and it's blocking a few people. jtimon at least.
606 2017-05-04T19:35:26  <gribble> https://github.com/bitcoin/bitcoin/issues/10189 | devtools/net: add a verifier for scriptable changes. Use it to make CNode::id private. by theuni · Pull Request #10189 · bitcoin/bitcoin · GitHub
607 2017-05-04T19:35:55  <sipa> i could use 10189 for some of the changes in 10195 as well
608 2017-05-04T19:38:07  <jtimon> cfields: not really blocking (scripted-diff PRs can be done without the script for travis being merged and review should be similar), but yeah, I think it's ready to go too. I drafted some documentation, maybe that helps: https://0bin.net/paste/mTtIoaSoedZ3al-y#dWgD7aBKElhKofVS5nHJBvuvlDLRFmgRy8a03KroOFM
609 2017-05-04T19:41:38  <sipa> jtimon: looks like something for developer-notes.md
610 2017-05-04T19:41:40  <cfields> jtimon: thanks!
611 2017-05-04T19:42:00  <cfields> jtimon: not sure what 3) is saying through
612 2017-05-04T19:42:22  <jtimon> sipa: right, well, cfields feel free to use whatever parts of it make sense to you
613 2017-05-04T19:42:57  <cfields> jtimon: lgtm, I'm just not following that one part
614 2017-05-04T19:43:54  <jtimon> well, I needed point 3 because I was some times editing the script trying to do more specific things, so I didn't want to keep changes done by an earlier version of the script or by me manually because I stupid and I forget the commit should only do what the script does and nothing else or something
615 2017-05-04T19:44:13  <jtimon> maybe we can keep it out, I don't know
616 2017-05-04T19:45:22  <cfields> jtimon: oh, i see.
617 2017-05-04T19:47:04  <cfields> jtimon: maybe just say "1) Writer: apply your script to a clean tree and commit with an appropriate message:"
618 2017-05-04T19:47:30  <cfields> then list the other steps. I was confused because you mention committing before running the script :)
619 2017-05-04T19:50:26  <praxeology> While upgrading the chainstate to keying on id+o.ix, does it pretty much freeze the init process of the program?
620 2017-05-04T19:50:37  <sipa> praxeology: yeah...
621 2017-05-04T19:50:45  <sipa> praxeology: and it takes a while
622 2017-05-04T19:51:03  <praxeology> Sounds like you are using the same db, not trying to make a new one?
623 2017-05-04T19:51:09  <sipa> indeed
624 2017-05-04T19:51:11  <jtimon> cfields: yeah that may do it without explicitly saying the "git checkout ." trick
625 2017-05-04T19:51:32  <sipa> praxeology: it's an in-place upgrade, and it is interruotible
626 2017-05-04T19:53:09  <praxeology> interruptible?  so you make the new entries, then delete the old ones?  and if interrupt, you delete the new ones?
627 2017-05-04T19:56:10  <sipa> it adds and deletes simultaneously, atomically
628 2017-05-04T19:56:19  <sipa> you can just kill it at any point
629 2017-05-04T19:57:25  <praxeology> so the new code can handle lookup and  editing of both kinds of entries?
630 2017-05-04T20:00:03  *** tripleslash has quit IRC
631 2017-05-04T20:00:04  <praxeology> I guess it doesn't matter that much if  I understand how it works if I'm not going to help code review/test/code the progress UI
632 2017-05-04T20:00:47  <sipa> praxeology: no, at startup it just iterates through all old entries and converts them
633 2017-05-04T20:01:03  <sipa> if you interrupt during that conversion, you need to continue later
634 2017-05-04T20:01:16  *** tripleslash has joined #bitcoin-core-dev
635 2017-05-04T20:01:18  <sipa> but by interruptible i mean you don't lose progress from doing that
636 2017-05-04T20:01:56  <praxeology> "need to continue later" or else your client doesn't process new blocks/transactions?
637 2017-05-04T20:02:22  <gmaxwell> when the software restarts it will continue where it left off.
638 2017-05-04T20:02:34  <gmaxwell> this is all in the init process before it even makes any connections to anything.
639 2017-05-04T20:02:48  <sipa> praxeology: by interrupt i mean you quit during the conversion
640 2017-05-04T20:02:59  <sipa> praxeology: the node can't do anything until the conversion is complete
641 2017-05-04T20:03:09  <praxeology> ok I see
642 2017-05-04T20:04:05  *** jtimon has quit IRC
643 2017-05-04T20:04:13  <sipa> it would be possible to do the conversion on the fly, but it would result in a long term slowdown
644 2017-05-04T20:04:38  <praxeology> also more complicated code that only need be used once
645 2017-05-04T20:05:26  <praxeology> Are you making the upgrade optional?
646 2017-05-04T20:05:30  <sipa> no
647 2017-05-04T20:06:07  <sipa> the new database code cannot read or write the old format anymore, and supporting that would be complicated and slow
648 2017-05-04T20:06:10  <gmaxwell> It's optional: don't use newer software if you don't want it upgrading.
649 2017-05-04T20:06:13  <gmaxwell> :)
650 2017-05-04T20:06:24  <praxeology> Alrighty... I can see why this would be a delayed commit to the main branch.
651 2017-05-04T20:07:48  *** Dyaheon has quit IRC
652 2017-05-04T20:08:03  <praxeology> I wish I could clone myself
653 2017-05-04T20:08:15  <gmaxwell> praxeology: we do not do development in master. (few large OSS projects that use Git do)-- things that go into master are at least believed to be done and more or less releasable.
654 2017-05-04T20:08:35  *** Dyaheon has joined #bitcoin-core-dev
655 2017-05-04T20:09:32  <praxeology> Yea I know.  I am saying that I see why this feature is/may be pushed back to later and later release versions due to the requirement of locking up the node for... how long does it take to process that 2GB of data?
656 2017-05-04T20:10:18  <sipa> maybe an hour on slow hardware
657 2017-05-04T20:10:29  <sipa> slow disk, mostly
658 2017-05-04T20:10:47  <gmaxwell> 15 minutes minutes on fast hardware, I'd guess?
659 2017-05-04T20:10:53  <sipa> yeah
660 2017-05-04T20:10:54  <praxeology> and then is leveldb optimized to handle such a db transformation?  And does it compress well after deleting all the old entries?
661 2017-05-04T20:11:23  <sipa> nope, it seems
662 2017-05-04T20:11:31  <sipa> but there is an explicit call to ask it to compact
663 2017-05-04T20:11:36  <sipa> which i'll experiment with
664 2017-05-04T20:11:47  <praxeology> Why not create a new db?
665 2017-05-04T20:12:05  <sipa> Why create a new db?
666 2017-05-04T20:12:22  <praxeology> if leveldb has bad performance w/ this sort of transformation
667 2017-05-04T20:12:58  <sipa> calling db.CompactRange(); is much simpler than dealing with multiple database at once during the conversion
668 2017-05-04T20:13:28  <praxeology> because you want atomic?
669 2017-05-04T20:13:32  <gmaxwell> praxeology: it doesn't have bad performance.
670 2017-05-04T20:13:40  <sipa> praxeology: no...
671 2017-05-04T20:14:00  <sipa> praxeology: the argument you're bringing up in favor of multiple databases is because leveldb doesn't deal well with the conversion
672 2017-05-04T20:14:15  <sipa> i offer an alternative to having multiple databases at once which also deals well with the conversion
673 2017-05-04T20:14:27  <sipa> which is simpler
674 2017-05-04T20:14:31  <praxeology> ok
675 2017-05-04T20:14:43  <gmaxwell> You need to ask it to free the space or its very lazy about it, thats all.
676 2017-05-04T20:15:45  <gmaxwell> Unrelated, while watching a debug=1 logging rescan.. why is it battering out leveldb activity during rescan?
677 2017-05-04T20:16:01  <sipa> gmaxwell: huh
678 2017-05-04T20:16:55  <gmaxwell> https://0bin.net/paste/VZ72NOA3G8n-lgpF#FXx9b3te-4BbUbBH7BfGjjpXOOWUiJmL1YToCU6HRHG
679 2017-05-04T20:17:17  <praxeology> ah, so CompactRange() will just clean up a sorted key range of the db?  Which could be done incrementally along with a process that alters the format with a sorted key iterator
680 2017-05-04T20:17:22  <gmaxwell> the repeatedbash is the some instrumentation I added to count repeated hashings of the block header.. every 6 or so of those is processing a new block.
681 2017-05-04T20:17:40  <gmaxwell> praxeology: ya, thats what he's going to have it do now.
682 2017-05-04T20:17:58  <sipa> gmaxwell: that looks like just background compactions
683 2017-05-04T20:18:05  <gmaxwell> e.g. every time the leading byte of the key changes, compact.
684 2017-05-04T20:21:09  <praxeology> I've heard rumors that leveldb is unreliable.  Is that still true?
685 2017-05-04T20:21:50  <sipa> praxeology: see my answer here https://bitcoin.stackexchange.com/a/51446/208
686 2017-05-04T20:22:57  *** tw2006 has quit IRC
687 2017-05-04T20:25:07  <gmaxwell> praxeology: a lot of those rumors are a result of several things (1) (primary) leveldb has agressive checking and will actually catch lower level corruption that other things miss,  (2) there there is no windows filesystem interface for leveldb and the one we previously used was incorrect, leading to unnecessary corruption with unclean shutdowns, (3) there is a usenix paper that panned leveldb f
688 2017-05-04T20:25:13  <gmaxwell> or not handling crashes given worst case behavior of the the VFS contract that they inferred.
689 2017-05-04T20:25:45  <gmaxwell> (3) appears to be completely inconsequential in practice, e.g. I left systems running for months in constant high load plus crash loops and it always recovered.
690 2017-05-04T20:26:19  <gmaxwell> When we do get reliablity reports from users we sometimes investigate them in detail, and frequently find actual disk corruption.
691 2017-05-04T20:26:37  <gmaxwell> Unfortunately, there is a lot of windows-grade hardware out there.
692 2017-05-04T20:26:38  <sipa> or overheating CPUs
693 2017-05-04T20:27:03  <praxeology> I would guess memory failure would be more common that cpu failure in my experience
694 2017-05-04T20:27:32  <gmaxwell> yes, esp laptops, many brands of laptop cannot be run at full speed for an hour at a time without actually ending up with apparent 'memory' corruption. (presumably cpu caches).
695 2017-05-04T20:28:00  <sipa> praxeology: i would think so too, but that's not very consistent with what we're seeing (for example, for some people, running with -par=1 actually improves things)
696 2017-05-04T20:29:09  <gmaxwell> well it could be ram which behaves more poorly with the cpu taxing the power rails and making things hot. who knows.
697 2017-05-04T20:30:23  <praxeology> I've had terrible experience with cheap consumer memory from lower cost computer brands, when operating under load
698 2017-05-04T20:30:53  <sipa> i think we're seeing fewer db corruptions than a few years ago
699 2017-05-04T20:31:07  <praxeology> Are you saying there are still issues with crash recovery in windows?
700 2017-05-04T20:31:08  <sipa> that may be due to better windows env support in leveldb
701 2017-05-04T20:31:20  <sipa> or due to people running on better hardware
702 2017-05-04T20:31:49  <sipa> or just fewer people running bitcoin core on windows
703 2017-05-04T20:32:09  <sipa> not all corruption reports are on windows, to be clear
704 2017-05-04T20:32:17  <sipa> but historically those were a majority
705 2017-05-04T20:32:53  <praxeology> I have like 3 windows machines running core on windows, and they haven't corrupted.  But I have all very reliable machines that don't crash or shutdown unexpectedly
706 2017-05-04T20:34:47  <sipa> praxeology: ha, that's good to know
707 2017-05-04T20:35:01  <praxeology> Alright well best of luck w/ the utxo index change.  I'd consider help testing or maybe doing some UI work on it.  But I am very busy
708 2017-05-04T20:35:12  <sipa> thanks regardless!
709 2017-05-04T20:35:29  *** jtimon has joined #bitcoin-core-dev
710 2017-05-04T20:36:41  <praxeology> Maybe if you point me to the code and ask me for help directly I'll add it to my todo list and maybe I will do it if my priorities make it there.
711 2017-05-04T20:46:40  *** chjj has quit IRC
712 2017-05-04T20:47:52  <BlueMatt> jonasschnelli: you might kill me for #10340, sorry :/
713 2017-05-04T20:47:53  <gribble> https://github.com/bitcoin/bitcoin/issues/10340 | Add harmless missing cs_wallet lock in qt CoinControlDialog by TheBlueMatt · Pull Request #10340 · bitcoin/bitcoin · GitHub
714 2017-05-04T20:47:54  <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #10340: Add harmless missing cs_wallet lock in qt CoinControlDialog (master...2017-05-fix-mapwallet-zap-runtime) https://github.com/bitcoin/bitcoin/pull/10340
715 2017-05-04T20:54:57  *** Guyver2 has quit IRC
716 2017-05-04T20:59:24  *** chjj has joined #bitcoin-core-dev
717 2017-05-04T21:23:58  *** tw2006 has joined #bitcoin-core-dev
718 2017-05-04T21:28:29  *** tw2006 has quit IRC
719 2017-05-04T21:45:15  *** laurentmt has joined #bitcoin-core-dev
720 2017-05-04T21:57:22  *** goksinen has quit IRC
721 2017-05-04T21:57:30  <luke-jr> hmm, master doesn't seem to build for me
722 2017-05-04T21:58:27  <sipa> what error?
723 2017-05-04T22:00:26  <luke-jr> wallet/rpcwallet.cpp:732:98: error: taking address of temporary [-fpermissive]
724 2017-05-04T22:02:14  <luke-jr> I'm not entirely clear why this shouldn't be okay
725 2017-05-04T22:02:23  <luke-jr> it's returning a reference
726 2017-05-04T22:03:24  <gmaxwell> you can't return a reference to something on the stack.. which is what that sounds like?
727 2017-05-04T22:03:59  <luke-jr> no, the univalue method is returning a reference to a member
728 2017-05-04T22:04:01  <luke-jr> const std::string* account = request.params[0].get_str() != "*" ? &request.params[0].get_str() : nullptr;
729 2017-05-04T22:04:10  <luke-jr> GCC doesn't like us taking the pointer of it for some reason
730 2017-05-04T22:04:18  <sipa> that's invalid
731 2017-05-04T22:04:28  <sipa> it's taking a pointer to a temporary
732 2017-05-04T22:04:42  <sipa> ah, wait
733 2017-05-04T22:05:36  <luke-jr> FWIW, this didn't fail until I disabled ccache, so I suspect it's a new error
734 2017-05-04T22:05:54  <luke-jr> GCC 5.4
735 2017-05-04T22:06:35  *** elkalamar has joined #bitcoin-core-dev
736 2017-05-04T22:11:33  *** goksinen has joined #bitcoin-core-dev
737 2017-05-04T22:16:27  *** goksinen has quit IRC
738 2017-05-04T22:24:58  *** tw2006 has joined #bitcoin-core-dev
739 2017-05-04T22:29:41  *** tw2006 has quit IRC
740 2017-05-04T22:37:35  <ryanofsky> i think i added the line causing that error, but i also don't see why it's an error
741 2017-05-04T22:40:09  *** justanotheruser has joined #bitcoin-core-dev
742 2017-05-04T22:58:44  *** vicenteH` has quit IRC
743 2017-05-04T23:14:15  *** juscamarena_ has quit IRC
744 2017-05-04T23:14:20  *** jannes has quit IRC
745 2017-05-04T23:25:42  *** tw2006 has joined #bitcoin-core-dev
746 2017-05-04T23:30:26  *** tw2006 has quit IRC
747 2017-05-04T23:32:54  *** Dyaheon has quit IRC
748 2017-05-04T23:33:21  *** Dyaheon has joined #bitcoin-core-dev
749 2017-05-04T23:43:43  *** str4d has joined #bitcoin-core-dev
750 2017-05-04T23:57:11  *** str4d has quit IRC
751 2017-05-04T23:58:31  *** abpa has quit IRC