1 2018-04-06T00:00:03  *** weez17 has quit IRC
  2 2018-04-06T00:00:47  *** weez17 has joined #bitcoin-core-dev
  3 2018-04-06T00:04:13  *** promag has quit IRC
  4 2018-04-06T00:04:55  *** dafunkiz_ has quit IRC
  5 2018-04-06T00:06:27  *** Lauda has quit IRC
  6 2018-04-06T00:13:22  *** Lauda has joined #bitcoin-core-dev
  7 2018-04-06T00:13:38  *** dafunkiz_ has joined #bitcoin-core-dev
  8 2018-04-06T00:16:16  *** drexl has quit IRC
  9 2018-04-06T00:22:07  *** Sinclair6 has joined #bitcoin-core-dev
 10 2018-04-06T00:22:37  *** Victorsueca has quit IRC
 11 2018-04-06T00:22:50  *** Randolf has joined #bitcoin-core-dev
 12 2018-04-06T00:23:53  *** Victorsueca has joined #bitcoin-core-dev
 13 2018-04-06T00:25:57  *** grafcaps has quit IRC
 14 2018-04-06T00:25:59  *** Madars has quit IRC
 15 2018-04-06T00:28:35  *** Lauda has quit IRC
 16 2018-04-06T00:30:12  *** Lauda has joined #bitcoin-core-dev
 17 2018-04-06T00:30:30  *** AaronvanW has quit IRC
 18 2018-04-06T00:31:04  *** AaronvanW has joined #bitcoin-core-dev
 19 2018-04-06T00:33:51  *** arbitrary_guy has quit IRC
 20 2018-04-06T00:34:11  *** arbitrary_guy has joined #bitcoin-core-dev
 21 2018-04-06T00:37:06  *** AaronvanW has quit IRC
 22 2018-04-06T00:40:07  *** nitramiz has joined #bitcoin-core-dev
 23 2018-04-06T00:40:49  *** moneyball has quit IRC
 24 2018-04-06T00:43:09  *** grafcaps has joined #bitcoin-core-dev
 25 2018-04-06T01:06:34  *** dafunkiz_ has quit IRC
 26 2018-04-06T01:08:46  *** Randolf has quit IRC
 27 2018-04-06T01:10:03  *** isis_ is now known as isis
 28 2018-04-06T01:11:48  *** Randolf has joined #bitcoin-core-dev
 29 2018-04-06T01:15:21  *** andytoshi has quit IRC
 30 2018-04-06T01:16:34  *** DrOlmer has quit IRC
 31 2018-04-06T01:16:53  *** DrOlmer has joined #bitcoin-core-dev
 32 2018-04-06T01:20:42  <aj> right, different VMs, but same test_bitcion.exe, under wine coinselector_tests/knapsack_solver_test takes 581.8 seconds, under windows server 2016 w/ updates it takes 31.4s
 33 2018-04-06T01:21:40  <sipa> is there a way to do profiling in wine?
 34 2018-04-06T01:22:59  *** Madars has joined #bitcoin-core-dev
 35 2018-04-06T01:22:59  <aj> i figure i'll try a newer version of wine than what's in trusty first
 36 2018-04-06T01:29:34  *** dafunkiz_ has joined #bitcoin-core-dev
 37 2018-04-06T01:31:11  *** shesek` has joined #bitcoin-core-dev
 38 2018-04-06T01:39:41  <aj> same version of wine on xenial seems no better, unsurprisingly; but, wine 3.0 at home seems much faster
 39 2018-04-06T01:42:05  *** AaronvanW has joined #bitcoin-core-dev
 40 2018-04-06T01:43:24  <aj> total of 236s for the entire test suite under wine 3.0 vs 193s under windows server; different hardware though
 41 2018-04-06T01:45:57  *** shesek` has quit IRC
 42 2018-04-06T01:47:40  *** AaronvanW has quit IRC
 43 2018-04-06T01:49:22  *** Madars has quit IRC
 44 2018-04-06T01:49:33  *** zigen has joined #bitcoin-core-dev
 45 2018-04-06T01:51:32  *** meshcollider has quit IRC
 46 2018-04-06T02:07:59  <aj> wine1.8 from some random ppa on xenial also gets the entire test suite in 377s or so, different hardware yet again
 47 2018-04-06T02:08:58  <aj> oh, bugger, that's wine3.0 as well, should have paid more attention when installing
 48 2018-04-06T02:10:22  <bitcoin-git> [bitcoin] lachlangreenbank opened pull request #12898: Comment cleanup and consistency (master...patch-1) https://github.com/bitcoin/bitcoin/pull/12898
 49 2018-04-06T02:10:52  *** Krellan has quit IRC
 50 2018-04-06T02:15:02  *** nitramiz has quit IRC
 51 2018-04-06T02:15:34  *** dafuq_ has quit IRC
 52 2018-04-06T02:15:38  *** dafunkiz_ has quit IRC
 53 2018-04-06T02:16:06  *** murrayn has joined #bitcoin-core-dev
 54 2018-04-06T02:22:44  *** unholymachine has joined #bitcoin-core-dev
 55 2018-04-06T02:26:01  *** Murch has quit IRC
 56 2018-04-06T02:28:53  *** booyah has quit IRC
 57 2018-04-06T02:30:34  *** booyah has joined #bitcoin-core-dev
 58 2018-04-06T02:32:25  *** nitramiz has joined #bitcoin-core-dev
 59 2018-04-06T02:34:05  *** moneyball has joined #bitcoin-core-dev
 60 2018-04-06T02:34:43  *** Madars has joined #bitcoin-core-dev
 61 2018-04-06T02:35:21  *** cryptojanitor has quit IRC
 62 2018-04-06T02:50:37  *** Derrekito has quit IRC
 63 2018-04-06T02:51:28  *** Derrekito has joined #bitcoin-core-dev
 64 2018-04-06T02:53:00  *** karimofthecrop has joined #bitcoin-core-dev
 65 2018-04-06T02:57:44  *** Madars has quit IRC
 66 2018-04-06T02:58:17  *** Derrekito has quit IRC
 67 2018-04-06T03:12:52  *** moneyball has quit IRC
 68 2018-04-06T03:52:50  *** zigen has quit IRC
 69 2018-04-06T03:53:22  *** zigen has joined #bitcoin-core-dev
 70 2018-04-06T03:54:43  *** zigen has quit IRC
 71 2018-04-06T03:54:57  *** zigen has joined #bitcoin-core-dev
 72 2018-04-06T03:55:47  *** moneyball has joined #bitcoin-core-dev
 73 2018-04-06T03:56:02  *** Madars has joined #bitcoin-core-dev
 74 2018-04-06T04:00:19  *** zigen has quit IRC
 75 2018-04-06T04:00:47  *** zigen has joined #bitcoin-core-dev
 76 2018-04-06T04:05:19  *** zigen has quit IRC
 77 2018-04-06T04:05:56  *** zigen has joined #bitcoin-core-dev
 78 2018-04-06T04:11:48  *** dafunkiz_ has joined #bitcoin-core-dev
 79 2018-04-06T04:14:26  *** zigen has quit IRC
 80 2018-04-06T04:26:19  *** dafunkiz_ has quit IRC
 81 2018-04-06T04:36:09  *** indistylo has joined #bitcoin-core-dev
 82 2018-04-06T04:45:03  *** nitramiz has quit IRC
 83 2018-04-06T04:45:49  *** dafunkiz_ has joined #bitcoin-core-dev
 84 2018-04-06T04:58:32  *** rex4539 has joined #bitcoin-core-dev
 85 2018-04-06T05:00:32  *** arubi has quit IRC
 86 2018-04-06T05:01:30  *** arubi has joined #bitcoin-core-dev
 87 2018-04-06T05:02:54  <karl2> sipa: you can use gprof with wine binaries as normal, I found out the other day. It wasn't particularly exciting stuff when I tried to track down job #2 timeout thing the other day, though. Maybe running different wine versions will trigger it, like aj is saying!
 88 2018-04-06T05:02:58  <gribble> https://github.com/bitcoin/bitcoin/issues/2 | Long-term, safe, store-of-value · Issue #2 · bitcoin/bitcoin · GitHub
 89 2018-04-06T05:03:30  *** karl2 is now known as kallewoof
 90 2018-04-06T05:06:35  *** geezas has quit IRC
 91 2018-04-06T05:08:31  <bitcoin-git> [bitcoin] AkioNak opened pull request #12899: macOS: Prevent Xcode 9.3 build warnings (master...preventxcodebuildwarnings) https://github.com/bitcoin/bitcoin/pull/12899
 92 2018-04-06T05:14:40  *** clarkmoody has quit IRC
 93 2018-04-06T05:16:23  *** rex4539 has quit IRC
 94 2018-04-06T05:18:53  *** karimofthecrop has quit IRC
 95 2018-04-06T05:21:20  *** zigen has joined #bitcoin-core-dev
 96 2018-04-06T05:23:42  *** zigen has quit IRC
 97 2018-04-06T05:23:54  *** zigen has joined #bitcoin-core-dev
 98 2018-04-06T05:25:34  *** Bridgette4Marks has quit IRC
 99 2018-04-06T05:27:55  *** rex4539 has joined #bitcoin-core-dev
100 2018-04-06T05:29:27  *** Soligor has quit IRC
101 2018-04-06T05:31:59  *** dafunkiz_ has quit IRC
102 2018-04-06T05:32:07  *** moneyball has quit IRC
103 2018-04-06T05:32:23  *** Soligor has joined #bitcoin-core-dev
104 2018-04-06T05:34:47  *** dafunkiz_ has joined #bitcoin-core-dev
105 2018-04-06T05:36:11  *** Evel-Knievel has quit IRC
106 2018-04-06T05:42:19  *** aruns has joined #bitcoin-core-dev
107 2018-04-06T05:42:42  *** indistylo has quit IRC
108 2018-04-06T06:00:58  *** jtimon has quit IRC
109 2018-04-06T06:04:38  *** Madisen45Reynold has joined #bitcoin-core-dev
110 2018-04-06T06:12:46  *** zigen has quit IRC
111 2018-04-06T06:13:21  *** zigen has joined #bitcoin-core-dev
112 2018-04-06T06:17:17  <bitcoin-git> [bitcoin] martintibor40 opened pull request #12900: Update README.md (master...patch-1) https://github.com/bitcoin/bitcoin/pull/12900
113 2018-04-06T06:17:54  <bitcoin-git> [bitcoin] fanquake closed pull request #12900: Update README.md (master...patch-1) https://github.com/bitcoin/bitcoin/pull/12900
114 2018-04-06T06:17:56  *** zigen has quit IRC
115 2018-04-06T06:24:45  *** zigen has joined #bitcoin-core-dev
116 2018-04-06T06:26:04  *** zigen has quit IRC
117 2018-04-06T06:26:39  *** zigen has joined #bitcoin-core-dev
118 2018-04-06T06:26:48  *** cheese_ has joined #bitcoin-core-dev
119 2018-04-06T06:29:51  *** Cheeseo has quit IRC
120 2018-04-06T06:30:57  *** zigen has quit IRC
121 2018-04-06T06:34:15  *** zigen has joined #bitcoin-core-dev
122 2018-04-06T06:38:46  *** zigen has quit IRC
123 2018-04-06T06:44:51  *** dafunkiz_ has quit IRC
124 2018-04-06T07:03:18  *** rods1 has quit IRC
125 2018-04-06T07:03:46  *** rods1 has joined #bitcoin-core-dev
126 2018-04-06T07:07:27  *** aruns has quit IRC
127 2018-04-06T07:24:14  *** grafcaps has quit IRC
128 2018-04-06T07:25:51  *** booyah_ has joined #bitcoin-core-dev
129 2018-04-06T07:26:03  *** booyah has quit IRC
130 2018-04-06T07:29:27  *** dermoth has joined #bitcoin-core-dev
131 2018-04-06T07:34:30  *** zarez has joined #bitcoin-core-dev
132 2018-04-06T07:49:37  *** timothy has joined #bitcoin-core-dev
133 2018-04-06T08:05:03  *** setpill has joined #bitcoin-core-dev
134 2018-04-06T08:07:47  *** Madisen45Reynold has quit IRC
135 2018-04-06T08:13:17  *** atroxes has quit IRC
136 2018-04-06T08:15:43  *** atroxes has joined #bitcoin-core-dev
137 2018-04-06T08:20:56  *** murrayn has quit IRC
138 2018-04-06T08:21:20  *** murrayn has joined #bitcoin-core-dev
139 2018-04-06T08:21:21  *** murrayn has joined #bitcoin-core-dev
140 2018-04-06T08:34:09  *** unholymachine has quit IRC
141 2018-04-06T08:35:28  *** SopaXorzTaker has joined #bitcoin-core-dev
142 2018-04-06T08:37:05  *** unholymachine has joined #bitcoin-core-dev
143 2018-04-06T08:40:15  *** promag has joined #bitcoin-core-dev
144 2018-04-06T08:42:10  *** promag has quit IRC
145 2018-04-06T08:43:47  *** rex4539 has quit IRC
146 2018-04-06T08:46:13  *** rex4539 has joined #bitcoin-core-dev
147 2018-04-06T08:47:13  *** lman has joined #bitcoin-core-dev
148 2018-04-06T08:47:36  *** murrayn has quit IRC
149 2018-04-06T08:48:02  *** murrayn has joined #bitcoin-core-dev
150 2018-04-06T08:58:13  *** Victorsueca has quit IRC
151 2018-04-06T08:59:23  *** Victorsueca has joined #bitcoin-core-dev
152 2018-04-06T09:02:11  *** zigen has joined #bitcoin-core-dev
153 2018-04-06T09:02:36  *** Eldora75Hettinge has joined #bitcoin-core-dev
154 2018-04-06T09:06:37  *** zigen has quit IRC
155 2018-04-06T09:11:39  <bitcoin-git> [bitcoin] practicalswift opened pull request #12901: build: Show enabled sanitizers in configure output (master...print-enabled-sanitizers-in-configure-output) https://github.com/bitcoin/bitcoin/pull/12901
156 2018-04-06T09:13:25  *** Eldora75Hettinge has quit IRC
157 2018-04-06T09:15:57  *** davec_ has quit IRC
158 2018-04-06T09:17:21  *** DrOlmer has quit IRC
159 2018-04-06T09:17:38  *** zigen has joined #bitcoin-core-dev
160 2018-04-06T09:18:10  *** DrOlmer has joined #bitcoin-core-dev
161 2018-04-06T09:23:22  *** grafcaps has joined #bitcoin-core-dev
162 2018-04-06T09:25:24  *** cryptojanitor has joined #bitcoin-core-dev
163 2018-04-06T09:27:13  *** zigen has quit IRC
164 2018-04-06T09:27:59  *** grafcaps has quit IRC
165 2018-04-06T09:36:56  *** davec has joined #bitcoin-core-dev
166 2018-04-06T09:37:43  *** zigen has joined #bitcoin-core-dev
167 2018-04-06T09:39:58  *** gwillen has quit IRC
168 2018-04-06T09:42:13  *** gwillen has joined #bitcoin-core-dev
169 2018-04-06T09:44:20  *** zigen has quit IRC
170 2018-04-06T09:55:35  *** arbitrary_guy has quit IRC
171 2018-04-06T10:02:37  *** Conner40VonRuede has joined #bitcoin-core-dev
172 2018-04-06T10:08:11  *** jtimon has joined #bitcoin-core-dev
173 2018-04-06T10:17:27  *** maxmatics has joined #bitcoin-core-dev
174 2018-04-06T10:18:00  <maxmatics> hello world.
175 2018-04-06T10:32:01  *** arbitrary_guy has joined #bitcoin-core-dev
176 2018-04-06T10:39:57  *** arbitrary_guy has quit IRC
177 2018-04-06T10:41:32  *** maxmatics has quit IRC
178 2018-04-06T10:44:02  *** Victorsueca has quit IRC
179 2018-04-06T10:45:03  *** Victorsueca has joined #bitcoin-core-dev
180 2018-04-06T11:04:20  *** pyericz has joined #bitcoin-core-dev
181 2018-04-06T11:05:21  *** pyericz has quit IRC
182 2018-04-06T11:10:03  *** AaronvanW has joined #bitcoin-core-dev
183 2018-04-06T11:11:42  *** Aaronvan_ has joined #bitcoin-core-dev
184 2018-04-06T11:15:42  *** AaronvanW has quit IRC
185 2018-04-06T11:19:53  *** Chris_Stewart_5 has joined #bitcoin-core-dev
186 2018-04-06T11:23:27  *** Giszmo has quit IRC
187 2018-04-06T11:26:58  *** Giszmo has joined #bitcoin-core-dev
188 2018-04-06T11:31:57  *** jojeyh has joined #bitcoin-core-dev
189 2018-04-06T11:37:34  *** mrannanay has joined #bitcoin-core-dev
190 2018-04-06T11:43:15  *** pyericz has joined #bitcoin-core-dev
191 2018-04-06T11:44:03  *** grafcaps has joined #bitcoin-core-dev
192 2018-04-06T11:48:26  *** grafcaps has quit IRC
193 2018-04-06T11:49:07  *** Conner40VonRuede has quit IRC
194 2018-04-06T11:50:00  *** esotericnonsense has quit IRC
195 2018-04-06T11:50:02  *** esotericnonsens_ has joined #bitcoin-core-dev
196 2018-04-06T11:58:27  *** zautomata1 has quit IRC
197 2018-04-06T12:02:36  *** Jaleel25Walter has joined #bitcoin-core-dev
198 2018-04-06T12:18:27  *** Giszmo has quit IRC
199 2018-04-06T12:27:09  *** Evel-Knievel has joined #bitcoin-core-dev
200 2018-04-06T12:27:15  *** pyericz has quit IRC
201 2018-04-06T12:29:35  *** Jaleel25Walter has quit IRC
202 2018-04-06T12:36:24  *** Chris_Stewart_5 has quit IRC
203 2018-04-06T12:40:23  *** esotericnonsens_ is now known as esotericnonsense
204 2018-04-06T12:51:50  *** jojeyh has quit IRC
205 2018-04-06T12:51:58  *** zarez has quit IRC
206 2018-04-06T13:02:44  *** Blaze52Cormier has joined #bitcoin-core-dev
207 2018-04-06T13:09:08  *** arubi_ has joined #bitcoin-core-dev
208 2018-04-06T13:10:44  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #12898: Comment cleanup and consistency (master...patch-1) https://github.com/bitcoin/bitcoin/pull/12898
209 2018-04-06T13:12:20  *** arubi has quit IRC
210 2018-04-06T13:14:20  *** zarez has joined #bitcoin-core-dev
211 2018-04-06T13:17:14  *** Blaze52Cormier has quit IRC
212 2018-04-06T13:18:14  *** qu4ku has joined #bitcoin-core-dev
213 2018-04-06T13:18:35  *** zarez has quit IRC
214 2018-04-06T13:24:10  *** Chris_Stewart_5 has joined #bitcoin-core-dev
215 2018-04-06T13:27:02  *** wxss has joined #bitcoin-core-dev
216 2018-04-06T13:30:05  *** satwo has joined #bitcoin-core-dev
217 2018-04-06T13:32:10  *** Aaronvan_ has quit IRC
218 2018-04-06T13:34:40  *** zarez has joined #bitcoin-core-dev
219 2018-04-06T13:39:17  *** zarez has quit IRC
220 2018-04-06T13:40:39  *** Deinogalerix21 has joined #bitcoin-core-dev
221 2018-04-06T13:43:13  *** qu4ku_ has joined #bitcoin-core-dev
222 2018-04-06T13:43:27  *** AaronvanW has joined #bitcoin-core-dev
223 2018-04-06T13:44:18  *** Aaronvan_ has joined #bitcoin-core-dev
224 2018-04-06T13:44:37  *** Aaronvan_ has quit IRC
225 2018-04-06T13:45:15  *** Aaronvan_ has joined #bitcoin-core-dev
226 2018-04-06T13:46:54  *** qu4ku has quit IRC
227 2018-04-06T13:48:08  *** AaronvanW has quit IRC
228 2018-04-06T13:50:04  <bitcoin-git> [bitcoin] sdaftuar opened pull request #12902: [qa] Handle potential cookie race when starting node (master...2018-04-improve-dbcrash-restarts) https://github.com/bitcoin/bitcoin/pull/12902
229 2018-04-06T13:51:46  *** Deinogalerix21 has quit IRC
230 2018-04-06T13:53:16  *** satwo has quit IRC
231 2018-04-06T14:02:10  *** satwo has joined #bitcoin-core-dev
232 2018-04-06T14:02:41  *** Hillary82Hartman has joined #bitcoin-core-dev
233 2018-04-06T14:03:37  *** griswaalt has joined #bitcoin-core-dev
234 2018-04-06T14:04:27  *** Victorsueca has quit IRC
235 2018-04-06T14:05:43  *** Victorsueca has joined #bitcoin-core-dev
236 2018-04-06T14:09:06  *** Dizzle has joined #bitcoin-core-dev
237 2018-04-06T14:10:14  *** qu4ku_ has quit IRC
238 2018-04-06T14:10:31  *** qu4ku has joined #bitcoin-core-dev
239 2018-04-06T14:13:19  *** satwo has quit IRC
240 2018-04-06T14:16:50  *** grafcaps has joined #bitcoin-core-dev
241 2018-04-06T14:20:14  *** Hillary82Hartman has quit IRC
242 2018-04-06T14:21:05  *** grafcaps has quit IRC
243 2018-04-06T14:22:38  *** griswaalt has quit IRC
244 2018-04-06T14:28:35  *** geezas has joined #bitcoin-core-dev
245 2018-04-06T14:33:02  *** d9b4bef9 has quit IRC
246 2018-04-06T14:33:24  *** moneyball has joined #bitcoin-core-dev
247 2018-04-06T14:34:08  *** d9b4bef9 has joined #bitcoin-core-dev
248 2018-04-06T14:36:26  <stevenroose> Does bitcoin core have a utxo cache? If so, could anyone point me to the file where it is defined?
249 2018-04-06T14:36:36  *** satwo has joined #bitcoin-core-dev
250 2018-04-06T14:38:54  <stevenroose> Aha, I found -dbcache, which is the size in MiB for the utxo db cache
251 2018-04-06T14:40:04  <stevenroose> I thought the cuckoocache was used as a sigcache.
252 2018-04-06T14:40:15  <stevenroose> Is it also used for the utxo cache?
253 2018-04-06T14:41:31  <sdaftuar> stevenroose: no, it is not.  see src/coins.h and src/coins.cpp for more information about the utxo cache.
254 2018-04-06T14:44:52  <stevenroose> sdaftuar: thanks!
255 2018-04-06T14:45:17  <stevenroose> from the init code, I see that the -dbcache is split in 3, of which one is "chain state cache", what is that one for?
256 2018-04-06T14:49:37  *** satwo has quit IRC
257 2018-04-06T14:52:13  *** Giszmo has joined #bitcoin-core-dev
258 2018-04-06T14:59:42  *** zarez has joined #bitcoin-core-dev
259 2018-04-06T15:00:49  *** nitramiz has joined #bitcoin-core-dev
260 2018-04-06T15:07:02  *** rex4539 has quit IRC
261 2018-04-06T15:07:22  *** rex4539 has joined #bitcoin-core-dev
262 2018-04-06T15:09:26  <bitcoin-git> [bitcoin] sdaftuar opened pull request #12904: [qa] Ensure bitcoind processes are cleaned up when tests end (master...2018-04-always-kill-bitcoind) https://github.com/bitcoin/bitcoin/pull/12904
263 2018-04-06T15:09:32  *** nitramiz has quit IRC
264 2018-04-06T15:20:21  *** Murch has joined #bitcoin-core-dev
265 2018-04-06T15:24:47  *** Guyver2 has joined #bitcoin-core-dev
266 2018-04-06T15:25:38  *** grafcaps has joined #bitcoin-core-dev
267 2018-04-06T15:28:40  *** CubicEar_ has quit IRC
268 2018-04-06T15:29:09  *** CubicEarths has joined #bitcoin-core-dev
269 2018-04-06T15:30:18  *** grafcaps has quit IRC
270 2018-04-06T15:33:57  *** CubicEarths has quit IRC
271 2018-04-06T15:36:20  <sipa> stevenroose: for the UTXO set the layers are: (1) disk (with OS cache etc) (2) LevelDB's cache (3) Bitcoin Core's pcoinsTip (a CCoinsViewCache object)
272 2018-04-06T15:36:40  <setpill> is there any way to actively check for conflicting transactions in the mempool for a given tx?
273 2018-04-06T15:37:39  <sipa> stevenroose: there is another data set, the block index which is loaded entirely in memory, but is stored in a separate LevelDB database
274 2018-04-06T15:38:09  <sipa> so the 3 pieces -dbcache is split over is the blockindex leveldb cache, the chainstate leveldb cache (=utxo), and pcoinTip
275 2018-04-06T15:38:28  <setpill> i suppose trying to rebroadcast would give an error in some cases, but i am not sure how reliable that is, and i would prefer just simply checking without broadcasting
276 2018-04-06T15:39:22  *** qu4ku has quit IRC
277 2018-04-06T15:41:46  *** zarez has quit IRC
278 2018-04-06T15:44:14  *** Randolf has quit IRC
279 2018-04-06T15:45:52  *** qu4ku has joined #bitcoin-core-dev
280 2018-04-06T15:57:53  *** qu4ku_ has joined #bitcoin-core-dev
281 2018-04-06T15:57:54  *** qu4ku has quit IRC
282 2018-04-06T16:04:43  *** Victorsueca has quit IRC
283 2018-04-06T16:05:54  *** Victorsueca has joined #bitcoin-core-dev
284 2018-04-06T16:06:08  *** Dizzle has quit IRC
285 2018-04-06T16:09:34  *** unholymachine has quit IRC
286 2018-04-06T16:09:55  *** grafcaps has joined #bitcoin-core-dev
287 2018-04-06T16:12:08  *** Aaronvan_ is now known as AaronvanW
288 2018-04-06T16:12:09  *** moneyball has quit IRC
289 2018-04-06T16:14:21  *** zarez has joined #bitcoin-core-dev
290 2018-04-06T16:16:09  *** dgenr8 has quit IRC
291 2018-04-06T16:16:22  *** dgenr8 has joined #bitcoin-core-dev
292 2018-04-06T16:20:06  *** zarez has quit IRC
293 2018-04-06T16:20:55  *** moneyball has joined #bitcoin-core-dev
294 2018-04-06T16:25:53  *** qu4ku_ has quit IRC
295 2018-04-06T16:31:04  *** promag has joined #bitcoin-core-dev
296 2018-04-06T16:31:29  *** Giszmo has quit IRC
297 2018-04-06T16:32:55  *** wxss has quit IRC
298 2018-04-06T16:38:13  *** promag has quit IRC
299 2018-04-06T16:38:14  *** geezas has quit IRC
300 2018-04-06T16:40:17  *** Randolf has joined #bitcoin-core-dev
301 2018-04-06T16:41:51  *** twistedline_ has quit IRC
302 2018-04-06T16:42:10  *** setpill has quit IRC
303 2018-04-06T16:47:01  *** Giszmo has joined #bitcoin-core-dev
304 2018-04-06T16:48:05  *** jb55 has joined #bitcoin-core-dev
305 2018-04-06T16:48:18  *** promag has joined #bitcoin-core-dev
306 2018-04-06T16:49:27  *** twistedline has joined #bitcoin-core-dev
307 2018-04-06T17:01:41  *** rex4539 has quit IRC
308 2018-04-06T17:13:54  *** williamayd has joined #bitcoin-core-dev
309 2018-04-06T17:14:15  *** promag has quit IRC
310 2018-04-06T17:16:35  *** promag has joined #bitcoin-core-dev
311 2018-04-06T17:17:05  *** DrOlmer has quit IRC
312 2018-04-06T17:17:20  *** promag has quit IRC
313 2018-04-06T17:17:37  *** DrOlmer has joined #bitcoin-core-dev
314 2018-04-06T17:18:32  *** williamayd has quit IRC
315 2018-04-06T17:19:14  *** lman has quit IRC
316 2018-04-06T17:22:21  *** rex4539 has joined #bitcoin-core-dev
317 2018-04-06T17:25:31  *** promag has joined #bitcoin-core-dev
318 2018-04-06T17:29:48  *** promag has quit IRC
319 2018-04-06T17:29:50  *** Murch has quit IRC
320 2018-04-06T17:32:57  *** Murch has joined #bitcoin-core-dev
321 2018-04-06T17:35:03  *** Victorsueca has quit IRC
322 2018-04-06T17:35:47  *** promag has joined #bitcoin-core-dev
323 2018-04-06T17:36:15  *** Victorsueca has joined #bitcoin-core-dev
324 2018-04-06T17:36:30  *** williamayd has joined #bitcoin-core-dev
325 2018-04-06T17:39:53  *** williamayd has quit IRC
326 2018-04-06T17:40:52  *** dafunkiz_ has joined #bitcoin-core-dev
327 2018-04-06T17:43:25  *** promag has quit IRC
328 2018-04-06T17:43:32  *** isis is now known as isis_
329 2018-04-06T17:47:32  *** vicenteH has quit IRC
330 2018-04-06T17:47:41  *** jnewbery has quit IRC
331 2018-04-06T17:49:11  *** Murch has quit IRC
332 2018-04-06T17:50:27  *** Murch has joined #bitcoin-core-dev
333 2018-04-06T17:52:27  *** Chris_Stewart_5 has quit IRC
334 2018-04-06T17:57:45  *** arubi_ is now known as arubi
335 2018-04-06T17:59:42  *** jb551 has joined #bitcoin-core-dev
336 2018-04-06T17:59:45  *** Murch has quit IRC
337 2018-04-06T18:00:19  *** Murch has joined #bitcoin-core-dev
338 2018-04-06T18:02:18  *** SopaXorzTaker has quit IRC
339 2018-04-06T18:05:22  *** arubi has quit IRC
340 2018-04-06T18:05:53  *** arubi has joined #bitcoin-core-dev
341 2018-04-06T18:08:31  *** dafunkiz_ has quit IRC
342 2018-04-06T18:10:50  *** dafunkiz_ has joined #bitcoin-core-dev
343 2018-04-06T18:11:11  *** booyah_ is now known as booyah
344 2018-04-06T18:13:59  *** promag has joined #bitcoin-core-dev
345 2018-04-06T18:14:05  *** jnewbery has joined #bitcoin-core-dev
346 2018-04-06T18:15:19  *** jb551 has quit IRC
347 2018-04-06T18:16:32  *** dafunki__ has joined #bitcoin-core-dev
348 2018-04-06T18:19:17  *** promag has quit IRC
349 2018-04-06T18:19:29  *** dafunkiz_ has quit IRC
350 2018-04-06T18:21:01  *** drexl has joined #bitcoin-core-dev
351 2018-04-06T18:24:38  *** moneyball has quit IRC
352 2018-04-06T18:26:16  *** moneyball has joined #bitcoin-core-dev
353 2018-04-06T18:28:25  *** promag has joined #bitcoin-core-dev
354 2018-04-06T18:28:45  <bitcoin-git> [bitcoin] sdaftuar opened pull request #12905: [rpcwallet] Clamp walletpassphrase value at 100M seconds (master...2018-04-wallet-encryption-timeout) https://github.com/bitcoin/bitcoin/pull/12905
355 2018-04-06T18:33:42  *** AhNo1 has joined #bitcoin-core-dev
356 2018-04-06T18:34:05  *** AhNo1 has quit IRC
357 2018-04-06T18:35:02  *** d9b4bef9 has quit IRC
358 2018-04-06T18:38:08  *** d9b4bef9 has joined #bitcoin-core-dev
359 2018-04-06T18:39:28  *** Aaronvan_ has joined #bitcoin-core-dev
360 2018-04-06T18:42:44  *** AaronvanW has quit IRC
361 2018-04-06T18:47:59  *** commavir_ has quit IRC
362 2018-04-06T18:48:33  *** commavir has joined #bitcoin-core-dev
363 2018-04-06T18:49:00  *** AaronvanW has joined #bitcoin-core-dev
364 2018-04-06T18:49:37  *** Murch has quit IRC
365 2018-04-06T18:50:57  <sdaftuar> setpill: unfortunately i don't think we have great tools right now. in the next release, we'll have an rpc called "testmempoolaccept" which you could use to determine whether a given transaction would be accepted to your mempool, which might be along the lines of what you'd want?
366 2018-04-06T18:51:09  <sdaftuar> but it's tricky in general, because dealing with transaction chains is not easy
367 2018-04-06T18:51:35  *** Aaronvan_ has quit IRC
368 2018-04-06T18:52:38  <sdaftuar> for instance, if someone sends you a transaction that depends on another unconfirmed transaction, and then a third transaction conflicts with the parent and evicts it from the mempool, it's hard to tell that your transaction is indirectly conflicted as well
369 2018-04-06T18:53:16  *** Murch has joined #bitcoin-core-dev
370 2018-04-06T18:55:33  *** Randolf has quit IRC
371 2018-04-06T18:59:10  *** sturles has quit IRC
372 2018-04-06T19:00:37  *** promag has quit IRC
373 2018-04-06T19:00:56  *** promag has joined #bitcoin-core-dev
374 2018-04-06T19:02:34  *** Chris_Stewart_5 has joined #bitcoin-core-dev
375 2018-04-06T19:03:50  <stevenroose> sipa: the block index I understand
376 2018-04-06T19:04:02  <stevenroose> "the chainstate leveldb cache (=utxo), and pcoinTip" -> so what exactly is the difference there?
377 2018-04-06T19:04:16  <stevenroose> they are both used to cache the utxo set, right?
378 2018-04-06T19:04:37  *** promag has quit IRC
379 2018-04-06T19:07:28  *** sturles has joined #bitcoin-core-dev
380 2018-04-06T19:10:59  <jamesob> does POTENTIAL DEADLOCK DETECTED being logged by a node during functional test runs indicate something is definitely out of the ordinary?
381 2018-04-06T19:19:16  *** promag has joined #bitcoin-core-dev
382 2018-04-06T19:20:10  *** Chris_Stewart_5 has quit IRC
383 2018-04-06T19:21:10  *** promag has quit IRC
384 2018-04-06T19:21:21  *** dafunki__ has quit IRC
385 2018-04-06T19:21:32  <stevenroose> sipa: oh you meant with (=utxo) that it's just the leveldb cache for the store that has the utxo data
386 2018-04-06T19:21:58  <stevenroose> ok, so the only thing core maintains itself is the CCoinsViewCache, right?
387 2018-04-06T19:23:43  *** dafunkiz_ has joined #bitcoin-core-dev
388 2018-04-06T19:26:11  *** dafunkiz_ has quit IRC
389 2018-04-06T19:32:06  *** jnewbery has quit IRC
390 2018-04-06T19:32:52  *** jamesob has quit IRC
391 2018-04-06T19:33:58  *** jnewbery has joined #bitcoin-core-dev
392 2018-04-06T19:35:27  *** jamesob has joined #bitcoin-core-dev
393 2018-04-06T19:45:13  *** cryptojanitor has quit IRC
394 2018-04-06T19:45:13  *** Victorsueca has quit IRC
395 2018-04-06T19:46:24  *** Victorsueca has joined #bitcoin-core-dev
396 2018-04-06T19:47:15  <sipa> stevenroose: yes, but that's thr most important source of speedups
397 2018-04-06T19:47:35  <sipa> the leveldb cache helps ob systems with very slow i/o
398 2018-04-06T19:48:08  <stevenroose> sipa: the coinsCache map is from outpoint to utxo entry, right?
399 2018-04-06T19:48:32  <stevenroose> Doesn't that mean that for a new tx, the txhash is potentially added a lot of times?
400 2018-04-06T19:49:22  <stevenroose> in btcd, we have a structure where we map txid to utxoentry that has a (potentially sparse) map from index to output
401 2018-04-06T19:49:32  <stevenroose> but it
402 2018-04-06T19:49:53  <stevenroose> 's currently not cached, and indeed, that's one of the mayor performance bottlenecks
403 2018-04-06T19:51:34  <drexl> when an opcode has inputs, do these come from the stack?
404 2018-04-06T19:51:58  *** moneyball has quit IRC
405 2018-04-06T19:52:41  <sipa> stevenroose: i believe btcd's design was based on bitcoin core's previous
406 2018-04-06T19:52:57  <sipa> we switched from per-tx to per-txout in 0.15
407 2018-04-06T19:53:37  <sipa> leveldb deduplicates multiple key-value pairs with keys that share a prefix anyway, so on disk it's not all that impactful
408 2018-04-06T19:53:47  *** jamesob has quit IRC
409 2018-04-06T19:54:14  <sipa> and it simplifies the in-memory cache and serialization overhead for read/writes significantly
410 2018-04-06T19:54:53  <stevenroose> simplifies it a lot indeed
411 2018-04-06T19:55:16  *** rex4539 has quit IRC
412 2018-04-06T19:55:21  <stevenroose> any numbers on extra memory usage /utxo stored?
413 2018-04-06T19:55:28  <sipa> 500 MB on disk extra
414 2018-04-06T19:56:02  <stevenroose> is disk more important than memory?
415 2018-04-06T19:56:09  <sipa> imho, no
416 2018-04-06T19:56:16  *** jamesob has joined #bitcoin-core-dev
417 2018-04-06T19:56:17  <sipa> it's a small constant factor extra disk
418 2018-04-06T19:56:30  <stevenroose> I mean for initila sync mostly, a big memory cache can be very significant, no?
419 2018-04-06T19:56:30  <sipa> but it makes memory usage for the cache faster, more effective, and more efficient
420 2018-04-06T19:56:48  <stevenroose> hmm
421 2018-04-06T19:57:15  <sipa> because now we don't need to load unrelated other unspent outputs into memory
422 2018-04-06T19:57:19  <sipa> when one output is being spent
423 2018-04-06T19:57:30  *** Krellan has joined #bitcoin-core-dev
424 2018-04-06T19:59:16  <stevenroose> oh, but you don't have to do that anyway, right?
425 2018-04-06T19:59:26  <stevenroose> the entry map (index -> output) is sparce
426 2018-04-06T19:59:50  <stevenroose> also on disk, you don't store entire txs, only the unspent outputs
427 2018-04-06T19:59:55  <sipa> yes
428 2018-04-06T20:00:02  *** moneyball has joined #bitcoin-core-dev
429 2018-04-06T20:00:05  <sipa> but then you need a complex operation to write changes to disk
430 2018-04-06T20:00:17  <sipa> and i don't see how you can easily perform the freshness optimization on that
431 2018-04-06T20:00:36  *** andytoshi has joined #bitcoin-core-dev
432 2018-04-06T20:01:04  *** andytoshi has joined #bitcoin-core-dev
433 2018-04-06T20:01:07  *** cryptojanitor has joined #bitcoin-core-dev
434 2018-04-06T20:01:08  <sipa> (that's the idea that if you create a utxo in memory, and then later spend it, in memory, it can be deleted from the cache, and nothing ever needs to hit disk, because both the create-txout and spent-txout operations are idempotent)
435 2018-04-06T20:01:27  *** rex4539 has joined #bitcoin-core-dev
436 2018-04-06T20:02:14  <stevenroose> hmm, I only just started looking into this more deeply, but I don't see how storing utxos grouped per tx change that
437 2018-04-06T20:02:38  <sipa> well it means you can only do that optimization if all utxos of a tx are spent before a flush
438 2018-04-06T20:04:17  <sipa> i guess you can have a hybrid where you store them per-txout on disk, but with shared txids in memory
439 2018-04-06T20:04:27  <sipa> i looked into doing that, but the memory usage savings are tiny
440 2018-04-06T20:05:17  <stevenroose> I don't know what optimization you talk about tbh. let's say you add a tx with two outputs (you always add whole txs right? I don't think single outpoints make much sense), so you git txid -> (o1, o2)
441 2018-04-06T20:05:42  <sipa> yup
442 2018-04-06T20:06:01  <stevenroose> then, before a flush, o2 gets spent, so you keep txid -> (o1) and then when you flush, I don't see the overhead over writing txid1 -> o over txid -> (o1)
443 2018-04-06T20:06:18  <stevenroose> (opposite overhead)
444 2018-04-06T20:06:23  *** meshcollider has joined #bitcoin-core-dev
445 2018-04-06T20:06:38  <sipa> that's actually a pretty significant overhead
446 2018-04-06T20:06:53  <sipa> you need a dynamically allocated structure for variable-length output array
447 2018-04-06T20:07:47  <sipa> it's hard to compare what we're talking about because there are so many variations, depending on how you do things in memory vs on disk
448 2018-04-06T20:10:38  *** harrymm has quit IRC
449 2018-04-06T20:10:58  <stevenroose> I realize, trying to see if it's worth turning the whole structure here upside down :)
450 2018-04-06T20:11:19  <sipa> https://github.com/bitcoin/bitcoin/pull/10195
451 2018-04-06T20:18:28  <stevenroose> Hmm, not reassuring :)
452 2018-04-06T20:19:09  *** dafunkiz_ has joined #bitcoin-core-dev
453 2018-04-06T20:20:21  <sipa> what could be done is change the in-memory representation slightly where you have two maps, one txid->(int,coinbase,height,#unspents) and another (int,vout)->(amount,script)
454 2018-04-06T20:20:43  <sipa> rather than the single map we currently have (txid,vout)->(amount,script,coinbase,height)
455 2018-04-06T20:21:13  <sipa> but it becomes a lot more complicated in the presence of more complicated cache flushing strategies
456 2018-04-06T20:22:11  <sipa> perhaps if average txn over time gain more txouts this becomes better
457 2018-04-06T20:23:38  *** harrymm has joined #bitcoin-core-dev
458 2018-04-06T20:23:42  <stevenroose> well, mostly to avoid changing all the db code etc, I will probably start with a cache on the current system
459 2018-04-06T20:24:05  <stevenroose> the argument of txs becoming bigger could make sense, though
460 2018-04-06T20:24:47  <stevenroose> if we want actual unlinkability with CT, coinjoin-like structures will become increasingly common
461 2018-04-06T20:25:45  <stevenroose> but yeah the simplicity of a txout based structure is also very compelling
462 2018-04-06T20:26:02  <sipa> in any case, i'm very skeptical that any attempts to share the txids and other tx metadata in memory are worthwhile
463 2018-04-06T20:26:50  <sipa> and unfortunately the CCoinsViewCache code is pretty complicated as it takes advantage of a bunch of tricks that are specific to utxo data
464 2018-04-06T20:27:15  <sipa> so it's not trivial to just drop in another cache design
465 2018-04-06T20:27:30  <sipa> you may want to talk to eklitzke
466 2018-04-06T20:28:21  <stevenroose> "a bunch of tricks that are specific to utxo data" hmm
467 2018-04-06T20:28:56  <sipa> well in particular the freshness optimization
468 2018-04-06T20:29:00  <sipa> that i mentioned above
469 2018-04-06T20:29:08  <stevenroose> I was going through it a bit, will certainly do some more consideration before i dive into coding
470 2018-04-06T20:29:21  <stevenroose> yeah, I'll look into that
471 2018-04-06T20:29:37  <sipa> it seems like a very hard first project if you're not already somewhat familiar with the codebase :)
472 2018-04-06T20:29:51  <stevenroose> how does it handle crashes? keeping latest flushed height or so and rebuilding newer blocks from disk in  case of crash?
473 2018-04-06T20:29:58  <sipa> ah!
474 2018-04-06T20:31:18  <sipa> https://github.com/bitcoin/bitcoin/pull/10148
475 2018-04-06T20:31:31  *** ThinkOfANick has joined #bitcoin-core-dev
476 2018-04-06T20:32:37  <stevenroose> thanks!
477 2018-04-06T20:33:02  <stevenroose> when talking about cached "chain state", I suppose new blocks are always directly written to disk, no?
478 2018-04-06T20:33:12  <sipa> the chain state is the utxo set
479 2018-04-06T20:33:22  <sipa> blocks are stored completely independently
480 2018-04-06T20:33:23  <stevenroose> ok, that clears it up
481 2018-04-06T20:34:40  *** mohsen_ has joined #bitcoin-core-dev
482 2018-04-06T20:34:52  <sipa> basically on disk there is a marker with a hash of a block that means "utxos created or spent after this block MAY be present" and another that means "all utxos created or spent before this block MUST be on disk"
483 2018-04-06T20:34:54  *** mohsen_ has quit IRC
484 2018-04-06T20:35:12  <sipa> and at startup all blocks before the first and the second hash are replayed and applied to the UTXO set
485 2018-04-06T20:35:42  <sipa> this approach means we don't need to write the whole cache utxo set in one atomic operation, and also don't need to remember old deleted entries
486 2018-04-06T20:37:01  *** setpill has joined #bitcoin-core-dev
487 2018-04-06T20:38:08  <ThinkOfANick> sipa: Wait, why not remember old entries?
488 2018-04-06T20:38:27  <sipa> ThinkOfANick: because you want to save memory
489 2018-04-06T20:38:43  *** ThinkOfANick has quit IRC
490 2018-04-06T20:38:59  <stevenroose> sipa: I'm having difficulty to see how those two block locators are not the same
491 2018-04-06T20:39:16  <setpill> sdaftuar: perhaps ill have to resort to adding all the "from" addresses of the entire chain as watch-only addresses...
492 2018-04-06T20:39:29  <sipa> stevenroose: you have 1 million modified UTXO entries in memory
493 2018-04-06T20:39:32  *** mohsen_ has joined #bitcoin-core-dev
494 2018-04-06T20:39:50  *** ThinkOfANick has joined #bitcoin-core-dev
495 2018-04-06T20:39:50  <sipa> stevenroose: you can't construct a single batch to write them all at once to memory, as that batch would be gigabytes in side
496 2018-04-06T20:39:56  <sipa> so you write part of it
497 2018-04-06T20:39:58  <sipa> *size
498 2018-04-06T20:40:20  <sipa> say you last full flush was at block 400000
499 2018-04-06T20:40:21  <sipa> the current tip is 450000
500 2018-04-06T20:40:36  <sipa> that means your cache may include entries from up to block 450000
501 2018-04-06T20:40:39  <stevenroose> (partial flush, right?)
502 2018-04-06T20:40:48  <stevenroose> ah full
503 2018-04-06T20:40:52  <stevenroose> srry didn't see
504 2018-04-06T20:41:11  <sipa> you're now doing a partial flush, writing just some subset of UTXO cache entries to disk
505 2018-04-06T20:41:39  <sipa> that may include UTXOs created up to block 450000 (or may miss things that were created between 400000 and 450000)
506 2018-04-06T20:41:58  <sipa> but it's not guaranteed to contain everything up to 450000 (in fact, because you know it's partial, it can't be)
507 2018-04-06T20:42:07  <sipa> so the range you write is 400000..450000
508 2018-04-06T20:42:12  <sipa> and if a crash happens then and there
509 2018-04-06T20:42:24  <sipa> at startup you'll need to replay everything between 400000 and 450000
510 2018-04-06T20:43:07  <sipa> because all utxo creation and spending operations are idempotent, it never hurts to replay an operation that was already applied
511 2018-04-06T20:43:21  <stevenroose> I see, I just don't see how the 450000 is not just the same as the tip.. you also said "utxos created or spent after this block MAY be present" that must have been "before" then?
512 2018-04-06T20:43:38  <sipa> 450000 is the tip
513 2018-04-06T20:43:41  <sipa> in this scenario
514 2018-04-06T20:43:43  <stevenroose> yeah
515 2018-04-06T20:43:50  <sipa> no
516 2018-04-06T20:44:00  <sipa> utxos created or spent after 400000 MAY be present
517 2018-04-06T20:44:08  <sipa> utxos created or spent before 450000 MUSt be present
518 2018-04-06T20:44:13  <sipa> so the range is 400000..450000
519 2018-04-06T20:44:27  *** ThinkOfANick has left #bitcoin-core-dev
520 2018-04-06T20:44:35  <sipa> eh, i guess this is vague
521 2018-04-06T20:44:35  <stevenroose> > utxos created or spent before 450000 MUSt be present
522 2018-04-06T20:44:44  <stevenroose> yeah that's where you lost me
523 2018-04-06T20:44:46  <sipa> let me reformulate
524 2018-04-06T20:45:06  <sipa> all operations up to block 400000 are guaranteed to be on disk
525 2018-04-06T20:45:07  <stevenroose> you said "MUST be before" and "MAY be after" so I assume that is the same one
526 2018-04-06T20:45:20  <sipa> all operations between 400000 and 450000 may be present on disk, but are not guaranteed to be
527 2018-04-06T20:45:35  <sipa> my "may" and "must" were very confusing before
528 2018-04-06T20:45:44  <stevenroose> yeah, that's what confused me
529 2018-04-06T20:45:56  <sipa> but things after 450000 are guaranteed to not be on disk
530 2018-04-06T20:46:05  <stevenroose> so now let's assume there is already a persistent chain tip indicator, then you only need to keep one, right?
531 2018-04-06T20:46:17  <sipa> well this is the chain tip indicator
532 2018-04-06T20:46:24  <sipa> instead of a tip, it's a range
533 2018-04-06T20:46:43  <stevenroose> that's why I was confused for it to be two. ok ok, yeah I got it. I just assumed you would always need a chaintip anyways
534 2018-04-06T20:46:55  <sipa> in the ideal scenario the two are the same
535 2018-04-06T20:46:58  <sipa> after a full flush
536 2018-04-06T20:47:03  <stevenroose> one last thing I'm missing
537 2018-04-06T20:47:35  <stevenroose> how to know when to update that consistence height
538 2018-04-06T20:47:38  <stevenroose> (the first one of the range)
539 2018-04-06T20:47:45  <sipa> ah, right now it's very simple
540 2018-04-06T20:48:19  <sipa> when we start a flush operation, we check what the previous lower-height (the 400000) was, and update it to (that lower height, current tip)
541 2018-04-06T20:48:34  <sipa> after a flush operation completes, it's replaced with (current tip, current tip)
542 2018-04-06T20:49:10  <stevenroose> oh, but then you still need full flushes?
543 2018-04-06T20:49:16  <stevenroose> so you can't update in partial writes?
544 2018-04-06T20:49:29  <sipa> well we only have full flushes now
545 2018-04-06T20:49:36  <sipa> but they're implemented as a sequence of partial flushes
546 2018-04-06T20:49:53  <sipa> longer term i want a system where we have a background thread that's constantly flushing
547 2018-04-06T20:50:09  <sipa> and is always "running behind" on the tip
548 2018-04-06T20:50:25  <sipa> to give the memory db a chance to cache creates/spends that cancel each other out before writing
549 2018-04-06T20:50:46  <sipa> but keeping track of which is the lower hash in the range in that system is more complicated
550 2018-04-06T20:51:18  <sipa> it's basically the lowest height of which you either have an unwritten create or unwritten spend
551 2018-04-06T20:51:33  <sipa> but it's more tricky in the presence of reorganizations
552 2018-04-06T20:51:44  <sipa> https://github.com/bitcoin/bitcoin/blob/master/src/txdb.cpp#L102L137 <- current flushing to disk logic
553 2018-04-06T20:52:37  <stevenroose> oooooooh
554 2018-04-06T20:53:05  <stevenroose> ok, I thought you were doing a partial flush when the cache was full
555 2018-04-06T20:53:18  <stevenroose> (like LRI fashion or so)
556 2018-04-06T20:53:22  <sipa> nope
557 2018-04-06T20:53:29  <stevenroose> but it's just a way to reduce the size of the transaction
558 2018-04-06T20:53:38  <stevenroose> ldb transaction
559 2018-04-06T20:53:49  <sipa> i've experimented with various approaches for MRU eviction from the cache etc
560 2018-04-06T20:53:51  <stevenroose> oh yeah than you can just do all batches and update the pointer
561 2018-04-06T20:54:03  *** dafunkiz_ has quit IRC
562 2018-04-06T20:54:03  <sipa> but they're basically all slower than what we're doing now (on fast hw at least)
563 2018-04-06T20:55:01  <stevenroose> sipa: yeah you'd have the problem of knowing to what hash it's consistent. you'd need to keep heights in entries and iterate over all entries once and a while to see what the most recent dirty one is
564 2018-04-06T20:55:26  <sipa> the *least* recent dirty one, yes
565 2018-04-06T20:55:40  <sipa> thankfully, utxo entries already have a height
566 2018-04-06T20:55:43  <stevenroose> excuseer :p
567 2018-04-06T20:56:07  <sipa> unfortunately, that's the creation height and not really the modification height (which may differ in the case of a spend or a reorg)
568 2018-04-06T20:56:54  <stevenroose> yeah I keep having a hard time picturing reorg handling there
569 2018-04-06T20:57:10  <stevenroose> because when you don't have a txindex and you delete an entry, it's impossible to get it back :D
570 2018-04-06T20:57:21  <sipa> oh, we have undo data
571 2018-04-06T20:57:24  <sipa> the *.rev files
572 2018-04-06T20:57:26  <stevenroose> do you keep like "revert objects" for the last X blocks
573 2018-04-06T20:57:33  <stevenroose> ah
574 2018-04-06T20:57:40  *** dafunkiz_ has joined #bitcoin-core-dev
575 2018-04-06T20:58:20  <stevenroose> how many are there? (thinking hardfork races here where two chains constantly catch up with each other and fuck old nodes)
576 2018-04-06T20:58:39  <sipa> there is one per block
577 2018-04-06T20:58:46  <sipa> and we prune them along with the blocks themselves
578 2018-04-06T20:59:04  <stevenroose> (technically wouldnt be a hardfork in that case though)
579 2018-04-06T20:59:44  <stevenroose> wait, why keep for all? or are they only like pointers to the actual data?
580 2018-04-06T21:00:05  <sipa> because we need to be able to reorg?
581 2018-04-06T21:00:22  <stevenroose> I haven't seen any of that data in btcd's codebase, let me dig to that tomorrow :)
582 2018-04-06T21:00:24  *** jigawatt has left #bitcoin-core-dev
583 2018-04-06T21:00:40  <stevenroose> yeah I know, but well, reorging over half the chain is kinda unlikely, isn't it?
584 2018-04-06T21:00:48  <sipa> yes
585 2018-04-06T21:00:55  <stevenroose> I'd say at least before the last checkpoint makes no sense..
586 2018-04-06T21:01:04  <sipa> checkpoints need to go away
587 2018-04-06T21:01:09  <stevenroose> oh
588 2018-04-06T21:01:28  <sipa> but yes, sure, it's unlikely that deep reorgs happen
589 2018-04-06T21:01:42  <stevenroose> what's the fundamental problem with checkpoints?
590 2018-04-06T21:01:52  <sipa> they confuse people
591 2018-04-06T21:01:54  <sipa> :)
592 2018-04-06T21:02:09  <sipa> they're seen as a security measure
593 2018-04-06T21:02:42  <stevenroose> more as an efficiency tool :p I mean you can skip verification up to that point
594 2018-04-06T21:03:02  <sipa> we have assumevalid for that now, which is far less invasive
595 2018-04-06T21:03:12  <stevenroose> (f.e. when syncing a node I always "ask my friends for the latest block they trust" (i.e. check some explorers) and do --addcheckpoint
596 2018-04-06T21:03:16  <sipa> it doesn't restrict which chain is valid
597 2018-04-06T21:03:30  <sipa> it just skips validation for any block that is an ancestor of a known valid block
598 2018-04-06T21:03:44  <sipa> but if the best chain we see is different than the assumevalid one, we'll accept it (after validating)
599 2018-04-06T21:04:05  <stevenroose> ok yeah that's a better version of a checkpoint
600 2018-04-06T21:04:31  <sipa> yes, assumevalid is updated from time to time, but we haven't modified checkpoints in years
601 2018-04-06T21:04:39  <stevenroose> but checkpoints are also usefull against eclipse attacks when you're just getting started
602 2018-04-06T21:04:46  <sipa> no
603 2018-04-06T21:05:10  <stevenroose> at least they let you know something is up, no?
604 2018-04-06T21:05:16  <sipa> they're useful against being spammed with low difficulty headers
605 2018-04-06T21:05:25  <sipa> but that's independent of eclipse attacks
606 2018-04-06T21:05:48  <sipa> we need backward headers sync to remove that dependency on checkpoints
607 2018-04-06T21:06:12  <stevenroose> backward header sync?
608 2018-04-06T21:06:23  *** laurentmt has joined #bitcoin-core-dev
609 2018-04-06T21:06:28  <sipa> first learn the best header, and only then download headers along that path
610 2018-04-06T21:06:51  <sipa> as opposed to downloading whatever header people give you, hoping that indeed it'll turn out to be one with more work than your current one
611 2018-04-06T21:07:01  <sipa> (that's how it works now)
612 2018-04-06T21:07:53  <stevenroose> how can you learn the best header?
613 2018-04-06T21:08:17  <sipa> using a yet to be devised protocol :)
614 2018-04-06T21:08:18  <stevenroose> asking all peers? (heighest checkpoint? :p)
615 2018-04-06T21:08:22  <stevenroose> oh
616 2018-04-06T21:08:29  *** laurentmt has quit IRC
617 2018-04-06T21:09:06  <sipa> there are some ideas about random sampling, where someone can send you a merkle sum tree over all their headers, and then you randomly query it a number of times to see if they indeed have the distribution of pow they claim
618 2018-04-06T21:09:09  <stevenroose> I recently thought about a backwards sync mechanism for initila utxo building. but I guess it's kinda not worth the effort when there is a good utxo cache
619 2018-04-06T21:10:20  <sipa> and once you've done enough queries, you know they actually have a chain with a certain amount of work
620 2018-04-06T21:10:31  <sipa> and if that amount of work is good enough, you can start downloading the actual headers
621 2018-04-06T21:10:59  <setpill> sipa: wouldn't just believing the claimed amount of accumulated work + ban on lie work?
622 2018-04-06T21:11:10  <stevenroose> are checkpoints really that bad?
623 2018-04-06T21:11:36  <sipa> stevenroose: people seem to misunderstand that if checkpoints ever have an effect, bitcoin is broken
624 2018-04-06T21:11:56  <sipa> i'm much more comfortable to have much weaker assumptions about correctness of the code
625 2018-04-06T21:12:06  <sipa> (which includes the checkpoints)
626 2018-04-06T21:12:17  <sipa> setpill: how is that better than what we have now?
627 2018-04-06T21:13:15  <sipa> stevenroose: i don't think they're terrible, but we also don't really need them anymore, except for this tiny DoS concern
628 2018-04-06T21:13:16  <setpill> sipa: last blocks are likely to have more pow behind them so are more expensive to maliciously craft
629 2018-04-06T21:13:29  <stevenroose> "if checkpoints ever have an effect, bitcoin is broken" you mean that when code breaks validity of an old block that no one will validate because checkpoints?
630 2018-04-06T21:14:10  <sipa> stevenroose: i mean that if checkpoints ever prevent the network from reorging to an attacker chain, it's clear that the concept of PoW itself is brokenb
631 2018-04-06T21:15:38  *** promag has joined #bitcoin-core-dev
632 2018-04-06T21:15:41  <stevenroose> and what if they prevent *a new node* from syncing a wrong chain? that's what their main use is imho
633 2018-04-06T21:16:08  *** dafunkiz_ has quit IRC
634 2018-04-06T21:16:10  <sipa> stevenroose: that's not what they do
635 2018-04-06T21:16:18  <sipa> stevenroose: they just prevent OOM
636 2018-04-06T21:16:30  <stevenroose> I mean even as a spam vector, a decent miner right now can prob create a quite significantly long chain that is use in size (1MB blocks) and has legit work
637 2018-04-06T21:16:46  <stevenroose> s/use/huge/
638 2018-04-06T21:16:46  <sipa> yes, absolutely - that's exactly the one thing they still do
639 2018-04-06T21:17:00  <stevenroose> OOM?
640 2018-04-06T21:17:07  <sipa> out of memory
641 2018-04-06T21:17:15  <sipa> also, not actually blocks, just headers
642 2018-04-06T21:17:31  <sipa> we don't download block data until a chain of validated headers is actually the best chain
643 2018-04-06T21:18:18  <stevenroose> yeah true, so that would only work if eclipsed
644 2018-04-06T21:18:25  <sipa> oh, and there is a known min amount of work
645 2018-04-06T21:18:31  <sipa> independently of checkpoints
646 2018-04-06T21:18:52  <stevenroose> "min amount of work at height x"?
647 2018-04-06T21:18:53  <sipa> so we never accept a headers chain until it passes that point
648 2018-04-06T21:19:00  <stevenroose> oh llike that
649 2018-04-06T21:19:02  <stevenroose> cumulative
650 2018-04-06T21:19:06  *** dafunkiz_ has joined #bitcoin-core-dev
651 2018-04-06T21:19:09  <stevenroose> thats neat
652 2018-04-06T21:19:59  <setpill> sipa: interesting, i hadnt heard of that; is that documented somewhere?
653 2018-04-06T21:21:51  <sipa> -minimumchainwork cmdline option
654 2018-04-06T21:22:59  <stevenroose> well, thanks for the insights :)
655 2018-04-06T21:23:54  <sipa> yw!
656 2018-04-06T21:26:22  <setpill> sipa: ahh, so its another "checkpoint-esque" thing, as in a hardcoded value that gets updated periodically?
657 2018-04-06T21:27:05  <setpill> for a second i was under the impression pow inflation somehow had a lower bound ^^'
658 2018-04-06T21:27:55  <sipa> setpill: there is a minimum difficulty, but it's trivial
659 2018-04-06T21:28:19  <setpill> yeah and wont help much against a malicious chain
660 2018-04-06T21:28:34  *** owowo has quit IRC
661 2018-04-06T21:29:18  <sipa> a single modern CPU thread can create a minimum-difficulty block in a few minute
662 2018-04-06T21:29:41  <sipa> modern HW miners can make 1000s per second
663 2018-04-06T21:30:55  <setpill> yeah, so i thought there was a higher-than-that, actually useful lower bound on difficulty based on something i was unaware of
664 2018-04-06T21:31:21  <sipa> there is also the max-divide-by-4 rule for difficulty changes
665 2018-04-06T21:33:34  <stevenroose> sipa: is that a consensus rule??
666 2018-04-06T21:33:40  <sipa> yes
667 2018-04-06T21:33:42  <stevenroose> was not aware of that
668 2018-04-06T21:33:47  <sipa> always has been
669 2018-04-06T21:33:51  <stevenroose> is there a max-multiply-by-x one?
670 2018-04-06T21:33:55  <sipa> yes, by 4
671 2018-04-06T21:34:07  <stevenroose> aha
672 2018-04-06T21:34:07  <sipa> neither rule has ever been hit
673 2018-04-06T21:34:16  <sipa> on mainnet at least
674 2018-04-06T21:34:33  <stevenroose> I hate the zero-diff-after-20-mins testnet rule
675 2018-04-06T21:34:42  <sipa> haha
676 2018-04-06T21:34:43  *** owowo has joined #bitcoin-core-dev
677 2018-04-06T21:35:08  <stevenroose> I used to try and testnet mine with an old butterfly labs jalapeno but somehow was never hitting one
678 2018-04-06T21:35:26  <sipa> you need to set your timestamp in the future :)
679 2018-04-06T21:35:27  <stevenroose> btcd didn't notify gbt long polls on the 20 minute hit
680 2018-04-06T21:35:50  *** promag has quit IRC
681 2018-04-06T21:35:52  <stevenroose> also
682 2018-04-06T21:36:57  *** Giszmo has quit IRC
683 2018-04-06T21:43:49  *** timothy has quit IRC
684 2018-04-06T21:44:20  *** Guyver2 has quit IRC
685 2018-04-06T21:47:05  *** grafcaps has quit IRC
686 2018-04-06T21:47:06  *** Randolf has joined #bitcoin-core-dev
687 2018-04-06T21:54:47  *** jb55 has quit IRC
688 2018-04-06T21:56:02  *** Giszmo has joined #bitcoin-core-dev
689 2018-04-06T22:10:16  *** grafcaps has joined #bitcoin-core-dev
690 2018-04-06T22:12:34  *** dafunkiz_ has quit IRC
691 2018-04-06T22:18:02  *** dafunkiz_ has joined #bitcoin-core-dev
692 2018-04-06T22:24:44  *** vicenteH has joined #bitcoin-core-dev
693 2018-04-06T22:26:09  *** meshcollider has quit IRC
694 2018-04-06T22:34:01  *** d9b4bef9 has quit IRC
695 2018-04-06T22:34:04  *** promag has joined #bitcoin-core-dev
696 2018-04-06T22:35:08  *** d9b4bef9 has joined #bitcoin-core-dev
697 2018-04-06T22:36:02  *** d9b4bef9 has quit IRC
698 2018-04-06T22:37:14  *** d9b4bef9 has joined #bitcoin-core-dev
699 2018-04-06T22:39:13  *** cryptapus has quit IRC
700 2018-04-06T22:45:27  *** dafunkiz_ has quit IRC
701 2018-04-06T22:47:14  *** grafcaps has quit IRC
702 2018-04-06T22:51:26  *** dafunkiz_ has joined #bitcoin-core-dev
703 2018-04-06T22:51:43  *** cryptapus has joined #bitcoin-core-dev
704 2018-04-06T22:51:43  *** cryptapus has quit IRC
705 2018-04-06T22:51:43  *** cryptapus has joined #bitcoin-core-dev
706 2018-04-06T22:53:28  *** meshcollider has joined #bitcoin-core-dev
707 2018-04-06T22:58:26  *** rex4539 has quit IRC
708 2018-04-06T22:58:39  *** CubicEarths has joined #bitcoin-core-dev
709 2018-04-06T23:03:20  *** CubicEarths has quit IRC
710 2018-04-06T23:03:49  *** CubicEarths has joined #bitcoin-core-dev
711 2018-04-06T23:05:44  <jtimon> sipa: re protocol to know the best chain: couldn't compact pow proofs be better for that than random sampling?
712 2018-04-06T23:06:20  <jtimon> hi stevenroose
713 2018-04-06T23:06:34  *** dafunkiz_ has quit IRC
714 2018-04-06T23:08:44  *** CubicEarths has quit IRC
715 2018-04-06T23:08:48  <sipa> jtimon: compact pow proofs need consensus rules
716 2018-04-06T23:08:52  <sipa> bitcoin doesn't have those
717 2018-04-06T23:09:27  *** dafunkiz_ has joined #bitcoin-core-dev
718 2018-04-06T23:09:54  <jtimon> sure, I mean assuming a compact proofs sf
719 2018-04-06T23:14:07  *** setpill has quit IRC
720 2018-04-06T23:16:25  <intcat> is there any (u)txo commitment design (close to becoming) a BIP?
721 2018-04-06T23:16:45  <intcat> ive read some ml interaction between peter todd and bram cohen but it's a few years old and im not sure how much has happened on that since
722 2018-04-06T23:33:38  *** Randolf has quit IRC
723 2018-04-06T23:35:36  *** grafcaps has joined #bitcoin-core-dev
724 2018-04-06T23:37:41  *** promag has quit IRC
725 2018-04-06T23:40:47  *** dafunkiz_ has quit IRC
726 2018-04-06T23:41:08  *** harrymm has quit IRC
727 2018-04-06T23:44:49  *** arbitrary_guy has joined #bitcoin-core-dev
728 2018-04-06T23:46:00  *** dafunkiz_ has joined #bitcoin-core-dev
729 2018-04-06T23:48:29  *** Victorsueca has quit IRC
730 2018-04-06T23:49:55  *** Victorsueca has joined #bitcoin-core-dev
731 2018-04-06T23:51:16  *** harrymm has joined #bitcoin-core-dev
732 2018-04-06T23:54:54  *** Sinclair6 has quit IRC