1 2017-02-25T00:02:22  <cfields> sipa: want me to PR that? Or you have something else in mind?
  2 2017-02-25T00:02:59  <BlueMatt> sipa: ok, here's a theory......"10:39:08 Prune:" is the prune called by AcceptBlock's FlushStateToDisk call (seems obvious given the Pre-allocating blk just above)....now, that flush call hit fDoFullFlush triggering a pcoinsTip->Flush(). OK, so now in CCoinsViewDB::BatchWrite we iterate over the entries in mapCoins, allocating memory for each one in the leveldb batch, erasing *AS WE GO*...at some point it ran out of memory, throwing
  3 2017-02-25T00:02:59  <BlueMatt> bad_alloc and leaving pcoinsTip in an inconsistent state, but std::bad_alloc is not an std::runtime_error so the catch at the end of FlushStateToDisk didnt catch it. OK, phew, so now we go to connect the next block and fail to connect its inputs (without allocating new disk space). I /think/ that explains your debug.log without hand waving about hardware errors
  4 2017-02-25T00:04:06  *** alpalp has joined #bitcoin-core-dev
  5 2017-02-25T00:04:06  *** alpalp has joined #bitcoin-core-dev
  6 2017-02-25T00:04:36  <sipa> BlueMatt: sounds like a plausible theory
  7 2017-02-25T00:04:40  <sipa> cfields: PR what?
  8 2017-02-25T00:04:49  <cfields> BlueMatt: that was my reading of it as well
  9 2017-02-25T00:05:07  *** MarcoFalke has quit IRC
 10 2017-02-25T00:05:11  <cfields> sipa: https://github.com/theuni/bitcoin/commit/28afe574567ba838d46959047282460dbab39b91
 11 2017-02-25T00:05:24  <sipa> if that's the problem (and that seems to become more plausible as dbcache gets set higher, as you get the duplication only at flush time)
 12 2017-02-25T00:05:43  <sipa> we should probably make the batch construction not erase things for ccoinscache
 13 2017-02-25T00:06:12  <BlueMatt> sipa: yes, an easy way to make this particular, very specific issue, less likely is to not erase as we go
 14 2017-02-25T00:06:19  <BlueMatt> ofc that means more memory usage, but...better than blowing up
 15 2017-02-25T00:08:15  <sipa> BlueMatt: alternatively, we could fail to flush, and still delete everything
 16 2017-02-25T00:08:18  <sipa> from the cache
 17 2017-02-25T00:08:36  <BlueMatt> sipa: hmm?
 18 2017-02-25T00:08:57  <sipa> ... and then reset pcoinsTip->hashBlock and chainActive to the last succesfully flushed block (which may be 1000s ago)
 19 2017-02-25T00:09:21  <BlueMatt> sipa: in that case we must AbortNode, I think?
 20 2017-02-25T00:09:28  <sipa> oh, absolutely
 21 2017-02-25T00:10:18  <sipa> but i'm worried we may not be able to to instantly abort (due to locks being held all over the place while the OOM occurs)
 22 2017-02-25T00:11:10  <BlueMatt> yea, I mean ideally we'd broaden the catch in FlushStateToDisk and use the AbortNode there
 23 2017-02-25T00:11:23  <BlueMatt> just this time dont let the coins map get into an inconsistent state
 24 2017-02-25T00:12:20  <sipa> cfields: looks good
 25 2017-02-25T00:13:17  <BlueMatt> cfields: oh, somehow I missed that that /is/ only for bad_alloc
 26 2017-02-25T00:13:32  <BlueMatt> yea, lets totally do that!
 27 2017-02-25T00:13:33  <cfields> <BlueMatt> wait, does this apply to more than bad_alloc?
 28 2017-02-25T00:13:33  <cfields> <cfields> no
 29 2017-02-25T00:13:35  <cfields> :)
 30 2017-02-25T00:13:44  <BlueMatt> oh, reading comprehension fail
 31 2017-02-25T00:18:08  <cfields> sipa: want to switch prevector to new[] and ditch realloc()? Or call get_new_handler()() if they return null?
 32 2017-02-25T00:18:23  <cfields> (or just leave it alone and let something write to 0 :p)
 33 2017-02-25T00:18:41  <sipa> cfields: just switch it to new[]
 34 2017-02-25T00:18:57  <cfields> ok
 35 2017-02-25T00:18:59  <BlueMatt> it would suck to drop realloc?
 36 2017-02-25T00:19:20  <BlueMatt> just assert(new_ptr)
 37 2017-02-25T00:19:39  <sipa> how often does realloc actually help?
 38 2017-02-25T00:20:02  <BlueMatt> no idea, but the difference between get_new_handler()() and assert(new_ptr) is minimal
 39 2017-02-25T00:20:51  <sipa> well i don't like making prevector assume that OOMs are fatal
 40 2017-02-25T00:20:59  <sipa> that's something application level
 41 2017-02-25T00:21:29  <BlueMatt> sipa: yea, and in our application they always are :p
 42 2017-02-25T00:22:46  <sipa> still, it's nicer to have that knowledge in only one place
 43 2017-02-25T00:26:14  <BlueMatt> sipa: internet says realloc is useful, though in our usage of prevector its likely not super noticeable
 44 2017-02-25T00:26:22  <BlueMatt> so I'm fine with whatever
 45 2017-02-25T00:30:42  <sipa> BlueMatt: oh, i just realized... calling get_new_handler()() in prevector doesn't imply it's going to fail
 46 2017-02-25T00:31:58  <BlueMatt> huh? you mean doesnt imply its going to std::terminate instead of throwing or so?
 47 2017-02-25T00:31:58  <sipa> yes
 48 2017-02-25T00:32:05  <sipa> so, i don't care.
 49 2017-02-25T00:32:05  <BlueMatt> ok, if you prefer that, do that
 50 2017-02-25T00:41:07  <bitcoin-git> [bitcoin] jtimon opened pull request #9855: RPC: Use integer satoshis instead BTC with decimals (master...0.15-rpc-satoshis) https://github.com/bitcoin/bitcoin/pull/9855
 51 2017-02-25T00:48:00  *** jl2012 has quit IRC
 52 2017-02-25T00:54:25  *** PaulCape_ has quit IRC
 53 2017-02-25T00:56:11  *** PaulCapestany has joined #bitcoin-core-dev
 54 2017-02-25T00:56:50  *** gk_-1wm_- has joined #bitcoin-core-dev
 55 2017-02-25T00:56:51  *** gk_-1wm_- has left #bitcoin-core-dev
 56 2017-02-25T01:00:51  *** echonaut has quit IRC
 57 2017-02-25T01:01:00  *** abpa has quit IRC
 58 2017-02-25T01:01:07  *** echonaut has joined #bitcoin-core-dev
 59 2017-02-25T01:17:28  *** bsm1175322 has quit IRC
 60 2017-02-25T01:18:02  *** CodeShark has quit IRC
 61 2017-02-25T01:18:17  *** CodeShark has joined #bitcoin-core-dev
 62 2017-02-25T01:25:11  *** Chris_Stewart_5 has quit IRC
 63 2017-02-25T01:30:09  *** dodomojo has joined #bitcoin-core-dev
 64 2017-02-25T01:33:13  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 65 2017-02-25T01:35:02  *** arowser has quit IRC
 66 2017-02-25T01:39:34  *** arowser has joined #bitcoin-core-dev
 67 2017-02-25T01:56:55  *** arowser has quit IRC
 68 2017-02-25T01:58:40  *** arowser has joined #bitcoin-core-dev
 69 2017-02-25T02:11:33  *** jtimon has quit IRC
 70 2017-02-25T02:20:02  *** gk_-1wm has joined #bitcoin-core-dev
 71 2017-02-25T02:20:03  *** gk_-1wm has left #bitcoin-core-dev
 72 2017-02-25T02:34:54  *** droark has joined #bitcoin-core-dev
 73 2017-02-25T02:36:43  *** droark has quit IRC
 74 2017-02-25T02:39:22  *** alpalp has quit IRC
 75 2017-02-25T02:43:25  *** Giszmo has quit IRC
 76 2017-02-25T02:44:53  *** jl2012 has joined #bitcoin-core-dev
 77 2017-02-25T02:57:55  *** wasi has quit IRC
 78 2017-02-25T02:58:32  *** wasi has joined #bitcoin-core-dev
 79 2017-02-25T03:02:09  *** AaronvanW has quit IRC
 80 2017-02-25T03:02:43  *** AaronvanW has joined #bitcoin-core-dev
 81 2017-02-25T03:08:23  *** alpalp has joined #bitcoin-core-dev
 82 2017-02-25T03:08:23  *** alpalp has joined #bitcoin-core-dev
 83 2017-02-25T03:35:24  <sipa> cfields, BlueMatt: crap, it seems i reported this exact same bug a year ago already...
 84 2017-02-25T03:59:44  *** droark has joined #bitcoin-core-dev
 85 2017-02-25T04:01:58  *** dodomojo has quit IRC
 86 2017-02-25T04:06:44  <cfields> sipa: heh. Well, at least we can get it in for 0.14
 87 2017-02-25T04:06:59  <cfields> sipa: does that mean it takes 1+ years to sync your rpi? :)
 88 2017-02-25T04:10:48  *** wudayoda has quit IRC
 89 2017-02-25T04:12:03  *** wudayoda has joined #bitcoin-core-dev
 90 2017-02-25T04:14:01  *** Ylbam has quit IRC
 91 2017-02-25T04:18:34  <sipa> cfields: it's a new one
 92 2017-02-25T04:18:35  *** alpalp has quit IRC
 93 2017-02-25T04:18:48  <sipa> it took 2 days to sync to the point where it was
 94 2017-02-25T04:18:54  *** alpalp has joined #bitcoin-core-dev
 95 2017-02-25T04:18:54  *** alpalp has joined #bitcoin-core-dev
 96 2017-02-25T04:29:22  *** alpalp has quit IRC
 97 2017-02-25T04:31:12  *** dodomojo has joined #bitcoin-core-dev
 98 2017-02-25T04:43:54  *** dodomojo has quit IRC
 99 2017-02-25T04:49:03  *** dodomojo has joined #bitcoin-core-dev
100 2017-02-25T04:49:56  *** dodomojo has quit IRC
101 2017-02-25T05:06:28  <cfields> sipa: i have no clue how representative the CCheckQueueSpeedPrevectorJob bench is, but there's a noticible difference when switching to new[] and dropping realloc
102 2017-02-25T05:07:37  <cfields> sipa: for the sake of being conservative this late in the release cycle, i think get_new_handler()() is likely to have much less impact
103 2017-02-25T05:10:31  *** mr_burdell has joined #bitcoin-core-dev
104 2017-02-25T05:13:00  *** BashCo_ has joined #bitcoin-core-dev
105 2017-02-25T05:14:08  *** roasbeef_ has joined #bitcoin-core-dev
106 2017-02-25T05:14:12  *** davec__ has joined #bitcoin-core-dev
107 2017-02-25T05:14:17  *** gmaxwell_ has joined #bitcoin-core-dev
108 2017-02-25T05:14:32  *** AdrianG_ has joined #bitcoin-core-dev
109 2017-02-25T05:14:47  *** wump has joined #bitcoin-core-dev
110 2017-02-25T05:16:42  *** davec has quit IRC
111 2017-02-25T05:16:43  *** Chris_Stewart_5 has quit IRC
112 2017-02-25T05:16:44  *** jamoes has quit IRC
113 2017-02-25T05:16:44  *** jcorgan has quit IRC
114 2017-02-25T05:16:45  *** Aleph0 has quit IRC
115 2017-02-25T05:16:45  *** haakonn has quit IRC
116 2017-02-25T05:16:48  *** murchandamus has quit IRC
117 2017-02-25T05:16:48  *** adiabat has quit IRC
118 2017-02-25T05:16:48  *** wumpus has quit IRC
119 2017-02-25T05:16:49  *** goregrin1 has quit IRC
120 2017-02-25T05:16:49  *** Guest74039 has quit IRC
121 2017-02-25T05:16:50  *** achow101 has quit IRC
122 2017-02-25T05:16:53  *** _mn3monic has quit IRC
123 2017-02-25T05:16:54  *** roasbeef has quit IRC
124 2017-02-25T05:16:54  *** midnightmagic has quit IRC
125 2017-02-25T05:16:54  *** gmaxwell has quit IRC
126 2017-02-25T05:16:54  *** so has quit IRC
127 2017-02-25T05:16:55  *** BashCo has quit IRC
128 2017-02-25T05:16:55  *** justanotheruser has quit IRC
129 2017-02-25T05:16:55  *** waxwing has quit IRC
130 2017-02-25T05:16:56  *** gluytium_ has quit IRC
131 2017-02-25T05:16:57  *** Squidicc has quit IRC
132 2017-02-25T05:16:58  *** warren has quit IRC
133 2017-02-25T05:16:59  *** Squidicuz has joined #bitcoin-core-dev
134 2017-02-25T05:17:00  *** so_ has joined #bitcoin-core-dev
135 2017-02-25T05:17:00  *** Chris_Stewart_5 has joined #bitcoin-core-dev
136 2017-02-25T05:17:07  *** warren has joined #bitcoin-core-dev
137 2017-02-25T05:17:10  *** waxwing has joined #bitcoin-core-dev
138 2017-02-25T05:17:21  *** achow101 has joined #bitcoin-core-dev
139 2017-02-25T05:17:29  *** goregrind has joined #bitcoin-core-dev
140 2017-02-25T05:17:32  *** gmaxwell_ has quit IRC
141 2017-02-25T05:17:33  *** gmaxwell_ has joined #bitcoin-core-dev
142 2017-02-25T05:17:36  *** murchandamus has joined #bitcoin-core-dev
143 2017-02-25T05:17:44  *** gmaxwell_ is now known as gmaxwell
144 2017-02-25T05:18:01  *** adiabat has joined #bitcoin-core-dev
145 2017-02-25T05:18:21  *** jcorgan has joined #bitcoin-core-dev
146 2017-02-25T05:18:57  *** midnightmagic has joined #bitcoin-core-dev
147 2017-02-25T05:19:01  *** jamoes has joined #bitcoin-core-dev
148 2017-02-25T05:19:11  *** gluytium has joined #bitcoin-core-dev
149 2017-02-25T05:19:53  *** justanotheruser has joined #bitcoin-core-dev
150 2017-02-25T05:20:23  *** michagogo has quit IRC
151 2017-02-25T05:21:17  *** haakonn has joined #bitcoin-core-dev
152 2017-02-25T05:21:41  *** haakonn is now known as Guest48161
153 2017-02-25T05:24:44  *** michagogo has joined #bitcoin-core-dev
154 2017-02-25T05:29:19  <sipa> cfields: ack
155 2017-02-25T05:30:30  <bitcoin-git> [bitcoin] theuni opened pull request #9856: Terminate immediately when allocation fails (master...bad_alloc_terminate2) https://github.com/bitcoin/bitcoin/pull/9856
156 2017-02-25T05:32:36  <cfields> damn fanquake, you're fast!
157 2017-02-25T05:35:04  *** fanquake has joined #bitcoin-core-dev
158 2017-02-25T05:40:13  *** fanquake1 has joined #bitcoin-core-dev
159 2017-02-25T05:41:15  *** fanquake has quit IRC
160 2017-02-25T05:41:15  *** Guest48161 has quit IRC
161 2017-02-25T05:41:16  *** so_ has quit IRC
162 2017-02-25T05:42:30  *** Chris_Stewart_5 has quit IRC
163 2017-02-25T05:42:31  *** Squidicuz has quit IRC
164 2017-02-25T05:42:31  *** Squidicuz has joined #bitcoin-core-dev
165 2017-02-25T05:42:34  *** so has joined #bitcoin-core-dev
166 2017-02-25T05:43:12  *** Chris_Stewart_5 has joined #bitcoin-core-dev
167 2017-02-25T05:46:40  *** haakonn_ has joined #bitcoin-core-dev
168 2017-02-25T05:59:35  <cfields> sipa: well i guess that discussion has been rendered moot. assert for 0.14, and do an efficient specialized unsigned char* prevector for master?
169 2017-02-25T05:59:49  <sipa> cfields: ok
170 2017-02-25T06:20:04  *** chjj has joined #bitcoin-core-dev
171 2017-02-25T06:54:11  *** roasbeef_ is now known as roasbeef
172 2017-02-25T07:00:04  *** dermoth has quit IRC
173 2017-02-25T07:01:01  *** dermoth has joined #bitcoin-core-dev
174 2017-02-25T07:09:06  *** _mn3monic has joined #bitcoin-core-dev
175 2017-02-25T07:12:07  *** lclc has joined #bitcoin-core-dev
176 2017-02-25T07:15:22  *** nickler has quit IRC
177 2017-02-25T07:17:10  *** lclc has quit IRC
178 2017-02-25T07:20:15  *** nickler has joined #bitcoin-core-dev
179 2017-02-25T07:36:50  *** lclc has joined #bitcoin-core-dev
180 2017-02-25T07:45:15  *** lclc has quit IRC
181 2017-02-25T07:46:07  <wump> bah, #9856 is really ugly :/
182 2017-02-25T07:46:09  <gribble> https://github.com/bitcoin/bitcoin/issues/9856 | Terminate immediately when allocation fails by theuni · Pull Request #9856 · bitcoin/bitcoin · GitHub
183 2017-02-25T07:46:15  *** wump is now known as wumpus
184 2017-02-25T07:47:19  *** lclc has joined #bitcoin-core-dev
185 2017-02-25T07:48:35  *** fanquake1 has quit IRC
186 2017-02-25T07:52:00  <wumpus> I understand wanting to terminate immediately on a memory error in some cases, but can't you just catch the exception at some point instead of registereing a global handler?
187 2017-02-25T07:52:43  <sipa> wumpus: the problem is exactly that it is being caught... and then we continue in an inconsistent state
188 2017-02-25T07:52:59  <wumpus> well then crash the program at the catch site!
189 2017-02-25T07:53:12  <sipa> that's what this does
190 2017-02-25T07:53:34  <sipa> tell the program to crash whenever new fails
191 2017-02-25T07:53:39  <wumpus> no, this uses some poorly documented feature of c++ to change a global callback for every failed allocation
192 2017-02-25T07:53:48  <wumpus> instead of just in the consensus code or wherever it is actually dangerous
193 2017-02-25T07:55:06  <wumpus> some allocation errors, either in bitcoin itself or in libraries may actually be recoverable, and they would now cause a crash
194 2017-02-25T07:56:13  <sipa> oh, sure, longer term i think we can find something more specific
195 2017-02-25T07:56:39  <sipa> but as long as we have "catch std::exception& e" everywhere, which catches everything, that's pretty hard
196 2017-02-25T07:56:42  *** lclc has quit IRC
197 2017-02-25T07:56:54  <wumpus> I think this is a bad direction to go in with the code base
198 2017-02-25T07:56:59  <wumpus> shoudln't we be addressing that then?
199 2017-02-25T07:57:14  <sipa> in 0.14?
200 2017-02-25T07:57:28  *** nohup has joined #bitcoin-core-dev
201 2017-02-25T07:57:40  <wumpus> for 0.14 this is ok, but this is a PR to master
202 2017-02-25T07:58:17  <sipa> hmm, you think this should go in just 0.14?
203 2017-02-25T07:58:28  *** lclc has joined #bitcoin-core-dev
204 2017-02-25T07:58:42  <wumpus> the way the PR is now? yes, I think so. This just too aspecific an hack
205 2017-02-25T07:58:47  <sipa> i think terminating when OOM is a very reasonable thing, and certainly more reasonable than trying to continuing
206 2017-02-25T07:59:01  <wumpus> I think terminating on OOM in select cases is a reasonable thing
207 2017-02-25T08:00:07  <sipa> there may be more user-friendly things to do in some cases, but it's very hard... allocation failures can occur pretty much everywhere, and it seems really hard to avoid inconsistent states
208 2017-02-25T08:00:36  <wumpus> sure, it's difficult
209 2017-02-25T08:00:45  <wumpus> but this just feels like giving up on any kind of sane error handling
210 2017-02-25T08:01:20  <sipa> i don't think so... OOM is different from disk errors, for example
211 2017-02-25T08:01:29  <sipa> you can cleanly recover from that, usually
212 2017-02-25T08:01:33  <wumpus> I really dislike global handlers like this, and I don't think much software uses this
213 2017-02-25T08:01:52  <wumpus> it just changes the expectation of the c++ standard that allocations return an exception
214 2017-02-25T08:02:17  <wumpus> which is adding surprising behavior at a low level
215 2017-02-25T08:04:27  <sipa> i'm surprised to read that
216 2017-02-25T08:04:46  <gmaxwell> on many systems allocation failures are just silently ignored and the program just crashes when it tries to use the memory.
217 2017-02-25T08:04:52  *** zrts_ has joined #bitcoin-core-dev
218 2017-02-25T08:04:59  <sipa> i think that most software just doesn't try to catch bad_alloc
219 2017-02-25T08:05:13  <sipa> and it just crashes
220 2017-02-25T08:05:18  <gmaxwell> In virtually none of the software could an allocation even possibly be handled, esp as almost everything we do involves allocation.
221 2017-02-25T08:05:44  <wumpus> ok ,never mind
222 2017-02-25T08:06:00  <wumpus> seems you all agree, just do this
223 2017-02-25T08:06:35  <gmaxwell> but I agree that just fixing the badalloc is not a full solution. What about other exceptions? If bad alloc can cause spurrious block invalidations how long until the next thing.
224 2017-02-25T08:07:17  <sipa> well, i think we should get rid of the catch std::exception and catch ... everywhere
225 2017-02-25T08:07:20  <wumpus> yes if the code is not exception-safe, that should be addressed
226 2017-02-25T08:07:28  <sipa> and only catch expected exceptions
227 2017-02-25T08:07:32  <gmaxwell> So I am of the opinion that bad alloc should just crash but also that the software needs to deal better with unexpected exceptions in block validation... as I mentioned before this is the third case of exceptions causing bogus block rejection that we've had.
228 2017-02-25T08:07:56  <sipa> but by doing that, we'd still just crash in case of an OOM
229 2017-02-25T08:08:06  <sipa> at least with this PR, we write to debug.log what happened
230 2017-02-25T08:08:26  <wumpus> but unlike AbortNode() no attempt is made to show a dialog to the user
231 2017-02-25T08:08:47  <gmaxwell> can't show a dialog without allocating.
232 2017-02-25T08:09:06  <sipa> well in many cases, that would be possible
233 2017-02-25T08:09:06  <wumpus> well the point is: maybe it needs less allocation than the original allocation request
234 2017-02-25T08:09:10  <wumpus> not all allocations are equal
235 2017-02-25T08:09:13  <sipa> indeed
236 2017-02-25T08:09:22  <sipa> typically those that trip an OOM are large ones
237 2017-02-25T08:09:32  <wumpus> someone might be trying to allocate a 4GB buffer, which fails, then after that the program could continue keeping in mind that it can't have that buffer
238 2017-02-25T08:09:37  <gmaxwell> That is a fair point. Also if it did something like shut down the mempool it might free enough to display a message.
239 2017-02-25T08:10:14  <gmaxwell> I wonder what the distribution of vm_overcommit settings is, it used to be ubiquitious, the fact that sipa even saw this suggests that it isn't anymore.
240 2017-02-25T08:11:04  <sipa> for this specific issue, it may be possible to catch the bad_alloc inside the leveldb wrapper, and if so, mark the db object as read-only
241 2017-02-25T08:11:11  <gmaxwell> apparently some hurestic is now the default.
242 2017-02-25T08:11:16  <wumpus> in many cases showing a simple modal dialog woudl still work, especially on windows where it's sort of a direct sytem operation. It should at least we attempted
243 2017-02-25T08:11:24  <sipa> that's fair
244 2017-02-25T08:11:39  <sipa> but i think it requires too many changes for 0.14
245 2017-02-25T08:11:42  <gmaxwell> I have no issue with that, so long as if it fails the system still shuts down.
246 2017-02-25T08:11:42  <wumpus> and we have a general 'A fatal issuee has occured check debug.log' already in AbortNode
247 2017-02-25T08:12:16  <sipa> i really want to think harder about possible inconsistent states
248 2017-02-25T08:12:27  <wumpus> well an allocation failure in qt will just propagate the std::alloc_error
249 2017-02-25T08:12:31  <wumpus> qt is exception safe
250 2017-02-25T08:12:41  <gmaxwell> sipa: I'm kinda surprised that cfield's patch works.  IIRC from gdbing boost asio it used to try to allocate 4TB of memory and the fail. I'm surprised nothing else in boost does.
251 2017-02-25T08:12:57  <wumpus> so just catch that and abort() if there is one
252 2017-02-25T08:13:03  <sipa> gmaxwell: it's c++11
253 2017-02-25T08:13:07  <sipa> boost does not assume c++11
254 2017-02-25T08:13:29  <sipa> for example, do we want to prevent writing to mempool.dat after some unhandled system error, to prevent corrupting it?
255 2017-02-25T08:14:02  <wumpus> yes
256 2017-02-25T08:14:16  <sipa> if catching of the exception occurs higher up, there is less risk of those things
257 2017-02-25T08:14:19  *** chjj has quit IRC
258 2017-02-25T08:14:26  <sipa> but what about writes that happen in unrelated background threads
259 2017-02-25T08:14:34  <sipa> the wallet gets flushed in a background thread
260 2017-02-25T08:14:44  <wumpus> set a flag and don't flush anything anymore
261 2017-02-25T08:14:56  <wumpus> I mean all the databases are atomic right?
262 2017-02-25T08:15:09  <wumpus> you can call writes, and as long as you don't flush, they'll be ignored on shutdown
263 2017-02-25T08:15:28  <sipa> but a flag where? an exception can only propagate up to its threads' main function
264 2017-02-25T08:15:36  <wumpus> a global flag
265 2017-02-25T08:15:41  <wumpus> checked before each database flag
266 2017-02-25T08:15:43  <wumpus> flush*
267 2017-02-25T08:16:15  <sipa> if an OOM occurs inside the background runner thread, it will - if uncaught - just kill the background runner
268 2017-02-25T08:16:18  <wumpus> a kind of alarm state, which makes sure nothing is permanently committed
269 2017-02-25T08:16:36  <wumpus> catch the exception and call a handler to set that flag?
270 2017-02-25T08:16:47  <sipa> that's really ugly...
271 2017-02-25T08:17:34  <wumpus> why is that ugly? you should catch exceptions such as std::alloc anyway to warn the user, log something , and initiaite shutdown
272 2017-02-25T08:17:38  <sipa> doesn't leveldb have some background writer thread?
273 2017-02-25T08:17:58  <wumpus> why is leveldb's background thread a problem here?
274 2017-02-25T08:18:06  <wumpus> we're concerned with corruptions of our own state, right?
275 2017-02-25T08:18:22  <wumpus> leveldb won't be writing anything that we haven't committed
276 2017-02-25T08:18:48  <wumpus> so as soon as there is the  suspicion of inconsistent state, don't write anything to leveldb anymore
277 2017-02-25T08:20:29  <sipa> i'm worried about an OOM caused by a thread we don't control
278 2017-02-25T08:20:44  <wumpus> then the inconsistent state won't be committed to disk, and leveldb should be fine. Same for the wallet
279 2017-02-25T08:20:45  <sipa> but i assume that would just cause a crash, as we can't catch a bad_alloc there anyway
280 2017-02-25T08:21:26  <wumpus> well yes the writer thread could die in the middle of writing, sure, but there's no way to avoid that. The database needs t obe able to handle that, also for other crash robustness
281 2017-02-25T08:22:19  <wumpus> what it won't do, however, is write inconsistent state if we haven't told it to
282 2017-02-25T08:22:29  <sipa> yes, indeed
283 2017-02-25T08:23:34  <sipa> i think you're right - with some changes it may be possible to do this safely
284 2017-02-25T08:23:53  <sipa> if we make sure there is no code that may unintentionally catch a bad_alloc
285 2017-02-25T08:24:02  <wumpus> right
286 2017-02-25T08:24:12  <sipa> and that all the catches that do, also call AbortNode, which sets this flag
287 2017-02-25T08:24:24  <wumpus> I think there should be a rule: if you catch a general std::exception, you can't continue there, and need to stop the program
288 2017-02-25T08:24:47  <wumpus> as you have no idea what kind of error you have and whether it's recoverable
289 2017-02-25T08:24:51  <sipa> right
290 2017-02-25T08:24:58  <wumpus> (so that's for top-level handlers only)
291 2017-02-25T08:25:52  <sipa> we have 53 places where std::exception is caught
292 2017-02-25T08:26:06  <sipa> and 22 where ... is caught (even worse, as you can't even inspect the object)
293 2017-02-25T08:26:13  <wumpus> the ptential worry is in the networking code... some parsing problems throw exceptions, we need to be really sure to catch them and continue otherwise there's a DoS
294 2017-02-25T08:26:56  <wumpus> e.g. serialization throws std::runtime_error instead of something specific
295 2017-02-25T08:27:29  <sipa> serialization throws std::ios_base
296 2017-02-25T08:27:36  <sipa> serialization throws std::ios_base::failure
297 2017-02-25T08:27:59  <wumpus> okay That's always an i/o error. Better than I thought
298 2017-02-25T08:28:12  <sipa> there are some other things that can throw that
299 2017-02-25T08:28:31  <wumpus> catching that and just reporting it and continuing in the network code would be ok
300 2017-02-25T08:28:52  <sipa> network code or net_handling?
301 2017-02-25T08:29:06  *** panicstr has joined #bitcoin-core-dev
302 2017-02-25T08:29:18  <sipa> net handling should catch serialization failures, and disconnect the offending peer
303 2017-02-25T08:29:29  <wumpus> well the packet processing. It should make sure that one node is disconnected and not thewhole program comes tumbling down
304 2017-02-25T08:29:35  <wumpus> yes
305 2017-02-25T08:29:43  <sipa> right
306 2017-02-25T08:29:53  <sipa> in fact, it's that except catch this is the culprit here
307 2017-02-25T08:30:02  <sipa> *that exact
308 2017-02-25T08:30:24  <wumpus> right, it should be more specific
309 2017-02-25T08:30:42  <wumpus> catching ios::base and potentially a few others instead of std::exception would go a long way
310 2017-02-25T08:30:48  <sipa> agree
311 2017-02-25T08:33:22  <wumpus> and some exceptions like std::bad_alloc should initiate a quick shutdown. It could still be a DoS, but the program is probably in such a bad state when it happens that continuing with the other nodes is hopeless.
312 2017-02-25T08:33:53  <wumpus> (though theoretically it could stil be a botched alloc due to the remote peer specifying a very large buffer which would never occur in reality)
313 2017-02-25T08:36:44  <wumpus> so I guess uploading executables for rc2 is no longer necessary and we should merge #9856 and do rc3?
314 2017-02-25T08:36:45  <gribble> https://github.com/bitcoin/bitcoin/issues/9856 | Terminate immediately when allocation fails by theuni · Pull Request #9856 · bitcoin/bitcoin · GitHub
315 2017-02-25T08:42:58  <wumpus> speaking about memory usage: another report of running out of memory with 0.14. #9857
316 2017-02-25T08:42:59  <gribble> https://github.com/bitcoin/bitcoin/issues/9857 | Sync problem crashes Bitcoin: EXCEPTION: St9bad_alloc · Issue #9857 · bitcoin/bitcoin · GitHub
317 2017-02-25T08:43:09  <wumpus> maybe 0.14 increased steady-state memory usage?
318 2017-02-25T08:49:22  <sipa> maybe
319 2017-02-25T08:49:37  <sipa> the reuse of mempool memory by coincache may contribute to it
320 2017-02-25T08:49:59  <sipa> though that doesn't affect worst-case memory usage, it may increase the average
321 2017-02-25T08:50:02  *** lclc has quit IRC
322 2017-02-25T08:50:55  <sipa> for RPi we should recommend lowering the default settings, i'm afraid
323 2017-02-25T08:51:58  *** lclc has joined #bitcoin-core-dev
324 2017-02-25T08:56:43  <wumpus> bah, what really helps when creating a gdb parachute is propagating the exit code, if not it will just always return success https://github.com/bitcoin/bitcoin/pull/9851/commits/6d7a789fb510d540906955c377143b92acb1fbb6 . Forgot this and might have been spinning for nothing for a while :p
325 2017-02-25T09:05:32  *** lclc has quit IRC
326 2017-02-25T09:07:02  <wumpus> uhm. Caught that issue in the act now. Unfortunately, the traceback is useless. Gdb near-crashes while generating it!
327 2017-02-25T09:07:10  *** lclc has joined #bitcoin-core-dev
328 2017-02-25T09:07:13  *** nabu has joined #bitcoin-core-dev
329 2017-02-25T09:07:27  <wumpus> "Error while running hook_stop: Cannot access memory at address 0x24"
330 2017-02-25T09:08:01  <wumpus> okay, so much for gdb...
331 2017-02-25T09:11:07  <wumpus> that leaves trying to make it cough up a core dump, and uploading that somewhere...
332 2017-02-25T09:12:22  <jeremyrubin> wumpus: cuckoocache in theory may have increase memory usage a bit
333 2017-02-25T09:13:06  <jeremyrubin> (if you're curious for all steadystate memory usage increases)
334 2017-02-25T09:13:29  <wumpus> by how much?
335 2017-02-25T09:13:35  <jeremyrubin> unclear
336 2017-02-25T09:14:00  <jeremyrubin> it always uses 100% of whatever's it's allocated
337 2017-02-25T09:14:25  <jeremyrubin> whereas the previous one only uses space when it has entries (in theory)
338 2017-02-25T09:14:42  <sipa> the default max is 32 MB instead of 40 MB though
339 2017-02-25T09:14:47  <sipa> so at peak, it's smaller
340 2017-02-25T09:14:48  <jeremyrubin> so at startup, would be using 32 MB whereas the prior was 0
341 2017-02-25T09:15:53  <jeremyrubin> it's just unclear how long you have to be on the network to experience a full cache w/ old code
342 2017-02-25T09:16:32  <jeremyrubin> anyways I'm pretty sure this isn't the cause :)
343 2017-02-25T09:17:23  <sipa> 1GB of memory is not much
344 2017-02-25T09:18:10  <wumpus> indeed, runninga node with 1GB of memory has always required tweaking
345 2017-02-25T09:18:26  <sipa> it's more surprising to me that 0.13.2 worked, though
346 2017-02-25T09:18:33  <sipa> he's even reducing dbcache
347 2017-02-25T09:18:47  *** lclc has quit IRC
348 2017-02-25T09:19:04  <wumpus> I do see increased reporting of out-of-memory errors with 0.14
349 2017-02-25T09:19:41  <jeremyrubin> yeah, totally could be the sigcache, could definitely put someone over-edge
350 2017-02-25T09:19:49  <jeremyrubin> especially if they set the value higher
351 2017-02-25T09:19:57  <jeremyrubin> e.g., 128 mb
352 2017-02-25T09:20:14  <jeremyrubin> prior version you could set it to a GB and run happily for a while
353 2017-02-25T09:20:28  <wumpus> they don't touch that setting, just dbcache
354 2017-02-25T09:20:43  <wumpus> possibly they should. We should update the recommedations for low-memory nodes.
355 2017-02-25T09:21:28  <jeremyrubin> I don't think practically they'd need to adjust it down, just be less agressive adjusting it up -- not much benefit in making it too large
356 2017-02-25T09:22:32  <jeremyrubin> Can we (by any chance) fast track sipa's hack to get cuckoocache back to be non pow 2?
357 2017-02-25T09:23:14  <jeremyrubin> it sucks if they're picking between 8 16 32 MB sizes, some more resolution there would be good
358 2017-02-25T09:23:40  <wumpus> which # is that?
359 2017-02-25T09:24:09  <jeremyrubin> #9533
360 2017-02-25T09:24:11  <gribble> https://github.com/bitcoin/bitcoin/issues/9533 | Allow non-power-of-2 signature cache sizes by sipa · Pull Request #9533 · bitcoin/bitcoin · GitHub
361 2017-02-25T09:26:05  <wumpus> jeremyrubin: would help to have more than a concept ACK there from you
362 2017-02-25T09:26:43  <jeremyrubin> k I'll do some more thorough review
363 2017-02-25T09:26:47  <wumpus> thanks
364 2017-02-25T09:28:59  <wumpus> also like gmaxwell I wonder what the performance difference would be, 8 multiplication would be slower than 8 logical AND
365 2017-02-25T09:31:37  <jeremyrubin> I'm not currently set up to run great performance tests currently (will take some work to reset up my benching environment)
366 2017-02-25T09:31:57  <jeremyrubin> But honestly performance was fine even when it was %
367 2017-02-25T09:32:11  <wumpus> 'fine' hehe
368 2017-02-25T09:32:17  <jeremyrubin> *shrug*
369 2017-02-25T09:32:20  <sipa> jeremyrubin: i'm sure that for the sigcache it doesn't matter
370 2017-02-25T09:32:33  <sipa> but maybe it does for other things in the future
371 2017-02-25T09:32:36  <wumpus> well it's obviously a compromise between memory usage and performance
372 2017-02-25T09:32:47  <jeremyrubin> I mostly changed it to hash_mask to make sipa happy :p
373 2017-02-25T09:33:02  <wumpus> neither '&' or '* >>' is 'better', but we need to be careful what to choose
374 2017-02-25T09:33:15  <sipa> well you violated the core tenet of never using integer division by a variable!
375 2017-02-25T09:33:23  <sipa> it invokes the wrath of maxwell
376 2017-02-25T09:33:31  <jeremyrubin> It was checked to not be 0 ;)
377 2017-02-25T09:33:58  <jeremyrubin> anyways...
378 2017-02-25T09:34:04  <wumpus> division? where?
379 2017-02-25T09:34:18  <jeremyrubin> (old version of cuckoocache used % in the hash computation)
380 2017-02-25T09:34:33  <wumpus> I saw only multiplies, if there's a division, I'm going to NACK this one
381 2017-02-25T09:34:33  <wumpus> ooh :p
382 2017-02-25T09:35:08  <sipa> and that PR brings back the flexibility of variable sizes, but with a * and a >> instead of a %
383 2017-02-25T09:35:22  <jeremyrubin> I recall benchmarking and not being able to see a difference between %, &, >>
384 2017-02-25T09:35:39  <wumpus> yes using % would be more straightforward code readability wise but division is even worse than multiplication
385 2017-02-25T09:35:44  <jeremyrubin> yeah
386 2017-02-25T09:35:54  <jeremyrubin> so I think it's fairly safe to take this one.
387 2017-02-25T09:36:00  <wumpus> yes
388 2017-02-25T09:36:07  <jeremyrubin> My one suggestion sipa is to convert size to a uint64_t
389 2017-02-25T09:36:11  <jeremyrubin> like the member
390 2017-02-25T09:36:20  <jeremyrubin> so you can get rid of all the casts in the hash part
391 2017-02-25T09:36:34  <wumpus> would this result in more efficient code on 32-bit though?
392 2017-02-25T09:36:41  <wumpus> as it guarantees that the top bits are zero
393 2017-02-25T09:36:50  <jeremyrubin> I'm not sure
394 2017-02-25T09:37:01  <jeremyrubin> but they have to be cast to uint64_t in hashing part now
395 2017-02-25T09:37:17  <jeremyrubin> so my thought is not having to do that cast every hash is slightly more efficient code
396 2017-02-25T09:37:34  <wumpus> casts don't generally generate code
397 2017-02-25T09:37:34  *** panicstr has quit IRC
398 2017-02-25T09:37:48  <jeremyrubin> uint32_t to uint64_t even?
399 2017-02-25T09:37:59  <jeremyrubin> there are instructions that auto-0 it to the alu?
400 2017-02-25T09:38:02  <jeremyrubin> that's fine then
401 2017-02-25T09:38:14  <wumpus> no. on 32-bit, it just means 'assume top bits are 0'. On 64 bit evrything is stored in 64-bit registers and the top bits are always 0
402 2017-02-25T09:38:37  <jeremyrubin> cool, then that's fine as is
403 2017-02-25T09:38:43  <wumpus> at least on x86_64 you can't load something into the lower 32 bits without zeroing the upper bits
404 2017-02-25T09:39:11  <jeremyrubin> I'd consider it an Ack from me then.
405 2017-02-25T09:39:59  <jeremyrubin> The tests are pretty agressive on cuckoocache, so if the hashing idea were broken in some way it would either catch it or it would be broken in a very minor way.
406 2017-02-25T09:40:00  <wumpus> of course the considereations depend on the specific architecture, but in general casts are very cheap or generate no extra code at all especially between big types. Arithmetic with 8-bit and 16-bit values can result in extra 'ANDs' in some architectures you're right
407 2017-02-25T09:40:18  <wumpus> but then it's not so much the cast that generates the code
408 2017-02-25T09:40:53  <wumpus> yes, having good tests is great!
409 2017-02-25T09:41:40  <wumpus> the state of testing in bitcoin core improved a lot last year
410 2017-02-25T09:42:06  <jeremyrubin> *cough* merge my checkqueue tests *cough* :P
411 2017-02-25T09:42:21  <wumpus> which reminds me I should merge #9847
412 2017-02-25T09:42:23  <gribble> https://github.com/bitcoin/bitcoin/issues/9847 | Extra test vector for BIP32 by sipa · Pull Request #9847 · bitcoin/bitcoin · GitHub
413 2017-02-25T09:42:26  <wumpus> jeremyrubin: oh, aren't they yet?
414 2017-02-25T09:42:54  <jeremyrubin> #9497
415 2017-02-25T09:42:56  <gribble> https://github.com/bitcoin/bitcoin/issues/9497 | CCheckQueue Unit Tests by JeremyRubin · Pull Request #9497 · bitcoin/bitcoin · GitHub
416 2017-02-25T09:42:58  <wumpus> usually test-only pulls are merged very quickly
417 2017-02-25T09:43:02  *** d9b4bef9 has quit IRC
418 2017-02-25T09:43:25  <wumpus> then again I cannot really keep up anymore with the volume of pulls
419 2017-02-25T09:43:52  <jeremyrubin> It's actually fine that they weren't, I found a mixed up loop condition (such that one test wasn't doing much) when I was sitting with kalle
420 2017-02-25T09:43:55  <wumpus> which is good and bad
421 2017-02-25T09:43:59  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f19afdbfb4cb...6206252e5073
422 2017-02-25T09:44:00  <bitcoin-git> bitcoin/master 30aedcb Pieter Wuille: BIP32 extra test vector
423 2017-02-25T09:44:00  <bitcoin-git> bitcoin/master 6206252 Wladimir J. van der Laan: Merge #9847: Extra test vector for BIP32...
424 2017-02-25T09:44:01  <sipa> but division is even worse than multiplication -> nehalem can issue a 64-bit mul every 2 cycles; a division every 19-69 cycles
425 2017-02-25T09:44:08  *** d9b4bef9 has joined #bitcoin-core-dev
426 2017-02-25T09:44:14  <jeremyrubin> But that's all fixed now
427 2017-02-25T09:44:19  <bitcoin-git> [bitcoin] laanwj closed pull request #9847: Extra test vector for BIP32 (master...bip32up) https://github.com/bitcoin/bitcoin/pull/9847
428 2017-02-25T09:44:33  <wumpus> sipa: yeah, that would classify as 'bad' in my eyes :)
429 2017-02-25T09:44:37  <sipa> wumpus: the mul or the div?
430 2017-02-25T09:44:40  <wumpus> the div
431 2017-02-25T09:44:46  <jeremyrubin> nehalem?
432 2017-02-25T09:44:51  <sipa> yes, absolutely
433 2017-02-25T09:45:08  <sipa> but 'even worse' sounds like multiplication is already pretty bad... every 2 cycles is pretty good i'd say :)
434 2017-02-25T09:45:23  <wumpus> divs have historically been really bad, some CPUs don't even have a specific instruction for it, but even those that have it's generally terrible
435 2017-02-25T09:45:42  <wumpus> because of the underlying computational structure of course, not because those CPUs are badly implemented or so :)
436 2017-02-25T09:45:49  <jeremyrubin> ah, nehalem:  an intel arch name I'd thoroughly forgotten :)
437 2017-02-25T09:46:32  <wumpus> 2 cycles is pretty good, espeically if it's *every* 2 cycles
438 2017-02-25T09:46:47  <jeremyrubin> and that hash routine should pipeline well
439 2017-02-25T09:46:54  <sipa> the latency is higher than 2 cycles
440 2017-02-25T09:46:55  <wumpus> usually mul is quite quick but the number of multiplication units is limited, so issuing a lot will bottleneck on that
441 2017-02-25T09:47:01  <wumpus> but not on nehalem, apparently :)
442 2017-02-25T09:47:05  <sipa> but the throughput is once every 2 cycles
443 2017-02-25T09:47:19  <sipa> same on skylaky, btw
444 2017-02-25T09:47:22  <wumpus> what would be the throughput for logical and?
445 2017-02-25T09:47:53  <sipa> 0.5 or 1, depending on the type of inputs
446 2017-02-25T09:47:54  <wumpus> can't be much faster than per 2 cycles
447 2017-02-25T09:47:55  <wumpus> oh!
448 2017-02-25T09:48:05  <sipa> 0.25 for and with a constant, even
449 2017-02-25T09:48:21  <sipa> http://www.agner.org/optimize/instruction_tables.pdf
450 2017-02-25T09:48:22  <wumpus> modern CPU architectures are so fast we can almost ignore computation overhead. Too bad memory didn't keep up :)
451 2017-02-25T09:48:42  <jeremyrubin> Also if you want to really make that hashing faster, could switch to balke2b or something
452 2017-02-25T09:49:34  <wumpus> sipa: thanks , that seems like a very useful document
453 2017-02-25T09:50:09  <jeremyrubin> wumpus: +1 thanks!
454 2017-02-25T09:50:58  <sipa> LOCK ADD... 18 cycles!
455 2017-02-25T09:51:15  *** Ylbam has joined #bitcoin-core-dev
456 2017-02-25T09:53:25  <wumpus> ieeeh
457 2017-02-25T09:57:38  <jeremyrubin> so I'm trying to write some test software
458 2017-02-25T09:57:47  <jeremyrubin> and I want to force segwit active
459 2017-02-25T09:58:05  <jeremyrubin> seems that isn't trivial to do?
460 2017-02-25T09:58:10  <sipa> regtest?
461 2017-02-25T09:59:19  <jeremyrubin> it seems to still require some blocks to vote for it before it can be made active?
462 2017-02-25T09:59:36  <sipa> yes
463 2017-02-25T09:59:55  <sipa> 3*144 blocks
464 2017-02-25T10:00:47  <jeremyrubin> hmm... would be kinda nice to not have to do that, similar to how I can set BIPxxxHeight = 0.
465 2017-02-25T10:01:39  *** lclc has joined #bitcoin-core-dev
466 2017-02-25T10:06:35  *** MarcoFalke has joined #bitcoin-core-dev
467 2017-02-25T10:08:00  *** jl2012 has quit IRC
468 2017-02-25T10:14:13  <jeremyrubin> would it be safe to change VersionBitsState to read VersionBitsDeploymentInfo for an override state parameter?
469 2017-02-25T10:17:10  *** lclc has quit IRC
470 2017-02-25T10:19:02  <jeremyrubin> no I guess I'd neet to change GetStateFor
471 2017-02-25T10:19:06  *** lclc has joined #bitcoin-core-dev
472 2017-02-25T10:34:04  *** Squidicc has joined #bitcoin-core-dev
473 2017-02-25T10:34:55  *** lclc has quit IRC
474 2017-02-25T10:36:42  *** Squidicuz has quit IRC
475 2017-02-25T10:36:50  *** lclc has joined #bitcoin-core-dev
476 2017-02-25T10:39:19  *** nohup has quit IRC
477 2017-02-25T10:46:53  *** Victor_sueca is now known as Victorsueca
478 2017-02-25T10:50:53  *** panicstr has joined #bitcoin-core-dev
479 2017-02-25T11:01:22  *** lclc has quit IRC
480 2017-02-25T11:08:13  *** MarcoFalke has left #bitcoin-core-dev
481 2017-02-25T11:22:43  *** MarcoFalke has joined #bitcoin-core-dev
482 2017-02-25T11:35:20  *** panicstr has quit IRC
483 2017-02-25T11:56:11  *** lclc has joined #bitcoin-core-dev
484 2017-02-25T12:05:08  *** lclc has quit IRC
485 2017-02-25T12:06:57  *** lclc has joined #bitcoin-core-dev
486 2017-02-25T12:13:59  *** AaronvanW has quit IRC
487 2017-02-25T12:14:16  *** AaronvanW has joined #bitcoin-core-dev
488 2017-02-25T12:14:16  *** AaronvanW has joined #bitcoin-core-dev
489 2017-02-25T12:17:36  *** AaronvanW has quit IRC
490 2017-02-25T12:18:10  *** AaronvanW has joined #bitcoin-core-dev
491 2017-02-25T12:18:10  *** AaronvanW has joined #bitcoin-core-dev
492 2017-02-25T12:29:01  *** alpalp has joined #bitcoin-core-dev
493 2017-02-25T12:29:01  *** alpalp has joined #bitcoin-core-dev
494 2017-02-25T12:31:41  *** wudayoda has quit IRC
495 2017-02-25T12:33:51  *** wudayoda has joined #bitcoin-core-dev
496 2017-02-25T12:42:13  <btcdrak> jeremyrubin you could just turn down the number of blocks required on regtest from 144 to something lower like 10.
497 2017-02-25T12:51:52  *** lclc has quit IRC
498 2017-02-25T13:01:53  *** jl2012 has joined #bitcoin-core-dev
499 2017-02-25T13:28:16  *** AaronvanW has quit IRC
500 2017-02-25T13:28:17  *** Aaronvan_ has joined #bitcoin-core-dev
501 2017-02-25T13:54:09  *** vFSgrcFGBJHg has joined #bitcoin-core-dev
502 2017-02-25T13:55:15  *** lclc has joined #bitcoin-core-dev
503 2017-02-25T14:00:02  *** lclc has quit IRC
504 2017-02-25T14:02:25  *** lclc has joined #bitcoin-core-dev
505 2017-02-25T14:06:02  *** Aaronvan_ has quit IRC
506 2017-02-25T14:06:36  *** AaronvanW has joined #bitcoin-core-dev
507 2017-02-25T14:06:36  *** AaronvanW has joined #bitcoin-core-dev
508 2017-02-25T14:11:12  *** AaronvanW has quit IRC
509 2017-02-25T14:14:46  *** lclc has quit IRC
510 2017-02-25T14:16:02  *** alpalp has quit IRC
511 2017-02-25T14:16:59  *** lclc has joined #bitcoin-core-dev
512 2017-02-25T14:32:07  *** AaronvanW has joined #bitcoin-core-dev
513 2017-02-25T14:32:08  *** AaronvanW has joined #bitcoin-core-dev
514 2017-02-25T14:38:28  *** Guyver2 has joined #bitcoin-core-dev
515 2017-02-25T14:46:46  *** panicstr has joined #bitcoin-core-dev
516 2017-02-25T14:51:56  *** panicstr has quit IRC
517 2017-02-25T15:03:20  *** arowser has quit IRC
518 2017-02-25T15:04:56  *** arowser has joined #bitcoin-core-dev
519 2017-02-25T15:13:22  *** alpalp has joined #bitcoin-core-dev
520 2017-02-25T15:24:58  *** alpalp has quit IRC
521 2017-02-25T15:30:08  *** alpalp has joined #bitcoin-core-dev
522 2017-02-25T15:30:08  *** alpalp has joined #bitcoin-core-dev
523 2017-02-25T15:34:35  *** Giszmo has joined #bitcoin-core-dev
524 2017-02-25T15:42:40  *** MarcoFalke has quit IRC
525 2017-02-25T15:45:25  <cfields> woohoo, 0.13 ec2 sync finally finished :)
526 2017-02-25T15:51:16  *** AaronvanW has quit IRC
527 2017-02-25T15:58:45  <cfields> for those interested... ec2 xlarge 4core, 16gb ram. All default settings:
528 2017-02-25T15:59:02  <cfields> v0.13.2: 01:12:39:43
529 2017-02-25T15:59:02  <cfields> v0.14.0: 00:06:24:17
530 2017-02-25T16:01:09  <paveljanik> nice!
531 2017-02-25T16:03:53  <cfields> Setting a reasonable dbcache cuts the 0.14 time in half. Unsure what it does for 0.13. Similar, I assume
532 2017-02-25T16:05:02  <cfields> Actually no... 0.13 is doing lots more validation. So it'd still be much more than half.
533 2017-02-25T16:05:36  *** AaronvanW has joined #bitcoin-core-dev
534 2017-02-25T16:10:43  *** lclc has quit IRC
535 2017-02-25T16:18:05  *** str4d has joined #bitcoin-core-dev
536 2017-02-25T16:48:24  <wumpus> FYI rc2 executables are online: https://bitcoin.org/bin/bitcoin-core-0.14.0/test.rc2/
537 2017-02-25T17:06:36  *** AaronvanW has quit IRC
538 2017-02-25T17:06:47  <Lauda> sync is faster with 0.14.0?
539 2017-02-25T17:10:26  *** AaronvanW has joined #bitcoin-core-dev
540 2017-02-25T17:10:27  *** AaronvanW has joined #bitcoin-core-dev
541 2017-02-25T17:15:23  *** Squidicc is now known as squidicuz
542 2017-02-25T17:41:22  *** alpalp has quit IRC
543 2017-02-25T18:14:27  *** alpalp has joined #bitcoin-core-dev
544 2017-02-25T18:14:28  *** alpalp has joined #bitcoin-core-dev
545 2017-02-25T18:54:22  *** Chris_Stewart_5 has quit IRC
546 2017-02-25T18:57:22  *** CodeShark has quit IRC
547 2017-02-25T18:57:23  *** GreenIsMyPepper has quit IRC
548 2017-02-25T18:58:26  *** CodeShark has joined #bitcoin-core-dev
549 2017-02-25T18:59:56  *** GreenIsMyPepper has joined #bitcoin-core-dev
550 2017-02-25T19:11:03  *** Chris_Stewart_5 has joined #bitcoin-core-dev
551 2017-02-25T19:23:49  *** LeMiner has joined #bitcoin-core-dev
552 2017-02-25T19:31:23  *** wudayoda has quit IRC
553 2017-02-25T19:32:00  *** wudayoda has joined #bitcoin-core-dev
554 2017-02-25T19:57:47  <bitcoin-git> [bitcoin] jameshilliard opened pull request #9858: remove TestBlockValidity from CreateNewBlock critical path (master...remove-tbv) https://github.com/bitcoin/bitcoin/pull/9858
555 2017-02-25T20:16:45  *** laurentmt has joined #bitcoin-core-dev
556 2017-02-25T20:36:30  *** chjj has joined #bitcoin-core-dev
557 2017-02-25T20:39:30  *** chjj has quit IRC
558 2017-02-25T20:39:52  *** chjj has joined #bitcoin-core-dev
559 2017-02-25T20:54:18  *** crudel has quit IRC
560 2017-02-25T20:57:55  *** justan0theruser has joined #bitcoin-core-dev
561 2017-02-25T21:00:14  *** str4d has quit IRC
562 2017-02-25T21:01:02  *** justanotheruser has quit IRC
563 2017-02-25T21:01:53  *** lclc has joined #bitcoin-core-dev
564 2017-02-25T21:08:01  *** AaronvanW has quit IRC
565 2017-02-25T21:11:47  *** laurentmt has quit IRC
566 2017-02-25T21:12:04  *** AaronvanW has joined #bitcoin-core-dev
567 2017-02-25T21:17:42  *** lclc has quit IRC
568 2017-02-25T21:19:54  *** lclc has joined #bitcoin-core-dev
569 2017-02-25T21:22:50  *** crudel has joined #bitcoin-core-dev
570 2017-02-25T21:40:27  *** lclc has quit IRC
571 2017-02-25T21:42:43  *** AaronvanW has quit IRC
572 2017-02-25T21:44:57  *** owowo has quit IRC
573 2017-02-25T21:48:48  <cfields> luke-jr: jinx :)
574 2017-02-25T21:48:54  <luke-jr> ?
575 2017-02-25T21:49:08  <cfields> luke-jr: 9858
576 2017-02-25T21:54:11  *** owowo has joined #bitcoin-core-dev
577 2017-02-25T21:54:20  <luke-jr> i c ☺
578 2017-02-25T22:09:11  *** wumpus has quit IRC
579 2017-02-25T22:11:29  *** wumpus has joined #bitcoin-core-dev
580 2017-02-25T22:19:29  *** owowo has quit IRC
581 2017-02-25T22:24:45  <luke-jr> would be nice if everything that went into stable branches were daggy fixes.
582 2017-02-25T22:25:35  <sipa> daggy?
583 2017-02-25T22:27:33  <luke-jr> sipa: http://wiki.secondlife.com/wiki/Creating_a_version_control_repository#The_.22Daggy.22_Fix
584 2017-02-25T22:29:04  *** Guyver2 has quit IRC
585 2017-02-25T22:31:17  <luke-jr> basically means bugfix commits are parented to the bug-introducing commit, allowing a merge into each branch of the same fix commit
586 2017-02-25T22:35:35  *** JackH has joined #bitcoin-core-dev
587 2017-02-25T22:37:16  <gmaxwell> are really that many of our fixes ones where there is a obvious bug inducing commit which is recent enough for that to be useful?
588 2017-02-25T22:41:34  <luke-jr> gmaxwell: maybe not. as a middle-ground, I typically just try to rebase to the most recent branching point that is a clean merge to master
589 2017-02-25T22:42:01  <luke-jr> eg, everything I do now would be based on 9828f9a at the latest (if possible)
590 2017-02-25T22:42:24  <luke-jr> (which I have simple-tagged as "branch-0.14" in my repo)
591 2017-02-25T22:56:47  *** owowo has joined #bitcoin-core-dev
592 2017-02-25T22:56:47  *** owowo has joined #bitcoin-core-dev
593 2017-02-25T23:17:02  *** justanotheruser has joined #bitcoin-core-dev
594 2017-02-25T23:19:27  *** justan0theruser has quit IRC