1 2015-10-21T00:36:20  *** Thireus has quit IRC
  2 2015-10-21T01:14:19  *** Ylbam has quit IRC
  3 2015-10-21T02:53:57  *** CodeShark has joined #bitcoin-core-dev
  4 2015-10-21T02:54:54  *** CodeShark_ has joined #bitcoin-core-dev
  5 2015-10-21T02:54:58  *** berndj has quit IRC
  6 2015-10-21T02:55:10  *** berndj has joined #bitcoin-core-dev
  7 2015-10-21T02:55:31  *** berndj is now known as Guest60457
  8 2015-10-21T02:55:49  *** CodeShark has quit IRC
  9 2015-10-21T02:55:49  *** CodeShark_ has quit IRC
 10 2015-10-21T02:56:10  *** CodeShark has joined #bitcoin-core-dev
 11 2015-10-21T02:56:53  *** Guest60457 has quit IRC
 12 2015-10-21T02:57:11  *** berndj-blackout has joined #bitcoin-core-dev
 13 2015-10-21T02:57:43  *** berndj-blackout has joined #bitcoin-core-dev
 14 2015-10-21T02:58:41  *** berndj-blackout has joined #bitcoin-core-dev
 15 2015-10-21T03:00:37  *** berndj-blackout has quit IRC
 16 2015-10-21T03:01:11  *** berndj-blackout has joined #bitcoin-core-dev
 17 2015-10-21T03:02:58  *** berndj-blackout has quit IRC
 18 2015-10-21T03:03:11  *** berndj-blackout has joined #bitcoin-core-dev
 19 2015-10-21T03:07:54  *** berndj-blackout has quit IRC
 20 2015-10-21T03:08:12  *** berndj-blackout has joined #bitcoin-core-dev
 21 2015-10-21T03:09:09  *** berndj-blackout has quit IRC
 22 2015-10-21T03:29:33  *** CodeShark has quit IRC
 23 2015-10-21T03:29:52  *** CodeShark has joined #bitcoin-core-dev
 24 2015-10-21T04:31:57  *** CodeShark_ has joined #bitcoin-core-dev
 25 2015-10-21T04:36:24  *** CodeShark has quit IRC
 26 2015-10-21T05:01:24  *** CodeShark has joined #bitcoin-core-dev
 27 2015-10-21T05:10:53  *** Thireus has joined #bitcoin-core-dev
 28 2015-10-21T06:16:16  *** Ylbam has joined #bitcoin-core-dev
 29 2015-10-21T06:21:45  *** Thireus has quit IRC
 30 2015-10-21T06:42:05  *** midnightmagic has quit IRC
 31 2015-10-21T06:47:03  *** midnightmagic has joined #bitcoin-core-dev
 32 2015-10-21T06:50:42  <GitHub9> [bitcoin] laanwj pushed 14 new commits to master: https://github.com/bitcoin/bitcoin/compare/c6de5cc88614...3b20e239c602
 33 2015-10-21T06:50:43  <GitHub9> bitcoin/master 78b82f4 Suhas Daftuar: Reverse the sort on the mempool's feerate index
 34 2015-10-21T06:50:43  <GitHub9> bitcoin/master 49b6fd5 Pieter Wuille: Add Mempool Expire function to remove old transactions...
 35 2015-10-21T06:50:44  <GitHub9> bitcoin/master 9c9b66f Matt Corallo: Fix calling mempool directly, instead of pool, in ATMP
 36 2015-10-21T06:50:48  <GitHub173> [bitcoin] laanwj closed pull request #6722: Limit mempool by throwing away the cheapest txn and setting min relay fee to it (master...mempoollimit) https://github.com/bitcoin/bitcoin/pull/6722
 37 2015-10-21T06:50:58  <BlueMatt> heyyyyyyyy
 38 2015-10-21T06:57:13  <jonasschnelli> Nice
 39 2015-10-21T06:58:12  <jonasschnelli> Finally
 40 2015-10-21T07:00:01  *** midnightmagic has quit IRC
 41 2015-10-21T07:04:57  *** Thireus has joined #bitcoin-core-dev
 42 2015-10-21T07:05:00  <phantomcircuit> BlueMatt, great now i can start reviewing it
 43 2015-10-21T07:05:04  *** midnightmagic has joined #bitcoin-core-dev
 44 2015-10-21T07:05:05  * phantomcircuit flees for his life
 45 2015-10-21T07:05:16  <BlueMatt> phantomcircuit: Ive done that before :/
 46 2015-10-21T07:08:58  <wumpus> gmaxwell: would be interesting to check at least; switching to tcmalloc is as simple as LD_PRELOAD a library, don't know about jemalloc
 47 2015-10-21T07:09:37  <gmaxwell> same thing for jemalloc.
 48 2015-10-21T07:09:55  *** Arnavion has quit IRC
 49 2015-10-21T07:10:08  *** AtashiCon has quit IRC
 50 2015-10-21T07:11:19  *** midnightmagic has quit IRC
 51 2015-10-21T07:12:59  <phantomcircuit> i wonder how much it would mess up things to put all the global config things into a locked object
 52 2015-10-21T07:13:01  <wumpus> good as a workaround but I'd prefer a solution that doesn't involve making changes at that level. C++ makes it possible to use own memory pools/allocators for specific things, that can be discarded in one block (say) at the end of the function, that seems to be a more predictable way
 53 2015-10-21T07:15:09  <gmaxwell> I wish the standard API just let you hint short and long lived objects.
 54 2015-10-21T07:15:12  <wumpus> phantomcircuit: which things aren't protected by mutexes that should be ?
 55 2015-10-21T07:15:29  * phantomcircuit looks at gmaxwell 
 56 2015-10-21T07:16:02  <wumpus> at the least you'd want a configuration object per module, not single global one, there is already too much lock centralization
 57 2015-10-21T07:19:02  <phantomcircuit> wumpus, i believe we do bad things with stuff like fReindex and nPruneTarget
 58 2015-10-21T07:22:59  *** midnightmagic has joined #bitcoin-core-dev
 59 2015-10-21T07:29:05  <wumpus> gmaxwell: yes, unfortunately C and C++ are very much lacking with regard to giving hints to your compiler or runtime, it presupposes them to be much more important
 60 2015-10-21T07:29:10  <wumpus> s/important/smart
 61 2015-10-21T07:51:00  <btcdrak> gmaxwell: what are the deployment options for SG?
 62 2015-10-21T07:51:10  <btcdrak> err, SW.
 63 2015-10-21T07:51:43  <gmaxwell> btcdrak: a flag day swap of the network and all software to use a different transaction seralization.
 64 2015-10-21T07:52:02  <gmaxwell> I don't think it can be done.
 65 2015-10-21T07:52:19  <btcdrak> gmaxwell: if we're going to have a hard fork, we may as well take the opportunity then...
 66 2015-10-21T07:52:21  * btcdrak runs
 67 2015-10-21T07:52:43  <phantomcircuit> btcdrak, there's a huge list of things that should absolutely be in a flag day
 68 2015-10-21T07:54:19  <btcdrak> SW is a big win, solves a lot of problems and would make a flagday worth the hassle. If we do end up with a blocksize increase, SW should be added at the same time imo.
 69 2015-10-21T07:54:48  * wumpus throws a hard frok after btcdrak
 70 2015-10-21T07:55:35  <Luke-Jr> gmaxwell: I thought I had convinced sipa a softfork was possible for SW a week or two ago. Or is that just too ugly?
 71 2015-10-21T07:55:39  * btcdrak has never tried to cross-dress before
 72 2015-10-21T07:56:31  <phantomcircuit> btcdrak, the blocksize thing is spv backwards compatible, the only other hard fork proposal im aware of that is, is for commitment schemes in the merkle tree root
 73 2015-10-21T07:56:59  <phantomcircuit> note: i dont actually see that as being very useful
 74 2015-10-21T07:57:06  <btcdrak> Luke-Jr: I'm sure I had a conversation with someone recently, maaku or BlueMatt maybe, about how SW could be a soft fork.
 75 2015-10-21T07:57:19  <phantomcircuit> spv clients can all be updated remotely by the devs since they're apps in an app store thingie
 76 2015-10-21T07:57:38  * Luke-Jr agrees SPV-backward-compatible is not a particularly desirable attribute.
 77 2015-10-21T07:58:25  <phantomcircuit> SW cannot really be a soft fork, you can commit to the sw mtr as a sf but that only gets you the uninteresting part of it
 78 2015-10-21T07:58:44  <phantomcircuit> oh but actually you can force the tx to commit to the sw txid in the script sig
 79 2015-10-21T07:58:50  <phantomcircuit> ha you could but... please no
 80 2015-10-21T07:59:26  <Luke-Jr> phantomcircuit: would it really be worse than this thing cdecker is proposing?
 81 2015-10-21T07:59:42  <gmaxwell> btcdrak: it cannot be a softfork. only a weak approximation of it can be, which is what the decker email is.  But that approximation doesn't get you benefits like being able to sync blocks without downloading signatures. (itself likely elimiating most of the need for bloom filtered lite wallets, among other benefits);  ... and the approximation has costs like a considerable utxo size overhead, a
 82 2015-10-21T07:59:48  <gmaxwell> nd doubling the amount of transaction hashing we have to do.
 83 2015-10-21T08:00:18  <Luke-Jr> gmaxwell: I don't see why you can't sync without downloading signatures in the softfork.
 84 2015-10-21T08:00:51  <gmaxwell> Luke-Jr: because you can't verify a txout you learn without the whole transaction.
 85 2015-10-21T08:01:13  <gmaxwell> (unless in a softfork you mean extensionblock like softforks)
 86 2015-10-21T08:02:24  <Luke-Jr> gmaxwell: the witness wouldn't be part of the "whole transaction"; it would be stored in a separate piece of data the block commits to
 87 2015-10-21T08:02:50  <btcdrak> SW was quite well received in this thread: https://www.reddit.com/r/Bitcoin/comments/3ngtx5/could_the_segregated_witness_part_of_the/
 88 2015-10-21T08:03:05  <phantomcircuit> block commits to the normal txid as normal and the sw id & script hash separately
 89 2015-10-21T08:03:20  <gmaxwell> Luke-Jr: hm. interesting, you have a point. So you'd basically have a scriptpubkey that says "go look elsewhere for the signature"
 90 2015-10-21T08:03:35  <Luke-Jr> right
 91 2015-10-21T08:04:11  <gmaxwell> still ends up needing to store two IDs.
 92 2015-10-21T08:04:29  <gmaxwell> hm. or maybe it doesn't.
 93 2015-10-21T08:05:36  <phantomcircuit> gmaxwell, segregated witness id should match transaction id if you simply require all the scripts in the normal transaction to be OP_1
 94 2015-10-21T08:05:37  <gmaxwell> Luke-Jr: damn you're right. This is pretty cool.
 95 2015-10-21T08:05:58  <gmaxwell> phantomcircuit: yea.
 96 2015-10-21T08:06:29  <Luke-Jr> ☺
 97 2015-10-21T08:06:31  <gmaxwell> phantomcircuit: empty is better.
 98 2015-10-21T08:07:07  <gmaxwell> So the idea there would be a new p2sh like scriptpubkey which requires the scriptsig to be empty.  The signatures are 'external', and commited in another hashtree commited to by the block.
 99 2015-10-21T08:08:21  <gmaxwell> it's identical to SW over whatever span of tx history is exclusively using this scriptpubkey type..
100 2015-10-21T08:09:06  <Luke-Jr> (which could be enforced by the softfork, if it was desirable)
101 2015-10-21T08:10:18  *** rubensayshi has joined #bitcoin-core-dev
102 2015-10-21T08:11:09  <gmaxwell> Luke-Jr: then it turns it into a flagday which is the hard thing to avoid.
103 2015-10-21T08:11:19  <Luke-Jr> sure
104 2015-10-21T08:11:28  <Luke-Jr> personally I think we might as well just do it in a hardfork
105 2015-10-21T08:11:35  <Luke-Jr> but a softfork is possible
106 2015-10-21T08:12:03  <gmaxwell> the problem is that it is not very reasonable to demand the entire bitcoin ecosystem cut cold to new code all at once.
107 2015-10-21T08:12:18  <gmaxwell> (I mean, you could try to demand it but the result would be epic amounts of failure)
108 2015-10-21T08:12:30  <gmaxwell> esp at the high degree that people reimplement everything themselves.
109 2015-10-21T08:12:36  <Luke-Jr> we'd need to ship a version with both code
110 2015-10-21T08:12:52  <Luke-Jr> but that's true of any hardfork
111 2015-10-21T08:13:14  <gmaxwell> Sure, yea yea, the issue isn't us. it's j random wallet thingy that hardly works at all _currently_.  It might even manage to ship code for both modes, but then the other one just won't actually work.
112 2015-10-21T08:13:38  <Luke-Jr> oh, you mean tx creation :D
113 2015-10-21T08:13:56  <gmaxwell> because outside of bitcoin core most software (but by no means all) in the ecosystem is developed by teams of one or two people, and shipped with little to no systematic testing.
114 2015-10-21T08:14:04  <gmaxwell> not just creation but handling.
115 2015-10-21T08:14:18  <gmaxwell> Not how we still have no remotely decent block explorer thingy for elements alpha.
116 2015-10-21T08:14:30  <Luke-Jr> yeah, I guess that's a big problem
117 2015-10-21T08:15:16  * Luke-Jr ponders if we can do the softfork way that remains tx-compatible with a future hardfork
118 2015-10-21T08:18:03  <gmaxwell> I think it's almost that already, without special considerations.
119 2015-10-21T08:18:50  <Luke-Jr> maybe
120 2015-10-21T09:01:54  *** randy-waterhouse has joined #bitcoin-core-dev
121 2015-10-21T09:20:01  <GitHub36> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/3b20e239c602...0fbfc5106cd9
122 2015-10-21T09:20:01  <GitHub36> bitcoin/master 41db8c4 Wladimir J. van der Laan: http: Restrict maximum size of request line + headers...
123 2015-10-21T09:20:02  <GitHub36> bitcoin/master 0fbfc51 Wladimir J. van der Laan: Merge pull request #6859...
124 2015-10-21T09:20:10  <GitHub199> [bitcoin] laanwj closed pull request #6859: http: Restrict maximum size of http + headers (master...2015_10_max_http_headers) https://github.com/bitcoin/bitcoin/pull/6859
125 2015-10-21T09:58:37  <gmaxwell> Is there a reason that for fee purposes we are not using max(size, sigops*BLOCK_MAX_BYTES/BLOCK_MAX_SIGOPS) as the size of a transaction?
126 2015-10-21T09:58:45  <GitHub192> [bitcoin] domob1812 opened pull request #6863: [Test Suite] Fix test for null tx input (master...null-txin-test) https://github.com/bitcoin/bitcoin/pull/6863
127 2015-10-21T10:02:41  <gmaxwell> I'd like to see this go in https://github.com/bitcoin/bitcoin/pull/6622  I've had this in testing on several nodes (including one acting as a gateway to the outside world) for several others for over a month now.
128 2015-10-21T10:02:58  <gmaxwell> Does anyone have any views on what else we need? for #6622?
129 2015-10-21T10:07:36  <gmaxwell> libevent related compile fail on one of my older fedora hosts,   error: ‘EVENT_LOG_WARN’ was not declared in this scope
130 2015-10-21T10:17:29  <btcdrak> gmaxwell: if this is the first time compiling since libevent merge you need to do a git clean -dfx
131 2015-10-21T10:21:37  <gmaxwell> btcdrak: fwiw, in my case no concern but you might want to take care in advising people to git clean -d :)
132 2015-10-21T10:21:53  <gmaxwell> "omg I was keeping my wallet in my source directory!"
133 2015-10-21T10:25:32  <wumpus> at least mention what it does
134 2015-10-21T10:26:06  <gmaxwell> yea.
135 2015-10-21T10:26:47  <gmaxwell> So in any case, cleaning doesn't make that go away. I see some other EVENT logging stuff appears to be ifdef guarded.
136 2015-10-21T10:26:53  <wumpus> but after larger changes to the build system it's good advice
137 2015-10-21T10:27:07  <gmaxwell> this appears to have libevent-2.0.so.5.1.6
138 2015-10-21T10:28:01  <wumpus> I don't know what the minimum version of libevent is that is supported, may be too old
139 2015-10-21T10:29:45  <gmaxwell> well hacking out that one like got it to build.
140 2015-10-21T10:29:47  <wumpus> though if it just errors about the logging stuff, commenting that out may make it compile further
141 2015-10-21T10:29:53  <wumpus> good
142 2015-10-21T10:29:56  <gmaxwell> er one line.
143 2015-10-21T10:30:01  <gmaxwell> testing now.
144 2015-10-21T10:30:17  <gmaxwell> warren: what libevent version does current RHES have?
145 2015-10-21T10:30:28  <wumpus> it's not necessary, it's just nice, redirects libevent errors to our debug log, likely you could put some #ifdef guard around it...
146 2015-10-21T10:30:36  <gmaxwell> unit tests pass.
147 2015-10-21T10:31:19  <wumpus> #if LIBEVENT_VERSION_NUMBER >= 0x....
148 2015-10-21T10:36:30  <warren> gmaxwell: RHEL7 you mean?
149 2015-10-21T10:36:52  <gmaxwell> warren: Yes, whatever version is most current.
150 2015-10-21T10:37:11  <warren> libevent-2.0.21-4.el7
151 2015-10-21T10:37:37  <warren> Keep in mind that if <whatever> is important you need to actually look at the source package as they often will forward port things.
152 2015-10-21T10:37:50  <gmaxwell> warren: okay, thats newer than what this host has in any case.
153 2015-10-21T10:38:43  <warren> wow, that's the same version in Fedora 22.  I guess it didn't change upstream in a while.
154 2015-10-21T10:39:44  <gmaxwell> the particular host I was trying on is my oldest still running fedora box, with F19.
155 2015-10-21T10:40:38  *** Naphex has quit IRC
156 2015-10-21T10:42:04  *** Naphex has joined #bitcoin-core-dev
157 2015-10-21T10:42:32  *** Naphex has quit IRC
158 2015-10-21T10:42:33  *** Naphex has joined #bitcoin-core-dev
159 2015-10-21T10:47:28  <wumpus> 2.0.21 is the one-to-latest stable of libevent, they have been working on 2.1 for a long time but it's still in alpha
160 2015-10-21T10:58:35  <GitHub79> [bitcoin] MarcoFalke opened pull request #6864: [qt] Use monospace font (master...MarcoFalke-2015-qtMonospace) https://github.com/bitcoin/bitcoin/pull/6864
161 2015-10-21T11:13:59  *** randy-waterhouse has quit IRC
162 2015-10-21T11:40:40  <btcdrak> gmaxwell: omg
163 2015-10-21T12:03:09  *** Arnavion has joined #bitcoin-core-dev
164 2015-10-21T12:03:44  *** AtashiCon has joined #bitcoin-core-dev
165 2015-10-21T12:45:48  *** moli has joined #bitcoin-core-dev
166 2015-10-21T12:47:19  *** molly has quit IRC
167 2015-10-21T12:48:19  *** ParadoxSpiral has joined #bitcoin-core-dev
168 2015-10-21T12:53:56  *** treehug88 has joined #bitcoin-core-dev
169 2015-10-21T13:04:57  *** berndj has joined #bitcoin-core-dev
170 2015-10-21T13:06:40  <jgarzik> gmaxwell, ok to merge #6622, IMO
171 2015-10-21T13:13:59  *** treehug88 has quit IRC
172 2015-10-21T13:24:57  *** treehug88 has joined #bitcoin-core-dev
173 2015-10-21T13:33:39  *** Thireus has quit IRC
174 2015-10-21T13:33:53  *** Thireus has joined #bitcoin-core-dev
175 2015-10-21T13:38:39  *** Thireus has quit IRC
176 2015-10-21T13:51:02  *** Thireus has joined #bitcoin-core-dev
177 2015-10-21T14:00:49  *** wangchun has quit IRC
178 2015-10-21T14:22:05  *** treehug88 has quit IRC
179 2015-10-21T14:26:20  *** ghtdak has quit IRC
180 2015-10-21T14:38:10  *** ghtdak has joined #bitcoin-core-dev
181 2015-10-21T14:41:18  *** MarcoFalke has joined #bitcoin-core-dev
182 2015-10-21T14:43:19  *** treehug88 has joined #bitcoin-core-dev
183 2015-10-21T14:52:13  *** MarcoFalke has quit IRC
184 2015-10-21T14:52:25  *** MarcoFalke has joined #bitcoin-core-dev
185 2015-10-21T14:55:49  *** sipa has joined #bitcoin-core-dev
186 2015-10-21T15:27:05  *** MarcoFalke has quit IRC
187 2015-10-21T15:47:51  *** wangchun has joined #bitcoin-core-dev
188 2015-10-21T15:48:26  *** wangchun has quit IRC
189 2015-10-21T15:48:46  *** wangchun has joined #bitcoin-core-dev
190 2015-10-21T16:02:50  *** rubensayshi has quit IRC
191 2015-10-21T16:27:18  *** treehug88 has quit IRC
192 2015-10-21T16:47:32  *** jl2012 has joined #bitcoin-core-dev
193 2015-10-21T17:19:52  *** paveljanik has joined #bitcoin-core-dev
194 2015-10-21T17:19:52  *** paveljanik has joined #bitcoin-core-dev
195 2015-10-21T17:28:07  *** Anduck has quit IRC
196 2015-10-21T17:28:59  *** Anduck has joined #bitcoin-core-dev
197 2015-10-21T17:29:50  *** teward has quit IRC
198 2015-10-21T17:34:56  *** CodeShark__ has joined #bitcoin-core-dev
199 2015-10-21T17:37:09  *** teward has joined #bitcoin-core-dev
200 2015-10-21T17:39:26  <gmaxwell> cfields_: Any thoughts on #6622?  If you look at it-- it's largely orthorgonal with your overall networking work, happening at a higher level. (later rate limiting support would probably compliment it nicely)
201 2015-10-21T17:44:08  *** gribble has quit IRC
202 2015-10-21T17:44:42  <cfields_> gmaxwell: hmm
203 2015-10-21T17:46:27  <gmaxwell> wow, we never went and set TCP_NODELAY on our sockets? we should do that right away-- perhaps even in backports.  We've talked about it before but apparently never did it.
204 2015-10-21T17:46:37  *** gribble has joined #bitcoin-core-dev
205 2015-10-21T17:50:24  <jgarzik> gmaxwell, not that I disagree, but... what's your specific motivation to use NODELAY & disable Nagel?
206 2015-10-21T17:52:23  <gmaxwell> jgarzik: now that we've removed all the gratitious sleeps from the networking, nagle is almost certantly slowing our performance. Consider how chatting the bitcoin protocol is.  It likely also makes the traffic more bursty, which isn't good for other users sharing the network.
207 2015-10-21T17:53:14  <gmaxwell> Basically all of our writes should be large enough that nagle shouldn't be saving us much, and the smaller writes we to tend to be latency important.
208 2015-10-21T17:53:29  <jgarzik> gmaxwell, nod
209 2015-10-21T17:53:43  <jgarzik> gmaxwell, just theory or is someone reporting this?
210 2015-10-21T17:53:56  <jgarzik> certainly appears low hanging fruit
211 2015-10-21T17:54:09  <cfields_> gmaxwell: agree that the logic for deciding what to avoid sending belongs at a higher level. The implementation itself seems a bit naive, though?
212 2015-10-21T17:54:52  <gmaxwell> jgarzik: Suhas mentioned in an email some testing that seemed impacted by it.  I'm just surprised we hadn't done this already, historical analysis suggests it's somewhat my fault! :(
213 2015-10-21T17:55:24  <sdaftuar> jgarzik: yeah tcp_nodelay speeds up block relay by a round trip
214 2015-10-21T17:55:26  <gmaxwell> (jgarzik you brought it up previously, and I (thinking of the 100ms sleeps) said we had much more low hanging fruit. :P )
215 2015-10-21T17:56:06  <jgarzik> yep it was an issue when I rewrote low level network reads/writes code
216 2015-10-21T17:56:26  <gmaxwell> I'd previously gone around to miner software and p2pool and had them fix it, even before it had been brought up in bitcoin core.
217 2015-10-21T17:56:29  <jgarzik> thanks.  I was curious where it would first trip up ppl
218 2015-10-21T17:56:50  <btcdrak> can someone with rights please restart this travis job please? https://travis-ci.org/bitcoin/bitcoin/jobs/86431798
219 2015-10-21T17:58:17  <cfields_> btcdrak: just force-push
220 2015-10-21T17:58:25  <btcdrak> not my PR
221 2015-10-21T18:00:30  <morcos> wumpus: sipa: i've been trying to learn a bit more about this intensive memory usage in getblocktemplate.  i still haven't quite figured out why the framentation is so bad that nothing can get clean up, but there are a few things happening
222 2015-10-21T18:00:56  <morcos> during CreateNewBlock, we fetch all the coins needed for all the txs in our mempool.
223 2015-10-21T18:01:20  <morcos> there is a 2x hit for this, as we make a copy in pcoinstip (assuming they aren't already cached there) and a copy in the view in miner.cpp
224 2015-10-21T18:02:04  <morcos> also note that the dbcache size we set here will have just been ignored as we're loading up pcoinstip.  so if we're over that it'll just get flushed again once we're done with CNB
225 2015-10-21T18:02:53  <morcos> also because the rpc call is running in a different thread, if the populating of the cache in pcoinstip trips the loadfactor of the CCoinsMap.  Then we'll end up moving the CCoinsMap memory to a different arena.
226 2015-10-21T18:03:26  <wumpus> right, that is how it will happen with the coinsview caches
227 2015-10-21T18:03:38  <morcos> so both the size of the coins in the miner.cpp view and the size in the coins of pcoinstip could possibly be replicated in #rpcthreads + 1
228 2015-10-21T18:04:11  <wumpus> indeed
229 2015-10-21T18:04:12  <gmaxwell> morcos: sipa and BlueMatt have been been working on the view cache memory usage.
230 2015-10-21T18:04:49  <morcos> i'm curious as to what they're doing
231 2015-10-21T18:04:50  <wumpus> it seems a bit redundant that the child cache also stores everything
232 2015-10-21T18:04:53  <jgarzik> morcos, RE fragmentation part of that has to do with the low level allocator that how that memory is returned to the OS.  Older allocators use sbrk(), which make it impossible to release memory unless there is absolutely no data structure, including hidden-from-app mgmt ADTs, within the memory range.  Ditto newer mmap-based allocators, which will not munmap unless there is perfect reclamation by app + libc.
233 2015-10-21T18:04:57  <morcos> yes, thats what i was thinking
234 2015-10-21T18:05:14  <jgarzik> So you're stuck even if libc allocation structs are left behind
235 2015-10-21T18:05:28  <jgarzik> (typically two pointers, in negative offsets from your DS)
236 2015-10-21T18:05:29  <morcos> wumpus
237 2015-10-21T18:06:00  <wumpus> especially as this is a read-only view
238 2015-10-21T18:06:03  <morcos> wumpus: which is the child cache?  The cache in view has to store everything b/c it might get modified by in mempool txs
239 2015-10-21T18:06:12  <wumpus> oh it isn't?
240 2015-10-21T18:06:29  <morcos> however the cache in pcoinstip doesn't need to be populated
241 2015-10-21T18:06:50  <morcos> view (in miner.cpp) is backed by pcoinstip is backed by the database
242 2015-10-21T18:06:52  <wumpus> that's the parent cache
243 2015-10-21T18:07:36  <wumpus> and sure - the modified coins need to be stored
244 2015-10-21T18:07:47  <morcos> i think its kind of silly that you set a dbcache size, but then can blow through that inside CreateNewBlock
245 2015-10-21T18:08:09  <morcos> of course its also made much worse by the organization of the mining code
246 2015-10-21T18:08:24  <morcos> no reason to go through every single tx
247 2015-10-21T18:09:09  <wumpus> e.g. the coins that are the same in the child and parent cache don't need to be stored twice
248 2015-10-21T18:09:38  <wumpus> only if the one in the child cache starts to deviate because ti is changed by a mempool transaction
249 2015-10-21T18:09:49  <morcos> yes, thats what i'm suggesting.. when you do a fetch from the child cache, just pass them down from the grandparent cache without storing them intermediately in the parent cache
250 2015-10-21T18:10:38  <wumpus> right, assuming they aren't changed in the parent cache, they don't need to be cached there
251 2015-10-21T18:10:55  <morcos> this is related to #5967
252 2015-10-21T18:11:14  <morcos> where i change the assumptions that the parent cache can't be flushed
253 2015-10-21T18:13:26  <morcos> slight shift of subject, how do most miners use bitcoind now?
254 2015-10-21T18:13:36  <morcos> they do use getblocktemplate?
255 2015-10-21T18:14:00  <wumpus> is there another way?
256 2015-10-21T18:14:01  <morcos> and in effect have CreateNewBlock control their mining algorithm, or they do something else?
257 2015-10-21T18:14:03  *** CodeShark_ has quit IRC
258 2015-10-21T18:14:04  *** CodeShark has quit IRC
259 2015-10-21T18:14:08  *** CodeShark__ has quit IRC
260 2015-10-21T18:14:15  <morcos> i don't know i thought i saw chatter somewhere about miners not using it
261 2015-10-21T18:14:18  *** CodeShark has joined #bitcoin-core-dev
262 2015-10-21T18:14:25  <morcos> but i dont' knwo what else you would do
263 2015-10-21T18:14:41  <wumpus> I think they all use getblocktemplate in some form or another, although some may have made large changes to the code
264 2015-10-21T18:14:55  <morcos> i'm wondering how to think about rewriting the mining code
265 2015-10-21T18:15:05  <morcos> it could use a substantial rewrite
266 2015-10-21T18:15:32  <morcos> but i guess it doesn't have to be thought of as consensus critical if the testing/submitting aspects of it haven't changed?
267 2015-10-21T18:15:34  *** CodeShark_ has joined #bitcoin-core-dev
268 2015-10-21T18:16:01  <wumpus> block validation is consensus critical, creating blocks from the mempool isn't
269 2015-10-21T18:16:08  <gmaxwell> morcos: I suspect you misunderstood whatever that chatter was (or someone was confused); everyone uses GBT to get work from bitcoind.  Many parties also do other stuff as well.
270 2015-10-21T18:16:17  <gavinand1esen> morcos: +1 . I'd suggest starting with a clean sheet of paper, and figuring out what would be best for solo miners / mining pools
271 2015-10-21T18:16:31  <morcos> it seems like there is some low hanging fruit.  once you already have 999,900 bytes in a block, you don't need to scan the remaining 2GB of txs just to see if you find a magical 100 byte tx for instance
272 2015-10-21T18:16:51  <morcos> gmaxwell: ok good... i definitely didn't understand whatever i read
273 2015-10-21T18:16:57  <gmaxwell> so long as the resulting block is valid I don't think it matters how it was done; so I think it's fine to rewrite.
274 2015-10-21T18:17:29  <gmaxwell> morcos: might have been a comment that few/no one is using GBT to go from pools to mining devices; which is the case.
275 2015-10-21T18:17:52  <wumpus> yeah it obviously matters that it generates correct blocks, but the exact algorithm is not set in stone
276 2015-10-21T18:18:19  <gmaxwell> morcos: as far as rewrite goes, being able to get createnewblock out of the latency critical path for mining would be the biggest win from the user's perspective.
277 2015-10-21T18:18:30  <morcos> does it make sense to start with a simple improvement to the existing algorithm, and then separately or later try to use more advanced logic that takes ancestor packages into account?  would it make sense to ever support 2 different algorithms
278 2015-10-21T18:19:13  <morcos> gmaxwell: can you explain that a bit more?  you mean once a new block comes in, the time it takes to have a new potential header to work on?
279 2015-10-21T18:19:15  <gmaxwell> (e.g. by returning an empty template when there isn't one precomputed, then computing a template in the background.).. and thats independant of the algorithims, but it means that more computationally expensive algorithims would be reasonable.
280 2015-10-21T18:19:16  <morcos> even if its not optimal
281 2015-10-21T18:19:48  <morcos> yes, ok that makes a lot of sense
282 2015-10-21T18:20:38  <gmaxwell> morcos: what I was thinking was: there is a cached block template, when a new block comes in, it's no longer valid (as we have a new block) so its flushed. If there is no cached template available, just generate one that has no transactions (fast), and trigger creating a template in the background.... and the background one can get recomputed on a timer or on new-block arrival, so long as GBT re
283 2015-10-21T18:20:44  <gmaxwell> quests keep coming in.
284 2015-10-21T18:20:59  <gmaxwell> we cache CNB right now, but it's "too late" :)
285 2015-10-21T18:21:27  <jgarzik> +1
286 2015-10-21T18:21:45  <jgarzik> that's been the general template for a rewrite, for years
287 2015-10-21T18:22:12  *** Guest98736 is now known as petertodd
288 2015-10-21T18:22:31  <morcos> gmaxwell: yes, thats what we were just discussing...
289 2015-10-21T18:22:32  *** CodeShark_ has quit IRC
290 2015-10-21T18:25:32  <morcos> so a new thread for template generation is ok?
291 2015-10-21T18:26:46  <gmaxwell> morcos: I think so!
292 2015-10-21T18:26:55  <jgarzik> practically required
293 2015-10-21T18:26:56  <wumpus> I don't see why not, but make sure to only do the work if a node is actually mining
294 2015-10-21T18:27:22  <morcos> what should a call to getblocktemplate do if cs_main is locked?  seems too bad that you really dont' care for a lot of the cs_main locks
295 2015-10-21T18:27:39  <morcos> but if the chain is being updated, you might prefer to wait for that to happen before the template is returned
296 2015-10-21T18:27:42  <jgarzik> does it need cs_main if it's simply returning a cached version?
297 2015-10-21T18:28:04  <jgarzik> seems like the RPC call should return cached version, and b/g compute thread is what needs the lock
298 2015-10-21T18:28:05  <morcos> i suppose maybe you could lock the template if you know you're activating a new chain
299 2015-10-21T18:28:28  <morcos> ok i guess i need to learn more about how notification of new tips works anyway
300 2015-10-21T18:28:55  <jgarzik> if a new block comes in, the cache is invalidated + regenerated as an empty template + kicks off new gen thread
301 2015-10-21T18:28:59  <morcos> but yes i agree in theory you should just need a tiny lock on whether your template is switched to the new one
302 2015-10-21T18:29:01  <gavinand1esen> I was very close to refactoring the block validation code a tiny bit so the cs_main lock was released while transaction validity was checked... theoretically not a difficult change
303 2015-10-21T18:29:33  <morcos> gavinand1esen: that seems scary
304 2015-10-21T18:29:35  *** Thireus has quit IRC
305 2015-10-21T18:29:41  <gavinand1esen> that's why theoretically....
306 2015-10-21T18:29:52  <jgarzik> should just need a quick swap upon new-block, not a long term lock
307 2015-10-21T18:30:16  <morcos> jgarzik: isn't the technical word for that "tiny"?
308 2015-10-21T18:30:38  <jgarzik> can be lockless technically :)
309 2015-10-21T18:31:25  <gmaxwell> yes, the cache could be RCU, but I don't think thats needed. :)
310 2015-10-21T18:31:40  <jgarzik> I was thinking more std::atomic
311 2015-10-21T18:33:19  <morcos> ok all sounds good... i'll give it a shot
312 2015-10-21T18:34:11  <gavinand1esen> morcos: I was mapping out the work needed to validate blocks in parallel (as part of prep work for looking at broadcasting 'weak blocks'); was pondering an extension to CCoinsViewCache that used leveldb read-only snapshots to validate against the UTXO set as of the last block, which could let validation against two different CCoinsViewCache's happen in parallel without the cs_main lock
313 2015-10-21T18:35:00  <gavinand1esen> morcos: that idea might work for a cs_main-free CreateNewBlock, too....
314 2015-10-21T18:35:40  *** gavinand1esen is now known as gavinandresen
315 2015-10-21T18:36:25  <sipa> signature validation already happens without the cs_main lock
316 2015-10-21T18:41:35  <morcos> sipa: you mean b/c they are happening in other threads?  but the cs_main lock is held and any further processing waits for them to finish
317 2015-10-21T18:42:17  <morcos> sipa: btw, did you see my comments about doing threaded signature checking in ATMP in addition to in ConnectBlock?
318 2015-10-21T18:42:40  <morcos> it seems thats when the actually work is being done a lot of the time anyway
319 2015-10-21T18:44:36  <sipa> morcos: i did not see that, but it certainly may make sense
320 2015-10-21T18:44:52  <sipa> i'd like to move signature checking to be something fully asynchronous though
321 2015-10-21T18:45:17  <sipa> with a queue that is worked on by threads, and notification callbacks happen when validatiin succeeds/fails
322 2015-10-21T18:46:04  <morcos> oh...  huh, so a block could just assume that all signatures were valid and keep processing the rest of the blcok, but only at the very end wait
323 2015-10-21T18:46:11  <morcos> wow that would be way better
324 2015-10-21T18:46:21  <sipa> it's nontrivial :)
325 2015-10-21T18:46:27  <morcos> not for you!
326 2015-10-21T18:46:44  <jcorgan> i think it would be a great idea...if sipa does it :-)
327 2015-10-21T18:46:52  <gmaxwell> morcos: yes, ... behold the sadness of CHECKSIG NOT, however.
328 2015-10-21T18:47:21  <morcos> hmm... so you don't even know whether you want it to be valid or not
329 2015-10-21T18:47:35  <gmaxwell> if all of script is async then that goes away.
330 2015-10-21T18:47:36  <morcos> i thought processors were good at branching. :)
331 2015-10-21T18:48:00  <morcos> what do you mean goes away?
332 2015-10-21T18:48:23  <gmaxwell> I mean that if the thing you dispatch is the whole script and not just ecdsa then there is no chance that you really want it to fail.
333 2015-10-21T18:48:31  <jgarzik> processors try to predict branching because they suck at it ;p
334 2015-10-21T18:48:44  <morcos> oh...
335 2015-10-21T18:48:49  <gmaxwell> but then the state that needs to be dispatched is large.
336 2015-10-21T18:49:19  <jgarzik> The kernel compiles Berkeley packet filters in a JIT - would be amusing to JIT scripts & sigs
337 2015-10-21T18:49:40  <gmaxwell> jgarzik: by amusing you mean the tremendous fun in finding all the attacks...
338 2015-10-21T18:49:43  <gmaxwell> :P
339 2015-10-21T18:50:44  <gmaxwell> morcos: in Elements alpha we turned CHECKSIG into CHECKSIGVERIFY  (if you want a checksig thats allowed to fail, wrap it in an if).
340 2015-10-21T18:50:57  <sipa> gmaxwell: i mean parallellizing script validation
341 2015-10-21T18:51:02  <sipa> not signature validation
342 2015-10-21T18:51:18  <gmaxwell> sipa: k. then the negation problem goes away... but lots of data to manage.
343 2015-10-21T18:52:19  <gmaxwell> (the CHECKSIGVERIFY vs CHECKSIG is also motivated by batch EC signature checking, which is faster and only gives you a pass/fail for the whole batch).
344 2015-10-21T18:52:20  <morcos> i'd feel better about if we more formally declared what the state that can affect script validation is
345 2015-10-21T18:52:43  <morcos> yeah do we need CHECKSIG?
346 2015-10-21T18:54:04  <gmaxwell> No, there is no need for CHECKSIG--- in theory we could softfork convert existing checksig into a CHECKSIGVERIFY; though some risk of breaking some crazy existing scriptpubkey and making it unspendable.
347 2015-10-21T18:54:45  <gmaxwell> So thats probably not advisable, but any future checksig like operators will certantly be VERIFY only.
348 2015-10-21T18:55:09  <morcos> wouldn't that not be a soft fork because of the CHECKSIG NOT's
349 2015-10-21T18:55:34  <morcos> it seems like you could just bump tx version if you didn't want to worry about existing scriptpubkeys
350 2015-10-21T18:56:16  <gmaxwell> morcos: it would be a soft fork: If you had a CHECKSIG NOT with a bad input  you'd just get denied before you got to the NOT.
351 2015-10-21T18:57:02  <gmaxwell> Less agressive but probably equally useful is CHECKSIG requiring that the only signature that is allowed to fail is the zero length signature. (so you can decide if it fails or not without doing anything but checking the length)
352 2015-10-21T18:58:15  <gmaxwell> morcos: key criteria for a soft-fork is that it cannot turn any invalid thing valid. So a soft fork that just added additional reasons for the script to fail when excuting a checksig would be a softfork (if not an advisable one!)
353 2015-10-21T18:58:49  <gmaxwell> the only-zero-length can fail also has the advantage of being less potentially incompatible with existing scriptpubkeys.
354 2015-10-21T18:59:00  <morcos> gmaxwell: yes, i just got confused about whether verify shortcircuited or not
355 2015-10-21T18:59:32  <morcos> ah so its the tx version of the spending tx that matters
356 2015-10-21T18:59:36  <gmaxwell> As far as version, it's a little complicated. We don't have versions on txouts in transactions.  Though we do track the transaction version in the utxo set at least.
357 2015-10-21T19:00:23  <gmaxwell> so one could use that utxo version to decide what rules to apply. alternatively one could use the spending transaction, version... and if you need to make a signature with the old rules to have it pass, then use the old version.
358 2015-10-21T19:00:30  <gmaxwell> So I think that would work.
359 2015-10-21T19:00:39  <morcos> yeah, but its more messy than i hope
360 2015-10-21T19:00:40  <morcos> d
361 2015-10-21T19:01:08  <jgarzik> one step at a time - just getting a b/g regen would be nice
362 2015-10-21T19:01:09  <jgarzik> ;p
363 2015-10-21T19:01:25  <gmaxwell> Not really sure if it's worth it for the existing checksig... esp since for ECDSA with our current encoding it only gives you the async gains.  For the schnorr in elements it allows a 2x verification speedup from batch verification.
364 2015-10-21T19:22:45  <gmaxwell> sipa: can you please get cdecker to not call his work "normalized transaction ids"
365 2015-10-21T19:22:55  <gmaxwell> (or have you met with him already?)
366 2015-10-21T19:40:26  <sipa> gmaxwell: just did
367 2015-10-21T19:40:31  <sipa> why not?
368 2015-10-21T20:19:02  *** MarcoFalke has joined #bitcoin-core-dev
369 2015-10-21T20:29:28  *** belcher has joined #bitcoin-core-dev
370 2015-10-21T20:36:14  <gmaxwell> sipa: because of the instant automatic assumption that this results in a transaction ID that cannot change.
371 2015-10-21T20:36:37  <gmaxwell> sipa: did you see that luke figured out how to make SW a soft-fork, rather elegantly too?
372 2015-10-21T20:37:56  <sipa> no, but i did talk about it with cdecker
373 2015-10-21T20:44:23  <sipa> gmaxwell: well it'd be a second id... transactions would have a txid and an ntxid
374 2015-10-21T20:45:16  <CodeShark> +1 to getting rid of scriptSig :)
375 2015-10-21T20:46:15  <gmaxwell> sipa: but the ntxid is not actually non-malleable. So if you write software expecting that it is, you will be disappointed (and maybe even suffer funds loss).
376 2015-10-21T20:47:28  <gmaxwell> It's non-malleable against certian kinds of operations under certian conditions, and quite useful. But we had this problem before that people assume this ID is actually more useful for accounting purposes than it is (with your prior writeup)
377 2015-10-21T20:48:28  <sipa> anything except sighash flags that isn't covered?
378 2015-10-21T20:49:34  <gmaxwell> signer just modifying the transaction; e.g. I pay you and it takes two hours to confirm, and during that time I RBF the payment (to boost the fees, for example).  You still get paid all the same but now your software is confused when the ntxid changed out from under you.
379 2015-10-21T20:49:58  <CodeShark> you can also stick in some NO_OPs into the script just to mess with the system :)
380 2015-10-21T20:50:04  <gmaxwell> And if you go look at the malleability paper from fc2015 thats the kind of thing wallets were doing with txid and getting horribly confused.
381 2015-10-21T20:50:08  <CodeShark> or write a different script with equivalent logic
382 2015-10-21T20:50:09  <sipa> gmaxwell: that's not malleability
383 2015-10-21T20:50:34  <sipa> but fair enough
384 2015-10-21T20:50:37  <sipa> do you have a better idea?
385 2015-10-21T20:50:39  <sipa> name
386 2015-10-21T20:51:17  <CodeShark> so the main feature is that malleating the transaction requires creating a new signature/new signatures?
387 2015-10-21T20:51:36  <gmaxwell> I think we should do the SW instead, and then it's just the same old txid and we do not introduce yet another ID that people need to deal with which is supposted to help malleabiity but still can't actually be used as a payment identifier.
388 2015-10-21T20:52:01  <sipa> SW doesn't help against these
389 2015-10-21T20:52:23  <sipa> it also changes txid on resigning/modifying a transaction
390 2015-10-21T20:52:29  <gmaxwell> Indeed, it doesn't but it doesn't introduce yet another ID which also doesn't do what people expect it to do unconditionally (because the expectation is unreasonable)
391 2015-10-21T20:53:14  <gmaxwell> and it has a number of additional benefits. (e.g. allowing private lite wallets that don't download signatures; or syncing the history without syncing the signatures)
392 2015-10-21T20:53:36  <sipa> i fear this is a lot more work than you think
393 2015-10-21T20:53:40  <sipa> not just in bitcoin core
394 2015-10-21T20:54:00  <gmaxwell> And not increasing the size of the utxo set.
395 2015-10-21T20:54:12  <sipa> it means we need new messages for blocks and transactions
396 2015-10-21T20:54:16  <sipa> to relay extra data
397 2015-10-21T20:54:19  <sipa> store that on disk
398 2015-10-21T20:54:24  <gmaxwell> You can use it in a softfork manner without updating. Actually the bitcoin protocol _used_ to relay extra data outside of transactions but that got removed.
399 2015-10-21T20:54:26  <sipa> go fetch it from peers that have it
400 2015-10-21T20:54:44  <gmaxwell> Yes, it complicates relay; which is the downside compared to the approach in EA.
401 2015-10-21T20:55:07  <sipa> it introduces a new type of block witholding
402 2015-10-21T20:55:17  <gmaxwell> sipa: I don't think it does.
403 2015-10-21T20:55:39  <sipa> if an old client gives you a block without the witness you need to go fetch it elsewhere
404 2015-10-21T20:55:56  <gmaxwell> sipa: or you just require that the network of updated nodes be connected.
405 2015-10-21T20:56:13  <sipa> so you stop accepting blocks from old nodes? yuck...
406 2015-10-21T20:56:15  <gmaxwell> E.g. you wouldn't fetch a block from a older protocol version.
407 2015-10-21T20:56:26  <gmaxwell> which, of course, could be deployed first.
408 2015-10-21T20:58:03  *** paveljanik has quit IRC
409 2015-10-21T20:58:26  <gmaxwell> In exchange you: Avoid giving the public two txids both of which don't do what they expect, you avoid increasing the UTXO set size by 30%, you allow new sync methods that don't transfer the signatures (saving 2/3rd the bandwidth), and you avoid double hashing transaction data to compute two IDs on it.
410 2015-10-21T20:59:32  <morcos> 9 blocks in the last 20 mins?
411 2015-10-21T21:00:41  <sipa> and you get a single txid that equally doesn't do what you expect, pretty much need everyone to upgrade anyway, or you need to keep supporting chains of transactions in which some are old-version which breaks even the intended effect
412 2015-10-21T21:01:53  <sipa> while ntxid is strictly better at accomplishing the goal of malleability protection, and is a simple new opcode to softfork in
413 2015-10-21T21:02:05  <sipa> (with the downsides you mentioned, yes)
414 2015-10-21T21:02:09  <gmaxwell> sipa: the txid is functonally the same as the ntxid but at least it's not confusing people by being something new that they'll understand as existing to accomplish something that cannot be accomplished.
415 2015-10-21T21:02:25  <gmaxwell> sipa: it is strictly inferior.
416 2015-10-21T21:02:35  <sipa> ntxid is recursive; sw is not - if some transactions in the chain do not use sw, it breaks; ntxid does not break
417 2015-10-21T21:03:22  <gmaxwell> sipa: okay point... This is true, but the cost is a 30% increase in utxo size. I don't think that is reasonable.
418 2015-10-21T21:03:48  <gmaxwell> thats a significant and perminant increase in the total operating cost of the system.
419 2015-10-21T21:04:48  <gmaxwell> as far as recursive case, it's not so simple-- for contracts all the participants are using the new style scriptpubkeys and you can enforce it so its fine.
420 2015-10-21T21:05:21  <sipa> hmm, i thought i did the math before, and it was 20% extra
421 2015-10-21T21:05:25  <sipa> now it seems to be 50% extra
422 2015-10-21T21:05:31  <gmaxwell> for ntxid none of the existing software is using these IDs, so gain none of the protection, and nodes need reference rewriting to fixup transactions when their parents change.
423 2015-10-21T21:06:04  <sipa> yeah, that fixup is annoying
424 2015-10-21T21:07:07  <gmaxwell> I agree its somewhat less annyoing then having to provide an upgraded relay mechenism, but we need to upgrade relay regardless... and doing weird topology things to assure connectivity, but these are one time and short term costs.
425 2015-10-21T21:07:16  <sipa> this makes no sense
426 2015-10-21T21:07:40  <sipa> the average serialized size of txouts in the utxo set (excluding txid) is 7 bytes
427 2015-10-21T21:07:50  <sipa>   "transactions": 7386379,
428 2015-10-21T21:07:50  <sipa>   "txouts": 32564578,
429 2015-10-21T21:07:50  <sipa>   "bytes_serialized": 480114635,
430 2015-10-21T21:08:13  <gmaxwell> My 30% was just 22+22+22+4+4+32  vs that plus 32 more.
431 2015-10-21T21:08:44  <gmaxwell> sipa: f2pools empty pubkey attack... but I didn't think it was that many.
432 2015-10-21T21:09:11  <sipa> also, i though the total size was way larger than 480 MB
433 2015-10-21T21:09:23  <gmaxwell> oh indeed, something there is screwed. lemme check.
434 2015-10-21T21:24:16  <gmaxwell> Myserty solved, obfscuation change broke the stats by cherry-picking bugged code from sipa's addrindex branch.
435 2015-10-21T21:24:19  <gmaxwell> Author: Pieter Wuille <pieter.wuille@gmail.com>
436 2015-10-21T21:24:21  <gmaxwell> Date:   Wed Oct 7 17:12:24 2015 -0700
437 2015-10-21T21:24:41  <gmaxwell> -                stats.nSerializedSize += 32 + slValue.size();
438 2015-10-21T21:24:41  <gmaxwell> +                stats.nSerializedSize += 32 + pcursor->GetKeySize();
439 2015-10-21T21:25:50  <GitHub125> [bitcoin] sipa opened pull request #6865: Fix chainstate serialized_size computation (master...fixchainsize) https://github.com/bitcoin/bitcoin/pull/6865
440 2015-10-21T21:31:11  <sipa> so, it's a 22% increase to add ntxid at this point
441 2015-10-21T21:31:24  <sipa>   "bytes_serialized": 1065420089
442 2015-10-21T21:33:49  <btcdrak> sipa: that's a lot
443 2015-10-21T21:34:22  <gmaxwell> indeed, its a figure that is artifically lowered by the spam attacks...
444 2015-10-21T21:34:27  *** devrandom has quit IRC
445 2015-10-21T21:35:18  <sipa> lowered?
446 2015-10-21T21:35:44  <sipa> ah, the 22% is lowered - yes
447 2015-10-21T22:19:06  <GitHub36> [bitcoin] MarcoFalke opened pull request #6866: [trivial] fix white space in rpc help messages (master...MarcoFalke-2015-rpcWhitespace) https://github.com/bitcoin/bitcoin/pull/6866
448 2015-10-21T22:50:39  *** sipa has quit IRC
449 2015-10-21T22:50:48  *** sipa has joined #bitcoin-core-dev
450 2015-10-21T22:58:41  *** molly has joined #bitcoin-core-dev
451 2015-10-21T23:01:19  *** moli has quit IRC
452 2015-10-21T23:22:17  *** droark has joined #bitcoin-core-dev
453 2015-10-21T23:22:25  *** droark has quit IRC
454 2015-10-21T23:46:22  <gmaxwell> sdaftuar: your headers first relay test... can you try that with nagle disabled? :P (e.g. is most of the benefit from the change nagle?  (trivial patch)
455 2015-10-21T23:48:25  <sdaftuar> gmaxwell: basically all the benefit is nagle
456 2015-10-21T23:48:47  <sdaftuar> if we turn it off, the existing relay code works as it's supposed to (one round trip instead of two)
457 2015-10-21T23:49:25  <sdaftuar> #6494 wasn't intended to "fix" that bug, it's intended to make relaying generally work better in the case of reorgs
458 2015-10-21T23:49:50  <sdaftuar> it just happens to not trigger nagle
459 2015-10-21T23:49:59  <gmaxwell> yea, obviously #6494 is good and we should do it.
460 2015-10-21T23:50:50  <gmaxwell> oh actually it was gavin that was testing it sorry.
461 2015-10-21T23:55:37  <GitHub20> [bitcoin] gmaxwell opened pull request #6867: Set TCP_NODELAY on P2P sockets. (master...nodelay) https://github.com/bitcoin/bitcoin/pull/6867