1 2015-11-11T00:00:17  <sipa> you could start by not downloading them :)
  2 2015-11-11T00:00:26  <sipa> s/downloading/getdata'ing/
  3 2015-11-11T00:00:38  <gmaxwell> yea, not getdata is an obvious first step and need no protocol handling.
  4 2015-11-11T00:01:32  <gmaxwell> but INVs actually use a lot of bandwidth... on the order of 5% of the size of all transactions per peer.
  5 2015-11-11T00:04:59  <CodeShark> an extra protocol message rather than negotiating it during the handshake?
  6 2015-11-11T00:05:49  <CodeShark> presumably miners would not want to connect to peers that won't relay unconfirmed transactions
  7 2015-11-11T00:05:54  <gmaxwell> CodeShark: shouldn't be a handshake thing, since you may with to adaptive cork and uncork.
  8 2015-11-11T00:06:19  <gmaxwell> er adaptively*
  9 2015-11-11T00:06:56  <CodeShark> if a node does not intend to accept nor relay unconfirmed transactions, it can't really hurt for others to know this
 10 2015-11-11T00:11:03  <CodeShark> one would hope that of 8 randomly chosen nodes at least one is relaying transactions
 11 2015-11-11T00:13:46  <CodeShark> furthermore, given a more efficient block propagation protocol (i.e. one that does not require resending txs to nodes that already have them in their mempool) the main benefit would be only for transactions that take a very long time to confirm or never get confirmed
 12 2015-11-11T00:15:06  <phantomcircuit> gmaxwell, filterload an empty filter
 13 2015-11-11T00:15:08  <phantomcircuit> tada
 14 2015-11-11T00:15:11  <CodeShark> heh
 15 2015-11-11T00:15:28  <wumpus> but then you won't receive any transactions in blocks either?
 16 2015-11-11T00:15:36  <CodeShark> right
 17 2015-11-11T00:15:39  <phantomcircuit> wumpus, getdata MSG_BLOCK
 18 2015-11-11T00:15:45  <phantomcircuit> isn't filtered at all
 19 2015-11-11T00:15:48  <CodeShark> true
 20 2015-11-11T00:16:11  <phantomcircuit> (i run with empty filters on mining full nodes)
 21 2015-11-11T00:16:24  <wumpus> nice
 22 2015-11-11T00:18:16  <gmaxwell> CodeShark: you wouldn't be listening (or at least not node-network and advertising yourself) while also doing this.
 23 2015-11-11T00:19:45  <gmaxwell> CodeShark: thats assuming spherical cow efficiency, and again; it leaves the user saddled with the inefficienct gossiping of transactions.  If you don't care about unconfirmed transactions all that is a waste of time.
 24 2015-11-11T00:20:29  <gmaxwell> beyond the bandwidth efficiency gains, not taking lose transactions removes an attack vector.
 25 2015-11-11T00:21:19  <wumpus> empty filterload should indeed work, the filter is checked before relaying transactions to a node
 26 2015-11-11T00:21:39  <gmaxwell> wumpus: won't that screw up relaying blocks?
 27 2015-11-11T00:22:27  <phantomcircuit> gmaxwell, nope
 28 2015-11-11T00:22:33  <wumpus> that's what I thought too, but phantomcircuit corrected me
 29 2015-11-11T00:23:00  <gmaxwell> phantomcircuit: care to clean up a patch that adds a blocksonly=1 that turns off node network and does the empty filterloads?
 30 2015-11-11T00:25:17  <wumpus> only getdata w/ MSG_FILTERED_BLOCK will have the filter applied
 31 2015-11-11T00:25:29  <gmaxwell> yup. duh.
 32 2015-11-11T00:26:29  <gmaxwell> phantomcircuit: should also check that the neighbor supports bloomfilters and not do it otherwise. And not getdata transactions even if INVed except from whitebind peers, I guess.
 33 2015-11-11T00:27:26  <wumpus> MSG_FILTERED_BLOCK with an empty filter could be handled very quickly, but it doesn't seem that we special case it :-)
 34 2015-11-11T00:29:00  <gmaxwell> hm. I thought I added some special case handling of empty filters before.
 35 2015-11-11T00:29:03  <wumpus> yes you probably want to disconnect from peers that don't support bloom filters
 36 2015-11-11T00:29:49  <wumpus> maybe I missed it
 37 2015-11-11T00:30:58  <wumpus> looks like there is no way to query isEmpty from outside CBloomFilter
 38 2015-11-11T00:31:23  <gmaxwell> inside it there is an isEmpty test that shortcuts contains.
 39 2015-11-11T00:31:48  <gmaxwell> (it was added as the 'cover change' for that divide by zero crash... but it's an actual optimization too)
 40 2015-11-11T00:31:55  *** belcher has quit IRC
 41 2015-11-11T00:32:16  <wumpus> what I meant was more something that bypasses *everything* and just returns empty if you query with an empty filter
 42 2015-11-11T00:32:29  *** zooko has quit IRC
 43 2015-11-11T00:32:59  <wumpus> now it still happily reads the blocks from disk, just to discard every transaction
 44 2015-11-11T00:33:03  <gmaxwell> Would be reasonable to bubble up the full/empty state.
 45 2015-11-11T00:33:06  <phantomcircuit> wumpus, it is special cased
 46 2015-11-11T00:33:11  <gmaxwell> phantomcircuit: in the wrong place.
 47 2015-11-11T00:33:16  <wumpus> phantomcircuit: oh? where?
 48 2015-11-11T00:33:36  <phantomcircuit> it's special cased in the bloom filtering code
 49 2015-11-11T00:34:11  <gmaxwell> Existing special casing inside the bloom object itself was really just done to make that divide by zero bug unreachable. But as wumpus notes it means you'll read the block and test every transaction against the filter, the shortcutting is a bit late.
 50 2015-11-11T00:35:21  <phantomcircuit> yes agreed
 51 2015-11-11T00:37:35  <wumpus> it's interesting how with three people in a low-traffic IRC channel we're stlll managing to talk past each other half the time :-)
 52 2015-11-11T00:44:30  *** BashCo has quit IRC
 53 2015-11-11T00:45:56  <gmaxwell> Following this thinking, might be interesting to support fee-rate filtering for inv.
 54 2015-11-11T01:15:11  *** zooko has joined #bitcoin-core-dev
 55 2015-11-11T01:34:57  <gmaxwell> Bleh, three days of AFLing univalue on 48 cores and it's still finding new paths.
 56 2015-11-11T01:35:32  <gmaxwell> I hate parsers.
 57 2015-11-11T01:35:46  <sipa> haha
 58 2015-11-11T01:40:36  *** MetaTrading5 has joined #bitcoin-core-dev
 59 2015-11-11T01:58:27  *** lecusemble has quit IRC
 60 2015-11-11T02:07:31  <dcousens>  gmaxwell AFLing?
 61 2015-11-11T02:07:36  <dcousens> (guessing some kind of fuzzing)
 62 2015-11-11T02:10:02  *** lecusemble has joined #bitcoin-core-dev
 63 2015-11-11T02:10:34  <sipa> https://en.wikipedia.org/wiki/American_fuzzy_lop_%28fuzzer%29
 64 2015-11-11T02:10:43  <dcousens> ta
 65 2015-11-11T02:14:25  *** Ylbam has quit IRC
 66 2015-11-11T02:14:26  *** CodeShark has quit IRC
 67 2015-11-11T02:29:44  *** sipa has quit IRC
 68 2015-11-11T02:29:45  *** sipa has joined #bitcoin-core-dev
 69 2015-11-11T02:30:00  <gmaxwell> can someone kick metatrading5 ? he's PM spamming.
 70 2015-11-11T02:30:13  *** ChanServ sets mode: +o sipa
 71 2015-11-11T02:30:17  *** sipa sets mode: +b *!*MetaTradi@198.46.233.*
 72 2015-11-11T02:30:18  *** MetaTrading5 was kicked by sipa (MetaTrading5)
 73 2015-11-11T02:31:21  *** dcousens has quit IRC
 74 2015-11-11T03:07:15  *** tulip has joined #bitcoin-core-dev
 75 2015-11-11T03:17:52  *** tulip has quit IRC
 76 2015-11-11T03:19:32  *** tulip has joined #bitcoin-core-dev
 77 2015-11-11T03:43:51  <phantomcircuit> jtimon, there wouldn't happen to be a library somewhere of all the pure functions in bitcoin core would there?
 78 2015-11-11T03:51:55  *** tulip has quit IRC
 79 2015-11-11T03:52:42  *** tulip has joined #bitcoin-core-dev
 80 2015-11-11T04:50:13  *** tulip has quit IRC
 81 2015-11-11T05:00:46  *** gribble has quit IRC
 82 2015-11-11T05:08:01  *** NLNico has joined #bitcoin-core-dev
 83 2015-11-11T05:09:35  *** gribble has joined #bitcoin-core-dev
 84 2015-11-11T06:00:37  <GitHub119> [bitcoin] sipa opened pull request #6983: Update libsecp256k1 (master...secp256k1new) https://github.com/bitcoin/bitcoin/pull/6983
 85 2015-11-11T06:29:24  *** ParadoxSpiral has joined #bitcoin-core-dev
 86 2015-11-11T06:36:47  *** ParadoxSpiral has quit IRC
 87 2015-11-11T06:52:17  *** skyraider has quit IRC
 88 2015-11-11T07:03:10  <warren> Running Bitcoin Core v0.11.2rc1 with blank data dir on Windows 10 ... it connected to 9 peers including a local peer.  Stuck at  0 blocks for the past 10 minutes.  what should I try?
 89 2015-11-11T07:04:46  <sipa> warren: does it have any headers?
 90 2015-11-11T07:04:55  <sipa> warren: what does getchaintips RPC report?
 91 2015-11-11T07:05:44  <warren> ah, it's working now
 92 2015-11-11T07:38:16  <gmaxwell> wumpus: I guess you did more security review on miniupnpc? ... I went to review that update and I wanted to jump off a snprintf bridge. :)
 93 2015-11-11T07:42:26  *** gribble has quit IRC
 94 2015-11-11T07:43:47  *** dgenr8 has quit IRC
 95 2015-11-11T07:44:07  *** davec has quit IRC
 96 2015-11-11T07:44:13  *** dgenr8 has joined #bitcoin-core-dev
 97 2015-11-11T07:44:24  *** davec has joined #bitcoin-core-dev
 98 2015-11-11T07:45:09  *** Thireus has quit IRC
 99 2015-11-11T07:45:24  *** gribble has joined #bitcoin-core-dev
100 2015-11-11T07:48:57  *** Ylbam has joined #bitcoin-core-dev
101 2015-11-11T07:53:35  *** NLNico has quit IRC
102 2015-11-11T08:05:29  *** andytoshi has quit IRC
103 2015-11-11T08:06:18  *** NLNico has joined #bitcoin-core-dev
104 2015-11-11T08:13:03  *** NLNico has quit IRC
105 2015-11-11T08:13:04  *** andytoshi has joined #bitcoin-core-dev
106 2015-11-11T08:14:37  *** Thireus has joined #bitcoin-core-dev
107 2015-11-11T08:19:16  *** go1111111 has quit IRC
108 2015-11-11T08:22:26  *** go1111111 has joined #bitcoin-core-dev
109 2015-11-11T08:23:58  *** judomaster has joined #bitcoin-core-dev
110 2015-11-11T08:29:42  <wumpus> gmaxwell: heh it's things like that make me so fed up with C. in this case snprintf resturn specification is different per platform, so it could have been OK on windows, but on Linux it returns the length that the output would be without buffer limit. So if the application sends that many bytes to the network, and you control an arbitrary string input, you have a configurable memory contents leak :)
111 2015-11-11T08:30:14  *** NLNico has joined #bitcoin-core-dev
112 2015-11-11T08:30:44  <wumpus> gmaxwell: which happens to be very useful if you need  to know the stack cookie and function addresses
113 2015-11-11T08:34:45  *** CodeShark has joined #bitcoin-core-dev
114 2015-11-11T08:40:21  <Luke-Jr> wait what? snprintf doesn't do that on Windows? :/
115 2015-11-11T08:40:42  <midnightmagic> miniupnpd (the other side of the coin) I'm not sure is any better. in the event that a client requests an impermissible port, rather than say something useful it instead says that doing so would conflict with an already existing mapping (which doesn't exist)
116 2015-11-11T08:41:01  <sipa>        Concerning the return value of snprintf(), SUSv2 and C99 contradict each other: when snprintf() is called with size=0 then SUSv2 stipulates an unspecified return value less  than
117 2015-11-11T08:41:05  <sipa>        1,  while  C99  allows  str to be NULL in this case, and gives the return value (as always) as the number of characters that would have been written in case the output string has
118 2015-11-11T08:41:09  <sipa>        been large enough.
119 2015-11-11T08:41:42  <sipa> snprintf doesn't exist in C89
120 2015-11-11T08:42:33  <Luke-Jr> If len > count, count characters are stored in buffer, no null-terminator is appended, and a negative value is returned. <-- wtf Microsoft
121 2015-11-11T08:43:25  <sipa> That obeys the SUSv2 standard.
122 2015-11-11T08:43:27  <wumpus> Luke-Jr: not just the return value, in multiple ways it doesn't behave the same on all platforms. we had issues due to that was well, one of the reasons why bitcoin core switched to a typesafe library for formatting that doesn't rely on libc
123 2015-11-11T08:43:41  *** sipa sets mode: -o sipa
124 2015-11-11T08:44:49  <wumpus> Luke-Jr: anyhow as C programmer you're supposed to know all these trivia, and if not you'll create awful security holes, there is zero leeway for human error
125 2015-11-11T08:45:16  <Luke-Jr> If buffer is a null pointer and count is zero, len is returned as the count of characters required to format the output, not including the terminating null. To make a successful call with the same argument and locale parameters, allocate a buffer holding at least len + 1 characters.
126 2015-11-11T08:45:19  <Luke-Jr> hm
127 2015-11-11T08:45:50  <wumpus> yep.
128 2015-11-11T08:47:10  <sipa> Luke-Jr: the 'best practice' way to use snprintf is with a wrapper that checks the return value, and if larger than the buffer size, reallocate to exactly that size, and if not, double it... and then retry
129 2015-11-11T08:47:26  <Luke-Jr> "Also, some buggy older systems ignore the length and overrun the buffer (e.g., 64-bit Solaris 7)." -.-
130 2015-11-11T08:47:40  <wumpus> another pet peeve with *printf is how the formatting characters are different per platform, at least for not-so-obscure types like 64 bit integers, size_t's, etc...
131 2015-11-11T08:48:06  <sipa> or how putting type information in a string is just crazy
132 2015-11-11T08:48:09  <wumpus> and then there is a header which sould privide handy macros for that.. **but it doesn't exist on every platform**
133 2015-11-11T08:48:13  <wumpus> GRR...
134 2015-11-11T08:48:22  <Luke-Jr> wumpus: PRId64
135 2015-11-11T08:48:22  <wumpus> sipa: oh yes
136 2015-11-11T08:48:32  <wumpus> Luke-Jr: no need to tell me :p
137 2015-11-11T08:48:44  <Luke-Jr> what platform doesn't have it? :p
138 2015-11-11T08:48:51  <wumpus> Luke-Jr: I find those issues because my head is filled with nonsense like that
139 2015-11-11T08:49:20  <sipa> hey let's bypass the minimal type safety C provides, because converting any data to a string should be possible with a single function call
140 2015-11-11T08:51:23  <wumpus> sipa: yes and initiallly to an arbitrary-length string, buffer limiting was introduced later, hey because it's an afterthought right :-)
141 2015-11-11T08:52:28  <sipa> i think when these functions were designed, the programmer and the user were considered to be one, and if you're writing outside of your buffer, well then you're just stupid and making your computer do weird things, right?
142 2015-11-11T08:52:31  <wumpus> and then the format string vulnerabilities, in which someone accidentally passes a user-provided string to the first argument to *printf.  Those functions birthed whole classes of bugs.
143 2015-11-11T08:52:50  <Luke-Jr> heh
144 2015-11-11T08:53:10  <CodeShark> when these functions were designed they were just a convenience to avoid having to write directly to io devices in assembly language :p
145 2015-11-11T08:53:28  <wumpus> sipa: true, many of it was before the networked world. What does it matter if you crash your MSX home computer :)
146 2015-11-11T08:55:00  <Luke-Jr> oh well, is it really a security issue if I just say I require C99? :P
147 2015-11-11T08:55:14  <Luke-Jr> then it's Windows at fault, right? <.<
148 2015-11-11T08:55:57  <wumpus> Luke-Jr: then you exclude windows, which, now in 2015, still doesn't implement C99. I used to be angry about that, but now udnerstand they rightly have moved on from C :)
149 2015-11-11T08:56:30  <wumpus> s/windows/MS compiler/
150 2015-11-11T08:56:38  <CodeShark> type safety was a luxury back in the early 70's :p
151 2015-11-11T08:56:40  <Luke-Jr> I already excluded MS compiler <.<
152 2015-11-11T08:56:58  <Luke-Jr> I'm not sure MingW has a workaround for this snprintf thing though
153 2015-11-11T08:57:08  <wumpus> but be careful: mingw behaves cameleon-like, in some things it's like posix in some ways it's like MSVC
154 2015-11-11T08:57:17  <CodeShark> or even the late 70's for that matter :p
155 2015-11-11T08:57:41  <wumpus> Luke-Jr: it behaves in the windows way
156 2015-11-11T08:59:05  <wumpus> CodeShark: but all those ideas did originate from them, and other languages such as PASCAL were ahead of C in some ways
157 2015-11-11T08:59:17  <wumpus> s/them/the 70's/
158 2015-11-11T08:59:56  <CodeShark> true - PASCAL was far more heavily typed...if only it hadn't been so verbose :)
159 2015-11-11T09:00:00  <wumpus> e.g. ADA, another dinosaur, has very strict types and array bounds checking
160 2015-11-11T09:01:41  <CodeShark> ADA wasn't very practical for systems level programming, though
161 2015-11-11T09:02:09  <CodeShark> but it did introduce a number of very important paradigms
162 2015-11-11T09:02:30  <Luke-Jr> no, it looks like MingW is C99-compatible, at least with -std=c99
163 2015-11-11T09:03:42  <sipa> it links against msvc's runtime, no?
164 2015-11-11T09:03:44  <wumpus> right, I just mean that it is not the case that no one back then thought about security. Maybe it was the step from e.g. mainframes to home computers that lost a lot of knowledge. Suddenly doing things "fast" was more important than being robust :)
165 2015-11-11T09:04:00  <Luke-Jr> sipa: in some cases, it will override stuff
166 2015-11-11T09:04:45  <wumpus> Luke-Jr: please don't assume that affects strprintf
167 2015-11-11T09:05:06  <Luke-Jr> you mean snprintf?
168 2015-11-11T09:05:11  <wumpus> ye
169 2015-11-11T09:05:17  <Luke-Jr> wumpus: I checked the headers
170 2015-11-11T09:05:24  <Luke-Jr> it has an override *and* a comment explaining this
171 2015-11-11T09:05:49  <wumpus> ok. Well you checked. I just said 'don't assume' :)
172 2015-11-11T09:05:55  <Luke-Jr> ☺
173 2015-11-11T09:06:05  <wumpus> maybe add a test that makes sure
174 2015-11-11T09:06:22  <Luke-Jr> well, part of why I checked is, my code should have crashed badly if it got Windows behaviour :P
175 2015-11-11T09:14:08  <GitHub126> [bitcoin] jonasschnelli opened pull request #6984: don't enforce maxuploadtarget's disconnect for whitelisted peers (master...2015/11/maxupload_whitebind) https://github.com/bitcoin/bitcoin/pull/6984
176 2015-11-11T09:19:01  *** fanquake has joined #bitcoin-core-dev
177 2015-11-11T09:26:17  *** rubensayshi has joined #bitcoin-core-dev
178 2015-11-11T09:30:43  *** CodeShark has quit IRC
179 2015-11-11T09:32:24  *** judomaster has quit IRC
180 2015-11-11T09:54:59  <warren> I have a Windows 10 machine 80% sync'ed now, shut it down for now.
181 2015-11-11T09:55:07  <warren> wumpus: how did you simulate system crashes?
182 2015-11-11T09:55:21  <jonasschnelli> would this additional doc make sense? https://github.com/bitcoin/bitcoin/commit/f6b96ce09ec827bb9be0fd926b199009dd08bfd4
183 2015-11-11T09:55:23  <warren> will play with it tomorrow, g'nite
184 2015-11-11T09:55:31  <jonasschnelli> If yes, anyone up to "fix the english"?
185 2015-11-11T09:55:54  <wumpus> warren: NotMyFault is nice for that https://technet.microsoft.com/en-us/sysinternals/bb963901.aspx
186 2015-11-11T09:56:04  <fanquake> jonasschnelli Can do shortly
187 2015-11-11T09:56:52  <wumpus> it offers a menu of different crashes, e.g. from the kernel, from userspace, from I/O, and as a bonus you can even change the color of the BSOD :-)
188 2015-11-11T09:56:56  <jonasschnelli> cool. You could do a commit on top of my branch https://github.com/jonasschnelli/bitcoin/commits/2015/11/MarcoFalke-2015-maxupload
189 2015-11-11T09:57:20  <jonasschnelli> fanquake: than i can open a PR with both commits
190 2015-11-11T09:57:56  <jonasschnelli> fanquake: or if to complicated, just write comments. Whatever is best for you.
191 2015-11-11T09:58:12  <sipa> warren: set a large dbcache, and start closing bitcoin-qt right before the crash
192 2015-11-11T09:58:47  <sipa> warren: that will make sure leveldb is writing stuff out during the crash
193 2015-11-11T09:59:04  <fanquake> jonasschnelli ok
194 2015-11-11T10:00:12  <fanquake> wumpus pushed a commit to bump zeromq.
195 2015-11-11T10:00:38  <wumpus> thanks
196 2015-11-11T10:01:01  <wumpus> fanquake: thanks for the mail
197 2015-11-11T10:04:30  *** d4de has quit IRC
198 2015-11-11T10:07:06  <wumpus> fanquake: re: #1040 I still think we should move the manpage from contrib/debian to a place where it gets installed on 'make install', it's not just useful for debian
199 2015-11-11T10:09:33  <wumpus> .TH BITCOIND "1" "January 2011" "bitcoind 3.19"    ... hah, and the build tooling should fill those in automatically
200 2015-11-11T10:10:45  * sipa remembers the 100 Mbyte blockchain
201 2015-11-11T10:11:56  <wumpus> hehe, oh no, 100 MB, that must be the end, we'll never be able to handle a chain that big!
202 2015-11-11T10:12:09  <sipa> sorry, the 95.4 MiB blockchain
203 2015-11-11T10:12:43  <fanquake> *used to be able to cpu mine .1 btc a day*
204 2015-11-11T10:13:08  <fanquake> wumpus i can open a pull to amend some of the man page issue if you'd like
205 2015-11-11T10:13:21  <sipa> *used to be able to GPU mine 50 BTC a day*
206 2015-11-11T10:14:42  <fanquake> I hacked together a 5970 + 5830 to mine for a while. Missed the 50btc a day ship however.
207 2015-11-11T10:21:07  <wumpus> fanquake:It'd be nice to close that lnog-standing (but minor) issue
208 2015-11-11T10:23:50  * wumpus didn't even know we had a 'getaddednodeinfo' RPC call 
209 2015-11-11T10:24:20  <fanquake> heh, it's always interesting going over all the "older" issues
210 2015-11-11T10:26:39  <wumpus> especially nice if issues are solved not because someone actively went on solving the issue, but as by-product of another change
211 2015-11-11T10:27:12  <wumpus> (that's the positive counterpart of 'changes introduce unexpected issues', they also close unexpected issues sometimes!)
212 2015-11-11T10:29:57  <fanquake> jonasschnelli How did you go with the trusty gitian build?
213 2015-11-11T10:30:58  <wumpus> for https://github.com/bitcoin/bitcoin/issues//2368 "Consider reducing 'getblocks' limit below 500" I have no idea whether it's still relevant, but the limit is still hardcoded to 500
214 2015-11-11T10:33:42  <wumpus> apparently it never killed us, but the issue may be based on a misunderstanding? that getblocks returns block data, instead of INVs?
215 2015-11-11T10:35:04  <wumpus> fanquake: what vmbuilder are you using?
216 2015-11-11T10:35:48  <sipa> wumpus: in old code we sent getdatas instantly for all invs we received
217 2015-11-11T10:36:38  <wumpus> yes that seems to be the place to solve the issue, not in getblocks
218 2015-11-11T10:36:43  <wumpus> ok, closing then
219 2015-11-11T10:37:34  <fanquake> wumpus debian8 base, latest gitianbuilder, is that what you mean?
220 2015-11-11T10:38:02  <wumpus> fanquake: but on debian you have to build vmbuilder from source, right?
221 2015-11-11T10:38:47  <fanquake> ah sorry, I'm using vm-builder_0.12.4+bzr489
222 2015-11-11T10:38:52  <wumpus> (e.g. the one included wasn't new enough to support ubuntu, this was described in the doc/gitian-building.md)
223 2015-11-11T10:38:54  <fanquake> building from source yes
224 2015-11-11T10:39:58  <fanquake> Ok. I see there is a newer version available from http://archive.ubuntu.com/ubuntu/pool/universe/v/vm-builder/. I'll send a pull to update the gitian guide to use that version.
225 2015-11-11T10:40:37  <wumpus> mine (on Ubuntu 14.04) is also 0.12.4 though. But I suppose using a newer one may be worth a try, as it's failing while building the image.
226 2015-11-11T10:41:43  <fanquake> is yours 489 or 494 (latest)?
227 2015-11-11T10:42:56  <wumpus> building VM images is, unfortunately, still somewhat of a black art, needing root and support from the OS and loopback devices etc. The linux kernel library project https://github.com/lkl seems promising for the future as it'd simplify doing all the filesystems manipulation etc from userspace.
228 2015-11-11T10:44:33  <wumpus> Version: 0.12.4+bzr489-0ubuntu2
229 2015-11-11T10:45:13  <wumpus> " The currently supported distros are: * Ubuntu Dapper, Gutsy, Hardy, Intrepid, Jaunty, and Karmic." oh no... either they stopped updating the software, or they stopped updating the package description
230 2015-11-11T10:46:46  <GitHub21> [bitcoin] fanquake opened pull request #6985: [gitian] Use latest vm-builder ( 0.12.4+bzr494 ) on Debian (master...patch-3) https://github.com/bitcoin/bitcoin/pull/6985
231 2015-11-11T10:47:17  <wumpus> "apt-get python-vm-bulder" gets me 96 perl packages, boggles
232 2015-11-11T10:48:58  <wumpus> oh it gets all of libvirt, even the daemon... whyy
233 2015-11-11T10:50:37  <wumpus> anyhow what I was trying to find out: Ubuntu 15.10 also still has 0.12.4+bzr489-0ubuntu2, with the same package description. I vaguely remember the author saying they will do a new package for 16.04 tho, the code is almost-unmaintained and what they recommend is using the ready-made daily ubuntu cloud images
234 2015-11-11T10:50:56  <wumpus> (but that'd make deterministic verification really hard for us)
235 2015-11-11T10:53:27  <wumpus> anyway, it should be able to make 14.04/trusty images, I've used it for that (on Ubuntu) multiple times
236 2015-11-11T10:55:01  <fanquake> jonasschnelli You can grab the commit from https://github.com/fanquake/bitcoin/commits/fixupdoc
237 2015-11-11T10:55:31  <fanquake> I  cherrypicked you doc commit and added
238 2015-11-11T10:57:59  <fanquake> Another depends to consider bumping is qt. They've released a 5.5.1 bugfix, just looking through the notes to see if any of our issues might have been solved.
239 2015-11-11T10:58:11  <fanquake> Release notes https://wiki.qt.io/Change-files-in-Qt-5.5.1
240 2015-11-11T11:08:35  <fanquake> wumpus looks like the newer vmbuilder has fixed it. The build no longer errors out, and finishes successfully,
241 2015-11-11T11:11:41  <wumpus> awesome!
242 2015-11-11T11:13:05  <fanquake> You can pull the commit in #6985 into your PR if you want.
243 2015-11-11T11:13:28  <wumpus> I will, thanks
244 2015-11-11T11:14:05  *** randy-waterhouse has quit IRC
245 2015-11-11T11:15:43  <wumpus> oh nm, you already filed a pull, I'll just merge that - it's sort of independent of using a new release for building
246 2015-11-11T11:16:46  <jonasschnelli> fanquake: thanks! Will have a look soon.
247 2015-11-11T11:20:44  <GitHub144> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/de7d4591a7ce...38a4f267e82a
248 2015-11-11T11:20:44  <GitHub144> bitcoin/master 7791395 Michael: [gitian] Use vm-builder_0.12.4+bzr494 on Debian
249 2015-11-11T11:20:45  <GitHub144> bitcoin/master 38a4f26 Wladimir J. van der Laan: Merge pull request #6985...
250 2015-11-11T11:20:54  <GitHub105> [bitcoin] laanwj closed pull request #6985: [gitian] Use latest vm-builder ( 0.12.4+bzr494 ) on Debian (master...patch-3) https://github.com/bitcoin/bitcoin/pull/6985
251 2015-11-11T11:22:12  <wumpus> (btw apart from being a bad fit for the gitian process, the pre-built cloud images are awfully convenient - ready-built qcow2 images that can be configured in a few seconds - only remining timesink is apt-get install of toolchains)
252 2015-11-11T11:23:54  <GitHub113> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/38a4f267e82a...cb841e75135e
253 2015-11-11T11:23:54  <GitHub113> bitcoin/master 7267843 Jorge Timón: Globals: Make AcceptBlockHeader static (Fix #6163)...
254 2015-11-11T11:23:55  <GitHub113> bitcoin/master cb841e7 Wladimir J. van der Laan: Merge pull request #6982...
255 2015-11-11T11:23:59  <GitHub58> [bitcoin] laanwj closed pull request #6982: Globals: Make AcceptBlockHeader static (Fix #6163) (AcceptBlockHeader) (master...fix-6163) https://github.com/bitcoin/bitcoin/pull/6982
256 2015-11-11T11:29:01  <wumpus> fanquake: re: https://wiki.qt.io/Change-files-in-Qt-5.5.1   doesn't even seem complete yet, I clicked 'qtbase' and it complains about an invalid branch :-)
257 2015-11-11T11:29:10  *** Guyver2 has joined #bitcoin-core-dev
258 2015-11-11T11:34:20  <fanquake> wumpus yea, you have to click on changes-5.5.1 up the top, and you should get to http://code.qt.io/cgit/qt/qtbase.git/tree/dist/changes-5.5.1
259 2015-11-11T11:34:42  <fanquake> Not sure why it doesn't just redirect straight there though.
260 2015-11-11T11:37:58  <wumpus> likely very fresh
261 2015-11-11T11:41:09  * wumpus is hopeful about the lack of complaints about 0.11.2rc1/0.10.4rc1
262 2015-11-11T11:41:47  <fanquake> *hopes that doesn't just reflect a lack of testing :o"
263 2015-11-11T11:41:58  <wumpus> sshhh
264 2015-11-11T11:42:12  <fanquake> Does someone have download counts for the RCs?
265 2015-11-11T11:42:25  <fanquake> Would be interesting to see the numbers
266 2015-11-11T11:45:07  <wumpus> I don't have access to the logs I think
267 2015-11-11T11:55:09  <GitHub40> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/cb841e75135e...3ac70609345a
268 2015-11-11T11:55:10  <GitHub40> bitcoin/master 17c4d9d Cory Fields: build: Split hardening/fPIE options out...
269 2015-11-11T11:55:10  <GitHub40> bitcoin/master 69d0513 Cory Fields: build: Use fPIC rather than fPIE for qt objects....
270 2015-11-11T11:55:11  <GitHub40> bitcoin/master 3ac7060 Wladimir J. van der Laan: Merge pull request #6978...
271 2015-11-11T11:55:19  <GitHub127> [bitcoin] laanwj closed pull request #6978: Alternative fix for #6248 (qt+fPIE) (master...qt-pie) https://github.com/bitcoin/bitcoin/pull/6978
272 2015-11-11T12:03:50  *** evoskuil has quit IRC
273 2015-11-11T12:19:52  <jtimon> phantomcircuit: a library with all the pure functions in bitcoin? what do you mean by pure functions?
274 2015-11-11T12:21:35  <phantomcircuit> jtimon, no side effects, ie inputs -> outputs no state modified
275 2015-11-11T12:29:57  <GitHub104> [bitcoin] jtimon opened pull request #6986: Globals: Don't call Params() from miner.cpp (master...global-chainparams-miner) https://github.com/bitcoin/bitcoin/pull/6986
276 2015-11-11T12:33:22  <GitHub19> [bitcoin] jonasschnelli opened pull request #6987: [doc] add documentation about reducing traffic (master...2015/11/doc_traffic) https://github.com/bitcoin/bitcoin/pull/6987
277 2015-11-11T12:34:44  <jtimon> phantomcircuit: oh, I see, cfields and I have been calling that "library ready" code. libbitcoinconsensus is obviously library ready and at a higher level there's some more that will be required for a complete libconsensus but not that many
278 2015-11-11T12:35:30  <jtimon> phantomcircuit: I mean, wait, no "library ready" is even more strict since it doesn't even use the state as input
279 2015-11-11T12:36:54  <jtimon> for example, checkTransaction checkBlockHeader and checkBlock, those are almost ready apart from uses of error(), Params() and other things that are trivial to solve (if, eventually, one day, we want to do that)
280 2015-11-11T12:38:17  <jtimon> phantomcircuit: anyway I don't know of any library with functions from bitcoin core besides libbitcoinconsensus with its VerifyScript()
281 2015-11-11T12:38:18  <wumpus> as long as all inputs and outputs are passed to/from the function explicitly, it qualifies as 'pure function'. This also means no i/o!
282 2015-11-11T12:40:36  <wumpus> (although ofc. people in functional languages like to cheat by passing 'the state of the world' as input and as output of a function, so that i/o can exist in their 'pure' world :-) )
283 2015-11-11T12:42:34  <jtimon> wumpus: well, ContextualCheckBlock, for example, doesn't modify the state, but it takes stateful structures explicitly as parameter CBlockIndex* (containing a pointer to a chain index) and a separated VerifyBlock would currently also take at least a CCoinsViewCache like ConnectBlock does. It may be a pure function, but it's not ready for libconsensus
284 2015-11-11T12:46:32  <jtimon> phantomcircuit: anyway, I'm curious about the question, what motivated you to ask me that?
285 2015-11-11T12:48:36  <wumpus> probably refering to the RPC calls that are (either truly or effectively) pure functions
286 2015-11-11T12:49:12  <wumpus> parts of the raw tx API, validateaddress (if no wallet), etc
287 2015-11-11T12:51:04  <phantomcircuit> jtimon, looking for easy things to fuzz :)
288 2015-11-11T12:51:38  <jtimon> oh, I see
289 2015-11-11T12:52:27  <jtimon> phantomcircuit: so the name of the functions is enough for you, right?
290 2015-11-11T12:54:31  <jtimon> as said just from the top of my head checkTransaction checkBlockHeader and checkBlock come to mind (although they are quite simple anyway), probably wumpus has more names "at hand"
291 2015-11-11T12:55:15  *** MarcoFalke has joined #bitcoin-core-dev
292 2015-11-11T12:56:26  <wumpus> off the top of my head: everyting in utilstrencodings.h, base58 parsing/generation, serialization/deserialization primitives, uint256 and other encapsulated classes
293 2015-11-11T12:56:41  <jtimon> in fact we could easily expose those 3 in libbitcoinconsensus right now (assuming we want to)
294 2015-11-11T12:57:30  <wumpus> everything in src/crypto
295 2015-11-11T12:58:10  <jtimon> mhmm, I think base58 is still coupled with the global Params() for Base58Prefix ...
296 2015-11-11T12:58:35  <wumpus> CScript, CScriptCompressor etc
297 2015-11-11T12:58:42  <jtimon> everything in current libbitcoinconsensus includes everything in src/crypto
298 2015-11-11T12:58:43  <wumpus> jtimon: read-only though, that doesn't matter for fuzzing
299 2015-11-11T12:59:01  <jtimon> wumpus: gotcha
300 2015-11-11T12:59:51  <wumpus> essentailly you want functions that are idempotent, not necessarily pure
301 2015-11-11T13:06:33  * wumpus thinks it would be nice if someone collected release notes "Notable changes" into an organized documentation, people go through the trouble of documenting things for release notes , but after a release it's essentially forgotten
302 2015-11-11T13:17:11  *** MarcoFalke has quit IRC
303 2015-11-11T13:17:32  <wumpus> should probably add the part about torcontrol to doc/tor.md as well
304 2015-11-11T13:19:08  *** BashCo has joined #bitcoin-core-dev
305 2015-11-11T13:20:52  <jtimon> wumpus: if #6625 was merged right now it wouldn't compile because transaction_tests.cpp now requires "#include "main.h" // For minRelayTxFee and DEFAULT_MIN_RELAY_TX_FEE"
306 2015-11-11T13:21:54  <jtimon> wumpus: can I push the rebased and fixed version or preemptively fix it in the old version without rebasing?
307 2015-11-11T13:22:11  <jtimon> s/can/should
308 2015-11-11T13:24:28  <jtimon> btw gmaxwell a re-"gmaxwell was here" on #6625 (a subset of #6672 without libconsensus related movements) would be appreciated
309 2015-11-11T13:25:37  <jtimon> after this little fix
310 2015-11-11T13:26:09  <harding> wumpus: you mean just extracting the "Notable changes" sections from all past releases into a single document hosted somewhere?
311 2015-11-11T13:28:25  <jtimon> wumpus: gmaxwell: never mind there's also new uses of MAX_BLOCK_SIZE in net.cpp so I have to rebase and fix more things
312 2015-11-11T13:36:22  *** jgarzik has quit IRC
313 2015-11-11T13:43:53  <jtimon> wumpus: oh, no, that was what I already fixed but didn't squashed, so my question remains, rebase and fix or preemptively fix ?
314 2015-11-11T13:53:35  <wumpus> harding: not literally - I just meant, it's easy to lose track of documentation that was written just for the release notes
315 2015-11-11T13:54:36  <wumpus> harding: but e.g. jonasschelli is doing the right thing in #6987 by also creating a file in doc
316 2015-11-11T13:58:36  <jonasschnelli> harding: https://github.com/bitcoin/bitcoin/pull/6987 could also be something for bitcoin.org?
317 2015-11-11T13:59:28  <wumpus> yes, was about to mention that, there is also overlap with the documentation on bitcoin.org (att least the bitcoin core section)
318 2015-11-11T14:01:26  <harding> wumpus: ah.  We do link to the Bitcoin Core documentation directory ( https://bitcoin.org/en/bitcoin-core/help#documentation ), although not prominently.
319 2015-11-11T14:01:45  <harding> jonasschnelli: I do think that we could intergrate 6987 to useful effect into https://bitcoin.org/en/full-node
320 2015-11-11T14:02:17  *** rubensayshi_ has joined #bitcoin-core-dev
321 2015-11-11T14:02:49  <wumpus> jtimon: I'm not sure - what is the problem to be fixed?
322 2015-11-11T14:05:08  <jtimon> wumpus previously main.h was only included for CheckTransaction so I removed it in https://github.com/jtimon/bitcoin/commit/7ec074dca2ec60d7f1c0711be31d165121ead600#diff-4f6d60b6976522cec2974a0aea9a5ab3L14 but in the future (rebased version) it will also be needed for minRelayTxFee and DEFAULT_MIN_RELAY_TX_FEE and cannot be removed (just change the comment instead)
323 2015-11-11T14:07:16  *** pigeons has quit IRC
324 2015-11-11T14:07:24  *** pigeons has joined #bitcoin-core-dev
325 2015-11-11T14:07:25  *** gribble has quit IRC
326 2015-11-11T14:07:47  *** pigeons is now known as Guest78100
327 2015-11-11T14:08:05  <jtimon> eventually (assuming minRelayTxFee and DEFAULT_MIN_RELAY_TX_FEE ever move to the policy dir) we will be able to remove that main.h include, but not just yet
328 2015-11-11T14:08:15  *** rubensayshi has quit IRC
329 2015-11-11T14:08:16  *** NLNico has quit IRC
330 2015-11-11T14:08:16  *** Ylbam has quit IRC
331 2015-11-11T14:09:46  <wumpus> ok... well if you need the include then keep it
332 2015-11-11T14:10:58  <wumpus> I mean this in a test, anything goes :)
333 2015-11-11T14:11:07  <jtimon> yeah, the question is whether to rebase or not, the problem is only visible if I rebase
334 2015-11-11T14:13:06  <jtimon> I mean I have already rebased, for me it's simpler to just push the rebased version, but it's worse for reviewers. I can easily "backport" the fix but it's kind of weird to have that  "// For minRelayTxFee and DEFAULT_MIN_RELAY_TX_FEE" comment without rebasing...
335 2015-11-11T14:15:57  *** NLNico has joined #bitcoin-core-dev
336 2015-11-11T14:15:58  *** Ylbam has joined #bitcoin-core-dev
337 2015-11-11T14:20:25  *** jgarzik has joined #bitcoin-core-dev
338 2015-11-11T14:21:43  *** gribble has joined #bitcoin-core-dev
339 2015-11-11T14:28:40  <jtimon> wumpus: I had to rebase anyway because both #6382 and #5970 depend on #6625 (in fact, that's how I discovered the problem)
340 2015-11-11T14:33:15  <wumpus> ok, yes rebasing makes sense then
341 2015-11-11T14:33:28  <jtimon> wumpus: ok, thanks
342 2015-11-11T14:46:08  <GitHub18> [bitcoin] MarcoFalke opened pull request #6988: [doc] Fix FIXME for libblkmaker in release-notes.md (master...MarcoFalke-2015-docFixme) https://github.com/bitcoin/bitcoin/pull/6988
343 2015-11-11T15:08:16  *** jgarzik has quit IRC
344 2015-11-11T15:09:08  *** jgarzik has joined #bitcoin-core-dev
345 2015-11-11T15:10:14  *** skyraider has joined #bitcoin-core-dev
346 2015-11-11T15:12:57  *** zooko has quit IRC
347 2015-11-11T15:25:26  *** zooko has joined #bitcoin-core-dev
348 2015-11-11T15:42:39  *** ParadoxSpiral has joined #bitcoin-core-dev
349 2015-11-11T16:02:41  *** bsm1175321 has quit IRC
350 2015-11-11T16:03:07  *** Thireus has quit IRC
351 2015-11-11T16:04:35  <morcos> phantomcircuit: ping, want to understand your privacy concerns with respect to wallet txs in mempool
352 2015-11-11T16:05:14  <morcos> is the idea that you don't want people to be able to go around polling nodes trying to get them to respond to a getdata for a tx that should have been evicted
353 2015-11-11T16:05:50  <morcos> and if they responded, they have a good probability of being the tx author?
354 2015-11-11T16:07:36  <morcos> i don't think i was suggesting changing that, but I do think it is important for users to know that their tx may no longer be in mempools.  it'll be an indication to them that they may want to rebroadcast it or RBF it.
355 2015-11-11T16:08:23  <morcos> so perhaps the mempool needs to let the wallet know it evicted or something similar and the wallet can inform hte user
356 2015-11-11T16:11:51  *** bsm1175321 has joined #bitcoin-core-dev
357 2015-11-11T16:12:10  <phantomcircuit> morcos, wallet should poll the mempool if anything
358 2015-11-11T16:12:39  <phantomcircuit> the potential to leak which transactions are in the local wallet is pretty high even if we're careful
359 2015-11-11T16:12:54  <sipa> it already does, and reports 'conflicted' for those transactions, afaikM
360 2015-11-11T16:13:14  <morcos> sipa: yes, but it would be nice if it could identify the reason the tx is no longer in the mempool
361 2015-11-11T16:13:26  <morcos> how to know, hey it got evicted
362 2015-11-11T16:13:26  <sipa> right
363 2015-11-11T16:13:46  *** skyraider_ has joined #bitcoin-core-dev
364 2015-11-11T16:15:33  <phantomcircuit> sipa, iirc the mempool notifies the wallet of all changes, right?
365 2015-11-11T16:15:56  *** BashCo has quit IRC
366 2015-11-11T16:16:10  *** wump has joined #bitcoin-core-dev
367 2015-11-11T16:16:26  <morcos> during tomorrows meeting we should try to game plan an order to review some of these open pulls we'd like to get in for 0.12.  I think if we put the effort in we can have a much better and more complete/consistent release.  There are quite a few things that are half way between fixes and new features that in my opinion really should be in the release, so we don't get complaints like this about conflicted txs
368 2015-11-11T16:16:58  <jgarzik> phantomcircuit, sipa: how/where is the mempool/wallet connection made, now?   signals?
369 2015-11-11T16:17:11  *** zooko has quit IRC
370 2015-11-11T16:17:11  *** skyraider has quit IRC
371 2015-11-11T16:17:12  *** isis has quit IRC
372 2015-11-11T16:17:13  *** wumpus has quit IRC
373 2015-11-11T16:17:14  <jgarzik> That was one area of open question in the mempool-janitor stuff from year+ ago.
374 2015-11-11T16:18:11  *** skyraider_ is now known as skyraider
375 2015-11-11T16:18:18  <morcos> there might be a notification via signal on ATMP, but i dont' think there is any way to inform of eviction is there?
376 2015-11-11T16:18:20  *** wump is now known as wumpus
377 2015-11-11T16:18:36  <sipa> my opinion back then was that mempool transactions from the wallet should be protected from eviction; i don't think that's a good idea anymore due to the privacy concerns that raises
378 2015-11-11T16:19:26  *** isis has joined #bitcoin-core-dev
379 2015-11-11T16:19:45  <morcos> yeah SyncWithWallets signals the wallet on txs entering the mempool or removing it for being mined or removing it for being conflicted
380 2015-11-11T16:20:05  <sipa> that was more applicable to randon eviction though, as now the mempool aims to be some sort of subset of most likely to be mined transactions
381 2015-11-11T16:20:23  <morcos> perhaps we can call it within ATMP for evicted txs?
382 2015-11-11T16:20:56  <sipa> and if your wallet tx is not in that, it probably means those transactions are effectively at risk of not confirming
383 2015-11-11T16:22:08  *** go1111111 has quit IRC
384 2015-11-11T16:23:24  <phantomcircuit> sipa, even with random eviction it's a bad idea...
385 2015-11-11T16:23:42  <phantomcircuit> especially when combined with "mempool"
386 2015-11-11T16:24:07  <sipa> phantomcircuit: the privacy concern is always there, but with random eviction, being evicted does not have any meaning for the wallet user
387 2015-11-11T16:24:10  <sipa> now it does
388 2015-11-11T16:24:33  *** go1111111 has joined #bitcoin-core-dev
389 2015-11-11T16:25:24  <morcos> anyway, looks a bit messy to add it, but this is the kind of thing that I think is worth trying to get right before a major release
390 2015-11-11T16:25:45  <sipa> perhaps the criterion for 'conflicted' should actually be that a conficting transaction is found; if not we can just call it "unclear" or something
391 2015-11-11T16:27:28  <jgarzik> RE meeting - I would like two buckets for 0.12 - "must have" and "really want to have, if it's ready in time, but not a release blocker"
392 2015-11-11T16:27:50  <jgarzik> too easy to continue slipping a release for one more awesome change
393 2015-11-11T16:28:20  <jgarzik> time-based releases are different from feature releases - they are more lumpy
394 2015-11-11T16:28:36  <sipa> the must haves should probably mostly be "fix known regressions"
395 2015-11-11T16:28:52  <sipa> the chain limits and wallet/mempool interactions probably belong there
396 2015-11-11T16:30:13  <jgarzik> +1
397 2015-11-11T16:30:25  <morcos> Yes, thats why we should use the next 2 weeks to get in all the really want to haves!  and then we can worry about the must haves after.. :)
398 2015-11-11T16:30:59  <jgarzik> ;p
399 2015-11-11T16:31:16  <morcos> I will be sad if we don't merge a good subset of the performance improvements.  But even just for those, its useful to decide on an order to knock them out, since they interact to some degree.
400 2015-11-11T16:32:59  <sipa> morcos: i hope a large part of them (if not all) go in
401 2015-11-11T16:37:07  *** Thireus has joined #bitcoin-core-dev
402 2015-11-11T16:37:13  *** BashCo has joined #bitcoin-core-dev
403 2015-11-11T16:42:32  *** randy-waterhouse has joined #bitcoin-core-dev
404 2015-11-11T16:43:14  *** zooko has joined #bitcoin-core-dev
405 2015-11-11T16:49:59  <wumpus> morcos: I think it's important to make a distinction between fixes that need to get in and performance improvements that would be nice, but may also be risky
406 2015-11-11T16:51:59  <morcos> wumpus: yep agreed, i'm just hoping that if we work together we'll be able to get a lot in before the deadline sneaks up on us (says the guy that needs to start reviewing instead of creating more things to be reviewed)
407 2015-11-11T16:52:32  <wumpus> agreed
408 2015-11-11T16:52:46  <wumpus> just thinking that it's important to prioritize, there is so much open
409 2015-11-11T16:52:51  <sipa> morcos: i am right to say that #5967 (if it works as intended) would fix the problem of ModifyNewCoins being unusable for coinbases?
410 2015-11-11T16:53:49  <morcos> sipa: i was wondering about that and it was on my list to go back and check it
411 2015-11-11T16:55:08  <sipa> it explicitly creates the case where a child cache has a non-fresh entry for something the parent cache does not have, in its tests
412 2015-11-11T16:55:19  <morcos> entirely possible, but i want to carefully think about combining 5967 with modify new coins, probably worth doing even if its not going to be merged as just another way of making sure the thinking is correct
413 2015-11-11T16:55:46  <morcos> yeah that sounds right
414 2015-11-11T17:05:52  <sipa> morcos: writing a new coin to a child cache while the parent does not have it but the grandparent does is identical to the cache where the parent had it when the coin was written, and then had the parent flushed
415 2015-11-11T17:06:25  <sipa> so i believe that 5967 exactly covers the unnecessary assumption, and tests it
416 2015-11-11T17:06:43  <morcos> sipa: before i try to parse your statement, thinking about 5967 made me realize removing the assert without 5967 would have been broken with the ModifyNewCoins
417 2015-11-11T17:07:35  <morcos> sipa: ok yes i agree with your statement and i think 5967 would allow ModifyNewCoins to be used in both cases and just not flag coinbases as FRESH
418 2015-11-11T17:08:34  <sipa> morcos: you would have caught it by adding ModifyNewCoin calls in the unit tests
419 2015-11-11T17:08:57  <morcos> sipa: ha ha. exactly!
420 2015-11-11T17:09:44  <morcos> ok so do you want to do 5967 as well then?
421 2015-11-11T17:10:02  <morcos> I will perhaps put them all in one pull so it can be tested all together (once i write the unit tests)
422 2015-11-11T17:10:20  <sipa> morcos: i'd say first do ModifyNewCoins as-is, adding unit tests for it
423 2015-11-11T17:10:57  <sipa> morcos: then just removing the assert + ModifyNewCoins for coinbases without fresh flag should fa
424 2015-11-11T17:11:21  <sipa> fail
425 2015-11-11T17:11:38  *** gribble has quit IRC
426 2015-11-11T17:11:47  <morcos> sure, ok that makes sense.  separate PR can add 5967 and change ModifyNewCoins, and that can be added or not later
427 2015-11-11T17:12:28  <GitHub96> [bitcoin] laanwj opened pull request #6990: http: speed up shutdown (master...2015_11_threadexit) https://github.com/bitcoin/bitcoin/pull/6990
428 2015-11-11T17:15:02  <GitHub188> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/3ac70609345a...44d7b561bc75
429 2015-11-11T17:15:02  <GitHub188> bitcoin/master 0817bf7 MarcoFalke: [doc] Fix FIXME for libblkmaker in release-notes.md
430 2015-11-11T17:15:03  <GitHub188> bitcoin/master 44d7b56 Wladimir J. van der Laan: Merge pull request #6988...
431 2015-11-11T17:15:07  <GitHub175> [bitcoin] laanwj closed pull request #6988: [doc] Fix FIXME for libblkmaker in release-notes.md (master...MarcoFalke-2015-docFixme) https://github.com/bitcoin/bitcoin/pull/6988
432 2015-11-11T17:18:33  <sipa> morcos: realizing that you could have introduced a bug is a good reason to add a test that would detect it :)
433 2015-11-11T17:22:07  <GitHub109> [bitcoin] laanwj closed pull request #6300: getblockhash: Return RPC_NOT_FOUND when nHeight > chainActive.Height() (master...master) https://github.com/bitcoin/bitcoin/pull/6300
434 2015-11-11T17:22:56  *** gribble has joined #bitcoin-core-dev
435 2015-11-11T17:27:56  <GitHub144> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/44d7b561bc75...c9743062a37c
436 2015-11-11T17:27:57  <GitHub144> bitcoin/master 6a776fa Jonas Schnelli: [doc] add documentation how to reduce traffic
437 2015-11-11T17:27:57  <GitHub144> bitcoin/master 42bae94 fanquake: [doc] Improve lanaguge in reducetraffic.md
438 2015-11-11T17:27:58  <GitHub144> bitcoin/master c974306 Wladimir J. van der Laan: Merge pull request #6987...
439 2015-11-11T17:28:06  <GitHub82> [bitcoin] laanwj closed pull request #6987: [doc] add documentation about reducing traffic (master...2015/11/doc_traffic) https://github.com/bitcoin/bitcoin/pull/6987
440 2015-11-11T17:30:37  <GitHub59> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/c9743062a37c...2f71b07d5ed6
441 2015-11-11T17:30:37  <GitHub59> bitcoin/master 40b77d4 Wladimir J. van der Laan: Always allow getheaders from whitelisted peers...
442 2015-11-11T17:30:38  <GitHub59> bitcoin/master 2f71b07 Wladimir J. van der Laan: Merge pull request #6974...
443 2015-11-11T17:30:47  <GitHub183> [bitcoin] laanwj closed pull request #6974: Always allow getheaders from whitelisted peers (master...2015_11_whitelisted_allow_headers) https://github.com/bitcoin/bitcoin/pull/6974
444 2015-11-11T17:35:20  <morcos> gmaxwell: how much do you care about this effectiveSize = max(size/MAX_BLOCK_SIZE,sigOps/MAX_BLOCK_SIGOPS) thing?
445 2015-11-11T17:36:20  <morcos> its not that difficult to implement, but it touches a lot of things, assuming we make descendant packages track it which i think is important.  its too risky to make the default mining prioritization differ a lot from the eviction code.
446 2015-11-11T17:37:57  <morcos> but then does it also affect package limits, are they 101kbs of actual size or 101kbs of effective size?  etc..   and do you sum that by summing maxes, etc. etc..   all doable, but i think i need to work on cleaning up these other pulls first
447 2015-11-11T17:39:45  *** trippysalmon has joined #bitcoin-core-dev
448 2015-11-11T17:39:51  *** rubensayshi_ has quit IRC
449 2015-11-11T17:44:38  <GitHub2> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/2f71b07d5ed6...cbf9609c71c6
450 2015-11-11T17:44:38  <GitHub2> bitcoin/master 7085728 Wladimir J. van der Laan: doc: there is no libboost-base-dev, add missing sudo...
451 2015-11-11T17:44:39  <GitHub2> bitcoin/master cbf9609 Wladimir J. van der Laan: Merge pull request #6969...
452 2015-11-11T17:44:43  <GitHub167> [bitcoin] laanwj closed pull request #6969: doc: there is no libboost-base-dev, add missing sudo in release notes (master...2015_11_docfix) https://github.com/bitcoin/bitcoin/pull/6969
453 2015-11-11T17:57:05  <morcos> sipa: any reason why the BatchWrite for CCoinsViewTest writes all entries and not just the dirty ones?  I think it should only write DIRTY ones.. I'm not sure what the right way to keep it in sync with what CCoinsViewDB does, but that makes the most sense to me.
454 2015-11-11T17:58:11  <sipa> it only writes dirty ones afaik?
455 2015-11-11T17:58:30  <sipa> ah, the test
456 2015-11-11T17:58:35  <sipa> i missed that
457 2015-11-11T18:00:25  <morcos> ok, i'm changing it to follow same logic as CCoinsViewDB...
458 2015-11-11T18:00:29  <gmaxwell> morcos: I don't care about the specific thing done, but something needs to be done about the sigops flooding attack. The obvious next alternative is to threshold it, but I'm doubtful a threshold can be set that won't randomly bite people with lots of p2pkh outputs.
459 2015-11-11T18:05:41  *** jl2012 has quit IRC
460 2015-11-11T18:12:31  *** davec has quit IRC
461 2015-11-11T18:19:55  *** trippysalmon has quit IRC
462 2015-11-11T18:28:28  *** NLNico has quit IRC
463 2015-11-11T18:34:43  *** jl2012 has joined #bitcoin-core-dev
464 2015-11-11T18:55:32  <GitHub3> [bitcoin] petertodd opened pull request #6991: Minor: Clarify 'fee' field in fundrawtransaction help text (master...clarify-fundrawtransaction-help) https://github.com/bitcoin/bitcoin/pull/6991
465 2015-11-11T19:03:59  <jgarzik> btcdrak, release manager != project lead...    bitcoin core is intentionally decentralized ever since Satoshi turned things over to Gavin.
466 2015-11-11T19:04:26  <jgarzik> I've been release manager before, for example.
467 2015-11-11T19:08:21  *** onetime has joined #bitcoin-core-dev
468 2015-11-11T19:15:02  *** Arnavion has quit IRC
469 2015-11-11T19:16:30  *** Arnavion has joined #bitcoin-core-dev
470 2015-11-11T19:16:35  *** gribble has quit IRC
471 2015-11-11T19:21:35  *** onetime has quit IRC
472 2015-11-11T19:23:46  *** gribble has joined #bitcoin-core-dev
473 2015-11-11T19:28:20  *** davec has joined #bitcoin-core-dev
474 2015-11-11T19:39:10  *** JackH has quit IRC
475 2015-11-11T20:28:29  <GitHub131> [bitcoin] sdaftuar opened pull request #6992: [WIP] [Mempool] Add space for priority transactions (master...add-priority-to-mempool) https://github.com/bitcoin/bitcoin/pull/6992
476 2015-11-11T20:42:20  *** jgarzik has quit IRC
477 2015-11-11T20:50:47  *** jgarzik has joined #bitcoin-core-dev
478 2015-11-11T20:50:57  *** jgarzik has quit IRC
479 2015-11-11T21:06:19  *** jgarzik has joined #bitcoin-core-dev
480 2015-11-11T21:06:19  *** jgarzik has joined #bitcoin-core-dev
481 2015-11-11T21:29:08  *** dcousens has joined #bitcoin-core-dev
482 2015-11-11T21:49:52  *** jgarzik has quit IRC
483 2015-11-11T22:18:19  <michagogo> 12:50:56 <wumpus> (but that'd make deterministic verification really hard for us) <-- How come? AFAIK it doesn't matter if you use a daily image or a release image or debootstrap or whatever, as soon as you upgrade it should all be the same...
484 2015-11-11T22:18:48  <michagogo> I mean, it's not like vmbuilder is deterministic, IIRC
485 2015-11-11T22:21:39  *** gribble has quit IRC
486 2015-11-11T22:34:34  *** Guyver2 has quit IRC
487 2015-11-11T22:35:08  *** evoskuil has joined #bitcoin-core-dev
488 2015-11-11T22:38:58  *** gribble has joined #bitcoin-core-dev
489 2015-11-11T22:45:57  <gmaxwell> michagogo: you're suggesting using daily images for buids? if so thats a terrible idea, as it would make it much easier to compromise everyone's builder images.
490 2015-11-11T22:46:16  <michagogo> gmaxwell: I wasn't, no
491 2015-11-11T22:47:01  <michagogo> 12:50:37 <wumpus> anyhow what I was trying to find out: Ubuntu 15.10 also still has 0.12.4+bzr489-0ubuntu2, with the same package description. I vaguely remember the author saying they will do a new package for 16.04 tho, the code is almost-unmaintained and what they recommend is using the ready-made daily ubuntu cloud images
492 2015-11-11T22:48:25  <michagogo> gmaxwell: also, I don't think the idea was "use today's image each time you build", I think it was "use today's image when you want to create the base VM"
493 2015-11-11T22:49:49  <michagogo> So I wasn't the one bringing it up, but I don't think I really see any reason not to, as I said above
494 2015-11-11T22:51:34  <michagogo> (Of course, I might just be missing something obvious)
495 2015-11-11T23:05:50  *** ParadoxSpiral has quit IRC
496 2015-11-11T23:07:10  <jtimon> dcousens: hopefully you like #6597 now, it seems I coded that too fast and it was hurting performance for no good reason
497 2015-11-11T23:13:21  <dcousens> jtimon: nice
498 2015-11-11T23:14:14  <dcousens> jtimon: I'm still 100% over the implications of the change (will keep running through it in my head) but, the code looks good.
499 2015-11-11T23:17:13  <jtimon> dcousens: I think the change you chosed as an example doesn't actually hurt performance since non-genesis block were comparing the hashes anyway (just after checking the pow instead of before), but there were other places (which I'm not sure I had to touch even before rebasing and testing #6382 again) where your arguments certainly applied
500 2015-11-11T23:17:49  <dcousens> jtimon: indeed, it compares the hash right after, but, this is cleaner :)
501 2015-11-11T23:17:56  <dcousens> and at least, we only do it once now
502 2015-11-11T23:18:03  <dcousens> well, less*
503 2015-11-11T23:18:56  <jtimon> and it's when throwing an error...thanks for the review!
504 2015-11-11T23:20:27  <gmaxwell> sipa: it was the right thing to do, but I think splitting the libsecp256k1 PR now means that no one is going to comment on the first part. :-/
505 2015-11-11T23:21:11  <jtimon> gmaxwell: I'm afraid I can just concept ack on the first part
506 2015-11-11T23:21:55  <sipa> gmaxwell: by first part you mean "the earlier PR", or "the PR whose commits now go first" ?
507 2015-11-11T23:23:32  <jtimon> I meant #6983
508 2015-11-11T23:28:42  <gmaxwell> sipa: the latter
509 2015-11-11T23:31:40  *** jgarzik has joined #bitcoin-core-dev
510 2015-11-11T23:31:58  *** jgarzik has quit IRC
511 2015-11-11T23:33:00  *** zooko has quit IRC
512 2015-11-11T23:47:18  <sipa> gmaxwell: meh
513 2015-11-11T23:59:06  *** evoskuil has quit IRC