1 2018-08-24T00:05:20  *** plankers has quit IRC
  2 2018-08-24T00:06:49  *** plankers has joined #bitcoin-core-dev
  3 2018-08-24T00:09:34  *** zivl has quit IRC
  4 2018-08-24T00:20:56  *** phwalkr has quit IRC
  5 2018-08-24T00:32:19  *** lnostdal has joined #bitcoin-core-dev
  6 2018-08-24T00:37:46  *** profmac has joined #bitcoin-core-dev
  7 2018-08-24T00:38:08  *** promag has quit IRC
  8 2018-08-24T00:38:36  *** Krellan has joined #bitcoin-core-dev
  9 2018-08-24T00:46:11  *** promag has joined #bitcoin-core-dev
 10 2018-08-24T01:07:09  *** rex4539 has quit IRC
 11 2018-08-24T01:07:15  *** Rootsudo has quit IRC
 12 2018-08-24T01:07:44  *** rex4539 has joined #bitcoin-core-dev
 13 2018-08-24T01:08:00  *** Rootsudo has joined #bitcoin-core-dev
 14 2018-08-24T01:08:03  *** AaronvanW has quit IRC
 15 2018-08-24T01:10:47  *** promag has quit IRC
 16 2018-08-24T01:17:35  *** promag has joined #bitcoin-core-dev
 17 2018-08-24T01:22:12  *** promag has quit IRC
 18 2018-08-24T01:46:43  *** Magma has quit IRC
 19 2018-08-24T01:46:48  *** dongcarl has quit IRC
 20 2018-08-24T01:46:49  *** adam3us has quit IRC
 21 2018-08-24T01:46:49  *** warren has quit IRC
 22 2018-08-24T01:46:49  *** pindarhk_ has quit IRC
 23 2018-08-24T01:46:55  *** warren has joined #bitcoin-core-dev
 24 2018-08-24T01:47:02  *** dongcarl has joined #bitcoin-core-dev
 25 2018-08-24T01:47:04  *** nanotube has quit IRC
 26 2018-08-24T01:50:57  *** adam3us has joined #bitcoin-core-dev
 27 2018-08-24T01:52:04  *** nanotube has joined #bitcoin-core-dev
 28 2018-08-24T01:58:48  *** lone3lf has quit IRC
 29 2018-08-24T02:28:44  *** vexbuy has quit IRC
 30 2018-08-24T02:28:44  *** rafalcpp has quit IRC
 31 2018-08-24T02:28:44  *** Lightsword has quit IRC
 32 2018-08-24T02:28:45  *** helo has quit IRC
 33 2018-08-24T02:28:45  *** instagibbs has quit IRC
 34 2018-08-24T02:28:46  *** niska has quit IRC
 35 2018-08-24T02:30:03  *** vexbuy has joined #bitcoin-core-dev
 36 2018-08-24T02:30:03  *** rafalcpp has joined #bitcoin-core-dev
 37 2018-08-24T02:30:03  *** Lightsword has joined #bitcoin-core-dev
 38 2018-08-24T02:30:03  *** helo has joined #bitcoin-core-dev
 39 2018-08-24T02:30:03  *** instagibbs has joined #bitcoin-core-dev
 40 2018-08-24T02:30:03  *** niska has joined #bitcoin-core-dev
 41 2018-08-24T02:31:56  *** keymone has quit IRC
 42 2018-08-24T02:31:56  *** BashCo has quit IRC
 43 2018-08-24T02:31:56  *** wumpus has quit IRC
 44 2018-08-24T02:31:56  *** gribble has quit IRC
 45 2018-08-24T02:31:56  *** thaumavorio has quit IRC
 46 2018-08-24T02:31:57  *** treyzania has quit IRC
 47 2018-08-24T02:31:57  *** BlueMatt has quit IRC
 48 2018-08-24T02:31:57  *** bitbee has quit IRC
 49 2018-08-24T02:31:57  *** Madars has quit IRC
 50 2018-08-24T02:31:57  *** berndj has quit IRC
 51 2018-08-24T02:31:57  *** andytoshi has quit IRC
 52 2018-08-24T02:32:02  *** GAit has quit IRC
 53 2018-08-24T02:32:02  *** ossifrage has quit IRC
 54 2018-08-24T02:32:02  *** justanotheruser has quit IRC
 55 2018-08-24T02:32:02  *** Lauda has quit IRC
 56 2018-08-24T02:32:02  *** phantomcircuit has quit IRC
 57 2018-08-24T02:32:02  *** gwillen has quit IRC
 58 2018-08-24T02:32:02  *** rev_strangehope has quit IRC
 59 2018-08-24T02:32:02  *** jamesob has quit IRC
 60 2018-08-24T02:32:02  *** dqx has quit IRC
 61 2018-08-24T02:43:32  *** sipa has quit IRC
 62 2018-08-24T02:47:32  *** BGL has quit IRC
 63 2018-08-24T02:49:39  *** gribble has joined #bitcoin-core-dev
 64 2018-08-24T02:50:36  *** berndj has joined #bitcoin-core-dev
 65 2018-08-24T02:50:41  *** bitbee has joined #bitcoin-core-dev
 66 2018-08-24T02:53:51  *** rev_strangehope has joined #bitcoin-core-dev
 67 2018-08-24T02:55:36  *** BashCo has joined #bitcoin-core-dev
 68 2018-08-24T02:58:23  *** justan0theruser has joined #bitcoin-core-dev
 69 2018-08-24T03:01:33  *** justan0theruser is now known as justanotheruser
 70 2018-08-24T03:05:50  *** lnostdal has quit IRC
 71 2018-08-24T03:05:50  *** Varunram has quit IRC
 72 2018-08-24T03:05:50  *** Giszmo has quit IRC
 73 2018-08-24T03:05:50  *** e4xit has quit IRC
 74 2018-08-24T03:05:50  *** farmerwampum has quit IRC
 75 2018-08-24T03:05:51  *** games_ has quit IRC
 76 2018-08-24T03:05:51  *** xHire has quit IRC
 77 2018-08-24T03:05:51  *** ryanofsky has quit IRC
 78 2018-08-24T03:05:51  *** molz has quit IRC
 79 2018-08-24T03:05:58  *** dongcarl has quit IRC
 80 2018-08-24T03:05:59  *** luke-jr has quit IRC
 81 2018-08-24T03:05:59  *** unholymachine has quit IRC
 82 2018-08-24T03:05:59  *** MarcoFalke has quit IRC
 83 2018-08-24T03:05:59  *** atroxes has quit IRC
 84 2018-08-24T03:05:59  *** windsok has quit IRC
 85 2018-08-24T03:05:59  *** petertodd has quit IRC
 86 2018-08-24T03:05:59  *** jl2012 has quit IRC
 87 2018-08-24T03:05:59  *** karelb has quit IRC
 88 2018-08-24T03:05:59  *** meshcollider has quit IRC
 89 2018-08-24T03:05:59  *** sturles has quit IRC
 90 2018-08-24T03:05:59  *** davex__ has quit IRC
 91 2018-08-24T03:06:37  *** lnostdal has joined #bitcoin-core-dev
 92 2018-08-24T03:06:37  *** Varunram has joined #bitcoin-core-dev
 93 2018-08-24T03:06:37  *** Giszmo has joined #bitcoin-core-dev
 94 2018-08-24T03:06:37  *** e4xit has joined #bitcoin-core-dev
 95 2018-08-24T03:06:37  *** farmerwampum has joined #bitcoin-core-dev
 96 2018-08-24T03:06:37  *** games_ has joined #bitcoin-core-dev
 97 2018-08-24T03:06:37  *** xHire has joined #bitcoin-core-dev
 98 2018-08-24T03:06:37  *** ryanofsky has joined #bitcoin-core-dev
 99 2018-08-24T03:06:37  *** molz has joined #bitcoin-core-dev
100 2018-08-24T03:06:55  *** grubles has quit IRC
101 2018-08-24T03:07:57  *** dongcarl has joined #bitcoin-core-dev
102 2018-08-24T03:07:57  *** luke-jr has joined #bitcoin-core-dev
103 2018-08-24T03:07:57  *** unholymachine has joined #bitcoin-core-dev
104 2018-08-24T03:07:57  *** MarcoFalke has joined #bitcoin-core-dev
105 2018-08-24T03:07:57  *** meshcollider has joined #bitcoin-core-dev
106 2018-08-24T03:07:57  *** windsok has joined #bitcoin-core-dev
107 2018-08-24T03:07:57  *** atroxes has joined #bitcoin-core-dev
108 2018-08-24T03:07:57  *** petertodd has joined #bitcoin-core-dev
109 2018-08-24T03:07:57  *** jl2012 has joined #bitcoin-core-dev
110 2018-08-24T03:07:57  *** karelb has joined #bitcoin-core-dev
111 2018-08-24T03:07:57  *** sturles has joined #bitcoin-core-dev
112 2018-08-24T03:07:57  *** davex__ has joined #bitcoin-core-dev
113 2018-08-24T03:08:14  *** plankers has quit IRC
114 2018-08-24T03:08:19  *** grubles has joined #bitcoin-core-dev
115 2018-08-24T03:08:26  *** grubles has quit IRC
116 2018-08-24T03:08:26  *** grubles has joined #bitcoin-core-dev
117 2018-08-24T03:12:21  *** Guest46194 has joined #bitcoin-core-dev
118 2018-08-24T03:21:41  *** Krellan has quit IRC
119 2018-08-24T03:22:28  *** plankers has joined #bitcoin-core-dev
120 2018-08-24T03:25:25  *** cornfeedhobo has quit IRC
121 2018-08-24T03:28:57  *** jhfrontz has quit IRC
122 2018-08-24T03:41:02  *** cornfeedhobo has joined #bitcoin-core-dev
123 2018-08-24T03:43:14  *** esotericnonsense has quit IRC
124 2018-08-24T03:43:22  *** BlueMatt has joined #bitcoin-core-dev
125 2018-08-24T03:48:17  *** BlueMatt has quit IRC
126 2018-08-24T03:50:06  *** BlueMatt has joined #bitcoin-core-dev
127 2018-08-24T03:57:45  *** moxuan has joined #bitcoin-core-dev
128 2018-08-24T04:01:06  *** plankers has quit IRC
129 2018-08-24T04:07:43  *** ken2812221 has quit IRC
130 2018-08-24T04:07:54  *** BGL has joined #bitcoin-core-dev
131 2018-08-24T04:08:20  *** esotericnonsense has joined #bitcoin-core-dev
132 2018-08-24T04:15:31  *** ken2812221 has joined #bitcoin-core-dev
133 2018-08-24T04:17:51  <ken2812221> Is there any existing document that teach us how to add new source files?
134 2018-08-24T04:20:23  <ken2812221> I think I can add those for MSVC.
135 2018-08-24T04:26:33  *** moxuan has quit IRC
136 2018-08-24T04:27:45  *** moxuan has joined #bitcoin-core-dev
137 2018-08-24T04:28:46  *** plankers has joined #bitcoin-core-dev
138 2018-08-24T04:29:44  *** ossifrage has joined #bitcoin-core-dev
139 2018-08-24T04:40:14  *** echonaut9 has quit IRC
140 2018-08-24T04:40:31  *** echonaut has joined #bitcoin-core-dev
141 2018-08-24T04:54:39  *** harrymm has quit IRC
142 2018-08-24T04:56:13  *** moxuan has quit IRC
143 2018-08-24T04:57:29  *** vexbuy has quit IRC
144 2018-08-24T04:58:05  *** vexbuy has joined #bitcoin-core-dev
145 2018-08-24T04:58:16  *** harrymm has joined #bitcoin-core-dev
146 2018-08-24T05:00:00  *** plankers has quit IRC
147 2018-08-24T05:00:44  *** Rootsudo has quit IRC
148 2018-08-24T05:01:57  *** ken2812221 has quit IRC
149 2018-08-24T05:02:35  *** vexbuy has quit IRC
150 2018-08-24T05:03:13  *** harrymm has quit IRC
151 2018-08-24T05:05:17  *** vexbuy has joined #bitcoin-core-dev
152 2018-08-24T05:08:01  *** moxuan has joined #bitcoin-core-dev
153 2018-08-24T05:09:54  *** sipa has joined #bitcoin-core-dev
154 2018-08-24T05:14:24  *** moxuan has quit IRC
155 2018-08-24T05:15:22  *** moxuan has joined #bitcoin-core-dev
156 2018-08-24T05:16:23  *** moxuan has quit IRC
157 2018-08-24T05:18:41  *** moxuan has joined #bitcoin-core-dev
158 2018-08-24T05:40:54  *** kallewoof has quit IRC
159 2018-08-24T06:08:27  *** vexbuy has quit IRC
160 2018-08-24T06:09:03  *** vexbuy has joined #bitcoin-core-dev
161 2018-08-24T06:10:21  *** ken2812221 has joined #bitcoin-core-dev
162 2018-08-24T06:13:39  *** vexbuy has quit IRC
163 2018-08-24T06:17:08  *** sipa has quit IRC
164 2018-08-24T06:17:57  *** owowo has joined #bitcoin-core-dev
165 2018-08-24T06:18:44  *** sipa has joined #bitcoin-core-dev
166 2018-08-24T06:22:26  *** vexbuy has joined #bitcoin-core-dev
167 2018-08-24T06:24:12  *** harrymm_ has joined #bitcoin-core-dev
168 2018-08-24T06:54:06  *** SopaXorzTaker has joined #bitcoin-core-dev
169 2018-08-24T06:56:35  *** vexbuy has quit IRC
170 2018-08-24T06:57:14  *** vexbuy has joined #bitcoin-core-dev
171 2018-08-24T07:00:28  *** ken2812221 has quit IRC
172 2018-08-24T07:00:53  *** ken2812221 has joined #bitcoin-core-dev
173 2018-08-24T07:01:22  *** vexbuy has quit IRC
174 2018-08-24T07:03:41  *** Krellan has joined #bitcoin-core-dev
175 2018-08-24T07:05:50  *** profmac has quit IRC
176 2018-08-24T07:08:01  *** d9b4bef9 has quit IRC
177 2018-08-24T07:09:07  *** d9b4bef9 has joined #bitcoin-core-dev
178 2018-08-24T07:15:22  *** csknk has joined #bitcoin-core-dev
179 2018-08-24T07:19:14  *** profmac has joined #bitcoin-core-dev
180 2018-08-24T07:22:21  *** Guyver2 has joined #bitcoin-core-dev
181 2018-08-24T07:33:24  *** ken2812221 has quit IRC
182 2018-08-24T07:41:24  *** gribble has quit IRC
183 2018-08-24T07:41:46  *** ken2812221 has joined #bitcoin-core-dev
184 2018-08-24T07:44:13  *** gribble has joined #bitcoin-core-dev
185 2018-08-24T08:02:35  *** IGHOR has joined #bitcoin-core-dev
186 2018-08-24T08:06:11  *** setpill has joined #bitcoin-core-dev
187 2018-08-24T08:06:22  *** vexbuy has joined #bitcoin-core-dev
188 2018-08-24T08:08:14  *** Ylbam_ has joined #bitcoin-core-dev
189 2018-08-24T08:09:24  *** Ylbam_ has quit IRC
190 2018-08-24T08:11:54  *** Krellan has quit IRC
191 2018-08-24T08:12:13  *** Krellan has joined #bitcoin-core-dev
192 2018-08-24T08:14:11  *** ylbam has joined #bitcoin-core-dev
193 2018-08-24T08:26:19  * luke-jr wonders why gitian builds seem to ignore cached depends sometimes
194 2018-08-24T08:32:43  *** vexbuy has quit IRC
195 2018-08-24T08:40:17  *** moxuan has quit IRC
196 2018-08-24T08:40:27  *** vexbuy has joined #bitcoin-core-dev
197 2018-08-24T08:40:28  *** Zenton has joined #bitcoin-core-dev
198 2018-08-24T08:40:42  *** profmac has quit IRC
199 2018-08-24T08:45:27  *** vexbuy has quit IRC
200 2018-08-24T09:04:38  *** promag has joined #bitcoin-core-dev
201 2018-08-24T09:08:53  *** zivl has joined #bitcoin-core-dev
202 2018-08-24T09:09:29  *** vexbuy has joined #bitcoin-core-dev
203 2018-08-24T09:31:24  *** vexbuy has quit IRC
204 2018-08-24T09:32:20  *** AaronvanW has joined #bitcoin-core-dev
205 2018-08-24T09:51:47  *** vexbuy has joined #bitcoin-core-dev
206 2018-08-24T09:52:32  *** vexbuy has joined #bitcoin-core-dev
207 2018-08-24T09:59:04  *** rex4539 has quit IRC
208 2018-08-24T10:20:43  *** Krellan has quit IRC
209 2018-08-24T10:22:01  *** Krellan has joined #bitcoin-core-dev
210 2018-08-24T10:32:48  *** profmac has joined #bitcoin-core-dev
211 2018-08-24T10:55:44  *** rex4539 has joined #bitcoin-core-dev
212 2018-08-24T11:07:01  *** d9b4bef9 has quit IRC
213 2018-08-24T11:08:07  *** d9b4bef9 has joined #bitcoin-core-dev
214 2018-08-24T11:12:54  *** ken2812221 has quit IRC
215 2018-08-24T11:37:55  *** spinza has joined #bitcoin-core-dev
216 2018-08-24T12:01:38  *** profmac has quit IRC
217 2018-08-24T12:06:11  *** promag has quit IRC
218 2018-08-24T12:13:55  *** berndj has quit IRC
219 2018-08-24T12:16:46  *** berndj has joined #bitcoin-core-dev
220 2018-08-24T12:24:28  *** ken2812221 has joined #bitcoin-core-dev
221 2018-08-24T12:39:31  *** vexbuy has quit IRC
222 2018-08-24T12:49:08  *** Guyver2 has quit IRC
223 2018-08-24T12:51:56  *** vexbuy has joined #bitcoin-core-dev
224 2018-08-24T12:54:52  *** profmac has joined #bitcoin-core-dev
225 2018-08-24T12:57:10  *** ylbam has quit IRC
226 2018-08-24T12:58:05  *** Chris_Stewart_5 has joined #bitcoin-core-dev
227 2018-08-24T13:02:11  *** setpill has quit IRC
228 2018-08-24T13:37:49  <cfields> luke-jr: rc2 required rebuilding qt to fix determinism. Is that what you're seeing?
229 2018-08-24T13:38:08  <cfields> jonasschnelli: macOS detached signature reminder :)
230 2018-08-24T14:15:27  *** Chris_Stewart_5 has quit IRC
231 2018-08-24T14:17:33  <MarcoFalke> ken2812221: A new section in developer notes?
232 2018-08-24T14:18:06  *** vexbuy_ has joined #bitcoin-core-dev
233 2018-08-24T14:21:46  <luke-jr> cfields: I'm seeing all the deps being rebuilt for basically the same commit
234 2018-08-24T14:21:48  *** vexbuy has quit IRC
235 2018-08-24T14:29:20  *** Chris_Stewart_5 has joined #bitcoin-core-dev
236 2018-08-24T14:32:44  *** jhfrontz has joined #bitcoin-core-dev
237 2018-08-24T14:37:36  *** vexbuy_ has quit IRC
238 2018-08-24T14:37:36  *** Krellan has quit IRC
239 2018-08-24T14:38:12  *** vexbuy has joined #bitcoin-core-dev
240 2018-08-24T14:38:17  *** Krellan has joined #bitcoin-core-dev
241 2018-08-24T14:39:41  *** vexbuy has quit IRC
242 2018-08-24T14:39:43  *** michaelsdunn1 has joined #bitcoin-core-dev
243 2018-08-24T14:39:52  <jonasschnelli> cfields: oh. One sec
244 2018-08-24T14:41:54  *** profmac has quit IRC
245 2018-08-24T14:44:05  <cfields> luke-jr: hmm, yea, that's not right
246 2018-08-24T14:45:01  *** Victorsueca has quit IRC
247 2018-08-24T14:45:08  <jonasschnelli> cfields: done -> https://github.com/bitcoin-core/bitcoin-detached-sigs/pull/11
248 2018-08-24T14:45:19  <luke-jr> cfields: is there any way to get debug output to see why it's not using the cached builds?
249 2018-08-24T14:45:25  <cfields> jonasschnelli: thanks!
250 2018-08-24T14:46:15  *** Victorsueca has joined #bitcoin-core-dev
251 2018-08-24T14:46:19  <cfields> luke-jr: sure, but gitian makes it kinda tough. Can you drop to a shell in the builder?
252 2018-08-24T14:47:00  <cfields> luke-jr: first thing, try building one more time and seeing if it happens again
253 2018-08-24T14:47:16  <cfields> IIRC bionic's glibc was updated this/last week, so that might've triggered it
254 2018-08-24T14:49:46  <luke-jr> oh
255 2018-08-24T14:52:21  <luke-jr> pretty sure I've seen it multiple times in the last few days, but I haven't been watching closely enough to be sure
256 2018-08-24T14:52:22  <cfields> (depends does something like COMPILER_SEED=`$CC --version --verbose`
257 2018-08-24T14:52:26  <luke-jr> so I'll do a few more just in case
258 2018-08-24T14:52:26  <cfields> )
259 2018-08-24T14:52:43  <cfields> and if it changes, it invalidates all current builds
260 2018-08-24T14:52:48  <cfields> ok
261 2018-08-24T14:53:22  <luke-jr> (also, the cache files are replaced with the same filename)
262 2018-08-24T14:53:33  <cfields> oh, that's weird
263 2018-08-24T14:53:51  <cfields> the filename contains the hash of the determined build recipe. So that shouldn't be happening.
264 2018-08-24T14:54:19  <cfields> maybe some filesystem thing keeps them from sending correctly to the builder?
265 2018-08-24T14:55:03  <luke-jr> no sign of trouble there that I could tell
266 2018-08-24T14:56:30  <luke-jr> hmm
267 2018-08-24T14:57:01  <luke-jr> I have cache/bitcoin-linux-0.17/bitcoin-linux-0.17/, possibly from manual copy-from-target on failures
268 2018-08-24T14:57:22  <luke-jr> it's possible since the glibc bump, I've only had failures..
269 2018-08-24T14:57:47  <cfields> hmm?
270 2018-08-24T14:58:11  <luke-jr> cfields: when gitian fails, it doesn't copy cache changes back to the host; I'd intended to manually do that, but it's possible they ended up in the wrong place
271 2018-08-24T14:58:23  <luke-jr> combined with the glibc update you mentioned, that may explain why it's rebuilding every time
272 2018-08-24T14:58:29  <cfields> ah, makes sense
273 2018-08-24T15:03:16  *** keymone has joined #bitcoin-core-dev
274 2018-08-24T15:04:46  *** jamesob has joined #bitcoin-core-dev
275 2018-08-24T15:11:15  <cfields> gitian builders: detached sigs for 0.17.0rc2 are up
276 2018-08-24T15:11:34  *** as1nc has joined #bitcoin-core-dev
277 2018-08-24T15:11:34  *** MaxHastings_ has joined #bitcoin-core-dev
278 2018-08-24T15:11:49  *** Dizzle has joined #bitcoin-core-dev
279 2018-08-24T15:12:03  *** rex4539 has quit IRC
280 2018-08-24T15:13:02  <cfields> fanquake/achow101/jonasschnelli/provoostenator/luke-jr/wumpus/MarcoFalke/fivepiece: ^^
281 2018-08-24T15:13:41  <jonasschnelli>  /o/
282 2018-08-24T15:14:24  <MaxHastings_> Hey. I just have a quick question. Are all the known vulnerabilities for SPV wallets listed here? https://bitcoin.org/en/developer-guide#simplified-payment-verification-spv I've been told on the Bitcoin slack that there are more.
283 2018-08-24T15:18:43  <as1nc> Hello guys, just published a 2018 simple guide to set up a bitcoin core node on, feedback really appreciated. https://gist.github.com/larafale/b4df34a97c7134cf1579539caf2db2c2
284 2018-08-24T15:19:23  <jonasschnelli> MaxHastings_: does it mention that the current used bloomfilter false-positive-rates do not protect privacy? (See Jonas Nicks master thesis from 2015)
285 2018-08-24T15:19:52  *** profmac has joined #bitcoin-core-dev
286 2018-08-24T15:20:06  <jonasschnelli> MaxHastings_: there are eventually non-protocol vulnerabilities (implementation issues)
287 2018-08-24T15:20:54  <jonasschnelli> as1nc: nice. You could mention pruning?
288 2018-08-24T15:21:09  <jonasschnelli> (and or electrum personal server)
289 2018-08-24T15:21:25  <jonasschnelli> But its OT in this channel... so use #bitcoin
290 2018-08-24T15:24:16  <luke-jr> cfields: can you sign PPC Windows builds too?
291 2018-08-24T15:24:22  <luke-jr> jk ;)
292 2018-08-24T15:25:22  <cfields> luke-jr: eh? does that exist?
293 2018-08-24T15:25:31  <luke-jr> cfields: dunno, hence jk
294 2018-08-24T15:25:37  <cfields> oh, phew
295 2018-08-24T15:25:50  <luke-jr> XD
296 2018-08-24T15:26:05  <luke-jr> (working on POWER8+ gitian binaries)
297 2018-08-24T15:26:08  <luke-jr> (for Linux)
298 2018-08-24T15:26:14  <cfields> because if they couldn't pull off small-endian arm, BE ppc wouldn't stand a chance :)
299 2018-08-24T15:26:23  <luke-jr> who said BE?
300 2018-08-24T15:26:31  <luke-jr> afaik even when Windows did support PPC, it was LE
301 2018-08-24T15:26:38  <cfields> yeah yeah, that's why I specified
302 2018-08-24T15:26:46  <cfields> wait, they actually did support PPC at one point?
303 2018-08-24T15:26:50  <MaxHastings_> jonasschnelli: Ah I see I did not know bloom filters were ineffective for keeping user privacy. Are there any other missing vulnerabilities on that page other than privacy concerns? I was told on Bitcoin slack that the SPV client could be tricked to change its consensus rules by malicious nodes.
304 2018-08-24T15:27:30  <luke-jr> cfields: NT supported a lot of archs
305 2018-08-24T15:27:58  <sipa> MaxHastings_: by definition, yes
306 2018-08-24T15:28:02  <cfields> huh. I used win embedded a few places, so I guess that makes sense. I never would've thought ppc though.
307 2018-08-24T15:28:14  <sipa> MaxHastings_: they rely on the majority of the hashrate
308 2018-08-24T15:28:54  <luke-jr> "Windows NT 3.51 added support for the PowerPC processor in 1995, specifically PReP-compliant systems such as the IBM Power Series desktops/laptops and Motorola PowerStack series; but despite meetings between Michael Spindler and Bill Gates, not on the Power Macintosh as the PReP compliant Power Macintosh project failed to ship."
309 2018-08-24T15:29:19  <luke-jr> not really sure what that means XD
310 2018-08-24T15:29:38  <as1nc> jonasschnelli, yes but i really want to incite people to txindex their chain so they can benefit the full spectrum of bitcoin capabilities. lightning for exemple require a txindexed chain right ?
311 2018-08-24T15:30:14  <sipa> as1nc: i don't think so (or that's temporary in any case)
312 2018-08-24T15:30:17  <jonasschnelli> Not sure. C-Lightning doesn't IMO
313 2018-08-24T15:30:59  <as1nc> lnd does i think
314 2018-08-24T15:31:38  <sipa> i'm sure that's temporary until better protocols to communicate exist
315 2018-08-24T15:32:22  <sipa> (like bip157)
316 2018-08-24T15:34:05  <MaxHastings_> sipa: Good to know.
317 2018-08-24T15:37:00  <gmaxwell> c-lightning doesn't.
318 2018-08-24T15:37:08  <gmaxwell> as1nc: txindex is non-viable long term.
319 2018-08-24T15:37:44  <gmaxwell> Anytime you think "X _requires_ txindex" you should think "X will eventually force people to depend on a centeralized service".
320 2018-08-24T15:38:52  <molz> lnd doesn't require txindex anymore
321 2018-08-24T15:40:42  *** Zenton has quit IRC
322 2018-08-24T15:41:00  <as1nc> ok interesting, I'm txindexing because I want to provide services for people. but the only case i find indexing worth it for personnal use is SPV wallets. does it makes sense ?
323 2018-08-24T15:41:26  <as1nc> i use electrum pointed to my node
324 2018-08-24T15:41:43  <sipa> as1nc: txindex is purely a local feature; it is not exposed over the network
325 2018-08-24T15:43:31  <as1nc> sipa, yes
326 2018-08-24T15:44:53  *** phwalkr has joined #bitcoin-core-dev
327 2018-08-24T15:47:37  *** phwalkr has quit IRC
328 2018-08-24T16:06:34  *** BashCo has quit IRC
329 2018-08-24T16:07:16  *** BashCo has joined #bitcoin-core-dev
330 2018-08-24T16:09:56  <Dizzle> as1nc: yep, it makes sense, for both personal use or providing centralized services. As for lnd, if you don't have a txindex, lnd basically falls back to rescanning blocks when it needs to know a tx's status, so a bit of a performance drag in some cases.
331 2018-08-24T16:10:09  *** rex4539 has joined #bitcoin-core-dev
332 2018-08-24T16:11:30  <gmaxwell> how is scanning blocks going to be a performance drag? thats how a node works anyways.
333 2018-08-24T16:21:46  <Dizzle> gmaxwell: at lnd level, it's done manually instead of regularly i.e. specifically to find what may have befallen the tx - https://github.com/lightningnetwork/lnd/blob/c9bead7c21b1b83d69894935f4e1269ceffbecc7/chainntnfs/bitcoindnotify/bitcoind.go#L535 - so instead of a quick index lookup in Core or btcd, it's back and forth RPC and comparison looping.
334 2018-08-24T16:25:39  <as1nc> Dizzle: thx, so it make sense to txindex your chain for personnal use, even if, like gmaxwell said, its not viable long run
335 2018-08-24T16:32:09  *** BashCo has quit IRC
336 2018-08-24T16:34:29  *** BashCo has joined #bitcoin-core-dev
337 2018-08-24T16:39:54  *** phwalkr has joined #bitcoin-core-dev
338 2018-08-24T16:42:36  <gmaxwell> Dizzle: I don't see any back and forth there, it just gets the block.
339 2018-08-24T16:43:10  <gmaxwell> txindex lowers performance a lot, so I'm still not seeing how you're saying the lnd performance is worth without it.
340 2018-08-24T16:44:18  <luke-jr> if only bech32 addresses were shorter, we could have called it belch32
341 2018-08-24T16:44:54  *** phwalkr has quit IRC
342 2018-08-24T16:52:34  *** peevsie has joined #bitcoin-core-dev
343 2018-08-24T16:55:45  <Dizzle> gmaxwell: only saying the performance is worse without txindex for the cases when historical lookup of a tx's status is needed. I mean, that's what the index is useful for. The back and forth I'm referring to is getblockhash+getblock RPC calls for each height in turn, check for a match, repeat.
344 2018-08-24T17:01:45  <gmaxwell> Dizzle: and my point is that you're missing the forrest for the trees. fetching the list of transactions in a block is cheap. maintaining the txindex slows down block processing a lot.
345 2018-08-24T17:02:10  <gmaxwell> So I think what you're describing costs several milliseconds per block to save a millisecond on some blocks.
346 2018-08-24T17:06:39  *** Urgo has joined #bitcoin-core-dev
347 2018-08-24T17:10:44  <luke-jr> how does symbol-check not complain about aarch using symbols from glibc 2.17?
348 2018-08-24T17:10:55  *** peevsie has quit IRC
349 2018-08-24T17:14:47  <luke-jr> …oh, gitian disables it for aarch XD
350 2018-08-24T17:18:50  *** Chris_Stewart_5 has quit IRC
351 2018-08-24T17:26:33  *** AaronvanW has quit IRC
352 2018-08-24T17:30:23  *** Victorsueca has quit IRC
353 2018-08-24T17:31:46  *** Victorsueca has joined #bitcoin-core-dev
354 2018-08-24T17:36:51  *** AaronvanW has joined #bitcoin-core-dev
355 2018-08-24T17:41:56  *** AaronvanW has quit IRC
356 2018-08-24T17:42:48  *** Zenton has joined #bitcoin-core-dev
357 2018-08-24T17:48:53  *** BashCo has quit IRC
358 2018-08-24T17:50:09  *** BashCo has joined #bitcoin-core-dev
359 2018-08-24T17:53:21  *** esotericnonsense has quit IRC
360 2018-08-24T17:53:32  *** BashCo has quit IRC
361 2018-08-24T17:59:31  *** BashCo has joined #bitcoin-core-dev
362 2018-08-24T18:02:25  <jnewbery> gmaxell: "txindex lowers performance a lot" - is that still true after #13033 ?
363 2018-08-24T18:02:28  <gribble> https://github.com/bitcoin/bitcoin/issues/13033 | Build txindex in parallel with validation by jimpo · Pull Request #13033 · bitcoin/bitcoin · GitHub
364 2018-08-24T18:03:39  *** esotericnonsense has joined #bitcoin-core-dev
365 2018-08-24T18:08:13  *** michaelsdunn1 has quit IRC
366 2018-08-24T18:08:48  <gmaxwell> jnewbery: maybe it is no longer "a lot", but it will still be more costly than checking the txids in a block, since it does that and a lot more.
367 2018-08-24T18:09:21  <gmaxwell> jnewbery: I missed that PR, so thanks for pointing that otu.
368 2018-08-24T18:09:45  <BlueMatt> I mean I dunno what the IOPS cost of it is, but if you're running with huge dbcache it should be almost free modulo the fact that we fsync the block files all the time (which we should stop doing, really)
369 2018-08-24T18:10:13  <sipa> BlueMatt: once per hour
370 2018-08-24T18:10:29  <sipa> and only the ones written to afaik
371 2018-08-24T18:10:31  *** BashCo_ has joined #bitcoin-core-dev
372 2018-08-24T18:10:45  <BlueMatt> not for block files, no, every time we move to a new one
373 2018-08-24T18:10:47  <BlueMatt> last I checked
374 2018-08-24T18:11:25  <BlueMatt> yea, every time we move to a new file we FlushBlockFile
375 2018-08-24T18:11:30  <BlueMatt> which fsync's, afaics
376 2018-08-24T18:11:38  <gmaxwell> BlueMatt: your comment is frustrating me. Above we are having a discussion where the argument was made that a whole txindex should be enabled to replace simply checking each block for a transaction.
377 2018-08-24T18:11:54  <BlueMatt> ah, I missed context, sorry
378 2018-08-24T18:11:56  <BlueMatt> yea, obviously not
379 2018-08-24T18:11:57  *** BashCo has quit IRC
380 2018-08-24T18:12:10  *** michaelsdunn1 has joined #bitcoin-core-dev
381 2018-08-24T18:12:13  <gmaxwell> perhaps txindex no longer slows down reindex by 20% or whatever. but yea...
382 2018-08-24T18:12:38  <sipa> performance isn't really the reason not to use it - it just doesn't scale well
383 2018-08-24T18:12:44  <BlueMatt> anyway, as a separate point, spinning-rust-ibd-with-huge-dbcache the hang after each "Leaving block file" entry is really annoying
384 2018-08-24T18:12:58  <BlueMatt> sipa: well, *also* performance :p
385 2018-08-24T18:18:58  *** BashCo_ has quit IRC
386 2018-08-24T18:19:57  *** Krellan has quit IRC
387 2018-08-24T18:22:43  *** BashCo has joined #bitcoin-core-dev
388 2018-08-24T18:23:12  *** as1nc has quit IRC
389 2018-08-24T18:23:14  <BlueMatt> sipa: wait, isnt extended-encoding the *only* way you can encode a 0-input 1-output transaction?
390 2018-08-24T18:23:34  <BlueMatt> reliably, at least
391 2018-08-24T18:23:50  <BlueMatt> short of psbt
392 2018-08-24T18:25:32  <sipa> BlueMatt: there is no reliable way to encode 0-input transactions
393 2018-08-24T18:25:55  <BlueMatt> sure, if you use extended-encoding
394 2018-08-24T18:26:08  <sipa> that's still potentially ambiguous
395 2018-08-24T18:26:14  <BlueMatt> the use-all-data heuristic should handle it fine
396 2018-08-24T18:26:35  <BlueMatt> and doesnt-need-to-be-backwards-compatible APIs (eg rust-bitcoin's deserializer) can *only* handle it that way
397 2018-08-24T18:27:02  <sipa> i'm confused
398 2018-08-24T18:27:17  <sipa> it is not legal to encode a tx without witnesses using extended encoding
399 2018-08-24T18:27:23  <sipa> so there should never be any ambiguity
400 2018-08-24T18:27:47  <BlueMatt> but every deserializer I've ever seen handles it fine, and its the only way to have no ambiguity in decoding 0-input txn
401 2018-08-24T18:28:00  <BlueMatt> so I think we should revise the spec to match implementations?
402 2018-08-24T18:28:03  <BlueMatt> or am I confusing myself
403 2018-08-24T18:28:34  *** leishman has joined #bitcoin-core-dev
404 2018-08-24T18:28:59  <sipa> sigh, so make an exception to the only-extended-if-has-witness rule in case there are 0 inputs?
405 2018-08-24T18:29:16  <sipa> that's fine by me - i mostly care about only having a single encoding for actual transactions
406 2018-08-24T18:29:29  <sipa> using the raw tx format for partial transactions was always a hack at best
407 2018-08-24T18:29:51  <BlueMatt> yes, agreed, but the only reliably hack is always-use-extended, sadly :(
408 2018-08-24T18:30:03  <BlueMatt> but, ok, fair, only doing it for 0-inputs seems reasonable to me
409 2018-08-24T18:30:15  <sipa> we can also make the serializer do the same
410 2018-08-24T18:30:24  <sipa> always use extended encoding when 0 inputs
411 2018-08-24T18:30:27  <BlueMatt> would seem prudent to me :(
412 2018-08-24T18:30:33  <sipa> ... or psbt
413 2018-08-24T18:30:36  *** BashCo has quit IRC
414 2018-08-24T18:30:39  <sipa> but everything at its time
415 2018-08-24T18:30:43  <BlueMatt> yea, well hopefully people migrate to psbt
416 2018-08-24T18:30:45  <BlueMatt> yea
417 2018-08-24T18:30:52  *** andytoshi has joined #bitcoin-core-dev
418 2018-08-24T18:37:57  *** AaronvanW has joined #bitcoin-core-dev
419 2018-08-24T18:40:23  *** leishman has quit IRC
420 2018-08-24T18:41:07  *** Empact has joined #bitcoin-core-dev
421 2018-08-24T18:42:26  *** AaronvanW has quit IRC
422 2018-08-24T18:45:38  <andytoshi> achow101: how do you use PSBT when nobody knows the full set of inputs up front?
423 2018-08-24T18:46:11  <andytoshi> in particular, can you use PSBT to avoid the 0-input serialization ambiguity
424 2018-08-24T18:46:58  <sipa> right
425 2018-08-24T18:47:00  *** SopaXorzTaker has quit IRC
426 2018-08-24T18:47:14  <sipa> for 0-input things people should just use a list of amounts/scriptpubkeys
427 2018-08-24T18:47:50  <sipa> (i've mentioned extended psbt to support 0 inputs just as a container format, though)
428 2018-08-24T18:48:26  <andytoshi> if we're going to propose an extension BIP to add p2c fields and industry/private reserved IDs, we should throw that in the pile
429 2018-08-24T18:49:24  <sipa> in any case, PSBT's serialization supports 0 inputs fine
430 2018-08-24T18:49:46  <sipa> (because it specifies using non-witness serialization for the tx, there is never any ambiguity)
431 2018-08-24T18:49:51  <andytoshi> oh right
432 2018-08-24T18:50:06  <andytoshi> ok i forgot that, never mind me, sorry
433 2018-08-24T18:50:22  <sipa> it's not really usable to use for the various PSBT roles just because there isn't anything to be done with it
434 2018-08-24T18:50:41  <gmaxwell> https://bitcoinperf.com/timeline/#/?exe=3,4,2,1&base=1+23&ben=reindex.522000.dbcache=2048.mem-usage&env=1&revs=50&equid=off&quarts=on&extr=on  < I think this should be blocking 0.17.
435 2018-08-24T18:50:53  <gmaxwell> 20% memory usage increase for unknown reasons isn't cool.
436 2018-08-24T18:51:57  <BlueMatt> sipa: wait, so psbt is not usable with the existing deserializer?
437 2018-08-24T18:52:04  <sipa> BlueMatt: yes it is
438 2018-08-24T18:52:07  <BlueMatt> cause 0-input non-witness is ambiguous?
439 2018-08-24T18:52:15  <jamesob> gmaxwell: still bisecting
440 2018-08-24T18:52:22  <sipa> BlueMatt: not if it is known to be non-witness
441 2018-08-24T18:52:31  <BlueMatt> unless you give it a flag saying "it is 0-input, and not-witness-encoded, i promise"
442 2018-08-24T18:52:46  <BlueMatt> ah, i guess we have that flag, other deserializers need it, though
443 2018-08-24T18:52:49  <sipa> BlueMatt: which is what the implementation in core does
444 2018-08-24T18:52:50  <BlueMatt> which is super obnoxious
445 2018-08-24T18:52:55  <jamesob> running some tests to ensure increased mem usage isn't some quirk with the benchmarking framework
446 2018-08-24T18:53:13  <sipa> BlueMatt: well, deal with it
447 2018-08-24T18:54:12  <BlueMatt> lolk, but why doesnt it just serialize a list of outputs?
448 2018-08-24T18:54:31  <sipa> BlueMatt: because it needs to serialize inputs too
449 2018-08-24T18:55:07  <BlueMatt> I mean in no-input cases
450 2018-08-24T18:55:17  <sipa> PSBT isn't designed for no-input cases
451 2018-08-24T18:55:23  <BlueMatt> I thought psbt was supposed to simplify the deserialization ambiguity, not work around it with more flags :(
452 2018-08-24T18:55:35  <sipa> it's to simplify the signing procedure
453 2018-08-24T18:55:43  <sipa> BlueMatt: i don't understand why this is hard
454 2018-08-24T18:55:53  <BlueMatt> its not "hard", just really annoying
455 2018-08-24T18:55:53  <sipa> you know there is no witness, don't try to deserialize as a witness
456 2018-08-24T18:56:08  <BlueMatt> yes, well I presume most deserializers dont have a flag for that
457 2018-08-24T18:56:10  <BlueMatt> cause they dont need it
458 2018-08-24T18:56:14  <BlueMatt> at least rust-bitcoin currently doesnt
459 2018-08-24T18:56:18  <sipa> ?
460 2018-08-24T18:56:20  <BlueMatt> which means plumbing through a ton of flags stuff
461 2018-08-24T18:56:28  <sipa> how do you deal with a tx from a non-witness peer?
462 2018-08-24T18:56:40  <gmaxwell> jamesob: well I also recently noticed the infinite dbcache case using much more dbcache size than I remember.
463 2018-08-24T18:56:42  <BlueMatt> you never see 0-input txn on peer network?
464 2018-08-24T18:56:45  <BlueMatt> so no ambiguity there
465 2018-08-24T18:57:00  <sipa> yes, but you should reject if it were to contain a witness
466 2018-08-24T18:57:02  <jamesob> gmaxwell: ah okay, so that corroborates
467 2018-08-24T18:57:10  <sipa> i guess that can be done after deserialization, fine
468 2018-08-24T18:57:41  <sipa> but really, is this worth arguing over?
469 2018-08-24T18:57:56  <sipa> deserialization of 0-input is a pain, i'm sorry for that
470 2018-08-24T18:58:00  <BlueMatt> I mean is it too late to fix the psbt spec? its just annoying to have so many serialization/deserialization modes
471 2018-08-24T18:58:07  <sipa> yes, it is far too late
472 2018-08-24T18:58:25  <BlueMatt> if the thing already has a "this is a 0-input tx" indicator, then why not have just used a list of outputs instead
473 2018-08-24T18:58:31  <BlueMatt> plus nVserion/nLockTime
474 2018-08-24T18:58:43  <sipa> no it doesn't have a this is a 0-input tx
475 2018-08-24T18:59:10  <sipa> that indicator is the lack of inputs
476 2018-08-24T18:59:36  <BlueMatt> ehh, whatever, I guess can be dealt with
477 2018-08-24T18:59:37  <sipa> and again, psbt isn't designed for 0 inputs; it's designed to simplify coordination of signing, which by definition needs at least 1 input
478 2018-08-24T18:59:46  <sipa> it just so happens that it is unambiguous even with 0 inputs
479 2018-08-24T19:00:21  <gmaxwell> sipa: uh, isn't "I create a PSBT with the outputs I want, and give it to you so you can add inputs (and your change)" a supported use case
480 2018-08-24T19:00:22  <achow101> BlueMatt: there are no flags in psbt. the global unsigned tx is always non-witness
481 2018-08-24T19:00:40  <achow101> the input txs are self descriptive as to witness or not as they are always valid network txs
482 2018-08-24T19:01:03  <sipa> gmaxwell: not initially no, but there is no reason why it couldn't
483 2018-08-24T19:01:05  <BlueMatt> achow101: wait, huh? 0-input isnt "valid network txs"
484 2018-08-24T19:01:27  <achow101> BlueMatt: input txs are txs that already exist on the network .they cannot be 0-input
485 2018-08-24T19:01:28  <gmaxwell> BlueMatt: he's talking about prevout inputs.
486 2018-08-24T19:01:29  <sipa> BlueMatt: the inputs
487 2018-08-24T19:01:35  <gmaxwell> for checking fees.
488 2018-08-24T19:01:39  <BlueMatt> ah
489 2018-08-24T19:02:10  <BlueMatt> anyway, I'll stop complaining since I obviously didnt read the spec in detail
490 2018-08-24T19:02:41  <sipa> gmaxwell: in PSBT role terminology that would be someone who takes the output of a Creator, and then "creates" a new PSBT for a different transaction
491 2018-08-24T19:02:57  <sipa> gmaxwell: but in the workflow, all steps operate on the same "proposed transaction"
492 2018-08-24T19:03:10  <sipa> when there are 0 inputs, there isn't really a proposed transaction
493 2018-08-24T19:03:22  <sipa> though the same format could easily be used to support that
494 2018-08-24T19:04:02  <achow101> gmaxwell: psbt's workflows are currently only defined for the signing stage. it assumes that all inputs and outputs have been decided and negotiated in some previous step
495 2018-08-24T19:04:36  <BlueMatt> ah, so there are never any 0-input txn encoded in it at all?
496 2018-08-24T19:04:46  <sipa> BlueMatt: in the PSBT workflow, no
497 2018-08-24T19:04:47  <achow101> but you could still use psbt for negotiating the inputs. I'm not sure why you would though
498 2018-08-24T19:05:05  <BlueMatt> sipa: why didnt you say that in the first place? :)
499 2018-08-24T19:05:06  <sipa> but the format can be used for that, if you want
500 2018-08-24T19:05:14  <achow101> BlueMatt: no, there is not. but it technically still works and will be deserialized properly
501 2018-08-24T19:05:25  <sipa> 18:55:17 < sipa> PSBT isn't designed for no-input cases
502 2018-08-24T19:05:29  <sipa> 18:55:35 < sipa> it's to simplify the signing procedure
503 2018-08-24T19:07:44  <sipa> it's kind of annyong to think about PSBT-the-container-format (which supports 0 input fine, if you're willing to do the decode-knowing-there-is-no-witness thing) and PSBT-the-procedure-for-signing (which clearly doesn't) as separate
504 2018-08-24T19:08:48  <BlueMatt> well I mean tell me to shut up cause I havent read it in detail, but I think in a new container-format we should be encoding 0-input with extended serialization
505 2018-08-24T19:09:11  <BlueMatt> cause thats what we just said, above, too
506 2018-08-24T19:09:38  <sipa> i think that's insane.
507 2018-08-24T19:10:04  <sipa> PSBT implementations aren't even required to support segwit
508 2018-08-24T19:10:06  <achow101> BlueMatt: why? there are no witnesses to serialize ever
509 2018-08-24T19:10:37  <sipa> just add a flag to your deserializer
510 2018-08-24T19:10:47  <achow101> BlueMatt: a psbt has two types of txs: the unsigned tx (the one you want to sign) and the input txs (ones already on the network).
511 2018-08-24T19:10:58  <achow101> the unsigned tx, by definition, has no signatures. therefore it never has witnesses
512 2018-08-24T19:11:26  <achow101> thus it is completely unnecessary to serialize the unsigned tx with extended serialization as it can never have a witness
513 2018-08-24T19:11:38  <BlueMatt> sipa: we just discussed, above, encoding 0-input with extended serialization, since thats the *only* way to do it that is non-ambiguous
514 2018-08-24T19:11:50  <BlueMatt> and otherwise createrawtransaction+fundrawtransaction are...not ideal
515 2018-08-24T19:11:59  <gmaxwell> I'm just really confused by this discussion.  I expected that PSBT was the fix for the createrawtransaction ambiguity case.
516 2018-08-24T19:12:13  <gmaxwell> and now you're saying that it isn't but it is?
517 2018-08-24T19:12:14  <achow101> BlueMatt: it is completely unambiguous because the unsigned tx is always non-witness
518 2018-08-24T19:12:34  <gmaxwell> so this discussion is pointless.
519 2018-08-24T19:12:40  <sipa> BlueMatt: this is the *first* time ever in my life i hear the suggestion to use extended serialization for 0 input
520 2018-08-24T19:12:44  <luke-jr> s/unsigned/no-input/
521 2018-08-24T19:13:06  <sipa> BlueMatt: it's a cool trick, i admit
522 2018-08-24T19:13:07  <BlueMatt> sipa: well its either that or we have to pass a flag into signrawtransaction (and any equivalent APIs in any bitcoin library anywhere)
523 2018-08-24T19:13:24  <sipa> but i had always lived under the assumption that our deserializer failed on that
524 2018-08-24T19:13:34  <sipa> it was just accidentally removed in a refactor
525 2018-08-24T19:13:34  <BlueMatt> of which there are many, any I generally dont trust most bitcoin libraries...for obvious reasons
526 2018-08-24T19:13:41  <achow101> BlueMatt: why are you signing a 0-input tx?
527 2018-08-24T19:13:45  <achow101> what is there to even sign?
528 2018-08-24T19:13:51  <sipa> achow101: not the point
529 2018-08-24T19:13:54  <BlueMatt> sipa: for p2p, it probably should, because, you know, 0-input is bogus on the p2p/consensus layer
530 2018-08-24T19:14:01  <BlueMatt> achow101: not signing, funding
531 2018-08-24T19:14:25  <BlueMatt> achow101: right now the suggested workflow of createrawtransaction, fundrawtransaction, signrawtransaction, sendrawtransaction is hit by the ambiguity bug
532 2018-08-24T19:14:29  <gmaxwell> achow101: See our rpc, "fundrawtransaction"  --- that isn't  "Fun! Draw Transaction." :P
533 2018-08-24T19:14:46  <sipa> BlueMatt: createraw and fundraw are merged into one in PSBT workflow
534 2018-08-24T19:14:57  <BlueMatt> its (mostly) fine right now, but my understanding was the psbt was addressing exactly that use-case but across multiple signers like multisig and the like
535 2018-08-24T19:15:15  <luke-jr> achow101: you don't know it's zero input yet
536 2018-08-24T19:15:20  <gmaxwell> the reason they exist seperately to begin with is that there are workflows where they need to be seperate. E.g. coinjoins.
537 2018-08-24T19:15:43  <sipa> FFS people
538 2018-08-24T19:15:44  <BlueMatt> sipa: fair, but you just mentioned that the existing container could be extended to split that out, and my point is, "is the container well-defined for the ambiguity bug"
539 2018-08-24T19:15:46  <gmaxwell> or partial coincontrols. ("spend _this_ coin plus, whatever is needed")
540 2018-08-24T19:15:48  <sipa> can you have commented on this 9 months ago?
541 2018-08-24T19:16:06  <sipa> BlueMatt: YES IT IS. NO WITNESSES IN THE UNSIGNED PSBT TRANSACTION
542 2018-08-24T19:16:11  <sipa> i'm tired of this discussion
543 2018-08-24T19:16:29  <gmaxwell> sipa:  I read the spec and saw no evidence that 0 inputs wasn't a supported case!
544 2018-08-24T19:16:36  <sipa> the fact that you want to reuse your heuristic decoder for raw transactions in PSBT may mean you need to add a flag to disable that feature
545 2018-08-24T19:16:50  <sipa> in a post-PSBT world, nobody ever needs to implement such a heuristic again
546 2018-08-24T19:17:20  <achow101> "Type: Unsigned Transaction PSBT_GLOBAL_UNSIGNED_TX = 0x00 ... The transaction must be in the old serialization format (without witnesses)"
547 2018-08-24T19:17:24  <sipa> because actual transactions will always have witnesses (or use unambiguously old serialization for things without)
548 2018-08-24T19:17:30  <sipa> and PSBTs will never have witnesses
549 2018-08-24T19:17:44  <gmaxwell> it's surprising to me that you and achow didn't except PSBT to be a superset of the applications of the existing raw transactions.
550 2018-08-24T19:17:48  *** profmac has quit IRC
551 2018-08-24T19:18:06  <gmaxwell> achow101: great, and the old tx format supports 0 inputs.
552 2018-08-24T19:18:13  <achow101> gmaxwell: exactly
553 2018-08-24T19:18:20  <sipa> gmaxwell: not... really
554 2018-08-24T19:18:22  <achow101> gmaxwell: but it is _always_ the old serialization. no ambiguity there
555 2018-08-24T19:19:03  *** MaxHastings_ has quit IRC
556 2018-08-24T19:19:42  <sipa> this discussion only happened because someone only has a heuristic tx decoder in their software, and finds it annoying to add a way to disable that heuristic
557 2018-08-24T19:19:51  <gmaxwell> achow101: sure, I agree.  Sipa was asking why wasn't this commented on 9 months ago.  And my reply is because the shortfall (that 0 input PSBT aren't a supported usecase) isn't something I could extract from the spec.
558 2018-08-24T19:20:11  <luke-jr> achow101: the ambiguity is because you don't know if it's 0-input or 1+ input segwit until you parse it correctly
559 2018-08-24T19:20:20  <sipa> luke-jr: you're missing context
560 2018-08-24T19:20:22  <gmaxwell> luke-jr: it's not ambigius in PSBT.
561 2018-08-24T19:20:24  <luke-jr> ok
562 2018-08-24T19:20:38  <gmaxwell> In PSBT the internal tx never has witnesses.
563 2018-08-24T19:21:30  <achow101> gmaxwell: the format supports 0-inputs just fine. I think there's even a test for it. the processes that we also defined in the bip do not.
564 2018-08-24T19:21:56  <sipa> gmaxwell: i think i always thought of PSBT as a better unambiguous serialization... but the BIP describes both a serialization and the procedures around it
565 2018-08-24T19:22:08  <achow101> but the processes don't matter that much. you can use the psbt format for whatever steps that occur before signing and then use the processes defined in the bip
566 2018-08-24T19:22:19  <sipa> and those procedures don't really map well to the part where you're still deciding what exactly to sign
567 2018-08-24T19:22:23  <gmaxwell> I never read that stuff, it's not normative (doesn't change the meaning of the datastructures), and expirence from other BIPs suggests tha people ignore them. (see, for example BIP32)
568 2018-08-24T19:22:37  <gmaxwell> (or at least don't read it carefully)
569 2018-08-24T19:22:39  <sipa> gmaxwell: well, then you were right
570 2018-08-24T19:22:49  <sipa> that part of PSBT supports 0 inputs fine :)
571 2018-08-24T19:33:08  *** csknk has quit IRC
572 2018-08-24T19:41:15  <gmaxwell> sipa: thanks! confusion resolved!!
573 2018-08-24T19:47:01  *** vexbuy has joined #bitcoin-core-dev
574 2018-08-24T19:53:43  *** plankers has joined #bitcoin-core-dev
575 2018-08-24T20:01:00  *** BlueMatt has left #bitcoin-core-dev
576 2018-08-24T20:03:10  *** vexbuy has quit IRC
577 2018-08-24T20:04:20  *** vexbuy has joined #bitcoin-core-dev
578 2018-08-24T20:13:59  <jnewbery> sipa: performance isn't really the reason not to use it - [txindex] just doesn't scale well" - scale well in what regard? Space requirements should be linear with blockchain size for a full archival node, right?
579 2018-08-24T20:15:17  <sipa> jnewbery: i don't think it is realistic that everyone needs to store the whole chain
580 2018-08-24T20:16:26  <sipa> or have fast access to it
581 2018-08-24T20:16:38  <jnewbery> I agree! But for those that do, txindex is just a constant factor increase in space requirements
582 2018-08-24T20:16:43  <sipa> sure
583 2018-08-24T20:16:57  <jnewbery> ie txindex scales as well as archival nodes scale
584 2018-08-24T20:17:14  <sipa> but building solutions that rely on having a fully-indexed blockchain available isn't scalable
585 2018-08-24T20:17:31  <sipa> you don't need many archival nodes
586 2018-08-24T20:18:10  <sipa> furthermore, nothing really _needs_ access to the full chain if designed well, in my opinion
587 2018-08-24T20:18:27  <sipa> except for debugging purposes
588 2018-08-24T20:18:46  <jnewbery> a block explorer?
589 2018-08-24T20:18:56  <sipa> for example
590 2018-08-24T20:19:02  <sipa> (i count that as debug purposes)
591 2018-08-24T20:19:19  <sipa> not something to rely upon for production
592 2018-08-24T20:20:22  <jnewbery> I agree that products shouldn't *rely* on block explorers, but there's demand for them now and I can't imagine there'd ever not be demand for them
593 2018-08-24T20:20:38  <sipa> sadly enough :(
594 2018-08-24T20:22:25  <sipa> however, in general i think we should discourage building on things like txindex
595 2018-08-24T20:22:39  <jamesob> and in any case it seems like there'll always be use-cases which require rescan; ie when you become interested in an output you weren't keeping tabs on before
596 2018-08-24T20:23:02  <sipa> as i fear we may end up with a less scalable ecosystem if everyone assumes it's easy to just look up whatever transaction
597 2018-08-24T20:23:40  <sipa> and of course there are use cases - it wouldn't be there otherwise
598 2018-08-24T20:24:34  <sipa> jamesob: i think that's either sign of a badly designed system, or an emergency (for which i think it's fine to need some index, you could also pay someone to look things up for you)
599 2018-08-24T20:25:26  <jamesob> yeah, maybe so
600 2018-08-24T20:25:30  *** roasbeef has joined #bitcoin-core-dev
601 2018-08-24T20:26:41  <roasbeef> lnd doesn't require the txindex, it'll detect if the full node has it on or not, and fall back to just fetching blocks it needs manually, eventually if the gcs filters are exposed on RPC it could also fetch those before going for the full block it needs
602 2018-08-24T20:29:26  <jnewbery> It's fair to say that 'assuming that it's easy to just look up whatever transaction' isn't a scalable architecture to build a product on, but I don't think the txindex in itself is unscalable now that we #13033
603 2018-08-24T20:29:28  <gribble> https://github.com/bitcoin/bitcoin/issues/13033 | Build txindex in parallel with validation by jimpo · Pull Request #13033 · bitcoin/bitcoin · GitHub
604 2018-08-24T20:31:33  <roasbeef> jnewbery: things could also be split up into distinct database, or even really just like memory mapped direct hash tables, shoving everything in leveldb makes other routine stuff slower as you end up with thousands of files that all need to be memory mapped and also to watch out for fd limits, then on top of that you end up with a handfull of disk seeks due to the file and sstable structure to read something like the offset of a txid in a block
605 2018-08-24T20:33:20  <jamesob> roasbeef: jimpo's txindex change already moves indices into separate leveldbs
606 2018-08-24T20:33:22  <roasbeef> Dizzle: we now continually also update a "height hint" so really a checkpoint of the state of that outpoint/txid of its best known state, so like if we know it isn't conf'd at height 100, but it was broadcast at height 10 we know now to start at height 10
607 2018-08-24T20:33:47  <sipa> roasbeef: BIP157 will improve things further, i assume?
608 2018-08-24T20:33:55  <Dizzle> roasbeef: nice
609 2018-08-24T20:35:09  <jnewbery> ok, yes - that is fair, although in practice I'm not sure how much performance impact there is. If it's truly a performance bottleneck, then I assume #13243 gives us the infrastructure to store txindex somewhere else
610 2018-08-24T20:35:11  <gribble> https://github.com/bitcoin/bitcoin/issues/13243 | Make reusable base class for auxiliary indices by jimpo · Pull Request #13243 · bitcoin/bitcoin · GitHub
611 2018-08-24T20:35:35  <roasbeef> sipa: yeh, either if it's going over p2p network, or just even getting the filters over rpc from the full node, as it can fetch the filters in batch, and also cache filters on its end
612 2018-08-24T20:35:59  *** profmac has joined #bitcoin-core-dev
613 2018-08-24T20:37:07  <jamesob> jnewbery: indexes are already isolated from other dbs in /datadir/indexes/[index name]
614 2018-08-24T20:37:12  <roasbeef> also eventually the stuff about creating larger like hierarchical multi-block filters can help as well when the node has been offline for a long-ish time
615 2018-08-24T20:45:50  <gmaxwell> Under a no-address reuse assumption (lol), hierarchical filters are never a win.
616 2018-08-24T20:46:02  *** d9b4bef9 has quit IRC
617 2018-08-24T20:46:27  <gmaxwell> 13:16:57 < jnewbery> ie txindex scales as well as archival nodes scale
618 2018-08-24T20:46:48  <gmaxwell> I don't expect that archive nodes will be a thing in 10 years, outside of academic curiousity and what not.
619 2018-08-24T20:47:08  *** d9b4bef9 has joined #bitcoin-core-dev
620 2018-08-24T20:47:19  <gmaxwell> No more than traces of all announced transactions from all peers in the network are a thing now.
621 2018-08-24T20:48:33  <gmaxwell> I expect that new nodes will sync up using a months old UTXO state thats vallidated according to the assumevalid model, then catch up.
622 2018-08-24T20:52:18  <jnewbery> Seems possible, but for the next 10 years I expect at least some people will want to run archival nodes!
623 2018-08-24T20:52:33  <gmaxwell> oh absolutely, and we should do what we can to make that work well too.
624 2018-08-24T20:52:33  <sipa> i hope they do.
625 2018-08-24T20:52:41  <sipa> but i hope no services realpy need them
626 2018-08-24T20:53:06  <gmaxwell> I've pushed a lot on optimizations related to e.g. reindex/ibd in the last year specifically because once we're syncing from AV-UTXO there will be much less motivation to improve these things.
627 2018-08-24T20:54:14  <jnewbery> to be clear - I'm not arguing that we should *encourage* the use of txindex or needing a full blockchain. I just wanted understand what was meant by "txindex is unscalable".
628 2018-08-24T20:56:21  <sipa> right; i think it's more "an ecosystem relying on txindex doesn't scale well"
629 2018-08-24T20:56:30  <jnewbery> yes, makes sense
630 2018-08-24T20:56:36  <sipa> txindex as an algorithmic thing itself scales as well as it could i think
631 2018-08-24T20:57:03  <gmaxwell> Well I'm sure it's lacking a ton of possible optimizations.
632 2018-08-24T20:57:30  <sipa> sure, but no dramatic complexity changes
633 2018-08-24T20:58:07  <gmaxwell> no, just constant factors.
634 2018-08-24T20:58:31  <gmaxwell> it's not the txindex structure itself isn't scable, it's that operations over the chain history aren't scalable.
635 2018-08-24T21:00:37  <jnewbery> sipa, has your opinion changed at all since this: I both hate and love this patch. I hate making it easy to build infrastructure that relies on a fully-indexed blockchain (which shouldn't be necessary), as it may remove the incentive to build more scalable solutions. On the other hand, in cases where the alternative is relying on a trusted third party, this approach is certainly preferable, and
636 2018-08-24T21:00:43  <jnewbery> would allow an RPC-based blockexplorer, for example.
637 2018-08-24T21:00:44  <jnewbery> (#2802)
638 2018-08-24T21:00:46  <gribble> https://github.com/bitcoin/bitcoin/issues/2802 | Add an address-based transaction index by sipa · Pull Request #2802 · bitcoin/bitcoin · GitHub
639 2018-08-24T21:01:27  <gmaxwell> RPC based explorer also needs a reversed spentness index.
640 2018-08-24T21:03:41  <gmaxwell> jnewbery: it's an interesting question of if people would actually use it if it was provided.  If they would I think it would be good to have. ... at least for the next 10 years, _some_ users would stop bleeding privacy wise.
641 2018-08-24T21:07:29  *** Krellan has joined #bitcoin-core-dev
642 2018-08-24T21:07:58  <sipa> jnewbery: i have the same love-hate relation with it :)
643 2018-08-24T21:09:47  <jnewbery> good to know. Thanks!
644 2018-08-24T21:09:58  <instagibbs> https://github.com/bitcoin/bitcoin/pull/14055 :( another RC?
645 2018-08-24T21:10:42  <instagibbs> we(not me clearly, someone else) kind of broke the call. Probably is a way around it(walletupdatepsbt?)
646 2018-08-24T21:10:55  <instagibbs> walletprocesspsbt
647 2018-08-24T21:11:12  <gmaxwell> Well I think unless it turns out that the memory usage increase is fictional, we'll have another RC anyways.
648 2018-08-24T21:12:35  <instagibbs> phew
649 2018-08-24T21:16:26  *** farmerwampum has quit IRC
650 2018-08-24T21:33:45  *** andytoshi has quit IRC
651 2018-08-24T21:49:27  *** Chris_Stewart_5 has joined #bitcoin-core-dev
652 2018-08-24T21:51:34  *** marcinja has quit IRC
653 2018-08-24T21:56:29  *** dendisuhubdy has joined #bitcoin-core-dev
654 2018-08-24T21:59:42  *** Victorsueca has quit IRC
655 2018-08-24T22:00:54  *** Victorsueca has joined #bitcoin-core-dev
656 2018-08-24T22:05:48  <luke-jr> woo finally got powerpc64 builds out of gitian working
657 2018-08-24T22:06:23  <luke-jr> just in time for bluematt to close his PR :x
658 2018-08-24T22:14:39  *** dendisuhubdy has quit IRC
659 2018-08-24T22:16:32  *** michaelsdunn1 has quit IRC
660 2018-08-24T22:19:11  *** jungly has joined #bitcoin-core-dev
661 2018-08-24T22:19:38  *** meshcollider_ has joined #bitcoin-core-dev
662 2018-08-24T22:34:54  *** Chris_Stewart_5 has quit IRC
663 2018-08-24T22:35:20  *** murrayn has joined #bitcoin-core-dev
664 2018-08-24T22:36:53  *** promag has joined #bitcoin-core-dev
665 2018-08-24T22:38:47  *** jungly has quit IRC
666 2018-08-24T22:56:18  *** profmac has quit IRC
667 2018-08-24T23:00:23  *** leishman has joined #bitcoin-core-dev
668 2018-08-24T23:01:18  *** AaronvanW has joined #bitcoin-core-dev
669 2018-08-24T23:09:22  *** Dizzle has quit IRC
670 2018-08-24T23:16:45  *** leishman has quit IRC
671 2018-08-24T23:18:31  *** Chris_Stewart_5 has joined #bitcoin-core-dev
672 2018-08-24T23:24:47  <luke-jr> is it preferable to de-bundle libpng from depends/qt, or patch the libpng bundled with depends/qt?
673 2018-08-24T23:25:44  <luke-jr> cfields: ^
674 2018-08-24T23:36:54  *** plankers has quit IRC
675 2018-08-24T23:48:13  <luke-jr> probably debundling, since Qt's copy is missing the files needed :/
676 2018-08-24T23:48:55  * luke-jr ponders using virtfs for gitian cache
677 2018-08-24T23:51:54  <gmaxwell> jonasschnelli: so this protocol's use of a seperate chacha20 for the sizes is effectively halving the performance of our cypher.
678 2018-08-24T23:53:03  <gmaxwell> jonasschnelli: the use of a message number as the nonce (instead of using a continious stream) will also decrease our performance for AVX by maybe 20%.
679 2018-08-24T23:54:06  <sipa> gmaxwell: that's a strange gratuitous performance reduction, indeed
680 2018-08-24T23:54:12  <sipa> i wonder why openssh did that
681 2018-08-24T23:55:44  <gmaxwell> maybe because they thought they would read ahead lengths and thus schedule the other runs more efficiently?
682 2018-08-24T23:57:29  <gmaxwell> https://tools.ietf.org/html/draft-josefsson-ssh-chacha20-poly1305-openssh-00#section-3
683 2018-08-24T23:59:14  <sipa> the separation of the length and data cipher makes sense, but there's no need to discard 28 bytes from each output of K_1