1 2016-09-14T00:04:16  *** zxzzt has joined #bitcoin-core-dev
  2 2016-09-14T00:08:43  *** fengling has joined #bitcoin-core-dev
  3 2016-09-14T00:13:41  *** fengling has quit IRC
  4 2016-09-14T00:33:31  *** droark has quit IRC
  5 2016-09-14T00:37:42  *** fengling has joined #bitcoin-core-dev
  6 2016-09-14T00:39:14  *** justanotheruser has quit IRC
  7 2016-09-14T00:39:34  *** justanotheruser has joined #bitcoin-core-dev
  8 2016-09-14T00:40:45  *** justanotheruser has quit IRC
  9 2016-09-14T00:41:05  *** justanotheruser has joined #bitcoin-core-dev
 10 2016-09-14T00:42:40  *** fengling has quit IRC
 11 2016-09-14T00:46:54  *** Ylbam has quit IRC
 12 2016-09-14T00:52:45  *** fengling has joined #bitcoin-core-dev
 13 2016-09-14T00:57:28  *** fengling has quit IRC
 14 2016-09-14T00:59:37  *** Giszmo has quit IRC
 15 2016-09-14T01:04:38  *** justanotheruser is now known as hughmungus
 16 2016-09-14T01:04:49  *** hughmungus is now known as justanotheruser
 17 2016-09-14T02:04:38  *** fengling has joined #bitcoin-core-dev
 18 2016-09-14T02:07:24  *** btcdrak has quit IRC
 19 2016-09-14T02:49:05  *** fengling has quit IRC
 20 2016-09-14T03:06:33  *** fengling has joined #bitcoin-core-dev
 21 2016-09-14T03:20:06  *** Alopex has quit IRC
 22 2016-09-14T03:21:11  *** Alopex has joined #bitcoin-core-dev
 23 2016-09-14T03:22:49  *** netsin has joined #bitcoin-core-dev
 24 2016-09-14T03:31:24  *** netsin has joined #bitcoin-core-dev
 25 2016-09-14T03:35:12  *** Alopex has quit IRC
 26 2016-09-14T03:36:17  *** Alopex has joined #bitcoin-core-dev
 27 2016-09-14T03:47:28  *** netsin has quit IRC
 28 2016-09-14T03:53:11  *** Alopex has quit IRC
 29 2016-09-14T03:54:17  *** Alopex has joined #bitcoin-core-dev
 30 2016-09-14T04:00:33  *** netsin has joined #bitcoin-core-dev
 31 2016-09-14T04:07:02  *** Alopex has quit IRC
 32 2016-09-14T04:08:07  *** Alopex has joined #bitcoin-core-dev
 33 2016-09-14T04:21:15  *** aj_ is now known as aj
 34 2016-09-14T04:23:38  *** netsin has joined #bitcoin-core-dev
 35 2016-09-14T04:30:12  *** netsin has quit IRC
 36 2016-09-14T04:31:31  *** Ismar has joined #bitcoin-core-dev
 37 2016-09-14T04:36:42  *** Ismar has quit IRC
 38 2016-09-14T04:49:07  *** Alopex has quit IRC
 39 2016-09-14T04:50:12  *** Alopex has joined #bitcoin-core-dev
 40 2016-09-14T04:51:31  *** justanotheruser has quit IRC
 41 2016-09-14T05:00:25  *** Squidicc has quit IRC
 42 2016-09-14T05:00:34  *** Squidicuz has joined #bitcoin-core-dev
 43 2016-09-14T05:02:17  *** justanotheruser has joined #bitcoin-core-dev
 44 2016-09-14T05:03:01  *** Alopex has quit IRC
 45 2016-09-14T05:04:07  *** Alopex has joined #bitcoin-core-dev
 46 2016-09-14T05:05:06  *** Squidicuz has quit IRC
 47 2016-09-14T05:09:40  *** btcdrak has joined #bitcoin-core-dev
 48 2016-09-14T05:13:00  *** dcousens has left #bitcoin-core-dev
 49 2016-09-14T05:13:05  *** dcousens has joined #bitcoin-core-dev
 50 2016-09-14T05:47:31  *** kadoban has quit IRC
 51 2016-09-14T05:56:16  *** owowo has quit IRC
 52 2016-09-14T06:01:32  *** Squidicuz has joined #bitcoin-core-dev
 53 2016-09-14T06:13:02  *** Alopex has quit IRC
 54 2016-09-14T06:14:07  *** Alopex has joined #bitcoin-core-dev
 55 2016-09-14T06:25:25  *** Madars_ is now known as Madars
 56 2016-09-14T06:32:20  *** assder has joined #bitcoin-core-dev
 57 2016-09-14T06:32:59  <GitHub186> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/fa7caf6d9116...57b34599b2de
 58 2016-09-14T06:32:59  <GitHub186> bitcoin/master 1b6bcdd Jonas Schnelli: Remove maxuploadtargets recommended minimum
 59 2016-09-14T06:33:00  <GitHub186> bitcoin/master 57b3459 Jonas Schnelli: Merge #8712: Remove maxuploadtargets recommended minimum...
 60 2016-09-14T06:33:09  <GitHub42> [bitcoin] jonasschnelli closed pull request #8712: Remove maxuploadtargets recommended minimum (master...2016/09/rem_maxupt_min) https://github.com/bitcoin/bitcoin/pull/8712
 61 2016-09-14T06:43:22  *** owowo has joined #bitcoin-core-dev
 62 2016-09-14T06:48:49  <jonasschnelli> I guess the Travis master issue is due to a random flickering in walletbackup.py
 63 2016-09-14T06:50:42  <jonasschnelli> Would be nice if anyone could review/ACK/NACK https://github.com/bitcoin/bitcoin/pull/7783 (nested RPC commands).
 64 2016-09-14T06:51:38  <dcousens> jonasschnelli: how would you extend it to the RPC?
 65 2016-09-14T06:52:10  <jonasschnelli> dcousens: there are no plans to extend it to the RPC layer... the PR once had support for RPC but we decided to keep it QT only for now.
 66 2016-09-14T06:52:40  <jonasschnelli> Moving the function RPCExecuteCommandLine(std::string &strResult, const std::string &strCommand) to a core class would be trivial.
 67 2016-09-14T06:53:12  <jonasschnelli> We could either have a special RPC command that result in parsing and executing multiple nested commands...but I don't think this would be clever.
 68 2016-09-14T06:53:25  <jonasschnelli> The Qt version is "client-side".
 69 2016-09-14T06:53:40  <sipa> i'd have liked that, but it seems i was alone with that :)
 70 2016-09-14T06:53:41  <jonasschnelli> A better approach for the RPC layer would be to add it into bitcoin-cli
 71 2016-09-14T06:54:13  <jonasschnelli> Having it server-side i just fear some uncontrollable resource usage.
 72 2016-09-14T06:54:32  <jonasschnelli> I think its better to start with it client-side QT only, and if it turns to be useful, add it to bitcoin-cli.
 73 2016-09-14T06:54:33  <sipa> rpc already has uncontrollable resource usage
 74 2016-09-14T06:54:40  <dcousens> Personally I just always used bash,  but I never use the QT UI so *shrug*
 75 2016-09-14T06:54:42  <sipa> fair enough :)
 76 2016-09-14T06:55:16  <jonasschnelli> dcousens: Adding in in Qt has less of exposures... a better test-bed .:)
 77 2016-09-14T06:55:26  <jonasschnelli> *it
 78 2016-09-14T06:55:59  <dcousens> As for the RPC extension, I think a special RPC command, say a mechanism to chain calls - makes sense, would feel like the idea by wumpus, but won't break all our RPC implementations
 79 2016-09-14T06:56:32  <dcousens> s/break/require changing
 80 2016-09-14T06:57:16  <sipa> well if we turn it into an RPC thing, we'd arguably want more discussion about what the "query language" will look like
 81 2016-09-14T06:57:17  <dcousens> as for resource usage, RPC is already meant to be a "safe space", haha
 82 2016-09-14T06:57:33  <sipa> as it may be harder to make incompatible changes latee
 83 2016-09-14T06:58:01  <dcousens> sipa: absolutely, the amount of time to get right may not be worth the extra line/rpc call for the few times this matters :P
 84 2016-09-14T06:58:53  <jonasschnelli> The query language is pretty homebrew and non-standard..
 85 2016-09-14T06:59:11  <jonasschnelli> Also,... once we have it in RPC, people are going to see it as part of the API...
 86 2016-09-14T06:59:17  <jonasschnelli> changes at this point will probably more "difficult".
 87 2016-09-14T06:59:35  <jonasschnelli> Lets find out through Qt how it could work in RPC. :)
 88 2016-09-14T06:59:37  <dcousens> ultimately any bulk usage of the RPC ends up seeing the piped network latency being completely insignificant... and chaining the commands won't lower the JSON bulk
 89 2016-09-14T06:59:59  <dcousens> unless their doing it over an open network I suppose
 90 2016-09-14T07:00:03  <sipa> jonasschnelli: yes, agree there
 91 2016-09-14T07:00:15  <wumpus> what did I break?
 92 2016-09-14T07:00:39  <jonasschnelli> ;-)
 93 2016-09-14T07:01:29  <sipa> wumpus: ?
 94 2016-09-14T07:01:48  <wumpus> oh that, yes that's a simple extension to JSON batching that would allow chaining of return values
 95 2016-09-14T07:03:13  <wumpus> don't know whether to propose it or not, I'm fairly sure their reply will be 'if you need this level of sophistication don't use JSON-RPC you noob'
 96 2016-09-14T07:03:35  <sipa> who is they?
 97 2016-09-14T07:03:59  <wumpus> the people who made the JSON RPC standard
 98 2016-09-14T07:04:02  <dcousens> I think if the RTT time is more significant than the encoding of the data itself, and you want to optimize RPC, its very likely you have control over the node anyway, just write a small HTTP wrapper that does that over localhost for you/
 99 2016-09-14T07:05:11  <dcousens> or hell... write a RPC command that does what you want
100 2016-09-14T07:05:28  <sipa> hah, yes
101 2016-09-14T07:05:34  <sipa> that's my approach usually :)
102 2016-09-14T07:05:41  * sipa zZzZ
103 2016-09-14T07:05:42  <dcousens> as is mine :S
104 2016-09-14T07:05:59  <wumpus> the point is that we don't want the API clogged full of 'chain A and B' commands
105 2016-09-14T07:06:07  <wumpus> but for a local patch sure
106 2016-09-14T07:06:30  <wumpus> in the end  I think no one really has any convincing use-cases for this
107 2016-09-14T07:06:48  <dcousens> yup, we're all just lazy cause it means 1 more line
108 2016-09-14T07:07:04  <jonasschnelli> I think the RPC layer is for app to app communication
109 2016-09-14T07:07:15  <wumpus> I've had approximately zero replies on me JSON-RPC extension proposal, though I think it makes a lot of sense *if* people are seriously considering this
110 2016-09-14T07:07:18  <jonasschnelli> The Qt console is supposed to be a human command line interface
111 2016-09-14T07:07:53  *** OdicforceSounds has joined #bitcoin-core-dev
112 2016-09-14T07:07:55  <wumpus> yes, the RPC layer is for app to app communication, it is an API
113 2016-09-14T07:07:57  <jonasschnelli> The convenience of nested commands are probably on the client side
114 2016-09-14T07:08:02  <dcousens> wumpus: I'm against the idea,  just wasn't sure if the Qt change (#7783) was relevant
115 2016-09-14T07:08:16  <wumpus> dcousens: huh?
116 2016-09-14T07:08:28  <wumpus> no, qt has nothing to do with it
117 2016-09-14T07:08:39  <wumpus> that' just a user convenience
118 2016-09-14T07:08:47  <dcousens> indeed, hence ACK :)
119 2016-09-14T07:09:14  <wumpus> I mean with bitcoin-cli you have full access to bash' scripting/variable munging/etc capabilities, in the Qt console you don't, so this adds a bit of shell power to it
120 2016-09-14T07:09:21  <sipa> wumpus: oh, i had no idea that that chaining value idea was your proposal
121 2016-09-14T07:09:48  *** rubensayshi has joined #bitcoin-core-dev
122 2016-09-14T07:10:10  <gmaxwell> or the rpc console could just link in a lua/python/js engine, all of which are embedable.
123 2016-09-14T07:10:14  <wumpus> this is  a completely orthogonal concern from being able to chain values at the server side to avoid latency
124 2016-09-14T07:10:22  <wumpus> gmaxwell: :-(
125 2016-09-14T07:10:51  <gmaxwell> heh
126 2016-09-14T07:10:54  <wumpus> no interpreters in bitcoin please
127 2016-09-14T07:11:04  <wumpus> (apart from the one we really need)
128 2016-09-14T07:11:06  <dcousens> no more*
129 2016-09-14T07:11:32  <wumpus> having a powerful interpreter in the target executable makes exploitation so much easier
130 2016-09-14T07:12:02  <dcousens> maybe the CLI could just have an interactive mode with a LUA shell or something?
131 2016-09-14T07:12:04  <gmaxwell> well it could be a seperate interface, e.g. fancy-console.
132 2016-09-14T07:12:14  *** OdicforceSounds has quit IRC
133 2016-09-14T07:12:15  <dcousens> that way its not in the core as such
134 2016-09-14T07:12:19  <wumpus> dcousens: yes, that has been proposed, we could jut have a python-based interactive CLI
135 2016-09-14T07:12:24  <gmaxwell> I've only heard of five or six instances where the Js enable ethereum wallet stuff was used to trick people into sending away all their funds. :P
136 2016-09-14T07:12:26  <dcousens> bitcoin-cli*, that is
137 2016-09-14T07:12:47  <wumpus> dcousens: it could be based on ncurses, ipython, etc, that idea is old as the hills, yet no one wrote one :)
138 2016-09-14T07:12:55  <gmaxwell> but yes, not having it in the main binary would be obvious. I dunno, I think the utility is all that great.
139 2016-09-14T07:12:56  <wumpus> I thnk the current tools are jut ok
140 2016-09-14T07:12:57  <dcousens> wumpus: but jonasschnelli wrote this :P
141 2016-09-14T07:13:13  <wumpus> gmaxwell: right, it just doesn't matter that much, people like talking about this but it's no itch to scratch
142 2016-09-14T07:13:19  <wumpus> dcousens: yes, he did, and it works
143 2016-09-14T07:13:21  <wumpus> it's awesome
144 2016-09-14T07:13:24  <dcousens> haha,indeed
145 2016-09-14T07:13:41  <wumpus> let's not try to shed-paint it to death
146 2016-09-14T07:14:01  <gmaxwell> the standard python bitcoin json library that people get linked to is not at all reliable (gets disconnected and then throws exceptions, and has to be wrapped to be usable) and I don't see anyone even complaining about it.
147 2016-09-14T07:14:31  <wumpus> s/json/json-rpc   and you're right
148 2016-09-14T07:15:05  <dcousens> I've found it much easier to just always use a well-tested rpc library than use something "bitcoin tailored"
149 2016-09-14T07:15:23  <wumpus> which one are you using for your projects?
150 2016-09-14T07:15:25  <gmaxwell> json-rpc, yes.
151 2016-09-14T07:15:50  <wumpus> are there any RPC mechanisms that don't suck?
152 2016-09-14T07:16:34  <dcousens> wumpus: myself? as most of my work is in JS, I ended up using my own (haha) simply because all the batch interfaces sucked with error handling for well tested ones
153 2016-09-14T07:16:40  <wumpus> this presentation talks about people using MQTT for bitcoin wallet control, among other things:  https://media.defcon.org/DEF%20CON%2024/DEF%20CON%2024%20presentations/DEFCON-24-Lucas-Lundgren-Light-Weight%20Protocol-Critical-Implications.pdf   of course, without authentication or encryption and on the public internet :)
154 2016-09-14T07:17:11  <wumpus> tl.dr: don't do that
155 2016-09-14T07:17:17  <luke-jr> [06:53:12] <jonasschnelli> We could either have a special RPC command that result in parsing and executing multiple nested commands…but I don't think this would be clever. <-- this would be useful if the RPC client was over a netwrok, to avoid round-trips and such
156 2016-09-14T07:17:29  <dcousens> but, my point was, even my own implementation has nothing to do bitcoin as such, its just an RPC implementation
157 2016-09-14T07:17:43  <jonasschnelli> Luke-Jr: Yes. The server-side nesting could save bandwidth...
158 2016-09-14T07:18:17  <luke-jr> FWIW, I both dislike and like the idea at the same time. >_<
159 2016-09-14T07:18:22  <gmaxwell> wumpus: What did we do in the embeded python json-rpc used in the tests to make it reliable? (or did we not and the test just run fast enough that it isn't an issue?)
160 2016-09-14T07:18:28  <wumpus> server-side promise pipelining aves some bandwidth, but it mostly saves *latency*
161 2016-09-14T07:18:45  <jonasschnelli> Example: with RPC nesting, you could get the first outputs value of the second transactions of the bestblock with just one command ---> bandwith: a couple of bytes:
162 2016-09-14T07:19:05  <jonasschnelli> s/:/.
163 2016-09-14T07:19:16  <luke-jr> it's easier to support server-side expressions when bitcoind is split from the consensus stuff someday
164 2016-09-14T07:19:22  <wumpus> gmaxwell: which issue do you exactly mean?
165 2016-09-14T07:19:42  <luke-jr> ("support" in a human manner, not technical)
166 2016-09-14T07:20:34  <wumpus> gmaxwell: we did add some code to AuthServiceProxy._request to handle connection loss
167 2016-09-14T07:21:16  <luke-jr> I think there's some RPC mechanism which does expressions already BTW
168 2016-09-14T07:21:38  <wumpus> luke-jr: yes, it's old as the hills, "promise pipelining" is usually what it's called
169 2016-09-14T07:21:47  <wumpus> luke-jr: capnproto does it for example
170 2016-09-14T07:22:04  <wumpus> I think even CORBA could do it in the 90's, but don't look for that, it'll turn you crazy
171 2016-09-14T07:22:36  <wumpus> XCB (X11's RPC protocol) can do it too
172 2016-09-14T07:22:41  <gmaxwell> wumpus: varrious forms of connection loss; bitcoind either times out the persistant connection then any other interaction with the proxy object throws an exception, or I believe any case where bitcoind returns an error. I don't have a detailed list because I've always just soloved it by creating a wrapper that catches the exception and reconnects.
173 2016-09-14T07:23:28  <wumpus> gmaxwell: the high-level problem is that AuthServiceProxy uses python's http mechanism in an unconventional way: it opens a connection to a HTTP server, and instead of immediately queuing a bunch of requests, it keeps the connection open
174 2016-09-14T07:23:55  <wumpus> gmaxwell: which in principle is fine, web browsers also do that, but it needs code to handle the case where the remote server hung up on you
175 2016-09-14T07:24:14  <wumpus> gmaxwell: stock AuthServiceProxy has no code for that, our version (in the tests) does
176 2016-09-14T07:24:28  <luke-jr> it used to :x
177 2016-09-14T07:25:29  <wumpus> no, it assumes a unwordly friendly RPC server that never disconnects. Which used to be the case, approximately, before the switch to evhttpd, which introduced (like any other HTTP server in the world) connection timeouts
178 2016-09-14T07:25:53  <wumpus> (but network problems can *also* result in the http connection being closed, you just can't make the assumption it will always be there in the real world)
179 2016-09-14T07:26:17  <dcousens> wumpus: fwiw, I occassionally sync an address index from genesis just to see how fast it goes,  and its never the throughput/bandwidth that slows it up,  its the disk thrashing,  even in just a query-of txindex case
180 2016-09-14T07:26:42  <dcousens> s/address index/script index
181 2016-09-14T07:27:01  <gmaxwell> wumpus: yea, it was unreliable before-- just more obvious now.
182 2016-09-14T07:27:10  <wumpus> dcousens: thanks for actually benchmarking to find bottlenecks instead of making assumptions :) ("it must be RPC overhead")
183 2016-09-14T07:27:28  *** Cheeseo has quit IRC
184 2016-09-14T07:27:45  <luke-jr> oh, I see what it was: the older version that I thought worked simply never reused the connection :|
185 2016-09-14T07:28:07  <wumpus> luke-jr: that's also a valid way to do it, if you don't have too many requests or theyr're sparse
186 2016-09-14T07:28:30  <wumpus> it's how people usually use python's http: just open a connection per requests. The whole keep-alive thing is an optimization, not required
187 2016-09-14T07:28:42  <wumpus> it brings a lot of state and complexity
188 2016-09-14T07:29:32  <luke-jr> yet another unmaintained jgarzik project: https://github.com/jgarzik/python-bitcoinrpc/pull/49
189 2016-09-14T07:29:42  <gmaxwell> though without it, I imagine performance is poor (well even with it, performance is poor)
190 2016-09-14T07:30:09  <gmaxwell> it stinks mostly because everywhere recommends it, and it falls flat on its face... and I'm not sure what to recommend instead.
191 2016-09-14T07:30:54  <wumpus> well "performance is poor" is relative. keep-alive only helps with repeated requests. In many cases of using RPC you don't really care about performance of repeated reuqests at all
192 2016-09-14T07:31:43  <wumpus> many people just use the RPC through curl, or bitcoin-cli, which involves executing an external process per connection per command, and they don't complain either
193 2016-09-14T07:32:33  <dcousens> wumpus: heh, I'm using 16 concurrent RPC connections w/ 300 batched in each
194 2016-09-14T07:32:38  <wumpus> I think the problem is recommending using a broken-ish) library when the underlying mechanism is so trivial
195 2016-09-14T07:33:31  <dcousens> needless to say I found where my custom RPC commands were missing LOCKs pretty quick
196 2016-09-14T07:33:37  <gmaxwell> right, but "walk through the last N blocks to gather data" taking longer in python than it takes me to modify bitcoind and recompile feels a little silly.
197 2016-09-14T07:34:00  <wumpus> yes, if you're one of the few people that needs to do really dense requesting, there are various optimizations that help
198 2016-09-14T07:34:07  <wumpus> but that's not what JSON RPC was designed for in the first place
199 2016-09-14T07:34:31  <wumpus> it's a high-overhead protocol that was designed to be simple to use, not for low-latency high-throughput application
200 2016-09-14T07:34:32  <dcousens> wumpus: my point was, the RPC isn't whats slow, so no point changing it
201 2016-09-14T07:35:01  <wumpus> something like capnproto is designed for the latter
202 2016-09-14T07:35:28  <wumpus> howeer it's much harder to use, bind, and have basic setup, which is why companie generally offer JSON RPC or REST APIs
203 2016-09-14T07:35:43  <dcousens> lest we forget SOAP
204 2016-09-14T07:35:50  <wumpus> heh :)
205 2016-09-14T07:36:17  <gmaxwell> author of capnproto didn't seem to regard sending uninitilized memory in struct padding to remote peers as a serious security problem when I spoke to him a year or two ago.
206 2016-09-14T07:37:00  <wumpus> yes, security is another thing that is usually compromised on when latency is the most important thing
207 2016-09-14T07:37:19  <gmaxwell> in any case, after seeing that the python stuff was slow I tested with curl and it didn't appear to be the bitcoind side that was slow.
208 2016-09-14T07:37:25  <wumpus> it's a difficult choice, I think JSON-RPC was a fairly good choice as these things go
209 2016-09-14T07:37:41  <gmaxwell> I agree.
210 2016-09-14T07:38:55  * jonasschnelli thinks we should fork to use CORBA as our bitcoin scripting language and SOAP as http API :}
211 2016-09-14T07:39:04  <luke-jr> ._.
212 2016-09-14T07:39:04  *** dermoth has quit IRC
213 2016-09-14T07:39:13  <luke-jr> jonasschnelli: you forgot /s
214 2016-09-14T07:39:38  <jonasschnelli> Ah... I forgot to migrate the Qt client to a chrome html/css extension. :)
215 2016-09-14T07:39:46  <wumpus> the network and JSON processing on the bitcoind side is fast: the most common bottleneck would be the locking, which has nothing to do with the RPC mechanism. Or reading/deserializing blocks. Or...
216 2016-09-14T07:40:03  <wumpus> jonasschnelli: hehe, maybe if bitcoin was designed 10 years earlier
217 2016-09-14T07:40:09  <dcousens> jonasschnelli: I wonder when a ECDSA library will be written with pure CSS
218 2016-09-14T07:40:18  <jonasschnelli> lol
219 2016-09-14T07:40:32  <wumpus> dcousens: at least you can write a ECDSA library with just x86 mov instructions
220 2016-09-14T07:40:34  <luke-jr> >_<
221 2016-09-14T07:40:47  <luke-jr> https://github.com/xoreaxeaxeax/movfuscator
222 2016-09-14T07:40:56  <wumpus> yea
223 2016-09-14T07:41:10  <luke-jr> no need to write a new one, just build libsecp256k1..
224 2016-09-14T07:41:50  <jonasschnelli> How large will it be?
225 2016-09-14T07:41:59  <wumpus> moon-sized
226 2016-09-14T07:42:41  <jonasschnelli> Would be fun open the moon-sized library in IDA-PRO
227 2016-09-14T07:42:41  <luke-jr> dunno, never tried it
228 2016-09-14T07:43:40  <wumpus> yes I think that's the only reason why anyone would do something like that, throw off people trying to analyze it 'this can't be real x86 code'
229 2016-09-14T07:44:23  <luke-jr> s/people/software/
230 2016-09-14T07:44:28  <luke-jr> eg virus scanners
231 2016-09-14T07:44:39  <wumpus> well not just software, software is much easier to throw off
232 2016-09-14T07:45:00  <luke-jr> is it? ☺
233 2016-09-14T07:45:13  <dcousens> jonasschnelli: its possible to de-movuscate,isn't it? in which case eh?
234 2016-09-14T07:45:31  <luke-jr> dcousens: IIRC this one intentionally makes it hard
235 2016-09-14T07:46:31  *** PatBoy has quit IRC
236 2016-09-14T07:48:31  <wumpus> then again if the virus is 1000MB it kind of inhibits its own spread
237 2016-09-14T07:49:21  *** PatBoy has joined #bitcoin-core-dev
238 2016-09-14T07:54:00  *** MarcoFalke has joined #bitcoin-core-dev
239 2016-09-14T08:00:51  *** laurentmt has joined #bitcoin-core-dev
240 2016-09-14T08:02:29  *** laurentmt has quit IRC
241 2016-09-14T08:03:55  *** dermoth has joined #bitcoin-core-dev
242 2016-09-14T08:11:37  *** dermoth has quit IRC
243 2016-09-14T08:13:20  *** dermoth has joined #bitcoin-core-dev
244 2016-09-14T08:15:20  *** dermoth has quit IRC
245 2016-09-14T08:16:47  <GitHub179> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/57b34599b2de...881d7eaf29f7
246 2016-09-14T08:16:48  <GitHub179> bitcoin/master 36fa01f Cory Fields: net: only delete CConnman if it's been created...
247 2016-09-14T08:16:48  <GitHub179> bitcoin/master 881d7ea Wladimir J. van der Laan: Merge #8715: net: only delete CConnman if it's been created...
248 2016-09-14T08:17:02  <GitHub0> [bitcoin] laanwj closed pull request #8715: net: only delete CConnman if it's been created (master...fix-connman-shutdown) https://github.com/bitcoin/bitcoin/pull/8715
249 2016-09-14T08:18:15  *** davec has quit IRC
250 2016-09-14T08:30:10  *** davec has joined #bitcoin-core-dev
251 2016-09-14T08:50:34  *** dermoth has joined #bitcoin-core-dev
252 2016-09-14T08:51:25  *** dermoth has quit IRC
253 2016-09-14T08:56:12  *** AaronvanW has joined #bitcoin-core-dev
254 2016-09-14T08:57:13  *** Ylbam has joined #bitcoin-core-dev
255 2016-09-14T09:01:52  *** dermoth has joined #bitcoin-core-dev
256 2016-09-14T09:02:43  <jonasschnelli> Is there a reason why the last key in out HD scheme (and in BIP32) is non-hardened? Its m/0'/0'/0 for the first key and not m/0'/0'/0'
257 2016-09-14T09:02:58  <jonasschnelli> Or does it just don't matter at the last level
258 2016-09-14T09:24:25  <gmaxwell> because the wallet itself isn't intended to iterate at that level. The part below is subkeys.
259 2016-09-14T09:24:49  <gmaxwell> So every key the wallet gives out is strong against unzipping, but it could support repeated payment arrangements without address reuse.
260 2016-09-14T09:24:52  <gmaxwell> IIRC.
261 2016-09-14T09:28:27  *** MarcoFalke has left #bitcoin-core-dev
262 2016-09-14T09:34:46  *** netzin has joined #bitcoin-core-dev
263 2016-09-14T09:36:09  *** netzin has quit IRC
264 2016-09-14T09:40:35  *** MarcoFalke has joined #bitcoin-core-dev
265 2016-09-14T09:56:03  <MarcoFalke> wumpus: Can you elaborate? (random.random() *is* deterministic)
266 2016-09-14T09:56:13  <MarcoFalke> I think the seed is the current time, so we may not know that
267 2016-09-14T09:56:22  <MarcoFalke> But knowing the seed is not required.
268 2016-09-14T09:56:24  <wumpus> MarcoFalke: I like test that simply test the same thing every time
269 2016-09-14T09:56:32  <wumpus> there is no need to randomize here
270 2016-09-14T09:56:43  <MarcoFalke> So just set it to usehd=0
271 2016-09-14T09:56:52  <wumpus> just make every odd bitcoind a legacy wallet and every even one a HD one, or so
272 2016-09-14T09:57:55  <wumpus> I just don't see the point of random() here, there's a fair chance that it ends up testing all-hd or all-non-hd
273 2016-09-14T10:01:36  <MarcoFalke> Which is a legit case to test
274 2016-09-14T10:01:50  <MarcoFalke> But I can remove it, so we don't need the import random
275 2016-09-14T10:01:53  <wumpus> sure, but not randomly
276 2016-09-14T10:02:17  <wumpus> I like deterministic tests, esp in travis
277 2016-09-14T10:02:31  <MarcoFalke> But you want them to be constant?
278 2016-09-14T10:02:36  <wumpus> yes
279 2016-09-14T10:03:01  <MarcoFalke> Have you seen the print(extra_args) to make it reproducible in case of failure?
280 2016-09-14T10:03:05  <wumpus> otherwise there's a larger chance it will fail due to reasons unrelated to the code changes
281 2016-09-14T10:03:12  <wumpus> which confuses people
282 2016-09-14T10:03:32  *** rubensayshi has quit IRC
283 2016-09-14T10:04:03  <wumpus> tests run in travis should preferably be constant and deterministic. I'm ok with a random fuzzing jackpot testing method for running locally, but the tests running for every pull should be the same tests every time
284 2016-09-14T10:07:03  <wumpus> if you want to test with a combination of legacy and HD wallets in a test that's good, but just hardcode that
285 2016-09-14T10:09:20  <MarcoFalke> fine
286 2016-09-14T10:09:44  <MarcoFalke> Then we should write some new tests which do "parameter fuzzing" some time
287 2016-09-14T10:10:05  <MarcoFalke> Sadly this depends somewhat on rewrinting the arg parser
288 2016-09-14T10:10:23  <wumpus> sorry that I feel so strongly about htis, but I've seen too many random travis failures in the last months :)
289 2016-09-14T10:15:27  *** fengling has joined #bitcoin-core-dev
290 2016-09-14T10:37:58  *** laurentmt has joined #bitcoin-core-dev
291 2016-09-14T10:38:07  *** laurentmt has quit IRC
292 2016-09-14T10:55:38  *** Samdney has left #bitcoin-core-dev
293 2016-09-14T10:58:38  *** mol has joined #bitcoin-core-dev
294 2016-09-14T10:59:40  <wumpus> cfields_: oh shit oh shit oh shit https://github.com/bitcoin/bitcoin/pull/8653#issuecomment-246973266
295 2016-09-14T11:00:21  <wumpus> bitcoin core win64 cross-build is in a state of crap with ubuntu 16.04
296 2016-09-14T11:01:04  <wumpus> I thought I did this before, and didn't have any of these issues, but I must hae imagined it
297 2016-09-14T11:02:09  *** molz has quit IRC
298 2016-09-14T11:06:12  <GitHub17> [bitcoin] andersoyvind opened pull request #8720: Minor change in section name (master...patch-1) https://github.com/bitcoin/bitcoin/pull/8720
299 2016-09-14T11:39:18  *** JackH has joined #bitcoin-core-dev
300 2016-09-14T11:44:17  *** cryptapus has joined #bitcoin-core-dev
301 2016-09-14T11:44:17  *** cryptapus has joined #bitcoin-core-dev
302 2016-09-14T12:13:41  *** Chris_Stewart_5 has joined #bitcoin-core-dev
303 2016-09-14T12:29:10  *** Chris_Stewart_5 has quit IRC
304 2016-09-14T12:30:40  *** Chris_Stewart_5 has joined #bitcoin-core-dev
305 2016-09-14T12:35:45  *** cryptapus has quit IRC
306 2016-09-14T13:00:41  *** Alina-malina has joined #bitcoin-core-dev
307 2016-09-14T13:12:28  <GitHub88> [bitcoin] laanwj opened pull request #8722: bitcoin-cli: More detailed error reporting (master...2016_09_cli_http_error) https://github.com/bitcoin/bitcoin/pull/8722
308 2016-09-14T13:22:41  <GitHub10> [bitcoin] jonasschnelli opened pull request #8723: [Wallet] Add support for flexible BIP32/HD keypath-scheme (master...2016/09/hd_flex) https://github.com/bitcoin/bitcoin/pull/8723
309 2016-09-14T13:22:53  * jonasschnelli thinks he should review Luke-Jr multiwallet PR
310 2016-09-14T13:23:22  <wumpus> jonasschnelli: whatevery you do don't look at #8653
311 2016-09-14T13:24:41  <jonasschnelli> wumpus: hehe.. I did already and started disable --hardening wherever I can to avoid "possible startup problems". :P
312 2016-09-14T13:25:24  <wumpus> my new favourite number is "zu"
313 2016-09-14T13:26:11  <jonasschnelli> :-)
314 2016-09-14T13:26:18  <wumpus> I think it's that posix mode of gcc that completely throws everything off track
315 2016-09-14T13:26:35  <wumpus> (on windows)
316 2016-09-14T13:27:17  <wumpus> I'm going to locally revert the usage of c++11 threads, compile with the -win32 compiler then see what happens, I think everything will be fine
317 2016-09-14T13:27:48  *** Cheeseo has joined #bitcoin-core-dev
318 2016-09-14T13:27:48  *** Cheeseo has joined #bitcoin-core-dev
319 2016-09-14T13:29:10  *** cryptapus has joined #bitcoin-core-dev
320 2016-09-14T13:29:11  *** cryptapus has joined #bitcoin-core-dev
321 2016-09-14T13:32:17  <jonasschnelli> wumpus: Not sure if this helps.. but in another project I'm using mingw.thread.h (https://github.com/digitalbitbox/dbb-app/blob/master/src/mingw/mingw.thread.h)
322 2016-09-14T13:32:41  *** Chris_Stewart_5 has quit IRC
323 2016-09-14T13:33:20  <jonasschnelli> Then just include it with a #ifdef WIN32
324 2016-09-14T13:33:24  <jonasschnelli> Probably a huge hack
325 2016-09-14T13:34:17  *** Giszmo has joined #bitcoin-core-dev
326 2016-09-14T13:36:06  *** Guyver2 has joined #bitcoin-core-dev
327 2016-09-14T13:49:49  <wumpus> jonasschnelli: that makes sense; but I don't get it, why isn't it part of mingw-w64 itself?
328 2016-09-14T13:50:35  <wumpus> well it's less of a hack than fully fledged POSIX emulation mode on windows
329 2016-09-14T13:54:16  <wumpus> it's supposed to just support it, and on trusty it doe
330 2016-09-14T13:54:28  <wumpus> (it=std::mutex and friends)
331 2016-09-14T13:56:15  <jonasschnelli> wumpus: Yes. I don't know why its not part of mingw itself... I just added them and it worked fine, though, I'm not using stuff like the --enable-hardening we do at Core
332 2016-09-14T13:56:53  <wumpus> I doubt this has anything to do with hardening, I think with hardening it just detects the crazy memory corruption that happens sooner
333 2016-09-14T13:57:48  <GitHub93> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/881d7eaf29f7...a82e5d8220bb
334 2016-09-14T13:57:48  <GitHub93> bitcoin/master 1111ddb MarcoFalke: gitignore: Remove unused lines
335 2016-09-14T13:57:49  <GitHub93> bitcoin/master a82e5d8 MarcoFalke: Merge #8714: [qa] gitignore: Remove unused lines...
336 2016-09-14T13:58:03  <GitHub7> [bitcoin] MarcoFalke closed pull request #8714: [qa] gitignore: Remove unused lines (master...Mf1609-qaUnused) https://github.com/bitcoin/bitcoin/pull/8714
337 2016-09-14T13:58:43  <wumpus> also there's a mutex header in ./lib/gcc/x86_64-w64-mingw32/5.3-win32/include/c++/mutex
338 2016-09-14T13:58:56  <wumpus> maybe we need to #define something
339 2016-09-14T13:58:57  *** AaronvanW has quit IRC
340 2016-09-14T13:59:33  *** Chris_Stewart_5 has joined #bitcoin-core-dev
341 2016-09-14T14:00:33  <wumpus> well at least that's one more 'trivial' I've removed from a pull title
342 2016-09-14T14:02:43  <jonasschnelli> heh.. yes. That happens quick..
343 2016-09-14T14:05:32  <MarcoFalke> Hmm. Github allows us to write to branches that have a pull request open against bitcoin.
344 2016-09-14T14:05:34  <MarcoFalke> c.f. https://github.com/bitcoin/bitcoin/pull/8720/commits
345 2016-09-14T14:05:48  <MarcoFalke> Might come in handy but I'd prefer to disable it
346 2016-09-14T14:05:57  <MarcoFalke> (if possible)
347 2016-09-14T14:08:07  *** AaronvanW has joined #bitcoin-core-dev
348 2016-09-14T14:08:20  *** Chris_St1 has joined #bitcoin-core-dev
349 2016-09-14T14:09:12  <jonasschnelli> MarcoFalke: so you did amend commit force push to 8720?
350 2016-09-14T14:09:24  <MarcoFalke> Jup: https://github.com/blog/2247-improving-collaboration-with-forks
351 2016-09-14T14:09:34  <MarcoFalke> Apparently not possible to disable
352 2016-09-14T14:09:39  <jonasschnelli> Heh... that's handy but evil.
353 2016-09-14T14:10:47  *** AaronvanW_ has joined #bitcoin-core-dev
354 2016-09-14T14:10:58  *** Chris_Stewart_5 has quit IRC
355 2016-09-14T14:14:10  *** AaronvanW has quit IRC
356 2016-09-14T14:15:04  *** AaronvanW__ has joined #bitcoin-core-dev
357 2016-09-14T14:18:15  *** AaronvanW_ has quit IRC
358 2016-09-14T14:20:41  <wumpus> that can be really useful
359 2016-09-14T14:21:03  <wumpus> was of course already possible to do that locally, but ok
360 2016-09-14T14:21:11  <jonasschnelli> Yes. Thinking of all there cases where a nit holds back a PR
361 2016-09-14T14:21:52  <jonasschnelli> I think its much better to have a more public way then locally alter during the merge
362 2016-09-14T14:22:09  <wumpus> before the merge*
363 2016-09-14T14:24:17  <wumpus> if you do a local merge you can simply add your own commits before doing it, that's pretty public
364 2016-09-14T14:24:43  <jonasschnelli> It's public, but only in the commit list and not on the PR
365 2016-09-14T14:25:16  <wumpus> right. Just wanted to be clear I didn't mean anything as sneaky as squashing it into the merge commit, or into the author's commits
366 2016-09-14T14:25:24  <MarcoFalke> Also quite fiddly trying to emulate github-merge.py
367 2016-09-14T14:25:41  <MarcoFalke> I think it is fine to use the new feature when we announce in in the pr that is affected
368 2016-09-14T14:25:43  <wumpus> yes I've always intended to add a local mode to github-merge.py
369 2016-09-14T14:25:57  <wumpus> (I have a local version of github-merge but it's really messy)
370 2016-09-14T14:33:01  <GitHub197> [bitcoin] MarcoFalke opened pull request #8724: [qa] walletbackup: Sync blocks inside the loop (master...Mf1609-qaWalletBackupFix) https://github.com/bitcoin/bitcoin/pull/8724
371 2016-09-14T14:39:35  *** Chris_St1 has quit IRC
372 2016-09-14T14:55:07  <jonasschnelli> windows_hell, nice branch name
373 2016-09-14T14:56:10  * jonasschnelli wishes we could include a ubuntu vm and the linux binaries for the windows package
374 2016-09-14T14:56:39  *** Chris_St1 has joined #bitcoin-core-dev
375 2016-09-14T14:57:19  <wumpus> well there seems to be a shitload of issues, for now the best recommendation would be to just use trusty. The "zu" issue doesn't exist with the gitian executables does it?
376 2016-09-14T14:58:43  <wumpus> let's first try to figure out why the tests crash...
377 2016-09-14T15:01:36  *** AaronvanW__ is now known as AaronvanW
378 2016-09-14T15:01:37  <jonasschnelli> Yes. We should probably flag certain distros not compatible with out cross compile process or so
379 2016-09-14T15:08:30  *** Chris_St1 has quit IRC
380 2016-09-14T15:16:17  <wumpus> it only seemed like bitcoin-qt was working: I had disabled wallet support, apparently it works then
381 2016-09-14T15:16:33  <wumpus> windows is truly a mess on gcc 5.3
382 2016-09-14T15:16:58  <wumpus> the test crash before it even get to the BasicTestingSetup constructor
383 2016-09-14T15:17:06  <wumpus> at least I can reproduce it on wine....
384 2016-09-14T15:18:39  <achow101> Is this windows problem the same thing as I was having a while back?
385 2016-09-14T15:21:48  *** Chris_St1 has joined #bitcoin-core-dev
386 2016-09-14T15:31:25  *** Chris_St1 has quit IRC
387 2016-09-14T15:34:14  *** Chris_St1 has joined #bitcoin-core-dev
388 2016-09-14T15:41:28  <wumpus> there are at least three seperate issues here
389 2016-09-14T15:42:18  <achow101> the thing with mutex and not compiling
390 2016-09-14T15:42:22  <wumpus> 1) the std::mutex/thread c++11 compilation issue 2) something trange with libevent (the 'zu' issue) 3)  something with hardening
391 2016-09-14T15:43:07  <wumpus> some of these may partially overlap in cause or effect
392 2016-09-14T15:43:12  <wumpus> but it's a mess
393 2016-09-14T15:45:49  <achow101> Yeah, it looks like the same problem, it happened on ubuntu 15.10 (wily)
394 2016-09-14T15:47:01  <achow101> This is the discussion if it's of any help: https://botbot.me/freenode/bitcoin-core-dev/2016-08-08/?msg=70975840&page=1
395 2016-09-14T15:48:12  <wumpus> right - it's still an issue on 16.04
396 2016-09-14T15:50:29  *** dermoth has quit IRC
397 2016-09-14T15:51:50  *** achow101 has quit IRC
398 2016-09-14T15:52:11  *** achow101 has joined #bitcoin-core-dev
399 2016-09-14T15:57:18  *** spudowiar has joined #bitcoin-core-dev
400 2016-09-14T16:00:21  *** Guyver2 has quit IRC
401 2016-09-14T16:01:49  *** MarcoFalke has left #bitcoin-core-dev
402 2016-09-14T16:07:29  *** Chris_St1 has quit IRC
403 2016-09-14T16:11:15  <wumpus> ok, phew, at least verified that the libevent `zu` issue doesn't exist in the released 0.13.0 executables
404 2016-09-14T16:15:24  <sipa> good
405 2016-09-14T16:16:01  <wumpus> well travis RPC tests pass on windows, so it'd be surprising, but I don't understand why it suddenly starts happening with a certain gcc version
406 2016-09-14T16:16:51  <wumpus> the libevent version didn't change since 0.13.0, neither did our use of it
407 2016-09-14T16:20:51  *** spudowiar has quit IRC
408 2016-09-14T16:23:25  *** Chris_St1 has joined #bitcoin-core-dev
409 2016-09-14T16:28:17  *** aalex has quit IRC
410 2016-09-14T16:32:12  *** aalex has joined #bitcoin-core-dev
411 2016-09-14T16:32:58  <wumpus> achow101: which patches? I've also been testing w/ Ubuntu 16.04 and windows 10
412 2016-09-14T16:37:21  *** neha has quit IRC
413 2016-09-14T16:39:18  *** neha has joined #bitcoin-core-dev
414 2016-09-14T16:53:03  <GitHub12> [bitcoin] rebroad opened pull request #8725: Split debug for estimatefee into {estimatefee,2} (master...MoreGranularDebug2) https://github.com/bitcoin/bitcoin/pull/8725
415 2016-09-14T16:53:21  *** spudowiar has joined #bitcoin-core-dev
416 2016-09-14T16:56:33  <GitHub174> [bitcoin] rebroad opened pull request #8726: Move a bunch of fairly verbose debug messages from mempool to mempool2 (master...MoreGranularDebug3) https://github.com/bitcoin/bitcoin/pull/8726
417 2016-09-14T16:59:29  *** Guyver2 has joined #bitcoin-core-dev
418 2016-09-14T17:01:45  <GitHub66> [bitcoin] rebroad opened pull request #8727: Move logic for TX INVs together (master...MoreGranularDebug4) https://github.com/bitcoin/bitcoin/pull/8727
419 2016-09-14T17:05:23  <GitHub85> [bitcoin] rebroad opened pull request #8728: More granular debug5 (master...MoreGranularDebug5) https://github.com/bitcoin/bitcoin/pull/8728
420 2016-09-14T17:18:44  <GitHub29> [bitcoin] rebroad opened pull request #8729: More granular debug6 (master...MoreGranularDebug6) https://github.com/bitcoin/bitcoin/pull/8729
421 2016-09-14T17:31:56  *** dermoth has joined #bitcoin-core-dev
422 2016-09-14T17:32:38  *** rebroad has joined #bitcoin-core-dev
423 2016-09-14T17:34:16  <GitHub64> [bitcoin] laanwj opened pull request #8730: depends: Add libevent compatibility patch for windows (master...2016_09_libevent_windows_gcc_531) https://github.com/bitcoin/bitcoin/pull/8730
424 2016-09-14T17:35:39  <wumpus> sheesh how many debug pulls do we need...
425 2016-09-14T17:36:53  <paveljanik> 8(
426 2016-09-14T17:37:11  <paveljanik> 6+1. The last one combines all prev 8)
427 2016-09-14T17:37:48  <wumpus> we really should add a suggestion not to spam pull requests
428 2016-09-14T17:39:02  <wumpus> especially if they're all related
429 2016-09-14T17:48:22  <paveljanik> well, I know how he feels. Sometimes you know it is much easier to make it in steps. But doing steps separately and then all in one PR is nonsense...
430 2016-09-14T17:49:31  <paveljanik> let's close the steps PR and comment only the final one.
431 2016-09-14T17:49:49  *** Chris_St1 has quit IRC
432 2016-09-14T17:50:23  <paveljanik> rebroad, is it ok for you?
433 2016-09-14T17:56:43  <rebroad> paveljanik, hi
434 2016-09-14T17:57:37  <rebroad> paveljanik, I find the whole process quite confusing... in my experience my smaller pull requests get merged but the larger ones don't... so this way I'm trying to hedge my bets and provide small ones and a larger one to work out which is preferred.
435 2016-09-14T17:58:03  <rebroad> paveljanik, not sure what you mean by "the final one"
436 2016-09-14T17:58:24  <paveljanik> the final one - the one containing all (almost) the previous
437 2016-09-14T17:58:27  <rebroad> do you mean the one with all the commits in? and close the others?
438 2016-09-14T17:58:32  <cfields_> wumpus: ok, diving into the win32 mess. I'd written it off as user error, but looks like I need to familiarize myself with what's going on now.
439 2016-09-14T17:58:54  <paveljanik> 8729
440 2016-09-14T17:58:55  <rebroad> paveljanik, I'm happy to do this, but I'm not sure if this is would the other devs would prefer also
441 2016-09-14T17:59:32  <paveljanik> well, some changes are wrong, yes, but they can tell you in one PR.
442 2016-09-14T17:59:54  <rebroad> paveljanik, wrong?
443 2016-09-14T18:00:09  <paveljanik> yes
444 2016-09-14T18:00:20  <paveljanik> lets wait for comments
445 2016-09-14T18:00:41  <paveljanik> I have already added approx. 5 comments.
446 2016-09-14T18:00:52  <rebroad> paveljanik, ah, thank you
447 2016-09-14T18:01:00  <GitHub154> [bitcoin] rebroad opened pull request #8731: Debug headers received ("block" for new block announcement, "block2" … (master...DebugHeadersReceived) https://github.com/bitcoin/bitcoin/pull/8731
448 2016-09-14T18:01:32  <rebroad> paveljanik, is there a way to easily go to my pull requests that have had comments left? i.e. is there an inbox for these comments somewhere on github?
449 2016-09-14T18:02:43  <paveljanik> rebroad, yes, please read https://help.github.com/articles/about-discussions-in-issues-and-pull-requests/
450 2016-09-14T18:02:58  <rebroad> thanks
451 2016-09-14T18:03:04  <paveljanik> and especially "Further reading"
452 2016-09-14T18:04:32  <rebroad> paveljanik, what did you mean by "why?" in #8728?
453 2016-09-14T18:07:04  <paveljanik> expanded there...
454 2016-09-14T18:08:28  <rebroad> paveljanik, #8727 you asked what other inv.type I can see.. not sure why you are asking this
455 2016-09-14T18:09:06  <rebroad> I just moved the code that was doing the same thing as the code above into the same if statement.. which is needed to avoid messier code with the later commits also
456 2016-09-14T18:11:42  <wumpus> this is starting to be annoying, please stop pushing debug pull requests
457 2016-09-14T18:11:54  <rebroad> paveljanik, #8725 have added a description now... I guess the "why" is a reasonable question given it was absent.
458 2016-09-14T18:12:17  <rebroad> wumpus, what is your issue with debug pull requests?
459 2016-09-14T18:12:17  <wumpus> most debugging information is only intersting for yourself during a debugging session, there is no reason to merge them upstream
460 2016-09-14T18:12:39  <paveljanik> there are many of them and useless - wumpus wrote it above...
461 2016-09-14T18:12:39  <wumpus> a) there are way too many already b) it's mostly ego-commits, not intersting to other people
462 2016-09-14T18:13:28  <rebroad> wumpus, from the pull requests I have seen I would say that various developers have found a number of my debug pull requests useful, and they facilitate better understanding of what is happening. It would be unreasonable to expect everyone to have to make their own changed to obtain useful debug information.
463 2016-09-14T18:13:49  <wumpus> try to be focused when you submit pulls upsteam. Every one of us likely has tons of patches adding personal debugging information to solve specific problems, but it's not in general useful for others and you're wasting review time
464 2016-09-14T18:14:46  <wumpus> it wouldn't be a problem if this was one or two pulls, but you keep pushing this on
465 2016-09-14T18:15:02  <rebroad> wumpus, i just spend 3 days having to rebase all these debug commits due to changes that were made recently.. I am getting very very fed up with the amount of time wasted rebasing when I could be spending that time adding functionality to bitcoin. this is why this needs to get merged!
466 2016-09-14T18:15:08  <wumpus> do you really expect people to review "more granular debug6"?
467 2016-09-14T18:15:20  <rebroad> i have a life, which I'd like to spend living rather than rebasing
468 2016-09-14T18:15:40  <wumpus> that's not a reason to merge it though, to merge it it needs to actually be considered useful, the project doesn't exist to save you work
469 2016-09-14T18:15:44  <wumpus> we all have a life
470 2016-09-14T18:15:59  <wumpus> it's kind of insulting to suggest you're the only one
471 2016-09-14T18:16:01  <paveljanik> so why do you spend your time with useless stuff instead of The functionality?
472 2016-09-14T18:16:05  <wumpus> and you can just burden us with your shit
473 2016-09-14T18:16:12  <rebroad> wumpus, it's not just saving me work. I wouldn't write it if I thought it was only useful for me
474 2016-09-14T18:16:34  <rebroad> i agree that some of the commits are trivial...
475 2016-09-14T18:16:42  <rebroad> wumpus, what do you mean by "ego-commits"?
476 2016-09-14T18:16:42  <wumpus> well some of them may be useful to others
477 2016-09-14T18:16:47  <wumpus> but it's the sheer volume
478 2016-09-14T18:17:08  <wumpus> everyone tries to make time to write and review patches that add functionality and fix bugs
479 2016-09-14T18:17:10  <wumpus> not just debugging spam
480 2016-09-14T18:17:48  <wumpus> <paveljanik> so why do you spend your time with useless stuff instead of The functionality? <- very good question
481 2016-09-14T18:18:07  <wumpus> we have 402 issues and 140 pull requests open at this moment
482 2016-09-14T18:18:26  <paveljanik> yup, pick one of the issues and try to solve it!
483 2016-09-14T18:18:39  <wumpus> lots of actual problems that need to be solved, that are experienced by users
484 2016-09-14T18:18:56  <rebroad> wumpus, you say sheer volume, but I'm not sure what you mean. do you mean pull reqs, or lines of code?
485 2016-09-14T18:19:00  <wumpus> and yes, some of them may be bullshit, but let's try not to add too much more
486 2016-09-14T18:19:09  <wumpus> rebroad: number of pull requests inthis case
487 2016-09-14T18:19:35  <wumpus> (same could hold for lines of changed code, in some cases)
488 2016-09-14T18:20:26  <rebroad> wumpus, ok, I will close the smaller ones as paveljanik suggests
489 2016-09-14T18:20:41  <wumpus> yes just merge some together if they are related to the same concern
490 2016-09-14T18:20:56  <wumpus> and at least make some effort to name/describe them
491 2016-09-14T18:21:02  <wumpus> 'more granular debug5' really sucks as name
492 2016-09-14T18:21:22  <GitHub9> [bitcoin] rebroad closed pull request #8484: More granular debug (master...MoreGranularDebug) https://github.com/bitcoin/bitcoin/pull/8484
493 2016-09-14T18:21:26  <rebroad> wumpus, you don't mince your words, do you :)
494 2016-09-14T18:21:39  <wumpus> rebroad: sorry this has been annoying me for a while
495 2016-09-14T18:22:01  <rebroad> spending 3 days rebasing has been annoying me too ...
496 2016-09-14T18:22:07  <wumpus> well then don
497 2016-09-14T18:22:08  *** Guyver2_ has joined #bitcoin-core-dev
498 2016-09-14T18:22:08  <wumpus> 't
499 2016-09-14T18:22:37  <rebroad> if i need to understand the code and how it functions I do... unless these commits get merged
500 2016-09-14T18:22:57  <rebroad> ok, I am oversimplifying a little
501 2016-09-14T18:23:06  <rebroad> but it all helps, IMHO
502 2016-09-14T18:23:08  <wumpus> you can understand code without logging everything, or by adding temporary logging
503 2016-09-14T18:23:17  <wumpus> or by stepping through with a debugger
504 2016-09-14T18:23:27  <wumpus> many ways taht don't involve adding logging statements to the upstream code
505 2016-09-14T18:23:41  <rebroad> part of me needs to disagree with you as otherwise I've spent a lot of time for very little reason :-s
506 2016-09-14T18:24:16  <wumpus> I'm not saying it's all useless
507 2016-09-14T18:24:41  *** Guyver2 has quit IRC
508 2016-09-14T18:24:42  *** Guyver2_ is now known as Guyver2
509 2016-09-14T18:24:43  <rebroad> there are some more commits yet to come that might help to reveal the usefulness of the so far seemingly useless ones
510 2016-09-14T18:24:45  <wumpus> just try to have some discipline, spend time explaining to others why something is useful, instead of just adding pulls with very little description
511 2016-09-14T18:26:25  <rebroad> but yes, the mempool2 and estimatefee2 ones were a bit trivial... estimatefee2 more so... but still useful to an extent...  I'd still like to know what you mean by ego-commits... i do wonder how much ego is behind my struggle to get things merged.
512 2016-09-14T18:26:51  <wumpus> and maybe explain the overall plan, people get lost if they only see details instead of where something is going
513 2016-09-14T18:26:58  <rebroad> wumpus, you are right.. i do need to be less lazy when it comes to explaining
514 2016-09-14T18:27:35  <rebroad> wumpus, thank you for your feedback and patience
515 2016-09-14T18:27:38  <wumpus> an ego-commit is something that is useful to yourself, for example some feature that you need, or some piece of information that you specifically need, but without regard whether it's useful for other users
516 2016-09-14T18:28:13  <rebroad> wumpus, I would find it hard to believe that some features are useful to only one person
517 2016-09-14T18:28:14  <wumpus> and then to stop having to rebase it yourself you try to push it upstream
518 2016-09-14T18:29:05  <wumpus> well it tends to happen if something is only a perceived need, someone is really held up about something
519 2016-09-14T18:29:21  <rebroad> there seems to be a general attitude among many developers that "debug" commits are a waste of time.. I don't quite understand why that is the popular viewpoint
520 2016-09-14T18:29:35  <wumpus> maybe it's useful to one other person :) but all code needs to be maintained, so there has to be some threashold
521 2016-09-14T18:29:49  <wumpus> well debugging is something kind of personal to developers
522 2016-09-14T18:29:55  <wumpus> what information do you need
523 2016-09-14T18:29:57  <wumpus> for what you're doing
524 2016-09-14T18:30:10  <wumpus> for the problem you're trying to find
525 2016-09-14T18:30:24  <rebroad> a lot of people, non-developers especially often ask for more feedback on what their full-node is doing... they want to see what it is doing...
526 2016-09-14T18:30:38  <rebroad> so in a sense, this debug info might benefit non-developers more than developers
527 2016-09-14T18:30:40  <wumpus> e.g. to find a wine issue I"m now instrumenting some code to log every executed instruction
528 2016-09-14T18:30:51  <wumpus> that's not something anyone would ever want upstream :-)
529 2016-09-14T18:31:03  <rebroad> i could code something prettier like a matrix style green scrolly thing.. but that would move away from actual useful information then!
530 2016-09-14T18:31:30  <wumpus> if I became really obsessed about this, and wanted this a default part of bitcoin, and became angry if other developers disagreed, that would be an ego commit
531 2016-09-14T18:32:14  <wumpus> yes on a high level I agree with you, it'd be nice to have some more insight into what a node is doing
532 2016-09-14T18:32:28  <wumpus> I kind of like statoshi's approach
533 2016-09-14T18:33:07  <wumpus> but I'm not sure e.g. splitting up debug into a zillion categories, or adding lots of messages, is the way to go
534 2016-09-14T18:33:29  <wumpus> haha if you'd add that to the GUI it may well be accepted
535 2016-09-14T18:33:42  <rebroad> if it's granular enough, then no one should need to add adhoc debug message as you are suggesting... there'd be a debug message already there that they can activate just by editing bitcoin.conf
536 2016-09-14T18:33:44  <wumpus> jonasschnelli did a screensaver-like thing once that showed statistics
537 2016-09-14T18:33:49  <rebroad> and they'd not need to recompile etc
538 2016-09-14T18:33:57  <rebroad> which wastes a lot of cpy, energy, etc...
539 2016-09-14T18:33:59  <rebroad> cpu
540 2016-09-14T18:34:00  <wumpus> but gave up on it unfortuantely
541 2016-09-14T18:34:12  <jonasschnelli> https://github.com/bitcoin/bitcoin/pull/5896
542 2016-09-14T18:34:27  <rebroad> :)
543 2016-09-14T18:34:28  <jonasschnelli> I haven't gave it up... I just came with a more stable solution for a first start
544 2016-09-14T18:34:30  <jonasschnelli> The mempool stats
545 2016-09-14T18:34:40  <rebroad> I love the mempool graphs..
546 2016-09-14T18:34:41  <jonasschnelli> Once this will be merged, more can be added
547 2016-09-14T18:34:56  <jonasschnelli> Nodes / Tx / Mempool just don't fit in one screem
548 2016-09-14T18:34:57  <rebroad> although it would be great if I could point to the graph and it would tell me the X-Y co-ords of the point closest to the pointer
549 2016-09-14T18:34:59  <kanzure> not clear to me if rebroad has used gdb and code stepping
550 2016-09-14T18:35:01  <wumpus> well i think there's always someone that wants to add an ad-hoc debug messgae, I don't think it's possible to be complete in that regard
551 2016-09-14T18:35:44  <kanzure> if there needs to be metrics for user display then probably a more general frmaewokr would be useful, rather than individual debugger additions
552 2016-09-14T18:35:46  <rebroad> kanzure, i have used gdb.. but i think it's of limited use, in my experience and not as clear as some good debug messages, imho
553 2016-09-14T18:36:01  <wumpus> the linux kernel has something called 'tracepoints' which allows inserting print or other hooks at certain points to print information on the fly
554 2016-09-14T18:36:06  <wumpus> I think a similar thing exists for user space
555 2016-09-14T18:36:53  <kanzure> wumpus: probably there's a very noisy compiler option for this...
556 2016-09-14T18:36:56  <wumpus> most annoying about gdb is <value optimized out>
557 2016-09-14T18:37:45  <GitHub167> [bitcoin] rebroad closed pull request #8728: More granular debug5 (master...MoreGranularDebug5) https://github.com/bitcoin/bitcoin/pull/8728
558 2016-09-14T18:38:04  <kanzure> -finstrument-functions
559 2016-09-14T18:38:06  <kanzure> hehehe
560 2016-09-14T18:39:16  <wumpus> another one is systemtap probe points
561 2016-09-14T18:39:53  <GitHub28> [bitcoin] rebroad closed pull request #8725: Split debug for estimatefee into {estimatefee,2} (master...MoreGranularDebug2) https://github.com/bitcoin/bitcoin/pull/8725
562 2016-09-14T18:40:08  <GitHub62> [bitcoin] rebroad closed pull request #8726: Move a bunch of fairly verbose debug messages from mempool to mempool2 (master...MoreGranularDebug3) https://github.com/bitcoin/bitcoin/pull/8726
563 2016-09-14T18:40:22  <kanzure> cool.
564 2016-09-14T18:40:28  <GitHub25> [bitcoin] rebroad closed pull request #8727: Move logic for TX INVs together (master...MoreGranularDebug4) https://github.com/bitcoin/bitcoin/pull/8727
565 2016-09-14T18:41:02  <wumpus> or gdb's "The dynamic printf command dprintf combines a breakpoint with formatted printing of your program's data to give you the effect of inserting printf calls into your program on-the-fly, without having to recompile it."
566 2016-09-14T18:41:26  <rebroad> someone on github mentioned that everytime i change the pull title, or close/open that 1200 emails get sent... is this true?
567 2016-09-14T18:41:49  <achow101> rebroad: yes.
568 2016-09-14T18:41:51  <wumpus> well I think 1200 is kind of overstating it
569 2016-09-14T18:42:03  <rebroad> if so.... this sounds a little verbose... i agree with paveljanik's comment that things can be concise to one person while verbose to another
570 2016-09-14T18:42:23  <rebroad> which is why I felt more granular was the answer..
571 2016-09-14T18:42:27  <kanzure> it's the same thing with mailing lists too. your email will be sent to many thousands of people.
572 2016-09-14T18:42:27  <wumpus> but at least the maintainers and people that have posted to an issue get a notification
573 2016-09-14T18:42:43  <achow101> wumpus: I don't think it is actually overstating it. Anyone who watches the repo will get an email and there are 1206 watchers
574 2016-09-14T18:42:47  <kanzure> and if everyone is reading their email then a 5 minute email ~= economic catastrophe of time spent reading
575 2016-09-14T18:43:29  <wumpus> achow101: I'm not sure watchers get a mail for everything that happens in a repostiory, but you could be right
576 2016-09-14T18:43:47  <kanzure> wumpus: are you a watcher?
577 2016-09-14T18:44:00  <achow101> I'm watching and I get mails for every comment and commit to every pr or issue
578 2016-09-14T18:44:20  <kanzure> i sort of assumed all the regulars were using the watchers feature.
579 2016-09-14T18:44:42  <kanzure> ah maybe it's something about org members on github. nevermind.
580 2016-09-14T18:44:46  <wumpus> kanzure: I'm repo owner, so I get a lot of mail from github, but I didn't realize watchers get all that too
581 2016-09-14T18:44:51  <rebroad> paveljanik, was it you who sent me a github article earlier on here? i am struggling to find the link you quoted earlier :-s
582 2016-09-14T18:45:12  <paveljanik> rebroad, see topic, it contains a link to log of this channel...
583 2016-09-14T18:45:13  <kanzure> you want https://help.github.com/articles/about-discussions-in-issues-and-pull-requests/
584 2016-09-14T18:45:19  <wumpus> I don't amange to read most of it though
585 2016-09-14T18:45:21  *** Guyver2_ has joined #bitcoin-core-dev
586 2016-09-14T18:45:59  <wumpus> still looking for a way to filter when someone @'s me from the rest of the github spam
587 2016-09-14T18:46:41  *** Guyver2 has quit IRC
588 2016-09-14T18:46:45  *** Guyver2_ is now known as Guyver2
589 2016-09-14T18:46:49  <kanzure> wumpus: i think github is supposed to do that on its homepage, but IIRC nobody ever views the github homepage :)
590 2016-09-14T18:46:56  <achow101> Does anyone think that the pull request review feature on github might be useful?
591 2016-09-14T18:47:13  <wumpus> really those should end up in my personal mailbox instead of with the bulk
592 2016-09-14T18:47:31  <wumpus> achow101: haven't looked at it yet
593 2016-09-14T18:47:43  *** Soligor has joined #bitcoin-core-dev
594 2016-09-14T18:47:45  *** achow101_ has joined #bitcoin-core-dev
595 2016-09-14T18:48:05  <achow101> part of it looks like basically preventing PRs from merging without ACKs and ACKs are built in
596 2016-09-14T18:48:30  <paveljanik> wumpus, github mails contain this weird CC: ... Comment <comment@noreply.github.com> or Mention or ...
597 2016-09-14T18:48:34  <wumpus> ok that sounds fairly useful
598 2016-09-14T18:48:37  <kanzure> achow101: are those ACKs tied to specific commit ids?
599 2016-09-14T18:48:58  <achow101> Dunno, haven't fully checked it out yet. I just read over the help article they linked
600 2016-09-14T18:49:21  <wumpus> paveljanik: that could be key
601 2016-09-14T18:53:28  *** Chris_St1 has joined #bitcoin-core-dev
602 2016-09-14T18:53:31  *** achow101_ has quit IRC
603 2016-09-14T18:55:07  <wumpus> paveljanik: yes, searching for to:mention@noreply.github.com seems to do what I want
604 2016-09-14T18:55:58  *** MarcoFalke has joined #bitcoin-core-dev
605 2016-09-14T19:00:09  *** Chris_St1 has quit IRC
606 2016-09-14T19:00:11  <GitHub131> [bitcoin] rebroad opened pull request #8733: More granular debug [WIP] (master...MoreGranularDebug.wip) https://github.com/bitcoin/bitcoin/pull/8733
607 2016-09-14T19:00:28  *** Chris_Stewart_5 has joined #bitcoin-core-dev
608 2016-09-14T19:00:44  <rebroad> this is the version with 22 commit.... hopefuly if anyone can be bothered to look they can get a clearer idea of where it was all going..
609 2016-09-14T19:00:46  *** achow101_ has joined #bitcoin-core-dev
610 2016-09-14T19:03:17  <wumpus> much better
611 2016-09-14T19:13:10  <GitHub147> [bitcoin] MarcoFalke closed pull request #8731: Debug headers received ("block" for new block announcement, "block2" … (master...DebugHeadersReceived) https://github.com/bitcoin/bitcoin/pull/8731
612 2016-09-14T19:19:50  *** spudowiar has quit IRC
613 2016-09-14T19:21:19  <btcdrak> wow, what is this new "review" thing on Github
614 2016-09-14T19:21:25  *** BunBun has joined #bitcoin-core-dev
615 2016-09-14T19:22:16  <BunBun> quit
616 2016-09-14T19:22:19  *** BunBun has quit IRC
617 2016-09-14T19:27:43  *** achow101_ has quit IRC
618 2016-09-14T19:30:27  *** Chris_Stewart_5 has quit IRC
619 2016-09-14T19:31:42  *** assder has quit IRC
620 2016-09-14T19:40:09  <GitHub198> [bitcoin] rebroad closed pull request #8596: [WIP] Feeler code and debugging. (master...FeelerFixes) https://github.com/bitcoin/bitcoin/pull/8596
621 2016-09-14T19:45:32  *** Chris_Stewart_5 has joined #bitcoin-core-dev
622 2016-09-14T19:51:06  *** timothy has quit IRC
623 2016-09-14T19:51:09  *** drizztbsd has joined #bitcoin-core-dev
624 2016-09-14T19:51:42  *** drizztbsd is now known as timothy
625 2016-09-14T20:00:47  *** achow101_ has joined #bitcoin-core-dev
626 2016-09-14T20:00:58  *** achow101_ has quit IRC
627 2016-09-14T20:07:46  *** spudowiar has joined #bitcoin-core-dev
628 2016-09-14T20:23:31  *** aalex has quit IRC
629 2016-09-14T20:25:24  <cfields_> wumpus: ok, i've finally had enough of the system package crap that we can't control. I'm adding native toolchains to depends, along with a BOOTSTRAP option, false by default. Bootstrapping can be done in gitian and uploaded, so that they're simply fetched 99% of the time.
630 2016-09-14T20:29:37  <sipa> cfields_: so gcc being part of the build for all platforms?
631 2016-09-14T20:30:12  <cfields_> sipa: that's what I'm considering, yes
632 2016-09-14T20:31:24  <sipa> awesome.
633 2016-09-14T20:32:07  <cfields_> I'm sure it'll spiral downward with arguments about where to stop, but gcc/binutils seems like a reasonable start.
634 2016-09-14T20:33:11  *** aalex has joined #bitcoin-core-dev
635 2016-09-14T20:33:35  <sipa> cfields_: do you know nixos?
636 2016-09-14T20:34:02  <cfields_> sipa: not really
637 2016-09-14T20:34:48  <cfields_> sipa: looks interesting. Is it intended to be deterministic across builders?
638 2016-09-14T20:35:24  <sipa> it is not
639 2016-09-14T20:35:53  <sipa> but it uses hashes of build instructions to identify packages
640 2016-09-14T20:36:19  <cfields_> ah, nice. sounds familiar :)
641 2016-09-14T20:36:33  <MarcoFalke> Isn't this what docker does?
642 2016-09-14T20:37:46  <sipa> docker builds from source?
643 2016-09-14T20:40:32  <cfields_> docker just does whatever you tell it to. but yes, i'd assume it uses the hash of the recipe for checkpointing
644 2016-09-14T20:50:39  <luke-jr> btcdrak: paveljanik: where do you see the new review thing⁇
645 2016-09-14T20:50:57  <btcdrak> luke-jr: look on the diff tab
646 2016-09-14T20:51:34  <luke-jr> cfields_: we need to support building outside gitian anyway, so not sure I see the point
647 2016-09-14T20:51:35  <btcdrak> luke-jr: "the files changed" tab.
648 2016-09-14T20:52:19  <luke-jr> huh
649 2016-09-14T20:52:22  <luke-jr> interesting
650 2016-09-14T20:58:16  <MarcoFalke> luke-jr: I think the goal is to solve the cross compile issues with depends.
651 2016-09-14T21:00:40  <MarcoFalke> So if you fetch the (gitian-built) toolchain, you can just build outside gitian like you do today
652 2016-09-14T21:22:42  *** spudowiar has quit IRC
653 2016-09-14T21:35:10  *** Guyver2 has quit IRC
654 2016-09-14T22:29:20  *** laurentmt has joined #bitcoin-core-dev
655 2016-09-14T22:40:38  *** laurentmt has quit IRC
656 2016-09-14T22:55:52  *** achow101_ has joined #bitcoin-core-dev
657 2016-09-14T23:02:42  *** wjx has joined #bitcoin-core-dev
658 2016-09-14T23:21:28  *** MarcoFalke has quit IRC
659 2016-09-14T23:21:55  *** davec has quit IRC
660 2016-09-14T23:22:25  *** davec has joined #bitcoin-core-dev
661 2016-09-14T23:49:18  *** afk11 has joined #bitcoin-core-dev
662 2016-09-14T23:49:18  *** afk11 has quit IRC
663 2016-09-14T23:49:18  *** afk11 has joined #bitcoin-core-dev
664 2016-09-14T23:59:53  *** achow101_ has quit IRC