1 2016-06-02T00:13:49  *** kadoban has quit IRC
  2 2016-06-02T00:32:09  *** TomMc has quit IRC
  3 2016-06-02T00:50:47  *** belcher has quit IRC
  4 2016-06-02T00:51:15  *** Ylbam has quit IRC
  5 2016-06-02T00:52:46  *** kadoban has joined #bitcoin-core-dev
  6 2016-06-02T00:56:17  *** frankenmint has quit IRC
  7 2016-06-02T01:12:14  *** justanotheruser has quit IRC
  8 2016-06-02T01:12:37  *** justanotheruser has joined #bitcoin-core-dev
  9 2016-06-02T01:41:01  *** Alopex has quit IRC
 10 2016-06-02T01:42:06  *** Alopex has joined #bitcoin-core-dev
 11 2016-06-02T01:54:13  *** xiangfu has joined #bitcoin-core-dev
 12 2016-06-02T02:17:45  *** moli has joined #bitcoin-core-dev
 13 2016-06-02T02:20:43  *** fengling has joined #bitcoin-core-dev
 14 2016-06-02T02:23:00  *** Chris_Stewart_5 has quit IRC
 15 2016-06-02T02:25:08  *** fengling has quit IRC
 16 2016-06-02T02:29:41  *** fengling has joined #bitcoin-core-dev
 17 2016-06-02T02:52:01  *** Alopex has quit IRC
 18 2016-06-02T02:53:06  *** Alopex has joined #bitcoin-core-dev
 19 2016-06-02T03:17:47  *** jcorgan has quit IRC
 20 2016-06-02T03:21:02  *** Alopex has quit IRC
 21 2016-06-02T03:22:07  *** Alopex has joined #bitcoin-core-dev
 22 2016-06-02T03:25:29  *** fengling has quit IRC
 23 2016-06-02T03:37:26  *** fengling has joined #bitcoin-core-dev
 24 2016-06-02T03:58:17  *** jtimon has quit IRC
 25 2016-06-02T04:22:29  *** gribble has quit IRC
 26 2016-06-02T04:23:12  *** grassass has joined #bitcoin-core-dev
 27 2016-06-02T04:23:27  *** molz has joined #bitcoin-core-dev
 28 2016-06-02T04:26:41  *** moli has quit IRC
 29 2016-06-02T04:31:51  *** gribble has joined #bitcoin-core-dev
 30 2016-06-02T04:34:52  *** kadoban has quit IRC
 31 2016-06-02T04:39:44  *** fengling has quit IRC
 32 2016-06-02T04:44:44  *** ghtdak has quit IRC
 33 2016-06-02T04:46:01  *** Alopex has quit IRC
 34 2016-06-02T04:47:06  *** Alopex has joined #bitcoin-core-dev
 35 2016-06-02T04:47:53  *** ghtdak has joined #bitcoin-core-dev
 36 2016-06-02T04:58:59  *** fengling has joined #bitcoin-core-dev
 37 2016-06-02T05:44:30  <gmaxwell> Our current addnode behavior is more than a little obnoxious.  We won't bring up an addnode if it would leave us with more than the maximum outbound connections, even if none of our existing connections are addnoded.  This means that if your network burps you can end up disconnecting all your addnoded peers, and then fill them back in with random peers.
 38 2016-06-02T05:45:25  <gmaxwell> Even sitting around manually disconnectnode-ing outbounds that weren't my addnodes took quite a while before it managed to try the addnode before connecting something else.
 39 2016-06-02T05:50:16  *** imacomput has joined #bitcoin-core-dev
 40 2016-06-02T05:50:48  <gmaxwell> I think the behavior should be, we should track if peers are addnodeed.  If we have unconnected addnodes and less han MAX connected addnodes, and less than MAX+1 total connected. ... try adding an addnode.
 41 2016-06-02T05:51:48  <btcdrak> gmaxwell: I have the same complaint. I just never articulated it.
 42 2016-06-02T05:52:06  <gmaxwell> I think we introduced this misbehavior around 0.9.x or something.
 43 2016-06-02T05:52:18  <gmaxwell> a long time ago addnode would just go over the connection limit.
 44 2016-06-02T05:52:56  <gmaxwell> ... and then seperately, every N seconds,  if we have >MAX outbound connections, pick one to evict. (where it would avoid evicting whitelisted and addnode peers, then ones that have recently sent us blocks, then txn, the prefer to keep long uptime).
 45 2016-06-02T05:53:39  <gmaxwell> The same eviction process could then be used to rotate the outbound connections. (just randomly try to connect to too many, and then let eviction do its thing.)
 46 2016-06-02T05:55:50  *** imacomput has quit IRC
 47 2016-06-02T06:02:00  *** calibre720 has joined #bitcoin-core-dev
 48 2016-06-02T06:03:26  <sipa> gmaxwell: indeed, and we already have eviction code
 49 2016-06-02T06:04:11  <sipa> just adding a bool to select for incoming or outgoing would be a large part already
 50 2016-06-02T06:08:12  <gmaxwell> the current eviction code only evicts incoming. I think our strategy for outgoing should be somewhat different... in particular, we have far fewer outgoing.
 51 2016-06-02T06:08:44  <sipa> for rotation we'd need better logic
 52 2016-06-02T06:08:59  <sipa> but for preferring addnode over random outgoing maybe not so much
 53 2016-06-02T06:09:14  <midnightmagic> +1 for addnode prioritization :-)
 54 2016-06-02T06:10:47  <gmaxwell> yes. I think sort by whitelist/addnode/most-recent-to-send-block/dice would be fine for evicting with the addnode case.
 55 2016-06-02T06:11:39  <gmaxwell> kinda odd that it would let you partition it by kicking off all your useful peers in favor of some dumb addnodes you plugged in.. but ::shrugs::
 56 2016-06-02T06:14:13  *** baldur has quit IRC
 57 2016-06-02T06:17:40  <sipa> yes, if you had to be whitelisted in order to get an addnode, we could just have addnode bypass the connection limit
 58 2016-06-02T06:18:03  <sipa> maybe something for post host authentication
 59 2016-06-02T06:20:25  <gmaxwell> right.
 60 2016-06-02T06:20:28  <gmaxwell> well baby steps
 61 2016-06-02T06:25:27  <midnightmagic> .w 2
 62 2016-06-02T06:26:50  <gmaxwell> sipa: could just be a feature of the p2p auth stuff, once we have that. If mutual auth is successful it bypasses limits.
 63 2016-06-02T06:27:09  <gmaxwell> (or at least a non-file descriptor related limit)
 64 2016-06-02T06:28:14  *** randy-waterhouse has joined #bitcoin-core-dev
 65 2016-06-02T06:30:45  <gmaxwell> sipa: another coner case to consider: say you set 8 addnode peers. They all ban you.
 66 2016-06-02T06:34:29  <midnightmagic> can I ask for a ptr to where the p2p auth is being worked on?
 67 2016-06-02T06:35:50  <sipa> midnightmagic: blocked by first having encryption in; see bip 151
 68 2016-06-02T06:37:16  <midnightmagic> ok
 69 2016-06-02T06:38:11  <gmaxwell> I guess that banning example suggests having some kind of addnode backoff... and making sure the auto evict runs slow enough after the connect that it won't kick a good peer to make room for a connection that is banned.
 70 2016-06-02T06:40:55  *** Giszmo has quit IRC
 71 2016-06-02T06:42:40  *** BashCo has quit IRC
 72 2016-06-02T06:43:38  <midnightmagic> I see a note of the mitm mitigation not being addressed and identity authentication: was this discussed somewhere that I missed?
 73 2016-06-02T06:44:19  <midnightmagic> (I would be happy with "sometime a few weeks ago in -wizards" or like, I don't mind doing my own legwork but would appreciate a general pointer)
 74 2016-06-02T06:49:17  <sipa> to address mitm, you need authentication
 75 2016-06-02T06:50:18  <sipa> jonas started on an authentication bip as well in parallel with bip 151
 76 2016-06-02T06:50:30  <sipa> but that was postponed to first get encryption done
 77 2016-06-02T06:52:14  <jonasschnelli> sipa, midnightmagic: there is some stuff from greg: https://people.xiph.org/~greg/auth1.txt
 78 2016-06-02T06:52:32  <jonasschnelli> Also the auth bip should probably be done before starting with the bip151 implementation
 79 2016-06-02T06:52:54  <midnightmagic> ok
 80 2016-06-02T07:03:26  <gmaxwell> I think it's fine to implement before auth is done, so long as whomever is implementing doesn't mind taking some small risk of disruption if changes to the auth design change the encryption.
 81 2016-06-02T07:05:05  <midnightmagic> gmaxwell: any thought to using proof-of-storage as ephemeral key ratchets? :-)
 82 2016-06-02T07:05:30  <midnightmagic> er. that was meant generally, not just to gmax. :)
 83 2016-06-02T07:06:28  <midnightmagic> and, would a group sig in log or constant space be possible for client group-membership proof non-interactively?
 84 2016-06-02T07:06:47  *** molly has joined #bitcoin-core-dev
 85 2016-06-02T07:09:30  *** molz has quit IRC
 86 2016-06-02T07:10:55  <gmaxwell> I don't think group sigs fit well in this context, unfortunately.  The server can tell if you are peer X by basically giving you a dummy keyring where all the keys are junk except yours.
 87 2016-06-02T07:11:18  <gmaxwell> you can't tell that it did this to you if it was successful in deanoning you
 88 2016-06-02T07:11:56  <gmaxwell> (the log size group sigs also take linear time to sign/verify... we're not actually super bandwidth constrained in this application-- so I think a log sized one doesn't buy us much)
 89 2016-06-02T07:12:13  <gmaxwell> The constant sized ones use pairing crypto, so then there is a bunch of extra crypto to bring in.
 90 2016-06-02T07:12:47  <midnightmagic> thank you
 91 2016-06-02T07:14:54  <gmaxwell> the chaumtoken auth was the best private auth I could come up with so far.
 92 2016-06-02T07:19:34  *** Ylbam has joined #bitcoin-core-dev
 93 2016-06-02T07:30:16  <jouke> I'm running a node on ipv4 and onion. I have a lot of peers that have the same "addr" IP, and my addresslocal says: "127.0.0.1". Are those connected trough TOR but are advertising that IP? The subver of those peers are different as well.
 94 2016-06-02T07:30:37  *** randy-waterhouse has quit IRC
 95 2016-06-02T07:35:17  <gmaxwell> if someone connects into you via onion, their addr will be 127.0.0.1:<something>  and the addrlocal should be your onion address.
 96 2016-06-02T07:35:27  <gmaxwell> though they can send whatever they want in addrlocal.
 97 2016-06-02T07:38:14  <jouke> Does the addrlocal in the gepeerinfo field specify on which IP address that node is connected to
 98 2016-06-02T07:38:42  <gmaxwell> No it is whatever address the remote party claims they think is your address.
 99 2016-06-02T07:38:48  <jouke> Oh right
100 2016-06-02T07:38:59  <jouke> Thanks.
101 2016-06-02T07:39:50  <jouke> And the addr field is the address where my nodes comunicates to?
102 2016-06-02T07:40:06  <gmaxwell> yes, which is 127.0.0.1 for inbound onion peers.
103 2016-06-02T07:41:31  <jouke> I have 57 peers with the same addr IP :-/
104 2016-06-02T07:42:02  <gmaxwell> presumably 52.51.something.
105 2016-06-02T07:42:07  <jouke> No
106 2016-06-02T07:42:12  <jouke> Those I have as well
107 2016-06-02T07:42:16  <jouke> On other nodes
108 2016-06-02T07:42:39  <jouke> It is is actualy a /32 IP
109 2016-06-02T07:43:04  <midnightmagic> that sounds super annoying.
110 2016-06-02T07:43:10  <gmaxwell> What do you mean by a "/32 IP"  ... all IPs are /32... because they're hosts not networks. :P
111 2016-06-02T07:43:33  <jouke> The bytessent/bytesrecv ratio isn't as skewed as with those 52.51
112 2016-06-02T07:43:58  <jouke> gmaxwell: just one and the same IP, not multiple IP's from the same subnet.
113 2016-06-02T07:44:16  <gmaxwell> I strongly recommend reporting abusive hosts to their providers, fwiw.
114 2016-06-02T07:44:55  <jouke> Hetzner...
115 2016-06-02T07:44:58  <midnightmagic> maybe it's a tor exit node or something
116 2016-06-02T07:45:19  <gmaxwell> midnightmagic: well thats easily checked. :) (indeed, don't report tor exits)
117 2016-06-02T07:46:13  <jouke> It's not in here: https://check.torproject.org/exit-addresses
118 2016-06-02T07:46:57  <midnightmagic> Ah. Hetzner. Why am I not surprised. :-/
119 2016-06-02T07:49:52  <jouke> I was thinking of having a script that would look at the upload/download ratio and ban nodes with a bad ratio. But wouldn't have worked in this case.
120 2016-06-02T07:50:58  <gmaxwell> you can do varrious adhoc things personally, but most don't work broader than that.. too easily avoided. the 52.x sybils could start sending junk traffic... and just waste more of your bandwidth.
121 2016-06-02T07:51:27  <gmaxwell> They case no harm except privacy loss, and the solution to that is to fix things so they can't hurt privacy.
122 2016-06-02T07:52:56  <gmaxwell> though if they're irritating you, http://0bin.net/paste/EH8fTyfotJSU3Udw#TCHJfA7qDt5odgG+gW4dG9aUEYkJQo6oOTmsuXBAJke  works.
123 2016-06-02T07:53:49  *** pmienk has quit IRC
124 2016-06-02T07:55:06  <gmaxwell> ah seems that misses 52.51.180.197 and 52.51.204.60
125 2016-06-02T07:56:37  <GitHub50> [bitcoin] jonasschnelli opened pull request #8135: [OSX] fix make deploy when compiling on OSX (master...2016/06/makedeploy) https://github.com/bitcoin/bitcoin/pull/8135
126 2016-06-02T08:05:44  *** pmienk has joined #bitcoin-core-dev
127 2016-06-02T08:06:53  <jouke> gmaxwell: heh, thanks :)
128 2016-06-02T08:10:27  *** jannes has joined #bitcoin-core-dev
129 2016-06-02T08:29:19  *** Guyver2 has joined #bitcoin-core-dev
130 2016-06-02T08:31:32  *** randy-waterhouse has joined #bitcoin-core-dev
131 2016-06-02T08:32:40  *** molz has joined #bitcoin-core-dev
132 2016-06-02T08:35:32  *** molly has quit IRC
133 2016-06-02T08:38:45  *** jyap has quit IRC
134 2016-06-02T08:39:09  *** jyap has joined #bitcoin-core-dev
135 2016-06-02T08:39:09  *** jyap has joined #bitcoin-core-dev
136 2016-06-02T08:45:41  <GitHub104> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/715e9fd7454f...58725ba89dfc
137 2016-06-02T08:45:41  <GitHub104> bitcoin/master 2692e1b fanquake: [Doc] Simplify OS X build notes...
138 2016-06-02T08:45:42  <GitHub104> bitcoin/master 58725ba Jonas Schnelli: Merge #8029: [Doc] Simplify OS X build notes...
139 2016-06-02T08:45:45  <GitHub56> [bitcoin] jonasschnelli closed pull request #8029: [Doc] Simplify OS X build notes (master...osx-build-notes) https://github.com/bitcoin/bitcoin/pull/8029
140 2016-06-02T09:11:59  <GitHub85> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/58725ba89dfc...ee1533e262e7
141 2016-06-02T09:11:59  <GitHub85> bitcoin/master 16698cb UdjinM6: PR #7772 is not enough to fix the issue with QCompleter, use event filter instead of `connect`
142 2016-06-02T09:12:00  <GitHub85> bitcoin/master ee1533e Jonas Schnelli: Merge #8129: Fix RPC console auto completer...
143 2016-06-02T09:12:09  <GitHub132> [bitcoin] jonasschnelli closed pull request #8129: Fix RPC console auto completer (master...fixRPCAutoCompleter_bitcoin) https://github.com/bitcoin/bitcoin/pull/8129
144 2016-06-02T09:13:24  *** BashCo has joined #bitcoin-core-dev
145 2016-06-02T09:19:39  *** paveljanik has joined #bitcoin-core-dev
146 2016-06-02T09:19:39  *** paveljanik has joined #bitcoin-core-dev
147 2016-06-02T09:19:52  <iniana> What can I do to debug the painfully slow sync of my core node on a linux machine? Currently at height 410k and it progresses around 1 block every 3 seconds on average.
148 2016-06-02T09:27:38  <jonasschnelli> iniana: what processor? how many RAM? disk stype?
149 2016-06-02T09:28:04  <jonasschnelli> 1 block every 3 seconds is very likely disk speed or CPU.
150 2016-06-02T09:28:34  <jonasschnelli> If you have enough memory, you should try to pass -dbache=4000 (4GB)
151 2016-06-02T09:28:47  <jonasschnelli> Even passing -dbcache=2000 can help
152 2016-06-02T09:28:55  <jonasschnelli> (default is 300)
153 2016-06-02T09:33:09  <iniana> jonasschnelli: blockchain is stored on hdd, dbcache was set to 2000. I have a feeling it is because it is run within a Qubes VM. Is there anyway to measure disk I/O?
154 2016-06-02T09:34:08  <jonasschnelli> iniana: I guess you can use iostat on linux
155 2016-06-02T09:34:41  <jonasschnelli> iniana: did you enable multicore on your VM? But yes,.. running in a VM can slow down the whole thing.
156 2016-06-02T09:35:01  <jonasschnelli> Best fix: don't stress it and wait a couple of days. :)
157 2016-06-02T09:36:35  <iniana> jonasschnelli: yes, set to 8 VCPUs, 8GB max memory, 4GB initial memory. Am at height 410k, so should only have to wait a couple of hours :)
158 2016-06-02T09:36:54  <jonasschnelli> iniana: yes. What CPU?
159 2016-06-02T09:37:39  <jonasschnelli> On my Xeon/SSD I could sync a node in a VirtualBox VM within 3-4h.
160 2016-06-02T09:37:44  <iniana> core i7, 2yo, not sure on exact specs
161 2016-06-02T09:38:03  <jonasschnelli> iniana: try run Core with --debug=bench
162 2016-06-02T09:38:22  <jonasschnelli> You will see if its the disk latency or the ecdsa
163 2016-06-02T09:39:26  <iniana> I managed to sync a pruned node on a whonix VM (tor-only) a lot faster than what I am experiencing now. Only difference is that was on SSD. But I have used the same HDD before and it was a lot faster then. Some combo of VM and HDD doesn't mix I guess.
164 2016-06-02T09:39:40  <iniana> Ah great! Will try that debug flag
165 2016-06-02T09:42:06  <phantomcircuit> iniana, does qubes pass the cpu flags?
166 2016-06-02T09:42:20  <phantomcircuit> also what version are you running?
167 2016-06-02T09:43:25  <iniana> phantomcircuit: What are the cpu flags? Am running Qubes 3.1, Core 0.12.1
168 2016-06-02T09:47:32  <iniana> The following line seems to be the one that takes longest to generate: - Connect 647 transactions: 597.49ms (0.923ms/tx, 0.509ms/txin) [102.29s]
169 2016-06-02T09:48:16  <iniana> also - Verify 4484 txins: 2407.61ms (0.537ms/txin) [104.76s]
170 2016-06-02T09:48:37  *** cryptapus has joined #bitcoin-core-dev
171 2016-06-02T10:01:42  <wumpus> looks like most time is spent on verification
172 2016-06-02T10:01:50  <wumpus> how many cores are allocated to that VM?
173 2016-06-02T10:02:36  <wumpus> bitcoin core makes heavy use of paralellism for verification
174 2016-06-02T10:04:23  <wumpus> ok 8 VCPUs should be fine
175 2016-06-02T10:05:18  <sipa> u
176 2016-06-02T10:07:25  <iniana> Hmm.. There was a sequence of a couple of hundred blocks which were processed very quickly (10/s maybe) after which it resumed with 1 block per 3s again.
177 2016-06-02T10:13:40  <sipa> are you reindexing or downloading?
178 2016-06-02T10:13:53  <iniana> downloading
179 2016-06-02T10:14:15  <sipa> that will explain the speed differences
180 2016-06-02T10:14:31  <sipa> we download multiple blocks at once
181 2016-06-02T10:14:51  <sipa> but they don't arrivs in the right order
182 2016-06-02T10:17:20  <iniana> Are transactions that can't be verified yet during sync added to the mempool or someplace else?
183 2016-06-02T10:17:20  <sipa> so validation only progresses when the first missing one arrices
184 2016-06-02T10:17:52  <sipa> you mean transactions in blocks we download? no
185 2016-06-02T10:18:11  <sipa> the mempool only contains validated transactions
186 2016-06-02T10:18:40  <iniana> ah ok
187 2016-06-02T10:18:55  *** tErik_mc has joined #bitcoin-core-dev
188 2016-06-02T10:18:57  <sipa> say you're synced up to block 10000
189 2016-06-02T10:19:13  <sipa> and you downloading the 100 blocks that follow
190 2016-06-02T10:19:30  <sipa> and 10001 though 10099 have arrived
191 2016-06-02T10:19:36  <sipa> but 10000 has not
192 2016-06-02T10:19:46  <sipa> then 10000 arrvives
193 2016-06-02T10:20:03  <sipa> and suddenly you can validate 100 blocks at once
194 2016-06-02T10:25:24  <iniana> Still weird that it is progressing this slowly though. I mean, it was the same speed when reindexing a previously synced data directory I had copied over. It had all the data and did no downloading. How come there was that sequence of blocks that could be progressed so quickly?
195 2016-06-02T10:27:12  <iniana> When I restarted bitcoin core recently the verification of the previous 288 blocks was also that slow.
196 2016-06-02T10:29:58  <sipa> during reindex you also saw some sequence of blocks that wete fast?
197 2016-06-02T10:30:19  <sipa> not all blocks have the same number of transactions
198 2016-06-02T10:30:43  <sipa> the initial 288 being slow is due to the cache mot being warmed up
199 2016-06-02T10:31:00  <sipa> the database entries still have to be loaded from disk
200 2016-06-02T10:31:21  <iniana> no, during reindex I never saw a fast sequence, though I could have missed it.
201 2016-06-02T10:32:16  <phantomcircuit> sipa, the p2p block fetching logic needs to be hardened against people being annoying
202 2016-06-02T10:32:25  <phantomcircuit> there's some very very very slow nodes
203 2016-06-02T10:32:29  <iniana> Sure, not all blocks are the same size, but from a couple of hundred blocks at height 411k, I bet most of them had a lot of transactions.
204 2016-06-02T10:32:38  <phantomcircuit> but worse there's some that are ridiculously bursty
205 2016-06-02T10:32:49  <phantomcircuit> they'll be fast for some blocks and then take 10 seconds to respond to another
206 2016-06-02T10:55:43  *** Thireus1 has joined #bitcoin-core-dev
207 2016-06-02T10:56:41  *** jtimon has joined #bitcoin-core-dev
208 2016-06-02T10:56:53  *** Thireus1 has quit IRC
209 2016-06-02T10:57:09  *** Thireus1 has joined #bitcoin-core-dev
210 2016-06-02T11:02:06  <sipa> iniana: you have high dbcache set in both?
211 2016-06-02T11:03:00  <iniana> sipa: yes, 2000
212 2016-06-02T11:05:00  *** calibre720 has quit IRC
213 2016-06-02T11:05:52  <sipa> whenever the dbcache fills up, it needs tp be written to disk, and afterwards processing is slower for a while
214 2016-06-02T11:06:18  <sipa> you can see when this happens when the cache size in the UodateTip lines falls back to 0
215 2016-06-02T11:06:33  <sipa> but that causes sudden slowness, not sudden fastness
216 2016-06-02T11:08:24  *** fengling has quit IRC
217 2016-06-02T11:10:55  <iniana> Cache currently at 1258MiB, and it is consistently slow.
218 2016-06-02T11:11:14  <sipa> are you running with debug=bench ?
219 2016-06-02T11:11:20  <iniana> yes
220 2016-06-02T11:14:09  *** xiangfu has quit IRC
221 2016-06-02T11:14:32  *** calibre720 has joined #bitcoin-core-dev
222 2016-06-02T11:15:00  <iniana> sipa: Here is a snippet from my log http://pastebin.com/6jWv29uj
223 2016-06-02T11:16:50  <sipa> all your time is lost on loading database entries
224 2016-06-02T11:17:12  <sipa> either those are blocks that spend a lot of very old inouts that are not cached
225 2016-06-02T11:17:18  <sipa> or your disk is very slow
226 2016-06-02T11:17:31  <sipa> *old inputs
227 2016-06-02T11:20:29  <iniana> iostat reports 7.8M read and 2.1M write for that particular block deivce. Doesn't feel like it is the bottleneck.
228 2016-06-02T11:22:06  <sipa> they're all very small reads
229 2016-06-02T11:23:24  <sipa> what kind of disk?
230 2016-06-02T11:23:31  <iniana> sipa: ok. I guess this is the performance I have to live with until I upgrade to an SSD?
231 2016-06-02T11:24:13  <sipa> i have a spinning disk, and numbers are far better than what you're seeing
232 2016-06-02T11:24:21  <sipa> 18s to load all the inputs is crazy
233 2016-06-02T11:24:49  <iniana> yes, I have used the same disk on a non-virtual machine with much better results.
234 2016-06-02T11:25:05  <sipa> ah, maybe it's due to the VM setup
235 2016-06-02T11:25:29  <wumpus> yes VM can make some difference
236 2016-06-02T11:25:49  <wumpus> there are some curious interactions between fdatasync and VMs for example
237 2016-06-02T11:26:09  <wumpus> probably because the VM 'disk' is one file in the host OS
238 2016-06-02T11:26:20  <sipa> wumpus: this is slowness in reads, not writes
239 2016-06-02T11:26:25  <sipa> so sync is not the issue
240 2016-06-02T11:26:32  <wumpus> reads shouldn't be affected by VM
241 2016-06-02T11:26:39  <wumpus> not much, at least
242 2016-06-02T11:27:19  <wumpus> unless it is some weird compressed image format
243 2016-06-02T11:27:45  <iniana> I'm using Qubes (xen) if that helps
244 2016-06-02T11:29:08  <wumpus> what can make a difference in i/o speed is to make sure that you're using a paravirtualized disk, and not e.g. emulating IDE
245 2016-06-02T11:30:01  <phantomcircuit> iniana, 200+ms to load a block from disk is terrrible
246 2016-06-02T11:32:25  <iniana> Could be the way I attach the block device to my VM (qvm-block), will explore this further.
247 2016-06-02T11:34:00  <wumpus> yes qemu allows configuring a lot of things related to emulated devices
248 2016-06-02T11:35:03  <wumpus> I'd say it is only worth it if you intend to do a lot of syncing, I mean once your node syncs the i/o and CPU usage drops by lots and this all matters very little
249 2016-06-02T11:36:13  *** airmace312 has joined #bitcoin-core-dev
250 2016-06-02T11:36:16  * airmace312 is selling coins if you are intrested in buying please private msg me and we can negotiate the rate so if intrested please msg me for a good deal thanks
251 2016-06-02T11:36:28  *** ChanServ sets mode: +o wumpus
252 2016-06-02T11:36:52  *** wumpus sets mode: +b *!*@103-224-130-8.flip.co.nz
253 2016-06-02T11:37:02  *** airmace312 was kicked by wumpus (no selling/buying here)
254 2016-06-02T11:37:18  *** ChanServ sets mode: -o wumpus
255 2016-06-02T11:37:36  <iniana> wumpus: I guess, but it would be annoying if when I have to turn off the machine only to have to wait for hours until it is synced up again when I turn it on. Plus its fun getting to know how stuff works :)
256 2016-06-02T11:38:34  <wumpus> sure, as a learning experience it makes sense
257 2016-06-02T11:48:14  <sipa> iniana: at your sync speed, it takes around 1-2 hours to sync a week of blockchain data
258 2016-06-02T11:48:18  <sipa> that's terrible
259 2016-06-02T11:48:28  <sipa> some people sync the whole chain in 4 hours
260 2016-06-02T11:48:47  <iniana> sipa: yes, its very frustrating
261 2016-06-02T11:48:49  <sipa> but i don't have any advice but checking your vm disk setup
262 2016-06-02T11:49:32  *** randy-waterhouse has quit IRC
263 2016-06-02T11:55:28  <phantomcircuit> iniana, it's xen based
264 2016-06-02T11:57:01  <iniana> phantomcircuit: yes
265 2016-06-02T11:57:14  <phantomcircuit> i would say
266 2016-06-02T11:57:19  <phantomcircuit> "good luck" fixing that issue
267 2016-06-02T11:57:20  <phantomcircuit> :/
268 2016-06-02T11:57:46  <iniana> You say I shouldn't waste my time?
269 2016-06-02T11:58:21  <phantomcircuit> i'd say go ask the qubes people
270 2016-06-02T11:58:32  <GitHub8> [bitcoin] jonasschnelli opened pull request #8136: Log/report in 10% steps during VerifyDB (master...2016/06/init_checkblocks) https://github.com/bitcoin/bitcoin/pull/8136
271 2016-06-02T12:00:07  <iniana> How would attaching the disk as a samba or nfs share instead of via xen work out? Has anyone tried loading blockchain data from a remote server that way?
272 2016-06-02T12:02:17  *** fengling has joined #bitcoin-core-dev
273 2016-06-02T12:03:44  <sipa> iniana: no, you need mmap support
274 2016-06-02T12:03:53  <sipa> network filesystems don't offer that
275 2016-06-02T12:04:24  <phantomcircuit> sipa, do we still require mmap?
276 2016-06-02T12:04:30  <sipa> leveldb does
277 2016-06-02T12:04:48  <phantomcircuit> hmm
278 2016-06-02T12:04:59  <sipa> i think
279 2016-06-02T12:05:06  <kinlo> isn't requiring mmap a good thing from a performance point of view?
280 2016-06-02T12:05:13  <sipa> yes
281 2016-06-02T12:05:17  <phantomcircuit> "meh"
282 2016-06-02T12:05:27  <sipa> i think you can disable it
283 2016-06-02T12:05:43  <sipa> i think we may not use mmap on 32-bit
284 2016-06-02T12:07:04  *** fengling has quit IRC
285 2016-06-02T12:07:22  <wumpus> loading the blockchain data from a network filesystem should work, just not having the database directories there (if it doesn't support mmap)
286 2016-06-02T12:07:50  <wumpus> I've succesfully used a network block device to store both though
287 2016-06-02T12:07:59  *** Chris_Stewart_5 has joined #bitcoin-core-dev
288 2016-06-02T12:08:29  <wumpus> haven't tried samba nor nfs
289 2016-06-02T12:09:36  <jonasschnelli> wumpus: is there a startup argument to set the blockfiles directory?
290 2016-06-02T12:09:37  <wumpus> I've also shared synced node data between windows and linux locally, linux' ntfs implementation surprisingly does work good enough for leveldb
291 2016-06-02T12:10:19  <jonasschnelli> Or do you need some ln -s?
292 2016-06-02T12:10:21  <wumpus> no, but you can put stuff all over the place with symbolic linnks
293 2016-06-02T12:10:43  <jonasschnelli> Right...
294 2016-06-02T12:11:09  <wumpus> being able to specify different directories as arguments would be mildly useful, although for people that don't know what they're doing it's a foot shooting cannon
295 2016-06-02T12:13:15  <wumpus> (e.g. we only aquire a lock on the data directory, not on the subdirectories individually, so if you can specify them elsewhere someone may do crazy things like point two bitcoinds at the same place)
296 2016-06-02T12:21:07  *** cryptapus_ has joined #bitcoin-core-dev
297 2016-06-02T12:21:07  *** cryptapus has quit IRC
298 2016-06-02T12:22:24  *** cryptapus_ is now known as cryptapus
299 2016-06-02T12:28:14  *** MrHodl has joined #bitcoin-core-dev
300 2016-06-02T12:32:29  <jonasschnelli> hmm.. travis is strange today:
301 2016-06-02T12:32:30  <jonasschnelli> W: Failed to fetch https://apt.dockerproject.org/repo/dists/ubuntu-trusty/main/binary-amd64/Packages  Hash Sum mismatch
302 2016-06-02T12:34:24  <wumpus> scary
303 2016-06-02T12:36:06  <wumpus> happens every time?
304 2016-06-02T12:43:53  <sipa> try pressing more buttond
305 2016-06-02T12:46:17  <wumpus> these are not even our own downloads, but travis pre-installing packages that fails
306 2016-06-02T12:46:37  <wumpus> so less scary, probably just something with the repository that is misconfigured
307 2016-06-02T12:48:36  *** achow101 has joined #bitcoin-core-dev
308 2016-06-02T12:48:38  <wumpus> docketprojects's package repository that is, not our repository
309 2016-06-02T12:49:52  *** Guyver2 has quit IRC
310 2016-06-02T12:57:45  *** BashCo has quit IRC
311 2016-06-02T13:03:47  <GitHub10> [bitcoin] pstratem opened pull request #8137: Improve CWallet API with new AccountMove function. (master...2016-06-02-cwallet-accountmove) https://github.com/bitcoin/bitcoin/pull/8137
312 2016-06-02T13:03:48  *** fengling has joined #bitcoin-core-dev
313 2016-06-02T13:06:27  *** Chris_Stewart_5 has quit IRC
314 2016-06-02T13:08:44  *** fengling has quit IRC
315 2016-06-02T13:12:00  <wumpus> pushing more buttons doesn't seem to help
316 2016-06-02T13:28:22  *** MarcoFalke has joined #bitcoin-core-dev
317 2016-06-02T13:36:05  *** cryptapus_ has joined #bitcoin-core-dev
318 2016-06-02T13:38:52  *** cryptapus has quit IRC
319 2016-06-02T13:40:33  *** cryptapus_ has quit IRC
320 2016-06-02T13:43:44  *** cryptapus_ has joined #bitcoin-core-dev
321 2016-06-02T13:46:03  *** jannes has quit IRC
322 2016-06-02T13:46:09  *** jannes_ has joined #bitcoin-core-dev
323 2016-06-02T13:49:27  *** zooko has joined #bitcoin-core-dev
324 2016-06-02T13:58:51  <MarcoFalke> sipa, but this would cause an include main.h in wallet.h?
325 2016-06-02T13:59:24  <sipa> MarcoFalke: why in wallet.h?
326 2016-06-02T13:59:36  <sipa> the constant would just disappear from the .h fr
327 2016-06-02T13:59:52  <sipa> wallet.cpp would call main to query the current relay fee
328 2016-06-02T14:00:39  <MarcoFalke> Oh, you mean current relay fee and not minimum relay fee.
329 2016-06-02T14:01:04  <sipa> even the minimum relay fee is configurable, so indeed :)
330 2016-06-02T14:01:26  <sipa> or even the minimum confirmable feerate
331 2016-06-02T14:01:39  <sipa> as we should never create outputs that are uneconomical to spend
332 2016-06-02T14:01:51  <sipa> and the fee estimate is the best guess for that
333 2016-06-02T14:04:50  *** Giszmo has joined #bitcoin-core-dev
334 2016-06-02T14:05:12  *** robs has quit IRC
335 2016-06-02T14:05:41  *** fengling has joined #bitcoin-core-dev
336 2016-06-02T14:06:08  *** Thireus1 has quit IRC
337 2016-06-02T14:10:10  <phantomcircuit> sipa, im seeing pull tester failures on master
338 2016-06-02T14:10:22  <phantomcircuit> can you (or someone else) check?
339 2016-06-02T14:10:24  *** fengling has quit IRC
340 2016-06-02T14:10:25  <phantomcircuit> wumpus, ^
341 2016-06-02T14:11:01  <wumpus> still the apt-get issue?
342 2016-06-02T14:11:05  <wumpus> nothing we can do there
343 2016-06-02T14:12:07  <wumpus> docker needs to wake up on this https://github.com/docker/docker/issues/23203
344 2016-06-02T14:13:09  <sipa> MarcoFalke: also, unrelated... i don't think we should care much about header file dependencies (apart from making compilation time and memory larger, they don't hurt modularity)... what matters is dependencies between (.h,.c) pairs on eachother, and wallet.cpp already depends on main.cpp
345 2016-06-02T14:13:16  <phantomcircuit> wumpus, no im testing locally
346 2016-06-02T14:13:49  <MarcoFalke> ok
347 2016-06-02T14:14:37  <sipa> the reasoning being: you already can't use the functionality from wallet.{cpp,h} without having main.{cpp,h} available, so there already is a dependency
348 2016-06-02T14:16:04  *** Thireus1 has joined #bitcoin-core-dev
349 2016-06-02T14:16:41  *** Thireus1 has quit IRC
350 2016-06-02T14:21:59  *** TomMc has joined #bitcoin-core-dev
351 2016-06-02T14:22:31  <jonasschnelli> MarcoFalke: sipa: what about introducing a nodemodel.{cpp/h} that could act as a wallet/node API?
352 2016-06-02T14:22:52  <sipa> how is that different from clientmodel?
353 2016-06-02T14:23:00  <jonasschnelli> clientmodel is GUI
354 2016-06-02T14:23:06  <jonasschnelli> nodemodel would be core/wallet
355 2016-06-02T14:23:12  <sipa> i don't understand
356 2016-06-02T14:23:27  <sipa> clientmodel is the abstraction of the client/node that the GUI uses to communicate with core
357 2016-06-02T14:23:31  <jonasschnelli> Main reason for this could be to have all node interaction in a single file (better readability, better source for later decoupling)
358 2016-06-02T14:23:48  <jonasschnelli> Yes. It could also be clientmodel...
359 2016-06-02T14:23:48  <sipa> ah, you mean a lower level
360 2016-06-02T14:23:51  <jonasschnelli> yes.
361 2016-06-02T14:24:04  <sipa> what sort of code would move there?
362 2016-06-02T14:24:14  <jonasschnelli> Fee estimation, relay fee, etc.
363 2016-06-02T14:24:19  <jonasschnelli> broadcast
364 2016-06-02T14:25:36  <sipa> i wouldn't put wallet-related things there (as those shouldn't be lumped together long term)
365 2016-06-02T14:26:00  <sipa> but if there are pieces of code shared between gui and rpc, it makes sense to move them to a common abstraction layer below
366 2016-06-02T14:28:58  <wumpus> wallet.cpp can depend on main.cpp, but not the other way around
367 2016-06-02T14:30:38  <wumpus> I'm not sure adding all node interaction in a single file is a step forward; we'tre trying to modularize things beyond that, e.g. net is different from blockchain handling
368 2016-06-02T14:31:41  <jonasschnelli> Yes. I agree, move it to a single file would only increase code readability, ... and not sure if this is worth a PR.
369 2016-06-02T14:32:48  <sipa> wumpus: agree
370 2016-06-02T14:33:33  <sipa> i think we're perhaps better off spending time to move code from RPC calls (especially the complicated code inside RPCs that grabs cs_main) to somewhere in the core code instead, so that the RPC is just a single call
371 2016-06-02T14:33:34  <wumpus> clientmodel has way too many responsibilities
372 2016-06-02T14:34:00  <wumpus> (but it's fine for the GUI, and was a big step forward from satoshi's stuff)
373 2016-06-02T14:34:40  <wumpus> yes, that makes sense, ideally only main functions would lock cs_main
374 2016-06-02T14:34:49  <wumpus> on the other hand at this point I'd not like moving things into main
375 2016-06-02T14:34:57  <wumpus> main has the same issue
376 2016-06-02T14:35:03  <wumpus> but after net refactors etc, sure
377 2016-06-02T14:36:43  <wumpus> I don't think RPC functions necessarily need to be just a single call, what matters is that operations that need to happen under cs_main lock and act on main data structures would be better off in main, but these could be the minimal possible encapsulated operations
378 2016-06-02T14:36:54  <sipa> wumpus: +1
379 2016-06-02T14:37:18  <sipa> anything that grabs a lock should be moved to the module that manages that lock
380 2016-06-02T14:37:39  <sipa> (in practice, that often means introducing callback functions etc... though)
381 2016-06-02T14:37:45  *** cryptapus_ is now known as cryptapus
382 2016-06-02T14:37:50  <wumpus> aren't we happy with c++11 lambdas then
383 2016-06-02T14:38:07  <sipa> i should learn to use those :)
384 2016-06-02T14:38:11  <sipa> but yes
385 2016-06-02T14:38:39  <wumpus> cfields taught me how to use them in Zurich
386 2016-06-02T14:39:11  <wumpus> I'm impressed, the functionality is so flexible it seems almost un-c++
387 2016-06-02T14:40:42  <wumpus> (e.g. how code in a lambda can use variables from the enclosing function is more remniscent of dynamic languages)
388 2016-06-02T14:41:22  <wumpus> phantomcircuit: if you run it locally you should have an error message?
389 2016-06-02T14:41:33  <gmaxwell> GCC nested functions can do that too.
390 2016-06-02T14:44:08  <phantomcircuit> wumpus, i have lots of error messages
391 2016-06-02T14:44:13  <wumpus> didn't know that! there's not really precedent, c++ nested classes cannot access members from enclosing classes, like in java static inner classes
392 2016-06-02T14:44:21  <phantomcircuit> im checking again after a git clean -fdx
393 2016-06-02T14:44:31  <wumpus> phantomcircuit: so the autotests fails?
394 2016-06-02T14:44:48  <wumpus> or the qa tests?
395 2016-06-02T14:44:49  <phantomcircuit> wumpus, rpc pull tests fail
396 2016-06-02T14:44:54  <phantomcircuit> make check tests pass
397 2016-06-02T14:45:19  <MarcoFalke> which rpc-test?
398 2016-06-02T14:45:29  <phantomcircuit> a bunch of the wallet related ones
399 2016-06-02T14:45:40  <phantomcircuit> i'll clarify in a minute when im done recompiling
400 2016-06-02T14:45:58  *** Chris_Stewart_5 has joined #bitcoin-core-dev
401 2016-06-02T14:52:09  *** Guyver2 has joined #bitcoin-core-dev
402 2016-06-02T14:54:08  <phantomcircuit> and now everything passes
403 2016-06-02T14:54:09  <phantomcircuit> >.>
404 2016-06-02T14:54:11  <phantomcircuit> nvm
405 2016-06-02T14:58:37  *** moli has joined #bitcoin-core-dev
406 2016-06-02T15:01:04  <GitHub38> [bitcoin] jonasschnelli opened pull request #8138: Add maximal amount-of-transactions limit to checkblock/CVerifyDB (master...2016/06/verify_db) https://github.com/bitcoin/bitcoin/pull/8138
407 2016-06-02T15:01:09  *** molz has quit IRC
408 2016-06-02T15:05:52  *** jl2012 has quit IRC
409 2016-06-02T15:06:12  *** jl2012 has joined #bitcoin-core-dev
410 2016-06-02T15:06:13  *** zooko has quit IRC
411 2016-06-02T15:07:04  *** eragmus has quit IRC
412 2016-06-02T15:07:13  *** fengling has joined #bitcoin-core-dev
413 2016-06-02T15:09:26  *** eragmus has joined #bitcoin-core-dev
414 2016-06-02T15:12:04  *** fengling has quit IRC
415 2016-06-02T15:15:09  *** slackircbridge has quit IRC
416 2016-06-02T15:15:21  *** slackircbridge has joined #bitcoin-core-dev
417 2016-06-02T15:19:13  <sipa> ping for review: https://github.com/bitcoin/bitcoin/pull/7948 (RPC: augment getblockchaininfo bip9_softforks data)
418 2016-06-02T15:20:03  <sipa> ping for review: https://github.com/bitcoin/bitcoin/pull/7967 ([RPC] add feerate option to fundrawtransaction)
419 2016-06-02T15:23:43  *** cryptapus has quit IRC
420 2016-06-02T15:23:51  *** Chris_Stewart_5 has quit IRC
421 2016-06-02T15:50:25  *** BashCo has joined #bitcoin-core-dev
422 2016-06-02T15:51:32  *** cryptapus has joined #bitcoin-core-dev
423 2016-06-02T15:56:17  <sipa> phantomcircuit: are you seeing this:
424 2016-06-02T15:56:25  <sipa> https://www.zerobin.net/?340fa0188dd35804#2Hkj8AQ8nQtI3LWQpGNrAHGnmRVeCiqVvm9BkG2+HpM=
425 2016-06-02T16:00:53  <sipa> MarcoFalke: ^
426 2016-06-02T16:04:22  <MarcoFalke> sipa: any zombie bitcoinds? ps -A |grep bitcoind
427 2016-06-02T16:04:27  <sipa> nope
428 2016-06-02T16:04:40  <MarcoFalke> reproducible?
429 2016-06-02T16:04:48  <sipa> retried the test walletbackup.py itself and got a different error
430 2016-06-02T16:05:05  <sipa> now trying rpc-tests.py again, and it seems to run ok
431 2016-06-02T16:05:19  <sipa> at least it doesn't instantly fail like before
432 2016-06-02T16:05:51  <MarcoFalke> Have you been running ./rpc-tests.py in parallel?
433 2016-06-02T16:06:06  <MarcoFalke> (This doesn't work)
434 2016-06-02T16:06:13  <sipa> no
435 2016-06-02T16:06:28  <sipa> if you mean multiple instances of rpc-tests.py simultaneously, no
436 2016-06-02T16:06:32  <MarcoFalke> jup
437 2016-06-02T16:07:16  *** erasmospunk has joined #bitcoin-core-dev
438 2016-06-02T16:08:43  *** fengling has joined #bitcoin-core-dev
439 2016-06-02T16:13:24  *** fengling has quit IRC
440 2016-06-02T16:28:20  *** Chris_Stewart_5 has joined #bitcoin-core-dev
441 2016-06-02T16:37:16  *** erasmospunk has quit IRC
442 2016-06-02T16:37:50  *** BashCo has quit IRC
443 2016-06-02T16:45:04  *** Thireus1 has joined #bitcoin-core-dev
444 2016-06-02T16:53:52  *** Chris_Stewart_5 has quit IRC
445 2016-06-02T16:57:21  *** Chris_Stewart_5 has joined #bitcoin-core-dev
446 2016-06-02T17:04:07  *** kadoban has joined #bitcoin-core-dev
447 2016-06-02T17:06:35  *** erasmospunk has joined #bitcoin-core-dev
448 2016-06-02T17:10:25  *** fengling has joined #bitcoin-core-dev
449 2016-06-02T17:14:57  <GitHub121> [bitcoin] sipa pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/ee1533e262e7...ec45cc5e2766
450 2016-06-02T17:14:58  <GitHub121> bitcoin/master 84c13e7 Wladimir J. van der Laan: chain: Add assertion in case of missing records in index db
451 2016-06-02T17:14:58  <GitHub121> bitcoin/master 6030625 Wladimir J. van der Laan: test: Add more thorough test for dbwrapper iterators...
452 2016-06-02T17:14:59  <GitHub121> bitcoin/master 269a440 Matt Corallo: Add test for dbwrapper iterators with same-prefix keys.
453 2016-06-02T17:15:02  <GitHub78> [bitcoin] sipa closed pull request #7992: Extend #7956 with one more test. (master...16-5-7956-update) https://github.com/bitcoin/bitcoin/pull/7992
454 2016-06-02T17:15:04  *** fengling has quit IRC
455 2016-06-02T17:15:14  <GitHub30> [bitcoin] sipa closed pull request #8051: Seemingly fix walletbackup.py failure (master...fixwalletbackup) https://github.com/bitcoin/bitcoin/pull/8051
456 2016-06-02T17:17:35  <GitHub20> [bitcoin] sipa opened pull request #8139: Fix interrupted HTTP RPC connection workaround for Python 3.5+ (master...fixwalletbackup) https://github.com/bitcoin/bitcoin/pull/8139
457 2016-06-02T17:25:30  *** MarcoFalke has left #bitcoin-core-dev
458 2016-06-02T17:26:25  *** Chris_Stewart_5 has quit IRC
459 2016-06-02T17:31:06  *** molz has joined #bitcoin-core-dev
460 2016-06-02T17:34:01  *** moli has quit IRC
461 2016-06-02T17:41:15  *** Thireus1 has quit IRC
462 2016-06-02T17:42:48  *** Chris_Stewart_5 has joined #bitcoin-core-dev
463 2016-06-02T17:57:42  *** calibre720 has quit IRC
464 2016-06-02T18:11:47  *** fengling has joined #bitcoin-core-dev
465 2016-06-02T18:16:24  *** fengling has quit IRC
466 2016-06-02T18:32:12  *** erasmospunk has quit IRC
467 2016-06-02T18:39:07  *** Thireus1 has joined #bitcoin-core-dev
468 2016-06-02T18:42:30  *** frankenmint has joined #bitcoin-core-dev
469 2016-06-02T18:43:49  *** frankenm_ has joined #bitcoin-core-dev
470 2016-06-02T18:44:26  *** frankenm_ is now known as frankenmint_
471 2016-06-02T18:45:47  *** frankenm_ has joined #bitcoin-core-dev
472 2016-06-02T18:46:09  *** erasmospunk has joined #bitcoin-core-dev
473 2016-06-02T18:46:10  *** slackircbridge has quit IRC
474 2016-06-02T18:46:27  *** slackircbridge has joined #bitcoin-core-dev
475 2016-06-02T18:47:01  *** franken__ has joined #bitcoin-core-dev
476 2016-06-02T18:47:14  *** laurentmt has joined #bitcoin-core-dev
477 2016-06-02T18:47:27  *** frankenmint has quit IRC
478 2016-06-02T18:47:54  *** Thireus1 has quit IRC
479 2016-06-02T18:48:45  *** frankenmint_ has quit IRC
480 2016-06-02T18:49:14  *** TheFactory7 has joined #bitcoin-core-dev
481 2016-06-02T18:50:28  *** frankenm_ has quit IRC
482 2016-06-02T18:50:33  *** GAit has joined #bitcoin-core-dev
483 2016-06-02T18:50:39  *** frankenm_ has joined #bitcoin-core-dev
484 2016-06-02T18:51:05  *** frankenm_ is now known as frankenmint_
485 2016-06-02T18:51:47  *** franken__ has quit IRC
486 2016-06-02T18:55:46  <GitHub100> [bitcoin] mrbandrews opened pull request #8141: Continuing port of java comparison tool (master...ba-comptool) https://github.com/bitcoin/bitcoin/pull/8141
487 2016-06-02T18:56:01  *** moli has joined #bitcoin-core-dev
488 2016-06-02T18:56:22  *** jannes_ has quit IRC
489 2016-06-02T18:57:13  *** Hoshea has joined #bitcoin-core-dev
490 2016-06-02T18:58:07  *** molz has quit IRC
491 2016-06-02T18:58:46  *** BakSAj has joined #bitcoin-core-dev
492 2016-06-02T18:58:52  <BakSAj> hi
493 2016-06-02T18:59:17  <wumpus> hi
494 2016-06-02T18:59:54  <BakSAj> bitcoin core live meeting now?
495 2016-06-02T18:59:59  <wumpus> yes
496 2016-06-02T19:00:04  <BakSAj> good :-)
497 2016-06-02T19:00:10  <sipa> yes
498 2016-06-02T19:00:13  <BakSAj> any outlook on merging segwit to master?
499 2016-06-02T19:00:23  <wumpus> #startmeeting
500 2016-06-02T19:00:23  <lightningbot> Meeting started Thu Jun  2 19:00:23 2016 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
501 2016-06-02T19:00:23  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
502 2016-06-02T19:00:35  <luke-jr> aww, too fast
503 2016-06-02T19:00:59  <luke-jr> I was going to propose a topic very unseriously explicitly outside the meeting. :P now it's time to be serious
504 2016-06-02T19:01:09  <petertodd> hi
505 2016-06-02T19:01:34  <wumpus> I guess segwit is the recurring topic, any other topic proposals?
506 2016-06-02T19:01:34  <gmaxwell> jtimon: Cory: morcos: sdaftuar: btcdrak: phantomcircuit: jonasschnelli:
507 2016-06-02T19:01:42  <jonasschnelli> Here!
508 2016-06-02T19:01:47  <sipa> cfields_!
509 2016-06-02T19:02:02  <gmaxwell> I can give some updates on compactblock testing, since it seems that Matt isn't around (or if he shows up, I expect he can)
510 2016-06-02T19:02:42  <wumpus> great
511 2016-06-02T19:03:17  <wumpus> #topic segwit review
512 2016-06-02T19:03:22  *** davec has quit IRC
513 2016-06-02T19:03:49  <wumpus> let's start with that then
514 2016-06-02T19:04:01  <sipa> My plan right now is making the BIP9/GBT changes, removing segnet, removing the positive witness flag, and then creating a parallel rebase
515 2016-06-02T19:04:12  <sipa> with has a clean history but leads to the same tree
516 2016-06-02T19:04:19  <sipa> if that is something wanted at this point
517 2016-06-02T19:04:20  <wumpus> 111 comments on PR #7910, that must be a record
518 2016-06-02T19:04:32  <luke-jr> "positive witness flag"?
519 2016-06-02T19:05:27  <sipa> luke-jr: originally, there was a flag to the serialize code to say "serialize witnesses!"; at some point we realized that the opposite was better, as almost always you want to serialize witnesses, and there are just a few exceptions
520 2016-06-02T19:05:48  <sipa> so i introduced both temporarily, and required that exactly one of both was set
521 2016-06-02T19:05:51  <luke-jr> oh, I thought having it explicit was a good idea
522 2016-06-02T19:06:23  <sipa> luke-jr: one reason for the opposite is that a failure where you miss the positive flag will not be detected usually, but failure to set a negative flag will
523 2016-06-02T19:06:30  <sipa> luke-jr: i thought the same thing initially
524 2016-06-02T19:06:59  <instagibbs> sipa, are you removing new wallet functionality as well?
525 2016-06-02T19:07:07  <sipa> instagibbs: ?
526 2016-06-02T19:07:09  <luke-jr> sipa: what downsides are there to the current explicit flag?
527 2016-06-02T19:07:25  <sipa> 21:06:22 < sipa> luke-jr: one reason for the opposite is that a failure where you miss the positive flag will not be detected usually, but failure to set a negative flag will
528 2016-06-02T19:07:37  <instagibbs> I assume you don't want users to have segwit addresses pre-rollout
529 2016-06-02T19:07:41  <petertodd> luke-jr: indeed, I might have very well written it with a separate CWitnessTx class :)
530 2016-06-02T19:07:53  <luke-jr> sipa: I don't understand that answer. :<
531 2016-06-02T19:07:55  <instagibbs> (unless that would still be a testing branch)
532 2016-06-02T19:08:31  <sipa> luke-jr: if we use a positive flag only, and we miss setting that flag somewhere, it won't easily be detected, as wherever that data goes, it will likely also accept tx without witness
533 2016-06-02T19:08:45  <luke-jr> right, I'm talking about where we have both positive and negative flags..
534 2016-06-02T19:08:51  <sipa> ah, that is also an option
535 2016-06-02T19:08:55  *** davec has joined #bitcoin-core-dev
536 2016-06-02T19:09:05  <sipa> it's just more code changes shattered all over the place
537 2016-06-02T19:09:43  <luke-jr> less likely to have it accidentally wrong, though
538 2016-06-02T19:09:49  <gmaxwell> With segwit in place there really isn't much further concern about getting the wrong flags, it should only have been an issue with the migration where support had to be added in many places (like RPCs) that might have less test coverage.
539 2016-06-02T19:10:11  <gmaxwell> If you get it wrong in the future, the thing you changed simply won't work right. And hopefully you'll be testing the thing you just changed.
540 2016-06-02T19:10:20  <gmaxwell> I don't have a strong or strongly held opinion however.
541 2016-06-02T19:10:50  <luke-jr> we have a lot of outstanding PRs that may need to be updated that might not conflict
542 2016-06-02T19:10:50  <jtimon> ack on cleaning history while removing the testnet
543 2016-06-02T19:11:02  <wumpus> instagibbs: well in principle, master is attesting branch
544 2016-06-02T19:11:35  <sipa> there are a few things we need in first, though
545 2016-06-02T19:11:37  <petertodd> jtimon: you mean segnet?
546 2016-06-02T19:11:39  <wumpus> jtimon: you mean removing the segnet? don't make him remove testnet too :)
547 2016-06-02T19:11:44  <instagibbs> oh a testing, not attesting
548 2016-06-02T19:12:03  <jtimon> petertodd: I believe sipa meant removing segwit's testnet
549 2016-06-02T19:12:13  <gmaxwell> sipa: in any case, what do you think will let you get the code merged sooner?
550 2016-06-02T19:12:13  <sipa> #7749 and #8083
551 2016-06-02T19:12:27  <wumpus> but yes ACK on cleaning up the history before merge
552 2016-06-02T19:12:37  <gmaxwell> I like cleaning history for sure.
553 2016-06-02T19:12:43  <gmaxwell> Future me thanks you.
554 2016-06-02T19:12:57  *** fengling has joined #bitcoin-core-dev
555 2016-06-02T19:13:03  <sipa> well it definitely has to happen before merge
556 2016-06-02T19:13:05  <jonasschnelli> As said, we can ACK the sha256sum of the diff (if someone cares about integrity of an ACK)
557 2016-06-02T19:13:08  <sipa> the question whether the time is now for that
558 2016-06-02T19:13:17  <luke-jr> (GBT VB as well)
559 2016-06-02T19:13:21  <sipa> jonasschnelli: git internally has tree hashes
560 2016-06-02T19:13:35  <jonasschnelli> sipa: Nice. That should work.
561 2016-06-02T19:14:13  <sipa> So please review #7749 and #8083
562 2016-06-02T19:14:26  <luke-jr> did we get anywhere with the fundrawtransaction issue?
563 2016-06-02T19:14:40  <sipa> luke-jr: i can't remember the conclusion there
564 2016-06-02T19:14:49  <jonasschnelli> Re 8083: im finalizing the seeder part (nits and filterbitmap-whitelist)
565 2016-06-02T19:14:49  <wumpus> q: is everything in the segwit PR meant to go in at once, or will it be split up in to, say, a pull with consensus/network changes, following up with wallet, and GUI changes
566 2016-06-02T19:15:03  <wumpus> #action review #7749 and #8083
567 2016-06-02T19:15:07  <luke-jr> sipa: IIRC, 1) it's a problem; 2) we won't change consensus code to fix it
568 2016-06-02T19:15:26  <luke-jr> I don't know of any actual resolution to the problem :/
569 2016-06-02T19:15:41  <gmaxwell> wumpus: If sipa does the "rewrite history to result in the same code thing" then if it is is PR split they will need to go in one right after another to preserve the "same code" property.
570 2016-06-02T19:15:56  <gmaxwell> (I think)
571 2016-06-02T19:16:02  <sipa> wumpus: that's a good question; the history is broken into sections already
572 2016-06-02T19:16:06  <sipa> so we can decide later
573 2016-06-02T19:16:21  <sipa> though... maybe not
574 2016-06-02T19:16:26  <gmaxwell> I suppose sipa should show "same code" at one point, then split, and the remaining parts could then change.
575 2016-06-02T19:16:38  <sipa> the unit/rpc/p2p tests rely on the wallet code
576 2016-06-02T19:16:39  <wumpus> gmaxwell: I don't have a strong opinion either way, if this change is sufficiently atomic it makes sense to do it at once, but see e.g. instagibbs's comment about wallet addresses
577 2016-06-02T19:17:01  *** laurentmt has quit IRC
578 2016-06-02T19:17:12  <wumpus> maybe hide it behind an option in the beginning?
579 2016-06-02T19:17:23  <gmaxwell> My recommendation for the wallet parts was to just hide the changed functionality there behind a hidden configuration option that the tests could use.
580 2016-06-02T19:17:28  <jtimon> well, the wallet code can be maybe be introduced disabled for users with a constant to be removed later or something?
581 2016-06-02T19:17:29  <sipa> maybe we can disable addwitnessaddress before the softfork
582 2016-06-02T19:17:32  <wumpus> gmaxwell: right
583 2016-06-02T19:17:37  <sipa> that would make sense
584 2016-06-02T19:17:44  *** fengling has quit IRC
585 2016-06-02T19:17:47  <wumpus> jtimon: hey we're all saying the same :)
586 2016-06-02T19:17:51  <jtimon> wumpus: yes
587 2016-06-02T19:17:52  <sipa> yay, agreement
588 2016-06-02T19:17:57  <sipa> i have another question
589 2016-06-02T19:18:03  <gmaxwell> sipa: what is the meaning of life?
590 2016-06-02T19:18:06  <sipa> 42
591 2016-06-02T19:18:06  <jonasschnelli> :)
592 2016-06-02T19:18:14  <gmaxwell> thats an answer, not a question!
593 2016-06-02T19:18:32  <luke-jr> he has both the answer and a question
594 2016-06-02T19:18:34  <gmaxwell> We're going to need to build a bigger computer...
595 2016-06-02T19:18:36  <sipa> jl2012 brought up the issue that our witness program definition is limited to 16 versions, and that is not easy to extend without introducing a new witness storage
596 2016-06-02T19:18:57  <sipa> there is an easy solution, by allowing witness programs to be slightly larger
597 2016-06-02T19:19:01  <sipa> but this is a hard forking change
598 2016-06-02T19:19:11  <luke-jr> sipa: it is? I thought the version could be any length?
599 2016-06-02T19:19:14  <sipa> which, if done unconditionally, could hardfork testnet
600 2016-06-02T19:19:22  <sipa> luke-jr: nope, has to be OP_0 through OP_16
601 2016-06-02T19:19:26  <luke-jr> :/
602 2016-06-02T19:19:32  <luke-jr> why limit it?
603 2016-06-02T19:19:42  <jtimon> then would be something to move to the later hardfork, no?
604 2016-06-02T19:20:01  <sipa> jtimon: i don't like relying on that
605 2016-06-02T19:20:08  <gmaxwell> Oh ... how'd that happen? in any case, you can simply use 0..16 and then use another byte after that one.
606 2016-06-02T19:20:16  <luke-jr> jtimon: we cannot assume there will ever be a hardfork.
607 2016-06-02T19:20:18  <sipa> gmaxwell: max 32 bytes can follow
608 2016-06-02T19:20:30  <gmaxwell> okay thats broken.
609 2016-06-02T19:20:43  * luke-jr doesn't see the purpose of restricting the witness-triggering sPK like that
610 2016-06-02T19:20:45  <jtimon> well, the plan was to deploy segwit as a softfork
611 2016-06-02T19:21:08  <sipa> jtimon: changing it is a HF with respect to the current SW code
612 2016-06-02T19:21:08  <gmaxwell> My assumption was that it would be arbritary and extended by just adding more bytes when the space ran out.
613 2016-06-02T19:21:13  <sipa> jtimon: not with respect to bitcoin
614 2016-06-02T19:21:34  <sipa> jtimon: so we can change it before merge while leaving SW a SF
615 2016-06-02T19:21:36  <jtimon> sipa: oh, I see, it would still be a SF to bitcoin. ok
616 2016-06-02T19:21:54  <sipa> gmaxwell, luke-jr: the reason was that constant size utxos are nice
617 2016-06-02T19:21:55  <gmaxwell> sipa: testnet segwit rules can be changed so activiation doesn't begin until later.
618 2016-06-02T19:22:06  <gmaxwell> So at worst that would be a reindex for testnet nodes, no?
619 2016-06-02T19:22:11  <sipa> gmaxwell: ... segwit is already activated on testnet
620 2016-06-02T19:22:16  <gmaxwell> yes, so?
621 2016-06-02T19:22:16  <luke-jr> sipa: they're already non-constant size.
622 2016-06-02T19:22:22  <sipa> luke-jr: ?
623 2016-06-02T19:22:23  <gmaxwell> sipa: move it forward, reindex.
624 2016-06-02T19:22:24  <jtimon> testnet4 ?
625 2016-06-02T19:22:27  <gmaxwell> luke-jr: he means in the future.
626 2016-06-02T19:22:36  <gmaxwell> sipa: in any case, 16 is unacceptably too few.
627 2016-06-02T19:22:40  <sipa> agree
628 2016-06-02T19:23:26  <sipa> gmaxwell: hmm, ok
629 2016-06-02T19:23:31  <instagibbs> The bip doesn't read like it would be a HF, but maybe I'm being obtuse
630 2016-06-02T19:23:37  <gmaxwell> I don't think constant size is as interesting as constant bound. so if you wanted to say that the version was limited to N bytes for some small N that would be fine. 4294967296 versions should be enough for anyone.
631 2016-06-02T19:23:38  <luke-jr> gmaxwell: in the future, if we softfork out the current long sPKs, we can softfork a limit on witness sPK length as well
632 2016-06-02T19:23:52  *** frankenmint_ is now known as frankenmint
633 2016-06-02T19:23:58  <sipa> fair enough, will do
634 2016-06-02T19:24:09  <sipa> limit to 36 or 40 bytes or so
635 2016-06-02T19:24:10  <jtimon> instagibbs: it would be a HF only for testnet3 which has already deployed "old segwit"
636 2016-06-02T19:24:31  <instagibbs> jtimon, no I mean, anything where version is 1+ has no consensus meaning yet
637 2016-06-02T19:24:39  <gmaxwell> sipa: cool (also, testnet behavior has never been in a merged much less released version, I don't mind breaking it)
638 2016-06-02T19:24:41  <instagibbs> when it's not simply 0
639 2016-06-02T19:25:15  <sipa> instagibbs: the problem is that something of the form "1 + 33 bytes" is not treated as a witness program, and is not allowed to have any witness data
640 2016-06-02T19:25:32  <sipa> we can extend the definition of a witness program, but it would require a new witness structure
641 2016-06-02T19:25:48  <sipa> as old nodes won't relay witness data with something they don't treat as a witness output
642 2016-06-02T19:25:59  <sipa> (and that rule is necessary to prevent spam)
643 2016-06-02T19:26:26  *** laurentmt has joined #bitcoin-core-dev
644 2016-06-02T19:26:27  <sipa> anyway, ok, will just change the rule
645 2016-06-02T19:26:36  <sipa> other topics?
646 2016-06-02T19:26:42  <wumpus> yes
647 2016-06-02T19:26:45  <luke-jr> old nodes won't relay anything with witness data they can't verify anyway..
648 2016-06-02T19:26:48  <sipa> (or more segwit review related points)
649 2016-06-02T19:26:56  <wumpus> #topic compactblock testing
650 2016-06-02T19:27:00  <sipa> luke-jr: old nodes in this case being witness v0 nodes
651 2016-06-02T19:27:05  <instagibbs> luke-jr, I'm not convinced, but I think fixing now is better anyways so whatever
652 2016-06-02T19:27:06  <luke-jr> sipa: yes, I also mean these
653 2016-06-02T19:27:11  <wumpus> @gmaxwell
654 2016-06-02T19:27:13  <luke-jr> IMO make any length limit a relay policy only.
655 2016-06-02T19:27:21  <sipa> we'll discuss further in #segwit-dev
656 2016-06-02T19:27:24  <luke-jr> k
657 2016-06-02T19:27:27  <instagibbs> ack
658 2016-06-02T19:28:00  <jtimon> compackblock
659 2016-06-02T19:28:23  <gmaxwell> Okay. So matt has built a parallel relay network that uses compact blocks and the UDP network block coding stuff.
660 2016-06-02T19:28:24  <sipa> i rebased BlueMatt's compactblock patch on top of the shared_ptr mempool change, and gmaxwell and i have been succesfully running it across the internet
661 2016-06-02T19:28:38  <sipa> ^ and that's more interesting
662 2016-06-02T19:28:54  <gmaxwell> ^ a number of people-- including some large miners-- are running both Sipa's rebase,  and Matt's PR without the rebase on public nodes.
663 2016-06-02T19:29:16  <gmaxwell> Which are also connecting to matt's nodes.
664 2016-06-02T19:29:31  <wumpus> good to hear
665 2016-06-02T19:29:33  <gmaxwell> (I got bored with the simulated topology lab testing, this is potentially more interesting)
666 2016-06-02T19:29:39  <sipa> 2016-06-02 18:36:13.412286 Successfully reconstructed block 0000000000000000014a6a924544dd097d538314281bebd95c89e50726e7d2ef with 1 txn prefilled, 2708 txn from mempool and 0 txn requested
667 2016-06-02T19:29:43  <sipa> 2016-06-02 18:37:46.728092 Successfully reconstructed block 000000000000000001943282946e9579fd84def95df577ebb8bcda3b8d9f4d06 with 1 txn prefilled, 1529 txn from mempool and 0 txn requested
668 2016-06-02T19:29:47  <sipa> 2016-06-02 18:43:32.713909 Successfully reconstructed block 000000000000000000499e1485726cd87003e7a6400118f8c061171748665612 with 1 txn prefilled, 1102 txn from mempool and 3 txn requested
669 2016-06-02T19:29:52  <wumpus> yes, always good to test with real world data as well
670 2016-06-02T19:29:55  *** GAit has quit IRC
671 2016-06-02T19:30:03  <gmaxwell> I don't know that there is much to report, it's working as expected.  :)   Sipa's rebase on the sharedptr is pretty nice.
672 2016-06-02T19:30:23  <instagibbs> gmaxwell, the rebase includes the predicted transactions?
673 2016-06-02T19:30:29  <BakSAj> nice
674 2016-06-02T19:30:31  <gmaxwell> As it also eliminates transaction duplication from the relay pool, and eliminates a fair bit of transaction copying.
675 2016-06-02T19:30:31  <wumpus> is this branch available somewhere?
676 2016-06-02T19:30:33  <sipa> instagibbs: it only sends the coinbase directly
677 2016-06-02T19:30:38  <instagibbs> sipa, ah k
678 2016-06-02T19:30:52  <gmaxwell> instagibbs: no, right now I don't believe anyone is running something with fancy prediction.  I think we'll leave that out in the initial PR. Easily added later.
679 2016-06-02T19:30:52  <sipa> wumpus: https://github.com/sipa/bitcoin/commits/compactblocks
680 2016-06-02T19:31:39  <wumpus> #link sipa's rebase of compactblocks on top of PR #8126: https://github.com/sipa/bitcoin/commits/compactblocks
681 2016-06-02T19:31:54  <wumpus> #action review PR #8126
682 2016-06-02T19:32:01  <gmaxwell> if other developers here are interested in running nodes connected to these nodes, lemme know and I'll give you addresses to connect to.
683 2016-06-02T19:32:17  <wumpus> I'm interested
684 2016-06-02T19:32:28  <gmaxwell> I should take an action to setup a couple on published addresses too, for people to connect to without asking. :)
685 2016-06-02T19:32:59  <wumpus> yes, that always works best :)
686 2016-06-02T19:33:15  <wumpus> any other topics?
687 2016-06-02T19:33:19  <luke-jr> is sipa's rebase different enough that we ought to switch to reviewing that instead?
688 2016-06-02T19:33:20  <gmaxwell> Yes, though they may get DDOS attacked, which is harmless but would waste time sorting out the issue. :)
689 2016-06-02T19:33:41  <sipa> luke-jr: it's less code than the original :)
690 2016-06-02T19:33:55  <wumpus> gmaxwell: you mean thoroughly stress-tested :)
691 2016-06-02T19:34:03  * gmaxwell stabs
692 2016-06-02T19:34:16  <gmaxwell> Does anyone know the current CFPF status?
693 2016-06-02T19:34:34  <wumpus> #topic current CPFP status
694 2016-06-02T19:34:43  <sipa> gmaxwell: review #7598
695 2016-06-02T19:34:51  <luke-jr> afaik no show-stoppers found, but more review needed; there's a dep PR to get in first though
696 2016-06-02T19:34:52  <wumpus> #action  review #7598
697 2016-06-02T19:35:08  <sipa> it's a blocker/dependency for CPFP (#7600)
698 2016-06-02T19:36:18  *** BashCo has joined #bitcoin-core-dev
699 2016-06-02T19:37:33  <gmaxwell> I've been struggling a bit with too many PRs outstanding at once that I want to test.
700 2016-06-02T19:37:35  <wumpus> #link CPFP is that like 'strong AI should be here in less than 20 years'
701 2016-06-02T19:37:41  <wumpus> EH
702 2016-06-02T19:37:47  <luke-jr> :<
703 2016-06-02T19:37:48  <wumpus> #link https://github.com/bitcoin/bitcoin/pull/7600
704 2016-06-02T19:38:01  <wumpus> (sorry, copy paste messup)
705 2016-06-02T19:38:04  <gmaxwell> Merging them is a pain.  (thanks to sipa for merging a lot of things recently!)
706 2016-06-02T19:38:48  <sipa> i've been going through all PRs... there are so many decent-but-not-quite-finished ones in the queue...
707 2016-06-02T19:38:50  <wumpus> I've lost overview a bit
708 2016-06-02T19:38:58  <wumpus> any PRs that should be close to be able to be merged?
709 2016-06-02T19:39:17  <wumpus> sipa: yes, I've noticed
710 2016-06-02T19:39:21  <jonasschnelli> IMO https://github.com/bitcoin/bitcoin/pull/7957 can be merged (save, tool only)
711 2016-06-02T19:39:29  <gmaxwell> Mostly my minor difficulty there just comes from many things touching the same parts, and a lot of that was actually my fault. (e.g. opening three PRs at once that all conflicted with each other. :) )
712 2016-06-02T19:39:51  <luke-jr> I'd like to see key generation type merged so there's no risk of other wallet upgrades conflicting it since these wallets are in the wild
713 2016-06-02T19:39:56  <sipa> 17:19:13 < sipa> ping for review: https://github.com/bitcoin/bitcoin/pull/7948 (RPC: augment getblockchaininfo bip9_softforks data)
714 2016-06-02T19:39:59  <sipa> 17:20:03 < sipa> ping for review: https://github.com/bitcoin/bitcoin/pull/7967 ([RPC] add feerate option to fundrawtransaction)
715 2016-06-02T19:40:23  <wumpus> jonasschnelli: agree, but that one is probably not blocking anything
716 2016-06-02T19:40:37  <sipa> also 7997
717 2016-06-02T19:41:07  <jonasschnelli> I'm requesting permission to merge [docs] and [tools] PRs
718 2016-06-02T19:41:20  <wumpus> jonasschnelli: sure
719 2016-06-02T19:42:09  <gmaxwell> sounds fine, I know we sometimes don't get enough review on those, esp docs. Please feel empowered to nag people to review things.
720 2016-06-02T19:42:11  <jonasschnelli> Okay. Will try to focus on trivial docs PRs, so wumpus and sipa can take care about the corish ones.
721 2016-06-02T19:42:24  <luke-jr> https://github.com/bitcoin/bitcoin/pull/7935 is ready IMO
722 2016-06-02T19:42:46  <wumpus> the problem with doc pulls is usually that they get review comments but the author disappears for large spans of time
723 2016-06-02T19:42:47  <sipa> luke-jr: agree
724 2016-06-02T19:43:10  <sipa> luke-jr: will do final review and reack
725 2016-06-02T19:43:28  <cfields_> luke-jr: that looked good to me last time I checked. I'll re-review as well.
726 2016-06-02T19:44:52  <sipa> also #7825 is good
727 2016-06-02T19:45:26  <sipa> and #7942
728 2016-06-02T19:46:23  <jonasschnelli> Also have a look at https://github.com/bitcoin/bitcoin/pull/7946 (could speed up things a little bit)
729 2016-06-02T19:46:26  <wumpus> #7942 has an unaddressed comment by sipa
730 2016-06-02T19:46:56  <sipa> tiny nit :)
731 2016-06-02T19:47:19  *** skang404 has joined #bitcoin-core-dev
732 2016-06-02T19:47:21  <jonasschnelli> nit is nit!
733 2016-06-02T19:47:38  <wumpus> that's not always clear to me whether it should block merging
734 2016-06-02T19:47:46  <wumpus> (I usually at least wait for the author to respond)
735 2016-06-02T19:48:08  <sipa> the author is active, he probably just missed it
736 2016-06-02T19:48:16  <sipa> jonasschnelli just pinged
737 2016-06-02T19:48:21  <wumpus> ok good
738 2016-06-02T19:48:24  <luke-jr> oh, topic: 0.11.next
739 2016-06-02T19:49:16  <jonasschnelli> luke-jr, I guess we already discussed the 0.11 maintenance?
740 2016-06-02T19:49:16  <wumpus> ok
741 2016-06-02T19:49:21  <wumpus> #topic 0.11.next
742 2016-06-02T19:49:53  <luke-jr> jonasschnelli: ?
743 2016-06-02T19:50:12  <sipa> 0.11 goes to critical fixes only when 0.13 is released, right?
744 2016-06-02T19:50:13  <jtimon> luke-jr: 0.11.next is supposed to include csv but not segwit, right?
745 2016-06-02T19:50:33  <jonasschnelli> I had in mind we we BP to 0.12, 0.13 and only security stuff to 0.11?
746 2016-06-02T19:50:37  <luke-jr> jtimon: unless it gets delayed until segwit is merged, I guess
747 2016-06-02T19:50:52  <wumpus> is there any urgent reason to do a 0.11 release?
748 2016-06-02T19:50:53  <luke-jr> sipa: unless someone decides to maintain it longer
749 2016-06-02T19:51:00  <luke-jr> wumpus: CSV support, at least
750 2016-06-02T19:51:05  <wumpus> right
751 2016-06-02T19:51:20  <jtimon> wumpus: is there any reason not to do it while things are backported and tested?
752 2016-06-02T19:51:24  <gmaxwell> looking at the network I'm not seeing any evidence of need to maintain 0.11 extensively, also we called for people running older versions in operations and got crickets in response, AFAIK.
753 2016-06-02T19:51:33  *** belcher has joined #bitcoin-core-dev
754 2016-06-02T19:51:35  * jonasschnelli checking the seeder
755 2016-06-02T19:51:36  <wumpus> jtimon: developer overhead
756 2016-06-02T19:51:49  <sipa> jtimon: is it tested?
757 2016-06-02T19:51:50  <jtimon> wumpus: well, that's luke-jr's problem, isn't it?
758 2016-06-02T19:52:33  <luke-jr> gmaxwell: we did? I don't recall seeing the "call for", and I know for a fact that Eligius relies on 0.11 for now
759 2016-06-02T19:52:58  <jonasschnelli> 88  0.11  nodes
760 2016-06-02T19:53:06  <jtimon> sipa: I don't know, I'm not reviewing or testing myself, but if luke-jr gets review and testing...
761 2016-06-02T19:53:19  <sipa> luke-jr: perhaps the time is better spent in upgrading eligius then :)
762 2016-06-02T19:53:28  <luke-jr> jtimon: which is unlikely, hence the history of git/rc only stable stuff :P
763 2016-06-02T19:53:37  <luke-jr> sipa: probably.
764 2016-06-02T19:53:49  <gmaxwell> esp since master will hopefully have CPFP soon.
765 2016-06-02T19:53:51  <luke-jr> jonasschnelli: what?
766 2016-06-02T19:53:56  <wumpus> yes, interest in older major versions is extrememly low
767 2016-06-02T19:54:09  <btcdrak> there is a backport PR for 0.11 for CSV etc. but we sort of semi decided back then it was not urgent and much more risky.
768 2016-06-02T19:54:14  <luke-jr> I see 1768 0.11 nodes.
769 2016-06-02T19:54:28  <btcdrak> the BIP68 backport in particular is complex
770 2016-06-02T19:54:40  <jtimon> my only point is that we shouldn't use the "we only promise to maintain the last 2 versions" as an artificial limitation beyond review and testing. If people are interested in working on that...
771 2016-06-02T19:54:41  <gmaxwell> well in particular, interest in _updates_ for old versions is low...
772 2016-06-02T19:54:55  <wumpus> gmaxwell: yes that's what I mean...
773 2016-06-02T19:54:59  <luke-jr> we should remove the promise if we're not going to uphold it.
774 2016-06-02T19:55:12  <wumpus> jtimon: the problem is that people are not interested
775 2016-06-02T19:55:17  <gmaxwell> what promise?
776 2016-06-02T19:55:26  <jtimon> well "promise"
777 2016-06-02T19:55:35  <gmaxwell> also, not "not going"-- not able.. without people to test you really can't provide good releases.
778 2016-06-02T19:55:48  <jtimon> wumpus: by people you mean users or reviewers/testers?
779 2016-06-02T19:55:50  <gmaxwell> not doing someone a service to put out a barely tested update.
780 2016-06-02T19:55:57  <wumpus> I mean if luke-jr really wants to handle a release by himself I'm not going to protest
781 2016-06-02T19:56:05  <gmaxwell> ^ agreed.
782 2016-06-02T19:56:05  <luke-jr> https://bitcoincore.org/en/lifecycle/ mentions something; whether promise or able or not, it should be updated if we can't do it
783 2016-06-02T19:56:10  <btcdrak> the CSV backport PR was https://github.com/bitcoin/bitcoin/pull/7716
784 2016-06-02T19:56:22  <btcdrak> we did pretty much decide not to merge it.
785 2016-06-02T19:56:23  <luke-jr> wumpus: I can't get testing/reviews by myself.
786 2016-06-02T19:56:28  <jonasschnelli> luke-jr: sorry,.. wrong file: we have 743 0.11x nodes and 1786 0.12.x nodes... so yes. CSV for 0.11 makes sense.
787 2016-06-02T19:56:41  <luke-jr> I can maintain stable branches, but releases seem unlikely to work out at this point
788 2016-06-02T19:56:54  <wumpus> but there's so much happening on master right now, and 0.13 release is near, I can't promise I will be able to spend any time on it (except gitian building / uploading executables)
789 2016-06-02T19:57:23  <jonasschnelli> Yes. 0.11 is certainly not something for wumpus (would be a waste of his time)
790 2016-06-02T19:57:57  <gmaxwell> Without people interested in using it, we can't get much platform qualification which is a lot of the difference between a branch and a release.
791 2016-06-02T19:57:59  <wumpus> so if you want to do this: please create your own 0.11 branch, tag, do the release notes etc
792 2016-06-02T19:58:00  <sipa> jtimon: i have 1093 'good' 0.11 nodes
793 2016-06-02T19:58:09  <sipa> eh, jonasschnelli:
794 2016-06-02T19:58:33  <jonasschnelli> good is not good enough...
795 2016-06-02T19:58:34  <jonasschnelli> cat dnsseed.dump | grep " 100.00% 100.00% 100.00% " | grep "Satoshi:0.11." | wc -l
796 2016-06-02T19:59:05  <sipa> anyway, besided the point
797 2016-06-02T19:59:14  <sipa> we can't do releases without people interested in running and testing
798 2016-06-02T19:59:17  <wumpus> yes, meeting time over
799 2016-06-02T19:59:21  <sipa> oh, that too
800 2016-06-02T19:59:24  <wumpus> #endmeeting
801 2016-06-02T19:59:24  <lightningbot> Meeting ended Thu Jun  2 19:59:24 2016 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
802 2016-06-02T19:59:24  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-06-02-19.00.html
803 2016-06-02T19:59:24  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-06-02-19.00.txt
804 2016-06-02T19:59:24  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-06-02-19.00.log.html
805 2016-06-02T19:59:35  <jtimon> maybe we can merge it into the 0.11 branch without doing a release?
806 2016-06-02T20:00:01  <wumpus> jtimon: sure. luke-jr can run his own 0.11 branch, I could just take that over
807 2016-06-02T20:00:18  *** cryptapus has quit IRC
808 2016-06-02T20:00:18  <gmaxwell> could be done, I suppose. But thing is: soft forks are more or less safe to run without, but a broken update may be less safe.
809 2016-06-02T20:00:27  <gmaxwell> not that I think there is a lot of risk there.
810 2016-06-02T20:00:37  <jtimon> and if it ever gets tested, release, otherwise there's never a 0.11.next
811 2016-06-02T20:00:42  <wumpus> well if no one runs it, it doesn't create much risk either :)
812 2016-06-02T20:00:56  <gmaxwell> Good point.
813 2016-06-02T20:01:48  <luke-jr> wumpus: should I continue to maintain stable branches in a separate repo (the old one was on Gitorious which is dead, so a new location needs deciding if so), or would it make sense to do it on the main repo now (would require push access for me, at least in the stable branches)?
814 2016-06-02T20:01:58  <luke-jr> [20:00:18] <gmaxwell> could be done, I suppose. But thing is: soft forks are more or less safe to run without, but a broken update may be less safe. <-- my main concern with backporting of the recent softforks
815 2016-06-02T20:02:43  <wumpus> luke-jr: well you don't strictly need push access; I could just pull the 0.11 branch from your repo when you say so
816 2016-06-02T20:02:43  <jtimon> can't we just merge PRs in the 0.11 branch in the main repo?
817 2016-06-02T20:02:47  <btcdrak> luke-jr: morcos specific concern was the safety of BIP68 backport
818 2016-06-02T20:02:54  <luke-jr> wumpus: that'd work too I guess
819 2016-06-02T20:02:59  <btcdrak> the backports were done in #7716
820 2016-06-02T20:03:05  <jtimon> or that, I'll shut up, you people coordinate
821 2016-06-02T20:03:52  <btcdrak> The codebase is significantly different between 0.11 and 0.12 with regards to BIP68
822 2016-06-02T20:03:58  <luke-jr> it probably would make sense to have a separate repo for stable in general, though, so we can't accidentally confuse PRs against the wrong branch
823 2016-06-02T20:05:20  <wumpus> at least the github merge script now automatically gets the branch to merge against from github
824 2016-06-02T20:05:39  <gmaxwell> luke-jr: is there any thing I could help do to get eligius off of 0.11 and to 0.12.1  (and maybe master with CPFP?)
825 2016-06-02T20:07:20  <luke-jr> gmaxwell: my current target is Knots 0.12.1 including CPFP there, so the big step is backporting CPFP in a way that can be turned on/off (which AIUI, the CPFP dep makes easier); I don't think there's a good way to divide this work, however
826 2016-06-02T20:08:31  <gmaxwell> luke-jr: ugh.
827 2016-06-02T20:08:59  <gmaxwell> I don't see what benefit you get from 0.12.1 with such a large and invasive backport.
828 2016-06-02T20:09:36  <luke-jr> as opposed to everything else in master? :x
829 2016-06-02T20:10:48  <gmaxwell> which has a lot more eyes on it, and in the mining relevant codepaths, the changes for CPFP are probably the bulk of the changes.
830 2016-06-02T20:11:08  <gmaxwell> also, look ahead a bit, and you'd have to forward port that backport onto segwit.
831 2016-06-02T20:13:35  <luke-jr> hmm
832 2016-06-02T20:14:27  *** fengling has joined #bitcoin-core-dev
833 2016-06-02T20:15:06  <BakSAj> i understand the need for backporting in enterprise software, where upgrades might get messy, but what is the exact point in btc where process is quite simple...?
834 2016-06-02T20:15:09  <luke-jr> not sure it'd be less work to forward-port Knots to master either, though
835 2016-06-02T20:15:47  *** laurentmt has quit IRC
836 2016-06-02T20:15:54  <luke-jr> BakSAj: consensus systems carry even more risk in upgrades, than enterprise
837 2016-06-02T20:16:01  <gmaxwell> BakSAj: there are many large businesses that use Bitcoin too, and some that have complex customizations.
838 2016-06-02T20:16:05  <sipa> luke-jr: what other kinds of changes does Knots have?
839 2016-06-02T20:16:43  <gmaxwell> BakSAj: unfortunately, though we know these deployments exist-- we seldom hear from people involved with them.
840 2016-06-02T20:18:43  <luke-jr> sipa: http://bitcoinknots.org/ has the full list, but relevant to Eligius.. *skims over list* #7149, #7533, and spamfilter; so maybe easier than backporting CPFP
841 2016-06-02T20:18:48  <BakSAj> i wonder if they do patching
842 2016-06-02T20:19:08  <BakSAj> otherwise its just a waste of dev that can go to master version
843 2016-06-02T20:19:24  *** fengling has quit IRC
844 2016-06-02T20:19:32  <luke-jr> would be nice to get #7149 reviewed and merged finally.
845 2016-06-02T20:20:58  <btcdrak> This was the summary regarding the backports, for the record https://bitcoincore.org/en/meetings/2016/03/31/#softfork-backports
846 2016-06-02T20:28:00  <BakSAj> ok, thanks for clarification
847 2016-06-02T20:29:20  <BakSAj> i hope 0.12.1 will activate soon, so you guys can move on with segwit and get lightning finally
848 2016-06-02T20:29:54  <BakSAj> anybody happen to know when e.g. f2pool moves to 0.12.1 ?
849 2016-06-02T20:34:53  *** rubensayshi has quit IRC
850 2016-06-02T20:35:40  *** face has quit IRC
851 2016-06-02T20:36:06  *** erasmospunk has quit IRC
852 2016-06-02T20:37:12  *** skang404 has quit IRC
853 2016-06-02T20:37:47  *** skang404 has joined #bitcoin-core-dev
854 2016-06-02T20:41:12  *** Hoshea has quit IRC
855 2016-06-02T20:43:35  *** dgenr8 has joined #bitcoin-core-dev
856 2016-06-02T20:45:39  *** BakSAj has quit IRC
857 2016-06-02T21:15:49  *** fengling has joined #bitcoin-core-dev
858 2016-06-02T21:20:44  *** fengling has quit IRC
859 2016-06-02T21:40:36  *** AaronvanW has quit IRC
860 2016-06-02T22:10:48  *** GAit has joined #bitcoin-core-dev
861 2016-06-02T22:17:20  *** fengling has joined #bitcoin-core-dev
862 2016-06-02T22:21:44  *** fengling has quit IRC
863 2016-06-02T22:45:45  <GitHub51> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/ec45cc5e2766...f972b04d63eb
864 2016-06-02T22:45:45  <GitHub51> bitcoin/master 0bf6f30 Pedro Branco: Prevent multiple calls to ExtractDestination
865 2016-06-02T22:45:45  <GitHub51> bitcoin/master f972b04 Pieter Wuille: Merge #7825: Prevent multiple calls to ExtractDestination...
866 2016-06-02T22:45:49  <GitHub30> [bitcoin] sipa closed pull request #7825: Prevent multiple calls to ExtractDestination (master...enhancement/prevent-multiple-calls-extractdestination) https://github.com/bitcoin/bitcoin/pull/7825
867 2016-06-02T22:53:48  *** spudowiar is now known as spudowiar-banned
868 2016-06-02T22:54:49  *** spudowiar-banned has quit IRC
869 2016-06-02T22:58:24  *** spudowiar has joined #bitcoin-core-dev
870 2016-06-02T23:01:23  *** Guyver2 has quit IRC
871 2016-06-02T23:06:52  *** GAit has quit IRC
872 2016-06-02T23:18:33  *** fengling has joined #bitcoin-core-dev
873 2016-06-02T23:18:43  *** spudowiar is now known as spudowiar-ban
874 2016-06-02T23:18:54  *** randy-waterhouse has joined #bitcoin-core-dev
875 2016-06-02T23:20:50  *** spudowiar-ban is now known as spudowiar
876 2016-06-02T23:21:40  *** kadoban is now known as kadoban_1ufYVpS
877 2016-06-02T23:23:24  *** fengling has quit IRC
878 2016-06-02T23:27:14  <GitHub138> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f972b04d63eb...a82f03393a32
879 2016-06-02T23:27:14  <GitHub138> bitcoin/master 9805f4a Kaz Wesley: mapNextTx: use pointer as key, simplify value...
880 2016-06-02T23:27:15  <GitHub138> bitcoin/master a82f033 Pieter Wuille: Merge #7997: replace mapNextTx with slimmer setSpends...
881 2016-06-02T23:27:19  <GitHub69> [bitcoin] sipa closed pull request #7997: replace mapNextTx with slimmer setSpends (master...setSpends) https://github.com/bitcoin/bitcoin/pull/7997
882 2016-06-02T23:36:32  *** kadoban_1ufYVpS is now known as kadoban
883 2016-06-02T23:39:39  *** randy-waterhouse has quit IRC
884 2016-06-02T23:54:04  *** AaronvanW has joined #bitcoin-core-dev