1 2016-09-08T00:28:43  *** justanotherus3r is now known as justanotheruser
  2 2016-09-08T00:36:51  *** Ylbam has quit IRC
  3 2016-09-08T00:57:37  *** Giszmo has quit IRC
  4 2016-09-08T00:58:01  *** Giszmo has joined #bitcoin-core-dev
  5 2016-09-08T01:24:37  *** Lauda has quit IRC
  6 2016-09-08T01:28:09  *** justanotheruser has quit IRC
  7 2016-09-08T01:32:39  *** Lauda has joined #bitcoin-core-dev
  8 2016-09-08T01:38:29  *** Chris_Stewart_5 has quit IRC
  9 2016-09-08T01:39:56  <veleiro> found the problem. couldnt get the depends build to work, but i removed /usr/local/lib/libboost* and /usr/local/include/boost* and reinstalled libboost-all-dev. thanks for the advice sipa
 10 2016-09-08T02:08:00  *** justanotheruser has joined #bitcoin-core-dev
 11 2016-09-08T02:10:03  *** Samdney has left #bitcoin-core-dev
 12 2016-09-08T02:33:26  *** Alopex has quit IRC
 13 2016-09-08T02:34:31  *** Alopex has joined #bitcoin-core-dev
 14 2016-09-08T02:39:40  *** fengling has joined #bitcoin-core-dev
 15 2016-09-08T02:54:33  *** dcousens has quit IRC
 16 2016-09-08T02:56:26  *** dcousens has joined #bitcoin-core-dev
 17 2016-09-08T02:57:44  *** Giszmo has quit IRC
 18 2016-09-08T03:13:07  *** droark has joined #bitcoin-core-dev
 19 2016-09-08T03:17:10  *** dcousens has quit IRC
 20 2016-09-08T03:17:22  *** btcdrak has quit IRC
 21 2016-09-08T03:21:04  *** jtimon has quit IRC
 22 2016-09-08T03:36:07  *** justan0theruser has joined #bitcoin-core-dev
 23 2016-09-08T03:37:19  *** justanotheruser has quit IRC
 24 2016-09-08T04:17:16  *** Alopex has quit IRC
 25 2016-09-08T04:18:21  *** Alopex has joined #bitcoin-core-dev
 26 2016-09-08T04:35:11  *** Alopex has quit IRC
 27 2016-09-08T04:36:16  *** Alopex has joined #bitcoin-core-dev
 28 2016-09-08T05:11:31  *** Alopex has quit IRC
 29 2016-09-08T05:12:36  *** Alopex has joined #bitcoin-core-dev
 30 2016-09-08T05:22:17  *** Alopex has quit IRC
 31 2016-09-08T05:23:22  *** Alopex has joined #bitcoin-core-dev
 32 2016-09-08T05:32:41  <cfields> jeremyrubin: started reviewing your checkqueue, but the emplacer distracted me. Will pick up where I left off tomorrow.
 33 2016-09-08T05:50:51  *** Ylbam has joined #bitcoin-core-dev
 34 2016-09-08T06:35:16  *** kadoban has quit IRC
 35 2016-09-08T06:38:04  *** arubi_ has joined #bitcoin-core-dev
 36 2016-09-08T06:41:34  *** arubi has quit IRC
 37 2016-09-08T06:47:59  *** btcdrak has joined #bitcoin-core-dev
 38 2016-09-08T06:51:58  *** owowo has quit IRC
 39 2016-09-08T06:52:17  *** BashCo has quit IRC
 40 2016-09-08T06:55:30  *** paveljanik has quit IRC
 41 2016-09-08T07:04:12  *** rubensayshi has joined #bitcoin-core-dev
 42 2016-09-08T07:12:00  *** BashCo has joined #bitcoin-core-dev
 43 2016-09-08T07:34:14  <wumpus> let's try to make cfields network refactor (https://github.com/bitcoin/bitcoin/pull/8085/files) happen , it's been open so long now, line-wise there's a lot of changes so he keeps having to rebase it
 44 2016-09-08T07:35:44  <wumpus> /home/user/projects/bitcoin/bitcoin/src/test/addrman_tests.cpp:336:471: warning: stack frame size of 328360 bytes in function 'addrman_tests::addrman_delete::test_method' [-Wframe-larger-than=]    whoa :) luckily only in the tests
 45 2016-09-08T07:37:58  <sipa> what is a reasonable max frame size?
 46 2016-09-08T07:38:05  <wumpus> largest in the normal core code is a stack frame size of 67256 bytes in function 'ThreadSocketHandler', which is peculiar but not that bad
 47 2016-09-08T07:39:03  <wumpus> 64k or so for non-recursive code? most is far below that, even <10kB
 48 2016-09-08T07:40:34  <wumpus> whopping 'warning: stack frame size of 984520 bytes in function 'net_tests::caddrdb_read::test_method'
 49 2016-09-08T07:41:46  <sipa> 8654 introduces a 9 kB struct in CTransactionSignatureChecker
 50 2016-09-08T07:41:55  <wumpus> peanuts
 51 2016-09-08T07:42:09  <sipa> 984kB sounds like mild overkill, indeex
 52 2016-09-08T07:42:11  <sipa> indeed
 53 2016-09-08T07:42:36  <sipa> also, agree on making the net refactor happen
 54 2016-09-08T07:50:55  <jonasschnelli> Yes. Net factor should happen... I have the PR running since a couple of days...
 55 2016-09-08T07:51:14  *** moli has quit IRC
 56 2016-09-08T08:12:31  *** jannes has joined #bitcoin-core-dev
 57 2016-09-08T08:36:19  *** blkdb has quit IRC
 58 2016-09-08T08:36:23  *** gribble has quit IRC
 59 2016-09-08T08:44:36  *** Guyver2 has joined #bitcoin-core-dev
 60 2016-09-08T08:59:02  *** blkdb has joined #bitcoin-core-dev
 61 2016-09-08T09:03:04  *** gribble has joined #bitcoin-core-dev
 62 2016-09-08T09:05:18  *** MarcoFalke has joined #bitcoin-core-dev
 63 2016-09-08T09:13:35  <GitHub87> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/ec139a5621a9...ddc308068d69
 64 2016-09-08T09:13:35  <GitHub87> bitcoin/master f71d4a3 Jeremy Rubin: Minimal fix to slow prevector tests as stopgap measure
 65 2016-09-08T09:13:36  <GitHub87> bitcoin/master ddc3080 MarcoFalke: Merge #8671: Minimal fix to slow prevector tests as stopgap measure...
 66 2016-09-08T09:13:50  <GitHub67> [bitcoin] MarcoFalke closed pull request #8671: Minimal fix to slow prevector tests as stopgap measure (master...simple_faster_tests) https://github.com/bitcoin/bitcoin/pull/8671
 67 2016-09-08T09:25:04  *** blkdb has quit IRC
 68 2016-09-08T09:26:26  *** blkdb has joined #bitcoin-core-dev
 69 2016-09-08T09:41:39  *** blkdb has quit IRC
 70 2016-09-08T09:43:15  *** blkdb has joined #bitcoin-core-dev
 71 2016-09-08T10:06:53  *** Giszmo has joined #bitcoin-core-dev
 72 2016-09-08T10:13:38  *** blkdb has quit IRC
 73 2016-09-08T10:20:12  *** JackH has quit IRC
 74 2016-09-08T10:21:08  *** blkdb has joined #bitcoin-core-dev
 75 2016-09-08T10:30:04  *** blkdb has quit IRC
 76 2016-09-08T10:39:58  *** cryptapus has joined #bitcoin-core-dev
 77 2016-09-08T10:45:02  *** blkdb has joined #bitcoin-core-dev
 78 2016-09-08T10:53:06  *** moli has joined #bitcoin-core-dev
 79 2016-09-08T10:56:48  *** owowo has joined #bitcoin-core-dev
 80 2016-09-08T11:12:12  *** BashCo_ has joined #bitcoin-core-dev
 81 2016-09-08T11:15:07  *** BashCo has quit IRC
 82 2016-09-08T11:20:14  *** blkdb has quit IRC
 83 2016-09-08T11:25:36  *** blkdb has joined #bitcoin-core-dev
 84 2016-09-08T11:38:19  *** blkdb has quit IRC
 85 2016-09-08T11:40:47  *** translatoree3 has joined #bitcoin-core-dev
 86 2016-09-08T11:41:16  <translatoree3> Hello, need help with contributing a translation
 87 2016-09-08T11:41:54  <translatoree3> I've seen transfix, so translations there get credit on the github bitcoin repo?
 88 2016-09-08T11:42:04  <translatoree3> I'd like to get credit as I have a github account
 89 2016-09-08T11:42:54  <translatoree3> is there a way to send a pull request if the Transifex process doesn't give contributor credit on github repo?
 90 2016-09-08T11:44:56  *** blkdb has joined #bitcoin-core-dev
 91 2016-09-08T11:58:25  *** cryptapus_ has joined #bitcoin-core-dev
 92 2016-09-08T12:01:04  *** blkdb has quit IRC
 93 2016-09-08T12:01:18  *** laurentmt has joined #bitcoin-core-dev
 94 2016-09-08T12:02:52  *** cryptapus has quit IRC
 95 2016-09-08T12:04:33  *** molz has joined #bitcoin-core-dev
 96 2016-09-08T12:05:43  <GitHub24> [bitcoin] bitcoinsSG opened pull request #8683: fix incorrect file name bitcoin.qrc  (master...patch-1) https://github.com/bitcoin/bitcoin/pull/8683
 97 2016-09-08T12:07:40  *** translatoree3 has quit IRC
 98 2016-09-08T12:08:16  *** moli has quit IRC
 99 2016-09-08T12:12:37  *** laurentmt has quit IRC
100 2016-09-08T12:17:22  *** jtimon has joined #bitcoin-core-dev
101 2016-09-08T12:19:53  *** dermoth_ has joined #bitcoin-core-dev
102 2016-09-08T12:20:17  *** dermoth has quit IRC
103 2016-09-08T12:20:19  *** dermoth_ is now known as dermoth
104 2016-09-08T12:21:02  *** blkdb has joined #bitcoin-core-dev
105 2016-09-08T12:34:05  *** Chris_Stewart_5 has joined #bitcoin-core-dev
106 2016-09-08T12:36:05  *** MrBTCNor has joined #bitcoin-core-dev
107 2016-09-08T12:41:54  *** blkdb has quit IRC
108 2016-09-08T12:47:26  *** blkdb has joined #bitcoin-core-dev
109 2016-09-08T12:57:45  *** kadoban has joined #bitcoin-core-dev
110 2016-09-08T12:59:15  *** vega4 has quit IRC
111 2016-09-08T13:11:06  *** fengling has quit IRC
112 2016-09-08T13:14:17  *** cryptapus_ has quit IRC
113 2016-09-08T13:14:29  *** cryptapus_ has joined #bitcoin-core-dev
114 2016-09-08T13:14:29  *** cryptapus_ has joined #bitcoin-core-dev
115 2016-09-08T13:16:25  *** cryptapus_ is now known as cryptapus
116 2016-09-08T13:19:07  *** Samdney has joined #bitcoin-core-dev
117 2016-09-08T13:19:53  *** Guyver2_ has joined #bitcoin-core-dev
118 2016-09-08T13:19:59  *** vega4 has joined #bitcoin-core-dev
119 2016-09-08T13:22:36  *** Guyver2 has quit IRC
120 2016-09-08T13:22:38  *** Guyver2_ is now known as Guyver2
121 2016-09-08T13:23:54  *** blkdb has quit IRC
122 2016-09-08T13:24:27  *** veleiro has quit IRC
123 2016-09-08T13:25:44  <sipa> cfields: nice work with the network refactor, and sorry for not having the courage earlier to read through 34 commits (it wasn't all that bad)
124 2016-09-08T13:31:41  * sipa is slightly annoyed with the introduction of many 'for('s and 'if('s, but seems we're not consistent about that already anyway
125 2016-09-08T13:35:30  <cfields> sipa: thanks. np about review. I haven't been pushy about it because it's pretty rough to get through. The next round should be much easier since after this PR it's mostly self-contained
126 2016-09-08T13:35:57  <cfields> and yes, sorry about the fors and ifs. I know that one bugs you. That's a habit that I can't seem to get out of
127 2016-09-08T13:36:29  <sipa> i realize it's just preference, and i'm sure the existing spaces bother you too :p
128 2016-09-08T13:36:29  <cfields> mm, I should get vim to fix that for me
129 2016-09-08T13:37:16  <cfields> not at all, they're only reminders that i forgot to space my changes around it (while re-reading post-commit, of course)
130 2016-09-08T13:41:41  *** TomMc has joined #bitcoin-core-dev
131 2016-09-08T13:59:12  *** Sanakov has joined #bitcoin-core-dev
132 2016-09-08T14:05:38  *** Guyver2_ has joined #bitcoin-core-dev
133 2016-09-08T14:08:04  *** Guyver2 has quit IRC
134 2016-09-08T14:08:12  *** Guyver2_ is now known as Guyver2
135 2016-09-08T14:08:32  *** fengling has joined #bitcoin-core-dev
136 2016-09-08T14:15:26  *** fengling has quit IRC
137 2016-09-08T14:17:43  *** assder has joined #bitcoin-core-dev
138 2016-09-08T14:23:29  *** blkdb has joined #bitcoin-core-dev
139 2016-09-08T14:28:18  *** BashCo_ has quit IRC
140 2016-09-08T14:35:12  *** face_ has joined #bitcoin-core-dev
141 2016-09-08T14:39:33  *** face_ is now known as face
142 2016-09-08T14:48:44  <jeremyrubin> cfields: be sure to look on the latest; I fixed it i beleive
143 2016-09-08T14:54:36  *** MrBTCNor has quit IRC
144 2016-09-08T14:56:34  <cfields> jeremyrubin: see github comments
145 2016-09-08T14:58:56  *** Sanakov has quit IRC
146 2016-09-08T15:02:35  <morcos> cfields: i was just trying to review #8660, so this is also a regression caused by https://github.com/bitcoin/bitcoin/pull/7946?
147 2016-09-08T15:03:11  <jeremyrubin> cfields: yeah you weren't looking at the one i sent you yest
148 2016-09-08T15:03:29  <cfields> morcos: aha, probably so
149 2016-09-08T15:04:24  <morcos> cfields: i wonder if we need to think a bit more carefully about that pull.  is there anything other than the RPC tests that might be depending on wallets being synced onces cs_main is released
150 2016-09-08T15:04:37  <morcos> for instance you could imagine someone using bitcoin core has come to depend on that behavior
151 2016-09-08T15:04:54  <morcos> not saying it shouldn't eventually be changed, but seems like something that requires a bit more thought
152 2016-09-08T15:05:13  <cfields> morcos: think more about 7946, you mean?
153 2016-09-08T15:05:16  <morcos> yeah
154 2016-09-08T15:06:11  <jeremyrubin> cfields: oops I'm wrong you were sorry -- i need to fix my github notifs they suck
155 2016-09-08T15:06:11  <sipa> morcos: 8660? sure you have the right number?
156 2016-09-08T15:06:14  <morcos> what the RPC tests are uncovering is that the wallet can now be out of sync with the blockchain.
157 2016-09-08T15:06:16  <cfields> morcos: now that you mention it, i wonder if blocknotify is affected
158 2016-09-08T15:06:41  <jeremyrubin> sipa: morcos is referring to the already merged one
159 2016-09-08T15:06:42  <morcos> yeoops 8680
160 2016-09-08T15:06:49  <jeremyrubin> oops nvm
161 2016-09-08T15:07:37  <cfields> ok nm, blocknotify uses the same signal, so the wallet would already be synced there
162 2016-09-08T15:07:49  <sipa> cfields: for 8680, isn't it possible to use ping/waitforpong instead?
163 2016-09-08T15:08:01  <morcos> sipa: the issues isn't fixing it for the RPC tests
164 2016-09-08T15:08:36  <morcos> the issue is whether there are things other than the rpc tests that were relying on wallet always being synced with blockchain once cs_main was released
165 2016-09-08T15:09:17  <cfields> sipa: isn't that just punting the problem to "now we assume we're all synced" at a different point?
166 2016-09-08T15:09:28  <morcos> also ping/waitforpong would happen to work i think with the current architecture, but not necessarily in the future right..
167 2016-09-08T15:10:11  <sipa> morcos: i don't think we can get rid of the synchronizing behaviour of ping/pong without breaking half the network software
168 2016-09-08T15:11:01  <morcos> sipa: there is a difference between it always synchronizing network behavior and synchronizing other thigns right?  it just happens that the processing of messages happens in the same thread as the wallet syncing code now
169 2016-09-08T15:11:08  *** vega4 has quit IRC
170 2016-09-08T15:11:31  <morcos> so you couldn't get to the pong until you'd synced the wallets, but that doesn't seem like something that would necessarily always be true
171 2016-09-08T15:11:32  *** rubensayshi has quit IRC
172 2016-09-08T15:11:36  <sipa> morcos: i see
173 2016-09-08T15:11:51  *** fengling has joined #bitcoin-core-dev
174 2016-09-08T15:11:54  <sipa> yes, i think we should prepare for a case where wallet syncing happens entirely in the background
175 2016-09-08T15:12:05  <sipa> and can lag behind more than one block
176 2016-09-08T15:12:30  *** vega4 has joined #bitcoin-core-dev
177 2016-09-08T15:12:31  <sipa> and network synchronizing messages don't change that
178 2016-09-08T15:12:33  <cfields> i suppose that's why sdaftuar was thinking more in terms of wait_for_wallet()
179 2016-09-08T15:12:57  *** mol has joined #bitcoin-core-dev
180 2016-09-08T15:13:03  <morcos> sipa: agreed.  and i think #7946 has forced that issue to be we have to think about that for 0.13.   b/c as far as RPC is concerned the wallet syncing can be happening several blocks behind now.
181 2016-09-08T15:13:30  <morcos> sorry, for master, not 0.13.  i'm still behind.  :)
182 2016-09-08T15:15:44  <BlueMatt> so what probably should happen, to avoid changing the api, is that any time you call into wallet it blocks until it has a consistent state with current blockchain
183 2016-09-08T15:15:48  <cfields> in the event that we start moving towards more async behavior, i think it's only natural to start introducing blocking rpcs
184 2016-09-08T15:16:11  <BlueMatt> otherwise you get an inconsistent rpc api - getblockheight will tell you something and then wallet will be behind that
185 2016-09-08T15:16:15  <BlueMatt> which is a hugely surprising api change
186 2016-09-08T15:16:16  <cfields> heh
187 2016-09-08T15:16:27  *** molz has quit IRC
188 2016-09-08T15:16:29  <morcos> cfields: i thought you just typed super fast
189 2016-09-08T15:16:37  <BlueMatt> moving wallet sync out of main thread is an important goal - and unlocking cs_main for it is a first step
190 2016-09-08T15:16:46  *** fengling has quit IRC
191 2016-09-08T15:17:19  <BlueMatt> but an api change like that is probaly just not ok
192 2016-09-08T15:17:30  <sipa> that does make sense
193 2016-09-08T15:17:47  <sipa> just have the wallet rcps wait for sync
194 2016-09-08T15:18:05  <morcos> but what exactly should they wait for
195 2016-09-08T15:18:26  <BlueMatt> being up-to-date with the chain state at the call entry
196 2016-09-08T15:18:29  <morcos> as you're catching up the blockheight could still be increasing i think
197 2016-09-08T15:18:33  <BlueMatt> or, maybe, lock cs_main...
198 2016-09-08T15:19:03  <cfields> morcos: you'd need to pass in the synced-to-x height to the wallet rpc :(
199 2016-09-08T15:20:40  <morcos> i guess what i just described is an existing problem, so maybe thats ok, b/c cs_main is released anyway
200 2016-09-08T15:21:15  <BlueMatt> i mean, ok, what makes the rpc consistent? if the rpc request blocks, then its not crazy to make the state returned consistent with where the chainstate was when you entered the rpc, because otherwise the client is multithreaded and thats gonna be broken no matter what we do
201 2016-09-08T15:21:20  <cfields> morcos: but, if you've specifically waited for a certain height/hash, and it's blocked/returned until that's hit, then you're at least synced to that height for the next call
202 2016-09-08T15:21:33  <cfields> (ignoring invalidate/reconsider, which ruin all of that)
203 2016-09-08T15:21:36  <morcos> cfields: yes. agreed. i think we should fix it that much.
204 2016-09-08T15:22:26  <cfields> morcos: well, the new rpcs in #8680 do all that we've described here. We could just wrap the old ones around those...
205 2016-09-08T15:23:29  <morcos> cfields: yes that sounds right...  i think we need to call getblockheight from the wallet calls and then use your new calls to wait for that
206 2016-09-08T15:23:58  <sipa> well the wallet can remember independently up to what block hash it is synced
207 2016-09-08T15:24:08  <sipa> which afaik it does not do now
208 2016-09-08T15:24:45  <morcos> sipa: i think you want to avoid querying the wallet mid-block
209 2016-09-08T15:25:09  <BlueMatt> sipa: oh? I thought it did?
210 2016-09-08T15:25:13  <BlueMatt> It used to at least be on-disk, too
211 2016-09-08T15:25:21  <sipa> morcos: eh, yes, how does that matter?
212 2016-09-08T15:25:30  <sipa> BlueMatt: yes, but that isn't uodated for every block
213 2016-09-08T15:26:01  <BlueMatt> ahh, well, ok, lets update it (in memory) for every block :p
214 2016-09-08T15:26:05  <sipa> agree
215 2016-09-08T15:27:34  <morcos> sipa: if the wallet rpcs aren't synced using cory's mechanism then when you check the wallet you don't know if its in the middle of syncing some next block (that maybe you weren't waiting for)
216 2016-09-08T15:27:53  <morcos> but actually, i guess you still don't know that
217 2016-09-08T15:27:54  <morcos> shoot
218 2016-09-08T15:29:07  <morcos> so maybe we need to lock the wallet around the SyncWithWallets loops?
219 2016-09-08T15:30:06  <sipa> hmm?
220 2016-09-08T15:30:17  <cfields> brb
221 2016-09-08T15:32:11  *** bsm117532 has quit IRC
222 2016-09-08T15:32:35  *** bsm117532 has joined #bitcoin-core-dev
223 2016-09-08T15:32:53  <morcos> sipa: isn't there nothing that stops a getbalance() call from happening in the middle of a SyncWithWallets() loop for a given blcok
224 2016-09-08T15:33:00  <sipa> so i'm not sure the out of syncness matters to external observers... if you used to wait for a block and then later do and RPC query, it has always been possible that yet another block got processed in between
225 2016-09-08T15:33:10  <morcos> yes
226 2016-09-08T15:33:17  <morcos> but now half of another block could be processed
227 2016-09-08T15:33:18  <sipa> what matters is consistency of wallet rpcs with its own state of the chain
228 2016-09-08T15:33:22  <morcos> which seems somehow worse
229 2016-09-08T15:33:38  <BlueMatt> sipa: its very, very, strange to get a balance response that is valid as of the middle of a block, but will change before the end of the block
230 2016-09-08T15:33:43  <BlueMatt> I'm not sure that ever used to be pososible
231 2016-09-08T15:33:54  <morcos> it didn't it was protected by cs_main previously
232 2016-09-08T15:34:03  <sipa> what is the middle of a block?
233 2016-09-08T15:34:11  <sipa> how can the middle of a block be observed?
234 2016-09-08T15:34:31  <morcos> don't you just loop through the txs and call SyncWithWallets individually for each one
235 2016-09-08T15:34:34  <sipa> the only problem is using main's tip to determine wallet confirmations etc
236 2016-09-08T15:34:51  <sipa> the wallet should use its own tip idea to determine confirmations, instead of main's
237 2016-09-08T15:34:53  <morcos> why can't you ask for the balance in the rpc thread while you are in the middle of that loop
238 2016-09-08T15:35:33  <sipa> i think that's fine; it won't be counted as confirmed until it's done
239 2016-09-08T15:35:46  <sipa> (with the change to use the wallet's idea of the tip)
240 2016-09-08T15:36:11  <morcos> sipa: its hard to imagine an exact problem, but it just seems an odd change
241 2016-09-08T15:36:21  <morcos> whats the downside to locking the wallet around that loop?
242 2016-09-08T15:36:34  <sipa> ah, now i understand what you mean
243 2016-09-08T15:36:45  <sipa> to prevent a partially updated wallet
244 2016-09-08T15:37:01  <sipa> i think that's fine, yes
245 2016-09-08T15:37:09  *** Chris_Stewart_5 has quit IRC
246 2016-09-08T15:37:19  <sipa> but i think it can be avoided
247 2016-09-08T15:37:29  <morcos> now someone just has to remember these ideas we've discussed.  on that note, lunch time.  :)
248 2016-09-08T15:37:30  <BlueMatt> indeed, but we should :)
249 2016-09-08T15:37:51  <sipa> enjoy nourishment
250 2016-09-08T15:42:08  <cfields> back. all fixed? :)
251 2016-09-08T15:43:17  <cfields> morcos: lunch thought: i suppose we'll need the same fencing for individual transactions + wallet interactions
252 2016-09-08T15:44:38  *** spudowiar has joined #bitcoin-core-dev
253 2016-09-08T15:52:16  *** Chris_Stewart_5 has joined #bitcoin-core-dev
254 2016-09-08T15:54:40  *** spudowiar has quit IRC
255 2016-09-08T15:55:09  *** AaronvanW has quit IRC
256 2016-09-08T15:58:22  *** AaronvanW has joined #bitcoin-core-dev
257 2016-09-08T15:58:23  *** AaronvanW has joined #bitcoin-core-dev
258 2016-09-08T16:02:59  *** jannes has quit IRC
259 2016-09-08T16:03:18  *** Sosumi has quit IRC
260 2016-09-08T16:04:47  *** Chris_Stewart_5 has quit IRC
261 2016-09-08T16:13:29  *** fengling has joined #bitcoin-core-dev
262 2016-09-08T16:18:26  *** fengling has quit IRC
263 2016-09-08T16:34:28  *** spudowiar has joined #bitcoin-core-dev
264 2016-09-08T16:39:47  *** PaulCapestany has quit IRC
265 2016-09-08T16:40:25  *** PaulCapestany has joined #bitcoin-core-dev
266 2016-09-08T16:51:02  *** vega4 has quit IRC
267 2016-09-08T17:11:19  *** Chris_Stewart_5 has joined #bitcoin-core-dev
268 2016-09-08T17:15:03  *** fengling has joined #bitcoin-core-dev
269 2016-09-08T17:19:46  *** fengling has quit IRC
270 2016-09-08T17:22:12  *** BashCo has joined #bitcoin-core-dev
271 2016-09-08T17:27:42  *** jtimon has quit IRC
272 2016-09-08T17:29:41  *** laurentmt has joined #bitcoin-core-dev
273 2016-09-08T17:30:16  *** laurentmt has quit IRC
274 2016-09-08T18:00:53  <morcos> cfields: how do you mean?
275 2016-09-08T18:04:06  *** laurentmt has joined #bitcoin-core-dev
276 2016-09-08T18:08:02  *** jyap has quit IRC
277 2016-09-08T18:11:25  <cfields> morcos: now that i'm poking around, i can't come up with a realistic example
278 2016-09-08T18:12:33  <cfields> morcos: i was thinking about cases where the mempool holds a tx that the wallet isn't in sync with yet, where the mempool presence would lead the rpc caller to expect a wallet entry
279 2016-09-08T18:16:30  *** fengling has joined #bitcoin-core-dev
280 2016-09-08T18:19:53  *** jtimon has joined #bitcoin-core-dev
281 2016-09-08T18:21:26  *** fengling has quit IRC
282 2016-09-08T18:22:52  <sipa> i believe we've had a related discussion to this before
283 2016-09-08T18:23:21  <sipa> about whether or not external notifications should be sent before or after wallet updates
284 2016-09-08T18:23:32  <sipa> or something like that
285 2016-09-08T18:23:58  <sipa> because in some cases you want the update as soon as possible, but if you're going to use it to trigger a wallet request, the wallet should be updated first
286 2016-09-08T18:28:40  <wumpus> probably there should be an external notification for the core as well as the wallet
287 2016-09-08T18:29:41  <wumpus> if you want to be updated of the wallet processing some information, you'd listen for the wallet notification, if you want to know as soon as possible after the core rpocessed something you'd listen for the core notification
288 2016-09-08T18:30:12  <sipa> yup, agree
289 2016-09-08T18:30:31  <sipa> if the wallet is going to be asynchronously updated from the rest, it should have independent notifications too
290 2016-09-08T18:30:40  <wumpus> and in the hypothetical case with with multiple wallets you'd want to register for *that* wallet you're interested in
291 2016-09-08T18:30:41  <wumpus> right
292 2016-09-08T18:30:47  *** jyap has joined #bitcoin-core-dev
293 2016-09-08T18:30:47  *** jyap has joined #bitcoin-core-dev
294 2016-09-08T18:31:26  <wumpus> it would make a lot of sense for wallet processing to be asynchronous
295 2016-09-08T18:31:49  *** Giszmo has quit IRC
296 2016-09-08T18:31:49  <wumpus> and indeed, in that case there's no use in getting a signal after the wallet got the signal
297 2016-09-08T18:31:58  <wumpus> it still has to start processing
298 2016-09-08T18:32:24  <wumpus> what you need is a notification from the wallet itself that it processed
299 2016-09-08T18:32:55  *** laurentmt has quit IRC
300 2016-09-08T18:37:14  <jonasschnelli> we have one! -walletnotify *duck*
301 2016-09-08T18:39:07  <sipa> yes, but we'd also need a notification from the wallet that it processed a block
302 2016-09-08T18:40:22  *** Samdney has quit IRC
303 2016-09-08T18:41:05  <jonasschnelli> Yes. Maybe an signal short after when the wallet bestblock was updated.
304 2016-09-08T18:41:11  <wumpus> right
305 2016-09-08T18:42:08  <jonasschnelli> Though this works agains in the opposite direction then the possible plan of decoupling the wallet process.
306 2016-09-08T18:42:23  <jonasschnelli> (if the core side is consuming the event)
307 2016-09-08T18:42:56  <sipa> parse error
308 2016-09-08T18:44:24  <jonasschnelli> In case we want the wallet be capable of running without the validation (utxo-set/mempool) we should be carefully add signals (or lets say additional coupling between those elements)
309 2016-09-08T18:44:44  <wumpus> no, the wallet would be emitting the event
310 2016-09-08T18:45:00  <jtimon> I'll be afk during the meeting but will read later
311 2016-09-08T18:45:02  <wumpus> the core side doesn't subscribe to it or anything
312 2016-09-08T18:45:35  <wumpus> 'has wallet processed block' is only something that external clients may care about, maybe the GUI, but certainly not the core
313 2016-09-08T18:45:44  <kanzure> this seems to be someone from a law firm talking about "veil of decentralization" and fork types https://www.youtube.com/watch?v=7aV-k_6nZ8g&t=43m10s
314 2016-09-08T18:45:49  <kanzure> oh crud.
315 2016-09-08T18:45:50  <jonasschnelli> Ok. Right. This would make sense.
316 2016-09-08T18:45:52  <kanzure> please ignore.
317 2016-09-08T18:46:28  * kanzure wanders off
318 2016-09-08T18:47:43  *** arubi_ has quit IRC
319 2016-09-08T18:50:13  *** arubi has joined #bitcoin-core-dev
320 2016-09-08T18:50:37  *** pmienk_ has quit IRC
321 2016-09-08T18:51:25  <GitHub36> [bitcoin] jl2012 opened pull request #8685: Discourage P2WSH with too big script or stack (master...bigp2wsh) https://github.com/bitcoin/bitcoin/pull/8685
322 2016-09-08T18:54:09  *** pmienk has joined #bitcoin-core-dev
323 2016-09-08T18:55:52  *** spudowiar has quit IRC
324 2016-09-08T18:56:19  *** jcorgan has joined #bitcoin-core-dev
325 2016-09-08T18:58:06  *** Giszmo has joined #bitcoin-core-dev
326 2016-09-08T18:59:49  <wumpus> meeting time?
327 2016-09-08T18:59:54  <jonasschnelli> ack
328 2016-09-08T18:59:58  <wumpus> #startmeeting
329 2016-09-08T18:59:58  <lightningbot> Meeting started Thu Sep  8 18:59:58 2016 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
330 2016-09-08T18:59:58  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
331 2016-09-08T19:00:09  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
332 2016-09-08T19:00:14  <morcos> here
333 2016-09-08T19:00:19  <CodeShark> here
334 2016-09-08T19:00:24  <jeremyrubin> here
335 2016-09-08T19:00:28  <BlueMatt> topic: sing morcos happy birthday
336 2016-09-08T19:00:29  <petertodd> here
337 2016-09-08T19:00:30  <btcdrak> here
338 2016-09-08T19:00:31  <cfields> thanks, here
339 2016-09-08T19:00:33  <kanzure> here
340 2016-09-08T19:00:35  <petertodd> BlueMatt: ack!
341 2016-09-08T19:00:40  <wumpus> happy birthday morcos
342 2016-09-08T19:00:41  <jeremyrubin> leaked PII
343 2016-09-08T19:00:46  <btcdrak> gmaxwell you missed jl2012
344 2016-09-08T19:00:47  <kanzure> wumpus: no doxxing :)
345 2016-09-08T19:00:51  <jeremyrubin> Alex sing your ssn!
346 2016-09-08T19:00:52  <petertodd> kanzure: lol
347 2016-09-08T19:01:01  <michagogo> Happy birthday!
348 2016-09-08T19:01:02  <luke-jr> morcos: happy birthday https://www.youtube.com/watch?v=dQw4w9WgXcQ
349 2016-09-08T19:01:02  <morcos> thanks
350 2016-09-08T19:01:12  <sipa> morcos: congrats
351 2016-09-08T19:01:14  <jcorgan> here in spirit only
352 2016-09-08T19:01:24  <btcdrak> saved by DCMA filters "The uploader has not made this video available in your country."
353 2016-09-08T19:01:25  <jonasschnelli> morcos: hey! happy birthday.
354 2016-09-08T19:01:27  <petertodd> kanzure: happy birthday to anyone who considers themselves born on this date
355 2016-09-08T19:01:39  <kanzure> much better.
356 2016-09-08T19:01:43  <petertodd> btcdrak: the copyright on happy birthday got overturned :)
357 2016-09-08T19:01:53  <btcdrak> petertodd: click the link
358 2016-09-08T19:02:07  <wumpus> anyone with proposed topics?
359 2016-09-08T19:02:08  <petertodd> btcdrak: oh, that's a great song
360 2016-09-08T19:02:12  *** cryptapus has quit IRC
361 2016-09-08T19:02:27  <sipa> wumpus: just one: we have quite a queue of things for 0.13.1, and i'd like to encourage people to review
362 2016-09-08T19:02:31  <BlueMatt> real topic: segwit-cb bip
363 2016-09-08T19:02:32  <btcdrak> wumpus: birthday cake
364 2016-09-08T19:02:35  <kanzure> not a topic proposal, but i would like to eventually get a resolution on the after_failure weirdness
365 2016-09-08T19:03:01  <jonasschnelli> I just wanted to let you know that there will be two hack days on monday and tuesday 10th and 11th of October after the SB conference in Milan.
366 2016-09-08T19:03:17  <petertodd> jonasschnelli: I'll be there
367 2016-09-08T19:03:30  <gmaxwell> btcdrak: the list was just based the top participants in the past.
368 2016-09-08T19:03:30  <jonasschnelli> More info and registration will follow...
369 2016-09-08T19:03:33  <wumpus> yes last week there was an ACTION for "Support for compact blocks together with segwit" (#8393), there has been a bit of review activity in last days, what's the status there?
370 2016-09-08T19:04:05  <gmaxwell> wumpus: I've been doing some testing. There aren't many segwit transactions on testnet currently. I was going to call for people to create more once I got more testing setup.
371 2016-09-08T19:04:06  <sipa> BlueMatt: i haven't reviewed the changes for the bip you suggested - does it require any code changes to the bitcoin core pr?
372 2016-09-08T19:04:32  <wumpus> #info jonasschnelli: I just wanted to let you know that there will be two hack days on monday and tuesday 10th and 11th of October after the SB conference in Milan. More info and registration will follow...
373 2016-09-08T19:04:40  <BlueMatt> sipa: possibly, two options, though, one minor, one slightly more
374 2016-09-08T19:04:45  <btcdrak> maybe we can ask roasbeef to help tx generation there
375 2016-09-08T19:04:57  <wumpus> gmaxwell: more segwit transactions would help, yes :)
376 2016-09-08T19:05:30  <sipa> can we pick a topic?
377 2016-09-08T19:05:45  <wumpus> #link re: queue of things for 0.13.1, link is https://github.com/bitcoin/bitcoin/milestones/0.13.1
378 2016-09-08T19:06:15  <wumpus> #topic segwit-cb bip
379 2016-09-08T19:06:18  <btcdrak> jl2012: is everything you are working correctly tagged (or not) for 0.13.1?
380 2016-09-08T19:06:44  <BlueMatt> so, to quote github issue "The last commit changes the protocol entirely, adding a cmpctack message. This has the advantage that you could implement receiving of some version of compactlocks without implementing sending that encoding, as well as simplifying the protocol slightly (instead of having to check if the current protocol version is higher-priority according to your probably-compile-time list of supported version you know
381 2016-09-08T19:06:44  <BlueMatt> which version you're using directly from the ack message) at the expensee of complicating the implementation somewhat (now you have to add support for another message type and special-case version 1). The last commit is definitely not worth it if we dont anticipate adding more than one or so more versions, but might be worth it if we anticipate compact blocks version 4, 5, 6 at some point. I'll bring this up in the IRC meeting later
382 2016-09-08T19:06:50  <BlueMatt> today."
383 2016-09-08T19:07:11  <BlueMatt> essentially, the current proposal is that you annoucne the set of compact block versions you want at startup (BIP text says before you send any pong or other compact block messages)
384 2016-09-08T19:07:22  <BlueMatt> and each sendcmpct announce implies that you are willing to encode to those
385 2016-09-08T19:07:33  <jl2012> btcdrak: including those already tagged, I think 8685, 8654 and 8635 are also for 0.13.1
386 2016-09-08T19:07:41  <BlueMatt> and you send them in the priority of what you want to receive
387 2016-09-08T19:07:58  <BlueMatt> and the version you use to send is the first one you receive from the other side that you also sent
388 2016-09-08T19:08:01  <jl2012> at least we may consider to include in 0.13.1
389 2016-09-08T19:08:17  <BlueMatt> and you use the highest-priority one the other side also announced to decode
390 2016-09-08T19:08:33  <BlueMatt> (comment text from above link https://github.com/bitcoin/bips/pull/423#issuecomment-245629813 )
391 2016-09-08T19:08:53  <sipa> i'm not a fan of changing the code again after all testing, but i do agree it's a cleaner solution, and will make things easier for future extensions
392 2016-09-08T19:08:55  <BlueMatt> so this is obviously somewhat complicated
393 2016-09-08T19:09:16  <BlueMatt> and the solution would be to introduce a cmpctack which you use to pick the one you want to encode to
394 2016-09-08T19:09:34  <sipa> cmpctack containing the version you picked?
395 2016-09-08T19:09:40  <wumpus> jl2012: (tagged. number of pulls for 0.13.1 does seem to be getting a bit out of hand)
396 2016-09-08T19:09:57  <gmaxwell> can we have some discussion about this outside of the meeting, I want to ask some questions but I think it'll be a design rathole right now. :)
397 2016-09-08T19:10:02  <BlueMatt> simplifying the which-do-i-use-to-decode code from "for each sendcmpct msg received, is this higher priority than the previously-highest-prirotiy one" to "the one I saw in a sendcmpct"
398 2016-09-08T19:10:14  <BlueMatt> in practice the proposed cmpctack is way more code, but a bit simpler
399 2016-09-08T19:10:21  <sipa> BlueMatt: ok, i'll review and adapt the pr
400 2016-09-08T19:10:32  <BlueMatt> s/""the one I saw in a sendcmpct"/""the one I saw in a cmpctack"/g
401 2016-09-08T19:10:32  <sipa> gmaxwell: ok
402 2016-09-08T19:10:38  <BlueMatt> I, personally, dont really like the cmpctack idea
403 2016-09-08T19:10:52  <sipa> so what alternative do you propose?
404 2016-09-08T19:10:52  <sdaftuar> i do like the cmpctack idea!
405 2016-09-08T19:10:52  <BlueMatt> certainly if we plan on having lots of versions, it is simpler protocol-wise
406 2016-09-08T19:10:56  <jl2012> wumpus: most of mine are pretty trivial. I am no able to do more than that anyway
407 2016-09-08T19:11:15  <BlueMatt> if we only ever have version 1 and 2 and maybe like a 3, then the previous proposal seems perfectly ok to me
408 2016-09-08T19:11:27  <morcos> My viewpoint is that we suffer a history of technical debt, and we have a chance now while compact blocks are new to kind of clean up the protocol messages with a bit less fuss
409 2016-09-08T19:11:36  <sipa> morcos: agree
410 2016-09-08T19:11:38  <BlueMatt> sipa: either way I'm proposing to switch the priority order to first-is-highest from last-is-highest
411 2016-09-08T19:11:39  <wumpus> jl2012: agreed
412 2016-09-08T19:11:42  <morcos> so we shoudl take the added changes now to be happier later with a better design
413 2016-09-08T19:11:51  <luke-jr> would it ever make sense to support a per-block encoding? for example, if nodes at some point want to pass blocks along as-is from peer A to other peers when possible
414 2016-09-08T19:12:01  <BlueMatt> note that we have to introduce a backwards compatibility hack if we do cmpctack
415 2016-09-08T19:12:17  <gmaxwell> I think it's fine to clean things up. But at some point the correct 'upgrade' is to just introduce a seperate mechenism sendcmpt2 and drop the old one, rather than extending.
416 2016-09-08T19:12:25  <sipa> BlueMatt: just say that if no ack is ever sent, it is implicitly for v1
417 2016-09-08T19:12:25  <luke-jr> BlueMatt: do we? old CBs will die with segwit anyway..
418 2016-09-08T19:13:38  <BlueMatt> sipa: this implies you have to announce sendcmpct version 1
419 2016-09-08T19:13:39  <gmaxwell> past some point trying to create a forever design just guarentees technical debt of a different kind. :)
420 2016-09-08T19:13:48  <BlueMatt> which the proposal for creating a cmpctack would not do
421 2016-09-08T19:14:03  <morcos> we should maybe do as gmaxwell said and discuss after meeting, but i don't think we actually need a hack, you just need to still tell 0.13 nodes that you support v1 and they only understand that by sending them a sendcmpct 1
422 2016-09-08T19:14:08  <morcos> but it doesn't hurt to send that to everyone
423 2016-09-08T19:14:12  <BlueMatt> alternatively: compact block version 3 can be called something other than compact blocks
424 2016-09-08T19:14:15  <BlueMatt> then you can do whatever :P
425 2016-09-08T19:14:19  <sipa> BlueMatt: hahaha
426 2016-09-08T19:14:27  <sipa> i could live with that too.
427 2016-09-08T19:14:29  <gmaxwell> BlueMatt: yea, thats what I was saying, effectively.
428 2016-09-08T19:14:59  <gmaxwell> besides the general framework here has limitations, further latency optinization should basically be a non-goal, because the fiber approach is vastly better in that respect.
429 2016-09-08T19:15:12  <BlueMatt> anyway, its taken 15 minutes to explain what the issues are, so maybe decide later, or let other topics go ahead first
430 2016-09-08T19:15:39  <wumpus> I don't think any other topics have been (seriously) proposed
431 2016-09-08T19:16:15  <wumpus> so go ahead
432 2016-09-08T19:16:29  <jl2012> I got to go. See you
433 2016-09-08T19:16:38  <morcos> proposed topic: picking a segwit rollout date and announcing this in a wider format
434 2016-09-08T19:16:38  <wumpus> see you later jl2012
435 2016-09-08T19:16:47  <sipa> jl2012: good night
436 2016-09-08T19:16:52  <sipa> BlueMatt: how about just sending sendcmpct2 for v2 :)
437 2016-09-08T19:17:00  * sipa hides
438 2016-09-08T19:17:16  <gmaxwell> morcos: I think the blocker there was basically having all the things merged in 0.13 branch that we believe would be needed on our end.
439 2016-09-08T19:17:18  <BlueMatt> alternatively: version negotiation protocol examples are at https://github.com/bitcoin/bips/blob/833dea41c4c757087ef4c35e3b19259ba2f80128/bip-0152.mediawiki#Sample_Version_Implementation and https://github.com/bitcoin/bips/blob/0d9b12cf285762e8ff661fe17e3261d014af1581/bip-0152.mediawiki#Sample_Version_Implementation
440 2016-09-08T19:17:19  <cfields> topic suggestion: rpc sync assumptions
441 2016-09-08T19:17:36  <btcdrak> need review of these backport #8679, should be simple enough
442 2016-09-08T19:18:02  *** fengling has joined #bitcoin-core-dev
443 2016-09-08T19:18:07  <BlueMatt> though I think the second misses the fact that you have to use sendcmpct instead of cmpctack if its version 1
444 2016-09-08T19:18:21  <instagibbs_> oh hi. znc bouncer is broke or something.
445 2016-09-08T19:18:34  <morcos> gmaxwell: yeah i've been a bit out of touch, and i can see that makes sense..   but i also think it would be nice to give as much warning to the community as possible as to when we are proposing to actually release this
446 2016-09-08T19:19:08  <gmaxwell> morcos: yes, a message that says "This will happen soon, our side waiting for X" to give people a chance to raise any concerns would be reasonable, I think.
447 2016-09-08T19:19:12  <instagibbs_>  /release/activate/ ?
448 2016-09-08T19:19:21  <morcos> instagibbs_: yes, both
449 2016-09-08T19:19:25  <CodeShark> we've been giving that message for several weeks already ;)
450 2016-09-08T19:19:30  <wumpus> #topic picking a segwit rollout date and announcing this in a wider format
451 2016-09-08T19:19:56  <instagibbs_> I've been pointing out the remaining milestone list, but it's a bit opaque for people who aren't actively reviewing stuff
452 2016-09-08T19:19:58  <gmaxwell> I went around to soem forks and asked for what their scheduling looked like and the response I got was basically 'after it's deployed in the network'
453 2016-09-08T19:20:05  <sipa> we can't propose a rollout date before knowing when we can have 0.13.1 out, and there are quite a few things to work out for that
454 2016-09-08T19:20:24  <morcos> sipa: yeah i suppose i agree
455 2016-09-08T19:20:26  <CodeShark> I'd rather not pile on additional scheduling issues externally unless we're confident
456 2016-09-08T19:20:56  <achow101> is a 0.12.2 backport still happening?
457 2016-09-08T19:21:01  <instagibbs_> Would the amount of lead time differ once we've merged all remaining issuess?
458 2016-09-08T19:21:07  <wumpus> yes it seems 0.13.1 is a lot more work than expected
459 2016-09-08T19:21:17  <wumpus> achow101: I don't think so
460 2016-09-08T19:21:51  <btcdrak> achow101: segwit needs compact block relay, so very unlikely.
461 2016-09-08T19:22:15  <achow101> ok
462 2016-09-08T19:22:17  <luke-jr> "needs"?
463 2016-09-08T19:22:22  <wumpus> achow101: getting this into 0.13 in the first place and assuring it is correct is a lot of work, I doubt anyone can pile up the extra work for 0.12
464 2016-09-08T19:22:26  *** fengling has quit IRC
465 2016-09-08T19:22:51  <BlueMatt> luke-jr: yea, what? segwit doesnt NEED it
466 2016-09-08T19:23:00  <instagibbs_> And no one seems to be demanding it, more importantly
467 2016-09-08T19:23:05  <BlueMatt> just might be painful if you dont have it
468 2016-09-08T19:23:15  <btcdrak> unless you are happy with bigger blocks being relayed without it...
469 2016-09-08T19:23:24  <btcdrak> anyway. weeds.
470 2016-09-08T19:23:33  <sipa> yes, weeds
471 2016-09-08T19:24:07  <instagibbs_> Which PRs need the most attention at this point
472 2016-09-08T19:24:07  <gmaxwell> achow101: no we pretty much decided to not do a 0.12. backport over a month ago.
473 2016-09-08T19:24:10  <wumpus> weeds?
474 2016-09-08T19:24:18  <sipa> wumpus: "we're getting into the weeds"
475 2016-09-08T19:24:24  <wumpus> ohh
476 2016-09-08T19:24:34  <gmaxwell> achow101: not worth the risk and resource investment, and no one was jumping to do it. From measurements and feedback we've found that virtually no one uses backport releases.
477 2016-09-08T19:24:35  <instagibbs_> mfw pieter is explaining English idioms
478 2016-09-08T19:24:42  <CodeShark> in the netherlands that might have a different meaning ;)
479 2016-09-08T19:24:45  <jonasschnelli> heh
480 2016-09-08T19:24:50  <wumpus> yes, 0.12 just isn't a very important topic right now, let's focus on moving forward
481 2016-09-08T19:24:56  <wumpus> CodeShark: yes :)
482 2016-09-08T19:25:20  <luke-jr> also, if someone really needs 0.12, they can put it behind a 0.13.1 node
483 2016-09-08T19:25:21  <wumpus> next topic?
484 2016-09-08T19:25:47  <wumpus> #topic rpc sync assumptions
485 2016-09-08T19:25:57  <gmaxwell> luke-jr: precisely, and any sw backport would create the disruption people were trying to avoid in staying with an older version.
486 2016-09-08T19:26:29  <gmaxwell> wumpus: I don't understand why people thought it was a problem that getbalance could return a "mid block" response.
487 2016-09-08T19:26:32  <morcos> just to be clear, the change to reduce cs_main locks is only for 0.14, so we don't have to worry about rpc sync assumptions for 0.13.1 right?
488 2016-09-08T19:26:35  <cfields> so, marcos pointed out that the reason that the rpc tests are failing and fix is needed is that we broke some timing assumptions by optimizing some stuff
489 2016-09-08T19:27:10  <cfields> so the question is: are those assumptions ok to break (just fix the tests), or are they part of the api?
490 2016-09-08T19:27:28  <cfields> wow, re-reading that, that's incredibly vague :)
491 2016-09-08T19:27:33  <sipa> morcos: yes
492 2016-09-08T19:27:36  <jonasschnelli> Can you make a clear example?
493 2016-09-08T19:27:46  <gmaxwell> I think the question should be what do we think the best behavior is. And if the best behavior changes the API, we should do it and document the change.
494 2016-09-08T19:27:57  <jeremyrubin> WalletSync expected to occur in test
495 2016-09-08T19:28:02  <jonasschnelli> Do we assume the wallet has processed a transaction after getting informed of a blockupdate?
496 2016-09-08T19:28:04  <jeremyrubin> *under the lock
497 2016-09-08T19:28:08  <luke-jr> does "mid block" include "numbers are being changed, so the value is unlike either the previous OR the next correct value"?
498 2016-09-08T19:28:09  <wumpus> gmaxwell: I don't know either, there's no guarantee that blocks are atomically processed by the wallet
499 2016-09-08T19:28:25  <sipa> we can make it atomic, as pointed out by morcos earlier
500 2016-09-08T19:28:26  <jonasschnelli> s/blockupdate/tipupdate
501 2016-09-08T19:28:32  <wumpus> but is it necessary?
502 2016-09-08T19:28:32  <morcos> luke-jr: it reflects the full effect of some of the txs in the block but not all
503 2016-09-08T19:28:48  <gmaxwell> my understanding is that the tests are doing things like handing a node a block, then instantly checking the wallet, and expecting it to be fully consistent with the block right away. Is this understanding correct?
504 2016-09-08T19:28:55  <sipa> my alternative is that we make the wallet aware of the current tip it knows about, and we let confirmations be computed based on that
505 2016-09-08T19:29:07  <sipa> that means that mid-block update you can see unconfirmed transactions
506 2016-09-08T19:29:12  <BlueMatt> jonasschnelli: ex: you do a getblockheight rpc call - under previous versions if you then do a getbalance that balance is up to date to that block, this is no longer true
507 2016-09-08T19:29:19  <wumpus> the tests need proper synchronization commands, some of those need to be implemented, I don't think we need to change the whole API for that
508 2016-09-08T19:29:24  <morcos> wumpus: well before we get to the atomic block question, there is the question of whether if you wait for the blockchain to be synced to some height, and then query the wallet whether you are getting wallet values of at least that height (mid-block or not)
509 2016-09-08T19:29:25  <cfields> gmaxwell: correct. as soon as the height/hash is reflected, they assume the wallet is synced with that height/hash
510 2016-09-08T19:29:32  <wumpus> (besides adding those syncronization commands)
511 2016-09-08T19:29:54  <jonasschnelli> Can we not just have the wallets bestblock in getwalletinfo and use it for sync with getchaintips?
512 2016-09-08T19:29:54  <BlueMatt> sipa: I think mid-block updates is a blatant violation of the "principal of least surprise"
513 2016-09-08T19:30:00  <gmaxwell> morcos: okay, I think sipa's suggestion would address that esp with coupled with a way of asking for the wallet's tip.
514 2016-09-08T19:30:10  <wumpus> morcos: well it would make sense for the wallet to track where it is, synchronization-wise, absolutely
515 2016-09-08T19:30:22  <gmaxwell> BlueMatt: uh then we need to remove unconfirmed transactions too?!
516 2016-09-08T19:30:24  <wumpus> morcos:  and for thtat info to be available externally
517 2016-09-08T19:30:41  <luke-jr> showing a best-wallet-block would make the whole mid-block state even more broken IMO (what best-block will it show mid-state?)
518 2016-09-08T19:30:44  <sipa> BlueMatt: there can always be unconfirmed transactions in the wallet
519 2016-09-08T19:30:47  <BlueMatt> and, further, if wallet is allowed to return something that is not up to date with the start of the call, this changes literally our entire api....so now anyone using the rpc has to go re-audit all of their uses of it???
520 2016-09-08T19:30:53  <sipa> luke-jr: the previous one
521 2016-09-08T19:30:56  <BlueMatt> sipa: huh? I mean things mid-block...am I missing what you said?
522 2016-09-08T19:31:01  <gmaxwell> luke-jr: it will update its state when its done processing updates, of course.
523 2016-09-08T19:31:03  <morcos> fixing the mid-block thing seems pretty trivial to me, why wouldnt' we just do that
524 2016-09-08T19:31:07  <jonasschnelli> luke-jr: the bestblock should only be updated after fully processing? Right?
525 2016-09-08T19:31:23  <BlueMatt> sipa: i was under the impression you indicated that you would see txn which come later in that block as unconfirmed, while txn earlier in the block are marked confirmed
526 2016-09-08T19:31:28  <gmaxwell> Why are people regarding mid block as a bug?!?
527 2016-09-08T19:31:31  <sipa> BlueMatt: no
528 2016-09-08T19:31:44  <gmaxwell> at _any_ time a transaction can show up and appear in the wallet, unrelated to any blocks.
529 2016-09-08T19:31:51  <morcos> gmaxwell: b/c its not a feature?
530 2016-09-08T19:31:57  <sipa> BlueMatt: mid-update you would see the transactions of the new block as unconfirmed
531 2016-09-08T19:32:00  <luke-jr> that seems the most obvious behaviour, but if you look at bestblock+balance, I would think them a pair, yet balance might be bestblock+partOfNextBlock in reality?
532 2016-09-08T19:32:02  <morcos> i mean does anyone want that
533 2016-09-08T19:32:03  <BlueMatt> ahh, yes, ok, so i was just confused....txn which appear in wallet before the block has processed seems reasonable
534 2016-09-08T19:32:07  <gmaxwell> Unless you want to remove unconfirmed transactions that will always happen.
535 2016-09-08T19:32:27  <sipa> i also don't think there is any problem with grabbing a wallet lock during the update
536 2016-09-08T19:32:36  <wumpus> so if changing the balance mid-block is regarded as a bug, that should apply to all other state too: the transaction list, for example. It should hold all updates until it processed the entire block
537 2016-09-08T19:32:36  <sipa> which would prevent seeing a mid-update state
538 2016-09-08T19:32:42  <sipa> i'm just questioning if it is necessary
539 2016-09-08T19:32:56  <gmaxwell> it seems strange to me to hurt concurrency to protect a property that we don't actually have (due to unconfirmed transactions)
540 2016-09-08T19:33:01  <wumpus> I don't think using the lock for that is a good thing
541 2016-09-08T19:33:21  <wumpus> agree with gmaxwell
542 2016-09-08T19:33:23  <jonasschnelli> From the wallet perspective processing doesn't matter, you just want to see confirmations... can slowly appear on a tx list IMO
543 2016-09-08T19:33:23  <cfields> wumpus: right, that made me cringe. That seems like a major layer violation
544 2016-09-08T19:33:36  <morcos> wumpus: gmaxwell: but can you explain what concurrency we are hurting?
545 2016-09-08T19:33:44  <morcos> i'm not suggesting we put cs_main covering those again
546 2016-09-08T19:33:49  <wumpus> morcos: you have to hold the lock *all* the time while processing the block
547 2016-09-08T19:33:53  <BlueMatt> gmaxwell: if it were listing your *confirmed* balance as mid-block, then it would be an issue
548 2016-09-08T19:33:54  <sipa> wumpus: no
549 2016-09-08T19:34:00  <gmaxwell> Wallet processing shouldn't stall when the node is processing a block, at least not any more than strictly necessary.
550 2016-09-08T19:34:01  <BlueMatt> which I believe is what people are worried about
551 2016-09-08T19:34:07  <sipa> wumpus: the proposal is to grab cs_wallet during the wallet updating for the block
552 2016-09-08T19:34:10  * luke-jr wonders if there's a good way to use listsinceblock
553 2016-09-08T19:34:17  <sipa> wumpus: which happens in its entirety after main processes the block
554 2016-09-08T19:34:19  <morcos> i'm just suggesting we use cs_wallet or some other lock that prevents a wallet specific call from returning until you are no longer in the middle of a loop that calls SyncWithWallets (on a given block)
555 2016-09-08T19:34:36  <sipa> wumpus: cs_main isn't even held at the point
556 2016-09-08T19:34:36  <wumpus> sipa: I know, but you'd still be holding the wallet lock longer than necessary
557 2016-09-08T19:34:45  <BlueMatt> to be fair, all of this should move onto a background thread in the future anyway
558 2016-09-08T19:34:50  <gmaxwell> morcos: so now wallet calls will stall block processing? (when the block processing waits to take that lock?)
559 2016-09-08T19:34:50  <wumpus> BlueMatt: agreed on that
560 2016-09-08T19:34:59  <sipa> yes, and it may be harder to maining if we parallellize/asynchronize things more
561 2016-09-08T19:35:02  <wumpus> wallet block processing should be async at some point
562 2016-09-08T19:35:16  <BlueMatt> so this would be a temporary fix that we should consider something which will happen on a background thread...lets not get too focused on blocking block processing with it
563 2016-09-08T19:35:22  <sipa> morcos: do you think there is a problem with showing the transactions from the being-processed-block as unconfirmed during the update?
564 2016-09-08T19:35:24  <morcos> gmaxwell: hmmm, yes i suppose while thats in the main thread thats a bad idea
565 2016-09-08T19:35:36  <jonasschnelli> IMO the wallet should process a copy of the block on its own, own thread
566 2016-09-08T19:35:43  <sipa> jonasschnelli: yes yes
567 2016-09-08T19:35:47  <CodeShark> ^]
568 2016-09-08T19:35:49  <wumpus> sipa: probably it will already be in the wallet as unconfirmed anyhow
569 2016-09-08T19:35:50  <sipa> jonasschnelli: i think everyone agrees on that
570 2016-09-08T19:35:52  <gmaxwell> BlueMatt: I'm fine with temporary fixes, I'm just confused as to why anything needs to be fixed here except some unrealistic expectations in the tests.
571 2016-09-08T19:36:05  <sipa> gmaxwell: i believe the current situation is broken
572 2016-09-08T19:36:20  <gmaxwell> sipa: great. I'd like to understand why.
573 2016-09-08T19:36:25  <sipa> mid-update you can see half of the block reflected as confirmed transactions, and miss other transactions from that same block
574 2016-09-08T19:36:32  <BlueMatt> gmaxwell: two issues: getbalance, by default, shows your *confirmed* balance, so you expect that to be consistent...right now it is not clear (I maybe wrong) that it might not show *half* of your actually confirmed balance
575 2016-09-08T19:36:51  <morcos> sipa: i think i need to go look more carefully at what SyncWithWallets does with txs from the block, but for instance, coudl you end up removing a conflicted tx, then showing a balance, without having addd the replacement tx yet.  i think yes, and i think thats would be confusing
576 2016-09-08T19:36:53  <gmaxwell> okay, I agree showing confirmed for some and not others is odd. Showing some and not others, however I think is fine, and consistent with the normal behavior.
577 2016-09-08T19:37:05  <cfields> let's not focus on the details here. The question at hand (mine, anyway), is whether the blocking behavior from 0.13 is considered part of the api, or if it's ok to deviate. If the answer is the latter, we can just fix the tests and move on.
578 2016-09-08T19:37:19  <sipa> cfields: blocking what behaviour?
579 2016-09-08T19:37:27  <sipa> 0.13 does not have this change
580 2016-09-08T19:37:52  <sipa> cfields: i think there is something that needs to be fixed in master, that is deeper than fixing up tests
581 2016-09-08T19:37:56  <sipa> cfields: but it may not be much
582 2016-09-08T19:38:11  <cfields> sipa: right, 0.13 doesn't give up the lock
583 2016-09-08T19:38:21  <sipa> indeed, 0.13 is totally fine
584 2016-09-08T19:38:27  <BlueMatt> gmaxwell: second, we are making an API change here, which it seems to me is probably going to break clients: previously if you called getblockheigh, and then getbalance, getbalance will be up-to-date at least as of that block...this is no longer true. I can absolutely see clients having assumed this
585 2016-09-08T19:38:29  <wumpus> "okay, I agree showing confirmed for some and not others is odd" absolutely. This would break the assumption that curtip-block which tx was confirmed in is the number of confirmations
586 2016-09-08T19:38:37  <gmaxwell> cfields: That the wallet blocks shouldn't be part of the api. Certian consistency properties might reasonably be.  I'm actually dubious that seeing confirmations incrementally is actually a problem.
587 2016-09-08T19:38:50  *** instagibbs_ has quit IRC
588 2016-09-08T19:38:57  <gmaxwell> BlueMatt: no, thats not really true either, since the chain can reorg between those two calls.
589 2016-09-08T19:39:07  <cfields> sipa: right. and now master has changed that behavior. So if the behavior is considered to be part of the api, we need to revert it
590 2016-09-08T19:39:10  <jonasschnelli> Why can't we not just use SyncWithWallets or mempool and add a SyncBlockWithWallet(blockcopy) for added and removed tips? Process it in a wallet-thread (similar to periodic flushes) and use cs_wallet there?
591 2016-09-08T19:39:11  <morcos> ok, how about i write up an issue
592 2016-09-08T19:39:20  <cfields> (I don't believe that and I'd -1 it. Just clarifying)
593 2016-09-08T19:39:43  <morcos> i don't think we should revert the change
594 2016-09-08T19:39:58  <wumpus> we shouldn't revert anything that prevents future paralellization/concurrency
595 2016-09-08T19:40:02  *** assder has quit IRC
596 2016-09-08T19:40:12  <sipa> i think the step was a step in the right direction, and we should continue it
597 2016-09-08T19:40:22  <sipa> but we have time in 0.14 to figure out exactly how
598 2016-09-08T19:40:23  <wumpus> if there is a need to 'force everything to wait for thewallet' that's a big -1 from me too
599 2016-09-08T19:40:25  <morcos> i think we should make it so that the existing rpc calls returns omething that make sense.  two issues 1) once you've waited for a certain height, that once you ask for balance you get a balance of at least that height 2) whether mid-block updates are ok
600 2016-09-08T19:40:29  <gmaxwell> how does this concurrency interact with the wallet's mempool interaction. The wallet cares if tx are in mempool or not, will a wallet look unconfirmed and fallen out of the mempool briefly while it's confirming?
601 2016-09-08T19:40:32  <morcos> 1) needs fixing, 2) needs more investigation
602 2016-09-08T19:40:55  <cfields> agreed. ok, that answers my question.
603 2016-09-08T19:41:03  <morcos> 1) can be fixed with cfields existing code from 8680 without harming any concurrency
604 2016-09-08T19:41:04  <sipa> morcos: 1) is easily fixed by reporting the wallet height rather than the core height
605 2016-09-08T19:41:10  <jonasschnelli> morcos: isn't 1) and 2) solvable from the wallet side?
606 2016-09-08T19:41:17  <wumpus> wallet height and core height are different things
607 2016-09-08T19:41:25  <sdaftuar> sipa: yes, but that would be api-breaking right?
608 2016-09-08T19:41:29  <wumpus> it has always been possible to confound these, but that has to change
609 2016-09-08T19:41:32  <sipa> sdaftuar: i believe not
610 2016-09-08T19:41:34  <sdaftuar> i think that's the right idea though
611 2016-09-08T19:41:57  <gmaxwell> Well it means that someone might need to look in a different place for the wallet height, no?
612 2016-09-08T19:42:04  <sipa> sdaftuar: if we make 0.14 report the wallet height,  i believe it will look equivalent to 0.13
613 2016-09-08T19:42:09  <wumpus> gmaxwell: yes, it'd need to be a wallet RPC
614 2016-09-08T19:42:24  <morcos> sipa: the issue is people probably already use getblockcount and then ask for balances
615 2016-09-08T19:42:32  <wumpus> either on getwalletinfo or getwalletblockcount
616 2016-09-08T19:42:37  <morcos> but they do that in their code
617 2016-09-08T19:42:38  <sipa> morcos: so, make getblockcount by default report the wallet height
618 2016-09-08T19:42:46  <wumpus> bah
619 2016-09-08T19:42:48  <morcos> that seems like a crazy change
620 2016-09-08T19:42:53  <gmaxwell> morcos: thats okay, we would release note that in 0.14. If it's the right behavior to change we shouldn't hesitate to do so here.
621 2016-09-08T19:42:58  <sipa> why? it's exactly what we've been doing all the time
622 2016-09-08T19:43:02  <wumpus> change getblockcount to a wallet call?!
623 2016-09-08T19:43:10  <jonasschnelli> no
624 2016-09-08T19:43:13  <sdaftuar> that means you need wallet support to use that rpc?
625 2016-09-08T19:43:15  <CodeShark> bad idea
626 2016-09-08T19:43:15  <morcos> i think we're getting a bit too worked up
627 2016-09-08T19:43:16  <sdaftuar> blech
628 2016-09-08T19:43:16  <wumpus> what if there is no wallet built in?
629 2016-09-08T19:43:25  <sipa> sigh
630 2016-09-08T19:43:26  <jonasschnelli> no blockcount!
631 2016-09-08T19:43:26  <gmaxwell> that woudl be kind of odd, but it's 99% of the time used as a wallet call, ... and we have getblockchaininfo....
632 2016-09-08T19:43:29  <wumpus> what if there are multiple wallets? ask them all, return the max value?
633 2016-09-08T19:43:38  <BlueMatt> gmaxwell: certain people have reorg handling code, this api change will not trigger their reorg handling code and will still break clients!
634 2016-09-08T19:43:42  <sipa> so deprepcate the RPC
635 2016-09-08T19:43:49  <jonasschnelli> Just give out the bestblockhash in getwalletinfo
636 2016-09-08T19:43:49  <BlueMatt> it is absolutely not unreasonable for this change to break rpc clients
637 2016-09-08T19:43:50  <sipa> and introduce a wallet-specific one and main-specific one
638 2016-09-08T19:43:53  <morcos> its easy enough to make wallet balance calls wait on their own until either the wallet reports a height that matches chainactive height or using cfields mechanism, that slows nothing other than that wallet call and solves 1
639 2016-09-08T19:44:00  <cfields> why not just have getblockcount block until the block is finished processing (all signals, not wallet specific)? New apis can be async
640 2016-09-08T19:44:08  <wumpus> yes, there needs to be a wallet-specific one
641 2016-09-08T19:44:12  <sipa> cfields: that's just delaying the problem
642 2016-09-08T19:44:20  <BlueMatt> what morcos said
643 2016-09-08T19:44:22  <gmaxwell> BlueMatt: look, the API is not, should not, and cannot be a suicide pact. We're talking about an change in a major version, and one that would only require minor changes. _WE CAN CHANGE THE API_.
644 2016-09-08T19:44:23  <sipa> cfields: because that's no longer possible if the wallet works asynchronously
645 2016-09-08T19:44:25  <BlueMatt> just block wallet calls until they are caught up
646 2016-09-08T19:44:44  <gmaxwell> Especially in a minor way like "get your height for purpose X this way instead of that"
647 2016-09-08T19:44:54  <wumpus> wallet processing blocking wallet calls is OK
648 2016-09-08T19:44:55  <BlueMatt> there is a simple solution to this that doesnt require all the users audit their codebase
649 2016-09-08T19:44:58  <BlueMatt> that isnt even a big deal
650 2016-09-08T19:44:58  <wumpus> wallet processing blocking core calls is not
651 2016-09-08T19:45:06  <BlueMatt> but you're arguing we change the api because its simpler?
652 2016-09-08T19:45:22  <BlueMatt> just block wallet rpc calls until its caught up at the start of the cs_wallet lock
653 2016-09-08T19:45:35  <BlueMatt> yes, wumpus, dont think anyone is arguing for that
654 2016-09-08T19:45:36  <gmaxwell> I haven't heard anything suggested that doesn't require having the wallet and block processing block each other.
655 2016-09-08T19:45:43  <BlueMatt> noooo
656 2016-09-08T19:45:45  <BlueMatt> wait, wut?
657 2016-09-08T19:45:45  <morcos> wumpus: yes, see cory's code in 8680, can easily adopt all wallet calls to use that (or ask the wallet for its height but then they might have to poll) and weight until it hits what chainactive was at the start of the call
658 2016-09-08T19:46:00  <BlueMatt> the proposal is that rpc calls might block until the wallet has caught up to where main chainstate is
659 2016-09-08T19:46:03  <morcos> gmaxwell: yeah i think you're misunderstanding
660 2016-09-08T19:46:10  <sipa> morcos: yup... but at some point we'd want to get rid of that too, i think
661 2016-09-08T19:46:13  <BlueMatt> the wallet processing can still be in a background thread, or on the main thread, or whatever
662 2016-09-08T19:46:17  <wumpus> morcos: yes, that makes sense
663 2016-09-08T19:46:19  <gmaxwell> and I think it's insane to degrade concurrency for an obscure property that anyone who wants can retain by using an appropritate call to ask where the wallet is vs where the blockchain is.
664 2016-09-08T19:46:20  <BlueMatt> it just has to catch up before the rpc will return
665 2016-09-08T19:46:37  <BlueMatt> gmaxwell: we arent degregading concurrency except that rpc calls might block, afaict
666 2016-09-08T19:46:46  <BlueMatt> and then we're returning more up-to-date info anyway
667 2016-09-08T19:46:50  <BlueMatt> so thats not even so bad
668 2016-09-08T19:46:57  <morcos> might block but would still return before they would have in 0.13!!!
669 2016-09-08T19:47:01  <cfields> BlueMatt: and it only blocks as long sa it would've before
670 2016-09-08T19:47:08  <BlueMatt> true
671 2016-09-08T19:47:38  <gmaxwell> what was suggested above was the block processing taking the wallet lock; which would bidirectionally block each other.
672 2016-09-08T19:47:48  <sipa> gmaxwell: no
673 2016-09-08T19:47:52  <morcos> gmaxwell: that was a proposal to solve problem 2 (the mid block)
674 2016-09-08T19:47:54  <BlueMatt> that was to a different issue
675 2016-09-08T19:47:57  <BlueMatt> there are two issues
676 2016-09-08T19:48:01  <BlueMatt> well, potential issues
677 2016-09-08T19:48:12  <sipa> the proposal was that _the wallet_ would grab _the wallet lock_ while it was updating its state for a new block
678 2016-09-08T19:48:25  <morcos> that problem needs more investigation to determine a) whether it needs solving and b) whether there is a simple solution.  i agree with your objection to my first suggested solution
679 2016-09-08T19:49:15  <morcos> sipa: yes but thats only a good idea when thats not running in teh same thread as block processing in the middle of block processing, otherwise some other independent wallet call holds up block processing
680 2016-09-08T19:49:17  <sipa> are there other topics? i don't think we need to figure this out completely right now
681 2016-09-08T19:49:20  <gmaxwell> morcos: Sounds fine to me.
682 2016-09-08T19:49:21  <wumpus> I think it's better to take this outside the meeting. Any other topics?
683 2016-09-08T19:49:28  <morcos> wumpus: ha , too late!
684 2016-09-08T19:49:48  <wumpus> morcos: heh same idea
685 2016-09-08T19:50:00  <cfields> this can be handled with signals btw, no need to grab actual locks. StartProcessing() -> block rpc -> FinishedProcessing() unblock
686 2016-09-08T19:50:36  <cfields> yes, later is fine. That got more heated than I expected :)
687 2016-09-08T19:50:59  <cfields> though, we still have the question of what to do about the tests.
688 2016-09-08T19:51:13  <sipa> morcos: yes, i'm not saying that's what we should or shouldn't do... just clarifying what the idea was
689 2016-09-08T19:51:14  <morcos> does 8680 fix the tests or not really?
690 2016-09-08T19:51:26  <wumpus> cfields: I think your pull is fine for that, as a temp fix at least
691 2016-09-08T19:51:28  <morcos> i think 8680 seems reasonable to me
692 2016-09-08T19:51:33  <wumpus> right
693 2016-09-08T19:51:39  <sipa> let's merge 8680 to fix the annoyance with the test
694 2016-09-08T19:51:42  <cfields> morcos: i believe so, but not 100% because of the nature
695 2016-09-08T19:51:47  <sipa> but open a tracker issue to reconsider
696 2016-09-08T19:51:52  <cfields> (i'm not 100%, sorry)
697 2016-09-08T19:51:53  <wumpus> #action merge  8680 to fix the annoyance with the test
698 2016-09-08T19:51:54  <michagogo> <i>8m warning</i>
699 2016-09-08T19:52:14  <wumpus> michagogo: looks like we're out of topics sooner than out of time
700 2016-09-08T19:52:19  <cfields> ok, I can slim that down to only the rpc used by the tests then
701 2016-09-08T19:52:25  <wumpus> cfields: ack
702 2016-09-08T19:52:33  <sipa> cfields: thanks
703 2016-09-08T19:52:49  <wumpus> #endmeeting
704 2016-09-08T19:52:49  <lightningbot> Meeting ended Thu Sep  8 19:52:49 2016 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
705 2016-09-08T19:52:49  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-09-08-18.59.html
706 2016-09-08T19:52:49  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-09-08-18.59.txt
707 2016-09-08T19:52:49  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-09-08-18.59.log.html
708 2016-09-08T19:53:11  * michagogo wishes he had more time lately
709 2016-09-08T19:53:35  *** instagibbs_ has joined #bitcoin-core-dev
710 2016-09-08T19:53:42  <michagogo> 2-3 hour commutes are not very fun.
711 2016-09-08T19:53:47  <wumpus> michagogo: yes we've been missing you lately!
712 2016-09-08T19:54:14  <michagogo> (Where lately = >1 year, I think…)
713 2016-09-08T19:54:31  <instagibbs_> michagogo: work from home, your hair will get fuller, and silkier with the reduced stress
714 2016-09-08T19:54:37  <MarcoFalke> michagogo: Let someone drive for you and use the time to catch up on stuff.
715 2016-09-08T19:54:53  <michagogo> It's a good thing I scripted gitian builds, so at least I can kick those off and let them run at home
716 2016-09-08T19:55:06  <michagogo> instagibbs_: unfortunately that's not an option for various reasons
717 2016-09-08T19:55:59  <michagogo> MarcoFalke: I wish that were feasible, but I'm provided with free public transportation and don't have a car
718 2016-09-08T19:56:05  <phantomcircuit> which pr changed the wallet handling to be background?
719 2016-09-08T19:56:11  *** vega4 has joined #bitcoin-core-dev
720 2016-09-08T19:56:18  <gmaxwell> Sorry for contributing to confusion above.  I probably misunderstood people, but it sounded to me that people were saying that the wallet operating non-concurrently was part of the API and that we couldn't change it.  I agree its a minor part of the api but think we must change it.  Making all the blocks 'confirmedness' updates to the wallet atomic seems reasonable to me (though I'm dubious anyt
721 2016-09-08T19:56:24  <gmaxwell> hing would actually be broken by that which isn't already broken)
722 2016-09-08T19:56:27  <jonasschnelli> phantomcircuit: i guess there is no PR for that?
723 2016-09-08T19:56:43  <michagogo> I try to figure out how to get rides whenever I can, but that's not very often
724 2016-09-08T19:56:56  <phantomcircuit> jonasschnelli: o.O ?
725 2016-09-08T19:57:15  <jonasschnelli> It was just a discussion.
726 2016-09-08T19:58:15  *** vega4 has quit IRC
727 2016-09-08T19:58:16  <gmaxwell> I don't think "would be an api change" should ever be an argument against something in a major release, and it was kind of worrying me to hear people talking that way... esp. since the current api stinks in a number of minor ways and needs improving. "Would be a rather disruptive API change relative to the benefits" would be a fine argument, but it wasn't what I was hearing, maybe it was implici
728 2016-09-08T19:58:22  <gmaxwell> t. Though I think making the wallet and block processing mutually concurrent has a lot of obvious benefits.
729 2016-09-08T19:58:22  <morcos> phantomcircuit: 7964 (its not background, it just happens outside of cs_main now)
730 2016-09-08T19:58:29  *** Samdney has joined #bitcoin-core-dev
731 2016-09-08T19:59:07  <CodeShark> gmaxwell: if it breaks deployed apps, it's a potential issue - otherwise, it isn't
732 2016-09-08T19:59:09  <sipa> gmaxwell: absolutely - i think everything that was proposed would never grab cs_main and cs_wallet at the same time (even though they're in the same thread for now)
733 2016-09-08T19:59:39  <morcos> gmaxwell: i think thats a fair summary.   in the future it probably makes sense for someone (like me in this case) to write up a small issue before the meeting, so people aren't trying to read about a somewhat nuanced issue in interlaced realtime chat
734 2016-09-08T20:00:28  <jonasschnelli> gmaxwell: I guess it depends from which angle you are looking at it. Current users expect (API) that getblockchaininfo represents the state of the wallet...
735 2016-09-08T20:00:35  <jonasschnelli> Which I agree we should keep for 0.13.x
736 2016-09-08T20:00:52  <phantomcircuit> morcos: that seems to be something about c++11 consistency
737 2016-09-08T20:01:02  <jonasschnelli> But afterwards, I think is a good signal to have own wallet process state calls
738 2016-09-08T20:01:36  <CodeShark> I'm in favor of any change that further decouples the wallet from the rest of the app ;)
739 2016-09-08T20:01:40  <jonasschnelli> To slowly make people aware of that the wallet might gets more of its own state/processes
740 2016-09-08T20:01:51  *** spudowiar has joined #bitcoin-core-dev
741 2016-09-08T20:02:14  <michagogo> ??? https://usercontent.irccloud-cdn.com/file/JhPBIVrq/IMG_3841.PNG
742 2016-09-08T20:03:07  <BlueMatt> gmaxwell: sorry, indeed that was my implied argument
743 2016-09-08T20:03:16  <BlueMatt> gmaxwell: I agree, making an api change is generally fine
744 2016-09-08T20:03:28  <BlueMatt> but making one that is pervasive across the entire api for such a minor gain seems insane
745 2016-09-08T20:03:49  <BlueMatt> and generally making a massively pervasive change across the entire api isnt ideal unless its easy to explain how to change your clients to work with it
746 2016-09-08T20:03:51  <gmaxwell> jonasschnelli: I think we should keep things for 0.13, sure.  though take care, there is no way to perform multiple RPCs atomically. So if you call getblockchaininfo then interact with the wallet, the world might have changed out from under you, .. in any version. :P
747 2016-09-08T20:04:42  <BlueMatt> gmaxwell: and, generally, I'd prefer for things like that to have an -wrapentireapitomakeitbackwardscompatible option
748 2016-09-08T20:05:02  <gmaxwell> BlueMatt: okay, my view is that having the wallet and node be concurrent is not a minor gain. (esp since that also means multiple wallets being concurrent with each other, ultimately)
749 2016-09-08T20:05:04  <jonasschnelli> Yes. Indeed. But users expect – in normal non reorg operations – that getblockchaininfo at least reports a height already processed by a wallet
750 2016-09-08T20:05:10  <jonasschnelli> *the wallet
751 2016-09-08T20:05:18  <sipa> well, one question is where do people "wait" for getblockcount to go up, and then take wallet action based on it?
752 2016-09-08T20:05:24  <sipa> i guess we don't know
753 2016-09-08T20:05:30  <MarcoFalke> I am fine with breaking the wallet api, but we should make it explicit and not silent.
754 2016-09-08T20:05:30  <BlueMatt> gmaxwell: oops, sure, I agree this case is a major gain
755 2016-09-08T20:05:38  <sipa> but it seems much more something unit tests do, because they now exactly what is going to happen
756 2016-09-08T20:05:42  <MarcoFalke> Eg. include the blockhash/height in every wallet call
757 2016-09-08T20:05:44  <BlueMatt> if it werent also trivial to keep the api consistent I might have a different view
758 2016-09-08T20:05:45  <cfields> gmaxwell: to be clear, I don't think the concern (from me, anyway) was "is this an api change", so I suppose I framed the question very poorly. The concern was more "did we break reasonable assumptions by making this more async?"
759 2016-09-08T20:05:55  <jonasschnelli> I guess the problem really apperars when you listen to notifications (ZMQ)
760 2016-09-08T20:06:01  <sipa> jonasschnelli: it does not
761 2016-09-08T20:06:20  <jonasschnelli> Right.. we face the problem during the RPC tests...
762 2016-09-08T20:06:32  <kanzure> if the wallet does its own block processing, that might be interesting, it could even consume the same notifications as third-party apps would be expected to (like zeromq or rpc communication)
763 2016-09-08T20:06:35  <sipa> we can control when the notification is sent out
764 2016-09-08T20:06:43  <jonasschnelli> But I guess you will run more often into this concurrency issues when calling RPC off a ZMQ not.
765 2016-09-08T20:06:44  <morcos> phantomcircuit: oops.  7946
766 2016-09-08T20:07:05  <sipa> i think it may be the case that the only ones affected by the asynchronicity is unit tests
767 2016-09-08T20:07:06  <phantomcircuit> morcos: ty
768 2016-09-08T20:07:25  <sipa> (that's independent from the partial update thing, which we can fix in various other ways)
769 2016-09-08T20:07:30  *** pmienk has quit IRC
770 2016-09-08T20:08:00  <kanzure> wait why would anyone wait for getblockcount to go up...? there is no conceivable reason i could see to do that.
771 2016-09-08T20:08:21  <sipa> "i do X, and i wait for Y to happen" is something you can't do in production env
772 2016-09-08T20:08:42  <kanzure> we are talking about syncing two separate processes that are supposedly monitoring a blockchain right?
773 2016-09-08T20:08:59  <sipa> no
774 2016-09-08T20:09:06  <gmaxwell> cfields: basically the tests failing proved its an API change, maybe not an important one, but absolutely a change.
775 2016-09-08T20:09:17  <sipa> we're even talking about something that happens entirely within one thread :)
776 2016-09-08T20:09:22  <cfields> kanzure: quick example: tracking how many confirmations your txs have and displaying on your site
777 2016-09-08T20:09:45  <kanzure> cfields: that would make sense to me, but sipa is talking about one thread?
778 2016-09-08T20:09:55  <cfields> not saying it's wise, but I can imagine that happening in the wild
779 2016-09-08T20:10:13  <phantomcircuit> kanzure: you need more imagination
780 2016-09-08T20:10:18  <sipa> kanzure: oh, ignoring the external process of course
781 2016-09-08T20:10:23  <phantomcircuit> listsinceblock
782 2016-09-08T20:10:27  <kanzure> i have recently spent a bunch of time writing and validating software that does blockchain syncing between two stores where one is considered truthy and the other is considered a laggard
783 2016-09-08T20:10:29  <sipa> kanzure: but all wallet updates and main updates happen within one thread in core
784 2016-09-08T20:10:49  <kanzure> listsinceblock is also something you shouldn't use. you should instead compute the most recent common block and then get the list of different blocks that your laggy side needs to consider.
785 2016-09-08T20:11:16  <phantomcircuit> kanzure: listsinceblock is the thing people (not me) have been suggesting for years
786 2016-09-08T20:11:17  <kanzure> looking at a block counter is backwards because it could go up and down
787 2016-09-08T20:11:33  <kanzure> and treating it as a monotonic clock is what i would expect most application developers to do heh
788 2016-09-08T20:12:25  <kanzure> i didn't look at the origiating issue that brought up this topic, so i'm sure my context is wrong here
789 2016-09-08T20:12:34  <kanzure> here is a toy i was writing a few days ago https://gist.github.com/kanzure/2fa531afaf03fddd6568eb0212ac8c4c
790 2016-09-08T20:13:41  <phantomcircuit> kanzure: wallet processing of blocks is put into the background, wallet rpc calls can then return results reflecting a partial processing of a block
791 2016-09-08T20:13:51  <CodeShark> fully async json-over-websockets API: https://github.com/ciphrex/CoinSocket ;)
792 2016-09-08T20:13:56  <phantomcircuit> ie tx a/b are both in block c but only a shows in the rpc calls
793 2016-09-08T20:14:05  <kanzure> btw when people above are talking about "unit tests" they mean the rpc tests right?
794 2016-09-08T20:14:12  <sipa> kanzure: yes
795 2016-09-08T20:14:25  *** cryptapus_afk is now known as cryptapus
796 2016-09-08T20:14:28  <cfields> kanzure: unit tests in bitcoind-speak :)
797 2016-09-08T20:15:02  <kanzure> re: async, then yes i would suggest using zeromq notifications in the testing framework to get information about internal state or for how long to wait (instead of sleeps etc)
798 2016-09-08T20:15:11  <phantomcircuit> kanzure: they're talking about the functional tests
799 2016-09-08T20:15:15  <phantomcircuit> not the unit tests
800 2016-09-08T20:16:04  *** paveljanik has joined #bitcoin-core-dev
801 2016-09-08T20:17:03  <cfields> kanzure: the question isn't about how to get the info, it's about what the caller expects. you can query for the current height, then call into the wallet and expect to see tx's from that height, only to find that they haven't been processed yet.
802 2016-09-08T20:17:41  <kanzure> right... so you could naively sleep to wait for the wallet to be done, or you can query the wallet state (instead of chain height of the non-wallet components)
803 2016-09-08T20:17:48  <cfields> so who's wrong? the rpc for returning a height without fully processing? or the user for expecting them to be in sync?
804 2016-09-08T20:18:10  <kanzure> er in this case isn't the rpc test also the user?
805 2016-09-08T20:18:52  *** fengling has joined #bitcoin-core-dev
806 2016-09-08T20:18:53  <sipa> so i believe there is a slight difference between normal users and the rpc tests
807 2016-09-08T20:18:54  <cfields> yea, i was just phrasing more generally
808 2016-09-08T20:19:14  <sipa> in that rpc tests control the entire world, and use more simplistic means to measure progress
809 2016-09-08T20:19:35  <cfields> sipa: yes. i doubt many rpc users/scripts are hammering on block-height in tightish loops :)
810 2016-09-08T20:19:38  <cfields> hope not, anyway
811 2016-09-08T20:19:39  <kanzure> so- for example- imagine you are operating a lot of bitcoin infrastructure. you do a bunch of per-block processing. when you are running functional/integration tests, you can't run your test checks immediately after issuing a command because things take a while. so when do you expect to run? well you setup a hook and you ask your infrastructure to ping your tester as soon as the data's available, or you poll for some internal state.
812 2016-09-08T20:20:10  <kanzure> (and polling is considered not ideal)
813 2016-09-08T20:20:51  <sipa> and non-polling methods exist
814 2016-09-08T20:21:00  <sipa> we have -walletnotify for exactly this
815 2016-09-08T20:21:58  <kanzure> walletnotify should certainly be used by the rpc tests
816 2016-09-08T20:22:26  <sipa> this raises another point: walletnotify should probably fire after all of a block's transactions are processed
817 2016-09-08T20:23:04  <kanzure> i would like zeromq notifications and -*notify notifications to be unified somehow, and eventually have much more distinct notifications for different events going on
818 2016-09-08T20:23:16  <kanzure> changing walletnotify behavior and when it fires is sort of unkind to the users that rely on its existing behavior :)
819 2016-09-08T20:23:46  *** fengling has quit IRC
820 2016-09-08T20:23:57  <cfields> kanzure: and #8680 introduces blocking calls. So you could "./bitcoin-cli waitfornewblock && ./dosomething.sh", which would essentially be blocknotify. Obviously the same could be done for wallet stuff
821 2016-09-08T20:24:00  <CodeShark> I would like all pub/sub stuff in a separate layer ;)
822 2016-09-08T20:24:28  <CodeShark> especially if it requires deep knowledge of account structure
823 2016-09-08T20:24:43  <kanzure> i feel like users are going to not understand how to use waitfornewblock. how are you going to stop them from treating that as a monotonic counter for each new lbock...?
824 2016-09-08T20:26:51  <cfields> kanzure: that's a hack, hidden for users, just for tests. It's a pretty bad idea to take action on an event so vague :)
825 2016-09-08T20:26:59  <kanzure> btw i (sheepishly) made a one-line proposal about throwing a component into bitcoind to keep track of external application state, specifically to store a range or diff-set of blocks that an external application should be aware about, and then the implementation proposal would be "please integrate against this" and we give the user an endpoint for saying "my application has processed the latest diff-workload x".  but this might be too ...
826 2016-09-08T20:27:05  <kanzure> ... kitchen-sinky.
827 2016-09-08T20:27:08  <kanzure> oh it's for testing. hm.
828 2016-09-08T20:27:30  <cfields> i was just using it as an example of an async approach.
829 2016-09-08T20:29:53  <kanzure> (the "diff-workload x" is specifically referring to a set of new blocks some of hwich might override previous blocks the other application was aware about. this makes the interface explicitly expose a concept of reorgs.)
830 2016-09-08T20:30:18  <kanzure> CodeShark: in your link i'm not seeing any tests for that library...
831 2016-09-08T20:30:46  <CodeShark> the whole thing needs to be repackaged
832 2016-09-08T20:32:00  <CodeShark> just giving it as an example of how I think an API should work
833 2016-09-08T20:32:31  <kanzure> cfields: fwiw once i switched to notifications inside my own tests (for some external infrastructure) re: bitcoind integration, everything went much faster and i had much more information about failures. far less timeouts too.
834 2016-09-08T20:32:31  <cfields> kanzure: what would bitcoind do with the knowledge that the application is done processing?
835 2016-09-08T20:32:36  <sipa> can we talk about solutions instead of vague design concerns?
836 2016-09-08T20:33:05  <kanzure> cfields: nothing. it's for the benefit of the application. once bitcoind is informed about the application's new state, it can discard the old diff information. that's about all it would do...
837 2016-09-08T20:34:42  <CodeShark> two subscription layers - one low level, the other much higher level
838 2016-09-08T20:35:05  <CodeShark> the low level layer only gives you basic events like new tip or new tx
839 2016-09-08T20:35:06  *** sipa has left #bitcoin-core-dev
840 2016-09-08T20:35:27  <CodeShark> dunno why sipa doesn't like to discuss architecture :(
841 2016-09-08T20:36:05  <CodeShark> it
842 2016-09-08T20:36:20  <CodeShark> it's a serious problem for the ecosystem right now that there's no good way for people to build apps
843 2016-09-08T20:36:41  *** MarcoFalke has left #bitcoin-core-dev
844 2016-09-08T20:37:27  *** sipa has joined #bitcoin-core-dev
845 2016-09-08T20:40:58  <cfields> sipa: the more I think about it, the more I think that there needs to be a wallet rpc for getbalanceat(height)
846 2016-09-08T20:41:33  <cfields> i can't think of anything that doesn't introduce more problems except for explicit/atomic calls
847 2016-09-08T20:41:44  <kanzure> should be a blockhash not a height
848 2016-09-08T20:42:10  <cfields> sure
849 2016-09-08T20:43:24  <sipa> cfields: that makes sense
850 2016-09-08T20:43:38  <cfields> I think the reason today's discussion devolved so quickly is because "getblockcount" is impossible to define, because there's no global height. So the only fix is to ensure that we're asking a specific interface a specific question. Then there's no way of being out of sync because you've specified your constraints
851 2016-09-08T20:43:58  * sipa goes for a walk, without phonr
852 2016-09-08T20:44:03  <sipa> be back in a while
853 2016-09-08T20:44:08  <cfields> cya
854 2016-09-08T20:45:00  <CodeShark> cfields: right
855 2016-09-08T20:46:15  <CodeShark> there's no such thing as "what's the current state of the network?"
856 2016-09-08T20:46:18  <cfields> then it's just a matter of how to handle the api. returning "come back later", blocking until there's an answer, or some callback mechanism.
857 2016-09-08T20:47:51  <cfields> arguably if you need a callback, you should be using a different interface. as kanzure suggested above. I'm of the opinion that blocking is the way to go because that lets the user chain his own callback
858 2016-09-08T20:48:25  <kanzure> blocking in what context? rpc ?
859 2016-09-08T20:48:29  <CodeShark> depends on what model you're after - I think async is generally cleaner but requires a bit more clientside framework
860 2016-09-08T20:49:03  <cfields> kanzure: yes, rpc blocks for a specified time (or forever) and returns when it has an answer, or timeout, or shutdown
861 2016-09-08T20:49:32  <kanzure> oh. i've had so may issues with rpc socket timeouts that i haven't traced down yet that i would recommend not considering that direction heh.
862 2016-09-08T20:49:43  <kanzure> (but also i haven't been able to make good reports about these problems, so they don't really count.)
863 2016-09-08T20:50:18  <cfields> kanzure: heh, i was actually surprised i didn't bump into those during testing. good to know :)
864 2016-09-08T20:50:37  <kanzure> well i think what i was doing could be considered "torture testing" and weird stuff, so.... yeah..
865 2016-09-08T20:51:28  <kanzure> to some degree aren't we already tracking peer state on the p2p layer? e.g. like which chains the peers seem to be following lately?
866 2016-09-08T20:51:47  <kanzure> or is that purely a function of the banning behavior heh
867 2016-09-08T20:53:19  <cfields> lately?
868 2016-09-08T20:53:28  <kanzure> (by virtue that you could ban everyone feeding you unhelpful data, as opposed to tracking which blocks you think peers have already)
869 2016-09-08T20:54:37  <kanzure> nevermind. just an abstract thought, it's not important, i was thinking that "peer communication" is sometimes similar to "application communication" but more one-way. anyway, just a minor distraction.
870 2016-09-08T20:56:43  <cfields> we keep tabs on what they shouldn't be sending, but i don't remember to what extent
871 2016-09-08T20:59:02  <CodeShark> a peer can query for inventory at any time regardless of whether it is in agreement with the node's best chain
872 2016-09-08T20:59:14  *** instagibbs_ has quit IRC
873 2016-09-08T20:59:26  *** e4xit_ has joined #bitcoin-core-dev
874 2016-09-08T21:00:10  <CodeShark> I think peers get sent invs whenever there are tip changes regardless
875 2016-09-08T21:00:59  *** e4xit has quit IRC
876 2016-09-08T21:01:00  *** e4xit_ is now known as e4xit
877 2016-09-08T21:15:00  *** Chris_Stewart_5 has quit IRC
878 2016-09-08T21:20:21  *** fengling has joined #bitcoin-core-dev
879 2016-09-08T21:25:06  *** fengling has quit IRC
880 2016-09-08T21:29:39  *** cryptapus is now known as cryptapus_afk
881 2016-09-08T21:53:11  <gmaxwell> "(and polling is considered not ideal)
882 2016-09-08T21:53:13  <gmaxwell> "
883 2016-09-08T21:53:41  <gmaxwell> This bugs me. Polling is highly reliable. And we're taking about things that have inherent latencies in real of seconds to tends of minutes.
884 2016-09-08T21:54:34  <phantomcircuit> polling is the correct way to interact with the wallet
885 2016-09-08T21:54:37  <gmaxwell> I do not know why people eschew polling as often as they do... there are contexts where its exactly the right tool and keeping track of a wallet state is likely one of them.
886 2016-09-08T21:54:41  <phantomcircuit> if you're not polling you're doing it wrong
887 2016-09-08T22:03:40  *** spudowiar has quit IRC
888 2016-09-08T22:08:45  <kanzure> well one trivial example is in scenarios where you have long-term polling, you will miss the update and spend extra time waiting
889 2016-09-08T22:08:56  <kanzure> this is apparent in scenarios with, say, exponential backoff
890 2016-09-08T22:11:25  <phantomcircuit> kanzure, call listtransactions
891 2016-09-08T22:11:37  <phantomcircuit> does the list contain a transaction you have previously seen?
892 2016-09-08T22:11:42  <kanzure> instead of hoping you catch wallet state transitions by polling, what about just having the wallet notify you directly of each change event? then you consume the feed/queue of events.
893 2016-09-08T22:11:43  <phantomcircuit> yes? great do nothing
894 2016-09-08T22:11:52  <phantomcircuit> no? increase the list size
895 2016-09-08T22:12:19  <phantomcircuit> handle reorgs? need to call the blockchain things and do a bunch of work
896 2016-09-08T22:12:45  <kanzure> which application are you imagining for listtransactions in particular?
897 2016-09-08T22:12:59  *** da2ce7_ has quit IRC
898 2016-09-08T22:13:16  <phantomcircuit> anything which needs to know that a payment has been made
899 2016-09-08T22:13:40  <kanzure> i don't know what the behavior of listtransactions is. how does it behave when a transaction has been removed from your chaintip..?
900 2016-09-08T22:14:14  *** da2ce7 has joined #bitcoin-core-dev
901 2016-09-08T22:15:00  <phantomcircuit> kanzure: it's buried in the list and has -1 confirmations
902 2016-09-08T22:15:40  <kanzure> for how long...? it occurs to me that listtransactions is probably a wallet-specific rpc command and that's why i know so little about it.
903 2016-09-08T22:17:11  <kanzure> getting notified when a transaction is about to be permanently removed from that list sonuds like a useful thing, but you could probably infer that detail from polling regularly and knowing the lifecycle i guess.
904 2016-09-08T22:17:34  <phantomcircuit> kanzure, listtransactions only shows you transactions in your wallet
905 2016-09-08T22:17:53  <phantomcircuit> nothing is ever removed from the list there's just a parameter which truncates the list
906 2016-09-08T22:18:11  <phantomcircuit> listtransactions "*" 4294967295
907 2016-09-08T22:18:21  <phantomcircuit> will give you all the transactions your wallet has ever seen
908 2016-09-08T22:19:23  <kanzure> incremental updates and messaging would not require so much data transfer all at once. :)
909 2016-09-08T22:19:47  <phantomcircuit> kanzure, messaging is unreliable
910 2016-09-08T22:19:53  <phantomcircuit> polling is inherently reliable
911 2016-09-08T22:19:54  <phantomcircuit> fin
912 2016-09-08T22:20:33  <sipa> we shouldn't make assumptions about how people use the api
913 2016-09-08T22:21:51  *** fengling has joined #bitcoin-core-dev
914 2016-09-08T22:23:54  <kanzure> my above concerns are mostly about polling interval
915 2016-09-08T22:26:46  *** fengling has quit IRC
916 2016-09-08T22:34:42  <kanzure> and blockheight can't be a heartbeat
917 2016-09-08T22:52:22  *** TomMc has quit IRC
918 2016-09-08T22:53:53  *** Guyver2 has quit IRC
919 2016-09-08T22:54:13  <BlueMatt> sipa: https://github.com/TheBlueMatt/bitcoin/commit/96cef326f9184aefd1c562f64a21298a15f0adde simplifies the total diff from master to be more readable, to me...it matches the bip, but another way to look at it is that you have two booleans: which version of compact blocks they want (which you use to send to them) and whether they will send us segwit cmpctblocks...you simply introduce the the do-they-support-v2 bool as an additional
920 2016-09-08T22:54:13  <BlueMatt> check before requesting cmpcblocks and otherwise dont need the complicated other semantics
921 2016-09-08T22:54:57  <BlueMatt> (warning: mostly untested...it compiles, though :p)
922 2016-09-08T23:02:48  <BlueMatt> ehh, nvm, need to fix a bug there, but I'm tired so will do it later/tomorrow
923 2016-09-08T23:06:46  *** e4xit has joined #bitcoin-core-dev
924 2016-09-08T23:17:21  *** Chris_Stewart_5 has joined #bitcoin-core-dev
925 2016-09-08T23:23:32  *** fengling has joined #bitcoin-core-dev
926 2016-09-08T23:23:40  *** TomMc has joined #bitcoin-core-dev
927 2016-09-08T23:26:52  *** Ylbam has quit IRC
928 2016-09-08T23:28:26  *** fengling has quit IRC
929 2016-09-08T23:37:00  *** TomMc has quit IRC
930 2016-09-08T23:40:26  *** jtimon has quit IRC