1 2020-12-11T00:00:20  *** jonatack <jonatack!~jon@> has joined #bitcoin-core-dev
  2 2020-12-11T00:09:52  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@> has joined #bitcoin-core-dev
  3 2020-12-11T00:10:25  *** belcher <belcher!~belcher@unaffiliated/belcher> has quit IRC (Ping timeout: 264 seconds)
  4 2020-12-11T00:10:33  *** belcher_ <belcher_!~belcher@unaffiliated/belcher> has joined #bitcoin-core-dev
  5 2020-12-11T00:14:18  <luke-jr> my gitians all match, pushed
  6 2020-12-11T00:15:03  *** chri_eb <chri_eb!~chris@gateway/tor-sasl/chrieb/x-28824719> has quit IRC (Ping timeout: 240 seconds)
  7 2020-12-11T00:15:23  *** sipa <sipa!~pw@gateway/tor-sasl/sipa1024> has quit IRC (Ping timeout: 240 seconds)
  8 2020-12-11T00:15:24  *** sdaftuar <sdaftuar!~sdaftuar@gateway/tor-sasl/sdaftuar> has quit IRC (Ping timeout: 240 seconds)
  9 2020-12-11T00:15:24  *** virtu <virtu!~virtu@gateway/tor-sasl/virtu> has quit IRC (Ping timeout: 240 seconds)
 10 2020-12-11T00:15:24  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has quit IRC (Ping timeout: 240 seconds)
 11 2020-12-11T00:15:24  *** yanmaani <yanmaani!~yanmaani@gateway/tor-sasl/yanmaani> has quit IRC (Ping timeout: 240 seconds)
 12 2020-12-11T00:15:43  *** andrewtoth_ <andrewtoth_!~andrewtot@gateway/tor-sasl/andrewtoth> has quit IRC (Ping timeout: 240 seconds)
 13 2020-12-11T00:15:44  *** az0re <az0re!~az0re@gateway/tor-sasl/az0re> has quit IRC (Ping timeout: 240 seconds)
 14 2020-12-11T00:15:44  *** jb55 <jb55!~jb55@gateway/tor-sasl/jb55> has quit IRC (Ping timeout: 240 seconds)
 15 2020-12-11T00:15:44  *** vasild <vasild!~vd@gateway/tor-sasl/vasild> has quit IRC (Ping timeout: 240 seconds)
 16 2020-12-11T00:15:44  *** k3tan <k3tan!~pi@gateway/tor-sasl/k3tan> has quit IRC (Ping timeout: 240 seconds)
 17 2020-12-11T00:16:03  *** ghost43 <ghost43!~daer@gateway/tor-sasl/daer> has quit IRC (Ping timeout: 240 seconds)
 18 2020-12-11T00:17:06  *** theStack <theStack!~honeybadg@vps1648322.vs.webtropia-customer.com> has quit IRC (Ping timeout: 272 seconds)
 19 2020-12-11T00:17:36  *** theStack <theStack!~honeybadg@vps1648322.vs.webtropia-customer.com> has joined #bitcoin-core-dev
 20 2020-12-11T00:20:31  *** belcher_ <belcher_!~belcher@unaffiliated/belcher> has quit IRC (Ping timeout: 246 seconds)
 21 2020-12-11T00:23:16  *** chri_eb <chri_eb!~chris@gateway/tor-sasl/chrieb/x-28824719> has joined #bitcoin-core-dev
 22 2020-12-11T00:23:24  *** k3tan <k3tan!~pi@gateway/tor-sasl/k3tan> has joined #bitcoin-core-dev
 23 2020-12-11T00:24:04  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has joined #bitcoin-core-dev
 24 2020-12-11T00:24:04  *** jb55 <jb55!~jb55@gateway/tor-sasl/jb55> has joined #bitcoin-core-dev
 25 2020-12-11T00:24:04  *** vasild <vasild!~vd@gateway/tor-sasl/vasild> has joined #bitcoin-core-dev
 26 2020-12-11T00:24:06  *** ghost43 <ghost43!~daer@gateway/tor-sasl/daer> has joined #bitcoin-core-dev
 27 2020-12-11T00:24:42  *** yanmaani <yanmaani!~yanmaani@gateway/tor-sasl/yanmaani> has joined #bitcoin-core-dev
 28 2020-12-11T00:26:00  *** sdaftuar <sdaftuar!~sdaftuar@gateway/tor-sasl/sdaftuar> has joined #bitcoin-core-dev
 29 2020-12-11T00:27:34  *** sipa <sipa!~pw@gateway/tor-sasl/sipa1024> has joined #bitcoin-core-dev
 30 2020-12-11T00:29:01  *** kristapsk <kristapsk!~KK@gateway/tor-sasl/kristapsk> has joined #bitcoin-core-dev
 31 2020-12-11T00:32:20  *** virtu <virtu!~virtu@gateway/tor-sasl/virtu> has joined #bitcoin-core-dev
 32 2020-12-11T00:32:23  *** az0re <az0re!~az0re@gateway/tor-sasl/az0re> has joined #bitcoin-core-dev
 33 2020-12-11T00:36:46  <achow101> why did we keep changing the sdk?
 34 2020-12-11T00:36:51  <achow101> *macOS sdk
 35 2020-12-11T00:37:09  <achow101> 0.19 is 10.11, 0.20 is 10.14, and 0.21 is long_xcode_name
 36 2020-12-11T00:38:12  *** twistedline_ <twistedline_!~twisted@2601:14d:8500:a77d:2985:36c3:6c1f:47b8> has joined #bitcoin-core-dev
 37 2020-12-11T00:38:23  *** twistedline <twistedline!~twisted@unaffiliated/twistedline> has quit IRC (Ping timeout: 258 seconds)
 38 2020-12-11T00:42:43  *** vasild <vasild!~vd@gateway/tor-sasl/vasild> has quit IRC (Ping timeout: 240 seconds)
 39 2020-12-11T00:45:05  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)
 40 2020-12-11T00:46:15  <fanquake> achow: just to piss everyone off
 41 2020-12-11T00:46:52  <fanquake> sipa: welcome back to the gitian building world
 42 2020-12-11T00:48:42  <sipa> :)
 43 2020-12-11T00:49:17  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
 44 2020-12-11T00:49:18  <bitcoin-git> [bitcoin] dongcarl opened pull request #20619: guix: Quality of life improvements (master...2020-12-guix-fixups) https://github.com/bitcoin/bitcoin/pull/20619
 45 2020-12-11T00:49:19  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
 46 2020-12-11T00:50:07  *** peterrizzo_ <peterrizzo_!~peterrizz@ool-44c18924.dyn.optonline.net> has quit IRC (Quit: peterrizzo_)
 47 2020-12-11T00:57:51  *** promag_ <promag_!~promag@> has joined #bitcoin-core-dev
 48 2020-12-11T00:59:13  *** promag <promag!~promag@> has quit IRC (Ping timeout: 260 seconds)
 49 2020-12-11T01:01:21  *** promag <promag!~promag@> has joined #bitcoin-core-dev
 50 2020-12-11T01:01:22  *** promag_ <promag_!~promag@> has quit IRC (Read error: Connection reset by peer)
 51 2020-12-11T01:06:08  *** promag_ <promag_!~promag@> has joined #bitcoin-core-dev
 52 2020-12-11T01:09:58  *** promag <promag!~promag@> has quit IRC (Ping timeout: 256 seconds)
 53 2020-12-11T01:18:54  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@mx-ll-171.5.29-209.dynamic.3bb.co.th> has joined #bitcoin-core-dev
 54 2020-12-11T01:20:16  *** twistedline_ <twistedline_!~twisted@2601:14d:8500:a77d:2985:36c3:6c1f:47b8> has quit IRC (Read error: Connection reset by peer)
 55 2020-12-11T01:20:23  *** twistedline_ <twistedline_!~twisted@2601:14d:8500:a77d:2985:36c3:6c1f:47b8> has joined #bitcoin-core-dev
 56 2020-12-11T01:30:50  *** peterrizzo_ <peterrizzo_!~peterrizz@ool-44c18924.dyn.optonline.net> has joined #bitcoin-core-dev
 57 2020-12-11T01:34:13  *** jeremyrubin <jeremyrubin!~jr@c-73-15-215-148.hsd1.ca.comcast.net> has quit IRC (Ping timeout: 260 seconds)
 58 2020-12-11T01:36:37  *** larryruane_ <larryruane_!uid473749@gateway/web/irccloud.com/x-ucsaxcwppqjkhsjq> has quit IRC (Quit: Connection closed for inactivity)
 59 2020-12-11T01:46:08  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
 60 2020-12-11T01:46:09  <bitcoin-git> [bitcoin] fanquake pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/da957cd62ecc...736eb4d80838
 61 2020-12-11T01:46:10  <bitcoin-git> bitcoin/master c5e3e74 Hennadii Stepanov: sync: Improve CheckLastCritical()
 62 2020-12-11T01:46:11  <bitcoin-git> bitcoin/master cb23fe0 Hennadii Stepanov: [skip ci] sync: Check precondition in LEAVE_CRITICAL_SECTION() macro
 63 2020-12-11T01:46:12  <bitcoin-git> bitcoin/master e1e68b6 Hennadii Stepanov: test: Fix inconsistent lock order in wallet_tests/CreateWallet
 64 2020-12-11T01:46:16  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
 65 2020-12-11T01:46:33  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
 66 2020-12-11T01:46:33  <bitcoin-git> [bitcoin] fanquake merged pull request #19982: test: Fix inconsistent lock order in wallet_tests/CreateWallet (master...200920-leave-cs) https://github.com/bitcoin/bitcoin/pull/19982
 67 2020-12-11T01:46:34  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
 68 2020-12-11T01:46:59  <fanquake> I assume now that we are using cirrus, adding things like [skip ci] to commit messages is pointless ?
 69 2020-12-11T01:47:12  <fanquake> I never really saw the point of doing it anyways
 70 2020-12-11T01:47:50  <andytoshi> travis still runs some lints that check every commit
 71 2020-12-11T01:48:44  <fanquake> Yea. Although hopefully that is going away shortly: #20467
 72 2020-12-11T01:48:45  <gribble> https://github.com/bitcoin/bitcoin/issues/20467 | Move travis linter job to cirrus · Issue #20467 · bitcoin/bitcoin · GitHub
 73 2020-12-11T01:49:08  <fanquake> I think someone else could jump in and  take that on if they wanted, given it's been 2 weeks
 74 2020-12-11T01:50:19  *** vasild <vasild!~vd@gateway/tor-sasl/vasild> has joined #bitcoin-core-dev
 75 2020-12-11T01:50:31  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@mx-ll-171.5.29-209.dynamic.3bb.co.th> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)
 76 2020-12-11T01:57:44  *** dr-orlovsky <dr-orlovsky!~dr-orlovs@> has quit IRC (Ping timeout: 256 seconds)
 77 2020-12-11T02:04:52  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@> has joined #bitcoin-core-dev
 78 2020-12-11T02:05:40  *** az0re <az0re!~az0re@gateway/tor-sasl/az0re> has quit IRC (Quit: Leaving)
 79 2020-12-11T02:17:37  *** miketwen_ <miketwen_!~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com> has quit IRC (Ping timeout: 264 seconds)
 80 2020-12-11T02:35:23  <fanquake> Someone could also port the Travis check in the gitian.sigs repo to Cirrus (if we want to keep it). That should be very easy
 81 2020-12-11T02:54:36  *** jeremyrubin <jeremyrubin!~jr@c-73-15-215-148.hsd1.ca.comcast.net> has joined #bitcoin-core-dev
 82 2020-12-11T02:56:57  *** twistedline_ <twistedline_!~twisted@2601:14d:8500:a77d:2985:36c3:6c1f:47b8> has quit IRC (Remote host closed the connection)
 83 2020-12-11T02:57:21  *** twistedline_ <twistedline_!~twisted@2601:14d:8500:a77d:2985:36c3:6c1f:47b8> has joined #bitcoin-core-dev
 84 2020-12-11T02:58:24  *** Eagle[TM] <Eagle[TM]!~EagleTM@unaffiliated/eagletm> has joined #bitcoin-core-dev
 85 2020-12-11T03:00:38  *** EagleTM <EagleTM!~EagleTM@unaffiliated/eagletm> has quit IRC (Ping timeout: 256 seconds)
 86 2020-12-11T03:14:26  *** twistedline_ <twistedline_!~twisted@2601:14d:8500:a77d:2985:36c3:6c1f:47b8> has quit IRC (Ping timeout: 264 seconds)
 87 2020-12-11T03:19:37  *** twistedline <twistedline!~twisted@2601:14d:8500:a77d:2985:36c3:6c1f:47b8> has joined #bitcoin-core-dev
 88 2020-12-11T03:23:53  *** peterrizzo_ <peterrizzo_!~peterrizz@ool-44c18924.dyn.optonline.net> has quit IRC (Quit: peterrizzo_)
 89 2020-12-11T03:42:16  *** twistedline <twistedline!~twisted@unaffiliated/twistedline> has quit IRC (Read error: Connection reset by peer)
 90 2020-12-11T03:44:34  *** twistedline <twistedline!~twisted@2601:14d:8500:a77d:2985:36c3:6c1f:47b8> has joined #bitcoin-core-dev
 91 2020-12-11T04:01:11  *** justanotheruser <justanotheruser!~justanoth@unaffiliated/justanotheruser> has quit IRC (Ping timeout: 272 seconds)
 92 2020-12-11T04:02:45  *** k3tan <k3tan!~pi@gateway/tor-sasl/k3tan> has quit IRC (Remote host closed the connection)
 93 2020-12-11T04:03:16  *** k3tan <k3tan!~pi@gateway/tor-sasl/k3tan> has joined #bitcoin-core-dev
 94 2020-12-11T04:49:22  *** JackSparrow <JackSparrow!~JackSparr@s91904426.blix.com> has quit IRC (Remote host closed the connection)
 95 2020-12-11T05:09:17  *** gac410 <gac410!~gac410@> has joined #bitcoin-core-dev
 96 2020-12-11T05:33:50  *** ryanofsky <ryanofsky!russ@jumpy.yanofsky.org> has quit IRC (Quit: ZNC 1.7.5 - https://znc.in)
 97 2020-12-11T05:35:18  *** harding <harding!quassel@2600:3c03::f03c:91ff:fe7b:78d1> has quit IRC (Remote host closed the connection)
 98 2020-12-11T05:35:21  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)
 99 2020-12-11T05:36:03  *** ryanofsky <ryanofsky!~russ@jumpy.yanofsky.org> has joined #bitcoin-core-dev
100 2020-12-11T05:39:27  *** harding <harding!~quassel@newmail.dtrt.org> has joined #bitcoin-core-dev
101 2020-12-11T06:00:31  *** jb55 <jb55!~jb55@gateway/tor-sasl/jb55> has quit IRC (Remote host closed the connection)
102 2020-12-11T06:01:02  *** jb55 <jb55!~jb55@gateway/tor-sasl/jb55> has joined #bitcoin-core-dev
103 2020-12-11T06:05:38  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has joined #bitcoin-core-dev
104 2020-12-11T07:30:38  *** jeremyrubin <jeremyrubin!~jr@c-73-15-215-148.hsd1.ca.comcast.net> has quit IRC (Ping timeout: 260 seconds)
105 2020-12-11T07:38:38  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)
106 2020-12-11T07:40:03  *** virtu <virtu!~virtu@gateway/tor-sasl/virtu> has quit IRC (Ping timeout: 240 seconds)
107 2020-12-11T07:43:41  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has joined #bitcoin-core-dev
108 2020-12-11T07:46:40  *** virtu <virtu!~virtu@gateway/tor-sasl/virtu> has joined #bitcoin-core-dev
109 2020-12-11T08:17:19  <jonasschnelli> macOS gitian build on master seems to fail: https://bitcoin.jonasschnelli.ch/gitian/build/359
110 2020-12-11T08:17:38  <jonasschnelli> Fetching v2.1.1.tar.gz from https://github.com/al45tair/mac_alias/archive/ (stalls)
111 2020-12-11T08:33:16  <wumpus> achow101: I noticed that the sqlite wallet backend recreates the prepared queries for every batch, is there a specific reason for this? I guess it has to do with multi-threading?
112 2020-12-11T08:34:07  *** mj_node <mj_node!7a001982@> has joined #bitcoin-core-dev
113 2020-12-11T08:35:51  <mj_node> Folks, I was 95% done syncing my node, and unplugged my external SSD accidently, I tried to restart bitcoind, but it's resuming from block 0, can anyone help? I tried to re-builld, etc.. the [blocks] folder has all the raw blk.dat but I just cant get it to use them.
114 2020-12-11T08:36:37  <sipa> mj_node: run with -reindex
115 2020-12-11T08:37:24  <sipa> it'll still restart the validation from 0, but it won't redownload everything
116 2020-12-11T08:41:16  <wumpus> normally unless a really high dbcache value is used it shouldn't go back that far on a crash, though unplugging storage while running can cause all kinds of corruption so it's hard to say-in any case a reindex is all you can do, try to keep the device plugged this time :-)
117 2020-12-11T08:42:22  <mj_node> I tried that -reindex doesnt work, its started to re-download blocks, I launched again re-index it spent hours going through the blocks but still says "loadBlockIndexDB: last block file = 136", when I have over 2000 block files
118 2020-12-11T08:42:40  <wumpus> ok, wipe everything and start over then
119 2020-12-11T08:42:43  <mj_node> somehow I think block\index content is the problem
120 2020-12-11T08:42:53  <mj_node> Im on 1mbps conneciton ,ehehe
121 2020-12-11T08:42:54  <wumpus> it's corrupted beyond repair
122 2020-12-11T08:42:58  <mj_node> ouch
123 2020-12-11T08:43:17  <mj_node> just to understand, is it because the blk.dat files are unique/
124 2020-12-11T08:43:37  <mj_node> and custom to each ?
125 2020-12-11T08:44:58  <sipa> mj_node: presumably when it started redownloading it started overwriting the block files you already had
126 2020-12-11T08:45:33  <mj_node> sipa: yes exactly that's what happened, so even a command like 'reconsider block' doesnt work?
127 2020-12-11T08:45:34  <wumpus> not necessarily, they contain public information after all, though the blocks don't come in in sequental order so they'll be in different orders in the files on different nodes, the "block index" database contains pointers to where every block is
128 2020-12-11T08:46:02  <mj_node> @wumpus hmm so surely I should be able to rebuild the block index from my raw blocks on the SDD no?
129 2020-12-11T08:46:11  <wumpus> yes, that's what reindex does
130 2020-12-11T08:46:22  <sipa> mj_node: redownloading corrupted the blocks you already had
131 2020-12-11T08:46:31  <wumpus> but not if the first blocks were overwritten as sipa says
132 2020-12-11T08:46:54  <mj_node> got it -thanks guys, appreciate the explaination, another 20 days i guess.
133 2020-12-11T08:46:55  <sipa> so you have a few good blocks at the beginning, then overwritten garbage, and then all old good blocks
134 2020-12-11T08:47:05  <sipa> 20?!
135 2020-12-11T08:47:15  <mj_node> slow wifi....
136 2020-12-11T08:47:16  <sipa> what hardware is this?
137 2020-12-11T08:47:28  <sipa> ow
138 2020-12-11T08:47:38  <mj_node> ehehhehe, I'm in quarantine with shitty connection
139 2020-12-11T08:47:49  <sipa> if network is the bottleneck i can't help you
140 2020-12-11T08:48:15  <sipa> mj_node: here is something you couldnl try (it's a moonshot, though)
141 2020-12-11T08:48:17  <wumpus> you might want to copy the block files from someone else on physical storage
142 2020-12-11T08:48:38  <sipa> start downloading again in another directory
143 2020-12-11T08:48:58  <sipa> until you have the first few block files (as many as you possibly had overwritten)
144 2020-12-11T08:49:22  <wumpus> in any case, this isn't a support channel, unless you're doing development and having questions about the code for that reason this is not the place, use #bitcoin next time
145 2020-12-11T08:49:23  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has quit IRC (Ping timeout: 240 seconds)
146 2020-12-11T08:49:40  <sipa> then copy those over the same-named ones in your real dir
147 2020-12-11T08:49:46  <sipa> and then do a reindex
148 2020-12-11T08:49:52  <sipa> also, yeah, what wumpus said
149 2020-12-11T08:49:55  <mj_node> @wumpus sorry for that, and I will make sure to go to #bitcoin-core-dev
150 2020-12-11T08:50:02  <mj_node> #bitcoin i mean...
151 2020-12-11T08:50:02  *** Pavlenex <Pavlenex!~Thunderbi@> has joined #bitcoin-core-dev
152 2020-12-11T08:50:26  *** mj_node <mj_node!7a001982@> has left #bitcoin-core-dev
153 2020-12-11T08:57:08  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has joined #bitcoin-core-dev
154 2020-12-11T09:13:03  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)
155 2020-12-11T09:13:25  <wumpus> support for rescanning with missing block files, and in general for disjointed block directories and downloading only missing blocks would be an interesting feature, and likely required for more eleborate pruning strategies, but I think there's a few things prventing this from working right now
156 2020-12-11T09:20:27  <vasild> the way blocks and undo are stored now in blk and rev files would make this messy
157 2020-12-11T09:20:29  <vasild> put everything in sqlite!
158 2020-12-11T09:21:21  <wumpus> I don't think the storage format is the problem, I think this kind of corruption is easier to handle the simpler your data format is
159 2020-12-11T09:21:50  *** jeremyrubin <jeremyrubin!~jr@c-73-15-215-148.hsd1.ca.comcast.net> has joined #bitcoin-core-dev
160 2020-12-11T09:22:38  <wumpus> heh :)
161 2020-12-11T09:23:21  <wumpus> ideally it would get the longest headers chain from P2P *then* start reconstructing
162 2020-12-11T09:23:42  <wumpus> it's easier to puzzle what fits where then
163 2020-12-11T09:24:14  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
164 2020-12-11T09:24:15  <bitcoin-git> [bitcoin] MarcoFalke pushed 5 commits to master: https://github.com/bitcoin/bitcoin/compare/736eb4d80838...6a4806367177
165 2020-12-11T09:24:15  <bitcoin-git> bitcoin/master 91d6195 Suhas Daftuar: Simplify and clarify extra outbound peer counting
166 2020-12-11T09:24:16  <bitcoin-git> bitcoin/master 3cc8a7a Suhas Daftuar: Use conn_type to identify block-relay peers, rather than m_tx_relay == nul...
167 2020-12-11T09:24:16  <bitcoin-git> bitcoin/master daffaf0 Suhas Daftuar: Periodically make block-relay connections and sync headers
168 2020-12-11T09:24:18  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
169 2020-12-11T09:24:34  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
170 2020-12-11T09:24:34  <bitcoin-git> [bitcoin] MarcoFalke merged pull request #19858: Periodically make block-relay connections and sync headers (master...2020-08-blocks-only-rotation) https://github.com/bitcoin/bitcoin/pull/19858
171 2020-12-11T09:24:36  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
172 2020-12-11T09:30:14  <wumpus> instead of having 'import blocks' as a discrete initialization phase, consider the unplaced but known blocks already on disk as another block source like P2P (thinking of it I suppose it *almost* works that way already, after reindex-chainstate two-phase reindex process)
173 2020-12-11T09:31:23  <wumpus> it'd be a lot of un-fun fiddling around with broken block directories and such to develop and test this, doesen't sound like fun :)
174 2020-12-11T09:33:24  *** promag_ is now known as promag
175 2020-12-11T09:33:27  <promag> cfields: ping?
176 2020-12-11T09:40:24  <wumpus> at the least it'd need to rescan *all* `blk*` files, not stop when one is missing and delete the rest, e.g. get rid of this sledgehammer: https://github.com/bitcoin/bitcoin/blob/master/src/init.cpp#L658
177 2020-12-11T09:43:38  *** jeremyrubin <jeremyrubin!~jr@c-73-15-215-148.hsd1.ca.comcast.net> has quit IRC (Ping timeout: 260 seconds)
178 2020-12-11T09:46:13  *** AaronvanW <AaronvanW!~AaronvanW@unaffiliated/aaronvanw> has quit IRC (Remote host closed the connection)
179 2020-12-11T09:49:59  <gwillen> wumpus: one thing that occurred to me, when I had a corrupted block directory myself and was waiting to see if core could recover (it could not), is that you have to be slightly careful
180 2020-12-11T09:50:30  <gwillen> you don't want any risk of ending up in a situation where your block files on disk actually contain two slightly different copies of the same block
181 2020-12-11T09:50:42  <gwillen> and you checked one of them, but one or more indices ends up pointing at the other one
182 2020-12-11T09:51:08  <gwillen> this seems like the sort of thing that could slip in, by being more liberal in what one accepts from the block files
183 2020-12-11T09:52:23  <gwillen> (especially considered that it is Generally Regarded As Safe to grab this stuff from someone else to bootstrap, trusting that core will check it on startup before using it)
184 2020-12-11T09:53:03  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has quit IRC (Ping timeout: 240 seconds)
185 2020-12-11T09:54:59  <aj> isn't that what `-loadblock=<file>` is for?
186 2020-12-11T10:02:23  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
187 2020-12-11T10:02:24  <bitcoin-git> [bitcoin] Psyruss77 opened pull request #20623: Create msbuild.yml (master...master) https://github.com/bitcoin/bitcoin/pull/20623
188 2020-12-11T10:02:24  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
189 2020-12-11T10:03:09  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
190 2020-12-11T10:03:09  <bitcoin-git> [bitcoin] fanquake closed pull request #20623: Create msbuild.yml (master...master) https://github.com/bitcoin/bitcoin/pull/20623
191 2020-12-11T10:03:10  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
192 2020-12-11T10:05:08  *** kexkey_ <kexkey_!~kexkey@static-198-54-132-93.cust.tzulo.com> has quit IRC (Ping timeout: 260 seconds)
193 2020-12-11T10:12:52  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has joined #bitcoin-core-dev
194 2020-12-11T10:16:48  *** AaronvanW <AaronvanW!~AaronvanW@unaffiliated/aaronvanw> has joined #bitcoin-core-dev
195 2020-12-11T10:49:51  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
196 2020-12-11T10:49:51  <bitcoin-git> [bitcoin] jnewbery opened pull request #20624: net processing: Remove nStartingHeight check from block relay (master...2020-12-remove-starting-height) https://github.com/bitcoin/bitcoin/pull/20624
197 2020-12-11T10:49:52  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
198 2020-12-11T10:54:35  <aj> jnewbery: is the logic for 20624 right? we never go back into IBD without stopping and restarting, but could conceivably get more than two weeks behind (eg if we've got an unattended node whose network was down)
199 2020-12-11T10:59:46  <jnewbery> aj: potentially, yes. We could get out of IBD, lose connectivity and fall behind, and then start catching up again and relaying old blocks to peers. That's mostly also the case now (the 2000 blocks behind nStartingHeight won't stop us from relaying old blocks to peers if we've been connected to them for a while)
200 2020-12-11T11:00:53  <aj> jnewbery: presuming we were connected a while, they should be in the same state as us (either both current, or both out of date), so relaying is probably okay; if the network was down, when it came back up, we'd reconnect and choose new starting heights though?
201 2020-12-11T11:01:15  <jnewbery> I think the worst case is this: we get out of IBD, lose connectivity and fall behind, and then connect to new peers that are on the best tip. We start catching up and then relay headers to peers that are ahead of us. The worst case is 80 bytes for each header, and that peer wouldn't download the block.
202 2020-12-11T11:01:53  *** belcher <belcher!~belcher@unaffiliated/belcher> has joined #bitcoin-core-dev
203 2020-12-11T11:02:54  <aj> jnewbery: i guess my impression is that if this works okay if you fall behind, it should work okay if you're still in IBD?
204 2020-12-11T11:04:34  <jnewbery> the 'this' working ok being 'checking that you're within 2000 blocks of the peer's starting height'?
205 2020-12-11T11:05:13  <aj> jnewbery: jnewbery: "sending headers, relaying blocks" -- if it's okay to do when you've been out of IBD but are 2001 blocks behind, it should also be okay if you're in IBD?
206 2020-12-11T11:06:19  <aj> jnewbery: (the "?" there is doing a lot of work...)
207 2020-12-11T11:07:40  <jnewbery> maybe. I think it's best that we just avoid relaying any inventory when we're in IBD
208 2020-12-11T11:08:45  <aj> jnewbery: so i think the argument against doing it in IBD is you could be on a relatively low-work but high-height fork from genesis, and end up spamming lots of headers/blocks and waste lots of b/w before everyone realises there's a higher-work fork and switches to that
209 2020-12-11T11:10:48  <jnewbery> I
210 2020-12-11T11:11:17  <aj> jnewbery: doing a high-height fork from 2000 blocks ago is expensive, though; and maybe finishing IBD generally is good enough there -- worst case there's a long, low-work chain from a year or two ago, _and_ you've been running but disconnected for that time, but the number of nodes doing that are going to be small, even compared to the number of nodes syncing from genesis at any point in time?
211 2020-12-11T11:11:18  <jnewbery> I'm confused. Are you arguing against the change in 20624?
212 2020-12-11T11:11:36  <aj> jnewbery: no, i'm trying to understand it
213 2020-12-11T11:14:31  <jnewbery> I think worst worst case is you send headers from that long chain from a year or two ago. Two years of headers is ~8MB
214 2020-12-11T11:15:01  <sdaftuar> aj: in practice, if you're out of IBD, you'll be syncing headers with all your peers shortly after the connection is established
215 2020-12-11T11:15:32  <sdaftuar> aj: because the criteria for doing that lines up pretty closely (iirc). While in IBD, we purposely sync headers from only one peer to avoid duplication.
216 2020-12-11T11:16:10  <sdaftuar> so i think if we are announcing blocks to peers while catching up after coming back online (so IBD is false, but we're in fact behind), it's no big deal, because we ought to know our peers' header chain anyway
217 2020-12-11T11:16:21  <sdaftuar> and therefore won't actually be announcing to them, since we'll know they know the blocks already
218 2020-12-11T11:16:40  *** filchef <filchef!~filchef@> has joined #bitcoin-core-dev
219 2020-12-11T11:17:15  *** filchef <filchef!~filchef@> has quit IRC (Client Quit)
220 2020-12-11T11:18:14  <sdaftuar> i guess it's worth testing that there's not some slippage at the beginning of a connection, if they are slow to respond to our getheaders and we are connecting blocks, maybe we'd blast them with useless data?  not sure how likely that is
221 2020-12-11T11:18:36  *** Neoma33Crooks <Neoma33Crooks!~Neoma33Cr@static.> has joined #bitcoin-core-dev
222 2020-12-11T11:20:37  *** AaronvanW <AaronvanW!~AaronvanW@unaffiliated/aaronvanw> has quit IRC (Ping timeout: 264 seconds)
223 2020-12-11T11:20:51  *** provoostenator <provoostenator!~quassel@provoostenator.sprovoost.nl> has quit IRC (Remote host closed the connection)
224 2020-12-11T11:20:55  <jnewbery> sdaftuar: are you talking about after removing the nStartingHeight check? I think if we're out of IBD, we can't be connecting that many blocks?
225 2020-12-11T11:21:23  <sdaftuar> (just realized "while in ibd, we purposely sync headers from only one peer" is a bit imprecise -- what i meant was that we wait until we have completed header sync from one peer before syncing with others, so we could be connecting blocks whiel still doing header download, and not yet have tried syncing headers with our other peers)
226 2020-12-11T11:21:31  *** provoostenator <provoostenator!~quassel@provoostenator.sprovoost.nl> has joined #bitcoin-core-dev
227 2020-12-11T11:21:50  <sdaftuar> jnewbery: yes i'm talking about that check (well your PR), trying to explain the IBD vs nonIBD distinction better
228 2020-12-11T11:23:16  *** Neoma33Crooks <Neoma33Crooks!~Neoma33Cr@static.> has quit IRC (Ping timeout: 256 seconds)
229 2020-12-11T11:23:47  <sdaftuar> hm! the behavior around exactly how we starting syncing headers witha  peer is not quite what i remember.
230 2020-12-11T11:25:40  <sdaftuar> it still may not matter much, but it looks like if we fall behind a bunch (say because we lost our network for a while, and then it came back up) that we would only sync headers from one peer until our tip is close to current, before syncing with new ones
231 2020-12-11T11:28:01  <sdaftuar> anyway, i think that's a long-winded way of me thinking that not-relay in IBD is the right behavior, and relaying while not in IBD even if we happen to be a bit behind is probably ok.
232 2020-12-11T11:30:52  <jnewbery> I need to understand the "start block sync" and FindNextBlocksToDownload() logic better
233 2020-12-11T11:32:25  <sdaftuar> the basic idea is that when we start up, we pick a first peer to start syncing headers from. as soon as we have headers that indicate there is a tip >= work of our current tip, we start downloading towards it from any peer that has it.  also, once our headers chain is close to current (time within a day of current time), we sync headers from all peers.
234 2020-12-11T11:33:15  <sdaftuar> and then as those peers respond with their headers (which should be quick, if our headers chain is the correct one -- a single header with their best tip is typical) we'll download blocks from them as well, since we'll know they have the blocks we need.
235 2020-12-11T11:36:38  <jnewbery> That certainly makes sense conceptually. I just gind that mapping that design to the various bits of logic and state in SendMessages() and elsewhere is a bit tricky
236 2020-12-11T11:38:35  <aj> jnewbery: "gind" ?
237 2020-12-11T11:38:51  <aj> find
238 2020-12-11T11:39:22  <jnewbery> *find
239 2020-12-11T12:00:05  <michaelfolkson> I'm looking at bech32 https://bitcoin.stackexchange.com/questions/100508/can-you-break-down-what-data-is-encoded-into-a-bech32-address
240 2020-12-11T12:00:33  <michaelfolkson> bech32 addresses start with bc representing Bitcoin and then 1 which is just a separator
241 2020-12-11T12:00:52  <michaelfolkson> But regtest bech32 start bcrt
242 2020-12-11T12:01:11  <michaelfolkson> Does anyone know what the `rt` represent?
243 2020-12-11T12:01:35  <wumpus> regtest
244 2020-12-11T12:02:34  <michaelfolkson> Huh, that was easy. Why does regtest have a rt separator yet testnet and signet both start tb1 an don't need a separator?
245 2020-12-11T12:02:51  <wumpus> this comes from https://github.com/bitcoin/bitcoin/blob/master/src/chainparams.cpp#L446
246 2020-12-11T12:03:39  <aj> michaelfolkson: 1 is always the separator in bech32
247 2020-12-11T12:04:10  <wumpus> both testnet and signet start with 'tb' i don't know why it was chosen to use the same there, probably because they are both test networks and there can potentially already be many of them
248 2020-12-11T12:04:12  <michaelfolkson> Oh so regtest always starts bcrt1?
249 2020-12-11T12:04:29  <michaelfolkson> bcrt is human readable and 1 is the separator
250 2020-12-11T12:04:36  <wumpus> yes
251 2020-12-11T12:05:33  <michaelfolkson> Ok thanks
252 2020-12-11T12:06:21  <michaelfolkson> This is the context on why testnet and signet both start tb https://github.com/bitcoin/bitcoin/pull/18267#discussion_r491150895
253 2020-12-11T12:06:32  <wumpus> this is the best reference of course: https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki
254 2020-12-11T12:07:05  <wumpus> michaelfolkson: thanks for looking it up so it was as i guessed
255 2020-12-11T12:08:06  <michaelfolkson> Presumably non-default signets will be encouraged to not start tb
256 2020-12-11T12:08:27  <michaelfolkson> Although can't force them
257 2020-12-11T12:08:34  <aj> michaelfolkson: nah, that would require patching the code
258 2020-12-11T12:08:48  <michaelfolkson> Ohhh non-default signets will also start tb?
259 2020-12-11T12:08:57  <aj> michaelfolkson: there used to be config options for it, -signet_hrp= or so
260 2020-12-11T12:09:38  <wumpus> seems from that discussion that all new test networks start with tb, as it's for testing, address overlap is not critical
261 2020-12-11T12:09:52  <wumpus> that regtest has its own is a historical artifact then
262 2020-12-11T12:09:56  <aj> yeah, and makes it easier to port wallets to different testnets
263 2020-12-11T12:10:03  <wumpus> exactly
264 2020-12-11T12:10:15  *** peterrizzo <peterrizzo!~peterrizz@ool-44c18924.dyn.optonline.net> has joined #bitcoin-core-dev
265 2020-12-11T12:10:15  <aj> meanwhile regtest is kind-of bitcoin-core specific, and there's no point having wallets work with it
266 2020-12-11T12:10:47  *** mj_node <mj_node!~mj_node@> has joined #bitcoin-core-dev
267 2020-12-11T12:11:05  <michaelfolkson> Yeah treating regtest differently to testnet, signet makes sense as not meant to be a public network
268 2020-12-11T12:11:42  <michaelfolkson> Though choosing tb versus bcrt is a bit peculiar
269 2020-12-11T12:11:56  <aj> i noticed today that apparently when p2sh was being deployed, regtest didn't even exist
270 2020-12-11T12:12:04  <wumpus> the public signet is a public network
271 2020-12-11T12:12:34  <michaelfolkson> And presumably the non-default signets would also be public(ish) networks
272 2020-12-11T12:12:42  <michaelfolkson> Otherwise use regtest
273 2020-12-11T12:13:09  <wumpus> yes definitely easier to share
274 2020-12-11T12:15:07  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has joined #bitcoin-core-dev
275 2020-12-11T12:16:01  *** mj_node is now known as symbiosis
276 2020-12-11T12:17:01  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has quit IRC (Remote host closed the connection)
277 2020-12-11T12:17:16  *** gac410 <gac410!~gac410@> has quit IRC (Remote host closed the connection)
278 2020-12-11T12:17:48  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has joined #bitcoin-core-dev
279 2020-12-11T12:20:55  *** Victor_sueca <Victor_sueca!~Victorsue@unaffiliated/victorsueca> has joined #bitcoin-core-dev
280 2020-12-11T12:22:19  *** kristapsk <kristapsk!~KK@gateway/tor-sasl/kristapsk> has quit IRC (Remote host closed the connection)
281 2020-12-11T12:22:20  *** ghost43 <ghost43!~daer@gateway/tor-sasl/daer> has quit IRC (Read error: Connection reset by peer)
282 2020-12-11T12:22:21  *** kristapsk_ <kristapsk_!~KK@gateway/tor-sasl/kristapsk> has joined #bitcoin-core-dev
283 2020-12-11T12:22:29  *** ghost43_ <ghost43_!~daer@gateway/tor-sasl/daer> has joined #bitcoin-core-dev
284 2020-12-11T12:23:05  *** Victorsueca <Victorsueca!~Victorsue@unaffiliated/victorsueca> has quit IRC (Ping timeout: 240 seconds)
285 2020-12-11T12:29:36  *** Pavlenex <Pavlenex!~Thunderbi@> has quit IRC (Ping timeout: 240 seconds)
286 2020-12-11T12:37:01  <jonatack> "// Download if this is a nice peer, or we have no nice peers and this one might do."
287 2020-12-11T12:37:11  <jonatack> love some of these comments :)
288 2020-12-11T12:37:59  *** AmberJ_ <AmberJ_!~AmberJ_@> has joined #bitcoin-core-dev
289 2020-12-11T12:43:21  *** dr-orlovsky <dr-orlovsky!~dr-orlovs@> has joined #bitcoin-core-dev
290 2020-12-11T12:46:23  <wumpus> it's cute
291 2020-12-11T12:58:20  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)
292 2020-12-11T13:09:29  <michaelfolkson> What you looking at jonatack?
293 2020-12-11T13:10:31  <jonatack> michaelfolkson: grep the codebase ;)
294 2020-12-11T13:11:26  <michaelfolkson> So demanding of my typing fingers
295 2020-12-11T13:12:01  *** kali1 <kali1!~kali@i16-les01-ntr-31-36-36-40.sfr.lns.abo.bbox.fr> has joined #bitcoin-core-dev
296 2020-12-11T13:12:29  *** kali1 <kali1!~kali@i16-les01-ntr-31-36-36-40.sfr.lns.abo.bbox.fr> has left #bitcoin-core-dev
297 2020-12-11T13:17:37  <jonatack> you'd be forgiven for thinking that line was from a jane austen novel instead
298 2020-12-11T13:21:16  <aj> "It is a truth universally acknowledged, that a high-bandwidth archive node must be in want of an inbound connection" ?
299 2020-12-11T13:24:03  *** symbiosis <symbiosis!~mj_node@> has quit IRC (Quit: Leaving)
300 2020-12-11T13:25:36  <vasild> "uhoh, different"
301 2020-12-11T13:30:37  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has joined #bitcoin-core-dev
302 2020-12-11T13:34:49  *** Guyver2 <Guyver2!Guyver@guyver2.xs4all.nl> has joined #bitcoin-core-dev
303 2020-12-11T13:35:44  *** vasild_ <vasild_!~vd@gateway/tor-sasl/vasild> has joined #bitcoin-core-dev
304 2020-12-11T13:35:45  *** vasild <vasild!~vd@gateway/tor-sasl/vasild> has quit IRC (Disconnected by services)
305 2020-12-11T13:35:45  *** vasild_ is now known as vasild
306 2020-12-11T13:39:11  *** vincenzopalazzo <vincenzopalazzo!~vincent@2001:b07:6474:9d49:849d:db24:7f93:fb8a> has joined #bitcoin-core-dev
307 2020-12-11T13:40:02  *** mol_ <mol_!~mol@unaffiliated/molly> has joined #bitcoin-core-dev
308 2020-12-11T13:40:05  *** owowo <owowo!~ovovo@unaffiliated/ovovo> has quit IRC (Ping timeout: 240 seconds)
309 2020-12-11T13:42:41  *** molz_ <molz_!~mol@unaffiliated/molly> has quit IRC (Ping timeout: 258 seconds)
310 2020-12-11T13:44:55  *** owowo <owowo!~ovovo@> has joined #bitcoin-core-dev
311 2020-12-11T13:46:50  *** da39a3ee5e6b4b0d <da39a3ee5e6b4b0d!~da39a3ee5@2403:6200:8876:bbcd:f950:50c:999:2b9d> has quit IRC (Ping timeout: 264 seconds)
312 2020-12-11T13:49:58  *** peterrizzo <peterrizzo!~peterrizz@ool-44c18924.dyn.optonline.net> has quit IRC (Quit: peterrizzo)
313 2020-12-11T14:03:26  *** peterrizzo_ <peterrizzo_!~peterrizz@ool-44c18924.dyn.optonline.net> has joined #bitcoin-core-dev
314 2020-12-11T14:10:39  *** Pavlenex <Pavlenex!~Thunderbi@> has joined #bitcoin-core-dev
315 2020-12-11T14:24:23  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has quit IRC (Ping timeout: 240 seconds)
316 2020-12-11T14:34:48  *** Pavlenex1 <Pavlenex1!~Thunderbi@> has joined #bitcoin-core-dev
317 2020-12-11T14:36:26  *** davterra <davterra!~davterra@> has joined #bitcoin-core-dev
318 2020-12-11T14:37:25  *** Pavlenex <Pavlenex!~Thunderbi@> has quit IRC (Ping timeout: 264 seconds)
319 2020-12-11T14:37:26  *** Pavlenex1 is now known as Pavlenex
320 2020-12-11T14:54:45  <wumpus> jonasschnelli: strange, you have a mismatch for the gitian macos signed build
321 2020-12-11T14:55:57  <wumpus> but the unsigned one matches
322 2020-12-11T15:00:26  *** miketwenty1 <miketwenty1!~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com> has joined #bitcoin-core-dev
323 2020-12-11T15:07:14  <jonasschnelli> wumpus: that’s really strange.
324 2020-12-11T15:07:21  <jonasschnelli> Let me do it again
325 2020-12-11T15:11:47  *** AaronvanW <AaronvanW!~AaronvanW@unaffiliated/aaronvanw> has joined #bitcoin-core-dev
326 2020-12-11T15:14:14  *** sr_gi <sr_gi!~sr_gi@> has quit IRC (Read error: Connection reset by peer)
327 2020-12-11T15:14:39  *** sr_gi <sr_gi!~sr_gi@> has joined #bitcoin-core-dev
328 2020-12-11T15:15:57  <wumpus> it shouldn't even be possible, a difference would invalidate the code-signing right?
329 2020-12-11T15:16:07  <wumpus> or is there scope for malleability
330 2020-12-11T15:18:11  *** Pavlenex1 <Pavlenex1!~Thunderbi@> has joined #bitcoin-core-dev
331 2020-12-11T15:18:54  <jonasschnelli> The only reason I could think of is that I have built it before the sigs where pushed (then it would have took the rc2 detached sig).
332 2020-12-11T15:19:18  <jonasschnelli> But I very much doubt that I did this
333 2020-12-11T15:20:37  *** Pavlenex <Pavlenex!~Thunderbi@> has quit IRC (Ping timeout: 265 seconds)
334 2020-12-11T15:20:37  *** Pavlenex1 is now known as Pavlenex
335 2020-12-11T15:22:21  <wumpus> it doesn't verify what it is attaching?
336 2020-12-11T15:25:21  *** Pavlenex <Pavlenex!~Thunderbi@> has quit IRC (Quit: Pavlenex)
337 2020-12-11T15:27:40  <jonasschnelli> I don’t think so. It just takes the newest signature from the 0.21 branch (signature repository)
338 2020-12-11T15:28:06  <jonasschnelli> A check against the release/tag should probably be added.
339 2020-12-11T15:28:30  <jonasschnelli> I investigate as soon as when I’m back on my desk
340 2020-12-11T15:28:47  <wumpus> I mean a check that the signature checks out against what it's attached to
341 2020-12-11T15:29:23  <wumpus> the windows one has a check like that IIRC
342 2020-12-11T15:31:13  <jonasschnelli> wumpus: that’s not the case AFAIK. I don’t know if you can verify the signature on Linux. Probably possible but maybe complicated to add.
343 2020-12-11T15:31:45  <wumpus> right, thanks
344 2020-12-11T15:33:49  *** Pavlenex <Pavlenex!~Thunderbi@> has joined #bitcoin-core-dev
345 2020-12-11T15:38:36  *** mol_ <mol_!~mol@unaffiliated/molly> has quit IRC (Ping timeout: 240 seconds)
346 2020-12-11T15:46:46  *** Pavlenex <Pavlenex!~Thunderbi@> has quit IRC (Quit: Pavlenex)
347 2020-12-11T15:48:49  <luke-jr> is the CI failing to merge with master before running? or did we lose a bunch of Cirrus instances?
348 2020-12-11T15:49:17  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
349 2020-12-11T15:49:17  <bitcoin-git> [bitcoin] ryanofsky closed pull request #19195: wallet: ScanForWalletTransactions cleanup (master...2020-06-06-scanforwallettransactions-cleanup) https://github.com/bitcoin/bitcoin/pull/19195
350 2020-12-11T15:49:18  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
351 2020-12-11T15:50:12  <luke-jr> also does this error make sense to anyone? only happens on bitcoinbuilds :/ https://bitcoinbuilds.org/index.php?ansilog=98434a62-d0b5-4596-bfde-5e5504998bc3.log
352 2020-12-11T15:53:10  *** belcher <belcher!~belcher@unaffiliated/belcher> has quit IRC (Ping timeout: 272 seconds)
353 2020-12-11T15:54:17  <wumpus> luke-jr: it's passing two arguments to a function that takes one?
354 2020-12-11T15:54:33  <wumpus> e.g. "CreateChainParams(gArgs, gArgs.GetChainName());"
355 2020-12-11T15:54:35  *** peterrizzo_ <peterrizzo_!~peterrizz@ool-44c18924.dyn.optonline.net> has quit IRC (Read error: Connection reset by peer)
356 2020-12-11T15:54:57  *** peterrizzo_ <peterrizzo_!~peterrizz@ool-44c18924.dyn.optonline.net> has joined #bitcoin-core-dev
357 2020-12-11T15:55:10  *** mol <mol!~mol@unaffiliated/molly> has joined #bitcoin-core-dev
358 2020-12-11T15:55:31  *** miketwen_ <miketwen_!~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com> has joined #bitcoin-core-dev
359 2020-12-11T15:56:54  *** reallll <reallll!~belcher@unaffiliated/belcher> has joined #bitcoin-core-dev
360 2020-12-11T15:58:45  *** miketwenty1 <miketwenty1!~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com> has quit IRC (Ping timeout: 240 seconds)
361 2020-12-11T16:00:45  *** reallll is now known as belcher
362 2020-12-11T16:00:58  <wumpus> oh the two-arg version is correct in master, is this some rebase/merge problem maybe?
363 2020-12-11T16:01:25  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has joined #bitcoin-core-dev
364 2020-12-11T16:01:53  <wumpus> as somehow it ends up with an old chainparams.h with single-argument CreateChainParams
365 2020-12-11T16:13:27  *** Pavlenex <Pavlenex!~Thunderbi@> has joined #bitcoin-core-dev
366 2020-12-11T16:22:21  *** Pavlenex <Pavlenex!~Thunderbi@> has quit IRC (Quit: Pavlenex)
367 2020-12-11T16:25:05  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has quit IRC (Remote host closed the connection)
368 2020-12-11T16:27:47  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has joined #bitcoin-core-dev
369 2020-12-11T16:34:07  *** Bob_ <Bob_!~bob@> has joined #bitcoin-core-dev
370 2020-12-11T16:35:01  <jonasschnelli> wumpus: I did the build again and getting now the “correct” hash.
371 2020-12-11T16:35:09  <jonasschnelli> New build: https://bitcoin.jonasschnelli.ch/gitian/build/360
372 2020-12-11T16:35:40  <jonasschnelli> Old build (the one I pushed to gitian.signs): https://bitcoin.jonasschnelli.ch/gitian/build/358
373 2020-12-11T16:37:01  <wumpus> jonasschnelli: happy to hear that! time to upload the rc binaries then
374 2020-12-11T16:37:01  <luke-jr> wumpus: I see. So that confirms a CI issue too >_<
375 2020-12-11T16:37:05  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has quit IRC (Quit: = "")
376 2020-12-11T16:37:14  <luke-jr> jonasschnelli: bitcoinbuilds is apparently not merging master before running CI
377 2020-12-11T16:39:54  <jonasschnelli> luke-jr: AFAIK is does
378 2020-12-11T16:40:27  <jonasschnelli> Someone, my old (invalid) build has different bitcoin-osx-unsigned.tar.gz as input
379 2020-12-11T16:40:30  <jonasschnelli> I don't know why
380 2020-12-11T16:40:43  <jonasschnelli> (old invalid build): https://bitcoin.jonasschnelli.ch/gitian/builds/360/bitcoin-dmg-signer-build.assert
381 2020-12-11T16:41:25  <jonasschnelli> https://bitcoin.jonasschnelli.ch/gitian/builds/358/bitcoin-dmg-signer-build.assert
382 2020-12-11T16:41:42  <jonasschnelli> very strange
383 2020-12-11T16:42:19  <luke-jr> jonasschnelli: see https://bitcoinbuilds.org/index.php?ansilog=98434a62-d0b5-4596-bfde-5e5504998bc3.log
384 2020-12-11T16:42:36  <jonasschnelli> I was just checking and came to the same conclusion luke-jr... hmm..
385 2020-12-11T16:44:09  <luke-jr> jonasschnelli: if I fix my PR will it mess up your ability to troubleshoot?
386 2020-12-11T16:44:17  <jonasschnelli> luke-jr: your right... so yes. bitcoinbuilds currently builds PR branches as they are (not merged on master)
387 2020-12-11T16:44:41  <jonasschnelli> luke-jr: no. I'll know how to fix it.
388 2020-12-11T16:44:43  <luke-jr> k
389 2020-12-11T16:48:27  *** Bob_ <Bob_!~bob@> has left #bitcoin-core-dev
390 2020-12-11T16:49:43  *** Pavlenex <Pavlenex!~Thunderbi@> has joined #bitcoin-core-dev
391 2020-12-11T16:49:56  *** Pavlenex <Pavlenex!~Thunderbi@> has quit IRC (Client Quit)
392 2020-12-11T16:50:38  *** kljasdfvv <kljasdfvv!~flack@p200300d46f24de007f9b1b51e45d0773.dip0.t-ipconnect.de> has quit IRC (Quit: Konversation terminated!)
393 2020-12-11T16:52:21  *** jeremyrubin <jeremyrubin!~jr@c-73-15-215-148.hsd1.ca.comcast.net> has joined #bitcoin-core-dev
394 2020-12-11T16:54:16  *** Barno7 <Barno7!~bob@> has joined #bitcoin-core-dev
395 2020-12-11T16:55:06  *** harrigan <harrigan!~harrigan@ptr-93-89-242-235.ip.airwire.ie> has quit IRC (Ping timeout: 256 seconds)
396 2020-12-11T16:55:13  <sipa> wumpus: treating the block data as an import sounds like a good idea, and not too hard
397 2020-12-11T17:02:25  <achow101> wumpus: yes, the prepared statement for each batch is so that we don't have issues where there are multiple batches for the same database. Although I'm pretty sure that we never have multiple batches writing to the same db at the same time anyeays.
398 2020-12-11T17:03:37  <wumpus> sipa: great!
399 2020-12-11T17:04:06  *** mj_node <mj_node!~mj_node@> has joined #bitcoin-core-dev
400 2020-12-11T17:04:41  <wumpus> achow101: right, thanks
401 2020-12-11T17:06:30  *** potato <potato!~Thunderbi@240d:1a:3d4:7d00:6195:91a4:15a2:31e8> has joined #bitcoin-core-dev
402 2020-12-11T17:11:26  *** Pavlenex <Pavlenex!~Thunderbi@> has joined #bitcoin-core-dev
403 2020-12-11T17:13:59  *** alko89 <alko89!~alko89@unaffiliated/alko89> has quit IRC (Quit: ZNC 1.7.5 - https://znc.in)
404 2020-12-11T17:14:42  *** alko89 <alko89!~alko89@unaffiliated/alko89> has joined #bitcoin-core-dev
405 2020-12-11T17:28:59  *** wumpus <wumpus!~ircclient@pdpc/supporter/professional/wumpus> has quit IRC (Quit: freebsd 12.2 update)
406 2020-12-11T17:31:29  *** Pavlenex <Pavlenex!~Thunderbi@> has quit IRC (Quit: Pavlenex)
407 2020-12-11T17:34:19  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has joined #bitcoin-core-dev
408 2020-12-11T17:38:08  *** wumpus <wumpus!~ircclient@pdpc/supporter/professional/wumpus> has joined #bitcoin-core-dev
409 2020-12-11T17:38:27  <wumpus> rc3 binaries up https://bitcoincore.org/bin/bitcoin-core-0.21.0/test.rc3/
410 2020-12-11T17:57:36  <achow101> for some reason gverify is giving a mismatch for darosior's 0.20.2rc1 win and osx builds, but manual inspection does not show a difference
411 2020-12-11T17:57:58  <luke-jr> O.o
412 2020-12-11T18:00:09  <luke-jr> achow101: where do you see it?
413 2020-12-11T18:00:28  <achow101> locally
414 2020-12-11T18:01:04  <luke-jr> I also see no difference
415 2020-12-11T18:01:25  <roconnor> does gverify look at filename dates or other file attributes?
416 2020-12-11T18:02:00  <darosior> Hmm, gverify passes on my end
417 2020-12-11T18:02:16  <luke-jr> achow101: could the signature be invalid/rejected for some reason?
418 2020-12-11T18:02:27  <achow101> luke-jr: i don't think so
419 2020-12-11T18:02:33  <darosior> Oh no, it does not
420 2020-12-11T18:02:46  <darosior> Not for Windows, good catch..
421 2020-12-11T18:03:00  <achow101> roconnor: iirc it expects the same filenames
422 2020-12-11T18:03:32  <luke-jr> oh, found it
423 2020-12-11T18:03:37  *** samuel-pedraza <samuel-pedraza!a5169307@gateway/web/cgi-irc/kiwiirc.com/ip.> has joined #bitcoin-core-dev
424 2020-12-11T18:03:42  <luke-jr> - release: v0.20.2rc1-win-unsigned
425 2020-12-11T18:03:47  <luke-jr> I bet tha'ts it
426 2020-12-11T18:04:03  *** vasild <vasild!~vd@gateway/tor-sasl/vasild> has quit IRC (Ping timeout: 240 seconds)
427 2020-12-11T18:04:09  <achow101> ugh
428 2020-12-11T18:04:20  <achow101> ok that doesn't matter
429 2020-12-11T18:04:39  *** vasild <vasild!~vd@gateway/tor-sasl/vasild> has joined #bitcoin-core-dev
430 2020-12-11T18:05:02  <prusnak> Big Sur says Bitcoin Core installed from bitcoin-0.21.0rc3-osx.dmg is broken and should be moved to the bin; that said this happens on M1 system, so maybe on x86-64 it's fine
431 2020-12-11T18:05:13  <prusnak> maybe jonasschnelli can confirm? ^
432 2020-12-11T18:05:33  <achow101> prusnak: is it the notarization warning?
433 2020-12-11T18:05:57  <prusnak> no, it's different than the error with rc2 (which i assume was notarization error)
434 2020-12-11T18:07:11  <luke-jr> where did the dmg come from?
435 2020-12-11T18:07:23  <luke-jr> could we have ended up with a signature that doesn't match a non-deterministic bin?
436 2020-12-11T18:08:28  <dongcarl> Just tested on x86_64, same error
437 2020-12-11T18:08:50  <luke-jr> :/
438 2020-12-11T18:09:02  <luke-jr> what about the unsigned dmg?
439 2020-12-11T18:09:02  <achow101> any more detail on that error?
440 2020-12-11T18:10:48  <dongcarl> https://nextcloud.carl.homeserver.net/s/WXdFHLwwTygXZW4
441 2020-12-11T18:11:33  <luke-jr> dongcarl: what is that and why does it demand I run JS?
442 2020-12-11T18:12:04  <dongcarl> Direct link: https://nextcloud.carl.homeserver.net/s/WXdFHLwwTygXZW4/preview
443 2020-12-11T18:12:43  <dongcarl> It's just the screenshot of the error when I open from the DMG (didn't drag to Applications)
444 2020-12-11T18:12:49  <luke-jr> does that question mark on the bottom left tell anything more?
445 2020-12-11T18:13:27  <achow101> hmm, damaged?
446 2020-12-11T18:13:47  <sipa> damaged could mean the signature check fails, i guess
447 2020-12-11T18:13:50  <dongcarl> https://nextcloud.carl.homeserver.net/s/7YTDtmrgaqPCYde/preview
448 2020-12-11T18:14:01  <dongcarl> code signing went wrong
449 2020-12-11T18:14:05  <luke-jr> sounds like it
450 2020-12-11T18:14:11  <luke-jr> dongcarl: can you verify the unsigned DMG is okay?
451 2020-12-11T18:14:24  <luke-jr> ping jonasschnelli
452 2020-12-11T18:14:48  <dongcarl> luke-jr: that's not uploaded yet I don't think
453 2020-12-11T18:15:08  <dongcarl> Happy to test if anyone has one
454 2020-12-11T18:15:15  <achow101> dongcarl: uploading it soon
455 2020-12-11T18:17:02  <achow101> https://github.com/achow101/bitcoin/releases/tag/v0.21.0rc3
456 2020-12-11T18:17:45  *** samuel-pedraza <samuel-pedraza!a5169307@gateway/web/cgi-irc/kiwiirc.com/ip.> has quit IRC (Quit: Connection closed)
457 2020-12-11T18:18:36  <dongcarl> achow101: works
458 2020-12-11T18:19:21  <prusnak> same here (on m1 system)
459 2020-12-11T18:20:02  <luke-jr> sounds like jonasschnelli signed the wrong file, or committed the wrong sig
460 2020-12-11T18:20:19  <luke-jr> or our combining process is suddnely broke
461 2020-12-11T18:21:20  <achow101> i think he signed the wrong file
462 2020-12-11T18:23:33  <roconnor> maybe I should keep my nose out of this, but what are the consequences of signing the wrong thing?  AFAIK publish signatures are unrevokable.
463 2020-12-11T18:23:43  <roconnor> *published
464 2020-12-11T18:23:55  <dongcarl> who does the windows codesigning?
465 2020-12-11T18:23:58  <achow101> me
466 2020-12-11T18:24:09  <dongcarl> roconnor: You might be thinking of notarization instead of codesigning?
467 2020-12-11T18:24:23  <roconnor> maybe.  What's the difference?
468 2020-12-11T18:24:35  <achow101> roconnor: in theory there shouldn't be any consequences because code signing is only done to shut up os warnings.
469 2020-12-11T18:24:46  <roconnor> ah.
470 2020-12-11T18:24:56  <achow101> but in practice, it both doesn't do that anymore, and it does convey some level of "this software is trusted" because users don't quite understand why we do that
471 2020-12-11T18:25:22  <luke-jr> maybe we should just stop signing then?
472 2020-12-11T18:25:24  <achow101> the consequence is that the software that is signed is malicious and some user is tricked into signing it
473 2020-12-11T18:25:39  <achow101> *using it
474 2020-12-11T18:25:53  <roconnor> maybe these code signatures are revokable?
475 2020-12-11T18:26:01  <luke-jr> roconnor: these days, to *actually* shut macOS up, you have to opt in to Apple privacy violations (it tells them every time the app is opened, and which app it was)
476 2020-12-11T18:26:13  <achow101> luke-jr: code signing is required for notarization, and notarization is now the thing to make macOS not give a warning
477 2020-12-11T18:26:34  <luke-jr> achow101: but notarization is problematic, and we don't do it
478 2020-12-11T18:26:36  *** jonatack <jonatack!~jon@> has quit IRC (Ping timeout: 240 seconds)
479 2020-12-11T18:26:47  <luke-jr> so if the signature-alone is useless, why bother?
480 2020-12-11T18:26:59  <achow101> iirc there's 2 levels of warnings, with signature alone being slightly less aggressive?
481 2020-12-11T18:27:06  <achow101> and at least it still works on older macOS
482 2020-12-11T18:27:17  <luke-jr> older macOS that we dropped support for? :P
483 2020-12-11T18:27:55  <dongcarl> luke-jr: Without signing, the warning is much scarier... something like "unknown developer"
484 2020-12-11T18:29:28  <achow101> I think there's an older macOS we support that doesn't do the notarization check
485 2020-12-11T18:30:11  <luke-jr> anyway, jonas can prob fix this easily
486 2020-12-11T18:30:13  *** harrigan <harrigan!~harrigan@ptr-93-89-242-235.ip.airwire.ie> has joined #bitcoin-core-dev
487 2020-12-11T18:34:05  *** kexkey <kexkey!~kexkey@static-198-54-132-158.cust.tzulo.com> has joined #bitcoin-core-dev
488 2020-12-11T18:45:08  *** jb55 <jb55!~jb55@gateway/tor-sasl/jb55> has quit IRC (Remote host closed the connection)
489 2020-12-11T18:46:01  *** jb55 <jb55!~jb55@gateway/tor-sasl/jb55> has joined #bitcoin-core-dev
490 2020-12-11T18:46:43  *** troygiorshev <troygiorshev!~troygiors@d67-193-140-136.home3.cgocable.net> has quit IRC (Ping timeout: 260 seconds)
491 2020-12-11T18:50:21  *** jonatack <jonatack!~jon@> has joined #bitcoin-core-dev
492 2020-12-11T18:58:24  *** troygiorshev <troygiorshev!~troygiors@d67-193-140-136.home3.cgocable.net> has joined #bitcoin-core-dev
493 2020-12-11T19:03:09  *** jonatack <jonatack!~jon@> has quit IRC (Read error: Connection reset by peer)
494 2020-12-11T19:08:03  <jonasschnelli> sorry.. was afk.
495 2020-12-11T19:08:06  <jonasschnelli> Reading backlog
496 2020-12-11T19:09:08  <jonasschnelli> I codesigned 5e3a08ae8195190d6f1b12e3e1e9d710e7ad385941a6e8d04e3391f12deddb11  bitcoin-0.21.0rc3-osx-unsigned.tar.gz
497 2020-12-11T19:10:10  *** chri_eb <chri_eb!~chris@gateway/tor-sasl/chrieb/x-28824719> has quit IRC (Quit: chri_eb)
498 2020-12-11T19:13:21  <jonasschnelli> achow101, luke-jr: I just checked the DMG is built initially and it works, signature is correct
499 2020-12-11T19:13:36  <jonasschnelli> I got 998dddf3c0f9b568fc0c39e61e3d61d2843dfb968016b7ceaf23aca94ace2542  bitcoin-osx-signed.dmg as the only one
500 2020-12-11T19:15:23  *** yanmaani <yanmaani!~yanmaani@gateway/tor-sasl/yanmaani> has quit IRC (Ping timeout: 240 seconds)
501 2020-12-11T19:15:45  <jonasschnelli> But yes,... 46cfa036d365d69db2a3b78377621d6b214f2d78f3082f9c7ebd7a9b89cfc599  bitcoin-osx-signed.dmg has an invalid code signature
502 2020-12-11T19:15:47  *** yanmaani <yanmaani!~yanmaani@gateway/tor-sasl/yanmaani> has joined #bitcoin-core-dev
503 2020-12-11T19:16:06  <jonasschnelli> but I signed 5e3a08ae8195190d6f1b12e3e1e9d710e7ad385941a6e8d04e3391f12deddb11  bitcoin-0.21.0rc3-osx-unsigned.tar.gz
504 2020-12-11T19:16:08  <jonasschnelli> let me try agai
505 2020-12-11T19:17:48  *** justanotheruser <justanotheruser!~justanoth@unaffiliated/justanotheruser> has joined #bitcoin-core-dev
506 2020-12-11T19:20:26  <jonasschnelli> I double checked and I can confirm that I have signed 5e3a08ae8195190d6f1b12e3e1e9d710e7ad385941a6e8d04e3391f12deddb11  bitcoin-0.21.0rc3-osx-unsigned.tar.gz
507 2020-12-11T19:20:44  <achow101> is signing deterministic?
508 2020-12-11T19:20:46  <jonasschnelli> The signature is not deterministic,... doing it again gives me a different file/hash
509 2020-12-11T19:20:58  <jonasschnelli> I can't verify what went wrong
510 2020-12-11T19:22:06  <jonasschnelli> I have verified the signed 5e3a08ae8195190d6f1b12e3e1e9d710e7ad385941a6e8d04e3391f12deddb11  bitcoin-0.21.0rc3-osx-unsigned.tar.gz and it works
511 2020-12-11T19:22:46  <jonasschnelli> I don't know what to do,... I can sign and push again. But I rather know why it happend
512 2020-12-11T19:23:29  <achow101> How can you check what was signed?
513 2020-12-11T19:23:55  <jonasschnelli> I keep the files
514 2020-12-11T19:24:44  <achow101> ah
515 2020-12-11T19:25:20  <achow101> I guess for an rc it's fine to leave it, but in the future we should test the signed binary first
516 2020-12-11T19:26:30  *** jonatack <jonatack!~jon@> has joined #bitcoin-core-dev
517 2020-12-11T19:26:46  <wumpus> we'll correct it for the next rc (or final) I guess, it's too bad the sig verification can't run in linux
518 2020-12-11T19:26:58  <achow101> we can't verify the signature after signing?
519 2020-12-11T19:27:49  <sipa> we have a linux based signing tool, right?
520 2020-12-11T19:28:13  <achow101> only for windows afaik
521 2020-12-11T19:28:14  <sipa> i'd be surprised if it doesn't have most of the code needed for verification too
522 2020-12-11T19:28:20  <sipa> oh
523 2020-12-11T19:28:55  <achow101> for windows we can (and do I think) verify the sig after signing
524 2020-12-11T19:30:01  <sipa> so the osx signature is created on osx, but stapled onto the binary in gitian/linux ?
525 2020-12-11T19:30:07  <sipa> *macos
526 2020-12-11T19:31:27  <achow101> yes
527 2020-12-11T19:33:02  <luke-jr> jonasschnelli: can you upload or compare 998dddf3c0f9b568fc0c39e61e3d61d2843dfb968016b7ceaf23aca94ace2542 with 46cfa036d365d69db2a3b78377621d6b214f2d78f3082f9c7ebd7a9b89cfc599 ?
528 2020-12-11T19:33:15  <luke-jr> I guess our combiner is malfunctioning?
529 2020-12-11T19:33:34  *** luke <luke!~luke@bitnomial/staff/luke> has joined #bitcoin-core-dev
530 2020-12-11T19:34:41  <jonasschnelli> luke-jr: 46cfa036d365d69db2a3b78377621d6b214f2d78f3082f9c7ebd7a9b89cfc599  is here -> https://bitcoin.jonasschnelli.ch/gitian/build/360
531 2020-12-11T19:39:19  <jonasschnelli> luke-jr: I don't have 46cfa036d365d69db2a3b78377621d6b214f2d78f3082f9c7ebd7a9b89cfc599
532 2020-12-11T19:39:43  <jonasschnelli> my first gitian build went somehow wrong: https://bitcoin.jonasschnelli.ch/gitian/build/358
533 2020-12-11T19:40:18  <jonasschnelli> the build log from the signer was actually a full macOS unsigned build: https://bitcoin.jonasschnelli.ch/gitian/builds/358/build_osx_signer.log (for whatever reason)
534 2020-12-11T19:41:11  <jonasschnelli> I think the dmg hash mismatch has nothing to do with the invalid signature
535 2020-12-11T19:46:25  <jonasschnelli> hmm...
536 2020-12-11T19:47:04  <jonasschnelli> I think what I need to do before pushing the macOS signatures is doing a gbuild with gitian-descriptors/gitian-osx-signer.yml with the detached signature
537 2020-12-11T19:47:13  <jonasschnelli> as sort of a dbl-check
538 2020-12-11T19:51:55  <luke-jr> would still be nice to figure out why it didn't work this time :x
539 2020-12-11T19:53:51  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
540 2020-12-11T19:53:51  <bitcoin-git> [bitcoin] vova557 opened pull request #20628: Владелец (master...patch-10) https://github.com/bitcoin/bitcoin/pull/20628
541 2020-12-11T19:53:52  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
542 2020-12-11T19:54:25  *** greypw <greypw!~greypw@unaffiliated/greypw> has quit IRC (Quit: I'll be back!)
543 2020-12-11T19:54:35  <jonasschnelli> luke-jr: Indeed. I'm still trying to figure that out.
544 2020-12-11T19:54:49  *** greypw <greypw!~greypw@unaffiliated/greypw> has joined #bitcoin-core-dev
545 2020-12-11T20:02:19  *** davec <davec!~davec@072-183-054-196.res.spectrum.com> has quit IRC (Quit: leaving)
546 2020-12-11T20:03:45  *** miketwen_ <miketwen_!~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com> has quit IRC (Ping timeout: 240 seconds)
547 2020-12-11T20:11:34  <jonasschnelli> one of the main problem is, that I currently can't test the detached signature before pushing it to the git repository
548 2020-12-11T20:11:52  <jonasschnelli> I try now to modify the gitian descriptor to allow that
549 2020-12-11T20:12:55  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
550 2020-12-11T20:12:55  <bitcoin-git> [bitcoin] jonasschnelli closed pull request #20628: Владелец (master...patch-10) https://github.com/bitcoin/bitcoin/pull/20628
551 2020-12-11T20:12:56  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
552 2020-12-11T20:13:07  <achow101> jonasschnelli: you can put it in gitian-builder/inputs/signature
553 2020-12-11T20:13:11  <achow101> and make a local tag
554 2020-12-11T20:13:51  <jonasschnelli> and set --commit signature=<tag> to that custom tag?
555 2020-12-11T20:14:17  <achow101> i just set it to the actual version name
556 2020-12-11T20:14:30  <achow101> and delete it later for the actual build
557 2020-12-11T20:15:03  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has quit IRC (Ping timeout: 240 seconds)
558 2020-12-11T20:15:04  <jonasschnelli> but AFAIK the signer always pulls https://github.com/bitcoin-core/bitcoin-detached-sigs.git?
559 2020-12-11T20:15:27  <achow101> it pulls but it shouldn't do anything
560 2020-12-11T20:15:29  <jonasschnelli> and overwrites the git?
561 2020-12-11T20:15:50  <achow101> it doesn't overwrite
562 2020-12-11T20:15:54  <jonasschnelli> ah.
563 2020-12-11T20:16:05  <jonasschnelli> I added now a part to the descriptor that allows putting a signatures.tar.gz in inputs
564 2020-12-11T20:16:11  <jonasschnelli> and if there,... it'll take it
565 2020-12-11T20:17:50  <luke-jr> jonasschnelli: -u bitcoin=/path/to/local/bitcoin
566 2020-12-11T20:18:03  <luke-jr> tells gbuild where to pull from
567 2020-12-11T20:18:11  <luke-jr> I always have ti pull from local
568 2020-12-11T20:18:41  <jonasschnelli> luke-jr: https://github.com/bitcoin/bitcoin/blob/master/contrib/gitian-descriptors/gitian-osx-signer.yml#L11
569 2020-12-11T20:18:58  <jonasschnelli> I guess it always pulls it from there.
570 2020-12-11T20:19:02  <midnight> Oh nice, no changes necessary in the gitian signing env and I'm getting matches.
571 2020-12-11T20:19:04  <luke-jr> not if you specify -u
572 2020-12-11T20:19:21  <luke-jr>     -u, --url PAIRS                  comma separated list of DIRECTORY=URL pairs
573 2020-12-11T20:19:32  <luke-jr> not very useful --help lol
574 2020-12-11T20:19:38  <jonasschnelli> I do that for bitcoin=
575 2020-12-11T20:19:53  <jonasschnelli> but wasn't aware you can also do it then for signature=
576 2020-12-11T20:20:00  <luke-jr> it's no different ;)
577 2020-12-11T20:20:16  <jonasschnelli> However,.. I'm going for the additional input file
578 2020-12-11T20:20:20  <midnight> just me and wumpus I think with the windows sigs for the rc but
579 2020-12-11T20:20:20  <jonasschnelli> Seems dummysave
580 2020-12-11T20:20:21  * midnight shrug
581 2020-12-11T20:21:05  <luke-jr> midnight: ?
582 2020-12-11T20:21:29  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has joined #bitcoin-core-dev
583 2020-12-11T20:21:32  <midnight> luke-jr: working nicely still, in spite of my semi-custom build environment.
584 2020-12-11T20:21:47  <luke-jr> midnight: I see lots of windows sigs for 0.21.0rc3
585 2020-12-11T20:21:59  <midnight> hrm
586 2020-12-11T20:24:24  <jonasschnelli> midnight: I guess semi-custom is good. Better than if everyone uses the same host/env with the same script
587 2020-12-11T20:24:44  <luke-jr> my attempt to semi-custom was met with mismatching
588 2020-12-11T20:24:58  <luke-jr> but my idea of semi-custom was a ppc64le VM image :P
589 2020-12-11T20:24:58  <jonasschnelli> have you figured out why?
590 2020-12-11T20:25:02  <midnight> jonasschnelli: The maintenance of the custom(ish) build env is entirely on me, and I figure it's better to arrive at similar results this way.
591 2020-12-11T20:25:19  <luke-jr> jonasschnelli: apparently the ppc64le compilers do not output the same objects as the x86_64 compilers, even cross compiling
592 2020-12-11T20:25:32  <luke-jr> no idea why :/
593 2020-12-11T20:25:59  <midnight> triangulation and all that.
594 2020-12-11T20:26:24  <luke-jr> you would *think* i686-w64-mingw64-gcc (or whatever) would produce the same stuff on x86_64 and ppc64le, but apparently not
595 2020-12-11T20:28:40  *** belcher <belcher!~belcher@unaffiliated/belcher> has quit IRC (Ping timeout: 272 seconds)
596 2020-12-11T20:30:03  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
597 2020-12-11T20:30:03  <bitcoin-git> [bitcoin] dongcarl opened pull request #20629:  depends: Improve id string robustness  (master...2020-12-improve-depends-id-string) https://github.com/bitcoin/bitcoin/pull/20629
598 2020-12-11T20:30:14  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
599 2020-12-11T20:32:45  <jonasschnelli> i'm doing the 0.20.2 mac signatures asap (need to track-down the previous issue first)
600 2020-12-11T20:44:10  *** belcher <belcher!~belcher@unaffiliated/belcher> has joined #bitcoin-core-dev
601 2020-12-11T20:46:44  <jonasschnelli> is there a way to speeup gitians "Upgrading system, may take a while (log in var/install.log)"?
602 2020-12-11T20:46:50  <jonasschnelli> Re-create the base system?
603 2020-12-11T20:46:57  <jonasschnelli> s/system/image
604 2020-12-11T20:47:32  <sipa> iirc, yes
605 2020-12-11T20:47:42  <sipa> creating an image after the update speeds things up
606 2020-12-11T20:48:18  <jonasschnelli> the install part takes >10mins here.. annoying for testing
607 2020-12-11T20:48:47  <wumpus> yes, the "Upgrading image" takes longer than the actual builds here
608 2020-12-11T20:48:59  <jonasschnelli> ideed
609 2020-12-11T20:49:04  <jonasschnelli> +n
610 2020-12-11T20:49:42  <wumpus> I have no idea how to speed it up, I hacked it once to upgrade the base image but then somehow ended up with two versions of every package and non-deterministic results, so yea, if regenerating the base image solves it I'd definitely recommend that path
611 2020-12-11T20:50:18  <jonasschnelli> Okay... I'll try that
612 2020-12-11T20:50:37  <wumpus> fairly sure that works for one of {KVM, LXC}, don't know which one, the other always gets the ancient image
613 2020-12-11T20:51:28  <wumpus> (they both generate the base image in completely different ways)
614 2020-12-11T20:53:09  <wumpus> it would make sense to look into it some day but also guix builds are around the corner right
615 2020-12-11T20:54:13  <sipa> Any Day Now(tm)
616 2020-12-11T20:55:02  <wumpus> releases are sufficiently rare anyway, but the extra delays are mostly annoying when testing/iterating something
617 2020-12-11T21:04:25  *** tylerchambers <tylerchambers!uid477594@gateway/web/irccloud.com/x-raafweykxwvatycu> has joined #bitcoin-core-dev
618 2020-12-11T21:10:43  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has quit IRC (Ping timeout: 240 seconds)
619 2020-12-11T21:11:59  <jonasschnelli> I'm really confused
620 2020-12-11T21:12:21  <jonasschnelli> I have downloaded again 5e3a08ae8195190d6f1b12e3e1e9d710e7ad385941a6e8d04e3391f12deddb11  bitcoin-osx-unsigned.tar.gz ...
621 2020-12-11T21:12:28  <jonasschnelli> ./detached-sig-create.sh -s "Bitcoin"
622 2020-12-11T21:12:49  *** bitdex <bitdex!~bitdex@gateway/tor-sasl/bitdex> has joined #bitcoin-core-dev
623 2020-12-11T21:12:49  <jonasschnelli> verified code-signature  codesign -v dist/Bitcoin-Qt.app
624 2020-12-11T21:13:00  <jonasschnelli> uploaded signature.tar.gz
625 2020-12-11T21:13:06  <jonasschnelli> did a gitian signer build
626 2020-12-11T21:13:11  <jonasschnelli> downloaded the dmg
627 2020-12-11T21:13:14  *** luke <luke!~luke@bitnomial/staff/luke> has quit IRC (Quit: sleep)
628 2020-12-11T21:13:17  <jonasschnelli> verified the signature and I get:
629 2020-12-11T21:13:27  <jonasschnelli> * /Volumes/Bitcoin-Core 4/Bitcoin-Qt.app: invalid signature (code or signature have been modified)
630 2020-12-11T21:13:44  <achow101> huh. did apple change something recently?
631 2020-12-11T21:14:31  <sipa> jonasschnelli: this is an old version?
632 2020-12-11T21:14:37  <sipa> of bitcoin core?
633 2020-12-11T21:14:58  <jonasschnelli> sipa: no.. still trying to figure out what happend for 0.21.0rc3
634 2020-12-11T21:15:22  <jonasschnelli> AFAIK 5e3a08ae8195190d6f1b12e3e1e9d710e7ad385941a6e8d04e3391f12deddb11 was the unsigned tarball (bitcoin-osx-unsigned.tar.gz)
635 2020-12-11T21:16:03  <achow101> yep
636 2020-12-11T21:16:22  <jonasschnelli> I redid everything and still getting invalid signatures
637 2020-12-11T21:16:49  <jonasschnelli> Might it be possible that there is an edge-case with the signature size or the attaching of the signature?
638 2020-12-11T21:20:11  <achow101> No issues on 10.15.7, so maybe just a big sur problem
639 2020-12-11T21:21:24  <achow101> nvm, didn't click far enough
640 2020-12-11T21:22:51  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has joined #bitcoin-core-dev
641 2020-12-11T21:22:51  <bitcoin-git> [bitcoin] jonasschnelli opened pull request #20630: Allow providing local signatures in gitian osx signer (master...2020/12/macos_gitian_signer) https://github.com/bitcoin/bitcoin/pull/20630
642 2020-12-11T21:22:52  *** bitcoin-git <bitcoin-git!~bitcoin-g@x0f.org> has left #bitcoin-core-dev
643 2020-12-11T21:25:29  <jonasschnelli> Here is the gist (how I did it): https://gist.github.com/jonasschnelli/39f0b03981e1bbffff929f538cf139a9
644 2020-12-11T21:25:44  <jonasschnelli> I have absolutely no clue why the signatures are wrong
645 2020-12-11T21:28:03  <jonasschnelli> if anyone like to go down the rabbit hole:
646 2020-12-11T21:28:28  <jonasschnelli> my local signed and detached signature directory: https://bitcoin.jonasschnelli.ch/bitcoin-osx-unsigned-nowsigned.zip
647 2020-12-11T21:28:46  <jonasschnelli> ended up in build: https://bitcoin.jonasschnelli.ch/bitcoin-osx-signed3.dmg
648 2020-12-11T21:28:51  <jonasschnelli> (both files are in the gist above)
649 2020-12-11T21:29:18  <jonasschnelli> maybe cfields has some insights?
650 2020-12-11T21:30:02  <jonasschnelli> I need some sleep and try it again with the 0.20.2rc1 release (and see what happens there)
651 2020-12-11T21:30:46  * jonasschnelli afk
652 2020-12-11T21:34:37  *** Guyver2 <Guyver2!Guyver@guyver2.xs4all.nl> has quit IRC (Quit: Going offline, see ya! (www.adiirc.com))
653 2020-12-11T21:36:37  *** vincenzopalazzo <vincenzopalazzo!~vincent@2001:b07:6474:9d49:849d:db24:7f93:fb8a> has quit IRC (Quit: Leaving)
654 2020-12-11T21:48:31  *** eugene-ff <eugene-ff!~eugene_ff@2604:2000:1383:472b:759a:a9e9:8a19:69d2> has joined #bitcoin-core-dev
655 2020-12-11T22:00:07  <midnight> gah, xcode is a 7+ GB download.
656 2020-12-11T22:00:20  <sipa> yes :(
657 2020-12-11T22:03:33  <sipa> and we only need 47 MB from it
658 2020-12-11T22:11:57  *** Talkless <Talkless!~Talkless@mail.dargis.net> has joined #bitcoin-core-dev
659 2020-12-11T22:29:50  <achow101> interesting. there's a single byte difference between jonasschnelli signing result and his gitian result
660 2020-12-11T22:34:49  <sipa> what byte?
661 2020-12-11T22:35:43  <achow101> https://0bin.net/paste/NuOSAQ26#QxBJq7yNS7ynp6oMsXzvKQWT0vXTLYboU2QRfyBqu2d
662 2020-12-11T22:36:39  <sipa> if you fix it, does it verify?
663 2020-12-11T22:36:53  <achow101> yes, the original verifies
664 2020-12-11T22:38:55  <sipa> what do you mean by signing result and gitian result?
665 2020-12-11T22:40:13  <sipa> the output of the macos signing tool, and the result of gitian applying the extracted signature?
666 2020-12-11T22:40:14  <achow101> the results from https://bitcoin.jonasschnelli.ch/bitcoin-osx-unsigned-nowsigned.zip which come from the signing process. it contains a fully signed binary
667 2020-12-11T22:40:42  <sipa> got it
668 2020-12-11T22:40:43  <achow101> the gitian result is https://bitcoin.jonasschnelli.ch/bitcoin-osx-signed3.dmg which comes from gitian builder combining the unsigned with the signature.tar.gz from the above zip
669 2020-12-11T22:40:55  <sipa> so the extraction/reattaching process doesn't result in the exact same binary
670 2020-12-11T22:41:18  <achow101> yeah
671 2020-12-11T22:43:09  <sipa> can you run "pagestuff -p" on both?
672 2020-12-11T22:45:32  <achow101> identical
673 2020-12-11T22:47:13  <sipa> so
674 2020-12-11T22:48:38  <sipa> the detached-sig-create.sh tool uses pagestuff -p | tail -2 | grep offset | sed 's/[^0-9]*//g' to figure out the offset
675 2020-12-11T22:49:17  <sipa> and the size of the sign file for the size
676 2020-12-11T22:49:25  <achow101> the sizes are off by one byte
677 2020-12-11T22:49:29  <sipa> aha
678 2020-12-11T22:49:34  <sipa> that's what i was going to ask next
679 2020-12-11T22:49:41  <sipa> i think this is the cause
680 2020-12-11T22:50:14  <achow101> x86_64-apple-darwin-otool -l Bitcoin-Qt for segment "__LINKEDIT" says "vmsize 0x000000000007b000" but on the other it's "vmsize 0x000000000007c000"
681 2020-12-11T22:50:14  <sipa> which one is bigger?
682 2020-12-11T22:50:26  <achow101> gitian one is bigger
683 2020-12-11T22:50:33  <achow101> wait no, that's backwards
684 2020-12-11T22:50:38  <achow101> signed one is bigger, gitian is smaller
685 2020-12-11T22:52:34  <sipa> that's a 4 kB difference though
686 2020-12-11T22:52:36  <sipa> not 1 byte
687 2020-12-11T22:53:18  <achow101> The files are the same size
688 2020-12-11T22:53:30  <achow101> and diffoscope only sees the difference on that single byte
689 2020-12-11T22:54:05  <sipa> what if you run the same on the unsigned gitian output (= the input to the macos signing tool)
690 2020-12-11T22:54:57  <achow101> not sure what you mean
691 2020-12-11T22:55:07  <sipa> x86_64-apple-darwin-otool -l Bitcoin-Qt
692 2020-12-11T22:55:15  <sipa> oh
693 2020-12-11T22:55:16  <sipa> nvm
694 2020-12-11T22:56:32  <sipa> yeah, what if you run that on the unsigned Bitcoin-Qt binary?
695 2020-12-11T22:57:53  <achow101> __LINKEDIT segment is the same size as the gitian result
696 2020-12-11T22:58:48  <sipa> so
697 2020-12-11T22:59:12  <sipa> the codesigning tool modified the __LINKEDIT segment, but applying the detached signature didn't do the same?
698 2020-12-11T22:59:50  <achow101> seems so
699 2020-12-11T23:01:56  <sipa> if you run "pagestuff -p Bitcoin-Qt | tail -2 | grep size | sed 's/[^0-9]*//g'", what do you get?
700 2020-12-11T23:02:03  <sipa> on the various version
701 2020-12-11T23:06:05  <achow101> ah wait, I was looking at the wrong file earlier
702 2020-12-11T23:06:54  <sipa> where do the linux binaries pagestuff and codesign_allocate come from?
703 2020-12-11T23:07:07  <achow101> the otool on the unsigned binary gives a way smaller __LINKEDIT, presumably because the code signature is involved in there
704 2020-12-11T23:07:43  <sipa> i guess that's expected
705 2020-12-11T23:08:09  <achow101> they come from one of the depends packages, not sure which
706 2020-12-11T23:08:35  <sipa> ah i see
707 2020-12-11T23:11:15  <achow101> the sizes from that pagestuff command are the same for both gitian and signed
708 2020-12-11T23:11:28  <sipa> and the offsets too?
709 2020-12-11T23:11:41  <achow101> not the same for unsigned, but that's expected because the section size it looks for doesn't exist as it's the code sig
710 2020-12-11T23:11:54  <sipa> yeah
711 2020-12-11T23:11:56  <achow101> same offests
712 2020-12-11T23:14:45  <sipa> ok, so the byte that changed is in the MP_MACH_HEADERS section, in the very beginning of the file
713 2020-12-11T23:14:50  <sipa> while the codesig is at the very end
714 2020-12-11T23:15:16  <achow101> presumably that section is a table
715 2020-12-11T23:21:56  <achow101> so the problem must be with codesign_allocate because that's what sets that size value
716 2020-12-11T23:22:49  *** davterra <davterra!~davterra@> has quit IRC (Quit: Leaving)
717 2020-12-11T23:25:40  <sipa> the codesign tool has a --detached option to construct detached signatures directly
718 2020-12-11T23:26:00  <sipa> why is the tooling doing signing directly, and then extracting the signatures from the result?
719 2020-12-11T23:26:16  <sipa> this may be a reason for a difference
720 2020-12-11T23:26:31  <achow101> this workflow probably was setup before that existed
721 2020-12-11T23:27:27  <sipa> is it possible that the codesign_allocate tool is out of date?
722 2020-12-11T23:28:08  <achow101> possibly
723 2020-12-11T23:29:05  <sipa> what else does otool or anything else say about __LINKEDIT?
724 2020-12-11T23:32:22  <achow101> nothing else afaict
725 2020-12-11T23:35:30  *** mol <mol!~mol@unaffiliated/molly> has quit IRC (Ping timeout: 272 seconds)
726 2020-12-11T23:38:17  <sipa> achow101: does macos come with a codesign_allocate tool too?
727 2020-12-11T23:39:38  *** AmberJ_ <AmberJ_!~AmberJ_@> has quit IRC (Remote host closed the connection)
728 2020-12-11T23:40:26  <achow101> yes
729 2020-12-11T23:40:30  <sipa> the native_cctools depends package is only a few months old, and most changes since have been for apple ARM stuff it seems
730 2020-12-11T23:43:43  <sipa> achow101: if you run "codesign_allocate -i <unsigned Bitcoin-Qt filename> -a x86_64 225312 -o <some temp file>
731 2020-12-11T23:43:49  <sipa> on macos
732 2020-12-11T23:44:12  <sipa> and then inspect that temp file, does it have the correct __LINKEDIT vmsize?
733 2020-12-11T23:44:23  <achow101> will try
734 2020-12-11T23:46:16  *** peterrizzo_ <peterrizzo_!~peterrizz@ool-44c18924.dyn.optonline.net> has quit IRC (Quit: peterrizzo_)
735 2020-12-11T23:48:38  *** mol <mol!~mol@unaffiliated/molly> has joined #bitcoin-core-dev
736 2020-12-11T23:48:44  *** justanotheruser <justanotheruser!~justanoth@unaffiliated/justanotheruser> has quit IRC (Ping timeout: 258 seconds)
737 2020-12-11T23:52:30  <achow101> It's the correct size
738 2020-12-11T23:53:39  <sipa> hmm, we should try updating to the latest native_cctools package i guess and see if that fixes it
739 2020-12-11T23:57:47  <sipa> an alternative is making jonasschnelli downgrade his codesign_allocate tool to one that's compatible with the native_ccttols we use, and then tell his codesign tool to use that
740 2020-12-11T23:59:03  *** vasild <vasild!~vd@gateway/tor-sasl/vasild> has quit IRC (Ping timeout: 240 seconds)
741 2020-12-11T23:59:52  <achow101> I wonder if we're just running into some weird edge case now