19:01:19 #startmeeting 19:01:19 Meeting started Thu Sep 29 19:01:19 2016 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:19 Useful Commands: #action #agreed #help #info #idea #link #topic. 19:01:26 hi 19:01:36 meeting time! 19:01:46 meeting! 19:01:56 #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier 19:02:04 (oh already started) 19:02:07 here 19:02:07 hi 19:02:12 hi 19:02:20 any proposed topics? 19:02:24 topic proposal: pruning and blockrelay 19:02:56 hi 19:03:11 policy against uncompressed keys or not 19:03:11 #topic pruning and blockrelay 19:03:38 I think we should add a service flag for block relay with a min-height 19:03:52 NODE_PRUNENETWORK or something 19:03:57 there have been multiple ideas around that 19:04:35 IMO whatever we do, we should recognise that w/ segwit's larger blocks we can expect a lot of full nodes to run out of disk space quite soon 19:04:35 the easiest is just to add a flag that says you relay valid blocks and transactions, but not historical blocks more than a few deep 19:04:35 I guess people slowly start to prune the blockchain to a max of 80GB or similar... but I guess not everyone is aware of the fact that you don't relay then 19:04:36 it would be nice to support more than one range, e.g. also archive nodes that host part of the old blocks 19:04:49 it becomes harder when you want multiple ranger 19:04:57 do we have a reason for more than one range? 19:05:00 We could introduce another message type... blockrange or so 19:05:01 it becomes even harder when you want to support sharding in an efficient way 19:05:08 I'm not sure why itb ecomes hard, just add a query message that returns what ranges are supported 19:05:11 sipa: what do you mean by sharding exactly? 19:05:25 petertodd: you'd configure your node to maintain a certain % of blocks 19:05:45 wumpus: query, yes, why not, or just inform like we do with sendheaders 19:05:47 sipa: see, given that the bitcoin protocol can't be safely sharded right now, I think we can safely say that we don't need to support sharding in block relay yet 19:05:55 sipa: doing so might even be dangerous if people start using it 19:06:04 petertodd: not in block relay 19:06:08 petertodd: for block archival 19:06:16 sipa: but why shard vs. keep ranges? 19:06:23 sipa: (ranges of full blocks) 19:06:25 BitTorrent already does this. Surely we can learn from that? 19:06:38 luke-jr: I don't think so - bittorrent is a very different problem than bitcoin 19:06:38 this is for letting other peers know what ranges of blocks are hosted, I don't think this should affect releay 19:06:41 so, i've been running statistics on what block depths are being requested from nodes 19:06:47 petertodd: learn from it, not use it directly 19:07:01 petertodd: BitTorrent's problem isn't very different from IBD 19:07:08 sipa: interesting.. do you have the stats public available somewhere 19:07:16 I wanted to do this a long time 19:07:20 luke-jr: so, the thing is bittorrent has the problem of a diverse set of files, we just don't have that problem and can optimise differently because everyone needs access t othe same set of data 19:08:03 there are something like 4 meaningful 'ranges' 1) the top 2 blocks (just relay) 2) up to ~2500 blocks deep... requested very often 3) up to ~10000 deep... requested a few times more than the next range 4) the rest 19:08:15 otoh bittorrent has a fixed block size :) 19:08:22 wumpus: so do we *ducks* 19:08:38 sipa: that probably corresponds to how long people leave their nodes offline :) 19:08:51 inb4 Bittorrent XT 19:09:04 jonasschnelli: they're not available, and the ranges i gave above are just from me quickly glancing over the result 19:09:04 btcdrak: I use Bittorrent Unlimited myself 19:09:05 What about fingerprinting issued in conjunction with available ranges? 19:09:16 *issues 19:09:20 jonasschnelli: make them powers of two? 19:09:33 well 4 ranges can be done with 2 service bit flags 19:09:45 gmaxwell: you've worked on these ideas before, comments? 19:09:47 But would that work with the flexible pruning option based on MB? 19:10:05 jonasschnelli: sure, just find the biggest range less than the pruning amount 19:10:12 jonasschnelli: you'd change your service bits on the fly 19:10:20 why would the ranges need to be in the flags? 19:10:30 sorry, I missed that the meeting started. 19:10:32 Yes. Why? Better add an explicit message for the range 19:10:34 how would you otherwise discover what nodes to connect to? 19:10:45 just randomly try? 19:10:45 jonasschnelli: oh right, you mean if MB != blocks... sorry. 19:10:49 I think you'll need a service flag to show support for the protocol, but not what ranges you have 19:10:52 query or inform the other node if proto-ver > NODE_PRUNENETWORK 19:11:00 well that can be negotiated later, like bittorrent does I guess 19:11:08 wumpus: well you do want addr messages to contain this information 19:11:18 I doubt bitcoin has 'service flags' in its tracker what blocks nodes have 19:11:18 sipa: so the nice thing about bitcoin, is just randomly try will probably work fairly often due to the low number of ranges out there 19:11:26 as that changes all the time anyhow 19:11:29 I was strongly of the view that we needed to signal at least two ranges. Sipa's latest measurements make me think at least three are needed. 19:11:31 s/bitcoin/bittorrent/ 19:11:37 I think informing other nodes ranges over addr is another thing... 19:11:46 A first step would be a information after connect 19:11:55 yes, addr is another thing 19:12:22 I think ranges in service bits are no big deal, the harder question is what to do about the history. having nodes with 150GB of history in order to serve the last range is not very viable. 19:12:23 could be done later if an efficient way is needed to *locate* peers with certain ranges 19:12:31 but that seems premature optimization 19:12:36 We will need to redo addr sometime relatively soon in any case, as our messages are not compatible with HS-NG. 19:12:52 gmaxwell: oh, you mean Tor's new hidden services standard right? 19:12:56 petertodd: yes. 19:13:02 (also I2P though thats not new) 19:13:18 I think the number of ranges should be variable 19:13:24 redesigning addr is a different topic 19:13:35 also necessary, but again, doesn't need to be on one heap 19:13:45 wumpus: when I'm saying ranges I am specifically referring to the top-N zomes. 19:14:14 well, so if we add service bits for recent history ranges, that should be possible to implement as a separate feature to archival history ranges, and it'd be a big first step 19:14:22 I think it should be possible to, say, only host the first 20GB of blocks 19:14:37 historic only nodes 19:14:37 I don't see why it should be restricted to only recent history 19:14:38 I don't think it's likely we'll see the two different features collide, so maybe implement recent history ranges first 19:14:55 or I mean first 20GB + last 144 blocks 19:15:07 For history storage, I was previously working on a proposal where nodes could signal a small (32 bit) seed and a size and from that everyone would know what parts of the history they would store. I was so far unable to unify two different schemes, one which was computationally efficient to figure out who had what, and one which never required a peer to fetch a block it had previously deleted. 19:15:15 so very quick breakdown: out of 7M requested blocks, 100k were for the tip, range 2-2500 has around 200-2000 requests per block, and from 10000 to genesis deep there are around 20 per block 19:16:19 I think for now we should not worry about the old history part and only worry about Top-n vs everything, as that fits into the pruning we already have and can be accomplished purely with service bits. 19:16:22 the bittorrent problem is different in that there the goal of each node is to have everything 19:16:25 so a social consideration here, is we can think in terms of recent history as "if there's a flaw, how much would we ever reorg w/o just saying bitcoin has failed?" 19:16:44 petertodd: thats partly why we have the 288 block maximum amount of pruning. 19:17:13 gmaxwell: indeed, and that's only two days... 19:17:36 Using multiple service bits for 4 ranges seems to be a hackish-design IMO 19:17:49 at 100 blocks any reorg will _necessarily_ cause unrecoverable losses. So 288 basically gives a day plus an extra day for overhead. 19:17:50 there's also a natural time criteria from how the difficulty adjustments reduce your resistance to 51% attack - if your node is offline longer, the minimum attacker size to fool you goes down 19:17:52 strangely enough: i see much more requests around 1000 deep than around 100 deep 19:18:04 jonasschnelli: I don't see anything hackish. 19:18:05 jonasschnelli: I also think it's a strange use of service bits 19:18:07 I'd prefere using a single service bit to state pruned blockchain and then a new message (or append something to version?) 19:18:24 sipa: probably because people don't turn their nodes on and off every day 19:18:25 sipa: you probably want to filter out the bitnodes spider, as I believe it requests a block to check the node is working. 19:18:35 gmaxwell: ah. 19:18:53 petertodd: someone who hasn't turned their node on will request all of 0 to -1000. so it will not make 1000 greater. 19:19:07 jonasschnelli: NAK. 19:19:07 gmaxwell: oh! I didn't know we did that 19:19:09 i'm a bit surprised people think there is no need to have the available block ranges indicated in addr messages 19:19:30 (whether through service bits, or some extension) 19:19:30 I think there is a need... but it could be a second step 19:19:38 jonasschnelli: appending to version should be unnecessary, that's also a hack :) 19:19:50 jonasschnelli: if it's a second step, we need to extend addr, and the whole management of addresses 19:19:51 Okay. Agree. What about a new message type? 19:19:55 blockrange 19:19:58 jonasschnelli: you don't understand. 19:20:18 jonasschnelli: look at pieter's request figures, if nodes are effectively forced to go to peers that have everything whenever they connect becuase if they don't know they'll be able to fetch any blocks at all, then it will put lots more load on them.. causing people to stop offering blocks... causing more pressure on what remains. 19:20:56 jonasschnelli: the point of having it in service bits is so nodes can find peers that have the range they need 19:20:58 but addr information gets old really fast 19:21:17 wumpus: much less so with feeler connections 19:21:26 nodes may dynamically change what blocks they have, so there will always be cases of nodes connecting and realizing they have nothing to offer each other 19:21:27 Okay. I see the point. 19:21:29 (presumably, i don't have numbers) 19:21:54 just like currently nodes will try to connect into black holes that no longer host a node 19:22:24 so another interesting thing here is that ranges are queried linearly - you download blocks in a roughly linear fashion - so we could take advantage of that by making sure that nodes with one range keep track of nodes with adjacent ranges 19:22:29 sipa: sure, feeler connections make it somewhat better 19:22:35 wumpus: yes, sometimes the data is wrong. But there is a big difference between having 80% of the nodes on the network giving you no idea if they'll be useful at all until after you connect, vs a suggestion that might sometimes be wrong. 19:22:41 but I don't think addr is a very up-to-date information source 19:22:46 thus, as you sync the first time, ask nodes with the range you're syncing at this moment for the next range you need 19:23:08 wumpus: if ranges are deterministic, they don't need to be up to date 19:23:15 petertodd: yes, any sharding plan wouldn't randomly distribute the kept blocks, but keep randomly distributed ranges 19:23:32 wumpus: I don't know if you realize that sipa and I are not thinking in terms of absolute ranges here. but nodes saying "I keep the last 288" or "I keep the last 2016" or "I have all of history". 19:23:38 gmaxwell: but indeed this is a different problem from the bittorrent problem where everyone's goal is to have everything 19:23:56 gmaxwell: well that's sharding... maybe that is something to postpone for later 19:24:01 sipa: sure, I'm more talking about how the linearity affects the network p2p design - prefentially peering with peers with the adjacent range may even be a reasonable design 19:24:03 wumpus: eh, everyone needs to get everything 19:24:03 there, nodes can just connect randomly and have a high change the other nodes has something to offer them 19:24:04 wumpus: and I wouldn't expect that data to go out of date fast.. pretty much only when nodes go up and down. 19:24:04 oh, nvm, i'm misreading 19:24:20 luke-jr: only initially 19:24:28 oh, I see the distinction 19:24:41 luke-jr: bittorrent nodes don't throw away blocks, generally 19:24:53 f(best-height, seed-in-addr) -> ranges 19:25:12 for the spreading the history around, as mentioned I came up with concrete schemes (based on consistent hashes) that have nice properties. 19:25:30 i wonder whether we need to have that in the first go at this 19:26:04 I think a first simple solution that allow to extend it further would be appriciated. 19:26:10 even just having serve-everything and server-the-last-288-and-relay-at-tip would be a good addition 19:26:15 making the ranges deterministic makes some sense, on the other hand, it does restrict the flexibilty of nodes to choose what ranges they host, it means everything has to be got right in first try 19:26:23 sipa: thats what I am saying. 19:26:27 sipa: agree 19:26:30 I do not think we can do better immediately anyways. 19:26:45 21:18:07 < jonasschnelli> I'd prefere using a single service bit to state pruned blockchain and then a new message (or append something to version?) 19:26:47 sipa: though your latest figures suggest that the 2016 depth is important too. 19:26:48 21:19:07 < gmaxwell> jonasschnelli: NAK. 19:27:10 if nodes attempt to maintain a few connections to peers that have the next range after they have, maybe it doesn't matter exactly what the ranges actually are? any given node would have a few connections to the next range, and anyone syncing from them could ask for those connections 19:27:15 sipa: my understanding of jonasschnelli comment was there should be a bit that says "I relay blocks but don't have history" I am NAK on that. 19:27:36 as there is no scope for later optimization, because all nodes have to agree what ranges are implied 19:27:40 We could add a service bit that says "I relay only the last 288 blocks" 19:27:50 jannes: yes that would be the initial idea 19:27:54 jonasschnelli* 19:28:08 gmaxwell: how is that different from what i suggested? 19:28:11 21:26:10 < sipa> even just having serve-everything and server-the-last-288-and-relay-at-tip would be a good addition 19:28:21 I think my initial idea with the general pruning sevice bit and a new message type is to complex and inflexible 19:28:28 jonasschnelli: yes, that would be better, though pieter's data suggests that there are a LOT of requests at 1000. I think if I had that data I would have been suggesting the maximum pruning should be 2016, and then had the bit at that dep. 19:28:50 sipa: the ability to relay blocks at depth -10. 19:29:04 gmaxwell: less than 2% of blocks requested from my node are at the tip 19:29:22 (but the tip is still 100x more frequent than any other individual depth) 19:29:50 gmaxwell: "a service bit to indicate pruned blockchain" implies you can serve 288 deep :) 19:30:08 gmaxwell: re: maximum pruning depth, it's reasonable for that to be a similar % of the total data that storing the UTXO set takes - if you have 10GB of UTXO, 2GB of block data isn't a big change 19:30:11 yes, you could define it as that 19:30:15 I don't think there is any remaining disagreement on using bit(s) to signal I have a top-n. But I have some doubt on N. it needs to capture the largest amount of the block realy bandwidth without being unduely pruning incompatible. 19:30:37 288 is the minimum pruning amount in bitcoin core already so it'd be a valid choice 19:30:44 as a first pass, i wonder if you preferentially downloaded from pruned peers whenever you were behind by less than 288 blocks, that would take enough load of peers serving full history? 19:30:50 morcos: absolutely. 19:31:01 Good idea 19:31:09 yes, that would make sense 19:31:23 unfortunately, sipa's data suggests that 288 sheds less traffic than measurements years ago suggested. 19:31:43 maybe i should compute statistics in bytes rather than blocks 19:31:45 gmaxwell: it wasn't clear to me what the integral from 1 to 288 was compared to 288 to inf 19:31:46 well it is a compromise 19:31:53 putting the threshold higher makes some peers completely useless 19:31:57 to see what percentage of bandwidth is needed in 1-288 19:31:58 which reduces morcos 's argument 19:32:04 Yes. I guess you convinced me to use two service bits then. -288 and -2016 19:32:16 which is why it might be useful to use two bits and be able to signal 1-288, 1-2016... and perhaps start encouraging people to not prune shorter than 2016. 19:32:37 i think we're getting into a design discussion here 19:32:47 my number are very premature and not well analysed 19:32:50 it'd also be possible to add a 288-flag now, and then consider a 2016 flag later 19:32:54 sipa: indeed, thought that was the input you requested from me. 19:32:57 wumpus: yes, thats what i'm saying 19:32:59 wumpus: yes! indeed. 19:33:03 Agree with wumpus 19:33:04 if it turns out to be necessary 19:33:10 wumpus: ACK 19:33:15 yes, i think just a 1-288 one seems useful 19:33:16 good :) 19:33:16 Start with a simple tip-288 relay, and get some experience 19:33:23 wumpus: it looks pretty clearly necessary but no need to do everything at once. 19:33:30 wumpus: basically advice is, turn your node on at least once every two days 19:33:54 petertodd: yes 19:33:55 petertodd: we really should have cron mode for the daemon where it just syncs up and shuts off. :P 19:34:06 bitcoind -oneshot 19:34:08 :P 19:34:17 gmaxwell: heh, that's not a crazy idea - I'd use it on my laptop 19:34:18 didn't we once had a proposal for the pause option? 19:34:24 right, there's a flag that quits after reindex, but none that exits after sync 19:34:31 would be easy to add tho 19:34:41 we could just ask for the utxo set, shoudl we discuss ideas how to do that 19:34:51 ^ yes :) 19:34:58 make -oneshot run in the foreground with a progress bar :) 19:34:59 without utxo commitment that's a no-go 19:35:05 thanks codeshark 19:35:20 wumpus: +1 19:35:42 morcos: pointless when we were unable to get past the discussion for the security model change to not validate the past history based on proof of work. 19:35:48 and lets not underestimate how dangerous UTXO commitments can be - I'm very dubious about committing to the (U)TXO set more recently than maybe a month or two 19:35:51 would be great to query utxo for quick sync, then go backwards in time fetching blocks to increase security...but yes, this is a design discussion 19:35:53 i was making a joke, sorry 19:36:18 alas, quick sync doesn't look feasible in the nearterm 19:36:20 ok, next topic? 19:36:36 but since that was brought up... Can we talk about removing checkpoints? 19:36:55 #topic removing checkpoints 19:36:59 what % of transactions are before the last checkpoint 19:37:01 does anyone know? 19:37:03 someone should write up a design proposal for that to be evaluated 19:37:18 Right now they're used for two things, preventing header flooding with low difficulty headers; and skipping signatures in earlier blocks. 19:37:21 gmaxwell: just removing checkpoints, or assuming sigs are valid if buried deep enough? 19:37:27 gmaxwell: and 3) estimating progress 19:37:45 keeping something for estimating progress would make sense 19:37:47 i think 1) remains needed and 3) remains useful 19:37:50 that doesn't need to be checkpoints 19:38:06 because very few percentage of the transactions are below the checkpoint .. since libsecp256k1 (and I expect the checkqueue)-- my point two is basically pointless, and I think it could just be removed 19:38:29 I think on a desktop it only adds 15-20 minutes to the sync. 19:38:29 gmaxwell: I'd ACK simply removing checkpoints entirely; I'm not happy to see them replaced with another scheme to skip sig checking 19:38:33 a block-height-to-relative-difficulty map would have much less of a stigma 19:38:46 eh, verification difficulty that is 19:38:52 gmaxwell: really? 19:38:54 petertodd: I think we could remove CP from reason two without implementing the replcement. 19:39:06 petertodd: morcos is right that needs a design proposal outside of the meeting. 19:39:12 i'm a bit confused about how to think about checkpoints for signature skipping 19:39:22 sipa: I benchmarked before but I'm going off of memory, I could be wildly wrong. I will test again if there is interest. 19:39:52 Removing checkpoints would slow down (maybe insignificant) a scan in a possible SPV hybrid mode? 19:39:54 For reason (1) the only answer I have is that I think we should proposal a bit to perpetually increase the minimum difficulty from 1 to something else. 19:40:00 for instance the recent ISM change caused us to do less validation for certain blocks in our history (blocks in a softfork between the 75% and 95% thresholds) 19:40:10 jonasschnelli: SPV mode won't validate *anything* at all 19:40:28 (with a checkpoint like bypass of that new rule, for existing blocks that break it) As little as 100,000 would eliminate the header flooding vulenrablity. 19:40:33 Yes. But assume we would add an SPV hibrid mode in oder to received payment during IBD 19:40:50 One would need to download 400k headers without a checkpoint at h400k 19:40:53 maybe checkpoints should just be disabled by default before complete removal? 19:41:00 jonasschnelli: i think you're confused 19:41:07 for Sipa's (3) reason for 'checkpoints' I don't give a darn, use chicken bones for progress estimation for all I care. :P it's historical accident that checkpoints and progress use the same data structure. 19:41:24 gmaxwell: :) +1 19:41:28 gmaxwell: yes, my point too 19:41:34 gmaxwell: agree, those could be completely separated 19:41:36 gmaxwell: ACK checken bones 19:41:36 Might as well fit a cubic spline to the height vs txn count... and store the parameters. 19:41:53 right 19:42:09 gmaxwell: heh, if we do that with floating point math that has the advantage that it _can't_ be used for consensus :) 19:42:18 * sipa now remembers a song our student organization wrote to the melody of staying alive, called 'cubic spline' 19:42:21 so my proposal, if there is interest, is that I'll measure the performance impact of removing the signature skippingentirely (esp post checkqueue). And if it's not awful, we'll remove. 19:42:37 +1 19:42:44 gmaxwell: i'm unconvinced 19:42:52 it doesn't hurt to benchmark 19:43:00 and maybe I'll tender a proposal to up the minimum difficulty, but I'd like to know what people think about that. 19:43:04 measuring is always better than making assumptions 19:43:11 with a replacement for sig skipping that isn't based on checkpoints we could significantly improve things 19:43:39 sipa: I don't think such a replacement can exist without changing the security assumptions; I'd *rather* have checkpoints than trusting hashing power for that 19:43:44 the last checkpoint currently is very old for the very reason that we've been planning to replace it 19:43:47 sipa: would you like to help work on a proposal for that? it has been controversial in the past. I'd like to do something good, because otherwise imprudent attempts will be adopted instead. 19:44:15 so it's unfair to use the "the last checkpoint is old" as a given; it's something we've affected indirectly 19:44:19 sipa: though what checkpoints should do is say "Something big has changed; you can disable checkpoints with --no-checkpoints, but you should find out what this means before doing so." 19:44:29 (for example Bitcoin Classic's current behavior simply looks at block header timestamps and ignores signatures when they're more than 24 hours (*par) old by the local clock. It's easily exploited and makes me sad. 19:44:40 petertodd: it's my opinion that on a timescale of months, it doesn't matter 19:44:53 IF you can guarantee it's actually a timescale of months 19:44:55 yes that makes me sad too 19:44:58 sipa: on a timescale of months, checkpoints shouldn't matter either... 19:45:06 anything based on time seems very brittle 19:45:27 petertodd: look at the current hashrate; what's 3 months worth of chain work at that hashrate 19:45:31 wumpus: and anything based on work isn't much better if you're running an old client, and mining has advanced significantly 19:45:37 sipa: I (hope) I'm not confused. If we would add a SPV hybrid mode directly fetch blocks at the tip (in order to received payments), no available checkpoint would result in downloading all headers *losing* maybe 3-4mins before you can start using SPV... minor issue though, I agree 19:45:38 sipa: that assumes you know what the current hashrate is 19:45:41 wumpus: the prior proposals were based on work, e.g. skip if the best chain you see dominates the next conflicted chain at that hight by N months of work. 19:45:44 gmaxwell: How have we solved the problem that checkpoints were originally created for? You have an excerpt in here: https://en.bitcoin.it/wiki/Bitcoin_Core_0.11_(ch_5):_Initial_Block_Download#Checkpoints 19:45:45 sipa: your node might be surrounded by sybils 19:45:52 wumpus: with a 'minimum total work' coded in as part of the release proces. 19:46:00 Chris_Stewart_5: headers first sync 19:46:08 Chris_Stewart_5: 0.10 19:46:08 Chris_Stewart_5: headers first sync. 19:46:09 gmaxwell: right. Well, at the least it should be measured whether such a change is really worth it. 19:46:16 petertodd: yes, i know... 19:46:21 so, let's measure. 19:46:25 and discuss later 19:46:37 Chris_Stewart_5: and the signature skipping behavior in checkpoints was actually a result of a bug fixed years ago.. mlock being used on all allocations making script validation INSANELY slow. 19:46:41 so much of the verification overhead is looking up UTXOs 19:46:44 sipa: okay. 19:46:47 something you'll not avoid 19:47:20 Chris_Stewart_5: but then with chain growth we became dependant on it to keep sync times reasonable. but libsecp256k1 made signature validation >5x faster. 19:47:21 especially for recent blocks 19:47:38 if you do any benchmarking please look at the recent blocks, not the first N 19:48:04 wumpus: it's still a major speed up on existing blocks. 19:48:07 in a side node: i've already updated my logging to measure bandwidth vs blockdepth instead of just count. 19:48:11 So header sync solves the attack of flooding disk space, but not having your entire network hijacked, correct? 19:48:32 Chris_Stewart_5: huh? 19:48:41 gmaxwell: sure, could be 19:48:42 Chris_Stewart_5: isolation can be resolved by simply knowing what the total work of the best chain was at release. 19:49:00 Chris_Stewart_5: sorry, this was discussed prior times removing checkpoints had come up, I haven't completely described the background. 19:49:19 gmaxwell: Thanks for the explanation, i'll keep digging. 19:49:45 Chris_Stewart_5: ah, you mean being isolated and being fed a wrong chain, sorry I was imaginging some wacky things at having your network hijacked :) 19:51:01 ok, next topic? 19:51:01 wumpus: just the "you got a faithful bitcoin core download but the attacker controls your network"... but that doesn't need a checkpoint to fix, a simple partitioning detction that knows the total work of the best chain at releast time is sufficient. 19:51:01 Thanks for the discussion. 19:51:13 #topic segwit against uncompressed keys or not 19:51:17 (10 minutes to go) 19:51:24 (9 minutes to go) 19:51:27 so to be clear, *just* segwit right? 19:51:30 does anyone still use uncompressed keys? 19:51:33 yes, only segwit 19:51:39 CodeShark: armory does 19:51:42 seems uncontroversial 19:51:49 I'm happy to ACK that given just segwit 19:51:57 having segwit enforce uncompressed keys would delay segwit adoption for armory users 19:52:03 *compressed 19:52:05 it's in #8499 19:52:09 achow101: why? just compress them 19:52:13 gmaxwell: yes, though we had a lot of trouble with partitioning detection, I remember some code being stripped out and such. But anyhow, yes that's the better approach if it can be gotten to work. 19:52:22 achow101: sigh, does armory still not do that? 19:52:30 luke-jr: we have to change the whole wallet structure (it's still going to happen anyways) 19:53:34 gmaxwell: without too much false positives 19:53:34 achow101: why? 19:53:34 achow101: alan said somewhere in 2013 he was implementing it... 19:53:34 alan's gone now.. 19:53:34 afaik the only downside to using compressed keys is it changes the address, which segwit is changing anyway 19:53:34 it's not a very complicated change 19:53:34 armory still uses uncompressed keys?! 19:53:34 there's no reason you'd need to change the wallet structure I can see 19:53:34 in any case this only applies to segwit, not to old transactions 19:53:34 the plan is to have a new wallet structure with bip32 that supports segwit and compressed keys 19:53:41 wumpus: "you're partitioned until you see a header chain with at least work X" is a pretty simple critera. :P 19:53:44 luke-jr: it had fixed size records in its wallet format for pubkeys 19:54:05 achow101: well if a new wallet format is needed for segwit anyway, it doesn't matter right? 19:54:10 achow101: oh god please do not use uncompressed keys with segwit. why would you do that? 19:54:13 sipa: zero-pad it? 19:54:35 sipa: well no, we don't need a new wallet for segwit as it could still work with the old one with a little bit of hacking 19:54:48 that was the original plan 19:54:48 achow101: no less than compressed could 19:55:15 sipa: or store the uncompressed key, and compress it at address-generation/signing 19:55:26 achow101: why cant the same hack that indicates segwit is in use indicate compressed.. you just chop off some bytes of the key pretty much. 19:55:43 btw, uncompressed keys account for 0.7% of used keys in succesful sigs on the network (in the past 2 hours) 19:55:44 it could be done entirely inside the process that seralizes the segwit scriptpubkey. 19:56:06 gmaxwell: idk. ask goatpig 19:56:06 achow101: okay 19:56:09 * michagogo pokes his head in belatedly 19:56:10 I think we should encourage all wallets to use compressed keys - achow101, if you need help with this I'd be willing to help 19:56:25 agree - we should help 19:56:27 yes, lots of people would be glad to help. 19:56:32 instead of just yell 19:56:50 well I offered to help armory move off uncompressed keys to alan several times, including offering to pay to do it. 19:56:56 so please don't say anyone just yelled. 19:58:39 I initially designed my account structures to only use compressed keys - but later added a compressed bit to support legacy stuff 19:59:06 CodeShark: what legacy stuff specifically? legacy armory users? 19:59:08 CodeShark: bah,it's kind of sad that to hear some things seem to be going back instead of forward :) 19:59:18 yes, to support other wallets 19:59:27 it's time 19:59:50 but I think we really do need to prod all wallets to move to compressed keys 20:00:07 there's really no reason to continue to support uncompressed keys - other than perhaps some migration tools 20:00:15 #endmeeting