19:00:21 <wumpus> #startmeeting
19:00:21 <lightningbot> Meeting started Thu Oct 31 19:00:21 2019 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:21 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:00:43 <jnewbery> hi
19:00:47 <warren> hi
19:00:51 <kanzure> hi
19:00:53 <achow101> hi
19:00:56 <amiti> hi
19:00:59 <moneyball> hi
19:01:01 <bitcoin-git> [13bitcoin] 15MarcoFalke pushed 2 commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/811c381ab650...222b7d0ca795
19:01:02 <bitcoin-git> 13bitcoin/06master 14c5377ff 15Suhas Daftuar: [qa] Add shrinkdebugfile=0 to regtest bitcoin.conf
19:01:02 <bitcoin-git> 13bitcoin/06master 14222b7d0 15MarcoFalke: Merge #17330: test: Add shrinkdebugfile=0 to regtest bitcoin.conf
19:01:13 <jeremyrubin> im unsure about DST
19:01:17 <jeremyrubin> Oh
19:01:20 <sipa> hi
19:01:21 <bitcoin-git> [13bitcoin] 15MarcoFalke merged pull request #17330: test: Add shrinkdebugfile=0 to regtest bitcoin.conf (06master...062019-10-shrinkdebugfile0) 02https://github.com/bitcoin/bitcoin/pull/17330
19:01:46 <provoostenator> hi
19:02:18 <wumpus> yes, DST in europe changed
19:02:32 <wumpus> it's an hour earlier here
19:03:05 <meshcollider> wumpus: don't forget to ping everyone :p
19:03:07 <meshcollider> hi
19:03:17 <fanquake> hi
19:03:37 <wumpus> four proposed topics in https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a : "moneyball: follow-up on travis and #16148" "umpus: close Boost → C++11 migration project for now" "proposed by dongcarl: MacOS, notarization https://github.com/bitcoin/bitcoin/issues/15774" "jeremyrubin: epoch mempool"
19:03:39 <gribble> https://github.com/bitcoin/bitcoin/issues/16148 | Travis timeouts · Issue #16148 · bitcoin/bitcoin · GitHub
19:03:56 <bitcoin-git> [13bitcoin] 15sdaftuar opened pull request #17332: [p2p] Proof-of-concept: Improve DoS-resistance to low-work headers chains (06master...062019-10-no-checkpoints-cleanedup) 02https://github.com/bitcoin/bitcoin/pull/17332
19:04:01 <digi_james> Hi
19:04:16 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball kvaciral ariard digi_james amiti fjahr
19:04:29 <wumpus> #topic High priority for review
19:04:47 <wumpus> currently 7 blockers, 7 chasing concept ACK
19:04:49 <wumpus> https://github.com/bitcoin/bitcoin/projects/8
19:05:23 <BlueMatt> plz2add #16974
19:05:24 <gribble> https://github.com/bitcoin/bitcoin/issues/16974 | Walk pindexBestHeader back to ChainActive().Tip() if it is invalid by TheBlueMatt · Pull Request #16974 · bitcoin/bitcoin · GitHub
19:06:28 <wumpus> BlueMatt: added (to blocker I guess?)
19:06:40 <BlueMatt> yea
19:07:08 <BlueMatt> it blocks post-headers-over-dns rust progress (since I want to lean heavily on "I have a better header than blocks, but my peers dont have blocks" as a criteria for searching for alternate block download methods)
19:07:27 <wumpus> yes, makes sense
19:07:59 <wumpus> nothing else too add/remove?
19:08:51 <wumpus> #topic close Boost → C++11 migration project for now (wumpus)
19:09:23 <warren> too much change required?
19:09:34 <wumpus> so it looks like all the low-hanging fruit for replacing boost has been picked now
19:10:07 <wumpus> what remains is hard to replace with C++11 (results in very verbose code, locale dependent legacy C, or introduces harder to maintain platform-specific code)
19:10:19 <jeremyrubin> It's basically just multiindex and boost::thread now right?
19:10:22 <wumpus> it's kind of a time wasting trap for developers now (regarding PRs like #17245)
19:10:24 <gribble> https://github.com/bitcoin/bitcoin/issues/17245 | wallet: Remove Boost from DecodeDumpTime by elichai · Pull Request #17245 · bitcoin/bitcoin · GitHub
19:10:52 <wumpus> nah, people stumble over the sleep and date/time handling stuff every time
19:11:21 <wumpus> #17307 might still be worthwhile
19:11:23 <gribble> https://github.com/bitcoin/bitcoin/issues/17307 | Stop using `boost::thread_group` · Issue #17307 · bitcoin/bitcoin · GitHub
19:11:31 <jeremyrubin> What about just checking in those specific copies of boost library code
19:11:48 <sipa> after 17307, won't removing boost::threa dbe a lot easier?
19:11:49 <jeremyrubin> Or are they too large/dependent on the rest of boost types
19:11:50 <wumpus> but it needs to be focused, we don't want to close the same PRs time after time that don't really make it
19:12:11 <wumpus> I think having that project open sends the wrong messag
19:12:15 <wumpus> we can't replace boost right now
19:12:36 <jeremyrubin> 17307, having worked on replacing thread_group in the checkqueue before, scares me a bit
19:12:37 <sipa> agree there
19:12:49 <wumpus> there's no need to replace, say, small string handling functions with our own impelentation, before we have a vision to replace the rest
19:13:33 <sipa> right; until we have a reasonable way to remove multiindex (which i'm not sure will ever happen), getting rid of boost entirely is not really useful
19:13:38 <dongcarl> Just so it's out there... we can maybe run bcp at some point when there's only one or two things left https://www.boost.org/doc/libs/1_71_0/tools/bcp/doc/html/index.html
19:13:49 <jeremyrubin> dongcarl: ++
19:13:55 <sipa> i do think there are individual improvements possible that dkn't go all the way, like removing boost::thread
19:14:13 <jnewbery> I agree that if it's not a priority then the project should be closed. It can always be re-opened later if necessary.
19:14:21 <wumpus> but in any case it doesn't seem like it needs a big coordinated project anymore
19:14:25 <sipa> how many non-headers-only boost libs are we still using?
19:14:39 <sipa> wumpus: agree
19:14:45 <jnewbery> (leaving a comment in the project description explaining why it's been closed)
19:14:58 <wumpus> when C++17 is allowed, it can be reopened
19:15:28 <fanquake> Guess #13751 can be closed with the same rationale as well then.
19:15:30 <gribble> https://github.com/bitcoin/bitcoin/issues/13751 | Utils and libraries: Drops the boost/algorithm/string/split.hpp dependency by l2a5b1 · Pull Request #13751 · bitcoin/bitcoin · GitHub
19:15:35 <wumpus> fanquake: yes
19:15:50 <wumpus> there's not really a reason to do that right now
19:16:33 <warren> gitian binaries static link the libstdc++ so distro support won't change when C++17 happens right?
19:16:43 <dongcarl> warren: Yes
19:16:47 <wumpus> warren: correct, assumning we can keep the same GLIBC req
19:16:50 <wumpus> (runtime)
19:17:24 <wumpus> which I don't see why not
19:17:58 <wumpus> statically linking using musl libc is another thing that might be considered in the future
19:18:03 <wumpus> but that's a whole different topic
19:18:15 <dongcarl> should be able to keep it as long as we 1. stick to Bionic 2. write new hooks, ick, or 3. use guix
19:18:33 <cfields> Bionic?
19:18:53 <dongcarl> cfields: bionic has 1.27, which is what we've written hooks to support up to
19:19:02 <dongcarl> glibc 1.27 that is
19:19:03 <sipa> i read "books" instead of hooks. was confused
19:19:38 <bitcoin-git> [13bitcoin] 15fanquake closed pull request #13751: Utils and libraries: Drops the boost/algorithm/string/split.hpp dependency (06master...06patch/remove_boost_split) 02https://github.com/bitcoin/bitcoin/pull/13751
19:19:39 <wumpus> hehe
19:19:44 <jnewbery> Shall I go ahead and update the project description and then close?
19:19:50 <wumpus> jnewbery: yes, thanks
19:20:02 <jnewbery> ok done
19:20:05 <wumpus> seems no one strongly disagrees
19:20:20 <wumpus> #topic MacOS, notarization (dongcarl)
19:20:31 <jnewbery> updated description is visible here: https://github.com/bitcoin/bitcoin/projects?query=is%3Aclosed
19:20:59 <dongcarl> Right, here
19:21:03 <wumpus> #15774
19:21:04 <gribble> https://github.com/bitcoin/bitcoin/issues/15774 | macOS App Notarization · Issue #15774 · bitcoin/bitcoin · GitHub
19:21:08 <dongcarl> Info available here: https://developer.apple.com/documentation/xcode/notarizing_your_app_before_distribution/customizing_the_notarization_workflow
19:21:11 <wumpus> jnewbery: great!
19:21:18 <dongcarl> Mostly just to make people aware of what's going to happen
19:21:30 <dongcarl> We're going to have a step in the release process that can only happen on a mac
19:21:33 <cfields> dongcarl: I think we should consider rejecting Apple's new requirement.
19:21:38 <wumpus> discntinuing MacOS binaries?
19:21:56 <dongcarl> cfields: that's something I've thought about too
19:22:03 <provoostenator> It depends on what the step is
19:22:04 * dongcarl typing gimme a sec
19:22:14 <cfields> I've been putting something together, probably best not to thought-dump here.
19:22:18 <provoostenator> As long as it's easy to review the difference on a non-Mac I don't mind too much.
19:22:33 <wumpus> htey make it harder and harder to distribute binaries for open source software
19:22:41 <luke-jr> [19:16:33] <warren> gitian binaries static link the libstdc++ so distro support won't change when C++17 happens right? <-- gitian binaries don't matter in this regard
19:22:49 <luke-jr> distro support is for NATIVE compiles
19:23:01 <luke-jr> (also, gitian binaries *don't* work on all distros)
19:23:02 <dongcarl> After we submit the app for notarization it is actually not NEEDED to staple it to the binary
19:23:14 <dongcarl> Apparently GateKeeper will query the Apple servers
19:23:18 <wumpus> luke-jr: that was last topic, but yes
19:23:24 <dongcarl> Which is convenient, but gives them A LOT OF CONTROL
19:23:28 <provoostenator> I'm not a huge fan of Apple getting a call from all our useres
19:23:31 <luke-jr> oh, I missed the #topic line
19:23:48 <provoostenator> So better to staple if it's not too difficult
19:23:50 <cfields> the bigger issue is that it makes Apple the final arbiter of what people can run. In a potential emergency fork/uasf scenario, this puts them in charge. I think it would be unwise for us to participate in that.
19:24:05 <luke-jr> wumpus: is Apple dropping support for unsigned binaries entirely?
19:24:10 <cfields> (final arbiter once we start down the path, that is)
19:24:19 <wumpus> yes, they want the same control as they have on iOS
19:24:24 <provoostenator> What changes when we go down the path?
19:24:24 <luke-jr> ugh
19:24:37 <moneyball> luke-jr: no you can still install but you have to "right click open"
19:24:51 <luke-jr> moneyball: right now, yes; but after these new changes?
19:24:57 <cfields> I think it's a mistake to engage that, because we can't go back. And if they refuse to sign a soft-fork, presumably there'd be nothing we could do.
19:25:10 <provoostenator> I don't understand why we can't go back
19:25:12 <luke-jr> cfields: we could timebomb..
19:25:22 <provoostenator> Do they change the installation rules once you notarize once?
19:25:24 <luke-jr> Knots already expires 1-2 years after the release
19:25:24 <dongcarl> I'm less worried about them refusing to sign
19:25:38 <dongcarl> I'm more worried about them signing: we've audited this binary and it's malicious
19:25:38 <moneyball> luke-jr: I run MacOS Catalina, am able to install 0.18 fine, and 0.19 RC3 I can install but need to "right click Open"
19:25:40 <cfields> provoostenator: if we engage once, then it would be a regression not to continue.
19:25:47 <dongcarl> That's a bad message to users
19:26:00 <provoostenator> cfields: arguably we're just kicking that regression down the road
19:26:16 <moneyball> If we make no changes, I think at minimum we need to communicate on the download site that one must "right click Open" otherwise many users will be perplexed as I was
19:26:19 <wumpus> it's not a regression if we never start supporting it
19:26:22 <provoostenator> Because Catalina users will encounter the regression (right click to install) now
19:26:27 <provoostenator> Which we can delay
19:26:34 <luke-jr> As long as users can run binaries w/o the notary, that sounds preferable
19:26:40 <wumpus> I think it's a good point, for a project like us, to be principled about it
19:26:43 <cfields> provoostenator: yes,( unpopular opinion time...) which is why it's worth considering dropping MacOs
19:27:04 <luke-jr> cfields: why drop entirely?
19:27:05 <cfields> I'm not saying that we should. I'm saying it's worth a discussion.
19:27:15 <wumpus> only when they *require* this
19:27:18 <wumpus> not right now
19:27:26 <luke-jr> can't our DMG just tell them right click etc in the folde rview?
19:27:31 <warren> are there other reasons aside from this?
19:27:32 <wumpus> but it's a possiblility in the future
19:28:03 <jeremyrubin> is there no more self-signing?
19:28:03 <wumpus> if it really becomes a closed platform
19:28:11 <sipa> cfields: stop supporting for release binarier, or stop supporting the platform? (i know the distinction is kinda vague for us, but we can still make sure our code compiles and runs fine, even if we're not providing binaries)
19:28:22 <jeremyrubin> Can we have a binary which we get apple to sign which self-signs our binary?
19:28:24 <wumpus> sipa: you can still build your own
19:28:27 <wumpus> sipa: using homebrew
19:28:49 <achow101> What changed between 0.18.x and 0.19 that 0.19 now triggers the warning but 0.18.x doesn't?
19:28:54 <jeremyrubin> Then apple would only have to sign it once, and then that can self-sign future cores
19:28:58 <wumpus> achow101: nothing in our code
19:29:11 <cfields> sipa: yes, thanks for making the distinction. The first would be far less problematic.
19:29:23 <dongcarl> Here's the risk that I see: someone takes our codesigned binaries, gives it to Apple, Apple maybe mistakingly decides that the binary is malicious, and now everyone's GateKeeper will ask Apple about that codesigned binary, which will give them a "THIS IS MALICIOUS" popup on double-click
19:29:25 <achow101> wumpus: moneyball said that 0.18.x installs fine, but 0.19 doesn't, on the same system
19:29:48 <wumpus> achow101: only because iti's more recent I think
19:29:50 <sipa> cfields: devil's advocate... is this very different from say snap controlling our snap releases? sure, snap is not the only way you run bitcoind on ubuntu, but depending on the user's expertise there may not be that much difference in how much control those entities have over what consensus code gets adopted
19:29:51 <cfields> dongcarl: yes, and that's inevitible. Because we ARE bitcoin core. You know, the thing they're scanning their binaries for :)
19:29:55 <fanquake> I'd assume he'd already been running 0.18
19:31:02 <moneyball> fanquake: me? as an experiment i installed 0.18 yesterday to compare to the experience installing 0.19
19:31:06 <achow101> I think it would be worthwhile to investigate why that happens and see if we are able to avoid the warning without doing any apple special stuff
19:31:08 <luke-jr> [19:27:31] <warren> are there other reasons aside from this? <-- macOS only works on backdoored hardware; macOS is proprietary; etc
19:31:31 <fanquake> moneyball yea. I don't think the reinstalling 0.18 would make a difference though.
19:31:42 <cfields> achow101: yes, agree.
19:31:42 <wumpus> even google isn't aiming for that kind of total control of android
19:31:48 <fanquake> Otherwise suddenly everyone who updated to 10.15's software would start breaking.
19:31:57 <wumpus> sipa: your comparison would hold for the apple app store, not for things installed outside it
19:32:01 <fanquake> I'd assume the new checks only apply to "new" binaries you are trying to run.
19:32:13 <cfields> Admittedly I don't have enough info on this. But what I've read has really rubbed me the wrong way. So I'll shut up now and chime in on the issue instead :)
19:32:16 <fanquake> So if you'd already been running 0.18 previously, I don't think reinstalling it is going to cause an issue.
19:32:54 <sipa> wumpus: fair point; but even on android you need to explicitly enable non-play store apk sources; that seems fairly similar to the right click "open anyway"
19:33:07 <fanquake> cfields: I assume your opinion is that we aren't shipping a macOS binary for 0.19 then? Assuming we're already at rc3?
19:33:09 <cfields> sipa: I'll have to think about that. I think it's different.
19:33:17 <provoostenator> I don't think it's very effective for a small (in terms of macOS user base) project to make a stance at this stage.
19:33:19 <wumpus> sipa: yes; I definiely don't think we should stop with MacOS binaries as long as that's possible
19:33:40 <dongcarl> Seems this is stirring up a lot of conversation, perhaps we can come back to this as the last topic?
19:33:42 <cfields> provoostenator: then I suppose we'd need to involve more projects :)
19:33:55 <ryanofsky> another danger of us not providing binaries is that someone else starts submitting and providing them, like happened with the snap store
19:34:01 <sipa> cfields: as far as ideology goes, supporting osx release binaries without participating in the gatekeeper stuff perhaps has more impact than dropping support entirely
19:34:02 <cfields> fanquake: Nah, for 0.19 I think we should just build/sign as before.
19:34:07 <jeremyrubin> ryanofsky: +1
19:34:09 <provoostenator> cfields: well, for starters it could be useful to get a bunch of projects together and try to get a physical meeting with Apple
19:34:14 <wumpus> provoostenator: I'm not really interested in having it being effective as a political strategy in general
19:34:20 <achow101> I can (probably) find a mac that doesn't have core installed and try
19:34:21 <provoostenator> They might be open to actually fixing this.
19:34:31 <wumpus> provoostenator: it would be a bad idea for bitcoin, so for bitcoin we should reject it
19:34:33 <fanquake> cfields: with just a warning / opening instructions on the website / in the dmg?
19:34:40 <sipa> provoostenator: i very much doubt that
19:34:50 <fanquake> ^
19:35:20 <sipa> provoostenator: i suspect that from their perspective, the fact that people can run unvetted binaries is a historical legacy that needs to be fixed :)
19:35:23 <provoostenator> Well if the answer is "we don't want to fix this, we like a gated community" then that's a nice quote to start a boycott.
19:35:54 <ryanofsky> dongcarl, if we do decide to "staple" the apple signature, is it still easily possible for a user to verify the executed code was built reproducibly & signed by us?
19:36:25 <provoostenator> ^ that's the most important question imo
19:36:27 <cfields> ryanofsky: that's already how it operates. We "staple" on the detached sig. It's easily auditable.
19:36:32 <jeremyrubin> Odd question: if we ship a binary + a VM, can we just run it in the VM always?
19:36:37 <wumpus> I'm not going to upload any binaries that aren't possible to verify in a distributed way
19:36:40 <cfields> I assume it'd be the same here. If not, surely we could write a tool to handle it.
19:37:10 <ryanofsky> ok, that's good at least
19:37:21 <achow101> is it possible to "staple" without a mac?
19:37:47 <dongcarl> not possible to "staple" without a mac right now
19:37:50 <fanquake> You'd need Xcode and associated tools
19:37:51 <cfields> It's not possible to do the detached sigs without a mac either.. the tooling is our own.
19:37:52 <sipa> is it possible to undo the stapling on non-mac?
19:38:04 <cfields> Er, let me rephrase...
19:38:36 <cfields> There's no supported way sign on non-mac as we do now, but that didn't stop us from hacking something together :)
19:39:02 <achow101> someone would have to reverse engineer how it's "stapled" and write a tool for it for other platforms
19:39:03 <cfields> So I assume we could do something similar again. Might turn into a 3-stage gitian build, though :(
19:39:09 <dongcarl> what cfields said
19:39:18 <fanquake> Liking something we can take away from this discussion: https://trac.torproject.org/projects/tor/ticket/30126
19:39:22 <fanquake> *likely
19:39:41 <sipa> cfields: what if instead of doing the stapling in the 2nd/3rd gitian stage, we write a tool that can compare a stapled binary against a deterministic one?
19:39:49 <sipa> that could even remove the need for the 2nd stage we have now
19:40:05 <wumpus> cfields: adding a slight delay is usually not a problem
19:40:12 <dongcarl> sipa: A notarization strip tool?
19:40:16 <sipa> dongcarl: yes
19:40:37 <cfields> sipa: I don't think it's possible to be deterministic because of the timestamping, but maybe I'm misunderstanding.
19:40:54 <sipa> let me start over
19:40:56 <dongcarl> cfields: He's talking about our existing deterministic codesigned binary
19:41:21 <sipa> do we actually care that the codesigning/timestamping/notarizing step is deterministic?
19:41:26 <wumpus> yes
19:41:37 <sipa> the point of determinism is showing that the shipped binaries match our source code
19:41:49 <wumpus> at least the attach-signature step
19:41:58 <warren> presumably the timestamp is part of the stapled part, while the executable part could be verified after stapling?
19:42:11 <wumpus> the signing itself doesn't need to be determinsitic, it's only done once by one person
19:42:21 <sipa> if you could *strip* the codesigning from a binary, and compare the result of that with the deterministic unsigned result, don't we achieve the same thing?
19:42:42 <sipa> in that case the notarizing/codesigning could be done entirely outside of gitian, removing the 2 phase process we have now
19:43:02 <sipa> that of course only works if the stripping tool is easily auditable
19:43:12 <cfields> sipa: I see. You'd have to trust the stripping tool and be able to audit the diff, but that could work.
19:43:28 <sipa> right, it depends on how complicated the stripping is
19:43:35 <wumpus> so instead of uploading my own gitian results, I'd have to upload the binary someone else sends me
19:43:43 <cfields> sipa: IIRC there's a reason we didn't do it that way. Can't seem to remember though, maybe not.
19:44:02 <dongcarl> how many people do the codesigning right now?
19:44:11 <sipa> wumpus: you'd upload the codesigned binary, but the gitian results contain the non-codesigned hash
19:44:23 <sipa> the comparison tool strips the codesigning, and compares with gitian
19:44:23 <achow101> dongcarl: 2. cfields for windows, jonasschnelli for mac
19:45:47 <dongcarl> right, if it's just one person for each platform then perhaps stripping is not too bad?
19:46:02 <cfields> sipa: ah, at the time, it might've just been about the tooling. Easier to add than remove. That likely isn't an issue anymore though, there's better tooling now.
19:46:33 <wumpus> at least now it's possible to look up the sha256 of the binaries from the SHA256SUMS.asc in gitian, and see they match, this would add an extra step
19:46:39 <wumpus> I don't like it but ok...
19:46:46 <sipa> wumpus: agreed, it's annoying; but needing to comply with ever-increasing notarization/codesigning requirements is also annoying
19:47:13 <sipa> it's just an idea; no need to decide anything now
19:47:37 <wumpus> this wouoldn't avoid that right?
19:47:55 <fanquake> Apparently we wont actually need the secure timestamp or hardened runtime until January next year
19:48:00 <fanquake> https://developer.apple.com/news/?id=09032019a
19:48:16 <provoostenator> You can still put the sha of the signed binary in the gitian result, along with the sha of the stripped version.
19:48:17 <cfields> Bitcoin 0.20 when? :p
19:48:19 <wumpus> why even bother with bitcoincore.org uploads anymore, if you go through all these hoops, can't you put it on the app store directly?
19:48:20 <achow101> that's in two months...
19:48:24 <dongcarl> wumpus: this would avoid us having to reverse engineer each notarization/codesigning step and writing a tool to do it under our reproducible builds process
19:48:35 <jeremyrubin> Here's a reductionist viewpoint: It's not worth jumping through too many hoops for apple users, because ultimately if the person shipping your kernel decides you don't want them running bitcoin, or a version of bitcoin, you're already, to put it lightly, fucked
19:48:44 <fanquake> achow101 at least it means a 0.19.0 release is simpler...
19:48:53 <fanquake> and gives us time to figure out future tooling
19:49:05 <jeremyrubin> So this is maybe a problem we can't really solve for OSX users, short of giving them a liberated computer
19:49:31 <sipa> back to short-term issue: are our processes already using the secure timestamp and/or hardened runtime?
19:49:36 <provoostenator> wumpus: app stores come with additional problems though, like auto-update
19:49:36 <sipa> or how hard is it to do so
19:49:47 <wumpus> let's go to the next issue, already spent too much time on apple junk
19:50:04 <wumpus> two topics left and 10 minutes
19:50:07 <sipa> ok
19:50:12 <wumpus> #topic follow-up on travis and #16148 (moneyball)
19:50:13 <gribble> https://github.com/bitcoin/bitcoin/issues/16148 | Travis timeouts · Issue #16148 · bitcoin/bitcoin · GitHub
19:51:32 <wumpus> moneyball?
19:51:38 <moneyball> hi
19:51:50 <moneyball> my name is on the topic but it is from 1 month ago
19:51:57 <instagibbs> moneyball, don't timeout in 10 minutes ;)
19:51:57 <moneyball> when we discussed the same topic and said punt 1 month
19:51:58 <wumpus> oh
19:52:03 <moneyball> since then the PR was closed
19:52:13 <moneyball> maybe MarcoFalke can discuss that?
19:52:29 <moneyball> also we were wondering the status of jonasschnelli's related project
19:52:58 <wumpus> ok, might be better to move to jeremyrubin's topic for now then
19:53:03 <moneyball> sure
19:53:09 <wumpus> #topic  epoch mempool (jeremyrubin)
19:53:12 <jeremyrubin> Hi!
19:53:31 <jeremyrubin> So Epoch Mempool is a relatively big upgrade to the way the mempool tracks relatives state
19:53:44 <jeremyrubin> We replace a ton of std::sets with epoch tracking for touching txiters
19:54:07 <sipa> yay
19:54:21 <jeremyrubin> This is done because we have restrictive limits on the number of descendants, which is a problem for protocols like lightning or OP_STB
19:54:43 <jeremyrubin> The PR is just a start, but it's a good checkpoint on improvement progress
19:54:59 <jeremyrubin> (I will have follow ups that are further improvements)
19:55:30 <sipa> it's clear to me that that approach in theory at least should be possible and improve performance; i haven't looked at the code
19:55:30 <jeremyrubin> But the critical question is: what standard of "evidence" do we want to see to be comfortable with such an improvement not introducing regressions in performance?
19:55:41 <instagibbs> it's also a problem for services due to simple transaction pinning
19:55:47 <wumpus> is mempool performance a bottleneck?
19:55:58 <sipa> wumpus: for increasing the mempool limits, it is
19:56:12 <sipa> (package size, ancestor depth, ...)
19:56:19 <wumpus> okay
19:56:58 <instagibbs> sipa, is there a good writeup of "all" the reasons for the limits? There are 1 or 2 on suhas' writeup on the wiki, but likely not exhaustive of collective wisdom
19:57:07 <sipa> instagibbs: not that i know
19:57:08 <ariard> is this a change only in the way we traverse mempool sets or also in the way we compute entry feerate?
19:57:18 <jeremyrubin> Traversal only as of now
19:57:27 <jeremyrubin> The notes aren't really fully up-to-date either
19:57:34 <sipa> instagibbs: and sdaftuar likely knows better than i do
19:57:49 <instagibbs> I think I've hit up sdaftuar for all he currently has written done(which is better than 0!)
19:58:08 <ariard> hmmmm did you spend time to think how we compute feerate? improving this point first could ease traversal
19:58:34 <jeremyrubin> Feerate isn't really the chief issue here IMO
19:58:36 <sipa> ariard: what do you mean by compute feerate?
19:58:48 <instagibbs> https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Mempool-and-mining poached slides being referenced
19:58:50 <jeremyrubin> But it does cause a lot of traversal-ing
19:58:54 <ariard> tracking ancestor/descendant size/fees
19:59:11 <instagibbs> the modified feerate you mean
20:00:08 <jeremyrubin> In either case, the feerate stuff is easier to deal with as it (iirc) ends up being O(N)-ish if you traverse in the correct order.
20:00:28 <jeremyrubin> But the speed of those traversals you have to pay on adding a txn or removing one
20:00:30 <ariard> yes nModifiesFees
20:00:33 <wumpus> let's wrap up please, it's almost time to close the meeting
20:00:36 <jeremyrubin> Ok
20:00:59 <jeremyrubin> I guess please review #17268
20:01:01 <gribble> https://github.com/bitcoin/bitcoin/issues/17268 | Epoch Mempool by JeremyRubin · Pull Request #17268 · bitcoin/bitcoin · GitHub
20:01:13 <wumpus> #action review #17268
20:01:13 <luke-jr> jeremyrubin: 1) what is "epoch tracking" ?
20:01:15 <gribble> https://github.com/bitcoin/bitcoin/issues/17268 | Epoch Mempool by JeremyRubin · Pull Request #17268 · bitcoin/bitcoin · GitHub
20:01:21 <wumpus> #endmeeting