19:01:03 #startmeeting 19:01:03 Meeting started Thu Aug 8 19:01:03 2019 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:03 Useful Commands: #action #agreed #help #info #idea #link #topic. 19:01:07 hi 19:01:08 hi 19:01:09 hi 19:01:10 hi 19:01:10 hi 19:01:13 hi 19:01:13 hi 19:01:15 hi 19:01:16 hi 19:01:33 #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball kvaciral 19:01:40 hi 19:01:42 Hi 19:01:46 hi 19:01:58 hi 19:02:04 I'd rather stick with some GitHub specific stuff that works than to ditch GitHub specific stuff and make running the tests harder for developers 19:02:13 any last-minute proposed topics? 19:02:22 hey 19:02:25 https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a 19:02:26 there's one by MarcoFalke: Who wants to volunteer as co-maintainer of the Bitcoin Core flatpak 19:02:29 wumpus: could you remove my proposed? 19:02:47 dongcarl: moneyball already did 19:02:50 We can always keep the bitcoin-builds as a backup fallback ci 19:02:53 MarcoFalke, it really depends on how powerful their build servers are, for a relatively small price a very powerful server could be acquired and used for builds 19:03:02 #proposedmeetingtopic GitHub ci 19:03:42 #topic High priority for review 19:03:44 phantomcircuit: Not every developer has a credit card, even if they can afford the price of the ci 19:03:46 https://github.com/bitcoin/bitcoin/projects/8 19:03:56 I don't see why we can't just use both. would it not be possible to make bitcoinbuilds just read the same config file as github's? 19:04:02 6 blockers, 6 chasing concept ACKs 19:04:08 why are we talking about credit cards ? 19:04:08 Agree with achow101 19:04:08 (lets take the CI discussion to #bitcoin-builds plz) 19:04:21 wumpus: Sorry, will shut up now about ci 19:04:48 I think the BIP9 bury PR is very near merge-ready 19:04:52 #16060 19:04:56 https://github.com/bitcoin/bitcoin/issues/16060 | Bury bip9 deployments by jnewbery · Pull Request #16060 · bitcoin/bitcoin · GitHub 19:05:18 anything to add/remove/replace from the project? 19:05:22 It is still in the process of review and some fixups keep getting found 19:05:40 (re 16060) 19:05:50 MarcoFalke: mostly small nits on nits about RPC behavior, from what I've seen 19:05:55 #8994 can be removed from chasing concept ack? 19:05:58 https://github.com/bitcoin/bitcoin/issues/8994 | Testchains: Introduce custom chain whose constructor... by jtimon · Pull Request #8994 · bitcoin/bitcoin · GitHub 19:06:10 MarcoFalke, no what im saying is that we could collectively buy a server that could do the builds, in general things where you're buying "packaged" cpu time from third parties end up being very expensive 19:06:17 wumpus: there was at least a bug in the test case which I'm fixing now 19:06:25 I would appreciate rereview after I push that fix 19:06:47 jnewbery: sure 19:07:02 jnewbery: Will take another look in the next couple of days 19:07:35 Also, I plan to review sdaftuar's pr (it has been on my list for a while, but I haven't gotten to it yet) 19:08:05 thanks 19:08:40 wumpus: first topic? 19:08:53 we're done with high priority for review? 19:09:10 * jonasschnelli waves at #16202 (quick review, on the path to p2p encryption) 19:09:12 https://github.com/bitcoin/bitcoin/issues/16202 | Refactor network message deserialization by jonasschnelli · Pull Request #16202 · bitcoin/bitcoin · GitHub 19:10:02 #topic Who wants to volunteer as co-maintainer of the Bitcoin Core flatpak (MarcoFalke) 19:10:14 who currently maintains it? 19:10:21 can someone explain flatpak? 19:10:26 It isn't created yet 19:10:36 jonasschnelli: It is like a snap package 19:10:42 it's a distribution agnostic binary package distribution format 19:10:44 Different name, though 19:10:44 for linux 19:11:45 Okay. Sounds acceptable. 19:11:47 it's especially good for GUI applications, it has some features like sandboxing and desktop UI integration which simply dropping static binaries has not 19:12:21 I will be a repository hosted under the flatpak org, so not in our org. Which is why I am asking for co-maintainers. 19:12:24 see #16550 19:12:25 https://github.com/bitcoin/bitcoin/issues/16550 | Distribute via flatpak · Issue #16550 · bitcoin/bitcoin · GitHub 19:12:54 i'd love to help but can't promise to have any time and motivation for any extra work at the moment, sorry 19:13:06 Sure. Maybe jonasschnelli? 19:13:10 Or fanquake? 19:13:20 it seems flatpak is also usable independent of a central repo 19:13:34 while snap really needs canonical's infrastructure? 19:13:48 jup, snap needs the Ubuntu infrastructure 19:13:55 MarcoFalke: what’s required to maintain? Just uploading releases and managing issues? 19:14:04 flatpak has flathub, but it's not required 19:14:17 fanquake: Only requirement is that you are a less agressive jaywalker 19:14:23 (just reading and learning) 19:14:28 👀 19:14:40 Sorry... already at my timelimit 19:14:42 MarcoFalke: one must know how to spell aggressive correctly, however 19:15:09 right 19:15:14 Can’t promise anything 🏃‍♂️ 19:15:19 Is there an autocorrect for IRC? 19:16:15 fanquake: Cool, I'll sign you up. 19:16:28 hehe,switch to matrix, it has message editing 19:17:02 MarcoFalke: heh, I was worried that just by asking a question that would happen 19:17:44 wumpus: Next topic then? 19:18:05 #topic GitHub ci (MarcoFalke) 19:18:22 fanquake: I expect it will look like this: https://github.com/bitcoin-core/packaging/pulls?q=is%3Apr+author%3AMarcoFalke+is%3Aclosed 19:18:31 I.e. bump version every 3 months 19:18:39 So nothing too fancy 19:18:51 ok, next topic 19:18:53 MarcoFalke: ok 👍 19:19:26 I think everyone knows by now that travis isn't the most stable ci solution 19:19:33 PSA: first deadline (soft translation string freeze) for 0.19.0 is in less than a month, 2019-09-01 19:20:14 So in exploring alternatives, the GitHub ci might be a good medium-term replacement 19:20:14 MarcoFalke: definitely, it's not a good fit for how we're using it at least 19:20:41 MarcoFalke: is the GitHub CI public info? 19:20:47 Both, travis and the GitHub ci, will be GitHub-centered. That should be fine, imo. 19:20:49 https://github.com/features/actions 19:20:53 ah, i see it is 19:20:56 they just announced it today, it's in beta 19:20:57 I've never heard of github ci 19:20:57 moneyball: Yes, according to achow101 19:21:12 when Pieter, Cory, Matt, and I visited GitHub, they demo'd this 19:21:23 but if it can take over what travis does and does it beter, without too much work from our side, that'd be very useful 19:21:48 cfields has strong views on this 19:21:50 wumpus: Jup, agree 19:22:06 moneyball: any opinions about it from the demo you saw? 19:22:07 moneyball: cfields: Elaborate, pls 19:22:08 cfields can't be here today afaik 19:22:19 moneyball: can you summarise the strong feelings? 19:22:23 Also agree. 19:22:39 strong feelings in a good or bad way? 19:22:50 bad. 19:22:59 Concerns about increasing dependence on GitHub/Microsoft 19:23:06 Yes 19:23:47 Again, I don't think it gets worse. Also, we can use jonasschnelli's ci as a fallback (running in the background 24/7) 19:23:51 jonasschnelli: do you want to update on the state of your CI? As an alternative? 19:23:56 Which I agree is a valid point. My response was "What kind of productivity improvement over Travis would we need to see to want to move in that direction?" but I didn't really get a good answer. So, I'd pose that question here. 19:23:56 Update on the self hosted custom open source CI (bitcoinbuilds.org): it does now (since a week) successfully builds all PRs, mostly quicker than travis (though less envs). Integration with Github is doable and easy. But first lets see how it runs for at least 2 months. 19:24:21 Plan is to use the Github checks API when we think it has been proven to be reliable (2 month) 19:24:29 moneyball: "no false positives" 19:24:34 Another point made was concern that GitHub 3rd party API might suffer / degrade over time making 3rd party tools like Travis even worse (relative to GitHub CI) 19:24:42 (or at least, neglible amount of, the problem with travis is false positives) 19:24:53 one point is that it seems to provide far more entry points than a normal CI does (it does much more automation of workflows, not just CI), and starting to depend on those seems like it would make it harder to move away (if ever needed) 19:25:10 good point 19:25:17 Yeah, good point. 19:25:26 I wouldn't want to rely on it for automation of workflows 19:25:33 Heh. We’ve already got plenty of GH “turned off” as well. 19:25:40 I'm not sure if their announcement today means anyway can experiment with it not, but if not, then we have an invitation to experiment with it, if anyone is interested. 19:25:40 just taking over what travis does now 19:25:41 They tend to mix auto-fix (code format correction and stuff) with CI which is indeed not only good 19:25:46 as long as it's a drop-in replacement of one CI for another, i agree there isn't much impact 19:25:47 We should only use it for the purpose of ci builds 19:25:54 auto-fix ? no way 19:26:08 " You’re in the queue for the beta—we’ll let you know as soon as you have access. " 19:26:12 that sounds realllllly scary 19:26:14 damn 19:26:37 that's just an example of something you can configure 19:26:43 not a default thing 19:26:47 yes, all of the other stuff is just scary 19:26:49 so now you CI doesn't only have false positives, it can introduce bugs and backdoors into your code ! 19:26:51 wumpus: https://developer.github.com/apps/quickstart-guides/creating-ci-tests-with-the-checks-api/#step-26-automatically-fixing-rubocop-errors 19:27:34 Like self-driving cars for software development 19:27:41 yea, lt's disable it, as fanquake says we have plenty of gh features disabled, what is one more … though it's something to watch 19:27:43 yes, exactly 19:28:17 jonasschnelli: we've "tested" other CIs live - no reason not to do it with our own 19:29:03 So is the resolution here to just wait until we can look at GH CI properly? Maybe test it in production alongside Travis? 19:29:15 that sounds good to me 19:29:22 ack 19:29:24 ack 19:29:25 Does GH CI work with non-GH repos? 19:29:30 no 19:29:38 luke-jr: i highly doubt it 19:29:38 luke-jr: only if you mirror them to github, I guess 19:29:39 luke-jr: embrace, extend, ... 19:29:43 I'm with cfields on that then… 19:29:56 luke-jr: Does travis work with non-GH repos? 19:30:08 I mean, if it's the only thing working, fine temporarily I guess, but I'd rather get away from GH dependency, not more entrenched 19:30:17 luke-jr: definitely agree on that 19:30:18 MarcoFalke: not sure 19:30:31 I really thing we should just do both GH CI and bitcoinbuilds and just have them use the same config file 19:30:41 It looks like GitHub is taking over Git the same way gmail took over email... be aware 19:30:42 does anyone want to test the GitHub CI now? let me know and I will introduce 19:30:44 achow101: +1 19:30:52 I wonder how long until there's some kind of standard for CIs 19:30:56 Nice centralized development functionality comes at a price 19:31:01 so we aren't entirely dependent on Github, but for individual developers, they can still run CI checks 19:31:08 achow101: i like that idea; it also forces us to keep the configuration to be compatible with non-GH features 19:31:13 luke-jr: seems no, travis cannot use anything else than github, unless you have the enterprise version 19:31:17 moneyball: does it require permission to use? 19:31:20 jonasschnelli: could test in production, but have it report "success" or "neutral" rather than "success" or "failure" maybe? 19:31:25 luke-jr: during the beta, yes 19:31:37 achow101: "make cichecks" would be nice :D 19:31:56 luke-jr: can't run on all of the platforms on one machine 19:32:06 achow101: why not? :\ 19:32:11 qemu 19:32:17 aj: Yes. That would make sense. 19:33:16 so, perhaps _someone_ should try out the GitHub CI, and see if it's better than Travis, as a stop gap 19:33:20 dongcarl: from my testing a few years ago, qemu-user works pretty well for the tests, at least if you stick to platforms with the same endianness 19:33:41 it sounds like bitcoinbuilds is further along than GitHub CI? 19:33:53 Could someone share a writeup of how to run the functional tests in qemu, pls? 19:34:15 MarcoFalke: write a shell script that runs the bitcoind bin, and export BITCOIND=/path/to/script 19:34:26 moneyball: you said we have an invitation to experiment with github CI? 19:34:51 Yes 19:34:52 can we skip the queue? 19:35:06 Presumably since they offered to let us use it a month ago 19:35:30 Is there a zero-effort Travis yml -> GH CI transition? 19:35:37 i doubt that 19:35:40 dongcarl: no 19:35:44 :-/ 19:36:03 Why not stick to Travis till bitcoinbuilds is ready? 19:36:16 But the travis.yml mostly specifies the env, so it shouldn't be too hard either 19:36:41 dongcarl: because travis false positives too much 19:36:41 yeah, I imagine at least some of the scripts should be portable 19:36:51 I'd still like to run a ci with free minutes along of bitcoinbuilds 19:37:00 Me 2 19:37:16 "free minutes along of bitcoinbuilds"? 19:37:20 moneyball: can you ask them let some of us into the beta so we can try it out on our own repos first? 19:37:22 fee cpu time 19:37:24 I think centralized development tools are fine as long as there is an alternative running in parallel 19:37:31 *free 19:37:33 ah 19:37:41 I have lots of free ppc64le CPU time I think 19:37:49 The problem is lock-in, suddenly they charge for it or make it incompatible with your needs, ...and you'r screwed 19:37:50 jonasschnelli: Agreed. 19:37:58 achow101: yes would you like access? 19:38:10 sure 19:38:10 achow101, travis is mostly hitting fp on resource exhaustion issues 19:38:12 moneyball: they can't just enable the entire repo +forks? :/ 19:38:26 i don't know 19:38:28 yeah, let's ask them how this works 19:38:36 i will clarify with them 19:38:47 just enable it for anyone in the bitcoin and bitcoin-core orgs :) 19:38:53 Let's not enable the repo for now and keep testing in forks only 19:38:58 I'd also hope we consider some ethos and not further extend our reliance on github 19:40:16 jonasschnelli: I agree, it's just, if we don't rely on anything further then switching from travis to github for CI doesn't make a difference, they're both centralized and githyb only 19:40:40 wumpus: any other topics? 19:40:49 nope 19:41:02 wumpus: There is maybe no technical difference,... but the signal to the community/world is different 19:41:41 Ok, that is a point I buy. They might say the GitHub ci is "approved by Bitcoin Core" 19:41:52 Could someone share a writeup of how to run the functional tests in qemu <- I'll try if I get around to it, I think the most difficult part is setting up the faux-root filesystem for the target, though on some distros you can install that as package 19:42:03 MarcoFalke: yes, that's scary 19:42:21 "Bitcoin Core implemente GitHub CI after talks with their CEO,... a week after they acted in massive censorship" 19:42:40 (I know I overstretch it) 19:43:11 Then we’d all have to jump on Medium and explain our actions in an emoji filled post mortem. 19:43:58 might be a harder story to spin if we have our own CI at the same time 19:44:11 we could deploy both github CI and bitcoinbuilds simultaneously? 19:44:11 haha way to go, fly ahead of the rumors in the meeting 😅 19:44:24 "We're migrating to our own CI, but helping GitHub test theirs too" 19:44:27 I'll experiment with the ci a bit in the coming days to see if it fits our use case. We can discuss more after everyone had a look 19:44:41 thanks MarcoFalke 19:44:46 +1 19:45:25 yes, let's try that, maybe it's worse a match for our CI needs than travis is, then it's not even worth discussing further 19:45:46 #endmeeting