19:00:54 #startmeeting 19:00:54 Meeting started Thu Jan 24 19:00:54 2019 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:54 Useful Commands: #action #agreed #help #info #idea #link #topic. 19:00:55 hi 19:01:14 topic proposed by jnewbery: Chaincode summer residency: looking for (remote) mentors and recommendations for residents 19:01:20 suggested topic: globals and initialization order 19:01:50 hi 19:01:51 #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb 19:01:54 hi 19:01:55 hi 19:01:58 yo 19:02:03 \o 19:02:06 hi 19:02:29 #topic High priority for review 19:03:09 #11082 #14491 #14711 #14897 #15153 #15226 19:03:13 https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub 19:03:16 https://github.com/bitcoin/bitcoin/issues/14491 | Allow descriptor imports with importmulti by MeshCollider · Pull Request #14491 · bitcoin/bitcoin · GitHub 19:03:21 https://github.com/bitcoin/bitcoin/issues/14711 | Remove uses of chainActive and mapBlockIndex in wallet code by ryanofsky · Pull Request #14711 · bitcoin/bitcoin · GitHub 19:03:23 https://github.com/bitcoin/bitcoin/issues/14897 | randomize GETDATA(tx) request order and introduce bias toward outbound by naumenkogs · Pull Request #14897 · bitcoin/bitcoin · GitHub 19:03:25 https://github.com/bitcoin/bitcoin/issues/15153 | gui: Add Open Wallet menu by promag · Pull Request #15153 · bitcoin/bitcoin · GitHub 19:03:27 https://github.com/bitcoin/bitcoin/issues/15226 | Allow creating blank (empty) wallets (alternative) by achow101 · Pull Request #15226 · bitcoin/bitcoin · GitHub 19:03:55 anything to be added or removed? 19:04:20 can I have #15225 on hi prio? 19:04:23 https://github.com/bitcoin/bitcoin/issues/15225 | GUI: Change the receive button to respond to keypool state changing by achow101 · Pull Request #15225 · bitcoin/bitcoin · GitHub 19:04:27 regarding open wallet menu - there are concerns regarding blocking GUI - is this something to avoid or can it be improved in 0.18.1? 19:04:30 #15118 19:04:33 https://github.com/bitcoin/bitcoin/issues/15118 | Refactor block file logic by jimpo · Pull Request #15118 · bitcoin/bitcoin · GitHub 19:04:37 it's actually a blocker for 15226 19:05:07 14711 should be almost mergeable 19:05:13 +1 for #15118. It blocks #14121 19:05:15 https://github.com/bitcoin/bitcoin/issues/15118 | Refactor block file logic by jimpo · Pull Request #15118 · bitcoin/bitcoin · GitHub 19:05:19 https://github.com/bitcoin/bitcoin/issues/14121 | Index for BIP 157 block filters by jimpo · Pull Request #14121 · bitcoin/bitcoin · GitHub 19:05:22 though promag keeps commenting so I'm not sure xD 19:05:39 I am hoping for movement on #13932 but I think it needs love from achow101 more than review at present 19:05:42 https://github.com/bitcoin/bitcoin/issues/13932 | Additional utility RPCs for PSBT by achow101 · Pull Request #13932 · bitcoin/bitcoin · GitHub 19:05:57 gwillen: oh yeah, I forgot to update that 19:06:06 wumpus: :( 19:06:09 achow101: lmk if there's any way I can help! 19:07:16 promag: I don't mean that negatively! if there's issues there's issues no matter *when* you find them, just mean it postpones the merge if there's new review comments 19:07:58 ok added #15225 #15118 19:08:00 https://github.com/bitcoin/bitcoin/issues/15225 | GUI: Change the receive button to respond to keypool state changing by achow101 · Pull Request #15225 · bitcoin/bitcoin · GitHub 19:08:02 https://github.com/bitcoin/bitcoin/issues/15118 | Refactor block file logic by jimpo · Pull Request #15118 · bitcoin/bitcoin · GitHub 19:08:05 wumpus: thanks 19:08:09 let's also try to remove (merge) a few this weke 19:08:36 hi 19:09:02 8 is kind of a lot to have on the list, and closer to the release the focus shifts to the milestone instead of this list 19:10:00 #15141 is ready for review again btw 19:10:03 so 2019-02-01 is string freeze, 2019-02-15 is feature freeze for 0.18 19:10:04 https://github.com/bitcoin/bitcoin/issues/15141 | Rewrite DoS interface between validation and net_processing by sdaftuar · Pull Request #15141 · bitcoin/bitcoin · GitHub 19:10:04 oops 19:10:18 oh, wait i did type that correcly 19:10:26 sdaftuar: cool will review 19:10:27 hi 19:11:07 #topic Globals and initialization order (sipa) 19:11:21 hi! 19:11:56 we currently have a bit of a mix in the codebase for dealing with initialization order of globals 19:12:29 some things are explicitly initialized using init functions, called from main/test startup 19:12:42 some things are initialized just using global initializers 19:13:08 and some things are using once/init-on-first-use-block-scoped-statics 19:13:32 and mixing them is pretty fragile 19:14:16 I prefer explicit initializer functions unless it's simply setting a value, at least it's perfectly clear what the order is in which things get initialized 19:14:37 which is very important if things depend on each other 19:14:49 also things running before main() is quite annoying for debugging 19:14:52 the problem is that with explicit initializer functions, things don't work when called from global initializer 19:16:08 (and things that take significant time to run certainly shouldn't be called from global initializers, as they'll delay showing even, say, the help message, which doesn't need any initialization at all) 19:16:32 and i think we actually had a long-standing problem with the RNG, which was used possibly before being initialized (since #14955 it uses an init-on-first use construction, which should always be fine) 19:16:36 https://github.com/bitcoin/bitcoin/issues/14955 | Switch all RNG code to the built-in PRNG by sipa · Pull Request #14955 · bitcoin/bitcoin · GitHub 19:17:18 anyhow that's my opinion on it, I'm aware the codebase is quite crazy in that regard, we've had initialation order issues since the first release... 19:17:44 sipa: what's your preference? 19:17:49 so i'm getting more and more in favor of using this init-on-first-use construction in more places 19:17:57 meh 19:18:00 since it's compatible with everything 19:18:12 it's hard to reason about 19:18:35 I also like explicit initialization of non-trivial things 19:18:36 accidentally using something before something else will suddenly change the initialization order 19:18:38 instead of just fail 19:18:46 we've seen that with logging, for example 19:18:58 that's because logging using a global initialized 19:19:01 I'm in favor of having fewer globals :-) But otherwise haven't really developed a preference in C++ 19:19:28 I prefer explicit order initialization 19:19:49 initializing on first use also doesn't really regard initialization dependencies 19:19:49 wumpus: in a bad way? the only time I can think of order mattering is when we're pre-config-files-loaded 19:20:09 wumpus: how so? 19:20:10 like "we need to have the datadirectory set before writing to it" 19:20:13 (thinking RNG specifically) 19:20:38 oh, i'm not really talking about application level things 19:21:09 more things like RNG, logging objects (not actually setting the logfile, which happens later), libsecp, ... 19:21:21 syncronization debugging state 19:21:31 if it's really only setting a value to a data structure I agree it's different, but if there's extensive work that might depend on other or OS things, it gets hairy fast 19:21:51 only allocating a data structure on first use is fine... 19:22:03 It's also important to avoid undefined behavior over and above just avoiding doing the wrong thing. 19:22:24 basically the RNG right now can't use the SHA module, because the RNG is invoked from global constructors (and it works fine with it), and SHA needs explicit initialzation 19:22:46 but I think we agree that global constructors are bad 19:22:49 that's one thing :) 19:23:04 so i think there are two solutions to that... avoid global constructors everywhere, or make everything work fine on first use 19:23:55 (I here mean global constructors as 'runs before main', not the static initializers that run on first use) 19:24:03 wumpus: right 19:24:31 wumpus: so what's your opinion on solving this particular issue? 19:24:35 Or make SHA module's autodetect get resolved by the linker, using the GCC extension that does that. :P 19:25:20 (doesn't address the general question about dependencies between global constructors) 19:25:23 sipa: so move to initialization on first use or explicit initialization, whatever makes sense in the case, move away from global initializers that do anything more significant than assigning a constant value 19:25:27 we can get rid of all globals whose construction needs randomness, but making the SHA256 code autodetect on first use seems a simpler change 19:25:53 I really like how you got rid of CInit, for example 19:26:04 The downside of autodetect on first use is that it would make every call to sha256 slightly slower. :( 19:26:46 One way to compensate for that would be make sure that there is a batch sha256 function that does many of them and only does the detection once, and be sure to use it where possible. 19:27:12 gmaxwell: hmm? 19:27:38 i benchmarked it as a 1.8ns slowdown here to use an on-first-use construction 19:27:49 sipa: I assume 'autodetect on first use' means "Check a synchronized variable every time the function is run". 19:27:55 gmaxwell: nope! 19:28:24 it actually compiles to both a synchronized and unsynchronized variable, and in the initialized case, only the latter is checked 19:28:33 Okay, 1.8ns doesn't sound that terrible. What the runtime of the function on that host? 19:28:36 nice 19:28:55 sipa: how does it know its initialized? 19:29:26 though this is a compiler implementation specific thing, clang and gcc might not do it the same way? 19:29:37 gmaxwell: it's just a boolean; when the boolean is false, it means it could be initialized or not (to be checked using synchronized primitives), if it is true, it is guaranteed to be initialized 19:30:22 from cppreference.com:If multiple threads attempt to initialize the same static local variable concurrently, the initialization occurs exactly once (similar behavior can be obtained for arbitrary functions with std::call_once). 19:30:26 Note: usual implementations of this feature use variants of the double-checked locking pattern, which reduces runtime overhead for already-initialized local statics to a single non-atomic boolean comparison. 19:30:41 good to know 19:31:02 in any case, I think that seems an obviously better way to handle it. Residual performance concerns could be handled by my above batching suggestion (which would be a win regardless because of function call overheads). MOST places where we'd have this concern wouldn't be a big performance bottleneck like sha256 is. 19:31:19 exactly 19:31:25 maybe sha256 is just special here 19:31:41 and we can at least decide for the general case 19:31:45 agree 19:32:02 enough on this topic, i think 19:32:05 ok! 19:32:23 well I could see the same problem showing up for crc32 and being worse, because 1.8ns would be like a 2x slowdown for it. :P but otherwise I can't come up with much where a 1.8ns slowdown would matter. 19:32:39 #topic Chaincode summer residency (jnewbery) 19:32:40 could have the calls be pointers that get changed on initialisation (I thought we already did?) 19:32:56 hi 19:33:06 luke-jr: that means the overhead of an indirect function call every time, that's worse than checking a boolean 19:33:19 we're hosting the next iteration of the Chaincode residency this summer 19:33:34 it'll be in our new office in Midtown Manhattan 19:33:41 details are here: https://residency.chaincode.com/ 19:34:00 great! 19:34:05 we'll take care of sourcing residents, paying travel/accommodation/stipend and hosting them in our office 19:34:20 nice! 19:34:35 nice! 19:34:48 it's 2-3 weeks of seminars followed by 2 months of project work. We're expecting the residents to make really meaningful contributions to Bitcoin Core and other Bitcoin/Lightning projects while they're here 19:35:05 I bring this up here because we need help for a couple of things: 19:35:51 1. We (Chaincode) will be doing the heavy lifting for the seminar series and hosting, but we need (remote) mentors for the 2 month project period. 19:36:03 what are the responsibilities of the mentors 19:36:31 each resident will be paired with a mentor. We're looking for 1 hour per week video calls with the resident to help guide them in their project 19:37:13 obviously chaincoders and their peers will be on hand for incidental questions during the week, and the mentor will be providing overall guidance helping with the project 19:37:28 Will the peers be chosen based on areas of knowledge 19:37:43 jnewbery, what's the action item here? ping you if interested in mentoring? 19:38:01 oh don't worry instagibbs, we've already signed you up 19:38:06 ;) 19:38:08 We'll try to pair residents with mentors who have overlapping interests obviously 19:38:16 instagibbs - I'll be reaching out individually 19:38:40 hah 19:38:50 ok, so check e-mails DMs 19:39:07 2. We're looking for recommendations for residents. If you know anyone who wants to immerse themselves in Bitcoin/Lightning over summer and is excited about making a real contribution to the project, please send them our way 19:39:23 Adam Jonas is taking the lead on organizing this one 19:39:41 So you can ping him or me if you have any questions 19:40:15 that's it! 19:40:37 ok! thanks for organizing this 19:40:46 We're really excited about this one. The longer format means we're expecting to have a lot of great contributions from the residents 19:41:09 hope so! 19:41:43 any other topics? 19:42:19 one reminder: I'd encourage people to use moneyball's #proposedmeetingtopic to propose meeting topics during the week! 19:42:33 Could I nag for review on #14929 ? it's getting forced to rebase faster than its being reviewed... 19:42:36 https://github.com/bitcoin/bitcoin/issues/14929 | net: Allow connections from misbehavior banned peers by gmaxwell · Pull Request #14929 · bitcoin/bitcoin · GitHub 19:43:06 gmaxwell: yep i'm actually in the middle of re-reviewing so i can re-ack 19:43:18 #action review #14929 19:43:19 add to high priority? 19:43:21 https://github.com/bitcoin/bitcoin/issues/14929 | net: Allow connections from misbehavior banned peers by gmaxwell · Pull Request #14929 · bitcoin/bitcoin · GitHub 19:43:35 ok 19:44:32 Thanks. 19:46:21 that... concludes the meeting, I guess 19:47:19 #endmeeting