12018-02-13T00:06:23 *** lnostdal has quit IRC
22018-02-13T00:14:44 *** mirese_ has joined #bitcoin-core-dev
32018-02-13T00:16:37 *** lnostdal has joined #bitcoin-core-dev
42018-02-13T00:17:46 *** mirese has quit IRC
52018-02-13T00:20:34 *** meshcollider has quit IRC
62018-02-13T00:26:21 *** rex_4539 has quit IRC
72018-02-13T00:33:21 *** Randolf has joined #bitcoin-core-dev
82018-02-13T00:50:17 *** StopAndDecrypt has joined #bitcoin-core-dev
92018-02-13T00:57:21 *** Randolf has quit IRC
102018-02-13T00:58:19 *** dabura667 has joined #bitcoin-core-dev
112018-02-13T01:02:24 *** meshcollider has joined #bitcoin-core-dev
122018-02-13T01:09:56 *** Jacinthe3Hermist has quit IRC
132018-02-13T01:13:27 *** grafcaps has quit IRC
142018-02-13T01:15:53 *** RezusMinus has joined #bitcoin-core-dev
152018-02-13T01:19:05 *** jonasschnelli has quit IRC
162018-02-13T01:19:47 <ProfMac> was there ever a time when GetBlockWork used the actual hash instead of nBits to calculate work?
172018-02-13T01:20:05 *** wolfspraul has quit IRC
182018-02-13T01:20:12 *** wolfspraul has joined #bitcoin-core-dev
192018-02-13T01:20:16 <bitcoin-git> [bitcoin] promag opened pull request #12419: wallet: Force distinct destinations in CWallet::CreateTransaction (master...2018-02-distinct-destinations) https://github.com/bitcoin/bitcoin/pull/12419
202018-02-13T01:20:16 <gmaxwell> no, that wouldn't be correct.
212018-02-13T01:20:29 <ProfMac> tell me more.
222018-02-13T01:20:37 *** jonasschnelli has joined #bitcoin-core-dev
232018-02-13T01:21:48 *** dabura667_ has joined #bitcoin-core-dev
242018-02-13T01:21:54 *** dabura667 has quit IRC
252018-02-13T01:22:13 <gmaxwell> the hash reflects more than actual work, by a factor of two on average. It also has very high variance. meaning that if you got a block with ten times that 'work' expected (which you'd get in 1/10) blocks you could just delay announcing it until a couple blocks had passed, then announce it and be very likely to reorg them out.
262018-02-13T01:22:48 <promag> gmaxwell: regarding duplicate outputs #12419
272018-02-13T01:22:49 <gribble> https://github.com/bitcoin/bitcoin/issues/12419 | Force distinct destinations in CWallet::CreateTransaction by promag · Pull Request #12419 · bitcoin/bitcoin · GitHub
282018-02-13T01:23:40 <ProfMac> Yes, gmaxwell, I agree with all that.
292018-02-13T01:23:51 <ProfMac> Thanks.
302018-02-13T01:24:23 <ProfMac> Not unlike "curling" or "corn hole"
312018-02-13T01:29:27 *** grafcaps has joined #bitcoin-core-dev
322018-02-13T01:36:17 *** wacawacawaca has joined #bitcoin-core-dev
332018-02-13T01:37:35 *** justanotheruser has joined #bitcoin-core-dev
342018-02-13T01:38:36 *** contrapumpkin is now known as Whale
352018-02-13T01:38:53 *** Whale is now known as contrapumpkin
362018-02-13T01:39:32 *** contrapumpkin is now known as Whale
372018-02-13T01:39:39 *** Whale is now known as contrapumpkin
382018-02-13T01:53:21 *** belcher has quit IRC
392018-02-13T02:34:26 <gmaxwell> n1bor: conversation continued in #bitcoin he posted bench logs which indicated to me that he was IO bound very badly. Today he followed up with,
402018-02-13T02:34:29 <gmaxwell> n1bor: 18:28:24 < murrayn> gmaxwell, just to follow up from last night: 1) i shutdown gracefully and restarted, and the reindex started from the beginning. From what you said, it shouldn't do that? and 2) it was definitely disk IO bound. doing it on an SSD this time. much better
412018-02-13T02:34:48 <gmaxwell> (1) was something sipa warned him about in here, that restarting with -reindex-chainstate would restart the reindex.
422018-02-13T02:36:34 *** TheRec_ has joined #bitcoin-core-dev
432018-02-13T02:36:47 *** TheRec has quit IRC
442018-02-13T02:38:51 *** AaronvanW has quit IRC
452018-02-13T02:39:21 <midnightmagic> doh
462018-02-13T02:40:44 *** lnostdal has quit IRC
472018-02-13T02:43:55 *** Randolf has joined #bitcoin-core-dev
482018-02-13T02:55:50 *** jamesob has joined #bitcoin-core-dev
492018-02-13T03:05:03 *** mandric has quit IRC
502018-02-13T03:10:44 <meshcollider> looks like contrib/bitcoin-cli.bash-completion needs an update, hasn't been updated for ages
512018-02-13T03:24:10 <esotericnonsense> k, so i'm looking at the JSON-RPC 2.0 spec stuff now. 'strict compliance' requires two main changes as far as i can see. one of them I've made gated behind a flag 'strictjsonrpcspec' (better names accepted)
522018-02-13T03:24:29 *** StopAndDecrypt has quit IRC
532018-02-13T03:24:29 *** StopAndDecrypt has joined #bitcoin-core-dev
542018-02-13T03:24:41 <esotericnonsense> that's the 'client must send jsonrpc=2.0, server must send jsonrpc=2.0, only one of 'error' or 'result' should be returned'
552018-02-13T03:25:32 <esotericnonsense> the other bit is that if the client does not send an id with a request, the server should send a blank response. within a batch it should just be skipped. so if you send [a id=1, b, c id=3] you should get back [aresult, cresult].
562018-02-13T03:26:14 *** StopAndDecrypt has quit IRC
572018-02-13T03:26:54 <esotericnonsense> i think that's enough to make it fully spec compliant. the former one is enough to get it to work with libjsonrpccpp, could probably write some unit tests based on the jsonrpc spec as well, manual tests i've done look fine
582018-02-13T03:27:15 *** StopAndDecrypt has joined #bitcoin-core-dev
592018-02-13T03:27:15 *** StopAndDecrypt has quit IRC
602018-02-13T03:27:15 *** StopAndDecrypt has joined #bitcoin-core-dev
612018-02-13T03:28:10 *** Giszmo has quit IRC
622018-02-13T03:29:52 *** StopAndDecrypt has quit IRC
632018-02-13T03:30:26 *** StopAndDecrypt has joined #bitcoin-core-dev
642018-02-13T03:30:39 *** StopAndDecrypt has quit IRC
652018-02-13T03:31:46 *** StopAndDecrypt has joined #bitcoin-core-dev
662018-02-13T03:34:40 *** dongcarl has joined #bitcoin-core-dev
672018-02-13T03:37:42 *** jamesob has quit IRC
682018-02-13T03:39:32 *** zautomata1 has joined #bitcoin-core-dev
692018-02-13T03:41:05 *** zautomata has quit IRC
702018-02-13T03:54:28 <bitcoin-git> [bitcoin] MarcoFalke reopened pull request #12349: shutdown: fix crash on shutdown with reindex-chainstate (master...fix-qt-shutdown) https://github.com/bitcoin/bitcoin/pull/12349
712018-02-13T04:11:45 *** shesek has quit IRC
722018-02-13T04:23:51 *** dongcarl_ has joined #bitcoin-core-dev
732018-02-13T04:24:33 *** dongcarl has quit IRC
742018-02-13T04:41:28 *** shesek has joined #bitcoin-core-dev
752018-02-13T04:41:28 *** shesek has joined #bitcoin-core-dev
762018-02-13T04:54:49 <kallewoof> What's going on here? Either that comment is wrong, or the check is wrong... https://github.com/bitcoin/bitcoin/blame/master/src/script/interpreter.cpp#L1233 (it says nOut out of range, but it checks nIn against txTo.vout.size()...)
772018-02-13T04:57:12 <kallewoof> I guess it's because single requires the in to sign the same index out. Just weird looking.
782018-02-13T05:06:28 <aj> kallewoof: yeah, "no vout corresponding to nIn" or something would make more sense to me as a comment
792018-02-13T05:06:38 <aj> kallewoof: code seems right at least
802018-02-13T05:07:18 *** RezusMinus has quit IRC
812018-02-13T05:09:39 <aj> kallewoof: wow, that comment used to be a log output saying nOut out of range
822018-02-13T05:13:29 *** cryptojanitor has quit IRC
832018-02-13T05:14:37 <kallewoof> aj: yeah agreed!
842018-02-13T05:15:42 *** goatpig has joined #bitcoin-core-dev
852018-02-13T05:18:46 *** rex_4539 has joined #bitcoin-core-dev
862018-02-13T05:36:21 *** mmgen has joined #bitcoin-core-dev
872018-02-13T05:53:22 *** Sinclair_ has joined #bitcoin-core-dev
882018-02-13T05:55:12 *** Sinclair6 has quit IRC
892018-02-13T06:15:37 *** sengehest has joined #bitcoin-core-dev
902018-02-13T06:22:10 *** dongcarl_ has quit IRC
912018-02-13T06:23:23 *** murrayn has quit IRC
922018-02-13T06:32:14 *** sengehest has quit IRC
932018-02-13T06:32:59 *** rex_4539 has quit IRC
942018-02-13T07:00:27 *** kallewoof has quit IRC
952018-02-13T07:04:35 *** kallewoof has joined #bitcoin-core-dev
962018-02-13T07:15:05 *** tripleslash has quit IRC
972018-02-13T07:15:33 *** mmgen has quit IRC
982018-02-13T07:16:39 *** Varunram has quit IRC
992018-02-13T07:16:39 *** Varunram has joined #bitcoin-core-dev
1002018-02-13T07:16:39 *** Varunram has joined #bitcoin-core-dev
1012018-02-13T07:20:05 *** tripleslash has joined #bitcoin-core-dev
1022018-02-13T07:30:10 *** bsm117532 has quit IRC
1032018-02-13T07:43:44 *** dongcarl has joined #bitcoin-core-dev
1042018-02-13T07:53:23 *** mmgen has joined #bitcoin-core-dev
1052018-02-13T08:01:30 *** jonasschnelli has quit IRC
1062018-02-13T08:01:30 *** jonasschnelli has joined #bitcoin-core-dev
1072018-02-13T08:04:35 *** murrayn has joined #bitcoin-core-dev
1082018-02-13T08:04:35 *** murrayn has joined #bitcoin-core-dev
1092018-02-13T08:05:05 *** merehap has quit IRC
1102018-02-13T08:09:23 *** merehap has joined #bitcoin-core-dev
1112018-02-13T08:25:50 *** dongcarl has quit IRC
1122018-02-13T08:29:21 *** baldur has quit IRC
1132018-02-13T08:49:37 *** n1bor has quit IRC
1142018-02-13T08:50:01 *** n1bor has joined #bitcoin-core-dev
1152018-02-13T09:00:09 *** timothy has joined #bitcoin-core-dev
1162018-02-13T09:18:19 *** timothy has quit IRC
1172018-02-13T09:24:49 *** timothy has joined #bitcoin-core-dev
1182018-02-13T09:32:07 <bitcoin-git> [bitcoin] laanwj pushed 5 new commits to master: https://github.com/bitcoin/bitcoin/compare/c997f8808256...2dbc4a4740cd
1192018-02-13T09:32:08 <bitcoin-git> bitcoin/master cc046f6 John Newbery: [tests] Reduce NodeConn connection logging from info to debug
1202018-02-13T09:32:08 <bitcoin-git> bitcoin/master c32cf9f John Newbery: [tests] Add P2PDataStore class...
1212018-02-13T09:32:09 <bitcoin-git> bitcoin/master 359d067 John Newbery: [tests] Fix flake8 warnings in invalidtxrequest
1222018-02-13T09:32:37 <bitcoin-git> [bitcoin] laanwj closed pull request #11771: [tests] Change invalidtxrequest to use BitcoinTestFramework (master...refactor_invalidtxrequest) https://github.com/bitcoin/bitcoin/pull/11771
1232018-02-13T09:59:26 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/2dbc4a4740cd...f4f4f51f1a94
1242018-02-13T09:59:26 <bitcoin-git> bitcoin/master a71c56a Luke Dashjr: clientversion: Use full commit hash for commit-based version descriptions...
1252018-02-13T09:59:27 <bitcoin-git> bitcoin/master f4f4f51 Wladimir J. van der Laan: Merge #11966: clientversion: Use full commit hash for commit-based version descriptions...
1262018-02-13T10:00:04 <bitcoin-git> [bitcoin] laanwj closed pull request #11966: clientversion: Use full commit hash for commit-based version descriptions (master...ver_full_commit_hash) https://github.com/bitcoin/bitcoin/pull/11966
1272018-02-13T10:02:45 *** timothy has quit IRC
1282018-02-13T10:17:13 *** timothy has joined #bitcoin-core-dev
1292018-02-13T10:28:05 *** dabura667_ has quit IRC
1302018-02-13T10:37:30 *** Scrat has joined #bitcoin-core-dev
1312018-02-13T10:37:50 *** baldur has joined #bitcoin-core-dev
1322018-02-13T10:39:15 *** laurentmt has joined #bitcoin-core-dev
1332018-02-13T10:40:18 <wumpus> hmmm
1342018-02-13T10:40:19 <wumpus> util.cpp:384:121: note: in instantiation of template class 'std::__1::pair<const std::__1::basic_string<char>, boost::interprocess::file_lock>' requested here
1352018-02-13T10:40:19 <wumpus> boost::interprocess::file_lock& lock = locks.emplace(pathLockFile.string(), pathLockFile.string().c_str()).first->second;
1362018-02-13T10:41:22 <wumpus> full error here: https://github.com/bitcoin/bitcoin/issues/12413
1372018-02-13T10:41:37 <wumpus> anyone have an idea what can cause this c++ error monstriosity?
1382018-02-13T10:46:38 *** larafale has joined #bitcoin-core-dev
1392018-02-13T10:53:14 *** Victorsueca has quit IRC
1402018-02-13T10:54:49 *** Victorsueca has joined #bitcoin-core-dev
1412018-02-13T11:03:25 <wumpus> so it looks like "static std::map<std::string, boost::interprocess::file_lock> locks;" is illegal with openbsd's clang + boost version combo
1422018-02-13T11:03:40 <bitcoin-git> [bitcoin] Sjors opened pull request #12421: [qt] navigate to transaction history page after send (master...2018/02/qt-goto-transactions-after-send) https://github.com/bitcoin/bitcoin/pull/12421
1432018-02-13T11:04:40 <Lauda> Will there be rc4?
1442018-02-13T11:05:13 *** epic has quit IRC
1452018-02-13T11:05:36 *** epic has joined #bitcoin-core-dev
1462018-02-13T11:08:47 *** belcher has joined #bitcoin-core-dev
1472018-02-13T11:11:29 <provoostenator> Lauda: there's two open PR's which I suspect need to be fixed before rc4: https://github.com/bitcoin/bitcoin/milestone/30
1482018-02-13T11:12:29 <Lauda> Alright, thanks. I'll wait for those.
1492018-02-13T11:17:31 <bitcoin-git> [bitcoin] laanwj opened pull request #12422: util: Use unique_ptr in locks map (master...2018_01_openbsd_util_fix) https://github.com/bitcoin/bitcoin/pull/12422
1502018-02-13T11:18:14 *** Guyver2 has joined #bitcoin-core-dev
1512018-02-13T11:24:45 <gmaxwell> wumpus: can you explain why it works inside a unique ptr? (I am just curious)
1522018-02-13T11:26:03 <wumpus> gmaxwell: no, I have no idea why it fails in the first place
1532018-02-13T11:26:28 <wumpus> tbh this is really over my limit of understanding C++
1542018-02-13T11:27:32 <wumpus> well, the point of having pointers in a map instead of whole objects makes it 'simpler' to work with layout in memory I suppose
1552018-02-13T11:28:03 <wumpus> as for some reason the whole lock objects can't be moved
1562018-02-13T11:28:12 <wumpus> and pointers, obviously, can
1572018-02-13T11:28:59 <gmaxwell> yea, I was wondering if values in maps had to be relocatable, but quick googling didn't answer the question for me.
1582018-02-13T11:29:56 <wumpus> it must be some edge case that was solved in either a newer version of clang, or in a different version of boost, as I've not seen the issue on other platforms
1592018-02-13T11:30:53 <wumpus> the 'map' here is used to avoid locking a directory multiple times I guess? this structure is write-only, no lookups are ever done in this map, so there should be no performance difference whatsoever
1602018-02-13T11:31:00 <wumpus> gah
1612018-02-13T11:31:20 <goatpig> from the look of the error log
1622018-02-13T11:31:24 <goatpig> this has to do with the ctor
1632018-02-13T11:31:27 <goatpig> not the map itself
1642018-02-13T11:31:41 <wumpus> so emplace returns the current record if one already exists, right?
1652018-02-13T11:31:46 *** hsmiths has quit IRC
1662018-02-13T11:32:02 *** hsmiths has joined #bitcoin-core-dev
1672018-02-13T11:32:14 <wumpus> I dont' think this construction even accomplishes what it is meant to accomplish
1682018-02-13T11:32:59 <goatpig> emplace is like insert afaik, returns a pair of <iter, bool>
1692018-02-13T11:33:20 <goatpig> the iterator points at the inserted element or the already existing one, the bool tells you if it was added (true) or fetched (false)
1702018-02-13T11:34:18 <wumpus> ok that depends on what does lock->try_lock does on a lock that is already locked
1712018-02-13T11:34:29 <goatpig> you mean within the same thread?
1722018-02-13T11:34:40 <goatpig> or another one?
1732018-02-13T11:34:40 <wumpus> within the same process
1742018-02-13T11:34:45 <goatpig> i think it throws
1752018-02-13T11:34:46 <goatpig> not sure
1762018-02-13T11:34:48 <wumpus> this is a per-process lock, not per-thread
1772018-02-13T11:35:01 *** timothy has quit IRC
1782018-02-13T11:35:14 <wumpus> I wonder what the intent is for LockDirectory being called on the same directory multiple times
1792018-02-13T11:35:25 <goatpig> but that wouldnt have anything to do compile, that would just be runtime error
1802018-02-13T11:35:31 <wumpus> I know.
1812018-02-13T11:35:40 <wumpus> I'm just doubting that this code does what it is meant to do, at all
1822018-02-13T11:35:53 <goatpig> typically that's to make sure you got the lock
1832018-02-13T11:35:56 <wumpus> separate from what it takes to compile it
1842018-02-13T11:36:04 <goatpig> now if the same threads is trying to get a lock multiple times
1852018-02-13T11:36:25 <goatpig> it presupposes the methods that are accessed can be used on their own
1862018-02-13T11:36:33 <wumpus> it should probably at least check the boolean it gets back from emplace
1872018-02-13T11:36:40 <wumpus> whether the lock already exists
1882018-02-13T11:36:54 <wumpus> otherwise it will lock an old lock another time
1892018-02-13T11:36:56 <goatpig> that part is like
1902018-02-13T11:36:59 <goatpig> idk id have to look at the code
1912018-02-13T11:37:02 <goatpig> but it seems pointless
1922018-02-13T11:37:14 <goatpig> why hold a reference to the lock in the scope? that's usless, the map perpetuates it
1932018-02-13T11:37:16 <wumpus> either that's a no-op, or it throws, but it's pointless
1942018-02-13T11:37:33 <goatpig> and that could be your compile issue
1952018-02-13T11:37:37 <goatpig> that these locks are not copiable
1962018-02-13T11:37:45 <goatpig> ie just get rid of the referencing altgother
1972018-02-13T11:37:46 <wumpus> defining the map itself causes the compile issue
1982018-02-13T11:37:53 <wumpus> referencing is not the problem
1992018-02-13T11:37:56 <goatpig> boost::interprocess::file_lock& lock = locks.emplace(pathLockFile.string(), pathLockFile.string().c_str()).first->second;
2002018-02-13T11:38:03 <goatpig> get rid of the reference and try
2012018-02-13T11:38:06 <gmaxwell> in general I wouldn't expect a lock to be copyable.
2022018-02-13T11:38:07 <goatpig> if that doesnt bork other stuff
2032018-02-13T11:38:16 <wumpus> goatpig: I tried, see discussion in https://github.com/bitcoin/bitcoin/issues/12413
2042018-02-13T11:38:19 <goatpig> i.e. just do locks.emplace(pathLockFile.string(), pathLockFile.string().c_str());
2052018-02-13T11:38:31 <gmaxwell> what the heck is this doing?
2062018-02-13T11:38:33 <gmaxwell> anyways?
2072018-02-13T11:38:43 <wumpus> goatpig: as I said above, defining the map itself already causes the error, even if you don't access it at all!
2082018-02-13T11:39:10 <wumpus> gmaxwell: that was also my question, what is the semantics supposed to be
2092018-02-13T11:39:27 <goatpig> you could just ninja that my making it a map of shared_ptr<lock>
2102018-02-13T11:39:33 <goatpig> by*
2112018-02-13T11:39:47 <gmaxwell> maybe we can remove it all entirely? thats always the best fix when possible. :P
2122018-02-13T11:39:51 <wumpus> goatpig: that's what I did in #12322 (but with unique_lock)
2132018-02-13T11:39:52 <gribble> https://github.com/bitcoin/bitcoin/issues/12322 | Docs: Remove step making cloned repository world-writable for Windows build. by murrayn · Pull Request #12322 · bitcoin/bitcoin · GitHub
2142018-02-13T11:40:00 <wumpus> eh #12422
2152018-02-13T11:40:02 <gribble> https://github.com/bitcoin/bitcoin/issues/12422 | util: Use unique_ptr in locks map (fix OpenBSD 6.2 build) by laanwj · Pull Request #12422 · bitcoin/bitcoin · GitHub
2162018-02-13T11:40:08 *** jl2012 has quit IRC
2172018-02-13T11:40:19 * gmaxwell channels some contributor and considers just submitting a PR to remove stuff he doesn't understand
2182018-02-13T11:40:25 *** jl2012 has joined #bitcoin-core-dev
2192018-02-13T11:40:48 <goatpig> wumpus: sure, dunno what the code really intents. i'd prefer unique_ptr when possible
2202018-02-13T11:40:59 <goatpig> but if it's trying to reference the lock, then it should be using shared_ptrs
2212018-02-13T11:41:08 <wumpus> no, shared_ptre is not necessary
2222018-02-13T11:41:11 *** wacawacawaca has quit IRC
2232018-02-13T11:41:14 <wumpus> the lock does not leave the scope at all
2242018-02-13T11:41:25 <goatpig> the lock exists within the map
2252018-02-13T11:41:29 <wumpus> yes
2262018-02-13T11:41:47 <wumpus> and after the change, it exists inside a unique pointer
2272018-02-13T11:41:57 <goatpig> if you kill the map, if there is a thread with the ptr/reference, it will blow up
2282018-02-13T11:41:59 <wumpus> which is unique, inside the map
2292018-02-13T11:42:10 <wumpus> no one has a reference to it
2302018-02-13T11:42:11 *** timothy has joined #bitcoin-core-dev
2312018-02-13T11:42:14 <wumpus> please read the code
2322018-02-13T11:42:21 <wumpus> no reference leaves the function
2332018-02-13T11:43:00 <goatpig> checking
2342018-02-13T11:43:02 <wumpus> the destructor will be called when the process has shut down, after the main function, by automatic cleanup
2352018-02-13T11:43:12 <wumpus> by that time no one will be using the data dir anymore
2362018-02-13T11:43:49 <wumpus> the map exists to hold on to the objects until the process shuts down
2372018-02-13T11:45:57 <goatpig> so the locks persist the process?
2382018-02-13T11:46:01 <goatpig> or rather
2392018-02-13T11:46:11 <goatpig> no method tries to clean them up
2402018-02-13T11:46:37 <wumpus> as I said, they'll be automatically cleaned up by the runtime framework
2412018-02-13T11:47:34 <wumpus> <gmaxwell> in general I wouldn't expect a lock to be copyable. <- they are; it's only moving them at most, not copying
2422018-02-13T11:50:11 <goatpig> wumpus: is it assumed that code gets accessed by only a single thread?
2432018-02-13T11:50:21 <gmaxwell> goatpig: this isn't an ordinary lock
2442018-02-13T11:50:26 <gmaxwell> it's a file locking thing
2452018-02-13T11:50:36 <wumpus> goatpig: yes, it's only used during initialization, by the init thread
2462018-02-13T11:50:42 <gmaxwell> used to keep seperate processes from muckign about with the same data directories.
2472018-02-13T11:50:50 <wumpus> goatpig: oh... I'm actually not sure anymore
2482018-02-13T11:50:51 <goatpig> i understand that much, but it's not thread safe, so im assuming this is every hit once
2492018-02-13T11:51:01 <goatpig> err
2502018-02-13T11:51:02 *** dgy_ has quit IRC
2512018-02-13T11:51:02 <wumpus> wallet directories complicated this
2522018-02-13T11:51:03 <goatpig> hit ever once
2532018-02-13T11:51:23 <goatpig> i mean the semantics of the method suggest this is accessed just to check the lock is held at all
2542018-02-13T11:51:23 <wumpus> as soon as dynamic opening of wallets is supported, this can be called from the RPC thread
2552018-02-13T11:51:33 <wumpus> but right now it's only used from init
2562018-02-13T11:52:18 <wumpus> it's a good question, the function as it is is certainly not thread safe
2572018-02-13T11:52:22 <goatpig> you are adding elements to a map. ideally it should check the map has the element, otherwise grab a mutex, add the element, then go back to regular execution
2582018-02-13T11:52:23 *** timothy has quit IRC
2592018-02-13T11:53:08 <wumpus> it should grab a mutex before accessing the map at all
2602018-02-13T11:53:12 <gmaxwell> ^
2612018-02-13T11:53:14 <wumpus> *if* it's supposed to be thread safe
2622018-02-13T11:53:19 <goatpig> ok, sure
2632018-02-13T11:53:38 <gmaxwell> and its trivial here in any case.. since the purpose of the map is just lifetime management for the contained objects.
2642018-02-13T11:53:45 <wumpus> yes
2652018-02-13T11:54:09 <goatpig> that would be if the method was not meant to return state of the lock
2662018-02-13T11:54:22 <goatpig> the semantics just suggest to me that it could be accessed by another thread
2672018-02-13T11:54:46 <wumpus> I think the intent is to return true if the process has a lock on the directory, whether it's new or not. Not sure the code matches that intent, at the moment.
2682018-02-13T11:54:52 <gmaxwell> nah its just init only now. though wumpus is right that dynamic wallet opening/closing will cause accesses from rpc.
2692018-02-13T11:55:38 <wumpus> I've asked meshcollider (who wrote the code) in the issue just in case
2702018-02-13T11:55:59 <gmaxwell> for wallet opening we probably want it to return true if it takes a lock and false if it already has it or can't take it.
2712018-02-13T11:56:37 <meshcollider> wumpus: I think ryanofsky suggested the currently used approach as a comment on the PR
2722018-02-13T11:56:41 <wumpus> gmaxwell: it's a lock on a directory, not on the wallet, bdb takes care of not opening the same wallet multiple times
2732018-02-13T11:56:53 <wumpus> gmaxwell: there can be multiple wallets in a directory
2742018-02-13T11:57:13 <gmaxwell> wumpus: ah point.
2752018-02-13T11:57:20 <gmaxwell> Carry on then.
2762018-02-13T11:59:48 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2772018-02-13T12:06:10 <wumpus> we could certainly use unit tests for that function
2782018-02-13T12:08:01 *** Chris_Stewart_5 has quit IRC
2792018-02-13T12:08:50 *** rodarmor has quit IRC
2802018-02-13T12:09:25 *** rodarmor has joined #bitcoin-core-dev
2812018-02-13T12:10:01 <wumpus> meshcollider: ok adding jnewbery to reviewers
2822018-02-13T12:12:55 <meshcollider> jnewbery or ryanofsky?
2832018-02-13T12:13:11 <meshcollider> wumpus: your PR looks good to me
2842018-02-13T12:13:41 <wumpus> eh, sorry, yes ryanofsky, added the wrong one
2852018-02-13T12:13:51 *** jamesob has joined #bitcoin-core-dev
2862018-02-13T12:15:02 <meshcollider> wumpus: didn't even think about the dynamic loading/unloading case / thread safety when I wrote it, apologies
2872018-02-13T12:15:17 <meshcollider> Was just trying to fix the issue for 0.16
2882018-02-13T12:15:43 <wumpus> meshcollider: no worries, we found that issue before it became a problem at least
2892018-02-13T12:21:35 *** cryptojanitor has joined #bitcoin-core-dev
2902018-02-13T12:32:44 * wumpus working on a unit test for LockDataDirectory - part of it will only work on UNIX because fork()
2912018-02-13T12:46:09 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2922018-02-13T12:50:57 *** Chris_Stewart_5 has quit IRC
2932018-02-13T13:05:11 *** Chris_Stewart_5 has joined #bitcoin-core-dev
2942018-02-13T13:11:22 <provoostenator> You have got to be kidding... anyone had to jump through these hoops before? http://htmlpreview.github.io/?https://github.com/JKSH/QtSdkRepoChooser/blob/master/doc/index.html
2952018-02-13T13:14:58 *** Pavle has joined #bitcoin-core-dev
2962018-02-13T13:15:11 <wumpus> no, not really, though I've had to build qt for embedded devices, the source code is huge, it takes a long time and it's a pain to configure for cross compile
2972018-02-13T13:15:35 <wumpus> it pretty much regards itself as an operating system in itself
2982018-02-13T13:19:29 *** lnostdal has joined #bitcoin-core-dev
2992018-02-13T13:21:03 *** Pavle has quit IRC
3002018-02-13T13:22:29 <wumpus> we can be really happy that linux distributions simply package that stuff pre-built
3012018-02-13T13:28:02 *** jtimon has quit IRC
3022018-02-13T13:34:13 *** timothy has joined #bitcoin-core-dev
3032018-02-13T13:56:17 *** jamesob has quit IRC
3042018-02-13T13:58:38 *** SopaXorzTaker has joined #bitcoin-core-dev
3052018-02-13T14:09:49 *** jamesob has joined #bitcoin-core-dev
3062018-02-13T14:18:18 <midnightmagic> /w/w 59
3072018-02-13T14:18:51 *** dermoth has quit IRC
3082018-02-13T14:19:41 *** dermoth has joined #bitcoin-core-dev
3092018-02-13T14:20:01 <cfields> wumpus: seems it's moot now, but I think std::piecewise_construct might be what's missing for the emplace
3102018-02-13T14:20:52 *** laurentmt has quit IRC
3112018-02-13T14:22:23 *** Chris_Stewart_5 has quit IRC
3122018-02-13T14:23:05 <cfields> as usual, spoke too soon. Red herring. Nevermind.
3132018-02-13T14:25:20 <wumpus> I'll look at that one, never heard of it
3142018-02-13T14:26:52 <cfields> wumpus: see for example: https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp#L574
3152018-02-13T14:27:37 *** timothy has quit IRC
3162018-02-13T14:31:57 *** rex_4539 has joined #bitcoin-core-dev
3172018-02-13T14:36:27 *** Victorsueca has quit IRC
3182018-02-13T14:37:48 *** Victorsueca has joined #bitcoin-core-dev
3192018-02-13T14:38:31 *** laurentmt has joined #bitcoin-core-dev
3202018-02-13T14:39:33 *** timothy has joined #bitcoin-core-dev
3212018-02-13T14:40:54 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3222018-02-13T14:42:07 *** meshcollider has quit IRC
3232018-02-13T14:49:49 *** jamesob has quit IRC
3242018-02-13T14:50:14 *** ken2812221 has quit IRC
3252018-02-13T14:50:39 *** ken2812221 has joined #bitcoin-core-dev
3262018-02-13T14:50:46 *** ossifrage has quit IRC
3272018-02-13T14:51:20 *** ossifrage has joined #bitcoin-core-dev
3282018-02-13T14:56:33 *** dermoth has quit IRC
3292018-02-13T14:56:33 *** mmgen has quit IRC
3302018-02-13T14:56:33 *** ghost43 has quit IRC
3312018-02-13T14:56:54 *** ghost43 has joined #bitcoin-core-dev
3322018-02-13T14:57:07 *** dermoth has joined #bitcoin-core-dev
3332018-02-13T14:57:16 *** mmgen has joined #bitcoin-core-dev
3342018-02-13T14:58:55 *** rex_4539 has quit IRC
3352018-02-13T15:01:49 *** rex_4539 has joined #bitcoin-core-dev
3362018-02-13T15:02:21 <hkjn0> a bit lazy q: am I reading depends/ docs right that there's currently no way to use other platforms than x86_64 to build e.g. bdb?
3372018-02-13T15:02:27 *** alfa has joined #bitcoin-core-dev
3382018-02-13T15:03:15 <hkjn0> so if I'm compiling code on armv7l and want wallet support, best thing to do is to get on x86_64 instead, and crosscompile towards arm as necessary?
3392018-02-13T15:03:57 *** ghost43 has quit IRC
3402018-02-13T15:03:59 <hkjn0> (I did get bdb4.8 compiled from source on armv7l outside of the depends system, but had trouble getting it to be noticed by the build system..)
3412018-02-13T15:04:09 *** ghost43 has joined #bitcoin-core-dev
3422018-02-13T15:06:13 <wumpus> hkjn0: the depends system assumes it's building on x86_64, however if you want to build db4 outside the depends system there's the script contrib/install_db4.sh and the flags BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include"
3432018-02-13T15:07:41 <wumpus> would be nice if the depends system also supported building *from* other architectures, though
3442018-02-13T15:10:02 *** Giszmo has joined #bitcoin-core-dev
3452018-02-13T15:10:18 <hkjn0> ah, great, I bet I was just misisng BDB_LIBS or somesuch.. will try out the contrib/ script
3462018-02-13T15:10:33 <wumpus> esp. at some point we'd like to do gitian builds from other architectures
3472018-02-13T15:11:59 <hkjn0> right. I've been setting up these weird build environments already, would be happy to try to contribute to gitian work or otherwise support other build platforms, just need to understand more of how current system hangs together first.. :)
3482018-02-13T15:18:27 *** Chris_Stewart_5 has quit IRC
3492018-02-13T15:44:01 *** jamesob has joined #bitcoin-core-dev
3502018-02-13T15:46:47 *** owowo has quit IRC
3512018-02-13T15:49:22 *** dermoth has quit IRC
3522018-02-13T15:51:36 *** owowo has joined #bitcoin-core-dev
3532018-02-13T15:51:40 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3542018-02-13T15:55:30 *** grafcaps has quit IRC
3552018-02-13T16:05:21 *** jamesob has quit IRC
3562018-02-13T16:07:11 *** Chris_Stewart_5 has quit IRC
3572018-02-13T16:10:17 *** lnostdal has quit IRC
3582018-02-13T16:11:55 *** grafcaps has joined #bitcoin-core-dev
3592018-02-13T16:22:40 <cfields> depends doesn't assume build is x86_64. It's the only arch that really gets tested, though
3602018-02-13T16:23:09 <cfields> but, all of the machinery is there to auto-detect the build platform and try to cope.
3612018-02-13T16:24:46 <cfields> hkjn0: if you try building native on arm and run into issues, feel free to ping me
3622018-02-13T16:27:03 *** AaronvanW has joined #bitcoin-core-dev
3632018-02-13T16:28:25 <hkjn0> cfield, thanks! building with contrib/install_db4.sh + BDB_LIBS it suggested for ./configure as wumpus suggested seems to work fine so far
3642018-02-13T16:29:44 <hkjn0> (before that I tried a plain `make` in depends/ and it failed on armv7l, but it is possible I am missing some packages)
3652018-02-13T16:34:01 <cfields> hkjn0: well if you'd be willing, I'd be happy to step through the depends with you and try to get it patched to build
3662018-02-13T16:35:10 <cfields> like wumpus said, it'd be nice to have several working build arches
3672018-02-13T16:44:42 <hkjn0> cfields: sure, let me know how you'd like to proceed.. do we take it off the main channel to not clutter it up too much?
3682018-02-13T16:45:08 <cfields> hkjn0: sure, #bitcoin-build
3692018-02-13T16:52:33 *** Randolf has quit IRC
3702018-02-13T16:54:59 <michagogo> wumpus/cfields: Do you remember (on a describe-in-a-couple-sentences-off-the-top-of-your-head level) what the actual issue is with Xenial and why it's broken?
3712018-02-13T16:55:27 <michagogo> And does Artful(/Bionic) just work?
3722018-02-13T16:55:42 <michagogo> (with cross-compilation for Windows, I mean)
3732018-02-13T16:56:36 <michagogo> I tried looking through the issues on GitHub and got confused by the various different threads and things people tried and experienced
3742018-02-13T16:56:37 <cfields> michagogo: by default, the toolchain is configured for win threads rather than posix, which don't exist (at least with that gcc version)
3752018-02-13T16:56:41 *** Murch has joined #bitcoin-core-dev
3762018-02-13T16:56:52 <cfields> further, the ssp is busted
3772018-02-13T16:57:30 <michagogo> And were those fixed in later versions, e.g. Artful?
3782018-02-13T16:58:13 <cfields> I believe so. But we should be switching to our own toolchains for 0.17 anyway. Or if not a full switch, probably at least for mingw.
3792018-02-13T16:59:10 <michagogo> That sounds like it would be a good thing
3802018-02-13T17:00:24 <michagogo> I ask because last night and today I spent a bunch of hours trying to see how hard it would be to just backport the version of mingw-w64 that's in artful/bionic to xenial
3812018-02-13T17:01:16 <michagogo> I tried the straightforward "`backportpackage` into a PPA", but ran into chains of missing dependencies
3822018-02-13T17:02:00 <michagogo> I won't bore you with everything I tried and looked at, but at this point I've come to the realization that I don't know enough about Ubuntu packaging and how all that works, and beyond that, more important stuff
3832018-02-13T17:02:17 <cfields> michagogo: I can upload a stand-alone mingw toolchain soon, if it would help
3842018-02-13T17:02:21 <michagogo> Like, what is mingw-w64, vs what is gcc, vs what is gcc-mingw-w64
3852018-02-13T17:02:23 <cfields> heh
3862018-02-13T17:02:27 <michagogo> and how do all those interact
3872018-02-13T17:02:43 <cfields> right
3882018-02-13T17:02:56 <cfields> well, in case it helps...
3892018-02-13T17:03:25 <michagogo> In #ubuntu-devel someone said: 18:34:22 <rbasak> michagogo: I would step back and consider your approach. I don't know what problem you're solving, but there might be an easier way.
3902018-02-13T17:04:10 <michagogo> I explained that the version in xenial was broken, and they said this: 18:39:39 <rbasak> Also, if the reason it doesn't work in Xenial is due to a bug, we're quite happy in general to cherry-pick the fix for all Xenial users to benefit.
3912018-02-13T17:04:27 <michagogo> But somehow I suspect if it were that simple it would have already happened...
3922018-02-13T17:04:38 <cfields> to build a mingw-w64 toolchain, you need: mingw-w64 headers (think kernel headers), mingw-64 runtime (think glibc), gcc, and binutils. And obviously they need to be built in a way that lets them play nicely.
3932018-02-13T17:05:04 <michagogo> Right. So the thing is, I don't actually know the difference between mingw-w64 and gcc
3942018-02-13T17:05:09 <michagogo> Well, that's not entirely true
3952018-02-13T17:05:14 <michagogo> I sort of know what gcc is
3962018-02-13T17:05:34 <michagogo> What I don't know is, what is mingw-w64, and how does it interact with/based on/use gcc
3972018-02-13T17:06:23 *** dgenr8 has joined #bitcoin-core-dev
3982018-02-13T17:07:08 <cfields> mingw-w64 is a shim that allows for posix-y code to work on windows.
3992018-02-13T17:08:50 <michagogo> So what is gcc-mingw-w64?
4002018-02-13T17:09:20 <cfields> so you can build a compiler (gcc) that's able to produce win32 code, but it wouldn't do much good if you couldn't use standard unixy apis.
4012018-02-13T17:10:20 <cfields> so, for example, mingw-w64 can use either win or pthread threading models. Configured without pthread support, you'll have trouble with lots of applications. That's one of our issues.
4022018-02-13T17:13:11 <michagogo> So gcc-mingw-64, say, is a version of GCC that builds the mingw-w64 layer into the binaries it produces?
4032018-02-13T17:13:40 <cfields> it's probably easiest to just think of mingw-w64 as a libc for windows.
4042018-02-13T17:14:17 <michagogo> I guess at this point the knowledge that I'm missing is: 1) what _exactly_ the term "toolchain" refers to and how you build one
4052018-02-13T17:14:37 <michagogo> 2) How Ubuntu packages work together to make up a toolchain
4062018-02-13T17:15:09 <michagogo> And all kinds of other things related to those, like how exactly Ubuntu packaging works
4072018-02-13T17:15:22 <cfields> well, the packaging is an entirely different beast
4082018-02-13T17:16:02 <michagogo> I'm thinking I'll drop the issue for now, since I don't really have a ton of time and I suspect it would take me at least a few days to sort all these things out
4092018-02-13T17:16:46 <michagogo> I mean, regarding what rbasak said over in #ubuntu-devel, are the problems we have simple bugs with focused fixes that can be cherry-picked, or more fundamental problems that can't really be fixed without a wholesale upgrade to a newer version of MinGW?
4102018-02-13T17:17:27 <cfields> a toolchain is the group of progs that are required to produce a working binary. Typically that's target kernel/os headers in some form, target libc, host compiler, other host apps (in linux, binutils provides ar/ranlib/etc.), and a linker (also provided by binutils on linux)
4112018-02-13T17:17:37 <jcorgan> michagogo: it's been a little while since i've used gcc-mingw-w64 but i recall in ubuntu that installing that package pulls in all the accessories (runtime, binutils, etc.) that cfields mentioned
4122018-02-13T17:17:59 <michagogo> And actually, is the issue just with the MinGW component? Meaning, say one were to rebuild the gcc-mingw-w64 packages with the same GCC version and just an upgraded MinGW, would that work?
4132018-02-13T17:19:04 <jcorgan> in late to the conversation here, so not sure you're exact goal
4142018-02-13T17:19:27 <cfields> michagogo: the thread thing is just a config option. There's also the stack issue, which I haven't looked into for a while, but afaik that's no longer an issue in newer ubuntu versions. So it was either an upstream bug that's been fixed, or a config problem that ubuntu has fixed
4152018-02-13T17:19:52 <michagogo> jcorgan: Well, at the end of the day, here's my big-picture goal
4162018-02-13T17:20:02 <michagogo> Make https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md#building-for-64-bit-windows die in a fire
4172018-02-13T17:20:30 <michagogo> Specifically, the lines telling you to sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu zesty universe"; sudo apt update;sudo apt upgrade
4182018-02-13T17:20:54 <michagogo> Because besides that not working, because zesty is EOL, that's just going to hose your system
4192018-02-13T17:21:28 <jcorgan> ah, i only have experience going from LTS to LTS, never use the ones in between
4202018-02-13T17:21:56 <michagogo> Even doing that, you never ever just add the newer repository like that
4212018-02-13T17:21:59 <michagogo> There's an upgrader
4222018-02-13T17:22:37 <michagogo> So my thought was, well, can I get the packages from the newer version brought back to xenial?
4232018-02-13T17:23:03 <jcorgan> it might be possible, but that way lies madness
4242018-02-13T17:23:25 <michagogo> So it seems.
4252018-02-13T17:23:51 <michagogo> I tried just naively `backportpackage`ing into a PPA
4262018-02-13T17:24:09 <jcorgan> you're on zesty and need packages that only exist later than that?
4272018-02-13T17:24:22 <cfields> michagogo: yea, I'm not sure it's worth the trouble. We already have a plan for resolving this, and it means that we don't have to rely on Ubuntu anymore
4282018-02-13T17:24:24 <michagogo> Iâm not on zesty
4292018-02-13T17:25:11 <michagogo> Iâm actually not an active Ubuntu (or Linux) user day to day
4302018-02-13T17:25:13 <jcorgan> you could see if the packages you need are in the 'backports' repo
4312018-02-13T17:25:18 <michagogo> Nah
4322018-02-13T17:25:36 <michagogo> My idea was to *get* a newer mingw-w64 into -backports
4332018-02-13T17:26:25 <michagogo> I filed the âplease backportâ bugs
4342018-02-13T17:26:28 <cfields> michagogo: well, that certainly wouldn't be a bad thing. It's definitely not just us.
4352018-02-13T17:26:44 <jcorgan> i expect that would require specialized knowledge of how gcc/binutils/etc are packaged that a typical user wouldn't normally need to know
4362018-02-13T17:27:07 <jcorgan> and personally i'd look for an alternative way to solve whatever problem
4372018-02-13T17:27:32 <michagogo> I had the naive thought that, well, it should be easy to backport, because itâs a cross-compiler, not anything that the system uses
4382018-02-13T17:28:26 <michagogo> jcorgan: AIUI âwhatever problemâ is âmingw-w64 on Xenial is very brokenâ
4392018-02-13T17:30:38 *** Pavle has joined #bitcoin-core-dev
4402018-02-13T17:31:03 <jcorgan> personally i'd see if the must-be-xenial constraint can be relaxed. but if you wanted a challenge you could learn enough about how it works to help fix the issue. that may not be worth your time, but it could also be fun and very educational.
4412018-02-13T17:31:21 <jcorgan> (depends on you)
4422018-02-13T17:31:21 *** mrannanay has joined #bitcoin-core-dev
4432018-02-13T17:31:25 <michagogo> A no-changes backport failed, firstly, because it build-depends on gcc-7
4442018-02-13T17:32:08 <michagogo> Which must-be-Xenial constraint?
4452018-02-13T17:32:23 <michagogo> The thing is, 1) thatâs the latest LTS
4462018-02-13T17:32:51 <jcorgan> right, i use that extensively
4472018-02-13T17:32:54 <michagogo> And 2) thatâs the version that you get with WSL, and as far as I know trying to upgrade just breaks it
4482018-02-13T17:33:31 <jcorgan> ah, another piece of the picture i was unaware of
4492018-02-13T17:34:07 <michagogo> So the âconstraintâ is that we really want to let users on Xenial cross-compile is
4502018-02-13T17:34:12 <jcorgan> you're in a maze of twisty passages, all alike
4512018-02-13T17:34:15 <michagogo> cross-compile us*
4522018-02-13T17:36:01 <michagogo> I mean, itâs possible that if I knew Ubuntu packaging and had enough time I could try just reverting the changes that made it build with gcc 7
4532018-02-13T17:36:05 <jcorgan> so, getting a backport made of mingw does seem like the right way to go, but the number of people who are likely to know how to do that is probably small
4542018-02-13T17:36:13 <michagogo> And bring it back to 5.whatever
4552018-02-13T17:36:58 <michagogo> For all I know it might be just changing the build-depends line back to point at the gcc version thatâs in Xenial
4562018-02-13T17:37:06 <jcorgan> sure
4572018-02-13T17:38:00 <jcorgan> but there could also be some specific bug in gcc5 that caused gcc7 to be needed.
4582018-02-13T17:38:08 <michagogo> I doubt it
4592018-02-13T17:38:23 <michagogo> I mean, I shouldnât t say that
4602018-02-13T17:38:49 <jcorgan> that does sound like a (relatively) easy first thing to try, and you might get lucky
4612018-02-13T17:39:04 <michagogo> But Iâm pretty sure that the upgrade to gcc 6 in... yakkety or zesty, I think? And to 7 in bionic, was just part of a wholesale upgrade to those versions
4622018-02-13T17:39:52 <jcorgan> i'm going to be in pain when i start looking at migrating everything to 18.04, aren't I?
4632018-02-13T17:40:39 <michagogo> But besides not knowing even the basics of Ubuntu packaging, Iâd need the ability to go over all the other changes to the gcc-mingw-w64 source package and figure out which ones are related to the gcc 5/7 stuff, vs which ones should be there anyway
4642018-02-13T17:40:56 *** AaronvanW has quit IRC
4652018-02-13T17:40:56 <bitcoin-git> [bitcoin] ryanofsky opened pull request #12424: Fix rescan test failure due to unset g_address_type, g_change_type (master...pr/scang) https://github.com/bitcoin/bitcoin/pull/12424
4662018-02-13T17:41:01 <michagogo> jcorgan: define âeverythingâ
4672018-02-13T17:41:33 <jcorgan> most of the systems i use/support are 16.04, some still 14.04
4682018-02-13T17:41:33 *** AaronvanW has joined #bitcoin-core-dev
4692018-02-13T17:41:47 <jcorgan> it sounds like a lot of changes are coming in 18.04
4702018-02-13T17:41:53 <michagogo> No idea
4712018-02-13T17:42:06 <jcorgan> anyway, this is straying a bit from coredev
4722018-02-13T17:42:10 <michagogo> I mean, itâs 2 years of progress in the overall Linux ecosystem
4732018-02-13T17:42:17 <michagogo> Bringing everything up to date
4742018-02-13T17:42:38 <michagogo> So yeah, some things may breakâ¦
4752018-02-13T17:42:47 <jcorgan> well, the older LTS versions get new packages for everything that isn't a fundamental system change
4762018-02-13T17:43:24 <jcorgan> and the backports repo even lets you run newer kernels and video drivers
4772018-02-13T17:44:29 <jcorgan> it's usually the lower-level stuff like sysvinit/upstart/systemd that causes me pain
4782018-02-13T17:45:40 *** AaronvanW has quit IRC
4792018-02-13T17:47:43 <michagogo> Get new packages? What do you mean?
4802018-02-13T17:48:21 <michagogo> My understanding is that -updates only has bug fixes, ~never new wholesale versions of software that have features
4812018-02-13T17:48:47 *** farmerwampum has quit IRC
4822018-02-13T17:49:51 <jcorgan> maybe i'm thinking of PPAs, where authors make recent versions of their software for several different ubuntu releases
4832018-02-13T17:49:58 <michagogo> Yep
4842018-02-13T17:50:17 <michagogo> With a PPA you can give people who decide to use you whatever you want, whenever you want
4852018-02-13T17:50:51 <michagogo> (Including installing a backdoor on their systems. Itâs important that you trust the owners of PPAs you use.)
4862018-02-13T17:51:08 <jcorgan> well, i hope you figure out how to get teh mingw/WSL stuff working
4872018-02-13T17:51:18 *** Pavle has quit IRC
4882018-02-13T17:51:31 *** AaronvanW has joined #bitcoin-core-dev
4892018-02-13T17:51:32 <michagogo> Yeah, I think I'm just going to drop the issue at this point
4902018-02-13T17:52:08 <michagogo> cfields said that we're going to be using our own toolchain at some point (soon? ish?)
4912018-02-13T17:52:10 *** alfa has quit IRC
4922018-02-13T17:52:59 <cfields> michagogo: yes. Ubuntu versions/packages won't matter for gitian/travis after that.
4932018-02-13T17:53:26 <michagogo> cfields: I'm thinking mostly about the case of someone wanting to cross-build binaries for themselves
4942018-02-13T17:53:56 <michagogo> Either just not wanting to go through the hassle of setting up gitian, or with something like WSL
4952018-02-13T17:54:15 <michagogo> I really don't like the fact that we're giving users broken instructions, especially ones that (were they to work) have them completely mess up their system
4962018-02-13T17:54:15 <cfields> michagogo: sure. None of the new stuff will be required, ofc.
4972018-02-13T17:54:39 <michagogo> (https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md#building-for-64-bit-windows)
4982018-02-13T17:54:48 *** ProfMac has quit IRC
4992018-02-13T17:55:28 <michagogo> And of course, there's the fact that mingw is broken for everyone else
5002018-02-13T17:56:26 *** AaronvanW has quit IRC
5012018-02-13T17:56:30 *** S3RVO has joined #bitcoin-core-dev
5022018-02-13T18:04:08 *** timothy has quit IRC
5032018-02-13T18:04:56 *** harrymm has quit IRC
5042018-02-13T18:05:13 *** Randolf has joined #bitcoin-core-dev
5052018-02-13T18:07:00 *** PaulCapestany has quit IRC
5062018-02-13T18:10:21 *** PaulCapestany has joined #bitcoin-core-dev
5072018-02-13T18:18:02 *** harrymm has joined #bitcoin-core-dev
5082018-02-13T18:21:34 *** shesek has quit IRC
5092018-02-13T18:25:33 *** Murch has quit IRC
5102018-02-13T18:26:12 *** Murch has joined #bitcoin-core-dev
5112018-02-13T18:27:24 *** ProfMac has joined #bitcoin-core-dev
5122018-02-13T18:28:13 *** sengehest has joined #bitcoin-core-dev
5132018-02-13T18:29:19 *** Chris_Stewart_5 has joined #bitcoin-core-dev
5142018-02-13T18:29:39 *** S3RVO has quit IRC
5152018-02-13T18:32:06 *** ProfMac has quit IRC
5162018-02-13T18:33:32 *** PaulCape_ has joined #bitcoin-core-dev
5172018-02-13T18:34:16 *** PaulCapestany has quit IRC
5182018-02-13T18:35:02 *** shesek has joined #bitcoin-core-dev
5192018-02-13T18:35:02 *** shesek has joined #bitcoin-core-dev
5202018-02-13T18:39:36 *** Murch has quit IRC
5212018-02-13T18:42:20 *** Murch has joined #bitcoin-core-dev
5222018-02-13T18:45:35 <bitcoin-git> [bitcoin] richardkiss opened pull request #12425: Add some script tests (master...feature/bool_tests) https://github.com/bitcoin/bitcoin/pull/12425
5232018-02-13T18:47:38 *** promag has quit IRC
5242018-02-13T18:49:38 *** promag has joined #bitcoin-core-dev
5252018-02-13T18:50:42 *** promag has quit IRC
5262018-02-13T18:51:16 *** Murch has quit IRC
5272018-02-13T18:53:56 *** Murch has joined #bitcoin-core-dev
5282018-02-13T19:10:38 *** Victorsueca has quit IRC
5292018-02-13T19:11:48 *** Victorsueca has joined #bitcoin-core-dev
5302018-02-13T19:11:59 *** AaronvanW has joined #bitcoin-core-dev
5312018-02-13T19:16:33 *** sengehest has quit IRC
5322018-02-13T19:19:13 *** mirese has joined #bitcoin-core-dev
5332018-02-13T19:22:58 *** mirese_ has quit IRC
5342018-02-13T19:23:17 *** ProfMac has joined #bitcoin-core-dev
5352018-02-13T19:27:35 *** AaronvanW has quit IRC
5362018-02-13T19:31:48 *** Murch has quit IRC
5372018-02-13T19:36:17 *** Julie73Donnelly has quit IRC
5382018-02-13T19:37:03 *** mirese_ has joined #bitcoin-core-dev
5392018-02-13T19:37:33 *** mirese__ has joined #bitcoin-core-dev
5402018-02-13T19:40:03 *** Murch has joined #bitcoin-core-dev
5412018-02-13T19:40:11 *** mirese___ has joined #bitcoin-core-dev
5422018-02-13T19:40:34 *** mirese has quit IRC
5432018-02-13T19:41:30 *** mirese has joined #bitcoin-core-dev
5442018-02-13T19:41:50 *** mirese_ has quit IRC
5452018-02-13T19:43:51 *** dcousens has quit IRC
5462018-02-13T19:44:06 *** mirese__ has quit IRC
5472018-02-13T19:44:09 *** dcousens has joined #bitcoin-core-dev
5482018-02-13T19:44:40 *** mirese___ has quit IRC
5492018-02-13T19:46:05 *** jtimon has joined #bitcoin-core-dev
5502018-02-13T19:48:27 *** dcousens has quit IRC
5512018-02-13T19:49:05 *** dcousens has joined #bitcoin-core-dev
5522018-02-13T19:50:29 *** AaronvanW has joined #bitcoin-core-dev
5532018-02-13T19:52:42 *** Tobin19Abernathy has joined #bitcoin-core-dev
5542018-02-13T19:55:10 *** Dizzle has joined #bitcoin-core-dev
5552018-02-13T19:55:49 *** Murch has quit IRC
5562018-02-13T19:57:49 *** Randolf has quit IRC
5572018-02-13T19:58:14 *** promag has joined #bitcoin-core-dev
5582018-02-13T20:01:56 *** AaronvanW has quit IRC
5592018-02-13T20:01:57 *** SopaXorzTaker has quit IRC
5602018-02-13T20:04:05 *** tli has joined #bitcoin-core-dev
5612018-02-13T20:08:20 *** promag has quit IRC
5622018-02-13T20:08:54 *** promag has joined #bitcoin-core-dev
5632018-02-13T20:08:54 *** neha has quit IRC
5642018-02-13T20:10:27 *** Tobin19Abernathy has quit IRC
5652018-02-13T20:13:21 *** promag has quit IRC
5662018-02-13T20:15:53 *** neha has joined #bitcoin-core-dev
5672018-02-13T20:22:35 *** promag has joined #bitcoin-core-dev
5682018-02-13T20:23:16 *** tli has quit IRC
5692018-02-13T20:24:37 <gmaxwell> Note: if anyone here uses bitmessage, it has a remote code execution bug. (deserializing using eval)
5702018-02-13T20:25:01 <sipa> what is it written in?
5712018-02-13T20:26:06 <gmaxwell> Python.
5722018-02-13T20:26:32 <contrapumpkin> is today national facepalm security day or something
5732018-02-13T20:29:56 <esotericnonsense> deserializing using eval ???
5742018-02-13T20:30:03 <esotericnonsense> -_-.
5752018-02-13T20:31:23 *** promag has quit IRC
5762018-02-13T20:35:10 <instagibbs> the vulnerability is being exploited in the wild as well
5772018-02-13T20:35:38 <instagibbs> not sure about success, but trying to run windows executables, grab electrum wallets, the like
5782018-02-13T20:37:27 <gmaxwell> esotericnonsense: deserializing using eval is a thing I've seen many times w/ javascript, php, and python.
5792018-02-13T20:37:49 <contrapumpkin> java does its own version of that too, if you use their native serialization
5802018-02-13T20:38:45 *** tli has joined #bitcoin-core-dev
5812018-02-13T20:38:58 *** jamesob has joined #bitcoin-core-dev
5822018-02-13T20:39:14 <esotericnonsense> gmaxwell: oh sure it's a thing. i just feel like it's one of the most famous 'don't do this' footguns
5832018-02-13T20:39:31 <esotericnonsense> evidently not
5842018-02-13T20:44:41 *** jamesob_ has joined #bitcoin-core-dev
5852018-02-13T20:47:00 *** jamesob has quit IRC
5862018-02-13T20:48:19 <mmgen> wouldn't use of the json module protect against this?
5872018-02-13T20:49:40 <mmgen> first json.loads(), then deserialize?
5882018-02-13T20:52:27 <reardencode> mmgen: yes.
5892018-02-13T20:52:40 <mmgen> strange they didn't do that
5902018-02-13T20:52:46 *** Marcia15Hilpert has joined #bitcoin-core-dev
5912018-02-13T20:55:53 *** Randolf has joined #bitcoin-core-dev
5922018-02-13T20:59:07 <mmgen> the fix: https://github.com/Bitmessage/PyBitmessage/commit/3a8016d31f517775d226aa8b902480f4a3a148a9
5932018-02-13T21:00:17 *** Randolf has quit IRC
5942018-02-13T21:00:31 *** tli has quit IRC
5952018-02-13T21:02:31 <contrapumpkin> that still seems like an iffy fix, even though the scope for mischief is far more limited
5962018-02-13T21:03:14 <contrapumpkin> it'll basically call an arbitrary method specified by untrusted data on the specified class
5972018-02-13T21:03:14 *** meshcollider has joined #bitcoin-core-dev
5982018-02-13T21:03:27 <bitcoin-git> [bitcoin] MarcoFalke opened pull request #12426: qt: Initialize members in WalletModel (master...Mf1802-qtInitializeMembersWalletModel) https://github.com/bitcoin/bitcoin/pull/12426
5992018-02-13T21:05:17 *** promag has joined #bitcoin-core-dev
6002018-02-13T21:07:09 <mmgen> contrapumpkin: hopefully just an emergency fix. far better than nothing in any case
6012018-02-13T21:07:41 <contrapumpkin> yup :)
6022018-02-13T21:13:45 *** harrymm has quit IRC
6032018-02-13T21:17:33 <mmgen> seems like try: assert data[""
6042018-02-13T21:18:07 <mmgen> seems like try: assert data[""] in ('message','vote') would have been safer
6052018-02-13T21:19:10 <contrapumpkin> the hoops people go through to avoid writing a proper parser... ð¢
6062018-02-13T21:19:42 <mmgen> since 'message' and 'vote' are the only permissible values
6072018-02-13T21:19:53 <sipa> it's getting a bit off topic for this channel
6082018-02-13T21:19:59 <contrapumpkin> sorry :) I'll shut up
6092018-02-13T21:21:03 *** Victorsueca has quit IRC
6102018-02-13T21:21:07 <mmgen> sipa: sorry
6112018-02-13T21:21:47 <sipa> np!
6122018-02-13T21:22:18 *** Victorsueca has joined #bitcoin-core-dev
6132018-02-13T21:22:37 <mmgen> sipa: though it **was** gmaxwell who brought up the topic
6142018-02-13T21:23:46 <sipa> i'm aware, and i participated too; no blame here
6152018-02-13T21:23:57 <sipa> just noting that the discussion is getting too far removed from the topic here
6162018-02-13T21:24:46 <mmgen> mmgen: i know, i'll really shut up now ;)
6172018-02-13T21:27:07 *** harrymm has joined #bitcoin-core-dev
6182018-02-13T21:27:13 *** AaronvanW has joined #bitcoin-core-dev
6192018-02-13T21:29:12 <bitcoin-git> [bitcoin] sipa opened pull request #12427: Make signrawtransaction accept P2SH-P2WSH redeemscripts (master...201802_signrawp2shp2wsh) https://github.com/bitcoin/bitcoin/pull/12427
6202018-02-13T21:30:55 <gmaxwell> doh, how'd we miss that
6212018-02-13T21:32:19 <grafcaps> hah
6222018-02-13T21:38:25 *** mmgen has quit IRC
6232018-02-13T21:38:32 <sipa> gmaxwell: it was actually partially being addressed in #11708
6242018-02-13T21:38:36 <gribble> https://github.com/bitcoin/bitcoin/issues/11708 | Add P2SH-P2WSH support to signrawtransaction and listunspent RPC by MeshCollider · Pull Request #11708 · bitcoin/bitcoin · GitHub
6252018-02-13T21:39:39 *** tsunami_86 has joined #bitcoin-core-dev
6262018-02-13T21:41:05 <promag> sipa: is it possible to add a test for #12427?
6272018-02-13T21:41:07 <gribble> https://github.com/bitcoin/bitcoin/issues/12427 | Make signrawtransaction accept P2SH-P2WSH redeemscripts by sipa · Pull Request #12427 · bitcoin/bitcoin · GitHub
6282018-02-13T21:41:21 <sipa> promag: yes, will look into that
6292018-02-13T21:41:29 <sipa> just wanted to have the PR visible as soon as possible
6302018-02-13T21:42:37 *** tsunami_86 has quit IRC
6312018-02-13T21:46:44 *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
6322018-02-13T21:46:44 *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
6332018-02-13T21:51:09 <Chris_Stewart_5> sipa: Is the basic idea that if we can sign the p2sh redeem script, we should be able to sign P2SH(P2WSH())?
6342018-02-13T21:51:10 *** laurentmt has quit IRC
6352018-02-13T21:54:52 *** Murch has joined #bitcoin-core-dev
6362018-02-13T21:58:00 *** Murch has quit IRC
6372018-02-13T22:00:55 *** Murch has joined #bitcoin-core-dev
6382018-02-13T22:01:44 *** promag has quit IRC
6392018-02-13T22:03:23 *** cheese_ has joined #bitcoin-core-dev
6402018-02-13T22:04:03 *** harrymm has quit IRC
6412018-02-13T22:05:48 *** Cheeseo has quit IRC
6422018-02-13T22:10:27 *** Murch has quit IRC
6432018-02-13T22:10:49 *** Cogito_Ergo_Sum has quit IRC
6442018-02-13T22:11:26 *** Guyver2 has quit IRC
6452018-02-13T22:13:06 *** Murch has joined #bitcoin-core-dev
6462018-02-13T22:14:23 *** mrannanay has quit IRC
6472018-02-13T22:15:31 *** jamesob_ has quit IRC
6482018-02-13T22:15:53 *** Randolf has joined #bitcoin-core-dev
6492018-02-13T22:17:46 *** promag has joined #bitcoin-core-dev
6502018-02-13T22:19:01 *** Randolf has quit IRC
6512018-02-13T22:21:03 *** Murch has quit IRC
6522018-02-13T22:25:29 *** tryphe has quit IRC
6532018-02-13T22:25:54 *** tryphe has joined #bitcoin-core-dev
6542018-02-13T22:27:25 *** Murch has joined #bitcoin-core-dev
6552018-02-13T22:48:57 *** owowo has quit IRC
6562018-02-13T22:51:41 <eklitzke> this is kind of cool, i got a build of bitcoind working with systemtap markers (in this case I'm marking CCoinsViewCache flushes): https://gist.github.com/eklitzke/8bf6957fe886ddec36cde737d69ac6f5
6572018-02-13T22:56:32 *** PaulCape_ has quit IRC
6582018-02-13T22:59:03 *** PaulCapestany has joined #bitcoin-core-dev
6592018-02-13T22:59:11 *** dongcarl has joined #bitcoin-core-dev
6602018-02-13T23:00:52 <gmaxwell> eklitzke: whats the advantage of that?
6612018-02-13T23:04:29 *** Murch has quit IRC
6622018-02-13T23:05:30 *** Murch has joined #bitcoin-core-dev
6632018-02-13T23:13:33 *** Emcy has quit IRC
6642018-02-13T23:13:45 <eklitzke> i'm going to add more probes and then use that to measure some work i'm trying to do to the ccoinscacheview, e.g. i want to make the cache into a real writeback cache and get cache hit statistics out of it
6652018-02-13T23:14:42 <eklitzke> today if you wanted to get cache hit statistics from CCoinCacheView::GetCoin you'd have to hack up some existing rpc to dump the information (or dump it to a log), with stap/dtrace probes that would be much easier and less invasive
6662018-02-13T23:16:15 <warren> Does anyone have working gitian with qemu-kvm instead of lxc? I want to compare notes.
6672018-02-13T23:18:35 <gmaxwell> eklitzke: you'll defeat its purpose entirely
6682018-02-13T23:20:13 *** larafale has quit IRC
6692018-02-13T23:20:28 <gmaxwell> eklitzke: the cache's purpose for existing is not to act as a cache, it's purpose is to act as a buffer to prevent writes from hitting the database entirely. And also to allow atomic updates to keep the database consistent with the chain at a block level (this is not as important since we now use a replay for consistency). It also satisfies reads, but if you defeat that it doesn't make a huge p
6702018-02-13T23:20:34 <gmaxwell> erformance difference in IBD.
6712018-02-13T23:20:56 *** larafale has joined #bitcoin-core-dev
6722018-02-13T23:20:57 *** Emcy has joined #bitcoin-core-dev
6732018-02-13T23:21:46 <gmaxwell> okay maybe on a non-SSD the read cache matters somewhat more, to be fair I haven't benchmarked it with defeated read caching on a spinning disk.
6742018-02-13T23:21:47 <eklitzke> i have thought through these issues, you can increase the amount of coins in memory during ibd in a way that's safe and still batches disk writes
6752018-02-13T23:21:57 <eklitzke> yes this is primarily for spinning disks
6762018-02-13T23:22:04 <eklitzke> or worse, people's cloud vm instances
6772018-02-13T23:22:45 <sipa> eklitzke: i've experimented with various cache flushing strategies before which try to predict which entries will be more useful in the near future, and keep them longer
6782018-02-13T23:22:52 <gmaxwell> eklitzke: the purpose of the replay stuff we did in 0.15.x was in part to enable incremental flushing, so yes, I also see advantages there.
6792018-02-13T23:23:04 <sipa> in practice, anything that reduced the ability to avoid writes slowed things down
6802018-02-13T23:23:14 <gmaxwell> but what we did find is that trying to achieve a higher read hit rate is mostly pointless.
6812018-02-13T23:23:31 <gmaxwell> (on a SSD at least)
6822018-02-13T23:24:08 <gmaxwell> I fear we've more or less given up on performance on non-SSDs :( esp since many large spinning disks are shingled now and have horrifying performance.
6832018-02-13T23:24:51 <gmaxwell> Incremenal flushing would also give the advantage of getting rid of flushing latency spikes and perhaps better overlap IO and validation.
6842018-02-13T23:24:57 *** larafale has quit IRC
6852018-02-13T23:24:59 <eklitzke> the flushing code right now is pretty inefficient, anyway databases like mysql do this by having a dirty buffer pool watermark that causes them to flush (potentially a large amount of data), and then they retaing data in memory that was flushed to disk if it's not dirty
6862018-02-13T23:25:54 <eklitzke> if you kept exactly the same semantics wrt flushing a large amount of data but didn't automatically expire the old coins from memory that would be an improvement (in my opinion)
6872018-02-13T23:25:58 *** AaronvanW has quit IRC
6882018-02-13T23:26:20 <eklitzke> would love to chat with you about it next week, as i'm participating in the chaincode labs hacker residency program
6892018-02-13T23:26:25 <gmaxwell> eklitzke: We've measured that, it doesn't help.
6902018-02-13T23:26:34 *** AaronvanW has joined #bitcoin-core-dev
6912018-02-13T23:26:47 <eklitzke> i've also measured it and shown that it does
6922018-02-13T23:26:48 <gmaxwell> Thats the whole reason I brought the subject up-- so you don't assume read caching is important.
6932018-02-13T23:27:18 <gmaxwell> eklitzke: Interesting, I'd love to see those results.
6942018-02-13T23:27:19 <eklitzke> if there are any writeups or mailing list posts about experiments in this area i'd be interested to read them
6952018-02-13T23:27:39 <gmaxwell> eklitzke: probably in the PR introducing the replay on start there are some of them.
6962018-02-13T23:27:49 <eklitzke> ok, i'll take a look
6972018-02-13T23:28:57 *** AaronvanW has quit IRC
6982018-02-13T23:29:20 <sipa> eklitzke: i'll be at the residency on 22nd and 23rd
6992018-02-13T23:29:24 <gmaxwell> I'd offer you the specific code I benchmarked but I don't have access to the system with it anymore.
7002018-02-13T23:29:36 *** AaronvanW has joined #bitcoin-core-dev
7012018-02-13T23:30:08 *** jojeyh has quit IRC
7022018-02-13T23:31:02 <sipa> eklitzke: the bitcoin utxo set is pretty unique as a data set in that a very large number of newly written entries are almost immediately after deleted again
7032018-02-13T23:31:16 *** AaronvanW has quit IRC
7042018-02-13T23:31:26 *** Chris_Stewart_5 has quit IRC
7052018-02-13T23:31:35 <gmaxwell> We tested many configurations. For an example, one of the specific cases we tested was having flush leave the entries in but marking them as non-dirty and existing in the backing store, and seperately removing non-dirty entries when full. What appeared to be the case is that avoiding the read hit seemed basically irrelevant, because if i got read, that means it is getting spent, which means th
7062018-02-13T23:31:41 <gmaxwell> ere is going to be an erasure which must hit the disk.
7072018-02-13T23:31:51 *** AaronvanW has joined #bitcoin-core-dev
7082018-02-13T23:32:04 <TD-Linux> since many large spinning disks are shingled now and have horrifying performance. <- these are still really uncommon
7092018-02-13T23:32:33 <gmaxwell> TD-Linux: they seem to be 100% of people showing up and talking about their node performance on spinning disks, but now that I say that outloud...
7102018-02-13T23:32:34 <sipa> eklitzke: but we should talk more about this
7112018-02-13T23:33:03 <eklitzke> i would like to change some of the leveldb settings too, but that's another matter entirely; for instance leveldb is configured to only open 64 files at once, so during ibd or a reindex you see the process thrashing a lot in a pattern where it opens an lbd file, mmaps it, munmaps it, and closes it
7122018-02-13T23:33:09 <eklitzke> that's another matter entirely though
7132018-02-13T23:33:20 <gmaxwell> eklitzke: by all means if you have results, great.
7142018-02-13T23:33:38 *** TheRec_ is now known as TheRec
7152018-02-13T23:33:52 <gmaxwell> eklitzke: we suffer from FD exhaustion issues currently. :(
7162018-02-13T23:34:35 <eklitzke> yeah I talked to BlueMatt about it a bit yesterday, the thing is you can mmap a file, close the fd, and the mmap mapping is still valid
7172018-02-13T23:34:56 <gmaxwell> ah, cute.
7182018-02-13T23:34:57 <eklitzke> so if you didn't mind the extra memory overhead from the PTEs, you could potentially mmap all of the ldb files at once and not actually have 1000+ ldb files open
7192018-02-13T23:35:05 <eklitzke> doing *all* of them is probably too agressive
7202018-02-13T23:35:12 <gmaxwell> leveldb doesn't use mmap on 32 bit hosts.
7212018-02-13T23:35:27 <gmaxwell> and on 64-bit the only harm is the PTE as you note, which is pretty minor.
7222018-02-13T23:36:34 *** AaronvanW has quit IRC
7232018-02-13T23:36:42 *** rex_4539 has quit IRC
7242018-02-13T23:37:21 <gmaxwell> eklitzke: if you want to get into leveldb internals there are probably a lot of other gains. E.g. our keys are almost uniformly random, and so bisections in leveldb can probably be replaced with a secant search. (and if our keys aren't random enough, we could fix that) leveldb block size is tunable too
7252018-02-13T23:37:45 <gmaxwell> I think I found larger blocks to be a loss, but that might not be as true with a secant search.
7262018-02-13T23:38:19 <eklitzke> yeah I was actually surprised that leveldb itself wasn't doing the mmap trick I just described (or at least have it as a tuneable parameter), I want to dig deeper into their code
7272018-02-13T23:39:00 <promag> ryanofsky: friendly ping https://github.com/bitcoin/bitcoin/pull/11687#pullrequestreview-94272991
7282018-02-13T23:40:50 <gmaxwell> eklitzke: I have a believe which I've not been able to validate yet, that a lot of the performance limitations of the current dbcache is all the malloc activity... that often doesn't show up too clearly in profiles. We'd talked about changing the main dbcache to an open hashtable with fixed size entries (and some special indirection to an exception map for unusually large entries).. but it's a l
7292018-02-13T23:40:56 <gmaxwell> ot of work to just try it and see if it helps.
7302018-02-13T23:42:10 <eklitzke> interesting because glibc malloc already has systemtap probes bulitin, since 2.24 i think, so that would go well with my systemtap branch because you could write stap scripts that correlated activity in malloc with events in bitcoind
7312018-02-13T23:42:42 *** meshcollider has quit IRC
7322018-02-13T23:43:43 *** AaronvanW has joined #bitcoin-core-dev
7332018-02-13T23:55:13 *** vicenteH has quit IRC
7342018-02-13T23:58:02 *** Madscotslad has joined #bitcoin-core-dev
7352018-02-13T23:58:24 <Madscotslad> Wow nice to see a full IRC room its been a while :) hey folks.
7362018-02-13T23:58:51 <luke-jr> if you think this is full, you should see #bitcoin