00:59:53 Review requested: https://github.com/monero-project/monero/pull/6451 01:06:32 What is the preferred amount of lockable memory to dispel the warning? Is this some established value? 01:08:50 nice compact improvement in 6451 :) 01:12:15 Thanks TheCharlatan! 02:25:20 ah, love that 6451 sarang 09:49:52 Hello 09:50:53 I'd like to contribute to the project, but would like to learn, if you are willing to accept my change, before I even start working. References: 09:50:55 https://repo.getmonero.org/monero-project/ccs-proposals/-/merge_requests/138 09:51:09 https://repo.getmonero.org/monero-project/ccs-proposals/-/blob/b8033720941c69a1dd45c26bf6c1bb1d03863b77/mj-compil-time-reduction.md 09:51:42 The POC contribution can be easily done unfunded. 10:02:28 mjxmr: i think that some parts go a bit crazy with templates (-ftemplate-depth needs to be increased). do you think it would be worth going through with clang's profiler seeing if there's anything that should be changed? 10:04:43 asymptotically: I think, that using profilers is always a better idea, than using a gut feeling (same applies for speed optimisations). Otherwise a programmer falls into a dogmatic way of thinking, while I prefer to work evidence based. 10:06:02 However in this case, boost headers, I'm absolutely sure, that this is the major slowdown. Abuse of templates create the need of having to include the boost headers in a parent header. This can be avoided with some effort. 10:06:56 I am so sure, because I ate my teeth on this many times already. 10:08:32 also what does the ccache integration with cmake do? i thought you just had to set CC and CXX 10:10:04 Yes, you can do this of course, but not everybody is proficient enough to do this. My goal would be to set the CC / CXX automatically if ccache is detected in path. This is minor thing, yeah. 10:16:46 At our job we disable ccache for special final builds, to minimize the risk of ccache affecting the binaries that go to the customer, but we have it on by default. 10:19:09 ... during the development, I mean, so 90% of time. 10:36:08 asymptotically: "Abuse of templates create the need of having to include the boost headers in a parent header. " A poor man's solution to this is using precompiled headers of the boost headers collection. This helps to reduce the compilation time by up to 50% per header, but a lot more this can be achieved via wrappers. 10:40:49 Perhaps it wouldn't be a bad idea to start off with this, as the least intrusive measure? 10:41:30 I’d wait for feedback by moneromooo / vtnerd 10:43:36 Thanks for the tip. While I wait, I want to give mad props for cutting off the ASICs. This was a giant leap towards decentralisation. 11:29:08 A speed boost would be nice. Whether it's worth churn that can introduce bugs depends on how much is gained. 11:29:32 Last I tried, precompiled headers did not gain much, the patch end up rejected. Maybe your method might gain more though. 11:30:02 Someone else started working on moving some serialization away from headers. Can't recall the nick now though. 11:30:23 ccache works fine with monero, I use it all the time. 11:30:59 I did not know CLANG had a compile time profiler. Sounds nifty. 11:42:06 not sure if chromium is the only tool that can open the output :/ https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/ 12:12:22 moneromooo: What would you propose doing now? It might be faster to just start moving the serialization away, since I have a lot of free time until the mid-May. I don't know how I would find the contributor with over 2.200 forks currently. 12:14:06 But I see, that there is interest. I think I'll just start then :) 12:22:30 How about doing something else, since nobody's doing precmpiled headers AFAIK ? :) 12:23:08 IIRC when I tried it, the reason why it did not gain much at all was we had several directories, and the precompiled thing was being done one per directory. 12:24:04 Anyone remembers who was asking about changing serialization ? IIRC we talked about the speedup vs churn pros/cons too :) 12:24:16 This is also cool. But when was the last time, that you heard from this person, who offered moving the serialization away from headers? 12:24:55 Very roughly, I'd say... a couple months maybe ? 12:25:11 because I can work full time on this now, until mid-May, meaning I could quickly bring it into life. Then just an hour or two after work. 12:27:19 Well, your pick really. 12:27:52 I think I'll give it a shot. 12:28:44 Historically, I've been pushing against churn (ie, rewriting stuff for no good reason (aka some code standard says so)). I'm trying to do it less now :) 12:29:06 So build times are a good reason, if the speedup is substantial. I'm curious to see how much you get. 12:29:51 The point here is obviously not to create any differences in functionality. 12:30:32 but I know from my experience, that developers usually are careless, and need a janitor :) So if you assume no churn, the mess never gets cleaned up. 12:32:46 was it woodser ? 12:36:19 I don't think so. 13:18:08 A small finding: 13:18:10 enjo@tp:~/devel/github/monero-build$ make test 13:18:11 Running tests... 13:18:11 Test project /home/enjo/devel/github/monero-build 13:18:12 No tests were found!!! 13:20:23 make release-test 13:21:40 To run one particular tests, make -C $BUILDDIR/subtree test 13:21:54 eg: make -C build/release/foo/tests/unit_tests test 13:23:38 make release-test 13:23:39 make: *** No rule to make target 'release-test'. Stop. 13:23:47 building out of source. 13:24:48 You messed up your Makefile most likely. Check the README.md file. 13:24:53 See git diff 13:43:52 Ah ok. This can't be executed from an out of source directory. 13:45:29 It can if you know what you're doing. You hinted you know cmake, so look at the contents of Makefile. 13:46:33 it's been a couple of months since i last tried the tests but iirc it just worked when i did `cmake ..` instead of trying the makefile 13:48:05 Let's say, that I will address this at a later time. For now I will be happy if the tests execute and keep me from screwing things up. 15:39:27 moneromooo: For how long do the core_tests normally run? I feel like they hung since like an hour. 15:42:30 Sounds plausible. If you run them often, there's a (closed) patch from me on github that speeds them up a lot. 15:43:00 Takes ~ 10 min with the patch or so. 15:44:40 Thank you. 15:45:10 I will run them for the night then. 15:46:53 Could you give me a hint, which tests cover the class tx_source_entry? I think I will start from this one as a sample small enough. 15:47:10 Transaction tests. 15:47:18 Which is going to be most of them. 15:51:37 Great. Thanks! 17:14:15 mjxmr moneromooo selsta : I have a github issue related to serialization, the technique woud reduce compile times as things can be split into a cpp 17:14:56 https://github.com/monero-project/monero/issues/6406 17:15:00 this one? 17:15:00 other things can be done to reduce compile times, but doing so without re-organizing large parts of the code at once coudl be tricky 17:15:28 yes 17:16:08 someone could reduce it further if the macro stuff was a bit much, but the general idea of splitting header/cpp is somewhat straightforward 17:17:04 the biggest compilation issues are the epee server stuff, and its going to be tricky to refactor that without it being a full rewrite (it may have to be to achieve the goa) 17:17:38 I started some small stuff with hex encoding, trying to remove boost::lexical_cast from being included so often, its fairly heavy to pull in the inculdes 17:18:27 gcc and clang both have a flag for reporting internal time spent compiling which is useful too 17:18:54 the flag gives breakdowns on much time spent doing macro expansion, etc., so you can get an interna look at time spent 17:20:28 ah found the proposal was in the other tab 17:22:05 yeah moving them into a cpp should help, but the bulk is _probably_ epee, but its hard to profile for certain due to the ways things are compiled 17:23:47 the boost serialization stuff was separate than what I was working on, and has nearly the same problem+solution 18:14:19 vtnerd: Shall we coordinate our efforts? My concrete proposal is to completely wrap the boost::archive. Putting the serialization templates can't be achieved, as it turns out, but there's no problem in wrapping the boost:archive. 18:15:19 It will be however hard to prove improvements, since even if I wrap the b::archive in one header, it is still included by some other headers, yielding no improvement, until they are sorted out. 18:15:47 In other words, the project needs *months* of maintenance, just be sure, that you realize it. 18:16:32 We have to do it one by header type by one. 18:19:21 EDIT: "Putting the serialization templates INTO .CPP FILES can't be achieved, as it turns out" 19:25:31 vtnerd: Wrapping the serialization will make it easier to integrate different serialization techniques later on.