hyc, sech1, tevador, (and the likes of gingeropolous who helped test), thank yous soo much for RandomX. In my view this is groundbreaking work. There's been a lot of nonsense naysayers and you've all shown restraint which is admirable too. Great idea, great code, great delivery, and genuinely nice people. Thank you.

"Great idea, great code, great delivery, and genuinely nice people." This is why i'm here too.

I'm still trying to load randomx library from memory buffer into mmap and call randomx_alloc_cache without any errors.

thanks jtgrassie. and yes, sech1 and tevador are first class.

Finally, randomx_alloc_cache works

thanks cohcho , thought i had

it works now. yay

this a vega farm u think? xmr.nanopool.org/account/463tWEBn5X…d5225e44f8d39454d1d906e7b916929df7d

The last problem is that large constants are being used by absolute address within executable image (mov $0x5fb760,%rcx) instead of relative address to $pc.

cohcho I think that GCC has an option to compile everything as PIC (position independent code)

`gcc -static -fPIC ...` produces absolute offsets to rodata but everything else looks ok

`gcc -static-pie ...` is likely the option I need but it relies on some functions that initialize executable before actual main()

I'm trying to get what I should call externally in order to initialize executable but not run actual main()

for that -static-pie

You have to call entry point of the executable, C/C++ run static variable initialization there

then you can call other functions

I can't compile your two sentences into actual code, too ambiguous.

:)

And i think you're wrong because entry point will continue actual main() and to syscall with exit

If you're trying to loading executable binary into memory and do some stuff, you have to follow the normal procedure that OS does

some functions may still work fine if they don't use static variables

I need fully functional randomx library, some functions already tested with `gcc -static` binary.

if you check superscalar.cpp, you'll see a lot of constants in "MacroOp" class which are statically initialized. If you just load the code into memory, these won't be initialized and dataset initialization won't work.

so you have to either link the library properly or make .so/.dll file and load them properly, calling entry point

sech1: can you PR your optimization to RandomX repo?

yes, I'm trying to remove make my branch even with upstream before applying changes

.tr imi bag pula in masa de muist

miningpoolstats seems to be broken for monero: miningpoolstats.stream/monero

they decided that all pools have "invalid block height"

minexmr show Blockchain Height: 1979121

my node shows 1979122

or maybe minexmr just shows last mined block in the network, not current height

xmrchain.net is stuck

yes, minexmr changed to 122 and my node to 123

so it's ok

tevador tried to port my optimization to RandomX repo, but there are already too many differences, hashes don't match. I'll look into it later today.

Maybe it's some stupid one line bugfix somewhere

OK

tevador done tevador/RandomX #166

cool, so this actually reduces the overall overhead

optimizations in the inner loop would increase the overhead

yes, combined loop is ~70% faster than doing hash -> fill separately

and I added prefetch to it to hide L3 latency

because Intel CPUs benefit from turning off HW prefetcher, so I have to add it there explicitly

let's see the speed up on ryzens with disabled prefetch

let's figure out how to disable it first

got 4% speed up on 3700X

disabling pretetcher is a global setting? most other s/w will be worse

sech1: 2% faster on 1700 with disabled prefetch

hyc: yes, global

where do you disable it?

BIOS

wtf, what's it called in BIOS?

something like L1 Stream Prefetcher

all my asrock boards have it

hmm

I don't see any prefetch options on my board

could this help? --> 32ipi028l5q82yhj72224m8j.wpengine.n…DC2017-Optimizing-For-AMD-Ryzen.pdf

"Avoid software prefetch. Improve Instruction Cache (IC) & Op Cache (OC) hit rate by using

efficient hardware prefetch.

bad advice for RandomX

OK, was an idea ;)

if you guys need an alternative block explorer

For RandomX the advice would be "avoid hardware prefetch, improve hit rate by using efficient software prefetch"

HW prefetch knows nothing about where the next access would be, but RandomX code knows

is this valid for gpu rigs also ?

hardware prefetch is dumb, but works well in most cases because of spare bandwidth and sequential nature of most workloads

I would assume GPUs don't have hardware prefetch

They do actually

Vega GPUs have prefetch from memory to l2 cache

interesting, I guess there is no way to disable it?

I think this temphash should be typedef'd before merging tevador/RandomX #166#pullrequestreview-324935989

there's no good reason to leave it up to API user to screw this up

hyc: maybe we should change void* output to char output[RANDOMX_HASH_SIZE] also? github.com/tevador/RandomX/blob/5c0…ba0465708e2087e0/src/randomx.h#L239

or are there use cases when you WANT to pass a pointer?

typedef or just make it a struct containing uint64_t data[8]?

I prefer typedefs, because then you don't have to think about what (foo *) means

I think tevador can handle this refactoring, no need to mix it with this PR

as for the output[SIZE] good question.

yeah, I was originally thinking to do it after merging

ok, fair enough

the only I can think of about output[] is if you wanted to splat the output into the middle of a larger buffer

but in general, when you can take advantage of typechecking, you should.

that's why I prefer C++ more

stricter type checks + templates, but you can still write C-style code

yeah, unfortunately this change spilled into the API, which must support C

heh. my statement "take advantage of typechecking" was only in the context of the limits of C. I think C++ is overboard.

those new optimizations on master?

of xmrig?

saw an improvement with my 1920x, but not the 7402

yes

Crashes on early 1st gen Ryzens have something to do with opcache: xmrig/xmrig #1348#issuecomment-560122919

that guy managed to run it on his Ryzen without performance drop in the end

yeah that's me, working great now

I also have another 1700 that I dont have mobo for atm but changed it with the one I got working and opcache made it stable too

hyc sech1 tevador/RandomX #170

nice

didn't think about this solution

tevador check your DM

16.4 KH/s on 3950X: reddit.com/r/MoneroMining/comments/…rate_optimization_amd_3950x/f9csosz

is the result credible?

this guy doesn't seem to all that certain

3600 MHz 15-14-14-14-32 RAM can do this

dual rank RAM, even better

560 H/s per thread is a normal number for Ryzen @ 4 GHz, I get the same

ok, so that means DDR4 bandwidth is not a limiting factor then

16.4 KH/s = 16.4 GB/s bandwidth, far from the DDR4 limit

transactions per second is the bottleneck here

sure. bandwidth assumes sequentail access, not random

does someone here have FreeBSD?

on a VM

I want to fix: tevador/RandomX #149

virtualbox

first solution would be to remove CPU affinity support on FreeBSD

ah I can't access my VM at the moment. it's on the external SSD back home.

unless that struct has a different name there

first I will disable it and then when we have some test env, we can fix it

I always just comment that out, yeah. it's missing on android too

how to detect Android in the preprocessor?

__android__

oh, wait

it's all upper case

__ANDROID__

looks fine

looks like freeBSD is just missing #include <sys/cpuset.h>

or maybe instead, unix.com/man-page/freebsd/3/pthread_affinity_np

so add #include <pthread_np.h> and it should be fine

can be added later, I don't want to do blind changes without being able to test

right

well, I'll be back home in a few days, can followup on it then if nobody else does a freeBSD build before then

Damn, what a busy day - my gmail github folder has 50+ notifications today and counting

plus countless reddit/other forums notifications

I don't get reddit notifications. keeps life simpler ;)

this still needs solving

probably by disabling JIT altogether

or writing your own clear_cache() function in asm

that should only be a couple instructions

the main issue is that iOS will kill the process if it tries to set a page to be executable

AFAIK

ah, he never got far enough to even see if that's true

Technically, just calling a big enough function (64 KB of NOPs) will clear code cache

No need to use fancy instructions

they have this: developer.apple.com/documentation/b…nts/com_apple_security_cs_allow-jit

but AFAIK applications with this entitlement will never pass the App store review

Apple... What's even the point of supporting it? Proprietary closed eco-system vs open-source, permisionless crypto

because sheeple use their phones

the market share is pretty low

I don't see any of iphone users mining RandomX, lol

Android is winning market share btw

this was for a mobile wallet

Wallets can get away with VM interpreter

ios eco system sucks, the days of releasing on ios first are long gone. android is way dominant now

We totally forgot to celebrate one thing: RandomX on Monero is the largest CPU mining operation in history. Bitcoin was too small during CPU days, other CPU coins were much smaller too.

25% market share in Europe, 50% in US

but yea no one will mine on iOS lol

sadly, iphones still have the fastest mobile CPUs

That's going to change too

im surprised the US has such a large % EU needs to pick it up

Cortex A77 is promising

A77 is only really good if you're on 7nm. Apple has been in the lead even without being ahead on process.

2020 Macbooks will use ARM, Apple is ditching x86

haven't actually seen a usable ARM PC yet

yeah haven't really seen it yet

the current ARM SOCs all have next to no on-chip cache, and anemic DRAM support

so should we update monero source tree to use randomX 1.1.7 as well?

probably, but it will need some changes in miner code

because from this version, mining and verification use different API calls

ah. that sounds a bit annoying

if the API has changed like that, perhaps a larger version bump is warranted

it hasn't changed

there are 2 new API functions that can be used to mine faster

ok

but would be a waste not to use it in monerod

if moo finds a bug in -dev we might do a CLI point release anyway

Honestly, adding these 2 new calls to rx_slow_hash will create a big mess there

no, it should be added to the miner directly

miner.cpp?

yes

if it's there only, it should be fine

it's safer, fewer things can break that way

miner.cpp still calls rx_slow_hash, so it looks like bigger refactoring now.

and rx_slow_hash prepares data for mining threads, I don't see how to untangle it properly

in retrospect, it should have been v1.2.0 tag :P

there is no point to update from 1.1.6 then I think

if we don't use the new API