Why does the Saviour of NASA take a group achievement award and present it as a proof of individual glory? twitter.com/hyc_symas/status/1203709575226183683

what is the current size of the data.mdb file on an unpruned node? mine says 139 GB, which seems suspiciously high

tevador: mine is 103G

103,736,483,840 bytes

Then why is the blockchain.raw from getmonero.org only ~85gb?

because it's old

and data.mdb has a lot of spare space inside

it doesn't grow in size with every new block

I'm not sure what happened to my node, it seems to keep resizing the database file until I run out of disk space (144 GB partition)

That's odd. If you can repro on restart, add *lmdb*:DEBUG

This should logs things about resizing, what size it thinks it has, etc.

not sure what I can do now, not enough disk space to restart monerod 148498685952 data.mdb

is there a way to shrink it?

mdb_copy maybe. You'd need a second temporary disk.

otherwise I'll have to rm and resync from scratch

(which could be a USB flash one)

Actually, restarting once after deleting some random ccache file or /tmp file should be enough. It'll fail at first restart but might still give interesting logs when trying to resize.

FYI last messages in the console before I killed it: paste.debian.net/hidden/debe874f

so it wanted to make the file even larger

after restarting, still wants to resize by 1024 MB

Good. What DEBUG messages then ?

I managed to free 170 MB (lol) and seems to be syncing

a lot of spam in the logs

nothing interesting the logs besides: E !! WARNING: Insufficient free space to extend database !!: 167 MB available, 1024 MB needed

So it looks like it's actually using that allocated space. Really odd then.

is my monerod torrenting or what? :D

One thing you might try:

mdb_stats -a /path/to/monerod/lmdb

And compare with someone else who's got a non pruned db, see what DB has got the extra weight.

Assuming it's just one.

(I've only got ptuned nodes)

It won't tell us why, but might give a clue.

my data.mdb: 97G Jan 24 15:52 data.mdb

103353671680

mdb_env_open failed, error -30794 MDB_VERSION_MISMATCH: Database environment version mismatch

Build the mdb_stats from external/db_drivers/lmdb

97 GB here as well.

(there's a makefile in there IIRC)

Pretty similar...

Do you have some other program using hte DB ? Like a block explorer, etc ?

If so, the DB might need to keep track of another state if a long lived txn is live.

A very very large one though...

If not, I'm at the end of what I can usefully say about lmdb, hyc will have to take over.

mdb_stat -e paste.debian.net/hidden/79763682

yes, block explorer

If you kill that block explorer, I have a feeling the "occupancy" will fall back way below 90%.

already killed it

Though I'm not sure what happens if you SIGKILL. How the db knows it can drop the pages...

ACID databases must survive SIGKILL

ie, when a long lived read txn stops, the old state can be dropped, but if a process dies hard... hyc ? :)

Oh, it'd be consistent, but I'm not sure how the old state can be dropped.

file locks maybe.

That gets cleared by the OS on process end.

I know sqlite has a vacuum command

Oh god. I used to have to do that regularly on Firefox to keep it from getting too slow.

if there is no in-place shrink for mdb, I'll have to delete and resync from scratch

I guess this explains it pretty well: symas.com/understanding-lmdb-databa…e-file-sizes-and-memory-utilization

"We hope that this gives you a better understanding how LMDB is designed and why the size of .mdb files can vary between providers and consumers (even by gigabytes), even when all contain the exact same data as the others. "

essentially, my database file contains 40 GB of junk that can't be removed, oh well

fwiw gingeropolous also had weird issues on the node where he hosted onion explorer

if you use mdb_stat -r it will show any stale readers

and mdb_stat -rr will purge them

whatever block explorer gingeropolous is running tends to leave stale read txns all over the place

that's broken explorer code

mdb_stat -eff will show how much of the space is occupied by free pages

that's the danger of a DB engine that supports multi-process access - a process accesses it that has buggy code, and screws things up for other users

hyc: mdb_stat shows 17 readers from monerod, 4 from the block explorer

that doesn't sound problematic

mdb_stat -eff returns 10990719 Free pages

that's excessive

anyway, the only thing to worry about is if there are stale readers hanging around

if one page is 32K, that would explain the 40 GB over normal size

is that on an M1?

everything else only uses 4KB pages

it's an amd epyc based VPS

I don't think they have a 32KB pagesize

but you have almost 11M pages at 4KB each which would get you 40GB

yeah, correct, I misread that as 1M

so why does monerod insist on allocating additional 1 GB?

if there are already 40 GB of free pages

it's only comparing current size to estimated size of incoming batches

it doesn't know that 40GB of the size is freespace

that sounds like a bug

is there a way to shrink the file at least a few GB?