13:48:24 <jakoson0> Why does the Saviour of NASA take a group achievement award and present it as a proof of individual glory? twitter.com/hyc_symas/status/1203709575226183683
18:50:55 <tevador> what is the current size of the data.mdb file on an unpruned node? mine says 139 GB, which seems suspiciously high
19:24:13 <selsta> tevador: mine is 103G
19:25:57 <sech1> 103,736,483,840 bytes
19:46:24 <koork[m]> Then why is the blockchain.raw from getmonero.org only ~85gb?
19:49:54 <sech1> because it's old
19:50:13 <sech1> and data.mdb has a lot of spare space inside
19:50:33 <sech1> it doesn't grow in size with every new block
19:51:14 <tevador> I'm not sure what happened to my node, it seems to keep resizing the database file until I run out of disk space (144 GB partition)
20:27:00 <moneromooo> That's odd. If you can repro on restart, add *lmdb*:DEBUG
20:27:17 <moneromooo> This should logs things about resizing, what size it thinks it has, etc.
20:29:27 <tevador> not sure what I can do now, not enough disk space to restart monerod  148498685952 data.mdb
20:29:33 <tevador> is there a way to shrink it?
20:30:00 <moneromooo> mdb_copy maybe. You'd need a second temporary disk.
20:30:03 <tevador> otherwise I'll have to rm and resync from scratch
20:30:14 <moneromooo> (which could be a USB flash one)
20:31:36 <moneromooo> Actually, restarting once after deleting some random ccache file or /tmp file should be enough. It'll fail at first restart but might still give interesting logs when trying to resize.
20:33:40 <tevador> FYI last messages in the console before I killed it: https://paste.debian.net/hidden/debe874f/
20:34:02 <tevador> so it wanted to make the file even larger
20:37:44 <tevador> after restarting, still wants to resize by 1024 MB
20:39:08 <moneromooo> Good. What DEBUG messages then ?
20:40:24 <tevador> I managed to free 170 MB (lol) and seems to be syncing
20:40:35 <tevador> a lot of spam in the logs
20:42:11 <tevador> nothing interesting the logs besides: E !! WARNING: Insufficient free space to extend database !!: 167 MB available, 1024 MB needed
20:43:09 <tevador> https://paste.debian.net/hidden/30eff086/
20:44:54 <moneromooo> So it looks like it's actually using that allocated space. Really odd then.
20:46:06 <tevador> is my monerod torrenting or what? :D
20:51:47 <moneromooo> One thing you might try:
20:51:56 <moneromooo> mdb_stats -a /path/to/monerod/lmdb
20:52:17 <moneromooo> And compare with someone else who's got a non pruned db, see what DB has got the extra weight.
20:52:23 <moneromooo> Assuming it's just one.
20:52:28 <moneromooo> (I've only got ptuned nodes)
20:52:57 <moneromooo> It won't tell us why, but might give a clue.
20:53:13 <gingeropolous> my data.mdb:  97G Jan 24 15:52 data.mdb
20:53:34 <gingeropolous> 103353671680
20:53:40 <tevador> mdb_env_open failed, error -30794 MDB_VERSION_MISMATCH: Database environment version mismatch
20:53:54 <moneromooo> Build the mdb_stats from external/db_drivers/lmdb
20:53:57 <iDunk> 97 GB here as well.
20:54:13 <moneromooo> (there's a makefile in there IIRC)
20:55:58 <tevador> https://paste.debian.net/hidden/c598ecf0/
20:56:24 <iDunk> https://paste.debian.net/hidden/a166d454/
20:58:04 <moneromooo> Pretty similar...
20:58:55 <moneromooo> Do you have some other program using hte DB ? Like a block explorer, etc ?
20:59:15 <moneromooo> If so, the DB might need to keep track of another state if a long lived txn is live.
20:59:27 <moneromooo> A very very large one though...
21:01:37 <moneromooo> If not, I'm at the end of what I can usefully say about lmdb, hyc will have to take over.
21:03:36 <iDunk> mdb_stat -e  https://paste.debian.net/hidden/79763682/
21:06:04 <tevador> yes, block explorer
21:06:39 <moneromooo> If you kill that block explorer, I have a feeling the "occupancy" will fall back way below 90%.
21:06:48 <tevador> already killed it
21:07:24 <moneromooo> Though I'm not sure what happens if you SIGKILL. How the db knows it can drop the pages...
21:08:35 <tevador> ACID databases must survive SIGKILL
21:08:37 <moneromooo> ie, when a long lived read txn stops, the old state can be dropped, but if a process dies hard... hyc ? :)
21:09:00 <moneromooo> Oh, it'd be consistent, but I'm not sure how the old state can be dropped.
21:09:10 <moneromooo> file locks maybe.
21:09:24 <moneromooo> That gets cleared by the OS on process end.
21:09:40 <tevador> I know sqlite has a vacuum command
21:10:11 <moneromooo> Oh god. I used to have to do that regularly on Firefox to keep it from getting too slow.
21:11:00 * moneromooo afk for a bit
21:12:27 <tevador> if there is no in-place shrink for mdb, I'll have to delete and resync from scratch
21:18:33 <tevador> I guess this explains it pretty well: https://symas.com/understanding-lmdb-database-file-sizes-and-memory-utilization/
21:19:57 <tevador> "We hope that this gives you a better understanding how LMDB is designed and why the size of .mdb files can vary between providers and consumers (even by gigabytes), even when all contain the exact same data as the others. "
21:22:24 <tevador> essentially, my database file contains 40 GB of junk that can't be removed, oh well
21:23:45 <selsta> fwiw gingeropolous also had weird issues on the node where he hosted onion explorer
22:31:35 <hyc> if you use mdb_stat -r it will show any stale readers
22:31:42 <hyc> and mdb_stat -rr will purge them
22:32:04 <hyc> whatever block explorer gingeropolous is running tends to leave stale read txns all over the place
22:32:19 <hyc> that's broken explorer code
22:33:01 <hyc> mdb_stat -eff will show how much of the space is occupied by free pages
22:34:03 <hyc> that's the danger of a DB engine that supports multi-process access - a process accesses it that has buggy code, and screws things up for other users
23:43:49 <tevador> hyc: mdb_stat shows 17 readers from monerod, 4 from the block explorer
23:45:44 <hyc> that doesn't sound problematic
23:45:46 <tevador> mdb_stat -eff returns 10990719 Free pages
23:45:56 <hyc> that's excessive
23:47:00 <hyc> anyway, the only thing to worry about is if there are stale readers hanging around
23:47:08 <tevador> if one page is 32K, that would explain the 40 GB over normal size
23:47:26 <hyc> is that on an M1?
23:47:33 <hyc> everything else only uses 4KB pages
23:47:59 <tevador> it's an amd epyc based VPS
23:48:15 <hyc> I don't think they have a 32KB pagesize
23:48:56 <hyc> but you have almost 11M pages at 4KB each which would get you 40GB
23:50:04 <tevador> yeah, correct, I misread that as 1M
23:50:36 <tevador> so why does monerod insist on allocating additional 1 GB?
23:50:53 <tevador> if there are already 40 GB of free pages
23:50:56 <hyc> it's only comparing current size to estimated size of incoming batches
23:51:04 <hyc> it doesn't know that 40GB of the size is freespace
23:51:20 <tevador> that sounds like a bug
23:51:56 <tevador> is there a way to shrink the file at least a few GB?