Why does full node synchronization become really slow after 60%?

CPU is not really sweating much. But, there's a lot of HDD activity and network activity while my monerod node is not really synchronizing.

* CPU is not really sweating much. But, there are some network activity and a lot of HDD activity while my monerod node is not really synchronizing.

* CPU is not really doing much. But, there are some network activity and a lot of HDD activity while my monerod node is not really synchronizing.

What does this mean?

WARNING net.p2p.tx src/cryptonote_protocol/levin_notify.cpp:680 Unable to send transaction(s) to tor - no suitable outbound connections at height 1433360

an HDD will be much much slower than an SSD

at some point RingCt was introduced which slows down the syncing

also as time goes on there are more txs per unit of time so .......

Why does it put a lot of burden on HDD instead of CPU?

HDD can surely read and write much faster.

I can't properly explain it but it is true. When using an SSD the read and writes will no longer be the bottle neck but the cpu will still not be working very hard

SSD = ~12hrs. HDD will take several days

<tokineko[m] "Why does it put a lot of burden "> HDD's typically have a write speed of around 100-120 megabytes per sec. SATA interface is designed for 600, *and* typically your board is going to have multiple SATA ports.

In short, a single HDD doesn't really push the capacity of a CPU

> <@tokigami.kineko:matrix.org> Why does it put a lot of burden on HDD instead of CPU?

* HDD's typically have a write speed of around 100-120 megabytes per sec. The SATA interface they used is designed for 600, _and_ typically your board is going to have multiple SATA ports.

* In short, a single HDD doesn't really push the capacity of a CPU's specs

Still, I don't understand why monerod pushes HDD so hard for very slow synchronization.

HDD's are simply quite slow for reads and writes. Have you ever tried I don't know a good point of reference, downloading DLC back in the Xbox 360 (which had HDD back then). Or downloading a Steam game on a HDD? The expectation is that HDD's are slow

* HDD's are simply quite slow for reads and writes. Have you ever tried... I don't know a good point of reference, downloading DLC back in the Xbox 360 (which had HDD back then). Or downloading a Steam game on a HDD? The expectation is that HDD's are slow

* HDD's are simply quite slow for reads and writes. Have you ever tried... I don't know a good point of reference, downloading DLC back in the Xbox 360 days (which had HDD back then). Or downloading a Steam game on a HDD? The expectation is that HDD's are slow

* HDD's are simply quite slow for reads and writes. Have you ever tried... I don't know a good point of reference, downloading DLC back in the Xbox 360 days (which had HDD back then). Or booting a computer on HDD vs SSD? The expectation is that HDD's are slow

anybody here use TrezorOne?

as their hardware wallet?

Still, monerod is synchronizing too slow.

* Still, monerod is synchronizing too slowly. HDD may not be the real bottleneck....

HDD should be capable of far more than 20 blocks per minute.

* HDD should be capable of processing far more than 20 blocks per minute.

It's not like HDD is responsible for verifying blocks.

<tokineko[m] "It's not like HDD is responsible"> It is responsible of storing them

HDD can write far more blocks than 20 blocks per minute.

pigeons: could you please send me a pm?

<tokineko[m] "HDD can write far more blocks th"> Hey tokineko -- the issue here is not the amount if data, its the amount of random reads and writes (also known as IOPS) that are necessary for verifying block chain data. This is not exclusive to Monero, but Monero transactions are slightly more complex than, say, Bitcoin, and so take more of those random read and wrote operations.

Remember that verifying a block or transaction can require pulling data from any previous transaction or block from genesis, so your sync requires your computer to find and validate that data multiple times per transaction you verify, and multiple transactions per block you verify.

Verifying 20 blocks could mean verifying 2000 transactions across the entire history of Monero, which each have 11 ring members to verify among other data.

Each of those verification steps is extremely costly on a hard drive which had to physically spin and seek across the platters for each of those ops, and could have to go to completely different physical locations on the disk for transactions in the same block.

On an SSD your computer merely requests the locations from the SSD controller and it quickly and easily returns the data with only electrical (not physical) movement.

Storing the blocks and transactions is trivial, which is why the portion before RingCT (and then before the latest checkpoint) is vastly quicker.

Verification became much more expensive when RingCT was introduced, but has gotten 90% more efficient over time since that point.

Verification is the portion that is very, very hard on traditional hard drives and older CPUs.

Hopefully that helps clarify some of what you're experiencing 🙂

I wonder if triptych 100+ ringsizes will cause the read/write bottleneck to become unscalable over time

Maybe something like ringsize ~40 is better for this reason, because increasing ringsize gives diminishing returns on plausible deniability anyway

Multi-terabyte SSD is still expensive. That's why I dedicate a 6TB HDD to monero blockchain.

But the Monero block chain isn't even 100GB

it is 100GB already

well, it depends on how you count "GB"

97.5 proper GBs and 104 normie GBs

In decades, it may need more than 6 terabytes.

Do you expect to use another coin in decades? Maybe.

In decades, your HDD will probably be broken

In decades we'll have petabyte drives

In decades, we'll be using Monero 3.0 with quantum resistant sharting

Monero 3.0?

I doubt that a petabyte drive is going to exist or be affordable in a few decades.

What is `sharting`?

<cankerwort[m] "I wonder if triptych 100+ ringsi"> The big improvement with Triptych/Arcturus is that ring sizes scale *logarithmically* (no idea how to spell that lol) instead of *linearly*. So ~128 ring size is around the same verification time and size as current ring size 11.

It's a bit more complex than that and depends on the transaction itself, but that's a reasonable comparison.

<cankerwort[m]> Maybe something like ringsize ~40 is better for this reason, because increasing ringsize gives diminishing returns on plausible deniability anyway

I'd keep the verification performance similar and go off of that

So increase to a ring size that has similar verification performance as the current 'scheme'

Yeah, which IIRC is ~128

It seems my btrfs mount options are not optimized for monero.

reddit.com/r/Monero/comments/gbchd8/full_node_on_btrfs_vs_ext4 says ext4 is faster than btrfs for synchronizing.

copy on write may decrease write performance of monero blockchain on HDD?

Wouldn't it decrease all write performance? You're asking the disks to perform extra operations for each real actual operation.

So you're taking an already slow medium (HDD) and slowing it down even further. Just get a 1TB SSD and call it a day for a decade or two

I guess I may settle with ext4.

According to reddit.com/r/Monero/comments/gbchd8/full_node_on_btrfs_vs_ext4 , it is 15 times as fast to synchronize on ext4 than on btrfs.

Is it possible to write lmdb on raw disk?

Lets take this to #monero:matrix.org

Does anyone know file system and mount options best for blockchain synchronization on HDD?

'Oddly, no one who was directly involved with the SIR-C missions and currently still working the Lab, remembers the name Howard Chu, except for one who vaguely recalls Eugene Chu as having a brother names Howard'

Ed Caro, NASA Chief Engineer. cryptogazette.com/wp-content/uploads/2019/12/1-768x1445.png

This channel is now relayed to the Monero Social Matrix server