How to reduce the time to sync the Ethereum Geth node from a few weeks to a few hours?
Or “How to download geth chaindata folder?”
To create your own Ethereum full node you need to have a powerfull server with a fast storage, fast internet connection and enough time to synchronize node with the network. During the synchronization process, your server will read and write data on the storage many-many times more than the storage size, because of other users that are in a sync state as well.
Full sync benchmark on two i3.2xlarge AWS EC2 instances ($0.624 per Hour, 8 cores, 61 GiB RAM, ephemeral volume 1.9 TiB NVMe SSD, if you stop the instance — you will lose all data)
--cache=4096 --maxpeers=50 --syncmode=full
Version: Geth v1.9.0
Sync time: 6d 8h 7m
Disk size: 303GiB
Disk reads: 40.2TiB
Disk writes: 32.6TiB
AWS: using c5.xlarge instance with EBS io1 volume (Provisioned IOPS) you can spend more, than 5 weeks time and about $1300, AWS calculation is here
On premise: using Raspberry PI 4B 8GB, SSD 2TB storage attached through USB 3.0, you can spend more than 3 months.
There are few possible solutions:
- “Take a super fast server and to hope that the synchronization process goes quick” — Nope… It does not work in this way, because other nodes are not so fast, you still need a lot of time to sync
- “Download somewhere exported data following Blockchain Import/Export features” — Nope… It will take almost the same time as just syncronization over the Internet
- “Download chaindata folder and continue to sync the node with a pre-existing database” — Yes, it will work.
Ok, how big is geth chaindata folder now?
du -sh chaindata
It is about 1.1TB (December 2020) of data and continues to grow.
Regarding to the manual on chaindata.club,
we need to download full (1.1 TB) chaindata folder:
lftp -e “mirror ./ ./” http://api.chaindata.club
and run a new geth node with attached folder:
mkdir -p full-ethereum-node/gethmv chaindata full-ethereum-node/geth/docker run -d --rm --name full-node-ethereum --ulimit nofile=10000:10000 -p 8545:8545 -p 30303:30303 -v $PWD/full-ethereum-node:/root/.ethereum ethereum/client-go --http --http.addr "0.0.0.0" -—nousb --ipcdisable --syncmode full
Use this manual to add other nodes manually and start to sync with the network immediately.
This solution provides an opportunity to run the geth node faster and cheaper than synchronize it from genesis block by yourself.