How to reduce the time to sync the Ethereum Geth node from a few weeks to a few hours?

Problem definition

To create your own Ethereum full node you need to have a powerfull server with a fast storage, fast internet connection and enough time to synchronize node with the network. During the synchronization process, your server will read and write data on the storage many-many times more than the storage size, because of other users that are in a sync state as well.

Here is some benchmark results, regarding to the post from official Ethereum blog:

Full sync benchmark on two i3.2xlarge AWS EC2 instances ($0.624 per Hour, 8 cores, 61 GiB RAM, ephemeral volume 1.9 TiB NVMe SSD, if you stop the instance — you will lose all data)

with --cache=4096 --maxpeers=50 --syncmode=full

Version: Geth v1.9.0

Sync time: 6d 8h 7m

Disk size: 303GiB

Disk reads: 40.2TiB

Disk writes: 32.6TiB

Personal experience:
AWS:
using c5.xlarge instance with EBS io1 volume (Provisioned IOPS) you can spend more, than 5 weeks time and about $1300, AWS calculation is here
On premise: using Raspberry PI 4B 8GB, SSD 2TB storage attached through USB 3.0, you can spend more than 3 months.

Solution

There are few possible solutions:

  1. “Take a super fast server and to hope that the synchronization process goes quick” — Nope… It does not work in this way, because other nodes are not so fast, you still need a lot of time to sync
  2. “Download somewhere exported data following Blockchain Import/Export features” — Nope… It will take almost the same time as just syncronization over the Internet
  3. “Download chaindata folder and continue to sync the node with a pre-existing database” — Yes, it will work.

Ok, how big is geth chaindata folder now?

du -sh chaindata
1.1T chaindata

It is about 1.1TB (December 2020) of data and continues to grow.

Download chaindata

Regarding to the manual on chaindata.club,
we need to download full (1.1 TB) chaindata folder:

lftp -e “mirror ./ ./” http://api.chaindata.club

and run a new geth node with attached folder:

mkdir -p full-ethereum-node/gethmv chaindata full-ethereum-node/geth/docker run -d --rm --name full-node-ethereum --ulimit nofile=10000:10000 -p 8545:8545 -p 30303:30303 -v $PWD/full-ethereum-node:/root/.ethereum ethereum/client-go --http --http.addr "0.0.0.0" -—nousb --ipcdisable --syncmode full

Tips

Use this manual to add other nodes manually and start to sync with the network immediately.

Synchronisation process

Conclusion

This solution provides an opportunity to run the geth node faster and cheaper than synchronize it from genesis block by yourself.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store