Skip to main content

Running an Archive Node

Overview​

System Administrator Required

For running nodes basic knowledge of Linux/Ubuntu system administration is required.

An Archive Node is a type of Full Node that stores extended historical data of a blockchain. If you are creating a blockchain explorer or a similar application that requires access to historical data, using an Archive Node as an indexer is recommended.

Prerequisites​

We highly recommend install mytonctrl using the supported operating systems:

  • Ubuntu 20.04
  • Ubuntu 22.04
  • Debian 11

Please, use a non-root user with sudo privileges to install and run mytonctrl.

Hardware requirements​

  • 16 x Cores CPU
  • 128GB ECC Memory
  • 4TB SSD OR Provisioned 32+k IOPS storage
  • 1 Gbit/s network connectivity
  • 16 TB/month traffic on peak load
  • a public IP address (fixed IP address)

Note: 4TB assumes usage of zfs volume with compression enabled

Installation​

In general, you need the following steps to run an Archive Node:

  1. Install ZFS and Prepare Volume
  2. Install MyTonCtrl
  3. Run a Full Node on your server and stop validator process
  4. Download and restore dump data from https://archival-dump.ton.org
  5. Run Full Node with Configuring DB specs for Archive Node

Install ZFS and Prepare Volume​

Dumps come in form of ZFS Snapshots compressed using plzip, you need to install zfs on your host and restore the dump, see Oracle Documentation for more details.

Usually, it's a good idea to create a separate ZFS pool for your node on a dedicated SSD drive, this will allow you to easily manage storage space and backup your node.

  1. Install zfs
sudo apt install zfsutils-linux
  1. Create pool on your dedicated 4TB <disk> and name it data
sudo zpool create data <disk>
  1. Before restoring we highly recommend to enable compression on parent ZFS filesystem, this will save you a lot of space. To enable compression for the data volume enter using root account:
sudo zfs set compression=lz4 data

Install MyTonCtrl​

Please, use a Running Full Node to install mytonctrl.

Run an Archive Node​

Prepare the node​

  1. Before performing a restore, you must stop the validator using root account:
sudo -s
systemctl stop validator.service
  1. Make a backup of ton-work config files (we will need the /var/ton-work/db/config.json, /var/ton-work/keys, and /var/ton-work/db/keyring).
mv /var/ton-work /var/ton-work.bak

Download the dump​

  1. Request user and password credentials to gain access for downloading dumps in the @TONBaseChatEn Telegram chat.
  2. Here is an example command to download & restore the dump from the ton.org server:
wget --user <usr> --password <pwd> -c https://archival-dump.ton.org/dumps/latest.zfs.lz | pv | plzip -d -n <cores> | zfs recv data/ton-work

Size of the dump is ~1.5TB, so it will take some time to download and restore it.

Prepare and run the command:

  1. Install the tools if necessary (pv, plzip)
  2. Replace <usr> and <pwd> with your credentials
  3. Tell plzip to use as many cores as your machine allows to speed up extraction (-n)

Mount the dump​

  1. Mount zfs:
zfs set mountpoint=/var/ton-work data/ton-work && zfs mount data/ton-work
  1. Restore db/config.json, keys and db/keyring from backup to /var/ton-work
cp /var/ton-work.bak/db/config.json /var/ton-work/db/config.json
cp -r /var/ton-work.bak/keys /var/ton-work/keys
cp -r /var/ton-work.bak/db/keyring /var/ton-work/db/keyring
  1. Make sure that permissions for /var/ton-work and /var/ton-work/keys dirs promoted correctly:
  • The owner for the /var/ton-work/db dir should be validator user:
chown -R validator:validator /var/ton-work/db
  • The owner for the /var/ton-work/keys dir should be ubuntu user:
chown -R ubuntu:ubuntu /var/ton-work/keys

Update Configuration​

Update node configuration for the archive node.

  1. Open the node config file /etc/systemd/system/validator.service
nano /etc/systemd/system/validator.service
  1. Add storage settings for the node in the ExecStart line:
--state-ttl 315360000 --archive-ttl 315360000 --block-ttl 315360000
info

Please be patient once you start the node and observe the logs. Dumps come without DHT caches, so it will take your node some time to find other nodes and then sync with them. Depending on the age of the snapshot, your node might take from a few hours to several days to catch up with the network. This is normal.

Start the node​

  1. Start the validator by running the command: 
systemctl start validator.service
  1. Open mytonctrl from local user and check the node status using the status.

Node maintenance​

Node database requires cleansing from time to time (we advise once a week), to do so please perform following steps as root:

  1. Stop validator process (Never skip this!)
sudo -s
systemctl stop validator.service
  1. Remove old logs
find /var/ton-work -name 'LOG.old*' -exec rm {} +
  1. Remove temp files
rm -r /var/ton-work/db/files/packages/temp.archive.*
  1. Start validator process
systemctl start validator.service

Troubleshooting and backups​

If for some reason something does not work / breaks you can always roll back to @archstate snapshot on your ZFS filesystem, this is the original state from dump.

  1. Stop validator process (Never skip this!)
sudo -s
systemctl stop validator.service
  1. Check the snapshot name
zfs list -t snapshot
  1. Rollback to the snapshot
zfs rollback data/ton-work@dumpstate

If your Node works well then you can remove this snapshot to save storage space, but we do recommend to regularly snapshot your filesystem for rollback purposes because validator node has been known to corrupt data as well as config.json in some cases. zfsnap is a nice tool to automate snapshot rotation.

Need help?

Have question or need help? Please ask in the TON dev chat to get help from the community. MyTonCtrl developers also hang out there.

See Also​