Gateway HUB - How to recreate Postgress database in one of the nodes

The steps to follow in order to restore data on the faulty node Copied

1. Stop the Hub on all nodes:

   • If systemd is managing the Hub:

   sudo systemctl mask hub-orchestration
   sudo systemctl stop hub-orchestration
   

   • or run /opt/hub/current/bin/hub.sh stop on each node

2. Ensure that the Hub is stopped on each node. First run /opt/hub/latest/bin/hub.sh status then check that Postgres is effectively stopped: ps -edf|grep postgres:

3. On one healthy node, make a tarball of /opt/hub/postgres and a tarball of /opt/hub/pgwal and copy across to node

4. On the faulty node, rename /opt/hub/postgres to /opt/hub/postgres.<timestamp> and /opt/hub/pgwal to /opt/hub/pgwal.<timestamp>.

5. On the faulty node, unzip the tarballs copied from healthy node so that you end up with /opt/hub/postgres and /opt/hub/pgwal

6. On the faulty node, start Postgres manually by running /opt/hub/latest/services/postgres-timescale-12.6-1.7.4/start.sh and wait until you see the following in the logs:

   LOG:  database system is ready to accept connections
   LOG:  TimescaleDB background worker launcher connected to shared catalogs
   

7. On the faulty node, stop Postgres using a Ctrl-C

8. Restart the Hub normally using hubctl or systemd.