System migration process to another server

1. Preparing the DB server for migration

1.1. Preparing the postgresql DB server in a container for backup.

  • Open a terminal.

  • Find the active container with postgresql:

docker ps -a

Let it be 'postgres_myinstance'.

  • Connect to the postgres container with the discovered name:

docker exec -it postgres_myinstance bash
  • Copy the file (just in case)

cp /var/lib/postgresql/data/pg_hba.conf /var/lib/postgresql/data/pg_hba.conf.bkp
  • Modify pg_hba.conf, adding at the end the possibility to connect for replication from the address

echo "host replication all 172.27.0.0/16 trust" >> /var/lib/postgresql/data/pg_hba.conf.

The address is the one from which the new server will connect to the previous server.
In the example, connection without authorization from subnet 192.168.0.0/24 is allowed for replication needs:

host replication all 172.27.0.0/16 trust
  • exit to the host and restart the postgres container:

docker restart postgre_local

After the whole procedure the old DB should be protected from external connections. Shut it down entirely or eliminate the possibility of connecting to it without authorization.

2. Creating a DB server backup.

The address is the local address in the network allowed above for replication, the port is 5432 by default, set in /var/lib/postgresql/14/era${instance}/postgresql.conf. (make sure listen_addresses is commented out or includes the network interface of interest, port is commented out or set to 5432; otherwise use the value specified there).

/usr/lib/postgresql/14/bin/pg_basebackup -P -R \
  -X stream \
  -c fast -h 172.27.1.101 -p 5432 \
  -U platformpgadmin \
  -D /var/lib/postgresql/14/era${instance}

This directory must ultimately end up on the server with the future new DB server. It can replace the existing one or become a new directory and a new instance. In the latter case the daemon must be configured to start the new instance automatically.

3. Obtain the required values from the old version

3.1. Download the license from the previous version

  • Open a terminal.

  • In the volumes folder of the current system (chosen during installation, by default /opt/era${instance}/lib/_workdir/) take the r.lic file, for example copying it straight to the new server with rsync -avrh --progress ./r.lic user@destination:/tmp

3.2. Download the configuration from the previous version

  • Log in to the master domain as admin, open the configuration section, open the active configuration, and copy its content to a notepad.

3.3. Download or prepare to download the syncroot directory (/opt/era${instance}/syncroot).

3.4. Download or prepare to download the era_recpath directory (/opt/era${instance}/era_recpath).

3.5. Prepare the Incoplax installer launch string containing all options of the previous installation. (The installation string should be saved in the project file along with other system properties useful for recovering project information).

4. Installing Incoplax on the new server.

  • In the general case installation is performed on a different local address. In the special case the address may be identical, and then no configuration changes are required.

  • It is advisable, but not mandatory, to preserve as many basic installation parameters as possible so that as few additional edits as possible are needed. Server name, folder locations, instance name, passwords, cookies, options. It is better to take the previous installer launch string with all assigned parameters and change exactly what is necessary, i.e., the local address.

If directories change location, the corresponding changes must be taken into account when applying the example scripts.

The following considers the option of installing the platform with a new empty DB and subsequent reconnection to the restored DB.
The environment variable ${BACKUP_DATA_FOLDER}, used in the following examples, contains the path to the directory where the DB was backed up.

4.1. Postgres on the new server in a container.

After the system is installed, a container with postgres appears on the server (postgres_myinstance, since we preserve as many original values as possible).

You can connect to the container from the terminal:

docker exec -it postgres_myinstance bash

The configs and data are in the /var/lib/postgresql/14/data folder.
This directory inside the container is treated as a container volume and is mounted into a folder on the host.

4.1.1. Determine the location of the instance data directory on the host:

docker inspect postgres_myinstance

In the output find the directory with configs and data ("Source"):

...
    "Mounts": [
        {
            "Type": "volume",
            "Name": "pg_data_vol",
            "Source": "/var/lib/docker/volumes/pg_data_vol/_data",
            "Destination": "/var/lib/postgresql/data",
            "Driver": "local",
            "Mode": "z",
            "RW": true,
            "Propagation": ""
        }
    ],
...

4.1.2. Rename the directory to *.bkp (just in case, for recovery):

sudo mv /var/lib/docker/volumes/pg_data_vol/_data /var/lib/docker/volumes/pg_data_vol/_data.bkp

4.1.3. Remove the recovery binding to the master from the directory:

sudo rm ${BACKUP_DATA_FOLDER}/recovery.conf

4.1.4. Set the owner group for the directory (docker)

sudo chown root:docker ${BACKUP_DATA_FOLDER}

4.1.5. Move the directory into the volume position

sudo mv ${BACKUP_DATA_FOLDER} /var/lib/docker/volumes/pg_data_vol/_data

4.1.6. Restart the container

docker restart postgres_myinstance

4.1.7. Check that the container works correctly.
This can be done through pgadmin, also installed in the container.
Or in any other way, for example via psql inside the container:

docker exec -it postgres_myinstance bash

su postgres

psql -p 5432 -c "Select pg_is_in_recovery()"

psql -p 5432 -d r_domain_master -c "Select * FROM \"user\".users"

4.2. Postgres on the new server on the host.

After the platform is installed, the /var/lib/postgresql/14 folder with instance subfolders appears on the server.

4.2.1. We copy the directory with the backup here (or immediately make the backup from stage 2 into a new subdirectory here. The target subdirectory must not exist when the backup runs).

4.2.2. Make edits to the config directory files:

  • edit pg_hba.conf, removing unnecessary connections, in particular the trust one made for replication earlier.

  • edit postgresql.conf, changing the port to avoid conflicts with the default instance (uncomment the port option and assign, for example, 5442).

  • delete the recovery.conf file if present (to unbind from the master server):

rm recovery.conf
  • rename the current config, saving it just in case:

mv postgresql.auto.conf postgresql.auto.conf.bkp

4.2.3. Start the new instance:

/usr/lib/postgresql/14/bin/pg_ctl -D /var/lib/postgresql/14/era_myinstance start

4.2.4. Try to execute an SQL query against the instance:

psql -p 5442 -c "Select pg_is_in_recovery();"

It should return f. (If not — the instance started in replica mode, and it must be switched to master with a signal — the master.signal file in the config directory — configured in postgresql.conf).

4.2.5. Create an autostart service for the instance when Linux starts.

sudo bash -c "echo '[Unit]
Description=PostgreSQL instance era_01 service
After=network.target

[Service]
Type=forking

User=postgres
Group=postgres

OOMScoreAdjust=-1000

Environment=PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj
Environment=PG_OOM_ADJUST_VALUE=0

Environment=PGSTARTTIMEOUT=270

Environment=PGDATA=/var/lib/postgresql/14/era_myinstance

ExecStart=/usr/lib/postgresql/14/bin/pg_ctl start -D ${PGDATA} -s -w -t ${PGSTARTTIMEOUT}
ExecStop=/usr/lib/postgresql/14/bin/pg_ctl stop -D ${PGDATA} -s -m fast
ExecReload=/usr/lib/postgresql/14/bin/pg_ctl reload -D ${PGDATA} -s

TimeoutSec=300

[Install]
WantedBy=multi-user.target' >> /etc/systemd/system/postgresql_12_era_myinstance.service"

Enable the service and start the daemon:

sudo -S systemctl enable "postgresql_12_era_myinstance.service"

sudo -S systemctl daemon-reload

4.2.5. Reboot the machine to make sure the daemon works and the instance starts automatically sudo reboot

4.2.6. Make sure the instance is running

su postgres

psql -p 5442 -c "Select 1;"
Similarly to item 4.1, instead of creating a new instance you can replace the directory of an existing instance.

4.3. Installation option with connection to a new restored DB.

  • This option assumes preliminary installation of the DB server — either manually or with the platform installer up to the DB installation stage followed by interrupting the process.

  • Then the main instance (the directory with configs and data) is replaced, or a new instance is added and accordingly a service for autostart at machine boot is manually started and configured.

  • During subsequent platform installation the existing DB is used.

5. Configuring the new platform.

Connect to the new platform instance via the web interface. By default it is

Uploading the license

The license file downloaded in stage 3.1 can be uploaded:

  • either under the master domain administrator account directly in the Settings application.

  • or by placing/copying the r.lic file from the old system /opt/era_instance/lib/mdc1@172.27.1.101/lic/r.lic to the new one in the same place.

Copying the syncroot directory

The syncroot directory (item 3.3) can be copied directly from the previous server or from intermediate storage:

rsync -avrh --progress era@172.27.1.101:/opt/era_myinstance/syncroot /opt/era_myinstance/syncroot

Copying the era_recpath directory

The era_recpath directory (item 3.4) can be copied directly from the previous server or from intermediate storage:

rsync -avrh --progress era@172.27.1.101:/opt/era_myinstance/era_recpath /opt/era_myinstance/era_recpath

Connecting the new DB

The new DB is connected by modifying the configuration.

Configuration

At the new system installation stage two configuration files were produced:

  • from the new installation;

  • from the previous system.

The task of this stage is to merge them. Overall it should be the previous system configuration, but the following must be corrected in it:

  • The connection string to the master domain postgresql DB;

  • Server addresses;

  • Ports (only if changed relative to the previous installation);

  • Server name (only if changed relative to the previous installation);

  • Paths to directories inside the container (only if changed relative to the previous installation).

Custom complex configuration settings may also contain special values requiring correction when creating the merged configuration.
Use search through the file to find all mentions of the previous server address values, as well as other values deliberately changed during the new installation.

After saving the result as the new configuration, it will be validated, and on success displayed with the "Correct" status.
At that moment it can be activated.

System restart, state check

Check the system state (Master domain, Settings → System → State).
Wait until the system reaches the correct state without errors and warnings.

Reboot the server:

sudo reboot

Make sure that after the server reboot the platform loads as planned and records no errors, i.e.:

  • log in to the master domain,

  • poll the state,

  • make sure the state contains no problem notifications,

  • make sure the working domains are in place,

  • make sure communication scripts open and nested static audio files are accessible,

Installing the product layer

This is done by the usual procedure.
Then custom packages used in the system are uploaded and activated in the builder.