WarehousePG

Appearance

Installing WarehousePG Backup and Restore

whpg-backup provides the gpbackup and gprestore utilities. Build them from source, then install the binaries on every host in your WarehousePG cluster.

Prerequisites ​

  • Build host: Go 1.23 or later, plus sqlite3. Building directly on the WarehousePG coordinator avoids a cross-compile step, since it's already running the same OS as your segment hosts. If you build on a macOS workstation instead, use make build_linux to cross-compile for the cluster.
  • Every host in the cluster: sqlite3. gpbackup splits work between the coordinator and every segment host, so the sqlite3 runtime dependency needs to be present wherever gpbackup, gprestore, or gpbackup_helper runs, not just on the build host.

Building from source ​

  1. Clone the repository and change into it:

    bash
    git clone https://github.com/warehouse-pg/whpg-backup.git
cd whpg-backup

  2. Build the gpbackup, gprestore, and gpbackup_helper binaries:

    bash
    make depend
make build

    The binaries are placed in $HOME/go/bin. If you're cross-compiling from a macOS workstation, run make build_linux instead.

TIP

Add $HOME/go/bin to your PATH so you can run the binaries directly from the build host:

bash
export PATH=$PATH:$HOME/go/bin

Installing on the cluster ​

  1. Confirm sqlite3 is installed on every host in the cluster, including the coordinator, standby coordinator, and all segment hosts. It's installed by default on many platforms, but install it if it's missing.

  2. On the coordinator, create a file all_hosts which lists all hosts in the WHPG cluster. For example:

    ini
    cdw
scdw
sdw1
sdw2
sdw3

  3. Transfer the binaries to all hosts in the cluster and place them on $PATH, for example in $GPHOME/bin. Use the gpsync utility on WarehousePG 7, or the gpscp utility on WarehousePG 6:

    WHPG 7 ​

    bash
    gpsync -f all_hosts $HOME/go/bin/gpbackup $HOME/go/bin/gprestore $HOME/go/bin/gpbackup_helper =:$GPHOME/bin

    WHPG 6 ​

    bash
    gpscp -f all_hosts $HOME/go/bin/gpbackup $HOME/go/bin/gprestore $HOME/go/bin/gpbackup_helper =:$GPHOME/bin

  4. Verify the installation by checking the gpbackup and gprestore versions:

    bash
    gpbackup --version
gprestore --version

Upgrading ​

To upgrade to a new release, rebuild gpbackup, gprestore, and gpbackup_helper from the release tag you want, then redistribute them to every host the same way as a fresh install.

Upgrading to 1.34.1

Review the following before you upgrade to 1.34.1 from an earlier version:

  • On WarehousePG 7.4 and later, backups taken with 1.34.0 or earlier contain no data for coordinator-only tables, including incremental backups. After upgrading, take a new full backup of any database that has them.
  • If you use a custom storage plugin and have coordinator-only tables, make sure the plugin is installed on the coordinator and accepts backup_data and restore_data calls with content -1. Backing up or restoring a coordinator-only table now calls the plugin on the coordinator, once per table. The S3 storage plugin is not affected.
  • Scripts that pass an empty --plugin-config, for example --plugin-config "$CFG" with CFG unset, used to run without a plugin and now fail.
  • Backups taken with an empty --plugin-config on earlier versions are on the segment hosts' local disks, not on plugin storage. Restore them with gprestore without --plugin-config, or delete them to free the disk space.

  1. In your existing clone, fetch the latest tags and check out the release you want to upgrade to. Replace <tag> with the release tag, for example 1.34.1-WHPG:

    bash
    git fetch --tags
git checkout <tag>

  2. Rebuild the binaries:

    bash
    make depend
make build

  3. Transfer the rebuilt binaries to every host in the cluster, overwriting the existing ones. Use the same gpsync or gpscp command as in Installing on the cluster.

  4. Verify the upgrade by checking the version on the coordinator:

    bash
    gpbackup --version
gprestore --version

Note

gpbackup and gprestore are command-line utilities, not a running service, so there's nothing to stop or restart. Avoid upgrading while a backup or restore is in progress.