Backing Up/Restoring Kinetica

KAgent can be used to simplify the processes of backing up and restoring the Kinetica database. It is distributed separately from the database and can be installed and used to configure a Kinetica cluster following the instructions under Kinetica Installation with KAgent.

There are two interfaces to KAgent for backing up a Kinetica cluster:

Graphical User Interface
Command-Line Interface (described below)

Prerequisites

KAgent backup management has two requirements:

KAgent installed, with access to the cluster being managed
A properly configured Kinetica cluster

Note

Kinetica does not need to be offline to be backed up or restored. The backup process will put Kinetica into read-only mode, however, which will block operations requiring disk write access (table creation/modification, persist-backed ingestion, etc.).

Backing Up

All the data in Kinetica can be backed up in either an ad-hoc or scheduled fashion, using the command line. To learn about how to back up Kinetica using the KAgent GUI, consult Backups.

Backups are stored local to each node in the cluster. Those local backup file target directories can be mounted via NFS or similar external shared storage to consolidate those files to a single device. If multiple clusters are backed up to the same shared storage under the same backup directory, those backups will be able to be restored to any of the clusters in the group; e.g., if Cluster A and Cluster B are both backed up to the same shared location, the backups of Cluster A can be restored to Cluster B and vice versa.

The base command for creating a backup:

/opt/gpudb/kagent/bin/kagent cluster backup [--schedule <schedule>] [--backup-path <backup path>] <cluster name>

Tip

To list the backup schedule for the cluster:

/opt/gpudb/kagent/bin/kagent cluster backup --list-schedule ALL <cluster name>

Schedule

There are three options for schedule:

now -- Runs a single backup right now without creating or modifying the schedule; (default behavior)
A crontab schedule quoted string -- Will overwrite any existing backup schedule with the one specified.

For example, '0 0 1 1-3 *' will schedule backups at 12:00 AM on the 1st day of each month, January through March.

Consult the crontab documentation for details on schedule specification format.
never -- Clears the current backup schedule for the given cluster name

Backup Path

The backup path should be any valid file path on the Kinetica cluster nodes. If the directory does not exist on one or more nodes, KAgent will create it.

The default backup path is /opt/backups.

Under this backup path directory, KAgent will create a subdirectory with the name of the cluster as the directory name. KAgent will then create a snapshot subdirectory under the cluster-specific subdirectory, named with the date/time at which the backup was initiated, into which all backup files will be placed.

For instance, given the following backup command execution, run at 12:34:56 on January 2nd, 2019:

/opt/gpudb/kagent/bin/kagent cluster backup --backup-path /opt/backup mycluster

Backup files will be placed under this location, on each node:

/opt/backup/mycluster/snapshot.2019-01-02.12-34-56

Examples

To list backups scheduled for the mycluster cluster:

/opt/gpudb/kagent/bin/kagent cluster backup --list-schedule ALL mycluster

To create an immediate backup of the mycluster cluster in the (default) /opt/backups directory:

/opt/gpudb/kagent/bin/kagent cluster backup mycluster

To create a backup in /opt/backup for the cluster named mainkincluster scheduled for 22:00 on day 1 through 5 of every week:

/opt/gpudb/kagent/bin/kagent cluster backup --schedule '0 22 * * 1-5' --backup-path /opt/backup mainkincluster

To remove the backup scheduled for the mycluster cluster:

/opt/gpudb/kagent/bin/kagent cluster backup --schedule never mycluster

Listing

The backups available to be restored to a given cluster can be listed via command line or the KAgent GUI. See Snapshots for details on how to display a list of backups in the GUI.

The base command for listing backups available to a given cluster:

/opt/gpudb/kagent/bin/kagent cluster list-backups [--backup-path <backup path>] <cluster name>

Backup Path

The backup path should be the file path on each Kinetica cluster node that contains the backups to list.

The default backup path is /opt/backups.

KAgent will look on each cluster node for the directory named in the --backup-path parameter and list the contents of that directory.

For instance, given the following restore command execution:

/opt/gpudb/kagent/bin/kagent cluster list-backups --backup-path /opt/backup mycluster

A backup snapshot directory will be looked for under /opt/backup, on each node of the cluster mycluster.

Examples

To list all backups available to the cluster mycluster:

/opt/gpudb/kagent/bin/kagent cluster list-backups mycluster

This might show output like the following, for cluster snapshots under the default /opt/backups directory:

/mycluster/snapshot.2019-01-02.01-23-45
/mycluster/snapshot.2019-01-02.12-34-56

To list all backups available to the cluster clusterA, under a shared backup directory of /opt/backup:

/opt/gpudb/kagent/bin/kagent cluster list-backups --backup-path /opt/backup clusterA

This might show output like the following, for cluster snapshots under /opt/backup:

/clusterA/snapshot.2019-01-02.01-23-45
/clusterA/snapshot.2019-01-02.11-11-11
/clusterB/snapshot.2019-01-02.02-34-56
/clusterB/snapshot.2019-01-02.12-12-12

Note

That if multiple clusters are backed up to the same shared location, all clusters' backups will be listed, allowing for later restoration of the targeted cluster from any of the other clusters' backups.

Restoring

Backups made through KAgent are restored through KAgent, either via command line or the KAgent GUI. See Snapshots for details on restoring from snapshots using the GUI.

The base command for restoring from backup:

/opt/gpudb/kagent/bin/kagent cluster restore --restore-from <source cluster name>/<snapshot name> [--backup-path <backup path>] <target cluster name>

Restore From

The --restore-from parameter specifies which backup snapshot should be restored. It should be the path to the existing snapshot to restore from, including the directory that is the name of the cluster from which the backup was taken:

<source cluster name>/snapshot.<date>.<time>

This should be the same path that is displayed by the backup listing command.

Backup Path

The backup path should be the file path on each Kinetica cluster node that contains the backups to restore.

The default backup path is /opt/backups.

KAgent will look on each cluster node in the directory given in the --backup-path parameter for the directory named in the --restore-from parameter. If found, KAgent will restore the database on each node from its corresponding local snapshot.

For instance, given the following restore command execution:

/opt/gpudb/kagent/bin/kagent cluster restore --restore-from mycluster/snapshot.2019-01-02.12-34-56 mycluster

A backup snapshot directory will be looked for at this location, on each node:

/opt/backups/mycluster/snapshot.2019-01-02.12-34-56

Examples

To restore a backup of the mycluster cluster in the (default) /opt/backups directory that was initiated on January 2nd, 2019 at 12:34:56:

/opt/gpudb/kagent/bin/kagent cluster restore --restore-from mycluster/snapshot.2019-01-02.12-34-56 mycluster

To restore a backup in /tmp/kinetica-backups/ for the cluster named mainkincluster that was initiated on January 2nd, 2019 at 01:23:45:

/opt/gpudb/kagent/bin/kagent cluster restore --restore-from mainkincluster/snapshot.2019-01-02.01-23-34 --backup-path /tmp/kinetica-backups/ mainkincluster

To restore a snapshot taken from one cluster, clusterA, to another cluster, clusterB, using a backup in the (default) /opt/backups directory:

/opt/gpudb/kagent/bin/kagent cluster restore --restore-from clusterA/snapshot.2019-01-02.12-34-56 clusterB

Deleting Backups

Backups can be deleted manually from their locations in the cluster's snapshot directory.

The command for deleting a backup:

rm -rf <backup path>/<cluster name>/<snapshot name>

For example, to delete a backup of the mycluster cluster in the /opt/backups directory that was initiated on January 2nd, 2019 at 12:34:56, run the following command on each node in the cluster:

rm -rf /opt/backups/mycluster/snapshot.2019-01-02.12-34-56

See Listing for details on how to list the backups available to be deleted.