pgBackRest can be used entirely with command-line parameters but a configuration file is more practical for installations that are complex or set a lot of options. The default location for the configuration file is /etc/pgbackrest.conf.
The archive section defines parameters when doing async archiving. This means that the archive files will be stored locally, then a background process will pick them and move them to the backup.
Archive WAL segments asynchronously.
WAL segments will be copied to the local repo, then a process will be forked to compress the segment and transfer it to the remote repo if configured. Control will be returned to PostgreSQL as soon as the WAL segment is copied locally.
default: n
example: archive-async=y
Limit size (in bytes) of the PostgreSQL archive queue.
After the limit is reached, the following will happen:
- pgBackRest will notify PostgreSQL that the WAL was successfully archived, then DROP IT.
- A warning will be output to the Postgres log.
If this occurs then the archive log stream will be interrupted and PITR will not be possible past that point. A new backup will be required to regain full restore capability.
In asynchronous mode the entire queue will be dropped to prevent spurts of WAL getting through before the queue limit is exceeded again.
The purpose of this feature is to prevent the log volume from filling up at which point Postgres will stop completely. Better to lose the backup than have
PostgreSQL go down.
example: archive-queue-max=1073741824
Archive timeout.
Set maximum time, in seconds, to wait for each WAL segment to reach the pgBackRest archive repository. The timeout applies to the check and backup commands when waiting for WAL segments required for backup consistency to be archived.
default: 60
allowed: 0.1-86400
example: archive-timeout=30
The backup section defines settings related to backup.
Check that WAL segments are present in the archive before backup completes.
Checks that all WAL segments required to make the backup consistent are present in the WAL archive. It's a good idea to leave this as the default unless you are using another method for archiving.
This option must be enabled if archive-copy is enabled.
default: y
example: archive-check=n
Copy WAL segments needed for consistency to the backup.
This slightly paranoid option protects against corruption in the WAL segment archive by storing the WAL segments required for consistency directly in the backup. WAL segments are still stored in the archive so this option will use additional space.
On restore, the WAL segments will be present in pg_xlog/pg_wal and PostgreSQL will use them in preference to calling the restore_command.
The archive-check option must be enabled if archive-copy is enabled.
default: n
example: archive-copy=y
Backup from the standby cluster.
Enable backup from standby to reduce load on the primary cluster. This option requires that both the primary and standby hosts be configured.
default: n
example: backup-standby=y
Validate data page checksums.
Directs pgBackRest to validate all data page checksums while backing up a cluster. This option will be automatically enabled when the required C library is present and checksums are enabled on the cluster.
Failures in checksum validation will not abort a backup. Rather, warnings will be emitted in the log (and to the console with default settings) and the list of invalid pages will be stored in the backup manifest.
example: checksum-page=n
Hardlink files between backups.
Enable hard-linking of files in differential and incremental backups to their full backups. This gives the appearance that each backup is a full backup. Be careful, though, because modifying files that are hard-linked can affect all the backups in the set.
default: n
example: hardlink=y
Manifest save threshold during backup.
Defines how often the manifest will be saved during a backup (in bytes). Saving the manifest is important because it stores the checksums and allows the resume function to work efficiently. The actual threshold used is 1% of the backup size or manifest-save-threshold, whichever is greater.
default: 1073741824
example: manifest-save-threshold=5368709120
Allow resume of failed backup.
Defines whether the resume feature is enabled. Resume can greatly reduce the amount of time required to run a backup after a previous backup of the same type has failed. It adds complexity, however, so it may be desirable to disable in environments that do not require the feature.
default: y
example: resume=n
Force a checkpoint to start backup quickly.
Forces a checkpoint (by passing y to the fast parameter of pg_start_backup()) so the backup begins immediately. Otherwise the backup will start after the next regular checkpoint.
This feature only works in PostgreSQL >= 8.4.
default: n
example: start-fast=y
Stop prior failed backup on new backup.
This will only be done if an exclusive advisory lock can be acquired to demonstrate that the prior failed backup process has really stopped.
This feature relies on pg_is_in_backup() so only works on PostgreSQL >= 9.3.
The setting is disabled by default because it assumes that pgBackRest is the only process doing exclusive online backups. It depends on an advisory lock that only pgBackRest sets so it may abort other processes that do exclusive online backups. Note that base_backup and pg_dump are safe to use with this setting because they do not call pg_start_backup() so are not exclusive.
default: n
example: stop-auto=y
The expire section defines how long backups will be retained. Expiration only occurs when the number of complete backups exceeds the allowed retention. In other words, if retention-full is set to 2, then there must be 3 complete backups before the oldest will be expired. Make sure you always have enough space for retention + 1 backups.
Number of backups worth of continuous WAL to retain.
Note that the WAL segments required to make a backup consistent are always retained until the backup is expired regardless of how this option is configured.
If this value is not set, then the archive to expire will default to the retention-full (or retention-diff) value corresponding to the retention-archive-type if set to full (or diff). This will ensure that WAL is only expired for backups that are already expired.
This option must be set if retention-archive-type is set to incr. If disk space is at a premium, then this setting, in conjunction with retention-archive-type, can be used to aggressively expire WAL segments. However, doing so negates the ability to perform PITR from the backups with expired WAL and is therefore not recommended.
allowed: 1-9999999
example: retention-archive=2
Backup type for WAL retention.
If set to full pgBackRest will keep archive logs for the number of full backups defined by retention-archive. If set to diff (differential) pgBackRest will keep archive logs for the number of full and differential backups defined by retention-archive, meaning if the last backup taken was a full backup, it will be counted as a differential for the purpose of retention. If set to incr (incremental) pgBackRest will keep archive logs for the number of full, differential, and incremental backups defined by retention-archive. It is recommended that this setting not be changed from the default which will only expire WAL in conjunction with expiring full backups.
default: full
example: retention-archive-type=diff
Number of differential backups to retain.
When a differential backup expires, all incremental backups associated with the differential backup will also expire. When not defined all differential backups will be kept until the full backups they depend on expire.
allowed: 1-9999999
example: retention-diff=3
Number of full backups to retain.
When a full backup expires, all differential and incremental backups associated with the full backup will also expire. When the option is not defined a warning will be issued. If indefinite retention is desired then set the option to the max value.
allowed: 1-9999999
example: retention-full=2
The general section defines options that are common for many commands.
Buffer size for file operations.
Set the buffer size used for copy, compress, and uncompress functions. A maximum of 3 buffers will be in use at a time per process. An additional maximum of 256K per process may be used for zlib buffers.
default: 4194304
example: buffer-size=32768
Path to ssh client executable.
Use a specific SSH client when an alternate is desired or the ssh executable is not in $PATH.
default: ssh
example: cmd-ssh=/usr/bin/ssh
Use gzip file compression.
Backup files are compatible with command-line gzip tools.
default: y
example: compress=n
Compression level for stored files.
Sets the zlib level to be used for file compression when compress=y.
default: 6
allowed: 0-9
example: compress-level=9
Compression level for network transfer when compress=n.
Sets the zlib level to be used for protocol compression when compress=n and the database cluster is not on the same host as the backup. Protocol compression is used to reduce network traffic but can be disabled by setting compress-level-network=0. When compress=y the compress-level-network setting is ignored and compress-level is used instead so that the file is only compressed once. SSH compression is always disabled.
default: 3
allowed: 0-9
example: compress-level-network=1
Database query timeout.
Sets the timeout, in seconds, for queries against the database. This includes the pg_start_backup() and pg_stop_backup() functions which can each take a substantial amount of time. Because of this the timeout should be kept high unless you know that these functions will return quickly (i.e. if you have set startfast=y and you know that the database cluster will not generate many WAL segments during the backup).
default: 1800
allowed: 0.1-604800
example: db-timeout=600
Path where lock files are stored.
The lock path provides a location for pgBackRest to create lock files to prevent conflicting operations from being run concurrently.
default: /tmp/pgbackrest
example: lock-path=/backup/db/lock
Use a neutral umask.
Sets the umask to 0000 so modes in the repository are created in a sensible way. The default directory mode is 0750 and default file mode is 0640. The lock and log directories set the directory and file mode to 0770 and 0660 respectively.
To use the executing user's umask instead specify neutral-umask=n in the config file or --no-neutral-umask on the command line.
default: y
example: neutral-umask=n
Max processes to use for compress/transfer.
Each process will perform compression and transfer to make the command run faster, but don't set process-max so high that it impacts database performance.
default: 1
allowed: 1-96
example: process-max=4
Protocol timeout.
Sets the timeout, in seconds, that the local or remote process will wait for a new message to be received on the protocol layer. This prevents processes from waiting indefinitely for a message. The protocol-timeout option must be greater than the db-timeout option.
default: 1830
allowed: 0.1-604800
example: protocol-timeout=630
Path where transient data is stored.
This path is used to store acknowledgements from the asynchronous archive-push process. These files are generally very small (zero to a few hundred bytes) so not much space is required.
The data stored in the spool path is not strictly temporary since it can and should survive a reboot. However, loss of the data in the spool path is not a problem. pgBackRest will simply recheck each WAL segment to ensure it is safely archived.
default: /var/spool/pgbackrest
example: spool-path=/backup/db/spool
The log section defines logging-related settings.
IMPORTANT NOTE: Trace-level logging may expose secrets such as keys and passwords. Use with caution!
Level for console logging.
The following log levels are supported:
- off - No logging at all (not recommended)
- error - Log only errors
- warn - Log warnings and errors
- info - Log info, warnings, and errors
- detail - Log detail, info, warnings, and errors
- debug - Log debug, detail, info, warnings, and errors
- trace - Log trace (very verbose debugging), debug, info, warnings, and errors
default: warn
example: log-level-console=error
Level for file logging.
The following log levels are supported:
- off - No logging at all (not recommended)
- error - Log only errors
- warn - Log warnings and errors
- info - Log info, warnings, and errors
- detail - Log detail, info, warnings, and errors
- debug - Log debug, detail, info, warnings, and errors
- trace - Log trace (very verbose debugging), debug, info, warnings, and errors
default: info
example: log-level-file=debug
Level for stderr logging.
Specifies which log levels will output to
stderr rather than
stdout (specified by
log-level-console). The timestamp and process will not be output to
stderr.
The following log levels are supported:
- off - No logging at all (not recommended)
- error - Log only errors
- warn - Log warnings and errors
- info - Log info, warnings, and errors
- detail - Log detail, info, warnings, and errors
- debug - Log debug, detail, info, warnings, and errors
- trace - Log trace (very verbose debugging), debug, info, warnings, and errors
default: warn
example: log-level-stderr=error
Path where log files are stored.
The log path provides a location for pgBackRest to store log files. Note that if log-level-file=none then no log path is required.
default: /var/log/pgbackrest
example: log-path=/backup/db/log
Enable timestamp in logging.
Enables the timestamp in console and file logging. This option is disabled in special situations such as generating documentation.
default: y
example: log-timestamp=n
The repository section defines options used to configure the repository.
pgBackRest exe path on the backup host.
Required only if the path to pgbackrest is different on the local and backup hosts. If not defined, the backup host exe path will be set the same as the local exe path.
example: backup-cmd=/usr/lib/backrest/bin/pgbackrest
pgBackRest backup host configuration file.
Sets the location of the configuration file on the backup host. This is only required if the backup host configuration file is in a different location than the local configuration file.
default: /etc/pgbackrest.conf
example: backup-config=/etc/pgbackrest_backup.conf
Backup host when operating remotely via SSH.
Make sure that trusted SSH authentication is configured between the db host and the backup host.
When backing up to a locally mounted network filesystem this setting is not required.
example: backup-host=backup.domain.com
Backup server SSH port when backup-host is set.
Use this option to specify a non-default SSH port for the backup server.
example: backup-ssh-port=25
Backup host user when backup-host is set.
Defines the user that will be used for operations on the backup server. Preferably this is not the postgres user but rather some other user like backrest. If PostgreSQL runs on the backup server the postgres user can be placed in the backrest group so it has read permissions on the repository without being able to damage the contents accidentally.
default: backrest
example: backup-user=backrest
Repository cipher passphrase.
Passphrase used to encrypt/decrypt files of the repository.
example: repo-cipher-pass=zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO
Cipher used to encrypt the repository.
The following repository types are supported:
- none - The repository is not encrypted
- aes-256-cbc - Advanced Encryption Standard with 256 bit key length
default: none
example: repo-cipher-type=aes-256-cbc
Repository path where WAL segments and backups stored.
The repository is where pgBackRest stores backup and archives WAL segments.
If you are new to backup then it will be difficult to estimate in advance how much space you'll need. The best thing to do is take some backups then record the size of different types of backups (full/incr/diff) and measure the amount of WAL generated per day. This will give you a general idea of how much space you'll need, though of course requirements will likely change over time as your database evolves.
default: /var/lib/pgbackrest
example: repo-path=/backup/db/backrest
S3 repository bucket.
S3 bucket used to store the repository.
pgBackRest repositories can be stored in the bucket root by setting repo-path=/ but it is usually best to specify a prefix, such as /repo, so logs and other AWS generated content can also be stored in the bucket.
example: repo-s3-bucket=db-backup
S3 SSL CA File.
Use a CA file other than the system default.
example: repo-s3-ca-file=/etc/pki/tls/certs/ca-bundle.crt
S3 SSL CA Path.
Use a CA path other than the system default.
example: repo-s3-ca-path=/etc/pki/tls/certs
S3 repository endpoint.
The AWS end point should be valid for the selected region.
example: repo-s3-endpoint=s3.amazonaws.com
S3 repository host.
Connect to a host other than the end point. This is typically used for testing.
example: repo-s3-host=127.0.0.1
S3 repository access key.
AWS key used to access this bucket.
example: repo-s3-key=AKIAIOSFODNN7EXAMPLE
S3 repository secret access key.
AWS secret key used to access this bucket.
example: repo-s3-key-secret=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
S3 repository region.
The AWS region where the bucket was created.
example: repo-s3-region=us-east-1
Verify S3 server certificate.
Disables verification of the S3 server certificate. This should only be used for testing or other scenarios where a certificate has been self-signed.
default: y
example: repo-s3-verify-ssl=n
Type of storage used for the repository.
The following repository types are supported:
- cifs - Like posix, but disables links and directory fsyncs
- posix - Posix-compliant file systems
- s3 - AWS Simple Storage Service
default: posix
example: repo-type=cifs
The restore section defines settings used for restoring backups.
Restore only specified databases.
This feature allows only selected databases to be restored. Databases not specifically included will be restored as sparse, zeroed files to save space but still allow PostgreSQL to perform recovery. After recovery the databases that were not included will not be accessible but can be removed with the drop database command.
Note that built-in databases (template0, template1, and postgres) are always restored.
The --db-include option can be passed multiple times to specify more than one database to include.
example: db-include=db_main
Restore all symlinks.
By default symlinked directories and files are restored as normal directories and files in $PGDATA. This is because it may not be safe to restore symlinks to their original destinations on a system other than where the original backup was performed. This option restores all the symlinks just as they were on the original system where the backup was performed.
default: n
example: link-all=y
Modify the destination of a symlink.
Allows the destination file or path of a symlink to be changed on restore. This is useful for restoring to systems that have a different storage layout than the original system where the backup was generated.
example: link-map=pg_xlog=/data/xlog
Set an option in recovery.conf.
See http://www.postgresql.org/docs/X.X/static/recovery-config.html for details on recovery.conf options (replace X.X with your PostgreSQL version). This option can be used multiple times.
Note: The restore_command option will be automatically generated but can be overridden with this option. Be careful about specifying your own restore_command as pgBackRest is designed to handle this for you. Target Recovery options (recovery_target_name, recovery_target_time, etc.) are generated automatically by pgBackRest and should not be set with this option.
Since pgBackRest does not start PostgreSQL after writing the recovery.conf file, it is always possible to edit/check recovery.conf before manually restarting.
example: recovery-option=primary_conninfo=db.mydomain.com
Restore a tablespace into the specified directory.
Moves a tablespace to a new location during the restore. This is useful when tablespace locations are not the same on a replica, or an upgraded system has different mount points.
Since PostgreSQL 9.2 tablespace locations are not stored in pg_tablespace so moving tablespaces can be done with impunity. However, moving a tablespace to the data_directory is not recommended and may cause problems. For more information on moving tablespaces http://www.databasesoup.com/2013/11/moving-tablespaces.html is a good resource.
example: tablespace-map=ts_01=/db/ts_01
Restore all tablespaces into the specified directory.
By default tablespaces are restored into their original locations and while this behavior can be modified by with the tablespace-map open it is sometime preferable to remap all tablespaces to a new directory all at once. This is particularly useful for development or staging systems that may not have the same storage layout as the original system where the backup was generated.
The path specified will be the parent path used to create all the tablespaces in the backup.
example: tablespace-map-all=/data/tablespace
A stanza defines the backup configuration for a specific PostgreSQL database cluster. The stanza section must define the database cluster path and host/user if the database cluster is remote. Also, any global configuration sections can be overridden to define stanza-specific settings.
Indexing: All db- options can be indexed for configuring standby replicas. For example if a single standby replica is configured then index the db- options as db2- (e.g. db2-host, db2-path, etc). If an index is not specified (e.g. db-host) it will be aliased to db1- (e.g. db1-host).
pgBackRest exe path on the database host.
Required only if the path to pgbackrest is different on the local and database hosts. If not defined, the database host exe path will be set the same as the local exe path.
example: db-cmd=/usr/lib/backrest/bin/pgbackrest
pgBackRest database host configuration file.
Sets the location of the configuration file on the database host. This is only required if the database host configuration file is in a different location than the local configuration file.
default: /etc/pgbackrest.conf
example: db-config=/etc/pgbackrest_db.conf
Cluster host for operating remotely via SSH.
Used for backups where the database cluster host is different from the backup host.
example: db-host=db.domain.com
Cluster data directory.
This should be the same as the data_directory setting in postgresql.conf. Even though this value can be read from postgresql.conf or the database cluster it is prudent to set it in case those resources are not available during a restore or offline backup scenario.
The db-path option is tested against the value reported by PostgreSQL on every online backup so it should always be current.
example: db-path=/data/db
Cluster port.
Port that PostgreSQL is running on. This usually does not need to be specified as most database clusters run on the default port.
default: 5432
example: db-port=6543
Cluster unix socket path.
The unix socket directory that was specified when PostgreSQL was started. pgBackRest will automatically look in the standard location for your OS so there usually no need to specify this setting unless the socket directory was explicitly modified with the unix_socket_directory setting in postgressql.conf.
example: db-socket-path=/var/run/postgresql
Database server SSH port when db-host is set.
Use this option to specify a non-default SSH port for a database server.
example: db-ssh-port=25
Cluster host logon user when db-host is set.
This user will also own the remote pgBackRest process and will initiate connections to PostgreSQL. For this to work correctly the user should be the PostgreSQL database cluster owner which is generally postgres, the default.
default: postgres
example: db-user=db_owner