Restic Block
Table of Contents
Defined in /modules/blocks/restic.nix.
This block sets up a backup job using Restic. It is heavily based on the nixpkgs Restic module.
Provider Contracts
This block provides the following contracts:
backup contract under the
shb.restic.instancesoption. It is tested with contract tests.database backup contract under the
shb.restic.databasesoption. It is tested with contract tests.
As requested by those two contracts, when setting up a backup with Restic, a backup Systemd service and a restore script are provided.
Usage
The following examples assume usage of the sops block to provide secrets although any blocks providing the secrets contract works too.
One folder backed up manually
The following snippet shows how to configure
the backup of 1 folder to 1 repository.
We assume that the folder /var/lib/myfolder of the service myservice must be backed up.
shb.restic.instances."myservice" = {
request = {
user = "myservice";
sourceDirectories = [
"/var/lib/myfolder"
];
};
settings = {
enable = true;
passphrase.result = config.shb.sops.secret."passphrase".result;
repository = {
path = "/srv/backups/myservice";
timerConfig = {
OnCalendar = "00:00:00";
RandomizedDelaySec = "3h";
};
};
retention = {
keep_within = "1d";
keep_hourly = 24;
keep_daily = 7;
keep_weekly = 4;
keep_monthly = 6;
};
};
};
shb.sops.secret."passphrase".request =
config.shb.restic.instances."myservice".settings.passphrase.request;
The related systemd service is named restic-backups-myservice.service
and the script is restic-backups-myservice.
One folder backed up with contract
With the same example as before but assuming the myservice service
has a myservice.backup option that is a requester for the backup contract,
the snippet above becomes:
shb.restic.instances."myservice" = {
request = config.myservice.backup.request;
settings = {
enable = true;
passphrase.result = config.shb.sops.secret."passphrase".result;
repository = {
path = "/srv/backups/myservice";
timerConfig = {
OnCalendar = "00:00:00";
RandomizedDelaySec = "3h";
};
};
retention = {
keep_within = "1d";
keep_hourly = 24;
keep_daily = 7;
keep_weekly = 4;
keep_monthly = 6;
};
};
};
shb.sops.secret."passphrase".request =
config.shb.restic.instances."myservice".settings.passphrase.request;
One folder backed up to S3
Here we will only highlight the differences with the previous configuration.
This assumes you have access to such a remote S3 store, for example by using Backblaze.
shb.test.backup.instances.myservice = {
repository = {
- path = "/srv/pool1/backups/myfolder";
+ path = "s3:s3.us-west-000.backblazeb2.com/backups/myfolder";
timerConfig = {
OnCalendar = "00:00:00";
RandomizedDelaySec = "3h";
};
+ extraSecrets = {
+ AWS_ACCESS_KEY_ID.source="<path/to/access_key_id>";
+ AWS_SECRET_ACCESS_KEY.source="<path/to/secret_access_key>";
+ };
};
}
Multiple directories to multiple destinations
The following snippet shows how to configure backup of any number of folders to 3 repositories, each happening at different times to avoid I/O contention.
We will also make sure to be able to re-use as much as the configuration as possible.
A few assumptions:
2 hard drive pools used for backup are mounted respectively on
/srv/pool1and/srv/pool2.You have a backblaze account.
First, let’s define a variable to hold all the repositories we want to back up to:
repos = [
{
path = "/srv/pool1/backups";
timerConfig = {
OnCalendar = "00:00:00";
RandomizedDelaySec = "3h";
};
}
{
path = "/srv/pool2/backups";
timerConfig = {
OnCalendar = "08:00:00";
RandomizedDelaySec = "3h";
};
}
{
path = "s3:s3.us-west-000.backblazeb2.com/backups";
timerConfig = {
OnCalendar = "16:00:00";
RandomizedDelaySec = "3h";
};
}
];
Compared to the previous examples, we do not include the name of what we will back up in the repository paths.
Now, let’s define a function to create a backup configuration. It will take a list of repositories, a name identifying the backup and a list of folders to back up.
backupcfg = repositories: name: sourceDirectories {
enable = true;
backend = "restic";
keySopsFile = ../secrets/backup.yaml;
repositories = builtins.map (r: {
path = "${r.path}/${name}";
inherit (r) timerConfig;
}) repositories;
inherit sourceDirectories;
retention = {
keep_within = "1d";
keep_hourly = 24;
keep_daily = 7;
keep_weekly = 4;
keep_monthly = 6;
};
environmentFile = true;
};
Now, we can define multiple backup jobs to backup different folders:
shb.test.backup.instances.myfolder1 = backupcfg repos ["/var/lib/myfolder1"];
shb.test.backup.instances.myfolder2 = backupcfg repos ["/var/lib/myfolder2"];
The difference between the above snippet and putting all the folders into one configuration (shown below) is the former splits the backups into sub-folders on the repositories.
shb.test.backup.instances.all = backupcfg repos ["/var/lib/myfolder1" "/var/lib/myfolder2"];
Monitoring
A generic dashboard for all backup solutions is provided. See Backups Dashboard and Alert section in the monitoring chapter.
Maintenance
Manual Backup
To launch a backup manually, just run:
systemctl start <systemd-service-name>
You can easily discover the systemd service name you need by either listing the units:
systemctl list-units 'restic*'
Or by autocompleting the unit name with <TAB>:
systemctl start restic<TAB><TAB>
Note that the systemd services are of Type=simple which means the systemd service
will not wait for the backup completion to terminate.
If you want instead to wait for the backup to complete, use the --wait flag:
systemctl start --wait <systemd-service-name>
Restore
One command-line helper is provided per backup instance and repository pair which allows to:
list snapshots:
sudo <script> snapshotsto restore a snapshot:
sudo <script> restore <snapshot>to take a snapshot:
sudo <script> backupto execute a custom command where arguments are sent:
sudo <script> exec <args>for example to unlock the repo, run:sudo <script> unlock
The restore script has all the secrets needed to access the repo and the user running it needs to have correct permissions for privilege escalation.
In the multiple directories example above, the following 6 helpers are provided in the $PATH:
restic-myfolder1_srv_pool1_backups
restic-myfolder1_srv_pool2_backups
restic-myfolder1_s3_s3.us-west-000.backblazeb2.com_backups
restic-myfolder2_srv_pool1_backups
restic-myfolder2_srv_pool2_backups
restic-myfolder2_s3_s3.us-west-000.backblazeb2.com_backups
Discovering those is easy thanks to tab-completion.
One can then restore a backup from a given repository with:
restic-myfolder1_srv_pool1_backups restore <snapshot>
Troubleshooting
In case something bad happens with a backup, the official documentation has a lot of tips.
Tests
Specific integration tests are defined in /test/blocks/restic.nix.
Options Reference
-
shb.restic.enableDashboard -
Whether to enable the Backups SHB dashboard.
Type: boolean
Default:
trueExample:
trueDeclared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases -
Databases to backup following the database backup contract.
Type: attribute set of (submodule)
Default:
{ }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.request -
Request part of the database backup contract.
Options set by the requester module enforcing how to backup files.
Type: submodule
Default:
""Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.request.backupCmd -
Command that produces the database dump on stdout.
Type: string
Default:
""Example:
${pkgs.postgresql}/bin/pg_dumpall | ${pkgs.gzip}/bin/gzip --rsyncableDeclared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.request.backupName -
Name of the backup in the repository.
Type: string
Default:
"dump"Example:
"postgresql.sql"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.request.restoreCmd -
Command that reads the database dump on stdin and restores the database.
Type: string
Default:
""Example:
${pkgs.gzip}/bin/gunzip | ${pkgs.postgresql}/bin/psql postgresDeclared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.request.user -
Unix user doing the backups.
This should be an admin user having access to all databases.
Type: string
Default:
"root"Example:
"postgres"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.result -
Result part of the database backup contract.
Options set by the provider module that indicates the name of the backup and restore scripts.
Type: submodule
Default: { restoreScript = restic-backups-<name>_path_to_repository; backupService = restic-backups-<name>_path_to_repository.service; }
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.result.backupService -
Name of service backing up the database.
This script can be ran manually to backup the database:
$ systemctl start restic-backups-<name>_path_to_repository.serviceType: string
Default: restic-backups-<name>_path_to_repository.service
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.result.restoreScript -
Name of script that can restore the database. One can then list snapshots with:
$ restic-backups-<name>_path_to_repository snapshotsAnd restore the database with:
$ restic-backups-<name>_path_to_repository restore latestType: string
Default: restic-backups-<name>_path_to_repository
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings -
Settings specific to the Restic provider.
Type: submodule
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.enable -
Whether to enable SelfHostBlocks’ Restic block
A disabled instance will not backup data anymore but still provides the helper tool to restore snapshots .
Type: boolean
Default:
falseExample:
trueDeclared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.limitDownloadKiBs -
Limit download bandwidth to the given KiB/s amount.
Type: null or signed integer
Default:
nullExample:
8000Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.limitUploadKiBs -
Limit upload bandwidth to the given KiB/s amount.
Type: null or signed integer
Default:
nullExample:
8000Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.passphrase -
Encryption key for the backup repository.
Type: submodule
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.passphrase.request -
Request part of the secret contract.
Options set by the requester module enforcing some properties the secret should have.
Type: submodule
Default: { mode = 0400; owner = shb.restic.databases.<name>.request.user; group = root; restartUnits = [ shb.restic.databases.<name>.settings.repository ]; }
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.passphrase.request.group -
Linux group owning the secret file.
Type: string
Default:
"root"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.passphrase.request.mode -
Mode of the secret file.
Type: string
Default:
"0400"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.passphrase.request.owner -
Linux user owning the secret file.
Type: string
Default: shb.restic.databases.<name>.request.user
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.passphrase.request.restartUnits -
Systemd units to restart after the secret is updated.
Type: list of string
Default: [ shb.restic.databases.<name>.settings.repository ]
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.passphrase.result -
Result part of the secret contract.
Options set by the provider module that indicates where the secret can be found.
Type: submodule
Default:
{ path = "/run/secrets/secret"; }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.passphrase.result.path -
Path to the file containing the secret generated out of band.
This path will exist after deploying to a target host, it is not available through the nix store.
Type: absolute path
Default:
"/run/secrets/secret"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.repository -
Repositories to back this instance to.
Type: submodule
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.repository.path -
Repository location
Type: string
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.repository.secrets -
Secrets needed to access the repository where the backups will be stored.
See s3 config for an example and list for the list of all secrets.
Type: attribute set of (submodule)
Default:
{ }Example:
{ AWS_ACCESS_KEY_ID.source = <path/to/secret>; AWS_SECRET_ACCESS_KEY.source = <path/to/secret>; }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.repository.secrets.<name>.source -
File containing the value.
Type: absolute path
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.repository.secrets.<name>.transform -
An optional function to transform the secret.
Type: raw value
Default:
nullExample:
v: "prefix-$${v}-suffix"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.repository.timerConfig -
When to run the backup. See systemd.timer(5) for details.
Type: attribute set of (systemd option)
Default:
{ OnCalendar = "daily"; Persistent = true; }Example:
{ OnCalendar = "00:05"; Persistent = true; RandomizedDelaySec = "5h"; }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.databases.<name>.settings.retention -
For how long to keep backup files.
Type: attribute set of (signed integer or non-empty string)
Default:
{ keep_daily = 7; keep_hourly = 24; keep_monthly = 6; keep_weekly = 4; keep_within = "1d"; }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances -
Files to backup following the backup contract.
Type: attribute set of (submodule)
Default:
{ }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.request -
Request part of the backup contract.
Options set by the requester module enforcing how to backup files.
Type: submodule
Default:
""Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.request.excludePatterns -
File patterns to exclude.
Type: list of string
Default:
[ ]Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.request.hooks -
Hooks to run around the backup.
Type: submodule
Default:
{ }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.request.hooks.afterBackup -
Hooks to run after backup.
Type: list of string
Default:
[ ]Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.request.hooks.beforeBackup -
Hooks to run before backup.
Type: list of string
Default:
[ ]Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.request.sourceDirectories -
Directories to backup.
Type: non-empty (list of string)
Default:
[ "/var/lib/example" ]Example:
"/var/lib/vaultwarden"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.request.user -
Unix user doing the backups.
Type: string
Default:
""Example:
"vaultwarden"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.result -
Result part of the backup contract.
Options set by the provider module that indicates the name of the backup and restore scripts.
Type: submodule
Default: { restoreScript = restic-backups-<name>_path_to_repository; backupService = restic-backups-<name>_path_to_repository.service; }
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.result.backupService -
Name of service backing up the database.
This script can be ran manually to backup the database:
$ systemctl start restic-backups-<name>_path_to_repository.serviceType: string
Default: restic-backups-<name>_path_to_repository.service
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.result.restoreScript -
Name of script that can restore the database. One can then list snapshots with:
$ restic-backups-<name>_path_to_repository snapshots <snapshot 1> <metadata> <snapshot 2> <metadata>And restore the database with:
$ restic-backups-<name>_path_to_repository restore <snapshot 1>Type: string
Default: restic-backups-<name>_path_to_repository
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings -
Settings specific to the Restic provider.
Type: submodule
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.enable -
Whether to enable SelfHostBlocks’ Restic block
A disabled instance will not backup data anymore but still provides the helper tool to restore snapshots .
Type: boolean
Default:
falseExample:
trueDeclared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.limitDownloadKiBs -
Limit download bandwidth to the given KiB/s amount.
Type: null or signed integer
Default:
nullExample:
8000Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.limitUploadKiBs -
Limit upload bandwidth to the given KiB/s amount.
Type: null or signed integer
Default:
nullExample:
8000Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.passphrase -
Encryption key for the backup repository.
Type: submodule
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.passphrase.request -
Request part of the secret contract.
Options set by the requester module enforcing some properties the secret should have.
Type: submodule
Default: { mode = 0400; owner = shb.restic.instances.<name>.request.user; group = root; restartUnits = [ shb.restic.instances.<name>.settings.repository ]; }
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.passphrase.request.group -
Linux group owning the secret file.
Type: string
Default:
"root"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.passphrase.request.mode -
Mode of the secret file.
Type: string
Default:
"0400"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.passphrase.request.owner -
Linux user owning the secret file.
Type: string
Default: shb.restic.instances.<name>.request.user
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.passphrase.request.restartUnits -
Systemd units to restart after the secret is updated.
Type: list of string
Default: [ shb.restic.instances.<name>.settings.repository ]
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.passphrase.result -
Result part of the secret contract.
Options set by the provider module that indicates where the secret can be found.
Type: submodule
Default:
{ path = "/run/secrets/secret"; }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.passphrase.result.path -
Path to the file containing the secret generated out of band.
This path will exist after deploying to a target host, it is not available through the nix store.
Type: absolute path
Default:
"/run/secrets/secret"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.repository -
Repositories to back this instance to.
Type: submodule
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.repository.path -
Repository location
Type: string
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.repository.secrets -
Secrets needed to access the repository where the backups will be stored.
See s3 config for an example and list for the list of all secrets.
Type: attribute set of (submodule)
Default:
{ }Example:
{ AWS_ACCESS_KEY_ID.source = <path/to/secret>; AWS_SECRET_ACCESS_KEY.source = <path/to/secret>; }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.repository.secrets.<name>.source -
File containing the value.
Type: absolute path
Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.repository.secrets.<name>.transform -
An optional function to transform the secret.
Type: raw value
Default:
nullExample:
v: "prefix-$${v}-suffix"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.repository.timerConfig -
When to run the backup. See systemd.timer(5) for details.
Type: attribute set of (systemd option)
Default:
{ OnCalendar = "daily"; Persistent = true; }Example:
{ OnCalendar = "00:05"; Persistent = true; RandomizedDelaySec = "5h"; }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.instances.<name>.settings.retention -
For how long to keep backup files.
Type: attribute set of (signed integer or non-empty string)
Default:
{ keep_daily = 7; keep_hourly = 24; keep_monthly = 6; keep_weekly = 4; keep_within = "1d"; }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.performance -
Reduce performance impact of backup jobs.
Type: submodule
Default:
{ }Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.performance.ioPriority -
ionice priority, defaults to 7 for lowest priority IO. Only used for
restic backup,restic forgetandrestic checkcommands.Type: null or integer between 0 and 7 (both inclusive)
Default:
7Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.performance.ioSchedulingClass -
ionice scheduling class, defaults to best-effort IO. Only used for
restic backup,restic forgetandrestic checkcommands.Type: one of “idle”, “best-effort”, “realtime”
Default:
"best-effort"Declared by:
<selfhostblocks/modules/blocks/restic.nix> -
shb.restic.performance.niceness -
nice priority adjustment, defaults to 15 for ~20% CPU time of normal-priority process
Type: integer between -20 and 19 (both inclusive)
Default:
15Declared by:
<selfhostblocks/modules/blocks/restic.nix>