This commit is contained in:
2021-04-12 16:41:46 +02:00
parent 604354741e
commit da6d9c53aa
44 changed files with 12125 additions and 4743 deletions

38
docs/markdown/_toc.md Normal file
View File

@@ -0,0 +1,38 @@
[Home](/)
[Quick Start](/quick)
[Installation](/installation)
[Configuration](/config)
> :Collapse label=Locations
>
> [Overview](/location/overview)
> [Hooks](/location/hooks)
> [Excluding Files](/location/exclude)
> [Forget Policy](/location/forget)
> [Cron](/location/cron)
> [Docker Volumes](/location/docker)
> :Collapse label=Backend
>
> [Overview](/backend/overview)
> [Available Backends](/backend/available)
> :Collapse label=CLI
>
> [General](/cli/general)
> [Info](/cli/info)
> [Check](/cli/check)
> [Backup](/cli/backup)
> [Restore](/cli/restore)
> [Forget](/cli/forget)
> [Cron](/cli/cron)
> [Exec](/cli/exec)
> [Install](/cli/install)
> [Uninstall](/cli/uninstall)
> [Update](/cli/update)
[Examples](/examples)
[QA](/qa)
[Contributors](/contrib)

View File

@@ -0,0 +1,62 @@
# Available Backends
In theory [all the restic backends](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html) are supported.
Those tested are the following:
## Local
```yaml
backends:
name-of-backend:
type: local
path: /data/my/backups
```
## Backblaze
```yaml
backends:
name-of-backend:
type: b2
path: 'myAccount:myBucket/my/path'
B2_ACCOUNT_ID: backblaze_account_id
B2_ACCOUNT_KEY: backblaze_account_key
```
## S3 / Minio
```yaml
backends:
name-of-backend:
type: s3
path: s3.amazonaws.com/bucket_name
# Minio
# path: http://localhost:9000/bucket_name
AWS_ACCESS_KEY_ID: my_key
AWS_SECRET_ACCESS_KEY: my_secret
```
## SFTP
For SFTP to work you need to use configure your host inside of ~/.ssh/config as password prompt is not supported. For more information on this topic please see the [official docs](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#sftp) on the matter.
```yaml
backends:
name-of-backend:
type: sftp
path: my-host:/remote/path/on/the/server
```
## Rest Server
See [here](https://github.com/restic/rest-server) for how to install a rest server backend and [here](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server) for further documentation.
```yaml
backends:
name-of-backend:
type: rest
path: http://localhost:8000/repo_name
```
> :ToCPrevNext

View File

@@ -0,0 +1,16 @@
# 💽 Backends
Backends are the ouputs of the backup process. Each location needs at least one.
```yaml | .autorestic.yml
backends:
name-of-backend:
type: local
path: /data/my/backups
```
## Types
We restic supports multiple types of backends. See the [full list](/backend/available) for details.
> :ToCPrevNext

View File

@@ -0,0 +1,13 @@
# Backup
```bash
autorestic backup [-l, --location] [-a, --all]
```
Performs a backup of all locations if the `-a` flag is passed. To only backup some locations pass one or more `-l` or `--location` flags.
```bash
autorestic backup -l my-location
```
> :ToCPrevNext

View File

@@ -0,0 +1,15 @@
# Check
```bash
autorestic check [-b, --backend] [-a, --all]
```
Cheks if one or more backend are configured properly and initializes them if they are not already.
This is mostly an internal command, but useful to verify if a backend is configured correctly.
```bash
autorestic check -b my-backend
```
> :ToCPrevNext

11
docs/markdown/cli/cron.md Normal file
View File

@@ -0,0 +1,11 @@
# Cron
```bash
autorestic cron
```
This command is mostly intended to be triggered by an automated system like systemd or crontab.
It will run cron jobs es [specified in the cron section](/locations/cron) of a specific location.
> :ToCPrevNext

15
docs/markdown/cli/exec.md Normal file
View File

@@ -0,0 +1,15 @@
# Exec
```bash
autorestic exec [-b, --backend] [-a, --all] <command> -- [native options]
```
This is avery handy command which enables you to run any native restic command on desired backends. An example would be listing all the snapshots of all your backends:
```bash
autorestic exec -a -- snapshots
```
With `exec` you can basically run every cli command that you would be able to run with the restic cli. It only pre-fills path, key, etc.
> :ToCPrevNext

View File

@@ -0,0 +1,11 @@
# Forget
```bash
autorestic forget [-l, --location] [-a, --all] [--dry-run]
```
This will prune and remove old data form the backends according to the [keep policy you have specified for the location](/locations/forget)
The `--dry-run` flag will do a dry run showing what would have been deleted, but won't touch the actual data.
> :ToCPrevNext

View File

@@ -0,0 +1,29 @@
# General
## `--version`
Prints the current version
```bash
autorestic --version
```
## `--c, --config`
Specify the config file to be used.
If omitted `autorestic` will search for for a `.autorestic.yml` in the current directory and your home directory.
```bash
autorestic -c /path/to/my/config.yml
```
## `--ci`
> Available since version 0.22
Run the CLI in CI Mode, which means there will be no interactivity.
This can be useful when you want to run cron e.g. as all the output will be saved.
```bash
autorestic --ci
```

18
docs/markdown/cli/info.md Normal file
View File

@@ -0,0 +1,18 @@
# Info
Displays the config file that autorestic is referring to.
Useful when you want to quickly see what locations are being backed-up where.
**Pro tip:** if it gets a bit long you can read it more easily with `autorestic info | less` 😉
```bash
autorestic info
```
## With a custom file
```bash
autorestic -c path/to/some/config.yml info
```
> :ToCPrevNext

View File

@@ -0,0 +1,9 @@
# Install
Installs both restic and autorestic to `/usr/local/bin`.
```bash
autorestic install
```
> :ToCPrevNext

View File

@@ -0,0 +1,17 @@
# Restore
```bash
autorestic restore [-l, --location] [--from backend] [--to <out dir>]
```
This will restore all the locations to the selected target. If for one location there are more than one backends specified autorestic will take the first one.
## Example
```bash
autorestic restore -l home --from hdd --to /path/where/to/restore
```
This will restore the location `home` to the `/path/where/to/restore` folder and taking the data from the backend `hdd`
> :ToCPrevNext

View File

@@ -0,0 +1,9 @@
# Uninstall
Installs both restic and autorestic from `/usr/local/bin`.
```bash
autorestic uninstall
```
> :ToCPrevNext

View File

@@ -0,0 +1,11 @@
# Update
Autorestic can update itself! Super handy right? Simply run autorestic update and we will check for you if there are updates for restic and autorestic and install them if necessary.
```bash
autorestic update
```
Updates both restic and autorestic automagically.
> :ToCPrevNext

42
docs/markdown/config.md Normal file
View File

@@ -0,0 +1,42 @@
# 🎛 Config File
## Path
By default autorestic searches for a `.autorestic.yml` file in the current directory and your home folder.
- `./.autorestic.yml`
- `~/.autorestic.yml`
You can also specify a custom file with the `-c path/to/some/config.yml`
> **⚠️ WARNING ⚠️**
>
> Note that the data is automatically encrypted on the server. The key will be generated and added to your config file. Every backend will have a separate key. **You should keep a copy of the keys or config file somewhere in case your server dies**. Otherwise DATA IS LOST!
## Example configuration
```yaml | .autorestic.yml
locations:
home:
from: /home/me
to: remote
important:
from: /path/to/important/stuff
to:
- remote
- hdd
backends:
remote:
type: b2
path: 'myBucket:backup/home'
B2_ACCOUNT_ID: account_id
B2_ACCOUNT_KEY: account_key
hdd:
type: local
path: /mnt/my_external_storage
```
> :ToCPrevNext

8
docs/markdown/contrib.md Normal file
View File

@@ -0,0 +1,8 @@
# 🙋‍♀️🙋‍♂️ Contributors
This amazing people helped the project!
- @ChanceM [Docs]
- @EliotBerriot [Docs, Pruning, S3]
> :ToCPrevNext

21
docs/markdown/examples.md Normal file
View File

@@ -0,0 +1,21 @@
# 🐣 Examples
## Exec
### List all the snapshots for all the backends
```bash
autorestic exec -a -- snapshots
```
### Unlock a locked repository
If you accidentally cancelled a running operation this could be useful.
Only do this if you know what you are doing.
```bash
autorestic exec -b my-backend -- unlock
```
> :ToCPrevNext

27
docs/markdown/index.md Normal file
View File

@@ -0,0 +1,27 @@
# `autorestic`
High backup level CLI utility for [restic](https://restic.net/).
Autorestic is a wrapper around the amazing [restic](https://restic.net/). While being amazing the restic cli can be a bit overwhelming and difficult to manage if you have many different location that you want to backup to multiple locations. This utility is aimed at making this easier 🙂
<!-- ![Sketch](./docs/Sketch.png) -->
## ✈️ Roadmap
~~I would like to make the official `1.0` release in the coming months. Until then please feel free to file issues or feature requests so that the tool is as flexible as possible :)~~
As of version `0.18` crons are supported wich where the last feature missing for a `1.0`. Will test this for a few weeks and then it's time for the first "real" release! 🎉 Also we now have waaay better docs 📒
## 🌈 Features
- YAML config files, no CLI
- Incremental -> Minimal space is used
- Backup locations to multiple backends
- Snapshot policies and pruning
- Fully encrypted
- Pre/After hooks
- Exclude pattern/files
- Cron jobs for automatic backup
- Backup & Restore docker volumes
> :ToCPrevNext

View File

@@ -0,0 +1,11 @@
# 🛳 Installation
Linux & macOS. Windows is not supported. If you have problems installing please open an issue :)
Autorestic requires `curl`, `wget` and `bzip2` to be installed. For most systems these should be already installed.
```bash
curl -s https://raw.githubusercontent.com/CupCakeArmy/autorestic/master/install.sh | bash
```
> :ToCPrevNext

View File

@@ -0,0 +1,49 @@
# Cron
Often it is usefull to trigger backups autmatically. For this we can specify a `cron` attribute to each location.
> Available since version 0.18
```yaml | .autorestic.yml
locations:
my-location:
from: /data
to: my-backend
cron: '0 3 * * 0' # Every Sunday at 3:00
```
Here is a awesome website with [some examples](https://crontab.guru/examples.html) and an [explorer](https://crontab.guru/)
## Installing the cron
**This has to be done only once, regadless of now many cros you have in your config file.**
To actually enable cron jobs you need something to call `autorestic cron` on a timed shedule.
Note that the shedule has nothing to do with the `cron` attribute in each location.
My advise would be to trigger the command every 5min, but if you have a cronjob that runs only once a week, it's probably enough to shedule it once a day.
### Crontab
Here is an example using crontab, but systemd would do too.
First, open your crontab in edit mode
```bash
crontab -e
```
Then paste this at the bottom of the file and save it. Note that in this specific example the `.autorestic.yml` is located in `/srv/`. You need to modify that part of course to fit your config file.
```bash
# This is required, as it otherwise cannot find restic as a command.
PATH="/usr/local/bin:/usr/bin:/bin"
# Example running every 5 minutes
*/5 * * * * autorestic -c /srv/.autorestic.yml --ci cron
```
> The `--ci` option is not required, but recommended
Now you can add as many `cron` attributes as you wish in the config file ⏱
> :ToCPrevNext

View File

@@ -0,0 +1,58 @@
# Docker
autorestic supports docker volumes directly, without needing them to be mounted to the host filesystem.
> Available since version 0.13
Let see an example.
```yaml | docker-compose.yml
version: '3.7'
volumes:
data:
name: my-data
services:
api:
image: alpine
volumes:
- data:/foo/bar
```
```yaml | .autorestic.yml
locations:
hello:
from: 'volume:my-data'
to:
- remote
options:
forget:
keep-last: 14 # Useful for limitations explained belowd
backends:
remote: ...
```
Now you can backup and restore as always.
```bash
autorestic -l hello backup
```
```bash
autorestic -l hello restore
```
If the volume does not exist on restore, autorestic will create it for you and then fill it with the data.
## Limitations
Unfortunately there are some limitations when backing up directly from a docker volume without mounting the volume to the host:
1. Incremental updates are not possible right now due to how the current docker mounting works. This means that it will take significantely more space.
2. Exclude patterns and files also do not work as restic only sees a compressed tarball as source and not the actual data.
If you are curious or have ideas how to improve this, please [read more here](https://github.com/cupcakearmy/autorestic/issues/4#issuecomment-568771951). Any help is welcomed 🙂
> :ToCPrevNext

View File

@@ -0,0 +1,20 @@
# Excluding files
If you want to exclude certain files or folders it done easily by specifiyng the right flags in the location you desire to filter.
The flags are taken straight from the [restic cli exclude rules](https://restic.readthedocs.io/en/latest/040_backup.html#excluding-files) so you can use any flag used there.
```yaml
locations:
my-location:
from: /data
to: my-backend
options:
backup:
exclude:
- '*.nope'
- '*.abc'
exclude-file: .gitignore
```
> :ToCPrevNext

View File

@@ -0,0 +1,25 @@
# Forget/Prune Policies
Autorestic supports declaring snapshot policies for location to avoid keeping old snapshot around if you don't need them.
This is based on [Restic's snapshots policies](https://restic.readthedocs.io/en/latest/060_forget.html#removing-snapshots-according-to-a-policy), and can be enabled for each location as shown below:
> **Note** This is a full example, of course you also can specify only one of them
```yaml | .autorestic.yml
locations:
etc:
from: /etc
to: local
options:
forget:
keep-last: 5 # always keep at least 5 snapshots
keep-hourly: 3 # keep 3 last hourly shapshots
keep-daily: 4 # keep 4 last daily shapshots
keep-weekly: 1 # keep 1 last weekly shapshots
keep-monthly: 12 # keep 12 last monthly shapshots
keep-yearly: 7 # keep 7 last yearly shapshots
keep-within: '2w' # keep snapshots from the last 2 weeks
```
> :ToCPrevNext

View File

@@ -0,0 +1,18 @@
# Hooks
Sometimes you might want to stop an app/db before backing up data and start the service again after the backup has completed. This is what the hooks are made for. Simply add them to your location config. You can have as many commands as you wish.
```yml | .autorestic.yml
locations:
my-location:
from: /data
to: my-backend
hooks:
before:
- echo "Hello"
- echo "Human"
after:
- echo "kthxbye"
```
> :ToCPrevNext

View File

@@ -0,0 +1,27 @@
# 🗂 Locations
Locations can be seen as the input to the backup process. Generally this is simply a folder.
The paths can be relative from the config file. A location can have multiple backends, so that the data is secured across multiple servers.
```yaml | .autorestic.yml
locations:
my-location-name:
from: path/to/backup
to:
- name-of-backend
- also-backup-to-this-backend
```
## `from`
This is the source of the location.
#### How are paths resolved?
Paths can be absolute or relative. If relative they are resolved relative to the location of the config file. Tilde `~` paths are also supported for home folder resolution.
## `to`
This is einther a single backend or an array of backends. The backends have to be configured in the same config file.
> :ToCPrevNext

8
docs/markdown/qa.md Normal file
View File

@@ -0,0 +1,8 @@
# ❓ QA
## My config file was moved?
This happens when autorestic needs to write to the config file. This happend e.g. when we are generating a key for you.
Unfortunately during this process formatting and comments are lost. That is why autorestic will place a copy of your old config next to the one we are writing to.
> :ToCPrevNext

72
docs/markdown/quick.md Normal file
View File

@@ -0,0 +1,72 @@
# 🚀 Quickstart
## Installation
```bash
curl -s https://raw.githubusercontent.com/CupCakeArmy/autorestic/master/install.sh | bash
```
## Write a simple config file
```bash
vim .autorestic.yml
```
For a quick overview:
- `locations` can be seen as the inputs and `backends` the output where the data is stored and backed up.
- One `location` can have one or multiple `backends` for redudancy.
- One `backend` can also be the target for multiple `locations`.
- **Backup the config file as it will contain the generated keys**. If you don't have a copy of that keys, the backups are useless as they are encrypted and data will be not recoverable.
```yaml | .autorestic.yml
locations:
home:
from: /home/me
to: remote
important:
from: /path/to/important/stuff
to:
- remote
- hdd
backends:
remote:
type: s3
path: 's3.amazonaws.com/bucket_name'
AWS_ACCESS_KEY_ID: account_id
AWS_SECRET_ACCESS_KEY: account_key
hdd:
type: local
path: /mnt/my_external_storage
```
## Check
```bash
autorestic check -a
```
This checks if the config file has any issues. If this is the first time this can take longer as autorestic will setup the backends.
Now is good time to **backup the config**. After you run autorestic at least once we will add the generated encryption keys to the config.
## Backup
```bash
autorestic backup -a
```
This will do a backup of all locations.
## Restore
```bash
autorestic restore -l home --from hdd --to /path/where/to/restore
```
This will restore the location `home` from the backend `hdd` to the given path.
> :ToCPrevNext