This commit is contained in:
cupcakearmy 2020-05-17 14:52:30 +02:00
parent d20a2fda68
commit 734c962006
No known key found for this signature in database
GPG Key ID: D28129AE5654D9D9
26 changed files with 518 additions and 2 deletions

5
.gitignore vendored
View File

@ -17,4 +17,7 @@ Dockerfile
# Config # Config
.autorestic.yml .autorestic.yml
.autorestic.lock .autorestic.lock
.docker.yml .docker.yml
# Docs
.codedoc

37
docs/md/_toc.md Normal file
View File

@ -0,0 +1,37 @@
[Home](/)
[Quick Start](/quick)
[Installation](/installation)
[Configuration](/config)
> :Collapse label=Locations
>
> [Overview](/locations/overview)
> [Hooks](/locations/hooks)
> [Exluding Files](/locations/exclude)
> [Forget Policy](/locations/forget)
> [Cron](/locations/cron)
> [Docker Volumes](/locations/docker)
> :Collapse label=Backend
>
> [Overview](/backend/overview)
> [Available Backends](/backend/available)
> :Collapse label=CLI
>
> [Info](/cli/info)
> [Check](/cli/check)
> [Backup](/cli/backup)
> [Restore](/cli/restore)
> [Forget](/cli/forget)
> [Cron](/cli/cron)
> [Exec](/cli/exec)
> [Install](/cli/install)
> [Uninstall](/cli/uninstall)
> [Update](/cli/update)
[Examples](/examples)
[QA](/qa)
[Contributors](/contrib)

13
docs/md/cli/backup.md Normal file
View File

@ -0,0 +1,13 @@
# Backup
```bash
autorestic backup [-l, --location] [-a, --all]
```
Performes a backup of all locations if the `-a` flag is passed. To only backup some locations pass one or more `-l` or `--location` flags.
```bash
autorestic backup -l my-location
```
> :ToCPrevNext

15
docs/md/cli/check.md Normal file
View File

@ -0,0 +1,15 @@
# check
```bash
autorestic check [-b, --backend] [-a, --all]
```
Cheks if one or more backend are configured properly and initializes them if they are not already.
This is mostly an internal command, but useful to verify if a backend is configured correctly.
```bash
autorestic check -b my-backend
```
> :ToCPrevNext

11
docs/md/cli/cron.md Normal file
View File

@ -0,0 +1,11 @@
# Cron
```bash
autorestic cron
```
This command is modtly intended to be triggered by an automated system like systemd or crontab.
It will run cron jobs es [specified in the cron section](/locations/cron) of a specific location.
> :ToCPrevNext

15
docs/md/cli/exec.md Normal file
View File

@ -0,0 +1,15 @@
# Exec
```bash
autorestic exec [-b, --backend] [-a, --all] <command> -- [native options]
```
This is avery handy command which enables you to run any native restic command on desired backends. An example would be listing all the snapshots of all your backends:
```bash
autorestic exec -a -- snapshots
```
With `exec` you can basically run every cli command that you would be able to run with the restic cli. It only prefillst path, key, etc.
> :ToCPrevNext

11
docs/md/cli/forget.md Normal file
View File

@ -0,0 +1,11 @@
# Forget
```bash
autorestic forget [-l, --location] [-a, --all] [--dry-run]
```
This will prune and remove old data form the backends according to the [keep policy you have specified for the location](/locations/forget)
The `--dry-run` flag will do a dry run showing what would have been deleted, but won't touch the actual data.
> :ToCPrevNext

18
docs/md/cli/info.md Normal file
View File

@ -0,0 +1,18 @@
# info
Displays the config file that autorestic is refering to.
Usefull when you want to quickly see what locations are being backuped where.
**Pro tip:** if it gets a bit long you can read it more easily with `autorestic info | less` 😉
```bash
autorestic info
```
## With a custom file
```bash
autorestic -c path/to/some/config.yml info
```
> :ToCPrevNext

9
docs/md/cli/install.md Normal file
View File

@ -0,0 +1,9 @@
# Install
Installs both restic and autorestic to `/usr/local/bin`.
```bash
autorestic install
```
> :ToCPrevNext

17
docs/md/cli/restore.md Normal file
View File

@ -0,0 +1,17 @@
# Restore
```bash
autorestic restore [-l, --location] [--from backend] [--to <out dir>]
```
This will restore all the locations to the selected target. If for one location there are more than one backends specified autorestic will take the first one.
## Example
```bash
autorestic restore -l home --from hdd --to /path/where/to/restore
```
This will restore the location `home` to the `/path/where/to/restore` folder and taking the data from the backend `hdd`
> :ToCPrevNext

9
docs/md/cli/uninstall.md Normal file
View File

@ -0,0 +1,9 @@
# Uninstall
Installs both restic and autorestic from `/usr/local/bin`.
```bash
autorestic uninstall
```
> :ToCPrevNext

11
docs/md/cli/update.md Normal file
View File

@ -0,0 +1,11 @@
# Update
Autorestic can update itself! Super handy right? Simply run autorestic update and we will check for you if there are updates for restic and autorestic and install them if necessary.
```bash
autorestic update
```
Updates both restic and autorestic automagically.
> :ToCPrevNext

42
docs/md/config.md Normal file
View File

@ -0,0 +1,42 @@
# 🎛 Config File
## Path
By default autorestic searches for a `.autorestic.yml` file in the current directory and your home folder.
- `./.autorestic.yml`
- `~/.autorestic.yml`
You can also specify a custom file with the `-c path/to/some/config.yml`
> **⚠️ WARNING ⚠️**
>
> Note that the data is automatically encrypted on the server. The key will be generated and added to your config file. Every backend will have a separate key. **You should keep a copy of the keys or config file somewhere in case your server dies**. Otherwise DATA IS LOST!
## Example configuration
```yaml | .autorestic.yml
locations:
home:
from: /home/me
to: remote
important:
from: /path/to/important/stuff
to:
- remote
- hdd
backends:
remote:
type: b2
path: 'myBucket:backup/home'
B2_ACCOUNT_ID: account_id
B2_ACCOUNT_KEY: account_key
hdd:
type: local
path: /mnt/my_external_storage
```
> :ToCPrevNext

8
docs/md/contrib.md Normal file
View File

@ -0,0 +1,8 @@
# 🙋‍♀️🙋‍♂️ Contributors
This amazing people helped the project!
- @ChanceM [Docs]
- @EliotBerriot [Docs, Pruning, S3]
> :ToCPrevNext

19
docs/md/examples.md Normal file
View File

@ -0,0 +1,19 @@
# 🐣 Examples
## List all the snapshots for all the backends
```bash
autorestic -a exec snapshots
```
## Unlock a locked repository
If you accidentally cancelled a running operation this could be useful.
Only do this if you know what you are doing.
```bash
autorestic -b my-backend exec unlock
```
> :ToCPrevNext

27
docs/md/index.md Normal file
View File

@ -0,0 +1,27 @@
# autorestic
High backup level CLI utility for [restic](https://restic.net/).
Autorestic is a wrapper around the amazing [restic](https://restic.net/). While being amazing the restic cli can be a bit overwhelming and difficoult to manage if you have many different location that you want to backup to multiple locations. This utility is aimed at making this easier 🙂
<!-- ![Sketch](./docs/Sketch.png) -->
## ✈️ Roadmap
I would like to make the official `1.0` release in the coming months. Until then please feel free to file issues or feature requests so that the tool is as flexible as possible :)
## 🌈 Features
- YAML config files, no CLI
- Predictable
- Incremental -> Minimal space is used
- Backup locations to multiple backends
- Snapshot policies and pruning
- Simple interface
- Fully encrypted
- Pre/After hooks
- Exclude pattern/files
- Backup & Restore docker volumes
- ~~Seamless cron jobs for automatic backup~~ [in development](https://github.com/cupcakearmy/autorestic/issues/21).
> :ToCPrevNext

11
docs/md/installation.md Normal file
View File

@ -0,0 +1,11 @@
# 🛳 Installation
Linux & macOS. Windows is not supported. If you have problems installing please open an issue :)
Autorestic requires `curl`, `wget` and `bzip2` to be installed. For most systems these should be already installed.
```bash
curl -s https://raw.githubusercontent.com/CupCakeArmy/autorestic/master/install.sh | bash
```
> :ToCPrevNext

15
docs/md/locations/cron.md Normal file
View File

@ -0,0 +1,15 @@
# Cron
Often it is usefull to trigger backups autmatically. For this we can specify a `cron` attribute to each location.
```yaml | .autorestic.yml
locations:
my-location:
from: /data
to: my-backend
cron: '0 3 * * 0' # Every Sunday at 3:00
```
Here is a awesome website with [some examples](https://crontab.guru/examples.html) and an [explorer](https://crontab.guru/)
> :ToCPrevNext

View File

@ -0,0 +1,56 @@
# Docker
Since version 0.13 autorestic supports docker volumes directly, without needing them to be mounted to the host filesystem.
Let see an example.
```yaml | docker-compose.yml
version: '3.7'
volumes:
data:
name: my-data
services:
api:
image: alpine
volumes:
- data:/foo/bar
```
```yaml | .autorestic.yml
locations:
hello:
from: 'volume:my-data'
to:
- remote
options:
forget:
keep-last: 14 # Useful for limitations explained belowd
backends:
remote: ...
```
Now you can backup and restore as always.
```bash
autorestic -l hello backup
```
```bash
autorestic -l hello restore
```
If the volume does not exist on restore, autorestic will create it for you and then fill it with the data.
## Limitations
Unfortunately there are some limitations when backing up directly from a docker volume without mounting the volume to the host:
1. Incremental updates are not possible right now due to how the current docker mounting works. This means that it will take significantely more space.
2. Exclude patterns and files also do not work as restic only sees a compressed tarball as source and not the actual data.
If you are curious or have ideas how to improve this, please [read more here](https://github.com/cupcakearmy/autorestic/issues/4#issuecomment-568771951). Any help is welcomed 🙂
> :ToCPrevNext

View File

@ -0,0 +1,20 @@
# Excluding files
If you want to exclude certain files or folders it done easily by specifiyng the right flags in the location you desire to filter.
The flags are taken straight from the [restic cli exclude rules](https://restic.readthedocs.io/en/latest/040_backup.html#excluding-files) so you can use any flag used there.
```yaml
locations:
my-location:
from: /data
to: my-backend
options:
backup:
exclude:
- '*.nope'
- '*.abc'
exclude-file: .gitignore
```
> :ToCPrevNext

View File

@ -0,0 +1,25 @@
# Forget/Prune Policies
Autorestic supports declaring snapshot policies for location to avoid keeping old snapshot around if you don't need them.
This is based on [Restic's snapshots policies](https://restic.readthedocs.io/en/latest/060_forget.html#removing-snapshots-according-to-a-policy), and can be enabled for each location as shown below:
> **Note** This is a full example, of course you also can specify only one of them
```yaml | .autorestic.yml
locations:
etc:
from: /etc
to: local
options:
forget:
keep-last: 5 # always keep at least 5 snapshots
keep-hourly: 3 # keep 3 last hourly shapshots
keep-daily: 4 # keep 4 last daily shapshots
keep-weekly: 1 # keep 1 last weekly shapshots
keep-monthly: 12 # keep 12 last monthly shapshots
keep-yearly: 7 # keep 7 last yearly shapshots
keep-within: '2w' # keep snapshots from the last 2 weeks
```
> :ToCPrevNext

View File

@ -0,0 +1,18 @@
# Hooks
Sometimes you might want to stop an app/db before backing up data and start the service again after the backup has completed. This is what the hooks are made for. Simply add them to your location config. You can have as many commands as you wish.
```yml | .autorestic.yml
locations:
my-location:
from: /data
to: my-backend
hooks:
before:
- echo "Hello"
- echo "Human"
after:
- echo "kthxbye"
```
> :ToCPrevNext

View File

@ -0,0 +1,27 @@
# 🗂 Locations
Locations can be seen as the input to the backup process. Generally this is simply a folder.
The paths can be relative from the config file. A location can have multiple backends, so that the data is secured across multiple servers.
```yaml | .autorestic.yml
locations:
my-location-name:
from: path/to/backup
to:
- name-of-backend
- also-backup-to-this-backend
```
## `from`
This is the source of the location.
#### How are paths resolved?
Paths can be absolute or relative. If relative they are resolved relative to the location of the config file. Tilde `~` paths are also supported for home folder resolution.
## `to`
This is einther a single backend or an array of backends. The backends have to be configured in the same config file.
> :ToCPrevNext

8
docs/md/qa.md Normal file
View File

@ -0,0 +1,8 @@
# ❓ QA
## My config file was moved?
This happens when autorestic needs to write to the config file. This happend e.g. when we are generating a key for you.
Unforunately during this process formatting and comments are lost. That is why autorestic will place a copy of your old config next to the one we are writing to.
> :ToCPrevNext

69
docs/md/quick.md Normal file
View File

@ -0,0 +1,69 @@
# 🚀 Quickstart
## Installation
```bash
curl -s https://raw.githubusercontent.com/CupCakeArmy/autorestic/master/install.sh | bash
```
## Write a simple config file
```bash
vim .autorestic.yml
```
For a quick overview:
- `locations` can be seen as the inputs and `backends` the output where the data is stored and backed up.
- One `location` can have one or multiple `backends` for redudancy.
- One `backend` can also be the target for multiple `locations`
```yaml | .autorestic.yml
locations:
home:
from: /home/me
to: remote
important:
from: /path/to/important/stuff
to:
- remote
- hdd
backends:
remote:
type: s3
path: 's3.amazonaws.com/bucket_name'
AWS_ACCESS_KEY_ID: account_id
AWS_SECRET_ACCESS_KEY: account_key
hdd:
type: local
path: /mnt/my_external_storage
```
## Check [Optional]
```bash
autorestic check -a
```
This checks if the config file has any issues. If this is the first time this can take longer as autorestic will setup the backends.
## Backup
```bash
autorestic backup -a
```
This will do a backup of all locations.
## Restore
```bash
autorestic restore -l home --from hdd --to /path/where/to/restore
```
This will restore the location `home` from the backend `hdd` to the given path.
> :ToCPrevNext

View File

@ -5,9 +5,11 @@
"build:watch": "tsc -w", "build:watch": "tsc -w",
"dev": "tsnd --no-notify --respawn ./src/autorestic.ts", "dev": "tsnd --no-notify --respawn ./src/autorestic.ts",
"move": "mv bin/autorestic-linux bin/autorestic_linux_x64 && mv bin/autorestic-macos bin/autorestic_macos_x64", "move": "mv bin/autorestic-linux bin/autorestic_linux_x64 && mv bin/autorestic-macos bin/autorestic_macos_x64",
"bin": "yarn run build && pkg lib/autorestic.js --targets latest-macos-x64,latest-linux-x64 --out-path bin && yarn run move" "bin": "yarn run build && pkg lib/autorestic.js --targets latest-macos-x64,latest-linux-x64 --out-path bin && yarn run move",
"docs:build": "codedoc build"
}, },
"devDependencies": { "devDependencies": {
"@codedoc/cli": "^0.1.4",
"@types/js-yaml": "^3.12.1", "@types/js-yaml": "^3.12.1",
"@types/minimist": "^1.2.0", "@types/minimist": "^1.2.0",
"@types/node": "^12.11.7", "@types/node": "^12.11.7",