octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-12-24 11:55:28 +00:00
Code Issues Releases Wiki Activity
4,827 Commits 7 Branches 33 Tags 66 MiB
6253ff0187
Commit Graph

292 Commits

Author SHA1 Message Date
Andreas Oberritter
97f7855de3 Add new option --repository-file (default: $RESTIC_REPOSITORY_FILE) 
As an alternative to -r, this allows to read the repository URL
from a file in order to prevent certain types of information leaks,
especially for URLs containing credentials.

Fixes #1458, fixes #2900.
 2020-10-01 00:50:26 +02:00
MichaelEischer
fd02407863
Merge pull request #2849 from classmarkets/gcs-access-token 
gs: support authentication with access token
 2020-09-30 17:42:56 +02:00
Fred
206cadfab4 Hide password from repository URLs 2020-09-22 22:00:51 +02:00
yoshiera
3cf29a777d Fix nil check in rejectBySize 2020-09-21 19:20:24 +08:00
Alexander Neumann
96fd982f6a Prepare changelog for 0.10.0 2020-09-19 17:37:59 +02:00
Alexander Neumann
6ff0082c02
Merge pull request #2928 from MichaelEischer/init-copy-chunker 
init: Add `--copy-chunker-parameters` option
 2020-09-19 17:32:13 +02:00
Alexander Neumann
95c1d7d959
Merge pull request #2927 from restic/update-deps 
Update dependencies
 2020-09-19 17:29:35 +02:00
Michael Eischer
f003410402 init: Add --copy-chunker-params option 
This allows creating multiple repositories with identical chunker
parameters which is required for working deduplication when copying
snapshots between different repositories.
 2020-09-19 16:53:05 +02:00
Alexander Neumann
a10b44a265 Add note about fuse 2020-09-19 14:35:55 +02:00
Alexander Neumann
8ce0ce387f Add changelog 2020-09-19 14:22:31 +02:00
yoshiera
ac4b8c98ac Support excluding files by size 2020-09-19 19:52:09 +08:00
Michael Eischer
4f221c4022 Add changelog entry 2020-09-13 16:28:46 +02:00
MichaelEischer
88664ba222
Merge pull request #2893 from MichaelEischer/restore-preallocate 
restorer: Preallocate files
 2020-09-08 22:43:05 +02:00
Michael Eischer
8cc9514879 restorer: pre-allocate files before loading chunks 2020-09-07 21:41:47 +02:00
MichaelEischer
2ddb7ffb7e
Merge pull request #2255 from Kidswiss/tar 
Fix dumping issues with / and the first sub level
 2020-09-01 21:52:17 +02:00
MichaelEischer
55071ee367
Merge pull request #2859 from buschjost/stats-filter-by-tag-and-path 
Add filter by tag and path to stats command
 2020-08-31 22:11:01 +02:00
Oliver Buschjost
d3fcfeba3a Add filter by tag and path to stats command 2020-08-30 17:14:48 +02:00
Simon Beck
da4193c3ef Add changelog 2020-08-30 16:00:48 +02:00
MichaelEischer
ea81a0e282
Merge pull request #2606 from middelink/fix-323 
Add copy functionality.
 2020-08-30 10:18:24 +02:00
MichaelEischer
c34c731698
Merge pull request #2702 from MichaelEischer/fix-backup-dir-count 
backup: Fix reporting of directory count in summary
 2020-08-29 11:29:21 +02:00
Alexander Weiss
9867c4bbb4 Add flag --iexclude-file to backup 2020-08-28 21:04:41 +02:00
Michael Eischer
b25978a53c backup: Fix reporting of directory count in summary 
Previously the directory stats were reported immediately after calling
`SaveDir`. However, as the latter method saves the tree asynchronously
the stats were still initialized to their nil value. The stats are now
reported via a callback similar to the one used for the fileSaver.
 2020-08-27 22:43:51 +02:00
Pauline Middelink
7048cc3e58 Add copy functionality 
Add a copy command to copy snapshots between repositories. It allows the user
to specify a destination repository, password, password-file, password-command
or key-hint to supply the necessary details to open the destination repository.

You need to supply a list of snapshots to copy, snapshots which already exist
in the destination repository will be skipped.

Note, when using the network this becomes rather slow, as it needs to read the
blocks, decrypt them using the source key, then encrypt them again using the
destination key before finally writing them out to the destination repository.
 2020-08-26 22:17:52 +02:00
Michael Eischer
bc0501d72c Add changelog 2020-08-25 23:33:10 +02:00
Alexander Neumann
b1b3f1ecb6
Merge pull request #2674 from MichaelEischer/prune-strict-checks 
prune: Stricter error checks
 2020-08-23 10:30:41 +02:00
Hristo Trendev
51c22f4223 local backend: ignore not supported error on sync() 
Closes #2395
 2020-08-22 01:27:07 +02:00
Michael Eischer
d8b80e9862 Add changelog 2020-08-16 11:36:14 +02:00
Sébastien Gross
9be4fe3e84 cmd/mount: honour --no-lock flag 
Do not lock the repository if --no-lock global flag is set. This allows
to mount repositories which are archived on a read only system.

Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
 2020-08-03 19:45:39 +02:00
Alexander Weiss
0eb8553c87 add changelog for #2840 2020-08-01 20:43:18 +02:00
Alexander Neumann
b7b479b668
Merge pull request #2599 from MichaelEischer/tweak-mem-usage 
Reduce memory usage when searching for used blobs
 2020-08-01 13:49:02 +02:00
Alexander Neumann
2580eef2aa
Merge pull request #2318 from classmarkets/2175-named-keys 
Allow specifying user and host when adding keys
 2020-08-01 13:06:31 +02:00
Michael Eischer
2d7ab9115f Add changelog entry 2020-08-01 12:29:16 +02:00
Alexander Weiss
f8316948d1 Optimize FUSE - make command restic mount faster and consume less memory 
- Add Open() functionality to dir
- only access index for blobs when file is read
- Implement NodeOpener and put one-time file stuff there
- Add comment about locking as suggested by bazil.org/fuse

=> Thanks at Michael Eischer for suggesting the last two improvements
 2020-07-28 23:01:18 +02:00
MichaelEischer
3d530dfc91
Merge pull request #2827 from aawsome/archiver-test-contents 
Make self-healing work when backing up with parent snapshot
 2020-07-25 13:13:18 +02:00
MichaelEischer
c43f5b2664
Merge pull request #2733 from hibbert/support_irsa 
Enable support for EKS IRSA
 2020-07-25 12:58:43 +02:00
Andy Hibbert
38087e40d9 Update minio-go version so it supports EKS IRSA 2020-07-25 12:41:18 +02:00
Alexander Weiss
9175795fdb Check contents in archiver 
When backing up with a parent snapshot and the file is not changed, also
check if contents are still available in index.
 2020-07-25 08:18:28 +02:00
rawtaz
5d8d70542f
Merge pull request #2852 from MichaelEischer/drop-go-1.11 
Drop support for Go version 1.11
 2020-07-25 01:17:36 +02:00
Michael Eischer
7c23381a2b Drop support for Go version 1.11 2020-07-24 18:52:39 +02:00
MichaelEischer
34181b13a2
Merge pull request #2328 from MichaelEischer/no-repeated-checks 
Fix duplicate tree checks within `restic check`
 2020-07-22 22:08:02 +02:00
Peter Schultz
758b44b9c0 gs: support authentication with access token 
In the Google Cloud Storage backend, support specifying access tokens
directly, as an alternative to a credentials file. This is useful when
restic is used non-interactively by some other program that is already
authenticated and eliminates the need to store long lived credentials.

The access token is specified in the GOOGLE_ACCESS_TOKEN environment
variable and takes precedence over GOOGLE_APPLICATION_CREDENTIALS.
 2020-07-22 16:23:03 +02:00
MichaelEischer
82c908871d
Merge pull request #2812 from greatroar/chaining 
Chaining hash table for repository.Index
 2020-07-20 23:29:36 +02:00
Michael Eischer
0f67ae813a Add changelog 2020-07-20 22:51:53 +02:00
MichaelEischer
bd3e280f6d
Merge pull request #2546 from Summerdave/fix-backup-error-code 
Return an error when errors occured during backup
 2020-07-20 22:31:46 +02:00
David Sommer
5729d967f5 backup: Return exit status code 3 when failing to read source data 
The backup command used to return a zero exit code as long as a snapshot
could be created successfully, even if some of the source files could not
be read (in which case the snapshot would contain the rest of the files).

This made it hard for automation/scripts to detect failures/incomplete
backups by looking at the exit code. Restic now returns the following exit
codes for the backup command:

 - 0 when the command was successful
 - 1 when there was a fatal error (no snapshot created)
 - 3 when some source data could not be read (incomplete snapshot created)
 2020-07-20 22:19:01 +02:00
Alexander Neumann
f9f6124558 Update chunker 2020-07-20 22:07:02 +02:00
greatroar
7bda28f31f Chaining hash table for repository.Index 
These are faster to construct but slower to access. The allocation rate
is halved, the peak memory usage almost halved compared to standard map.

Benchmark results on linux/amd64, -benchtime=3s -count=20:

name                                            old time/op    new time/op    delta
PackerManager-8                                    178ms ± 0%     178ms ± 0%     ~     (p=0.231 n=20+20)
DecodeIndex-8                                      4.54s ± 0%     4.30s ± 0%   -5.20%  (p=0.000 n=18+17)
DecodeIndexParallel-8                              4.54s ± 0%     4.30s ± 0%   -5.22%  (p=0.000 n=19+18)
IndexHasUnknown-8                                 44.4ns ± 5%    50.5ns ±11%  +13.82%  (p=0.000 n=19+17)
IndexHasKnown-8                                   48.3ns ± 0%    51.5ns ±12%   +6.68%  (p=0.001 n=16+20)
IndexAlloc-8                                       758ms ± 1%     616ms ± 1%  -18.69%  (p=0.000 n=19+19)
IndexAllocParallel-8                               234ms ± 3%     204ms ± 2%  -12.60%  (p=0.000 n=20+18)
MasterIndexLookupSingleIndex-8                     122ns ± 0%     145ns ± 9%  +18.44%  (p=0.000 n=14+20)
MasterIndexLookupMultipleIndex-8                   369ns ± 2%     429ns ± 8%  +16.27%  (p=0.000 n=20+20)
MasterIndexLookupSingleIndexUnknown-8             68.4ns ± 5%    74.9ns ±13%   +9.47%  (p=0.000 n=20+20)
MasterIndexLookupMultipleIndexUnknown-8            315ns ± 3%     369ns ±11%  +17.14%  (p=0.000 n=20+20)
MasterIndexLookupParallel/known,indices=5-8        743ns ± 1%     816ns ± 2%   +9.87%  (p=0.000 n=17+17)
MasterIndexLookupParallel/unknown,indices=5-8      238ns ± 1%     260ns ± 2%   +9.14%  (p=0.000 n=19+20)
MasterIndexLookupParallel/known,indices=10-8      1.01µs ± 3%    1.11µs ± 2%   +9.79%  (p=0.000 n=19+20)
MasterIndexLookupParallel/unknown,indices=10-8     222ns ± 0%     269ns ± 2%  +20.83%  (p=0.000 n=16+20)
MasterIndexLookupParallel/known,indices=20-8      1.06µs ± 2%    1.19µs ± 2%  +12.95%  (p=0.000 n=19+18)
MasterIndexLookupParallel/unknown,indices=20-8     413ns ± 1%     530ns ± 1%  +28.19%  (p=0.000 n=18+20)
SaveAndEncrypt-8                                  30.2ms ± 1%    30.4ms ± 0%   +0.71%  (p=0.000 n=19+19)
LoadTree-8                                         540µs ± 1%     576µs ± 1%   +6.73%  (p=0.000 n=20+20)
LoadBlob-8                                        5.64ms ± 0%    5.64ms ± 0%     ~     (p=0.883 n=18+17)
LoadAndDecrypt-8                                  5.93ms ± 0%    5.95ms ± 1%     ~     (p=0.247 n=20+19)
LoadIndex-8                                       25.1ms ± 0%    24.5ms ± 1%   -2.54%  (p=0.000 n=18+17)

name                                            old speed      new speed      delta
PackerManager-8                                  296MB/s ± 0%   296MB/s ± 0%     ~     (p=0.229 n=20+20)
SaveAndEncrypt-8                                 139MB/s ± 1%   138MB/s ± 0%   -0.71%  (p=0.000 n=19+19)
LoadBlob-8                                       177MB/s ± 0%   177MB/s ± 0%     ~     (p=0.890 n=18+17)
LoadAndDecrypt-8                                 169MB/s ± 0%   168MB/s ± 1%     ~     (p=0.227 n=20+19)

name                                            old alloc/op   new alloc/op   delta
PackerManager-8                                   91.8kB ± 0%    91.8kB ± 0%     ~     (p=0.772 n=12+19)
IndexAlloc-8                                       786MB ± 0%     400MB ± 0%  -49.04%  (p=0.000 n=20+18)
IndexAllocParallel-8                               786MB ± 0%     401MB ± 0%  -49.04%  (p=0.000 n=19+15)
SaveAndEncrypt-8                                  21.0MB ± 0%    21.0MB ± 0%   +0.00%  (p=0.000 n=19+19)

name                                            old allocs/op  new allocs/op  delta
PackerManager-8                                    1.41k ± 0%     1.41k ± 0%     ~     (all equal)
IndexAlloc-8                                        977k ± 0%      907k ± 0%   -7.18%  (p=0.000 n=20+20)
IndexAllocParallel-8                                977k ± 0%      907k ± 0%   -7.17%  (p=0.000 n=19+15)
SaveAndEncrypt-8                                    73.0 ± 0%      73.0 ± 0%     ~     (all equal)
 2020-07-19 13:58:22 +02:00
greatroar
4cf1c8e8da Changelog entry for subquadratic reading in mount 2020-07-12 18:27:16 +02:00
Alexander Weiss
b5543cff5d Loop over index files for 'list blobs' 
=> reduces memory consumption a lot!
 2020-06-14 17:25:34 +02:00
MichaelEischer
74bc7141c1
Merge pull request #2781 from aawsome/reduce-index-memory 
Reduce index memory
 2020-06-14 10:58:28 +02:00