octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-12-28 04:56:04 +00:00
Code Issues Releases Wiki Activity
4,589 Commits 7 Branches 33 Tags 66 MiB
758b44b9c0
Commit Graph

250 Commits

Author SHA1 Message Date
Peter Schultz
758b44b9c0 gs: support authentication with access token 
In the Google Cloud Storage backend, support specifying access tokens
directly, as an alternative to a credentials file. This is useful when
restic is used non-interactively by some other program that is already
authenticated and eliminates the need to store long lived credentials.

The access token is specified in the GOOGLE_ACCESS_TOKEN environment
variable and takes precedence over GOOGLE_APPLICATION_CREDENTIALS.
 2020-07-22 16:23:03 +02:00
MichaelEischer
82c908871d
Merge pull request #2812 from greatroar/chaining 
Chaining hash table for repository.Index
 2020-07-20 23:29:36 +02:00
MichaelEischer
bd3e280f6d
Merge pull request #2546 from Summerdave/fix-backup-error-code 
Return an error when errors occured during backup
 2020-07-20 22:31:46 +02:00
David Sommer
5729d967f5 backup: Return exit status code 3 when failing to read source data 
The backup command used to return a zero exit code as long as a snapshot
could be created successfully, even if some of the source files could not
be read (in which case the snapshot would contain the rest of the files).

This made it hard for automation/scripts to detect failures/incomplete
backups by looking at the exit code. Restic now returns the following exit
codes for the backup command:

 - 0 when the command was successful
 - 1 when there was a fatal error (no snapshot created)
 - 3 when some source data could not be read (incomplete snapshot created)
 2020-07-20 22:19:01 +02:00
Alexander Neumann
f9f6124558 Update chunker 2020-07-20 22:07:02 +02:00
greatroar
7bda28f31f Chaining hash table for repository.Index 
These are faster to construct but slower to access. The allocation rate
is halved, the peak memory usage almost halved compared to standard map.

Benchmark results on linux/amd64, -benchtime=3s -count=20:

name                                            old time/op    new time/op    delta
PackerManager-8                                    178ms ± 0%     178ms ± 0%     ~     (p=0.231 n=20+20)
DecodeIndex-8                                      4.54s ± 0%     4.30s ± 0%   -5.20%  (p=0.000 n=18+17)
DecodeIndexParallel-8                              4.54s ± 0%     4.30s ± 0%   -5.22%  (p=0.000 n=19+18)
IndexHasUnknown-8                                 44.4ns ± 5%    50.5ns ±11%  +13.82%  (p=0.000 n=19+17)
IndexHasKnown-8                                   48.3ns ± 0%    51.5ns ±12%   +6.68%  (p=0.001 n=16+20)
IndexAlloc-8                                       758ms ± 1%     616ms ± 1%  -18.69%  (p=0.000 n=19+19)
IndexAllocParallel-8                               234ms ± 3%     204ms ± 2%  -12.60%  (p=0.000 n=20+18)
MasterIndexLookupSingleIndex-8                     122ns ± 0%     145ns ± 9%  +18.44%  (p=0.000 n=14+20)
MasterIndexLookupMultipleIndex-8                   369ns ± 2%     429ns ± 8%  +16.27%  (p=0.000 n=20+20)
MasterIndexLookupSingleIndexUnknown-8             68.4ns ± 5%    74.9ns ±13%   +9.47%  (p=0.000 n=20+20)
MasterIndexLookupMultipleIndexUnknown-8            315ns ± 3%     369ns ±11%  +17.14%  (p=0.000 n=20+20)
MasterIndexLookupParallel/known,indices=5-8        743ns ± 1%     816ns ± 2%   +9.87%  (p=0.000 n=17+17)
MasterIndexLookupParallel/unknown,indices=5-8      238ns ± 1%     260ns ± 2%   +9.14%  (p=0.000 n=19+20)
MasterIndexLookupParallel/known,indices=10-8      1.01µs ± 3%    1.11µs ± 2%   +9.79%  (p=0.000 n=19+20)
MasterIndexLookupParallel/unknown,indices=10-8     222ns ± 0%     269ns ± 2%  +20.83%  (p=0.000 n=16+20)
MasterIndexLookupParallel/known,indices=20-8      1.06µs ± 2%    1.19µs ± 2%  +12.95%  (p=0.000 n=19+18)
MasterIndexLookupParallel/unknown,indices=20-8     413ns ± 1%     530ns ± 1%  +28.19%  (p=0.000 n=18+20)
SaveAndEncrypt-8                                  30.2ms ± 1%    30.4ms ± 0%   +0.71%  (p=0.000 n=19+19)
LoadTree-8                                         540µs ± 1%     576µs ± 1%   +6.73%  (p=0.000 n=20+20)
LoadBlob-8                                        5.64ms ± 0%    5.64ms ± 0%     ~     (p=0.883 n=18+17)
LoadAndDecrypt-8                                  5.93ms ± 0%    5.95ms ± 1%     ~     (p=0.247 n=20+19)
LoadIndex-8                                       25.1ms ± 0%    24.5ms ± 1%   -2.54%  (p=0.000 n=18+17)

name                                            old speed      new speed      delta
PackerManager-8                                  296MB/s ± 0%   296MB/s ± 0%     ~     (p=0.229 n=20+20)
SaveAndEncrypt-8                                 139MB/s ± 1%   138MB/s ± 0%   -0.71%  (p=0.000 n=19+19)
LoadBlob-8                                       177MB/s ± 0%   177MB/s ± 0%     ~     (p=0.890 n=18+17)
LoadAndDecrypt-8                                 169MB/s ± 0%   168MB/s ± 1%     ~     (p=0.227 n=20+19)

name                                            old alloc/op   new alloc/op   delta
PackerManager-8                                   91.8kB ± 0%    91.8kB ± 0%     ~     (p=0.772 n=12+19)
IndexAlloc-8                                       786MB ± 0%     400MB ± 0%  -49.04%  (p=0.000 n=20+18)
IndexAllocParallel-8                               786MB ± 0%     401MB ± 0%  -49.04%  (p=0.000 n=19+15)
SaveAndEncrypt-8                                  21.0MB ± 0%    21.0MB ± 0%   +0.00%  (p=0.000 n=19+19)

name                                            old allocs/op  new allocs/op  delta
PackerManager-8                                    1.41k ± 0%     1.41k ± 0%     ~     (all equal)
IndexAlloc-8                                        977k ± 0%      907k ± 0%   -7.18%  (p=0.000 n=20+20)
IndexAllocParallel-8                                977k ± 0%      907k ± 0%   -7.17%  (p=0.000 n=19+15)
SaveAndEncrypt-8                                    73.0 ± 0%      73.0 ± 0%     ~     (all equal)
 2020-07-19 13:58:22 +02:00
greatroar
4cf1c8e8da Changelog entry for subquadratic reading in mount 2020-07-12 18:27:16 +02:00
Alexander Weiss
b5543cff5d Loop over index files for 'list blobs' 
=> reduces memory consumption a lot!
 2020-06-14 17:25:34 +02:00
MichaelEischer
74bc7141c1
Merge pull request #2781 from aawsome/reduce-index-memory 
Reduce index memory
 2020-06-14 10:58:28 +02:00
Alexander Weiss
7419844885 add changelog, benchmark, memory calculation 2020-06-14 07:50:15 +02:00
rawtaz
1d66bb9e62
Merge pull request #2784 from MichaelEischer/minio-sha256-changelog 
Add changelog for #2709
 2020-06-13 18:33:54 +02:00
Michael Eischer
0b2c31b05b Add changelog for #2709 2020-06-13 18:13:36 +02:00
Alexander Weiss
70347e95d5 disable index uploads for prune command 
+ modifications of changelog
 2020-06-12 09:24:38 +02:00
Alexander Weiss
91906911b0 Fix non-intuitive repository behavior 
- The SaveBlob method now checks for duplicates.
- Moves handling of pending blobs to MasterIndex.
  -> also cleans up pending index entries when they are saved in the index
  -> when using SaveBlob no need to care about index any longer
- Always check for full index and save it when storing packs.
  -> removes the need of an index uploader
  -> also removes the verbose "uploaded intermediate index" messages
- The Flush method now also saves the index
- Fix race condition when checking and saving full/non-finalized indexes
 2020-06-11 13:05:23 +02:00
Martin Michlmayr
5cc1760fdf
Fix typos 2020-05-16 14:05:26 +08:00
MichaelEischer
9790d8ce1c
Merge pull request #2668 from MichaelEischer/fix-stats-blobs-crash 
stats: Fix crash in blobs-per-file mode on missing blob
 2020-04-18 17:21:26 +02:00
Michael Eischer
4a400f94bb Add changelog 2020-04-12 22:27:09 +02:00
Michael Eischer
0ce81d88b6 stats: Fix crash in blobs-per-file mode on missing blob 
In a damaged repository with a missing blob, the error message tried to
dereference the subtreeID field of the current node, which is a file
however. Said field is set to nil for a file thus causing a segfault
when dereferenced.

Fix this by using the actual parentTreeID.
 2020-03-27 22:17:54 +01:00
rawtaz
cec7d581f3
Merge pull request #2610 from restic/remove-vendor 
Remove vendored dependencies
 2020-03-01 20:22:45 +01:00
Alexander Neumann
0c03a80fc4
Merge pull request #2592 from greatroar/sftp-ipv6 
Support IPv6 in SFTP backend
 2020-03-01 19:38:44 +01:00
Alexander Neumann
b1c77172c2 Add entry to changelog 2020-03-01 19:32:35 +01:00
greatroar
60e4a88f17 Changelog entry for SFTP w/ IPv6 addresses 2020-03-01 11:04:06 +01:00
greatroar
5cd0bce452 Honor RESTIC_CACHE_DIR on Mac and Windows 2020-02-28 15:44:32 +01:00
rawtaz
58bd165253
Merge pull request #2581 from aawsome/multiple-hostnames 
Allow multiple hostnames tags
 2020-02-27 08:35:23 +01:00
Alexander Weiss
9a9101d144 Support specifying multiple host flags for various commands 
The `dump`, `find`, `forget`, `ls`, `mount`, `restore`, `snapshots`,
`stats` and `tag` commands will now take into account multiple
`--host` and `-H` flags.
 2020-02-26 22:17:59 +01:00
Alexander Neumann
a40ac37550 Add changelog file 2020-02-26 20:55:12 +01:00
Igor Fedorenko
f17ffa0283 restorer: Allow writing target file blobs out of order 
Much simpler implementation that guarantees each required pack
is downloaded only once (and hence does not need to manage
pack cache). Also improves large file restore performance.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2020-02-26 16:14:45 +01:00
David Potter
71900248ab Correct #2531 (stats restore-size calculates hard links incorrectly) 2020-02-15 19:34:22 +01:00
David Potter
23055aaadf Correct #2537 (cmd_stats file counting issue) 2020-02-15 19:34:16 +01:00
curiousleo
27d6e5e186 Add changelog file. 2020-02-15 01:32:43 +01:00
Michael Eischer
294ca55967 Add changelog entry 2020-02-13 21:14:20 +01:00
rawtaz
680a14afa1
Merge pull request #2530 from restic/fix-sftp-mkdirall 
sftp: Use MkdirAll provided by the client
 2020-02-13 01:12:59 +01:00
rawtaz
bf199e5ca7
Minor speling corection. 2020-02-13 00:47:09 +01:00
Alexander Neumann
b58dfda212 Add entry to changelog 2020-02-12 21:29:26 +01:00
Daniel Hoffend
e7cdf2acbb fix backup --json total_bytes_processed output 
Closes #2429
 2020-02-09 02:02:45 +01:00
Alexander Neumann
6e9778ae0e Add changelog file 2020-01-06 21:46:22 +01:00
Alexander Neumann
6ec5dc8016 Fix template for new version of calens 
The `wrap` function has been renamed to `wrapIndent`
 2020-01-01 11:58:21 +01:00
streambinder
c56cbfe95b changelog: add pull-2423 2019-12-19 13:15:37 +01:00
Lorenz Bausch
d818b7618b
Display respective snapshot date when using restic find 2019-12-17 07:51:12 +01:00
Matt Holt
1c3812a6f6
Merge pull request #2342 from rpsene/master 
Add support for ppc64le
 2019-12-16 11:50:52 -07:00
Rafael Peria de Sene
ec91b80f09 Add support for ppc64le 
This commit adds support for cross-compiling Restic for ppc64le
as part of the release process.

Add a new file:   changelog/unreleased/issue-2277
Modifies:   helpers/build-release-binaries/main.go
Modifies:   run_integration_tests.go
 2019-12-15 22:52:13 -03:00
Eric Hamilton
d21a13f1ee Create issue-2390 2019-12-10 23:20:18 +01:00
Alexander Neumann
fe0361ffcb Move changelog entry for #2179 
For background on how this happened see
https://forum.restic.net/t/psa-attention-anyone-not-using-a-recent-master-ie-0-9-5-version-restic-will-not-detect-changed-files-if-mtime-is-reset-by-the-application-eg-ms-excel-and-xls-xlsx-files-on-macos/1772/8?u=fd0
 2019-11-24 15:15:19 +01:00
rawtaz
952469473f
Merge pull request #2486 from rawtaz/changelog-gitkeep 
changelog: Add .gitkeep to persist unreleased/ folder
 2019-11-24 14:59:39 +01:00
rawtaz
02108f202e
Merge pull request #2487 from rawtaz/2469-fix-diff-stats 
diff: Fix wrong bytes reported in diff stats (#2469)
 2019-11-23 12:15:28 +01:00
Leo R. Lundgren
c0a5530950 changelog: Fix typo in TEMPLATE. 2019-11-22 17:32:56 +01:00
Leo R. Lundgren
77ef92b95c diff: Fix wrong bytes reported in diff stats (#2469) 2019-11-22 17:26:01 +01:00
Leo R. Lundgren
69f75bbf70 changelog: Add .gitkeep to persist the unreleased/ folder. 2019-11-22 17:14:17 +01:00
Alexander Neumann
29b7b17491 Prepare changelog files for 0.9.6 2019-11-22 16:18:33 +01:00
Alexander Neumann
fb95426f64 Rename environment variable to AWS_DEFAULT_REGION 
This seems to be the correct name, at least the AWS cli uses it:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
 2019-11-22 15:24:42 +01:00