octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-06-10 21:02:23 +00:00
Code Issues Releases Wiki Activity
5,747 Commits 7 Branches 33 Tags 65 MiB
9b57fcc6b0
Commit Graph

5747 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
MichaelEischer
f2a3b3b4a1
Merge pull request #3537 from greatroar/dump-lru 
Check cap instead of len in bloblru
 2021-10-09 20:00:27 +02:00
MichaelEischer
58e8b34633
Merge pull request #3532 from garrmcnu/s3-credentials-config 
s3: Add warning if key ID or secret is empty
 2021-10-09 19:32:32 +02:00
MichaelEischer
a02cea6e83
Merge pull request #3539 from jnoxon/fix-ec2-metadata-v2 
get iam roles working with ec2 metadata v2
 2021-10-07 23:00:27 +02:00
Garry McNulty
708d7a2574 s3: Add warning if key ID or secret is empty 
Also add debug message if no credential types are available.

Closes #2388
 2021-10-06 23:13:40 +01:00
Jeff Noxon
6f4b5ab8d1 get iam roles working with ec2 metadata v2 2021-10-03 20:06:55 -05:00
greatroar
634a9c162d Check cap instead of len in bloblru 
restic dump uses bloblru.Cache to keep buffers alive, but also reuses
evicted buffers. That means large buffers may be used to store small
blobs, causing the cache to think it's using less memory than it
actually does.
 2021-10-03 09:34:17 +02:00
jtagcat
632ca2ef52 docs/example: useradd: restic now system account 
and: use /sbin/nologin
and: use long flags in the useradd command
and: download v0.12.1 instead of v0.9.6
 2021-10-03 00:51:06 +03:00
MichaelEischer
24088f8307
Merge pull request #3528 from greatroar/tree-insert 
internal/restic: Don't allocate in Tree.Insert
 2021-09-27 22:22:03 +02:00
greatroar
c892c0bab9 internal/restic: Don't allocate in Tree.Insert 
name         old time/op    new time/op    delta
BuildTree-8    34.6µs ± 4%     7.0µs ± 3%  -79.68%  (p=0.000 n=18+19)

name         old alloc/op   new alloc/op   delta
BuildTree-8    34.0kB ± 0%     0.9kB ± 0%  -97.37%  (p=0.000 n=20+20)

name         old allocs/op  new allocs/op  delta
BuildTree-8       108 ± 0%         1 ± 0%  -99.07%  (p=0.000 n=20+20)
 2021-09-26 18:08:48 +02:00
MichaelEischer
78dac2fd48
Merge pull request #3523 from greatroar/swift-v2 
Upgrade ncw/swift to v2
 2021-09-24 21:41:36 +02:00
Michael Eischer
5ea8bba1a1 swift: restore context err check for list() 2021-09-24 21:19:46 +02:00
MichaelEischer
a5e103a212
Merge pull request #3522 from greatroar/dump-lru 
Use LRU cache in restic dump
 2021-09-24 20:33:58 +02:00
greatroar
e7ec0453b1 Upgrade ncw/swift to v2 2021-09-24 19:08:37 +02:00
Uli Martens
1ebcb1d097 Add changelog entry for PR #3508 2021-09-24 15:45:09 +02:00
greatroar
fe04d024c7 Use LRU cache in restic dump 2021-09-24 15:45:08 +02:00
Uli Martens
718966a81a Move Blobcache into dedicated internal package 2021-09-24 15:45:00 +02:00
greatroar
4f33eca634 Remove unused Writer arg to internal/dump.writeDump 2021-09-24 15:40:42 +02:00
MichaelEischer
cc110c42e6
Merge pull request #2657 from mansam/add-skip-tls-verification-flag 
Add --insecure-tls flag to disable SSL cert verification
 2021-09-22 21:40:01 +02:00
Sam Lucidi
897d8e662c Add --insecure-tls flag to disable SSL cert verification 
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
 2021-09-21 10:52:40 -04:00
cimnine
4a95af5290 GitHub action for Docker build 2021-09-21 08:23:33 +02:00
cimnine
f28c8bc1c2 Multistage Docker build 2021-09-21 08:23:33 +02:00
MichaelEischer
1827b16ade
Merge pull request #3519 from greatroar/maphash 
Replace siphash by hash/maphash
 2021-09-19 19:46:03 +02:00
greatroar
8b758c78a3 Require Go 1.14 to build 2021-09-19 16:18:19 +02:00
greatroar
8d2996eaaa Replace siphash by hash/maphash 
In Go 1.17.1, maphash has become quite a bit faster than siphash, so we
can drop one third-party dependency. maphash is just an interface to the
standard Go map's hash function, which we already trust for other use
cases.

Benchmark results on linux/amd64, -benchtime=3s:

name                                             old time/op    new time/op    delta
IndexHasUnknown-8                                  50.6ns ±10%    41.0ns ±19%  -18.92%  (p=0.000 n=9+10)
IndexHasKnown-8                                    52.6ns ±12%    41.5ns ±12%  -21.13%  (p=0.000 n=9+10)
IndexMapHash-8                                     3.64µs ± 1%    2.00µs ± 0%  -45.09%  (p=0.000 n=10+9)
IndexAlloc-8                                        700ms ± 1%     601ms ± 6%  -14.18%  (p=0.000 n=8+10)
IndexAllocParallel-8                                205ms ± 5%     192ms ± 8%   -6.18%  (p=0.043 n=10+10)
MasterIndexAlloc-8                                  319ms ± 1%     279ms ± 5%  -12.58%  (p=0.000 n=10+10)
MasterIndexLookupSingleIndex-8                      156ns ± 8%     147ns ± 6%   -5.46%  (p=0.023 n=10+10)
MasterIndexLookupMultipleIndex-8                    150ns ± 7%     142ns ± 8%   -5.69%  (p=0.007 n=10+10)
MasterIndexLookupSingleIndexUnknown-8              74.4ns ± 6%    72.0ns ± 9%     ~     (p=0.175 n=10+9)
MasterIndexLookupMultipleIndexUnknown-8            67.4ns ± 9%    65.5ns ± 7%     ~     (p=0.340 n=9+9)
MasterIndexLookupParallel/known,indices=25-8        461ns ± 2%     445ns ± 2%   -3.49%  (p=0.000 n=10+10)
MasterIndexLookupParallel/unknown,indices=25-8      408ns ±11%     378ns ± 5%   -7.22%  (p=0.035 n=10+9)
MasterIndexLookupParallel/known,indices=50-8        479ns ± 1%     437ns ± 4%   -8.82%  (p=0.000 n=10+10)
MasterIndexLookupParallel/unknown,indices=50-8      406ns ± 8%     343ns ±15%  -15.44%  (p=0.001 n=10+10)
MasterIndexLookupParallel/known,indices=100-8       480ns ± 1%     455ns ± 5%   -5.15%  (p=0.000 n=8+10)
MasterIndexLookupParallel/unknown,indices=100-8     391ns ±18%     382ns ± 8%     ~     (p=0.315 n=10+10)
MasterIndexLookupBlobSize-8                        71.0ns ± 8%    57.2ns ±11%  -19.36%  (p=0.000 n=9+10)
PackerManager-8                                     254ms ± 1%     254ms ± 1%     ~     (p=0.285 n=15+15)

name                                             old speed      new speed      delta
IndexMapHash-8                                   1.12GB/s ± 1%  2.05GB/s ± 0%  +82.13%  (p=0.000 n=10+9)
PackerManager-8                                   208MB/s ± 1%   207MB/s ± 1%     ~     (p=0.281 n=15+15)

name                                             old alloc/op   new alloc/op   delta
IndexMapHash-8                                      0.00B          0.00B          ~     (all equal)
IndexAlloc-8                                        400MB ± 0%     400MB ± 0%     ~     (p=1.000 n=9+10)
IndexAllocParallel-8                                401MB ± 0%     401MB ± 0%   +0.00%  (p=0.000 n=10+10)
MasterIndexAlloc-8                                  258MB ± 0%     262MB ± 0%   +1.42%  (p=0.000 n=9+10)
PackerManager-8                                    73.1kB ± 0%    73.1kB ± 0%     ~     (p=0.382 n=13+13)

name                                             old allocs/op  new allocs/op  delta
IndexMapHash-8                                       0.00           0.00          ~     (all equal)
IndexAlloc-8                                         907k ± 0%      907k ± 0%   -0.00%  (p=0.000 n=10+10)
IndexAllocParallel-8                                 907k ± 0%      907k ± 0%   +0.00%  (p=0.009 n=10+10)
MasterIndexAlloc-8                                   327k ± 0%      317k ± 0%   -3.06%  (p=0.000 n=10+10)
PackerManager-8                                       744 ± 0%       744 ± 0%     ~     (all equal)
 2021-09-19 16:05:18 +02:00
MichaelEischer
58efe21eca
Merge pull request #3264 from amozoss/upstream-master 
Refactor backup progress
 2021-09-19 14:54:42 +02:00
MichaelEischer
71fcf48533
Merge pull request #2594 from greatroar/concurrent-restore-verify 
Concurrent restore --verify
 2021-09-19 14:52:31 +02:00
Michael Eischer
921e328b56 restore: Fix linting error 2021-09-19 14:41:07 +02:00
Michael Eischer
e62d4f622f Reword changelog 2021-09-19 14:41:07 +02:00
Michael Eischer
2cdc0719af restorer: Sanitize verify errors 2021-09-19 14:01:26 +02:00
greatroar
bdcdfaf6b4 restore --verify: buffer reuse consistency and comment 2021-09-19 13:11:27 +02:00
greatroar
2b94742ca5 Replace no-op closures in restorer by nil check 2021-09-19 13:11:16 +02:00
greatroar
d357744104 Handle canceled context in restore --verify properly 2021-09-19 13:11:05 +02:00
greatroar
d4225ec803 Simplify buffer growing in Restorer.verifyFile 
Suggested-by: Igor Fedorenko <igor@ifedorenko.com>
 2021-09-19 13:10:55 +02:00
greatroar
de8521ae56 Report number of successes from Restorer.VerifyFiles 
Previously, the number of attempts was reported.
 2021-09-19 13:10:44 +02:00
greatroar
bb066cf7d3 Concurrent Restorer.VerifyFiles 
Time to verify a 2GB snapshot down from 9.726s to 4.645s (-52%).
 2021-09-19 13:10:41 +02:00
greatroar
556424d61b Reuse buffer in Restorer.VerifyFiles 
Time to verify a 2GB snapshot down from 11.568s to 9.726s (-16%).
 2021-09-19 12:55:31 +02:00
greatroar
92ae951ffa Report timing from restic restore --verify 2021-09-19 12:53:09 +02:00
greatroar
973fa921cb Test and document Restorer.VerifyFiles 2021-09-19 12:52:11 +02:00
Michael Eischer
e0d615c264 archiver: Fix TestArchiverAbortEarlyOnError test 
This can be caused when the test has uploaded four blobs, then queues
two blobs for upload which are delayed. Then a seventh file can be
opened which lead to a test failure.
 2021-09-12 22:17:17 +02:00
MichaelEischer
ef5672a902
Merge pull request #3509 from ajaspers/patch-1 
Update instructions for reproducing build
 2021-09-12 21:40:37 +02:00
ajaspers
c0eddc9969 Update instructions for reproducing build 
Dependencies are fetched at build time and stored in the GOPATH. These paths end up being in the final binary.

Bump restic version to latest and go version to the 1.16.6, which was used to build restic 0.12.1.
 2021-09-12 09:02:57 -07:00
Michael Eischer
fbb0e6499a ui: consolidate backup ui in ui/backup package 2021-09-12 16:20:15 +02:00
Michael Eischer
503d4c3e2f Add changelog 2021-09-12 16:00:49 +02:00
Michael Eischer
cccb0d4064 ui: assert that ProgressPrinter interface is implemented 2021-09-12 15:26:40 +02:00
Michael Eischer
a144c986f2 backup: Reenable JSON status updates with redirected output 
After the refactoring status updates were no longer printed in quiet
mode or when the output is not an interactive terminal. However, the
JSON output is often piped to e.g. another program. Thus, don't set the
update frequency to 0 in that case. The status updates are still
disabled for backup --quiet.

This also reduces the status update frequency to 60fps compared to a
potentially much higher value before the refactoring.
 2021-09-12 15:26:40 +02:00
Michael Eischer
d62bfed65d ui: move SetDryRun to ProgressReporter 2021-09-12 15:25:58 +02:00
Michael Eischer
77b129ec74 ui: remove unused Summary field 2021-09-12 15:25:58 +02:00
Michael Eischer
3024239e40 ui/json: restore summary output 2021-09-12 15:25:58 +02:00
Michael Eischer
5ccf583b8a ui: restore a few comments 2021-09-12 15:25:58 +02:00
Michael Eischer
80cbaf6d38 ui: Simplify ReportTotal methods 2021-09-12 15:25:58 +02:00