octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-11-16 01:57:10 +00:00
Code Issues Releases Wiki Activity
4,770 Commits 7 Branches 33 Tags 66 MiB
1557c58eef
Commit Graph

4770 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
greatroar
9f7cd69f13 Move Index.FindBlob to tests 2020-04-29 10:57:01 +02:00
greatroar
f97a680887 Fix repository benchmarks 
BenchmarkLoad{AndDecrypt,Blob} were spending between 38% and 50% of
their time measuring SHA-256 performance in their checks.
 2020-04-28 07:57:29 +02:00
greatroar
42a3db05b0 Use Minio's optimized SHA-256 
internal/repository benchmarks on an Intel i7-3770k:

name               old speed      new speed       delta
PackerManager-8     209MB/s ± 1%    291MB/s ± 1%  +38.94%  (p=0.008 n=5+5)
SaveAndEncrypt-8    112MB/s ± 1%    135MB/s ± 1%  +20.25%  (p=0.008 n=5+5)
 2020-04-28 07:57:18 +02:00
MichaelEischer
070d43e290
Merge pull request #2704 from azak-azkaran/patch-1 
Update 030_preparing_a_new_repo.rst
 2020-04-23 20:48:34 +02:00
MichaelEischer
d4bd32a37e
Merge pull request #2640 from greatroar/simplify-loadblob-usage 
Simplify Repository.LoadBlob usage
 2020-04-23 20:23:35 +02:00
greatroar
e7d7b85d59 Merge Repository.{LoadBlob,loadBlob} 
Pushing the allocation logic down into the former loadBlob body means
that fewer allocations have to be performed:

name              old time/op    new time/op    delta
LoadTree-8           478µs ± 1%     481µs ± 2%    ~     (p=0.315 n=9+10)
LoadBlob-8          11.6ms ± 1%    11.6ms ± 2%    ~     (p=0.393 n=10+10)
LoadAndDecrypt-8    13.3ms ± 3%    13.3ms ± 3%    ~     (p=0.905 n=10+9)
LoadIndex-8         33.6ms ± 2%    33.2ms ± 1%  -1.15%  (p=0.028 n=10+9)

name              old alloc/op   new alloc/op   delta
LoadTree-8          41.2kB ± 0%    41.1kB ± 0%  -0.23%  (p=0.000 n=10+10)
LoadBlob-8          2.28kB ± 0%    2.18kB ± 0%  -4.21%  (p=0.000 n=10+10)
LoadAndDecrypt-8    2.10MB ± 0%    2.10MB ± 0%    ~     (all equal)
LoadIndex-8         5.22MB ± 0%    5.22MB ± 0%    ~     (p=0.631 n=10+10)

name              old allocs/op  new allocs/op  delta
LoadTree-8             652 ± 0%       651 ± 0%  -0.15%  (p=0.000 n=10+10)
LoadBlob-8            24.0 ± 0%      23.0 ± 0%  -4.17%  (p=0.000 n=10+10)
LoadAndDecrypt-8      30.0 ± 0%      30.0 ± 0%    ~     (all equal)
LoadIndex-8          30.2k ± 0%     30.2k ± 0%    ~     (p=0.610 n=10+10)

name              old speed      new speed      delta
LoadBlob-8        86.4MB/s ± 1%  85.9MB/s ± 2%    ~     (p=0.393 n=10+10)
LoadAndDecrypt-8  75.4MB/s ± 3%  75.4MB/s ± 3%    ~     (p=0.858 n=10+9)
 2020-04-23 10:04:20 +02:00
greatroar
be5a0ff59f Centralize buffer allocation and size checking in Repository.LoadBlob 
Benchmark results for internal/repository:

name              old time/op    new time/op    delta
LoadTree-8           479µs ± 2%     478µs ± 1%   ~     (p=0.780 n=10+9)
LoadBlob-8          11.6ms ± 2%    11.6ms ± 1%   ~     (p=0.631 n=10+10)
LoadAndDecrypt-8    13.2ms ± 2%    13.3ms ± 3%   ~     (p=0.631 n=10+10)

name              old alloc/op   new alloc/op   delta
LoadTree-8          41.2kB ± 0%    41.2kB ± 0%   ~     (all equal)
LoadBlob-8          2.28kB ± 0%    2.28kB ± 0%   ~     (all equal)
LoadAndDecrypt-8    2.10MB ± 0%    2.10MB ± 0%   ~     (all equal)

name              old allocs/op  new allocs/op  delta
LoadTree-8             652 ± 0%       652 ± 0%   ~     (all equal)
LoadBlob-8            24.0 ± 0%      24.0 ± 0%   ~     (all equal)
LoadAndDecrypt-8      30.0 ± 0%      30.0 ± 0%   ~     (all equal)

name              old speed      new speed      delta
LoadBlob-8        86.2MB/s ± 2%  86.4MB/s ± 1%   ~     (p=0.594 n=10+10)
LoadAndDecrypt-8  75.7MB/s ± 2%  75.4MB/s ± 3%   ~     (p=0.617 n=10+10)
 2020-04-23 10:04:20 +02:00
azak-azkaran
c5100d5632
Update 030_preparing_a_new_repo.rst 
fixed markdown for wasabi region link
 2020-04-23 09:24:19 +02:00
MichaelEischer
956a1b0f96
Merge pull request #2294 from BenWiederhake/debugsyms 
build: Preserve debug symbols in debug build
 2020-04-19 17:26:03 +02:00
Ben Wiederhake
fab626a3df build: Preserve debug symbols in debug and profile build 
Signed-off-by: Ben Wiederhake <BenWiederhake.GitHub@gmx.de>
 2020-04-19 14:48:40 +02:00
MichaelEischer
4f00564574
Merge pull request #2621 from greatroar/fixes 
Some small fixes
 2020-04-18 18:07:08 +02:00
MichaelEischer
f77477129f
Merge pull request #2584 from greatroar/mount-cache-uid-gid 
Cache uid and gid for top directories in internal/fuse
 2020-04-18 17:45:14 +02:00
greatroar
2e31120f89 Remove unused argument to restic.fakeFile 2020-04-18 17:40:13 +02:00
greatroar
8fb2c0d3c1 Typo in crypto test name 2020-04-18 17:39:06 +02:00
greatroar
072cf7b02d Fix debug messages in internal/fuse 2020-04-18 17:39:06 +02:00
greatroar
df66daa5c9 Fix context usage in backend tests 
Found by go vet. This is also the only complaint is has.
 2020-04-18 17:39:06 +02:00
MichaelEischer
9790d8ce1c
Merge pull request #2668 from MichaelEischer/fix-stats-blobs-crash 
stats: Fix crash in blobs-per-file mode on missing blob
 2020-04-18 17:21:26 +02:00
MichaelEischer
16710454f4
Merge pull request #2628 from MichaelEischer/one-element-pack-lists 
cache: Don't sort one element pack lists
 2020-04-18 17:09:06 +02:00
MichaelEischer
08ec6c9f17
Merge pull request #2677 from MichaelEischer/complain-about-invalid-indexes 
rebuild_index: Report invalid packs that were ignored
 2020-04-18 16:57:50 +02:00
MichaelEischer
7910ff4c0e
Merge pull request #2648 from nairb774/iowritestring 
termstatus: Use io.WriteString to output messages.
 2020-04-18 13:48:42 +02:00
MichaelEischer
c4da9d1e90
Merge pull request #2638 from greatroar/no-close-in-packer 
Don't Close in Packer.Finalize
 2020-04-18 13:07:21 +02:00
rawtaz
3ed61987a2
Merge pull request #2695 from restic/rawtaz-doc-exclude-file-tilde 
doc: Add note on tilde expansion in exclude files
 2020-04-18 13:01:17 +02:00
rawtaz
9dba7a2577
doc: Add note on tilde expansion in exclude files 
Explains to the reader that tilde expansion does not work in exclude files, and that they should instead use the $HOME variable.
 2020-04-18 12:47:47 +02:00
MichaelEischer
a1352906e2
Merge pull request #2622 from greatroar/optimize-packer-manager 
Fix PackerManager benchmark and optimize hashing.Writer
 2020-04-18 12:46:34 +02:00
MichaelEischer
b7c0d4d8bf
Merge pull request #2644 from greatroar/signal-notify-buffered 
Make all signal.Notify channels buffered
 2020-04-18 11:31:02 +02:00
MichaelEischer
f033850aa0
Merge pull request #2692 from MichaelEischer/fix-archiveraborttest 
archiver: Fix race condition triggered by TestArchiverAbortEarlyOnError
 2020-04-13 18:36:48 +02:00
Michael Eischer
bdf7ba20cb archiver: Fix race condition triggered by TestArchiverAbortEarlyOnError 
The Save methods of the BlobSaver, FileSaver and TreeSaver return early
on when the archiver is stopped due to an error. For that they select on
both the tomb.Dying() and context.Done() channels, which can lead to a
race condition when the tomb is killed due to an error: The tomb first
closes its Dying channel before canceling all child contexts.
Archiver.SaveDir only aborts its execution once the context was
canceled. When the tomb killing is paused between closing its Dying
channel and canceling the child contexts, this lets the
FileSaver/TreeSaver.Save methods return immediately, however, ScanDir
still reads further files causing the test case to fail.

As a killed tomb always cancels all child contexts and as the Savers
always use a context bound to the tomb, it is sufficient to just use
context.Done() as escape hatch in the Save functions. This fixes the
mismatch between SaveDir and Save.

Adjust the tests to use contexts bound to the tomb for all interactions
with the Savers.
 2020-04-13 18:23:17 +02:00
rawtaz
3ee6b8ec63
Merge pull request #2689 from MichaelEischer/fix-background-hang 
Fix shutdown hang when restic is started as background job
 2020-04-12 22:39:44 +02:00
Michael Eischer
4a400f94bb Add changelog 2020-04-12 22:27:09 +02:00
Michael Eischer
1a1c572bac Fix shutdown hang when restic is started as background job 
restic uses a cleanup hook to ensure that it restores the terminal
configuration to a sane state, when restic is interrupted while reading
a password from the terminal. However, this causes a problem, when
restic runs in a background job, as reconfiguring a terminal will cause
a SIGTTOU to be sent to restic pausing it. Therefore, restic seems to
hang on shutdown when it was running in the background.

This commit changes the behavior to only restore the terminal
configuration if restic was interrupted while reading a password from
the terminal. As reading a password from the terminal requires that
restic is in the foreground, this should avoid restic getting stopped.

Fixes #2298
Issue introduced in #402
 2020-04-12 22:27:09 +02:00
rawtaz
5a7c27ddb6
Merge pull request #2681 from MichaelEischer/optimize-debug 
Reduce memory usage and startup time of debug command
 2020-04-04 00:08:14 +02:00
Michael Eischer
fb842759fc debug: don't load the repository index 
The existing commands don't need a loaded repository index which can
take several minutes to load on larger repositories.
 2020-04-04 00:01:01 +02:00
Michael Eischer
7aa2f8a61e debug: get stdout/stderr from gopts/globalOptions 2020-04-04 00:01:01 +02:00
Michael Eischer
08bf3bae79 debug: explicitly pass stdout to dump functions 2020-04-03 23:32:44 +02:00
rawtaz
e7b741b2d7
Merge pull request #2682 from MichaelEischer/cleanup-cli-paramter-names 
Cleanup CLI parameter names for backup / global flags
 2020-04-03 21:35:34 +02:00
Michael Eischer
6bee62e346 Update doc excerpts for --help 
This adds some previously missing changes and the new paramters names
from the previous commit.
 2020-04-03 19:49:06 +02:00
Michael Eischer
bc74cd3ae5 backup/global: Use proper name for command line argument parameters 
Several paramters printed a generic "string" or "stringArray" name.
 2020-04-03 19:49:04 +02:00
Michael Eischer
90243ed1c4 rebuild_index: Report invalid packs that were ignored 2020-04-02 22:38:31 +02:00
Michael Eischer
0ce81d88b6 stats: Fix crash in blobs-per-file mode on missing blob 
In a damaged repository with a missing blob, the error message tried to
dereference the subtreeID field of the current node, which is a file
however. Said field is set to nil for a file thus causing a segfault
when dereferenced.

Fix this by using the actual parentTreeID.
 2020-03-27 22:17:54 +01:00
rawtaz
c03bc88b29
Merge pull request #2669 from MichaelEischer/doc-cifs-linux-bug 
doc: Warn about compatibility issues with CIFS and restic
 2020-03-26 23:01:06 +01:00
Brian Atkinson
b8da7b1f4d termstatus: Use io.WriteString to output messages. 
The previous implementation was repeating the implementation that is
found inside of io.WriteString. Simplify by making use of the stdlib's
implementation.
 2020-03-26 14:55:00 -07:00
Michael Eischer
f1b4d97945 doc: Warn about compatibility issues with CIFS and restic 
On Linux CIFS (SMB) seems to be incompatible with the async preemption
implementation of Go 1.14. CIFS seems not to restart syscalls (open,
read, chmod, readdir, ...) as expected by Go, which sets SA_RESTART for
its signal handler to have syscalls restarted automatically. This leads
to Go passing up lots of EINTR return codes to restic.

See https://github.com/restic/restic/issues/2659 for a detailed explanation.
 2020-03-26 21:52:37 +01:00
Peter Schultz
90fc639a67 Allow specifying user and host when adding keys 
The username and hostname for new keys can be specified with the new
--user and --host flags, respectively. The flags are used only by the
`key add` command and are otherwise ignored.

This allows adding keys with for a desired user and host without having
to run restic as that particular user on that particular host, making
automated key management easier.

Co-authored-by: James TD Smith <ahktenzero@mohorovi.cc>
 2020-03-23 13:11:10 +01:00
rawtaz
2b5a6d255a
Merge pull request #2660 from restic/rawtaz-delete-old-issue-template 
Delete ISSUE_TEMPLATE.md (not used anymore)
 2020-03-21 20:48:39 +01:00
rawtaz
f004dbe605
Delete ISSUE_TEMPLATE.md (not used anymore) 
Nowadays the ISSUE_TEMPLATE/ directory and its files are used for this feature.
 2020-03-21 20:30:19 +01:00
rawtaz
9efbe98879
Merge pull request #2623 from alrs/internal-restic-err-before-close 
internal/restic: close os.File after checking for error
 2020-03-18 22:23:20 +01:00
rawtaz
7d9300efca
Merge pull request #2637 from greatroar/unused 
Remove Go 1.5 compatibility code
 2020-03-18 22:22:15 +01:00
rawtaz
c38aaaa768
Merge pull request #2652 from greatroar/upgrade-fuse 
Upgrade bazil.org/fuse and remove restic mount --allow-root
 2020-03-18 22:21:50 +01:00
greatroar
8941041355 Upgrade bazil.org/fuse to version that fixes FreeBSD opBmap bug 2020-03-18 10:00:39 +01:00
greatroar
18fee4806f Remove broken --allow-root from restic mount 2020-03-17 23:35:06 +01:00