octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-12-04 19:03:46 +00:00
Code Issues Releases Wiki Activity
5,548 Commits 7 Branches 33 Tags 66 MiB
ea04f40eb3
Commit Graph

5548 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael Eischer
4036991c91 Update links to help wanted labels in CONTRIBUTING.md and link to forum 2020-10-09 16:15:43 +02:00
Michael Eischer
88c8e903d2 filter: Fix glob matching on absolute path marker on windows 
A pattern part containing "/" is used to mark a path or a pattern as
absolute. However, on Windows the path separator is "\" such that glob
patterns like "?" could match the marker. The code now explicitly skips
the marker when the pattern does not represent an absolute path.
 2020-10-09 16:11:05 +02:00
greatroar
740758a5fa Optimize filter pattern matching 
By replacing "**" with "", checking for this special path component can
be reduced to a length-zero check.

name                          old time/op    new time/op    delta
FilterLines-8                   44.7ms ± 5%    44.9ms ± 5%     ~     (p=0.631 n=10+10)
FilterPatterns/Relative-8       13.6ms ± 4%    13.4ms ± 5%     ~     (p=0.165 n=10+10)
FilterPatterns/Absolute-8       10.9ms ± 5%    10.7ms ± 4%     ~     (p=0.052 n=10+10)
FilterPatterns/Wildcard-8       53.7ms ± 5%    50.4ms ± 5%   -6.00%  (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-8     128ms ± 2%      95ms ± 1%  -25.54%  (p=0.000 n=10+10)

name                          old alloc/op   new alloc/op   delta
FilterPatterns/Relative-8       3.57MB ± 0%    3.57MB ± 0%     ~     (p=1.000 n=9+8)
FilterPatterns/Absolute-8       3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.903 n=9+8)
FilterPatterns/Wildcard-8       19.7MB ± 0%    19.7MB ± 0%   -0.00%  (p=0.022 n=10+9)
FilterPatterns/ManyNoMatch-8    3.57MB ± 0%    3.57MB ± 0%     ~     (all equal)

name                          old allocs/op  new allocs/op  delta
FilterPatterns/Relative-8        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Absolute-8        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Wildcard-8        88.7k ± 0%     88.7k ± 0%     ~     (all equal)
FilterPatterns/ManyNoMatch-8     22.2k ± 0%     22.2k ± 0%     ~     (all equal)
 2020-10-09 15:51:14 +02:00
MichaelEischer
862ee4b2c9
Merge pull request #2970 from labkode/fopenskip 
Skip fopen for file change check to avoid network penalty
 2020-10-09 11:43:54 +02:00
Hugo Gonzalez Labrador
958dc6aafc Skip fopen for file change check to avoid network penalty 2020-10-09 10:53:28 +02:00
MichaelEischer
88cc444779
Merge pull request #2934 from MichaelEischer/upgrade-backoff 
Upgrade github.com/cenkalti/backoff module
 2020-10-08 20:33:30 +02:00
MichaelEischer
4a424af1d5
Merge pull request #2989 from MichaelEischer/remove-local-chmod 
local: mark repository files as read-only and handle chmod errors
 2020-10-08 19:04:13 +02:00
Michael Eischer
bc65da2baf Add changelog 2020-10-08 14:45:26 +02:00
Michael Eischer
b79f18209f Upgrade github.com/cenkalti/backoff module 
We now use v4 of the module. `backoff.WithMaxRetries` no longer repeats
an operation endlessly when a retry count of 0 is specified. This
required a few fixes for the tests.
 2020-10-07 22:04:59 +02:00
Michael Eischer
8388e67c4b filter: cleanup path separator conversion 2020-10-07 21:14:07 +02:00
Michael Eischer
0acc3c5923 filter: special case patterns without globbing characters 
In case a part of a path is a simple string, we can just check for
equality without complex parsing in filepath.Match.

name                          old time/op    new time/op    delta
FilterLines-4                   34.8ms ±17%    41.2ms ±23%  +18.36%  (p=0.000 n=10+10)
FilterPatterns/Relative-4       21.7ms ± 6%    12.1ms ±23%  -44.46%  (p=0.000 n=10+10)
FilterPatterns/Absolute-4       10.0ms ± 5%     9.1ms ±11%   -9.80%  (p=0.006 n=10+9)
FilterPatterns/Wildcard-4       47.0ms ± 7%    42.2ms ± 5%  -10.19%  (p=0.000 n=9+10)
FilterPatterns/ManyNoMatch-4     190ms ± 1%     131ms ±20%  -31.47%  (p=0.000 n=8+10)

name                          old alloc/op   new alloc/op   delta
FilterPatterns/Relative-4       3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.870 n=9+9)
FilterPatterns/Absolute-4       3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.145 n=10+10)
FilterPatterns/Wildcard-4       14.3MB ± 0%    19.7MB ± 0%  +37.91%  (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4    3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.421 n=10+9)

name                          old allocs/op  new allocs/op  delta
FilterPatterns/Relative-4        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Absolute-4        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Wildcard-4        88.7k ± 0%     88.7k ± 0%     ~     (all equal)
FilterPatterns/ManyNoMatch-4     22.2k ± 0%     22.2k ± 0%     ~     (all equal)
 2020-10-07 20:55:43 +02:00
Michael Eischer
54a124de3b filter: explicitly test separate ListWithChild function 2020-10-07 20:47:52 +02:00
Michael Eischer
bcc3bddcf4 filter: only check whether a child path could match when necessary 
When checking excludes there is no need to test whether a child path
could also match the pattern, as it is by definition excluded.
Previously childMayMatch was calculated but then discarded. For simple
absolute paths this can account for half the time spent for checking
pattern matches.

name                          old time/op    new time/op    delta
FilterPatterns/Relative-4       23.3ms ± 9%    21.7ms ± 6%   -6.68%  (p=0.004 n=10+10)
FilterPatterns/Absolute-4       13.9ms ± 7%    10.0ms ± 5%  -27.61%  (p=0.000 n=10+10)
FilterPatterns/Wildcard-4       51.4ms ± 7%    47.0ms ± 7%   -8.51%  (p=0.001 n=9+9)
FilterPatterns/ManyNoMatch-4     551ms ± 9%     190ms ± 1%  -65.41%  (p=0.000 n=10+8)

name                          old alloc/op   new alloc/op   delta
FilterPatterns/Relative-4       3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.665 n=10+9)
FilterPatterns/Absolute-4       3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.480 n=9+10)
FilterPatterns/Wildcard-4       14.3MB ± 0%    14.3MB ± 0%     ~     (p=0.431 n=9+10)
FilterPatterns/ManyNoMatch-4    3.57MB ± 0%    3.57MB ± 0%     ~     (all equal)

name                          old allocs/op  new allocs/op  delta
FilterPatterns/Relative-4        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Absolute-4        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Wildcard-4        88.7k ± 0%     88.7k ± 0%     ~     (all equal)
FilterPatterns/ManyNoMatch-4     22.2k ± 0%     22.2k ± 0%     ~     (all equal)
 2020-10-07 20:47:52 +02:00
Michael Eischer
17c53efb0d filter: Optimize double wildcard expansion 
This only allocates a single slice to expand the double wildcard and
only copies the pattern prefix once.

name                          old time/op    new time/op    delta
FilterPatterns/Relative-4       22.7ms ± 5%    23.3ms ± 9%     ~     (p=0.353 n=10+10)
FilterPatterns/Absolute-4       14.2ms ±13%    13.9ms ± 7%     ~     (p=0.853 n=10+10)
FilterPatterns/Wildcard-4        266ms ±16%      51ms ± 7%  -80.67%  (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4     554ms ± 6%     551ms ± 9%     ~     (p=0.436 n=10+10)

name                          old alloc/op   new alloc/op   delta
FilterPatterns/Relative-4       3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.349 n=10+10)
FilterPatterns/Absolute-4       3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.073 n=10+9)
FilterPatterns/Wildcard-4        141MB ± 0%      14MB ± 0%  -89.89%  (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4    3.57MB ± 0%    3.57MB ± 0%     ~     (all equal)

name                          old allocs/op  new allocs/op  delta
FilterPatterns/Relative-4        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Absolute-4        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Wildcard-4        1.63M ± 0%     0.09M ± 0%  -94.56%  (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4     22.2k ± 0%     22.2k ± 0%     ~     (all equal)
 2020-10-07 20:47:48 +02:00
Michael Eischer
7959796269 filter: Special case for absolute paths 
name                          old time/op    new time/op    delta
FilterPatterns/Relative-4       23.6ms ±20%    22.7ms ± 5%     ~     (p=0.684 n=10+10)
FilterPatterns/Absolute-4       32.3ms ± 8%    14.2ms ±13%  -56.01%  (p=0.000 n=10+10)
FilterPatterns/Wildcard-4        334ms ±17%     266ms ±16%  -20.56%  (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4     709ms ± 7%     554ms ± 6%  -21.89%  (p=0.000 n=10+10)

name                          old alloc/op   new alloc/op   delta
FilterPatterns/Relative-4       3.57MB ± 0%    3.57MB ± 0%   +0.00%  (p=0.046 n=9+10)
FilterPatterns/Absolute-4       3.57MB ± 0%    3.57MB ± 0%     ~     (p=0.464 n=10+10)
FilterPatterns/Wildcard-4        141MB ± 0%     141MB ± 0%     ~     (p=0.163 n=9+10)
FilterPatterns/ManyNoMatch-4    3.57MB ± 0%    3.57MB ± 0%     ~     (all equal)

name                          old allocs/op  new allocs/op  delta
FilterPatterns/Relative-4        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Absolute-4        22.2k ± 0%     22.2k ± 0%     ~     (all equal)
FilterPatterns/Wildcard-4        1.63M ± 0%     1.63M ± 0%     ~     (p=0.072 n=10+10)
FilterPatterns/ManyNoMatch-4     22.2k ± 0%     22.2k ± 0%     ~     (all equal)
 2020-10-07 20:47:29 +02:00
Michael Eischer
375c2a56de filter: Parse filter patterns only once 
name                          old time/op    new time/op    delta
FilterPatterns/Relative-4       30.3ms ±10%    23.6ms ±20%  -22.12%  (p=0.000 n=10+10)
FilterPatterns/Absolute-4       49.0ms ± 3%    32.3ms ± 8%  -33.94%  (p=0.000 n=8+10)
FilterPatterns/Wildcard-4        345ms ± 9%     334ms ±17%     ~     (p=0.315 n=10+10)
FilterPatterns/ManyNoMatch-4     3.93s ± 2%     0.71s ± 7%  -81.98%  (p=0.000 n=9+10)

name                          old alloc/op   new alloc/op   delta
FilterPatterns/Relative-4       4.63MB ± 0%    3.57MB ± 0%  -22.98%  (p=0.000 n=9+9)
FilterPatterns/Absolute-4       8.54MB ± 0%    3.57MB ± 0%  -58.20%  (p=0.000 n=10+10)
FilterPatterns/Wildcard-4        146MB ± 0%     141MB ± 0%   -2.93%  (p=0.000 n=9+9)
FilterPatterns/ManyNoMatch-4     907MB ± 0%       4MB ± 0%  -99.61%  (p=0.000 n=9+9)

name                          old allocs/op  new allocs/op  delta
FilterPatterns/Relative-4        66.6k ± 0%     22.2k ± 0%  -66.67%  (p=0.000 n=10+10)
FilterPatterns/Absolute-4        88.7k ± 0%     22.2k ± 0%  -75.00%  (p=0.000 n=10+10)
FilterPatterns/Wildcard-4        1.70M ± 0%     1.63M ± 0%   -3.92%  (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4     4.46M ± 0%     0.02M ± 0%  -99.50%  (p=0.000 n=10+10)
 2020-10-07 20:47:27 +02:00
Michael Eischer
b8eacd1364 filter: Reduce redundant path and pattern splitting 
A single call to filter.List will split the path only once and also
split each search pattern only once and use it for both match and
childMatch.

name                          old time/op    new time/op    delta
FilterPatterns/Relative-4       62.1ms ±15%    30.3ms ±10%  -51.22%  (p=0.000 n=9+10)
FilterPatterns/Absolute-4        111ms ±10%      49ms ± 3%  -56.08%  (p=0.000 n=10+8)
FilterPatterns/Wildcard-4        393ms ±15%     345ms ± 9%  -12.30%  (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4     10.0s ± 3%      3.9s ± 2%  -60.53%  (p=0.000 n=10+9)

name                          old alloc/op   new alloc/op   delta
FilterPatterns/Relative-4       16.4MB ± 0%     4.6MB ± 0%  -71.76%  (p=0.000 n=10+9)
FilterPatterns/Absolute-4       31.4MB ± 0%     8.5MB ± 0%  -72.77%  (p=0.000 n=9+10)
FilterPatterns/Wildcard-4        168MB ± 0%     146MB ± 0%  -13.19%  (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4    3.23GB ± 0%    0.91GB ± 0%  -71.96%  (p=0.000 n=10+9)

name                          old allocs/op  new allocs/op  delta
FilterPatterns/Relative-4         178k ± 0%       67k ± 0%  -62.50%  (p=0.000 n=10+10)
FilterPatterns/Absolute-4         266k ± 0%       89k ± 0%  -66.67%  (p=0.000 n=10+10)
FilterPatterns/Wildcard-4        1.87M ± 0%     1.70M ± 0%   -9.47%  (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4     17.7M ± 0%      4.5M ± 0%  -74.87%  (p=0.000 n=9+10)
 2020-10-07 18:13:19 +02:00
Michael Eischer
e73c281142 filter: Benchmark absolute paths, wildcards and long filter lists 2020-10-07 17:54:36 +02:00
Michael Eischer
fdd3b14db3 filter: test some corner cases 2020-10-07 17:09:44 +02:00
Michael Eischer
f4282aa6fd local: mark repository files as read-only 
This is intended to prevent accidental modifications of data files.
Marking the files as read-only was accidentally removed in #1258.
 2020-10-07 12:29:37 +02:00
Michael Eischer
40ee17167e local: Ignore permission errors on chmod call in Save/Remove operation 
The file is already created with the proper permissions, thus the chmod
call is not critical. However, some file systems have don't allow
modifications of the file permissions. Similarly the chmod call in the Remove
operation should not prevent it from working.
 2020-10-07 12:29:37 +02:00
greatroar
f3e933f0c1 Use windows.SetConsoleCursorPosition in ui/termstatus 2020-10-07 11:18:54 +02:00
MichaelEischer
0ae02f3030
Merge pull request #2844 from aawsome/prune-integration-tests 
Add prune integration tests for many edge cases
 2020-10-06 23:16:41 +02:00
rawtaz
e401859afb
Merge pull request #2992 from lbausch/repo_wording 
Replace repo/repos with repository/repositories in `--no-lock` help text
 2020-10-06 23:03:55 +02:00
Lorenz Bausch
af4100e07d
Replace repo/repos with repository/repositories in --no-lock help text 2020-10-06 22:49:48 +02:00
Alexander Neumann
ea5bbe0857 Remove dysfunctional link to saythanks 2020-10-06 20:43:44 +02:00
Alexander Weiss
6822a58413 Add prune integration tests for many edge cases 2020-10-06 20:20:05 +02:00
MichaelEischer
09d39e260d
Merge pull request #2980 from greatroar/error-wrapping 
Remove repetitive error wrapping from internal/cache
 2020-10-06 15:19:00 +02:00
Michael Eischer
1579d2a8ec Remove some unused assignments 2020-10-06 14:55:13 +02:00
Michael Eischer
eba5dd831f Fix typos reported by misspell 2020-10-06 14:55:13 +02:00
Michael Eischer
9ffb698c8d Modernize context import in internal/fuse 
Migrated using `go fix`.
 2020-10-06 14:55:13 +02:00
Michael Eischer
efbb850d92 Remove a few redundant type specifiers 
This is the result of running `gofmt -s -w **/*.go`
 2020-10-06 14:55:13 +02:00
MichaelEischer
cfd57c480a
Merge pull request #2991 from restic/nonsense 
cache/check: Clarify error message when given arguments
 2020-10-06 00:40:12 +02:00
Leo R. Lundgren
e105a3f391 cache/check: Clarify error message when given arguments 2020-10-06 00:08:59 +02:00
rawtaz
5d8cfff3f2
Merge pull request #2988 from greatroar/snapshots-json 
Regression test for #2979
 2020-10-05 21:02:52 +02:00
greatroar
673dda77c0 Less repetitive error messages in internal/cache 
Many instances of errors.Wrap in this package would produce messages
like "Open: open <filename>: no such file or directory"; those now omit
the first "Open:" (or "Stat:", or "MkdirAll"). The function readVersion
now appends its own name to the error message, rather than the function
that failed, to make it easier to spot. Other function names (e.g.,
Load) are already added further up in the call chain.
 2020-10-05 20:28:54 +02:00
MichaelEischer
1ab4c710e1
Merge pull request #2987 from greatroar/no-password-for-help 
Don't read password for generate, help or self-update
 2020-10-05 20:16:19 +02:00
greatroar
feedf0ebce Don't read password for generate, help or self-update 
Fixes #2951.
 2020-10-05 19:54:07 +02:00
MichaelEischer
1a490acd67
Merge pull request #2982 from greatroar/archiver-error-handling 
Check error in archiver before calling Select
 2020-10-05 19:19:47 +02:00
Michael Eischer
187518a8a3 docs: Properly reproduce zip archives for windows binaries 2020-10-05 18:24:46 +02:00
Michael Eischer
a232c833dc docs: Update binary reproduction instructions and refer to older instructions 2020-10-05 18:24:46 +02:00
MichaelEischer
c84643c6a9
Merge pull request #2981 from greatroar/mkdircachedir 
Simplify cache directory creation
 2020-10-05 18:15:23 +02:00
MichaelEischer
8390a8aaf3
Merge pull request #2986 from greatroar/upgrade-xxhash 
Upgrade cespare/xxhash to 2.1.1
 2020-10-05 17:49:08 +02:00
greatroar
1e1a1f3078 Upgrade cespare/xxhash to 2.1.1 2020-10-05 17:25:39 +02:00
MichaelEischer
abe9fa261f
Merge pull request #2983 from greatroar/upgrade-siphash 
Upgrade siphash to 1.2.2
 2020-10-05 17:18:07 +02:00
MichaelEischer
6001b45bf7
Merge pull request #2984 from restic/snapshots-json 
snapshots: Make --json output [] instead of null when no snapshots
 2020-10-05 17:14:23 +02:00
greatroar
6bee0aafc2 Regression test for #2979 2020-10-05 15:08:09 +02:00
greatroar
8252ea8e3d Upgrade siphash to 1.2.2 
New version is potentially faster on 32-bit ARM.
 2020-10-05 14:56:13 +02:00
Leo R. Lundgren
137d20a06a snapshots: Make --json output [] instead of null when no snapshots 2020-10-05 13:14:10 +02:00
greatroar
c4e2203e45 Check error in archiver before calling Select 
The archiver first called the Select function for a path before checking
whether the Lstat on that path actually worked. The RejectFuncs in
exclude.go worked around this by checking whether they received a nil
os.FileInfo. Checking first is more obvious and requires less code.
 2020-10-05 11:11:04 +02:00