octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-11-26 06:46:34 +00:00
Code Issues Releases Wiki Activity
8,690 Commits 7 Branches 33 Tags 66 MiB
9c758313e3
Commit Graph

2338 Commits

Author SHA1 Message Date
Michael Eischer
6ec2b62ec5 fuse: cache fs.Node instances 
A particular node should always be represented by a single instance.
This is necessary to allow the fuse library to assign a stable nodeId to
a node. macOS Sonoma trips over the previous, unstable behavior when
using fuse-t.
 2024-09-14 18:11:44 +02:00
Damien Clark
4795143d6d cache: fix race condition in cache cleanup 
Fix multiple restic processes executing concurrently and racing to remove obsolete snapshots.

Co-authored-by: Michael Eischer <michael.eischer@fau.de>
 2024-09-14 18:07:46 +02:00
Michael Eischer
34fe73ea42 fs: retry preallocate on Linux if interrupted by signal 2024-09-07 16:39:40 +02:00
Srigovind Nayak
b69c6408a6
forget: make oldest snapshot marker more strict 
Now, a snapshot is only marked as oldest if it's the last in the list AND its values matches the last seen value for that bucket.

Also, updated the corresponding golden files for the tests.
 2024-09-07 15:07:23 +05:30
Srigovind Nayak
d656a50852
forget: update tests to reflect specific reasons for keeping oldest snapshots in a group 2024-09-07 15:07:23 +05:30
Srigovind Nayak
87f30bc787
forget: indicate why the oldest snapshot in a group is kept 
When the oldest snapshot in the
list is retained, the reason is now prefixed with "oldest" to clearly
indicate why it's being kept.
 2024-09-07 15:07:23 +05:30
Michael Eischer
9a6059eb71
Merge pull request #5032 from dropbigfish/master 
chore: fix some function name comments
 2024-09-01 21:52:26 +00:00
dropbigfish
6f9513d88c chore: fix some function names 
Signed-off-by: dropbigfish <fillfish@foxmail.com>
 2024-09-01 00:54:39 +08:00
Michael Eischer
b91ef3f1ff fs: remove dead code 2024-08-31 18:40:36 +02:00
Michael Eischer
e2bce1b9ee fs: move WindowsAttributes definition back to restic package 2024-08-31 18:40:36 +02:00
Michael Eischer
ebdd946ac1 fs: unexport nodeRestoreTimestamps 2024-08-31 18:40:36 +02:00
Michael Eischer
2aa1e2615b fs: fix comments 2024-08-31 18:40:36 +02:00
Michael Eischer
6c16733dfd fs: remove unused methods from File interface 2024-08-31 18:40:36 +02:00
Michael Eischer
f0329bb4e6 fs: replace statT with ExtendedFileInfo 2024-08-31 18:40:36 +02:00
Michael Eischer
6d3a5260d3 fs: unexport a several windows functions 2024-08-31 18:40:36 +02:00
Michael Eischer
cf051e777a fs: remove Readdir method from File interface 2024-08-31 18:20:41 +02:00
Michael Eischer
cc7f99125a minimize usage of internal/fs in tests 2024-08-31 18:20:41 +02:00
Michael Eischer
65a7157383 mount: use os instead of fs package 2024-08-31 18:20:41 +02:00
Michael Eischer
24f4e780f1 backend: consistently use os package for filesystem access 
The go std library should be good enough to manage the files in the
backend and cache folders.
 2024-08-31 18:20:40 +02:00
Michael Eischer
ca1e5e10b6 add proper constants for node type 2024-08-31 18:20:01 +02:00
Michael Eischer
7bb92dc7bd archiver: use ExtendedStat from FS interface 
With this change, NodeFromFileInfo is the last function that bypasses
the FS interface in the archiver.
 2024-08-31 18:05:09 +02:00
Michael Eischer
e79dca644e fs: unexport DeviceID 2024-08-31 18:04:53 +02:00
Michael Eischer
70fbad6623 archiver: minimize imports 2024-08-31 18:04:37 +02:00
Michael Eischer
6fd5d5f2d5 archiver: move helper functions to combine rejects 2024-08-31 18:04:22 +02:00
Michael Eischer
f1585af0f2 move include/exclude options to filter package 2024-08-31 18:04:07 +02:00
Michael Eischer
41c031a19e backup: move RejectFuncs to archiver package 2024-08-31 18:03:35 +02:00
Michael Eischer
f9dbcd2531 backup: convert reject funcs to use FS interface 
Depending on parameters the paths in a snapshot do not directly
correspond to real paths on the filesystem. Therefore, reject funcs must
use the FS interface to work correctly.
 2024-08-31 18:03:02 +02:00
Michael Eischer
c6fae0320e archiver: hide implementation details 2024-08-31 17:52:45 +02:00
Michael Eischer
507842b614 fs: remove Open method from FS interface 2024-08-31 17:37:25 +02:00
Michael Eischer
263709da8c fs: unexport isListxattrPermissionError 2024-08-31 17:37:25 +02:00
Michael Eischer
80ed863aab repository: remove redundant cleanup code 
The temp files used by the packer manager are either delete after
creation (unix) or marked as delete on close (windows). Thus, no
explicit cleanup is necessary.
 2024-08-31 17:37:25 +02:00
Michael Eischer
0ddb4441d7 fs: clean up helper functions 2024-08-31 17:37:25 +02:00
Michael Eischer
fc549c9462 cleanup imports 2024-08-31 17:37:25 +02:00
Michael Eischer
b9b32e5647 restic: extract Node filesystem code to fs package 2024-08-31 17:37:25 +02:00
Michael Eischer
a2e54eac64 restic: simplify nodeCreateFileAt 
The code to write the file content is never used.
 2024-08-31 17:37:25 +02:00
Michael Eischer
5644079707 restic: prepare extraction of fs code from Node 2024-08-31 17:37:25 +02:00
Michael Eischer
97f696b937 backend: remove dead code 2024-08-31 17:25:24 +02:00
Michael Eischer
af989aab4e backend/layout: unexport fields and simplify rest layout 2024-08-31 17:25:24 +02:00
Michael Eischer
6024597028 drop support for s3legacy layout 2024-08-31 17:25:24 +02:00
Michael Eischer
943b6ccfba index: remove support for legacy index format 2024-08-31 17:12:43 +02:00
Michael Eischer
a5533344f9
Merge pull request #5028 from MichaelEischer/windows-allow-specifying-volumes 
backup: support specifying volume instead of path on Windows
 2024-08-31 16:43:20 +02:00
Michael Eischer
ddf35a60ad
Merge pull request #5026 from MichaelEischer/fix-handling-invalid-filenames 
cache: Fix handling of invalid filenames
 2024-08-31 16:42:13 +02:00
Michael Eischer
4fcedb4bae backup: support specifying volume instead of path on Windows 
"C:" (volume name) versus "C:\" (path)
 2024-08-30 11:35:43 +02:00
Michael Eischer
a0f2dfbc19
Merge pull request #5019 from MichaelEischer/fix-windows-sd-race 
backup: Fix spurious "A Required Privilege Is Not Held by the Client" error
 2024-08-29 16:59:06 +02:00
Michael Eischer
0aadfe32bb
Merge pull request #5018 from MichaelEischer/rest-retry-http2-goaway 
rest: improve handling of HTTP2 goaway
 2024-08-29 16:58:04 +02:00
Michael Eischer
7bbf75237d
Merge pull request #5014 from MichaelEischer/configurable-slow-request-timeout 
Make timeout for slow requests configurable
 2024-08-29 16:52:24 +02:00
Michael Eischer
8eff4e0e5c cache: correctly ignore files whose filename is no ID 
this can for example be the case for temporary files created by the
backend implementation.
 2024-08-29 16:32:15 +02:00
Michael Eischer
9c70794886 fs: fix error handling for retried get/set of security descriptor 
The retry code path did not filter `ERROR_NOT_SUPPORTED`. Just call the
original function a second time to correctly follow the low privilege
code path.
 2024-08-26 19:36:43 +02:00
Michael Eischer
6fbfccc2d3 fs: fix race condition in get/set security descriptor 
Calling `Load()` twice for an atomic variable can return different
values each time. This resulted in trying to read the security
descriptor with high privileges, but then not entering the code path to
switch to low privileges when another thread has already done so
concurrently.
 2024-08-26 19:31:21 +02:00
Michael Eischer
2296fdf668 lock: introduce short delay between failed locking retries 
Failed locking attempts were immediately retried up to three times
without any delay between the retries. If a lock file is not found while
checking for other locks, with the reworked backend retries there is no
delay between those retries. This is a problem if a backend requires a
few seconds to reflect file deletions in the file listings. To work
around this problem, introduce a short exponentially increasing delay
between the retries. The number of retries is now increased to 4. This
results in delays of 5, 10 and 20 seconds between the retries.
 2024-08-26 16:31:42 +02:00
Michael Eischer
89d216ca76
Merge pull request #5011 from MichaelEischer/fix-canceled-retry 
backend/retry: don't trip circuit breaker if context is canceled
 2024-08-26 16:30:03 +02:00
Michael Eischer
e24dd5a162 backend/retry: don't trip circuit breaker if context is canceled 
When the context used for a load operation is canceled, then the result
is always an error independent of whether the file could be retrieved
from the backend. Do not false positively trip the circuit breaker in
this case.

The old behavior was problematic when trying to lock a repository. When
`Lock.checkForOtherLocks` listed multiple lock files in parallel and one
of them fails to load, then all other loads were canceled. This
cancelation was remembered by the circuit breaker, such that locking
retries would fail.
 2024-08-26 16:22:21 +02:00
Michael Eischer
36c4475ad9 rest: improve handling of HTTP2 goaway 
The HTTP client can only retry HTTP2 requests after receiving a GOAWAY
response if it can rewind the body. As we use a custom data type,
explicitly provide an implementation of `GetBody`.
 2024-08-26 15:44:17 +02:00
Michael Eischer
b8f409723d make timeout for slow requests configurable 2024-08-26 14:14:43 +02:00
Michael Eischer
311b27ced8 restic: cleanup redundant code in test case 2024-08-25 23:18:55 +02:00
Michael Eischer
43b36ad2b0 restore: test timestamps for macOS resource forks are restored correctly 2024-08-25 23:18:55 +02:00
Michael Eischer
2e55209b34 restic: restore timestamps after extended attributes 
restoring the xattr containing resource forks on macOS apparently
modifies the file modification timestamps. Thus, restore the timestamp
after xattrs.
 2024-08-25 23:18:55 +02:00
Michael Eischer
5cca6e66be
Merge pull request #4981 from konidev20/fix-gh-4934-cleanup-removed-snaphots-from-cache 
cache: clear snapshot files from cache during load index
 2024-08-16 19:04:59 +00:00
Srigovind Nayak
88174cd0a4
cache: remove redundant index file cleanup 
addressing code review comments
 2024-08-17 00:21:49 +05:30
Srigovind Nayak
b7d014b685
Revert "repository: removed redundant prepareCache method from Repository" 
This reverts commit 720609f8ba.
 2024-08-17 00:18:13 +05:30
Michael Eischer
74d3f92cc7
Merge pull request #4993 from MichaelEischer/fix-timeout-error 
backend: return correct error on upload/request timeout
 2024-08-15 22:07:37 +02:00
Michael Eischer
80f24584a5
Merge pull request #4998 from zmanda/ea_vss_fix 
Fix extended attributes handling for VSS snapshots
 2024-08-15 19:51:35 +00:00
Michael Eischer
8e00158c34
Merge pull request #5000 from deining/fix-typo 
Fix typos
 2024-08-15 19:42:14 +00:00
Michael Eischer
36b5580c1c
Merge pull request #4989 from plant99/progress-bar-for-restore-verify 
restore: Add progress bar to 'restore --verify'
 2024-08-15 19:34:05 +00:00
aneesh-n
19f487750e
Add test cases and handle volume GUID paths 
Gracefully handle errors while checking for EA and add debug logs.
 2024-08-11 19:25:58 -06:00
Shivashis Padhi
f1407afd1f
restore: Add progress bar to 'restore --verify' 2024-08-11 22:25:21 +02:00
Andreas Deininger
4401265e36 Fix typos 2024-08-11 21:38:15 +02:00
Srigovind Nayak
5fd984ba6f
cache: add test for the automated cache clear to cache backend 2024-08-11 23:41:07 +05:30
Srigovind Nayak
720609f8ba
repository: removed redundant prepareCache method from Repository 
* remove the prepareCache method from the Repository
* changed the signature of the SetIndex function to no longer return an error
 2024-08-11 23:41:07 +05:30
Srigovind Nayak
a23e7bfb82
cache: check for context cancellation before clearing cache 2024-08-11 23:41:07 +05:30
Srigovind Nayak
f66624f5bf
cache: backend add List method and a cache clear functionality 
* removes files which are no longer in the repository, including index files, snapshot files and pack files from the cache.

cache: fix ids set initialisation with NewIDSet()
 2024-08-11 23:40:52 +05:30
aneesh-n
849c441455
Gracefully handle invalid prepared volume names 2024-08-11 01:48:25 -06:00
aneesh-n
1d392a36f9
Fix extended attributes handling for VSS snapshots 2024-08-11 01:23:47 -06:00
Michael Eischer
3f5e2160de
Merge pull request #4938 from MichaelEischer/bump-go-version 
Bump go version to 1.21
 2024-08-10 19:57:59 +02:00
Michael Eischer
283225f15f
Merge pull request #4994 from MichaelEischer/less-flaky-watchdog-test 
backend: tweak timeouts to make watchdog timeout test less flaky
 2024-08-10 19:35:38 +02:00
Michael Eischer
400ae55940 replace deprecated usages of math/rand 2024-08-10 19:34:49 +02:00
Michael Eischer
0b19f6cf5a Switch back to sha256 from the std library 
The std library now also supports the sha assembly instructions on
ARM64. Thus, sha256-simd no longer provides a performance benefit.
 2024-08-10 19:16:10 +02:00
Michael Eischer
ad48751adb bump required go version to 1.21 2024-08-10 19:16:10 +02:00
Michael Eischer
86390b453d
Merge pull request #4980 from zmanda/unsupported-ea-handling 
Skip ExtendedAttribute processing in Windows for volumes that do not support EA
 2024-08-10 17:11:12 +00:00
Michael Eischer
fa35e72214 backend: tweak timeouts to make watchdog timeout test less flaky 2024-08-10 19:08:03 +02:00
Michael Eischer
05571286b2
Merge pull request #4946 from mikix/json-errors 
Improve error handling in --json mode
 2024-08-10 17:04:56 +00:00
aneesh-n
18e9d71d7a
Fix review comments 2024-08-10 10:38:04 -06:00
Michael Eischer
853a686994 backend: return correct error on upload/request timeout 2024-08-10 18:06:24 +02:00
Michael Eischer
a164789321
Merge pull request #4977 from MichaelEischer/warn-on-incomplete-metadata 
backup: warn but store store item if extended metadata is incomplete
 2024-08-10 17:59:25 +02:00
aneesh-n
71632a8197
Handle extended length paths 2024-08-05 16:03:43 -06:00
aneesh-n
85639f5159
Add handling for relative paths, vss paths, UNC paths 2024-08-04 13:19:13 -06:00
aneesh-n
c13725b5d0
Check EA support only for volumes, files and dirs 2024-08-04 11:05:40 -06:00
aneesh-n
89712f6640
Formatted 2024-08-04 10:36:13 -06:00
aneesh-n
9dedba6dfc
Address review comments 2024-08-04 10:23:39 -06:00
aneesh-n
8c8a066c0e
Correct the bitmask for fileSupportsExtendedAttributes and add link 2024-08-03 18:06:47 -06:00
aneesh-n
56113a8da7
Skip EA processing for volumes that do not support EA 2024-08-03 16:03:30 -06:00
Michael Eischer
73c9780321 backup: store but warn if extended metadata for item is incomplete 
Files were not included in the backup if the extended metadata for the
file could not be read. This is rather drastic. Instead settle on
returning a warning but still including the file in the backup.
 2024-08-03 22:25:29 +02:00
Michael Terry
88f59fc2d6 json: switch backup and restore errors from string to struct types 
This keeps backwards compatibility with the previous empty structs.
And maybe we'd want to put other fields into the inner struct later,
rather than the outer message.
 2024-08-03 16:21:44 -04:00
Michael Eischer
a48baf6f3a
Merge pull request #4976 from textaligncenter/backup-panic 
fix panic in fs_reader
 2024-08-03 19:56:15 +00:00
Michael Terry
a376323331 restore: print JSON versions of errors in --json mode 
Previously, they were printed as freeform text.

This also adds a ui.Terminal interface to make writing
tests easier and also adds a few tests.
 2024-08-03 15:18:46 -04:00
Michael Eischer
e622135e7e
Merge pull request #4974 from MichaelEischer/fix-cache-debug-log 
cache: fix confusing debug log
 2024-08-03 21:12:32 +02:00
textaligncenter
d8ea178e69 fix panic in fs_reader 2024-08-03 19:08:04 +00:00
Michael Terry
ad2585af67 backup: show actual error strings in --json mode 
Previously, an error JSON fragment would look like:
{"message_type": "error", "error": {}}

This is because encoding/json cannot marshal an error interface.
Instead, we now call .Error() to get the string value.
 2024-08-03 15:07:52 -04:00
Michael Terry
f4bdfea1c9 backup: print scanner errors to stderr, not stdout 2024-08-03 15:07:52 -04:00
Michael Eischer
d2f7c5a9c6
Merge pull request #4978 from konidev20/fix-gh-4949-repair-index-spurious-index 
rewrite: skip saving empty indexes during MasterIndex.Rewrite
 2024-08-03 18:53:57 +00:00