Michael Eischer
1d6d3656b0
repository: move backend.LoadAll to repository.LoadRaw
...
LoadRaw also includes improved context cancellation handling similar to the
implementation in repository.LoadUnpacked.
The removed cache backend test will be added again later on.
2024-05-18 21:26:00 +02:00
Michael Eischer
47232bf8b0
backend: move LimitReadCloser to util package
...
The helper is only intended for usage by backend implementations.
2024-05-18 21:26:00 +02:00
Michael Eischer
53d15bcd1b
retry: add circuit breaker to load method
...
If a file exhausts its retry attempts, then it is likely not accessible
the next time. Thus, immediately fail all load calls for that file to
avoid useless retries.
2024-05-18 19:59:26 +02:00
Michael Eischer
394c8ca3ed
rest/rclone/s3/sftp/swift: move short file detection behind feature gate
...
These backends tend to use a large variety of server implementations.
Some of those implementations might prove problematic with the new
checks.
2024-05-18 19:59:26 +02:00
Michael Eischer
6328b7e1f5
replace "too small" with "too short" in error messages
2024-05-18 19:59:26 +02:00
Michael Eischer
aeb7eb245c
retry: do not retry permanent errors
...
This is currently gated behind a feature flag as some unexpected
interactions might show up in the wild.
2024-05-18 19:59:26 +02:00
Michael Eischer
bf8cc59889
Use generic backend-error-redesign feature flag instead of http-timeouts
...
An individual flag for each change of the backend error handling would
be too finegrained. Thus, add a generic flag.
2024-05-18 19:54:52 +02:00
Michael Eischer
4740528a0b
backend: add tests for IsPermanentError
2024-05-18 19:54:52 +02:00
Michael Eischer
6a85df7297
backend: add IsPermanentError() method to interface
2024-05-18 19:54:52 +02:00
Michael Eischer
cfc420664a
mem: stricter handling of out of bounds requests
2024-05-18 19:54:52 +02:00
Michael Eischer
d40f23e716
azure/b2/gs/s3/swift: adapt cloud backend
2024-05-18 19:54:51 +02:00
Michael Eischer
e793c002ec
local: stricter handling of short files
2024-05-18 19:54:21 +02:00
Michael Eischer
b4895ebd76
rest: rework error reporting and report too short files
2024-05-18 19:54:21 +02:00
Michael Eischer
eaa3f81d6b
sftp: check for truncated files without an extra backend request
2024-05-18 19:54:21 +02:00
Michael Eischer
c6d74458ee
sftp: improve handling of too short files
2024-05-18 19:54:21 +02:00
Michael Eischer
7ed560a201
Merge pull request #4796 from MichaelEischer/parallel-dump-load
...
dump: Parallelize loading large files
2024-05-14 22:35:44 +02:00
Michael Eischer
92221c2a6d
Merge pull request #4708 from zmanda/windows-securitydesc
...
Back up and restore SecurityDescriptors on Windows
2024-05-12 14:14:39 +00:00
Michael Eischer
b5fdb1d637
Merge pull request #4782 from MichaelEischer/fix-sftp-performance
...
Fix sftp upload performance
2024-05-12 15:28:33 +02:00
Michael Eischer
ebd01a4675
backend: add tests for watchdogRoundTripper
2024-05-09 23:46:17 +02:00
Michael Eischer
8778670232
backend: cancel stuck http requests
...
requests that make no upload or download progress within a timeout are
canceled.
2024-05-09 23:46:17 +02:00
Michael Eischer
0987c731ec
backend: configure protocol-level connection health checks
...
This should detect a connection that is stuck for more than 2 minutes.
2024-05-09 23:46:17 +02:00
aneesh-n
a4fd1b91e5
Fix review comments
...
Change lowerPrivileges from bool to atomic.Bool.
Add missing cleanup from upstream go-winio.
Add handling for ERROR_NOT_ALL_ASSIGNED warning.
2024-05-06 16:54:08 -06:00
Michael Eischer
4d55a62ada
bloblru: add test for GetOrCompute
2024-05-05 12:00:25 +02:00
Michael Eischer
7cce667f92
fuse: switch to use bloblru.GetOrCompute
2024-05-05 11:38:42 +02:00
Michael Eischer
bd03af2feb
dump: add GetOrCompute to bloblru cache
2024-05-05 11:38:42 +02:00
Michael Eischer
45509eafc8
dump: load blobs of a file from repository in parallel
2024-05-05 11:38:42 +02:00
Michael Eischer
ffe5439149
Merge pull request #4605 from MichaelEischer/better-restorer-error-handling
...
Rework repository.StreamPacks & better restorer error handling
2024-05-01 16:37:41 +02:00
aneesh-n
672f6cd776
Fix review comments for privileges and security flags
2024-04-29 17:29:51 -06:00
aneesh-n
08c6945d61
Fix review comments
2024-04-29 16:21:38 -06:00
Aneesh N
3f76b902e5
Merge branch 'master' into windows-securitydesc
2024-04-29 14:40:34 -06:00
DRON-666
125dba23c5
Rearange code
2024-04-29 01:27:34 +03:00
DRON-666
7ee889bb0d
Use S_FALSE and MaxInt
2024-04-29 01:25:25 +03:00
DRON-666
90b168eb6c
isMountPointExcluded to isMountPointIncluded
2024-04-29 01:23:50 +03:00
DRON-666
24330c19a8
Use kebab case in option names
2024-04-29 01:21:33 +03:00
DRON-666
5703e5a652
Fix texts and comments
2024-04-29 01:18:46 +03:00
DRON-666
0a8f9c5d9c
vss: Add tests for "provider" option
2024-04-28 22:45:21 +03:00
DRON-666
bb0f93ef3d
vss: Add "provider" option
2024-04-28 22:45:21 +03:00
DRON-666
3bac1f0135
vss: Fix issues reported by linters
2024-04-28 22:45:21 +03:00
DRON-666
88c509e3e9
vss: Change ErrorHandler signature
...
We don't need `error` here: the only existing implementation
of `ErrorHandler` always call `Backup.Error` and all
implementations of `Backup.Error` always return nil.
2024-04-28 22:44:16 +03:00
DRON-666
9d3d915e2c
vss: Add some tests
2024-04-28 22:44:16 +03:00
DRON-666
c4f67c0064
vss: Add volume filtering
...
Add options to exclude all mountpoints and arbitrary volumes from snapshotting.
2024-04-28 22:44:15 +03:00
DRON-666
7470e5356e
vss: Add "timeout" option
...
Changing multiple "callAsyncFunctionAndWait" with fixed timeout
to calculated timeout based on deadline.
2024-04-28 22:44:15 +03:00
DRON-666
78dbc5ec58
vss: Add initial support for extended options
2024-04-28 22:44:15 +03:00
Michael Eischer
935327d480
sftp: slightly increase write concurrency
...
This should increase upload throughput for high latency links a bit.
2024-04-28 11:50:09 +02:00
Michael Eischer
669a669603
sftp: Fix upload performance issue
...
Since pkg/sftp 1.13.0 files were uploaded sequentially using 32kb chunks
instead of sending 64 chunks in parallel.
2024-04-28 11:48:26 +02:00
Michael Eischer
940a3159b5
let index.Each() and pack.Size() return error on canceled context
...
This forces a caller to actually check that the function did complete.
2024-04-22 22:39:32 +02:00
Michael Eischer
31624aeffd
Improve command shutdown on context cancellation
2024-04-22 22:31:38 +02:00
Michael Eischer
b15d867414
Merge pull request #4763 from MichaelEischer/refactor-prune
...
Refactor repair index / prune into the repository package
2024-04-22 22:24:53 +02:00
Michael Eischer
2e6c43c695
Merge pull request #4761 from MichaelEischer/fix-cache-race
...
cache: ignore ErrNotExist during cleanup of old files
2024-04-22 21:46:06 +02:00
Michael Eischer
f7632de3d6
Merge pull request #4772 from MichaelEischer/better-error-on-too-large-blob
...
repository: Better error message if blob is larger than 4GB
2024-04-22 21:45:06 +02:00
