Michael Eischer
8a486eafed
gs: Don't drop error when finishing upload
...
The error returned when finishing the upload of an object was dropped.
This could cause silent upload failures and thus data loss in certain
cases. When a MD5 hash for the uploaded blob is specified, a wrong
hash/damaged upload would return its error via the Close() whose error
was dropped.
2021-01-30 13:31:32 +01:00
MichaelEischer
4d576c2f79
Merge pull request #3217 from M4a1x/read-data-subset-fix
...
Fix missing rand seed for restic check --read-data-subset=x%
2021-01-29 23:34:13 +01:00
Max Stabel
f9e1fa26ff
Fix missing rand seed for restic check --read-data-subset=x%
2021-01-29 23:18:35 +01:00
MichaelEischer
fb3cf3f885
Merge pull request #3245 from aawsome/prune-fix-statistics-cacheable
...
prune: Fix statistics for --repack-cacheable-only
2021-01-29 23:14:49 +01:00
Alexander Weiss
e08e65dc30
prune: Simplify logic selecting packs to repack
2021-01-29 22:27:22 +01:00
Alexander Weiss
daeb4cdf8f
prune: Fix statistics for --repack-cacheable-only
2021-01-29 22:27:22 +01:00
Alexander Neumann
cdd704920d
azure: Pass data length to Azure libray
...
The azureAdapter was used directly without a pointer, but the Len()
method was only defined with a pointer receiver (which means Len() is
not present on a azureAdapter{}, only on a pointer to it).
2021-01-29 21:08:41 +01:00
Alexander Neumann
bbdf18c4a2
Merge pull request #3176 from MichaelEischer/backend-content-length
...
Pass upload size to backends and sanity check it
2021-01-29 20:33:44 +01:00
Michael Eischer
1f583b3d8e
backend: test that incomplete uploads fail
2021-01-29 13:51:53 +01:00
Michael Eischer
c73316a111
backends: add sanity check for the uploaded file size
...
Bugs in the error handling while uploading a file to the backend could
cause incomplete files, e.g. https://github.com/golang/go/issues/42400
which could affect the local backend.
Proactively add sanity checks which will treat an upload as failed if
the reported upload size does not match the actual file size.
2021-01-29 13:51:51 +01:00
Michael Eischer
4526d5d197
swift: explicitly pass upload size to library
...
This allows properly setting the content-length which could help the
server-side to detect incomplete uploads.
2021-01-29 13:50:46 +01:00
Michael Eischer
dca9b6f5db
azure: explicitly pass upload size
...
Previously the fallback from the azure library was to read the whole
blob into memory and use that to determine the upload size.
2021-01-29 13:50:46 +01:00
Alexander Neumann
a16ce65295
Merge pull request #3244 from MichaelEischer/better-damage-reports
...
Print more details about possible repository damages
2021-01-29 11:45:45 +01:00
rawtaz
de7e3a0648
Merge pull request #2823 from greatroar/trust-mtime
...
Add --ignore-ctime flag to backup and document change detection
2021-01-29 00:02:19 +01:00
greatroar
6bd8a2faaa
backup: Add --ignore-ctime option and document change detection
2021-01-28 23:42:10 +01:00
Michael Eischer
58b5679f14
prune: reword missing blobs error
...
The previous wording could be understood such that the prune run did
damage the repository.
2021-01-28 21:48:24 +01:00
Michael Eischer
7b8886c052
prune: report missing but unneeded pack files
...
This indicates a damaged repository so add some output to help with
debugging.
2021-01-28 21:46:01 +01:00
Michael Eischer
ff95999246
rebuild-index: report added/removed/reindexed files
...
This should help with investigating missing pack files.
2021-01-28 21:46:01 +01:00
Michael Eischer
b71c52797a
find: correctly expand multiple blob ids
...
For example `restic find --show-pack-id --blob f78dc991 5b9e4366 ddd8c7d4`
would previously only expand one blob if all of them belong to the same
file.
2021-01-28 21:21:54 +01:00
MichaelEischer
82140967d3
Merge pull request #3228 from aawsome/prune-all-trees
...
prune: Remove all unused trees
2021-01-28 21:04:27 +01:00
MichaelEischer
43cb26010a
Merge pull request #3242 from greatroar/fprintln
...
internal/ui/termstatus: Use Fprintln to get a newline
2021-01-28 20:34:06 +01:00
MichaelEischer
35033d9b79
Merge pull request #3177 from MichaelEischer/fix-2759
...
prune: don't print stacktrace on console
2021-01-28 20:28:45 +01:00
Alexander Neumann
84822d44d4
Merge pull request #2536 from MatthewVance/threat-model
...
Update threat model
2021-01-28 14:26:03 +01:00
Matthew Vance
58c7f4694d
Update threat model
2021-01-28 14:20:42 +01:00
Alexander Neumann
4d40c70214
Merge pull request #3211 from MichaelEischer/sftp-speedup
...
Speed-up caching/pack download via SFTP
2021-01-28 14:16:55 +01:00
Alexander Neumann
44169d0dc4
Merge pull request #3205 from MichaelEischer/fix-quiet-verbose
...
Properly check that --quiet and --verbose are not combined
2021-01-28 13:53:31 +01:00
Alexander Neumann
6aa7e9f9c6
Merge pull request #3174 from MichaelEischer/parallelize-lock-loading
...
Parallelize lock file loading
2021-01-28 13:52:12 +01:00
Alexander Neumann
bdfedf1f5b
Merge pull request #3173 from MichaelEischer/unify-index-loading
...
Unify index loading
2021-01-28 13:50:42 +01:00
greatroar
b9cfe6f68a
internal/ui/termstatus: Use Fprintln to get a newline
2021-01-28 13:30:10 +01:00
Alexander Neumann
72eec8c0c4
Merge pull request #3106 from MichaelEischer/parallel-tree-walk
...
Parallelize tree walk in prune and copy and add progress bar to check
2021-01-28 12:06:42 +01:00
Michael Eischer
68608a89ad
restic: add comment about StreamTrees shutdown
2021-01-28 11:10:50 +01:00
Michael Eischer
1e306be000
Add changelog entry
2021-01-28 11:10:50 +01:00
Michael Eischer
ddb7697d29
restic: Test progress reporting of StreamTrees
2021-01-28 11:10:50 +01:00
Michael Eischer
313ad0e32f
progress/counter: Fix test for final report call
2021-01-28 11:10:50 +01:00
Michael Eischer
e2b0072441
check: add progress bar to the tree structure check
2021-01-28 11:10:50 +01:00
Michael Eischer
505f8a2229
progress/counter: Support updating the progress bar maximum
2021-01-28 11:10:47 +01:00
Michael Eischer
eda8c67616
restic: let FindUsedBlobs handle multiple snapshots at once
2021-01-28 11:08:43 +01:00
Michael Eischer
258ce0c1e5
parallel: report progress for StreamTrees
...
This assigns an id to each tree root and then keeps track of how many
tree loads (i.e. trees referenced for the first time) are pending per
tree root. Once a tree root and its subtrees were fully processed there
are no more pending tree loads and the tree root is reported as
processed.
2021-01-28 11:08:43 +01:00
Michael Eischer
3d6a3e2555
copy: Remove treeCloner struct
2021-01-28 11:08:43 +01:00
Michael Eischer
0caad1e890
copy: parallelize tree walk
2021-01-28 11:08:43 +01:00
Michael Eischer
f2a1b125cb
restic: Actually parallelize FindUsedBlobs
2021-01-28 11:08:43 +01:00
Michael Eischer
6e03f80ca2
check: Split the parallelized tree loader into a reusable component
...
The actual code change is minimal
2021-01-28 11:08:43 +01:00
Michael Eischer
1d7bb01a6b
check: Cleanup tree loading and switch to use errgroup
...
The helper methods are now wired up in the Structure method.
2021-01-28 11:08:43 +01:00
Alexander Neumann
a4689eb3b9
Merge pull request #3199 from MichaelEischer/non-interactive-counter
...
Don't print progress on non-interactive terminals
2021-01-28 10:53:38 +01:00
Alexander Neumann
c5a66e9181
ui: Simlify channel receive
2021-01-28 10:42:02 +01:00
Alexander Weiss
d7dc19a496
prune: Always repack packs containing tree blobs
2021-01-15 16:42:04 +01:00
Michael Eischer
f3442ce8a5
Test that WriteTo of a backend's Load remains accessible
2021-01-03 22:23:53 +01:00
Michael Eischer
678e75e1c2
sftp: enforce use of optimized upload method
...
ReadFrom was already used by Save before, this just ensures that this
won't accidentally change in the future.
2021-01-03 22:23:53 +01:00
Michael Eischer
6b5b29dbee
limiter: add unit tests
2021-01-03 22:23:53 +01:00
Michael Eischer
f35f2c48cd
limiter: support WriteTo in LimitBackend for read rate limiting
2021-01-03 22:23:53 +01:00
