2
2
mirror of https://github.com/octoleo/restic.git synced 2024-12-25 04:05:58 +00:00
Commit Graph

1868 Commits

Author SHA1 Message Date
Michael Eischer
ed4a4f8748 check: exclude inaccessible files from the repair pack suggestion 2024-02-12 20:25:15 +01:00
Michael Eischer
4073299a7c check: fix missing error if blob is invalid 2024-02-12 20:20:13 +01:00
Michael Eischer
22a3cea1b3 checker: wrap all pack errors in ErrPackData 2024-02-12 20:19:32 +01:00
Michael Eischer
5b5d506472 backup: report files whose chunks failed to upload 2024-02-11 22:43:26 +01:00
Alexander Neumann
c0514dd8ba Fix linter errors (except for tests) 2024-02-10 22:58:10 +01:00
Michael Eischer
5957417b1f Apply changelog entry / documentation improvements from review 2024-02-04 18:55:41 +01:00
Michael Eischer
86b38a0b17 rename --no-verify-pack to --no-extra-verify 2024-02-04 17:01:05 +01:00
Michael Eischer
c32e5e2abb pack: verify integrity of pack file header 2024-02-04 15:31:42 +01:00
Michael Eischer
c97a271e89 repository: ask users to report corrupted data while saving blobs 2024-02-04 15:31:42 +01:00
Michael Eischer
193140525c repository: test verification of blobs/unpacked data 2024-02-04 15:31:42 +01:00
Michael Eischer
2dbb18128c repository: Allow skipping verification for tests
Some tests have to explicitly create pack files with blobs that don't
match their ID. For those blobs the builtin verification of the
repository must be disabled.
2024-02-03 18:22:47 +01:00
Michael Eischer
30a84e9003 backup: verify unpacked files before upload 2024-02-03 18:22:47 +01:00
Michael Eischer
c01a0c6da7 backup: verify blobs before upload
This only covers the blobs themselves, the pack header is not verified
so far. Unpacked files are also not covered by the integrity check.
2024-02-03 18:22:47 +01:00
Michael Eischer
16e3f79e8b repository: make repo.Options configurable for test repos 2024-02-03 18:22:47 +01:00
Michael Eischer
bb92b487f7 repository: fix repack test 2024-02-03 18:22:47 +01:00
Michael Eischer
e44e4b00a6
Merge pull request #4550 from ndecker/ls-ncdu
Ls ncdu
2024-01-27 12:27:35 +00:00
Michael Eischer
d4ed7c8858 walker: add tests for leaveDir 2024-01-27 13:17:33 +01:00
Michael Eischer
2c80cfa4a5 walker: fix missing leaveDir if directory is partially skipped 2024-01-27 13:17:33 +01:00
Michael Eischer
9ecbda059c walker: add callback to inform about leaving a directory 2024-01-27 13:17:32 +01:00
Michael Eischer
246559e654 check: cleanup s3 legacy detection 2024-01-27 13:02:04 +01:00
Michael Eischer
1dfd854769 lock: switch to repository.List 2024-01-27 13:02:04 +01:00
Michael Eischer
bfb56b78e1 replace some usages of restic.Repository with more specific interface
This should eventually make it easier to test the code.
2024-01-27 13:02:02 +01:00
Michael Eischer
3424088274
Merge pull request #4644 from MichaelEischer/refactor-repair-packs
Refactor and test `repair packs`
2024-01-27 13:00:51 +01:00
Michael Eischer
724ec179e3
Merge pull request #4648 from MichaelEischer/repository-removekey
repository: Introduce RemoveKey function
2024-01-27 13:00:42 +01:00
Michael Eischer
f0e1ad2285 fix linter warning 2024-01-27 12:51:45 +01:00
Michael Eischer
fd579421dd repository: deduplicate test 2024-01-27 12:51:45 +01:00
Michael Eischer
42c9318b9c repair pack: add tests 2024-01-27 12:51:45 +01:00
Michael Eischer
764b0bacd6 repair pack: add support for truncated files 2024-01-27 12:51:45 +01:00
Michael Eischer
7c351bc53c repair pack: reenable auto index updates
The method is not available on the restic.Repository interface that is
used for testing. Drop the call as a small amount of additional index
writes is not a problem.
2024-01-27 12:51:45 +01:00
Michael Eischer
feeab84204 repair pack: extract the repair logic into the repository package
Currently, the cmd/restic package contains a significant amount of code
that modifies repository internals. This code should in the mid-term
move into the repository package.
2024-01-27 12:51:45 +01:00
Michael Eischer
cb50832d50 index: let MasterIndex.Save also delete obsolete indexes 2024-01-27 12:51:08 +01:00
Michael Eischer
bedff1ed6d split deleteFiles into UI and logic parts 2024-01-27 12:51:08 +01:00
Michael Eischer
c13bf0b607 repository: Introduce RemoveKey function
This replaces directly removing keys via the backend.
2024-01-27 12:42:58 +01:00
Michael Eischer
446167ae80
Merge pull request #4643 from MichaelEischer/remove-redundant-poly1305-mask
Remove redundant poly1305 key masking
2024-01-23 19:46:40 +01:00
Michael Eischer
1419baf67a
Merge pull request #4645 from MichaelEischer/improve-lock-checking
lock: checkForOtherLocks processes each lock at most once
2024-01-23 19:46:00 +01:00
Michael Eischer
6bdca13603 ui/termstatus: simplify cleaning up on termination
`writeStatus` also cleans no longer used status lines.
The old code actually cleaned one line too much. However, as that line
was never used it makes no difference.
2024-01-21 21:27:27 +01:00
Michael Eischer
f1f34eb3e5 lock: checkForOtherLocks processes each lock at most once
If a lock could not be loaded, then restic would check all lock files
again. These repeated checks are not useful as the status of a lock file
cannot change unless its ID changes too. Thus, skip already check lock
files on retries.
2024-01-20 22:40:12 +01:00
Michael Eischer
fee83e1c09 Remove redundant poly1305 key masking
The implementation in crypto/poly1305 already performs the exact same
masking.
2024-01-20 12:36:59 +01:00
Michael Eischer
a763a5c67d s3: minor cleanups for archive storage class handling 2024-01-20 11:25:28 +01:00
Vladislav Belous
8ca58b487c S3: do not set storage class for metadata when using archive storage 2024-01-20 11:04:15 +01:00
Michael Eischer
2c310a526e repository: Replace StreamPack function with LoadBlobsFromPack method
LoadBlobsFromPack is now part of the repository struct. This ensures
that users of that method don't have to deal will internals of the
repository implementation.

The filerestorer tests now also contain far fewer pack file
implementation details.
2024-01-19 21:40:43 +01:00
Michael Eischer
6b7b5c89e9 repository: prepare StreamPack refactor 2024-01-19 21:40:43 +01:00
Michael Eischer
22d0c3f8dc check: Use PackBlobIterator instead of StreamPack
To only stream the content of a pack file once, check used StreamPack
with a custom pack load function. This combination was always brittle
and complicates using StreamPack everywhere else. Now that StreamPack
internally uses PackBlobIterator use that primitive instead, which is a
much better fit for what the check command requires.
2024-01-19 21:40:36 +01:00
Michael Eischer
fb422497af repository: split StreamPack implementation
Move the actual decoding of the pack data into a separate iterator.
2024-01-19 21:39:55 +01:00
Michael Eischer
54c5c72e5a
Merge pull request #4616 from MichaelEischer/fix-rest-connection-close
rest: fix and cleanup closing of http response body
2024-01-19 21:31:35 +01:00
Michael Eischer
bd883caae1 CI: enable bodyclose linter 2024-01-19 21:17:18 +01:00
Michael Eischer
b1a8fd1d03 rest: fix and cleanup closing of http response body
If client.Do returns an error, then there's no body that has to be
closed. For requests for which we are not interested in the response
body, immediately drain and close the body to make sure it isn't
forgotten later on.

This change in particular adds the missing `Close()` call for the
`List()` command.
2024-01-19 21:17:17 +01:00
Michael Eischer
fdcbb53017 walker: test skipping for root node 2024-01-19 21:16:06 +01:00
Michael Eischer
0b39940fdb walker: Remove ignoreTrees functionality
It was only used in two places:
- stats: apparently as a minor performance optimization, which is
  unlikely to be important
- find: filtered directories would be ignored. However, this
  optimization missed that it is possible that two directories have the
  exact same content. Such directories would be incorrectly ignored too.
  Example:
```
mkdir test test/a test/b
restic backup test
restic find latest test/b
-> incorrectly does not return anything
```

Thus, remove the functionality as it's apparently too complex to use
correctly.
2024-01-19 21:16:06 +01:00
Michael Eischer
7b2de84763
Merge pull request #4618 from MichaelEischer/workaround-rclone-list-errors
rclone: Workaround for incorrect "not found" errors while listing files
2024-01-09 18:28:31 +01:00