Commit Graph

996 Commits

Author SHA1 Message Date
MichaelEischer 58efe21eca
Merge pull request #3264 from amozoss/upstream-master
Refactor backup progress
2021-09-19 14:54:42 +02:00
greatroar 92ae951ffa Report timing from restic restore --verify 2021-09-19 12:53:09 +02:00
Michael Eischer fbb0e6499a ui: consolidate backup ui in ui/backup package 2021-09-12 16:20:15 +02:00
Michael Eischer a144c986f2 backup: Reenable JSON status updates with redirected output
After the refactoring status updates were no longer printed in quiet
mode or when the output is not an interactive terminal. However, the
JSON output is often piped to e.g. another program. Thus, don't set the
update frequency to 0 in that case. The status updates are still
disabled for backup --quiet.

This also reduces the status update frequency to 60fps compared to a
potentially much higher value before the refactoring.
2021-09-12 15:26:40 +02:00
Michael Eischer d62bfed65d ui: move SetDryRun to ProgressReporter 2021-09-12 15:25:58 +02:00
Dan Willoughby 448419990c Refactor backup progress
Move the shared logic into the progress

Allows logic to be shared with forth coming restore status
2021-09-12 15:25:58 +02:00
Alexander Neumann 8fe122d675
Merge pull request #3488 from MichaelEischer/rebuild-broken-index
Fix `rebuild-index` for damaged index
2021-09-07 17:00:23 +02:00
MichaelEischer fa5ca8af81
Merge pull request #2856 from aawsome/remove-readahead
Simplify cache logic
2021-09-04 20:25:49 +02:00
MichaelEischer b45d88e124
Merge pull request #3496 from juergenhoetzel/imporove-snapshot-filter-warning-message
Improve snapshot filter warning message
2021-09-03 23:42:27 +02:00
MichaelEischer bc4cbd775b
Merge pull request #2880 from aawsome/enhance-recover
Improve recover command
2021-09-03 23:40:43 +02:00
Alexander Weiss bce87922c0 Improve recover
- only save directories not referenced by any snapshot
- dont't write empty snapshot
- use progress bar
2021-09-03 21:36:57 +02:00
Alexander Weiss 81876d5c1b Simplify cache logic 2021-09-03 21:01:00 +02:00
greatroar 7f0aa49f45 cmd/restic: Streamline progress printing
* PrintProgress no longer does unnecessary Sprintf calls, and performs
  fewer allocations in general
* newProgressMax's callback checks whether the terminal supports
  line updates once instead of once per call
* the callback looks up the terminal width once per call instead of
  twice (on Windows)
* the status shortening now uses the Unicode-aware version from
  internal/ui/termstatus (future-proofing)
2021-09-03 11:48:22 +02:00
Juergen Hoetzel 409e4936af Improve snapshot filter warning message
Include the root-cause why the snapshot prefix is ignored.
2021-08-25 15:46:21 +02:00
Michael Eischer 877fc9f352 rebuild-index: test that invalid indexes are skipped and removed 2021-08-22 18:24:19 +02:00
Michael Eischer 64258a2c2a rebuild-index: Ignore broken index files
Previously, an e.g. truncated index could prevent rebuild-index from
working.
2021-08-22 18:23:47 +02:00
Alexander Neumann d90efd7704 Fix test 2021-08-09 10:30:10 +02:00
Alexander Neumann 9fe5a87785
Merge pull request #3429 from MichaelEischer/safe-key-change
key: Check that new key works before deleting the old one
2021-08-09 10:07:15 +02:00
erin 4126435663 resolve rawtaz's review comments
make majority of suggestions from review by @rawtaz verbatim, with one clarification on my part in changelog
2021-08-04 21:19:29 +02:00
Alexander Weiss d107a2cfdf Separate dry run tests 2021-08-04 21:19:29 +02:00
Ryan Hitchman 77bf148460 backup: add --dry-run/-n flag to show what would happen.
This can be used to check how large a backup is or validate exclusions.
It does not actually write any data to the underlying backend. This is
implemented as a simple overlay backend that accepts writes without
forwarding them, passes through reads, and generally does the minimal
necessary to pretend that progress is actually happening.

Fixes #1542

Example usage:

$ restic -vv --dry-run . | grep add
new       /changelog/unreleased/issue-1542, saved in 0.000s (350 B added)
modified  /cmd/restic/cmd_backup.go, saved in 0.000s (16.543 KiB added)
modified  /cmd/restic/global.go, saved in 0.000s (0 B added)
new       /internal/backend/dry/dry_backend_test.go, saved in 0.000s (3.866 KiB added)
new       /internal/backend/dry/dry_backend.go, saved in 0.000s (3.744 KiB added)
modified  /internal/backend/test/tests.go, saved in 0.000s (0 B added)
modified  /internal/repository/repository.go, saved in 0.000s (20.707 KiB added)
modified  /internal/ui/backup.go, saved in 0.000s (9.110 KiB added)
modified  /internal/ui/jsonstatus/status.go, saved in 0.001s (11.055 KiB added)
modified  /restic, saved in 0.131s (25.542 MiB added)
Would add to the repo: 25.892 MiB
2021-08-04 21:19:29 +02:00
Alexander Neumann 7049f1cbfc Set development version for 0.12.1 2021-08-03 11:45:39 +02:00
Alexander Neumann dc7a8aab24 Add version for 0.12.1 2021-08-03 11:45:36 +02:00
Alexander Neumann 77551597b2
Merge pull request #3416 from torfason/keep-hourly-within
Keep hourly within
2021-07-30 10:36:41 +02:00
Alexander Neumann 92f293cd0b
Merge pull request #3427 from MichaelEischer/find-packs-from-index
find: List missing pack files based on the index
2021-07-30 10:33:02 +02:00
Magnus Thor Torfason 74ebc650ab forget: Add --keep-within-hourly (and friends)
Allow keeping hourly/daily/weekly/monthly/yearly snapshots for a given time period.

This adds the following flags/parameters to restic forget:
  --keep-within-hourly duration
  --keep-within-daily duration
  --keep-within-weekly duration
  --keep-within-monthly duration
  --keep-within-yearly duration

Includes following changes:
  - Add tests for --keep-within-hourly (and friends)
  - Add documentation for --keep-within-hourly (and friends)
  - Add changelog for --keep-within-hourly (and friends)
2021-07-24 16:14:43 +00:00
Alexander Neumann c707d71b72
Merge pull request #3401 from MichaelEischer/goroutine-shutdown-cleanups
Goroutine shutdown cleanups
2021-07-11 16:32:28 +02:00
Michael Eischer 0880afe67b Use our generate command instead of cobra's complete command 2021-07-10 19:44:18 +02:00
Michael Eischer 3442dc87fb find: Address review comments 2021-07-06 21:04:34 +02:00
Michael Eischer 95b44490a0 find: search blob ids for pack in index if pack is missing
If a pack file is missing try to determine the contained pack ids based
on the repository index. This helps with assessing the damage to a
repository before running `rebuild-index`.
2021-07-06 21:04:34 +02:00
Michael Eischer 3caab3c7ac find: Print not found pack files 2021-07-06 21:04:34 +02:00
Michael Eischer 40745b4f82 find: stop file listing after resolving all pack files 2021-07-06 21:04:34 +02:00
Michael Eischer 6c01078f3d find: support resolving multiple pack ids to blobs
Just passing the list of blobs to packsToBlobs would also work in most
cases, however, it could cause unexpected results when multiple pack
files have the same prefix. Forget found prefixes to prevent this.
2021-07-06 21:04:34 +02:00
Michael Eischer aad8864835 prune: Add missing newlines in error descriptions 2021-06-20 14:25:40 +02:00
Michael Eischer 454b6d608e key: Add test that failed key saves don't break the repository 2021-06-13 13:46:48 +02:00
Michael Eischer 6add186867 key: Check that a new key file actually works 2021-06-12 23:09:19 +02:00
Michael Eischer fe43f53528 mount: enable fuse readahead
Apparently readahead was disabled by default. Enable readahead with the
Linux default size of 128kB. Larger values seem to have no effect.
This can speed up reading from the fuse mount by at least factor 5.

Speedup for a 1G random file stored in a local repository:
(Only one result shown, but times were quite stable, restarted restic
after each command)
$ dd if=/dev/urandom bs=1M count=1024 of=rand
$ shasum -a 256 tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d  tmp/rand

before: $ time shasum -a 256 fuse/snapshots/latest/tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d  fuse/snapshots/latest/tmp/rand

real    0m18.294s
user    0m4.522s
sys     0m3.305s

before: $ time cat fuse/snapshots/latest/tmp/rand > /dev/null
real    0m14.924s
user    0m0.000s
sys     0m4.625s

after:  $ time shasum -a 256 fuse/snapshots/latest/tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d  fuse/snapshots/latest/tmp/rand

real    0m6.106s
user    0m3.115s
sys     0m0.182s

after:  $ time cat fuse/snapshots/latest/tmp/rand > /dev/null
real    0m3.096s
user    0m0.017s
sys     0m0.241s
2021-06-12 17:07:30 +02:00
rawtaz fdbd65485e
Merge pull request #3402 from MichaelEischer/misc-fixes
Various small code cleanups
2021-05-24 11:30:31 +02:00
rawtaz 2daf033156
Merge pull request #3403 from MichaelEischer/fast-cat
cat: only load index if really necessary
2021-05-24 11:02:10 +02:00
MichaelEischer 5dad45f005
Merge pull request #3312 from soraxas/master
Bump cobra and add completions for fish
2021-05-23 13:58:27 +02:00
Michael Eischer 61b368ddea cat: only load index if really necessary 2021-05-23 13:11:55 +02:00
Michael Eischer fd8bce8184 backup: cleanly shutdown goroutines on error 2021-05-23 13:02:44 +02:00
greatroar d7322a5f36 restic ls --json: print "size":0 for empty files 2021-05-21 21:06:00 +02:00
Tin Lai 9cc1ecdd45
bump cobra and add completions for fish
Signed-off-by: Tin Lai <oscar@tinyiu.com>
2021-05-21 13:47:52 +10:00
Alexander Neumann af3de702c7
Merge pull request #3332 from restic/debug-1999
Merge `debug examine` to salvage damaged pack files
2021-05-18 09:38:40 +02:00
Alexander Neumann 226cd8d4d1
Merge pull request #3310 from MichaelEischer/copy-unstable-tree
`copy` raw tree blobs
2021-05-18 09:36:51 +02:00
Alexander Neumann 4cabad8c34
Merge pull request #3325 from MichaelEischer/fix-mintty-output
Fix windows terminal output for mintty
2021-05-18 09:29:24 +02:00
Michael Eischer 5767c65c62 delete: properly close fileChan if context is canceled 2021-05-17 21:05:54 +02:00
Michael Eischer 5a87a0ba0a find: use Str() to format short ids 2021-05-17 20:56:49 +02:00
Leo R. Lundgren 90d75651e6 backup: Improve wording for --one-file-system description 2021-05-15 00:06:27 +02:00