2
2
mirror of https://github.com/octoleo/restic.git synced 2024-06-11 21:32:21 +00:00
Commit Graph

1582 Commits

Author SHA1 Message Date
Michael Eischer
33b7c84a7a deduplicate string formatting of snapshot metadata
This removes the spurious ")" bracket at the end and normalizes the
metadata format used by the `ls` command.
2024-01-06 20:20:51 +01:00
Michael Manganiello
045aa64558 cmd/copy: Prefix hostname to snapshot display output
This change better resembles the output generated by `Snapshot.String()`,
which includes both username and hostname.

Closes #4506

Before:

```
$ restic copy --from-repo /srv/restic-repo
repository 3666882b opened (version 2, compression level auto)
repository 0085c387 opened (version 2, compression level auto)
created new cache in /home/mike/.cache/restic
[0:00] 100.00%  1 / 1 index files loaded
[0:00]          0 index files loaded

snapshot 32b39a20 of [/home/mike/data] at 2023-10-21 16:01:13.979948154 -0300 -03)
  copy started, this may take a while...
[0:00] 100.00%  1 / 1 packs copied
snapshot 10331fdd saved
```

After:

```
$ restic copy --from-repo /srv/restic-repo
repository 3666882b opened (version 2, compression level auto)
repository 0085c387 opened (version 2, compression level auto)
[0:00] 100.00%  1 / 1 index files loaded
[0:00]          0 index files loaded

snapshot 32b39a20 of [/home/mike/data] at 2023-10-21 16:01:13.979948154 -0300 -03 by mike@desktop)
  copy started, this may take a while...
[0:00] 100.00%  1 / 1 packs copied
snapshot a67bd1ee saved
```
2024-01-06 20:20:46 +01:00
Michael Eischer
4f6b1bb6f6 diff: document limitations regarding metadata 2024-01-06 20:12:47 +01:00
Michael Eischer
3549635243 diff: copy nodes before modifying them for bitrot detection 2024-01-06 20:12:47 +01:00
Daniel Danner
a7dc18e697 Add bitrot detection to "diff" command
This introduces a new modifier to the output of the diff command. It
appears whenever two files being compared only differ in their content
but not in their metadata. As far as we know, under normal
circumstances, this should only ever happen if some kind of bitrot has
happened in the source file. The prerequisite for this detection to work
is that the right-side snapshot of the comparison has been created with
"backup --force".
2024-01-06 20:12:47 +01:00
Michael Eischer
649a6409ee rewrite: cleanup tests 2023-12-24 15:36:22 +01:00
Michael Eischer
c31f5f986c rewrite: Minor cleanups 2023-12-24 15:36:22 +01:00
Michael Eischer
2730d05fce rewrite: Don't walk snapshot content if only metadata is modified 2023-12-24 15:36:22 +01:00
Michael Eischer
893d0d6325 rewrite: cleanup new metadata options and fix no parameters check 2023-12-24 15:36:22 +01:00
Gabriel Kabbe
a02d8d75c2 rewrite: Implement rewriting metadata 2023-12-24 15:36:22 +01:00
Gabriel Kabbe
7bf38b6c50 rewrite: Add test TestRewriteMetadata 2023-12-24 15:36:22 +01:00
Gabriel Kabbe
da1704b2d5 rewrite: Add tests
Pass nil instead of metadata to existing tests
2023-12-24 15:36:19 +01:00
Gabriel Kabbe
3026baea07 rewrite: Add structs for tracking metadata changes
Adds

  * snapshotMetadataArgs, which holds the new metadata as strings parsed from
    the command line

  * snapshotMetadata, which holds the new metadata converted to the
    correct types
2023-12-24 14:43:07 +01:00
Sébastien Gross
f7587be28f mount: detect mountpoint does not exist before opening the repository
Bug #1681 suggests that restic should not be nice to user and should
refrain from creating a mountpoint if it does not exist. Nevertheless,
it currently opens the repository before checking for the mountpoint's
existence. In the case of large or remote repositories, this process
can be time-consuming, delaying the inevitable outcome.

    /restic mount --repo=REMOTE --verbose /tmp/backup
    repository 33f14e42 opened (version 2, compression level max)
    [0:38] 100.00%  162 / 162 index files loaded
    Mountpoint /tmp/backup doesn't exist
    stat /tmp/backup: no such file or directory

    real	0m39.534s
    user	1m53.961s
    sys	0m3.044s

In this scenario, 40 seconds could have been saved if the nonexistence
of the path had been verified beforehand.

This patch relocates the mountpoint check to the beginning of the
runMount function, preceding the opening of the repository.

    /restic mount --repo=REMOTE --verbose /tmp/backup
    Mountpoint /tmp/backup doesn't exist
    stat /tmp/backup: no such file or directory

    real	0m0.136s
    user	0m0.018s
    sys	0m0.027s

Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
2023-12-24 11:54:18 +01:00
Michael Eischer
30e6ed038c
Merge pull request #4586 from Gelma/typos
Fix typos
2023-12-23 13:21:50 +00:00
Michael Kuhn
fd2fb233aa Fix repository not being printed when using repository file
When using `RESTIC_REPOSITORY_FILE` in combination with `restic init`,
the repository is missing in the output:
```
$ restic init
created restic repository 3c872be20f at
[...]
```
This is due to the code using `gopts.Repo`, which is empty in this case.
2023-12-23 13:49:22 +01:00
Michael Eischer
dbbd31bc3a CI: update golangci-lint
Necessary to properly support Go 1.21.
2023-12-23 13:41:30 +01:00
Andrea Gelmini
241916d55b
Fix typos 2023-12-06 13:11:55 +01:00
Michael Eischer
b72de5a883 Merge branch 'patch-release' 2023-11-12 11:52:21 +01:00
Tobias Speicher
ce53ea32c6
Split go_target into go_os and go_arch 2023-11-01 22:43:38 +01:00
Tobias Speicher
10cbc169c1
Use different function to be more consistent with other code 2023-11-01 22:18:37 +01:00
Tobias Speicher
d45cc52468
command version: add json output option 2023-10-31 19:31:07 +01:00
Alexander Neumann
59fe24cb2b Set development version for 0.16.2 2023-10-29 19:50:12 +01:00
Alexander Neumann
1a5efcf680 Add version for 0.16.2 2023-10-29 19:50:07 +01:00
Michael Eischer
ee305e6041 backup: rework error reporting for subcommand 2023-10-27 23:58:52 +02:00
Michael Eischer
317144c1d6 fs: merge command startup into CommandReader 2023-10-27 23:58:51 +02:00
Michael Eischer
7d879705ad fs: cleanup CommandReader implementation 2023-10-27 23:58:51 +02:00
Enrico204
37a312e505 restic-from-command: use standard behavior when no output and exit code 0 from command
The behavior of the new option should reflect the behavior of normal backups: when the command exit code is zero and there is no output in the stdout, emit a warning but create the snapshot. This commit fixes the integration tests and the ReadCloserCommand struct.
2023-10-27 23:58:51 +02:00
Enrico204
c0ca54dc8a restic-from-command: add tests 2023-10-27 23:58:51 +02:00
Enrico204
81f8d473df restic-from-command: abort snapshot on non-zero exit codes 2023-10-27 23:58:51 +02:00
Enrico204
4e5caab114 stdin-from-command: implemented suggestions in #4254
The code has been refactored so that the archiver is back to the original code, and the stderr is handled using a go routine to avoid deadlock.
2023-10-27 23:58:51 +02:00
Sebastian Hoß
25350a9c55 Extend SnapshotOptions w/ command data
In order to determine whether to save a snapshot, we need to capture the exit code returned by a command. In order to provide a nice error message, we supply stderr as well.

Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
a2b76ff34f Start command from --stdin-from-command
It acts similar to --stdin but reads its data from the stdout of the given command instead of os.Stdin.

Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
333fe1c3cf Align Stdin and StdinCommand in conditionals
In order to run with --stdin-from-command we need to short-circuit some functions similar to how it is handled for the --stdin flag. The only difference here is that --stdin-from-command actually expects that len(args) should be greater 0 whereas --stdin does not expect any args at all.

Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
a8657bde68 Add --stdin-from-command option
This new flag is added to the backup subcommand in order to allow restic to control the execution of a command and determine whether to save a snapshot if the given command succeeds.

Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Michael Eischer
731b3a4357 stats: fix hardlink tracking in a snapshot
inodes are only unique within a device. Use the HardlinkIndex from the
restorer instead of the custom (broken) hashmap to correctly account for
both inode and deviceID.
2023-10-27 23:40:42 +02:00
Michael Eischer
249605843b prune: get backend connection count via repository 2023-10-25 23:01:54 +02:00
Michael Eischer
c7b770eb1f convert MemorizeList to be repository based
Ideally, code that uses a repository shouldn't directly interact with
the underlying backend. Thus, move MemorizeList one layer up.
2023-10-25 23:01:35 +02:00
Michael Eischer
1b8a67fe76 move Backend interface to backend package 2023-10-25 23:00:18 +02:00
Alexander Neumann
27ec320eae Set development version for 0.16.1 2023-10-24 20:02:58 +02:00
Alexander Neumann
baca3f6303 Add version for 0.16.1 2023-10-24 20:02:55 +02:00
Michael Eischer
ead8dd0173
Merge pull request #4524 from MichaelEischer/fix-autocompletion
Don't load password when autocompleting CLI commands
2023-10-23 20:06:53 +00:00
Michael Eischer
cd09ef4614 don't load password when autocompleting CLI commands 2023-10-23 21:44:22 +02:00
Michael Eischer
54a4034ec0 Only show index progress bar if stdout is a terminal
This ensures that there is no interference when using restic for
scripting use cases.
2023-10-23 21:16:52 +02:00
Michael Eischer
d1d4510974 repair packs: Add stub feature flag implementation 2023-10-23 18:36:28 +02:00
Michael Eischer
a28940ea29 check: Suggest usage of restic repair packs for corrupted blobs
For now, the guide is only shown if the blob content does not match its
hash. The main intended usage is to handle data corruption errors when
using maximum compression in restic 0.16.0
2023-10-23 18:36:28 +02:00
Michael Eischer
db26dc75e1 repair packs: add experimental command
This allows recovering a repository from several cases of damaged blobs.
2023-10-23 18:36:28 +02:00
Michael Eischer
ceff4af1ac Disable log output from automaxprocs library 2023-10-22 17:09:46 +02:00
Michael Gmelin
34f3b13b7c Make key list command honor --no-lock
Fixes #4513
2023-10-19 22:09:35 +02:00
Michael Eischer
495982232c
Merge pull request #4511 from dnnr/inode-in-json
Include inode in JSON output for find and ls
2023-10-17 21:06:34 +00:00
Michael Eischer
f5d5e8fd0a automatically adjust GOMAXPROCS when run in a cpu-constrained container 2023-10-17 20:31:57 +02:00
Daniel Danner
a858ab254b Include inode in JSON output for find and ls
This introduces the inode attribute to the JSON output emitted for nodes
in `ls` and matches in `find`. There doesn't seem to be any discernible
reason to omit the inode and it can be useful in scripting scenarios.
2023-10-11 21:10:01 +02:00
Michael Eischer
91aef00df3 check: add index loading progress bar 2023-10-01 19:55:29 +02:00
Michael Eischer
b0da0f152f deduplicate index progress bar setup 2023-10-01 19:53:26 +02:00
Michael Eischer
75f6bd89ed hide index loading progress for JSON output 2023-10-01 19:53:26 +02:00
Michael Eischer
3fd0ad7448 repository: list index files only once 2023-10-01 19:53:26 +02:00
arjunajesh
ed65a7dbca implement progress bar for index loading 2023-10-01 19:52:59 +02:00
Michael Eischer
e60c5b2d7f
Merge pull request #4436 from MichaelEischer/add-snapshot-subfolder-syntax-to-help-text
Add snapshot subfolder syntax in help output of diff/dump/restore
2023-08-13 19:36:38 +02:00
Michael Eischer
a04964bb86
Merge pull request #4446 from 2000yeshu/feat-cat
Check for arguments before opening remote connection in cat command
2023-08-13 17:25:37 +00:00
Michael Eischer
fe54912a46 cat: extract parameter validation and add a test 2023-08-13 19:17:46 +02:00
Yakul Garg
705556f134
Check for arguments before opening remote connection in cat command
Signed-off-by: Yakul Garg <2000yeshu@gmail.com>
2023-08-12 23:54:40 +05:30
Michael Eischer
0fca028491 restore: Fix help text for --iinclude and --iexclude 2023-08-05 18:21:03 +02:00
Michael Eischer
3a478bc522 unify snapshotID spelling in command parameters 2023-08-05 12:35:45 +02:00
Michael Eischer
9a7704fa2b diff/dump/restore: describe snapshot:subfolder syntax in help text 2023-08-05 12:35:08 +02:00
Alexander Neumann
82e6e28781 Set development version for 0.16.0 2023-07-31 20:24:29 +02:00
Alexander Neumann
eb33e564c9 Add version for 0.16.0 2023-07-31 20:23:24 +02:00
Michael Eischer
3f919f2371 forget: simplify test 2023-07-28 19:21:40 +02:00
Michael Eischer
50b43fbac0 forget: replace --keep-* -1 with --keep-* unlimited
This ensures consistency with the `prune --max-unused unlimited` option.
2023-07-28 19:21:40 +02:00
Michael Eischer
6c5fc32967
Merge pull request #3225 from wojas/restic-cmd-more-help
Extend help output with short overview
2023-07-23 15:43:03 +00:00
Michael Eischer
ae13cf15c1 doc: move quickstart to docs 2023-07-23 17:32:37 +02:00
arjunajesh
32f5ee6f4e snapshots sorted by timestamp 2023-07-23 12:49:23 +02:00
Konrad Wojas
5bd8a6d7eb Extend help output with short overview
The list of subcommands can be confusing for first time users. This
adds a short overview of the most important commands to help get people
started.
2023-07-23 12:01:40 +02:00
Michael Eischer
25ff9fa893
Merge pull request #4334 from MichaelEischer/snapshot-subtree-syntax
Add support for snapshot:path syntax
2023-07-22 23:59:26 +02:00
Michael Eischer
76253b2a20 Hopefully fix flaky TestLockWaitCancel test 2023-07-22 23:23:17 +02:00
Michael Eischer
b8f4267a36 lock: add more debug information to flaky test 2023-07-22 23:23:17 +02:00
Michael Eischer
1f6883a05c lock: test stale lock refresh 2023-07-22 23:23:17 +02:00
Michael Eischer
8154f6a77a Ensure consistent naming for <snapshot>:<subfolder> syntax 2023-07-22 19:55:57 +02:00
Michael Eischer
233b841ad9 Add cat tree snapshot:subpath command
This command prints the tree for the given path.
2023-07-22 19:55:57 +02:00
Michael Eischer
85860e6e97 Add support for snapshot:subpath syntax
This snapshot specification syntax is supported by the cat, diff, dump,
ls and restore command.
2023-07-22 19:55:57 +02:00
Michael Eischer
2dd6769429 lock: Fix possible deadlock during refresh of stale lock
A delayed lock refresh could send a signal on the `refreshed` channel
while the `monitorLockRefresh` goroutine waits for a reply to its
`refreshLockRequest`. As the channels are unbuffered, this resulted in a
deadlock.
2023-07-22 19:55:44 +02:00
Michael Eischer
399f8e84a1 lock: use mem backend for locking tests
This hopefully fixes the timing related test failures which appear to be
caused by very slow fsync calls.
2023-07-22 19:55:44 +02:00
Michael Eischer
6a436d731d lock: rename test helper function 2023-07-22 19:55:44 +02:00
Michael Eischer
b2fcbc21cb lock: rework stale lock refresh to avoid data race 2023-07-22 19:55:44 +02:00
Michael Eischer
05e5e29a8c backup: don't pretend to lock repo for dry run
The dry run did not actually create lock files, but still ran all other
parts of the lock processing.
2023-07-22 19:55:44 +02:00
Michael Eischer
f490288738 lock: freeze backend operations while refreshing stale lock
Freeze new backend operations while trying to refresh a stale lock.
2023-07-22 19:55:44 +02:00
Michael Eischer
11eb88a2ea lock: cleanup 2023-07-22 19:55:44 +02:00
Michael Eischer
bee3231ed4 lock: try refreshing of stale locks
A stale lock may be refreshed if it continues to exist until after a
replacement lock has been created. This ensures that a repository was
not unlocked in the meantime.
2023-07-22 19:55:44 +02:00
Michael Eischer
47206a6579
Merge pull request #4351 from kenny-y-dev/human-readable-ls
Add --human-readable to ls and find output
2023-07-16 11:44:41 +00:00
Michael Eischer
89fbd39e59 Don't print stacktrace on invalid cacert option 2023-07-09 14:23:54 +02:00
Michael Eischer
1ce599d2ae Fix handling of empty cacert environment variable
This resulted in a "empty filename for root certificate supplied" error.
2023-07-09 14:23:54 +02:00
Michael Eischer
325fa916b5 stats: Add debug mode to collect repository statistics 2023-07-08 23:04:47 +02:00
Michael Eischer
cc3c218baf small cleanups for certificate environment variables 2023-07-08 09:44:59 +02:00
arjunajesh
c1578a2035 certificates can be passed through env vars 2023-07-08 09:44:59 +02:00
Michael Eischer
389f6ee74c backup: add minimal test for program versioni 2023-07-07 23:33:23 +02:00
Michael Eischer
bbac74b172 add program version to snapshot 2023-07-07 23:27:10 +02:00
greatroar
41a5bf357f cmd, ui: Move size parsing code and make it more robust 2023-07-02 21:44:50 +02:00
Michael Eischer
cbf87fbdb3 init: don't include password in debug log 2023-06-17 15:15:58 +02:00
Michael Eischer
50e0d5e6b5 backend: Hardcode backend scheme in Factory
Our ParseConfig implementations always expect a specific scheme, thus no
other scheme would work.
2023-06-17 15:15:58 +02:00
Michael Eischer
3a3cf608f5 b2/s3: Move config validation from ApplyEnvironment to Open/Create
Conceptually the backend configuration should be validated when creating
or opening the backend, but not when filling in information from
environment variables into the configuration.
2023-06-17 15:15:58 +02:00
Michael Eischer
9aa9e0d1ec local/sftp: move limiter setup into backend 2023-06-17 15:15:58 +02:00