Commit Graph

455 Commits

Author SHA1 Message Date
Zlatko Čalušić 134f834c60 Use lowercase in messages from check/tag commands, too 2017-10-27 21:06:34 +02:00
Konrad Wojas c5da90a5b7 Add --last flag to snapshots command
Add --last flag to snapshots command to only show the last entry for any
(hostname, paths) combination.

This makes it easier to check when various paths were last backed up.
2017-10-26 14:02:29 +08:00
Alexander Neumann 8d37b723ca Upadte vendored dependencies 2017-10-22 10:07:36 +02:00
TobyLL 9e3dde8ec7 Fix whitespace 2017-10-19 15:52:06 +01:00
TobyLL 9dba182e51 Ignore comments (lines starting with #) in the --files-from file 2017-10-19 15:48:22 +01:00
Alexander Neumann beaf55f1fc dump: Allow absolute paths 2017-10-16 20:22:01 +02:00
Alexander Neumann 0e7281eb71 Merge pull request #1346 from fawick/cmdCatFile
Add dump sub-command for dumping files to stdout
2017-10-16 20:13:44 +02:00
Alexander Neumann 897c923cc9 Retry failed backend requests 2017-10-14 15:56:25 +02:00
Alexander Neumann 67193e3deb Improve error message when creating lock failed 2017-10-14 15:56:25 +02:00
Fabian Wickborn d1d9c3f9d7 Renamed cmd_catfile.go to cmd_dump.go 2017-10-14 13:55:21 +02:00
Fabian Wickborn cd5cbe0910 Rename debug dump related variable and run function 2017-10-14 13:55:00 +02:00
Fabian Wickborn 814e992c0b Rename subcommand catfile to dump 2017-10-14 11:34:04 +02:00
Fabian Wickborn 87d084e18c Add subcommand dump 2017-10-12 20:24:34 +02:00
Alexander Neumann f2314b26ba Move 'dump' to 'debug dump' 2017-10-12 20:18:45 +02:00
Alexander Neumann 74dcf41f25 Disable 'dump' for non-debug builds 2017-10-12 20:14:48 +02:00
rmdashrf 32637a0328 Basic rate limiting implementation.
Added `--limit-upload` and `--limit-download` flags to rate limit
backups and restores.
2017-10-11 20:01:20 -07:00
Alexander Neumann c5553ec855 Merge pull request #1276 from fawick/supply_ca_cert
Add REST backend option to use CA root certificate
2017-10-08 09:47:23 +02:00
Alexander Neumann bb3ed54291 Merge pull request #1337 from mungomat/fuse_updateRepo
fuse: mount and backup in parallel (#1330)
2017-10-08 09:45:28 +02:00
Tobias Klein 02200acad0 fuse: mount and backup in parallel (#1330) 2017-10-07 11:48:48 +02:00
Alexander Neumann 1a2d190bdb Merge pull request #1334 from felix9/backup_tags
backup should use latest parent regardless of tags
2017-10-07 10:45:02 +02:00
Fabian Wickborn f0f17db847 Cache evaluated directories in isExcludedByFile
Fixes #1271.
2017-10-06 22:11:22 +02:00
Felix Lee 7b44fd0f9d fix #1143
Backup was choosing a parent snapshot that had the same tags, which
makes backup unnecessarily slow when there are newer snapshots with
different tags.

There's no reason parent has to have the same tags.

This change makes backup choose the newest snapshot instead.
2017-10-05 11:48:26 -07:00
Fabian Wickborn 69a6e622d0 Add REST backend option to use CA root certificate
Closes #1114.
2017-10-04 22:14:10 +02:00
Alexander Neumann 5d1c1f721e Merge pull request #1324 from antonlindstrom/password-feedback
Add password successful feedback
2017-10-04 21:31:24 +02:00
Alexander Neumann cdef55bb88 Merge pull request #1312 from hgfischer/issue-1119
nit: please remove dot imports #1119
2017-10-04 21:01:40 +02:00
Anton Lindstrom a7baea0522 Output password successful on terminal stdout
This removes the conditions that checks if the password is supplied
through environment variable or file and outputs password is successful
on terminal and when --quiet is not supplied.
2017-10-04 14:55:04 +02:00
Anton Lindstrom 55e6003749 Add password successful feedback
This adds some feedback when entering the password on the command line.
When the password is entered and supplied through stdin (and stdout is a
terminal) then the a message saying `password is correct` if correct is
printed.
2017-10-04 13:45:05 +02:00
Alexander Neumann 657a1d75af snapshots: Add short ID to JSON output
We're using the short ID in all output to users, so it should also be
included in the JSON output of the `snapshots` command.
2017-10-03 15:24:09 +02:00
Anton Lindstrom d46314648e Add explicit OK output to check command
This adds additional output to the check command when no errors were
found. It means that when all checks have been completed, the following
output is displayed:

	No errors were found

The output is added to make sure that it is easier to understand that no
errors were found.

Full example output:

	Create exclusive lock for repository
	Load indexes
	Check all packs
	Check snapshots, trees and blobs
	No errors were found
2017-10-03 12:57:51 +02:00
Alexander Neumann e45011af57 Merge pull request #1304 from jannickfahlbusch/showAmountOfSnapshots
Show the number of snapshots
2017-10-03 12:41:47 +02:00
Alexander Neumann fb09884893 Merge pull request #1311 from felix9/dump-no-trees
"dump" command no longer has a "trees" option
2017-10-03 12:40:24 +02:00
Alexander Neumann d1eecafa63 Merge pull request #1314 from jannickfahlbusch/fixTypo
Fix typo in comment
2017-10-03 12:39:24 +02:00
Alexander Neumann 034b0b8040 forget: Run prune for manually forgotten snapshots 2017-10-03 11:56:13 +02:00
Jannick Fahlbusch 3dcacb3730
Fix typo in comment 2017-10-02 18:19:22 +02:00
Herbert 033589a66b Use rtest on these as well to keep codebase consistent 2017-10-02 17:48:08 +02:00
Jannick Fahlbusch e46a647c45
Just print the number of snapshots 2017-10-02 17:05:59 +02:00
Herbert 3473c3f7b6 Remove all dot-imports 2017-10-02 15:06:39 +02:00
Jannick Fahlbusch 2f10e25738
Show the amount of snapshots 2017-10-01 17:14:54 +02:00
Felix Lee 555bd257bd dump command no longer has 'trees' option 2017-10-01 08:04:52 -07:00
Alexander Neumann dad1c87afe Rework generation of manpages and completion files
This commit removes the `manpages` and `autocomplet` commands and
replaces them with the more generic `generate` command. Also, zsh
completion file support was added.
2017-09-26 13:12:12 +02:00
Alexander Neumann 56fccecd06 prune: Repack mixed pack files 2017-09-24 21:54:53 +02:00
Alexander Neumann 62ba9f1950 check: Disable cache by default 2017-09-24 21:54:53 +02:00
Alexander Neumann 610b676444 Automatically exclude current restic cache 2017-09-24 21:54:53 +02:00
Alexander Neumann 9be24a1c9f Add cache
This commits adds rudimentary support for a cache directory, enabled by
default. The cache directory is created if it does not exist. The cache
is used if there's anything in it, newly created snapshot and index
files are written to the cache automatically.
2017-09-24 21:54:53 +02:00
Fabian Wickborn ea1ab96749 Always sort snapshots lists ascending by timestamp
Fixes #1219.
2017-09-24 13:01:13 +02:00
Alexander Neumann 9c6b7f688e Merge pull request #1270 from restic/sftp-allow-password-prompt
sftp: Allow password entry
2017-09-23 22:13:04 +02:00
Alexander Neumann fb9729fdb9 sftp: Allow password entry
This was a bit tricky: We start the ssh binary, but we want it to ignore
SIGINT. In contrast, restic itself should process SIGINT and clean up
properly. Before, we used `setsid()` to give the ssh process its own
process group, but that means it cannot prompt the user for a password
because the tty is gone.

So, now we're passing in two functions that ignore SIGINT just before
the ssh process is started and re-install it after start.
2017-09-23 11:43:33 +02:00
Alexander Neumann 45a09c76ff Allow suspending SIGINT handler 2017-09-23 11:12:56 +02:00
Dmitriy Morozov ae60188eb9 Add --compact option to forget 2017-09-22 16:32:59 -07:00
Alexander Neumann e7de3b5f9d Merge pull request #1266 from JaCoB1123/document_hostname_option
Add note about rescan to hostname flag (fixes #1221)
2017-09-22 15:40:24 +02:00
Jan Bader 727fb6eabe Add note about rescan to hostname flag (fixes #1221) 2017-09-22 14:29:04 +02:00
Alexander Neumann a4577769ae Merge pull request #1262 from mafgh/bytes
fix duplicate bytes in prune output
2017-09-22 11:59:10 +02:00
Stefan Völkel 7f927d4774 fix duplicate bytes in prune output 2017-09-22 10:07:24 +02:00
Tobias Klein 1a83a739dc fuse: added symlink 'latest' to snapshots-dir 2017-09-21 16:41:20 +02:00
Alexander Neumann d7e644272f prune: Add plausibility check 2017-09-19 10:50:07 +02:00
Alexander Neumann fca9a523e9 Merge pull request #1241 from restic/fix-timestamp-check
Use .Equal() instead of == for time.Time
2017-09-16 14:54:26 +02:00
Alexander Neumann a60e751217 Use .Equal() instead of == for time.Time
Closes #1238
2017-09-15 20:57:35 +02:00
Alexander Neumann 81c5d8a968 Correct bash completion file path 2017-09-15 20:45:16 +02:00
Alexander Neumann c8c01a5cae Merge pull request #1223 from mrzv/snapshots-compact
Add --compact option to snapshots
2017-09-13 14:02:31 +02:00
Alexander Neumann 0f25ef9498 Merge pull request #1230 from shayneholmes/update-short-help
Update style in short help commands
2017-09-11 22:25:31 +02:00
Alexander Neumann 36eaa22ed0 Merge pull request #1205 from mungomat/backup_time
Backup time
2017-09-11 19:01:41 +02:00
Shayne Holmes 9eb39cef05 Capitalize short help commands
Unify existing Cobra help command, and git-help's style.
2017-09-11 09:32:44 -07:00
Shayne Holmes ee6150f67c Change short help messages to imperative voice
Unify the output of `restic help`.
2017-09-11 09:26:13 -07:00
Alexander Neumann b1af544b1d Merge pull request #1224 from restic/improve-exclude-caches
Allow multiple exclude-if-present
2017-09-11 17:31:24 +02:00
Dmitriy Morozov 7a221f2473 Run changes through gofmt 2017-09-10 15:09:28 -07:00
Dmitriy Morozov bdbe956c5c Add --compact option to snapshots
With --compact, snapshots doesn't list directories and puts all tags on a single
line. This way each snapshot takes up exactly one line.
2017-09-10 13:06:43 -07:00
Alexander Neumann b6aeea425b Merge pull request #1196 from mungomat/forget_groupByTagsOnly
forget: group-by-tags-only
2017-09-10 20:52:15 +02:00
Alexander Neumann a8aa4eb06c Rename parameter filename -> path 2017-09-10 20:28:21 +02:00
Alexander Neumann 0b2947dedb Add test for rejectByPattern 2017-09-10 15:31:58 +02:00
Alexander Neumann 47ddd34266 Improve test 2017-09-10 15:20:41 +02:00
Alexander Neumann 2fdca5d310 Improve debug message 2017-09-10 15:16:05 +02:00
Alexander Neumann e5d4e33509 Improve error message if no targets specified 2017-09-10 15:14:11 +02:00
Alexander Neumann e117f613af Move device test into new RejectFunc 2017-09-10 15:13:40 +02:00
Alexander Neumann 0dfdf02885 Rework pattern excludes 2017-09-10 14:34:28 +02:00
Alexander Neumann 4a0129fc2b Rename excludeByFile -> rejectIfPresent 2017-09-10 14:25:58 +02:00
Alexander Neumann a9c705009c Move reject functions to new file 2017-09-10 14:25:25 +02:00
Alexander Neumann d937ad8cf6 Rename FilenameCheck to RejectFunc
We already have the opposite: pipe.SelectFunc(item string, fi
os.FileInfo) bool, so RejectFunc is a good name.
2017-09-10 14:21:51 +02:00
Alexander Neumann 36e70228f2 Handle invalid key file 2017-09-10 10:55:01 +02:00
Alexander Neumann c22c582546 Allow multiple exclude-if-present 2017-09-09 21:24:29 +02:00
Alexander Neumann ea75509d6e Print warning for non-existing items 2017-09-09 21:12:41 +02:00
Tobias Klein ed30bd7b76 gofmt 2017-09-09 18:19:19 +02:00
Tobias Klein 8f9ef4402b error in case of unknown grouping option 2017-09-09 15:55:37 +02:00
Tobias Klein 43ff971dfd new sub-option for backup: time
New option to specify the timestamp for a backup
2017-09-09 13:26:35 +02:00
Fabian Wickborn dbda892542 Add option to exclude directories with a tagfile
The option is named --exclude-if-present and accepts a parameter
filename[:content]. Directories are excluded and their contents is not
backed up if they contain a file with the specified name and,
optionally, that starts with the specified content. The tagfile itself
is never excluded.

There is also a shortcut --exclude-caches that works in the same way as
the likewise-named option of tar(1): Directories are recognized as cache
if they contain a file named "CACHEDIR.TAG.

Closes #317.
2017-09-09 09:57:42 +02:00
Tobias Klein 1073bfba37 flexible grouping option for the forget-command 2017-09-06 20:14:18 +02:00
Alexander Neumann fa2ee78a5c Merge pull request #1044 from lloeki/982-improve-restore
Improve restore
2017-09-04 21:51:12 +02:00
Tobias Klein e4a5cdc5bc forget: group-by-tags-only 2017-09-03 17:11:25 +02:00
Alexander Neumann 0ee1650f82 Merge pull request #1191 from prattmic/profile
debug: properly handle interrupted profiles
2017-09-03 09:49:46 +02:00
Alexander Neumann de8c64e767 Use deterministic date for man pages 2017-09-02 09:27:11 +02:00
Michael Pratt e4c469c149 debug: properly handle interrupted profiles
By default (i.e., without profile.NoShutdownHook), profile.Start listens
for SIGINT and will stop the profile and call os.Exit(0).

restic already listens for SIGINT and runs its own cleanup handlers
before calling os.Exit(0).

As is, these handlers are racing when an interrupt occurs, and in my
experience, restic tends to win the race, resulting in an unusable
profile.

Eliminate the race and properly stop profiles on interrupt by disabling
package profile's signal handler and instead stop the profile in a
restic cleanup handler.
2017-08-28 22:03:26 -07:00
Ricardo Seriani 62ed776a8c Change key rm command to key remove
Change key rm command to key remove, to follow manual and other commands
2017-08-17 11:03:26 -03:00
Loic Nageleisen f880ff21aa Fixing restore with excluded
An exclude filter is basically a 'wildcard but foo', so even if a
childMayMatch, other children of a dir may not, therefore childMayMatch
does not matter, but we should not go down unless the dir is selected
for restore.
2017-08-16 15:25:02 +02:00
Loic Nageleisen 4a36993c19 Smarter filter when children won't match
This improves restore performance by several orders of magniture by not
going through the whole tree recursively when we can anticipate that no
match will ever occur.
2017-08-16 15:25:02 +02:00
Alexander Neumann b67c178672 Merge pull request #1149 from restic/azure-support
Add Azure blob storage as backend
2017-08-09 21:30:35 +02:00
Dipta Das ba75a3884c Add Google Cloud Storage as backend
Environment variables:
GOOGLE_PROJECT_ID=gcp-project-id
GOOGLE_APPLICATION_CREDENTIALS=path-to-json-file

Environment variables for test:
RESTIC_TEST_GS_PROJECT_ID=gcp-project-id
RESTIC_TEST_GS_APPLICATION_CREDENTIALS=path-to-json-file
RESTIC_TEST_GS_REPOSITORY=gs:us-central1/test-bucket

Init repository:
$ restic -r gs🪣/[prefix] init
2017-08-06 21:47:55 +02:00
Alexander Neumann d91d89eef6 azure: Create container if it does not exist 2017-08-06 21:47:04 +02:00
Dipta Das 3a85b6b7c6 Add Azure Blob Storage as backend
Environment variables:
AZURE_ACCOUNT_NAME=storage-account-name
AZURE_ACCOUNT_KEY=storage-account-key

Environment variables for test:
RESTIC_TEST_AZURE_ACCOUNT_NAME=storage-account-name
RESTIC_TEST_AZURE_ACCOUNT_KEY=storage-account-key
RESTIC_TEST_AZURE_REPOSITORY=azure:restic-test-container

Init repository:
$ restic -r azure:container-name:/prefix/dir init
2017-08-06 21:47:04 +02:00
Alexander Neumann 6bc43a4198 manpage: Remove auto gen tag from man page 2017-08-06 21:31:01 +02:00
Alexander Neumann e348b3deeb manpage: Do not panic when no command is given 2017-08-06 21:01:49 +02:00
Alexander Neumann 739350fd8e backup: Do not print hostname in help text
This is necessary so that the manpage generation is deterministic and we
can test if the man pages are up to date when the CI tests run.
2017-08-05 12:05:53 +02:00
Alexander Neumann a3ab17b470 Add 'manpage' command to generate manual pages 2017-08-05 10:57:01 +02:00
Johannes Kohnen 37aad2e3aa Close exclude files and check errors 2017-08-01 17:34:27 +00:00
donat 893bc9f777 Error message in case PasswordFile is missing 2017-07-27 14:23:08 +03:00
Michael Stapelberg b89d3cc4d0 fix typo: explicitely → explicitly 2017-07-27 08:24:53 +02:00
Alexander Neumann 2e804511ca Merge pull request #1133 from middelink/fix-1132
Force restic to ask the password when adding a key.
2017-07-26 22:11:37 +02:00
Pauline Middelink c95e2b009e Simpify cmd_backup and cmd_init now we have the password in gopts. 2017-07-24 23:32:55 +02:00
Pauline Middelink d5615a67c8 Refactor password resolving.
Instead of determining the password lazily during ReadPassword(), do so now in
cobra.PersistentPreRunE() so we can store the result in the globalOptions and
reuse/override when applicable without having to worry about the environment
or flag options interfering.
2017-07-24 23:05:37 +02:00
Pauline Middelink d9b9bbd4a8 Force restic to ask the password when adding a key.
As `restic key add` uses the same `ReadPasswordTwice()` as the
rest of restic, it is sensitive to the environment variable
RESTIC_PASSWORD or --password-file= override.

When asking for the new key, temporary remove these 2 overrides, forcing
the password to be asked.
2017-07-24 22:00:44 +02:00
Alexander Neumann d780b1eede Rework withTestEnvironment
Switch from a function passed as a parameter to a cleanup function,
which is also executed when the test function panics, so no temporary
directories are left behind.
2017-07-24 21:32:34 +02:00
Pauline Middelink 1717391f6c Fuse testing leaves test mountpoint around. Move it under the testing tree which is removed after each test. 2017-07-24 20:33:39 +02:00
Alexander Neumann 23c903074c Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
Alexander Neumann 94b27e8933 Fix paths for tests 2017-07-23 14:39:57 +02:00
Alexander Neumann 6caeff2408 Run goimports 2017-07-23 14:21:03 +02:00
Alexander Neumann 83d1a46526 Moves files 2017-07-23 14:19:13 +02:00
Alexander Neumann b63399d606 Move things around for gb
This moves all restic source files to src/, and all vendored
dependencies to vendor/src.
2016-02-20 17:31:20 +01:00
Alexander Neumann 1287b307ac Add cleanup handler to restore terminal state
Closes #402
2016-02-13 18:29:26 +01:00
Alexander Neumann 4ec5050cbb backup: Hide status output for narrow terminals
closes #428
2016-02-10 17:28:48 +01:00
Rached Ben Mustapha ba35ca522a Handle pack loading errors in rebuild-index
Errors returned from backend.LoadAll() were not handled, leading to
these fatal errors from the unpacker trying to read the size from the end of
an empty buffer:

`seeking to read header length failed: bytes.Reader.Seek: negative position`

This change takes care of returning on error, as well as showing which pack
failed to load and validating pack integrity.
2016-02-07 18:30:47 +00:00
Alexander Neumann 2c3a6a6fa9 cmd_rebuild_index: Remove calls to GetReader() 2016-01-24 00:42:04 +01:00
Alexander Neumann 61551b0591 cmd_cat: Remove calls to GetReader() 2016-01-24 00:42:04 +01:00
Alexander Neumann f05a32509e Add "Test" prefix to backend test functions 2016-01-23 19:12:02 +01:00
Alexander Neumann 6ba56befad Abort fuse integration test on error
Before, the fuse integration test was run and the tests were never
finished, because the testing code did not detect any errors when the
fusermount binary returned an error. This commit fixes it.
2016-01-23 19:10:43 +01:00
Alexander Neumann 8b7bf8691d backend: Remove Get()
This is the first commit that removes the (redundant) Get() method of
the backend interface. Get(x, y) is equivalent to GetReader(x, y, 0, 0).
2016-01-23 13:13:05 +01:00
Alexander Neumann 1dd4c52a8b Add comments, configure flag library 2016-01-17 18:48:05 +01:00
Alexander Neumann 1922a4272c s3: fix usage
Ignore error response for existing bucket, add more debug code.
2016-01-17 18:48:05 +01:00
Alexander Neumann 7d5f8214cf use new backend open with config 2016-01-17 18:48:05 +01:00
Alexander Neumann 3ac1d0e4d1 add progress 2015-12-06 17:29:31 +01:00
Alexander Neumann 0e66a66bce read packs concurrently 2015-12-06 17:09:06 +01:00
Alexander Neumann a1440c819b Make NoLock a global option 2015-11-13 12:33:59 +01:00
Alexander Neumann 6edb7e02d0 Allow restoring and listing without locking 2015-11-10 22:13:53 +01:00
Alexander Neumann c79dcbd7c4 Allow checking a read-only repo with --no-lock 2015-11-10 21:41:22 +01:00
Alexander Neumann acba82c8f7 Merge pull request #252 from restic/repack-blobs
WIP: Repack blobs
2015-11-09 20:57:57 +01:00
Alexander Neumann 1f9aea9905 fix test on windows, reset read-only flag 2015-11-08 22:41:45 +01:00
Alexander Neumann 742d69bf4d Add another test for optimizing unused blobs 2015-11-08 22:38:17 +01:00
Alexander Neumann 5776b8f01c remove ConvertIndex 2015-11-08 22:27:13 +01:00
Alexander Neumann 2e6eee991d Add test for optimize command with old indexes 2015-11-08 22:21:08 +01:00
Alexander Neumann 43e2c9837e check: removing orphaned packs is handled in 'optimize' 2015-11-08 21:24:51 +01:00
Alexander Neumann c4fc7b52ae Add 'optimize' command that repacks blobs 2015-11-08 21:10:03 +01:00
Alexander Neumann cd948b56ac cmd_check: Don't display unused blobs by default 2015-11-08 20:46:52 +01:00
Alexander Neumann ea41a1045f Add integration test for error on readdirnames 2015-11-06 23:19:56 +01:00
Alexander Neumann 18d7f7f835 Read password from stdin if terminal is not a tty 2015-11-04 22:20:58 +01:00
Alexander Neumann 1fc0d78913 Refactor Index.Store() to take a PackedBlob 2015-11-02 19:07:03 +01:00
Alexander Neumann 50c2f2e87f cmd_cat: allow dumping raw tree blobs 2015-11-02 17:36:04 +01:00
Alexander Neumann fccde030d5 Refactor Index.Lookup() to return struct PackedBlob 2015-10-31 14:47:42 +01:00
Alexander Neumann b5976474dd backup: add debug output for excluded files/dirs 2015-10-27 22:34:30 +01:00
Alexander Neumann 00e05ae3c9 bugfix: close pack files after reading the header 2015-10-27 20:39:52 +01:00
Alexander Neumann 74cd134b54 rebuild index: remember already stored blobs 2015-10-25 22:34:22 +01:00
Alexander Neumann 734ae7fcb8 Add test for corner case
It was observed that a restic repository still contained overlapping
indexes after `rebuild-index` has been called. This is caused by
instantly forgetting that blobs have already been saved once a full
index has been written during index rebuilding.

This commit adds a (failing) test that shows the behaviour.
2015-10-25 21:51:57 +01:00
Alexander Neumann 7b8e42a763 Silence rebuild-index tests 2015-10-25 21:51:46 +01:00
Alexander Neumann efbce9f0fa rebuild-index: handle not yet indexed packs 2015-10-25 18:07:51 +01:00