1427 Commits

Author	SHA1	Message	Date
Alexander Neumann	027a51529d	prune: Improve error message for missing files ... This commit changes the error message so that a list of file names is printed. Before, just the raw map was printed, which is not a great user interface.	2021-01-31 11:31:27 +01:00
Alexander Neumann	f867e65bcd	Fix issues reported by staticcheck	2021-01-30 20:43:53 +01:00
Alexander Neumann	aef3658a5f	Address review comments	2021-01-30 20:02:37 +01:00
Alexander Neumann	200f09522d	Add more error checks	2021-01-30 20:02:37 +01:00
Alexander Neumann	cbd88c457a	backup: Improve error handling	2021-01-30 20:02:37 +01:00
Alexander Neumann	1a0eb05bfa	errcheck: Add more error checks	2021-01-30 20:02:37 +01:00
MichaelEischer	4d576c2f79	Merge pull request #3217 from M4a1x/read-data-subset-fix ... Fix missing rand seed for restic check --read-data-subset=x%	2021-01-29 23:34:13 +01:00
Max Stabel	f9e1fa26ff	Fix missing rand seed for restic check --read-data-subset=x%	2021-01-29 23:18:35 +01:00
Alexander Weiss	e08e65dc30	prune: Simplify logic selecting packs to repack	2021-01-29 22:27:22 +01:00
Alexander Weiss	daeb4cdf8f	prune: Fix statistics for --repack-cacheable-only	2021-01-29 22:27:22 +01:00
Alexander Neumann	a16ce65295	Merge pull request #3244 from MichaelEischer/better-damage-reports ... Print more details about possible repository damages	2021-01-29 11:45:45 +01:00
rawtaz	de7e3a0648	Merge pull request #2823 from greatroar/trust-mtime ... Add --ignore-ctime flag to backup and document change detection	2021-01-29 00:02:19 +01:00
greatroar	6bd8a2faaa	backup: Add --ignore-ctime option and document change detection	2021-01-28 23:42:10 +01:00
Michael Eischer	58b5679f14	prune: reword missing blobs error ... The previous wording could be understood such that the prune run did damage the repository.	2021-01-28 21:48:24 +01:00
Michael Eischer	7b8886c052	prune: report missing but unneeded pack files ... This indicates a damaged repository so add some output to help with debugging.	2021-01-28 21:46:01 +01:00
Michael Eischer	ff95999246	rebuild-index: report added/removed/reindexed files ... This should help with investigating missing pack files.	2021-01-28 21:46:01 +01:00
Michael Eischer	b71c52797a	find: correctly expand multiple blob ids ... For example `restic find --show-pack-id --blob f78dc991 5b9e4366 ddd8c7d4` would previously only expand one blob if all of them belong to the same file.	2021-01-28 21:21:54 +01:00
MichaelEischer	82140967d3	Merge pull request #3228 from aawsome/prune-all-trees ... prune: Remove all unused trees	2021-01-28 21:04:27 +01:00
MichaelEischer	35033d9b79	Merge pull request #3177 from MichaelEischer/fix-2759 ... prune: don't print stacktrace on console	2021-01-28 20:28:45 +01:00
Alexander Neumann	4d40c70214	Merge pull request #3211 from MichaelEischer/sftp-speedup ... Speed-up caching/pack download via SFTP	2021-01-28 14:16:55 +01:00
Alexander Neumann	44169d0dc4	Merge pull request #3205 from MichaelEischer/fix-quiet-verbose ... Properly check that --quiet and --verbose are not combined	2021-01-28 13:53:31 +01:00
Alexander Neumann	bdfedf1f5b	Merge pull request #3173 from MichaelEischer/unify-index-loading ... Unify index loading	2021-01-28 13:50:42 +01:00
Michael Eischer	e2b0072441	check: add progress bar to the tree structure check	2021-01-28 11:10:50 +01:00
Michael Eischer	505f8a2229	progress/counter: Support updating the progress bar maximum	2021-01-28 11:10:47 +01:00
Michael Eischer	eda8c67616	restic: let FindUsedBlobs handle multiple snapshots at once	2021-01-28 11:08:43 +01:00
Michael Eischer	258ce0c1e5	parallel: report progress for StreamTrees ... This assigns an id to each tree root and then keeps track of how many tree loads (i.e. trees referenced for the first time) are pending per tree root. Once a tree root and its subtrees were fully processed there are no more pending tree loads and the tree root is reported as processed.	2021-01-28 11:08:43 +01:00
Michael Eischer	3d6a3e2555	copy: Remove treeCloner struct	2021-01-28 11:08:43 +01:00
Michael Eischer	0caad1e890	copy: parallelize tree walk	2021-01-28 11:08:43 +01:00
Alexander Neumann	a4689eb3b9	Merge pull request #3199 from MichaelEischer/non-interactive-counter ... Don't print progress on non-interactive terminals	2021-01-28 10:53:38 +01:00
Alexander Weiss	d7dc19a496	prune: Always repack packs containing tree blobs	2021-01-15 16:42:04 +01:00
Michael Eischer	f3442ce8a5	Test that WriteTo of a backend's Load remains accessible	2021-01-03 22:23:53 +01:00
Michael Eischer	97388b3504	Properly check that --quiet and --verbose are not combined ... If --verbose is specified once, then globalOptions.Verbose == 1. Previously --quiet --verbose would silently ignore the --verbose flag.	2020-12-30 21:24:18 +01:00
MichaelEischer	e1efc193e1	Merge pull request #3139 from aawsome/prune-healing ... prune: Add healing of repository in some situations	2020-12-29 22:17:27 +01:00
Alexander Weiss	f0113139ea	prune: Correct error message	2020-12-29 20:20:05 +01:00
Alexander Weiss	f6df94a50e	prune: Add self-healing ... Allow prune to heal situations where blobs in the index are missing or the corresponding packfiles are damaged if those blobs are not needed.	2020-12-29 20:20:05 +01:00
MichaelEischer	31e56f1ad5	Merge pull request #3197 from SkYNewZ/fix/3183 ... Fix tag handling for multiple tag lists	2020-12-29 18:44:38 +01:00
greatroar	dec5008369	Warn when unlock fails instead of returning an error ... Only one caller was checking the error.	2020-12-29 17:48:20 +01:00
Quentin Lemaire	25ecf9eafb	fix(cmd_tag): Use restic.TagLists	2020-12-29 17:12:46 +01:00
Quentin Lemaire	e88f3fb80c	fix(cmd_backup): Use restic.TagLists	2020-12-29 17:12:46 +01:00
Michael Eischer	05a987b07c	Support values less than 1 for RESTIC_PROGRESS_FPS ... For example set the variable to 0.016666 to print the progress once per minute.	2020-12-29 16:32:18 +01:00
Michael Eischer	85fe5feadb	Unify progress report frequency calculation	2020-12-29 16:03:43 +01:00
Michael Eischer	969141b5e9	Honor RESTIC_PROGRESS_FPS env variable on non-interactive terminals ... This makes it possible to use the environment variable to also get regular progress updates on non-interactive terminals.	2020-12-29 16:03:43 +01:00
Michael Eischer	c2ef049f1b	ui/progress: don't print progress on non-interactive terminals ... This reverts to the old behavior of not printing progress updates on non-interactive terminals. It was accidentally changed in #3058.	2020-12-29 16:03:43 +01:00
greatroar	18531e3d6f	Portability fixes to internal/restic ... syscall.Mknod is not available on AIX.	2020-12-29 01:35:01 +01:00
DRON-666	332b1896d1	Some options fixes ... Add tests for bool type. Fix subtle bug in TestOptionsApplyInvalid. Fix options list formatting.	2020-12-23 23:26:04 +03:00
Alexander Weiss	d8d2cc6dd9	Simplify rebuild-index code	2020-12-22 23:01:58 +01:00
Alexander Weiss	68b74e359e	Count packs directly in RebuildIndexFiles	2020-12-22 23:01:58 +01:00
Michael Eischer	ccc84af73d	debug/list: parallelize index loading	2020-12-22 22:36:18 +01:00
Alexander Neumann	edf89e1c74	forget: Enable --dry-run together with --prune	2020-12-22 20:58:02 +01:00
Michael Eischer	cfea79d0c5	prune: don't print stacktrace on console	2020-12-19 14:28:48 +01:00
DRON-666	2ca76afc2b	dump: Add new option --archive	2020-12-19 01:09:47 +03:00
DRON-666	0256f95994	dump: Split tar and walk logic	2020-12-19 01:09:47 +03:00
Alexander Neumann	0d5b764f90	Merge pull request #3130 from aawsome/snapshots-parallel ... Make loading snapshots parallel	2020-12-06 21:08:18 +01:00
Alexander Weiss	e329623771	Remove LoadAllSnapshots	2020-12-06 05:22:27 +01:00
Alexander Weiss	5b9ee56335	Add ForAllSnapshots	2020-12-06 05:04:21 +01:00
Alexander Weiss	83c8a9b058	Bugfix: packSizeFromList should save size from List()	2020-12-05 20:58:36 +01:00
Michael Eischer	f72f6c9c80	Fix debug build	2020-11-29 18:44:36 +01:00
MichaelEischer	52b98f7f95	Merge pull request #3017 from greatroar/files-from0 ... Add backup options --files-from-verbatim and --files-from-raw	2020-11-29 18:15:21 +01:00
Alexander Neumann	04d856e601	Merge pull request #3136 from restic/rawtaz-copy-doc ... doc: Emphasize double transfer and duplication in copy command	2020-11-29 13:57:10 +01:00
Alexander Neumann	a7b49c4889	Merge pull request #3119 from restic/keep-mountpoints ... Keep mountpoints as empty directories for --one-file-system	2020-11-29 11:22:12 +01:00
Leo R. Lundgren	f70b10d0ee	doc: Emphasize double transfer and duplication in copy command	2020-11-28 19:49:34 +01:00
greatroar	55bf76ba0c	backup: Add --files-from-{verbatim,raw} options	2020-11-28 18:22:31 +01:00
Alexander Neumann	82ae942965	backup: Keep mountpoints for --one-file-system ... When a file system is mounted at a directory, lstat() returns attributes of the root node of the mounted file system, including the device ID of the other file system. The previous code used when --one-file-system is specified excluded the directory itself because of that. This commit changes the code so that mountpoints are kept as empty directories, its attributes set to the root note of the mounted file system. The behavior mimics `tar`, which does the same.	2020-11-28 17:00:31 +01:00
Alexander Neumann	f576d3d826	Add tests	2020-11-28 17:00:31 +01:00
Alexander Neumann	037f0a4c91	Refactor device ID checking	2020-11-28 17:00:31 +01:00
Alexander Neumann	a3105799c9	Print warning when unlocking the repo fails	2020-11-28 11:31:25 +01:00
Kyle Brennan	666768cd17	cat: Respect --no-lock flag	2020-11-28 11:30:02 +01:00
Alexander Weiss	aa7a5f19c2	Use BlobHandle in index methods	2020-11-22 20:41:12 +01:00
Alexander Weiss	92bd448691	Make BlobHandle substruct of Blob	2020-11-22 20:41:10 +01:00
Alexander Weiss	67c938f232	Use PackSize in prune	2020-11-21 22:13:54 +01:00
Alexander Weiss	4960b841e6	Use PackSize in rebuild-index	2020-11-21 22:13:54 +01:00
Alexander Neumann	110a32a08b	Merge pull request #3113 from aawsome/fix-prune-stats-duplicates ... Fix statistics in prune for duplicates	2020-11-19 20:34:11 +01:00
Alexander Weiss	cb5ec7ea6b	Fix statistics in prune for duplicates ... Note that this fix only solves the statistics problem, if all duplicates are marked for repacking. If not all duplicates are marked for repacking, we lack the information which The situation that not all duplicates are marked for repacking can occur when using the `max-repack-size` option	2020-11-18 22:30:22 +01:00
Michael Eischer	625410f003	Limit progress bar updates to once per second on non-terminal outputs ... The code accidentally checked whether stdin is a terminal instead of stdout, the former is not relevant here as the output is printed on stdout.	2020-11-18 22:12:07 +01:00
Alexander Neumann	d857fb6e59	Allow using --repository-file in init	2020-11-17 20:43:46 +01:00
Michael Eischer	1f43cac12d	check: Only track data blobs when unused blobs should be reported ... This improves the memory usage of check a lot as it now only has to track tree blobs when run using the default parameters.	2020-11-15 18:43:07 +01:00
Michael Eischer	3500f9490c	check: Simplify blob status tracking ... UnusedBlobs now directly reads the list of existing blobs from the repository index. This removes the need for the blobStatusExists flag, which in turn allows converting the blobRefs map into a BlobSet.	2020-11-15 18:42:42 +01:00
Michael Eischer	caac38ed27	check: Tweak subset percentage over 100% error message	2020-11-15 18:13:50 +01:00
fgma	8f9cea8cc0	Check data subset: check random percentage subset	2020-11-15 18:13:50 +01:00
Alexander Neumann	3c0c0c132b	Merge pull request #3006 from aawsome/new-rebuild-index ... Reimplement rebuild-index and remove /internal/index	2020-11-15 17:48:43 +01:00
kitone	3c03b35212	Fix #1681 should not try to create the mount point if it doesn't exist, rather return an error	2020-11-15 17:09:30 +01:00
Alexander Weiss	9607cad267	Remove internal/index	2020-11-15 07:05:09 +01:00
Alexander Weiss	30b6a0878a	Reimplement rebuild-index	2020-11-15 07:05:09 +01:00
Alexander Weiss	1ec628ddf5	Add extraObsolete to MasterIndex.Save	2020-11-15 07:05:09 +01:00
MichaelEischer	3601a9b6cd	Merge pull request #2690 from SkYNewZ/master ... Fix #2688: Handle comma-separated list tags when using backup command	2020-11-14 23:19:42 +01:00
Michael Eischer	fdec8051ab	tags: Tweak description of --add/set/remove options	2020-11-14 22:55:30 +01:00
MichaelEischer	333c5a19d4	Merge pull request #3082 from aawsome/check-sizes ... Check: check sizes of packs from index and packheader	2020-11-14 22:37:42 +01:00
Quentin Lemaire	a8ad6b9a4b	fix(tags): Change tags list type according to restic.TagList	2020-11-14 16:24:58 +00:00
Quentin Lemaire	b0882b3f3c	fix(snapshots): Update help message to better match the 'forget' command one	2020-11-14 15:48:57 +00:00
Quentin Lemaire	ae441d3134	fix(backup): Switch tags cobra type to handle comma-separated list	2020-11-14 15:48:56 +00:00
Alexander Weiss	80dcfca191	check: Check sizes computed from index and pack header	2020-11-14 00:42:49 +01:00
Alexander Neumann	59ea5a4208	diff: Correctly count top-level blobs	2020-11-13 21:11:21 +01:00
greatroar	63e32c44c0	Improve error reporting from restic diff ... Instead of a stacktrace, restic diff 111 222 now reports: Fatal: no matching ID found for prefix "111"	2020-11-11 16:40:40 +01:00
greatroar	21b787a4d1	Stop Counters where they're constructed and started	2020-11-09 13:03:31 +01:00
greatroar	ddca699cd2	Replace restic.Progress with new progress.Counter ... This fixes two race conditions while cleaning up the code.	2020-11-09 12:12:35 +01:00
Alexander Neumann	a8d21b5dcf	prune: Warn if no cache is present	2020-11-08 20:25:35 +01:00
Alexander Neumann	823d0afd6e	backup: Always print parent snapshot info	2020-11-08 20:25:35 +01:00
Alexander Neumann	47277c4b4c	Add comments, clarify computation	2020-11-06 20:23:30 +01:00
Alexander Weiss	d2e53730d6	Add test that repo.List is only called once	2020-11-06 20:23:30 +01:00
Alexander Weiss	fd33030556	Use in-memory index to rebuild index in prune	2020-11-06 20:23:30 +01:00
Alexander Neumann	5144141321	Merge pull request #2718 from aawsome/new-cleanup-command ... Reimplementation of prune	2020-11-05 10:12:19 +01:00
Alexander Neumann	d35d279455	Set development version for 0.11.0	2020-11-05 09:41:40 +01:00
Alexander Neumann	b7e1ece1e0	Add version for 0.11.0	2020-11-05 09:41:35 +01:00
fgma	916b2d303b	vss: fix potential crash (not reachable in restic) (#3045) ... HasSufficientPrivilegesForVSS() now returns an error	2020-11-04 22:14:18 +01:00
Alexander Neumann	1ca60bccfb	Refactor condition for MaxRepackBytes ... Don't depend on the string (opts.MaxRepackSize) for the condition, instead check if there's a (positive) limit configured.	2020-11-03 16:42:21 +01:00
Alexander Neumann	7f86eb4ec0	Move helper function	2020-11-03 16:42:21 +01:00
Alexander Neumann	c1a3de4a6e	Refactor max-unused calculation, add unlimited option ... Add a callback to the PruneOptions struct which calculates the number of bytes allowed to be unused after prune is done. This way, the logic is closer to the option parsing code. Also, add an explicit option `unlimited` for the use case when storage does not matter but bandwidth and time do. Internally, this sets the maximum number of unused bytes to MaxUint64. Rework the documentation slightly so that no more "packs" are mentioned and it talks about "files" instead. Make it clear in the documentation that the percentage given to `--max-unused` is relative to the whole repository size after pruning is done. If specified, it must be below 100%, otherwise the repository would contain 100% of unused data, which is pointless. I had a hard time coming up with the correct formula to calculate the maximum number of unused bytes based on the number of used bytes. For a fraction `p` (0 ≤ p < 1), a repo with `u` bytes used, and the number of unused bytes `x` the following holds: x ≤ p * (u+x) ⇔ x ≤ p*u + p*x ⇔ x - p*x ≤ p*u ⇔ x * (1-p) ≤ p*u ⇔ x ≤ p/(1-p) * u	2020-11-03 16:42:21 +01:00
Alexander Neumann	f8c4dd7b1a	Split packe rewrite logic into two case branches ... The comma is too sublte, let's split this into two separate branches.	2020-11-03 16:42:21 +01:00
Alexander Neumann	a5b80452fe	Add comment that usedBlobs is modified	2020-11-03 16:42:21 +01:00
Alexander Neumann	aff1e220f5	Split struct members, add comments	2020-11-03 16:42:21 +01:00
Alexander Neumann	095155d9ce	Remove RepackSmall	2020-11-03 16:42:21 +01:00
Alexander Weiss	b2f5381737	Make realistic forget --prune --dryrun	2020-11-03 16:42:21 +01:00
Alexander Weiss	7f9a0a5907	Reimplementation of prune	2020-11-03 16:42:21 +01:00
Alexander Weiss	3b591ed987	Add Verboseff	2020-11-03 16:42:21 +01:00
Alexander Neumann	21ba15577e	Return an error for invalid sizes	2020-11-03 11:53:49 +01:00
Alexander Neumann	6509c207f4	Merge pull request #2997 from MichaelEischer/faster-excludes ... Speedup exclude/include checking	2020-11-02 11:07:21 +01:00
Alexander Neumann	445b845267	Merge pull request #2978 from MichaelEischer/warn-tree-error ... Warn if backup failed to read tree blob	2020-11-02 10:14:12 +01:00
Alexander Neumann	3ff37215df	Merge pull request #2935 from MichaelEischer/upgrade-minio ... Upgrade minio SDK to version 7	2020-11-02 09:09:10 +01:00
MichaelEischer	31b8d7a639	Merge pull request #2274 from fgma/master ... Support for Volume Shadow Copy Service (VSS) on windows	2020-10-24 15:43:34 +02:00
fgma	5695f9ebd2	vss: Implement VSS support for Windows ... The VSS support works for 32 and 64-bit windows, this includes a check that the restic version matches the OS architecture as required by VSS. The backup operation will fail the user has not sufficient permissions to use VSS. Snapshotting volumes also covers mountpoints but skips UNC paths.	2020-10-24 11:35:57 +02:00
greatroar	3807d13bdc	restic mount: don't umount unless we actually tried to mount ... Previously, failure to open a repository would result in an "unable to umount" message.	2020-10-21 12:15:05 +02:00
Alexander Neumann	56883817d8	Merge pull request #2990 from MichaelEischer/fix-goreport-warnings ... Fix some goreport warnings	2020-10-12 20:44:56 +02:00
Michael Eischer	e638b46a13	Embed context into ReaderAt ... The io.Reader interface does not support contexts, such that it is necessary to embed the context into the backendReaderAt struct. This has the problem that a reader might suddenly stop working when it's contained context is canceled. However, this is now problem here as the reader instances never escape the calling function.	2020-10-09 22:39:07 +02:00
Michael Eischer	2964d2ad15	Skip unlocking for nil locks ... Now that lockRepo receives a context, it is possible that it is canceled before a lock was created. Thus `unlockRepo` must be able to handle this case.	2020-10-09 22:39:06 +02:00
Michael Eischer	0c9efa9c2a	Pass context to lockRepo	2020-10-09 22:39:06 +02:00
Michael Eischer	37a5e2d681	rest: use global context on repository creation	2020-10-09 22:39:06 +02:00
Michael Eischer	dc2e664209	integration_fuse_test: use global context ... No need to use the TODO context.	2020-10-09 22:37:56 +02:00
Michael Eischer	603bb0e309	restore: Use proper context while loading snapshot	2020-10-09 22:37:56 +02:00
Michael Eischer	27456f6545	debug: use proper context ... This allows the debug commands to be properly interrupted.	2020-10-09 22:37:56 +02:00
Michael Eischer	c458e114d4	pass context to Find / FindSnapshot ... This allows proper interruption of restic while it searches for snapshots or key files.	2020-10-09 22:37:56 +02:00
Michael Eischer	45e9a55c62	Wire context into backend layout detection	2020-10-09 22:37:24 +02:00
Michael Eischer	307a6ba3a3	Upgrade minio sdk to v7 ... This changes are primarily straightforward modifications to pass the parameters in the now expected way.	2020-10-09 22:37:24 +02:00
Michael Eischer	50da20d93d	Warn if backup failed to read tree blob	2020-10-09 22:36:27 +02:00
Alexander Neumann	30cb553c8d	Merge pull request #2932 from MichaelEischer/proper-rclone-create ... Call rclone.Create to create a new repository for the rclone backend	2020-10-09 21:29:15 +02:00
Michael Eischer	bcc3bddcf4	filter: only check whether a child path could match when necessary ... When checking excludes there is no need to test whether a child path could also match the pattern, as it is by definition excluded. Previously childMayMatch was calculated but then discarded. For simple absolute paths this can account for half the time spent for checking pattern matches. name old time/op new time/op delta FilterPatterns/Relative-4 23.3ms ± 9% 21.7ms ± 6% -6.68% (p=0.004 n=10+10) FilterPatterns/Absolute-4 13.9ms ± 7% 10.0ms ± 5% -27.61% (p=0.000 n=10+10) FilterPatterns/Wildcard-4 51.4ms ± 7% 47.0ms ± 7% -8.51% (p=0.001 n=9+9) FilterPatterns/ManyNoMatch-4 551ms ± 9% 190ms ± 1% -65.41% (p=0.000 n=10+8) name old alloc/op new alloc/op delta FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.665 n=10+9) FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.480 n=9+10) FilterPatterns/Wildcard-4 14.3MB ± 0% 14.3MB ± 0% ~ (p=0.431 n=9+10) FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal) name old allocs/op new allocs/op delta FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal) FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal) FilterPatterns/Wildcard-4 88.7k ± 0% 88.7k ± 0% ~ (all equal) FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)	2020-10-07 20:47:52 +02:00
Michael Eischer	375c2a56de	filter: Parse filter patterns only once ... name old time/op new time/op delta FilterPatterns/Relative-4 30.3ms ±10% 23.6ms ±20% -22.12% (p=0.000 n=10+10) FilterPatterns/Absolute-4 49.0ms ± 3% 32.3ms ± 8% -33.94% (p=0.000 n=8+10) FilterPatterns/Wildcard-4 345ms ± 9% 334ms ±17% ~ (p=0.315 n=10+10) FilterPatterns/ManyNoMatch-4 3.93s ± 2% 0.71s ± 7% -81.98% (p=0.000 n=9+10) name old alloc/op new alloc/op delta FilterPatterns/Relative-4 4.63MB ± 0% 3.57MB ± 0% -22.98% (p=0.000 n=9+9) FilterPatterns/Absolute-4 8.54MB ± 0% 3.57MB ± 0% -58.20% (p=0.000 n=10+10) FilterPatterns/Wildcard-4 146MB ± 0% 141MB ± 0% -2.93% (p=0.000 n=9+9) FilterPatterns/ManyNoMatch-4 907MB ± 0% 4MB ± 0% -99.61% (p=0.000 n=9+9) name old allocs/op new allocs/op delta FilterPatterns/Relative-4 66.6k ± 0% 22.2k ± 0% -66.67% (p=0.000 n=10+10) FilterPatterns/Absolute-4 88.7k ± 0% 22.2k ± 0% -75.00% (p=0.000 n=10+10) FilterPatterns/Wildcard-4 1.70M ± 0% 1.63M ± 0% -3.92% (p=0.000 n=10+10) FilterPatterns/ManyNoMatch-4 4.46M ± 0% 0.02M ± 0% -99.50% (p=0.000 n=10+10)	2020-10-07 20:47:27 +02:00
MichaelEischer	0ae02f3030	Merge pull request #2844 from aawsome/prune-integration-tests ... Add prune integration tests for many edge cases	2020-10-06 23:16:41 +02:00
Lorenz Bausch	af4100e07d	Replace repo/repos with repository/repositories in --no-lock help text	2020-10-06 22:49:48 +02:00
Alexander Weiss	6822a58413	Add prune integration tests for many edge cases	2020-10-06 20:20:05 +02:00
Michael Eischer	1579d2a8ec	Remove some unused assignments	2020-10-06 14:55:13 +02:00
Michael Eischer	eba5dd831f	Fix typos reported by misspell	2020-10-06 14:55:13 +02:00
Leo R. Lundgren	e105a3f391	cache/check: Clarify error message when given arguments	2020-10-06 00:08:59 +02:00
rawtaz	5d8cfff3f2	Merge pull request #2988 from greatroar/snapshots-json ... Regression test for #2979	2020-10-05 21:02:52 +02:00
MichaelEischer	1ab4c710e1	Merge pull request #2987 from greatroar/no-password-for-help ... Don't read password for generate, help or self-update	2020-10-05 20:16:19 +02:00
greatroar	feedf0ebce	Don't read password for generate, help or self-update ... Fixes #2951.	2020-10-05 19:54:07 +02:00
MichaelEischer	1a490acd67	Merge pull request #2982 from greatroar/archiver-error-handling ... Check error in archiver before calling Select	2020-10-05 19:19:47 +02:00
greatroar	6bee0aafc2	Regression test for #2979	2020-10-05 15:08:09 +02:00
Leo R. Lundgren	137d20a06a	snapshots: Make --json output [] instead of null when no snapshots	2020-10-05 13:14:10 +02:00
greatroar	c4e2203e45	Check error in archiver before calling Select ... The archiver first called the Select function for a path before checking whether the Lstat on that path actually worked. The RejectFuncs in exclude.go worked around this by checking whether they received a nil os.FileInfo. Checking first is more obvious and requires less code.	2020-10-05 11:11:04 +02:00
rawtaz	bbeb439f41	Merge pull request #2937 from andreaso/self-update-output-path-fix ... Don't require `self-update --output` placeholder file	2020-10-03 13:35:20 +02:00
Andreas Olsson	ce14df303b	Don't require self-update --output placeholder file ... This removes the requirement on `restic self-update --output` to point to a path of an existing file, to overwrite. In case the specified path does exist we still want to verify that it's a regular file, rather than a directory or a device, which gets overwritten. We also want to verify that a path to a new file exists within an existing directory. The alternative being running into that issue after the actual download, etc has completed. While at it I also replace `errors.Errorf` with the more appropriately verbose `errors.Fatalf`. Resolves #2491	2020-10-03 07:02:23 +02:00
MichaelEischer	3c6671b18b	Merge pull request #2973 from restic/rawtaz-repo2 ... Correct data type for --repo2 option	2020-10-02 22:22:41 +02:00
rawtaz	0cce6dc31c	Correct data type for --repo2 option	2020-10-02 21:10:52 +02:00
rawtaz	6253ff0187	Clarify max verbose level in option description ... Clarifies that the max verbosity level is 3, in the description of the --verbose option.	2020-10-02 20:25:34 +02:00
jtagcat	a687261804	--help: --compact: clarify description	2020-10-02 16:55:56 +03:00
Michael Eischer	61035d68bc	Add test for --repository-file	2020-10-01 00:50:27 +02:00
Andreas Oberritter	97f7855de3	Add new option --repository-file (default: $RESTIC_REPOSITORY_FILE) ... As an alternative to -r, this allows to read the repository URL from a file in order to prevent certain types of information leaks, especially for URLs containing credentials. Fixes #1458, fixes #2900.	2020-10-01 00:50:26 +02:00
rawtaz	34ea960559	Merge pull request #2966 from MichaelEischer/recover-help-typo ... recover: Fix typo in the command help	2020-09-30 18:16:26 +02:00
Michael Eischer	2a79c1a44d	recover: Fix typo in the command help ... Reported-by: Steve Divskinsy <stevesbrain@users.noreply.github.com>	2020-09-30 17:44:34 +02:00
Michael Eischer	aea9f7d286	clarify verbose password prompt	2020-09-30 17:25:54 +02:00
plumbeo	009bd907f2	Don’t print too many messages when deleting files ... Print detailed debug messages on file deletions only when --verbose=2 is used	2020-09-27 14:24:04 +02:00
MichaelEischer	14b312f00d	Merge pull request #2658 from creativeprojects/issue-2241 ... Don't echo authentication passwords (rest backend)	2020-09-22 22:17:53 +02:00
Fred	206cadfab4	Hide password from repository URLs	2020-09-22 22:00:51 +02:00
MichaelEischer	4875f7b659	Merge pull request #2614 from greatroar/simplify-fs ... Simplify internal/fs	2020-09-21 22:05:10 +02:00
yoshiera	3cf29a777d	Fix nil check in rejectBySize	2020-09-21 19:20:24 +08:00
Michael Eischer	9333f707fa	init: use Create method for rclone backend ... This properly issues the initial repository creation command Fixes #1896	2020-09-19 19:11:43 +02:00
Alexander Neumann	429f97b887	Set development version for 0.10.0	2020-09-19 17:38:47 +02:00
Alexander Neumann	40832b2927	Add version for 0.10.0	2020-09-19 17:38:26 +02:00
Michael Eischer	f003410402	init: Add --copy-chunker-params option ... This allows creating multiple repositories with identical chunker parameters which is required for working deduplication when copying snapshots between different repositories.	2020-09-19 16:53:05 +02:00
Michael Eischer	655430550b	Extract parameters for second repository from copy command	2020-09-19 16:07:55 +02:00
Leo R. Lundgren	311ad2d2d0	Improve wording for --password-file and related options	2020-09-19 15:47:32 +02:00
Leo R. Lundgren	ffe6dce7e7	Improve wording for --password-command and --password-command2 options	2020-09-19 14:23:25 +02:00
yoshiera	ac4b8c98ac	Support excluding files by size	2020-09-19 19:52:09 +08:00
greatroar	9abef3bf1a	Move internal/fs.TestChdir to internal/test.Chdir	2020-09-17 10:43:33 +02:00
Michael Eischer	f5c448aa65	diff: Optimize diff calculation for shared subtrees ... When the diff calculation compares two trees with identical id then no differences between them can ever show up. Optimize for that case by simply traversing the tree only once to collect all referenced blobs for a proper calculation of added and removed blobs. Just skipping the common subtrees is not possible as this would skew the results if the added or removed blobs are shared with one of the subtrees.	2020-09-13 16:28:46 +02:00
Michael Eischer	c0fc85d303	diff: Add integration test	2020-09-13 16:28:46 +02:00
Alexander Neumann	0c48e515f0	Merge pull request #2630 from MichaelEischer/fix-staticcheck ... Fix lots of small issues reported by staticcheck	2020-09-13 16:19:22 +02:00
rawtaz	d3a286928a	Merge pull request #2905 from MichaelEischer/fix-ls-help ... ls: Explicitly enforce that the user specifies a snapshot ID	2020-09-05 10:44:01 +02:00
Michael Eischer	1ede018ea6	error variable names should start with 'Err'	2020-09-05 10:07:17 +02:00
Michael Eischer	d0329cf3eb	Adjust comments to match name of exported methods	2020-09-05 10:07:16 +02:00
Michael Eischer	b22655367c	integration_test: Replace fprintf without format string	2020-09-05 10:07:16 +02:00
Michael Eischer	068a3ce23f	Remove redundant return	2020-09-05 10:07:16 +02:00
Michael Eischer	d19f05c960	Use buf.String() instead of string(buf.Bytes())	2020-09-05 10:07:16 +02:00
Michael Eischer	460e2ffbf6	Collapse a few boolean operations	2020-09-05 10:07:14 +02:00
Michael Eischer	49b6aac3fa	Use !bytes.Equal instead of bytes.Compare !=0 to check for inequality	2020-09-05 10:06:23 +02:00
Michael Eischer	2f8335554c	Remove a few unused variables	2020-09-05 10:06:23 +02:00
Michael Eischer	37113282ca	Merge a few variable declaration and initializations	2020-09-05 10:05:34 +02:00
Michael Eischer	337725c354	Use sort.Strings	2020-09-05 10:05:34 +02:00
MichaelEischer	2ddb7ffb7e	Merge pull request #2255 from Kidswiss/tar ... Fix dumping issues with / and the first sub level	2020-09-01 21:52:17 +02:00
Michael Eischer	81dcfea11a	dump: Mention tar output in command help text	2020-08-31 22:43:10 +02:00
MichaelEischer	55071ee367	Merge pull request #2859 from buschjost/stats-filter-by-tag-and-path ... Add filter by tag and path to stats command	2020-08-31 22:11:01 +02:00
Michael Eischer	bcf44a9c3f	dump: Properly handle file paths without / prefix ... filepath.Rel failed if the requested path did not start with a `/` e.g. `restic` instead of `/restic`.	2020-08-30 18:25:42 +02:00
Oliver Buschjost	a7b4c19abf	Fix display of warning about ignored filters	2020-08-30 17:14:56 +02:00
Oliver Buschjost	d3fcfeba3a	Add filter by tag and path to stats command	2020-08-30 17:14:48 +02:00
Michael Eischer	e69449bf2c	dump: Add test for splitPath	2020-08-30 16:00:48 +02:00
MichaelEischer	ea81a0e282	Merge pull request #2606 from middelink/fix-323 ... Add copy functionality.	2020-08-30 10:18:24 +02:00
Simon Beck	d6f739ec22	dump: Extract tar code to internal/dump	2020-08-29 21:14:34 +02:00
Simon Beck	b98598e55f	dump: Fix dumping issues with / and the first sub level ... There was an issue that prevented the dump command from working correctly when either: * `/` contained multiple nodes (e.g. `restic backup /`) * dumping a file in the first sublevel was attempted (e.g. `/foo`)	2020-08-29 21:14:29 +02:00
Michael Eischer	d5f86effa1	ls: Explicitly enforce that the user specifies a snapshot ID ... The help messages suggested that the `ls` command work without explicitly passing a snapshot ID. However, this was never the case: without a snapshot ID the command just failed with the error `Ignoring "", it is not a snapshot id`. Fixes #2299	2020-08-29 14:28:53 +02:00
Michael Eischer	412623b848	copy: Reuse buffer for downloaded blobs	2020-08-29 10:48:44 +02:00
Michael Eischer	9a4796594a	integration tests: Fix checking of wrong snapshot	2020-08-29 10:48:44 +02:00
Michael Eischer	15374d22e9	integration tests: Add basic tests for copy command	2020-08-29 10:48:44 +02:00
Michael Eischer	88ad58d6cd	integration tests: Redirect directory diff into intermediate buffer	2020-08-29 10:48:44 +02:00
Michael Eischer	591a8c4cdf	integration tests: Deduplicate backup test-data setup code	2020-08-29 10:48:44 +02:00
Michael Eischer	ec9a53b7e8	copy: Mark and skip previously copied snapshots ... Use the `Original` field of the copied snapshot to store a persistent snapshot ID. This can either be the ID of the source snapshot if `Original` was not yet set or the previous value stored in the `Original` field. In order to still copy snapshots modified using the tags command the source snapshot is compared to all snapshots in the destination repository which have the same persistent ID. Snapshots are only considered equal if all fields except `Original` and `Parent` match. That way modified snapshots are still copied while avoiding duplicate copies at the same time.	2020-08-29 10:48:44 +02:00
Alexander Weiss	9867c4bbb4	Add flag --iexclude-file to backup	2020-08-28 21:04:41 +02:00
Jason Cox	2447f3f110	Update usage strings to put flags before args ... The standard UNIX-style ordering of command-line arguments places optional flags before other positional arguments. All of restic's commands support this ordering, but some of the usage strings showed the flags after the positional arguments (which restic also parses just fine). This change updates the doc strings to reflect the standard ordering. Because the `restic help` command comes directly from Cobra, there does not appear to be a way to update the argument ordering in its usage string, so it maintains the non-standard ordering (positional arguments before optional flags).	2020-08-28 02:16:22 +00:00
Michael Eischer	b0a8c4ad6c	copy: Only process each tree once ... This speeds up copying multiple overlapping snapshots from one repository to another, as we only have to copy the changed parts of later snapshots.	2020-08-26 22:17:52 +02:00
Michael Eischer	908b23fda0	copy: Update for modernized repository interface	2020-08-26 22:17:52 +02:00
greatroar	4508d406ef	copy: Remove separate SaveIndex in restic copy ... Flush does this now.	2020-08-26 22:17:52 +02:00
Pauline Middelink	7048cc3e58	Add copy functionality ... Add a copy command to copy snapshots between repositories. It allows the user to specify a destination repository, password, password-file, password-command or key-hint to supply the necessary details to open the destination repository. You need to supply a list of snapshots to copy, snapshots which already exist in the destination repository will be skipped. Note, when using the network this becomes rather slow, as it needs to read the blocks, decrypt them using the source key, then encrypt them again using the destination key before finally writing them out to the destination repository.	2020-08-26 22:17:52 +02:00
Michael Eischer	17995dec7a	Unify progress bar of check and prune commands	2020-08-25 22:47:38 +02:00
Alexander Neumann	b1b3f1ecb6	Merge pull request #2674 from MichaelEischer/prune-strict-checks ... prune: Stricter error checks	2020-08-23 10:30:41 +02:00
Bruce Dillahunty	fe69b83074	Update key password prompt (#2847)	2020-08-19 21:42:08 +02:00
Michael Eischer	08d24ff99e	prune: Include ID of all missing blobs in error message	2020-08-16 11:36:14 +02:00
Michael Eischer	1c84aceb39	prune: Test for abort on damaged repositories	2020-08-16 11:36:12 +02:00
Michael Eischer	575ed9a47e	Test that rebuild-index errors when old index cannot be removed	2020-08-16 11:34:01 +02:00
Michael Eischer	8f811642c3	Add support for integration tests to wrap the backend	2020-08-16 11:34:01 +02:00
Michael Eischer	744a15247d	prune/rebuild-index: Fail if an old index cannot be removed ... The old behavior was problematic in the context of rebuild-index as it could leave old, possibly invalid index files behind without returning a fatal error. Prune calls rebuildIndex before removing any data from the repository. For this use case failing to delete an old index MUST be treated as a fatal error. Otherwise the index could still contain an old index file that refers to blobs/packs that were later on deleted by prune. Later backup runs will assume that the affected blobs already exist in the repository which results in a backup which misses data.	2020-08-16 11:34:01 +02:00
Michael Eischer	3ba19869be	prune: Abort if any used blobs are missing ... The previous check only approximately verified whether all required blobs were found. However, after forgetting a few snapshots the repository contains lots of unused blobs whose number can be sufficient to make up for missing packs. When coupled with a malfunctioning backend that temporarily returns broken data this could cause restic to regard the corresponding packs as invalid and thereby delete data that's still in use. This change lets restic play it safe and refuse to delete anything if data is missing.	2020-08-16 11:34:01 +02:00
aawsome	0fed6a8dfc	Use "pack file" instead of "data file" (#2885) ... - changed variable names, especially changed DataFile into PackFile - changed in some comments - always use "pack file" in docu	2020-08-16 11:16:38 +02:00
MichaelEischer	08dee8a52b	Merge pull request #2865 from greatroar/unused ... Dead code removal	2020-08-08 16:15:28 +02:00
MichaelEischer	84b6f1ec53	Merge pull request #2874 from MichaelEischer/fix-verbose ... Adjust description of --verbose=n parameter	2020-08-04 23:21:05 +02:00
MichaelEischer	06fb4ea3f0	Merge pull request #2805 from jwilk-forks/verbose ... Fix examples of --verbose with argument	2020-08-04 23:11:35 +02:00
Michael Eischer	e38d415173	Adjust description of --verbose=n parameter	2020-08-04 23:07:53 +02:00
greatroar	d81a396944	Dead code removal ... Found by running golangci-lint on the entire code base.	2020-08-04 08:38:57 +02:00
MichaelEischer	38a2f9c07b	Merge pull request #2864 from greatroar/dump-error ... Fix error handling in dump (err != err)	2020-08-03 20:09:42 +02:00
rawtaz	5af2815627	Merge pull request #2821 from renard/mount-nolock ... cmd/mount: honur --no-lock flag	2020-08-03 20:09:26 +02:00
Sébastien Gross	9be4fe3e84	cmd/mount: honour --no-lock flag ... Do not lock the repository if --no-lock global flag is set. This allows to mount repositories which are archived on a read only system. Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>	2020-08-03 19:45:39 +02:00
Michael Eischer	05116e4787	prune: Cleanup progress bar handling while repacking	2020-08-03 19:32:46 +02:00
Michael Eischer	04f79b9642	prune: Stop progress bar after searching used blobs	2020-08-03 19:31:49 +02:00
greatroar	8b358935a0	Fix error handling in dump (err != err)	2020-08-03 09:43:44 +02:00
MichaelEischer	66d089e239	Merge pull request #2841 from aawsome/optimize-getUsedBlobs ... Extract get used blobs in prune as separate function	2020-08-01 22:40:33 +02:00
Alexander Weiss	9762bec091	Use optimized getUsedBlobs in prune	2020-08-01 21:07:31 +02:00
Alexander Weiss	d3692f5b81	Delete files in parallel in rebuild-index	2020-08-01 20:43:18 +02:00
Alexander Weiss	1c0b61204b	Delete files in parallel in forget	2020-08-01 20:43:18 +02:00
Alexander Weiss	2ee654763b	Delete files in parallel	2020-08-01 20:39:24 +02:00
Alexander Neumann	b7b479b668	Merge pull request #2599 from MichaelEischer/tweak-mem-usage ... Reduce memory usage when searching for used blobs	2020-08-01 13:49:02 +02:00
Alexander Neumann	2580eef2aa	Merge pull request #2318 from classmarkets/2175-named-keys ... Allow specifying user and host when adding keys	2020-08-01 13:06:31 +02:00
Michael Eischer	184103647a	FindUsedBlobs: merge seen into blobs BlobSet ... The seen BlobSet always contained a subset of the entries in blobs. Thus use blobs instead and avoid the memory overhead of the second set. Suggested-by: Alexander Weiss <alex@weissfam.de>	2020-08-01 12:29:16 +02:00
Michael Eischer	3ce9893e0b	Don't print a stacktrace if some files could not be read	2020-07-28 23:52:48 +02:00
Alexander Weiss	9d1fb94c6c	make Lookup() return all blobs ... + simplify syntax	2020-07-25 21:18:34 +02:00
MichaelEischer	020cab8e08	Merge pull request #2787 from greatroar/no-blobsize-cache ... Remove blob size cache from restic mount	2020-07-25 20:46:35 +02:00
greatroar	07da61baee	Remove blob size cache from restic mount	2020-07-25 19:39:14 +02:00
Alexander Weiss	9175795fdb	Check contents in archiver ... When backing up with a parent snapshot and the file is not changed, also check if contents are still available in index.	2020-07-25 08:18:28 +02:00
Michael Eischer	ddf0b8cd0b	checker: Properly distinguish between data and tree blobs ... If a data blob and a tree blob with the same ID (= same content) exist, then the checker did not report a data or tree blob as unused when the blob of the other type was still in use.	2020-07-20 22:58:39 +02:00
David Sommer	5729d967f5	backup: Return exit status code 3 when failing to read source data ... The backup command used to return a zero exit code as long as a snapshot could be created successfully, even if some of the source files could not be read (in which case the snapshot would contain the rest of the files). This made it hard for automation/scripts to detect failures/incomplete backups by looking at the exit code. Restic now returns the following exit codes for the backup command: - 0 when the command was successful - 1 when there was a fatal error (no snapshot created) - 3 when some source data could not be read (incomplete snapshot created)	2020-07-20 22:19:01 +02:00
greatroar	58719e1f47	Replace mount's per-file cache by a global LRU cache	2020-07-12 18:27:16 +02:00
Jakub Wilk	4e6a9767de	Fix examples of --verbose with argument	2020-06-23 20:22:14 +02:00
Andreas Olsson	0fcef2ec23	Use https:// for linking CACHEDIR.TAG specs ... That site might not have supported https:// when those links were originally added. It does now. Also dropping the _spec.html_ ending of the url, there being a `<link rel="canonical" ...>` tag suggesting that that no longer being the preferred address.	2020-06-21 19:36:26 +02:00
Alexander Weiss	b5543cff5d	Loop over index files for 'list blobs' ... => reduces memory consumption a lot!	2020-06-14 17:25:34 +02:00
Michael Eischer	bb1e258bb7	cat/stats: Get stdout from globalOptions	2020-06-14 11:08:11 +02:00
Michael Eischer	182655bc88	Replace fmt.Printf/Println/Fprintf with wrapper functions ... cmd/restic/globals.go already provides Printf, Println and Warnf wrapper which get their output streams from the globalOptions object. This allows for stream replacements when testing.	2020-06-14 11:08:11 +02:00
MichaelEischer	dd7b4f54f5	Merge pull request #2709 from greatroar/minio-sha256 ... Use Minio's optimized SHA-256	2020-06-12 23:32:58 +02:00
MichaelEischer	735a8074d5	Merge pull request #2773 from aawsome/index-uploads+knownblobs ... Fix non-intuitive repo behavior	2020-06-12 22:41:04 +02:00
Alexander Weiss	70347e95d5	disable index uploads for prune command ... + modifications of changelog	2020-06-12 09:24:38 +02:00
Alexander Weiss	91906911b0	Fix non-intuitive repository behavior ... - The SaveBlob method now checks for duplicates. - Moves handling of pending blobs to MasterIndex. -> also cleans up pending index entries when they are saved in the index -> when using SaveBlob no need to care about index any longer - Always check for full index and save it when storing packs. -> removes the need of an index uploader -> also removes the verbose "uploaded intermediate index" messages - The Flush method now also saves the index - Fix race condition when checking and saving full/non-finalized indexes	2020-06-11 13:05:23 +02:00
Michael Eischer	087c770161	Revive hint to the unlock command if a repository is locked ... errors.Fatalf wraps a error and just keeps an error message as a string. This prevents the `restic.IsAlreadyLocked(err)` check from working as the error is no longer an ErrAlreadyLocked. Just add an additional remark to the error using `errors.WithMessage`.	2020-06-10 20:58:56 +02:00
nunoperalta	2c3360db98	Fix #2774 - make unable to umount message clearer	2020-06-07 15:57:39 +01:00
MichaelEischer	84475aa3a8	Merge pull request #2730 from greatroar/mount-build-tags ... Simplify build tags for restic mount	2020-05-23 20:33:43 +02:00
Martin Michlmayr	5cc1760fdf	Fix typos	2020-05-16 14:05:26 +08:00
greatroar	649cbec6c5	Simplify build tags for restic mount ... This command can only be built on Darwin, FreeBSD and Linux (and if we upgrade bazil.org/fuse, only FreeBSD and Linux: https://github.com/bazil/fuse/issues/224). Listing the few supported operating systems explicitly here makes porting restic to new platforms easier.	2020-05-12 11:30:41 +02:00
MichaelEischer	1ee2306033	Merge pull request #2717 from MichaelEischer/fix-flaky-password-prompt ... backup: Avoid race between password prompt and open repository message	2020-05-01 23:58:55 +02:00
Michael Eischer	c882a92cd6	backup: Avoid race between password prompt and open repository message ... `term.Print` sends the output via a channel to a goroutine which actually prints the message. This may race with the password prompt printed by `OpenRepository` resulting in a missing prompt.	2020-05-01 23:42:16 +02:00
Michael Eischer	d465b5b9ad	Revert "Darwin test fix: allow 1μs timestamp difference" ... This reverts commit f3016a9096.	2020-05-01 17:11:30 +02:00
greatroar	42a3db05b0	Use Minio's optimized SHA-256 ... internal/repository benchmarks on an Intel i7-3770k: name old speed new speed delta PackerManager-8 209MB/s ± 1% 291MB/s ± 1% +38.94% (p=0.008 n=5+5) SaveAndEncrypt-8 112MB/s ± 1% 135MB/s ± 1% +20.25% (p=0.008 n=5+5)	2020-04-28 07:57:18 +02:00
greatroar	be5a0ff59f	Centralize buffer allocation and size checking in Repository.LoadBlob ... Benchmark results for internal/repository: name old time/op new time/op delta LoadTree-8 479µs ± 2% 478µs ± 1% ~ (p=0.780 n=10+9) LoadBlob-8 11.6ms ± 2% 11.6ms ± 1% ~ (p=0.631 n=10+10) LoadAndDecrypt-8 13.2ms ± 2% 13.3ms ± 3% ~ (p=0.631 n=10+10) name old alloc/op new alloc/op delta LoadTree-8 41.2kB ± 0% 41.2kB ± 0% ~ (all equal) LoadBlob-8 2.28kB ± 0% 2.28kB ± 0% ~ (all equal) LoadAndDecrypt-8 2.10MB ± 0% 2.10MB ± 0% ~ (all equal) name old allocs/op new allocs/op delta LoadTree-8 652 ± 0% 652 ± 0% ~ (all equal) LoadBlob-8 24.0 ± 0% 24.0 ± 0% ~ (all equal) LoadAndDecrypt-8 30.0 ± 0% 30.0 ± 0% ~ (all equal) name old speed new speed delta LoadBlob-8 86.2MB/s ± 2% 86.4MB/s ± 1% ~ (p=0.594 n=10+10) LoadAndDecrypt-8 75.7MB/s ± 2% 75.4MB/s ± 3% ~ (p=0.617 n=10+10)	2020-04-23 10:04:20 +02:00
MichaelEischer	9790d8ce1c	Merge pull request #2668 from MichaelEischer/fix-stats-blobs-crash ... stats: Fix crash in blobs-per-file mode on missing blob	2020-04-18 17:21:26 +02:00
MichaelEischer	08ec6c9f17	Merge pull request #2677 from MichaelEischer/complain-about-invalid-indexes ... rebuild_index: Report invalid packs that were ignored	2020-04-18 16:57:50 +02:00
MichaelEischer	b7c0d4d8bf	Merge pull request #2644 from greatroar/signal-notify-buffered ... Make all signal.Notify channels buffered	2020-04-18 11:31:02 +02:00
Michael Eischer	1a1c572bac	Fix shutdown hang when restic is started as background job ... restic uses a cleanup hook to ensure that it restores the terminal configuration to a sane state, when restic is interrupted while reading a password from the terminal. However, this causes a problem, when restic runs in a background job, as reconfiguring a terminal will cause a SIGTTOU to be sent to restic pausing it. Therefore, restic seems to hang on shutdown when it was running in the background. This commit changes the behavior to only restore the terminal configuration if restic was interrupted while reading a password from the terminal. As reading a password from the terminal requires that restic is in the foreground, this should avoid restic getting stopped. Fixes #2298 Issue introduced in #402	2020-04-12 22:27:09 +02:00
rawtaz	5a7c27ddb6	Merge pull request #2681 from MichaelEischer/optimize-debug ... Reduce memory usage and startup time of debug command	2020-04-04 00:08:14 +02:00
Michael Eischer	fb842759fc	debug: don't load the repository index ... The existing commands don't need a loaded repository index which can take several minutes to load on larger repositories.	2020-04-04 00:01:01 +02:00
Michael Eischer	7aa2f8a61e	debug: get stdout/stderr from gopts/globalOptions	2020-04-04 00:01:01 +02:00
Michael Eischer	08bf3bae79	debug: explicitly pass stdout to dump functions	2020-04-03 23:32:44 +02:00
Michael Eischer	bc74cd3ae5	backup/global: Use proper name for command line argument parameters ... Several paramters printed a generic "string" or "stringArray" name.	2020-04-03 19:49:04 +02:00
Michael Eischer	90243ed1c4	rebuild_index: Report invalid packs that were ignored	2020-04-02 22:38:31 +02:00
Michael Eischer	0ce81d88b6	stats: Fix crash in blobs-per-file mode on missing blob ... In a damaged repository with a missing blob, the error message tried to dereference the subtreeID field of the current node, which is a file however. Said field is set to nil for a file thus causing a segfault when dereferenced. Fix this by using the actual parentTreeID.	2020-03-27 22:17:54 +01:00
Peter Schultz	90fc639a67	Allow specifying user and host when adding keys ... The username and hostname for new keys can be specified with the new --user and --host flags, respectively. The flags are used only by the `key add` command and are otherwise ignored. This allows adding keys with for a desired user and host without having to run restic as that particular user on that particular host, making automated key management easier. Co-authored-by: James TD Smith <ahktenzero@mohorovi.cc>	2020-03-23 13:11:10 +01:00
greatroar	18fee4806f	Remove broken --allow-root from restic mount	2020-03-17 23:35:06 +01:00
greatroar	47d4d5bf1b	Make all signal.Notify channels buffered	2020-03-12 20:59:39 +01:00
rawtaz	58bd165253	Merge pull request #2581 from aawsome/multiple-hostnames ... Allow multiple hostnames tags	2020-02-27 08:35:23 +01:00
Alexander Weiss	9a9101d144	Support specifying multiple host flags for various commands ... The `dump`, `find`, `forget`, `ls`, `mount`, `restore`, `snapshots`, `stats` and `tag` commands will now take into account multiple `--host` and `-H` flags.	2020-02-26 22:17:59 +01:00
David Potter	71900248ab	Correct #2531 (stats restore-size calculates hard links incorrectly)	2020-02-15 19:34:22 +01:00
David Potter	23055aaadf	Correct #2537 (cmd_stats file counting issue)	2020-02-15 19:34:16 +01:00
curiousleo	4214b1746e	Ensure Print{,f,ln} use global stdout	2020-02-15 01:32:43 +01:00
curiousleo	f6f240573a	Don't Printf already formatted output ... Fixes #2281.	2020-02-15 01:32:43 +01:00
Michael Eischer	78c518ccac	Print backup summary after status output is shutdown	2020-02-13 21:14:20 +01:00
Michael Eischer	42a3292bcf	Better name for jsonstatus package ... internal/ui/jsonstatus and termstatus sound similar but are not related in any way. Instead `internal/ui/backup` and `internal/ui/jsonstatus/status` are the counterparts. Rename the latter to `internal/ui/json/backup` to make this clear.	2020-02-13 21:14:20 +01:00
Erik Rigtorp	94f4f13388	Add documentation on exit status codes to man pages ... This is step one to start defining useful exit codes for all the commands.	2020-02-12 23:09:26 +01:00
rawtaz	299f5971f2	Merge pull request #2560 from brualan/master ... Two small improvements to code quality	2020-02-12 22:40:20 +01:00
Lars Lehtonen	94a4d45dfb	cmd/restic: fix a dropped error	2020-02-11 09:19:03 -08:00
Alexander Bruyako	688014487b	avoid "index out of range" error ... in case of len(format) == 0, we will get an "index out of range" error Usage of strings.HasSuffix is allowing to avoid that	2020-01-27 19:24:42 +03:00
Lorenz Bausch	d818b7618b	Display respective snapshot date when using restic find	2019-12-17 07:51:12 +01:00
Leo R. Lundgren	77ef92b95c	diff: Fix wrong bytes reported in diff stats (#2469)	2019-11-22 17:26:01 +01:00
Alexander Neumann	30519f01ff	Set development version for 0.9.6	2019-11-22 16:19:03 +01:00
Alexander Neumann	b723ca3de5	Add version for 0.9.6	2019-11-22 16:18:59 +01:00
Alexander Neumann	fb95426f64	Rename environment variable to AWS_DEFAULT_REGION ... This seems to be the correct name, at least the AWS cli uses it: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html	2019-11-22 15:24:42 +01:00
mdauphin	df500a372d	Add AWS_REGION env var to specify s3 region	2019-11-22 15:04:48 +01:00
rawtaz	26d1f9f4ba	Merge pull request #2307 from gary-kim/fix-2306-password-retries ... Allow multiple retries for interactive password input	2019-11-20 18:30:20 +01:00
Jean Kahrs	8066e93f47	updade message that is shown when a cache cleanup is advised	2019-07-02 10:49:49 +02:00
Fred Akalin	3ca424050f	Add upper bound for t in --read-data-subset=n/t (#2304) ... * Add upper bound for t in --read-data-subset=n/t * Add changelog entry	2019-06-29 14:34:53 -04:00
Gary Kim	fea2464d4d	Allow multiple retries for interactive password input ... Restic used to quit if the repository password was typed incorrectly once. Restic will now ask the user again for the repository password if typed incorrectly. The user will now get three tries to input the correct password before restic quits.	2019-06-13 20:11:02 +08:00
Garry McNulty	4429a66b5f	backup: Convert relative pathname for --stdin-filename to absolute (#2063)	2019-05-08 15:21:27 +02:00
Lorenz Bausch	298f490195	Fix typo	2019-05-02 11:51:35 +02:00
Alexander Neumann	c7762453cf	Set development version for 0.9.5	2019-04-23 13:19:43 +02:00
Alexander Neumann	303210aa08	Add version for 0.9.5	2019-04-23 13:19:39 +02:00
Martin Michlmayr	0561155963	Split paragraph from list in restic stats --help ... The help text for `restic stats` lists a number of modes in a list. Make sure the "more info" text is a separate paragraph rather than being part of the list.	2019-04-23 10:36:33 +07:00
Martin Michlmayr	f11789c437	Add missing opening parenthesis	2019-04-23 10:36:26 +07:00
Alexander Neumann	8cab0c121d	Merge pull request #2221 from classmarkets/2203-password-from-stdin ... Fix reading password from stdin	2019-04-22 20:59:59 +02:00
Alexander Neumann	5979414bcd	Merge pull request #2124 from Kidswiss/tar ... Ability to dump folders to tar via stdout	2019-04-22 20:58:59 +02:00
Simon Beck	cc8b690b52	Restore whole folder to sdtout as tar ... With this change it is possible to dump a folder to stdout as a tar. The It can be used just like the normal dump command: `./restic dump fa97e6e1 "/data/test/" > test.tar` Where `/data/test/` is a a folder instead of a file.	2019-04-22 20:44:33 +02:00
Jan Niklas Richter	733519d895	Refactor duplicate code for grouping snapshots ... This commit is a followup to the addition of the --group-by flag for the snapshots command. Adding the grouping code there introduced duplicated code (the forget command also does grouping). This commit refactors boths sides to only use shared code.	2019-04-22 14:57:53 +02:00
Jan Niklas Richter	c4475ac58f	Move snapshot grouping code into own function to deduplicate code ... This commit moves the code which is used to group snapshots in the snapshots command into an own function to deduplicate code shared by the snapshots command and forget command.	2019-04-22 14:37:58 +02:00
Jan Niklas Richter	c9fd9b5275	Fix json tags for grouped snapshot output ... This commit will add json tags to the structs for json output, so all json variables of the snapshot command output are lowercase and snake-case. Furthermore it adds some internal code changes based on the feedback in the pull request #2087.	2019-04-22 14:37:58 +02:00
Jan Niklas Richter	cadcab5a19	Add GroupBy option to snapshots command ... This commit adds a --group-by option to the snapshots command, which behaves similar to the --group-by option of forget. Valid option values are "host, paths, tags". If this option is given, the output of snapshots will be divided into multiple tables, according to the value given (i.e. "host" will create a table of snapshots for each host, that has a snapshot in the list). Also the JSON output will be grouped. The default behavior (when --group-by is not given) has not changed. More to this discussion can be found in issue #2037.	2019-04-22 14:37:58 +02:00
Peter Schultz	5715517e29	Fix reading password from stdin ... Reading the password from non-terminal stdin used io.ReadFull with a byte slice of length 1000. We are now using a Scanner to read one line of input, independent of its length. Additionally, if stdin is not a terminal, the password is read only once instead of twice (in an effort to detect typos). Fixes #2203 Signed-off-by: Peter Schultz <peter.schultz@classmarkets.com>	2019-04-15 10:05:49 +02:00
Alexander Neumann	ecc2458de8	Merge pull request #2230 from cdhowie/issue-2229 ... Extend find --show-pack-id to work with --tree	2019-04-14 19:38:38 +02:00
Alexander Neumann	2c6ba5d9ac	Merge pull request #2231 from cdhowie/issue-2224 ... find command: don't abort on tree load errors	2019-04-14 19:31:06 +02:00
Alexander Neumann	6b700d02f5	Merge pull request #2217 from restic/improve-memory-usage ... WIP: improve memory usage	2019-04-13 15:07:07 +02:00
Alexander Neumann	78a3ffcfb9	Merge pull request #2185 from d3zd3z/json-forget ... Json forget	2019-04-13 14:23:32 +02:00
Alexander Neumann	d51e9d1b98	Add []byte to repo.LoadAndDecrypt and utils.LoadAll ... This commit changes the signatures for repository.LoadAndDecrypt and utils.LoadAll to allow passing in a []byte as the buffer to use. This buffer is enlarged as needed, and returned back to the caller for further use. In later commits, this allows reducing allocations by reusing a buffer for multiple calls, e.g. in a worker function.	2019-04-13 13:38:39 +02:00
Jack Henschel	ecd7ee85e8	Mention Cache Directoy Tagging Standard in man page and help text ... This patch makes it more explicit what is meant by the CACHEDIR.TAG file. It not only has to have this particular name, but also a specific content (described at http://bford.info/cachedir/spec.html), which is not immediately obvious to the user.	2019-04-02 15:53:07 +02:00
Chris Howie	36f22a0feb	find command: don't abort on tree load errors	2019-03-29 22:58:12 -04:00
Chris Howie	f58a44b911	Extend find --show-pack-id to work with --tree	2019-03-29 22:56:36 -04:00
Heiko Bornholdt	db8f5864fc	Add --ignore-inode option to backup cmd ... revised version of https://github.com/restic/restic/pull/2047	2019-03-10 21:24:29 +01:00
David Brown	d19a29f79e	Include reasons in json output of forget ... This dumps the reasons as well as the list of keeps and removes with the output from the forget command.	2019-02-23 09:38:33 -07:00
David Brown	d9e22c2df1	Add test for --json support for forget command ... This adds a test of the json output of the forget command, by running it once, asking it to keep one snapshot, and verifying that the output has the right number of snapshots listed in the Keep and Remove fields of the result.	2019-02-22 22:00:20 -07:00
David Brown	4b0fb5af36	Add --json support to forget command ... Fixes #2184	2019-02-22 22:00:15 -07:00
Alexander Neumann	3f069ac404	Merge pull request 2153 from dmd/umount-message ... more accurate umount help message, for #815	2019-02-10 12:43:48 +01:00
Alexander Neumann	56e5467096	Merge pull request 2032 from j6s/feature.case-insensitive-exclude ... Add options for case insensitive includes & excludes	2019-02-10 12:40:33 +01:00
Matthew Holt	6cf13483b5	Add snapshot ID to summary output	2019-02-10 12:09:09 +01:00
Matthew Holt	f645306a18	Initial implementation of JSON progress reporter	2019-02-10 12:09:09 +01:00
Daniel M. Drucker	29a5bd5b30	more accurate umount help message, for #815	2019-01-28 16:53:35 -05:00
Johannes Hertenstein	cdc287a7f6	Store reference to reject function for insensitive pattern rejection	2019-01-19 11:08:13 +00:00
Johannes Hertenstein	deedc38129	Only convert iexclude & iinclude casing once	2019-01-19 11:04:53 +00:00
Alexander Neumann	60c7020bcb	Print message in verbose mode only ... Closes #2140	2019-01-10 21:28:22 +01:00
Alexander Neumann	b96ef48562	Set development version for 0.9.4	2019-01-06 21:40:32 +01:00
Alexander Neumann	cd9b2295f1	Add version for 0.9.4	2019-01-06 21:40:32 +01:00
Alexander Neumann	2996c110f1	Merge pull request 2017 from kylewlacy/fuse_default_permissions_option ... mount: Add default-permissions flag to set FUSE option	2019-01-06 21:19:27 +01:00
Alexander Neumann	4609b5c24d	Merge pull request 2094 from juergenhoetzel/password-command ... Add support for reading password from external command	2019-01-06 21:14:52 +01:00
Alexander Neumann	830511460a	mount: Turn on DefaultPermissions for --allow-other` ... This commit changes the logic slightly: checking the permissions in the fuse mount when nobody else besides the current user can access the fuse mount does not sense. The current user has access to the repo files in addition to the password, so they can access all data regardless of what the fuse mount does. Enabling `--allow-root` allows the root user to access the files in the fuse mount, for this user no permission checks will be done anyway. The code now enables `DefaultPermissions` automatically when `--allow-other` is set, it can be disabled with `--no-default-permissions` to restore the old behavior.	2019-01-06 20:55:49 +01:00
Alexander Neumann	1f246c5309	Merge pull request 2090 from plumbeo/within-hours ... Increase the granularity of the "keep within" snapshot retention policy	2019-01-06 15:04:55 +01:00
Matthew Holt	de54618852	cmd: Don't exit if printing to stdout or stderr fails (closes #2102)	2018-12-01 15:11:22 -07:00
Kyle Lacy	cf0883e16c	mount: Add "no-default-permissions" option ... This option restores the previous behavior of `mount` by disabling the "DefaultPermissions" FUSE option. This allows any user that can access the mountpoint to read any file from the snapshot. Normal FUSE rules apply, so `allow-root` or `allow-other` can be used to allow users besides the mounting user to access these files.	2018-11-26 21:16:01 -08:00
Kyle Lacy	a35a24b8b4	mount: Enable "DefaultPermissions" FUSE option by default ... This enforces the Unix permissions of the snapshot files within the mounted filesystem, which will only allow users to access snapshot files if they had access to the file outside of the snapshot.	2018-11-26 21:16:01 -08:00
Juergen Hoetzel	df7f72cdde	Add support for reading password from external command ... This allows reading the password from an password manager (like "pass"). Signed-off-by: Juergen Hoetzel <juergen@archlinux.org>	2018-11-26 20:18:31 +01:00
plumbeo	71891b340c	Support time ranges expressed in hours in snapshot retention policies ... Make restic forget --keep-within accept time ranges measured in hours and choose accordingly which snapshots to keep and which to forget. Add relative tests.	2018-11-26 14:27:42 +01:00
Alexander Neumann	2434ab2106	Merge pull request 2098 from cdhowie/key-hinting ... Add key hinting (#2097)	2018-11-25 17:52:54 +01:00
Chris Howie	1688713400	Add key hinting (#2097)	2018-11-25 09:13:18 -05:00
Johannes Hertenstein	5fe6de219d	Add tests for insensitive variants of filter methods	2018-11-25 13:32:16 +01:00
Johannes Hertenstein	c13f79da02	Add iexclude and iinclude options to restore command	2018-11-25 13:32:16 +01:00
Johannes Hertenstein	db82e6b80c	Add iexclude option to backup command	2018-11-25 13:32:16 +01:00
Alexander Neumann	6dc7cca597	Merge pull request 2086 from frebib/fix-2085 ... Allow --files-from multiple times	2018-11-25 13:21:29 +01:00
Joe Groocock	09e9b74cbd	Allow --files-from multiple times ... Fixes https://github.com/restic/restic/issues/2085	2018-11-25 12:55:03 +01:00
Alexander Neumann	d53595e43c	Merge pull request 2088 from garrmcnu/stats-latest-host ... stats: Add shorthand flag `-H` for host name filter	2018-11-24 19:05:53 +01:00
plumbeo	a7971a3ece	Use local time for user-provided time of backup	2018-11-20 21:28:19 +01:00
Garry McNulty	4b3c054257	stats: Add shorthand flag -H for host name filter ... The default value of the `--host` flag was set to 'H' (the shorthand version of the flag), this caused the snapshot lookup to fail. Also add shorthand `-H` for `backup` command. Closes #2040	2018-11-14 21:58:31 +00:00
Alexander Neumann	c8fc72364a	Merge pull request 2081 from restic/fix-2080 ... backup: Open repository before async status starts	2018-11-11 14:21:47 +01:00
Alexander Neumann	8471a359ee	dump: Use path instead of filepath ... Some time ago we changed the paths in the repo to always use a slash for separation, it seems we missed that the `dump` command still uses the `filepath` package, so on Windows backslashes are used. Closes #2079	2018-11-10 12:43:11 +01:00
Alexander Neumann	f9422ff4c7	backup: Open repository before async status starts ... Closes #2080	2018-11-10 12:38:29 +01:00
Alexander Neumann	c0572ca15f	backup: Add warning when patterns do not match any files	2018-11-03 14:51:30 +01:00
Alexander Neumann	c012fccd22	Display local time for all commands	2018-11-02 20:36:15 +01:00
Alexander Neumann	9310cd0cd6	Merge pull request #2056 from restic/recover-data ... Add new command 'recover'	2018-10-28 14:26:48 +01:00
Alexander Neumann	19fd0f101f	Merge pull request 2053 from andyjack/correct-doc-for-self-update ... Fix incorrect command name in long doc for self-update	2018-10-20 11:51:13 +02:00
Alexander Neumann	8c91c51d1b	Fix timestamps	2018-10-20 11:08:03 +02:00
Alexander Neumann	7e28bf7e97	Add 'recover' command	2018-10-20 11:08:03 +02:00
Gábor Lipták	d2ee58f2e9	Correct typo	2018-10-18 21:30:14 -04:00
Andy Jack	3f25537a06	Fix incorrect command name in long doc for self-update	2018-10-18 18:45:31 -04:00
Andreas Nüßlein	6eedd66c1a	tiny formatting fix ... I noticed this discrepancy when trying to parse the output of restic.	2018-10-17 09:41:11 +02:00
Alexander Neumann	4c2072d875	Merge pull request 1719 from ifedorenko/mt-restorer ... new optimized multithreaded restore implementation	2018-10-14 20:29:16 +02:00
Igor Fedorenko	9e24154ec9	restore: significantly reduce memory footprint ... reworked restore error callback to use file location path instead of much heavier Node. this reduced restore memory usage by as much as 50% in some of my tests. Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:39:42 +02:00
Igor Fedorenko	da57302fca	restore: Removed legacy restore implementation ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:37:30 +02:00
Igor Fedorenko	1869930d95	restore: New optimized multithreaded implementation ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:37:30 +02:00
Alexander Neumann	a432b42c81	self-update: Check current version before download	2018-10-14 17:29:46 +02:00
Alexander Neumann	7d0f2eaf24	self-update: Use correct path to restic binary ... Closes #2041	2018-10-14 17:29:19 +02:00
Alexander Neumann	41a4d67d93	Merge pull request 1962 from mholt/lsjson ... ls: Stream output when using --json option	2018-10-14 15:51:38 +02:00
Alexander Neumann	d7baa67acb	snapshots: Don't print empty line for --compact ... Closes #2038	2018-10-13 20:17:34 +02:00
Alexander Neumann	167397c18c	Set development version for 0.9.3	2018-10-13 13:48:31 +02:00
Alexander Neumann	be36c5f150	Add version for 0.9.3	2018-10-13 13:48:31 +02:00
Johannes Hertenstein	277cba4b32	Catch errors when walking cache directories	2018-10-08 15:47:34 +02:00
Johannes Hertenstein	ed651df19b	Use correct method for joining paths	2018-10-08 15:47:08 +02:00
Johannes Hertenstein	641dc65e6e	Output directory size in cache command	2018-10-07 14:37:51 +02:00
Daniel Bretoi	966e5a5575	use latest instead of newest in forget language	2018-10-06 12:02:22 -07:00
Alexander Neumann	5dbef3712e	Merge pull request #2027 from j6s/feature.cache-dir-information ... Print base directory with cache command	2018-10-06 12:40:48 +02:00
Johannes Hertenstein	9b8deb51ba	Print base directory with cache command	2018-10-05 21:23:57 +02:00
Johannes Hertenstein	0916ff71bd	Add defaults to command line help of --cache-dir	2018-10-04 19:50:05 +02:00
Alexander Neumann	585a5e3416	Use --host for all commands, deprecate --hostname	2018-10-03 14:12:35 +02:00
Alexander Neumann	fb31d66951	backup: Improve error message for invalid pattern	2018-10-03 11:12:51 +02:00
Stefano Fancello	e2da0a416c	Expand Glob (wildcards character) in paths in file in --files-from	2018-10-03 10:37:35 +02:00
Alexander Neumann	6d53e767d5	cache: Print message when new cache is created ... Sometimes, users run restic without retaining the local cache directories. This was reported several times in the past. Restic will now print a message whenever a new cache directory is created from scratch (i.e. it did not exist before), so users have a chance to recognize when the cache is not kept between different runs of restic.	2018-08-29 22:01:10 +02:00
Quentin Bourgeois	a065ada46a	Make generated man pages more easy to read with items ... Allow Cobra to output a more list items friendly format, this make reading of the documentation more easily using information from manpages.	2018-08-27 17:42:54 +02:00
Alexander Neumann	896089976a	Fix comment	2018-08-26 19:54:17 +02:00
Alexander Neumann	de307ea2ab	Merge pull request #1876 from restic/forget-explain ... forget: Add --explain	2018-08-25 21:48:44 +02:00
Alexander Neumann	76387b6cd0	Remove old text table implementation	2018-08-20 22:00:23 +02:00
Alexander Neumann	9fd3796d93	forget: Display reasons why snapshots are kept ... This change displays the reasons for keeping a snapshot in the table, unless `--compact` is specified.	2018-08-20 21:47:51 +02:00
Alexander Neumann	15ad0e5bc7	walk: Pass parent tree ID to WalkFunc	2018-08-19 23:28:04 +02:00
Mikael Berthe	1f27d17c0d	walker.Walk: Pass parent tree-id to WalkFunc	2018-08-19 23:28:04 +02:00
Mikael Berthe	8af918a1e4	find: Add support for multiple patterns or objects	2018-08-19 23:28:04 +02:00
Mikael Berthe	bb5425a1d8	find: Add support for blobs, IDs and packs lookups ... With --blob, --tree and --pack, the find command now lists the snapshots that contain a specific tree or blob, or the snapshots that contain blobs belonging to a given pack. It also displays the pack ID a blob belongs to. A list of IDs can be given, as long as the IDs are all of the same type.	2018-08-19 22:32:30 +02:00
Matthew Holt	22475729ce	Select specific Node fields for listing	2018-08-19 09:03:47 -06:00
Matthew Holt	04c67d700d	ls: Stream output when using --json option	2018-08-19 00:18:43 -06:00
kitone	48cc2f2188	fix: switch struct_type value to lower case	2018-08-18 14:57:50 +02:00
kitone	7925217e25	ls: Add JSON output support for restic ls cmd	2018-08-18 14:57:50 +02:00
Alexander Neumann	401a564486	Merge pull request #1961 from mholt/statsoutput ... stats: Show what was scanned and scanning mode used	2018-08-18 14:29:14 +02:00
Matthew Holt	2d89311d49	stats: Show what was scanned and scanning mode used	2018-08-17 17:15:30 -06:00
Alexander Neumann	5a25ad1972	Update version in build.go	2018-08-17 21:17:26 +02:00
Alexander Neumann	1ab5703404	prune: Fix calculation for removed bytes	2018-08-14 22:06:05 +02:00
Alexander Neumann	49d95e9a50	Merge pull request #1949 from restic/add-self-update ... Add command self-update	2018-08-13 22:02:34 +02:00
Alexander Neumann	5fee36fa84	Merge pull request #1941 from mholt/lsfilter ... ls: Implement directory filter, optionally subfolders	2018-08-13 21:58:32 +02:00
Alexander Neumann	6e1a3987b7	Add 'self-update' command ... This commit adds a command called `self-update` which downloads the latest released version of restic from GitHub and replacing the current binary with it. It does not rely on any external program (so it'll work everywhere), but still verifies the GPG signature using the embedded GPG public key. By default, the `self-update` command is hidden behind the `selfupdate` built tag, which is only set when restic is built using `build.go`. The reason for this is that downstream distributions will then not include the command by default, so users are encouraged to use the platform-specific distribution mechanism.	2018-08-12 23:34:47 +02:00
Alexander Neumann	9630398e3b	ls: Rework and simplify logic ... This commit introduces two functions: withinDir() and approachingMatchingTree() Both bind the list of directories with a closure, so we don't need to iterate over the list in the function passed to Walk(). This reduces the indentation level and since we can just use return, we don't need the breaks any more. The case that len(dirs) == 0 can also be handled by the functions with a return, which saves another indentation level. The main function body of the function passed to Walk() was reduced to three cases: * Within one of the dirs: Print the node, and if recursive operation is requested, directly return, so the walker continues recursive traversal * Approaching one of the dirs: don't print anything, but continue recursive traversal. * Nothing of the two: abort walking this branch of the tree.	2018-08-12 23:13:34 +02:00
Alexander Neumann	7e34de4c29	ls: Add comments	2018-08-12 22:18:44 +02:00
Alexander Neumann	ace5cc4ed3	ls: Only skip directory nodes ... Special case for Walk(): When SkipDir is returned for a non-dir node, the remaining nodes for the current tree are skipped. We don't want that.	2018-08-12 22:02:59 +02:00
Alexander Neumann	7f617cfd7f	ls: Use nodepath for filter	2018-08-12 22:01:38 +02:00
Alexander Neumann	0deb4e5994	ls: Check dirs before opening the repository ... Users get feedback instantly, and before any expensive network calls have been made.	2018-08-12 21:59:57 +02:00
Andreas Skielboe	b07bb3d8c3	Reject files excluded by name before calling lstat to improve scan speed ... Adds a SelectByName method to the archive and scanner which only require the filename as input, and can thus be run before calling lstat on the file. Can speed up scanning significantly if a lot of filename excludes are used.	2018-08-12 17:51:12 +02:00
Matthew Holt	11ce572894	Fix bug where some folder listings were empty	2018-08-11 17:17:43 -06:00
Matthew Holt	7a468d1226	Speed up nonrecursive queries; include exact filter match	2018-08-11 16:18:09 -06:00
Matthew Holt	00e2fd8b5f	Apply feedback and use SkipNode	2018-08-11 15:25:22 -06:00
Matthew Holt	ac3bd6b2eb	Replace Exitf with errors.Fatalf	2018-08-10 22:15:33 -06:00
Matthew Holt	156d85a29b	Minor fixes/tweaks; add docs	2018-08-10 22:10:02 -06:00
Matthew Holt	8c146eac4b	ls: Implement directory filter, optionally subfolders	2018-08-10 21:41:38 -06:00
Matthew Holt	6f5b0f3622	stat: Improve error message for bad snapshot ID (fixes #1933)	2018-08-08 16:49:36 -06:00
Alexander Neumann	3422c1ca83	Merge pull request #1729 from mholt/stats ... Implement `restic stats` command to get more info about a repository	2018-07-31 23:24:36 +02:00
Matthew Holt	f6b2731aa5	stats: Add manual doc, improve -h doc ... Also rename files-by-content to files-by-contents, once and for all	2018-07-31 22:54:10 +02:00
Alexander Neumann	3678ec9ad8	Print message for password being read from stdin ... Closes #1911	2018-07-31 20:21:18 +02:00
Alexander Neumann	12c797700e	make statsWalkSnapshot return a function	2018-07-27 21:44:59 +02:00
Matthew Holt	daca9d6815	Consolidate mode flags; use new Walk function	2018-07-27 21:27:40 +02:00
Matthew Holt	930602a444	Update comment now that question was answered	2018-07-27 21:27:39 +02:00
Matthew Holt	acb05e7855	Fix filepath uniqueness bug for blobs-per-file mode	2018-07-27 21:27:39 +02:00
Matthew Holt	a7b95d716a	Implement four counting modes	2018-07-27 21:27:39 +02:00
Matthew Holt	925b542eb0	Count unique files by blob sequence rather than tree ID	2018-07-27 21:27:39 +02:00
Matthew Holt	f7659bd8b0	stats: Initial implementation of stats command	2018-07-27 21:27:39 +02:00
Alexander Neumann	f715bef82f	Merge pull request #1899 from garrmcnu/check-cache-dir ... check: Use --cache-dir argument	2018-07-22 21:03:52 +02:00
Garry McNulty	7603ab7ac1	check: Update --cache-dir argument handling based on code review comments ... The temporary cache directory is created in the specified directory, or if not specified in the default temporary directory.	2018-07-22 18:24:11 +01:00
Garry McNulty	f1f69bc648	check: Use --cache-dir argument ... Closes #1880	2018-07-20 20:51:20 +01:00
Alexander Neumann	fb74de6360	Return an error when exclude files cannot be read	2018-07-18 21:39:07 +02:00
Alexander Neumann	b511f4dce2	Improve help message for check	2018-07-05 22:19:08 +02:00
Mikael Berthe	fbf8073dfc	Fix find -i (case-insensitive search)	2018-06-24 19:35:11 +02:00
Alexander Neumann	7ddf91b65c	Merge pull request #1772 from ifedorenko/restore-verify ... Restore verify	2018-06-23 14:50:29 +02:00
Alexander Neumann	8dae2de2ce	Merge pull request #1858 from restic/fix-1857 ... Allow excluding files with $ via --exclude-file	2018-06-22 21:22:23 +02:00
Alexander Neumann	025ec9dff5	Allow excluding files with $ via --exclude-file ... Previously it wasn't possbile to exclude files with a literal dollar sign (`$`) via exclude files, now users can write `$$` for that.	2018-06-22 20:46:04 +02:00
David Roman	2384c1cee7	Ignore ExcludeOtherFs if Stdin is true ... Closes: #1807	2018-06-21 23:12:38 +02:00
Steve Kriss	5088905502	add JSON output support for restic key list cmd ... Signed-off-by: Steve Kriss <steve@heptio.com>	2018-06-18 15:26:26 -07:00
Iku Iwasa	f080142137	Support NetBSD without fuse	2018-06-16 15:55:04 +09:00
Igor Fedorenko	e206680947	restore: New --verify flag to verify restored files content ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-06-11 07:57:22 -04:00
Igor Fedorenko	5fa6dc53cb	Refactor: introduced restorer tree visitor ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-06-11 07:57:22 -04:00
Igor Fedorenko	26be094f28	Refactor: moved restorer to separate package ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-06-11 07:57:22 -04:00
Alexander Neumann	321efec60c	Fix integration tests on Windows	2018-06-10 00:00:22 +02:00
Alexander Neumann	8f26fe271c	ls: Use walker for ls	2018-06-09 23:35:20 +02:00
Alexander Neumann	081743d0a5	find: Use walker.Walk	2018-06-09 23:35:20 +02:00
Alexander Neumann	ce01ca30d6	find: Correct tree pruning optimization ... The `find` command will now take care to only mark trees as "not found" when the pattern couldn't be found within any subtree. Closes #1825, #1823	2018-06-09 18:59:13 +02:00
Alexander Neumann	e2d347a698	find: Use OS independent slash-based format	2018-06-09 18:58:13 +02:00
Alexander Neumann	42ebb0a0a6	backup: Parse timestamp earlier	2018-06-09 18:21:12 +02:00
Alexander Neumann	bfd923e81e	rclone: Respect bandwith limits	2018-05-22 20:48:17 +02:00
Alexander Neumann	8d9d218d1c	list: Improve error message ... Before: $ restic list Fatal: type not specified After: $ restic list Fatal: type not specified, usage: list [blobs|packs|index|snapshots|keys|locks] Closes #1783	2018-05-17 19:41:56 +02:00
Mikael Berthe	a868a30f4d	Fix restic backup --one-file-system / ... This patch should fix the following panic when trying to backup the root filesystem with thre --one-file-system flag: % restic backup --one-file-system / (...) panic: item /, device id 2082 not found, allowedDevs: map[/:2082]	2018-05-16 13:44:14 +02:00
Alexander Neumann	cc627e832b	Add custom Duration type	2018-05-13 12:48:42 +02:00
Alexander Neumann	5a0f0e3faa	Add support for keeping a range of snapshots	2018-05-13 12:48:42 +02:00
Matthew Holt	b52f2aa9a4	forget: Add policy to keep snapshots before a date	2018-05-13 12:48:10 +02:00
Alexander Neumann	159badf5ba	Merge pull request #1778 from restic/fix-1771 ... archiver: Improve error handling	2018-05-13 00:13:54 +02:00
Alexander Neumann	903a3a31dc	Merge pull request #1776 from restic/fix-1775 ... Always use cleaned path for excludes	2018-05-13 00:13:52 +02:00
Alexander Neumann	581c62ee72	archiver: Improve error handling ... This commit changes how the worker goroutines for saving e.g. blobs interact. Before, it was possible to get stuck sending an instruction to archive a file or dir when no worker goroutines were available any more. This commit introduces a `done` channel for each of the worker pools, which is set to the channel returned by `tomb.Dying()`, so it is closed when the first worker returned an error.	2018-05-12 23:55:54 +02:00
Alexander Neumann	ef7747313d	backup: Use absolute paths for allowed devices	2018-05-11 21:32:44 +02:00
Alexander Neumann	18d4ac2fd9	backup: Always use cleaned path for excludes	2018-05-11 21:32:41 +02:00
Alexander Neumann	9180e2c48a	Remove unneeded file excludes	2018-05-11 21:26:10 +02:00
Gaige Lama	d3c0bd6d0e	Fix small typo ... "explicitely" should be "explicitly"	2018-05-11 10:55:12 -04:00
Mikael Berthe	ffc276a603	cat: Do not display a stack trace ... Don't show a stack trace when "restic cat snapshot" is invoked with invalid/nonexistent IDs.	2018-05-10 19:22:46 +02:00
Alexander Neumann	fca4fe4459	backup: Disable status output for --quiet	2018-05-02 21:24:18 +02:00
Alexander Neumann	d0974c155d	backup: Tune user interface a bit ... Make non-verbose mode a bit more verbose.	2018-05-01 22:18:32 +02:00
Alexander Neumann	8026e6fdfb	Merge pull request #1749 from restic/add-cache-cmd ... Add 'cache' command to list and cleanup cache dirs	2018-05-01 21:25:52 +02:00
Alexander Neumann	f928aeec34	Add 'cache' command	2018-05-01 16:27:44 +02:00
Alexander Neumann	f77bc0fae8	Use textfile.Read() to read files ... This converts the text to UTF-8 and removes a Byte Order Mark.	2018-05-01 16:25:53 +02:00
Alexander Neumann	d8bbe5dc84	Print repository ID after opening	2018-04-30 14:19:07 +02:00
Alexander Neumann	d926b9fd80	Add profile build flag	2018-04-30 14:19:04 +02:00
Alexander Neumann	4ba8d40282	Add block profile option	2018-04-30 14:18:45 +02:00
Alexander Neumann	4fb1401266	Fix --cacert help text	2018-04-29 13:17:33 +02:00
Alexander Neumann	1449d7dc29	Remove background checking code	2018-04-28 22:08:11 +02:00
Alexander Neumann	0e78ac92d8	Use new archiver code for backup	2018-04-28 22:08:11 +02:00
Alexander Neumann	9fac2ca832	Add flags to set verbosity	2018-04-25 14:42:45 +02:00
Alexander Neumann	a5c0cf2324	Add workaround for symlinked temp dir on darwin ... Chdir to the tempdir, then use os.Getwd() to get the name that filepath.Abs() uses (and stores in the Snapshot).	2018-04-25 14:42:45 +02:00
Alexander Neumann	38926d8576	Use new archiver code in tests	2018-04-25 14:42:45 +02:00
Alexander Neumann	b804279fe8	integration tests: Don't print anything to stdout	2018-04-22 11:37:05 +02:00
Alexander Neumann	a069467e72	ls: Improve output	2018-04-22 11:37:05 +02:00
Alexander Neumann	09365cc4ea	Add --trace-profile	2018-04-22 11:37:05 +02:00
Eri Bastos	2aa6b49651	Return exit code 130 when SIGINT is received	2018-04-20 21:09:50 +02:00
Matthew Holt	c5829e9ffc	key: Add flag for non-interactive password changes	2018-04-20 14:29:39 +02:00
Alexander Neumann	1c1fede399	Improve error message for orphaned pack files	2018-04-07 10:07:54 +02:00
Eri Bastos	901e1b129c	Fixed issue #1608 - Use --time argument properly ... Backups via stdin will now handle --time argument and pass it down as expected	2018-04-03 14:40:42 -03:00
Alexander Neumann	01fe719aff	check: Make sure temp cache dir is removed	2018-04-01 18:09:53 +02:00
Alexander Neumann	2c964df3e2	Merge pull request #1699 from restic/fix-incremental-backup-test ... Slightly increment size for TestIncrementalBackup	2018-04-01 14:33:12 +02:00
Alexander Neumann	a95eb33616	check: Use cache in temporary directory if possible ... Closes #1694	2018-04-01 13:59:27 +02:00
Alexander Neumann	4544a77172	Slightly increment size for TestIncrementalBackup ... This should make the test more reliable, it should hit the accidental "repo is has grown too much" way less often.	2018-04-01 13:49:42 +02:00
Alexander Neumann	fe99340e40	Add rclone backend	2018-04-01 10:16:31 +02:00
Alexander Neumann	d3f9c8b362	forget: Print policy	2018-03-30 10:24:26 +02:00
Leo R. Lundgren	923be90906	Make version output one line.	2018-03-21 21:49:03 +01:00
Bowen Song	34671d7c9b	Skip archiver.Scan before backup when --quiet is set	2018-03-20 20:44:10 +00:00
Alexander Neumann	8206f85d2e	Merge pull request #1552 from lawrencejones/use-auto-auth ... Automatically load Google auth	2018-03-18 19:53:30 +01:00
Alexander Neumann	dfb6d0fced	Reduce data for TestIncrementalBackup	2018-03-11 21:17:27 +01:00