1680 Commits

Author	SHA1	Message	Date
Alexander Weiss	39a26066f7	rebuild-index: add missing bar.Done()	2021-01-31 18:28:02 +01:00
MichaelEischer	b3dc127af5	Merge pull request #3207 from aawsome/filerestorer-fix-errorhandling ... Fix error handling in filerestorer and fix #3166	2021-01-31 17:47:46 +01:00
Alexander Weiss	6e942693ba	Fix #3166	2021-01-31 14:22:57 +01:00
Alexander Neumann	027a51529d	prune: Improve error message for missing files ... This commit changes the error message so that a list of file names is printed. Before, just the raw map was printed, which is not a great user interface.	2021-01-31 11:31:27 +01:00
Alexander Neumann	f867e65bcd	Fix issues reported by staticcheck	2021-01-30 20:43:53 +01:00
Alexander Neumann	aef3658a5f	Address review comments	2021-01-30 20:02:37 +01:00
Alexander Neumann	200f09522d	Add more error checks	2021-01-30 20:02:37 +01:00
Alexander Neumann	cbd88c457a	backup: Improve error handling	2021-01-30 20:02:37 +01:00
Alexander Neumann	1a0eb05bfa	errcheck: Add more error checks	2021-01-30 20:02:37 +01:00
MichaelEischer	4d576c2f79	Merge pull request #3217 from M4a1x/read-data-subset-fix ... Fix missing rand seed for restic check --read-data-subset=x%	2021-01-29 23:34:13 +01:00
Max Stabel	f9e1fa26ff	Fix missing rand seed for restic check --read-data-subset=x%	2021-01-29 23:18:35 +01:00
Alexander Weiss	e08e65dc30	prune: Simplify logic selecting packs to repack	2021-01-29 22:27:22 +01:00
Alexander Weiss	daeb4cdf8f	prune: Fix statistics for --repack-cacheable-only	2021-01-29 22:27:22 +01:00
Alexander Neumann	a16ce65295	Merge pull request #3244 from MichaelEischer/better-damage-reports ... Print more details about possible repository damages	2021-01-29 11:45:45 +01:00
rawtaz	de7e3a0648	Merge pull request #2823 from greatroar/trust-mtime ... Add --ignore-ctime flag to backup and document change detection	2021-01-29 00:02:19 +01:00
greatroar	6bd8a2faaa	backup: Add --ignore-ctime option and document change detection	2021-01-28 23:42:10 +01:00
Michael Eischer	58b5679f14	prune: reword missing blobs error ... The previous wording could be understood such that the prune run did damage the repository.	2021-01-28 21:48:24 +01:00
Michael Eischer	7b8886c052	prune: report missing but unneeded pack files ... This indicates a damaged repository so add some output to help with debugging.	2021-01-28 21:46:01 +01:00
Michael Eischer	ff95999246	rebuild-index: report added/removed/reindexed files ... This should help with investigating missing pack files.	2021-01-28 21:46:01 +01:00
Michael Eischer	b71c52797a	find: correctly expand multiple blob ids ... For example `restic find --show-pack-id --blob f78dc991 5b9e4366 ddd8c7d4` would previously only expand one blob if all of them belong to the same file.	2021-01-28 21:21:54 +01:00
MichaelEischer	82140967d3	Merge pull request #3228 from aawsome/prune-all-trees ... prune: Remove all unused trees	2021-01-28 21:04:27 +01:00
MichaelEischer	35033d9b79	Merge pull request #3177 from MichaelEischer/fix-2759 ... prune: don't print stacktrace on console	2021-01-28 20:28:45 +01:00
Alexander Neumann	4d40c70214	Merge pull request #3211 from MichaelEischer/sftp-speedup ... Speed-up caching/pack download via SFTP	2021-01-28 14:16:55 +01:00
Alexander Neumann	44169d0dc4	Merge pull request #3205 from MichaelEischer/fix-quiet-verbose ... Properly check that --quiet and --verbose are not combined	2021-01-28 13:53:31 +01:00
Alexander Neumann	bdfedf1f5b	Merge pull request #3173 from MichaelEischer/unify-index-loading ... Unify index loading	2021-01-28 13:50:42 +01:00
Michael Eischer	e2b0072441	check: add progress bar to the tree structure check	2021-01-28 11:10:50 +01:00
Michael Eischer	505f8a2229	progress/counter: Support updating the progress bar maximum	2021-01-28 11:10:47 +01:00
Michael Eischer	eda8c67616	restic: let FindUsedBlobs handle multiple snapshots at once	2021-01-28 11:08:43 +01:00
Michael Eischer	258ce0c1e5	parallel: report progress for StreamTrees ... This assigns an id to each tree root and then keeps track of how many tree loads (i.e. trees referenced for the first time) are pending per tree root. Once a tree root and its subtrees were fully processed there are no more pending tree loads and the tree root is reported as processed.	2021-01-28 11:08:43 +01:00
Michael Eischer	3d6a3e2555	copy: Remove treeCloner struct	2021-01-28 11:08:43 +01:00
Michael Eischer	0caad1e890	copy: parallelize tree walk	2021-01-28 11:08:43 +01:00
Alexander Neumann	a4689eb3b9	Merge pull request #3199 from MichaelEischer/non-interactive-counter ... Don't print progress on non-interactive terminals	2021-01-28 10:53:38 +01:00
Alexander Weiss	d7dc19a496	prune: Always repack packs containing tree blobs	2021-01-15 16:42:04 +01:00
Michael Eischer	f3442ce8a5	Test that WriteTo of a backend's Load remains accessible	2021-01-03 22:23:53 +01:00
Michael Eischer	97388b3504	Properly check that --quiet and --verbose are not combined ... If --verbose is specified once, then globalOptions.Verbose == 1. Previously --quiet --verbose would silently ignore the --verbose flag.	2020-12-30 21:24:18 +01:00
MichaelEischer	e1efc193e1	Merge pull request #3139 from aawsome/prune-healing ... prune: Add healing of repository in some situations	2020-12-29 22:17:27 +01:00
Alexander Weiss	f0113139ea	prune: Correct error message	2020-12-29 20:20:05 +01:00
Alexander Weiss	f6df94a50e	prune: Add self-healing ... Allow prune to heal situations where blobs in the index are missing or the corresponding packfiles are damaged if those blobs are not needed.	2020-12-29 20:20:05 +01:00
MichaelEischer	31e56f1ad5	Merge pull request #3197 from SkYNewZ/fix/3183 ... Fix tag handling for multiple tag lists	2020-12-29 18:44:38 +01:00
greatroar	dec5008369	Warn when unlock fails instead of returning an error ... Only one caller was checking the error.	2020-12-29 17:48:20 +01:00
Quentin Lemaire	25ecf9eafb	fix(cmd_tag): Use restic.TagLists	2020-12-29 17:12:46 +01:00
Quentin Lemaire	e88f3fb80c	fix(cmd_backup): Use restic.TagLists	2020-12-29 17:12:46 +01:00
Michael Eischer	05a987b07c	Support values less than 1 for RESTIC_PROGRESS_FPS ... For example set the variable to 0.016666 to print the progress once per minute.	2020-12-29 16:32:18 +01:00
Michael Eischer	85fe5feadb	Unify progress report frequency calculation	2020-12-29 16:03:43 +01:00
Michael Eischer	969141b5e9	Honor RESTIC_PROGRESS_FPS env variable on non-interactive terminals ... This makes it possible to use the environment variable to also get regular progress updates on non-interactive terminals.	2020-12-29 16:03:43 +01:00
Michael Eischer	c2ef049f1b	ui/progress: don't print progress on non-interactive terminals ... This reverts to the old behavior of not printing progress updates on non-interactive terminals. It was accidentally changed in #3058.	2020-12-29 16:03:43 +01:00
greatroar	18531e3d6f	Portability fixes to internal/restic ... syscall.Mknod is not available on AIX.	2020-12-29 01:35:01 +01:00
DRON-666	332b1896d1	Some options fixes ... Add tests for bool type. Fix subtle bug in TestOptionsApplyInvalid. Fix options list formatting.	2020-12-23 23:26:04 +03:00
Alexander Weiss	d8d2cc6dd9	Simplify rebuild-index code	2020-12-22 23:01:58 +01:00
Alexander Weiss	68b74e359e	Count packs directly in RebuildIndexFiles	2020-12-22 23:01:58 +01:00
Michael Eischer	ccc84af73d	debug/list: parallelize index loading	2020-12-22 22:36:18 +01:00
Alexander Neumann	edf89e1c74	forget: Enable --dry-run together with --prune	2020-12-22 20:58:02 +01:00
Michael Eischer	cfea79d0c5	prune: don't print stacktrace on console	2020-12-19 14:28:48 +01:00
DRON-666	2ca76afc2b	dump: Add new option --archive	2020-12-19 01:09:47 +03:00
DRON-666	0256f95994	dump: Split tar and walk logic	2020-12-19 01:09:47 +03:00
Alexander Neumann	0d5b764f90	Merge pull request #3130 from aawsome/snapshots-parallel ... Make loading snapshots parallel	2020-12-06 21:08:18 +01:00
Alexander Weiss	e329623771	Remove LoadAllSnapshots	2020-12-06 05:22:27 +01:00
Alexander Weiss	5b9ee56335	Add ForAllSnapshots	2020-12-06 05:04:21 +01:00
Alexander Weiss	83c8a9b058	Bugfix: packSizeFromList should save size from List()	2020-12-05 20:58:36 +01:00
Michael Eischer	f72f6c9c80	Fix debug build	2020-11-29 18:44:36 +01:00
MichaelEischer	52b98f7f95	Merge pull request #3017 from greatroar/files-from0 ... Add backup options --files-from-verbatim and --files-from-raw	2020-11-29 18:15:21 +01:00
Alexander Neumann	04d856e601	Merge pull request #3136 from restic/rawtaz-copy-doc ... doc: Emphasize double transfer and duplication in copy command	2020-11-29 13:57:10 +01:00
Alexander Neumann	a7b49c4889	Merge pull request #3119 from restic/keep-mountpoints ... Keep mountpoints as empty directories for --one-file-system	2020-11-29 11:22:12 +01:00
Leo R. Lundgren	f70b10d0ee	doc: Emphasize double transfer and duplication in copy command	2020-11-28 19:49:34 +01:00
greatroar	55bf76ba0c	backup: Add --files-from-{verbatim,raw} options	2020-11-28 18:22:31 +01:00
Alexander Neumann	82ae942965	backup: Keep mountpoints for --one-file-system ... When a file system is mounted at a directory, lstat() returns attributes of the root node of the mounted file system, including the device ID of the other file system. The previous code used when --one-file-system is specified excluded the directory itself because of that. This commit changes the code so that mountpoints are kept as empty directories, its attributes set to the root note of the mounted file system. The behavior mimics `tar`, which does the same.	2020-11-28 17:00:31 +01:00
Alexander Neumann	f576d3d826	Add tests	2020-11-28 17:00:31 +01:00
Alexander Neumann	037f0a4c91	Refactor device ID checking	2020-11-28 17:00:31 +01:00
Alexander Neumann	a3105799c9	Print warning when unlocking the repo fails	2020-11-28 11:31:25 +01:00
Kyle Brennan	666768cd17	cat: Respect --no-lock flag	2020-11-28 11:30:02 +01:00
Alexander Weiss	aa7a5f19c2	Use BlobHandle in index methods	2020-11-22 20:41:12 +01:00
Alexander Weiss	92bd448691	Make BlobHandle substruct of Blob	2020-11-22 20:41:10 +01:00
Alexander Weiss	67c938f232	Use PackSize in prune	2020-11-21 22:13:54 +01:00
Alexander Weiss	4960b841e6	Use PackSize in rebuild-index	2020-11-21 22:13:54 +01:00
Alexander Neumann	110a32a08b	Merge pull request #3113 from aawsome/fix-prune-stats-duplicates ... Fix statistics in prune for duplicates	2020-11-19 20:34:11 +01:00
Alexander Weiss	cb5ec7ea6b	Fix statistics in prune for duplicates ... Note that this fix only solves the statistics problem, if all duplicates are marked for repacking. If not all duplicates are marked for repacking, we lack the information which The situation that not all duplicates are marked for repacking can occur when using the `max-repack-size` option	2020-11-18 22:30:22 +01:00
Michael Eischer	625410f003	Limit progress bar updates to once per second on non-terminal outputs ... The code accidentally checked whether stdin is a terminal instead of stdout, the former is not relevant here as the output is printed on stdout.	2020-11-18 22:12:07 +01:00
Alexander Neumann	d857fb6e59	Allow using --repository-file in init	2020-11-17 20:43:46 +01:00
Michael Eischer	1f43cac12d	check: Only track data blobs when unused blobs should be reported ... This improves the memory usage of check a lot as it now only has to track tree blobs when run using the default parameters.	2020-11-15 18:43:07 +01:00
Michael Eischer	3500f9490c	check: Simplify blob status tracking ... UnusedBlobs now directly reads the list of existing blobs from the repository index. This removes the need for the blobStatusExists flag, which in turn allows converting the blobRefs map into a BlobSet.	2020-11-15 18:42:42 +01:00
Michael Eischer	caac38ed27	check: Tweak subset percentage over 100% error message	2020-11-15 18:13:50 +01:00
fgma	8f9cea8cc0	Check data subset: check random percentage subset	2020-11-15 18:13:50 +01:00
Alexander Neumann	3c0c0c132b	Merge pull request #3006 from aawsome/new-rebuild-index ... Reimplement rebuild-index and remove /internal/index	2020-11-15 17:48:43 +01:00
kitone	3c03b35212	Fix #1681 should not try to create the mount point if it doesn't exist, rather return an error	2020-11-15 17:09:30 +01:00
Alexander Weiss	9607cad267	Remove internal/index	2020-11-15 07:05:09 +01:00
Alexander Weiss	30b6a0878a	Reimplement rebuild-index	2020-11-15 07:05:09 +01:00
Alexander Weiss	1ec628ddf5	Add extraObsolete to MasterIndex.Save	2020-11-15 07:05:09 +01:00
MichaelEischer	3601a9b6cd	Merge pull request #2690 from SkYNewZ/master ... Fix #2688: Handle comma-separated list tags when using backup command	2020-11-14 23:19:42 +01:00
Michael Eischer	fdec8051ab	tags: Tweak description of --add/set/remove options	2020-11-14 22:55:30 +01:00
MichaelEischer	333c5a19d4	Merge pull request #3082 from aawsome/check-sizes ... Check: check sizes of packs from index and packheader	2020-11-14 22:37:42 +01:00
Quentin Lemaire	a8ad6b9a4b	fix(tags): Change tags list type according to restic.TagList	2020-11-14 16:24:58 +00:00
Quentin Lemaire	b0882b3f3c	fix(snapshots): Update help message to better match the 'forget' command one	2020-11-14 15:48:57 +00:00
Quentin Lemaire	ae441d3134	fix(backup): Switch tags cobra type to handle comma-separated list	2020-11-14 15:48:56 +00:00
Alexander Weiss	80dcfca191	check: Check sizes computed from index and pack header	2020-11-14 00:42:49 +01:00
Alexander Neumann	59ea5a4208	diff: Correctly count top-level blobs	2020-11-13 21:11:21 +01:00
greatroar	63e32c44c0	Improve error reporting from restic diff ... Instead of a stacktrace, restic diff 111 222 now reports: Fatal: no matching ID found for prefix "111"	2020-11-11 16:40:40 +01:00
greatroar	21b787a4d1	Stop Counters where they're constructed and started	2020-11-09 13:03:31 +01:00
greatroar	ddca699cd2	Replace restic.Progress with new progress.Counter ... This fixes two race conditions while cleaning up the code.	2020-11-09 12:12:35 +01:00
Alexander Neumann	a8d21b5dcf	prune: Warn if no cache is present	2020-11-08 20:25:35 +01:00
Alexander Neumann	823d0afd6e	backup: Always print parent snapshot info	2020-11-08 20:25:35 +01:00
Alexander Neumann	47277c4b4c	Add comments, clarify computation	2020-11-06 20:23:30 +01:00
Alexander Weiss	d2e53730d6	Add test that repo.List is only called once	2020-11-06 20:23:30 +01:00
Alexander Weiss	fd33030556	Use in-memory index to rebuild index in prune	2020-11-06 20:23:30 +01:00
Alexander Neumann	5144141321	Merge pull request #2718 from aawsome/new-cleanup-command ... Reimplementation of prune	2020-11-05 10:12:19 +01:00
Alexander Neumann	d35d279455	Set development version for 0.11.0	2020-11-05 09:41:40 +01:00
Alexander Neumann	b7e1ece1e0	Add version for 0.11.0	2020-11-05 09:41:35 +01:00
fgma	916b2d303b	vss: fix potential crash (not reachable in restic) (#3045) ... HasSufficientPrivilegesForVSS() now returns an error	2020-11-04 22:14:18 +01:00
Alexander Neumann	1ca60bccfb	Refactor condition for MaxRepackBytes ... Don't depend on the string (opts.MaxRepackSize) for the condition, instead check if there's a (positive) limit configured.	2020-11-03 16:42:21 +01:00
Alexander Neumann	7f86eb4ec0	Move helper function	2020-11-03 16:42:21 +01:00
Alexander Neumann	c1a3de4a6e	Refactor max-unused calculation, add unlimited option ... Add a callback to the PruneOptions struct which calculates the number of bytes allowed to be unused after prune is done. This way, the logic is closer to the option parsing code. Also, add an explicit option `unlimited` for the use case when storage does not matter but bandwidth and time do. Internally, this sets the maximum number of unused bytes to MaxUint64. Rework the documentation slightly so that no more "packs" are mentioned and it talks about "files" instead. Make it clear in the documentation that the percentage given to `--max-unused` is relative to the whole repository size after pruning is done. If specified, it must be below 100%, otherwise the repository would contain 100% of unused data, which is pointless. I had a hard time coming up with the correct formula to calculate the maximum number of unused bytes based on the number of used bytes. For a fraction `p` (0 ≤ p < 1), a repo with `u` bytes used, and the number of unused bytes `x` the following holds: x ≤ p * (u+x) ⇔ x ≤ p*u + p*x ⇔ x - p*x ≤ p*u ⇔ x * (1-p) ≤ p*u ⇔ x ≤ p/(1-p) * u	2020-11-03 16:42:21 +01:00
Alexander Neumann	f8c4dd7b1a	Split packe rewrite logic into two case branches ... The comma is too sublte, let's split this into two separate branches.	2020-11-03 16:42:21 +01:00
Alexander Neumann	a5b80452fe	Add comment that usedBlobs is modified	2020-11-03 16:42:21 +01:00
Alexander Neumann	aff1e220f5	Split struct members, add comments	2020-11-03 16:42:21 +01:00
Alexander Neumann	095155d9ce	Remove RepackSmall	2020-11-03 16:42:21 +01:00
Alexander Weiss	b2f5381737	Make realistic forget --prune --dryrun	2020-11-03 16:42:21 +01:00
Alexander Weiss	7f9a0a5907	Reimplementation of prune	2020-11-03 16:42:21 +01:00
Alexander Weiss	3b591ed987	Add Verboseff	2020-11-03 16:42:21 +01:00
Alexander Neumann	21ba15577e	Return an error for invalid sizes	2020-11-03 11:53:49 +01:00
Alexander Neumann	6509c207f4	Merge pull request #2997 from MichaelEischer/faster-excludes ... Speedup exclude/include checking	2020-11-02 11:07:21 +01:00
Alexander Neumann	445b845267	Merge pull request #2978 from MichaelEischer/warn-tree-error ... Warn if backup failed to read tree blob	2020-11-02 10:14:12 +01:00
Alexander Neumann	3ff37215df	Merge pull request #2935 from MichaelEischer/upgrade-minio ... Upgrade minio SDK to version 7	2020-11-02 09:09:10 +01:00
MichaelEischer	31b8d7a639	Merge pull request #2274 from fgma/master ... Support for Volume Shadow Copy Service (VSS) on windows	2020-10-24 15:43:34 +02:00
fgma	5695f9ebd2	vss: Implement VSS support for Windows ... The VSS support works for 32 and 64-bit windows, this includes a check that the restic version matches the OS architecture as required by VSS. The backup operation will fail the user has not sufficient permissions to use VSS. Snapshotting volumes also covers mountpoints but skips UNC paths.	2020-10-24 11:35:57 +02:00
greatroar	3807d13bdc	restic mount: don't umount unless we actually tried to mount ... Previously, failure to open a repository would result in an "unable to umount" message.	2020-10-21 12:15:05 +02:00
Alexander Neumann	56883817d8	Merge pull request #2990 from MichaelEischer/fix-goreport-warnings ... Fix some goreport warnings	2020-10-12 20:44:56 +02:00
Michael Eischer	e638b46a13	Embed context into ReaderAt ... The io.Reader interface does not support contexts, such that it is necessary to embed the context into the backendReaderAt struct. This has the problem that a reader might suddenly stop working when it's contained context is canceled. However, this is now problem here as the reader instances never escape the calling function.	2020-10-09 22:39:07 +02:00
Michael Eischer	2964d2ad15	Skip unlocking for nil locks ... Now that lockRepo receives a context, it is possible that it is canceled before a lock was created. Thus `unlockRepo` must be able to handle this case.	2020-10-09 22:39:06 +02:00
Michael Eischer	0c9efa9c2a	Pass context to lockRepo	2020-10-09 22:39:06 +02:00
Michael Eischer	37a5e2d681	rest: use global context on repository creation	2020-10-09 22:39:06 +02:00
Michael Eischer	dc2e664209	integration_fuse_test: use global context ... No need to use the TODO context.	2020-10-09 22:37:56 +02:00
Michael Eischer	603bb0e309	restore: Use proper context while loading snapshot	2020-10-09 22:37:56 +02:00
Michael Eischer	27456f6545	debug: use proper context ... This allows the debug commands to be properly interrupted.	2020-10-09 22:37:56 +02:00
Michael Eischer	c458e114d4	pass context to Find / FindSnapshot ... This allows proper interruption of restic while it searches for snapshots or key files.	2020-10-09 22:37:56 +02:00
Michael Eischer	45e9a55c62	Wire context into backend layout detection	2020-10-09 22:37:24 +02:00
Michael Eischer	307a6ba3a3	Upgrade minio sdk to v7 ... This changes are primarily straightforward modifications to pass the parameters in the now expected way.	2020-10-09 22:37:24 +02:00
Michael Eischer	50da20d93d	Warn if backup failed to read tree blob	2020-10-09 22:36:27 +02:00
Alexander Neumann	30cb553c8d	Merge pull request #2932 from MichaelEischer/proper-rclone-create ... Call rclone.Create to create a new repository for the rclone backend	2020-10-09 21:29:15 +02:00
Michael Eischer	bcc3bddcf4	filter: only check whether a child path could match when necessary ... When checking excludes there is no need to test whether a child path could also match the pattern, as it is by definition excluded. Previously childMayMatch was calculated but then discarded. For simple absolute paths this can account for half the time spent for checking pattern matches. name old time/op new time/op delta FilterPatterns/Relative-4 23.3ms ± 9% 21.7ms ± 6% -6.68% (p=0.004 n=10+10) FilterPatterns/Absolute-4 13.9ms ± 7% 10.0ms ± 5% -27.61% (p=0.000 n=10+10) FilterPatterns/Wildcard-4 51.4ms ± 7% 47.0ms ± 7% -8.51% (p=0.001 n=9+9) FilterPatterns/ManyNoMatch-4 551ms ± 9% 190ms ± 1% -65.41% (p=0.000 n=10+8) name old alloc/op new alloc/op delta FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.665 n=10+9) FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.480 n=9+10) FilterPatterns/Wildcard-4 14.3MB ± 0% 14.3MB ± 0% ~ (p=0.431 n=9+10) FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal) name old allocs/op new allocs/op delta FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal) FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal) FilterPatterns/Wildcard-4 88.7k ± 0% 88.7k ± 0% ~ (all equal) FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)	2020-10-07 20:47:52 +02:00
Michael Eischer	375c2a56de	filter: Parse filter patterns only once ... name old time/op new time/op delta FilterPatterns/Relative-4 30.3ms ±10% 23.6ms ±20% -22.12% (p=0.000 n=10+10) FilterPatterns/Absolute-4 49.0ms ± 3% 32.3ms ± 8% -33.94% (p=0.000 n=8+10) FilterPatterns/Wildcard-4 345ms ± 9% 334ms ±17% ~ (p=0.315 n=10+10) FilterPatterns/ManyNoMatch-4 3.93s ± 2% 0.71s ± 7% -81.98% (p=0.000 n=9+10) name old alloc/op new alloc/op delta FilterPatterns/Relative-4 4.63MB ± 0% 3.57MB ± 0% -22.98% (p=0.000 n=9+9) FilterPatterns/Absolute-4 8.54MB ± 0% 3.57MB ± 0% -58.20% (p=0.000 n=10+10) FilterPatterns/Wildcard-4 146MB ± 0% 141MB ± 0% -2.93% (p=0.000 n=9+9) FilterPatterns/ManyNoMatch-4 907MB ± 0% 4MB ± 0% -99.61% (p=0.000 n=9+9) name old allocs/op new allocs/op delta FilterPatterns/Relative-4 66.6k ± 0% 22.2k ± 0% -66.67% (p=0.000 n=10+10) FilterPatterns/Absolute-4 88.7k ± 0% 22.2k ± 0% -75.00% (p=0.000 n=10+10) FilterPatterns/Wildcard-4 1.70M ± 0% 1.63M ± 0% -3.92% (p=0.000 n=10+10) FilterPatterns/ManyNoMatch-4 4.46M ± 0% 0.02M ± 0% -99.50% (p=0.000 n=10+10)	2020-10-07 20:47:27 +02:00
MichaelEischer	0ae02f3030	Merge pull request #2844 from aawsome/prune-integration-tests ... Add prune integration tests for many edge cases	2020-10-06 23:16:41 +02:00
Lorenz Bausch	af4100e07d	Replace repo/repos with repository/repositories in --no-lock help text	2020-10-06 22:49:48 +02:00
Alexander Weiss	6822a58413	Add prune integration tests for many edge cases	2020-10-06 20:20:05 +02:00
Michael Eischer	1579d2a8ec	Remove some unused assignments	2020-10-06 14:55:13 +02:00
Michael Eischer	eba5dd831f	Fix typos reported by misspell	2020-10-06 14:55:13 +02:00
Leo R. Lundgren	e105a3f391	cache/check: Clarify error message when given arguments	2020-10-06 00:08:59 +02:00
rawtaz	5d8cfff3f2	Merge pull request #2988 from greatroar/snapshots-json ... Regression test for #2979	2020-10-05 21:02:52 +02:00
MichaelEischer	1ab4c710e1	Merge pull request #2987 from greatroar/no-password-for-help ... Don't read password for generate, help or self-update	2020-10-05 20:16:19 +02:00
greatroar	feedf0ebce	Don't read password for generate, help or self-update ... Fixes #2951.	2020-10-05 19:54:07 +02:00
MichaelEischer	1a490acd67	Merge pull request #2982 from greatroar/archiver-error-handling ... Check error in archiver before calling Select	2020-10-05 19:19:47 +02:00
greatroar	6bee0aafc2	Regression test for #2979	2020-10-05 15:08:09 +02:00
Leo R. Lundgren	137d20a06a	snapshots: Make --json output [] instead of null when no snapshots	2020-10-05 13:14:10 +02:00
greatroar	c4e2203e45	Check error in archiver before calling Select ... The archiver first called the Select function for a path before checking whether the Lstat on that path actually worked. The RejectFuncs in exclude.go worked around this by checking whether they received a nil os.FileInfo. Checking first is more obvious and requires less code.	2020-10-05 11:11:04 +02:00
rawtaz	bbeb439f41	Merge pull request #2937 from andreaso/self-update-output-path-fix ... Don't require `self-update --output` placeholder file	2020-10-03 13:35:20 +02:00
Andreas Olsson	ce14df303b	Don't require self-update --output placeholder file ... This removes the requirement on `restic self-update --output` to point to a path of an existing file, to overwrite. In case the specified path does exist we still want to verify that it's a regular file, rather than a directory or a device, which gets overwritten. We also want to verify that a path to a new file exists within an existing directory. The alternative being running into that issue after the actual download, etc has completed. While at it I also replace `errors.Errorf` with the more appropriately verbose `errors.Fatalf`. Resolves #2491	2020-10-03 07:02:23 +02:00
MichaelEischer	3c6671b18b	Merge pull request #2973 from restic/rawtaz-repo2 ... Correct data type for --repo2 option	2020-10-02 22:22:41 +02:00
rawtaz	0cce6dc31c	Correct data type for --repo2 option	2020-10-02 21:10:52 +02:00
rawtaz	6253ff0187	Clarify max verbose level in option description ... Clarifies that the max verbosity level is 3, in the description of the --verbose option.	2020-10-02 20:25:34 +02:00
jtagcat	a687261804	--help: --compact: clarify description	2020-10-02 16:55:56 +03:00
Michael Eischer	61035d68bc	Add test for --repository-file	2020-10-01 00:50:27 +02:00
Andreas Oberritter	97f7855de3	Add new option --repository-file (default: $RESTIC_REPOSITORY_FILE) ... As an alternative to -r, this allows to read the repository URL from a file in order to prevent certain types of information leaks, especially for URLs containing credentials. Fixes #1458, fixes #2900.	2020-10-01 00:50:26 +02:00
rawtaz	34ea960559	Merge pull request #2966 from MichaelEischer/recover-help-typo ... recover: Fix typo in the command help	2020-09-30 18:16:26 +02:00
Michael Eischer	2a79c1a44d	recover: Fix typo in the command help ... Reported-by: Steve Divskinsy <stevesbrain@users.noreply.github.com>	2020-09-30 17:44:34 +02:00
Michael Eischer	aea9f7d286	clarify verbose password prompt	2020-09-30 17:25:54 +02:00
plumbeo	009bd907f2	Don’t print too many messages when deleting files ... Print detailed debug messages on file deletions only when --verbose=2 is used	2020-09-27 14:24:04 +02:00
MichaelEischer	14b312f00d	Merge pull request #2658 from creativeprojects/issue-2241 ... Don't echo authentication passwords (rest backend)	2020-09-22 22:17:53 +02:00
Fred	206cadfab4	Hide password from repository URLs	2020-09-22 22:00:51 +02:00
MichaelEischer	4875f7b659	Merge pull request #2614 from greatroar/simplify-fs ... Simplify internal/fs	2020-09-21 22:05:10 +02:00
yoshiera	3cf29a777d	Fix nil check in rejectBySize	2020-09-21 19:20:24 +08:00
Michael Eischer	9333f707fa	init: use Create method for rclone backend ... This properly issues the initial repository creation command Fixes #1896	2020-09-19 19:11:43 +02:00
Alexander Neumann	429f97b887	Set development version for 0.10.0	2020-09-19 17:38:47 +02:00
Alexander Neumann	40832b2927	Add version for 0.10.0	2020-09-19 17:38:26 +02:00
Michael Eischer	f003410402	init: Add --copy-chunker-params option ... This allows creating multiple repositories with identical chunker parameters which is required for working deduplication when copying snapshots between different repositories.	2020-09-19 16:53:05 +02:00
Michael Eischer	655430550b	Extract parameters for second repository from copy command	2020-09-19 16:07:55 +02:00
Leo R. Lundgren	311ad2d2d0	Improve wording for --password-file and related options	2020-09-19 15:47:32 +02:00
Leo R. Lundgren	ffe6dce7e7	Improve wording for --password-command and --password-command2 options	2020-09-19 14:23:25 +02:00
yoshiera	ac4b8c98ac	Support excluding files by size	2020-09-19 19:52:09 +08:00
greatroar	9abef3bf1a	Move internal/fs.TestChdir to internal/test.Chdir	2020-09-17 10:43:33 +02:00
Michael Eischer	f5c448aa65	diff: Optimize diff calculation for shared subtrees ... When the diff calculation compares two trees with identical id then no differences between them can ever show up. Optimize for that case by simply traversing the tree only once to collect all referenced blobs for a proper calculation of added and removed blobs. Just skipping the common subtrees is not possible as this would skew the results if the added or removed blobs are shared with one of the subtrees.	2020-09-13 16:28:46 +02:00
Michael Eischer	c0fc85d303	diff: Add integration test	2020-09-13 16:28:46 +02:00
Alexander Neumann	0c48e515f0	Merge pull request #2630 from MichaelEischer/fix-staticcheck ... Fix lots of small issues reported by staticcheck	2020-09-13 16:19:22 +02:00
rawtaz	d3a286928a	Merge pull request #2905 from MichaelEischer/fix-ls-help ... ls: Explicitly enforce that the user specifies a snapshot ID	2020-09-05 10:44:01 +02:00
Michael Eischer	1ede018ea6	error variable names should start with 'Err'	2020-09-05 10:07:17 +02:00
Michael Eischer	d0329cf3eb	Adjust comments to match name of exported methods	2020-09-05 10:07:16 +02:00
Michael Eischer	b22655367c	integration_test: Replace fprintf without format string	2020-09-05 10:07:16 +02:00
Michael Eischer	068a3ce23f	Remove redundant return	2020-09-05 10:07:16 +02:00
Michael Eischer	d19f05c960	Use buf.String() instead of string(buf.Bytes())	2020-09-05 10:07:16 +02:00
Michael Eischer	460e2ffbf6	Collapse a few boolean operations	2020-09-05 10:07:14 +02:00
Michael Eischer	49b6aac3fa	Use !bytes.Equal instead of bytes.Compare !=0 to check for inequality	2020-09-05 10:06:23 +02:00
Michael Eischer	2f8335554c	Remove a few unused variables	2020-09-05 10:06:23 +02:00
Michael Eischer	37113282ca	Merge a few variable declaration and initializations	2020-09-05 10:05:34 +02:00
Michael Eischer	337725c354	Use sort.Strings	2020-09-05 10:05:34 +02:00
MichaelEischer	2ddb7ffb7e	Merge pull request #2255 from Kidswiss/tar ... Fix dumping issues with / and the first sub level	2020-09-01 21:52:17 +02:00
Michael Eischer	81dcfea11a	dump: Mention tar output in command help text	2020-08-31 22:43:10 +02:00
MichaelEischer	55071ee367	Merge pull request #2859 from buschjost/stats-filter-by-tag-and-path ... Add filter by tag and path to stats command	2020-08-31 22:11:01 +02:00
Michael Eischer	bcf44a9c3f	dump: Properly handle file paths without / prefix ... filepath.Rel failed if the requested path did not start with a `/` e.g. `restic` instead of `/restic`.	2020-08-30 18:25:42 +02:00
Oliver Buschjost	a7b4c19abf	Fix display of warning about ignored filters	2020-08-30 17:14:56 +02:00
Oliver Buschjost	d3fcfeba3a	Add filter by tag and path to stats command	2020-08-30 17:14:48 +02:00
Michael Eischer	e69449bf2c	dump: Add test for splitPath	2020-08-30 16:00:48 +02:00
MichaelEischer	ea81a0e282	Merge pull request #2606 from middelink/fix-323 ... Add copy functionality.	2020-08-30 10:18:24 +02:00
Simon Beck	d6f739ec22	dump: Extract tar code to internal/dump	2020-08-29 21:14:34 +02:00
Simon Beck	b98598e55f	dump: Fix dumping issues with / and the first sub level ... There was an issue that prevented the dump command from working correctly when either: * `/` contained multiple nodes (e.g. `restic backup /`) * dumping a file in the first sublevel was attempted (e.g. `/foo`)	2020-08-29 21:14:29 +02:00
Michael Eischer	d5f86effa1	ls: Explicitly enforce that the user specifies a snapshot ID ... The help messages suggested that the `ls` command work without explicitly passing a snapshot ID. However, this was never the case: without a snapshot ID the command just failed with the error `Ignoring "", it is not a snapshot id`. Fixes #2299	2020-08-29 14:28:53 +02:00
Michael Eischer	412623b848	copy: Reuse buffer for downloaded blobs	2020-08-29 10:48:44 +02:00
Michael Eischer	9a4796594a	integration tests: Fix checking of wrong snapshot	2020-08-29 10:48:44 +02:00
Michael Eischer	15374d22e9	integration tests: Add basic tests for copy command	2020-08-29 10:48:44 +02:00
Michael Eischer	88ad58d6cd	integration tests: Redirect directory diff into intermediate buffer	2020-08-29 10:48:44 +02:00
Michael Eischer	591a8c4cdf	integration tests: Deduplicate backup test-data setup code	2020-08-29 10:48:44 +02:00
Michael Eischer	ec9a53b7e8	copy: Mark and skip previously copied snapshots ... Use the `Original` field of the copied snapshot to store a persistent snapshot ID. This can either be the ID of the source snapshot if `Original` was not yet set or the previous value stored in the `Original` field. In order to still copy snapshots modified using the tags command the source snapshot is compared to all snapshots in the destination repository which have the same persistent ID. Snapshots are only considered equal if all fields except `Original` and `Parent` match. That way modified snapshots are still copied while avoiding duplicate copies at the same time.	2020-08-29 10:48:44 +02:00
Alexander Weiss	9867c4bbb4	Add flag --iexclude-file to backup	2020-08-28 21:04:41 +02:00
Jason Cox	2447f3f110	Update usage strings to put flags before args ... The standard UNIX-style ordering of command-line arguments places optional flags before other positional arguments. All of restic's commands support this ordering, but some of the usage strings showed the flags after the positional arguments (which restic also parses just fine). This change updates the doc strings to reflect the standard ordering. Because the `restic help` command comes directly from Cobra, there does not appear to be a way to update the argument ordering in its usage string, so it maintains the non-standard ordering (positional arguments before optional flags).	2020-08-28 02:16:22 +00:00
Michael Eischer	b0a8c4ad6c	copy: Only process each tree once ... This speeds up copying multiple overlapping snapshots from one repository to another, as we only have to copy the changed parts of later snapshots.	2020-08-26 22:17:52 +02:00
Michael Eischer	908b23fda0	copy: Update for modernized repository interface	2020-08-26 22:17:52 +02:00
greatroar	4508d406ef	copy: Remove separate SaveIndex in restic copy ... Flush does this now.	2020-08-26 22:17:52 +02:00
Pauline Middelink	7048cc3e58	Add copy functionality ... Add a copy command to copy snapshots between repositories. It allows the user to specify a destination repository, password, password-file, password-command or key-hint to supply the necessary details to open the destination repository. You need to supply a list of snapshots to copy, snapshots which already exist in the destination repository will be skipped. Note, when using the network this becomes rather slow, as it needs to read the blocks, decrypt them using the source key, then encrypt them again using the destination key before finally writing them out to the destination repository.	2020-08-26 22:17:52 +02:00
Michael Eischer	17995dec7a	Unify progress bar of check and prune commands	2020-08-25 22:47:38 +02:00
Alexander Neumann	b1b3f1ecb6	Merge pull request #2674 from MichaelEischer/prune-strict-checks ... prune: Stricter error checks	2020-08-23 10:30:41 +02:00
Bruce Dillahunty	fe69b83074	Update key password prompt (#2847)	2020-08-19 21:42:08 +02:00
Michael Eischer	08d24ff99e	prune: Include ID of all missing blobs in error message	2020-08-16 11:36:14 +02:00
Michael Eischer	1c84aceb39	prune: Test for abort on damaged repositories	2020-08-16 11:36:12 +02:00
Michael Eischer	575ed9a47e	Test that rebuild-index errors when old index cannot be removed	2020-08-16 11:34:01 +02:00
Michael Eischer	8f811642c3	Add support for integration tests to wrap the backend	2020-08-16 11:34:01 +02:00
Michael Eischer	744a15247d	prune/rebuild-index: Fail if an old index cannot be removed ... The old behavior was problematic in the context of rebuild-index as it could leave old, possibly invalid index files behind without returning a fatal error. Prune calls rebuildIndex before removing any data from the repository. For this use case failing to delete an old index MUST be treated as a fatal error. Otherwise the index could still contain an old index file that refers to blobs/packs that were later on deleted by prune. Later backup runs will assume that the affected blobs already exist in the repository which results in a backup which misses data.	2020-08-16 11:34:01 +02:00
Michael Eischer	3ba19869be	prune: Abort if any used blobs are missing ... The previous check only approximately verified whether all required blobs were found. However, after forgetting a few snapshots the repository contains lots of unused blobs whose number can be sufficient to make up for missing packs. When coupled with a malfunctioning backend that temporarily returns broken data this could cause restic to regard the corresponding packs as invalid and thereby delete data that's still in use. This change lets restic play it safe and refuse to delete anything if data is missing.	2020-08-16 11:34:01 +02:00
aawsome	0fed6a8dfc	Use "pack file" instead of "data file" (#2885) ... - changed variable names, especially changed DataFile into PackFile - changed in some comments - always use "pack file" in docu	2020-08-16 11:16:38 +02:00
MichaelEischer	08dee8a52b	Merge pull request #2865 from greatroar/unused ... Dead code removal	2020-08-08 16:15:28 +02:00
MichaelEischer	84b6f1ec53	Merge pull request #2874 from MichaelEischer/fix-verbose ... Adjust description of --verbose=n parameter	2020-08-04 23:21:05 +02:00
MichaelEischer	06fb4ea3f0	Merge pull request #2805 from jwilk-forks/verbose ... Fix examples of --verbose with argument	2020-08-04 23:11:35 +02:00
Michael Eischer	e38d415173	Adjust description of --verbose=n parameter	2020-08-04 23:07:53 +02:00
greatroar	d81a396944	Dead code removal ... Found by running golangci-lint on the entire code base.	2020-08-04 08:38:57 +02:00
MichaelEischer	38a2f9c07b	Merge pull request #2864 from greatroar/dump-error ... Fix error handling in dump (err != err)	2020-08-03 20:09:42 +02:00
rawtaz	5af2815627	Merge pull request #2821 from renard/mount-nolock ... cmd/mount: honur --no-lock flag	2020-08-03 20:09:26 +02:00
Sébastien Gross	9be4fe3e84	cmd/mount: honour --no-lock flag ... Do not lock the repository if --no-lock global flag is set. This allows to mount repositories which are archived on a read only system. Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>	2020-08-03 19:45:39 +02:00
Michael Eischer	05116e4787	prune: Cleanup progress bar handling while repacking	2020-08-03 19:32:46 +02:00
Michael Eischer	04f79b9642	prune: Stop progress bar after searching used blobs	2020-08-03 19:31:49 +02:00
greatroar	8b358935a0	Fix error handling in dump (err != err)	2020-08-03 09:43:44 +02:00
MichaelEischer	66d089e239	Merge pull request #2841 from aawsome/optimize-getUsedBlobs ... Extract get used blobs in prune as separate function	2020-08-01 22:40:33 +02:00
Alexander Weiss	9762bec091	Use optimized getUsedBlobs in prune	2020-08-01 21:07:31 +02:00
Alexander Weiss	d3692f5b81	Delete files in parallel in rebuild-index	2020-08-01 20:43:18 +02:00
Alexander Weiss	1c0b61204b	Delete files in parallel in forget	2020-08-01 20:43:18 +02:00
Alexander Weiss	2ee654763b	Delete files in parallel	2020-08-01 20:39:24 +02:00
Alexander Neumann	b7b479b668	Merge pull request #2599 from MichaelEischer/tweak-mem-usage ... Reduce memory usage when searching for used blobs	2020-08-01 13:49:02 +02:00
Alexander Neumann	2580eef2aa	Merge pull request #2318 from classmarkets/2175-named-keys ... Allow specifying user and host when adding keys	2020-08-01 13:06:31 +02:00
Michael Eischer	184103647a	FindUsedBlobs: merge seen into blobs BlobSet ... The seen BlobSet always contained a subset of the entries in blobs. Thus use blobs instead and avoid the memory overhead of the second set. Suggested-by: Alexander Weiss <alex@weissfam.de>	2020-08-01 12:29:16 +02:00
Michael Eischer	3ce9893e0b	Don't print a stacktrace if some files could not be read	2020-07-28 23:52:48 +02:00
Alexander Weiss	9d1fb94c6c	make Lookup() return all blobs ... + simplify syntax	2020-07-25 21:18:34 +02:00
MichaelEischer	020cab8e08	Merge pull request #2787 from greatroar/no-blobsize-cache ... Remove blob size cache from restic mount	2020-07-25 20:46:35 +02:00
greatroar	07da61baee	Remove blob size cache from restic mount	2020-07-25 19:39:14 +02:00
Alexander Weiss	9175795fdb	Check contents in archiver ... When backing up with a parent snapshot and the file is not changed, also check if contents are still available in index.	2020-07-25 08:18:28 +02:00
Michael Eischer	ddf0b8cd0b	checker: Properly distinguish between data and tree blobs ... If a data blob and a tree blob with the same ID (= same content) exist, then the checker did not report a data or tree blob as unused when the blob of the other type was still in use.	2020-07-20 22:58:39 +02:00
David Sommer	5729d967f5	backup: Return exit status code 3 when failing to read source data ... The backup command used to return a zero exit code as long as a snapshot could be created successfully, even if some of the source files could not be read (in which case the snapshot would contain the rest of the files). This made it hard for automation/scripts to detect failures/incomplete backups by looking at the exit code. Restic now returns the following exit codes for the backup command: - 0 when the command was successful - 1 when there was a fatal error (no snapshot created) - 3 when some source data could not be read (incomplete snapshot created)	2020-07-20 22:19:01 +02:00
greatroar	58719e1f47	Replace mount's per-file cache by a global LRU cache	2020-07-12 18:27:16 +02:00
Jakub Wilk	4e6a9767de	Fix examples of --verbose with argument	2020-06-23 20:22:14 +02:00
Andreas Olsson	0fcef2ec23	Use https:// for linking CACHEDIR.TAG specs ... That site might not have supported https:// when those links were originally added. It does now. Also dropping the _spec.html_ ending of the url, there being a `<link rel="canonical" ...>` tag suggesting that that no longer being the preferred address.	2020-06-21 19:36:26 +02:00
Alexander Weiss	b5543cff5d	Loop over index files for 'list blobs' ... => reduces memory consumption a lot!	2020-06-14 17:25:34 +02:00
Michael Eischer	bb1e258bb7	cat/stats: Get stdout from globalOptions	2020-06-14 11:08:11 +02:00
Michael Eischer	182655bc88	Replace fmt.Printf/Println/Fprintf with wrapper functions ... cmd/restic/globals.go already provides Printf, Println and Warnf wrapper which get their output streams from the globalOptions object. This allows for stream replacements when testing.	2020-06-14 11:08:11 +02:00
MichaelEischer	dd7b4f54f5	Merge pull request #2709 from greatroar/minio-sha256 ... Use Minio's optimized SHA-256	2020-06-12 23:32:58 +02:00
MichaelEischer	735a8074d5	Merge pull request #2773 from aawsome/index-uploads+knownblobs ... Fix non-intuitive repo behavior	2020-06-12 22:41:04 +02:00
Alexander Weiss	70347e95d5	disable index uploads for prune command ... + modifications of changelog	2020-06-12 09:24:38 +02:00
Alexander Weiss	91906911b0	Fix non-intuitive repository behavior ... - The SaveBlob method now checks for duplicates. - Moves handling of pending blobs to MasterIndex. -> also cleans up pending index entries when they are saved in the index -> when using SaveBlob no need to care about index any longer - Always check for full index and save it when storing packs. -> removes the need of an index uploader -> also removes the verbose "uploaded intermediate index" messages - The Flush method now also saves the index - Fix race condition when checking and saving full/non-finalized indexes	2020-06-11 13:05:23 +02:00
Michael Eischer	087c770161	Revive hint to the unlock command if a repository is locked ... errors.Fatalf wraps a error and just keeps an error message as a string. This prevents the `restic.IsAlreadyLocked(err)` check from working as the error is no longer an ErrAlreadyLocked. Just add an additional remark to the error using `errors.WithMessage`.	2020-06-10 20:58:56 +02:00
nunoperalta	2c3360db98	Fix #2774 - make unable to umount message clearer	2020-06-07 15:57:39 +01:00
MichaelEischer	84475aa3a8	Merge pull request #2730 from greatroar/mount-build-tags ... Simplify build tags for restic mount	2020-05-23 20:33:43 +02:00
Martin Michlmayr	5cc1760fdf	Fix typos	2020-05-16 14:05:26 +08:00
greatroar	649cbec6c5	Simplify build tags for restic mount ... This command can only be built on Darwin, FreeBSD and Linux (and if we upgrade bazil.org/fuse, only FreeBSD and Linux: https://github.com/bazil/fuse/issues/224). Listing the few supported operating systems explicitly here makes porting restic to new platforms easier.	2020-05-12 11:30:41 +02:00
MichaelEischer	1ee2306033	Merge pull request #2717 from MichaelEischer/fix-flaky-password-prompt ... backup: Avoid race between password prompt and open repository message	2020-05-01 23:58:55 +02:00
Michael Eischer	c882a92cd6	backup: Avoid race between password prompt and open repository message ... `term.Print` sends the output via a channel to a goroutine which actually prints the message. This may race with the password prompt printed by `OpenRepository` resulting in a missing prompt.	2020-05-01 23:42:16 +02:00
Michael Eischer	d465b5b9ad	Revert "Darwin test fix: allow 1μs timestamp difference" ... This reverts commit f3016a9096.	2020-05-01 17:11:30 +02:00
greatroar	42a3db05b0	Use Minio's optimized SHA-256 ... internal/repository benchmarks on an Intel i7-3770k: name old speed new speed delta PackerManager-8 209MB/s ± 1% 291MB/s ± 1% +38.94% (p=0.008 n=5+5) SaveAndEncrypt-8 112MB/s ± 1% 135MB/s ± 1% +20.25% (p=0.008 n=5+5)	2020-04-28 07:57:18 +02:00
greatroar	be5a0ff59f	Centralize buffer allocation and size checking in Repository.LoadBlob ... Benchmark results for internal/repository: name old time/op new time/op delta LoadTree-8 479µs ± 2% 478µs ± 1% ~ (p=0.780 n=10+9) LoadBlob-8 11.6ms ± 2% 11.6ms ± 1% ~ (p=0.631 n=10+10) LoadAndDecrypt-8 13.2ms ± 2% 13.3ms ± 3% ~ (p=0.631 n=10+10) name old alloc/op new alloc/op delta LoadTree-8 41.2kB ± 0% 41.2kB ± 0% ~ (all equal) LoadBlob-8 2.28kB ± 0% 2.28kB ± 0% ~ (all equal) LoadAndDecrypt-8 2.10MB ± 0% 2.10MB ± 0% ~ (all equal) name old allocs/op new allocs/op delta LoadTree-8 652 ± 0% 652 ± 0% ~ (all equal) LoadBlob-8 24.0 ± 0% 24.0 ± 0% ~ (all equal) LoadAndDecrypt-8 30.0 ± 0% 30.0 ± 0% ~ (all equal) name old speed new speed delta LoadBlob-8 86.2MB/s ± 2% 86.4MB/s ± 1% ~ (p=0.594 n=10+10) LoadAndDecrypt-8 75.7MB/s ± 2% 75.4MB/s ± 3% ~ (p=0.617 n=10+10)	2020-04-23 10:04:20 +02:00
MichaelEischer	9790d8ce1c	Merge pull request #2668 from MichaelEischer/fix-stats-blobs-crash ... stats: Fix crash in blobs-per-file mode on missing blob	2020-04-18 17:21:26 +02:00
MichaelEischer	08ec6c9f17	Merge pull request #2677 from MichaelEischer/complain-about-invalid-indexes ... rebuild_index: Report invalid packs that were ignored	2020-04-18 16:57:50 +02:00
MichaelEischer	b7c0d4d8bf	Merge pull request #2644 from greatroar/signal-notify-buffered ... Make all signal.Notify channels buffered	2020-04-18 11:31:02 +02:00
Michael Eischer	1a1c572bac	Fix shutdown hang when restic is started as background job ... restic uses a cleanup hook to ensure that it restores the terminal configuration to a sane state, when restic is interrupted while reading a password from the terminal. However, this causes a problem, when restic runs in a background job, as reconfiguring a terminal will cause a SIGTTOU to be sent to restic pausing it. Therefore, restic seems to hang on shutdown when it was running in the background. This commit changes the behavior to only restore the terminal configuration if restic was interrupted while reading a password from the terminal. As reading a password from the terminal requires that restic is in the foreground, this should avoid restic getting stopped. Fixes #2298 Issue introduced in #402	2020-04-12 22:27:09 +02:00
rawtaz	5a7c27ddb6	Merge pull request #2681 from MichaelEischer/optimize-debug ... Reduce memory usage and startup time of debug command	2020-04-04 00:08:14 +02:00
Michael Eischer	fb842759fc	debug: don't load the repository index ... The existing commands don't need a loaded repository index which can take several minutes to load on larger repositories.	2020-04-04 00:01:01 +02:00
Michael Eischer	7aa2f8a61e	debug: get stdout/stderr from gopts/globalOptions	2020-04-04 00:01:01 +02:00
Michael Eischer	08bf3bae79	debug: explicitly pass stdout to dump functions	2020-04-03 23:32:44 +02:00
Michael Eischer	bc74cd3ae5	backup/global: Use proper name for command line argument parameters ... Several paramters printed a generic "string" or "stringArray" name.	2020-04-03 19:49:04 +02:00
Michael Eischer	90243ed1c4	rebuild_index: Report invalid packs that were ignored	2020-04-02 22:38:31 +02:00
Michael Eischer	0ce81d88b6	stats: Fix crash in blobs-per-file mode on missing blob ... In a damaged repository with a missing blob, the error message tried to dereference the subtreeID field of the current node, which is a file however. Said field is set to nil for a file thus causing a segfault when dereferenced. Fix this by using the actual parentTreeID.	2020-03-27 22:17:54 +01:00
Peter Schultz	90fc639a67	Allow specifying user and host when adding keys ... The username and hostname for new keys can be specified with the new --user and --host flags, respectively. The flags are used only by the `key add` command and are otherwise ignored. This allows adding keys with for a desired user and host without having to run restic as that particular user on that particular host, making automated key management easier. Co-authored-by: James TD Smith <ahktenzero@mohorovi.cc>	2020-03-23 13:11:10 +01:00
greatroar	18fee4806f	Remove broken --allow-root from restic mount	2020-03-17 23:35:06 +01:00
greatroar	47d4d5bf1b	Make all signal.Notify channels buffered	2020-03-12 20:59:39 +01:00
rawtaz	58bd165253	Merge pull request #2581 from aawsome/multiple-hostnames ... Allow multiple hostnames tags	2020-02-27 08:35:23 +01:00
Alexander Weiss	9a9101d144	Support specifying multiple host flags for various commands ... The `dump`, `find`, `forget`, `ls`, `mount`, `restore`, `snapshots`, `stats` and `tag` commands will now take into account multiple `--host` and `-H` flags.	2020-02-26 22:17:59 +01:00
David Potter	71900248ab	Correct #2531 (stats restore-size calculates hard links incorrectly)	2020-02-15 19:34:22 +01:00
David Potter	23055aaadf	Correct #2537 (cmd_stats file counting issue)	2020-02-15 19:34:16 +01:00
curiousleo	4214b1746e	Ensure Print{,f,ln} use global stdout	2020-02-15 01:32:43 +01:00
curiousleo	f6f240573a	Don't Printf already formatted output ... Fixes #2281.	2020-02-15 01:32:43 +01:00
Michael Eischer	78c518ccac	Print backup summary after status output is shutdown	2020-02-13 21:14:20 +01:00
Michael Eischer	42a3292bcf	Better name for jsonstatus package ... internal/ui/jsonstatus and termstatus sound similar but are not related in any way. Instead `internal/ui/backup` and `internal/ui/jsonstatus/status` are the counterparts. Rename the latter to `internal/ui/json/backup` to make this clear.	2020-02-13 21:14:20 +01:00
Erik Rigtorp	94f4f13388	Add documentation on exit status codes to man pages ... This is step one to start defining useful exit codes for all the commands.	2020-02-12 23:09:26 +01:00
rawtaz	299f5971f2	Merge pull request #2560 from brualan/master ... Two small improvements to code quality	2020-02-12 22:40:20 +01:00
Lars Lehtonen	94a4d45dfb	cmd/restic: fix a dropped error	2020-02-11 09:19:03 -08:00
Alexander Bruyako	688014487b	avoid "index out of range" error ... in case of len(format) == 0, we will get an "index out of range" error Usage of strings.HasSuffix is allowing to avoid that	2020-01-27 19:24:42 +03:00
Lorenz Bausch	d818b7618b	Display respective snapshot date when using restic find	2019-12-17 07:51:12 +01:00
Leo R. Lundgren	77ef92b95c	diff: Fix wrong bytes reported in diff stats (#2469)	2019-11-22 17:26:01 +01:00
Alexander Neumann	30519f01ff	Set development version for 0.9.6	2019-11-22 16:19:03 +01:00
Alexander Neumann	b723ca3de5	Add version for 0.9.6	2019-11-22 16:18:59 +01:00
Alexander Neumann	fb95426f64	Rename environment variable to AWS_DEFAULT_REGION ... This seems to be the correct name, at least the AWS cli uses it: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html	2019-11-22 15:24:42 +01:00
mdauphin	df500a372d	Add AWS_REGION env var to specify s3 region	2019-11-22 15:04:48 +01:00
rawtaz	26d1f9f4ba	Merge pull request #2307 from gary-kim/fix-2306-password-retries ... Allow multiple retries for interactive password input	2019-11-20 18:30:20 +01:00
Jean Kahrs	8066e93f47	updade message that is shown when a cache cleanup is advised	2019-07-02 10:49:49 +02:00
Fred Akalin	3ca424050f	Add upper bound for t in --read-data-subset=n/t (#2304) ... * Add upper bound for t in --read-data-subset=n/t * Add changelog entry	2019-06-29 14:34:53 -04:00
Gary Kim	fea2464d4d	Allow multiple retries for interactive password input ... Restic used to quit if the repository password was typed incorrectly once. Restic will now ask the user again for the repository password if typed incorrectly. The user will now get three tries to input the correct password before restic quits.	2019-06-13 20:11:02 +08:00
Garry McNulty	4429a66b5f	backup: Convert relative pathname for --stdin-filename to absolute (#2063)	2019-05-08 15:21:27 +02:00
Lorenz Bausch	298f490195	Fix typo	2019-05-02 11:51:35 +02:00
Alexander Neumann	c7762453cf	Set development version for 0.9.5	2019-04-23 13:19:43 +02:00
Alexander Neumann	303210aa08	Add version for 0.9.5	2019-04-23 13:19:39 +02:00
Martin Michlmayr	0561155963	Split paragraph from list in restic stats --help ... The help text for `restic stats` lists a number of modes in a list. Make sure the "more info" text is a separate paragraph rather than being part of the list.	2019-04-23 10:36:33 +07:00
Martin Michlmayr	f11789c437	Add missing opening parenthesis	2019-04-23 10:36:26 +07:00
Alexander Neumann	8cab0c121d	Merge pull request #2221 from classmarkets/2203-password-from-stdin ... Fix reading password from stdin	2019-04-22 20:59:59 +02:00
Alexander Neumann	5979414bcd	Merge pull request #2124 from Kidswiss/tar ... Ability to dump folders to tar via stdout	2019-04-22 20:58:59 +02:00
Simon Beck	cc8b690b52	Restore whole folder to sdtout as tar ... With this change it is possible to dump a folder to stdout as a tar. The It can be used just like the normal dump command: `./restic dump fa97e6e1 "/data/test/" > test.tar` Where `/data/test/` is a a folder instead of a file.	2019-04-22 20:44:33 +02:00
Jan Niklas Richter	733519d895	Refactor duplicate code for grouping snapshots ... This commit is a followup to the addition of the --group-by flag for the snapshots command. Adding the grouping code there introduced duplicated code (the forget command also does grouping). This commit refactors boths sides to only use shared code.	2019-04-22 14:57:53 +02:00
Jan Niklas Richter	c4475ac58f	Move snapshot grouping code into own function to deduplicate code ... This commit moves the code which is used to group snapshots in the snapshots command into an own function to deduplicate code shared by the snapshots command and forget command.	2019-04-22 14:37:58 +02:00
Jan Niklas Richter	c9fd9b5275	Fix json tags for grouped snapshot output ... This commit will add json tags to the structs for json output, so all json variables of the snapshot command output are lowercase and snake-case. Furthermore it adds some internal code changes based on the feedback in the pull request #2087.	2019-04-22 14:37:58 +02:00
Jan Niklas Richter	cadcab5a19	Add GroupBy option to snapshots command ... This commit adds a --group-by option to the snapshots command, which behaves similar to the --group-by option of forget. Valid option values are "host, paths, tags". If this option is given, the output of snapshots will be divided into multiple tables, according to the value given (i.e. "host" will create a table of snapshots for each host, that has a snapshot in the list). Also the JSON output will be grouped. The default behavior (when --group-by is not given) has not changed. More to this discussion can be found in issue #2037.	2019-04-22 14:37:58 +02:00
Peter Schultz	5715517e29	Fix reading password from stdin ... Reading the password from non-terminal stdin used io.ReadFull with a byte slice of length 1000. We are now using a Scanner to read one line of input, independent of its length. Additionally, if stdin is not a terminal, the password is read only once instead of twice (in an effort to detect typos). Fixes #2203 Signed-off-by: Peter Schultz <peter.schultz@classmarkets.com>	2019-04-15 10:05:49 +02:00
Alexander Neumann	ecc2458de8	Merge pull request #2230 from cdhowie/issue-2229 ... Extend find --show-pack-id to work with --tree	2019-04-14 19:38:38 +02:00
Alexander Neumann	2c6ba5d9ac	Merge pull request #2231 from cdhowie/issue-2224 ... find command: don't abort on tree load errors	2019-04-14 19:31:06 +02:00
Alexander Neumann	6b700d02f5	Merge pull request #2217 from restic/improve-memory-usage ... WIP: improve memory usage	2019-04-13 15:07:07 +02:00
Alexander Neumann	78a3ffcfb9	Merge pull request #2185 from d3zd3z/json-forget ... Json forget	2019-04-13 14:23:32 +02:00
Alexander Neumann	d51e9d1b98	Add []byte to repo.LoadAndDecrypt and utils.LoadAll ... This commit changes the signatures for repository.LoadAndDecrypt and utils.LoadAll to allow passing in a []byte as the buffer to use. This buffer is enlarged as needed, and returned back to the caller for further use. In later commits, this allows reducing allocations by reusing a buffer for multiple calls, e.g. in a worker function.	2019-04-13 13:38:39 +02:00
Jack Henschel	ecd7ee85e8	Mention Cache Directoy Tagging Standard in man page and help text ... This patch makes it more explicit what is meant by the CACHEDIR.TAG file. It not only has to have this particular name, but also a specific content (described at http://bford.info/cachedir/spec.html), which is not immediately obvious to the user.	2019-04-02 15:53:07 +02:00
Chris Howie	36f22a0feb	find command: don't abort on tree load errors	2019-03-29 22:58:12 -04:00
Chris Howie	f58a44b911	Extend find --show-pack-id to work with --tree	2019-03-29 22:56:36 -04:00
Heiko Bornholdt	db8f5864fc	Add --ignore-inode option to backup cmd ... revised version of https://github.com/restic/restic/pull/2047	2019-03-10 21:24:29 +01:00
David Brown	d19a29f79e	Include reasons in json output of forget ... This dumps the reasons as well as the list of keeps and removes with the output from the forget command.	2019-02-23 09:38:33 -07:00
David Brown	d9e22c2df1	Add test for --json support for forget command ... This adds a test of the json output of the forget command, by running it once, asking it to keep one snapshot, and verifying that the output has the right number of snapshots listed in the Keep and Remove fields of the result.	2019-02-22 22:00:20 -07:00
David Brown	4b0fb5af36	Add --json support to forget command ... Fixes #2184	2019-02-22 22:00:15 -07:00
Alexander Neumann	3f069ac404	Merge pull request 2153 from dmd/umount-message ... more accurate umount help message, for #815	2019-02-10 12:43:48 +01:00
Alexander Neumann	56e5467096	Merge pull request 2032 from j6s/feature.case-insensitive-exclude ... Add options for case insensitive includes & excludes	2019-02-10 12:40:33 +01:00
Matthew Holt	6cf13483b5	Add snapshot ID to summary output	2019-02-10 12:09:09 +01:00
Matthew Holt	f645306a18	Initial implementation of JSON progress reporter	2019-02-10 12:09:09 +01:00
Daniel M. Drucker	29a5bd5b30	more accurate umount help message, for #815	2019-01-28 16:53:35 -05:00
Johannes Hertenstein	cdc287a7f6	Store reference to reject function for insensitive pattern rejection	2019-01-19 11:08:13 +00:00
Johannes Hertenstein	deedc38129	Only convert iexclude & iinclude casing once	2019-01-19 11:04:53 +00:00
Alexander Neumann	60c7020bcb	Print message in verbose mode only ... Closes #2140	2019-01-10 21:28:22 +01:00
Alexander Neumann	b96ef48562	Set development version for 0.9.4	2019-01-06 21:40:32 +01:00
Alexander Neumann	cd9b2295f1	Add version for 0.9.4	2019-01-06 21:40:32 +01:00
Alexander Neumann	2996c110f1	Merge pull request 2017 from kylewlacy/fuse_default_permissions_option ... mount: Add default-permissions flag to set FUSE option	2019-01-06 21:19:27 +01:00
Alexander Neumann	4609b5c24d	Merge pull request 2094 from juergenhoetzel/password-command ... Add support for reading password from external command	2019-01-06 21:14:52 +01:00
Alexander Neumann	830511460a	mount: Turn on DefaultPermissions for --allow-other` ... This commit changes the logic slightly: checking the permissions in the fuse mount when nobody else besides the current user can access the fuse mount does not sense. The current user has access to the repo files in addition to the password, so they can access all data regardless of what the fuse mount does. Enabling `--allow-root` allows the root user to access the files in the fuse mount, for this user no permission checks will be done anyway. The code now enables `DefaultPermissions` automatically when `--allow-other` is set, it can be disabled with `--no-default-permissions` to restore the old behavior.	2019-01-06 20:55:49 +01:00
Alexander Neumann	1f246c5309	Merge pull request 2090 from plumbeo/within-hours ... Increase the granularity of the "keep within" snapshot retention policy	2019-01-06 15:04:55 +01:00
Matthew Holt	de54618852	cmd: Don't exit if printing to stdout or stderr fails (closes #2102)	2018-12-01 15:11:22 -07:00
Kyle Lacy	cf0883e16c	mount: Add "no-default-permissions" option ... This option restores the previous behavior of `mount` by disabling the "DefaultPermissions" FUSE option. This allows any user that can access the mountpoint to read any file from the snapshot. Normal FUSE rules apply, so `allow-root` or `allow-other` can be used to allow users besides the mounting user to access these files.	2018-11-26 21:16:01 -08:00
Kyle Lacy	a35a24b8b4	mount: Enable "DefaultPermissions" FUSE option by default ... This enforces the Unix permissions of the snapshot files within the mounted filesystem, which will only allow users to access snapshot files if they had access to the file outside of the snapshot.	2018-11-26 21:16:01 -08:00
Juergen Hoetzel	df7f72cdde	Add support for reading password from external command ... This allows reading the password from an password manager (like "pass"). Signed-off-by: Juergen Hoetzel <juergen@archlinux.org>	2018-11-26 20:18:31 +01:00
plumbeo	71891b340c	Support time ranges expressed in hours in snapshot retention policies ... Make restic forget --keep-within accept time ranges measured in hours and choose accordingly which snapshots to keep and which to forget. Add relative tests.	2018-11-26 14:27:42 +01:00
Alexander Neumann	2434ab2106	Merge pull request 2098 from cdhowie/key-hinting ... Add key hinting (#2097)	2018-11-25 17:52:54 +01:00
Chris Howie	1688713400	Add key hinting (#2097)	2018-11-25 09:13:18 -05:00
Johannes Hertenstein	5fe6de219d	Add tests for insensitive variants of filter methods	2018-11-25 13:32:16 +01:00
Johannes Hertenstein	c13f79da02	Add iexclude and iinclude options to restore command	2018-11-25 13:32:16 +01:00
Johannes Hertenstein	db82e6b80c	Add iexclude option to backup command	2018-11-25 13:32:16 +01:00
Alexander Neumann	6dc7cca597	Merge pull request 2086 from frebib/fix-2085 ... Allow --files-from multiple times	2018-11-25 13:21:29 +01:00
Joe Groocock	09e9b74cbd	Allow --files-from multiple times ... Fixes https://github.com/restic/restic/issues/2085	2018-11-25 12:55:03 +01:00
Alexander Neumann	d53595e43c	Merge pull request 2088 from garrmcnu/stats-latest-host ... stats: Add shorthand flag `-H` for host name filter	2018-11-24 19:05:53 +01:00
plumbeo	a7971a3ece	Use local time for user-provided time of backup	2018-11-20 21:28:19 +01:00
Garry McNulty	4b3c054257	stats: Add shorthand flag -H for host name filter ... The default value of the `--host` flag was set to 'H' (the shorthand version of the flag), this caused the snapshot lookup to fail. Also add shorthand `-H` for `backup` command. Closes #2040	2018-11-14 21:58:31 +00:00
Alexander Neumann	c8fc72364a	Merge pull request 2081 from restic/fix-2080 ... backup: Open repository before async status starts	2018-11-11 14:21:47 +01:00
Alexander Neumann	8471a359ee	dump: Use path instead of filepath ... Some time ago we changed the paths in the repo to always use a slash for separation, it seems we missed that the `dump` command still uses the `filepath` package, so on Windows backslashes are used. Closes #2079	2018-11-10 12:43:11 +01:00
Alexander Neumann	f9422ff4c7	backup: Open repository before async status starts ... Closes #2080	2018-11-10 12:38:29 +01:00
Alexander Neumann	c0572ca15f	backup: Add warning when patterns do not match any files	2018-11-03 14:51:30 +01:00
Alexander Neumann	c012fccd22	Display local time for all commands	2018-11-02 20:36:15 +01:00
Alexander Neumann	9310cd0cd6	Merge pull request #2056 from restic/recover-data ... Add new command 'recover'	2018-10-28 14:26:48 +01:00
Alexander Neumann	19fd0f101f	Merge pull request 2053 from andyjack/correct-doc-for-self-update ... Fix incorrect command name in long doc for self-update	2018-10-20 11:51:13 +02:00
Alexander Neumann	8c91c51d1b	Fix timestamps	2018-10-20 11:08:03 +02:00
Alexander Neumann	7e28bf7e97	Add 'recover' command	2018-10-20 11:08:03 +02:00
Gábor Lipták	d2ee58f2e9	Correct typo	2018-10-18 21:30:14 -04:00
Andy Jack	3f25537a06	Fix incorrect command name in long doc for self-update	2018-10-18 18:45:31 -04:00
Andreas Nüßlein	6eedd66c1a	tiny formatting fix ... I noticed this discrepancy when trying to parse the output of restic.	2018-10-17 09:41:11 +02:00
Alexander Neumann	4c2072d875	Merge pull request 1719 from ifedorenko/mt-restorer ... new optimized multithreaded restore implementation	2018-10-14 20:29:16 +02:00
Igor Fedorenko	9e24154ec9	restore: significantly reduce memory footprint ... reworked restore error callback to use file location path instead of much heavier Node. this reduced restore memory usage by as much as 50% in some of my tests. Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:39:42 +02:00
Igor Fedorenko	da57302fca	restore: Removed legacy restore implementation ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:37:30 +02:00
Igor Fedorenko	1869930d95	restore: New optimized multithreaded implementation ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:37:30 +02:00
Alexander Neumann	a432b42c81	self-update: Check current version before download	2018-10-14 17:29:46 +02:00
Alexander Neumann	7d0f2eaf24	self-update: Use correct path to restic binary ... Closes #2041	2018-10-14 17:29:19 +02:00
Alexander Neumann	41a4d67d93	Merge pull request 1962 from mholt/lsjson ... ls: Stream output when using --json option	2018-10-14 15:51:38 +02:00
Alexander Neumann	d7baa67acb	snapshots: Don't print empty line for --compact ... Closes #2038	2018-10-13 20:17:34 +02:00
Alexander Neumann	167397c18c	Set development version for 0.9.3	2018-10-13 13:48:31 +02:00
Alexander Neumann	be36c5f150	Add version for 0.9.3	2018-10-13 13:48:31 +02:00
Johannes Hertenstein	277cba4b32	Catch errors when walking cache directories	2018-10-08 15:47:34 +02:00
Johannes Hertenstein	ed651df19b	Use correct method for joining paths	2018-10-08 15:47:08 +02:00
Johannes Hertenstein	641dc65e6e	Output directory size in cache command	2018-10-07 14:37:51 +02:00
Daniel Bretoi	966e5a5575	use latest instead of newest in forget language	2018-10-06 12:02:22 -07:00
Alexander Neumann	5dbef3712e	Merge pull request #2027 from j6s/feature.cache-dir-information ... Print base directory with cache command	2018-10-06 12:40:48 +02:00
Johannes Hertenstein	9b8deb51ba	Print base directory with cache command	2018-10-05 21:23:57 +02:00
Johannes Hertenstein	0916ff71bd	Add defaults to command line help of --cache-dir	2018-10-04 19:50:05 +02:00
Alexander Neumann	585a5e3416	Use --host for all commands, deprecate --hostname	2018-10-03 14:12:35 +02:00
Alexander Neumann	fb31d66951	backup: Improve error message for invalid pattern	2018-10-03 11:12:51 +02:00
Stefano Fancello	e2da0a416c	Expand Glob (wildcards character) in paths in file in --files-from	2018-10-03 10:37:35 +02:00
Alexander Neumann	6d53e767d5	cache: Print message when new cache is created ... Sometimes, users run restic without retaining the local cache directories. This was reported several times in the past. Restic will now print a message whenever a new cache directory is created from scratch (i.e. it did not exist before), so users have a chance to recognize when the cache is not kept between different runs of restic.	2018-08-29 22:01:10 +02:00
Quentin Bourgeois	a065ada46a	Make generated man pages more easy to read with items ... Allow Cobra to output a more list items friendly format, this make reading of the documentation more easily using information from manpages.	2018-08-27 17:42:54 +02:00
Alexander Neumann	896089976a	Fix comment	2018-08-26 19:54:17 +02:00
Alexander Neumann	de307ea2ab	Merge pull request #1876 from restic/forget-explain ... forget: Add --explain	2018-08-25 21:48:44 +02:00
Alexander Neumann	76387b6cd0	Remove old text table implementation	2018-08-20 22:00:23 +02:00
Alexander Neumann	9fd3796d93	forget: Display reasons why snapshots are kept ... This change displays the reasons for keeping a snapshot in the table, unless `--compact` is specified.	2018-08-20 21:47:51 +02:00
Alexander Neumann	15ad0e5bc7	walk: Pass parent tree ID to WalkFunc	2018-08-19 23:28:04 +02:00
Mikael Berthe	1f27d17c0d	walker.Walk: Pass parent tree-id to WalkFunc	2018-08-19 23:28:04 +02:00
Mikael Berthe	8af918a1e4	find: Add support for multiple patterns or objects	2018-08-19 23:28:04 +02:00
Mikael Berthe	bb5425a1d8	find: Add support for blobs, IDs and packs lookups ... With --blob, --tree and --pack, the find command now lists the snapshots that contain a specific tree or blob, or the snapshots that contain blobs belonging to a given pack. It also displays the pack ID a blob belongs to. A list of IDs can be given, as long as the IDs are all of the same type.	2018-08-19 22:32:30 +02:00
Matthew Holt	22475729ce	Select specific Node fields for listing	2018-08-19 09:03:47 -06:00
Matthew Holt	04c67d700d	ls: Stream output when using --json option	2018-08-19 00:18:43 -06:00
kitone	48cc2f2188	fix: switch struct_type value to lower case	2018-08-18 14:57:50 +02:00
kitone	7925217e25	ls: Add JSON output support for restic ls cmd	2018-08-18 14:57:50 +02:00
Alexander Neumann	401a564486	Merge pull request #1961 from mholt/statsoutput ... stats: Show what was scanned and scanning mode used	2018-08-18 14:29:14 +02:00
Matthew Holt	2d89311d49	stats: Show what was scanned and scanning mode used	2018-08-17 17:15:30 -06:00
Alexander Neumann	5a25ad1972	Update version in build.go	2018-08-17 21:17:26 +02:00
Alexander Neumann	1ab5703404	prune: Fix calculation for removed bytes	2018-08-14 22:06:05 +02:00
Alexander Neumann	49d95e9a50	Merge pull request #1949 from restic/add-self-update ... Add command self-update	2018-08-13 22:02:34 +02:00
Alexander Neumann	5fee36fa84	Merge pull request #1941 from mholt/lsfilter ... ls: Implement directory filter, optionally subfolders	2018-08-13 21:58:32 +02:00
Alexander Neumann	6e1a3987b7	Add 'self-update' command ... This commit adds a command called `self-update` which downloads the latest released version of restic from GitHub and replacing the current binary with it. It does not rely on any external program (so it'll work everywhere), but still verifies the GPG signature using the embedded GPG public key. By default, the `self-update` command is hidden behind the `selfupdate` built tag, which is only set when restic is built using `build.go`. The reason for this is that downstream distributions will then not include the command by default, so users are encouraged to use the platform-specific distribution mechanism.	2018-08-12 23:34:47 +02:00
Alexander Neumann	9630398e3b	ls: Rework and simplify logic ... This commit introduces two functions: withinDir() and approachingMatchingTree() Both bind the list of directories with a closure, so we don't need to iterate over the list in the function passed to Walk(). This reduces the indentation level and since we can just use return, we don't need the breaks any more. The case that len(dirs) == 0 can also be handled by the functions with a return, which saves another indentation level. The main function body of the function passed to Walk() was reduced to three cases: * Within one of the dirs: Print the node, and if recursive operation is requested, directly return, so the walker continues recursive traversal * Approaching one of the dirs: don't print anything, but continue recursive traversal. * Nothing of the two: abort walking this branch of the tree.	2018-08-12 23:13:34 +02:00
Alexander Neumann	7e34de4c29	ls: Add comments	2018-08-12 22:18:44 +02:00
Alexander Neumann	ace5cc4ed3	ls: Only skip directory nodes ... Special case for Walk(): When SkipDir is returned for a non-dir node, the remaining nodes for the current tree are skipped. We don't want that.	2018-08-12 22:02:59 +02:00
Alexander Neumann	7f617cfd7f	ls: Use nodepath for filter	2018-08-12 22:01:38 +02:00
Alexander Neumann	0deb4e5994	ls: Check dirs before opening the repository ... Users get feedback instantly, and before any expensive network calls have been made.	2018-08-12 21:59:57 +02:00
Andreas Skielboe	b07bb3d8c3	Reject files excluded by name before calling lstat to improve scan speed ... Adds a SelectByName method to the archive and scanner which only require the filename as input, and can thus be run before calling lstat on the file. Can speed up scanning significantly if a lot of filename excludes are used.	2018-08-12 17:51:12 +02:00
Matthew Holt	11ce572894	Fix bug where some folder listings were empty	2018-08-11 17:17:43 -06:00
Matthew Holt	7a468d1226	Speed up nonrecursive queries; include exact filter match	2018-08-11 16:18:09 -06:00
Matthew Holt	00e2fd8b5f	Apply feedback and use SkipNode	2018-08-11 15:25:22 -06:00
Matthew Holt	ac3bd6b2eb	Replace Exitf with errors.Fatalf	2018-08-10 22:15:33 -06:00
Matthew Holt	156d85a29b	Minor fixes/tweaks; add docs	2018-08-10 22:10:02 -06:00
Matthew Holt	8c146eac4b	ls: Implement directory filter, optionally subfolders	2018-08-10 21:41:38 -06:00
Matthew Holt	6f5b0f3622	stat: Improve error message for bad snapshot ID (fixes #1933)	2018-08-08 16:49:36 -06:00
Alexander Neumann	3422c1ca83	Merge pull request #1729 from mholt/stats ... Implement `restic stats` command to get more info about a repository	2018-07-31 23:24:36 +02:00
Matthew Holt	f6b2731aa5	stats: Add manual doc, improve -h doc ... Also rename files-by-content to files-by-contents, once and for all	2018-07-31 22:54:10 +02:00
Alexander Neumann	3678ec9ad8	Print message for password being read from stdin ... Closes #1911	2018-07-31 20:21:18 +02:00
Alexander Neumann	12c797700e	make statsWalkSnapshot return a function	2018-07-27 21:44:59 +02:00
Matthew Holt	daca9d6815	Consolidate mode flags; use new Walk function	2018-07-27 21:27:40 +02:00
Matthew Holt	930602a444	Update comment now that question was answered	2018-07-27 21:27:39 +02:00
Matthew Holt	acb05e7855	Fix filepath uniqueness bug for blobs-per-file mode	2018-07-27 21:27:39 +02:00
Matthew Holt	a7b95d716a	Implement four counting modes	2018-07-27 21:27:39 +02:00
Matthew Holt	925b542eb0	Count unique files by blob sequence rather than tree ID	2018-07-27 21:27:39 +02:00
Matthew Holt	f7659bd8b0	stats: Initial implementation of stats command	2018-07-27 21:27:39 +02:00
Alexander Neumann	f715bef82f	Merge pull request #1899 from garrmcnu/check-cache-dir ... check: Use --cache-dir argument	2018-07-22 21:03:52 +02:00
Garry McNulty	7603ab7ac1	check: Update --cache-dir argument handling based on code review comments ... The temporary cache directory is created in the specified directory, or if not specified in the default temporary directory.	2018-07-22 18:24:11 +01:00
Garry McNulty	f1f69bc648	check: Use --cache-dir argument ... Closes #1880	2018-07-20 20:51:20 +01:00
Alexander Neumann	fb74de6360	Return an error when exclude files cannot be read	2018-07-18 21:39:07 +02:00
Alexander Neumann	b511f4dce2	Improve help message for check	2018-07-05 22:19:08 +02:00
Mikael Berthe	fbf8073dfc	Fix find -i (case-insensitive search)	2018-06-24 19:35:11 +02:00
Alexander Neumann	7ddf91b65c	Merge pull request #1772 from ifedorenko/restore-verify ... Restore verify	2018-06-23 14:50:29 +02:00
Alexander Neumann	8dae2de2ce	Merge pull request #1858 from restic/fix-1857 ... Allow excluding files with $ via --exclude-file	2018-06-22 21:22:23 +02:00
Alexander Neumann	025ec9dff5	Allow excluding files with $ via --exclude-file ... Previously it wasn't possbile to exclude files with a literal dollar sign (`$`) via exclude files, now users can write `$$` for that.	2018-06-22 20:46:04 +02:00
David Roman	2384c1cee7	Ignore ExcludeOtherFs if Stdin is true ... Closes: #1807	2018-06-21 23:12:38 +02:00
Steve Kriss	5088905502	add JSON output support for restic key list cmd ... Signed-off-by: Steve Kriss <steve@heptio.com>	2018-06-18 15:26:26 -07:00
Iku Iwasa	f080142137	Support NetBSD without fuse	2018-06-16 15:55:04 +09:00
Igor Fedorenko	e206680947	restore: New --verify flag to verify restored files content ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-06-11 07:57:22 -04:00
Igor Fedorenko	5fa6dc53cb	Refactor: introduced restorer tree visitor ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-06-11 07:57:22 -04:00
Igor Fedorenko	26be094f28	Refactor: moved restorer to separate package ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-06-11 07:57:22 -04:00
Alexander Neumann	321efec60c	Fix integration tests on Windows	2018-06-10 00:00:22 +02:00
Alexander Neumann	8f26fe271c	ls: Use walker for ls	2018-06-09 23:35:20 +02:00
Alexander Neumann	081743d0a5	find: Use walker.Walk	2018-06-09 23:35:20 +02:00
Alexander Neumann	ce01ca30d6	find: Correct tree pruning optimization ... The `find` command will now take care to only mark trees as "not found" when the pattern couldn't be found within any subtree. Closes #1825, #1823	2018-06-09 18:59:13 +02:00
Alexander Neumann	e2d347a698	find: Use OS independent slash-based format	2018-06-09 18:58:13 +02:00
Alexander Neumann	42ebb0a0a6	backup: Parse timestamp earlier	2018-06-09 18:21:12 +02:00
Alexander Neumann	bfd923e81e	rclone: Respect bandwith limits	2018-05-22 20:48:17 +02:00
Alexander Neumann	8d9d218d1c	list: Improve error message ... Before: $ restic list Fatal: type not specified After: $ restic list Fatal: type not specified, usage: list [blobs|packs|index|snapshots|keys|locks] Closes #1783	2018-05-17 19:41:56 +02:00
Mikael Berthe	a868a30f4d	Fix restic backup --one-file-system / ... This patch should fix the following panic when trying to backup the root filesystem with thre --one-file-system flag: % restic backup --one-file-system / (...) panic: item /, device id 2082 not found, allowedDevs: map[/:2082]	2018-05-16 13:44:14 +02:00
Alexander Neumann	cc627e832b	Add custom Duration type	2018-05-13 12:48:42 +02:00
Alexander Neumann	5a0f0e3faa	Add support for keeping a range of snapshots	2018-05-13 12:48:42 +02:00
Matthew Holt	b52f2aa9a4	forget: Add policy to keep snapshots before a date	2018-05-13 12:48:10 +02:00
Alexander Neumann	159badf5ba	Merge pull request #1778 from restic/fix-1771 ... archiver: Improve error handling	2018-05-13 00:13:54 +02:00
Alexander Neumann	903a3a31dc	Merge pull request #1776 from restic/fix-1775 ... Always use cleaned path for excludes	2018-05-13 00:13:52 +02:00
Alexander Neumann	581c62ee72	archiver: Improve error handling ... This commit changes how the worker goroutines for saving e.g. blobs interact. Before, it was possible to get stuck sending an instruction to archive a file or dir when no worker goroutines were available any more. This commit introduces a `done` channel for each of the worker pools, which is set to the channel returned by `tomb.Dying()`, so it is closed when the first worker returned an error.	2018-05-12 23:55:54 +02:00
Alexander Neumann	ef7747313d	backup: Use absolute paths for allowed devices	2018-05-11 21:32:44 +02:00
Alexander Neumann	18d4ac2fd9	backup: Always use cleaned path for excludes	2018-05-11 21:32:41 +02:00
Alexander Neumann	9180e2c48a	Remove unneeded file excludes	2018-05-11 21:26:10 +02:00
Gaige Lama	d3c0bd6d0e	Fix small typo ... "explicitely" should be "explicitly"	2018-05-11 10:55:12 -04:00
Mikael Berthe	ffc276a603	cat: Do not display a stack trace ... Don't show a stack trace when "restic cat snapshot" is invoked with invalid/nonexistent IDs.	2018-05-10 19:22:46 +02:00
Alexander Neumann	fca4fe4459	backup: Disable status output for --quiet	2018-05-02 21:24:18 +02:00
Alexander Neumann	d0974c155d	backup: Tune user interface a bit ... Make non-verbose mode a bit more verbose.	2018-05-01 22:18:32 +02:00
Alexander Neumann	8026e6fdfb	Merge pull request #1749 from restic/add-cache-cmd ... Add 'cache' command to list and cleanup cache dirs	2018-05-01 21:25:52 +02:00
Alexander Neumann	f928aeec34	Add 'cache' command	2018-05-01 16:27:44 +02:00
Alexander Neumann	f77bc0fae8	Use textfile.Read() to read files ... This converts the text to UTF-8 and removes a Byte Order Mark.	2018-05-01 16:25:53 +02:00
Alexander Neumann	d8bbe5dc84	Print repository ID after opening	2018-04-30 14:19:07 +02:00
Alexander Neumann	d926b9fd80	Add profile build flag	2018-04-30 14:19:04 +02:00
Alexander Neumann	4ba8d40282	Add block profile option	2018-04-30 14:18:45 +02:00
Alexander Neumann	4fb1401266	Fix --cacert help text	2018-04-29 13:17:33 +02:00
Alexander Neumann	1449d7dc29	Remove background checking code	2018-04-28 22:08:11 +02:00
Alexander Neumann	0e78ac92d8	Use new archiver code for backup	2018-04-28 22:08:11 +02:00
Alexander Neumann	9fac2ca832	Add flags to set verbosity	2018-04-25 14:42:45 +02:00
Alexander Neumann	a5c0cf2324	Add workaround for symlinked temp dir on darwin ... Chdir to the tempdir, then use os.Getwd() to get the name that filepath.Abs() uses (and stores in the Snapshot).	2018-04-25 14:42:45 +02:00
Alexander Neumann	38926d8576	Use new archiver code in tests	2018-04-25 14:42:45 +02:00
Alexander Neumann	b804279fe8	integration tests: Don't print anything to stdout	2018-04-22 11:37:05 +02:00
Alexander Neumann	a069467e72	ls: Improve output	2018-04-22 11:37:05 +02:00
Alexander Neumann	09365cc4ea	Add --trace-profile	2018-04-22 11:37:05 +02:00
Eri Bastos	2aa6b49651	Return exit code 130 when SIGINT is received	2018-04-20 21:09:50 +02:00
Matthew Holt	c5829e9ffc	key: Add flag for non-interactive password changes	2018-04-20 14:29:39 +02:00
Alexander Neumann	1c1fede399	Improve error message for orphaned pack files	2018-04-07 10:07:54 +02:00
Eri Bastos	901e1b129c	Fixed issue #1608 - Use --time argument properly ... Backups via stdin will now handle --time argument and pass it down as expected	2018-04-03 14:40:42 -03:00
Alexander Neumann	01fe719aff	check: Make sure temp cache dir is removed	2018-04-01 18:09:53 +02:00
Alexander Neumann	2c964df3e2	Merge pull request #1699 from restic/fix-incremental-backup-test ... Slightly increment size for TestIncrementalBackup	2018-04-01 14:33:12 +02:00
Alexander Neumann	a95eb33616	check: Use cache in temporary directory if possible ... Closes #1694	2018-04-01 13:59:27 +02:00
Alexander Neumann	4544a77172	Slightly increment size for TestIncrementalBackup ... This should make the test more reliable, it should hit the accidental "repo is has grown too much" way less often.	2018-04-01 13:49:42 +02:00
Alexander Neumann	fe99340e40	Add rclone backend	2018-04-01 10:16:31 +02:00
Alexander Neumann	d3f9c8b362	forget: Print policy	2018-03-30 10:24:26 +02:00
Leo R. Lundgren	923be90906	Make version output one line.	2018-03-21 21:49:03 +01:00
Bowen Song	34671d7c9b	Skip archiver.Scan before backup when --quiet is set	2018-03-20 20:44:10 +00:00
Alexander Neumann	8206f85d2e	Merge pull request #1552 from lawrencejones/use-auto-auth ... Automatically load Google auth	2018-03-18 19:53:30 +01:00
Alexander Neumann	dfb6d0fced	Reduce data for TestIncrementalBackup	2018-03-11 21:17:27 +01:00
Alexander Neumann	c6c1dccc53	Reduce data set for TestRestore	2018-03-11 21:10:37 +01:00
Alexander Neumann	279566bafe	Reduce dataset for integration tests	2018-03-11 21:07:47 +01:00
Alexander Neumann	c67a8452f7	Disable polynomial check for chunker for tests	2018-03-11 20:59:40 +01:00
Lawrence Jones	0dfdc11ed9	Automatically load Google auth ... This change removes the hardcoded Google auth mechanism for the GCS backend, instead using Google's provided client library to discover and generate credential material. Google recommend that client libraries use their common auth mechanism in order to authorise requests against Google services. Doing so means you automatically support various types of authentication, from the standard GOOGLE_APPLICATION_CREDENTIALS environment variable to making use of Google's metadata API if running within Google Container Engine.	2018-03-11 17:11:25 +00:00
Eri Bastos	b9d643358a	Skip checking for password file existence if command is 'version' - Issue #1632	2018-03-08 17:55:03 -04:00
Jasper Lievisse Adriaanse	96311d1a2b	Add support for illumos/Solaris ... This does come without xattr/fuse support at this point. NB: not hooking up the integration tests as restic won't compile without cgo with Go < 1.10.	2018-03-04 20:11:29 +00:00
Alexander Neumann	7d59df1ab8	mount: Ignore non-existing locks ... Closes #1637	2018-02-25 13:11:03 +01:00
Igor Fedorenko	07d080830e	Add --read-data-subset flag to check command ... Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-02-18 23:31:27 -05:00
Alexander Neumann	c5575c7ed9	Strip spaces from items read via --files-from ... In #1590, it was mentioned that while lines read from exclude files via `--exclude-file` have leading and trailing spaces stripped, this is not the case for lines read via `--files-from`. This commit fixes that, spaces are always stripped.	2018-02-11 20:56:11 +01:00
Alexander Neumann	cbfa516575	Remove archiver progress "data processed" bandwith ... This commit removes the bandwidth displayed during backup process. It is misleading and seldomly correct, because it's neither the "read bandwidth" (only for the very first backup) nor the "upload bandwidth". Many users are confused about (and rightly so), c.f. #1581, #1033, #1591 We'll eventually replace this display with something more relevant when #1494 is done.	2018-01-28 09:30:26 +01:00
Alexander Neumann	5dc8d3588d	GS: Use generic http transport ... During the development of #1524 I discovered that the Google Cloud Storage backend did not yet use the HTTP transport, so things such as bandwidth limiting did not work. This commit does the necessary magic to make the GS library use our HTTP transport.	2018-01-27 20:12:34 +01:00
Alexander Neumann	c34db983d8	Read TLS client cert and key from the same file	2018-01-27 14:02:01 +01:00
Bryce Chidester	e805b968b1	Support for TLS client certificate authentication ... This adds --tls-client-cert and --tls-client-key parameters and enables use of that certificate/key pair when connecting to https servers.	2018-01-27 13:18:22 +01:00
Alexander Neumann	ff3de66ddf	Merge pull request #1582 from restic/optimize-debug-log ... Optimize debug logs	2018-01-26 21:57:18 +01:00
Alexander Neumann	f63c7048c7	index: Limit index files to 3000 pack files	2018-01-26 21:07:16 +01:00
Alexander Neumann	663c57ab4d	debug: Remove manual Str() call Log()	2018-01-25 20:49:41 +01:00
Alexander Neumann	9c55e8d69c	Merge pull request #1549 from MJDSys/more_index_lookup_avoids ... More optimizations to avoid calling Index.Lookup()	2018-01-24 20:53:30 +01:00
Matthew Dawson	df2c03a6a4	repository/master_index: Optimize Index.Lookup() ... When looking up a blob in the master index, with several indexes present in the master index, a significant amount of time is spent generating errors for each failed lookup. However, these errors are often used to check if a blob is present, but the contents are not inspected making the overhead of the error not useful. Instead, change Index.Lookup (and Index.LookupSize) to instead return a boolean denoting if the blob was found instead of an error. Also change all the calls to these functions to handle the new function signature. benchmark old ns/op new ns/op delta BenchmarkMasterIndexLookupSingleIndex-6 820 897 +9.39% BenchmarkMasterIndexLookupMultipleIndex-6 12821 2001 -84.39% BenchmarkMasterIndexLookupSingleIndexUnknown-6 5378 492 -90.85% BenchmarkMasterIndexLookupMultipleIndexUnknown-6 17026 1649 -90.31% benchmark old allocs new allocs delta BenchmarkMasterIndexLookupSingleIndex-6 9 9 +0.00% BenchmarkMasterIndexLookupMultipleIndex-6 59 19 -67.80% BenchmarkMasterIndexLookupSingleIndexUnknown-6 22 6 -72.73% BenchmarkMasterIndexLookupMultipleIndexUnknown-6 72 16 -77.78% benchmark old bytes new bytes delta BenchmarkMasterIndexLookupSingleIndex-6 160 160 +0.00% BenchmarkMasterIndexLookupMultipleIndex-6 3200 240 -92.50% BenchmarkMasterIndexLookupSingleIndexUnknown-6 1232 48 -96.10% BenchmarkMasterIndexLookupMultipleIndexUnknown-6 4272 128 -97.00%	2018-01-23 22:25:56 -05:00
Alexander Neumann	b0c6e53241	Fix calls to repo/backend.List() everywhere	2018-01-21 21:15:09 +01:00
Alexander Neumann	663e8284b2	init: Correct text (backend vs. repository) ... Closes #1565	2018-01-20 09:51:49 +01:00
Alexander Neumann	0bdb131521	Remove SuspendSignalHandler	2018-01-17 23:14:47 +01:00
Alexander Neumann	97565d68a2	fuse/mount: Add option for snapshot template	2018-01-15 14:47:17 +01:00
Alexander Neumann	92316a9853	restore: Improve error message for more than one ID	2018-01-06 22:44:18 +01:00
Alexander Neumann	c912b38bf0	forget: Also run prune when only IDs are forgotten	2018-01-01 21:27:40 +01:00
Alexander Neumann	18ecd9df30	Improve limiting HTTP based backends	2017-12-29 12:43:49 +01:00
Gábor Lipták	32505c3916	Correct golint warning on return	2017-12-26 17:09:41 -05:00
Alexander Neumann	84e493beba	b2: Warn when account ID or key is empty ... Closes #1490	2017-12-19 21:12:38 +01:00
Alexander Neumann	2f26fb8834	Only print cache warning for terminals	2017-12-13 19:58:10 +01:00
Alexander Neumann	d3ebe1311f	backup: Reject filenames with paths read from stdin	2017-12-13 19:55:53 +01:00
Alexander Neumann	3a5e040b7e	Merge pull request #1439 from armhold/propagate-context ... replace ad-hoc context.TODO() with gopts.ctx	2017-12-08 20:27:36 +01:00
George Armhold	1695c8ed55	use global context for check, debug, dump, find, forget, init, key, ... list, mount, tag, unlock commands gh-1434	2017-12-06 07:02:55 -05:00
George Armhold	366622f09a	use global context for cat and ls ... gh-1434	2017-12-06 05:48:39 -05:00
Alexander Neumann	0405e67f8b	Change diff output slightly	2017-12-05 22:26:56 +01:00
Alexander Neumann	df350e1f6e	Improve diff	2017-12-05 22:05:22 +01:00
Alexander Neumann	2424012d75	Add 'diff' command	2017-12-03 18:33:35 +01:00
Alexander Neumann	88607fc625	Correctly format all node types	2017-12-03 17:38:55 +01:00
Alexander Neumann	ad82781743	Reenable cache ... In 6341c7d72c, the cache was accidentally disabled due to a bug, this commit reenables the cache.	2017-12-03 15:52:57 +01:00
George Armhold	d886cb5c27	replace ad-hoc context.TODO() with gopts.ctx, so that cancellation ... can properly trickle down from cmd_*. gh-1434	2017-12-03 07:22:14 -05:00
Alexander Neumann	2579fe6b7b	Ignore SIGPIPE ... Handling SIGPIPE made restic abort when a TCP connection was reset by a server. This happened on DigitalOcean Spaces, which uses the s3 backend.	2017-11-30 21:23:43 +01:00
Alexander Neumann	eddb8549ef	backup: By default, do not save the access time ... This can be re-enabled with `--with-atime`.	2017-11-28 21:31:35 +01:00
Alexander Neumann	2567026ccb	Merge pull request #1447 from fawick/fixMultipleRejectIfPresent ... Fix multiple rejectIfPresent cancelling each other	2017-11-28 19:20:29 +01:00
Alexander Neumann	0cc8fc6f18	Merge pull request #1431 from n0npax/master ... fix #1411	2017-11-27 21:54:06 +01:00
Fabian Wickborn	27fadd2c6e	Document approach for multiple reject-if-present test	2017-11-27 21:38:15 +01:00
Alexander Neumann	dc38265b54	Merge pull request #1436 from restic/remove-old-cache ... Remove old cache directories	2017-11-27 21:37:05 +01:00
Fabian Wickborn	1ea518d5ef	cmd/restic: Use a dedicated cache for each rejectIfPresent	2017-11-27 17:33:53 +01:00
Fabian Wickborn	901cd5edef	cmd/restic: Add test for rejectIfPresent bug ... All RejectFuncs returned by rejectIfPresent share the same rejection cache and hence might cancel each other out.	2017-11-27 17:26:19 +01:00
Alexander Neumann	35a5307db3	Merge pull request #1415 from armhold/signals ... also handle SIGPIPE in cleanup routines	2017-11-24 21:28:08 +01:00
Alexander Neumann	6341c7d72c	cache: Add option to remove old cache dirs	2017-11-24 20:53:28 +01:00
Alexander Neumann	fa893ee477	cache: Add detection code for old cache dirs	2017-11-24 20:53:26 +01:00
n0npax	b12bba4e2a	fix #1411 ... replace panic during index save with fatal error	2017-11-19 11:40:47 +01:00
George Armhold	0ed2401711	exit 1 if received signal is other than SIGINT ... send cleanup msg to stderr, not stdout gh-1413	2017-11-09 07:16:01 -05:00
George Armhold	c347431907	also handle SIGPIPE in cleanup routines ... fixes gh-1413: restic fails to cleanup locks when bash pipeline fails	2017-11-05 07:14:27 -05:00
Björn Ketelaars	f63d7048f9	Fix test failing on OpenBSD #1307 ... Is seems that #1307 is similar to #1087, which describes a comparable observation on Apple's new filesystem. #1389 Has been committed and fixes the problem on Darwin. Although I'm not sure if the root cause of the issue is the same the solution is similar for OpenBSD, and leverages #1389.	2017-11-05 07:27:58 +01:00
Alexander Neumann	526aaca6f5	Merge pull request #1389 from wojas/fix-modtime-tests-on-apfs ... Fix modtime tests on APFS: allow 1us difference	2017-10-29 14:20:40 +01:00
Konrad Wojas	f3016a9096	Darwin test fix: allow 1μs timestamp difference ... On Darwin, allow a 1μs difference in restored timestamps, because macOS <10.13 cannot restore with nanosecond precision and the current version of Go (1.9.2) does not yet support the new syscall required for this. (#1087 #1389)	2017-10-29 20:48:07 +08:00
Zlatko Čalušić	134f834c60	Use lowercase in messages from check/tag commands, too	2017-10-27 21:06:34 +02:00
Konrad Wojas	c5da90a5b7	Add --last flag to snapshots command ... Add --last flag to snapshots command to only show the last entry for any (hostname, paths) combination. This makes it easier to check when various paths were last backed up.	2017-10-26 14:02:29 +08:00
Alexander Neumann	8d37b723ca	Upadte vendored dependencies	2017-10-22 10:07:36 +02:00
TobyLL	9e3dde8ec7	Fix whitespace	2017-10-19 15:52:06 +01:00
TobyLL	9dba182e51	Ignore comments (lines starting with #) in the --files-from file	2017-10-19 15:48:22 +01:00
Alexander Neumann	beaf55f1fc	dump: Allow absolute paths	2017-10-16 20:22:01 +02:00
Alexander Neumann	0e7281eb71	Merge pull request #1346 from fawick/cmdCatFile ... Add dump sub-command for dumping files to stdout	2017-10-16 20:13:44 +02:00
Alexander Neumann	897c923cc9	Retry failed backend requests	2017-10-14 15:56:25 +02:00
Alexander Neumann	67193e3deb	Improve error message when creating lock failed	2017-10-14 15:56:25 +02:00
Fabian Wickborn	d1d9c3f9d7	Renamed cmd_catfile.go to cmd_dump.go	2017-10-14 13:55:21 +02:00
Fabian Wickborn	cd5cbe0910	Rename debug dump related variable and run function	2017-10-14 13:55:00 +02:00
Fabian Wickborn	814e992c0b	Rename subcommand catfile to dump	2017-10-14 11:34:04 +02:00
Fabian Wickborn	87d084e18c	Add subcommand dump	2017-10-12 20:24:34 +02:00
Alexander Neumann	f2314b26ba	Move 'dump' to 'debug dump'	2017-10-12 20:18:45 +02:00
Alexander Neumann	74dcf41f25	Disable 'dump' for non-debug builds	2017-10-12 20:14:48 +02:00
rmdashrf	32637a0328	Basic rate limiting implementation. ... Added `--limit-upload` and `--limit-download` flags to rate limit backups and restores.	2017-10-11 20:01:20 -07:00
Alexander Neumann	c5553ec855	Merge pull request #1276 from fawick/supply_ca_cert ... Add REST backend option to use CA root certificate	2017-10-08 09:47:23 +02:00
Alexander Neumann	bb3ed54291	Merge pull request #1337 from mungomat/fuse_updateRepo ... fuse: mount and backup in parallel (#1330)	2017-10-08 09:45:28 +02:00
Tobias Klein	02200acad0	fuse: mount and backup in parallel (#1330)	2017-10-07 11:48:48 +02:00
Alexander Neumann	1a2d190bdb	Merge pull request #1334 from felix9/backup_tags ... backup should use latest parent regardless of tags	2017-10-07 10:45:02 +02:00
Fabian Wickborn	f0f17db847	Cache evaluated directories in isExcludedByFile ... Fixes #1271.	2017-10-06 22:11:22 +02:00
Felix Lee	7b44fd0f9d	fix #1143 ... Backup was choosing a parent snapshot that had the same tags, which makes backup unnecessarily slow when there are newer snapshots with different tags. There's no reason parent has to have the same tags. This change makes backup choose the newest snapshot instead.	2017-10-05 11:48:26 -07:00
Fabian Wickborn	69a6e622d0	Add REST backend option to use CA root certificate ... Closes #1114.	2017-10-04 22:14:10 +02:00
Alexander Neumann	5d1c1f721e	Merge pull request #1324 from antonlindstrom/password-feedback ... Add password successful feedback	2017-10-04 21:31:24 +02:00
Alexander Neumann	cdef55bb88	Merge pull request #1312 from hgfischer/issue-1119 ... nit: please remove dot imports #1119	2017-10-04 21:01:40 +02:00
Anton Lindstrom	a7baea0522	Output password successful on terminal stdout ... This removes the conditions that checks if the password is supplied through environment variable or file and outputs password is successful on terminal and when --quiet is not supplied.	2017-10-04 14:55:04 +02:00
Anton Lindstrom	55e6003749	Add password successful feedback ... This adds some feedback when entering the password on the command line. When the password is entered and supplied through stdin (and stdout is a terminal) then the a message saying `password is correct` if correct is printed.	2017-10-04 13:45:05 +02:00
Alexander Neumann	657a1d75af	snapshots: Add short ID to JSON output ... We're using the short ID in all output to users, so it should also be included in the JSON output of the `snapshots` command.	2017-10-03 15:24:09 +02:00
Anton Lindstrom	d46314648e	Add explicit OK output to check command ... This adds additional output to the check command when no errors were found. It means that when all checks have been completed, the following output is displayed: No errors were found The output is added to make sure that it is easier to understand that no errors were found. Full example output: Create exclusive lock for repository Load indexes Check all packs Check snapshots, trees and blobs No errors were found	2017-10-03 12:57:51 +02:00
Alexander Neumann	e45011af57	Merge pull request #1304 from jannickfahlbusch/showAmountOfSnapshots ... Show the number of snapshots	2017-10-03 12:41:47 +02:00
Alexander Neumann	fb09884893	Merge pull request #1311 from felix9/dump-no-trees ... "dump" command no longer has a "trees" option	2017-10-03 12:40:24 +02:00
Alexander Neumann	d1eecafa63	Merge pull request #1314 from jannickfahlbusch/fixTypo ... Fix typo in comment	2017-10-03 12:39:24 +02:00
Alexander Neumann	034b0b8040	forget: Run prune for manually forgotten snapshots	2017-10-03 11:56:13 +02:00
Jannick Fahlbusch	3dcacb3730	Fix typo in comment	2017-10-02 18:19:22 +02:00
Herbert	033589a66b	Use rtest on these as well to keep codebase consistent	2017-10-02 17:48:08 +02:00
Jannick Fahlbusch	e46a647c45	Just print the number of snapshots	2017-10-02 17:05:59 +02:00
Herbert	3473c3f7b6	Remove all dot-imports	2017-10-02 15:06:39 +02:00
Jannick Fahlbusch	2f10e25738	Show the amount of snapshots	2017-10-01 17:14:54 +02:00
Felix Lee	555bd257bd	dump command no longer has 'trees' option	2017-10-01 08:04:52 -07:00
Alexander Neumann	dad1c87afe	Rework generation of manpages and completion files ... This commit removes the `manpages` and `autocomplet` commands and replaces them with the more generic `generate` command. Also, zsh completion file support was added.	2017-09-26 13:12:12 +02:00
Alexander Neumann	56fccecd06	prune: Repack mixed pack files	2017-09-24 21:54:53 +02:00
Alexander Neumann	62ba9f1950	check: Disable cache by default	2017-09-24 21:54:53 +02:00
Alexander Neumann	610b676444	Automatically exclude current restic cache	2017-09-24 21:54:53 +02:00
Alexander Neumann	9be24a1c9f	Add cache ... This commits adds rudimentary support for a cache directory, enabled by default. The cache directory is created if it does not exist. The cache is used if there's anything in it, newly created snapshot and index files are written to the cache automatically.	2017-09-24 21:54:53 +02:00
Fabian Wickborn	ea1ab96749	Always sort snapshots lists ascending by timestamp ... Fixes #1219.	2017-09-24 13:01:13 +02:00
Alexander Neumann	9c6b7f688e	Merge pull request #1270 from restic/sftp-allow-password-prompt ... sftp: Allow password entry	2017-09-23 22:13:04 +02:00
Alexander Neumann	fb9729fdb9	sftp: Allow password entry ... This was a bit tricky: We start the ssh binary, but we want it to ignore SIGINT. In contrast, restic itself should process SIGINT and clean up properly. Before, we used `setsid()` to give the ssh process its own process group, but that means it cannot prompt the user for a password because the tty is gone. So, now we're passing in two functions that ignore SIGINT just before the ssh process is started and re-install it after start.	2017-09-23 11:43:33 +02:00
Alexander Neumann	45a09c76ff	Allow suspending SIGINT handler	2017-09-23 11:12:56 +02:00
Dmitriy Morozov	ae60188eb9	Add --compact option to forget	2017-09-22 16:32:59 -07:00
Alexander Neumann	e7de3b5f9d	Merge pull request #1266 from JaCoB1123/document_hostname_option ... Add note about rescan to hostname flag (fixes #1221)	2017-09-22 15:40:24 +02:00
Jan Bader	727fb6eabe	Add note about rescan to hostname flag (fixes #1221)	2017-09-22 14:29:04 +02:00
Alexander Neumann	a4577769ae	Merge pull request #1262 from mafgh/bytes ... fix duplicate bytes in prune output	2017-09-22 11:59:10 +02:00
Stefan Völkel	7f927d4774	fix duplicate bytes in prune output	2017-09-22 10:07:24 +02:00
Tobias Klein	1a83a739dc	fuse: added symlink 'latest' to snapshots-dir	2017-09-21 16:41:20 +02:00
Alexander Neumann	d7e644272f	prune: Add plausibility check	2017-09-19 10:50:07 +02:00
Alexander Neumann	fca9a523e9	Merge pull request #1241 from restic/fix-timestamp-check ... Use .Equal() instead of == for time.Time	2017-09-16 14:54:26 +02:00
Alexander Neumann	a60e751217	Use .Equal() instead of == for time.Time ... Closes #1238	2017-09-15 20:57:35 +02:00
Alexander Neumann	81c5d8a968	Correct bash completion file path	2017-09-15 20:45:16 +02:00
Alexander Neumann	c8c01a5cae	Merge pull request #1223 from mrzv/snapshots-compact ... Add --compact option to snapshots	2017-09-13 14:02:31 +02:00
Alexander Neumann	0f25ef9498	Merge pull request #1230 from shayneholmes/update-short-help ... Update style in short help commands	2017-09-11 22:25:31 +02:00
Alexander Neumann	36eaa22ed0	Merge pull request #1205 from mungomat/backup_time ... Backup time	2017-09-11 19:01:41 +02:00
Shayne Holmes	9eb39cef05	Capitalize short help commands ... Unify existing Cobra help command, and git-help's style.	2017-09-11 09:32:44 -07:00
Shayne Holmes	ee6150f67c	Change short help messages to imperative voice ... Unify the output of `restic help`.	2017-09-11 09:26:13 -07:00
Alexander Neumann	b1af544b1d	Merge pull request #1224 from restic/improve-exclude-caches ... Allow multiple exclude-if-present	2017-09-11 17:31:24 +02:00
Dmitriy Morozov	7a221f2473	Run changes through gofmt	2017-09-10 15:09:28 -07:00
Dmitriy Morozov	bdbe956c5c	Add --compact option to snapshots ... With --compact, snapshots doesn't list directories and puts all tags on a single line. This way each snapshot takes up exactly one line.	2017-09-10 13:06:43 -07:00
Alexander Neumann	b6aeea425b	Merge pull request #1196 from mungomat/forget_groupByTagsOnly ... forget: group-by-tags-only	2017-09-10 20:52:15 +02:00
Alexander Neumann	a8aa4eb06c	Rename parameter filename -> path	2017-09-10 20:28:21 +02:00
Alexander Neumann	0b2947dedb	Add test for rejectByPattern	2017-09-10 15:31:58 +02:00
Alexander Neumann	47ddd34266	Improve test	2017-09-10 15:20:41 +02:00
Alexander Neumann	2fdca5d310	Improve debug message	2017-09-10 15:16:05 +02:00
Alexander Neumann	e5d4e33509	Improve error message if no targets specified	2017-09-10 15:14:11 +02:00
Alexander Neumann	e117f613af	Move device test into new RejectFunc	2017-09-10 15:13:40 +02:00
Alexander Neumann	0dfdf02885	Rework pattern excludes	2017-09-10 14:34:28 +02:00
Alexander Neumann	4a0129fc2b	Rename excludeByFile -> rejectIfPresent	2017-09-10 14:25:58 +02:00
Alexander Neumann	a9c705009c	Move reject functions to new file	2017-09-10 14:25:25 +02:00
Alexander Neumann	d937ad8cf6	Rename FilenameCheck to RejectFunc ... We already have the opposite: pipe.SelectFunc(item string, fi os.FileInfo) bool, so RejectFunc is a good name.	2017-09-10 14:21:51 +02:00
Alexander Neumann	36e70228f2	Handle invalid key file	2017-09-10 10:55:01 +02:00
Alexander Neumann	c22c582546	Allow multiple exclude-if-present	2017-09-09 21:24:29 +02:00
Alexander Neumann	ea75509d6e	Print warning for non-existing items	2017-09-09 21:12:41 +02:00
Tobias Klein	ed30bd7b76	gofmt	2017-09-09 18:19:19 +02:00
Tobias Klein	8f9ef4402b	error in case of unknown grouping option	2017-09-09 15:55:37 +02:00
Tobias Klein	43ff971dfd	new sub-option for backup: time ... New option to specify the timestamp for a backup	2017-09-09 13:26:35 +02:00
Fabian Wickborn	dbda892542	Add option to exclude directories with a tagfile ... The option is named --exclude-if-present and accepts a parameter filename[:content]. Directories are excluded and their contents is not backed up if they contain a file with the specified name and, optionally, that starts with the specified content. The tagfile itself is never excluded. There is also a shortcut --exclude-caches that works in the same way as the likewise-named option of tar(1): Directories are recognized as cache if they contain a file named "CACHEDIR.TAG. Closes #317.	2017-09-09 09:57:42 +02:00
Tobias Klein	1073bfba37	flexible grouping option for the forget-command	2017-09-06 20:14:18 +02:00
Alexander Neumann	fa2ee78a5c	Merge pull request #1044 from lloeki/982-improve-restore ... Improve restore	2017-09-04 21:51:12 +02:00
Tobias Klein	e4a5cdc5bc	forget: group-by-tags-only	2017-09-03 17:11:25 +02:00
Alexander Neumann	0ee1650f82	Merge pull request #1191 from prattmic/profile ... debug: properly handle interrupted profiles	2017-09-03 09:49:46 +02:00
Alexander Neumann	de8c64e767	Use deterministic date for man pages	2017-09-02 09:27:11 +02:00
Michael Pratt	e4c469c149	debug: properly handle interrupted profiles ... By default (i.e., without profile.NoShutdownHook), profile.Start listens for SIGINT and will stop the profile and call os.Exit(0). restic already listens for SIGINT and runs its own cleanup handlers before calling os.Exit(0). As is, these handlers are racing when an interrupt occurs, and in my experience, restic tends to win the race, resulting in an unusable profile. Eliminate the race and properly stop profiles on interrupt by disabling package profile's signal handler and instead stop the profile in a restic cleanup handler.	2017-08-28 22:03:26 -07:00
Ricardo Seriani	62ed776a8c	Change key rm command to key remove ... Change key rm command to key remove, to follow manual and other commands	2017-08-17 11:03:26 -03:00
Loic Nageleisen	f880ff21aa	Fixing restore with excluded ... An exclude filter is basically a 'wildcard but foo', so even if a childMayMatch, other children of a dir may not, therefore childMayMatch does not matter, but we should not go down unless the dir is selected for restore.	2017-08-16 15:25:02 +02:00
Loic Nageleisen	4a36993c19	Smarter filter when children won't match ... This improves restore performance by several orders of magniture by not going through the whole tree recursively when we can anticipate that no match will ever occur.	2017-08-16 15:25:02 +02:00
Alexander Neumann	b67c178672	Merge pull request #1149 from restic/azure-support ... Add Azure blob storage as backend	2017-08-09 21:30:35 +02:00
Dipta Das	ba75a3884c	Add Google Cloud Storage as backend ... Environment variables: GOOGLE_PROJECT_ID=gcp-project-id GOOGLE_APPLICATION_CREDENTIALS=path-to-json-file Environment variables for test: RESTIC_TEST_GS_PROJECT_ID=gcp-project-id RESTIC_TEST_GS_APPLICATION_CREDENTIALS=path-to-json-file RESTIC_TEST_GS_REPOSITORY=gs:us-central1/test-bucket Init repository: $ restic -r gs🪣/[prefix] init	2017-08-06 21:47:55 +02:00
Alexander Neumann	d91d89eef6	azure: Create container if it does not exist	2017-08-06 21:47:04 +02:00
Dipta Das	3a85b6b7c6	Add Azure Blob Storage as backend ... Environment variables: AZURE_ACCOUNT_NAME=storage-account-name AZURE_ACCOUNT_KEY=storage-account-key Environment variables for test: RESTIC_TEST_AZURE_ACCOUNT_NAME=storage-account-name RESTIC_TEST_AZURE_ACCOUNT_KEY=storage-account-key RESTIC_TEST_AZURE_REPOSITORY=azure:restic-test-container Init repository: $ restic -r azure:container-name:/prefix/dir init	2017-08-06 21:47:04 +02:00
Alexander Neumann	6bc43a4198	manpage: Remove auto gen tag from man page	2017-08-06 21:31:01 +02:00
Alexander Neumann	e348b3deeb	manpage: Do not panic when no command is given	2017-08-06 21:01:49 +02:00
Alexander Neumann	739350fd8e	backup: Do not print hostname in help text ... This is necessary so that the manpage generation is deterministic and we can test if the man pages are up to date when the CI tests run.	2017-08-05 12:05:53 +02:00
Alexander Neumann	a3ab17b470	Add 'manpage' command to generate manual pages	2017-08-05 10:57:01 +02:00
Johannes Kohnen	37aad2e3aa	Close exclude files and check errors	2017-08-01 17:34:27 +00:00
donat	893bc9f777	Error message in case PasswordFile is missing	2017-07-27 14:23:08 +03:00
Michael Stapelberg	b89d3cc4d0	fix typo: explicitely → explicitly	2017-07-27 08:24:53 +02:00
Alexander Neumann	2e804511ca	Merge pull request #1133 from middelink/fix-1132 ... Force restic to ask the password when adding a key.	2017-07-26 22:11:37 +02:00
Pauline Middelink	c95e2b009e	Simpify cmd_backup and cmd_init now we have the password in gopts.	2017-07-24 23:32:55 +02:00
Pauline Middelink	d5615a67c8	Refactor password resolving. ... Instead of determining the password lazily during ReadPassword(), do so now in cobra.PersistentPreRunE() so we can store the result in the globalOptions and reuse/override when applicable without having to worry about the environment or flag options interfering.	2017-07-24 23:05:37 +02:00
Pauline Middelink	d9b9bbd4a8	Force restic to ask the password when adding a key. ... As `restic key add` uses the same `ReadPasswordTwice()` as the rest of restic, it is sensitive to the environment variable RESTIC_PASSWORD or --password-file= override. When asking for the new key, temporary remove these 2 overrides, forcing the password to be asked.	2017-07-24 22:00:44 +02:00
Alexander Neumann	d780b1eede	Rework withTestEnvironment ... Switch from a function passed as a parameter to a cleanup function, which is also executed when the test function panics, so no temporary directories are left behind.	2017-07-24 21:32:34 +02:00
Pauline Middelink	1717391f6c	Fuse testing leaves test mountpoint around. Move it under the testing tree which is removed after each test.	2017-07-24 20:33:39 +02:00
Alexander Neumann	23c903074c	Move restic package to internal/restic	2017-07-24 17:43:32 +02:00
Alexander Neumann	94b27e8933	Fix paths for tests	2017-07-23 14:39:57 +02:00
Alexander Neumann	6caeff2408	Run goimports	2017-07-23 14:21:03 +02:00
Alexander Neumann	83d1a46526	Moves files	2017-07-23 14:19:13 +02:00
Alexander Neumann	b63399d606	Move things around for gb ... This moves all restic source files to src/, and all vendored dependencies to vendor/src.	2016-02-20 17:31:20 +01:00
Alexander Neumann	1287b307ac	Add cleanup handler to restore terminal state ... Closes #402	2016-02-13 18:29:26 +01:00
Alexander Neumann	4ec5050cbb	backup: Hide status output for narrow terminals ... closes #428	2016-02-10 17:28:48 +01:00
Rached Ben Mustapha	ba35ca522a	Handle pack loading errors in rebuild-index ... Errors returned from backend.LoadAll() were not handled, leading to these fatal errors from the unpacker trying to read the size from the end of an empty buffer: `seeking to read header length failed: bytes.Reader.Seek: negative position` This change takes care of returning on error, as well as showing which pack failed to load and validating pack integrity.	2016-02-07 18:30:47 +00:00
Alexander Neumann	2c3a6a6fa9	cmd_rebuild_index: Remove calls to GetReader()	2016-01-24 00:42:04 +01:00
Alexander Neumann	61551b0591	cmd_cat: Remove calls to GetReader()	2016-01-24 00:42:04 +01:00
Alexander Neumann	f05a32509e	Add "Test" prefix to backend test functions	2016-01-23 19:12:02 +01:00
Alexander Neumann	6ba56befad	Abort fuse integration test on error ... Before, the fuse integration test was run and the tests were never finished, because the testing code did not detect any errors when the fusermount binary returned an error. This commit fixes it.	2016-01-23 19:10:43 +01:00
Alexander Neumann	8b7bf8691d	backend: Remove Get() ... This is the first commit that removes the (redundant) Get() method of the backend interface. Get(x, y) is equivalent to GetReader(x, y, 0, 0).	2016-01-23 13:13:05 +01:00
Alexander Neumann	1dd4c52a8b	Add comments, configure flag library	2016-01-17 18:48:05 +01:00
Alexander Neumann	1922a4272c	s3: fix usage ... Ignore error response for existing bucket, add more debug code.	2016-01-17 18:48:05 +01:00
Alexander Neumann	7d5f8214cf	use new backend open with config	2016-01-17 18:48:05 +01:00
Alexander Neumann	3ac1d0e4d1	add progress	2015-12-06 17:29:31 +01:00
Alexander Neumann	0e66a66bce	read packs concurrently	2015-12-06 17:09:06 +01:00
Alexander Neumann	a1440c819b	Make NoLock a global option	2015-11-13 12:33:59 +01:00
Alexander Neumann	6edb7e02d0	Allow restoring and listing without locking	2015-11-10 22:13:53 +01:00
Alexander Neumann	c79dcbd7c4	Allow checking a read-only repo with --no-lock	2015-11-10 21:41:22 +01:00
Alexander Neumann	acba82c8f7	Merge pull request #252 from restic/repack-blobs ... WIP: Repack blobs	2015-11-09 20:57:57 +01:00
Alexander Neumann	1f9aea9905	fix test on windows, reset read-only flag	2015-11-08 22:41:45 +01:00
Alexander Neumann	742d69bf4d	Add another test for optimizing unused blobs	2015-11-08 22:38:17 +01:00
Alexander Neumann	5776b8f01c	remove ConvertIndex	2015-11-08 22:27:13 +01:00
Alexander Neumann	2e6eee991d	Add test for optimize command with old indexes	2015-11-08 22:21:08 +01:00
Alexander Neumann	43e2c9837e	check: removing orphaned packs is handled in 'optimize'	2015-11-08 21:24:51 +01:00
Alexander Neumann	c4fc7b52ae	Add 'optimize' command that repacks blobs	2015-11-08 21:10:03 +01:00
Alexander Neumann	cd948b56ac	cmd_check: Don't display unused blobs by default	2015-11-08 20:46:52 +01:00
Alexander Neumann	ea41a1045f	Add integration test for error on readdirnames	2015-11-06 23:19:56 +01:00
Alexander Neumann	18d7f7f835	Read password from stdin if terminal is not a tty	2015-11-04 22:20:58 +01:00
Alexander Neumann	1fc0d78913	Refactor Index.Store() to take a PackedBlob	2015-11-02 19:07:03 +01:00
Alexander Neumann	50c2f2e87f	cmd_cat: allow dumping raw tree blobs	2015-11-02 17:36:04 +01:00
Alexander Neumann	fccde030d5	Refactor Index.Lookup() to return struct PackedBlob	2015-10-31 14:47:42 +01:00
Alexander Neumann	b5976474dd	backup: add debug output for excluded files/dirs	2015-10-27 22:34:30 +01:00
Alexander Neumann	00e05ae3c9	bugfix: close pack files after reading the header	2015-10-27 20:39:52 +01:00
Alexander Neumann	74cd134b54	rebuild index: remember already stored blobs	2015-10-25 22:34:22 +01:00
Alexander Neumann	734ae7fcb8	Add test for corner case ... It was observed that a restic repository still contained overlapping indexes after `rebuild-index` has been called. This is caused by instantly forgetting that blobs have already been saved once a full index has been written during index rebuilding. This commit adds a (failing) test that shows the behaviour.	2015-10-25 21:51:57 +01:00
Alexander Neumann	7b8e42a763	Silence rebuild-index tests	2015-10-25 21:51:46 +01:00
Alexander Neumann	efbce9f0fa	rebuild-index: handle not yet indexed packs	2015-10-25 18:07:51 +01:00
Alexander Neumann	88849c06a6	rebuild-index: Refactor a bit	2015-10-25 17:53:02 +01:00
Alexander Neumann	6aed9f268b	Add command rebuild-index	2015-10-25 17:24:52 +01:00
Alexander Neumann	96ecc26507	Let the checker return a list of hints along with errors	2015-10-25 16:26:50 +01:00
Alexander Neumann	650eab6a0e	Fix typo in dump usage	2015-10-25 13:19:35 +01:00
Alexander Neumann	7ab9915859	Fix 'dump' command	2015-10-12 22:42:31 +02:00
Alexander Neumann	86fcd170f6	Add and use MasterIndex	2015-10-12 22:34:12 +02:00
Alexander Neumann	de2c20be84	Dump individual indexes	2015-10-12 20:53:07 +02:00
Alexander Neumann	db85ab8aa0	Use the correct channel for sending errors	2015-10-11 19:13:45 +02:00
Chapuis Bertil	c765688779	find command integration tests	2015-08-28 19:31:05 +02:00
Chapuis Bertil	d4686ebcc5	Load the index and search subtree	2015-08-27 23:21:44 +02:00
Alexander Neumann	fcb769fa3b	Merge pull request #262 from restic/fix-tests ... Fix tests	2015-08-18 22:45:50 +02:00
Alexander Neumann	1a47ea4ab8	test helpers: add RemoveAll and ResetReadOnly ... This is mainly needed in Windows, where files and dirs cannot be removed unless they are writeable.	2015-08-18 21:40:40 +02:00
klauspost	a3570af500	Create separate disEntry.equals for Windows. ... Windows does not have UID/GID the same way as unix, so we don't attempt to compare them.	2015-08-17 11:10:12 +02:00
klauspost	d5dab39a4a	Disable FUSE test on Windows.	2015-08-17 11:02:04 +02:00
Klaus Post	4dc746dac2	Change repository type detection to first check if path is a directory that exists. ... The method of determining if a repository exists doesn't work on Windows, since the "url.Scheme" will contain the drive letter - "c" in "c:\backup", so as a first step we check if the URL can be opened as a file, and if so, we assume it is a 'local' type repository.	2015-08-14 15:39:16 +02:00
Klaus Post	fb3778abb6	Disable mount command on Windows. ... FUSE does not support Windows, so we disable it on Windows.	2015-08-14 15:33:11 +02:00
Alexander Neumann	dde0fd8421	Add option to use root as owner of files/dirs	2015-07-26 20:56:27 +02:00
Alexander Neumann	05e2afba0b	Merge pull request #249 from restic/reduce-fuse-memory-usage ... Reduce memory usage for fuse mount	2015-07-26 19:20:13 +02:00
Alexander Neumann	b85927576b	Address code review comments	2015-07-26 18:01:20 +02:00
Alexander Neumann	d1629e1e4e	fuse: move to top level	2015-07-26 17:20:26 +02:00
Alexander Neumann	bd746a0425	fuse: refactor and add tests for fuse.file	2015-07-26 16:43:42 +02:00
Alexander Neumann	9753c37e31	Remove dead code ... This removes dead code (functions that aren't called) detected with `deadcode`.	2015-07-26 14:56:34 +02:00
Alexander Neumann	55ddd5317d	Reduce memory usage for fuse mount ... This changes `repository.LoadBlob()` so that a destination buffer must be provided, which enables the fuse code to use a buffer from a `sync.Pool`. In addition, release the buffers when the file is closed. At the moment, the max memory usage is defined by the max file size that is read in one go (e.g. with `cat`). It could be further optimized by implementing a LRU caching scheme.	2015-07-26 14:25:59 +02:00
Alexander Neumann	054256468b	Fix message display for parent snapshot id	2015-07-25 18:10:23 +02:00
Alexander Neumann	5cdcc99eba	Use array instead of hash for backend.ID ... Since backend.ID is always a slice of constant length, use an array instead of a slice. Mostly, arrays behave as slices, except that an array cannot be nil, so use `*backend.ID` insteaf of `backend.ID` in places where the absence of an ID is possible (e.g. for the Subtree of a Node, which may not present when the node is a file node). This change allows to directly use backend.ID as the the key for a map, so that arbitrary data structures (e.g. a Set implemented as a map[backend.ID]struct{}) can easily be formed.	2015-07-25 18:01:57 +02:00
Alexander Neumann	deaca157fe	restore: ignore errors ... Also add a test for restoring a file that is owned by root. Closes #244.	2015-07-25 12:59:59 +02:00
Alexander Neumann	cc34401152	Record exclude patterns in snapshot ... This adds the exclude patterns used to create a backup in the snapshot, so we can later compute statistics (like git does) on the data structure, e.g. added/removed files etc. For that, we need the exclude pattern.	2015-07-22 22:43:32 +02:00
Alexander Neumann	9911d46996	fuse: rename child/children -> node/items	2015-07-21 21:34:59 +02:00
Alexander Neumann	3f4b5b8d48	fuse/mount: display symlinks properly	2015-07-21 21:25:05 +02:00
Alexander Neumann	1ac72b8813	Fix style issue	2015-07-21 21:24:06 +02:00
Alexander Neumann	99dae57b4f	fuse: use node.Type instead of node.Mode	2015-07-21 21:23:40 +02:00
Alexander Neumann	aeb5a694d3	Merge pull request #238 from restic/fix-checks-on-bsd ... Refactor skipping symlink timestamp checks on *bsd and darwin	2015-07-21 19:47:33 +02:00
Alexander Neumann	258b6a77ee	Refactor skipping symlink ModTime checks, add OpenBSD	2015-07-20 21:29:21 +02:00
Alexander Neumann	bd3ce5d4a3	Fix tests on freebsd	2015-07-20 21:08:29 +02:00
Alexander Neumann	1da89253cf	Add include filter for restore ... Include and exclude filter are mutually exclusive.	2015-07-20 19:21:01 +02:00
Alexander Neumann	c0337a2675	Add exclude filter to restorer and 'restore' command	2015-07-20 19:21:01 +02:00
Alexander Neumann	7fd52f9f57	Add exclude filter to archiver and 'backup' command	2015-07-20 19:21:01 +02:00
Matthieu Rakotojaona	3767eb2675	Unmount and remove directory for mount in tests	2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona	e44716381c	Unmount when closing application from cli	2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona	eadfcd3f9e	Add waitForMount for OSX	2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona	fe6f1c01f3	Make inodeFromBackendId more explicit	2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona	a4d122e5ae	Cleanup mount after test	2015-07-19 23:04:18 +02:00