Alexander Neumann
9a02f17cc2
archiver: Add tests for Save() for fs.Reader
2018-05-20 16:11:36 +02:00
Steve Kriss
b358dd369b
S3: rearrange credentials chain to be standard
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-16 16:49:33 -07:00
Steve Kriss
d67b9a32c6
S3: add file credentials to chain
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-16 16:35:14 -07:00
Alexander Neumann
347a645450
Fix double error message
2018-05-15 11:03:33 +02:00
Alexander Neumann
9f5565b0fc
Merge pull request #1735 from mholt/forget-max-age
...
forget: Add --max-age policy to set hard cutoff for removing snapshots
2018-05-14 21:18:06 +02:00
Alexander Neumann
060d8b57e0
Restructure TestApplyPolicy
2018-05-13 12:48:42 +02:00
Alexander Neumann
cc627e832b
Add custom Duration type
2018-05-13 12:48:42 +02:00
Alexander Neumann
5a0f0e3faa
Add support for keeping a range of snapshots
2018-05-13 12:48:42 +02:00
Matthew Holt
b52f2aa9a4
forget: Add policy to keep snapshots before a date
2018-05-13 12:48:10 +02:00
Alexander Neumann
60ea2435be
Improve error message for readdir/readdirnames
...
As mentioned in the forum[1], restic does not include the dir name when
readdir/readdirnames fails.
[1] https://forum.restic.net/t/readdirnames-readdirent-no-such-file-or-directory/653
2018-05-13 10:34:50 +02:00
Alexander Neumann
159badf5ba
Merge pull request #1778 from restic/fix-1771
...
archiver: Improve error handling
2018-05-13 00:13:54 +02:00
Alexander Neumann
e43c9202a6
archiver: Make sure backend error is passed up
2018-05-12 23:55:59 +02:00
Alexander Neumann
c5e75d1c98
archiver: Add test for early abort on unhandled error
2018-05-12 23:55:59 +02:00
Alexander Neumann
526956af35
archiver: Read files/dirs in order
2018-05-12 23:55:54 +02:00
Alexander Neumann
256104111d
archiver: Clarify names
2018-05-12 23:55:54 +02:00
Alexander Neumann
21c83b1725
archiver: Add high-level documentation
2018-05-12 23:55:54 +02:00
Alexander Neumann
581c62ee72
archiver: Improve error handling
...
This commit changes how the worker goroutines for saving e.g. blobs
interact. Before, it was possible to get stuck sending an instruction to
archive a file or dir when no worker goroutines were available any more.
This commit introduces a `done` channel for each of the worker pools,
which is set to the channel returned by `tomb.Dying()`, so it is closed
when the first worker returned an error.
2018-05-12 23:55:54 +02:00
Gaige Lama
a63989afcd
Ran gofmt on backend.go
...
I ran gofmt on backend.go, this appears to valid edit.
2018-05-11 11:07:16 -04:00
Alexander Neumann
ca4af43c03
archiver: Return low-level errors
...
This commit changes the archiver so that low-level errors saving data to
the repo are returned to the caller (instead of being handled by the
error callback function). This correctly bubbles up errors like a full
temp file system and makes restic abort early and makes all other worker
goroutines exit.
2018-05-10 21:30:09 +02:00
Alexander Neumann
1f2463f42e
archiver: Return correct error
2018-05-10 20:48:00 +02:00
Alexander Neumann
157c854d04
backup: Remove errored files from status display
2018-05-10 20:48:00 +02:00
Alexander Neumann
b2208bb9c2
Rework termstatus
...
This now keeps the cursor at the first column of the first status line
so that messages printed to stdout or stderr by some other part of the
progarm will still be visible. The message will overwrite the status
lines, but those are easily reprinted on the next status update.
2018-05-07 21:40:07 +02:00
Alexander Neumann
4c25495d68
backup: Hide percent until total size is known
2018-05-06 20:20:25 +02:00
Alexander Neumann
fd77646f8b
Fix panic for nil-pointer dereference
...
Closes #1755
2018-05-03 20:49:30 +02:00
Alexander Neumann
fca4fe4459
backup: Disable status output for --quiet
2018-05-02 21:24:18 +02:00
Alexander Neumann
26757ae2e5
Merge pull request #1750 from restic/archiver-fix-small-things
...
backup: Tune user interface a bit
2018-05-01 23:15:54 +02:00
Alexander Neumann
2218ecd049
archiver: Use lstat before open/fstat
...
The previous code tried to be as efficient as possible and only do a
single open() on an item to save, and then fstat() on the fd to find out
what the item is (file, dir, other). For normal files, it would then
start reading the data without opening the file again, so it could not
be exchanged for e.g. a symlink.
This behavior starts the watchdog on my machine when /dev is saved
with restic, and after a few seconds, the machine reboots.
This commit reverts the behavior to the strategy the old archiver code
used: run lstat(), then decide what to do. For normal files, open the
file and then run fstat() on the fd to verify it's still a normal file,
then start reading the data.
The downside is that for normal files we now do two stat() calls
(lstat+fstat) instead of only one. On the upside, this does not start
the watchdog. :)
2018-05-01 23:05:50 +02:00
Alexander Neumann
d0974c155d
backup: Tune user interface a bit
...
Make non-verbose mode a bit more verbose.
2018-05-01 22:18:32 +02:00
Alexander Neumann
8026e6fdfb
Merge pull request #1749 from restic/add-cache-cmd
...
Add 'cache' command to list and cleanup cache dirs
2018-05-01 21:25:52 +02:00
Alexander Neumann
f928aeec34
Add 'cache' command
2018-05-01 16:27:44 +02:00
Alexander Neumann
eb6650b201
Add textfile
2018-05-01 14:38:59 +02:00
Alexander Neumann
bc68d55e94
fs: Add TestTempFile
2018-05-01 14:38:41 +02:00
Alexander Neumann
ecbbd851a1
Merge pull request #1744 from restic/fix-parent-detect
...
backup: Clean target paths before finding parent
2018-05-01 11:13:19 +02:00
Bryce Chidester
e9f1721678
http backend: Parse the correct argument when loading --tls-client-cert
...
Previously, the function read from ARGV[1] (hardcoded) rather than the
value passed to it, the command-line argument as it exists in globalOptions.
Resolves #1745
2018-04-30 15:21:09 -07:00
Alexander Neumann
64d98945a6
backup: Clean target paths before finding parent
...
This resolves an issue described in the forum where restic could not
find a parent snapshot if the target path ends with a slash:
https://forum.restic.net/t/new-archiver-code-please-test/623/23
2018-04-30 22:03:11 +02:00
Alexander Neumann
c83c03ed63
archiver: Fix blocking on pipes
2018-04-30 15:34:58 +02:00
Alexander Neumann
19b9c881ca
fs: Add O_NONBLOCK
2018-04-30 15:34:58 +02:00
Alexander Neumann
4e34325035
archiver: Process dirs concurrently
2018-04-30 15:13:28 +02:00
Alexander Neumann
78bd591c7c
archiver: Improve buffer pool
2018-04-30 15:13:28 +02:00
Alexander Neumann
39ac12f6ea
archiver: Correct comment
2018-04-30 14:19:07 +02:00
Alexander Neumann
400730afca
archiver: Improve memory usage, tune buffer pool
2018-04-30 14:19:07 +02:00
Alexander Neumann
d80e108b03
backup: Clear status lines on finish
2018-04-30 14:19:07 +02:00
Alexander Neumann
846c2b6869
backup: Fix ETA calculation for >= 100%
2018-04-30 14:19:07 +02:00
Alexander Neumann
16c314ab7f
termstatus: Don't print status if in background
2018-04-28 22:08:11 +02:00
Alexander Neumann
0e78ac92d8
Use new archiver code for backup
2018-04-28 22:08:11 +02:00
Alexander Neumann
1af96fc6dd
Add termstatus
2018-04-27 21:42:15 +02:00
Alexander Neumann
38926d8576
Use new archiver code in tests
2018-04-25 14:42:45 +02:00
Alexander Neumann
f279731168
Add new archiver code
2018-04-25 14:42:45 +02:00
Alexander Neumann
76b616451f
Remove unneeded code
2018-04-23 21:40:33 +02:00
Alexander Neumann
fd12a3af20
Remove old archiver code
2018-04-23 21:40:33 +02:00
Alexander Neumann
a56b8fad87
repository: Improve buffer pooling
2018-04-22 11:37:05 +02:00
Alexander Neumann
b6f98bdb02
node: Fill minimal info
2018-04-22 11:37:05 +02:00
Alexander Neumann
c4b2486b7c
fs: Add interface and FS implementations
...
This adds two implementations of the new `FS` interface: One for the local
file system (`Local`) and one for a single file read from an
`io.Reader` (`Reader`).
2018-04-22 11:37:05 +02:00
Alexander Neumann
83ca08245b
checker: Check metadata size and blob sizes
2018-04-22 11:37:05 +02:00
Alexander Neumann
6a7c23d2ae
tree: Add convenience functions
2018-04-22 11:37:05 +02:00
Alexander Neumann
cc847a3d6d
tree: Improve error for pre-existing node
2018-04-22 11:37:05 +02:00
Alexander Neumann
baebf45e2e
FindLatestSnapshot: Use absolute paths
2018-04-22 11:37:05 +02:00
Alexander Neumann
fa4f438bc1
snapshot: Do not modify slice of paths
2018-04-22 11:37:05 +02:00
Alexander Neumann
4e0b2a8e3a
snapshot: correct error handling for filepath.Abs
2018-04-22 11:37:05 +02:00
Alexander Neumann
0532f08048
Add test.Helper, also works with Go 1.8
2018-04-22 11:37:05 +02:00
Alexander Neumann
a472868e06
fs: Add TestChdir()
2018-04-22 11:37:05 +02:00
Alexander Neumann
e4fdc5eb76
fs: Add IsRegularFile()
2018-04-22 11:37:05 +02:00
Alexander Neumann
09bd924710
Do not restore sockets, correct error handling
...
Closes #1730
2018-04-20 13:53:11 +02:00
Alexander Neumann
577faa7570
local/sftp: Handling non-existing dirs in List()
2018-04-10 21:35:30 +02:00
Alexander Neumann
1c1fede399
Improve error message for orphaned pack files
2018-04-07 10:07:54 +02:00
Eri Bastos
901e1b129c
Fixed issue #1608 - Use --time argument properly
...
Backups via stdin will now handle --time argument and pass it down as
expected
2018-04-03 14:40:42 -03:00
Alexander Neumann
e68a7fea8a
check: Allow filling the cache during check
...
Closes #1665
2018-04-01 13:59:27 +02:00
Alexander Neumann
2e7ec717c1
repository: Move cache preparation into function
2018-04-01 13:59:27 +02:00
Alexander Neumann
3f48e0e0f4
Add extra options to rclone
...
For details see https://github.com/restic/restic/pull/1657#issuecomment-377707486
2018-04-01 10:34:30 +02:00
Alexander Neumann
86f4b03730
Remove unneeded byte counters
2018-04-01 10:18:38 +02:00
Alexander Neumann
c43c94776b
rclone: Make concurrent connections configurable
2018-04-01 10:18:38 +02:00
Alexander Neumann
0b776e63e7
backend/rclone: Request random file name
...
When `/` is requested, rclone returns the list of all files in the
remote, which is not what we want (and it can take quite some time).
2018-04-01 10:18:38 +02:00
Alexander Neumann
737d93860a
Extend first timeout to 60 seconds.
2018-04-01 10:18:38 +02:00
Alexander Neumann
17312d3a98
backend/rest: Ensure base URL ends with slash
...
This makes it easier for rclone.
2018-04-01 10:18:38 +02:00
Alexander Neumann
4d5c7a8749
backend/rclone: Make sure rclone terminates
2018-04-01 10:18:38 +02:00
Alexander Neumann
fc0295016a
Address code review comments
2018-04-01 10:18:38 +02:00
Alexander Neumann
99b62c11b8
backend/rclone: Stop rclone in case of errors
2018-04-01 10:18:38 +02:00
Alexander Neumann
6d9a029e09
backend/rclone: Prefix all error messages
2018-04-01 10:18:38 +02:00
Alexander Neumann
065fe1e54f
backend/rclone: Skip test if binary is unavailable
2018-04-01 10:16:31 +02:00
Alexander Neumann
4dc0f24b38
backend/tests: Drain reader before returning error
2018-04-01 10:16:31 +02:00
Alexander Neumann
fe99340e40
Add rclone backend
2018-04-01 10:16:31 +02:00
Alexander Neumann
e377759c81
rest: Export Backend struct
2018-04-01 10:16:31 +02:00
Alexander Neumann
cabbbd2b14
backend/rest: Export Content-Types
2018-04-01 10:16:31 +02:00
Alexander Neumann
cf4cf94418
Move backend/sftp.StartForeground to backend/
2018-04-01 10:16:31 +02:00
Alexander Neumann
34f27edc03
Refactor SplitShellStrings
2018-04-01 10:16:31 +02:00
Alexander Neumann
345b6c4694
Move backend/sftp.SplitShellArgs to backend/
2018-04-01 10:16:31 +02:00
Alexander Neumann
594256bfa4
Merge pull request #1693 from restic/update-deps
...
Update dependencies
2018-03-30 17:19:08 +02:00
Alexander Neumann
673f0bbd6c
Update vendored library github.com/cenkalti/backoff
2018-03-30 11:45:27 +02:00
Alexander Neumann
d3f9c8b362
forget: Print policy
2018-03-30 10:24:26 +02:00
Alexander Neumann
e5c929b793
Fix rest-server tests
...
Since today, the rest-server needs to be explicitly told (via
`--no-auth`) that authentication is not necessary.
2018-03-24 18:06:21 +01:00
Alexander Neumann
8206f85d2e
Merge pull request #1552 from lawrencejones/use-auto-auth
...
Automatically load Google auth
2018-03-18 19:53:30 +01:00
Alexander Neumann
0279fd7212
Merge pull request #1669 from restic/make-tests-faster
...
Reduce test runtime
2018-03-13 19:50:34 +01:00
Alexander Neumann
b9ada91054
Reduce data for TestCreateSnapshot
2018-03-11 21:42:39 +01:00
Alexander Neumann
c67a8452f7
Disable polynomial check for chunker for tests
2018-03-11 20:59:40 +01:00
Alexander Neumann
5253ef218c
Remove TestParallelSaveWithDuplication
2018-03-11 19:49:48 +01:00
Alexander Neumann
0923976909
Remove TestArchiverDuplication
2018-03-11 19:44:25 +01:00
Lawrence Jones
0dfdc11ed9
Automatically load Google auth
...
This change removes the hardcoded Google auth mechanism for the GCS
backend, instead using Google's provided client library to discover and
generate credential material.
Google recommend that client libraries use their common auth mechanism
in order to authorise requests against Google services. Doing so means
you automatically support various types of authentication, from the
standard GOOGLE_APPLICATION_CREDENTIALS environment variable to making
use of Google's metadata API if running within Google Container Engine.
2018-03-11 17:11:25 +00:00
Alexander Neumann
e085713b35
Return the first password/key which works
...
Closes #1663
2018-03-11 14:12:21 +01:00
Alexander Neumann
e77d8c64a7
Merge pull request #1661 from restic/fix-rest-content-length
...
rest: Really set Content-Length HTTP header
2018-03-10 20:34:30 +01:00
Alexander Neumann
b3e1089cf9
Return error message for config decryption failure
...
See #1663
2018-03-09 21:05:35 +01:00
