Christian Kemper
32c2cafa89
Simplify creation of the Config by moving it to a separate function. Simplify the parsing logic
...
by sharing the handling of s3: and s3://
2016-02-14 09:18:22 -08:00
Christian Kemper
74608531c7
strip off the trailing slash for the object prefix
2016-02-14 07:16:50 -08:00
Christian Kemper
48f85fbb09
replaced if-else chain with switch
2016-02-14 07:01:14 -08:00
Christian Kemper
535dfaf097
address first round of review comments
2016-02-14 06:40:15 -08:00
Christian Kemper
8f5ff379b7
Introduced a configurable object path prefix for s3 repositories.
...
Prepends the object path prefix to all s3 paths and allows to have multiple independent
restic backup repositories in a single s3 bucket.
Removed the hardcoded "restic" prefix from s3 paths.
Use "restic" as the default object path prefix for s3 if no other prefix gets specified.
This will retain backward compatibility with existing s3 repository configurations.
Simplified the parse flow to have a single point where we parse the bucket name and the prefix within the bucket.
Added tests for s3 object path prefix and the new default prefix to config_test and location_test.
2016-02-14 06:05:38 -08:00
Alexander Neumann
eccbcb73a1
Merge pull request #425 from restic/fix-loadall
...
backend.LoadAll: return nil on expected error
2016-02-08 00:37:04 +01:00
Alexander Neumann
e781e1cf1d
Merge pull request #424 from restic/fix-backup
...
Fix backup of root directory
2016-02-08 00:06:19 +01:00
Alexander Neumann
e9a21c1dc6
backend.LoadAll: return nil on expected error
...
The current code returns io.ErrUnexpectedEOF, but it is the normal,
expected behaviour of the function LoadAll() to load until the item is
completely loaded. Therefore, the io.ErrUnexpectedEOF is not returned to
the caller.
2016-02-07 23:48:54 +01:00
Alexander Neumann
a37ed45534
Add test for LoadAll with too large buffer
...
LoadAll() should not pass on io.ErrUnexpectedEOF, since the occurrence
of this error is normal.
2016-02-07 23:48:03 +01:00
Alexander Neumann
26484d0c7b
pipe: Ignore excluded directories
...
When top-level directories are ignored, they were still added to the
top-level Dir{} object. This commit ignores them.
2016-02-07 23:23:16 +01:00
Alexander Neumann
68db75b4e3
pipe/archiver: Add more debug messages
2016-02-07 23:22:06 +01:00
Alexander Neumann
9048eb676b
pipe: join replaced paths with original path
...
When saving `/`, it was replaced with the contents, but without the
proper path. So `/` was replaced by [`boot`, `bin`, `home`, ...], but
without prefixing the entry name with the proper path.
2016-02-07 22:18:37 +01:00
Alexander Neumann
6a5b022939
archiver: Add error reporting for directories
...
When an error occurred while walking a directory, this error wasn't
reported to the user before.
2016-02-07 22:18:00 +01:00
Alexander Neumann
57a24b2cdf
Merge pull request #421 from restic/fix-fuse-with-special-names
...
fuse: Replace special node names with their content
2016-02-07 20:43:29 +01:00
Alexander Neumann
da47389483
Merge pull request #420 from restic/archiver-unique-paths
...
archiver: deduplicate list of paths to save
2016-02-07 20:31:18 +01:00
Alexander Neumann
4c329110c5
Merge pull request #419 from restic/fix-backup-root
...
Fix backup of "/"
2016-02-07 20:31:08 +01:00
Alexander Neumann
46fbae0d71
Merge pull request #418 from benmur/show-pack-name-on-rebuild-index-error
...
Handle pack loading errors in rebuild-index
2016-02-07 19:59:01 +01:00
Alexander Neumann
1835e988cf
fuse: Replace special node names with their content
...
This is the companion fix for #419 and allows mounting repositories with
special directory names directly below the snapshot.
Closes #403
2016-02-07 19:51:29 +01:00
Alexander Neumann
537347d9b5
archiver: deduplicate list of paths to save
2016-02-07 19:35:35 +01:00
Alexander Neumann
811dbfa52d
Make TestWalkerPath absolute before walking
2016-02-07 19:34:02 +01:00
Rached Ben Mustapha
ba35ca522a
Handle pack loading errors in rebuild-index
...
Errors returned from backend.LoadAll() were not handled, leading to
these fatal errors from the unpacker trying to read the size from the end of
an empty buffer:
`seeking to read header length failed: bytes.Reader.Seek: negative position`
This change takes care of returning on error, as well as showing which pack
failed to load and validating pack integrity.
2016-02-07 18:30:47 +00:00
Alexander Neumann
c6a1f2e2f3
pipe: Handle special paths gracefully
...
This fixes handling special paths "." and "/". When such a path name is
found, it is replaced by the contents of the path before walking.
2016-02-07 19:30:00 +01:00
Alexander Neumann
604c27f001
Test pipe walker for invalid paths
...
It was discovered that when restic is instructed to save `/`, the tree
structures in the repository contain an invalid node.
When saving the dir `/home/user`, the following structure is created:
snapshot
-> tree
nodes: ["user"]
[...]
When the root directory `/` is saved, the structure is as follows:
snapshot
-> tree
nodes: ["/"]
[...]
This behavior is caused by the walker in pipe.go sending a node with the
name "." to the archiver, so this commit adds a test for invalid node
names.
2016-02-07 19:29:44 +01:00
Alexander Neumann
0535490618
Merge pull request #413 from restic/reduce-travis-matrix
...
Reduce jobs run on Travis
2016-02-06 14:10:54 +01:00
Alexander Neumann
9c0fc4930b
Merge pull request #412 from restic/activate-hound
...
Enable HoundCI checking for Go
2016-02-06 13:37:13 +01:00
Alexander Neumann
a45f2cb205
Reduce jobs run on Travis
...
Skip building and testing restic with Go 1.3, 1.4 and 1.6 on osx.
2016-02-06 13:04:49 +01:00
Alexander Neumann
3fd5b5975a
Merge pull request #411 from restic/fix-appveyor-sourceforge
...
Always use NetCologne SourceForge mirror
2016-02-06 13:01:55 +01:00
Alexander Neumann
9175f0b6af
Enable HoundCI checking for Go
2016-02-05 21:15:46 +01:00
Alexander Neumann
1160d03279
Always use NetCologne SourceForge mirror
...
The one automatically selected by the SourceForge CDN fails currently:
appveyor DownloadFile http://downloads.sourceforge.net/project/gnuwin32/tar/1.13-1/tar-1.13-1-bin.zip -FileName tar.zip
Error downloading file: Unable to connect to the remote server Command exited with code 2
2016-02-05 21:13:14 +01:00
Alexander Neumann
789e0df49e
Merge pull request #408 from benmur/validate-pack-checksums
...
checker: Validate pack checksums before unpacking
2016-02-05 19:45:21 +01:00
Rached Ben Mustapha
83bbf21f1a
checker: Validate pack checksums before unpacking
...
This avoids reading a possibly invalid size at the end of a corrupted pack
2016-02-04 22:55:39 +01:00
Alexander Neumann
7a8054d678
Add link to record of talk at C4 Cologne
2016-02-04 19:56:41 +01:00
Alexander Neumann
c0bbb7254d
Merge pull request #406 from restic/fix-405
...
Test and fix for #405
2016-02-02 20:53:52 +01:00
Alexander Neumann
4f1f03cdb9
Move testing for known blobs to Archiver
...
This removes the list of in-flight blobs from the master index and
instead keeps a list of "known" blobs in the Archiver. "known" here
means: either already processed, or included in an index. This property
is tested atomically, when the blob is not in the list of "known" blobs,
it is added to the list and the caller is responsible to make this
happen (i.e. save the blob).
2016-02-01 23:50:56 +01:00
Alexander Neumann
382c766983
Move test for #405 : Test Archiver instead of Repo
2016-02-01 23:50:41 +01:00
Alexander Neumann
f5f6e9cf37
Add test to reproduce #405
2016-02-01 23:35:01 +01:00
Alexander Neumann
cf88b33383
Merge pull request #398 from restic/update-minio-go
...
Update minio-library
2016-01-29 13:14:37 +01:00
Alexander Neumann
57615edd3a
Merge pull request #400 from restic/update-go-appveyor
...
Update Go version for CI tests
2016-01-29 13:13:42 +01:00
Alexander Neumann
49cb88d158
Add note about lightning talk at FOSDEM
2016-01-28 22:40:52 +01:00
Alexander Neumann
1464d84cf5
Update minio-go
...
This should work with Go 1.3/1.4 again
2016-01-28 22:38:29 +01:00
Alexander Neumann
74ce027924
Merge pull request #397 from restic/remove-readcloser
...
Remove backend.ReadCloser, this is not used any more
2016-01-28 22:31:30 +01:00
Alexander Neumann
39f698886a
Update Travis Go version
2016-01-28 22:28:17 +01:00
Alexander Neumann
9f8f2bc874
Update Go version for appveyor
2016-01-28 00:00:28 +01:00
Alexander Neumann
f8daadc5ef
Update minio-library
...
This addresses #388
2016-01-27 23:23:47 +01:00
Alexander Neumann
f2371db2a9
repository: remove decryptReadCloser
2016-01-27 22:47:09 +01:00
Alexander Neumann
7d1775e000
Remove backend.ReadCloser
2016-01-27 22:35:18 +01:00
Alexander Neumann
ce4a7f16ca
Merge pull request #395 from restic/rework-backend-interface
...
WIP: Rework backend interface
2016-01-27 22:11:20 +01:00
Alexander Neumann
322eca86bc
mem backend: remove unused code
2016-01-27 21:33:48 +01:00
Alexander Neumann
3d06e6083a
CI: download minio for the correct os and arch
2016-01-26 23:52:39 +01:00
Alexander Neumann
b64006221c
CI: Download minio server, do not compile latest master
2016-01-26 23:50:27 +01:00