Matthew Dawson
539599d1f1
repository/index: Optimize index.Has()
...
When backing up several million files (>14M tested here) with few changes,
a large amount of time is spent failing to find an id in an index and creating
an error to signify this. Since this is checked using the Has method,
which doesn't use this error, this time creating the error is wasted.
Instead, directly check if the given id and type are present in the index.
This also avoids reporting all the packs containing this blob, further
reducing cpu usage.
2018-01-08 21:46:17 +01:00
Alexander Neumann
d77a326bb0
Add benchmark for Index.Has()
2018-01-08 21:10:30 +01:00
Alexander Neumann
b8620429e2
restorer: Set directory mode as last step
...
Closes #1512
2018-01-07 15:13:24 +01:00
Alexander Neumann
aab414b368
Add test case for #1512
2018-01-07 14:50:47 +01:00
Alexander Neumann
5f43e17918
Merge pull request #1535 from ifedorenko/check-retry
...
fixed restic-check does not retry backend.Test failures
2018-01-07 09:43:49 +01:00
Igor Fedorenko
8c550ca011
fixed restic-check does not retry backend.Test failures
...
added missing RetryBackend.Test implementation
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-06 23:22:35 -05:00
Alexander Neumann
032621289b
errors: Add "Fatal" prefix for Fatalf()
2018-01-06 23:27:47 +01:00
Alexander Neumann
b45fc89512
local/sftp: Create repo dirs on demand in Save()
2018-01-05 17:51:09 +01:00
Alexander Neumann
aabc0ccaa7
fs: Make HasPathPrefix work with relative paths
2018-01-05 17:38:52 +01:00
Alexander Neumann
18ecd9df30
Improve limiting HTTP based backends
2017-12-29 12:43:49 +01:00
Alexander Neumann
3f7d85360a
fuse: Only reload list of snapshots once per minute
2017-12-28 13:18:27 +01:00
Alexander Neumann
6c2b2a58ad
backend: Retry deletes
2017-12-22 22:41:28 +01:00
Alexander Neumann
bc04ce8e6b
Merge pull request #1482 from restic/fix-1478
...
backend: Only return top-level files for most dirs
2017-12-20 18:06:47 +01:00
Alexander Neumann
7d8765a937
backend: Only return top-level files for most dirs
...
Fixes #1478
2017-12-14 19:14:16 +01:00
Alexander Neumann
81a04656c5
fuse: Cleanup node name
2017-12-13 20:21:18 +01:00
Harshavardhana
27ccea6371
Since upgrade to minio-go 4.0 remove workaround
...
We added previously a code to fix the issue of chaining
credentials, we do not need this anymore since the
upstream minio-go already has this relevant change.
2017-12-09 02:01:42 -08:00
Alexander Neumann
8b3b7bc5ef
s3: Use context
2017-12-08 22:04:55 +01:00
Alexander Neumann
934ae1b559
Update to minio-go 4
2017-12-08 21:52:50 +01:00
Alexander Neumann
3a5e040b7e
Merge pull request #1439 from armhold/propagate-context
...
replace ad-hoc context.TODO() with gopts.ctx
2017-12-08 20:27:36 +01:00
George Armhold
1695c8ed55
use global context for check, debug, dump, find, forget, init, key,
...
list, mount, tag, unlock commands
gh-1434
2017-12-06 07:02:55 -05:00
George Armhold
0dc31c03e1
remove check for context.Canceled
...
gh-1434
2017-12-06 05:38:29 -05:00
Alexander Neumann
82ded35706
node: Correctly compare times
2017-12-03 18:33:35 +01:00
Alexander Neumann
88607fc625
Correctly format all node types
2017-12-03 17:38:55 +01:00
Alexander Neumann
7092af6329
fuse: Handle sockets/fifos/devs correctly
...
Closes #1463
2017-12-03 17:25:00 +01:00
George Armhold
20d78ab0d9
add context to repo.Flush() call in test code
...
gh-1434
2017-12-03 07:32:50 -05:00
George Armhold
be24237063
make retry code context-aware.
...
detect cancellation in backend, so that retry code does not keep trying
once user has hit ^c
gh-1434
2017-12-03 07:22:14 -05:00
George Armhold
d886cb5c27
replace ad-hoc context.TODO() with gopts.ctx, so that cancellation
...
can properly trickle down from cmd_*.
gh-1434
2017-12-03 07:22:14 -05:00
Alexander Neumann
0b44c629f2
retry: Remove file after failed save
2017-11-30 22:05:14 +01:00
Alexander Neumann
410efe0694
Merge pull request #1454 from restic/fix-cache-dir
...
Fix cache dir detection
2017-11-29 18:43:07 +01:00
Alexander Neumann
eddb8549ef
backup: By default, do not save the access time
...
This can be re-enabled with `--with-atime`.
2017-11-28 21:31:35 +01:00
Alexander Neumann
dc38265b54
Merge pull request #1436 from restic/remove-old-cache
...
Remove old cache directories
2017-11-27 21:37:05 +01:00
Alexander Neumann
0f5e38609f
restore: Fix restore to relative path
2017-11-26 18:36:48 +01:00
Alexander Neumann
27d29b9853
restorer: Make sure node names are clean
2017-11-26 15:28:45 +01:00
Alexander Neumann
8a171731ba
restorer: Add tests for invalid node names
2017-11-26 15:28:37 +01:00
Alexander Neumann
134abbd82b
rest: Use client for creating the repository
...
Before, creating a new repo via REST would use the defaut HTTP client,
which is not a problem unless the server uses HTTPS and a TLS
certificate which isn't signed by a CA in the system's CA store. In this
case, all commands work except the 'init' command, which fails with a
message like "invalid certificate".
2017-11-25 20:56:40 +01:00
Alexander Neumann
1ebf0e8de8
Merge pull request #1437 from restic/fix-1292
...
s3: Document and remove default prefix
2017-11-25 11:34:26 +01:00
Alexander Neumann
47b326b7b5
Merge pull request #1423 from harshavardhana/creds
...
Fix chaining of credentials for minio-go
2017-11-24 21:57:52 +01:00
Alexander Neumann
f79698dcdd
Merge pull request #1410 from armhold/deadlock2
...
unify behavior for max http connections across backends
2017-11-24 21:32:56 +01:00
Alexander Neumann
f4bab789b8
cache: Simplify cache dir creation
2017-11-24 20:53:26 +01:00
Alexander Neumann
fa893ee477
cache: Add detection code for old cache dirs
2017-11-24 20:53:26 +01:00
Alexander Neumann
262b0cd9d4
s3: Remove default prefix "/restic"
2017-11-21 21:33:09 +01:00
Alexander Neumann
e83ec17e95
s3: Correct comment
2017-11-20 22:21:39 +01:00
Alexander Neumann
ea593fca1b
cache: Correctly return dir for Windows/darwin
2017-11-20 06:11:18 +01:00
Alexander Neumann
fe1f151ae1
cache: Return error during default dir detection
2017-11-20 06:10:42 +01:00
Harshavardhana
41c8c946ba
Fix chaining of credentials for minio-go
...
chaining failed because chaining provider
was only looking for subsequent credentials
provider after an error. Writer a new
chaining provider which proceeds to fetch
new credentials also under situations where
providers do not return but instead return
no keys at all.
Fixes https://github.com/restic/restic/issues/1422
2017-11-18 02:51:12 -08:00
George Armhold
0268d0e7d6
swift backend: limit http concurrency in Save(), Stat(), Test(), Remove(),
...
List().
move comment regarding problematic List() backend api (it's s3's ListObjects
that has a problem, NOT swift's ObjectsWalk).
As per discussion in PR #1399 .
2017-11-02 18:29:32 -04:00
George Armhold
8515d093e0
swift backend: fix premature release of semaphore in Load() & document
...
concurrency issue in List().
refactor wrapReader from b2 -> semaphore so it can be used elsewhere.
As per discussion in PR #1399 .
2017-11-02 12:38:17 -04:00
George Armhold
99ac0da4bc
s3 backend: limit http concurrency in Save(), Stat(), Test(), Remove()
...
NB: List() is NOT currently limited, as it would cause deadlock due to
be.client.ListObjects() implementation.
as per discussion in PR #1399
2017-11-01 09:40:54 -04:00
Alexander Neumann
6d46824fb0
Pass in a nil buffer to Open()
2017-11-01 10:30:40 +01:00
Alexander Neumann
bb435b39d9
crypto: Rework Seal/Open to use sliceForAppend
2017-11-01 10:30:40 +01:00
