2
2
mirror of https://github.com/octoleo/restic.git synced 2024-11-27 07:16:40 +00:00
Commit Graph

60 Commits

Author SHA1 Message Date
Alexander Neumann
1fc0d78913 Refactor Index.Store() to take a PackedBlob 2015-11-02 19:07:03 +01:00
Alexander Neumann
f3f84b1544 Add ID handling for index 2015-11-02 18:52:13 +01:00
Alexander Neumann
60a34087c9 Move LoadIndexWithDecoder to index.go 2015-11-02 18:52:13 +01:00
Alexander Neumann
89a77ab2f9 Add Index.ListPack() 2015-11-02 18:52:13 +01:00
Alexander Neumann
fccde030d5 Refactor Index.Lookup() to return struct PackedBlob 2015-10-31 14:47:42 +01:00
Alexander Neumann
7711fcda69 use new index format for repository tests 2015-10-27 20:51:55 +01:00
Alexander Neumann
7717ea5cca Add benchmark for LoadJSONPack 2015-10-27 20:51:55 +01:00
Alexander Neumann
88849c06a6 rebuild-index: Refactor a bit 2015-10-25 17:53:02 +01:00
Alexander Neumann
9074c923ea index: add AddToSupersedes() 2015-10-25 17:06:56 +01:00
Alexander Neumann
461d54e43c Refactor repository.SaveIndex() 2015-10-25 17:05:54 +01:00
Alexander Neumann
04614c7527 Add test for packs in duplicate indexes 2015-10-25 15:35:33 +01:00
Alexander Neumann
f7ff5b766c Mark written indexes as finalized 2015-10-25 15:35:18 +01:00
Alexander Neumann
d9f9b77d68 Add Index.Packs() and IDSet.Equals() 2015-10-25 15:28:01 +01:00
Alexander Neumann
4b1a2caea7 Allow overwriting the IndexFull function for tests 2015-10-25 15:05:22 +01:00
Alexander Neumann
af0d6f58b9 Remove unneeded pointer to pack id 2015-10-25 14:35:08 +01:00
Alexander Neumann
2710d6399a Cleanup index code
The selectFn wasn't used any more, so remove it from generatePackList().
2015-10-25 14:26:04 +01:00
Alexander Neumann
1dd731fdb8 Handle concurrent access to the inFlight list 2015-10-14 20:50:54 +02:00
Alexander Neumann
6fa4be5af2 Regularly save intermediate indexes 2015-10-12 23:59:17 +02:00
Alexander Neumann
941b7025b6 Delete Index.Remove() 2015-10-12 22:51:11 +02:00
Alexander Neumann
4b2a4b03ec Remove Index.StoreInProgress() 2015-10-12 22:49:31 +02:00
Alexander Neumann
86fcd170f6 Add and use MasterIndex 2015-10-12 22:34:12 +02:00
Alexander Neumann
64fa89d406 Add error checks 2015-10-12 22:07:56 +02:00
Alexander Neumann
eb73182fcf Rework index decode and handling old format 2015-10-12 20:53:07 +02:00
Alexander Neumann
356bb62243 Add CreateEncryptedBlob and GetDecryptReader 2015-10-12 20:53:07 +02:00
Alexander Neumann
96f2165067 Allow loading index with old format 2015-10-12 20:53:07 +02:00
Alexander Neumann
7944e8e323 Update index format 2015-10-12 20:53:07 +02:00
Alexander Neumann
cc7acba02b Return the original backend ID on duplicate entries 2015-10-11 20:45:50 +02:00
Alexander Neumann
f188cf81dc Add more panic() calls for invalid conditions 2015-10-11 20:45:42 +02:00
Alexander Neumann
7db2369081 Shorten error message for tree errors 2015-10-11 20:22:52 +02:00
Philipp Serr
7b11660f4f Prevent concurrent processing of same blob
... by first adding a preliminary index entry and making this fail if
an index entry for the same blob already exists.

A preliminary index entry is characterized by not yet being associated
with a pack. Until now, these entries where added to the index just
like final index entries using index.Store, which silently overwrites
existing index entries.

This commit adds a new method index.StoreInProgress which refuses to
overwrite existing index entries and allows for creating preliminary
index entries only. The existing method index.Store has not been
changed and continues to silently overwrite existing index entries.
This distinction is important, as otherwise, it would be impossible to
update a preliminary index entry after the blob has been written to a
pack.

Resolves: restic#292
2015-09-27 16:56:49 +02:00
Alexander Neumann
d42ff509ba Small refactorings
* use uint instead of uint32 in packs/indexes
 * use ID.Str() for debug messages
 * add ParallelIDWorkFunc
2015-09-05 18:41:58 +02:00
Alexander Neumann
681d7851aa index: use backend.ID instead of string for maps 2015-09-05 18:41:58 +02:00
Alexander Neumann
b85927576b Address code review comments 2015-07-26 18:01:20 +02:00
Alexander Neumann
bd746a0425 fuse: refactor and add tests for fuse.file 2015-07-26 16:43:42 +02:00
Alexander Neumann
55ddd5317d Reduce memory usage for fuse mount
This changes `repository.LoadBlob()` so that a destination buffer must
be provided, which enables the fuse code to use a buffer from a
`sync.Pool`. In addition, release the buffers when the file is closed.
At the moment, the max memory usage is defined by the max file size that
is read in one go (e.g. with `cat`). It could be further optimized by
implementing a LRU caching scheme.
2015-07-26 14:25:59 +02:00
Alexander Neumann
5cdcc99eba Use array instead of hash for backend.ID
Since backend.ID is always a slice of constant length, use an array
instead of a slice. Mostly, arrays behave as slices, except that an
array cannot be nil, so use `*backend.ID` insteaf of `backend.ID` in
places where the absence of an ID is possible (e.g. for the Subtree of a
Node, which may not present when the node is a file node).

This change allows to directly use backend.ID as the the key for a map,
so that arbitrary data structures (e.g. a Set implemented as a
map[backend.ID]struct{}) can easily be formed.
2015-07-25 18:01:57 +02:00
Matthieu Rakotojaona
ca6b7ec533 Add Index.LookupSize 2015-07-19 23:04:18 +02:00
Alexander Neumann
54c4c29a89 Add checker and command 'check' to replace 'fsck' 2015-07-11 16:27:38 +02:00
Alexander Neumann
cbcf58f1c0 repository: Improve error message 2015-07-11 16:13:11 +02:00
Florian Weingarten
36a62cf77d Extract chunker 2015-07-08 16:58:23 -04:00
Alexander Neumann
5c1fe5784a repository: Load index in parallel 2015-07-05 11:07:10 +02:00
Alexander Neumann
14d252dfba Allow cancelling parallel workers on files 2015-07-05 11:07:10 +02:00
Alexander Neumann
0b531210eb Add repository.FilesInParallel 2015-07-05 11:07:10 +02:00
Alexander Neumann
885b27fdbb Add test and benchmark for LoadIndex 2015-07-05 11:07:10 +02:00
Alexander Neumann
58600e3bb8 repository: Add comments, silence golint 2015-07-05 11:07:10 +02:00
Alexander Neumann
9a02148daf repository: Rename Save() -> SaveAndEncrypt() 2015-07-05 11:07:10 +02:00
Alexander Neumann
03983f0907 repository: Rename Load() -> LoadAndDecrypt() 2015-07-05 11:07:10 +02:00
Alexander Neumann
c553a57e0d repository: Refactor Config 2015-07-05 11:07:10 +02:00
Alexander Neumann
867f6c8e24 repository: Rename receiver: s -> r
No functional changes.
2015-07-05 11:07:10 +02:00
Alexander Neumann
0ad3d71f01 repository: Add more debug to Create() 2015-06-28 13:52:22 +02:00