octoleo
/
restic
mirror of https://github.com/octoleo/restic.git
2
2
Fork 0
Code Issues Releases Wiki Activity
6,290 Commits 7 Branches 33 Tags 64 MiB
Commit Graph

6290 Commits

Author SHA1 Message Date
Michael Eischer 2ba14160de Add changelog for the optimized tree serialization 2022-07-23 14:49:08 +02:00
Michael Eischer 4a10ebed15 archiver: reduce memory usage for large files 
FutureBlob now uses a Take() method as a more memory-efficient way to
retrieve the futures result. In addition, futures are now collected
while saving the file. As only a limited number of blobs can be queued
for uploading, for a large file nearly all FutureBlobs already have
their result ready, such that the FutureBlob object just consumes
memory.
 2022-07-23 14:45:07 +02:00
Michael Eischer b817681a11 archiver: Incrementally serialize tree nodes 
That way it is not necessary to keep both the Nodes forming a Tree and
the serialized JSON version in memory.
 2022-07-23 14:45:07 +02:00
Michael Eischer c206a101a3 archiver: unify FutureTree/File into futureNode 
There is no real difference between the FutureTree and FutureFile
structs. However, differentiating both increases the size of the
FutureNode struct.

The FutureNode struct is now only 16 bytes large on 64bit platforms.
That way is has a very low overhead if the corresponding file/directory
was not processed yet.

There is a special case for nodes that were reused from the parent
snapshot, as a go channel seems to have 96 bytes overhead which would
result in a memory usage regression.
 2022-07-23 14:45:07 +02:00
Michael Eischer 32f4997733 archiver: remove unused fileInfo from progress callback 2022-07-23 14:16:23 +02:00
Michael Eischer dcb00fd2d1 archiver: cleanup Saver interface 2022-07-23 14:16:23 +02:00
Michael Eischer 79321a195c archiver: remove dead attribute from FutureNode 2022-07-23 14:16:23 +02:00
MichaelEischer 049f4c4144
Merge pull request #3730 from MichaelEischer/stricter-check 
Let `check` warn about legacy variants of the repo format version 1
 2022-07-23 14:14:50 +02:00
Michael Eischer 3bf53da672 Add changelog for stricter checks 2022-07-23 11:21:26 +02:00
Michael Eischer 5a6f2f9fa0 Fix S3 legacy layout migration 2022-07-23 11:19:32 +02:00
Michael Eischer 04e49924fb checker: Fix S3 legacy layout detection 2022-07-23 11:19:32 +02:00
Michael Eischer 768c890fcb check: Deprecate `--check-unused` 
Unused blobs are not a problem but rather expected to exist now that
prune by default does not remove every unused blob. However, the option
has caused questions from users whether a repository is damaged or not,
so just remove that option.

Note that the remaining code is left intact as it is still useful for
our test cases.
 2022-07-23 11:19:32 +02:00
Michael Eischer fcb3ddf181 check: Complain about usage of s3 legacy layout 2022-07-23 11:19:32 +02:00
Michael Eischer 8b8bd4e8ac check: complain about mixed pack files 2022-07-23 11:19:32 +02:00
MichaelEischer 443cc49afd
Merge pull request #3830 from MichaelEischer/cleanup-repo 
Extract Load/SaveTree/JSONUnpacked from repository
 2022-07-23 10:46:13 +02:00
MichaelEischer 1f5369e072
Merge pull request #3831 from MichaelEischer/move-code 
Move code out of the restic package and consolidate backend specific code
 2022-07-23 10:33:05 +02:00
MichaelEischer 827ab02eea
Merge pull request #3661 from rgammans/azure_sas_support 
add SAS authentication option for Azure repos
 2022-07-23 10:32:03 +02:00
MichaelEischer e9c39442fb
Merge pull request #3827 from MichaelEischer/backup-doc 
backup: clarify usage string
 2022-07-23 10:31:33 +02:00
Michael Eischer 9729e6d7ef backend: extract readerat from restic package 2022-07-17 15:29:09 +02:00
Michael Eischer c44b21d366 restorer: extract hardlinks index from restic package 2022-07-17 13:45:42 +02:00
Michael Eischer 8c11fc3ec9 crypto: move crypto buffer helpers 2022-07-17 13:42:23 +02:00
Michael Eischer a0cef9f247 limiter: move to internal/backend 2022-07-17 13:40:15 +02:00
Michael Eischer 163ab9c025 mock: move to internal/backend 2022-07-17 13:40:06 +02:00
Michael Eischer 89d3ce852b repository: extract Load/StoreJSONUnpacked 
A Load/Store method for each data type is much clearer. As a result the
repository no longer needs a method to load / store json.
 2022-07-17 13:22:00 +02:00
Michael Eischer fbcbd5318c repository: extract LoadTree/SaveTree 
The repository has no real idea what a Tree is. So these methods never
belonged there.
 2022-07-17 13:11:28 +02:00
MichaelEischer d9ea1e9ee2
Merge pull request #3290 from aawsome/prune-handle-duplicates 
prune: Handle duplicate blobs more efficiently
 2022-07-17 11:51:54 +02:00
Michael Eischer 715d457aad prune: code cleanups 2022-07-17 11:41:56 +02:00
Michael Eischer 9be1bd2acc prune: handle very high duplication of some blobs 
Suggested-By: Alexander Weiss <alex@weissfam.de>
 2022-07-17 11:39:56 +02:00
Alexander Weiss 7478cbf70e prune: Enhance treatment of duplicates 2022-07-17 00:22:23 +02:00
Michael Eischer b2043e8198 backup: clarify usage string 
Using the `--files-from` options it is possible to run `backup` without
specifying any source paths directly on the command line.
 2022-07-17 00:03:22 +02:00
Michael Eischer 5639c41b6a azure: Strip ? prefix from sas token 2022-07-16 23:55:18 +02:00
Roger Gammans 64a7ec5341 azure: add SAS authentication option 2022-07-16 23:55:18 +02:00
MichaelEischer 6cbeb4a9f9
Merge pull request #3825 from restic/rawtaz-doc-gdrive 
doc: Add note about using rclone for Google Drive
 2022-07-16 19:21:59 +02:00
rawtaz f5c219f5a2
doc: Add note about using rclone for Google Drive 
It wasn't clear that Google Cloud Storage and Google Drive are two different services and that one should use the rclone backend for the latter. This commit adds a note with this information.
 2022-07-16 13:22:38 +02:00
MichaelEischer d71b29221b
Merge pull request #3822 from JsBergbau/doc-max-compression 
Added hint for --compression max in migration process
 2022-07-12 23:02:45 +02:00
Michael Eischer 71ff6b77f0 doc: Rework hint to repack with max compression 2022-07-12 21:24:40 +02:00
MichaelEischer 6970d05d47
Merge pull request #3741 from lbausch/repository-wording 
Wording: change repo to repository
 2022-07-12 21:15:33 +02:00
MichaelEischer 3934480da4
Merge pull request #3819 from lbausch/restore-validate-patterns 
restore: validate include/exclude patterns
 2022-07-12 20:58:41 +02:00
MichaelEischer 71a0157c2c
Merge pull request #3820 from lbausch/patch-1 
Fix wording in changelog template
 2022-07-12 20:51:52 +02:00
Michael Eischer 2aad6f24b5 doc: update sample help output 2022-07-12 20:49:01 +02:00
Michael Eischer ec4dfa3c66 Wording: replace further repo occurrences with repository 2022-07-12 20:48:01 +02:00
JsBergbau 8d3f04aefa
Added hint for --compression max in migration process 
Added hint for --compression max in migration process. Since this is a onetime process users should be aware of this and consider this step.
 2022-07-12 00:08:26 +02:00
Lorenz Bausch b609523582
Add changelog entry 2022-07-09 22:24:56 +02:00
lbausch ac96a4138d
Fix wording in changelog template 2022-07-09 22:13:54 +02:00
Lorenz Bausch 7e36ec279d
Test restore fails when using invalid patterns 2022-07-08 20:09:26 +02:00
Lorenz Bausch be524f0b78
Add testRunRestoreAssumeFailure function 2022-07-08 20:09:25 +02:00
Lorenz Bausch 9a7db6675c
Restore: validate provided patterns 2022-07-08 20:09:25 +02:00
Lorenz Bausch d6e3c7f28e
Wording: change repo to repository 2022-07-08 20:05:35 +02:00
MichaelEischer 98a3125ce4
Merge pull request #3816 from mattxtaz/master 
Fix minor typo in docs
 2022-07-05 21:30:26 +02:00
mattxtaz 87d899c099 Fix minor typo in docs 2022-07-05 20:06:27 +01:00