octoleo
/
restic
mirror of https://github.com/octoleo/restic.git
2
2
Fork 0
Code Issues Releases Wiki Activity
6,159 Commits 7 Branches 33 Tags 64 MiB
Commit Graph

1345 Commits

Author SHA1 Message Date
Michael Eischer f414db987d gofmt all files 
Apparently the rules for comment formatting have changed with go 1.19.
 2022-08-19 19:12:26 +02:00
Michael Eischer af50fe9ac0 mount: Map slashes in tags to underscores 
Suggested-by: greatroar <>
 2022-08-19 18:17:57 +02:00
Michael Eischer 2ea6c82cf6 comment cleanup 
gofmt reformatted the comment
 2022-08-18 20:15:38 +02:00
Michael Eischer bb27f7408c forget: Fail test if duration parsing error is missing 2022-08-18 20:14:09 +02:00
Leo R. Lundgren 6f517858e8 forget: Error when invalid unit is given in duration policy 2022-08-10 13:37:26 +02:00
MichaelEischer 9ad3ad5972
Merge pull request #3850 from lbausch/go1.19 
Update tests to Go 1.19
 2022-08-07 14:56:17 +02:00
MichaelEischer 2930a102de
Merge pull request #3731 from metalsp0rk/feature/min-packsize-flag 
Feature: min packsize flag
 2022-08-07 14:54:45 +02:00
Michael Eischer f3fdc66b32
restic: Use stable sorting in snapshot policy 
sort.Sort is not guaranteed to be stable. Go 1.19 has changed the
sorting algorithm which resulted in changes of the sort order. When
comparing snapshots with identical timestamp but different paths and
tags lists, there is not meaningful order among them. So just keep their
order stable.
 2022-08-07 14:10:40 +02:00
Michael Eischer 0b7291b8b2 mount: Fix parent inode used by snapshots dir 2022-08-07 13:03:32 +02:00
greatroar cfa80e2c6b mount: remove unused inode field from root node 2022-08-07 13:03:26 +02:00
MichaelEischer 74ae76036f
Merge pull request #2913 from aawsome/mount-snapshot-slashes 
mount: Make snapshots dir structure customizable
 2022-08-07 12:27:59 +02:00
Michael Eischer caa17988a3 fuse: Redesign snapshot dirstruct 
Cleanly separate the directory presentation and the snapshot directory
structure. SnapshotsDir now translates the dirStruct into a format
usable by the fuse library and contains only minimal special case rules.
All decisions have moved into SnapshotsDirStructure which now creates a
fully preassembled tree data structure.
 2022-08-07 12:13:06 +02:00
Michael Eischer 1ed775e3a8 debug: support roundtripper logging also for release builds 
Different from debug builds do not use the eofDetectRoundTripper if
logging is disabled.
 2022-08-05 23:49:39 +02:00
Michael Eischer 38becfc436 debug: enable debug support for release builds 2022-08-05 23:49:39 +02:00
Michael Eischer 82c268c917 Remove unused hooks mechanism 2022-08-05 23:49:39 +02:00
Michael Eischer 7266f07c87 repository: StreamPack in parts if there are too large gaps 
For large pack sizes we might be only interested in the first and last
blob of a pack file. Thus stream a pack file in multiple parts if the
gaps between requested blobs grow too large.
 2022-08-05 23:48:36 +02:00
Michael Eischer 7f3b2be1e8 s3: Disable multipart uploads below 200MB 2022-08-05 23:48:36 +02:00
Michael Eischer 1b076cda97 rename option to --pack-size 2022-08-05 23:47:43 +02:00
Kyle Brennan 1e3f05c3f1 repository: prevent header overfill 2022-08-05 23:47:12 +02:00
Michael Eischer 0a6fa602c8 add option for setting min pack size 2022-08-05 23:47:12 +02:00
Michael Eischer 2db7733ee3 fuse: remove unused MetaDir 2022-08-05 23:46:46 +02:00
Michael Eischer f678f7cb04 fuse: cleanup test 2022-08-05 23:46:46 +02:00
Alexander Weiss 1751afae26 Make snapshots dirs in mount command customizable 2022-08-05 23:46:46 +02:00
Alexander Weiss 57f4003f2f Generalize fuse snapshot dirs implemetation 
+ allow "/" in tags and snapshot template
 2022-08-05 23:46:46 +02:00
Alexander Weiss 696c18e031 Add possibility to set snapshot ID (used in test) 2022-08-05 23:46:46 +02:00
MichaelEischer 04a8ee80fb
Merge pull request #3829 from MichaelEischer/prune-refactor 
Split prune into slightly small functions
 2022-08-05 23:29:52 +02:00
greatroar ad6ac680af internal/restic: Handle EINVAL for xattr on Solaris 
Also make the errors a bit less verbose by not prepending the operation,
since pkg/xattr already does that. Old errors looked like

    Listxattr: xattr.list /myfiles/.zfs/snapshot: invalid argument
 2022-08-01 12:45:17 +02:00
Michael Eischer 73053674d9 repository: Test fallback to existing blobs 2022-07-30 17:37:07 +02:00
Michael Eischer 623770eebb repository: try to recover from invalid blob while repacking 
If a blob that should be kept is invalid, Repack will now try to request
the blob using LoadBlob. Only return an error if that fails.
 2022-07-30 17:37:07 +02:00
greatroar 2bdc40e612 Speed up restic init over slow SFTP links 
pkg/sftp.Client.MkdirAll(d) does a Stat to determine if d exists and is
a directory, then a recursive call to create the parent, so the calls
for data/?? each take three round trips. Doing a Mkdir first should
eliminate two round trips for 255/256 data directories as well as all
but one of the top-level directories.

Also, we can do all of the calls concurrently. This may reintroduce some
of the Stat calls when multiple goroutines try to create the same
parent, but at the default number of connections, that should not be
much of a problem.
 2022-07-30 13:09:08 +02:00
greatroar 23ebec717c Remove stale comments from backend/sftp 
The preExec and postExec functions were removed in
0bdb131521 from 2018.
 2022-07-30 13:07:25 +02:00
Michael Eischer 4a10ebed15 archiver: reduce memory usage for large files 
FutureBlob now uses a Take() method as a more memory-efficient way to
retrieve the futures result. In addition, futures are now collected
while saving the file. As only a limited number of blobs can be queued
for uploading, for a large file nearly all FutureBlobs already have
their result ready, such that the FutureBlob object just consumes
memory.
 2022-07-23 14:45:07 +02:00
Michael Eischer b817681a11 archiver: Incrementally serialize tree nodes 
That way it is not necessary to keep both the Nodes forming a Tree and
the serialized JSON version in memory.
 2022-07-23 14:45:07 +02:00
Michael Eischer c206a101a3 archiver: unify FutureTree/File into futureNode 
There is no real difference between the FutureTree and FutureFile
structs. However, differentiating both increases the size of the
FutureNode struct.

The FutureNode struct is now only 16 bytes large on 64bit platforms.
That way is has a very low overhead if the corresponding file/directory
was not processed yet.

There is a special case for nodes that were reused from the parent
snapshot, as a go channel seems to have 96 bytes overhead which would
result in a memory usage regression.
 2022-07-23 14:45:07 +02:00
Michael Eischer 32f4997733 archiver: remove unused fileInfo from progress callback 2022-07-23 14:16:23 +02:00
Michael Eischer dcb00fd2d1 archiver: cleanup Saver interface 2022-07-23 14:16:23 +02:00
Michael Eischer 79321a195c archiver: remove dead attribute from FutureNode 2022-07-23 14:16:23 +02:00
Michael Eischer 5a6f2f9fa0 Fix S3 legacy layout migration 2022-07-23 11:19:32 +02:00
Michael Eischer 04e49924fb checker: Fix S3 legacy layout detection 2022-07-23 11:19:32 +02:00
Michael Eischer fcb3ddf181 check: Complain about usage of s3 legacy layout 2022-07-23 11:19:32 +02:00
Michael Eischer 8b8bd4e8ac check: complain about mixed pack files 2022-07-23 11:19:32 +02:00
MichaelEischer 443cc49afd
Merge pull request #3830 from MichaelEischer/cleanup-repo 
Extract Load/SaveTree/JSONUnpacked from repository
 2022-07-23 10:46:13 +02:00
MichaelEischer 1f5369e072
Merge pull request #3831 from MichaelEischer/move-code 
Move code out of the restic package and consolidate backend specific code
 2022-07-23 10:33:05 +02:00
Michael Eischer 9729e6d7ef backend: extract readerat from restic package 2022-07-17 15:29:09 +02:00
Michael Eischer c44b21d366 restorer: extract hardlinks index from restic package 2022-07-17 13:45:42 +02:00
Michael Eischer 8c11fc3ec9 crypto: move crypto buffer helpers 2022-07-17 13:42:23 +02:00
Michael Eischer a0cef9f247 limiter: move to internal/backend 2022-07-17 13:40:15 +02:00
Michael Eischer 163ab9c025 mock: move to internal/backend 2022-07-17 13:40:06 +02:00
Michael Eischer 89d3ce852b repository: extract Load/StoreJSONUnpacked 
A Load/Store method for each data type is much clearer. As a result the
repository no longer needs a method to load / store json.
 2022-07-17 13:22:00 +02:00
Michael Eischer fbcbd5318c repository: extract LoadTree/SaveTree 
The repository has no real idea what a Tree is. So these methods never
belonged there.
 2022-07-17 13:11:28 +02:00