octoleo
/
restic
mirror of https://github.com/octoleo/restic.git
2
2
Fork 0
Code Issues Releases Wiki Activity
6,305 Commits 7 Branches 33 Tags 65 MiB
Commit Graph

93 Commits

Author SHA1 Message Date
Alexander Weiss e329623771 Remove LoadAllSnapshots 2020-12-06 05:22:27 +01:00
Alexander Weiss 92bd448691 Make BlobHandle substruct of Blob 2020-11-22 20:41:10 +01:00
Alexander Weiss 67c938f232 Use PackSize in prune 2020-11-21 22:13:54 +01:00
Alexander Weiss cb5ec7ea6b Fix statistics in prune for duplicates 
Note that this fix only solves the statistics problem, if
all duplicates are marked for repacking.

If not all duplicates are marked for repacking, we lack the
information which

The situation that not all duplicates are marked for repacking can occur
when using the `max-repack-size` option
 2020-11-18 22:30:22 +01:00
Alexander Weiss 1ec628ddf5 Add extraObsolete to MasterIndex.Save 2020-11-15 07:05:09 +01:00
greatroar 21b787a4d1 Stop Counters where they're constructed and started 2020-11-09 13:03:31 +01:00
greatroar ddca699cd2 Replace restic.Progress with new progress.Counter 
This fixes two race conditions while cleaning up the code.
 2020-11-09 12:12:35 +01:00
Alexander Neumann a8d21b5dcf prune: Warn if no cache is present 2020-11-08 20:25:35 +01:00
Alexander Neumann 47277c4b4c Add comments, clarify computation 2020-11-06 20:23:30 +01:00
Alexander Weiss fd33030556 Use in-memory index to rebuild index in prune 2020-11-06 20:23:30 +01:00
Alexander Neumann 1ca60bccfb Refactor condition for MaxRepackBytes 
Don't depend on the string (opts.MaxRepackSize) for the condition,
instead check if there's a (positive) limit configured.
 2020-11-03 16:42:21 +01:00
Alexander Neumann 7f86eb4ec0 Move helper function 2020-11-03 16:42:21 +01:00
Alexander Neumann c1a3de4a6e Refactor max-unused calculation, add `unlimited` option 
Add a callback to the PruneOptions struct which calculates the number of
bytes allowed to be unused after prune is done. This way, the logic is
closer to the option parsing code.

Also, add an explicit option `unlimited` for the use case when storage
does not matter but bandwidth and time do. Internally, this sets the
maximum number of unused bytes to MaxUint64.

Rework the documentation slightly so that no more "packs" are
mentioned and it talks about "files" instead.

Make it clear in the documentation that the percentage given to
`--max-unused` is relative to the whole repository size after pruning is
done. If specified, it must be below 100%, otherwise the repository
would contain 100% of unused data, which is pointless.

I had a hard time coming up with the correct formula to calculate the
maximum number of unused bytes based on the number of used bytes. For a
fraction `p` (0 ≤ p < 1), a repo with `u` bytes used, and the number of
unused bytes `x` the following holds:

      x ≤ p * (u+x)
    ⇔ x ≤ p*u + p*x
    ⇔ x - p*x ≤ p*u
    ⇔ x * (1-p) ≤ p*u
    ⇔ x ≤ p/(1-p) * u
 2020-11-03 16:42:21 +01:00
Alexander Neumann f8c4dd7b1a Split packe rewrite logic into two case branches 
The comma is too sublte, let's split this into two separate branches.
 2020-11-03 16:42:21 +01:00
Alexander Neumann a5b80452fe Add comment that usedBlobs is modified 2020-11-03 16:42:21 +01:00
Alexander Neumann aff1e220f5 Split struct members, add comments 2020-11-03 16:42:21 +01:00
Alexander Neumann 095155d9ce Remove RepackSmall 2020-11-03 16:42:21 +01:00
Alexander Weiss b2f5381737 Make realistic forget --prune --dryrun 2020-11-03 16:42:21 +01:00
Alexander Weiss 7f9a0a5907 Reimplementation of prune 2020-11-03 16:42:21 +01:00
Michael Eischer 0c9efa9c2a Pass context to lockRepo 2020-10-09 22:39:06 +02:00
Michael Eischer 17995dec7a Unify progress bar of check and prune commands 2020-08-25 22:47:38 +02:00
Michael Eischer 08d24ff99e prune: Include ID of all missing blobs in error message 2020-08-16 11:36:14 +02:00
Michael Eischer 3ba19869be prune: Abort if any used blobs are missing 
The previous check only approximately verified whether all required
blobs were found. However, after forgetting a few snapshots the
repository contains lots of unused blobs whose number can be sufficient
to make up for missing packs.

When coupled with a malfunctioning backend that temporarily returns broken
data this could cause restic to regard the corresponding packs as
invalid and thereby delete data that's still in use. This change lets
restic play it safe and refuse to delete anything if data is missing.
 2020-08-16 11:34:01 +02:00
aawsome 0fed6a8dfc
Use "pack file" instead of "data file" (#2885) 
- changed variable names, especially changed DataFile into PackFile
- changed in some comments
- always use "pack file" in docu
 2020-08-16 11:16:38 +02:00
Michael Eischer 05116e4787 prune: Cleanup progress bar handling while repacking 2020-08-03 19:32:46 +02:00
Michael Eischer 04f79b9642 prune: Stop progress bar after searching used blobs 2020-08-03 19:31:49 +02:00
MichaelEischer 66d089e239
Merge pull request #2841 from aawsome/optimize-getUsedBlobs 
Extract get used blobs in prune as separate function
 2020-08-01 22:40:33 +02:00
Alexander Weiss 9762bec091 Use optimized getUsedBlobs in prune 2020-08-01 21:07:31 +02:00
Alexander Weiss 2ee654763b Delete files in parallel 2020-08-01 20:39:24 +02:00
Michael Eischer 184103647a FindUsedBlobs: merge seen into blobs BlobSet 
The seen BlobSet always contained a subset of the entries in blobs.
Thus use blobs instead and avoid the memory overhead of the second set.

Suggested-by: Alexander Weiss <alex@weissfam.de>
 2020-08-01 12:29:16 +02:00
Alexander Weiss 70347e95d5 disable index uploads for prune command 
+ modifications of changelog
 2020-06-12 09:24:38 +02:00
Erik Rigtorp 94f4f13388 Add documentation on exit status codes to man pages 
This is step one to start defining useful exit codes for all the commands.
 2020-02-12 23:09:26 +01:00
Alexander Neumann 1ab5703404 prune: Fix calculation for removed bytes 2018-08-14 22:06:05 +02:00
Alexander Neumann 663c57ab4d debug: Remove manual Str() call Log() 2018-01-25 20:49:41 +01:00
Alexander Neumann b0c6e53241 Fix calls to repo/backend.List() everywhere 2018-01-21 21:15:09 +01:00
Alexander Neumann 56fccecd06 prune: Repack mixed pack files 2017-09-24 21:54:53 +02:00
Stefan Völkel 7f927d4774 fix duplicate bytes in prune output 2017-09-22 10:07:24 +02:00
Alexander Neumann d7e644272f prune: Add plausibility check 2017-09-19 10:50:07 +02:00
Shayne Holmes 9eb39cef05 Capitalize short help commands 
Unify existing Cobra help command, and git-help's style.
 2017-09-11 09:32:44 -07:00
Alexander Neumann 6bc43a4198 manpage: Remove auto gen tag from man page 2017-08-06 21:31:01 +02:00
Alexander Neumann 23c903074c Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
Alexander Neumann 6caeff2408 Run goimports 2017-07-23 14:21:03 +02:00
Alexander Neumann 83d1a46526 Moves files 2017-07-23 14:19:13 +02:00