octoleo
/
restic
mirror of https://github.com/octoleo/restic.git
2
2
Fork 0
Code Issues Releases Wiki Activity
6,305 Commits 7 Branches 33 Tags 64 MiB
Commit Graph

93 Commits

Author SHA1 Message Date
Michael Eischer 6d2d297215 pass global context through cobra 2022-10-03 00:19:46 +02:00
Michael Eischer 928914f821 Prepare for context bound to lock lifetime 2022-10-03 00:19:46 +02:00
Michael Eischer 985722b102 Remove ctx from globalOptions 
Previously the global context was either accessed via gopts.ctx,
stored in a local variable and then used within that function or
sometimes both. This makes it very hard to follow which ctx or a wrapped
version of it reaches which method.

Thus just drop the context from the globalOptions struct and pass it
explicitly to every command line handler method.
 2022-10-03 00:19:46 +02:00
Michael Eischer 1ebd57247a repository: optimize MasterIndex.Each 
Sending data through a channel at very high frequency is extremely
inefficient. Thus use simple callbacks instead of channels.

> name                old time/op  new time/op  delta
> MasterIndexEach-16   6.68s ±24%   0.96s ± 2%  -85.64%  (p=0.008 n=5+5)
 2022-09-24 12:21:59 +02:00
mattxtaz 01ab36336f Fix typo with double percentage in help text 2022-08-07 20:21:05 +01:00
Michael Eischer 55a11c1396 Reword prune --repack-small description 2022-08-05 23:48:36 +02:00
Michael Eischer 176b387d98 Always repack very small pack files 2022-08-05 23:48:36 +02:00
Michael Eischer 324935cb80 Only repack small files if there are multiple of them 2022-08-05 23:48:34 +02:00
Michael Eischer 1b076cda97 rename option to --pack-size 2022-08-05 23:47:43 +02:00
Michael Eischer 6a6d313c9a prune: reduce priority of repacking small packs 2022-08-05 23:47:12 +02:00
Kyle Brennan 0269381b8d prune: add repack-small parameter 2022-08-05 23:47:12 +02:00
Michael Eischer e85a21eda2 prune: move code 2022-07-30 17:37:07 +02:00
Michael Eischer d0590b7841 prune: Add internal integrity check 
After repacking every blob that should be kept must have been repacked.
We have seen a few cases in which a single blob went missing, which
could have been caused by a bitflip somewhere. This sanity check might
help catch some of these cases.
 2022-07-30 17:37:07 +02:00
Michael Eischer 5cbde03eae prune: split into smaller functions 2022-07-30 17:37:07 +02:00
Alexander Weiss 7643237da5 prune: separate collecting/printing/pruning 2022-07-30 17:37:07 +02:00
Michael Eischer 715d457aad prune: code cleanups 2022-07-17 11:41:56 +02:00
Michael Eischer 9be1bd2acc prune: handle very high duplication of some blobs 
Suggested-By: Alexander Weiss <alex@weissfam.de>
 2022-07-17 11:39:56 +02:00
Alexander Weiss 7478cbf70e prune: Enhance treatment of duplicates 2022-07-17 00:22:23 +02:00
Michael Eischer ec4dfa3c66 Wording: replace further repo occurrences with repository 2022-07-12 20:48:01 +02:00
Michael Eischer ed8aa15376 repository: add Save method to MasterIndex interface 2022-07-02 18:38:56 +02:00
Alexander Neumann 74f7fe2b98
Merge pull request #3767 from MichaelEischer/fix-prune-empty-snapshot 
prune: Fix crash on snapshot loading error
 2022-05-29 16:52:45 +02:00
Michael Eischer c8e1ac4049 prune: Don't print stack trace if snapshot can't be loaded 2022-05-23 22:38:45 +02:00
Michael Eischer 173695104c prune: Fix crash on empty snapshot 2022-05-23 22:32:59 +02:00
Michael Eischer 381bd94c6c prune: Add option to repack uncompressed data 2022-05-09 22:31:30 +02:00
Michael Eischer 5406743102 prune: Automatically repack uncompressed trees for repo v2 
Tree packs are cached locally at clients and thus benefit a lot from
being compressed. Ensure this be having prune always repack pack files
containing uncompressed trees.
 2022-05-09 22:31:30 +02:00
Michael Eischer dbbeac7174 prune: Add unsafe option to recover from no free space 
The new option allows prune to operate with nearly no scratch space by only removing
no longer necessary pack files and first deleting the index before
rebuilding it. By first deleting the index it becomes safe to just
delete no longer necessary pack files. However, as a downside there's
now the risk that the repository becomes inaccessible if prune fails.

To recover from that problem a user might have to manually delete the
repository index and then run (a full) `rebuild-index` again.
 2022-04-30 19:21:07 +02:00
Michael Eischer f5609d1d3c prune: Fail early if too few backend connections 2022-04-23 11:32:52 +02:00
Michael Eischer 3d29083e60 copy/find/ls/recover/stats: Memorize snapshot listing before index 
These commands filter the snapshots according to some criteria which
essentially requires loading the index before filtering the snapshots.
Thus create a copy of the snapshots list beforehand and use it later on.
 2022-04-09 12:26:30 +02:00
Michael Eischer 2ec0f3303a backup/diff/dump/restore/stats: List snapshots before index 
During a backup the index is written before the corresponding snapshots.
To ensure that a concurrent/later restic run can read a snapshot's data,
restic thus must first load the snapshots and only afterwards the index.
Otherwise it is not possible to ensure that the loaded index is recent
enough to cover all of the snapshot's data.
 2022-04-09 12:24:09 +02:00
Michael Eischer f78bd14e28 repository: Remove pack implementation details from MasterIndex 2022-03-28 22:09:49 +02:00
Michael Eischer 537b4c310a copy: Implement by reusing repack 
The repack operation copies all selected blobs from a set of pack files
into new pack files. For prune the source and destination repositories
are identical. To implement copy, just use a different source and
destination repository.
 2022-03-26 20:47:15 +01:00
rawtaz abfbacf3d3
Merge pull request #3591 from MichaelEischer/prune-fix-max-repack 
prune: Handle --max-repack-size=0 as expected
 2022-01-13 03:53:20 +01:00
mattxtaz 6ff32ee4d3 Add missing colon in prune stats output and change padding to 14 chars to align the fields 2022-01-06 21:15:15 +00:00
Michael Eischer 0cfdb82ea4 prune: Handle --max-repack-size=0 as expected 
Previously the flag was ignored and `--max-repack-size=1` had to be
used.
 2021-12-27 15:48:56 +01:00
greatroar 7f0aa49f45 cmd/restic: Streamline progress printing 
* PrintProgress no longer does unnecessary Sprintf calls, and performs
  fewer allocations in general
* newProgressMax's callback checks whether the terminal supports
  line updates once instead of once per call
* the callback looks up the terminal width once per call instead of
  twice (on Windows)
* the status shortening now uses the Unicode-aware version from
  internal/ui/termstatus (future-proofing)
 2021-09-03 11:48:22 +02:00
Michael Eischer aad8864835 prune: Add missing newlines in error descriptions 2021-06-20 14:25:40 +02:00
Alexander Neumann 027a51529d prune: Improve error message for missing files 
This commit changes the error message so that a list of file names is
printed. Before, just the raw map was printed, which is not a great user
interface.
 2021-01-31 11:31:27 +01:00
Alexander Weiss e08e65dc30 prune: Simplify logic selecting packs to repack 2021-01-29 22:27:22 +01:00
Alexander Weiss daeb4cdf8f prune: Fix statistics for --repack-cacheable-only 2021-01-29 22:27:22 +01:00
Alexander Neumann a16ce65295
Merge pull request #3244 from MichaelEischer/better-damage-reports 
Print more details about possible repository damages
 2021-01-29 11:45:45 +01:00
Michael Eischer 58b5679f14 prune: reword missing blobs error 
The previous wording could be understood such that the prune run did
damage the repository.
 2021-01-28 21:48:24 +01:00
Michael Eischer 7b8886c052 prune: report missing but unneeded pack files 
This indicates a damaged repository so add some output to help with
debugging.
 2021-01-28 21:46:01 +01:00
MichaelEischer 82140967d3
Merge pull request #3228 from aawsome/prune-all-trees 
prune: Remove all unused trees
 2021-01-28 21:04:27 +01:00
MichaelEischer 35033d9b79
Merge pull request #3177 from MichaelEischer/fix-2759 
prune: don't print stacktrace on console
 2021-01-28 20:28:45 +01:00
Michael Eischer eda8c67616 restic: let FindUsedBlobs handle multiple snapshots at once 2021-01-28 11:08:43 +01:00
Alexander Weiss d7dc19a496 prune: Always repack packs containing tree blobs 2021-01-15 16:42:04 +01:00
Alexander Weiss f0113139ea prune: Correct error message 2020-12-29 20:20:05 +01:00
Alexander Weiss f6df94a50e prune: Add self-healing 
Allow prune to heal situations where blobs in the index are missing or
the corresponding packfiles are damaged if those blobs are not needed.
 2020-12-29 20:20:05 +01:00
Alexander Weiss 68b74e359e Count packs directly in RebuildIndexFiles 2020-12-22 23:01:58 +01:00
Michael Eischer cfea79d0c5 prune: don't print stacktrace on console 2020-12-19 14:28:48 +01:00