octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-12-12 14:17:57 +00:00
Code Issues Releases Wiki Activity
6,150 Commits 7 Branches 33 Tags 66 MiB
bb27f7408c
Commit Graph

6150 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael Eischer
bf81bf0795 repository: Properly set id for finalized index 
As MergeFinalIndex and index uploads can occur concurrently, it is
necessary for MergeFinalIndex to check whether the IDs for an index were
already set before merging it. Otherwise, we'd loose the ID of an index
which is set _after_ uploading it.
 2022-07-02 18:39:59 +02:00
Michael Eischer
e0a7852b8b repository: remove unused (Master)Index.Count 2022-07-02 18:39:58 +02:00
Michael Eischer
8ef2968f28 repository: remove unused index.ListPack 2022-07-02 18:39:12 +02:00
Michael Eischer
e4f20dea61 repository: inline index.encode 2022-07-02 18:39:12 +02:00
Michael Eischer
fe5a8e137a repository: remove unused index.Store 2022-07-02 18:39:12 +02:00
Michael Eischer
628ae799ca repository: make flushPacks private 2022-07-02 18:39:12 +02:00
Michael Eischer
ed8aa15376 repository: add Save method to MasterIndex interface 2022-07-02 18:38:56 +02:00
Michael Eischer
a77d5c4d11 repository: index saving belongs into the MasterIndex 2022-07-02 18:38:56 +02:00
MichaelEischer
19641bf828
Merge pull request #3810 from greatroar/revert-3786 
Revert "restic prune: Merge three loops over the index"
 2022-07-01 23:13:39 +02:00
greatroar
a0fa9c6e9f Revert "restic prune: Merge three loops over the index" 
This reverts commit 8bdfcf779f.
Should fix #3809. Also needed to make #3290 apply cleanly.
 2022-06-30 15:27:34 +02:00
greatroar
90d2c0502b cmd/restic, limiter: Move config knowledge to internal packages 
The GlobalOptions struct now embeds a backend.TransportOptions, so it
doesn't need to construct one in open and create. The upload and
download limits are similarly now a struct in internal/limiter that is
embedded in GlobalOptions.
 2022-06-22 18:29:58 +02:00
MichaelEischer
bc96879d41
Merge pull request #3785 from MichaelEischer/replace-tomb-usage 
Remove usage of tomb package
 2022-06-19 14:42:48 +02:00
MichaelEischer
307f14604f
Merge pull request #3795 from greatroar/sema 
backend: Move semaphores to a dedicated package
 2022-06-18 17:12:01 +02:00
MichaelEischer
19581dbc18
Merge pull request #3786 from greatroar/prune 
restic prune: Merge three loops over the index
 2022-06-18 16:54:50 +02:00
greatroar
8bdfcf779f restic prune: Merge three loops over the index 
There were three loops over the index in restic prune, to find
duplicates, to determine sizes (in pack.Size) and to generate packInfos.
These three are now one loop. This way, prune doesn't need to construct
a set of duplicate blobs, pack.Size doesn't need to contain special
logic for prune's use case (the onlyHdr argument) and pack.Size doesn't
need to construct a map only to have it immediately transformed into a
different map.

Some quick testing on a 160GiB local repo doesn't show running time or
memory use of restic prune --dry-run changing significantly.
 2022-06-18 10:40:33 +02:00
greatroar
910d917b71 backend: Move semaphores to a dedicated package 
... called backend/sema. I resisted the temptation to call the main
type sema.Phore. Also, semaphores are now passed by value to skip a
level of indirection when using them.
 2022-06-18 10:01:58 +02:00
MichaelEischer
2c893fe43c
Merge pull request #3798 from greatroar/errors 
all: Move away from pkg/errors, easy cases
 2022-06-17 19:01:40 +02:00
greatroar
f92ecf13c9 all: Move away from pkg/errors, easy cases 
github.com/pkg/errors is no longer getting updates, because Go 1.13
went with the more flexible errors.{As,Is} function. Use those instead:
errors from pkg/errors already support the Unwrap interface used by 1.13
error handling. Also:

* check for io.EOF with a straight ==. That value should not be wrapped,
  and the chunker (whose error is checked in the cases changed) does not
  wrap it.
* Give custom Error methods pointer receivers, so there's no ambiguity
  when type-switching since the value type will no longer implement error.
* Make restic.ErrAlreadyLocked private, and rename it to
  alreadyLockedError to match the stdlib convention that error type
  names end in Error.
* Same with rest.ErrIsNotExist => rest.notExistError.
* Make s3.Backend.IsAccessDenied a private function.
 2022-06-14 08:36:38 +02:00
MichaelEischer
0c0e7b6957
Merge pull request #3776 from wjiec/bugfix/maxkeys-in-search 
Limit number of key files tested while opening a repository
 2022-06-12 15:51:10 +02:00
Michael Eischer
c9ef873192 tweak password test count changelog 2022-06-12 15:39:06 +02:00
Michael Eischer
d08549f0ae fix flaky key test 2022-06-12 14:19:06 +02:00
Jayson Wang
f144920ed5 fix handling of maxKeys in SearchKey 2022-06-12 14:19:06 +02:00
Alexander Neumann
1dd4b9b60e
Merge pull request #3788 from greatroar/sftp-posix-rename 
backend/sftp: Support atomic rename
 2022-06-06 19:39:48 +02:00
Alexander Neumann
07114ccb21
Merge pull request #3789 from greatroar/fix-loadblob 
internal/repository: Fix LoadBlob + fuzz test
 2022-06-06 19:33:42 +02:00
greatroar
c9557b2822 internal/repository: Fix LoadBlob + fuzz test 
When given a buf that is big enough for a compressed blob but not its
decompressed contents, the copy at the end of LoadBlob would skip the
last part of the contents.

Fixes #3783.
 2022-06-06 17:02:28 +02:00
greatroar
fa8f02292e backend/sftp: Support atomic rename 
... if the server has posix-rename@openssh.com.
OpenSSH introduced this extension in 2008:
7c29661471
 2022-06-06 13:40:42 +02:00
Alexander Neumann
7d64aa7f57
Merge pull request #3784 from MichaelEischer/cleanup-migrate-check 
migrate: Cleanup option to request repository check
 2022-06-05 17:37:45 +02:00
Michael Eischer
853ceb3bec get rid of tomb package 2022-06-05 15:54:57 +02:00
Michael Eischer
e002b09d57 archiver: free workers once finished 2022-06-05 15:48:10 +02:00
Michael Eischer
408ac1a0c2 archiver: remove tomb usage 2022-06-05 15:47:52 +02:00
Michael Eischer
5eba1217e7 migrate: Cleanup option to request repository check 2022-06-04 23:45:00 +02:00
MichaelEischer
0cb6b3d80a
Merge pull request #3778 from greatroar/ellipsis 
cmd/restic: Remove trailing "..." from progress messages
 2022-06-04 19:16:00 +02:00
MichaelEischer
60ca6b1418
Merge pull request #3774 from greatroar/archiver-pool 
archiver: Remove cleanup goroutine from BufferPool
 2022-06-04 18:50:24 +02:00
greatroar
b7c990871f cmd/restic: Remove trailing "..." from progress messages 
These were added after message since the last refactor of the progress
printing code. Also skips an allocation in the common case.
 2022-05-31 19:06:26 +02:00
greatroar
0db1d11b2e archiver: Remove cleanup goroutine from BufferPool 
This isn't doing anything. Channels should get cleaned up by the GC when
the last reference to them disappears, just like all other data
structures. Also inlined BufferPool.Put in Buffer.Release, its only
caller.
 2022-05-29 17:09:16 +02:00
Alexander Neumann
74f7fe2b98
Merge pull request #3767 from MichaelEischer/fix-prune-empty-snapshot 
prune: Fix crash on snapshot loading error
 2022-05-29 16:52:45 +02:00
Alexander Neumann
d2c5843c68
Merge pull request #3704 from MichaelEischer/compression-migrations 
Support migration to repository format with compression
 2022-05-29 15:52:21 +02:00
Alexander Neumann
78a21bbccf
Merge pull request #3752 from MichaelEischer/fix-dir-sync-errors 
local: Ignore additional errors for directory syncing
 2022-05-29 12:54:51 +02:00
MichaelEischer
2ce8587598
Merge pull request #3771 from greatroar/id-marshaljson 
internal/restic: Custom ID.MarshalJSON
 2022-05-28 16:24:51 +02:00
Michael Eischer
a73fc31b50 Fix linter check 2022-05-28 16:13:46 +02:00
greatroar
dde8e9e296 internal/restic: Custom ID.MarshalJSON 
This skips an allocation. internal/archiver benchmarks, Linux/amd64:

name                     old time/op    new time/op    delta
ArchiverSaveFileSmall-8    3.94ms ± 6%    3.91ms ± 6%    ~     (p=0.947 n=20+20)
ArchiverSaveFileLarge-8     304ms ± 3%     301ms ± 4%    ~     (p=0.265 n=18+18)

name                     old speed      new speed      delta
ArchiverSaveFileSmall-8  1.04MB/s ± 6%  1.05MB/s ± 6%    ~     (p=0.803 n=20+20)
ArchiverSaveFileLarge-8   142MB/s ± 3%   143MB/s ± 4%    ~     (p=0.421 n=18+19)

name                     old alloc/op   new alloc/op   delta
ArchiverSaveFileSmall-8    17.9MB ± 0%    17.9MB ± 0%  -0.01%  (p=0.000 n=19+19)
ArchiverSaveFileLarge-8     382MB ± 2%     382MB ± 1%    ~     (p=0.687 n=20+19)

name                     old allocs/op  new allocs/op  delta
ArchiverSaveFileSmall-8       540 ± 1%       528 ± 0%  -2.19%  (p=0.000 n=19+19)
ArchiverSaveFileLarge-8     1.93k ± 3%     1.79k ± 4%  -7.06%  (p=0.000 n=20+20)
 2022-05-27 12:26:37 +02:00
Alexander Neumann
bc27c370e7 Update gopkg.in/yaml 
This fixes a panic in invalid input, but I think we aren't affected.
 2022-05-26 14:23:49 +02:00
Alexander Neumann
9e30152f3c
Merge pull request #3770 from lbausch/update-minio-go 
Update github.com/minio/minio-go/v7 to v7.0.27
 2022-05-26 14:21:19 +02:00
Lorenz Bausch
cc3f8d3732
Update github.com/minio/minio-go/v7 to v7.0.27 
This version adds support for Cloudflare R2, as discussed in #3757
 2022-05-26 13:05:13 +02:00
Michael Eischer
c8e1ac4049 prune: Don't print stack trace if snapshot can't be loaded 2022-05-23 22:38:45 +02:00
Michael Eischer
173695104c prune: Fix crash on empty snapshot 2022-05-23 22:32:59 +02:00
MichaelEischer
ded783dd61
Merge pull request #3762 from marigbede/marigbede-gs-documentation-fix 
Update 030_preparing_a_new_repo.rst
 2022-05-23 22:10:38 +02:00
Arigbede Moses
2aa75d6272
Update 030_preparing_a_new_repo.rst 2022-05-18 22:03:59 +01:00
MichaelEischer
88a8701fb5
Merge pull request #3734 from lbausch/validate-patterns 
Validate exclude patterns
 2022-05-14 16:20:15 +02:00
MichaelEischer
b2a2e5f727
Merge pull request #3753 from greatroar/indexmap-alloc 
repository: Re-tune indexmap allocation strategy
 2022-05-14 15:44:08 +02:00