Michael Eischer
20d8eed400
repository: streamPack: separate requests for gap larger than 1MB
...
With most cloud providers, traffic is much more expensive than API
calls. Thus slightly bias streamPack towards a bit more API calls in
exchange for slightly less traffic.
2024-04-22 21:21:23 +02:00
Michael Eischer
cf700d8794
repository: streamPack: reuse zstd decoder
2024-04-22 21:21:23 +02:00
Michael Eischer
666a0b0bdb
repository: streamPack: replace streaming with chunked download
...
Due to the interface of streamPack, we cannot guarantee that operations
progress fast enough that the underlying connections remains open. This
introduces partial failures which massively complicate the error
handling.
Switch to a simpler approach that retrieves the pack in chunks of 32MB.
If a blob is larger than this limit, then it is downloaded separately.
To avoid multiple copies in memory, an auxiliary interface
`discardReader` is introduced that allows directly accessing the
downloaded byte slices, while still supporting the streaming used by the
`check` command.
2024-04-22 21:21:23 +02:00
Michael Eischer
621012dac0
repository: Add blob loading fallback to LoadBlobsFromPack
...
Try to retrieve individual blobs via LoadBlob if streaming did not work.
2024-04-21 21:35:55 +02:00
Michael Eischer
228b35f074
Merge pull request #4769 from will-ca/patch-1
...
Tiny wording clarification in `restic-stats.1`.
2024-04-18 19:00:39 +00:00
will-ca
6aced61c72
Tiny docs wording clarification.
2024-04-18 07:29:55 +00:00
Michael Eischer
4d22412e0c
Merge pull request #4766 from coderwander/master
...
Fix struct names
2024-04-18 06:18:19 +00:00
coderwander
a82ed71de7
Fix struct names
...
Signed-off-by: coderwander <770732124@qq.com>
2024-04-18 10:02:09 +08:00
Michael Eischer
2173c69280
Merge pull request #4770 from testwill/close_files
...
fix: close files
2024-04-17 16:50:20 +00:00
Michael Eischer
001bb71676
repair packs: Properly close backup files
2024-04-17 18:32:30 +02:00
Michael Eischer
55d56db31b
Merge pull request #4743 from MichaelEischer/deprecate-s3legacy-layout
...
Deprecate s3legacy layout
2024-04-11 22:09:34 +02:00
Michael Eischer
591b421c4a
Deprecate s3legacy layout
2024-04-10 21:27:56 +02:00
Michael Eischer
8efc3a8b7d
Merge pull request #4668 from MichaelEischer/backup-xattr-parent-enoperm
...
backup: Ignore xattr.list permission error for parent directories
2024-04-10 21:25:28 +02:00
Michael Eischer
bf054c09d2
backup: Ignore xattr.list permission error for parent directories
...
On FreeBSD, limited users may not be able to even list xattrs for the
parent directories above the snapshot source paths. As this can cause
the backup to fail, just ignore those errors.
2024-04-10 20:46:15 +02:00
Michael Eischer
6091029fd6
Merge pull request #4756 from mgeisler/patch-1
...
doc: fix typo in 047_tuning_backup_parameters.rst
2024-04-07 19:24:09 +00:00
Martin Geisler
09d2183351
doc: fix typo in 047_tuning_backup_parameters.rst
2024-04-07 18:05:53 +02:00
Michael Eischer
a4b7ebecfc
Merge pull request #4750 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.40.0
...
build(deps): bump cloud.google.com/go/storage from 1.39.0 to 1.40.0
2024-04-03 20:10:41 +00:00
dependabot[bot]
ba136b31b8
build(deps): bump cloud.google.com/go/storage from 1.39.0 to 1.40.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.39.0 to 1.40.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.39.0...spanner/v1.40.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-03 20:03:22 +00:00
Michael Eischer
f328111a6e
Merge pull request #4751 from restic/dependabot/go_modules/golang.org/x/net-0.23.0
...
build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0
2024-04-03 19:52:29 +00:00
Michael Eischer
9fb017e67a
Merge pull request #4745 from MichaelEischer/full-id-key-list
...
key list: include full key id in JSON output
2024-04-03 21:50:02 +02:00
Michael Eischer
49f98f25fc
Merge pull request #4742 from MichaelEischer/consistent-rtest-import
...
Use consistent alias for interal/test package
2024-04-03 21:47:32 +02:00
dependabot[bot]
96c602a6de
build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.21.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.21.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-03 19:41:38 +00:00
Michael Eischer
1d0a20dd58
Merge pull request #4748 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.10.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.9.2 to 1.10.0
2024-04-03 19:34:47 +00:00
Michael Eischer
6cca1d5705
Merge pull request #4655 from ae-govau/unixsocket
...
Enhancement: option to send HTTP over unix socket
2024-04-03 19:29:21 +00:00
dependabot[bot]
f8a72ac2a3
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.9.2 to 1.10.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.9.2...sdk/azcore/v1.10.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-01 02:32:19 +00:00
Michael Eischer
5145c8f9c0
key list: include full key id in JSON output
2024-03-31 12:25:20 +02:00
Michael Eischer
831fc4413d
Merge pull request #4737 from stephan0307/3117
...
json output forget command: added id's in snapshots within reasons object
2024-03-29 13:11:24 +00:00
Stephan Paul
df07814ec2
forget json output: added id's in snapshots within reasons object
...
In order to evaluate the keep reasons for snapshots, there should be also the id's
to compare it with snapshots within the keep object. (See also Issue #3117 )
In order to avoid output parameters also changed function addJSONSnapshots to asJSONSnapshots
2024-03-29 11:30:00 +01:00
Michael Eischer
ec2b79834a
use consistent alias for interal/test package
2024-03-29 00:24:03 +01:00
Michael Eischer
510f6f06b0
Merge pull request #4709 from MichaelEischer/refactor-locking
...
Refactor locking into repository package
2024-03-28 23:53:09 +01:00
Michael Eischer
07eb6c315b
add changelog for locking refactor
2024-03-28 23:46:58 +01:00
Michael Eischer
5e98f1e2eb
repository: fix test setup race conditions
2024-03-28 23:17:02 +01:00
Michael Eischer
8155dbe711
correctly lock repository in integration tests
2024-03-28 23:17:02 +01:00
Michael Eischer
d18726cd70
ls: add missing read lock
...
As `ls` reads data from the repository, it must acquire a read lock
unless `--no-lock` was specified. The old behavior is equivalent to `ls
--no-lock`.
2024-03-28 23:17:02 +01:00
Michael Eischer
dc441c57a7
repository: unify repository initialization in tests
...
Tests should use a helper from internal/repository/testing.go to
construct a Repository object.
2024-03-28 23:17:02 +01:00
Michael Eischer
3ba1fa3cee
repository: remove a few global variables
2024-03-28 23:17:02 +01:00
Michael Eischer
044e8bf821
repository: parallelize lock tests
2024-03-28 23:17:02 +01:00
Michael Eischer
e8df50fa3c
repository: remove global list of locks
2024-03-28 22:46:33 +01:00
Michael Eischer
cbb5f89252
lock: move code to repository package
2024-03-28 22:46:33 +01:00
Michael Eischer
118a69a84b
lock: replace lockRepo(Exclusive) with openWith(Read/Write/Exclusive)Lock
...
The new functions much better convey the intent behind the lock
request. This allows cleanly integrating noLock (for read) and dryRun
(write/exclusive) handling.
There are only minor changes to existing behavior with two exceptions:
- `tag` no longer accepts the `--no-lock` flag. As it replaces files in
the repository, this always requires an exclusive lock.
- `debug examine` now returns an error if both `--extract-pack` and
`--no-lock` are given.
2024-03-28 22:46:33 +01:00
Michael Eischer
7f9ad1c3db
Merge pull request #4705 from MichaelEischer/snapshot-statistics
...
Store snapshot statistics & print snapshot size
2024-03-28 22:41:45 +01:00
Michael Eischer
71b6284155
Merge pull request #4006 from MichaelEischer/deviceID-only-for-hardlinks
...
archiver: only store deviceID for hardlinks
2024-03-28 22:33:28 +01:00
Michael Eischer
cf81f8ced6
stats: only check for hardlinks for files with more than one link
2024-03-28 21:29:27 +01:00
Michael Eischer
21cf38fe96
add changelog for deviceID only for hardlinks
2024-03-28 19:32:50 +01:00
Michael Eischer
d705741571
backup: test that deviceID is only stored for hardlinks
2024-03-28 19:32:50 +01:00
Michael Eischer
a9b3d86c4f
features: remove example feature
2024-03-28 19:12:07 +01:00
Michael Eischer
a26d6ffa72
archiver: move deviceID handling behind feature flag
2024-03-28 19:12:07 +01:00
Michael Eischer
2ba21fe72b
archiver: only store deviceID for hardlinks
...
The deviceID can change e.g. when backing up from filesystem snapshot.
It is only used for hardlink detection. Thus there it is not necessary
to store it for everything else.
2024-03-28 19:12:07 +01:00
Michael Eischer
870904d3ae
Merge pull request #4731 from facutuesca/powershell-completion-doc
...
doc: Add instructions to configure PowerShell completions
2024-03-28 17:46:55 +00:00
Facundo Tuesca
15555c9898
doc: Use consistent case for PROFILE env variable in PowerShell
2024-03-28 18:35:18 +01:00
