Michael Eischer
c0f34af9db
backup: hide files from status which are read completely but not saved
...
As the FileSaver is asynchronously waiting for all blobs of a file to be
stored, the number of active files is higher than the number of files
from which restic is reading concurrently. Thus to not confuse users,
only display files in the status from which restic is currently reading.
2022-10-30 10:29:12 +01:00
Michael Eischer
a571fc4aa1
add changelog for faster backups with small files
2022-10-30 10:29:12 +01:00
Michael Eischer
b52a8ff05c
ui: Properly clear lines no longer used for status
...
Previously, the old status text remained until it was overwritten.
2022-10-30 10:29:12 +01:00
Michael Eischer
b4de902596
archiver: Asynchronously complete FutureFile
...
After reading and chunking all data in a file, the FutureFile still has
to wait until the FutureBlobs are completed. This was done synchronously
which results in blocking the file saver and prevents the next file from
being read.
By replacing the FutureBlob with a callback, it becomes possible to
complete the FutureFile asynchronously.
2022-10-30 10:29:11 +01:00
Michael Eischer
47e05080a9
Merge pull request #3990 from MichaelEischer/lock-refresh-test
...
lock: add test to check that refreshing works
2022-10-30 10:15:44 +01:00
Michael Eischer
c7ace314f6
Merge pull request #3989 from greatroar/eachbypack
...
More compact data structure for Index.EachByPack
2022-10-30 00:02:55 +02:00
greatroar
0e8893dae9
index: Compact data structure for Index.EachByPack
2022-10-29 23:09:17 +02:00
greatroar
137f0bc944
repository: Fix benchmarkSaveAndEncrypt
2022-10-29 23:09:17 +02:00
Michael Eischer
01f0db4e56
return error if RESTIC_COMPRESSION env variable is invalid
2022-10-29 22:03:39 +02:00
Michael Eischer
7c87fb941c
Merge pull request #3986 from greatroar/counter
...
ui/progress: Load both values in a single Lock/Unlock
2022-10-29 21:50:55 +02:00
Michael Eischer
3b0bb02a68
Merge pull request #3977 from greatroar/progress
...
ui/backup: Replace channels with a mutex
2022-10-29 21:33:04 +02:00
rawtaz
af3f7c866f
Merge pull request #3988 from FelixBurkhard/FelixBurkhard-patch-1
...
Clearify what Azure account name means
2022-10-29 13:32:44 +02:00
Michael Eischer
24267e9a9d
lock: add test to check that refreshing works
2022-10-29 11:26:00 +02:00
Michael Eischer
8e51e1e605
shorten 'repository opened' output
2022-10-29 11:22:00 +02:00
FelixBurkhard
575d26ec87
Clearify what Azure account name means
...
When reading it first it was not clear to me the 'account name' meant the name of the
Azure Storage Account and not the Azure account itself.
2022-10-29 00:27:43 +02:00
greatroar
2dafda9164
ui/progress: Load both values in a single Lock/Unlock
...
We always need both values, except in a test, so we don't need to lock
twice and risk scheduling in between.
Also, removed the resetting in Done. This copied a mutex, which isn't
allowed. Static analyzers tend to trip over that.
2022-10-25 07:55:24 +02:00
Michael Eischer
f8910bc4ff
Merge pull request #3985 from saltsa/fix_lock_refresh
...
Fix bug in lock refresh monitoring
2022-10-24 22:59:18 +02:00
Joonas Aunola
b06427c9f6
fix Unix to UnixNano
2022-10-23 23:40:21 +03:00
greatroar
006380199e
cmd, ui: Deduplicate formatting utilities
2022-10-23 13:40:07 +02:00
greatroar
04216eb9aa
ui/backup: Replace channels with a mutex
...
The channel-based algorithm had grown quite complicated. This is easier
to reason about and likely to be more performant with very many
CompleteBlob calls.
2022-10-23 13:28:41 +02:00
Michael Eischer
4fea3a413d
show selected compression level when opening repository
2022-10-22 20:18:46 +02:00
Michael Eischer
b57d42905c
Merge pull request #3899 from MichaelEischer/less-prune-mem
...
Optimize prune memory usage
2022-10-22 18:56:02 +02:00
Michael Eischer
d966c52707
prune: allow gc of set of repacked blobs before index rebuild
2022-10-22 18:45:12 +02:00
Michael Eischer
1e2794fa55
add prune memory optimization changelog
2022-10-22 18:45:12 +02:00
Michael Eischer
68c9cb9c6a
prune: Shrink keepBlobs set if possible
...
As long as only a small fraction of the data in a repository is
rewritten, the keepBlobs set will be rather small after cleaning it up.
As golang maps do not shrink their memory usage, just copy the contents
over to a new map. However, only copy the map if the cleanup removed at
least half the entries.
2022-10-22 18:45:12 +02:00
Michael Eischer
c4fc5c97f9
prune: Use a single CountedBlobSet to track blobs
...
The set covers necessary, existing and duplicate blobs. This removes the
duplicate sets used to track whether all necessary blobs also exist.
This reduces the memory usage of prune by about 20-30%.
2022-10-22 18:45:12 +02:00
Michael Eischer
b21241ec1c
restic: Add CountedBlobSet type
...
This allows maintaining a usage counter for each blob.
2022-10-22 18:45:12 +02:00
Michael Eischer
ee6688a9f6
Merge pull request #3915 from plumbeo/compression-stats
...
restic stats: print uncompressed size in mode raw-data
2022-10-21 22:10:29 +02:00
Michael Eischer
27634a1a68
Merge pull request #3978 from MichaelEischer/fix-negative-pattern-example
...
Remove misleading wildcard from negative exclude pattern example
2022-10-21 22:04:30 +02:00
Michael Eischer
aa77702e49
Merge pull request #3971 from MichaelEischer/parallel-list
...
Unify ForAllIndex/Snapshot/Lock functions
2022-10-21 21:58:33 +02:00
Michael Eischer
6877aaa8aa
Merge pull request #3967 from MichaelEischer/archiver-extract-exclude-options
...
backup: extract exclude pattern options
2022-10-21 21:50:00 +02:00
Michael Eischer
2e9ee8577a
Merge pull request #3970 from MichaelEischer/split-retry-backend
...
Split backend package into smaller parts
2022-10-21 21:49:46 +02:00
Michael Eischer
59d46bb3f5
backup: extract exclude pattern options
...
This is a preparation to make the exclude options usable for the
upcoming `rewrite` command.
2022-10-21 21:40:59 +02:00
Michael Eischer
5c7a9a739a
backend: Split RetryBackend into own package
...
The RetryBackend tests depend on the mock backend. When the Backend
interface is eventually split from the restic package, this will lead to
a dependency cycle between backend and backend/mock. Thus split the
RetryBackend into a separate package to avoid this problem.
2022-10-21 21:38:17 +02:00
Michael Eischer
32603d49c4
backend: remove unused ErrorBackend
2022-10-21 21:36:05 +02:00
Michael Eischer
8c18c65b3b
backend: remove unused Paths variable
2022-10-21 21:36:05 +02:00
Michael Eischer
4ccd5e806b
backend: split layout code into own subpackage
2022-10-21 21:36:05 +02:00
Michael Eischer
b361284f28
Merge pull request #3979 from MichaelEischer/backup-less-time-now
...
backup: reduce calls to time.Now
2022-10-21 21:33:34 +02:00
Michael Eischer
738b2a0445
parallelize more List usages
2022-10-21 21:26:45 +02:00
Michael Eischer
ae45f3b04f
restic: Unify code to load Index/Lock/Snapshot
2022-10-21 21:25:11 +02:00
Michael Eischer
8e2695be0b
Merge pull request #3973 from MichaelEischer/speedup-integration-tests
...
speed-up integration tests by reducing the RetryBackend timeout
2022-10-21 21:17:35 +02:00
Michael Eischer
35d968bcde
Merge pull request #3969 from MichaelEischer/key-by-id
...
Port restic.Find to return IDs and identify keys by restic.ID
2022-10-21 21:15:40 +02:00
Michael Eischer
4133fee6f9
Merge pull request #3972 from MichaelEischer/fix-flaky-lock-cancel-test
...
lock: fix flaky TestLockFailedRefresh
2022-10-21 21:12:34 +02:00
Michael Eischer
c8c8391b21
Merge pull request #3974 from greatroar/cleanup
...
More cleanups and a micro-optimization
2022-10-21 21:11:37 +02:00
Michael Eischer
ee7c28f5e6
backup: reduce calls to time.Now
...
Archiver.Save queries the current time multiple times. This commit
removes one of these calls as they showed up while profiling a backup of
a nearly unchanged dataset containing 3 million files.
2022-10-21 20:55:01 +02:00
Michael Eischer
3e60d38a23
Remove misleading wildcard from negative exclude pattern example
...
There is no need to use a special wildcard `**` to demonstrate negative
patterns. Actually, it is both slower than the simpler variant and seems
to confuse users.
2022-10-21 20:48:45 +02:00
greatroar
9adae5521d
cache: Call interface method once
2022-10-21 14:32:46 +02:00
greatroar
201e5c7e74
backup: Clean up progress reporting code
2022-10-21 13:48:30 +02:00
plumbeo
a6f83e0011
Add changelog
2022-10-17 15:38:42 +02:00
plumbeo
bc945d0bf0
restic stats: add more compression statistics
...
Calculate and display compression ratio, space saving and progress
2022-10-17 15:38:38 +02:00
