2
2
mirror of https://github.com/octoleo/restic.git synced 2024-12-23 19:38:57 +00:00
restic/internal/repository
Michael Eischer 6f53ecc1ae adapt workers based on whether an operation is CPU or IO-bound
Use runtime.GOMAXPROCS(0) as worker count for CPU-bound tasks,
repo.Connections() for IO-bound task and a combination if a task can be
both. Streaming packs is treated as IO-bound as adding more worker
cannot provide a speedup.

Typical IO-bound tasks are download / uploading / deleting files.
Decoding / Encoding / Verifying are usually CPU-bound. Several tasks are
a combination of both, e.g. for combined download and decode functions.
In the latter case add both limits together. As the backends have their
own concurrency limits restic still won't download more than
repo.Connections() files in parallel, but the additional workers can
decode already downloaded data in parallel.
2022-07-03 12:19:26 +02:00
..
testdata internal/repository: Fix LoadBlob + fuzz test 2022-06-06 17:02:28 +02:00
doc.go Moves files 2017-07-23 14:19:13 +02:00
fuzz_test.go repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
index_parallel_test.go repository: Add test for ForAllIndexes 2020-12-22 22:36:18 +01:00
index_parallel.go adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00
index_test.go rebuild-index: correctly rebuild index for mixed packs 2022-07-02 19:24:02 +02:00
index.go rebuild-index: correctly rebuild index for mixed packs 2022-07-02 19:24:02 +02:00
indexmap_test.go repository: implement pack compression 2022-04-30 11:34:10 +02:00
indexmap.go repository: Re-tune indexmap allocation strategy 2022-05-11 21:22:14 +02:00
key.go Merge pull request #3798 from greatroar/errors 2022-06-17 19:01:40 +02:00
master_index_test.go repository: hide MasterIndex.FinalizeFullIndexes / FinalizeNotFinalIndexes 2022-07-02 18:39:59 +02:00
master_index.go adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00
packer_manager_test.go repository: Limit to a single pending pack file 2022-07-02 22:42:34 +02:00
packer_manager.go repository: Limit to a single pending pack file 2022-07-02 22:42:34 +02:00
packer_uploader.go repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
repack_test.go repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
repack.go adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00
repository_internal_test.go Use "pack file" instead of "data file" (#2885) 2020-08-16 11:16:38 +02:00
repository_test.go repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
repository.go adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00
testing.go repository: hide MasterIndex.FinalizeFullIndexes / FinalizeNotFinalIndexes 2022-07-02 18:39:59 +02:00