50053a85d3
rest: Adjust http2 missing eof test to golang >= 1.17.3, >= 1.16.10
...
The missing eof with http2 when a response included a content-length
header but no data, has been fixed in golang 1.17.3/1.16.10. Therefore
just drop the canary test and schedule it for removal once go 1.18 is
required as minimum version by restic.
2021-11-13 21:57:30 +01:00
f1cfb97237
Merge pull request #3514 from phcreery/rclone_timeout
...
rclone: extend timeout from 60s to 240s
2021-11-07 18:10:34 +01:00
cb81ee9396
Merge pull request #3474 from kitone/fix-issue-3382
...
Honor RESTIC_CACHE_DIR environment variable
2021-11-07 17:57:54 +01:00
b0e64deb27
rclone: Fix timeout calculation
2021-11-07 17:49:33 +01:00
43d173b042
rclone: add timeout option and documentation
2021-11-07 17:49:21 +01:00
15d7313387
local: Fix fd leak when encountering files directly inside data/
2021-11-06 19:44:57 +01:00
78c7dd53ef
Merge pull request #3526 from greatroar/dump-refactor
...
Refactor internal/dump + concurrent load/write
2021-11-05 22:38:39 +01:00
1b23675f21
cache --cleanup should handle directories created by restic check.
...
Because there is no guarantee that a cleanup of these directories will occur
after the "restic check", we extend the behavior to detect and manage these
specific cache directories and allow their cleanup too.
2021-11-04 15:10:38 +01:00
c71729dfc4
Refactor internal/dump + concurrent load/write
...
Package internal/dump has been reworked so its API consists of a single
type Dumper that handles tar and zip formats. Tree loading and node
writing happen concurrently.
2021-11-01 23:01:55 +01:00
fb5d9345a7
Merge pull request #3510 from MichaelEischer/fix-archiver-early-on-abort
...
archiver: Fix TestArchiverAbortEarlyOnError test
2021-10-16 15:37:41 +02:00
95eb859b54
Honor RESTIC_CACHE_DIR environment variable
...
Fix #3382 : restic check doesn't obey the RESTIC_CACHE_DIR environment variable
2021-10-10 16:00:02 +02:00
f2a3b3b4a1
Merge pull request #3537 from greatroar/dump-lru
...
Check cap instead of len in bloblru
2021-10-09 20:00:27 +02:00
708d7a2574
s3: Add warning if key ID or secret is empty
...
Also add debug message if no credential types are available.
Closes #2388
2021-10-06 23:13:40 +01:00
634a9c162d
Check cap instead of len in bloblru
...
restic dump uses bloblru.Cache to keep buffers alive, but also reuses
evicted buffers. That means large buffers may be used to store small
blobs, causing the cache to think it's using less memory than it
actually does.
2021-10-03 09:34:17 +02:00
c892c0bab9
internal/restic: Don't allocate in Tree.Insert
...
name old time/op new time/op delta
BuildTree-8 34.6µs ± 4% 7.0µs ± 3% -79.68% (p=0.000 n=18+19)
name old alloc/op new alloc/op delta
BuildTree-8 34.0kB ± 0% 0.9kB ± 0% -97.37% (p=0.000 n=20+20)
name old allocs/op new allocs/op delta
BuildTree-8 108 ± 0% 1 ± 0% -99.07% (p=0.000 n=20+20)
2021-09-26 18:08:48 +02:00
78dac2fd48
Merge pull request #3523 from greatroar/swift-v2
...
Upgrade ncw/swift to v2
2021-09-24 21:41:36 +02:00
5ea8bba1a1
swift: restore context err check for list()
2021-09-24 21:19:46 +02:00
a5e103a212
Merge pull request #3522 from greatroar/dump-lru
...
Use LRU cache in restic dump
2021-09-24 20:33:58 +02:00
e7ec0453b1
Upgrade ncw/swift to v2
2021-09-24 19:08:37 +02:00
fe04d024c7
Use LRU cache in restic dump
2021-09-24 15:45:08 +02:00
718966a81a
Move Blobcache into dedicated internal package
2021-09-24 15:45:00 +02:00
4f33eca634
Remove unused Writer arg to internal/dump.writeDump
2021-09-24 15:40:42 +02:00
897d8e662c
Add --insecure-tls flag to disable SSL cert verification
...
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
2021-09-21 10:52:40 -04:00
1827b16ade
Merge pull request #3519 from greatroar/maphash
...
Replace siphash by hash/maphash
2021-09-19 19:46:03 +02:00
8d2996eaaa
Replace siphash by hash/maphash
...
In Go 1.17.1, maphash has become quite a bit faster than siphash, so we
can drop one third-party dependency. maphash is just an interface to the
standard Go map's hash function, which we already trust for other use
cases.
Benchmark results on linux/amd64, -benchtime=3s:
name old time/op new time/op delta
IndexHasUnknown-8 50.6ns ±10% 41.0ns ±19% -18.92% (p=0.000 n=9+10)
IndexHasKnown-8 52.6ns ±12% 41.5ns ±12% -21.13% (p=0.000 n=9+10)
IndexMapHash-8 3.64µs ± 1% 2.00µs ± 0% -45.09% (p=0.000 n=10+9)
IndexAlloc-8 700ms ± 1% 601ms ± 6% -14.18% (p=0.000 n=8+10)
IndexAllocParallel-8 205ms ± 5% 192ms ± 8% -6.18% (p=0.043 n=10+10)
MasterIndexAlloc-8 319ms ± 1% 279ms ± 5% -12.58% (p=0.000 n=10+10)
MasterIndexLookupSingleIndex-8 156ns ± 8% 147ns ± 6% -5.46% (p=0.023 n=10+10)
MasterIndexLookupMultipleIndex-8 150ns ± 7% 142ns ± 8% -5.69% (p=0.007 n=10+10)
MasterIndexLookupSingleIndexUnknown-8 74.4ns ± 6% 72.0ns ± 9% ~ (p=0.175 n=10+9)
MasterIndexLookupMultipleIndexUnknown-8 67.4ns ± 9% 65.5ns ± 7% ~ (p=0.340 n=9+9)
MasterIndexLookupParallel/known,indices=25-8 461ns ± 2% 445ns ± 2% -3.49% (p=0.000 n=10+10)
MasterIndexLookupParallel/unknown,indices=25-8 408ns ±11% 378ns ± 5% -7.22% (p=0.035 n=10+9)
MasterIndexLookupParallel/known,indices=50-8 479ns ± 1% 437ns ± 4% -8.82% (p=0.000 n=10+10)
MasterIndexLookupParallel/unknown,indices=50-8 406ns ± 8% 343ns ±15% -15.44% (p=0.001 n=10+10)
MasterIndexLookupParallel/known,indices=100-8 480ns ± 1% 455ns ± 5% -5.15% (p=0.000 n=8+10)
MasterIndexLookupParallel/unknown,indices=100-8 391ns ±18% 382ns ± 8% ~ (p=0.315 n=10+10)
MasterIndexLookupBlobSize-8 71.0ns ± 8% 57.2ns ±11% -19.36% (p=0.000 n=9+10)
PackerManager-8 254ms ± 1% 254ms ± 1% ~ (p=0.285 n=15+15)
name old speed new speed delta
IndexMapHash-8 1.12GB/s ± 1% 2.05GB/s ± 0% +82.13% (p=0.000 n=10+9)
PackerManager-8 208MB/s ± 1% 207MB/s ± 1% ~ (p=0.281 n=15+15)
name old alloc/op new alloc/op delta
IndexMapHash-8 0.00B 0.00B ~ (all equal)
IndexAlloc-8 400MB ± 0% 400MB ± 0% ~ (p=1.000 n=9+10)
IndexAllocParallel-8 401MB ± 0% 401MB ± 0% +0.00% (p=0.000 n=10+10)
MasterIndexAlloc-8 258MB ± 0% 262MB ± 0% +1.42% (p=0.000 n=9+10)
PackerManager-8 73.1kB ± 0% 73.1kB ± 0% ~ (p=0.382 n=13+13)
name old allocs/op new allocs/op delta
IndexMapHash-8 0.00 0.00 ~ (all equal)
IndexAlloc-8 907k ± 0% 907k ± 0% -0.00% (p=0.000 n=10+10)
IndexAllocParallel-8 907k ± 0% 907k ± 0% +0.00% (p=0.009 n=10+10)
MasterIndexAlloc-8 327k ± 0% 317k ± 0% -3.06% (p=0.000 n=10+10)
PackerManager-8 744 ± 0% 744 ± 0% ~ (all equal)
2021-09-19 16:05:18 +02:00
58efe21eca
Merge pull request #3264 from amozoss/upstream-master
...
Refactor backup progress
2021-09-19 14:54:42 +02:00
921e328b56
restore: Fix linting error
2021-09-19 14:41:07 +02:00
2cdc0719af
restorer: Sanitize verify errors
2021-09-19 14:01:26 +02:00
bdcdfaf6b4
restore --verify: buffer reuse consistency and comment
2021-09-19 13:11:27 +02:00
2b94742ca5
Replace no-op closures in restorer by nil check
2021-09-19 13:11:16 +02:00
d357744104
Handle canceled context in restore --verify properly
2021-09-19 13:11:05 +02:00
d4225ec803
Simplify buffer growing in Restorer.verifyFile
...
Suggested-by: Igor Fedorenko <igor@ifedorenko.com>
2021-09-19 13:10:55 +02:00
de8521ae56
Report number of successes from Restorer.VerifyFiles
...
Previously, the number of attempts was reported.
2021-09-19 13:10:44 +02:00
bb066cf7d3
Concurrent Restorer.VerifyFiles
...
Time to verify a 2GB snapshot down from 9.726s to 4.645s (-52%).
2021-09-19 13:10:41 +02:00
556424d61b
Reuse buffer in Restorer.VerifyFiles
...
Time to verify a 2GB snapshot down from 11.568s to 9.726s (-16%).
2021-09-19 12:55:31 +02:00
973fa921cb
Test and document Restorer.VerifyFiles
2021-09-19 12:52:11 +02:00
e0d615c264
archiver: Fix TestArchiverAbortEarlyOnError test
...
This can be caused when the test has uploaded four blobs, then queues
two blobs for upload which are delayed. Then a seventh file can be
opened which lead to a test failure.
2021-09-12 22:17:17 +02:00
fbb0e6499a
ui: consolidate backup ui in ui/backup package
2021-09-12 16:20:15 +02:00
cccb0d4064
ui: assert that ProgressPrinter interface is implemented
2021-09-12 15:26:40 +02:00
d62bfed65d
ui: move SetDryRun to ProgressReporter
2021-09-12 15:25:58 +02:00
77b129ec74
ui: remove unused Summary field
2021-09-12 15:25:58 +02:00
3024239e40
ui/json: restore summary output
2021-09-12 15:25:58 +02:00
5ccf583b8a
ui: restore a few comments
2021-09-12 15:25:58 +02:00
80cbaf6d38
ui: Simplify ReportTotal methods
2021-09-12 15:25:58 +02:00
448419990c
Refactor backup progress
...
Move the shared logic into the progress
Allows logic to be shared with forth coming restore status
2021-09-12 15:25:58 +02:00
fa5ca8af81
Merge pull request #2856 from aawsome/remove-readahead
...
Simplify cache logic
2021-09-04 20:25:49 +02:00
b45d88e124
Merge pull request #3496 from juergenhoetzel/imporove-snapshot-filter-warning-message
...
Improve snapshot filter warning message
2021-09-03 23:42:27 +02:00
a29777f467
Merge pull request #3501 from greatroar/printprogress
...
Streamline progress printing in cmd/restic
2021-09-03 23:34:36 +02:00
81876d5c1b
Simplify cache logic
2021-09-03 21:01:00 +02:00
5aaa3e93c1
internal/ui/termstatus: Optimize and publish Truncate
...
name old time/op new time/op delta
TruncateASCII-8 347ns ± 1% 69ns ± 1% -80.02% (p=0.000 n=9+10)
TruncateUnicode-8 447ns ± 3% 348ns ± 1% -22.04% (p=0.000 n=10+10)
2021-09-03 11:48:22 +02:00
Michael Eischer
phcreery
kitone
greatroar
Alexander Neumann
Garry McNulty
greatroar
Uli Martens
Sam Lucidi
Dan Willoughby
Alexander Weiss