Commit Graph

6378 Commits

Author SHA1 Message Date
Michael Eischer 4ccd5e806b backend: split layout code into own subpackage 2022-10-21 21:36:05 +02:00
Michael Eischer b361284f28
Merge pull request #3979 from MichaelEischer/backup-less-time-now
backup: reduce calls to time.Now
2022-10-21 21:33:34 +02:00
Michael Eischer 8e2695be0b
Merge pull request #3973 from MichaelEischer/speedup-integration-tests
speed-up integration tests by reducing the RetryBackend timeout
2022-10-21 21:17:35 +02:00
Michael Eischer 35d968bcde
Merge pull request #3969 from MichaelEischer/key-by-id
Port restic.Find to return IDs and identify keys by restic.ID
2022-10-21 21:15:40 +02:00
Michael Eischer 4133fee6f9
Merge pull request #3972 from MichaelEischer/fix-flaky-lock-cancel-test
lock: fix flaky TestLockFailedRefresh
2022-10-21 21:12:34 +02:00
Michael Eischer c8c8391b21
Merge pull request #3974 from greatroar/cleanup
More cleanups and a micro-optimization
2022-10-21 21:11:37 +02:00
Michael Eischer ee7c28f5e6 backup: reduce calls to time.Now
Archiver.Save queries the current time multiple times. This commit
removes one of these calls as they showed up while profiling a backup of
a nearly unchanged dataset containing 3 million files.
2022-10-21 20:55:01 +02:00
greatroar 9adae5521d cache: Call interface method once 2022-10-21 14:32:46 +02:00
greatroar b513597546 internal/restic: Make FileType a uint8 instead of a string
The string form was presumably useful before the introduction of
layouts, but right now it just makes call sequences and garbage
collection more expensive (the latter because every string contains
a pointer to be scanned).
2022-10-16 10:59:01 +02:00
greatroar 22147e1e02 all: Minor cleanups
if x { return true } return false => return x

	fmt.Sprintf("%v", x) => fmt.Sprint(x) or x.String()

The fmt.Sprintf idiom is still used in the SecretString tests, where it
serves security hardening.
2022-10-16 10:50:39 +02:00
greatroar d03460010f internal/restic: Fix ID.UnmarshalJSON, ParseID
ID.UnmarshalJSON accepted non-JSON input with ' as the string delimiter.
Also, the error message for non-hex input was less informative than it
could be and it performed too many checks.

Changed ParseID to keep the error messages consistent.
2022-10-16 10:39:52 +02:00
Michael Eischer aa39bf3cf6 backend/test: remove duplicate test
The test is identical to the tests for the mem backend.
2022-10-15 23:15:07 +02:00
Michael Eischer 28e1c4574b mem: use cheaper hash for backend 2022-10-15 23:14:33 +02:00
Michael Eischer c3400d3c55 backend: speedup RetryBackend tests 2022-10-15 23:13:44 +02:00
Michael Eischer 99547518cd lock: fix flaky TestLockFailedRefresh
The comparison of the current time and the last lock refresh were using
seconds represented as integers. As the test only waits for up to one
second, the associated number truncation can cause the test to take
longer than once second and thus to fail.

Switch to nanoseconds to avoid this problem. This also slightly speeds
up the test.
2022-10-15 22:36:32 +02:00
Michael Eischer e10420553b speed-up integration tests by reducing the RetryBackend timeout
On my machine this decreases the runtime for `./cmd/restic` from 9.5s to
6.5s.
2022-10-15 22:29:58 +02:00
Michael Eischer 367f35db27
Merge pull request #3968 from MichaelEischer/cleanup-complete-blob
backup: Remove unused filename parameter from CompleteBlob callback
2022-10-15 16:11:16 +02:00
Michael Eischer 8d62a7adb4 identify keys by ID and not name 2022-10-15 16:07:43 +02:00
Michael Eischer 02634dce7a restic: change Find to return ids
That way consumers no longer have to manually convert the returned name
to an id.
2022-10-15 16:06:54 +02:00
Michael Eischer 964977677f backup: Remove unused filename parameter from CompleteBlob callback 2022-10-15 15:21:17 +02:00
Michael Eischer 258b487d8f
Merge pull request #3951 from MichaelEischer/rework-snapshot-filter
Rework snapshot filtering
2022-10-15 14:47:47 +02:00
Michael Eischer de9bc031df add changelog for ls handling of missing snapshots 2022-10-15 13:34:50 +02:00
Michael Eischer 246d3032ae restic: Don't list snapshots if FindSnapshot gets full id 2022-10-15 13:34:34 +02:00
Michael Eischer d8c00b9726 add comment 2022-10-15 13:34:21 +02:00
Michael Eischer a3113c6097 restic: Change FindSnapshot functions to return the snapshot 2022-10-15 13:34:04 +02:00
Michael Eischer b50f48594d restic: cleanup arguments of findLatestSnapshot 2022-10-15 13:33:48 +02:00
Michael Eischer 61e827ae4f restic: hide findLatestSnapshot 2022-10-15 13:33:32 +02:00
Michael Eischer fcad5e6f5d backup: use unified FindFilteredSnapshot 2022-10-15 13:33:29 +02:00
Michael Eischer 0aa73bbd39 ls: proper error handling for non-existent snapshot
Use restic.FindFilteredSnapshot to resolve the snapshot ID. This ensures
consistent behavior for all commands using initSingleSnapshotFilterOptions.
2022-10-15 13:32:00 +02:00
Michael Eischer a81f0432e9 restic: Add unified method to resolve a single snapshot 2022-10-15 13:31:45 +02:00
Michael Eischer 95a1bb4261 restic: Rework error handling of FindFilteredSnapshots and handle snapshotIDs
FindFilteredSnapshots no longer prints errors during snapshot loading on
stderr, but instead passes the error to the callback to allow the caller
to decide on what to do.

In addition, it moves the logic to handle an explicit snapshot list from
the main package to restic.
2022-10-15 13:31:26 +02:00
Michael Eischer cff22a5f01 dump: use correct help text for filter options 2022-10-15 13:31:10 +02:00
Michael Eischer 7a6dcb4831
Merge pull request #3966 from MichaelEischer/cleanup-walker-test
walker: Convert tests to use TreeJSONBuilder
2022-10-15 11:25:11 +02:00
Michael Eischer 7cf042118f walker: Convert tests to use TreeJSONBuilder
The old code marshalled the tree blobs different than other places in
restic. The hashed tree blob did not contain a final newline character.
2022-10-15 11:04:13 +02:00
Michael Eischer cea7191995
Merge pull request #3959 from MichaelEischer/buffered-backup-progress
backup: Use buffered channels to collect backup status
2022-10-15 10:57:19 +02:00
Michael Eischer ba688aad20
Merge pull request #3961 from greatroar/cleanup
Misc. cleanup
2022-10-14 21:49:35 +02:00
Michael Eischer 9c290a8093
Merge pull request #3960 from greatroar/errors
errors: Drop WithMessage
2022-10-14 21:41:28 +02:00
greatroar 0e155fd9a6 internal/restic: Fix UID/GID parsing
The helper function uidGidInt used strconv.ParseInt instead of
ParseUint, so it silently ignored some invalid user/group IDs.

Also, improve the error message. "Invalid UID" is more informative than
having "ParseInt" twice (*strconv.NumError displays the function name).

Finally, the user.User struct can be passed by pointer to get reduce
code size.
2022-10-14 18:21:00 +02:00
greatroar e0b743c64d internal/restic: Remove unused ID.EqualString 2022-10-14 18:20:11 +02:00
greatroar 6922360179 ui/backup: Remove unused ProgressReporter type, Progress field 2022-10-14 14:36:19 +02:00
greatroar d4aadfa389 all: Drop ctxhttp
This package is no longer needed, since we can use the stdlib's
http.NewRequestWithContext.

backend/rclone already did, but it needed a different error check due to
a difference between net/http and ctxhttp.

Also, store the http.Client by value in the REST backend (changed to a
pointer when ctxhttp was introduced) and use errors.WithStack instead
of errors.Wrap where the message was no longer accurate. Errors from
http.NewRequestWithContext will start with "net/http" or "net/url", so
they're easy to identify.
2022-10-14 14:33:49 +02:00
greatroar 16849d5361 internal/archiver: Missing argument to errors.Errorf 2022-10-14 14:18:52 +02:00
greatroar 09c14f33c8 internal/checker: Pass Error.Error pointer receiver 2022-10-14 14:13:32 +02:00
greatroar feb790f497 internal/restic: Use errors.New when no formatting is needed 2022-10-14 14:07:20 +02:00
greatroar ba44666704 errors: Drop WithMessage 2022-10-14 14:06:47 +02:00
Michael Eischer 1a6160d152
Merge pull request #3880 from MichaelEischer/archiver-savedir-cleanup
archiver: Improve handling of "file xxx already present" error
2022-10-08 21:48:14 +02:00
Michael Eischer 21b1d7a880
Merge pull request #3948 from MichaelEischer/split-index
repository: split index into a separate package
2022-10-08 21:41:57 +02:00
Michael Eischer 5278ab51c8 archiver: Check that duplicates are only ignored if identical 2022-10-08 21:38:36 +02:00
Michael Eischer 403b01b788 backup: Only return a warning for duplicate directory entries
The backup command failed if a directory contains duplicate entries.
Downgrade the severity of this problem from fatal error to a warning.
This allows users to still create a backup.
2022-10-08 21:38:21 +02:00
Michael Eischer d7d7b4ab27 archiver: refactor TreeSaverTest 2022-10-08 21:29:32 +02:00