mirror of
https://github.com/octoleo/restic.git
synced 2024-11-25 22:27:35 +00:00
Generate CHANGELOG.md for 0.10.0
This commit is contained in:
Alexander Neumann
parent
96fd982f6a
commit
ee6e981b4e
474
CHANGELOG.md
474
CHANGELOG.md
@ -1,3 +1,467 @@
|
||||
Changelog for restic 0.10.0 (2020-09-19)
|
||||
=======================================
|
||||
|
||||
The following sections list the changes in restic 0.10.0 relevant to
|
||||
restic users. The changes are ordered by importance.
|
||||
|
||||
Summary
|
||||
-------
|
||||
|
||||
* Fix #1863: Report correct number of directories processed by backup
|
||||
* Fix #2254: Fix tar issues when dumping `/`
|
||||
* Fix #2281: Handle format verbs like '%' properly in `find` output
|
||||
* Fix #2298: Do not hang when run as a background job
|
||||
* Fix #2389: Fix mangled json output of backup command
|
||||
* Fix #2390: Refresh lock timestamp
|
||||
* Fix #2429: Backup --json reports total_bytes_processed as 0
|
||||
* Fix #2469: Fix incorrect bytes stats in `diff` command
|
||||
* Fix #2518: Do not crash with Synology NAS sftp server
|
||||
* Fix #2531: Fix incorrect size calculation in `stats --mode restore-size`
|
||||
* Fix #2537: Fix incorrect file counts in `stats --mode restore-size`
|
||||
* Fix #2592: SFTP backend supports IPv6 addresses
|
||||
* Fix #2607: Honor RESTIC_CACHE_DIR environment variable on Mac and Windows
|
||||
* Fix #2668: Don't abort the stats command when data blobs are missing
|
||||
* Fix #2674: Add stricter prune error checks
|
||||
* Fix #2899: Fix possible crash in the progress bar of check --read-data
|
||||
* Chg #2482: Remove vendored dependencies
|
||||
* Chg #2546: Return exit code 3 when failing to backup all source data
|
||||
* Chg #2600: Update dependencies, require Go >= 1.13
|
||||
* Chg #1597: Honor the --no-lock flag in the mount command
|
||||
* Enh #1570: Support specifying multiple host flags for various commands
|
||||
* Enh #1680: Optimize `restic mount`
|
||||
* Enh #2072: Display snapshot date when using `restic find`
|
||||
* Enh #2175: Allow specifying user and host when creating keys
|
||||
* Enh #2277: Add support for ppc64le
|
||||
* Enh #2395: Ignore sync errors when operation not supported by local filesystem
|
||||
* Enh #2427: Add flag `--iexclude-file` to backup command
|
||||
* Enh #2569: Support excluding files by their size
|
||||
* Enh #2571: Self-heal missing file parts during backup of unchanged files
|
||||
* Enh #2858: Support filtering snapshots by tag and path in the stats command
|
||||
* Enh #323: Add command for copying snapshots between repositories
|
||||
* Enh #551: Use optimized library for hash calculation of file chunks
|
||||
* Enh #2195: Simplify and improve restore performance
|
||||
* Enh #2328: Improve speed of check command
|
||||
* Enh #2423: Support user@domain parsing as user
|
||||
* Enh #2576: Improve the chunking algorithm
|
||||
* Enh #2598: Improve speed of diff command
|
||||
* Enh #2599: Slightly reduce memory usage of prune and stats commands
|
||||
* Enh #2733: S3 backend: Add support for WebIdentityTokenFile
|
||||
* Enh #2773: Optimize handling of new index entries
|
||||
* Enh #2781: Reduce memory consumption of in-memory index
|
||||
* Enh #2786: Optimize `list blobs` command
|
||||
* Enh #2790: Optimized file access in restic mount
|
||||
* Enh #2840: Speed-up file deletion in forget, prune and rebuild-index
|
||||
|
||||
Details
|
||||
-------
|
||||
|
||||
* Bugfix #1863: Report correct number of directories processed by backup
|
||||
|
||||
The directory statistics calculation was fixed to report the actual number of processed
|
||||
directories instead of always zero.
|
||||
|
||||
https://github.com/restic/restic/issues/1863
|
||||
|
||||
* Bugfix #2254: Fix tar issues when dumping `/`
|
||||
|
||||
We've fixed an issue with dumping either `/` or files on the first sublevel e.g. `/foo` to tar.
|
||||
This also fixes tar dumping issues on Windows where this issue could also happen.
|
||||
|
||||
https://github.com/restic/restic/issues/2254
|
||||
https://github.com/restic/restic/issues/2357
|
||||
https://github.com/restic/restic/pull/2255
|
||||
|
||||
* Bugfix #2281: Handle format verbs like '%' properly in `find` output
|
||||
|
||||
The JSON or "normal" output of the `find` command can now deal with file names that contain
|
||||
substrings which the Golang `fmt` package considers "format verbs" like `%s`.
|
||||
|
||||
https://github.com/restic/restic/issues/2281
|
||||
|
||||
* Bugfix #2298: Do not hang when run as a background job
|
||||
|
||||
Restic did hang on exit while restoring the terminal configuration when it was started as a
|
||||
background job, for example using `restic ... &`. This has been fixed by only restoring the
|
||||
terminal configuration when restic is interrupted while reading a password from the
|
||||
terminal.
|
||||
|
||||
https://github.com/restic/restic/issues/2298
|
||||
|
||||
* Bugfix #2389: Fix mangled json output of backup command
|
||||
|
||||
We've fixed a race condition in the json output of the backup command that could cause multiple
|
||||
lines to get mixed up. We've also ensured that the backup summary is printed last.
|
||||
|
||||
https://github.com/restic/restic/issues/2389
|
||||
https://github.com/restic/restic/pull/2545
|
||||
|
||||
* Bugfix #2390: Refresh lock timestamp
|
||||
|
||||
Long-running operations did not refresh lock timestamp, resulting in locks becoming stale.
|
||||
This is now fixed.
|
||||
|
||||
https://github.com/restic/restic/issues/2390
|
||||
|
||||
* Bugfix #2429: Backup --json reports total_bytes_processed as 0
|
||||
|
||||
We've fixed the json output of total_bytes_processed. The non-json output was already fixed
|
||||
with pull request #2138 but left the json output untouched.
|
||||
|
||||
https://github.com/restic/restic/issues/2429
|
||||
|
||||
* Bugfix #2469: Fix incorrect bytes stats in `diff` command
|
||||
|
||||
In some cases, the wrong number of bytes (e.g. 16777215.998 TiB) were reported by the `diff`
|
||||
command. This is now fixed.
|
||||
|
||||
https://github.com/restic/restic/issues/2469
|
||||
|
||||
* Bugfix #2518: Do not crash with Synology NAS sftp server
|
||||
|
||||
It was found that when restic is used to store data on an sftp server on a Synology NAS with a
|
||||
relative path (one which does not start with a slash), it may go into an endless loop trying to
|
||||
create directories on the server. We've fixed this bug by using a function in the sftp library
|
||||
instead of our own implementation.
|
||||
|
||||
The bug was discovered because the Synology sftp server behaves erratic with non-absolute
|
||||
path (e.g. `home/restic-repo`). This can be resolved by just using an absolute path instead
|
||||
(`/home/restic-repo`). We've also added a paragraph in the FAQ.
|
||||
|
||||
https://github.com/restic/restic/issues/2518
|
||||
https://github.com/restic/restic/issues/2363
|
||||
https://github.com/restic/restic/pull/2530
|
||||
|
||||
* Bugfix #2531: Fix incorrect size calculation in `stats --mode restore-size`
|
||||
|
||||
The restore-size mode of stats was counting hard-linked files as if they were independent.
|
||||
|
||||
https://github.com/restic/restic/issues/2531
|
||||
|
||||
* Bugfix #2537: Fix incorrect file counts in `stats --mode restore-size`
|
||||
|
||||
The restore-size mode of stats was failing to count empty directories and some files with hard
|
||||
links.
|
||||
|
||||
https://github.com/restic/restic/issues/2537
|
||||
|
||||
* Bugfix #2592: SFTP backend supports IPv6 addresses
|
||||
|
||||
The SFTP backend now supports IPv6 addresses natively, without relying on aliases in the
|
||||
external SSH configuration.
|
||||
|
||||
https://github.com/restic/restic/pull/2592
|
||||
|
||||
* Bugfix #2607: Honor RESTIC_CACHE_DIR environment variable on Mac and Windows
|
||||
|
||||
On Mac and Windows, the RESTIC_CACHE_DIR environment variable was ignored. This variable can
|
||||
now be used on all platforms to set the directory where restic stores caches.
|
||||
|
||||
https://github.com/restic/restic/pull/2607
|
||||
|
||||
* Bugfix #2668: Don't abort the stats command when data blobs are missing
|
||||
|
||||
Runing the stats command in the blobs-per-file mode on a repository with missing data blobs
|
||||
previously resulted in a crash.
|
||||
|
||||
https://github.com/restic/restic/pull/2668
|
||||
|
||||
* Bugfix #2674: Add stricter prune error checks
|
||||
|
||||
Additional checks were added to the prune command in order to improve resiliency to backend,
|
||||
hardware and/or networking issues. The checks now detect a few more cases where such outside
|
||||
factors could potentially cause data loss.
|
||||
|
||||
https://github.com/restic/restic/pull/2674
|
||||
|
||||
* Bugfix #2899: Fix possible crash in the progress bar of check --read-data
|
||||
|
||||
We've fixed a possible crash while displaying the progress bar for the check --read-data
|
||||
command. The crash occurred when the length of the progress bar status exceeded the terminal
|
||||
width, which only happened for very narrow terminal windows.
|
||||
|
||||
https://github.com/restic/restic/pull/2899
|
||||
https://forum.restic.net/t/restic-rclone-pcloud-connection-issues/2963/15
|
||||
|
||||
* Change #2482: Remove vendored dependencies
|
||||
|
||||
We've removed the vendored dependencies (in the subdir `vendor/`). When building restic, the
|
||||
Go compiler automatically fetches the dependencies. It will also cryptographically verify
|
||||
that the correct code has been fetched by using the hashes in `go.sum` (see the link to the
|
||||
documentation below).
|
||||
|
||||
https://github.com/restic/restic/issues/2482
|
||||
https://golang.org/cmd/go/#hdr-Module_downloading_and_verification
|
||||
|
||||
* Change #2546: Return exit code 3 when failing to backup all source data
|
||||
|
||||
The backup command used to return a zero exit code as long as a snapshot could be created
|
||||
successfully, even if some of the source files could not be read (in which case the snapshot
|
||||
would contain the rest of the files).
|
||||
|
||||
This made it hard for automation/scripts to detect failures/incomplete backups by looking at
|
||||
the exit code. Restic now returns the following exit codes for the backup command:
|
||||
|
||||
- 0 when the command was successful - 1 when there was a fatal error (no snapshot created) - 3 when
|
||||
some source data could not be read (incomplete snapshot created)
|
||||
|
||||
https://github.com/restic/restic/issues/956
|
||||
https://github.com/restic/restic/issues/2064
|
||||
https://github.com/restic/restic/issues/2526
|
||||
https://github.com/restic/restic/issues/2364
|
||||
https://github.com/restic/restic/pull/2546
|
||||
|
||||
* Change #2600: Update dependencies, require Go >= 1.13
|
||||
|
||||
Restic now requires Go to be at least 1.13. This allows simplifications in the build process and
|
||||
removing workarounds.
|
||||
|
||||
This is also probably the last version of restic still supporting mounting repositories via
|
||||
fuse on macOS. The library we're using for fuse does not support macOS any more and osxfuse is not
|
||||
open source any more.
|
||||
|
||||
https://github.com/bazil/fuse/issues/224
|
||||
https://github.com/osxfuse/osxfuse/issues/590
|
||||
https://github.com/restic/restic/pull/2600
|
||||
https://github.com/restic/restic/pull/2852
|
||||
https://github.com/restic/restic/pull/2927
|
||||
|
||||
* Change #1597: Honor the --no-lock flag in the mount command
|
||||
|
||||
The mount command now does not lock the repository if given the --no-lock flag. This allows to
|
||||
mount repositories which are archived on a read only backend/filesystem.
|
||||
|
||||
https://github.com/restic/restic/issues/1597
|
||||
https://github.com/restic/restic/pull/2821
|
||||
|
||||
* Enhancement #1570: Support specifying multiple host flags for various commands
|
||||
|
||||
Previously commands didn't take more than one `--host` or `-H` argument into account, which
|
||||
could be limiting with e.g. the `forget` command.
|
||||
|
||||
The `dump`, `find`, `forget`, `ls`, `mount`, `restore`, `snapshots`, `stats` and `tag`
|
||||
commands will now take into account multiple `--host` and `-H` flags.
|
||||
|
||||
https://github.com/restic/restic/issues/1570
|
||||
|
||||
* Enhancement #1680: Optimize `restic mount`
|
||||
|
||||
We've optimized the FUSE implementation used within restic. `restic mount` is now more
|
||||
responsive and uses less memory.
|
||||
|
||||
https://github.com/restic/restic/issues/1680
|
||||
https://github.com/restic/restic/pull/2587
|
||||
https://github.com/restic/restic/pull/2787
|
||||
|
||||
* Enhancement #2072: Display snapshot date when using `restic find`
|
||||
|
||||
Added the respective snapshot date to the output of `restic find`.
|
||||
|
||||
https://github.com/restic/restic/issues/2072
|
||||
|
||||
* Enhancement #2175: Allow specifying user and host when creating keys
|
||||
|
||||
When adding a new key to the repository, the username and hostname for the new key can be
|
||||
specified on the command line. This allows overriding the defaults, for example if you would
|
||||
prefer to use the FQDN to identify the host or if you want to add keys for several different hosts
|
||||
without having to run the key add command on those hosts.
|
||||
|
||||
https://github.com/restic/restic/issues/2175
|
||||
|
||||
* Enhancement #2277: Add support for ppc64le
|
||||
|
||||
Adds support for ppc64le, the processor architecture from IBM.
|
||||
|
||||
https://github.com/restic/restic/issues/2277
|
||||
|
||||
* Enhancement #2395: Ignore sync errors when operation not supported by local filesystem
|
||||
|
||||
The local backend has been modified to work with filesystems which doesn't support the `sync`
|
||||
operation. This operation is normally used by restic to ensure that data files are fully
|
||||
written to disk before continuing.
|
||||
|
||||
For these limited filesystems, saving a file in the backend would previously fail with an
|
||||
"operation not supported" error. This error is now ignored, which means that e.g. an SMB mount
|
||||
on macOS can now be used as storage location for a repository.
|
||||
|
||||
https://github.com/restic/restic/issues/2395
|
||||
https://forum.restic.net/t/sync-errors-on-mac-over-smb/1859
|
||||
|
||||
* Enhancement #2427: Add flag `--iexclude-file` to backup command
|
||||
|
||||
The backup command now supports the flag `--iexclude-file` which is a case-insensitive
|
||||
version of `--exclude-file`.
|
||||
|
||||
https://github.com/restic/restic/issues/2427
|
||||
https://github.com/restic/restic/pull/2898
|
||||
|
||||
* Enhancement #2569: Support excluding files by their size
|
||||
|
||||
The `backup` command now supports the `--exclude-larger-than` option to exclude files which
|
||||
are larger than the specified maximum size. This can for example be useful to exclude
|
||||
unimportant files with a large file size.
|
||||
|
||||
https://github.com/restic/restic/issues/2569
|
||||
https://github.com/restic/restic/pull/2914
|
||||
|
||||
* Enhancement #2571: Self-heal missing file parts during backup of unchanged files
|
||||
|
||||
We've improved the resilience of restic to certain types of repository corruption.
|
||||
|
||||
For files that are unchanged since the parent snapshot, the backup command now verifies that
|
||||
all parts of the files still exist in the repository. Parts that are missing, e.g. from a damaged
|
||||
repository, are backed up again. This verification was already run for files that were
|
||||
modified since the parent snapshot, but is now also done for unchanged files.
|
||||
|
||||
Note that restic will not backup file parts that are referenced in the index but where the actual
|
||||
data is not present on disk, as this situation can only be detected by restic check. Please
|
||||
ensure that you run `restic check` regularly.
|
||||
|
||||
https://github.com/restic/restic/issues/2571
|
||||
https://github.com/restic/restic/pull/2827
|
||||
|
||||
* Enhancement #2858: Support filtering snapshots by tag and path in the stats command
|
||||
|
||||
We've added filtering snapshots by `--tag tagList` and by `--path path` to the `stats`
|
||||
command. This includes filtering of only 'latest' snapshots or all snapshots in a repository.
|
||||
|
||||
https://github.com/restic/restic/issues/2858
|
||||
https://github.com/restic/restic/pull/2859
|
||||
https://forum.restic.net/t/stats-for-a-host-and-filtered-snapshots/3020
|
||||
|
||||
* Enhancement #323: Add command for copying snapshots between repositories
|
||||
|
||||
We've added a copy command, allowing you to copy snapshots from one repository to another.
|
||||
|
||||
Note that this process will have to read (download) and write (upload) the entire snapshot(s)
|
||||
due to the different encryption keys used on the source and destination repository. Also, the
|
||||
transferred files are not re-chunked, which may break deduplication between files already
|
||||
stored in the destination repo and files copied there using this command.
|
||||
|
||||
To fully support deduplication between repositories when the copy command is used, the init
|
||||
command now supports the `--copy-chunker-params` option, which initializes the new
|
||||
repository with identical parameters for splitting files into chunks as an already existing
|
||||
repository. This allows copied snapshots to be equally deduplicated in both repositories.
|
||||
|
||||
https://github.com/restic/restic/issues/323
|
||||
https://github.com/restic/restic/pull/2606
|
||||
https://github.com/restic/restic/pull/2928
|
||||
|
||||
* Enhancement #551: Use optimized library for hash calculation of file chunks
|
||||
|
||||
We've switched the library used to calculate the hashes of file chunks, which are used for
|
||||
deduplication, to the optimized Minio SHA-256 implementation.
|
||||
|
||||
Depending on the CPU it improves the hashing throughput by 10-30%. Modern x86 CPUs with the SHA
|
||||
Extension should be about two to three times faster.
|
||||
|
||||
https://github.com/restic/restic/issues/551
|
||||
https://github.com/restic/restic/pull/2709
|
||||
|
||||
* Enhancement #2195: Simplify and improve restore performance
|
||||
|
||||
Significantly improves restore performance of large files (i.e. 50M+):
|
||||
https://github.com/restic/restic/issues/2074
|
||||
https://forum.restic.net/t/restore-using-rclone-gdrive-backend-is-slow/1112/8
|
||||
https://forum.restic.net/t/degraded-restore-performance-s3-backend/1400
|
||||
|
||||
Fixes "not enough cache capacity" error during restore:
|
||||
https://github.com/restic/restic/issues/2244
|
||||
|
||||
NOTE: This new implementation does not guarantee order in which blobs are written to the target
|
||||
files and, for example, the last blob of a file can be written to the file before any of the
|
||||
preceeding file blobs. It is therefore possible to have gaps in the data written to the target
|
||||
files if restore fails or interrupted by the user.
|
||||
|
||||
The implementation will try to preallocate space for the restored files on the filesystem to
|
||||
prevent file fragmentation. This ensures good read performance for large files, like for
|
||||
example VM images. If preallocating space is not supported by the filesystem, then this step is
|
||||
silently skipped.
|
||||
|
||||
https://github.com/restic/restic/pull/2195
|
||||
https://github.com/restic/restic/pull/2893
|
||||
|
||||
* Enhancement #2328: Improve speed of check command
|
||||
|
||||
We've improved the check command to traverse trees only once independent of whether they are
|
||||
contained in multiple snapshots. The check command is now much faster for repositories with a
|
||||
large number of snapshots.
|
||||
|
||||
https://github.com/restic/restic/issues/2284
|
||||
https://github.com/restic/restic/pull/2328
|
||||
|
||||
* Enhancement #2423: Support user@domain parsing as user
|
||||
|
||||
Added the ability for user@domain-like users to be authenticated over SFTP servers.
|
||||
|
||||
https://github.com/restic/restic/pull/2423
|
||||
|
||||
* Enhancement #2576: Improve the chunking algorithm
|
||||
|
||||
We've updated the chunker library responsible for splitting files into smaller blocks. It
|
||||
should improve the chunking throughput by 5-15% depending on the CPU.
|
||||
|
||||
https://github.com/restic/restic/issues/2820
|
||||
https://github.com/restic/restic/pull/2576
|
||||
https://github.com/restic/restic/pull/2845
|
||||
|
||||
* Enhancement #2598: Improve speed of diff command
|
||||
|
||||
We've improved the performance of the diff command when comparing snapshots with similar
|
||||
content. It should run up to twice as fast as before.
|
||||
|
||||
https://github.com/restic/restic/pull/2598
|
||||
|
||||
* Enhancement #2599: Slightly reduce memory usage of prune and stats commands
|
||||
|
||||
The prune and the stats command kept directory identifiers in memory twice while searching for
|
||||
used blobs.
|
||||
|
||||
https://github.com/restic/restic/pull/2599
|
||||
|
||||
* Enhancement #2733: S3 backend: Add support for WebIdentityTokenFile
|
||||
|
||||
We've added support for EKS IAM roles for service accounts feature to the S3 backend.
|
||||
|
||||
https://github.com/restic/restic/issues/2703
|
||||
https://github.com/restic/restic/pull/2733
|
||||
|
||||
* Enhancement #2773: Optimize handling of new index entries
|
||||
|
||||
Restic now uses less memory for backups which add a lot of data, e.g. large initial backups. In
|
||||
addition, we've improved the stability in some edge cases.
|
||||
|
||||
https://github.com/restic/restic/pull/2773
|
||||
|
||||
* Enhancement #2781: Reduce memory consumption of in-memory index
|
||||
|
||||
We've improved how the index is stored in memory. This change can reduce memory usage for large
|
||||
repositories by up to 50% (depending on the operation).
|
||||
|
||||
https://github.com/restic/restic/pull/2781
|
||||
https://github.com/restic/restic/pull/2812
|
||||
|
||||
* Enhancement #2786: Optimize `list blobs` command
|
||||
|
||||
We've changed the implementation of `list blobs` which should be now a bit faster and consume
|
||||
almost no memory even for large repositories.
|
||||
|
||||
https://github.com/restic/restic/pull/2786
|
||||
|
||||
* Enhancement #2790: Optimized file access in restic mount
|
||||
|
||||
Reading large (> 100GiB) files from restic mountpoints is now faster, and the speedup is
|
||||
greater for larger files.
|
||||
|
||||
https://github.com/restic/restic/pull/2790
|
||||
|
||||
* Enhancement #2840: Speed-up file deletion in forget, prune and rebuild-index
|
||||
|
||||
We've sped up the file deletion for the commands forget, prune and rebuild-index, especially
|
||||
for remote repositories. Deletion was sequential before and is now run in parallel.
|
||||
|
||||
https://github.com/restic/restic/pull/2840
|
||||
|
||||
|
||||
Changelog for restic 0.9.6 (2019-11-22)
|
||||
=======================================
|
||||
|
||||
@ -1360,11 +1824,11 @@ Details
|
||||
the latest version of restic.
|
||||
|
||||
Exploiting the vulnerability requires a Linux/Unix system which saves backups via restic and
|
||||
a Windows systems which restores files from the repo. In addition, the attackers need to be
|
||||
able to create files with arbitrary names which are then saved to the restic repo. For
|
||||
example, by creating a file named "..\test.txt" (which is a perfectly legal filename on Linux)
|
||||
and restoring a snapshot containing this file on Windows, it would be written to the parent of
|
||||
the target directory.
|
||||
a Windows systems which restores files from the repo. In addition, the attackers need to be able
|
||||
to create files with arbitrary names which are then saved to the restic repo. For example, by
|
||||
creating a file named "..\test.txt" (which is a perfectly legal filename on Linux) and
|
||||
restoring a snapshot containing this file on Windows, it would be written to the parent of the
|
||||
target directory.
|
||||
|
||||
We'd like to thank Tyler Spivey for reporting this responsibly!
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user