Commit Graph

46 Commits

Author SHA1 Message Date
Jakob Borg
e82ed6e3d3
style: gofumpt all the things (#9829)
Literally `gofumpt -w .` from the top level dir. Guaranteed to be minor
style changes only and nothing else.

@imsodin per request?
2024-11-19 11:32:56 +01:00
Gusted
356c5055ad
lib/sha256: Remove it (#9643)
### Purpose

Remove the `lib/sha256` package, because it's no longer necessary. Go's
standard library now has the same performance and is on par with
`sha256-simd` since [Since Go
1.21](1a64574f42).
Therefore using `sha256-simd` has no benefits anymore.

ARM already has optimized sha256 assembly code since
7b8a7f8272,
`sha256-simd` published their results before that optimized assembly was
implemented,
f941fedda8.
The assembly looks very similar and the benchmarks in the Go commit
match that of `sha256-simd`.

This patch removes all of the related code of `lib/sha256` and makes
`crypto/sha256` the 'default'.

Benchmark of `sha256-simd` and `crypto/sha256`:
<details>

```
cpu: AMD Ryzen 5 3600X 6-Core Processor
                │  simd.txt   │               go.txt                │
                │   sec/op    │    sec/op     vs base               │
Hash/8Bytes-12    63.25n ± 1%    73.38n ± 1%  +16.02% (p=0.002 n=6)
Hash/64Bytes-12   98.73n ± 1%   105.30n ± 1%   +6.65% (p=0.002 n=6)
Hash/1K-12        567.2n ± 1%    572.8n ± 1%   +0.99% (p=0.002 n=6)
Hash/8K-12        4.062µ ± 1%    4.062µ ± 1%        ~ (p=0.396 n=6)
Hash/1M-12        512.1µ ± 0%    510.6µ ± 1%        ~ (p=0.485 n=6)
Hash/5M-12        2.556m ± 1%    2.564m ± 0%        ~ (p=0.093 n=6)
Hash/10M-12       5.112m ± 0%    5.127m ± 0%        ~ (p=0.093 n=6)
geomean           13.82µ         14.27µ        +3.28%

                │   simd.txt   │               go.txt                │
                │     B/s      │     B/s       vs base               │
Hash/8Bytes-12    120.6Mi ± 1%   104.0Mi ± 1%  -13.81% (p=0.002 n=6)
Hash/64Bytes-12   618.2Mi ± 1%   579.8Mi ± 1%   -6.22% (p=0.002 n=6)
Hash/1K-12        1.682Gi ± 1%   1.665Gi ± 1%   -0.98% (p=0.002 n=6)
Hash/8K-12        1.878Gi ± 1%   1.878Gi ± 1%        ~ (p=0.310 n=6)
Hash/1M-12        1.907Gi ± 0%   1.913Gi ± 1%        ~ (p=0.485 n=6)
Hash/5M-12        1.911Gi ± 1%   1.904Gi ± 0%        ~ (p=0.093 n=6)
Hash/10M-12       1.910Gi ± 0%   1.905Gi ± 0%        ~ (p=0.093 n=6)
geomean           1.066Gi        1.032Gi        -3.18%
```

</details>


### Testing

Compiled and tested on Linux.

### Documentation

https://github.com/syncthing/docs/pull/874
2024-08-10 12:58:20 +01:00
Alexander Seiler
ddce692f72
all: Correct various typos (#8870) 2023-05-09 08:54:02 +02:00
Jakob Borg
a3c724f2c3
all: Add build constants for runtime.GOOS comparisons (#8442)
all: Add package runtimeos for runtime.GOOS comparisons

I grew tired of hand written string comparisons. This adds generated
constants for the GOOS values, and predefined Is$OS constants that can
be iffed on. In a couple of places I rewrote trivial switch:es to if:s,
and added Illumos where we checked for Solaris (because they are
effectively the same, and if we're going to target one of them that
would be Illumos...).
2022-07-28 19:36:39 +02:00
Jakob Borg
4b750b6dc3
all: Remove usage of deprecated io/ioutil (#7971)
As of Go 1.16 io/ioutil is deprecated. This replaces usage with the
corresponding functions in package os and package io.
2021-11-22 08:59:47 +01:00
André Colomb
41bfb7a330
Normalize CLI options to always use two dashes. (#8037)
Consistently use double dashes and fix typos -conf, -data-dir and
-verify.

Applies also to tests running the syncthing binary for consistency.

* Fix mismatched option name --conf in cli subcommand.

According to the source code comments, the cli option flags should
mirror those from the serve subcommand where applicable.  That one is
actually called --config though.

* cli: Fix help text option placeholders.

The urfave/cli package uses the Value field of StringFlag to provide a
default value, not to name the placeholder.  That is instead done with
backticks around some part of the Usage field.

* cli: Add missing --data flag in subcommand help text.

The urfave/cli based option parsing uses a fake flags collection to
generate help texts matching the used global options.  But the --data
option was omitted from it, although it is definitely required when
using --config as well.  Note that it cannot just be ignored, as some
debug stuff actually uses the DB:

syncthing cli --data=/bar --config=/foo debug index dump
2021-11-04 08:42:55 +01:00
Jakob Borg
d47745a86b
all: Update build constraints to Go 1.17 style (#7894) 2021-08-17 10:10:41 +02:00
Jakob Borg
f4372710bf
all: Remove crypto/md5 (#7493)
This is a mostly pointless change to make security scanners and static
analysis tools happy, as they all hate seeing md5. None of our md5 uses
were security relevant, but still. Only visible effect of this change is
that our temp file names for very long file names become slightly longer
than they were previously...
2021-03-17 22:22:49 +01:00
Simon Frei
3dd13c3994
test, lib/model: Various integration test updates & improvements (#6956) 2020-09-07 09:35:37 +02:00
Simon Frei
932d8c69de lib/fs: Properly handle case insensitive systems (fixes #1787, fixes #2739, fixes #5708)
With this change we emulate a case sensitive filesystem on top of
insensitive filesystems. This means we correctly pick up case-only renames
and throw a case conflict error when there would be multiple files differing
only in case.

This safety check has a small performance hit (about 20% more filesystem
operations when scanning for changes). The new advanced folder option
`caseSensitiveFS` can be used to disable the safety checks, retaining the
previous behavior on systems known to be fully case sensitive.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-07-28 11:15:11 +02:00
Jakob Borg
dd92b2b8f4
all: Tweak error creation (#6391)
- In the few places where we wrap errors, use the new Go 1.13 "%w"
  construction instead of %s or %v.

- Where we create errors with constant strings, consistently use
  errors.New and not fmt.Errorf.

- Remove capitalization from errors in the few places where we had that.
2020-03-03 22:40:00 +01:00
Jakob Borg
c2ddc83509 all: Revert the underscore sillyness 2019-02-02 12:16:27 +01:00
Jakob Borg
e187a5f5cb test: Add another variant of API timeout to skip when benchmarking 2018-05-30 13:23:52 +02:00
Jakob Borg
2fdf9bc55d test: Mend the transfer benchmark 2018-02-03 10:29:05 +01:00
Jakob Borg
6e148e20cd test: Mend tests for latest event changes etc 2017-11-13 00:25:07 +01:00
Jakob Borg
72c5f2e5c6 test: Updates for fs/osutil package changes 2017-11-13 00:25:07 +01:00
Jakob Borg
f7fc0c1d3e all: Update license url to https (ref #3976) 2017-02-09 08:04:16 +01:00
Jakob Borg
c4ba580cbb all: Remove symlink support on Windows, SymlinksEnabled config
After this change,

- Symlinks on Windows are always unsupported. Sorry.

- Symlinks are always enabled on other platforms. They are just a small
  file like anything else. There is no need to special case them. If you
  don't want to sync some symlinks, ignore them.

- The protocol doesn't differentiate between different "types" of
  symlinks. If that distinction ever does become relevant the individual
  devices can figure it out by looking at the destination when they
  create the link.

It's backwards compatible in that all the old symlink types are still
understood to be symlinks, and the new SYMLINK type is equivalent to the
old SYMLINK_UNKNOWN which was always a valid way to do it.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3962
LGTM: AudriusButkevicius
2017-02-07 08:34:24 +00:00
Jakob Borg
18cc7a663b lib: Remove osutil.Remove & osutil.RemoveAll (fixes #3513)
These are no longer required with Go 1.7. Change made by removing the
functions, doing a global s/osutil.Remove/os.Remove/.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3514
2016-08-16 10:01:58 +00:00
Jakob Borg
ea87bcefd6 lib/protocol, lib/model: Implement high precision time stamps (fixes #3305)
This adds a new nanoseconds field to the FileInfo, populates it during
scans and sets the non-truncated time in Chtimes calls.

The actual file modification time is defined as modified_s seconds +
modified_ns nanoseconds. It's expected that the modified_ns field is <=
1e9 (that is, all whole seconds should go in the modified_s field) but
not really enforced. Given that it's an int32 the timestamp can be
adjusted += ~2.9 seconds by the modified_ns field...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3431
2016-08-06 13:05:59 +00:00
Michael Ploujnikov
66f480519b lib/model: Refactor out scanning method from rwfolder.Serve loop 2016-03-30 06:53:47 +00:00
Jakob Borg
ac190b2e39 Change DB label format (index folders, devices) 2016-01-03 19:32:40 +01:00
Jakob Borg
6d11006b54 Generate ECDSA keys instead of RSA
This replaces the current 3072 bit RSA certificates with 384 bit ECDSA
certificates. The advantage is these certificates are smaller and
essentially instantaneous to generate. According to RFC4492 (ECC Cipher
Suites for TLS), Table 1: Comparable Key Sizes, ECC has comparable
strength to 3072 bit RSA at 283 bits - so we exceed that.

There is no compatibility issue with existing Syncthing code - this is
verified by the integration test ("h2" instance has the new
certificate).

There are browsers out there that don't understand ECC certificates yet,
although I think they're dying out. In the meantime, I've retained the
RSA code for the HTTPS certificate, but pulled it down to 2048 bits. I
don't think a higher security level there is motivated, is this matches
current industry standard for HTTPS certificates.
2015-11-27 09:15:12 +01:00
Mike Boone
342036408e Fix typos. 2015-11-11 21:20:34 -05:00
Jakob Borg
7705a6c1f1 mv internal lib 2015-08-09 09:35:26 +02:00
Jakob Borg
6e4d33c741 Don't run tests with audit on 2015-07-20 15:46:14 +02:00
Jakob Borg
c3d07d60ca Refactored integration tests
Added internal/rc to remote control a Syncthing process and made the
"awaiting sync" determination reliable.
2015-06-19 08:47:47 +02:00
Jakob Borg
3f59d6daff Add validation cache 2015-05-25 13:19:18 +02:00
Jakob Borg
03506db76c Fix rename with capitalization test 2015-05-08 10:02:19 +02:00
Lode Hoste
67f0c9bef0 Do not remove the file when renaming on a case-insensitive platform (fixes #1743) 2015-05-01 23:43:45 +02:00
Lode Hoste
58b15f9452 Limit alterfiles to a single operation per file 2015-05-01 13:03:03 +02:00
Lode Hoste
dedca59ac9 Add capitalization changes to integration tests 2015-05-01 12:11:57 +02:00
dartraiden
32a76901a9 typos and spelling correction 2015-04-29 15:59:47 +02:00
Audrius Butkevicius
50d377d9fe Fix integration tests 2015-04-28 00:09:44 +01:00
Audrius Butkevicius
d12db3e7b8 Add osutil.Glob to deal with Windows (fixes #1690) 2015-04-26 16:37:50 +01:00
Jakob Borg
54752deaa1 Move index to index-v0.11.0.db (new format) and centralize location config 2015-04-01 11:30:28 +02:00
Jakob Borg
32425c5561 MPLv2 2015-03-17 16:02:27 +01:00
Jakob Borg
e22c873ec4 Repair integration tests 2015-03-08 08:41:43 +01:00
Lode Hoste
758233f001 Do not error when chmod failes when permissions are ignored (fixes #1404). 2015-03-07 21:38:16 +01:00
Jakob Borg
d114648c16 fixup alterFiles 2015-03-01 21:38:04 +01:00
Lode Hoste
218c4c128c Alter files into directories and the other way around 2015-02-23 12:12:31 +01:00
Audrius Butkevicius
25c664b13a Improvements to integration tests 2015-01-22 00:18:08 +00:00
Audrius Butkevicius
e50a8917ec Add renames to integration tests 2015-01-13 22:07:14 +00:00
Jakob Borg
1e9110b763 Add debugging utility for manual directory comparison 2014-12-31 13:04:06 +01:00
Jakob Borg
6f3fbbbe49 Improve error checking in integration tests 2014-12-31 13:04:04 +01:00
Jakob Borg
2090530bbb Improve and clean up integration tests, benchmark. 2014-12-19 12:43:48 +01:00