Commit Graph

2215 Commits

Author SHA1 Message Date
Chih-Hsuan Yen
b806026990
lib/connections: Fix building with -tags noquic (#9009) 2023-07-28 10:08:50 +00:00
Emil Lundberg
b08b99e284
lib/api: Fix data race in TestCSRFRequired (#9006) 2023-07-26 21:33:45 +00:00
Jakob Borg
855c6dc67b
lib/api: Allow Bearer authentication style with API key (#9002)
Currently, historically, we look for the `X-API-Key` header to
authenticate with an API key. There's nothing wrong with this, but in
some scenarios it's easier to produce an `Authorization` header with a
`Bearer $token` content, which is nowadays more common. This change adds
support for both, so that we will accept an API key either in our custom
header or as a bearer token.
2023-07-26 13:13:06 +02:00
tomasz1986
f42f041f53
lib/ur: Don't report uptime if start time is in the past (fixes #7698) (#8996)
Currently, because of devices with unset RTC clock, the 100% percentile
for Uptime on [1] is calculated since the Unix epoch which is useless as
far as usage statistics are concerned. Thus, if the Syncthing start time
is set to a past date, assume that the clock is wrong and do not even
try to report the uptime.

[1] https://data.syncthing.net

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2023-07-22 21:25:03 +00:00
Christian Kujau
6b6b2c6194
lib/model: use WARN for "Unexpected folder" messages (#8998) 2023-07-22 21:17:32 +00:00
tomasz1986
d70eb569f2
lib/osutil: Skip setLowPriority in Windows if already lower (fixes #6597) (#8993) 2023-07-21 04:38:15 +00:00
deepsource-autofix[bot]
21c074cc2c
all: replace empty slice literal with var (#8990)
refactor: replace empty slice literal with `var`

An empty slice can be represented by `nil` or an empty slice literal. They are
functionally equivalent — their `len` and `cap` are both zero — but the `nil`
slice is the preferred style. For more information about empty slices,
see [Declaring Empty Slices](https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices).

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-07-18 14:44:37 +00:00
deepsource-autofix[bot]
f23c41221b
all: fix unused method receiver (#8988)
refactor: fix unused method receiver

Methods with unused receivers can be a symptom of unfinished refactoring or a bug. To keep 
the same method signature, omit the receiver name or '_' as it is unused.

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-07-18 14:34:50 +00:00
deepsource-autofix[bot]
24e230d455
all: unused parameter should be replaced by underscore (#8989)
refactor: unused parameter should be replaced by underscore

Unused parameters in functions or methods should be replaced with `_`
(underscore) or removed.

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-07-18 14:33:13 +00:00
Jakob Borg
df2ac7aaeb
gui, lib/api: Add possibility to feed through extra version information (#8980)
This adds an environment variable STVERSIONEXTRA that, when set, gets
added to the version information in the API and GUI.

The purpose of all this is to be able to communicate something about the
bundling or packaging, through the log & GUI and the end user, to the
potential person supporting it -- i.e., us. :) A wrapper can set this
variable to indicate that Syncthing is being run via `SyncTrayzor`,
`Syncthing-macOS`, etc., and thus indicate to the end user that the GUI
they are looking at is perhaps not the only source of truth and
management for this instance.
2023-07-16 17:43:10 +02:00
Jakob Borg
c44de2cd58
lib/fs: Clarify errors for Windows filenames (fixes #8968) (#8969)
With this change, error messages include the offending characters or
name parts. Examples:

    nul.txt: name is invalid, contains Windows reserved name: "nul"
    foo>bar.txt: name is invalid, contains Windows reserved character: ">"
    foo \bar.txt: name is invalid, must not end in space or period on Windows
2023-07-07 11:00:40 +00:00
guangwu
27aba3567b
all: Minor staticcheck fixes (#8939) 2023-06-19 06:50:53 +00:00
Jakob Borg
6b475bdb78
lib/config, gui: Disallow some options in combination with "untrusted" (fixes #8920) (#8921)
This prevents combining untrusted with introducer and auto-accept, and
also verifies that folders shared with untrusted devices have passwords
at config loading time.

Co-authored-by: Simon Frei <freisim93@gmail.com>
2023-06-14 09:24:31 +02:00
Simon Frei
bf6ffbbd67
Don't add empty device to config on init (#8933)
We usually want to ensure that our own device is present. However if the
given device ID is the empty ID, we shouldn't do that. This is a
legimate (though way too non-obvious) use-case when opening the config
without knowing/caring about the device ID.
2023-06-10 20:33:39 +00:00
Jakob Borg
ae176ea9cd
build: Tests should run with Go 1.20 on Windows (#8924)
Tests should run with Go 1.20 on Windows
2023-06-05 10:19:47 +02:00
Jakob Borg
81a4b22d43
lib/model: Improve test for unignored parent directories (#8926) 2023-06-04 15:32:03 +02:00
Anthony Goeckner
405cdedcd3
lib/model: Set platform data for new folders w/ ignorePerms (ref #8883) (#8907)
* Platform data (ownership, xattrs, etc.) is now set correctly for newly-received folders, even if the received folder has the NoPermissions flag.

* Call setPlatformData on receivers that have ignorePerms set to true.
2023-05-17 09:06:50 +02:00
Eng Zer Jun
089320aadc
lib: replace Readdir(-1) with os.ReadDir (#8901) 2023-05-11 15:35:52 +00:00
Will Rouesnel
b2fb2ef276
lib/api: Allow BindDN to exclude any username formatting (fixes #8899) (#8900)
This allows a syncthing instance to be locked to exactly 1 user without
needing search capability on the LDAP instance.
2023-05-10 07:52:02 +02:00
Ross Smith II
3adfe2f91f lib/fs: Fix root path handling for Windows (fixes #8778)
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2023-05-09 10:01:57 +00:00
Jakob Borg
1103a27337 all: Grand test refactor (fixes #8779, fixes #8799)
This fixes various test issues with Go 1.20.

- Most tests rewritten to use fakefs where possible
- Some tests that were already skipped, or dubious (invasive,
  unmaintainable, unclear what they even tested) have been removed
- Some actual code rewritten to better support testing in fakefs

Co-authored-by: Eric P <eric@kastelo.net>
2023-05-09 10:01:57 +00:00
Alexander Seiler
ddce692f72
all: Correct various typos (#8870) 2023-05-09 08:54:02 +02:00
Anthony Goeckner
7e31ec5417
lib/model: Set platform data, incl. copying ownership, for new folders w/ NoPermissions flag (#8883)
Platform data (ownership, xattrs, etc.) is now set correctly for newly-received folders, even if the received folder has the NoPermissions flag.
2023-05-02 11:11:39 +02:00
Eric P
9f131eee6b
lib/ignore: Properly handle non-existing included ignore-files (fixes #8764) (#8874)
In the sequence of loading ignores, the error File Does Not Exist is not being considered a fatal  error, since the .stignore file is allowed to not exist. However, included ignore files also tossed that same error in case those do not exist while in those cases it's considered an error and it should lead to the folder stopping. Changing the error when opening an included ignore file to something other than the regular does fix this issue, as in it now works again as described in the Documentation.
2023-04-20 15:00:55 +02:00
Jakob Borg
09efe03e1d lib/connections: Avoid using nil lanChecker
Otherwise it panics when someone calls Priority() on it...
2023-04-19 10:42:25 +02:00
Jakob Borg
9b660c1959
lib/config, lib/connections: Configurable protocol priority (ref #8626) (#8868)
This makes the various protocol priorities configurable among the other
options. With this, it's possible to prefer QUIC over TCP for WAN
connections, for example. Both sides need to be similarly configured for
this to work properly.

The default priority order remains the same as previously (TCP, QUIC,
Relay, with LAN better than WAN).

To make this happen I made each dialer & listener more priority aware,
and moved the check for whether a connection is LAN or not into the
dialer / listener -- this is the new "lanChecker" type that's passed
around.
2023-04-16 14:54:28 +02:00
Jakob Borg
f59ffc8ddd
lib/model: Improve path generation for auto accepted folders (fixes #8859) (#8860)
- Make sure we don't try to use empty last path components
- Create the directory to "reserve" it once we've decided to use it
2023-04-11 13:07:22 +02:00
Jakob Borg
4a8c691aef
lib/syncthing: Handle successful global migration (fixes #8851) (#8852)
lib/syncthing: Handle successfull global migration (fixes #8851)
2023-04-05 15:25:55 +02:00
Simon Frei
3a41d4afa5
lib/model: Set enc. trailer size on pull (ref #8563, #8556) (#8839)
In the original fix in #8563 I simply forgot this. Which meant #8556
wasn't actually fixed, as the trialer size would have been 0 (default),
and thus we would have still sent the inflated size to encrypted peers.
2023-03-28 22:47:24 +02:00
Simon Frei
6a66aee489
lib/model: Fix file size inconsistency due to enc. trailer (#8840)
lib/model: Fix file size inconsisency due to enc. trailer

Fixes a regression due to PR #8563, while arguable the bug was actually
introduced in a much older PR #7155, but didn't have any bad effects so
far:
We account for the encryption trailer in the db updater routine,
calculating the file-info size there. However there's no guarantee that
the file-info at this point is still the exact same as when it was
written. It was before, but isn't anymore since introducing the new
EncryptedTrailerSize field.
Fix: Adjust the size in the info at the same place where the trailer is
written, i.e. we definitely have the actual size on disk.
2023-03-28 22:02:59 +02:00
Jakob Borg
34b312b85b lib/ur: Fix custom releases URL comparison 2023-03-24 09:15:12 +01:00
Dimitri Papadopoulos Orfanos
526e21ae26
all: Fix typos found by codespell (#8833) 2023-03-21 08:07:28 +01:00
Jakob Borg
f378e63147
lib/protocol: Handle encrypted requests without encrypted hash (fixes #8277) (#8827)
The layout of the request differs based on whether it comes from an
untrusted device or a trusted device with encrypted enabled. Handle
both.

Closes #8819.
2023-03-18 10:22:18 +01:00
tomasz1986
358cf25cff
lib/config: Allow sub-second watcher delay (fixes #7859) (#7864)
Allow the watcher delay to take fractional values, effectively allowing
for much shorter delays. The minimum value is limited at 0.01, which
effectively translates to 10ms. This is required in order to guarantee
that there is still enough time to aggregate multiple single change
events.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-03-18 08:50:38 +01:00
Jakob Borg
466b56ded1
lib/protocol: Cache expensive key operations (fixes #8599) (#8820)
This adds a cache to the expensive key generation operations. It's fixes
size LRU/MRU stuff to keep memory usage bounded under absurd conditions.

Also closes #8600.
2023-03-12 20:06:59 +01:00
Simon Frei
da72df6ffc
lib: Correctly handle encrypted trailer size (fixes #8556) (#8563) 2023-03-10 14:14:14 +01:00
Jakob Borg
ab8e6a82ab
lib/api: Expose blocksHash in file info (#8810)
This adds the BlocksHash field from the FileInfo to our API output. It
can be useful for debugging, or for external tools. I'm intentionally
leaving it as an opaque base64 string because no meaning should be
derived from it: it's just a string.
2023-03-06 15:37:15 +01:00
entity0xfe
4558eef446
lib/discover: Don't leak relay-tokens to discovery (#8762)
Use an allowlist to send only the `id` query param to the discovery server.
2023-03-04 12:16:57 +01:00
Jakob Borg
cbec697e5f cmd/stupgrades: Cache should apply to HEAD as well as GET 2023-02-22 12:22:52 +01:00
Jakob Borg
8991ecf444 build: Add more GitHub Actions 2023-02-22 10:56:55 +01:00
Andreas Sommer
2f88dafa56
Only fail after chmod error if permissions differ (e.g. on config file) (#8771) 2023-02-20 15:41:10 +01:00
greatroar
38f2b34d29
all: Use new Go 1.19 atomic types (#8772) 2023-02-07 12:07:34 +01:00
Jakob Borg
99595ce3d9
build: Update quic-go and pfilter for Go 1.20 (fixes #8768) (#8769) 2023-02-02 22:00:50 +01:00
Jakob Borg
d5e4ef249f Add forgotten copyright notices 2023-02-01 22:59:41 +01:00
Jakob Borg
65cfefaa3c cmd, docker: Updates for infrastructure
These are some changes to the relay pool server, upgrade server, and
crash receiver to run under Kubernetes. It's been in production for a
while.
2023-01-31 11:17:52 +01:00
Jakob Borg
abdac2caa2
Handle relay connect timeout (fixes #8749) (#8755)
This makes sure the service manager doesn't interpret timeout errors, or any other error, as a signal to stop the service instead of restarting it.

I added it directly to our service utility function, as it may help catch other instances of the same problem... We would typically want timeouts etc to be a retryable error, unless it is the top level context that has timed out and we check for that specifically.
2023-01-19 11:15:18 +01:00
Jakob Borg
c4e69cd66c
gui, api: Indicate running under container (#8728)
This adds a word to the version string when running containerized. The
purpose is mostly to facilitate troubleshooting via screenshot by
"leaking" this rather important aspect of the setup. Additionally, the
version row gets "no-overflow-ellipsis" treatment so that the whole
thing is actually visible in the GUI and the (now useless) tooltip is
removed. In production releases this won't make a difference as the
whole thing will typically fit, but in odd setups it provides more info
up front.
2022-12-25 08:08:41 +01:00
Simon Frei
634a3d0e3b
lib/fs: Use io/fs errors as recommended in std lib (#8726) 2022-12-21 23:42:22 +01:00
Jakob Borg
ad0044fec8
lib/fs: Watching is unsupported on android/amd64 (fixes #8709) (#8710) 2022-12-21 22:01:00 +01:00
Simon Frei
d157d12037
lib/model: Only log at info level if setting change time fails (#8725) 2022-12-21 21:58:35 +01:00
Aleksey Vasenev
f9d68474ac
lib/model: Don't lower rescan interval from default on auto accepted enc folder (fixes #8572) (#8573) 2022-12-21 19:52:22 +01:00
Aleksey Vasenev
05738001ac
lib/model: Fixed adding empty items to device list (fixes #8646) (#8647) 2022-12-06 21:22:35 +01:00
André Colomb
ab0eb909a2
gui, lib/connections: Let the backend decide whether connection is local (fixes #8686) (#8694)
* lib/connections: Cache isLAN decision for later external access.

The check whether a remote device's address is on a local network
currently happens when handling the Hello message, to configure the
limiters.  Save the result to the ConnectionInfo and pass it out as
part of the model's ConnectionInfo struct in ConnectionStats().

* gui: Use provided connection attribute to distinguish LAN / WAN.

Replace the dumb IP address check which didn't catch common cases and
actually could contradict what the backend decided.  That could have
been confusing if the GUI says WAN, but the limiter is not actually
applied because the backend thinks it's a LAN.

Add strings for QUIC and relay connections to also differentiate
between LAN and WAN.

* gui: Redefine reception level icons for all connection types.

Move the mapping to the JS code, as it is much easier to handle
multiple switch cases by fall-through there.

QUIC is regarded no less than TCP anymore.  LAN and WAN make the
difference between levels 4 / 3 and 2 / 1:

{TCP,QUIC} LAN --> {TCP,QUIC} WAN --> Relay LAN --> Relay WAN -->
Disconnected.
2022-11-28 09:28:33 +01:00
greatroar
663106ef6e
lib/fs: Optimize WindowsInvalidFilename (#8687)
Replaced strings.Split with the new strings.Cut, which doesn't allocate.

name                           old time/op    new time/op    delta
WindowsInvalidFilenameValid-8     154ns ± 2%      89ns ± 0%   -42.09%  (p=0.000 n=10+9)
WindowsInvalidFilenameNUL-8       124ns ± 2%     124ns ± 1%      ~     (p=0.371 n=8+10)

name                           old alloc/op   new alloc/op   delta
WindowsInvalidFilenameValid-8     16.0B ± 0%      0.0B       -100.00%  (p=0.000 n=10+10)
WindowsInvalidFilenameNUL-8       19.0B ± 0%      3.0B ± 0%   -84.21%  (p=0.000 n=10+10)

name                           old allocs/op  new allocs/op  delta
WindowsInvalidFilenameValid-8      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=10+10)
WindowsInvalidFilenameNUL-8        2.00 ± 0%      1.00 ± 0%   -50.00%  (p=0.000 n=10+10)
2022-11-21 12:58:00 +01:00
Jakob Borg
2a8362d7af
build: Update all dependencies (fixes #8679) (#8680)
Minor API change for the LRU cache in casefs
2022-11-17 21:26:10 +01:00
Jakob Borg
6aa04118a6
lib/model: Correctly set xattrs on temp files (fixes #8667) (#8670) 2022-11-11 11:49:15 +01:00
Jakob Borg
a523fef78e
lib/model: Correctly handle xattrs on directories (fixes #8657) (#8658) 2022-11-09 06:54:04 +01:00
Simon Frei
ce2a68622c
lib/protocol: Ignore inode time when xattr&ownership is ignored (fixes #8654) (#8655)
lib/protocol: Ignore inode time when both xattr and ownership is ignored (fixes #8654)
2022-11-08 08:36:41 +01:00
Jakob Borg
a29605750d
lib/fs: Try to remove read only Windows files (fixes #3744) (#8650)
This happens when folders contain a custom icon.

Co-authored-by: Alexandre Alves <alexandrealvesdb.contact@gmail.com>
2022-11-07 21:33:17 +01:00
Jakob Borg
413c8cf4ea
lib/connections: Use adaptive write size for rate limited connections (fixes #8630) (#8631) 2022-11-03 15:44:46 +01:00
Jakob Borg
bf1e418e4a
lib/fs: Let xattr test avoid non-test attributes (fixes #8601) (#8628)
SELinux for example adds security.* attributes by default that we are
not allowed to touch, which causes the test to fail.
2022-11-03 11:57:30 +01:00
Jakob Borg
d3f50637d2
lib/model, lib/protocol: Fix file comparisons (fixes #8594) (#8603) 2022-10-16 17:04:28 +02:00
Jakob Borg
ed588ce335
lib/scanner: More sensible debug output (#8596)
Previous debug input didn't really give enough info to show what was
happening, while it also printed full block lists which are enormously
verbose. Now it consistently prints 1. what it sees on disk, 2. what it
got from CurrentFile (without blocks), 3. the action taken on that file.
2022-10-13 19:32:58 +02:00
Jakob Borg
87bf09ea40
lib/protocol: Show inode change time in FileInfo debug string (#8591) 2022-10-08 19:28:10 +02:00
Eric P
7a402409f1
lib/api: Add /rest/noauth/health health-check (fixes #8430) (#8585) 2022-10-06 21:28:49 +02:00
Eric P
c791dba392
api, gui: Prevent connection issues due to unsupported-upgrade (fixes #8569) (#8586)
There are some situations where an upgrade wouldn't be supported, even though the noUpgrade bool isn't set. So when handling the errors that are caused by this, when attempting an upgrade, it shouldn't lead to some sort of offline-message/restart/warning/etc...

I added some checks on specific errors related to this and return a 501 (Not Implemented) response instead, in case of an "UpgradeUnsupported"-error. Additionally, on the GUI-side, the 501-response is now not to be considered an error to act upon.
2022-10-06 21:27:08 +02:00
Simon Frei
a0c80e030a
lib/model: Fix warning log statement (ref #8583) (#8584) 2022-10-05 18:45:37 +02:00
entity0xfe
ad986f372d
cmd/strelaysrv: Add optional auth token (fixes #3987) (#8561)
* implement authentication via token for relaysrv

Make replaysrv check for a token before allowing clients to
join. The token can be set via the replay-uri.

* fix formatting

* key composite literal

* do not error out if auth material is provided but not needed

* remove unused method receiver

* clean up unused parameter in functions

* cleaner token handling, disable joining the pool if token is set.

* Keep backwards compatibility with older clients.

In prior versions of the protocol JoinRelayRequest did not have a
token field. Trying to unmarshal such a request will result in
an error. Return an empty JoinRelayRequest, that is a request
without token, instead.

Co-authored-by: entity0xfe <entity0xfe@my.domain>
2022-10-01 20:41:02 +01:00
Simon Frei
0935886045
lib/fs: Skip xattrs test on EOPNOTSUPP (fixes #8564) (#8565) 2022-10-01 08:48:00 +02:00
Eric P
3f2742a275
lib/versioner: Fix error in Trashcan restore (fixes: #7965) (#8549)
The restore function of Trash Can ran a rename at the end regardless of whether there was anything to rename. In this case, when the file-to-be-restored did not exist in the destination folder, this resulted in an error. I added a simple check, keeping track of whether the file existed prior to restoring it in the destination folder and depending on this value it will now return nil after the restoration to prevent the renaming function to kick off. Added a test for this specific edge-case as well.
2022-09-20 11:34:15 +02:00
Jakob Borg
6cac308bcd
all: Support syncing extended attributes (fixes #2698) (#8513)
This adds support for syncing extended attributes on supported
filesystem on Linux, macOS, FreeBSD and NetBSD. Windows is currently
excluded because the APIs seem onerous and annoying and frankly the uses
cases seem few and far between. On Unixes this also covers ACLs as those
are stored as extended attributes.

Similar to ownership syncing this will optional & opt-in, which two
settings controlling the main behavior: one to "sync" xattrs (read &
write) and another one to "scan" xattrs (only read them so other devices
can "sync" them, but not apply any locally).

Co-authored-by: Tomasz Wilczyński <twilczynski@naver.com>
2022-09-14 09:50:55 +02:00
greatroar
8065cf7e97
lib: Factor out getting IP address from net.Addr (#8538)
... and add fast paths for common cases.
2022-09-14 08:44:46 +02:00
Eric P
6e768a8387 lib/versioner: Fix cleaning behaviour (fixes #7988) (#8537)
The cleaning logic in util.go was used by Simple and Trashcan but only
really suited Trashcan since it works based on mtimes which Simple does
not use. The cleaning logic in util.go was moved to trashcan.go.
Staggered and Simple seemed to be able to benefit from the same base so
util.go now has the base for those two with an added parameter which
takes a function so it can still handle versioner-specific logic to
decide which files to clean up. Simple now also correctly cleans files
based on their time-stamp in the title together with a specific maximum
amount to keep. The Archive function in Simple.go was changed to get rid
of duplicated code.

Additionally the trashcan testcase which was used by Trashcan as well as
Simple was moved from versioner_test.go to trashcan_test.go to keep it
clean, there was no need to keep it in a separate test file
2022-09-13 19:21:42 +02:00
greatroar
152388b3a3
lib/tlsutil: Use crypto.Signer interface (#8526)
*rsa.PrivateKey and *ecdsa.PrivateKey are both Signers, which have a
method to get the public key. No need for the type switch.
2022-09-09 14:22:38 +02:00
luzpaz
837ffcfab5
all: Fix various user-facing and non-user-facing typos (#8509)
Found via `codespell -q 3 -S lang,./gui/default/vendor -L benchs,bu,inflight,ro`
2022-08-23 15:44:11 +02:00
Jakob Borg
b10d106a55
all: Modernize error wrapping (#8491)
This replaces old style errors.Wrap with modern fmt.Errorf and removes
the (direct) dependency on github.com/pkg/errors. A couple of cases are
adjusted by hand as previously errors.Wrap(nil, ...) would return nil,
which is not what fmt.Errorf does.
2022-08-16 10:01:49 +02:00
Jakob Borg
eb81f7400c
lib/fs: Cache user lookups (#8496) 2022-08-12 07:48:00 +02:00
Jakob Borg
06273875ae
all: Make scanning ownership opt-in (#8497) 2022-08-12 07:47:20 +02:00
André Colomb
4d4bfe8032
lib/model: Skip paused folders in aggregated completion loop (fixes #8219) (#8220)
Locally paused folders will fail on checkFolderRunningLocked() and
therefore abort the loop.  Avoid this by skipping paused folders
directly.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2022-08-10 08:50:19 +02:00
André Colomb
63de838f27
gui, api: Show internal config and state paths (fixes #8323) (#8324)
* lib/locations: Fix enum values camelCase.

* lib/locations: Remove unused FailuresFile.

* cmd/syncthing: Turn around role of locations storage.

Previously the locations package was used to provide default paths,
possibly with an overridden home directory.  Extra paths supplied on
the command line were handled and passed around in the options object.

To make the changed paths available to any other interested package,
override the location setting from the option if supplied, instead of
vice versa when not supplied.  Adapt code using this to read from the
locations package instead of passing through the options object.

* lib/locations: Refactor showPaths to locations package.

Generate a reusable string in locations.PrettyPrintPaths().
Enumerating all possible locations in different packages is error
prone, so add a new public function to generate the listing as a
string in the locations package.  Adapt cmd/syncthing --paths to use
that instead of its own console output.

* lib/locations: Include CSRF token in pretty printed paths.

* lib/api: New endpoint /rest/system/paths.

The paths should be available for troubleshooting from a running
instance.  Using the --paths CLI option is not easy in some
environments, so expose the locations mapping to a JSON endpoint.

Add utility function ListExpandedPaths() that also filters out any
entries which still contain variable placeholders.

* gui: List runtime paths in separate log viewer tab.

* Wrap paths.

* lib/syncthing: Utilize locations.Get() instead of passing an arg.

* Include base directories, move label to table caption.

* gui: Switch to hard-coded paths instead of iterating over all.

* gui: Break aboutModalView into tabs.

Use tabs to separate authors from included third-party software.

* gui: Move paths from log viewer to about modal.

* lib/locations: Adjust pretty print output order to match GUI.

* gui, authors: Remove additional bot names and fix indent.

The indentation changed because of the tabbed about dialog, fix the
authors script to respect that.

Skip Syncthing*Automation in authors list as well.

* Update AUTHORS list to remove bot names.

* Revert AUTHORS email order change.

* Do not emphasize DB and log file locations.

* Review line wrapping.

* review part 1: strings.Builder, naming

* Rename and extend locations.Set() with error handling.

Remodel the Override() function along the existing SetBaseDir() and
rename it to simply Set().  Make sure to use absolute paths when given
log file or GUI assets override options.  Add proper error reporting
if that goes wrong.

* Remove obsolete comment about empty logfile option.

* Don't filter out unexpanded baseDir placeholders, only ${timestamp}.

* Restore behavior regarding special "-" logfile argument.

If the option is given, but with empty value, assume the no log
file (same as "-").  Don't try to convert the special value to an
absolute path though and document this fact in a comment for the Set()
function.

* Use template to check for location key validity.

* Don't filter out timestamp placeholders.

* lib/api: Remove paths from /rest/system/status.

* lib/ur: Properly initialize map in failure data (fixes #8479)

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2022-08-10 08:25:13 +02:00
Jakob Borg
209e68c1ba
build: Update quic-go for Go 1.19 (#8483)
Also adds idle time and keepalive parameters because how this is
configured has changed in the new package version. The values are those
that seems like might already be default, if keep-alives were enabled,
which is not obvious from the doc comments.

Also, Go 1.19 gofmt reformatting of comments.
2022-08-03 15:43:26 +02:00
Jakob Borg
cc54488e55
lib/ur: Properly initialize map in failure data (fixes #8479) (#8480) 2022-08-03 10:41:26 +02:00
Jakob Borg
5977868165
lib/config: Use net.JoinHostPort instead of string manipulation (#8470) 2022-07-28 22:06:55 +02:00
Jakob Borg
585fb3f49b lib/api: Fix inverted logic in string comparison 2022-07-28 21:51:14 +02:00
deepsource-autofix[bot]
8e3f1190d1
lib/model: Use bytes.Equal instead of converting to string (#8469)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 20:00:07 +02:00
Jakob Borg
a3c724f2c3
all: Add build constants for runtime.GOOS comparisons (#8442)
all: Add package runtimeos for runtime.GOOS comparisons

I grew tired of hand written string comparisons. This adds generated
constants for the GOOS values, and predefined Is$OS constants that can
be iffed on. In a couple of places I rewrote trivial switch:es to if:s,
and added Illumos where we checked for Solaris (because they are
effectively the same, and if we're going to target one of them that
would be Illumos...).
2022-07-28 19:36:39 +02:00
deepsource-autofix[bot]
f2be9d1166
all: Fix nested if with else if (#8467)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 19:14:22 +02:00
Jakob Borg
dde275c6cc all: Unused errors 2022-07-28 19:08:51 +02:00
Jakob Borg
212258d213 lib/fs: Consolidate append in test 2022-07-28 19:05:24 +02:00
Jakob Borg
966db0d076 lib/api: Further (final?) unused parameters removal 2022-07-28 19:02:12 +02:00
Jakob Borg
6baa93e13f lib/api: String comparisons 2022-07-28 18:49:44 +02:00
Jakob Borg
0c8b22c696 lib/api: Remove unused parameters 2022-07-28 18:48:15 +02:00
deepsource-autofix[bot]
81d8fa1cb5
all: Fix unused method receiver (further) (#8466)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 17:55:29 +02:00
Jakob Borg
5958f42294 lib/model: Clarify normal shallow copy 2022-07-28 17:41:07 +02:00
Jakob Borg
8b4bd43306
lib/api: Missing error handling in config handler (#8463) 2022-07-28 17:35:43 +02:00
deepsource-autofix[bot]
755d21953f
all: Remove unused method receivers (#8462)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 17:32:45 +02:00
Jakob Borg
388e4db9cd
all: Combine sequential appends (#8461) 2022-07-28 17:28:24 +02:00
deepsource-autofix[bot]
5130c414da
all: Unused parameter should be replaced by underscore (#8464)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 17:17:29 +02:00
Jakob Borg
7bdb5faa9c
all: Remove or convert deprecated API usages (#8459) 2022-07-28 17:14:49 +02:00
deepsource-autofix[bot]
7e26f74f38
lib/config: Remove unnecessary use of fmt.Sprintf (#8458)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 17:00:41 +02:00
deepsource-autofix[bot]
0bdd0d595b
lib/model: Replace for loop with append (#8457)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 16:57:09 +02:00
deepsource-autofix[bot]
80ec4acb53
all: Fix check for empty string (#8456)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 16:51:03 +02:00
Jakob Borg
a2c5d901f2
cmd/syncthing, lib/config: Remove restartOnWakeup option & functionality (fixes #8448) (#8449) 2022-07-26 16:53:10 +02:00
Evan Spensley
c6a319d98b
gui: Add device sync status (fixes #7981) (#8401) 2022-07-26 10:15:19 +02:00
Jakob Borg
adce6fa473
all: Support syncing ownership (fixes #1329) (#8434)
This adds support for syncing ownership on Unixes and on Windows. The
scanner always picks up ownership information, but it is not applied
unless the new folder option "Sync Ownership" is set.

Ownership data is stored in a new FileInfo field called "platform data". This
is intended to hold further platform-specific data in the future
(specifically, extended attributes), which is why the whole design is a
bit overkill for just ownership.
2022-07-26 08:24:58 +02:00
André Colomb
a6dba7c6d6
lib/model, lib/config: Apply sensible defaults for auto-accepted encrypted folder (fixes #8296) (#8427)
* lib/model: Override scan config for auto-accepted encrypted folders.

Encrypted folders should not have the fs watcher enabled and rarely
benefit from a scheduled rescan.  The GUI adjusts the suggested
settings (watcher disabled, one day rescan interval) when accepting a
receive-encrypted folder.  Mirror that behavior to the auto-accept
case where the GUI is not involved.

Versioning also does not work well for encrypted folders, same
treatment.
2022-07-22 11:27:58 +02:00
Aroun
7cb8af9029
lib/osutil: Only announce address of interfaces which are up (fixes #7458) (#8422) 2022-07-07 19:19:29 +02:00
Simon Frei
3a6ebb8482
lib/config, lib/model: Warn about two-way introducer (fixes #8393) (#8395) 2022-06-20 18:36:45 +01:00
Jakob Borg
0e79b532cf
build: Update to patched github.com/gobwas/glob (fixes #8386) (#8387)
The glob package has seen very little activity the last few years so I'm
not holding my breath on getting this merged.
2022-06-15 21:49:15 +02:00
Simon Frei
5ac122b85f
lib/model: Don't include deleted items in kqueue warning threshold (ref #7855) (#8365) 2022-05-28 20:15:38 +02:00
greatroar
2ca8a5ac61
lib/assets: MIME types, time formats (#8351)
.eot and .woff2 weren't listed, but are present in vendored fontawesome.

.ttf and .woff are font/* according to IANA,
https://www.iana.org/assignments/media-types/media-types.xhtml#font.
This matches what mime.TypeByExtension returns.

.eot is from https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types.
mime.TypeByExtension doesn't recognize this extension.

* lib/assets: Use http.ParseTime

This understands the three time formats allowed in HTTP.
2022-05-22 22:10:18 +02:00
Simon Frei
e3078cc531
lib/model: Don't fail on temporary chmod (fixes #8355, ref #8235) (#8356) 2022-05-22 13:52:40 +02:00
greatroar
97291c9184
lib/api: Fix and optimize csrfManager (#8329)
An off-by-one error could cause tokens to be forgotten. Suppose

	tokens := []string{"foo", "bar", "baz", "quux"}
	i := 2
	token := tokens[i] // token == "baz"

Then, after

	copy(tokens[1:], tokens[:i+1])
	tokens[0] = token

we have

	tokens == []string{"baz", "foo", "bar", "baz"}

The short test actually relied on this bug.
2022-05-07 12:30:13 +02:00
Jakob Borg
f8c51d801a
lib/discover: Filter locally announced addresses (fixes #7289) (#8302) 2022-05-04 18:43:00 +02:00
Jakob Borg
ce0ded7c78
lib/connections: Correct race on loop variable (fixes #8320) (#8321) 2022-05-04 18:16:36 +02:00
greatroar
233d3e7f7b
lib/events: Remove unused method noopLogger.Stop (#8312)
This was needed for the old suture API, abandoned in
9524b51708.
2022-05-02 08:00:55 +02:00
greatroar
d00a30069a
lib/db: Constant/unused args and return values, double it.Release (#8259) 2022-04-27 20:32:44 +02:00
greatroar
49488c0e71
all: Clean up fmt.Errorf usage (#8309) 2022-04-27 20:30:13 +02:00
Simon Frei
6a7fc49c6b
lib/discover: Increase global discovery timeout (#8303) 2022-04-23 16:12:25 +02:00
André Colomb
1eda82b95f
lib/model: Improve remoteFolderState reporting (fixes #8266) (#8283) 2022-04-22 08:42:20 +02:00
Jakob Borg
623ec03dad
lib/model: Correct type of event data (fixes #8294) (#8295)
These things are fragile, every event should use an ${eventType}Data struct or something instead.
2022-04-21 15:45:31 +02:00
Simon Frei
3907cb0693
lib/model: Subscribe to correct event for fs watching (ref #8249) (#8287) 2022-04-17 12:41:25 +04:00
Jakob Borg
61dffabf97
cmd/syncthing, lib/logger: Add date to default log format (fixes #8272) (#8273)
This changes the default log format to include the date.
2022-04-15 07:46:14 +04:00
Eng Zer Jun
bc27aa12cd
all: use T.TempDir to create temporary test directory (#8280)
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-15 07:44:06 +04:00
André Colomb
0537b9546f
lib/model: Switch the remoteFolderState default value to valid (#8275)
Showing all folders from disconnected or paused remote devices as
unaccepted would be a lot of false positives.  As we cannot know
whether the remote has accepted while it doesn't have an active
connection, let's better report false negatives, as in assuming the
folders are accepted.
2022-04-13 18:15:58 +02:00
Simon Frei
0525c755f4
build: Bump quic-go to 0.26.0 for go1.18 update (#8231)
Merging because I want this in the RC, when we do the RC...
2022-04-12 16:27:29 +04:00
Simon Frei
bcd91f536e
lib/connections: Create the forgotten channel (ref #8263) (#8267) 2022-04-11 17:32:22 +04:00
André Colomb
0c46e0a9cc
gui, lib/model: Mark folders unaccepted by remote device (fixes #8202) (#8201) 2022-04-10 22:47:57 +02:00
Simon Frei
db72579f0e
lib: Get rid of buggy filesystem wrapping (#8257) 2022-04-10 20:55:05 +02:00
Jakob Borg
9b09bcc5f1
lib/connections: Always run a simple connection test (#7866) 2022-04-10 20:54:42 +02:00
Simon Frei
22e12904c9
lib/connections: Make request tests sequential (#8263) 2022-04-10 20:54:16 +02:00
Simon Frei
b947056e62
lib: Removal global connection registry (#8254) 2022-04-09 16:04:56 +02:00
Simon Frei
072fa46bfd
lib/connections, lib/model: Improve new conn handling (#8253) 2022-04-07 17:35:33 +02:00
Simon Frei
edc3a77b98
lib/fs, lib/model: Add warning about kqueue resource usage (fixes #7855) (#8249) 2022-04-05 21:32:06 +02:00
André Colomb
30fa462e33
all: Comments and cosmetics (#8218)
* lib/api: Note ItemStarted and ItemFinished for default filtering.

The reasoning why LocalChangeDetected and RemoteChangeDetected events
are not included in the event stream by default (without explicit
filter mask requested) also holds for the ItemStarted and ItemFinished
events.  They should be excluded as well when we start to break the
API compatibility for some reason.

* gui: Enumerate unused event types in the eventService.

Define constants for the unused event types as well, for completeness'
sake.  They are intentionally not handled in the GUI currently.

* cmd/syncthing: Harmonize uppercase CLI argument placeholders.

Use ALL-UPPERCASE and connecting dashes to distinguish argument
placeholders from literal argument options (e.g. "cpu" or "heap" for
profiling).  The dash makes it clear which words form a single
argument and where a new argument starts.

This style is already used for the "syncthing cli debug file" command.

* lib/model: Simplify event data structure.

Using map[string]interface{} is not necessary when all values are
known to be strings.
2022-04-02 20:36:19 +02:00
Simon Frei
55c513b827
lib/model: Clear folder error after loading ignores (fixes #8232) (#8238) 2022-03-27 21:29:40 +02:00
greatroar
0eca0ac45a
lib/model: Chmod to mode|0700, not 755 (#8235) 2022-03-26 12:07:07 +02:00
greatroar
4be867c560
all: Replace errors.Cause with errors.Is (#8236) 2022-03-26 12:05:57 +02:00
Greg
53926a1ae6
lib/fs: Do not follow symlinks in watcher on solaris (fixes #8020) (#8223) 2022-03-24 08:36:43 +01:00
greatroar
26eaedc491
lib/db, lib/discover: Minor cleanup (#8217) 2022-03-14 22:48:10 +01:00
greatroar
d0fd6c6c82
lib/db: Make err(Closed|NotFound) values (#8215) 2022-03-13 20:53:34 +01:00
greatroar
bd0acd04b1
lib/protocol: Use one mutex for rawConnect.awaiting and nextID (#8198)
Having a separate mutex for the three or four instructions needed to
fetch and increment nextID means the overhead exceeds the cost of this
operation.  nextID is now handled inside the critical section for
awaiting instead, while the more expensive channel creation has been
moved outside it.

This is mostly a simplification, though it may have minor performance
benefits in some situations. The single-threaded sender benchmark shows
no significant difference:

name               old speed      new speed      delta
RequestsRawTCP-8   55.3MB/s ± 7%  56.6MB/s ± 6%   ~     (p=0.190 n=10+10)
RequestsTLSoTCP-8  20.5MB/s ±20%  20.8MB/s ± 8%   ~     (p=0.604 n=10+9)
2022-02-28 09:13:30 +01:00
Jakob Borg
f890fe6fd3
lib/config: Improve clarity of free space errors (fixes #8180) (#8191) 2022-02-24 17:07:51 +01:00
tomasz1986
d35d7d2360
lib, gui: Removed unused cleanInterval from Staggered Versioning (ref #541) (#8161)
Staggered File Versioning used to have its own cleanInterval that
controlled how often file versions were cleaned. Nowadays, there is a
seperate setting called cleanupIntervalS responsible for the cleanup,
which applies to all File Versioning (except External). Thus, remove the
unneeded code and don't set the param up on new folders anymore.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2022-02-08 07:06:14 +01:00
André Colomb
1c74944cca
lib/api: Fix comments for endpoint parameters (#8163) 2022-02-07 08:51:09 +01:00
Simon Frei
518d5174e6
lib/model: Print device when a block request fails (#8159) 2022-02-01 20:20:55 +01:00
Simon Frei
635085d139
lib/db, lib/model: Remove filesystem state from FileSet (fixes #7850) (#8151) 2022-01-31 10:12:52 +01:00
Jonathan
d37a5b03f1
lib/fs: Handle permission change events on macos (fixes #7924) (#8150) 2022-01-30 17:21:21 +01:00
Simon Frei
8763fb05ec
lib/fs: Improve error messages checking file paths (fixes #7942) (#8148) 2022-01-29 18:47:55 +01:00
greatroar
a0fd619df3
lib/protocol: Require at least 3.125% savings from compression (#8133)
* lib/protocol: Require at least 3.125% savings from compression

The new lz4 library doesn't need its output buffer to be the maximum
size, unlike the old one (which would allocate if it weren't). It can
take a buffer that is of a smaller size and will report if compressed
data can fit inside the buffer (with a small chance of reporting a false
negative). Use that property to our advantage by requiring compressed
data to be at most n-n/32 = .96875*n bytes long for n input bytes.

* lib/protocol: Remove unused receivers

To make DeepSource happy.

* lib/protocol: Micro-optimize lz4Compress

Only write the length if compression was successful. This is a memory
write, so the compiler can't reorder it.

Only check the return value of lz4.CompressBlock.  Length-zero inputs
are always expanded by LZ4 compression (the library documents this),
so the check on len(src) isn't needed.
2022-01-24 19:36:58 +01:00
Jakob Borg
b6d1e16b4e
lib/protocol: Switch to a newer lz4 package (#8122) 2022-01-17 18:52:43 +01:00
Simon Frei
21d04b895a
lib, gui: Default ignores for new folders (fixes #7428) (#7530) 2022-01-13 23:38:21 +01:00
André Colomb
0cba3154f0
lib/model: Remove bogus fields from connections API endpoint (fixes #8103) (#8104)
* lib/model: Remove bogus fields from connections API endpoint.

Switch the returned data type for the /rest/system/connections element
"total" to use only the Statistics struct.  The other fields of the
ConnectionInfo struct are not populated and misleading.

* Lowercase JSON field names.

* lib/model: Get rid of ConnectionInfo.MarshalJSON().

It was missing the StartedAt field from the embedded Statistics
struct.  Just lowercasing the JSON attribute names can be done just as
easily with annotations.

* lib/model: Remove bogus startedAt field from totals.

Instead of using the Statistics type with one field empty, just switch
to a free-form map with the three needed fields.
2022-01-10 10:26:45 +01:00
André Colomb
5237337626
cmd/syncthing: Add --skip-port-probing (fixes #8090) (#8099)
* cmd/syncthing: Remove unnecessary function arguments.

The openGUI() function does not need a device ID to work, and there is
only one caller anyway which uses EmptyDeviceID.

The loadOrDefaultConfig() function is always called with the same
dummy values.

* cmd/syncthing: Avoid misleading info messages from monitor process.

In order to check whether panic reporting is enabled, the monitor
process utilizes the loadOrDefaultConfig() function.  In case there is
no config file yet, info messages may be logged during creation if the
config Wrapper, which is discarded immediately after.

Stop using the DefaultConfig() utility function from lib/syncthing and
directly generate a minimal config instead to avoid these.

Add comments to loadOrDefaultConfig() explaining its limited purpose.

* cmd/syncthing/generate: Always write updated config file.

Previously, an existing config file was left untouched unless either
of the --gui-user or --gui-password options was given.  Remove that
condition and simplify the checking code.

* lib/config: Factor out ProbeFreePorts().

* cmd/syncthing: Add option --skip-port-probing.

Applies to both the "generate" and "serve" subcommands, as well as the
deprecated --generate option, just as the --no-default-folder flag.
2022-01-07 11:19:17 +01:00
Gahl Saraf
cc39341eb9
lib: Fix panic due to closed event subscriptions on shutdown (#8079) 2021-12-22 20:16:21 +01:00
greatroar
eb857dbc45 lib/osutil: Use x/sys/windows for SetLowPriority 2021-11-27 15:35:07 +01:00
greatroar
286a25ae49 lib/upgrade: Use strings.Reader instead of bytes.Buffer 2021-11-27 15:35:07 +01:00
greatroar
ae70046b49 lib/protocol: Remove unused sorting boilerplate 2021-11-27 15:35:07 +01:00
greatroar
c366933416 lib/sync: Make the clock a function pointer 2021-11-27 15:35:07 +01:00
greatroar
6a9716e8a1 lib/model: Return index from deviceActivity.leastBusy
This way, we don't need a second loop over the Availabilities to remove
the selected item.
2021-11-26 12:07:43 +01:00
greatroar
8a1e54d58a lib/fs: Optimize Canonicalize
When pathSep is a constant, the compiler precomputes pathSep+pathSep and
".."+pathSep instead of emitting function calls to compute "//" and
"../". Benchmark results in lib/osutil:

name                old time/op    new time/op    delta
TraversesSymlink-8    8.86µs ± 3%    8.53µs ± 4%  -3.79%  (p=0.000 n=18+20)

name                old alloc/op   new alloc/op   delta
TraversesSymlink-8    1.06kB ± 0%    1.06kB ± 0%    ~     (all equal)

name                old allocs/op  new allocs/op  delta
TraversesSymlink-8      15.0 ± 0%      15.0 ± 0%    ~     (all equal)
2021-11-23 21:25:29 +01:00
greatroar
3e032c4da6 lib/fs: optimize Windows path checking/sanitizing
name                          old time/op    new time/op    delta
WindowsInvalidFilenameValid-8     875ns ± 1%     150ns ± 1%  -82.84%  (p=0.000 n=9+9)
WindowsInvalidFilenameNUL-8       276ns ± 4%     121ns ± 3%  -56.26%  (p=0.000 n=10+10)

name                          old alloc/op   new alloc/op   delta
WindowsInvalidFilenameValid-8     32.0B ± 0%     16.0B ± 0%  -50.00%  (p=0.000 n=10+10)
WindowsInvalidFilenameNUL-8       32.0B ± 0%     19.0B ± 0%  -40.62%  (p=0.000 n=10+10)

name                          old allocs/op  new allocs/op  delta
WindowsInvalidFilenameValid-8      2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
WindowsInvalidFilenameNUL-8        2.00 ± 0%      2.00 ± 0%     ~     (all equal)
2021-11-23 21:25:29 +01:00
Jakob Borg
1754c93370
lib/config, lib/ignore: Write Windows line endings (fixes #7115) (#8052) 2021-11-22 09:38:24 +01:00
Jakob Borg
4b750b6dc3
all: Remove usage of deprecated io/ioutil (#7971)
As of Go 1.16 io/ioutil is deprecated. This replaces usage with the
corresponding functions in package os and package io.
2021-11-22 08:59:47 +01:00
greatroar
bf89bffb0b
lib/config: Decouple VerifyConfiguration from Committer (#7939)
... and remove 8/10 implementations, which were no-ops. This saves code
and time copying configurations.
2021-11-22 08:45:29 +01:00
Jakob Borg
e2288fe441
lib/relay: Send SNI when the address is a host name (fixes #8014) (#8015) 2021-11-22 08:31:03 +01:00
greatroar
8265dac127
lib/nat: Fix race condition in Mapping (#8042)
The locking protocol in nat.Mapping was racy:

* Mapping.addressMap RLock'd, but then returned a map shared between
  caller and Mapping, so the lock didn't do anything.

* Operations inside Service.{verifyExistingMappings,acquireNewMappings}
  would lock the map for every update, but that means callers to
  Mapping.ExternalAddresses can be looping over the map while the
  Service methods are concurrently modifying it. When the Go runtime
  detects that happening, it panics.

* Mapping.expires was read and updated without locking.

The Service methods now lock the map once and release the lock only when
done.

Also, subscribers no longer get the added and removed addresses, because
none of them were using the information. This was changed for a previous
attempt to retain the fine-grained locking and not reverted because it
simplifies the code.
2021-11-22 08:29:44 +01:00
André Colomb
100870e142
cmd/syncthing: Implement generate as a subcommand with optional API credential setting (fixes #8021) (#8043)
Accept a subcommand as an alternative to the --generate option.  It
accepts a custom config directory through either the --home or
--config options, using the default location if neither is given.

Add the options --gui-user and --gui-password to "generate", but not
the "serve --generate" option form.  If either is given, an existing
config will not abort the command, but rather load, modify and save it
with the new credentials.  The password can be read from standard
input by passing only a single dash as argument.

Config modification is skipped if the value matches what's already in
the config.

* cmd/syncthing: Utilize lib/locations package in generate().
Instead of manually joining paths with "magic" file names, get them
from the centralized locations helper lib.

* cmd/syncthing: Simplify logging for --generate option.
Visible change: No more timestamp prefixes.
2021-11-18 22:57:59 +01:00
Jakob Borg
12fb7f2a0a lib/model: Correct "reverting folder" log entry 2021-11-17 12:52:10 +01:00
Jakob Borg
f1bf4d899a lib/model: Correct handling of fakefs cache
We looked under one cache key, then stored under another...
2021-11-17 12:52:10 +01:00
Simon Frei
591e4d8af1
gui, lib: Fix tracking deleted locally-changed on encrypted (fixes #7715) (#7726) 2021-11-10 09:46:21 +01:00
André Colomb
dec6f80d2b
lib/config: Move the bcrypt password hashing to GUIConfiguration (#8028)
What hash is used to store the password should ideally be an
implementation detail, so that every user of the GUIConfiguration
object automatically agrees on how to handle it.  That is currently
distribututed over the confighandler.go and api_auth.go files, plus
tests.

Add the SetHasedPassword() / CompareHashedPassword() API to keep the
hashing method encapsulated.  Add a separate test for it and adjust
other users and tests.  Remove all deprecated imports of the bcrypt
package.
2021-11-08 13:32:04 +01:00
André Colomb
ec8a748514
lib/syncthing: Clean up / refactor LoadOrGenerateCertificate() utility function. (#8025)
LoadOrGenerateCertificate() takes two file path arguments, but then
uses the locations package to determine the actual path.  Fix that
with a minimally invasive change, by using the arguments instead.
Factor out GenerateCertificate().

The only caller of this function is cmd/syncthing, which passes the
same values, so this is technically a no-op.

* lib/tlsutil: Make storing generated certificate optional.  Avoid
  temporary cert and key files in tests, keep cert in memory.
2021-11-07 23:59:48 +01:00
greatroar
db15e52743
lib/api: http.Request.BasicAuth instead of custom code (#8039) 2021-11-06 12:38:08 +01:00
André Colomb
41bfb7a330
Normalize CLI options to always use two dashes. (#8037)
Consistently use double dashes and fix typos -conf, -data-dir and
-verify.

Applies also to tests running the syncthing binary for consistency.

* Fix mismatched option name --conf in cli subcommand.

According to the source code comments, the cli option flags should
mirror those from the serve subcommand where applicable.  That one is
actually called --config though.

* cli: Fix help text option placeholders.

The urfave/cli package uses the Value field of StringFlag to provide a
default value, not to name the placeholder.  That is instead done with
backticks around some part of the Usage field.

* cli: Add missing --data flag in subcommand help text.

The urfave/cli based option parsing uses a fake flags collection to
generate help texts matching the used global options.  But the --data
option was omitted from it, although it is definitely required when
using --config as well.  Note that it cannot just be ignored, as some
debug stuff actually uses the DB:

syncthing cli --data=/bar --config=/foo debug index dump
2021-11-04 08:42:55 +01:00
greatroar
807a6b1022
lib/model: Optimize jobQueue performance and memory use (#8023)
By truncating time.Time to an int64 nanosecond count, we lose the
ability to precisely order timestamps before 1678 or after 2262, but we
gain (linux/amd64, Go 1.17.1):

name                      old time/op    new time/op    delta
JobQueuePushPopDone10k-8    2.85ms ± 5%    2.29ms ± 2%  -19.80%  (p=0.000 n=20+18)
JobQueueBump-8              34.0µs ± 1%    29.8µs ± 1%  -12.35%  (p=0.000 n=19+19)

name                      old alloc/op   new alloc/op   delta
JobQueuePushPopDone10k-8    2.56MB ± 0%    1.76MB ± 0%  -31.31%  (p=0.000 n=18+13)

name                      old allocs/op  new allocs/op  delta
JobQueuePushPopDone10k-8      23.0 ± 0%      23.0 ± 0%     ~     (all equal)

Results for BenchmarkJobQueueBump are with the fixed version, which no
longer depends on b.N for the amount of work performed. rand.Rand.Intn
is cheap at ~10ns per iteration.
2021-10-29 20:20:46 +02:00
Tomasz Wilczyński
296cc1bca2 lib/model: Limit the number of default hashers on Android (ref #2220)
Like Windows and Mac, Android is also an interactive operating system.
On top of that, it usually runs on much slower hardware than the other
two. Because of that, it makes sense to limit the number of hashes used
by default there too.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2021-10-29 10:06:52 +02:00
Simon Frei
951b058952 lib/model: Set mod. time after writing trailer in shortcut (ref #7992) 2021-10-24 16:50:30 +02:00
greatroar
8f8e8a9285 lib/protocol: Simplify codeToError, errorToCode
Also be explicit about the fact that ErrNoError is nil. That name isn't
used anywhere outside this file.
2021-10-22 18:40:40 +02:00
greatroar
46082f194c lib/protocol: Eliminate nativeModel on Unix 2021-10-22 18:40:40 +02:00
Simon Frei
517667c590
lib/model: Pull when a new connection is established (fixes #8012) (#8013) 2021-10-20 18:55:22 +02:00
greatroar
7c292cc812
lib/connections: Fix and optimize registry (#7996)
Registry.Get used a full sort to get the minimum of a list, and the sort
was broken because util.AddressUnspecifiedLess assumed it could find out
whether an address is IPv4 or IPv6 from its Network method. However,
net.(TCP|UDP)Addr.Network always returns "tcp"/"udp".
2021-10-06 10:52:51 +02:00
Simon Frei
c94b797f00
lib/protocol: Preserve sequence decrypting fileinfos (fixes #7994) (#7995) 2021-10-06 10:26:54 +02:00
Simon Frei
708a5c2070
lib/model: Write trailer when shortcutting on recv-enc (fixes #7991) (#7992) 2021-10-05 11:23:27 +02:00
Jakob Borg
92eaf52c21 lib/protocol: Test to lock down encryption key derivation
So that we don't inadvertently change how these things work.
2021-10-04 14:36:28 +02:00
tomasz1986
8d8f331a4a
lib/config: Always set mtime window to 2 on Android unless ext detected (ref #7963) (#7966)
The current detection is flawed, because it looks for a few specific
file systems like "msdos" or "fat" to set the mtime window, while in
reality Android seems to report names like "fuseblk", which can stand
for fat, ext4, or even f2fs.

At the moment, we set the mtime window only for a few known names used
for the fat filesystem. With this change, we take a safer approach of
always setting the time window unless we explicitly detect file systems
like ext2/ext3/ex4, which are known not to experience issues with moving
timestamps on Android.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2021-09-27 21:29:51 +02:00
greatroar
198028d627
lib/rand: Optimizations (#7964)
rand.secureSource.Uint64 no longer allocates. rand.String uses a
strings.Builder. Benchmark results on linux/amd64:

name            old time/op    new time/op    delta
SecureSource-8    69.1ns ± 3%    51.7ns ± 3%   -25.21%  (p=0.000 n=20+10)
String-8          2.66µs ± 2%    1.95µs ± 1%   -26.61%  (p=0.000 n=10+10)

name            old alloc/op   new alloc/op   delta
SecureSource-8     8.00B ± 0%     0.00B       -100.00%  (p=0.000 n=20+10)
String-8            288B ± 0%       32B ± 0%   -88.89%  (p=0.000 n=10+10)

name            old allocs/op  new allocs/op  delta
SecureSource-8      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=20+10)
String-8            33.0 ± 0%       1.0 ± 0%   -96.97%  (p=0.000 n=10+10)
2021-09-26 12:15:39 +02:00
Jakob Borg
73c5184518
build: Update all dependencies (#7968)
Also add a script step for how to do this, as frankly I forget each time
what the canonical way is.

`go run build.go update-deps`
2021-09-26 12:08:23 +02:00
greatroar
f96c211198
lib/db: Replace SipHash with hash/maphash (#7962) 2021-09-24 21:26:07 +02:00
greatroar
a4489dec30
lib/tlsutil: Allocate UnionedConnection in one go (#7912) 2021-09-21 08:40:34 +02:00
Simon Frei
721cd740d8
lib/model: Don't reset db while folder is running (fixes #7935) (#7936) 2021-09-11 17:14:47 +02:00
greatroar
de719ac409
lib/db: Inline sync.Once in releaser (#7938)
Having a pointer to a Once suggests that it is shared with other
objects, but it never is.
2021-09-10 09:58:17 +02:00
jtagcat
0ffa7f3f57
all: Clarify 'Cleaning data for folder' log message (#7937)
Instead of data (could be read as user data), use metadata.
2021-09-10 09:56:44 +02:00
greatroar
7fa141ea39
all: Unused args, retvals, assignments (#7926) 2021-09-08 00:11:16 +02:00
greatroar
13196ddd92 lib/relay/protocol: Merge two Sprintf calls 2021-09-06 15:30:56 +02:00
greatroar
eafb40460d lib/discover, lib/protocol: Buffer allocation 2021-09-06 15:30:56 +02:00
Simon Frei
4e2a9bb139
lib/model: Remove encryption pw on folder stop (#7925) 2021-09-03 09:54:47 +02:00
greatroar
3b2239357f
lib/model: Add missing error assignment + Remove unused argument (#7922) 2021-09-01 22:03:06 +02:00
Ross Smith II
7501bee430
lib/model: Typo in folder_recvonly.go (#7919) 2021-09-01 07:41:18 +02:00
greatroar
ed98039aa5
lib/fs: Optimize TempName + some cosmetic changes (#7911) 2021-08-29 10:47:53 +02:00