Commit Graph

1495 Commits

Author SHA1 Message Date
Jakob Borg
2bb5b2244b cmd/stupgrades: Basic process metrics 2024-06-03 19:50:28 +02:00
Jakob Borg
2f281799c1 cmd/stcrashreceiver: Ignore patterns, improve metrics 2024-06-03 19:50:28 +02:00
Jakob Borg
18a58a2ddc cmd/strelaypoolsrv: More compact response, improved metrics 2024-06-03 19:50:28 +02:00
Jakob Borg
f283215fce cmd/stdiscosrv: Add AMQP replication 2024-06-03 19:50:28 +02:00
Jakob Borg
ba6ac2f604
lib/geoip, cmd/relaypoolsrv, cmd/ursrv: Automatically manage GeoIP updates (#9342)
This adds a small package `geoip` which knows how to download and manage
the Maxmind GeoLite2 database we use. This removes the need for various
scripts to download and manage the geoip database, something that today
happens on Docker startup for the relay pool server and using various
hand written hacks for the usage reporting server.

The database is downloaded when needed and then refreshed on a
best-effort basis weekly.
2024-05-18 20:31:49 +03:00
Jakob Borg
07a9fa2dbd
all: Use own automaxprocs package that doesn't log (ref #9436) (#9437)
### Purpose

🤫
2024-02-27 13:05:19 +01:00
Thomas
aa559bf496
all: Use Linux container CPU quota (fixes #9357, fixes #9435) (#9436)
Go is not cgroup aware and by default will set GOMAXPROCS to the number
of available threads, regardless of whether it is within the allocated
quota. This behaviour causes high amount of CPU throttling and degraded
application performance.
2024-02-26 12:23:14 +00:00
Jakob Borg
2d968d46b7 cmd/syncthing: Remove legacy GOMAXPROCS handling (ref #9436) 2024-02-26 13:12:57 +01:00
bt90
35e153625c
cmd/ursrv: Add FreeBSD detection (#9351)
### Purpose

Classify `ports@freebsd` as `FreeBSD (3rd party)`
2024-01-16 17:14:12 +01:00
bt90
d5e1b99e6c
cmd/ursrv: Fix Arch detection (#9350)
### Purpose

Classify `syncthing@archlinux` as `Arch (3rd party)`
2024-01-16 17:13:34 +01:00
Daniel Padrta
9387e107b3
cmd/syncthing: Add CLI completion functionality (fixes #8616) (#9226)
### Purpose

This implements CLI completion using the Kongplete module. As a side
effect a CLI structure for syncthing/cli was created for kongplete to be
able to parse and implement CLI completion.

### Testing

I've tested the autocompletion manually, and it had worked, but I hadn't
added any tests so as to test it automatically. Additionally, I ran `go
run build.go test` with all tests passing.
2024-01-04 10:20:53 +00:00
Jakob Borg
aa901790b9
lib/api: Save session & CSRF tokens to database, add option to stay logged in (fixes #9151) (#9284)
This adds a "token manager" which handles storing and checking expired
tokens, used for both sessions and CSRF tokens. It removes the old,
corresponding functionality for CSRFs which saved things in a file. The
result is less crap in the state directory, and active login sessions
now survive a Syncthing restart (this really annoyed me).

It also adds a boolean on login to create a longer-lived session cookie,
which is now possible and useful. Thus we can remain logged in over
browser restarts, which was also annoying... :)

<img width="1001" alt="Screenshot 2023-12-12 at 09 56 34"
src="https://github.com/syncthing/syncthing/assets/125426/55cb20c8-78fc-453e-825d-655b94c8623b">

Best viewed with whitespace-insensitive diff, as a bunch of the auth
functions became methods instead of closures which changed indentation.
2024-01-04 10:07:12 +00:00
Jakob Borg
935a28c961
lib/model: Use a single lock (phase two: cleanup) (#9276)
Cleanup after #9275.

This renames `fmut` -> `mut`, removes the deadlock detector and
associated plumbing, renames some things from `...PRLocked` to
`...RLocked` and similar, and updates comments.

Apart from the removal of the deadlock detection machinery, no
functional code changes... i.e. almost 100% diff noise, have fun
reviewing.
2023-12-11 22:06:45 +01:00
Jakob Borg
c53a1f210c
cmd/syncthing: Better cli stdin handling (ref #9166) (#9281)
Seems to work for me, @AudriusButkevicius.
2023-12-11 21:15:52 +01:00
cjc7373
b71a930bfc
cmd/syncthing: Mostly replace urfave/cli command line parser with alecthomas/kong (#9166)
`syncthing cli` subcommand was using urfave/cli as the command parser.
This PR replace it with kong, which the main command uses.

Some help texts and error message format are changed. Other than that,
all the command usage and logic remains unchanged.

There's only one place which still uses urfave/cli, which is `syncthing
cli config`, because it uses some magic to dynamically build commands
from struct reflects. I used kong's `passthrough:""` tag to pass any
argument following `syncthing cli config` to urfave/cli parser.

This PR also fixes #9041

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2023-12-11 11:35:57 +01:00
Maximilian
16db6fcf3d
lib/nat, lib/upnp: IPv6 UPnP support (#9010)
This pull request allows syncthing to request an IPv6
[pinhole](https://en.wikipedia.org/wiki/Firewall_pinhole), addressing
issue #7406. This helps users who prefer to use IPv6 for hosting their
services or are forced to do so because of
[CGNAT](https://en.wikipedia.org/wiki/Carrier-grade_NAT). Otherwise,
such users would have to configure their firewall manually to allow
syncthing traffic to pass through while IPv4 users can use UPnP to take
care of network configuration already.

### Testing

I have tested this in a virtual machine setup with miniupnpd running on
the virtualized router. It successfully added an IPv6 pinhole when used
with IPv6 only, an IPv4 port mapping when used with IPv4 only and both
when dual-stack (IPv4 and IPv6) is used.

Automated tests could be added for SOAP responses from the router but
automatically testing this with a real network is likely infeasible.

### Documentation

https://docs.syncthing.net/users/firewall.html could be updated to
mention the fact that UPnP now works with IPv6, although this change is
more "behind the scenes".

---------

Co-authored-by: Simon Frei <freisim93@gmail.com>
Co-authored-by: bt90 <btom1990@googlemail.com>
Co-authored-by: André Colomb <github.com@andre.colomb.de>
2023-12-11 07:36:18 +01:00
Jakob Borg
2ae15aa454 cmd/stcrashreceiver: Add metrics for diskstore inventory 2023-11-27 08:24:59 +01:00
Jakob Borg
47bcf4f8f4 cmd/stcrashreceiver: Minor cleanup, stricter file permissions 2023-11-27 08:24:59 +01:00
Jakob Borg
a8b9096353 cmd/stcrashreceiver: Add metrics for incoming reports 2023-11-27 08:24:59 +01:00
Jakob Borg
5328380691 cmd/ursrv: Add metrics for incoming reports 2023-11-27 08:24:59 +01:00
Anatoli Babenia
b184d46d8a
cmd/ursrv: Add link to source code (#9224)
To see that https://data.syncthing.net is open source, study the code
and change it.
2023-11-15 08:51:23 +00:00
Jakob Borg
3f32c5cb4b cmd/ursrv: Anchor distribution expressions to avoid mismatches (ref #9141) 2023-11-15 09:32:46 +01:00
Jakob Borg
d0a6dc5b13 cmd/ursv: Report on copy range method 2023-11-15 08:48:00 +01:00
Jakob Borg
aaee0c126b cmd/stdiscorv: Expose build info in metrics 2023-11-14 09:31:53 +01:00
Jakob Borg
5e2b7825dc cmd/stdiscosrv: Metric for returned retry-after 2023-11-08 12:18:59 +01:00
Jakob Borg
58bd931d90 cmd/stdiscosrv: Account IPv4 & IPv6 2023-11-08 12:18:59 +01:00
vapatel2
854499382e
cmd/stdiscosrv: Prevent nil IPs from X-Forwarded-For (fixes #9189) (#9190)
### Purpose

Treat X-Forwarded-For as a comma-separated string to prevent nil IP being returned by the Discovery Server

### Testing

Unit Tests implemented

Testing with a Discovery Client can be done as follows:
```
A simple example to replicate this entails running Discovery with HTTP, use Nginx as a reverse proxy and hardcode (as an example) a list of IPs in the X-Forwarded-For header.
1. Send an Announcement with tcp://0.0.0.0:<some-port>
2. Query the DeviceID
3. Observe the returned IP Address is no longer nil; i.e.  `tcp://<nil>:<some-port>`
```
2023-11-08 11:10:23 +00:00
Jakob Borg
9ce6a73f42 Revert "cmd/stcrashreceiver: Aggregate slice out of bounds errors"
This reverts commit dc6a10dff4.
2023-10-16 08:08:23 +02:00
Jakob Borg
a405c21ebb cmd/stdiscosrv: Only attempt unescaping when there are %-encodings in the header (fixes #9143) 2023-10-14 12:30:29 +02:00
Jakob Borg
dc6a10dff4 cmd/stcrashreceiver: Aggregate slice out of bounds errors 2023-10-14 12:19:55 +02:00
Jakob Borg
d4c2acf6f6 cmd/stcrashreceiver: Propagate synthetic user ID for crashes 2023-10-14 12:19:55 +02:00
orangekame3
5eb20580b1
cmd/ursrv: Replace "2006-01-02" with time.DateOnly (#9157)
This commit replaces "2006-01-02" to time.DateOnly. time.DateOnly is
introduced since Go1.20
2023-10-11 10:32:19 +00:00
Jakob Borg
690b55360f
cmd/stdiscosrv: Handle unescaped cert header from Traefik (fixes #9143) (#9153) 2023-10-07 04:09:07 +02:00
Jakob Borg
4f6b86a1c0 cmd/stdiscosrv: Slightly tweak replication settings 2023-10-04 14:15:00 +02:00
bt90
7c579880eb
cmd/ursrv: Add linuxserver.io detection (#9145)
Detect linuxserver
2023-10-02 12:48:04 +02:00
bt90
f8a7a034a7
cmd/ursrv: Fix f-droid detection (#9142)
Fix f-droid detection
2023-09-29 17:42:44 +02:00
bt90
ceae56a860
cmd/ursrv: Support new android build user (#9141)
Support new android build user
2023-09-29 16:34:28 +02:00
Jakob Borg
6ed9c0c34c
lib/config: Accept pre-hashed password (fixes #9123) (#9124) 2023-09-24 19:23:49 +02:00
bt90
06ac10ee37
cmd/stdiscosrv: Deduplicate addresses (fixes #8482) (#9080) 2023-09-06 14:36:00 +02:00
Jakob Borg
c6334e61aa
all: Support multiple device connections (fixes #141) (#8918)
This adds the ability to have multiple concurrent connections to a single device. This is primarily useful when the network has multiple physical links for aggregated bandwidth. A single connection will never see a higher rate than a single link can give, but multiple connections are load-balanced over multiple links.

It is also incidentally useful for older multi-core CPUs, where bandwidth could be limited by the TLS performance of a single CPU core -- using multiple connections achieves concurrency in the required crypto calculations...

Co-authored-by: Simon Frei <freisim93@gmail.com>
Co-authored-by: tomasz1986 <twilczynski@naver.com>
Co-authored-by: bt90 <btom1990@googlemail.com>
2023-09-06 12:52:01 +02:00
Jakob Borg
a80e6be353 cmd/stdiscosrv: Streamline context handling 2023-08-30 09:36:27 +02:00
Jakob Borg
acc532fc60 cmd/stdiscosrv: Explicitly enable HTTP/2
The server supports it, but it's not negotiated unless explicitly
allowed in the TLS config NextProtos.
2023-08-30 09:09:52 +02:00
Jakob Borg
a04cc95005 cmd/stdiscosrv: Separate HTTPS and replication certificates 2023-08-23 13:43:54 +02:00
Jakob Borg
480fa4b915 cmd/stdiscosrv: Use larger database settings 2023-08-23 13:43:14 +02:00
Jakob Borg
92a4931850 cmd/stdiscosrv: Modernise TLS settings, remove excessive HTTP logging 2023-08-23 13:39:52 +02:00
Jakob Borg
bdfef9010f cmd/stdiscosrv: Serve compressed responses 2023-08-23 13:39:14 +02:00
Jakob Borg
462389934b cmd/stupgrades: Serve friendlier URLs for upgrade assets (fixes #9033) 2023-08-09 21:01:15 +02:00
Jakob Borg
319916124b
cmd/strelaysrv: Handle accept error with debug set (fixes #9001) (#9004) 2023-07-26 23:55:48 +01:00
deepsource-autofix[bot]
24e230d455
all: unused parameter should be replaced by underscore (#8989)
refactor: unused parameter should be replaced by underscore

Unused parameters in functions or methods should be replaced with `_`
(underscore) or removed.

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-07-18 14:33:13 +00:00
Jakob Borg
e4d0f9dd6c cmd/syncthing: Mention STVERSIONEXTRA in --help output (ref #8980) 2023-07-16 17:48:24 +02:00