Commit Graph

71 Commits

Author SHA1 Message Date
Jakob Borg
f283215fce cmd/stdiscosrv: Add AMQP replication 2024-06-03 19:50:28 +02:00
Jakob Borg
07a9fa2dbd
all: Use own automaxprocs package that doesn't log (ref #9436) (#9437)
### Purpose

🤫
2024-02-27 13:05:19 +01:00
Thomas
aa559bf496
all: Use Linux container CPU quota (fixes #9357, fixes #9435) (#9436)
Go is not cgroup aware and by default will set GOMAXPROCS to the number
of available threads, regardless of whether it is within the allocated
quota. This behaviour causes high amount of CPU throttling and degraded
application performance.
2024-02-26 12:23:14 +00:00
Jakob Borg
aaee0c126b cmd/stdiscorv: Expose build info in metrics 2023-11-14 09:31:53 +01:00
Jakob Borg
5e2b7825dc cmd/stdiscosrv: Metric for returned retry-after 2023-11-08 12:18:59 +01:00
Jakob Borg
58bd931d90 cmd/stdiscosrv: Account IPv4 & IPv6 2023-11-08 12:18:59 +01:00
vapatel2
854499382e
cmd/stdiscosrv: Prevent nil IPs from X-Forwarded-For (fixes #9189) (#9190)
### Purpose

Treat X-Forwarded-For as a comma-separated string to prevent nil IP being returned by the Discovery Server

### Testing

Unit Tests implemented

Testing with a Discovery Client can be done as follows:
```
A simple example to replicate this entails running Discovery with HTTP, use Nginx as a reverse proxy and hardcode (as an example) a list of IPs in the X-Forwarded-For header.
1. Send an Announcement with tcp://0.0.0.0:<some-port>
2. Query the DeviceID
3. Observe the returned IP Address is no longer nil; i.e.  `tcp://<nil>:<some-port>`
```
2023-11-08 11:10:23 +00:00
Jakob Borg
a405c21ebb cmd/stdiscosrv: Only attempt unescaping when there are %-encodings in the header (fixes #9143) 2023-10-14 12:30:29 +02:00
Jakob Borg
690b55360f
cmd/stdiscosrv: Handle unescaped cert header from Traefik (fixes #9143) (#9153) 2023-10-07 04:09:07 +02:00
Jakob Borg
4f6b86a1c0 cmd/stdiscosrv: Slightly tweak replication settings 2023-10-04 14:15:00 +02:00
bt90
06ac10ee37
cmd/stdiscosrv: Deduplicate addresses (fixes #8482) (#9080) 2023-09-06 14:36:00 +02:00
Jakob Borg
c6334e61aa
all: Support multiple device connections (fixes #141) (#8918)
This adds the ability to have multiple concurrent connections to a single device. This is primarily useful when the network has multiple physical links for aggregated bandwidth. A single connection will never see a higher rate than a single link can give, but multiple connections are load-balanced over multiple links.

It is also incidentally useful for older multi-core CPUs, where bandwidth could be limited by the TLS performance of a single CPU core -- using multiple connections achieves concurrency in the required crypto calculations...

Co-authored-by: Simon Frei <freisim93@gmail.com>
Co-authored-by: tomasz1986 <twilczynski@naver.com>
Co-authored-by: bt90 <btom1990@googlemail.com>
2023-09-06 12:52:01 +02:00
Jakob Borg
a80e6be353 cmd/stdiscosrv: Streamline context handling 2023-08-30 09:36:27 +02:00
Jakob Borg
acc532fc60 cmd/stdiscosrv: Explicitly enable HTTP/2
The server supports it, but it's not negotiated unless explicitly
allowed in the TLS config NextProtos.
2023-08-30 09:09:52 +02:00
Jakob Borg
a04cc95005 cmd/stdiscosrv: Separate HTTPS and replication certificates 2023-08-23 13:43:54 +02:00
Jakob Borg
480fa4b915 cmd/stdiscosrv: Use larger database settings 2023-08-23 13:43:14 +02:00
Jakob Borg
92a4931850 cmd/stdiscosrv: Modernise TLS settings, remove excessive HTTP logging 2023-08-23 13:39:52 +02:00
Jakob Borg
bdfef9010f cmd/stdiscosrv: Serve compressed responses 2023-08-23 13:39:14 +02:00
Jakob Borg
b49f535834 cmd/stdiscorv: Fix database test (fixes #8828)
The problem was that a statistics/cleanup run is triggered when the
database started and runs concurrently with the test. That cleanup run
removes old entries without valid addresses, and one of the test objects
matched this. The test object would thus randomly be removed in the
middle of the test, causing a failure. This fixes it so the object looks
recent when the cleaner-upper looks, and also uses a RAM database
(faster).
2023-03-24 09:15:12 +01:00
luzpaz
837ffcfab5
all: Fix various user-facing and non-user-facing typos (#8509)
Found via `codespell -q 3 -S lang,./gui/default/vendor -L benchs,bu,inflight,ro`
2022-08-23 15:44:11 +02:00
Jakob Borg
79f8bd0f33 all: Minor deprecation updates 2022-07-28 17:54:12 +02:00
deepsource-autofix[bot]
5130c414da
all: Unused parameter should be replaced by underscore (#8464)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-07-28 17:17:29 +02:00
Jakob Borg
479712cdc5
cmd/stdiscosrv: Don't start replication listener without peers (fixes #8143) (#8144)
The intention was that if no peers are given, we shouldn't start the
listener. We did that anyway, because:

- splitting an empty string on comma returns a slice with one empty
  string in it
- parsing the empty string as a device ID returns the empty device ID

so we end up with a valid replication peer which is the empty device ID.
2022-01-27 09:36:54 +01:00
Kebin Liu
083fa1803a
cmd/stdiscosrv: Support deploying behind Caddyserver (#8092) 2022-01-05 15:17:13 +01:00
greatroar
e7620e951d cmd/stdiscosrv: use strconv.Itoa 2021-11-27 15:35:07 +01:00
Simon Frei
5bdb6798a9
all: Regenerate proto (#7696) 2021-05-19 13:30:20 +02:00
Simon Frei
9524b51708
all: Implement suture v4-api (#6947) 2020-11-17 13:19:04 +01:00
Audrius Butkevicius
d507d932b8
all: Use protobuf to generate config structs (fixes #6734) (#6900) 2020-08-25 08:11:14 +02:00
Audrius Butkevicius
f310bbaaac
cmd/stdiscosrv: Don't abuse wrong header (#6749) 2020-06-16 10:58:25 +02:00
Audrius Butkevicius
f619a7f4cc
lib/connections: Try TCP punchthrough (fixes #4259) (#5753) 2020-06-16 09:17:07 +02:00
Jakob Borg
7fa699e159
build, lib/build: Build faster (#6538)
This changes the build script to build all the things in one go
invocation, instead of one invocation per cmd. This is a lot faster
because it means more things get compiled concurrently. It's especially
a lot faster when things *don't* need to be rebuilt, possibly because it
only needs to build the dependency map and such once instead of once per
binary.

In order for this to work we need to be able to pass the same ldflags to
all the binaries. This means we can't set the program name with an
ldflag.

When it needs to rebuild everything (go clean -cache):

    ( ./old-build -gocmd go1.14.2 build all 2> /dev/null; )  65.82s user 11.28s system 574% cpu 13.409 total
    ( ./new-build -gocmd go1.14.2 build all 2> /dev/null; )  63.26s user 7.12s system 1220% cpu 5.766 total

On a subsequent run (nothing to build, just link the binaries):

    ( ./old-build -gocmd go1.14.2 build all 2> /dev/null; )  26.58s user 7.53s system 582% cpu 5.853 total
    ( ./new-build -gocmd go1.14.2 build all 2> /dev/null; )  18.66s user 2.45s system 1090% cpu 1.935 total
2020-04-16 10:09:33 +02:00
Kevin Bushiri
5975772ed8
cmd/stdiscosrv: Only generate keypair if it doesn't exist (fixes #5809) (#6419) 2020-03-19 14:50:24 +01:00
Dan
ceb9475668 etc: Fix misleading comment in discosrv options file (#6258) 2020-01-06 22:43:41 +00:00
Jakob Borg
9084510e1b
cmd/stdiscosrv: Sort addresses before replication (fixes #6093) (#6094)
This makes sure addresses are sorted when coming in from the API. The
database merge operation still checks for correct ordering (which is
quick) and sorts if it isn't correct (legacy database record or
replication peer), but then does a copy first.

Tested with -race in production...
2019-10-18 10:50:19 +02:00
Jakob Borg
ad2d3702ae all: Upgrade github.com/gogo/protobuf and regenerate (fixes #6085) 2019-10-18 09:53:59 +02:00
Jakob Borg
4736cccda1
all: Update certificate lifetimes (fixes #6036) (#6078)
This adds a certificate lifetime parameter to our certificate generation
and hard codes it to twenty years in some uninteresting places. In the
main binary there are a couple of constants but it results in twenty
years for the device certificate and 820 days for the HTTPS one. 820 is
less than the 825 maximum Apple allows nowadays.

This also means we must be prepared for certificates to expire, so I add
some handling for that and generate a new certificate when needed. For
self signed certificates we regenerate a month ahead of time. For other
certificates we leave well enough alone.
2019-10-16 20:31:46 +02:00
Cyprien Devillez
6408a116f9 cmd/stdiscosrv: Add support for Traefik 2 as a reverse proxy (#6065) 2019-10-07 12:55:27 +01:00
Jakob Borg
67b8ef1f3e
cmd/*, lib/build: Set correct LongVersion (fixes #5993) (#5997)
The relay and discosrv didn't use the new lib/build package, now they
do. Conversely the lib/build package wasn't aware there might be other
users and hard coded the program name - now it's set by the build
script
2019-10-07 13:30:25 +02:00
Jakob Borg
80894948f6
build: Upgrade github.com/gogo/protobuf (#5994)
This is the result of:

- Changing build.go to take the protobuf version from the modules
  instead of hardcoded
- `go get github.com/gogo/protobuf@v1.3.0` to upgrade
- `go run build.go proto` to regenerate our code
2019-09-04 07:33:29 +01:00
Cromefire_
7b37d453f9 build, etc: Add systemd units and ufw rules for relay and discovery (fixes #5115) (#5350) 2019-08-08 18:04:52 +02:00
Jakob Borg
9fd270d78e
all: A few more interesting linter fixes (#5502)
A couple of minor bugs and simplifications
2019-02-02 12:09:07 +01:00
Jakob Borg
b01edca420
all: Update protobuf package 1.0.0 -> 1.2.0 (#5452)
Also adds a few file global options to keep the generated code similar
to what we already had.
2019-01-14 11:53:36 +01:00
Jakob Borg
944ddcf768
all: Become a Go module (fixes #5148) (#5384)
* go mod init; rm -rf vendor

* tweak proto files and generation

* go mod vendor

* clean up build.go

* protobuf literals in tests

* downgrade gogo/protobuf
2018-12-18 12:36:38 +01:00
Alexandre Viau
75f9ea623c cmd: Update prometheus_client (fixes #5280) (#5282) 2018-10-21 16:11:26 +01:00
Jakob Borg
8519a24ba6
cmd/*, lib/tlsutil: Refactor TLS stuff (fixes #5256) (#5276)
This changes the TLS and certificate handling in a few ways:

- We always use TLS 1.2, both for sync connections (as previously) and
  the GUI/REST/discovery stuff. This is a tightening of the requirements
  on the GUI. AS far as I can tell from caniusethis.com every browser from
  2013 and forward supports TLS 1.2, so I think we should be fine.

- We always greate ECDSA certificates. Previously we'd create
  ECDSA-with-RSA certificates for sync connections and pure RSA
  certificates for the web stuff. The new default is more modern and the
  same everywhere. These certificates are OK in TLS 1.2.

- We use the Go CPU detection stuff to choose the cipher suites to use,
  indirectly. The TLS package uses CPU capabilities probing to select
  either AES-GCM (fast if we have AES-NI) or ChaCha20 (faster if we
  don't). These CPU detection things aren't exported though, so the tlsutil
  package now does a quick TLS handshake with itself as part of init().
  If the chosen cipher suite was AES-GCM we prioritize that, otherwise we
  prefer ChaCha20. Some might call this ugly. I think it's awesome.
2018-10-21 14:17:50 +09:00
Jakob Borg
c1f1fd71fe cmd/stdiscosrv: Unflake test (fixes #5247) 2018-10-18 20:39:36 +09:00
Jakob Borg
9e00b619ab all, vendor: Switch back to non-forked thejerf/suture (#5171) 2018-09-08 12:56:56 +03:00
Jakob Borg
e384c822b6 cmd/stdiscosrv: Be more picky about allowed addresses (fixes #5151) (#5153)
Filter out ludicrous stuff both from explicitly announced addresses and
potential automatic replacements.
2018-08-30 18:06:35 +01:00
Paweł Rozlach
f6da436f4b cmd/stdiscosrv: Add replication heartbeats (fixes #5117) (#5120) 2018-08-15 16:52:20 +02:00
Jakob Borg
48795dba07
all: Don't let Suture capture panics (fixes #4758) (#5119)
Fork with new option.
2018-08-13 20:39:08 +02:00