The main() function is growing too big (142 lines as of the date of this
commit), so this attempts to extract some functionality out of there and
into their own functions to make it easier to reason about them and keep
functions short and concise.
--browser-only assumes syncthing is already running and will open the
user's browser in the URL:port currently used in the configuration if
the GUI is enabled.
Every time a JSON object is returned in an HTTP response, the
appropriate header needs to be set and the object itself needs to be
encoded. Doing this in every function is repetitive and error prone
(getDBFile and postDBScan, for instance, never set any headers).
This adds a helper function to centralize the appropriate JSON response
handling.
With this change, the behavior is as follows:
- SIGTERM / SIGINT: Exit cleanly (exit code 0)
- SIGHUP: Restart
This is the case both when hitting the monitor process or the inner
process, or if running with NORESTART (but then we won't restart,
obviously, just exit with code exitRestarting).
It enables "pkill -HUP syncthing" to do the right thing to restart all
Syncthings on package upgrade, for example.
Also fixes what I think migh thave been a bug where we did not use the
proxy for usage reports. And removes the BuildEnv field that we don't
need any more.
This replaces the current 3072 bit RSA certificates with 384 bit ECDSA
certificates. The advantage is these certificates are smaller and
essentially instantaneous to generate. According to RFC4492 (ECC Cipher
Suites for TLS), Table 1: Comparable Key Sizes, ECC has comparable
strength to 3072 bit RSA at 283 bits - so we exceed that.
There is no compatibility issue with existing Syncthing code - this is
verified by the integration test ("h2" instance has the new
certificate).
There are browsers out there that don't understand ECC certificates yet,
although I think they're dying out. In the meantime, I've retained the
RSA code for the HTTPS certificate, but pulled it down to 2048 bits. I
don't think a higher security level there is motivated, is this matches
current industry standard for HTTPS certificates.
- Move to ipinfo.io for geoip, rather than Telize. Telize has been closed
down. ipinfo.io has apparently got decent availability, and allows
1,000 requests per day on the free tier. Since requests are made by the
client, this should be more than enough (and the total across all clients
should still be less than this).
- Fix issue where one nonresponsive relay would cause 'data unavailable'
to be shown for many relays. This was caused by the relay status
promise not being correctly added to the list of things being waited
for before the map was rendered. Any delayed relay status requests
would therefore occur after the map was rendered, which was too late.
Knowing why a relay server failed to join the pool can be important. This
is typically an issue which must be investigated after it occurred, so
having logs available is useful.
Running with -debug permanently enabled is impractical, due to the amount
of traffic that is generated, particularly when data is being transferred.
Logging is limited to at most one message per minute, although one message
per hour is more likely.
This allows relaysrv to listen on an unprivileged port, with port
forwarding directing traffic from 443, thus providing an alternative
to using setcap cap_net_bind_service=+ep
We're going to need the db.Instance to keep some state, and for that to
work we need the same one passed around everywhere. Hence this moves the
leveldb-specific file opening stuff into the db package and exports the
dbInstance type.
'AlwaysLocalNets' was getting printed, but was getting used
when setting up connections. Now, the nets that should be
considered local are printed and used.
Overwriting configuration files is likely to happen if a
user syncs their home directories across computers. In this
case, the biggest risk is that all nodes will end up with
the same certificate and thus Device ID.
When the model prepares a folder for syncing, it checks to
see if the configuration files this instance is using are
getting synced. If the are getting synced, and they aren't
getting ignored, a warning is emitted. The model is used
so that when a new folder is added dynamically, a warning
is also emitted.
This will not prevent a user from shooting themselves in
the foot, and will not cover all cases (e.g. symlinks).
It should provide _something_ for many users in this
situation to go on, though.
Add WorkingDirectory to create and use the certificates within
/var/lib/syncthing-relaysrv. Add RootDirectory to chroot(2) the whole
thing into that directory.
This implements a new debug/trace infrastructure based on a slightly
hacked up logger. Instead of the traditional "if debug { ... }" I've
rewritten the logger to have no-op Debugln and Debugf, unless debugging
has been enabled for a given "facility". The "facility" is just a
string, typically a package name.
This will be slightly slower than before; but not that much as it's
mostly a function call that returns immediately. For the cases where it
matters (the Debugln takes a hex.Dump() of something for example, and
it's not in a very occasional "if err != nil" branch) there is an
l.ShouldDebug(facility) that is fast enough to be used like the old "if
debug".
The point of all this is that we can now toggle debugging for the
various packages on and off at runtime. There's a new method
/rest/system/debug that can be POSTed a set of facilities to enable and
disable debug for, or GET from to get a list of facilities with
descriptions and their current debug status.
Similarly a /rest/system/log?since=... can grab the latest log entries,
up to 250 of them (hardcoded constant in main.go) plus the initial few.
Not implemented in this commit (but planned) is a simple debug GUI
available on /debug that shows the current log in an easily pasteable
format and has checkboxes to enable the various debug facilities.
The debug instructions to a user then becomes "visit this URL, check
these boxes, reproduce your problem, copy and paste the log". The actual
log viewer on the hypothetical /debug URL can poll regularly for new log
entries and this bypass the 250 line limit.
The existing STTRACE=foo variable is still obeyed and just sets the
start state of the system.
Not necessarily the easiest way to fix just this bug, but the root cause
was using the (at that point uninitialized) cfg variable, so it seemed
sensible to just get rid of it to avoid that kind of crap.
The connections service no longer depends directly on the
syncthing model object, but on an interface instead. This
makes it drastically easier to write clients that handle
the model differently, but still want to benefit from
existing and future connections changes in the core.
This was motivated by burkemw3's interest in creating a
FUSE client that can present a view of the global model,
but not have all of the file data locally.
The actual decoupling was done by adding a connections.Model
interface. This interface is effectively an extension of the
protocol.Model interface that also handles connections
alongside the modified service.
This makes it so we can initialize the relay management and then give
that to the connection management, instead of the other way around.
This is important to me in the discovery revamp I'm doing, as otherwise
I get a circular dependency when constructing stuff, with relaying
depending on connection, connection depending on discovery, and
discovery depending on relaying.
With this fixed, discovery will depend on relaying, and connection will
depend on both discovery and relaying.
* syncthing/pr/1995:
Add switch to disable relays
Do not start relay service unless explicitly asked for, or global announcement server is running
Add dynamic relay lookup (DDoS relays.syncthing.net!)
Discovery clients now take an announcer, global discovery is delayed
Expose connection type and relay status in the UI
Add dependencies (fixes#1364)
Check relays for available devices
Add incoming connection relay service
Add unsubscribe to config
Connections have types
Large refactoring/feature commit
1. Change listen addresses to URIs
2. Break out connectionSvc to support listeners and dialers based on schema
3. Add relay announcement and lookups part of discovery service
I figured we're missing out on being cool and awesome by not having an
alphabetically based release code name like the big guys. This commit
fixes that. I've unilaterally decided on a theme of "$metal $bug"
because metals are kind of cool, and bugs, well, ...
This will decrease the risk of running out of file descriptors for the
database and other bad things, which could otherwise potentially happen
if we're serving lots of requests and scanning in parallel, etc.
Windows doesn't have a per process open file limit like Unix so we don't
need to worry about it there.
- Move the Go files into script/ instead of random places
- Rewrite check-contrib.sh into check-authors.go and check-copyright.go
- Clean up build.sh a little bit
This sends the Cache-Control header to allow caching of static resources,
and checks the If-Modified-Since header to allow browser to use the
cached resource on refresh. Also fixes some paths that caused redirects
(core//foo -> core/foo)
This captures the common pattern of writing to a temp file and moving it
to it's real name only if everything went well. It reduces the amount of
code in some places where we do this, but maybe not as much as I
expected because the upgrade thing is still a special snowflake...
This captures the common pattern of writing to a temp file and moving it
to it's real name only if everything went well. It reduces the amount of
code in some places where we do this, but maybe not as much as I
expected because the upgrade thing is still a special snowflake...
The reset of all folders failed when there was no data for a given
folder, as it was not returned by db.ListFolders then. But we don't
really care about that, we can "reset" it anyway...
The API never got the first few events ("Starting" etc) as it subscribed
too late. Instead, set up a subscription for it early on. If the API is
configured not to run this is unnecessary but doesn't hurt very much.
By doing it init(), the monitor process also gets a subscription thing
running, which is unnecessary (and really confused me when seeing it in
the debug output).
Prior to this change we schedule summaries on each state change, i.e.
scanning->idle and idle->scanning, which is unnecessary. Now we only do
it on index updates, plus the immediate one on going syncing->idle.
The "Invalid" config attribute is retained for errors discovered during
config loading (empty path, duplicate ID). This can only be set or
cleared at config loading time.
Errors discovered during runtime (I/O problems, etc) are now in the
folder state instead. Changes to these are sent as any other folder
state change.
This reverts the GC behavior to the Go default of triggering GC when the
heap has grown 100% compared to after the previous GC. We were setting
this to 25% to keep memory usage at a minimum, but it has a pretty
severe performance cost (especially when syncing large files) as we keep
triggering GC too often.
This documents the tweak in the `-help` message so users can decide for
themselves, and sticks to whatever the Go runtime developers thinks is
best for the default.
Replace the current mix of UpperCamelCase und lowerCamelCase with
consistent lowerCamelCase keys for the JSON API. Also adapt the frontend
so it works with the changed API.
Attention: this will break existing consumers of the API.
Integers are for numbers, enabling arithmetic like subtractions and for
loops without getting shot in the foot. Unsigneds are for bitfields.
- "int" for numbers that will always be laughably smaller than four
billion, and where we don't care about the serialization format.
- "int32" for numbers that will always be laughably smaller than four
billion, and will be serialized to four bytes.
- "int64" for numbers that may approach four billion or will be
serialized to eight bytes.
- "uint32" and "uint64" for bitfields, depending on required number of
bits and serialization format. Likewise "uint8" and "uint16", although
rare in this project since they don't exist in XDR.
- "int8", "int16" and plain "uint" are almost never useful.
