Commit Graph

935 Commits

Author SHA1 Message Date
Jakob Borg
44d30c83bf lib/config, cmd/syncthing: Handle committing configuration better (fixes #3077)
This slightly changes the interface used for committing configuration
changes. The two parts are now:

 - VerifyConfiguration, which runs synchronously and locked, and can
   abort the config change. These callbacks shouldn't *do* anything
   apart from looking at the config changes and saying yes or no. No
   change from previously.

 - CommitConfiguration, which runs asynchronously (one goroutine per
   call) *after* replacing the config and releasing any locks. Returning
   false from these methods sets the "requires restart" flag, which now
   lives in the config.Wrapper.

This should be deadlock free as the CommitConfiguration calls can take
as long as they like and can wait for locks to be released when they
need to tweak things. I think this should be safe compared to before as
the CommitConfiguration calls were always made from a random background
goroutine (typically one from the HTTP server), so it was always
concurrent with everything else anyway.

Hence the CommitResponse type is gone, instead you get an error back on
verification failure only, and need to explicitly check
w.RequiresRestart() afterwards if you care.

As an added bonus this fixes a bug where we would reset the "requires
restart" indicator if a config that did not require restart was saved,
even if we already were in the requires-restart state.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3386
2016-07-04 20:32:34 +00:00
Jakob Borg
7ff7b55732 cmd/strelaypoolsrv: Remove unused var (metalint) 2016-07-04 21:22:53 +02:00
Jakob Borg
44346b3a5a cmd/strelaypoolsrv: Fixup import in main 2016-07-04 14:58:29 +02:00
Jakob Borg
ce9dc809bc build, cmd/strelaypoolsrv: Build assets using standard script 2016-07-04 13:34:44 +02:00
Jakob Borg
59ce7c0424 cmd/strelaypoolsrv: Merge relaypoolsrv repo into main
* relaypoolsrv/master: (32 commits)
  Fetch deps of deps X_x
  Here we go with gvt bugs
  Screw godep
  Add solaris support back in
  Add font awesome
  No value is less than zero
  Screw solaris
  Godeps
  Refactor javascript, always show table, add sorting
  Add local geoip
  Update dependencies
  Hey look, had to check all code out on linux to fix the deps
  Update godeps, reduce amount of time spent testing a relay. Goddamit godeps.
  Add timeouts, deal with overlapping markers, add a table, increase circle radiuses
  Fix a couple of issues with the relays map (geoip, 'data unavailable')
  Rate infos are in kbps, not kBps
  Add support for header holding IP address
  Update relay parameters even if it already exists (fixes #3)
  Add missing space
  Add homepage
  ...
2016-07-04 13:33:57 +02:00
Jakob Borg
8d0019595f cmd/syncthing: Update code name for v0.14
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3384
2016-07-04 10:58:45 +00:00
aviau
6ff74cfcab build, cmd/stdiscosrv, cmd/strelaysrv: Rename binaries to add "st" prefix
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3371
2016-07-04 10:51:22 +00:00
Jakob Borg
fa0101bd60 lib/protocol, lib/discover, lib/db: Use protocol buffer serialization (fixes #3080)
This changes the BEP protocol to use protocol buffer serialization
instead of XDR, and therefore also the database format. The local
discovery protocol is also updated to be protocol buffer format.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3276
LGTM: AudriusButkevicius
2016-07-04 10:40:29 +00:00
Cedric Staniewski
a58f69be04 cmd/discosrv: Respect the listen address scheme (fixes #3346)
If the listen address scheme is set to tcp4:// or tcp6://, it needs to be
made sure that the remote address matches this scheme before it is added to
the database.

This prevents invalid URIs like tcp4://<IPv6 address>:<port> or tcp6://<IPv4
address>:<port>.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3378
2016-07-03 11:19:12 +00:00
Phil Davis
e194eb1f69 cmd/relaysrv: Typos in options
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3377
2016-07-03 08:44:41 +00:00
Jakob Borg
6d357211b2 lib/config: Remove "Invalid" attribute (fixes #2471)
This contains the following behavioral changes:

 - Duplicate folder IDs is now fatal during startup
 - Invalid folder flags in the ClusterConfig is fatal for the connection
   (this will go away soon with the proto changes, as we won't have any
   unknown flags any more then)
 - Empty path is a folder error reported at runtime

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3370
2016-07-02 19:38:39 +00:00
Lars K.W. Gohlke
af0bc95de5 lib/model: Refactor encapsulation of the folder scanning
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3017
2016-06-29 06:37:34 +00:00
Antony Male
7ef2743964 lib/events: Introduce per-subscription event IDs (fixes #3335)
Events API consumers rely on being able to detect that events were skipped
by the fact that the event ID has increased by more than 1. This is
documented, and is absolutely necessary when trying to maintain a local
model of Syncthing's state.

With the introduction of LocalChangeDetected, which is not exposed to the
Events API, this contract was broken.

This commit introduces separate concepts of a "Global ID" and a
"Subscription ID". The Global ID of an event is unique across all
subscriptions. The Subscription ID is local to a particular subscription,
and always increments by 1. They are both exposed over the Events API, but
the Subscription ID uses the key "id" for backwards compatibility, and
the "?since=xx" parameter refers to the Subscription ID (making the Global
ID for information only).

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3351
LGTM: calmh
2016-06-27 21:18:58 +00:00
Jakob Borg
9323f0faf8 lib/model: Refactor CheckFolderHealth into separate methods
While attempting to fix #2782 I thought the problem was the
CheckFolderHealth method, so I cleaned it up. That turned out not to be
the case, but I think this is better anyhow.

It also moves the "create folder and marker if the folder was empty in
the index" code to StartFolder where I think it makes better sense.

This is covered by a number of existing tests.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3343
2016-06-26 10:07:27 +00:00
Audrius Butkevicius
502bee9a09 lib/osutil: Return "/" as filesystem root on non-windows (fixes #3321)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3332
2016-06-20 20:25:00 +00:00
Daniel Harte
e71c78ae84 cmd/syncthing: Remove folder limit on /rest/system/browse
Previously limited to 10 results, now unlimited.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3279
LGTM: calmh, AudriusButkevicius
2016-06-08 07:09:50 +00:00
Jakob Borg
03a8027efc cmd/syncthing: Refactor out staticsServer (prev. embeddedStatic) a bit
The purpose of this operation is to separate the serving of GUI assets a
bit from the serving of the REST API. It's by no means complete. The end
goal is something like a combined server type that embeds a statics
server and an API server and wraps it in authentication and HTTPS and
stuff, plus possibly a named pipe server that only provides the API and
does not wrap in the same authentication etc.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3273
2016-06-07 07:46:45 +00:00
Jakob Borg
b7e186b370 cmd/discosrv: Fix lint warnings
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3275
2016-06-07 07:33:11 +00:00
Jakob Borg
4a69f3987f cmd/relaysrv: Fix lint warnings
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3274
2016-06-07 07:31:43 +00:00
Jakob Borg
5aacfd1639 cmd/syncthing: Make API serve loop more robust (fixes #3136)
This sacrifices the ability to return an error when creating the service
for being more persistent in keeping it running.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3270
LGTM: AudriusButkevicius, canton7
2016-06-06 22:12:23 +00:00
Jakob Borg
287cfee73c cmd/syncthing: Re-enable auto upgrade for dev builds (fixes #901)
As noted in the ticket I no longer agree that dev builds should not auto
upgrade. The main reason is that we give dev builds to users to test
specific fixes, and noone is happier by them being inadvertently stuck
on that version when a newer version including the fix is released.

For developers, it's first of all probably unlikely that development is
happening on a build that's older than release, and secondly STNOUPGRADE
can be set in the environment once and for all if it an issue.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3244
2016-06-02 13:01:59 +00:00
Jakob Borg
a6c465e929 cmd/relaysrv: Go 1.3 fix (we should probably drop that compatibility soon) 2016-06-02 14:48:25 +02:00
Audrius Butkevicius
becb5ab1dc cmd/relaysrv: Missed changes in the repo merge (README, systemd) 2016-06-02 14:42:57 +02:00
Audrius Butkevicius
49170bf2d8 cmd/relaysrv: Add number of routines 2016-06-02 14:39:19 +02:00
Majed Abdulaziz
b1205db7ac cmd/discosrv: Accept host names in announced addresses
GitHub-Pull-Request: https://github.com/syncthing/discosrv/pull/48
2016-06-02 14:34:04 +02:00
Jakob Borg
7a56e4a0e5 cmd/relaysrv: Copyright headers 2016-06-02 14:16:02 +02:00
Jakob Borg
d17608d0a0 cmd/relaysrv: vet: composite literal uses unkeyed fields 2016-06-02 14:10:55 +02:00
Jakob Borg
0af216fea0 cmd/relaysrv: Add build stamped version, print at startup 2016-06-02 14:09:36 +02:00
Jakob Borg
56a9964101 cmd/relaysrv: Merge relaysrv repo
* relaysrv/master: (60 commits)
  Add new dependencies
  Add more logging in the case of relaypoolsrv internal server error
  Dependency update
  Update deps
  Update packages, fix testutil. Goddamit godep.
  Typo
  Add signal handlers (fixes #15)
  Update readme (fixes #16)
  Limit number of connections (fixes #23)
  Enable extra logging in pool.go even when -debug not specified
  Add Antony Male to CONTRIBUTORS
  Allow extAddress to be set from the command line
  URLs should have Go units
  Add CORS headers
  Fix units
  Expose provided by in status endpoint
  Add ability to advertise provider
  Change the URL
  Rename relaysrv binary, see #11
  Jail the whole thing a bit more
  ...
2016-06-02 14:04:22 +02:00
Jakob Borg
532b4383bf cmd/discosrv: Add build stamped version, print at startup 2016-06-02 13:58:39 +02:00
Jakob Borg
7035ea3ab7 cmd/discosrv: Merge discosrv repo
* discosrv/master: (64 commits)
  Use atomics for statistics handling (fixes #45)
  Lower case JSON fields are nicer
  Change v13 to v2
  Remove explicit relay handling
  Update vendored github.com/cznic/ql (fixes #34)
  Defer fd.Close() (fixes #37)
  There is no "get dependencies" step
  Add vendor/golang.org/x/net/context
  Use Go 1.5 vendoring instead of Godeps
  Add debug performance logging per request
  Must close result sets
  Set Retry-After header
  Ignores
  lru.Cache is not concurrency safe
  We need a limit on the number of PostgreSQL connections
  Correct example DSN (fixes #29)
  Allow plain HTTP serving behind a proxy
  Fix Query/Answer stats
  Reduce our patience with slow clients somewhat
  Discovery server should print device ID of certificate at startup
  ...
2016-06-02 13:51:17 +02:00
Audrius Butkevicius
6f63909c65 lib/db,cmd/stindex: Expose VersionList and use it in stindex
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3231
2016-05-31 19:29:26 +00:00
Audrius Butkevicius
1612baca92 gui: /rest/system/browse with no arguments returns drives on Windows (ref #3201)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3203
2016-05-31 19:27:07 +00:00
Antony Male
6361172bea cmd/syncthing: Be more explicit about how assets should be cached (fixes #3182)
With the previous setup, browsers were free to use a local cache for any
length of time they pleased: we didn't set an 'Expires' header (or max-age
directive), and Cache-Control just said "you're free to cache this".

Therefore be more explicit: we don't mind if browsers cache things, but they
MUST revalidate everything on every request.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3221
2016-05-30 13:54:55 +00:00
Alexander Graf
7373d2eb3c cmd/syncthing: Fix upgrade of running syncthing from CLI (fixes #3193)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3198
2016-05-28 14:08:26 +00:00
Jakob Borg
da5010d37a cmd/syncthing: Use API to generate API Key and folder ID (fixes #3179)
Expose a random string generator in the API and use it when the GUI
needs random strings for API key and folder ID.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3192
2016-05-26 07:25:34 +00:00
Jakob Borg
e6b78e5d56 lib/rand: Break out random functions into separate package
The intention for this package is to provide a combination of the
security of crypto/rand and the convenience of math/rand. It should be
the first choice of random data unless ultimate performance is required
and the usage is provably irrelevant from a security standpoint.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3186
2016-05-26 07:02:56 +00:00
Audrius Butkevicius
410d700ae3 cmd/syncthing: Do not modify events (fixes #3002)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3190
2016-05-26 06:54:44 +00:00
Jakob Borg
98734375f2 cmd/syncthing: Correctly set, parse and compare modified time HTTP headers (fixes #3165)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3167
2016-05-23 12:16:14 +00:00
Jakob Borg
4bc2b3f369 gui: Set CSRF stuff earlier (fixes #3138)
We need to set these properties *before* Angular starts making requests,
and doing that from the response to a request is too late. The obvious
choice (to me) would be to use the angular $cookies service, but that
service isn't available until after initialization so we can't use it.
Instead, add a special file that is loaded by index.html and includes
the info we need before the JS app even starts running.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3152
2016-05-22 10:26:09 +00:00
Jakob Borg
a0ec4467fd cmd/syncthing: Emit new RemoteDownloadProgress event to track remote download progress
Without this the summary service doesn't know to recalculate completion
percentage for remote devices when DownloadProgress messages come in.
That means that completion percentage isn't updated in the GUI while
transfers of large files are ongoing. With this change, it updates
correctly.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3144
2016-05-22 07:52:08 +00:00
Jakob Borg
7dddc0de9e Use atomics for statistics handling (fixes #45)
This is one of those rare cases where that's actually cleaner, I
think...
2016-05-22 09:24:11 +09:00
Jakob Borg
bf7fcc612d cmd/syncthing: Enforce stricter CSRF policy on /rest GET requests (fixes #3134)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3137
2016-05-21 13:48:55 +00:00
Jakob Borg
1dbc310c9b cmd/syncthing: Rename event LocalDiskUpdated -> LocalChangeDetected
I think this better reflects what it means. Also tweaks the verbose
format to be more like our other things and lightly refactors the code
to not have the boolean and include the folder in the event.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3121
2016-05-19 07:01:43 +00:00
Nate Morrison
86ca58e2a9 lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-19 00:19:26 +00:00
Jakob Borg
6f743f3138 Revert "lib/model: Emit LocalDiskUpdated events on detecting local changes"
This reverts commit 5a7fad0bcd.
2016-05-14 10:55:24 +02:00
Nate Morrison
5a7fad0bcd lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-14 08:37:07 +00:00
Jakob Borg
0761d804a4 cmd/syncthing: Use random folder ID for default folder, limit random charset
This uses the same charset as the Javascript code, excluding confusing
characters like 0, O, I, 1, l etc.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3064
2016-05-09 09:43:40 +00:00
klemens
bd41e21c26 all: Correct spelling in comments
Skip-check: authors pr-build-mac

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3056
2016-05-08 10:54:22 +00:00
Jakob Borg
39899e40bf cmd/syncthing: Use ReadAll + json.Unmarshal in places were we care about consuming the reader
Because json.NewDecoder(r).Decode(&v) doesn't necessarily consume all
data on the reader, that means an HTTP connection can't be reused. We
don't do a lot of HTTP traffic where we read JSON responses, but the
discovery is one such place. The other two are for POSTs from the GUI,
where it's not exactly critical but still nice if the connection still
can be keep-alive'd after the request as well.

Also ensure that we call req.Body.Close() for clarity, even though this
should by all accounts not really be necessary.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3050
2016-05-06 22:01:56 +00:00