Commit Graph

264 Commits

Author SHA1 Message Date
Simon Frei
e7ae851900 lib/model: Debug and test fixes (#5613) 2019-03-22 14:43:47 +01:00
Simon Frei
289a02e994 lib/model: Integrate stat refs in folder (#5596) 2019-03-11 16:57:21 +00:00
Simon Frei
445637ebec lib/model: Pass fset & ignores on folder creation (#5592) 2019-03-11 07:28:54 +01:00
Simon Frei
3f3d2c814b
lib/model: Remove unused code (#5591) 2019-03-10 17:05:39 +01:00
Jakob Borg
9da3273eb8 lib/model: Clarify fileInfoBatch.flushIfFull criteria 2019-03-05 21:34:04 +01:00
Jakob Borg
f24676ba5a
lib/tlsutil: Enable TLS 1.3 when available, on test builds (fixes #5065) (#5558)
* lib/tlsutil: Enable TLS 1.3 when available, on test builds (fixes #5065)

This enables TLS 1.3 negotiation on Go 1.12 by setting the GODEBUG
variable. For now, this just gets enabled on test versions (those with a
dash in the version number).

Users wishing to enable this on production builds can set GODEBUG
manually.

The string representation of connections now includes the TLS version
and cipher suite. This becomes part of the log output on connections.
That is, when talking to an old client:

    Established secure connection .../TLS1.2-TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256

and now potentially:

    Established secure connection .../TLS1.3-TLS_AES_128_GCM_SHA256

(The cipher suite was there previously in the log output, but not the
TLS version.)

I also added this info as a new Crypto() method on the connection, and
propagate this out to the API and GUI, where it can be seen in the
connection address hover (although with bad word wrapping sometimes).

* wip

* wip
2019-02-26 11:49:02 +01:00
Simon Frei
722b3fce6a all: Hide implementations behind interfaces for mocked testing (#5548)
* lib/model: Hide implementations behind interfaces for mocked testing

* review
2019-02-26 08:09:25 +00:00
Simon Frei
d5ff2c41dc all: Get rid of fatal logging (#5537)
* cleanup Fatal in lib/config/config.go

* cleanup Fatal in lib/config/folderconfiguration.go

* cleanup Fatal in lib/model/model.go

* cleanup Fatal in cmd/syncthing/monitor.go

* cleanup Fatal in cmd/syncthing/main.go

* cleanup Fatal in lib/api

* remove Fatal methods from logger

* lowercase in errors.Wrap

* one less channel
2019-02-14 20:29:14 +00:00
Jakob Borg
04fdafa280 lib/db, lib/model: Remove dead code (#5517) 2019-02-08 16:42:58 +01:00
Simon Frei
5df8219bcb lib/model: Add progressEmitter to supervisor (model) (#5510) 2019-02-05 18:02:36 +00:00
Jakob Borg
c2ddc83509 all: Revert the underscore sillyness 2019-02-02 12:16:27 +01:00
Jakob Borg
9fd270d78e
all: A few more interesting linter fixes (#5502)
A couple of minor bugs and simplifications
2019-02-02 12:09:07 +01:00
Jakob Borg
2111386ee4
all: Fix some linter errors (#5499)
I'm working through linter complaints, these are some fixes. Broad
categories:

1) Ignore errors where we can ignore errors: add "_ = ..." construct.
you can argue that this is annoying noise, but apart from silencing the
linter it *does* serve the purpose of highlighting that an error is
being ignored. I think this is OK, because the linter highlighted some
error cases I wasn't aware of (starting CPU profiles, for example).

2) Untyped constants where we though we had set the type.

3) A real bug where we ineffectually assigned to a shadowed err.

4) Some dead code removed.

There'll be more of these, because not all packages are fixed, but the
diff was already large enough.
2019-02-02 10:11:42 +01:00
Simon Frei
24ffd8be99 all: Send Close BEP msg on intentional disconnect (#5440)
This avoids waiting until next ping and timeout until the connection is actually
closed both by notifying the peer of the disconnect and by immediately closing
the local end of the connection after that. As a nice side effect, info level
logging about dropped connections now have the actual reason in it, not a generic
timeout error which looks like a real problem with the connection.
2019-01-09 17:31:09 +01:00
Jakob Borg
1e71b00936
lib/model: Sanitize paths used for auto accepted folders (fixes #5411) (#5435) 2019-01-05 18:10:02 +01:00
Simon Frei
47e08797cb lib/model: Don't pull if ignores failed to load and cleanup (#5418) 2019-01-01 10:17:14 +01:00
Simon Frei
a09079ed25 all: Display list of locally changed items in UI (fixes #5336) (#5337) 2018-12-11 09:59:04 +01:00
Audrius Butkevicius
ff2cde469e lib/model: Allow limiting number of concurrent scans (fixes #2760) (#4888) 2018-12-05 08:40:05 +01:00
Simon Frei
2f9840ddae lib: Introduce fs.IsParent (fixes #5324) (#5326) 2018-11-22 11:16:45 +01:00
Simon Frei
4f27bdfc27 lib/model, lib/protocol: Handle request concurrency in model (#5216) 2018-11-13 08:53:55 +01:00
Simon Frei
d510e3cca3 all: Display errors while scanning in web UI (fixes #4480) (#5215) 2018-11-07 11:04:41 +01:00
Simon Frei
d10773c311 lib/db, lib/model: Resolve identical recv only items (fixes #5130) (#5230) 2018-10-10 12:43:07 +02:00
Jakob Borg
b50d57b7fd
lib/db: Refactor: use a Lowlevel type underneath Instance (ref #5198) (#5212)
This adds a thin type that holds the state associated with the
leveldb.DB, leaving the huge Instance type more or less stateless. Also
moves some keying stuff into the DB package so that other packages need
not know the keying specifics.

(This does not, yet, fix the cmd/stindex program, in order to keep the
diff size down. Hence the keying constants are still exported.)
2018-10-10 11:34:24 +02:00
Jakob Borg
f528923d1e lib/model, cmd/syncthing: Wait for folder restarts to complete (fixes #5233) (#5239)
* lib/model, cmd/syncthing: Wait for folder restarts to complete (fixes #5233)

This is the somewhat ugly - but on the other hand clear - fix for what
is really a somewhat thorny issue. To avoid zombie folder runners a new
mutex is introduced that protects the RestartFolder operation. I hate
adding more mutexes but the alternatives I can think of are worse.

The other part of it is that the POST /rest/system/config operation now
waits for the config commit to complete. The point of this is that until
the commit has completed we should not accept another config commit. If
we did, we could end up with two separate RestartFolders queued in the
background. While they are both correct, and will run without
interfering with each other, we can't guarantee the order in which they
will run. Thus it could happen that the newer config got committed
first, and the older config commited after that, leaving us with the
wrong config running.

* test

* wip

* hax

* hax

* unflake test

* per folder mutexes

* paranoia

* race
2018-10-05 09:26:25 +01:00
Simon Frei
a57fa9cfab lib/model: Polish (#5189) 2018-09-13 18:17:13 +02:00
Jakob Borg
9e00b619ab all, vendor: Switch back to non-forked thejerf/suture (#5171) 2018-09-08 12:56:56 +03:00
Jakob Borg
4cb6bb6f64 Merge branch 'release'
* release:
  lib/db: Fix inconsistency in sequence index (fixes #5149) (#5158)
2018-09-02 21:05:53 +02:00
Jakob Borg
b80da29b23 lib/db: Fix inconsistency in sequence index (fixes #5149) (#5158)
The problem here is that we would update the sequence index before
updating the FileInfos, which would result in a high sequence number
pointing to a low-sequence FileInfo. The index sender would pick up the
high sequence number, send the old file, and think everything was good.
On the receiving side the old file is a no-op and ignored. The file
remains out of sync until another update for it happens.

This fixes that by correcting the order of operations in the database
update: first we remove old sequence index entries, then we update the
FileInfos (which now don't have anything pointing to them) and then we
add the sequence indexes (which the index sender can see).

The other option is to add "proper" transactions where required at the
database layer. I actually have a branch for that, but it's literally
thousands of lines of diff and I'm putting that off for another day as
this solves the problem...
2018-09-02 21:02:28 +02:00
Jakob Borg
836ca50570
lib/db: Fix inconsistency in sequence index (fixes #5149) (#5158)
The problem here is that we would update the sequence index before
updating the FileInfos, which would result in a high sequence number
pointing to a low-sequence FileInfo. The index sender would pick up the
high sequence number, send the old file, and think everything was good.
On the receiving side the old file is a no-op and ignored. The file
remains out of sync until another update for it happens.

This fixes that by correcting the order of operations in the database
update: first we remove old sequence index entries, then we update the
FileInfos (which now don't have anything pointing to them) and then we
add the sequence indexes (which the index sender can see).

The other option is to add "proper" transactions where required at the
database layer. I actually have a branch for that, but it's literally
thousands of lines of diff and I'm putting that off for another day as
this solves the problem...
2018-09-02 20:58:32 +02:00
Audrius Butkevicius
aec66045ef
lib/config: Rewrite pending notifications (fixes #2291) 2018-08-25 11:36:10 +01:00
Simon Frei
03c0537340 lib/model: Fix regressions detecting deletes/ignores (fixes #5125, fixes #5127) (#5129) 2018-08-25 10:32:35 +02:00
Jakob Borg
a02db70a63 lib/model: Process download progress messages for all folder types (fixes #5131) (#5139) 2018-08-21 18:49:35 +01:00
Simon Frei
7c0798b622 lib/model: Always release the lock (#5126) 2018-08-15 16:44:59 +02:00
Simon Frei
ee42c46bd3 lib/model: Catch racy nil deref in ClusterConfig (#5106) 2018-08-15 16:44:20 +02:00
Simon Frei
885f6fcf28 lib/model: Always release the lock (#5126) 2018-08-15 16:33:03 +02:00
Jakob Borg
48795dba07
all: Don't let Suture capture panics (fixes #4758) (#5119)
Fork with new option.
2018-08-13 20:39:08 +02:00
Simon Frei
dfe4008607 lib/model: Catch racy nil deref in ClusterConfig (#5106) 2018-08-11 09:10:29 +02:00
Jakob Borg
f822b10550
all: Add receive only folder type (#5027)
Adds a receive only folder type that does not send changes, and where the user can optionally revert local changes. Also changes some of the icons to make the three folder types distinguishable.
2018-07-12 11:15:57 +03:00
Simon Frei
0f0290d574 lib/model, lib/weakhash: Abort pulling quicker on folder stop (ref #5028) 2018-07-04 08:07:33 +01:00
Simon Frei
7b0d8c2e77 lib/model: Release both locks when waiting for services to stop (fixes #5028) 2018-06-24 16:55:28 +01:00
Jakob Borg
b1b68ceedb
Add LocalFlags to FileInfo (#4952)
We have the invalid bit to indicate that a file isn't good. That's enough for remote devices. For ourselves, it would be good to know sometimes why the file isn't good - because it's an unsupported type, because it matches an ignore pattern, or because we detected the data is bad and we need to rescan it.

Or, and this is the main future reason for the PR, because it's a change detected on a receive only device. We will want something like the invalid flag for those changes, but marking them as invalid today means the scanner will rehash them. Hence something more fine grained is required.

This introduces a LocalFlags fields to the FileInfo where we can stash things that we care about locally. For example,

    FlagLocalUnsupported = 1 << 0 // The kind is unsupported, e.g. symlinks on Windows
    FlagLocalIgnored     = 1 << 1 // Matches local ignore patterns
    FlagLocalMustRescan  = 1 << 2 // Doesn't match content on disk, must be rechecked fully

The LocalFlags fields isn't sent over the wire; instead the Invalid attribute is calculated based on the flags at index sending time. It's on the FileInfo anyway because that's what we serialize to database etc.

The actual Invalid flag should after this just be considered when building the global state and figuring out availability for remote devices. It is not used for local file index entries.
2018-06-24 09:50:18 +02:00
Jakob Borg
35a75a95dc lib/model: Don't panic when rechecking file (fixes #5002) (#5003) 2018-06-13 18:07:52 +01:00
Simon Frei
1e2732aa21
lib/config, lib/model: Don't warn and return error (#4997) 2018-06-10 15:41:20 +02:00
Simon Frei
b7234785f8
lib/model: Wait for folder to stop (fixes #4981) (#4982) 2018-06-10 13:24:59 +02:00
Simon Frei
e2c44f519c
lib/config, lib/model: Handle shared with information in config (fixes #4870) (#4974) 2018-06-06 23:34:11 +02:00
Jakob Borg
119335930c lib/model: Refactor override implementation into sendOnlyFolder
I'm trying to slowly clean this up a bit, and moving functionality out
into the folder types and having those methods not reach into model is
part of it. That can mean takign some odd arguments in the meantime,
some of those should probably become interfaces or properties on folder
in the long term.
2018-05-21 09:09:47 +02:00
Simon Frei
d64d954721 lib/db: Fix prefixed walks (fixes #4925) (#4940) 2018-05-17 09:26:40 +02:00
Audrius Butkevicius
a48a31e3f5 lib/ignores: Fix ignore loading, report errors to UI (fixes #4901) (#4932) 2018-05-08 23:37:13 +02:00
Jakob Borg
2343c82c33 lib/model: Don't include unshared folders in ClusterConfig (fixes #4926)
Also fixes a data race where ClusterConfig would access folderFiles
without a lock. Tweaked the ClusterConfig unit test to verify the
behavior.
2018-05-08 08:19:34 +01:00
Simon Frei
a94aceb22f lib/model: Don't create folder root when paused (fixes #4903) (#4904) 2018-05-05 10:30:39 +02:00