Simon Frei
e782bab9fc
lib/config: Add some info to the folder marker missing (ref #5207 ) ( #6270 )
2020-01-16 15:30:29 +01:00
Simon Frei
71882765f2
lib/events, lib/model: Unflake test and prevent deadlock on event unsubscribing ( #6261 )
2020-01-11 08:14:05 +01:00
Simon Frei
08753ccabe
lib/model: Reset queue after all pulling is done ( fixes #5867 ) ( #6256 )
2020-01-08 12:21:22 +01:00
Simon Frei
f56a5545d4
gui, lib/model: Prevent negative sync completion ( fixes #4570 ) ( #6248 )
2020-01-03 14:07:57 +01:00
Jakob Borg
be0508cf26
lib/model, lib/protocol: Use error handling to avoid panic on non-started folder ( fixes #6174 ) ( #6212 )
...
This adds error returns to model methods called by the protocol layer.
Returning an error will cause the connection to be torn down as the
message couldn't be handled. Using this to signal that a folder isn't
currently available will then cause a reconnection a few moments later,
when it'll hopefully work better.
Tested manually by running with STRECHECKDBEVERY=0 on a nontrivially
sized setup. This panics reliably before this patch, but just causes a
disconnect/reconnect now.
2019-12-04 10:46:55 +01:00
Simon Frei
6fd5e78740
lib: Consistently unsubscribe from config-wrapper ( fixes #6133 ) ( #6205 )
2019-12-04 07:15:00 +01:00
Marcus Legendre
b3fd9a8d53
lib/ignore: Don't create empty ".stignore" files ( fixes #6190 ) ( #6197 )
...
This will:
1. prevent creation of a new .stignore if there are no ignore patterns
2. delete an existing .stignore if all ignore patterns are removed
2019-12-02 08:19:02 +01:00
Jakob Borg
e82a7e3dfa
all: Propagate errors from NamespacedKV ( #6203 )
...
As foretold by the prophecy, "once the database refactor is merged, then
shall appear a request to propagate errors from the store known
throughout the land as the NamedspacedKV, and it shall be good".
2019-11-30 13:03:24 +01:00
Jakob Borg
928767e316
lib/model: gofmt lol :(
2019-11-29 09:29:59 +01:00
Jakob Borg
c71116ee94
Implement database abstraction, error checking (ref #5907 ) ( #6107 )
...
This PR does two things, because one lead to the other:
- Move the leveldb specific stuff into a small "backend" package that
defines a backend interface and the leveldb implementation. This allows,
potentially, in the future, switching the db implementation so another
KV store should we wish to do so.
- Add proper error handling all along the way. The db and backend
packages are now errcheck clean. However, I drew the line at modifying
the FileSet API in order to keep this manageable and not continue
refactoring all of the rest of Syncthing. As such, the FileSet methods
still panic on database errors, except for the "database is closed"
error which is instead handled by silently returning as quickly as
possible, with the assumption that we're anyway "on the way out".
2019-11-29 09:11:52 +01:00
Simon Frei
606154b183
lib/model: Also send folder summary from sync-preparing (ref #6028 ) ( #6202 )
2019-11-29 08:30:17 +01:00
Jakob Borg
4e151d380c
lib/versioner: Reduce surface area ( #6186 )
...
* lib/versioner: Reduce surface area
This is a refactor while I was anyway rooting around in the versioner.
Instead of exporting every possible implementation and the factory and
letting the caller do whatever, this now encapsulates all that and
exposes a New() that takes a config.VersioningConfiguration.
Given that and that we don't know (from the outside) how a versioner
works or what state it keeps, we now just construct it once per folder
and keep it around. Previously it was recreated for each restore
request.
* unparam
* wip
2019-11-26 07:39:31 +00:00
Simon Frei
4d368a37e2
lib/model, lib/protocol: Add contexts sending indexes and download-progress ( #6176 )
2019-11-25 11:07:36 +01:00
Simon Frei
cf312abc72
lib: Wrap errors with errors.Wrap instead of fmt.Errorf ( #6181 )
2019-11-23 15:20:54 +00:00
Simon Frei
65d4dd32cb
lib/model: Also handle ServeBackground ( #6173 )
2019-11-22 21:30:16 +01:00
Simon Frei
57d668ed1d
lib/config: Do introductions in a single config change ( #6162 )
2019-11-21 08:41:41 +01:00
Simon Frei
90d85fd0a2
lib: Replace done channel with contexts in and add names to util services ( #6166 )
2019-11-21 08:41:15 +01:00
Simon Frei
0d14ee4142
lib/model: Don't info log repeat pull errors ( #6149 )
2019-11-19 09:56:53 +01:00
Simon Frei
28edf2f5bb
lib/model: Keep fmut locked while adding/starting/restarting folders ( #6156 )
2019-11-18 21:15:26 +01:00
Simon Frei
5edf4660e2
lib/model: Prevent cleanup-race in testing (ref #6152 ) ( #6155 )
2019-11-14 23:08:40 +01:00
Simon Frei
f80ce17497
lib/model: In tests prevent goroutine leaks and increase timeouts ( #6152 )
2019-11-13 10:21:54 +01:00
Simon Frei
ce72bee576
lib/model: Simplify pull error/retry logic ( fixes #6139 ) ( #6141 )
2019-11-11 15:50:28 +01:00
Simon Frei
85e6a77f25
lib/model: Remove some testing deadlocks ( #6138 )
2019-11-08 18:53:51 +01:00
Jakob Borg
88244b0c1f
lib/model: Add test for previous commit
2019-11-08 17:03:25 +01:00
Simon Frei
cd290d2d05
lib/model: Add initial deviceStatRefs on model creation ( fixes #6136 ) ( #6137 )
...
This is a regression introduced in PR #6005 / commit
f7b2e79fdc
2019-11-08 11:32:51 +00:00
Simon Frei
bee7cce081
lib/model: Add folders on start in model ( #6135 )
2019-11-08 10:56:16 +01:00
Ruslan Yevdokymov
3b5dff3f34
lib/model: Fix removal of a marker when there are still folders referencing it ( #6114 )
2019-10-30 15:11:07 +00:00
Simon Frei
bbdda059bd
lib/model: Check for symlinks before deleting during pull ( fixes #6090 ) ( #6100 )
2019-10-22 21:55:51 +02:00
Simon Frei
7b33294955
gui, lib/model: Add new state FolderPreparingSync ( fixes #6027 ) ( #6028 )
2019-10-16 09:08:54 +02:00
Lukas Lihotzki
96bb1c8e29
all, lib/logger: Refactor SetDebug calls ( #6054 )
2019-10-04 13:03:34 +02:00
Audrius Butkevicius
8fb576ed54
lib/model: Adjust blocks reported in usage reporting ( fixes #5995 ) ( #6037 )
...
* lib/model: Adjust blocks reported in usage reporting (fixes #5995 )
* Use variables, fix go.mod
2019-10-04 12:03:13 +01:00
Simon Frei
a2a14c8424
lib/model: Set empty version when unignoring deleted files (fixes 6038) ( #6039 )
2019-10-01 15:34:59 +02:00
Simon Frei
28b6e8b063
lib/db: Update db when only local flags change ( fixes #6008 ) ( #6007 )
2019-09-12 08:47:39 +02:00
Simon Frei
f7b2e79fdc
lib/model: Use read-locks wherever possible ( #6005 )
2019-09-12 05:55:23 +01:00
Jakob Borg
ebd2e5bf30
lib/model: Correctly handle manual rescan with ignore error ( fixes #5985 ) ( #5987 )
...
Assume a folder error was set due to bad ignores on the latest scan.
Previously, doing a manual rescan would result in:
1. Clearing the folder error, which schedules (immediately) an fs
watcher restart
2. Attempting to load the ignores, which fails, so we set a folder
error and bail.
3. Now the fs watcher restarts, as scheduled, so we trigger a scan.
Goto 1.
This change fixes this by not clearing the error until the error is
actually cleared, that is, if both the health check and ignore loading
succeeds.
2019-08-30 13:27:26 +01:00
Jakob Borg
24d4290d03
lib/model, lib/scanner: Pass a valid event logger ( fixes #5970 ) ( #5971 )
2019-08-21 08:05:43 +02:00
Simon Frei
77a5980747
lib/model: Do free disk space check later on pull ( fixes #5948 ) ( #5949 )
2019-08-16 09:40:53 +02:00
Simon Frei
b677464dfa
lib/model: Optimise locking around conn-close and puller states ( #5954 )
2019-08-16 09:35:19 +02:00
Simon Frei
b1c74860e8
all: Remove global events.Default (ref #4085 ) ( #5886 )
2019-08-15 16:29:37 +02:00
Simon Frei
6fa02d5081
lib/model: Fix a few more problematic locks (ref #5929 ) ( #5944 )
2019-08-13 09:04:43 +02:00
Simon Frei
86e35f1879
lib/model: Less locking in ClusterConfig ( #5943 )
2019-08-11 19:30:24 +02:00
Simon Frei
4a619e74f2
lib/model: Fix incorrect locking ( #5939 )
2019-08-11 16:10:30 +02:00
Simon Frei
8a06cf0973
lib/model: Unflake TestPullInvalidIgnored ( #5918 )
2019-08-01 11:07:41 +02:00
Simon Frei
05835ed81f
all: Remove potentially problematic errors from panics ( fixes #5839 ) ( #5912 )
2019-07-31 10:53:35 +02:00
Simon Frei
df522576ac
lib/model: Don't call t.Fatal in goroutines ( fixes #5901 ) ( #5903 )
2019-07-30 17:50:51 +02:00
Simon Frei
fd847d4efe
lib/model: Fix flakyness of TestRequestRemoteRenameChanged ( #5904 )
2019-07-28 22:29:31 +02:00
Jakob Borg
c1c976aa2b
lib/model: Don't panic on failed chmod-back on directory ( fixes #5836 ) ( #5896 )
...
* lib/model: Don't panic on failed chmod-back on directory (fixes #5836 )
This makes the "in writable dir"-wrapper log chmod-back errors instead
of panicking. To do that we need a logger so the function moved into the
model package which is also the only place it's used. The tests came
along.
(The test also exercised osutil.RenameOrCopy like some sort of
piggybacking. I removed that.)
2019-07-28 10:25:05 +02:00
Jakob Borg
669bcb748f
lib/config, lib/model: Don't save on every pending folder/device update ( fixes #5888 ) ( #5890 )
...
Wrapper methods generally don't save by themselves.
2019-07-27 11:05:00 +01:00
Simon Frei
7a4c88d4e4
lib: Add mtime window when comparing files ( #5852 )
2019-07-23 21:48:53 +02:00
Simon Frei
35f40e9a58
lib/model: Create new file-set after stopping folder ( fixes #5882 ) ( #5883 )
2019-07-23 20:39:25 +02:00
Simon Frei
6f08162376
lib/model: Remove incorrect/useless panics ( #5881 )
2019-07-23 10:51:16 +02:00
Simon Frei
942659fb06
lib/model, lib/nat: More service termination speedup ( #5884 )
2019-07-23 10:49:22 +02:00
Simon Frei
b5b54ff057
lib/model: No watch-error on missing folder ( fixes #5833 ) ( #5876 )
2019-07-19 19:41:16 +02:00
Simon Frei
1cb55904bc
lib/model: Prevent panic in NeedFolderFiles ( fixes #5872 ) ( #5875 )
2019-07-19 19:39:52 +02:00
Simon Frei
2b622d0774
lib/model: Close conn on dev pause ( fixes #5873 ) ( #5874 )
2019-07-19 19:37:29 +02:00
Simon Frei
82b70b9fae
lib/model, lib/protocol: Track closing connections ( fixes #5828 ) ( #5829 )
2019-07-14 11:03:55 +02:00
Simon Frei
20c8dbd9ed
lib/model: Fix integer conversion ( fixes #5837 ) ( #5851 )
2019-07-12 16:37:12 +02:00
Simon Frei
3446d50201
lib/model: Remove pointless error that watch hasn't started ( fixes #5833 ) ( #5834 )
2019-07-10 11:00:06 +02:00
Simon Frei
9fef1552fc
lib/db, lib/model: Remove folder info from panics (ref #5839 ) ( #5840 )
2019-07-10 10:57:49 +02:00
Simon Frei
ba056578ec
lib: Add util.Service as suture.Service template ( fixes #5801 ) ( #5806 )
2019-07-09 11:40:30 +02:00
Simon Frei
863fe23347
gui, lib/model: Fix download progress accounting ( fixes #5811 ) ( #5815 )
2019-06-30 09:23:47 +02:00
Simon Frei
1cf352a722
lib/model: NewFileSet outside fmut ( #5818 )
2019-06-29 08:49:30 +02:00
Simon Frei
b58f6ca886
lib/model: Correct/unify check if item changed ( #5819 )
2019-06-29 07:45:41 +02:00
Simon Frei
3c7e7e971d
lib/model: Make jobQueue.Jobs paginated ( fixes #5754 ) ( #5804 )
...
* lib/model: Make jobQueue.Jobs paginated (fixes #5754 )
* fix, no test yet
* add test
2019-06-27 19:25:38 +01:00
Audrius Butkevicius
afde0727fe
lib/versioner: Revert naming change ( fixes #5807 ) ( #5808 )
2019-06-25 08:56:11 +03:00
Simon Frei
cea5962417
lib/model: Unflake TestPullInvalidIgnoredSR/SO ( fixes #5796 ) ( #5799 )
2019-06-17 15:23:28 +03:00
Simon Frei
abd363e8bb
lib/model: Don't error on pulling deletion of invalid file ( fixes #5791 ) ( #5792 )
2019-06-14 08:48:14 +02:00
Simon Frei
13a746e0fb
lib/model: Prevent nil deref if folder stopped ( fixes #5780 ) ( #5778 )
2019-06-11 11:48:51 +02:00
Audrius Butkevicius
21f50e2f8f
lib/versioner: Use mtime for version cleanup ( fixes #5765 ) ( #5769 )
2019-06-11 09:16:55 +02:00
Simon Frei
1054ce9354
lib/model: Refactor sending indexes as suture service ( #5757 )
2019-06-10 13:27:22 +02:00
Jakob Borg
997bb5e7e1
all: Remove "large blocks" config ( #5763 )
...
We now always use large / variable blocks.
2019-06-06 15:57:38 +01:00
Simon Frei
e39d3f95dd
lib/protocol: Prioritize close msg and add close timeout ( #5746 )
2019-06-05 14:01:59 +08:00
Audrius Butkevicius
e714df013f
lib/connections: Add QUIC protocol support ( fixes #5377 ) ( #5737 )
2019-05-29 09:56:40 +02:00
Simon Frei
129df0613b
lib/model: Unflake folder restart with blocking conn test (ref #5707 ) ( #5744 )
2019-05-27 10:58:09 +01:00
Simon Frei
486230768e
lib/fs, lib/model: Add error channel to Watch to avoid panics ( fixes #5697 ) ( #5734 )
...
* lib/fs, lib/model: Add error channel to Watch to avoid panics (fixes #5697 )
* forgot unsupported watch
* and more non(-standard)-unixy fixes
* and windows test
* review
2019-05-25 20:08:26 +01:00
Simon Frei
d91da8feee
lib/model: Readd special handling of conn close in TestIssue5063 ( #5743 )
...
This partially reverts commit 64518b0f7e
2019-05-25 18:51:13 +01:00
Simon Frei
64518b0f7e
lib/model: Close connections when model is stopped ( #5733 )
2019-05-25 16:00:32 +02:00
Simon Frei
5b306510a0
lib/model: Consistently cleanup model in tests ( #5724 )
2019-05-19 14:29:07 +02:00
Simon Frei
1b2b970f32
lib/model, lib/testutils: Test closing a connection on folder restart ( #5707 )
2019-05-18 08:53:59 +02:00
Simon Frei
5ffbb7668d
lib/model: Fix test flakyness regression (ref #5592 ) ( #5718 )
2019-05-18 08:52:50 +02:00
Simon Frei
441ea109a1
lib/model: Refactor file deletions when pulling ( #5699 )
2019-05-17 18:29:54 +02:00
André Colomb
e4956358fb
lib/model: Remove superfluous check for IndexID in remote ClusterConfig ( #5717 )
...
The check in ClusterConfig() when iterating through announced devices
in a folder explicitly skips entries without a non-zero IndexID.
Therefore, the check for IndexID == 0 just below will never be true
and the intended cleanup of local index data will not happen.
Plainly remove that check to make the intended case distinction work.
2019-05-12 21:17:55 +02:00
Simon Frei
2b246eeb52
lib/model: Move test utilities to separate files ( #5694 )
2019-05-10 13:33:45 +02:00
Jakob Borg
31be810eb6
lib/model: Don't fail operation when fsync() fails ( fixes #5704 ) ( #5705 )
2019-05-09 21:20:29 +01:00
Simon Frei
b45d77b6be
lib/model: Fix regression deleting directories on pull (ref #5690 ) ( #5691 )
2019-05-06 20:55:26 +02:00
Simon Frei
79e67b7f79
lib/model: Remove individual pull errors on retry ( fixes #5659 ) ( #5690 )
2019-05-06 17:21:56 +02:00
Simon Frei
5fa8467756
lib/model: Consistent use of locks ( #5684 )
2019-05-02 18:55:39 +01:00
Simon Frei
5954b105cd
lib/model: Let fakeConnection call Model.Closed on close ( #5682 )
2019-05-02 14:24:55 +02:00
Simon Frei
defc5dca65
lib/model: Use correct lock ( #5683 )
2019-05-02 14:09:42 +02:00
Simon Frei
fe4daf242b
cmd, lib/db: Actually close goleveldb ( fixes #5505 ) ( #5671 )
2019-05-02 11:15:00 +02:00
Audrius Butkevicius
0ca1f26ff8
lib/versioner: Restore for all versioners, cross-device support ( #5514 )
...
* lib/versioner: Restore for all versioners, cross-device support
Fixes #4631
Fixes #4586
Fixes #1634
Fixes #5338
Fixes #5419
2019-04-28 23:30:16 +01:00
Simon Frei
5da41f75fa
lib/model, lib/protocol: Wait for reader/writer loops on close ( fixes #4170 ) ( #5657 )
...
* lib/protocol: Wait for reader/writer loops on close (fixes #4170 )
* waitgroup
* lib/model: Don't hold lock while closing connection
* fix comments
* review (lock once, func argument) and naming
2019-04-28 11:58:51 +01:00
Simon Frei
110806842c
lib/model: Pass the old not new fileinfo to deleteItemOnDisk ( fixes #5654 ) ( #5655 )
2019-04-23 20:46:28 +01:00
Simon Frei
926e9228ed
lib/scanner, lib/model: File -> item when logging error ( #5664 )
2019-04-21 16:19:59 +01:00
Simon Frei
86e72d9973
lib/model: Use RLock and legacy polish ( #5663 )
2019-04-21 13:21:36 +01:00
Simon Frei
3bea59b0d9
lib/model: Refactor progressEmitter to de-/activate by config ( fixes #4613 ) ( #5623 )
2019-04-13 14:20:51 +02:00
Simon Frei
fca895a632
lib/model: Fix block index calculation for recheckFile ( fixes #5649 ) ( #5648 )
2019-04-12 15:21:07 +02:00
Simon Frei
9b2a73f9ab
lib/model: Pause pull for at least as long as failed pull took ( fixes #5641 ) ( #5643 )
2019-04-10 16:22:23 +02:00
Simon Frei
c305265c62
lib/model: Request errors conforming to BEP specs ( #5642 )
2019-04-10 11:47:24 +02:00
Simon Frei
395e524e2d
lib/model: Update db on scan/pull in folder ( #5608 )
2019-04-07 13:29:17 +02:00
