Simon Frei
a20a5f61f0
lib/ur: Send unreported failures on shutdown ( #7164 )
2020-12-22 20:17:14 +01:00
Simon Frei
78bd0341a8
all: Handle errors opening db/creating file-set (ref #5907 ) ( #7150 )
2020-12-21 12:59:22 +01:00
Simon Frei
c845e245a1
lib: Close underlying conn in protocol ( fixes #7165 ) ( #7212 )
2020-12-21 11:40:51 +01:00
Simon Frei
d904dfa191
lib/model: Fix flaky test and add some scanning debug ( #7214 )
2020-12-20 18:13:35 +01:00
Simon Frei
fa40ccece1
lib: Consistently set suture logging ( #7202 )
2020-12-18 19:44:00 +01:00
Simon Frei
7919310dc6
lib/model: Unflake TestIgnoreDeleteUnignore ( #7208 )
2020-12-18 18:42:09 +01:00
André Colomb
7502997e7e
all: Store pending devices and folders in database ( fixes #7178 ) ( #6443 )
2020-12-17 19:54:31 +01:00
Jakob Borg
4142a431b5
model: Actually print folder description in "Overriding" log message
2020-12-12 12:32:24 +01:00
Simon Frei
240ae0c14f
lib/model: Unflake TestRequestReceiveEncryptedLocalNoSend ( #7167 )
2020-11-27 20:53:03 +01:00
Simon Frei
a9764fc16c
lib: Skip deleted, locally changed on recv-enc folders ( fixes #7153 ) ( #7154 )
2020-11-27 11:26:36 +01:00
Simon Frei
bf7d03d029
lib/model: Fix enc file size when pulling ( fixes #7152 ) ( #7155 )
2020-11-25 22:57:25 +01:00
Simon Frei
54b50e3d52
lib/fs, lib/model: Cover more windowsyness sanitizing paths ( fixes #7075 ) ( #7158 )
2020-11-25 22:52:46 +01:00
André Colomb
e6595c1ab9
lib/model: Simplify access to Folder and Device configuration. ( #7151 )
...
Use the accessor methods with a device argument instead of looking up
in a temporary map of the same data.
2020-11-24 22:20:50 +01:00
Simon Frei
100ef10d84
lib/model: Don't send locally changed on recv-enc ( fixes #7137 ) ( #7147 )
2020-11-24 21:49:45 +01:00
Simon Frei
db1f20603a
lib/model: Retain index info for new folders/devs (ref #7100 ) ( #7133 )
2020-11-20 15:53:13 +01:00
Simon Frei
24af89c8e2
all: Refactor preparing configuration ( #7127 )
2020-11-20 14:21:54 +01:00
Simon Frei
b43eccf2fe
lib/model: Never send unpaused folder without index info ( #7134 )
2020-11-20 14:13:50 +01:00
Simon Frei
e11b309379
lib/model, lib/util: Replace IsComplete with NoRestartErr (ref #6947 ) ( #7126 )
2020-11-18 13:43:57 +01:00
Simon Frei
2d3a535ced
lib/model: Handle cluster-config before folder start ( fixes #7122 ) ( #7123 )
2020-11-17 15:30:21 +01:00
Simon Frei
9524b51708
all: Implement suture v4-api ( #6947 )
2020-11-17 13:19:04 +01:00
Simon Frei
b8051fb37e
lib/model: Prevent test deadlock ( #7110 )
2020-11-10 12:32:07 +01:00
Simon Frei
31559e908b
all: Add untrusted folders behind feature flag (ref #62 ) ( #7055 )
2020-11-09 15:33:32 +01:00
Simon Frei
54643e86b5
lib/model: Fix locking when resending cluster-configs ( #7107 )
2020-11-09 14:05:21 +01:00
Simon Frei
1f1729ba43
lib/model: Add done chan to track folder-lifetime ( fixes #6664 ) ( #7094 )
2020-11-09 09:05:48 +01:00
Simon Frei
d4ce0dfd84
lib/model: Send indexes for newly shared folder ( fixes #7098 ) ( #7100 )
2020-11-09 08:58:46 +01:00
Simon Frei
cc9ea9db89
lib/folder: Clear pull errors when nothing is needed anymore ( #7093 )
2020-11-06 14:22:20 +01:00
Simon Frei
1c2be84e4e
lib/model: Pass device infos as struct ( fixes #7051 ) ( #7052 )
2020-10-22 13:05:31 +02:00
Simon Frei
5c91723ef2
lib/model: Handle index sender lifetime ( fixes #7034 ) ( #7038 )
2020-10-21 11:51:53 +02:00
Simon Frei
27c91c57d5
lib/db: Remove need for the right dev removing globals ( fixes #7036 ) ( #7044 )
2020-10-21 08:26:10 +02:00
Simon Frei
1a8c10a8d0
lib/model: Use winning version instead of merge on conflict ( #6995 )
2020-10-19 08:53:19 +02:00
Simon Frei
9d09fd6af3
all: Add failure reporting ( fixes #7028 ) ( #6925 )
2020-10-07 10:05:13 +02:00
Simon Frei
42de53c6c9
lib/model: Fix race in GlobalDirectory tests ( fixes #7021 ) ( #7022 )
2020-10-03 20:46:17 +02:00
Simon Frei
48da6f0f22
lib: Use uint64 for disk stats (ref #3930 ) ( #7019 )
2020-10-02 15:22:28 +02:00
Simon Frei
a20c6ca868
lib/model, lib/protocol: Send ClusterConfig on config change ( fixes #7020 ) ( #7018 )
2020-10-02 11:49:51 +02:00
Audrius Butkevicius
e027175446
all: Move remaining protos to use the vanity plugin ( #7009 )
2020-10-02 08:07:05 +02:00
Simon Frei
1b1d38183d
lib: Remove HelloResult type, same as Hello ( #7015 )
2020-09-29 12:17:38 +01:00
Jakob Borg
9e0b924d57
lib/fs: Be more clear about invalid file names (ref #7010 ) ( #7011 )
...
Add specific errors for the failures, resulting in this rather than just
the generic "invalid filename":
[MRIW7] 08:50:50 INFO: Puller (folder default, item "NUL"): syncing: filename is invalid: name is reserved
[MRIW7] 08:50:50 INFO: Puller (folder default, item "fail."): syncing: filename is invalid: name ends with space or period
[MRIW7] 08:50:50 INFO: Puller (folder default, item "sup:yo"): syncing: filename is invalid: name contains reserved character
[MRIW7] 08:50:50 INFO: default: Failed to sync 3 items
2020-09-28 10:22:50 +02:00
Simon Frei
8452fd2ab4
lib/model, lib/scanner: Prevent races aborting scans ( fixes #6994 ) ( #6997 )
2020-09-25 11:27:44 +02:00
Simon Frei
8210466b03
lib/model: Consider case conflicts when checking to be deleted items ( #6986 )
2020-09-12 07:45:50 +02:00
Simon Frei
08bebbe59b
lib/db, lib/syncthing: Don't repair DB on upgrade, but on error ( fixes #6917 ) ( #6971 )
2020-09-10 10:54:41 +02:00
Simon Frei
c5c23ed10f
lib/model: Consider existing file when handling symlink on windows ( #6977 )
2020-09-10 10:52:38 +02:00
Simon Frei
286698ccb1
lib/model: Handle symlink deletion on windows ( fixes #6972 ) ( #6976 )
2020-09-10 08:25:44 +02:00
Simon Frei
56d48d341f
lib/model: Fix case-only renames on pull ( #6978 )
2020-09-10 08:23:54 +02:00
Simon Frei
e3cd9219b8
lib/model: Don't fail over case-conflict on tempfile ( fixes #6973 ) ( #6975 )
2020-09-09 11:47:14 +02:00
Jakob Borg
674fca3868
lib/db, lib/protocol: Never need empty-version entries ( fixes #6961 ) ( #6962 )
...
Avoid havoc when discovering locally-deleted-upgraded files during repair / need calculation...
Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-09-07 20:18:25 +02:00
Simon Frei
e19728d8cc
lib/model: Correct completion perc. when globalBytes is 0 ( #6963 )
2020-09-07 20:03:18 +02:00
Simon Frei
3dd13c3994
test, lib/model: Various integration test updates & improvements ( #6956 )
2020-09-07 09:35:37 +02:00
Simon Frei
6365e6108f
lib/model: Don't acquire I/O token for send-only pull ( #6951 )
2020-09-03 13:59:45 +02:00
Simon Frei
540518a7b7
lib/model: Add missing lock on download-state ( fixes #6880 ) ( #6945 )
2020-08-30 08:03:10 +02:00
Jakob Borg
3ffbe45a6d
lib/db, lib/model: Avoid accounting mishap on folder restart ( fixes #6938 ) ( #6939 )
...
We created a new fileset before stopping the folder during restart. When
we create that fileset it loads the current metadata and sequence
numbers from the database. But the folder may have time to update those
before stopping, leaving the new fileset with bad data.
This would cause wrong accounting (forgotten files) and potentially
sequence reuse causing files not sent to other devices.
This change reuses the fileset on restart, skipping the issue entirely.
It also moves the creation of the fileset back under the lock so there
should be no chance of concurrency issues here.
2020-08-27 18:33:27 +02:00
Jakob Borg
3393db1f69
lib/model: Save config after auto accepting folders ( fixes #6922 ) ( #6937 )
2020-08-27 16:28:19 +02:00
Jakob Borg
145d87ce70
lib/model: Fixup spelling of protocol.FileIntf in test (ref #6463 )
2020-08-27 16:21:04 +02:00
Simon Frei
06ac631351
lib/model: Reduce fmut locking in ClusterConfig ( #6913 )
...
The FileSet.Drop operation in there needs to potentially update a whole lot of global lists, which can take a while (longer than the deadlock interval apparently)
2020-08-27 16:01:46 +02:00
Simon Frei
1fc2dbdeeb
lib/model: Add test for recv-only changes undone by remote ( #6463 )
...
Co-authored-by: Audrius Butkevicius <audrius.butkevicius@gmail.com>
2020-08-27 15:54:00 +02:00
Audrius Butkevicius
d507d932b8
all: Use protobuf to generate config structs ( fixes #6734 ) ( #6900 )
2020-08-25 08:11:14 +02:00
Simon Frei
a65b46debd
lib/model: Don't mark globally deleted items as recv-only ( fixes #6910 ) ( #6911 )
2020-08-21 12:34:22 +02:00
Simon Frei
b628460930
lib/model: Prevent localflag flipflopping on del. item (ref #6865 ) ( #6919 )
2020-08-21 10:09:48 +02:00
Simon Frei
cc1f6e4d4a
lib/db, lib/model: Cover exec-paths with debug logging ( #6918 )
2020-08-20 16:11:20 +02:00
Simon Frei
88599bc154
lib/model: Handle del. dir with locally changed items on pull ( fixes #6873 ) ( #6914 )
2020-08-20 10:56:29 +02:00
Audrius Butkevicius
bf9ff17267
all: Remove need to restart syncthing ( #6883 )
2020-08-18 09:26:33 +02:00
Audrius Butkevicius
e9bb17307d
lib/fs: Unwrap mtimeFile, get fd the "correct" way (ref #6875 ) ( #6877 )
2020-08-07 07:47:48 +02:00
Audrius Butkevicius
96e197e502
lib/model: Don't close file early ( fixes #6875 ) ( #6876 )
2020-08-03 21:54:42 +02:00
Simon Frei
a85bc1c3a6
lib/model: Check folder error before sync-waiting ( fixes #6793 ) ( #6847 )
2020-07-31 19:26:09 +02:00
Simon Frei
850dd4cd25
lib/db: Don't count invalid items to global state (ref #6850 ) ( #6863 )
2020-07-30 13:49:14 +02:00
Simon Frei
e46c8ab9ee
lib/model: Detect deleted RO items scanning on non-RO ( fixes #6864 ) ( #6865 )
2020-07-30 13:41:45 +02:00
Simon Frei
932d8c69de
lib/fs: Properly handle case insensitive systems ( fixes #1787 , fixes #2739 , fixes #5708 )
...
With this change we emulate a case sensitive filesystem on top of
insensitive filesystems. This means we correctly pick up case-only renames
and throw a case conflict error when there would be multiple files differing
only in case.
This safety check has a small performance hit (about 20% more filesystem
operations when scanning for changes). The new advanced folder option
`caseSensitiveFS` can be used to disable the safety checks, retaining the
previous behavior on systems known to be fully case sensitive.
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-07-28 11:15:11 +02:00
Jakob Borg
aedc2d788f
lib/model, lib/versioner: Drive version cleanup from scanner ( fixes #6313 ) ( #6693 )
...
This change adds a separate config for the cleanup interval, and runs that cleanup from the main folder loop.
2020-07-14 10:48:50 +02:00
Jakob Borg
0c61c66511
lib/api, lib/model: Improve folder completion API ( fixes #6075 ) ( #6808 )
2020-07-03 08:48:37 +02:00
Simon Frei
9c45ac381c
lib/model: Consider weak hash on recheckFile ( #6797 )
2020-06-26 16:47:03 +02:00
Simon Frei
4f06708330
lib/api, lib/model: Fixes around event request tracking ( #6774 )
2020-06-25 21:48:48 +02:00
Simon Frei
0648fb0626
lib/model: Don't force rescan already changed items ( #6798 )
2020-06-25 20:23:59 +02:00
Simon Frei
90e248615f
lib/scanner: Test weak hash consistency (ref #5556 ) ( #6794 )
...
Relevant much earlier changes:
9b1c592fb7bd1c29ee32
2020-06-25 14:47:35 +02:00
Audrius Butkevicius
689cf2a5ee
lib/ur: Normalise contract between syncthing and ursrv ( #6770 )
...
* Fix ui, hide report date
* Undo Goland madness
* UR now web scale
* Fix migration
* Fix marshaling, force tick on start
* Fix tests
* Darwin build
* Split "all" build target, add package name as a tag
* Remove pq and sql dep from syncthing, split build targets
* Empty line
* Revert "Empty line"
This reverts commit f74af2b067dadda8a343714123512bd545a643c3.
* Revert "Remove pq and sql dep from syncthing, split build targets"
This reverts commit 8fc295ad007c5bb7886c557f492dacf51be307ad.
* Revert "Split "all" build target, add package name as a tag"
This reverts commit f4dc88995106d2b06042f30bea781a0feb08e55f.
* Normalise contract types
* Fix build add more logging
2020-06-23 09:47:15 +01:00
Simon Frei
85794933d3
lib/model: Remove double error handling in performFinish ( #6780 )
2020-06-23 07:30:16 +02:00
Audrius Butkevicius
5fb3992275
lib/model: More connection closing ( #6778 )
...
* lib/model: More connection closing
* Revert "lib/model: More connection closing"
This reverts commit 5397c3a55c0de25fab93d4ec622530818f1b2a46.
* Add tests, fix one broken-ness
* Update lib/model/model.go
Co-authored-by: Simon Frei <freisim93@gmail.com>
* Update model.go
* Update model.go
Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-06-22 21:26:26 +01:00
Simon Frei
f66e57947b
lib/model: Use right db snap when scanning recvonly folder ( #6769 )
2020-06-21 09:28:29 +02:00
Simon Frei
2716898cb9
lib/model: Don't ignore stat failure in performFinish ( #6766 )
2020-06-19 23:47:29 +02:00
Simon Frei
22f0077262
lib/model: Don't stay scanning forever on fail ( #6761 )
2020-06-18 14:13:46 +01:00
Jakob Borg
946170f3fc
gui, lib/ignore: Handle editing ignores with error ( fixes #5425 ) ( #6757 )
...
This changes the error handling in loading ignores slightly:
- There is a new ParseError type that is returned as the error
(somewhere in the chain) when the problem was not an I/O error loading
the file, but some issue with the contents.
- If the file was read successfully but not parsed successfully we still
return the lines read (in addition to nil patterns and a ParseError).
- In the API, if the error IsParseError then we return a successful
HTTP response with the lines and the actual error included in the JSON
object.
- In the GUI, as long as the HTTP call to load the ignores was
successful we can edit the ignores. If there was an error we show this
as a validation error on the dialog.
Also some cleanup on the Javascript side as it for some reason used
jQuery instead of Angular for this editor...
2020-06-18 11:04:00 +02:00
Simon Frei
8cf9d91ed4
lib: Print nicely rounded durations ( #6756 )
2020-06-18 10:55:41 +02:00
Audrius Butkevicius
4812fd3ec1
all: Add copy-on-write filesystem support ( fixes #4271 ) ( #6746 )
2020-06-18 08:15:47 +02:00
Simon Frei
6976219d6d
lib/model: Check dir before deletion when pulling ( #6741 )
2020-06-16 15:20:08 +02:00
Simon Frei
a47546a1f1
lib/scanner, lib/model: Improve error handling when scanning ( #6736 )
2020-06-16 09:25:41 +02:00
Jakob Borg
5be13f62a2
lib/model: Fix minor flakiness in TestModTimeWindow
2020-06-16 06:32:37 +02:00
Simon Frei
1f8e6c55f6
lib/db: Refactor to use global list by version ( fixes #6372 ) ( #6638 )
...
Group the global list of files by version, instead of having one flat list for all devices. This removes lots of duplicate protocol.Vectors.
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-05-30 09:50:23 +02:00
Simon Frei
ed6bfc5417
lib/model: Don't increase pull pause on triggered pull ( #6690 )
2020-05-29 09:52:28 +02:00
Simon Frei
b784f5b9e3
lib/config, lib/model: Commit auto-accepted folders all at once ( #6684 )
2020-05-27 08:05:26 +02:00
Simon Frei
c3b5eba205
lib/model: Fix checking children when trying to delete a dir ( fixes #6646 ) ( #6647 )
2020-05-25 08:46:24 +02:00
Simon Frei
cf75329067
lib/model: Handle concurrently removed device in ClusterConfig ( #6666 )
2020-05-20 11:13:55 +02:00
Jakob Borg
5ffa012410
lib/model: Don't crash when taking rename shortcut ( fixes #6654 ) ( #6657 )
...
If we fail to take the rename shortcut we may crash on a later loop,
because we do trickiness with the indexes but the original buckets[key]
in "range buckets[key]" isn't re-evaluated so i exceeds the max index.
2020-05-17 08:48:35 +01:00
Jakob Borg
651ee2ce74
lib/model: Partial revert of rename fix ( fixes #6653 ) ( #6656 )
...
We can't just drop the snap because it's in use elsewhere. This should
be equally functional.
2020-05-16 23:05:33 +02:00
Audrius Butkevicius
a1c5b44c74
lib/model: Fix rename handling (ref #6650 ) ( #6652 )
2020-05-16 14:39:27 +02:00
Simon Frei
e5cc55ce09
lib/model: Close conns when devices are removed ( fixes #6564 ) ( #6641 )
2020-05-14 07:50:53 +02:00
Simon Frei
299b9d8883
lib/model: Adjust remote-rename-test to timer-based versions ( fixes #6625 ) ( #6644 )
2020-05-14 00:31:05 +02:00
Simon Frei
974551375e
lib/db: Dont add symlinks to blocks map ( fixes #6637 ) ( #6639 )
2020-05-13 20:38:21 +02:00
Jakob Borg
531ceb2b0f
Add indirection for large version vectors. ( #6376 )
...
This adds indirection of large version vectors in the same manner as we
already to block lists. The effect is the same: less duplicated data in
some situations.
To mitigate the impact for when this indirection
wouldn't be needed I've added an indirection cutoff for both blocks and
the new version vector stuff: we don't do the indirection at all for
small block lists or small version vectors, instead storing it directly
like we used to do. This is faster for small files and small setups.
2020-05-13 14:28:42 +02:00
Audrius Butkevicius
6201eebc98
lib/model: Add support for different puller block ordering ( #6587 )
...
* WIP
* Tests
* Header and format
* WIP
* Fix tests
* Goland you disappoint me
* Remove CC storage
* Update blockpullreorderer.go
2020-05-11 22:44:04 +01:00
Audrius Butkevicius
decb967969
all: Reorder sequences for better rename detection ( #6574 )
2020-05-11 20:15:11 +02:00
Simon Frei
a94951becd
lib/db, lib/model: Keep need stats in metadata (ref #5899 ) ( #6413 )
2020-05-11 15:07:06 +02:00
Simon Frei
57ea8a1bf5
lib/model: Prevent panic on test failure (ref #6618 ) ( #6620 )
2020-05-08 17:56:49 +02:00
