Simon Frei
c78fa42f31
lib/connections: Dial devices in parallel ( #7783 )
2021-06-25 11:38:04 +02:00
Simon Frei
dd39556759
lib: Revert needing invalid files ( fixes #7608 , ref #7476 ) ( #7609 )
2021-04-29 22:01:46 +02:00
Jakob Borg
81484699db
lib/model: Actually break puller loop on context cancel ( #7492 )
...
Current break does nothing (breaks the select).
2021-03-17 21:34:52 +01:00
Simon Frei
273ee09925
lib/db, lib/model: Allow needing invalid files ( fixes #7474 ) ( #7476 )
2021-03-15 07:58:01 +01:00
Simon Frei
31f859e909
lib/model: Return correct error in puller-iteration (ref #7424 ) ( #7461 )
2021-03-12 12:21:28 +01:00
Simon Frei
310fba4c12
lib: Return error from db.FileSet.Snapshot ( fixes #7419 , ref #5907 ) ( #7424 )
2021-03-07 13:43:22 +01:00
Simon Frei
c0f353c0e8
lib: Do not set ModifiedBy on meta only changes ( #7345 )
2021-02-08 15:30:39 +01:00
Simon Frei
c48eb4241a
lib/model: Fix child-check when deleting dirs in pull ( #7236 )
2021-01-02 21:40:37 +01:00
Simon Frei
a05dc6cc47
lib/model: Cleanup redundant filesystem variables in folders ( #7237 )
2020-12-27 22:26:25 +01:00
Simon Frei
bf7d03d029
lib/model: Fix enc file size when pulling ( fixes #7152 ) ( #7155 )
2020-11-25 22:57:25 +01:00
Simon Frei
9524b51708
all: Implement suture v4-api ( #6947 )
2020-11-17 13:19:04 +01:00
Simon Frei
31559e908b
all: Add untrusted folders behind feature flag (ref #62 ) ( #7055 )
2020-11-09 15:33:32 +01:00
Simon Frei
cc9ea9db89
lib/folder: Clear pull errors when nothing is needed anymore ( #7093 )
2020-11-06 14:22:20 +01:00
Simon Frei
1a8c10a8d0
lib/model: Use winning version instead of merge on conflict ( #6995 )
2020-10-19 08:53:19 +02:00
Simon Frei
48da6f0f22
lib: Use uint64 for disk stats (ref #3930 ) ( #7019 )
2020-10-02 15:22:28 +02:00
Audrius Butkevicius
e027175446
all: Move remaining protos to use the vanity plugin ( #7009 )
2020-10-02 08:07:05 +02:00
Jakob Borg
9e0b924d57
lib/fs: Be more clear about invalid file names (ref #7010 ) ( #7011 )
...
Add specific errors for the failures, resulting in this rather than just
the generic "invalid filename":
[MRIW7] 08:50:50 INFO: Puller (folder default, item "NUL"): syncing: filename is invalid: name is reserved
[MRIW7] 08:50:50 INFO: Puller (folder default, item "fail."): syncing: filename is invalid: name ends with space or period
[MRIW7] 08:50:50 INFO: Puller (folder default, item "sup:yo"): syncing: filename is invalid: name contains reserved character
[MRIW7] 08:50:50 INFO: default: Failed to sync 3 items
2020-09-28 10:22:50 +02:00
Simon Frei
8210466b03
lib/model: Consider case conflicts when checking to be deleted items ( #6986 )
2020-09-12 07:45:50 +02:00
Simon Frei
c5c23ed10f
lib/model: Consider existing file when handling symlink on windows ( #6977 )
2020-09-10 10:52:38 +02:00
Simon Frei
286698ccb1
lib/model: Handle symlink deletion on windows ( fixes #6972 ) ( #6976 )
2020-09-10 08:25:44 +02:00
Simon Frei
56d48d341f
lib/model: Fix case-only renames on pull ( #6978 )
2020-09-10 08:23:54 +02:00
Simon Frei
e3cd9219b8
lib/model: Don't fail over case-conflict on tempfile ( fixes #6973 ) ( #6975 )
2020-09-09 11:47:14 +02:00
Audrius Butkevicius
d507d932b8
all: Use protobuf to generate config structs ( fixes #6734 ) ( #6900 )
2020-08-25 08:11:14 +02:00
Simon Frei
88599bc154
lib/model: Handle del. dir with locally changed items on pull ( fixes #6873 ) ( #6914 )
2020-08-20 10:56:29 +02:00
Audrius Butkevicius
96e197e502
lib/model: Don't close file early ( fixes #6875 ) ( #6876 )
2020-08-03 21:54:42 +02:00
Simon Frei
a85bc1c3a6
lib/model: Check folder error before sync-waiting ( fixes #6793 ) ( #6847 )
2020-07-31 19:26:09 +02:00
Simon Frei
932d8c69de
lib/fs: Properly handle case insensitive systems ( fixes #1787 , fixes #2739 , fixes #5708 )
...
With this change we emulate a case sensitive filesystem on top of
insensitive filesystems. This means we correctly pick up case-only renames
and throw a case conflict error when there would be multiple files differing
only in case.
This safety check has a small performance hit (about 20% more filesystem
operations when scanning for changes). The new advanced folder option
`caseSensitiveFS` can be used to disable the safety checks, retaining the
previous behavior on systems known to be fully case sensitive.
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-07-28 11:15:11 +02:00
Jakob Borg
aedc2d788f
lib/model, lib/versioner: Drive version cleanup from scanner ( fixes #6313 ) ( #6693 )
...
This change adds a separate config for the cleanup interval, and runs that cleanup from the main folder loop.
2020-07-14 10:48:50 +02:00
Simon Frei
85794933d3
lib/model: Remove double error handling in performFinish ( #6780 )
2020-06-23 07:30:16 +02:00
Simon Frei
2716898cb9
lib/model: Don't ignore stat failure in performFinish ( #6766 )
2020-06-19 23:47:29 +02:00
Audrius Butkevicius
4812fd3ec1
all: Add copy-on-write filesystem support ( fixes #4271 ) ( #6746 )
2020-06-18 08:15:47 +02:00
Simon Frei
6976219d6d
lib/model: Check dir before deletion when pulling ( #6741 )
2020-06-16 15:20:08 +02:00
Simon Frei
1f8e6c55f6
lib/db: Refactor to use global list by version ( fixes #6372 ) ( #6638 )
...
Group the global list of files by version, instead of having one flat list for all devices. This removes lots of duplicate protocol.Vectors.
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-05-30 09:50:23 +02:00
Simon Frei
c3b5eba205
lib/model: Fix checking children when trying to delete a dir ( fixes #6646 ) ( #6647 )
2020-05-25 08:46:24 +02:00
Jakob Borg
5ffa012410
lib/model: Don't crash when taking rename shortcut ( fixes #6654 ) ( #6657 )
...
If we fail to take the rename shortcut we may crash on a later loop,
because we do trickiness with the indexes but the original buckets[key]
in "range buckets[key]" isn't re-evaluated so i exceeds the max index.
2020-05-17 08:48:35 +01:00
Simon Frei
299b9d8883
lib/model: Adjust remote-rename-test to timer-based versions ( fixes #6625 ) ( #6644 )
2020-05-14 00:31:05 +02:00
Audrius Butkevicius
6201eebc98
lib/model: Add support for different puller block ordering ( #6587 )
...
* WIP
* Tests
* Header and format
* WIP
* Fix tests
* Goland you disappoint me
* Remove CC storage
* Update blockpullreorderer.go
2020-05-11 22:44:04 +01:00
Audrius Butkevicius
decb967969
all: Reorder sequences for better rename detection ( #6574 )
2020-05-11 20:15:11 +02:00
Simon Frei
2e3975e956
lib/model: Improve errors when deleting dirs ( fixes #6575 ) ( #6586 )
2020-05-01 11:11:38 +02:00
Audrius Butkevicius
782bd08aad
lib/model: Add option to disable fsync ( #6588 )
...
* lib/model: Add option to disable fsync
* Fix test
* Dont open stuff for no reason
2020-05-01 08:36:46 +01:00
Jakob Borg
6c73617974
lib/model: Use semaphore to limit concurrent folder writes ( fixes #6541 ) ( #6573 )
2020-04-27 00:13:18 +02:00
Simon Frei
d3ed4de4ed
lib/model: Don't exit pullerRoutine on cancelled ctx ( fixes #6559 ) ( #6562 )
...
* lib/model: Don't exit pullerRoutine on cancelled ctx (fixes #6559 )
* actual fix
2020-04-21 18:55:14 +01:00
Simon Frei
6bbd24de12
lib/model: Refactor folder health/error handling ( fixes #6557 ) ( #6558 )
2020-04-21 10:15:59 +02:00
greatroar
2f26a95973
lib/db, lib/model: Code simplifications ( #6484 )
...
NamespacedKV.prefixedKey is still small enough to be inlined.
2020-03-31 14:32:24 +02:00
Jakob Borg
20aaa5927b
lib/protocol: Use BlocksHash to compare block lists when available ( #6401 )
...
This is an optimization for faster equal checks on block lists.
2020-03-10 14:46:49 +01:00
Simon Frei
f0e33d052a
lib: More contextification ( #6343 )
2020-02-24 21:57:15 +01:00
Jakob Borg
55937b61ca
lib/model: Add global request limiter ( fixes #6302 ) ( #6303 )
...
This adds a new config with the simple and concise name
maxConcurrentIncomingRequestKiB. This limits how many bytes we have "in
the air" in the form of response data being read and processed.
After some testing I think that not having this limiter is seldom a
great idea and thus I propose a default value of 256 MiB for this new
setting.
I also refactored the folder IO limiter to be a model/folder attribute
instead of a package global.
2020-02-01 08:02:18 +01:00
Jakob Borg
d91c4b010b
lib/config, lib/model: Limit concurrent pulls ( fixes #5914 ) ( #6290 )
...
Adds a new folder state "Waiting to Sync" in the same vein as the
existing "Waiting to Scan". This vastly improves performances in the
rare cases when there are lots and lots of folders operating.
2020-01-27 17:31:17 +01:00
Simon Frei
08f0e125ef
all: Transactionalize db.FileSet ( fixes #5952 ) ( #6239 )
2020-01-21 18:23:08 +01:00
Simon Frei
08753ccabe
lib/model: Reset queue after all pulling is done ( fixes #5867 ) ( #6256 )
2020-01-08 12:21:22 +01:00
