Commit Graph

405 Commits

Author SHA1 Message Date
Simon Frei
fc81e2b3d7 lib/model: Do folder watch operations under lock (fixes #5392) (#5395) 2018-12-21 12:06:21 +01:00
Simon Frei
a09079ed25 all: Display list of locally changed items in UI (fixes #5336) (#5337) 2018-12-11 09:59:04 +01:00
Jakob Borg
1b59960ff9
lib/model: Attempt to unflake Deregister tests (fixes #5362) (#5366) 2018-12-11 09:42:03 +01:00
Audrius Butkevicius
ff2cde469e lib/model: Allow limiting number of concurrent scans (fixes #2760) (#4888) 2018-12-05 08:40:05 +01:00
Simon Frei
2f9840ddae lib: Introduce fs.IsParent (fixes #5324) (#5326) 2018-11-22 11:16:45 +01:00
Simon Frei
db8777c29e lib/model: Add test for #5323 (#5325) 2018-11-13 08:36:16 +00:00
Simon Frei
33bed5b1ec lib/model: Don't compare permissions if IgnorePerms is true (fixes #5323) (#5322) 2018-11-13 08:54:49 +01:00
Simon Frei
4f27bdfc27 lib/model, lib/protocol: Handle request concurrency in model (#5216) 2018-11-13 08:53:55 +01:00
Simon Frei
d510e3cca3 all: Display errors while scanning in web UI (fixes #4480) (#5215) 2018-11-07 11:04:41 +01:00
Simon Frei
64a591610b lib/model: Check if files from queue are invalid (fixes #5291) (#5292) 2018-10-26 19:13:35 +01:00
Alexandre Viau
9745679c63 lib: chmod -x on progressemitter.go and errors.go (#5281) 2018-10-21 16:08:14 +01:00
Simon Frei
0690fe7585 lib/model: Unnecessary return (#5264) 2018-10-11 15:08:37 +02:00
Simon Frei
1b10607def lib/model: Don't check folder health if there is nothing to pull (fixes #2497) (#5255) 2018-10-11 11:33:21 +02:00
Jakob Borg
9d7a811e72
lib/model: Don't flake out on shortcutting files (ref #5258, #5257, #5234) (#5259)
In a recent change (#5201) this return disappeared. The effect is that
we first shortcut the file and then also treat it normally. This results
in to database updates after each other, which are bound to end up in
the same batch. This means we remove one sequence entry and add two.

Not marking the issues as fixed, because I need to do more testing and
there are other discrepancies...
2018-10-11 11:07:52 +02:00
Simon Frei
d10773c311 lib/db, lib/model: Resolve identical recv only items (fixes #5130) (#5230) 2018-10-10 12:43:07 +02:00
Simon Frei
523ac45456 lib/model: Treat failed rename like del&update (#5203) 2018-10-10 11:37:20 +02:00
Jakob Borg
b50d57b7fd
lib/db: Refactor: use a Lowlevel type underneath Instance (ref #5198) (#5212)
This adds a thin type that holds the state associated with the
leveldb.DB, leaving the huge Instance type more or less stateless. Also
moves some keying stuff into the DB package so that other packages need
not know the keying specifics.

(This does not, yet, fix the cmd/stindex program, in order to keep the
diff size down. Hence the keying constants are still exported.)
2018-10-10 11:34:24 +02:00
Jakob Borg
f12ca95af2 lib/model: Unflake TestFolderRestartZombies (fixes #5244) 2018-10-07 13:58:25 +02:00
Jakob Borg
f528923d1e lib/model, cmd/syncthing: Wait for folder restarts to complete (fixes #5233) (#5239)
* lib/model, cmd/syncthing: Wait for folder restarts to complete (fixes #5233)

This is the somewhat ugly - but on the other hand clear - fix for what
is really a somewhat thorny issue. To avoid zombie folder runners a new
mutex is introduced that protects the RestartFolder operation. I hate
adding more mutexes but the alternatives I can think of are worse.

The other part of it is that the POST /rest/system/config operation now
waits for the config commit to complete. The point of this is that until
the commit has completed we should not accept another config commit. If
we did, we could end up with two separate RestartFolders queued in the
background. While they are both correct, and will run without
interfering with each other, we can't guarantee the order in which they
will run. Thus it could happen that the newer config got committed
first, and the older config commited after that, leaving us with the
wrong config running.

* test

* wip

* hax

* hax

* unflake test

* per folder mutexes

* paranoia

* race
2018-10-05 09:26:25 +01:00
Simon Frei
c2b0d309fb lib/model: Prevent repeat db update (#5231) 2018-09-27 07:41:40 +02:00
Simon Frei
272fb3b444
all: Adjust windows perms in fs package (#5200) 2018-09-16 16:09:56 +02:00
Simon Frei
60eb9088ff lib/model: Extend shortcutFile (#5201) 2018-09-16 10:29:06 +01:00
Simon Frei
c8652222ef all: Check files on disk/in db when deleting/renaming (fixes #5194) (#5195) 2018-09-16 09:48:14 +02:00
Simon Frei
a57fa9cfab lib/model: Polish (#5189) 2018-09-13 18:17:13 +02:00
Jakob Borg
9e00b619ab all, vendor: Switch back to non-forked thejerf/suture (#5171) 2018-09-08 12:56:56 +03:00
Jakob Borg
4cb6bb6f64 Merge branch 'release'
* release:
  lib/db: Fix inconsistency in sequence index (fixes #5149) (#5158)
2018-09-02 21:05:53 +02:00
Jakob Borg
b80da29b23 lib/db: Fix inconsistency in sequence index (fixes #5149) (#5158)
The problem here is that we would update the sequence index before
updating the FileInfos, which would result in a high sequence number
pointing to a low-sequence FileInfo. The index sender would pick up the
high sequence number, send the old file, and think everything was good.
On the receiving side the old file is a no-op and ignored. The file
remains out of sync until another update for it happens.

This fixes that by correcting the order of operations in the database
update: first we remove old sequence index entries, then we update the
FileInfos (which now don't have anything pointing to them) and then we
add the sequence indexes (which the index sender can see).

The other option is to add "proper" transactions where required at the
database layer. I actually have a branch for that, but it's literally
thousands of lines of diff and I'm putting that off for another day as
this solves the problem...
2018-09-02 21:02:28 +02:00
Jakob Borg
836ca50570
lib/db: Fix inconsistency in sequence index (fixes #5149) (#5158)
The problem here is that we would update the sequence index before
updating the FileInfos, which would result in a high sequence number
pointing to a low-sequence FileInfo. The index sender would pick up the
high sequence number, send the old file, and think everything was good.
On the receiving side the old file is a no-op and ignored. The file
remains out of sync until another update for it happens.

This fixes that by correcting the order of operations in the database
update: first we remove old sequence index entries, then we update the
FileInfos (which now don't have anything pointing to them) and then we
add the sequence indexes (which the index sender can see).

The other option is to add "proper" transactions where required at the
database layer. I actually have a branch for that, but it's literally
thousands of lines of diff and I'm putting that off for another day as
this solves the problem...
2018-09-02 20:58:32 +02:00
Audrius Butkevicius
aec66045ef
lib/config: Rewrite pending notifications (fixes #2291) 2018-08-25 11:36:10 +01:00
Simon Frei
03c0537340 lib/model: Fix regressions detecting deletes/ignores (fixes #5125, fixes #5127) (#5129) 2018-08-25 10:32:35 +02:00
Jakob Borg
7dde6c7e3c lib/model: Clear out-of-space-errored files from queue (fixes #5143) 2018-08-25 10:26:10 +02:00
Simon Frei
cb0f4ce55a lib/model: Don't stop folder if out of disk space (fixes #2370) (#5099)
This removes the out of disk space check from CheckHealth. The disk space is now
only checked if there are files to pull, in which case pulling those files is
stopped, but everything else (dirs, links, deletes) keeps running -> can recover
disk space through pulling.
2018-08-25 10:16:38 +02:00
Jakob Borg
a02db70a63 lib/model: Process download progress messages for all folder types (fixes #5131) (#5139) 2018-08-21 18:49:35 +01:00
Simon Frei
165417c462 lib/model: Fixes on receive-only test setup and pulling (#5136) 2018-08-19 22:34:26 +01:00
Simon Frei
ff3cbdc90d lib/model: Check availability later to catch renames (#5097) 2018-08-19 19:03:20 +01:00
Simon Frei
9028969617
lib/model: Small fixes to test convenience functions (#5128) 2018-08-16 12:11:48 +02:00
Simon Frei
7c0798b622 lib/model: Always release the lock (#5126) 2018-08-15 16:44:59 +02:00
Simon Frei
ee42c46bd3 lib/model: Catch racy nil deref in ClusterConfig (#5106) 2018-08-15 16:44:20 +02:00
Simon Frei
885f6fcf28 lib/model: Always release the lock (#5126) 2018-08-15 16:33:03 +02:00
Simon Frei
dc57bab107 lib/model: Don't enter pulling state if we need nothing (fixes #4782) (#5118) 2018-08-13 20:39:25 +02:00
Jakob Borg
48795dba07
all: Don't let Suture capture panics (fixes #4758) (#5119)
Fork with new option.
2018-08-13 20:39:08 +02:00
Jakob Borg
b37c05c6b8
lib/model: Don't run watcher on recvonly tests (fixes #5110) (#5112) 2018-08-11 22:19:37 +02:00
Simon Frei
dfe4008607 lib/model: Catch racy nil deref in ClusterConfig (#5106) 2018-08-11 09:10:29 +02:00
Audrius Butkevicius
5161f03f02 lib/config: Fix aliased append, copy config inputs and outputs (fixes #5063) (#5069) 2018-07-26 23:14:12 +02:00
Audrius Butkevicius
9f87fd1fcf lib/model: More auto accept tests (#5014) 2018-07-15 20:26:20 +03:00
Simon Frei
5592b8b190 lib/model: Record error for unavailable files (#5066) 2018-07-14 14:09:23 +01:00
Jakob Borg
f822b10550
all: Add receive only folder type (#5027)
Adds a receive only folder type that does not send changes, and where the user can optionally revert local changes. Also changes some of the icons to make the three folder types distinguishable.
2018-07-12 11:15:57 +03:00
Simon Frei
6b82538e62 lib/model: Also handle missing parent dir non-regular items (#5048)
This is an improvement of PR #4493 and related to (and maybe fixing) #4961
and #4475. Maybe fixing, because there is no clear reproducer for that
problem.

The previous PR added a mechanism to resurrect missing parent directories,
if there is a valid child file to be pulled. The same mechanism does not
exist for dirs and symlinks, even though a missing parent can happen for
those items as well. Therefore this PR extends the resurrection to all types
of pulled items.

In addition I moved the IsDeleted branch while iterating over
processDirectly to the existing IsDeleted branch in the WithNeed iteration.
This saves one pointless assignment and IsDeleted query. Also
2018-07-10 18:40:06 +03:00
Simon Frei
0f0290d574 lib/model, lib/weakhash: Abort pulling quicker on folder stop (ref #5028) 2018-07-04 08:07:33 +01:00
Simon Frei
7b0d8c2e77 lib/model: Release both locks when waiting for services to stop (fixes #5028) 2018-06-24 16:55:28 +01:00