Commit Graph

1262 Commits

Author SHA1 Message Date
Simon Frei
eed1edcca0 cmd/syncthing: Ensure myID is set by making it local (fixes #5859) (#5862) 2019-07-17 07:19:14 +02:00
Simon Frei
0025e9ccfb all: Refactor cmd/syncthing creating lib/syncthing (ref #4085) (#5805)
* add skeleton for lib/syncthing

* copy syncthingMain to lib/syncthing (verbatim)

* Remove code to deduplicate copies of syncthingMain

* fix simple build errors

* move stuff from main to syncthing with minimal mod

* merge runtime options

* actually use syncthing.App

* pass io.writer to lib/syncthing for auditing

* get rid of env stuff in lib/syncthing

* add .Error() and comments

* review: Remove fs interactions from lib

* and go 1.13 happened

* utility functions
2019-07-14 11:43:13 +01:00
Simon Frei
82b70b9fae lib/model, lib/protocol: Track closing connections (fixes #5828) (#5829) 2019-07-14 11:03:55 +02:00
Simon Frei
def4b8cee5 lib/config: Error on empty folder path (fixes #5853) (#5854) 2019-07-14 11:03:14 +02:00
Aurélien Rainone
f1a7dd766e all: Add comment to ensure correct atomics alignment (fixes #5813)
Per the sync/atomic bug note:

> On ARM, x86-32, and 32-bit MIPS, it is the caller's
> responsibility to arrange for 64-bit alignment of 64-bit words
> accessed atomically. The first word in a variable or in an
> allocated struct, array, or slice can be relied upon to be
> 64-bit aligned.

All atomic accesses of 64-bit variables in syncthing code base are
currently ok (i.e they are all 64-bit aligned).

Generally, the bug is triggered because of incorrect alignement
of struct fields. Free variables (declared in a function) are
guaranteed to be 64-bit aligned by the Go compiler.

To ensure the code remains correct upon further addition/removal
of fields, which would change the currently correct alignment, I
added the following comment where required:

     // atomic, must remain 64-bit aligned

See https://golang.org/pkg/sync/atomic/#pkg-note-BUG.
2019-07-13 14:05:39 +01:00
Simon Frei
20c8dbd9ed lib/model: Fix integer conversion (fixes #5837) (#5851) 2019-07-12 16:37:12 +02:00
xduugu
4b3f9b1af9 lib/versioner: Replace multiple placeholders in a single token in external command (fixes #5849)
* lib/versioner: Add placeholder to provide the absolute file path to external commands

This commit adds support for a new placeholder, %FILE_PATH_FULL%, to the
command of the external versioner. The placeholder will be replaced by
the absolute path of the file that should be deleted.

* Revert "lib/versioner: Add placeholder to provide the absolute file path to external commands"

This reverts commit fb48962b947358e90a76e87a7794adb6057de476.

* lib/versioner: Replace all placeholders in external command (fixes #5849)

Before this commit, only these placeholders were replaced that span a
whole word, for example "%FOLDER_PATH%". Words that consisted of more
than one placeholder or additional characters, for example
"%FOLDER_PATH%/%FILE_PATH%", were left untouched.

* fixup! lib/versioner: Replace all placeholders in external command (fixes #5849)
2019-07-12 08:45:39 +01:00
Simon Frei
3446d50201 lib/model: Remove pointless error that watch hasn't started (fixes #5833) (#5834) 2019-07-10 11:00:06 +02:00
Simon Frei
9fef1552fc lib/db, lib/model: Remove folder info from panics (ref #5839) (#5840) 2019-07-10 10:57:49 +02:00
Simon Frei
485acda63b lib/relay: Call the proper Error method (ref #5806) (#5841) 2019-07-09 22:29:19 +01:00
Simon Frei
ba056578ec
lib: Add util.Service as suture.Service template (fixes #5801) (#5806) 2019-07-09 11:40:30 +02:00
Simon Frei
863fe23347 gui, lib/model: Fix download progress accounting (fixes #5811) (#5815) 2019-06-30 09:23:47 +02:00
Simon Frei
1cf352a722 lib/model: NewFileSet outside fmut (#5818) 2019-06-29 08:49:30 +02:00
Simon Frei
b58f6ca886 lib/model: Correct/unify check if item changed (#5819) 2019-06-29 07:45:41 +02:00
Simon Frei
3c7e7e971d lib/model: Make jobQueue.Jobs paginated (fixes #5754) (#5804)
* lib/model: Make jobQueue.Jobs paginated (fixes #5754)

* fix, no test yet

* add test
2019-06-27 19:25:38 +01:00
Audrius Butkevicius
afde0727fe lib/versioner: Revert naming change (fixes #5807) (#5808) 2019-06-25 08:56:11 +03:00
Simon Frei
bf744ded31 cmd/syncthing, lib/db: Exit/close db faster (fixes #5781) (#5782)
This adds a 10s timeout on closing the db and additionally cancels active
db iterators and waits for them to terminate before closing the db.
2019-06-17 15:27:25 +03:00
Simon Frei
cea5962417 lib/model: Unflake TestPullInvalidIgnoredSR/SO (fixes #5796) (#5799) 2019-06-17 15:23:28 +03:00
Simon Frei
02752af862 lib/protocol: Don't block on Close (fixes #5794) (#5795) 2019-06-14 19:04:41 +02:00
Simon Frei
abd363e8bb
lib/model: Don't error on pulling deletion of invalid file (fixes #5791) (#5792) 2019-06-14 08:48:14 +02:00
Simon Frei
13a746e0fb lib/model: Prevent nil deref if folder stopped (fixes #5780) (#5778) 2019-06-11 11:48:51 +02:00
Audrius Butkevicius
21f50e2f8f lib/versioner: Use mtime for version cleanup (fixes #5765) (#5769) 2019-06-11 09:16:55 +02:00
Audrius Butkevicius
b7c70a9817 lib/fs: Enhance mtimefs, use everywhere (fixes #5777) (#5776) 2019-06-11 08:27:12 +02:00
Jakob Borg
42ce6be9b9
lib/ur: Implement crash (panic) reporting (fixes #959) (#5702)
* lib/ur: Implement crash (panic) reporting (fixes #959)

This implements a simple crash reporting method. It piggybacks on the
panic log files created by the monitor process, picking these up and
uploading them from the usage reporting routine.

A new config value points to the crash receiver base URL, which defaults
to "https://crash.syncthing.net/newcrash" (following the pattern of
"https://data.syncthing.net/newdata" for usage reports, but allowing us
to separate the service as required).
2019-06-11 08:19:11 +02:00
Mingxuan Lin
eb4fe808c5 lib/fs: Fallback EvalSymlinks method on windows (fixes #5609) (#5611) 2019-06-10 14:33:53 +02:00
Simon Frei
1054ce9354
lib/model: Refactor sending indexes as suture service (#5757) 2019-06-10 13:27:22 +02:00
Audrius Butkevicius
ee746263fb
lib/connections: Do not leak FDs, fix address copy (fixes #5767) (#5768)
* lib/connections: Do not leak FDs, fix address copy (fixes #5767)

* build

* Update quic_listen.go

* Update quic_listen.go
2019-06-09 22:14:00 +01:00
Jakob Borg
41ff4b323e lib/api: Correct logic for errors.jons in support bundle (fixes #5766)
The err check should always be done with != and have the error case
first, to follow the pattern and avoid mistakes like these.
2019-06-09 09:35:05 +02:00
Jakob Borg
997bb5e7e1 all: Remove "large blocks" config (#5763)
We now always use large / variable blocks.
2019-06-06 15:57:38 +01:00
Jakob Borg
ca2fa8de4e lib/build: Version 1.2 will be the Fermium Flea 2019-06-06 14:45:07 +02:00
Simon Frei
e39d3f95dd lib/protocol: Prioritize close msg and add close timeout (#5746) 2019-06-05 14:01:59 +08:00
Simon Frei
6664e01acf lib/protocol: Return from ClusterConfig when closed (#5752) 2019-05-29 12:14:00 +02:00
Jakob Borg
5ce5b2c94a
lib/config: Refactor migrations a bit (#5750)
This breaks out config migrations to a separate concept, making it
(imho) slightly easier to maintain and get an overview.
2019-05-29 11:37:44 +02:00
Audrius Butkevicius
e714df013f lib/connections: Add QUIC protocol support (fixes #5377) (#5737) 2019-05-29 09:56:40 +02:00
Simon Frei
3775a64d5c lib/protocol: Don't send anything else before cluster config (#5741) 2019-05-27 11:15:34 +01:00
Simon Frei
129df0613b lib/model: Unflake folder restart with blocking conn test (ref #5707) (#5744) 2019-05-27 10:58:09 +01:00
Simon Frei
486230768e lib/fs, lib/model: Add error channel to Watch to avoid panics (fixes #5697) (#5734)
* lib/fs, lib/model: Add error channel to Watch to avoid panics (fixes #5697)

* forgot unsupported watch

* and more non(-standard)-unixy fixes

* and windows test

* review
2019-05-25 20:08:26 +01:00
Simon Frei
9e6db72535 lib/protocol: Don't call receiver after calling Closed (fixes #4170) (#5742)
* lib/protocol: Don't call receiver after calling Closed (fixes #4170)

* review
2019-05-25 20:08:07 +01:00
Simon Frei
d91da8feee lib/model: Readd special handling of conn close in TestIssue5063 (#5743)
This partially reverts commit 64518b0f7e.
2019-05-25 18:51:13 +01:00
Simon Frei
64518b0f7e
lib/model: Close connections when model is stopped (#5733) 2019-05-25 16:00:32 +02:00
Simon Frei
5d35b2c540 lib/protocol: Test for Close on blocking send deadlock (ref #5442) (#5732) 2019-05-23 21:42:02 +01:00
Simon Frei
5b306510a0
lib/model: Consistently cleanup model in tests (#5724) 2019-05-19 14:29:07 +02:00
Simon Frei
1b2b970f32 lib/model, lib/testutils: Test closing a connection on folder restart (#5707) 2019-05-18 08:53:59 +02:00
Simon Frei
5ffbb7668d lib/model: Fix test flakyness regression (ref #5592) (#5718) 2019-05-18 08:52:50 +02:00
Simon Frei
441ea109a1
lib/model: Refactor file deletions when pulling (#5699) 2019-05-17 18:29:54 +02:00
Jakob Borg
2c866277a2 lib/api, lib/connections, gui: Show connection error for disconnected devices (fixes #3345) (#5727)
* lib/api, lib/connections, gui: Show connection error for disconnected devices (fixes #3345)

This adds functionality in the connetions service to track the last
error per address. That is in turn exposed in the /rest/system/status
API method, as that is also where we already show the listener status
from the connection service.

The GUI uses this info where it lists addresses, showing errors (if any)
in red underneath each address.

I also slightly refactored the existing status method on the connection
service to have a better name and return typed information.

* ok

* review

* formatting

* review
2019-05-16 22:11:45 +01:00
André Colomb
e4956358fb lib/model: Remove superfluous check for IndexID in remote ClusterConfig (#5717)
The check in ClusterConfig() when iterating through announced devices
in a folder explicitly skips entries without a non-zero IndexID.
Therefore, the check for IndexID == 0 just below will never be true
and the intended cleanup of local index data will not happen.

Plainly remove that check to make the intended case distinction work.
2019-05-12 21:17:55 +02:00
Simon Frei
dfbbb286fc lib/fs: Consider win83 for root path as well when watching (ref #5706) (#5709) 2019-05-11 10:06:04 +02:00
Simon Frei
fbd445fe0a lib/api: Clean up after test, enables test caching (#5710) 2019-05-11 08:41:32 +02:00
Simon Frei
2b246eeb52
lib/model: Move test utilities to separate files (#5694) 2019-05-10 13:33:45 +02:00
Simon Frei
2558b021e5 lib/fs: Remove \\?\ for drive letters when watching (fixes #5578) (#5701) 2019-05-10 09:09:58 +02:00
Jakob Borg
31be810eb6 lib/model: Don't fail operation when fsync() fails (fixes #5704) (#5705) 2019-05-09 21:20:29 +01:00
Jakob Borg
62a6d619e7 Merge branch 'release'
* release:
  lib/fs: Revert removal of \\?\ for drive letters (fixes #5695)
2019-05-08 17:38:08 +02:00
Jakob Borg
a04fcfe749 lib/fs: Revert removal of \\?\ for drive letters (fixes #5695)
This reverts commit ca823bd591 from #5633.

This un-fixes bug #5578.
2019-05-08 17:31:52 +02:00
Simon Frei
283f39ae5f lib/protocol: Revert unreleased changes related to closing connections (#5688)
This reverts commits:
    ec7c88ca55
    19b51c9b92
    5da41f75fa
    04b927104f
2019-05-08 08:08:26 +02:00
Simon Frei
b45d77b6be lib/model: Fix regression deleting directories on pull (ref #5690) (#5691) 2019-05-06 20:55:26 +02:00
Simon Frei
79e67b7f79 lib/model: Remove individual pull errors on retry (fixes #5659) (#5690) 2019-05-06 17:21:56 +02:00
Simon Frei
5fa8467756 lib/model: Consistent use of locks (#5684) 2019-05-02 18:55:39 +01:00
Simon Frei
5954b105cd
lib/model: Let fakeConnection call Model.Closed on close (#5682) 2019-05-02 14:24:55 +02:00
Simon Frei
defc5dca65
lib/model: Use correct lock (#5683) 2019-05-02 14:09:42 +02:00
Simon Frei
9f358ecae0
lib/protocol: Refactor to pass only relevant argument to writeMessage (#5681) 2019-05-02 14:09:19 +02:00
Simon Frei
fe4daf242b
cmd, lib/db: Actually close goleveldb (fixes #5505) (#5671) 2019-05-02 11:15:00 +02:00
Simon Frei
ec7c88ca55 lib/protocol: Fix yet another deadlock (fixes #5678) (#5679)
* lib/protocol: Fix yet another deadlock (fixes #5678)

* more consistency

* read deadlock

* naming

* more naming
2019-05-02 09:21:07 +01:00
Simon Frei
19b51c9b92 lib/protocol: Don't close asyncMessage.done on success (fixes #5674) (#5675) 2019-04-29 17:52:57 +02:00
Audrius Butkevicius
0ca1f26ff8
lib/versioner: Restore for all versioners, cross-device support (#5514)
* lib/versioner: Restore for all versioners, cross-device support

Fixes #4631
Fixes #4586
Fixes #1634
Fixes #5338
Fixes #5419
2019-04-28 23:30:16 +01:00
Jakob Borg
2984d40641 lib/ignore: Additional test case (#5672) 2019-04-28 21:20:11 +01:00
Simon Frei
5da41f75fa lib/model, lib/protocol: Wait for reader/writer loops on close (fixes #4170) (#5657)
* lib/protocol: Wait for reader/writer loops on close (fixes #4170)

* waitgroup

* lib/model: Don't hold lock while closing connection

* fix comments

* review (lock once, func argument) and naming
2019-04-28 11:58:51 +01:00
Simon Frei
04b927104f lib/protocol: Don't send any messages before cluster config (#5646)
* lib/model: Send cluster config before releasing pmut

* reshuffle

* add model.connReady to track cluster-config status

* Corrected comments/strings

* do it in protocol
2019-04-23 20:47:11 +01:00
Simon Frei
110806842c lib/model: Pass the old not new fileinfo to deleteItemOnDisk (fixes #5654) (#5655) 2019-04-23 20:46:28 +01:00
Simon Frei
e302ccf4b4
lib/fs: Improve IsParent (#5658) 2019-04-22 11:12:32 +02:00
Simon Frei
926e9228ed lib/scanner, lib/model: File -> item when logging error (#5664) 2019-04-21 16:19:59 +01:00
Simon Frei
86e72d9973 lib/model: Use RLock and legacy polish (#5663) 2019-04-21 13:21:36 +01:00
Simon Frei
3bea59b0d9
lib/model: Refactor progressEmitter to de-/activate by config (fixes #4613) (#5623) 2019-04-13 14:20:51 +02:00
Simon Frei
fca895a632 lib/model: Fix block index calculation for recheckFile (fixes #5649) (#5648) 2019-04-12 15:21:07 +02:00
Simon Frei
79360e2205
lib/fs: Add test that symlinks are skipped on walk (#5644) 2019-04-10 22:36:37 +02:00
Simon Frei
9b2a73f9ab
lib/model: Pause pull for at least as long as failed pull took (fixes #5641) (#5643) 2019-04-10 16:22:23 +02:00
Simon Frei
c305265c62 lib/model: Request errors conforming to BEP specs (#5642) 2019-04-10 11:47:24 +02:00
Simon Frei
ca823bd591
lib/fs: When watching remove \\?\ for drive letters (fixes #5578) (#5633) 2019-04-09 09:02:04 +02:00
Simon Frei
395e524e2d lib/model: Update db on scan/pull in folder (#5608) 2019-04-07 13:29:17 +02:00
Jakob Borg
f0f8bf7784 lib/config: Round times stored for pending folders/devices (fixes #5554) 2019-03-27 20:35:42 +01:00
Jakob Borg
bf3834e367
lib/protocol: Use constants instead of init time hashing (fixes #5624) (#5625)
This constructs the map of hashes of zero blocks from constants instead
of calculating it at startup time. A new test verifies that the map is
correct.
2019-03-27 20:20:30 +01:00
Simon Frei
8d1eff7e41 lib/model: Missing queue.Done and reset between pulls (fixes #5332) (#5626) 2019-03-27 20:19:35 +01:00
Simon Frei
0cff66fcbc lib/model: Optimize puller for meta only changes (#5622) 2019-03-27 08:36:58 +00:00
Simon Frei
43a5be1c4b lib/model: Send item finished even after deregistering (fixes #5362) (#5620) 2019-03-26 21:31:33 +01:00
Simon Frei
b50039a920 cmd/syncthing, lib/api: Separate api/gui into own package (ref #4085) (#5529)
* cmd/syncthing, lib/gui: Separate gui into own package (ref #4085)

* fix tests

* Don't use main as interface name (make old go happy)

* gui->api

* don't leak state via locations and use in-tree config

* let api (un-)subscribe to config

* interface naming and exporting

* lib/ur

* fix tests and lib/foldersummary

* shorter URVersion and ur debug fix

* review

* model.JsonCompletion(FolderCompletion) -> FolderCompletion.Map()

* rename debug facility https -> api

* folder summaries in model

* disassociate unrelated constants

* fix merge fail

* missing id assignement
2019-03-26 19:53:58 +00:00
Simon Frei
d4e81fff8a lib/model: Remove unnecessary arguments in pulling functions (#5619) 2019-03-25 12:59:22 +01:00
Simon Frei
e31a116e6e lib/model: Pass correct file info to deleteItemOnDisk (fixes #5616) (#5617) 2019-03-25 12:42:39 +01:00
Simon Frei
e7ae851900 lib/model: Debug and test fixes (#5613) 2019-03-22 14:43:47 +01:00
Simon Frei
1a6d023ba8 lib/model: Run recvonly tests in temporary dirs (#5587) 2019-03-20 17:38:03 +00:00
Simon Frei
50d8c43e7c lib/config: Set UseLargeBlocks to true by default (fixes #5599) (#5600) 2019-03-12 12:59:26 +00:00
Simon Frei
289a02e994 lib/model: Integrate stat refs in folder (#5596) 2019-03-11 16:57:21 +00:00
Simon Frei
445637ebec lib/model: Pass fset & ignores on folder creation (#5592) 2019-03-11 07:28:54 +01:00
Simon Frei
3f3d2c814b
lib/model: Remove unused code (#5591) 2019-03-10 17:05:39 +01:00
Simon Frei
189e44488e lib/model: Introduce must test utility (#5586)
* lib/model: Introduce must test utility

* nice
2019-03-09 18:45:36 +00:00
Simon Frei
27ff20faa3 lib/model: Introduce waitForState test utility (#5585)
* lib/model: Introduce waitForState test utility

* folder id as param to waitForState
2019-03-09 10:36:55 +00:00
Simon Frei
b1564e53e4 lib/model: Improve test utilities (#5584) 2019-03-08 20:29:09 +00:00
Simon Frei
3a75b63776
lib/model: Use temp dir from osutils in tests (#5581) 2019-03-07 16:34:41 +01:00
Simon Frei
8e238c8e48
lib/model: Check before replacing existing file on pull (fixes #5571) (#5567) 2019-03-07 15:15:14 +01:00
Jakob Borg
9da3273eb8 lib/model: Clarify fileInfoBatch.flushIfFull criteria 2019-03-05 21:34:04 +01:00
Simon Frei
bd37f6da17 lib/model: Optimize dbUpdaterRoutine (#5576) 2019-03-05 21:32:37 +01:00