Commit Graph

728 Commits

Author SHA1 Message Date
Jakob Borg
77970d5113
refactor: use modern Protobuf encoder (#9817)
At a high level, this is what I've done and why:

- I'm moving the protobuf generation for the `protocol`, `discovery` and
`db` packages to the modern alternatives, and using `buf` to generate
because it's nice and simple.
- After trying various approaches on how to integrate the new types with
the existing code, I opted for splitting off our own data model types
from the on-the-wire generated types. This means we can have a
`FileInfo` type with nicer ergonomics and lots of methods, while the
protobuf generated type stays clean and close to the wire protocol. It
does mean copying between the two when required, which certainly adds a
small amount of inefficiency. If we want to walk this back in the future
and use the raw generated type throughout, that's possible, this however
makes the refactor smaller (!) as it doesn't change everything about the
type for everyone at the same time.
- I have simply removed in cold blood a significant number of old
database migrations. These depended on previous generations of generated
messages of various kinds and were annoying to support in the new
fashion. The oldest supported database version now is the one from
Syncthing 1.9.0 from Sep 7, 2020.
- I changed config structs to be regular manually defined structs.

For the sake of discussion, some things I tried that turned out not to
work...

### Embedding / wrapping

Embedding the protobuf generated structs in our existing types as a data
container and keeping our methods and stuff:

```
package protocol

type FileInfo struct {
  *generated.FileInfo
}
```

This generates a lot of problems because the internal shape of the
generated struct is quite different (different names, different types,
more pointers), because initializing it doesn't work like you'd expect
(i.e., you end up with an embedded nil pointer and a panic), and because
the types of child types don't get wrapped. That is, even if we also
have a similar wrapper around a `Vector`, that's not the type you get
when accessing `someFileInfo.Version`, you get the `*generated.Vector`
that doesn't have methods, etc.

### Aliasing

```
package protocol

type FileInfo = generated.FileInfo
```

Doesn't help because you can't attach methods to it, plus all the above.

### Generating the types into the target package like we do now and
attaching methods

This fails because of the different shape of the generated type (as in
the embedding case above) plus the generated struct already has a bunch
of methods that we can't necessarily override properly (like `String()`
and a bunch of getters).

### Methods to functions

I considered just moving all the methods we attach to functions in a
specific package, so that for example

```
package protocol

func (f FileInfo) Equal(other FileInfo) bool
```

would become

```
package fileinfos

func Equal(a, b *generated.FileInfo) bool
```

and this would mostly work, but becomes quite verbose and cumbersome,
and somewhat limits discoverability (you can't see what methods are
available on the type in auto completions, etc). In the end I did this
in some cases, like in the database layer where a lot of things like
`func (fv *FileVersion) IsEmpty() bool` becomes `func fvIsEmpty(fv
*generated.FileVersion)` because they were anyway just internal methods.

Fixes #8247
2024-12-01 16:50:17 +01:00
Jakob Borg
49aa9399be Repair config tests 2015-08-09 11:46:28 +02:00
Lode Hoste
dfaa999291 Stop folder when running out of disk space (fixes #2057)
& tweaks by calmh
2015-08-09 10:37:23 +02:00
Jakob Borg
7705a6c1f1 mv internal lib 2015-08-09 09:35:26 +02:00
KAMADA Ken'ichi
3f3b2f4c99 Fix editing address lists in the Settings dialog.
Setting "Sync Protocol Listen Addresses" and "Global Discovery
Server" in the Settings dialog does not work.  This bug seems to
have been introduced in 26d52be.
2015-08-09 15:52:22 +09:00
Jakob Borg
257d1afdf8 Enable browser caching of static resources
This sends the Cache-Control header to allow caching of static resources,
and checks the If-Modified-Since header to allow browser to use the
cached resource on refresh. Also fixes some paths that caused redirects
(core//foo -> core/foo)
2015-08-08 13:50:18 +02:00
Jakob Borg
e312fdd4f8 Handle multiple case insensitivity prefixes in ignores (fixes #2134) 2015-08-08 11:58:20 +02:00
Jakob Borg
9b6681d543 Use grid instead of three-column (fixes #2130) 2015-08-08 11:37:19 +02:00
Jakob Borg
bcc04623c1 Fix editing device address (fixes #2129) 2015-08-08 11:29:13 +02:00
Lode Hoste
219a25fe80 Rename 'delete' to 'remove' (fixes #1007) 2015-08-03 20:01:26 +02:00
Jakob Borg
b1dd704819 Rebuild assets 2015-08-02 09:41:22 +02:00
Jakob Borg
19451e0654 Translation and docs update 2015-08-02 09:19:32 +02:00
Jakob Borg
fa922d7792 Send index immediately on local change 2015-08-02 08:08:24 +02:00
Jakob Borg
bbe1de3119 Merge pull request #2100 from AudriusButkevicius/memory
Use protocol provided buffer for requests (fixes #1157)
2015-08-02 08:07:52 +02:00
Audrius Butkevicius
1977c526e4 Use protocol provided buffers for requests (fixes #1157) 2015-08-01 12:35:47 +01:00
Lode Hoste
9b9fe0d65c Reduce scanning effort 2015-07-31 21:32:49 +02:00
Jakob Borg
d3972b88f2 Remove set.ReplaceWithDelete (dead code) 2015-07-28 21:09:43 +04:00
Jakob Borg
a5f6e3bba0 Update translations and docs 2015-07-26 11:25:40 +02:00
Jakob Borg
d170660c25 Usage -> Statistics 2015-07-24 12:04:16 +02:00
Audrius Butkevicius
2202aaed51 Merge pull request #2087 from calmh/norestart
Add folders without restart (fixes #2063)
2015-07-24 08:06:36 +01:00
Jakob Borg
1acfa291a0 Link to usage data (ref syncthing/website#17) 2015-07-24 08:53:33 +02:00
Jakob Borg
21accd534c Add folders without restart (fixes #2063) 2015-07-24 08:20:57 +02:00
Jakob Borg
76480adda5 Add devices without restart (fixes #2083) 2015-07-22 10:43:47 +02:00
Jakob Borg
e205f8afbb Don't error integration tests on unexpected EOF at Stop() 2015-07-22 10:43:33 +02:00
Jakob Borg
12eab4a8ba Optionally ignore remote deletes (fixes #1254) 2015-07-21 20:39:19 +02:00
Jakob Borg
da4ebb6535 Determine conflict winner based on change type and modification time (fixes #1848) 2015-07-21 15:39:20 +02:00
Jakob Borg
491452a19d Improve performance for syncing many small files quickly
Without this, as soon as we'd touched 1000 files in the last minute
(which can happen), we got stuck doing cache cleaning all the time,
burning a lot of CPU time.
2015-07-20 15:30:44 +02:00
Jakob Borg
7d3257b222 Use soft shutdown when running tests 2015-07-20 15:05:15 +02:00
Jakob Borg
1836ef2884 Squashed commit of pull request #1981
Conflicts:
	gui/scripts/syncthing/core/controllers/syncthingController.js
	internal/auto/gui.files.go
2015-07-20 14:48:03 +02:00
Jakob Borg
f0684d83e9 Add osutil.AtomicWriter
This captures the common pattern of writing to a temp file and moving it
to it's real name only if everything went well. It reduces the amount of
code in some places where we do this, but maybe not as much as I
expected because the upgrade thing is still a special snowflake...
2015-07-20 14:27:14 +02:00
Jakob Borg
7683096fe1 Add dva 2015-07-20 14:25:07 +02:00
Jakob Borg
bb438bfb17 Squashed commit of pull request #1990
commit 4eb3ff55ba964fde84f4ac98c7304b2b883f03e1
Merge: ddb3fea a04b005
Author: Brian R. Becker <brbecker@gmail.com>
Date:   Sat Jul 11 20:45:30 2015 -0700

    Merge remote-tracking branch 'upstream/master'

commit ddb3fea0d994794375bd3a71a417240b27cb9d09
Author: Brian R. Becker <brbecker@gmail.com>
Date:   Mon Jun 22 11:36:58 2015 -0700

    Corrected spelling in GUI message
2015-07-20 14:22:36 +02:00
Jakob Borg
a11aa295de Squashed commit of pull request #1875
commit d60fbce311414f3ade16c7b1f6525b1d02f01699
Author: Jacek Szafarkiewicz <szafar@linux.pl>
Date:   Mon Jun 1 11:16:36 2015 +0200

    Correct order of deb files

commit 3b2ecfcc45343701ccb98483a62f70c102a70785
Merge: f4daebb c23a601
Author: Jacek Szafarkiewicz <szafar@linux.pl>
Date:   Mon Jun 1 11:15:06 2015 +0200

    Merge github.com:syncthing/syncthing

    Conflicts:
    	build.go

commit f4daebb8510f0b40b15e404817a7799e0c3b592c
Author: Jacek Szafarkiewicz <szafar@linux.pl>
Date:   Tue May 26 12:58:25 2015 +0200

    Add me to AUTHORS

commit 9e77f4bea01cc826132a5fbc6ca412cce5327e97
Author: Jacek Szafarkiewicz <szafar@linux.pl>
Date:   Tue May 26 12:57:40 2015 +0200

    Add systemd files to deb packate
2015-07-20 14:18:07 +02:00
Jakob Borg
59e829e595 Translation & docs update 2015-07-19 13:34:11 +02:00
Jakob Borg
78dca5fe8b Assets & translations 2015-07-16 14:03:21 +02:00
Jakob Borg
f3f6b03d85 Don't let folder ID escape into HTML tag ID:s (fixes #2059) 2015-07-16 08:13:10 +02:00
Jakob Borg
8fab7ec5e3 Decrease timing sensitivity of ignore.TestCache 2015-07-14 12:12:57 +02:00
Daniel Bergmann
569314be45 Added select-on-click for remote 'Device ID' and 'API Key'
Now these uneditable boxes match the behaviour when clicking on the ID
box in Actions > Show ID.
2015-07-13 17:01:13 +01:00
Jakob Borg
909d60464e Revert "Merge pull request #2053 from calmh/atomicwriter" (fixes #2058)
This reverts commit b611f72e08, reversing
changes made to a04b005e93.
2015-07-13 12:47:32 +02:00
Jakob Borg
d855abf8b0 Translation & docs update 2015-07-13 08:24:04 +02:00
Jakob Borg
44e3bec42e Add osutil.AtomicWriter
This captures the common pattern of writing to a temp file and moving it
to it's real name only if everything went well. It reduces the amount of
code in some places where we do this, but maybe not as much as I
expected because the upgrade thing is still a special snowflake...
2015-07-12 14:28:59 +10:00
Jakob Borg
a04b005e93 Revert "Let suture logging bubble upwards"
This reverts commit 1b837116e6.
2015-07-11 11:12:20 +10:00
Jakob Borg
1b837116e6 Let suture logging bubble upwards 2015-07-11 10:52:57 +10:00
Jakob Borg
0c28216ee5 Make sure connection is added to m.protoConn and m.rawConn before it's Start()ed (fixes #2034) 2015-07-10 16:43:41 +10:00
Jakob Borg
b78c515724 Debug output on request errors 2015-07-10 16:01:10 +10:00
Audrius Butkevicius
1ded554a15 Fix advanced option saving (fixes #2042) 2015-07-09 21:58:58 +01:00
Jakob Borg
1da3a57fe7 Asset rebuild 2015-07-05 19:44:55 +02:00
Jakob Borg
57a5d13c47 Translation and docs update 2015-07-05 11:24:21 +02:00
Jakob Borg
500b96240b Don't show Failed Items on folder masters 2015-07-05 11:21:15 +02:00
Jakob Borg
fddc4c2fc0 Rebuild assets 2015-07-05 11:13:35 +02:00