Commit Graph

250 Commits

Author SHA1 Message Date
Jakob Borg
47fa4b0a2c cmd/syncthing, lib/db, lib/model, lib/protocol: Implement delta indexes (fixes #438)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3427
2016-07-23 12:46:31 +00:00
Jakob Borg
7b7e35d339 lib/protocol: Hello message length is an int16
It used to be an int32, but that's unnecessary and the spec now says
int16. Also relaxes the size requirement to that which fits in a signed
int16 instead of limiting to 1024 bytes, to allow for future growth.

As reported in
https://forum.syncthing.net/t/difference-between-documented-and-implemented-protocol/7798

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3406
2016-07-17 21:41:20 +00:00
Jakob Borg
e440d30028 lib/protocol: Allow unknown message types
This lets us add message types in the future, for authentication or
other purposes, without completely breaking old clients. I see this as
similar behavior to adding fields to messages - newer clients must
simple be aware that older ones may ignore the message and act
accordingly.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3390
2016-07-05 09:29:28 +00:00
Jakob Borg
aa50ef4069 lib/model: Invalidate files with trailing white space on Windows (fixes #3227)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3383
2016-07-04 10:44:30 +00:00
Jakob Borg
fa0101bd60 lib/protocol, lib/discover, lib/db: Use protocol buffer serialization (fixes #3080)
This changes the BEP protocol to use protocol buffer serialization
instead of XDR, and therefore also the database format. The local
discovery protocol is also updated to be protocol buffer format.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3276
LGTM: AudriusButkevicius
2016-07-04 10:40:29 +00:00
Jakob Borg
d507126101 lib/protocol: Understand older/newer Hello messages (fixes #3287)
This is in preparation for future changes, but also improves the
handling when talking to pre-v0.13 clients. It breaks out the Hello
message and magic from the rest of the protocol implementation, with the
intention that this small part of the protocol will survive future
changes.

To enable this, and future testing, the new ExchangeHello function takes
an interface that can be implemented by future Hello versions and
returns a version indendent result type. It correctly detects pre-v0.13
protocols and returns a "too old" error message which gets logged to the
user at warning level:

   [I6KAH] 09:21:36 WARNING: Connecting to [...]:
     the remote device speaks an older version of the protocol (v0.12) not
     compatible with this version

Conversely, something entirely unknown will generate:

   [I6KAH] 09:40:27 WARNING: Connecting to [...]:
     the remote device speaks an unknown (newer?) version of the protocol

The intention is that in future iterations the Hello exchange will
succeed on at least one side and ExchangeHello will return the actual
data from the Hello together with ErrTooOld and an even more precise
message can be generated.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3289
2016-06-09 10:50:14 +00:00
Jakob Borg
ac40b27c79 lib/connections: Handle wrapped connection in SetTCPOptions (fixes #3223)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3225
2016-05-31 08:11:57 +00:00
Nate Morrison
86ca58e2a9 lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-19 00:19:26 +00:00
Jakob Borg
6f743f3138 Revert "lib/model: Emit LocalDiskUpdated events on detecting local changes"
This reverts commit 5a7fad0bcd.
2016-05-14 10:55:24 +02:00
Nate Morrison
5a7fad0bcd lib/model: Emit LocalDiskUpdated events on detecting local changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3055
2016-05-14 08:37:07 +00:00
klemens
bd41e21c26 all: Correct spelling in comments
Skip-check: authors pr-build-mac

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3056
2016-05-08 10:54:22 +00:00
Jakob Borg
2e840134d2 lib/protocol: Add Request benchmarks over raw and TLS encrypted TCP channels
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3040
2016-05-04 23:07:07 +00:00
Jakob Borg
66e1be33cf lib/protocol: Delete erroneously checked in test binary
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3039
2016-05-04 22:09:07 +00:00
Audrius Butkevicius
49387f9494 lib/protocol: Clean up error values, unused flags
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3025
2016-04-30 04:35:38 +00:00
AudriusButkevicius
1a5f524ae4 lib/model, lib/protocol: Implement temporary indexes (fixes #950)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2252
2016-04-15 10:59:41 +00:00
Jakob Borg
f3ac421266 lib/protocol: Comment the bit numbers for flags in IndexMessage 2016-04-10 10:47:30 +00:00
Audrius Butkevicius
2eb8a9ef56 all: Dead code cleanup 2016-04-09 01:10:31 +00:00
Jakob Borg
f5f0e46016 lib: Use bytes.Equal instead of bytes.Compare where possible 2016-03-31 15:12:46 +00:00
Audrius Butkevicius
b4f9a55e6e protocol: Add "Hello" message at connection start, also for unauthed peers 2016-03-25 20:29:07 +00:00
Lars K.W. Gohlke
16c3d39fd2 Add folder label in addition to ID (fixes #966)
An auto generated ID is suggested on folder creation to reduce conflicts with
folders created on other devices.
2016-03-11 09:48:46 +00:00
Lars K.W. Gohlke
9b1bebc9b2 Correct path to genxdr after the change to Go1.5+ vendoring 2016-03-09 12:43:16 +00:00
Jakob Borg
83c29e1945 Fix tests on Go 1.3 2016-03-08 09:07:18 +01:00
Laurent Etiemble
c8b6e6fd9b Increase maximum allowed file size to 10 Mblocks
Upgrade FileInfo up to 10000000 blocks. 1310 GB files can be shared.
Increase limit when unmarshaling XDR.
Increase the size of message.
2016-03-04 16:24:54 +01:00
Jakob Borg
a6a9af4f02 Fix marshalling tests for Go 1.6 2016-03-04 14:16:42 +01:00
Jakob Borg
e1ac740ac4 Use v2 of XDR package (actual changes) 2016-02-02 15:33:46 +01:00
Jakob Borg
4feeaf1641 Use v2 of XDR package (auto generated) 2016-02-02 12:44:33 +01:00
Jakob Borg
eb55d19786 Clean up error handling a bit in protocol.readMessage 2016-02-02 10:18:19 +01:00
Jakob Borg
1353e3916f A couple of protocol tests 2016-01-20 11:37:48 -08:00
Jakob Borg
11d4986517 Humanize serialization of version vectors (again) 2016-01-20 11:14:08 -08:00
Jakob Borg
8b3d75b339 Undo the hash algorithm additions; retain flag checks 2016-01-12 14:35:00 +01:00
Jakob Borg
357089a438 Mend protocol tests, for sure 2016-01-12 14:35:00 +01:00
Jakob Borg
9595687bce Improve protocol tests, close handling 2016-01-12 09:30:02 +01:00
Jakob Borg
acdddc0b79 Don't leak sendIndexes on disconnect (fixes #2589)
Adds a Closed() method on protocol.Connection and clears up
wireformatConnection a little too.
2016-01-11 17:57:25 +01:00
Audrius Butkevicius
ff4bab4c07 Silence the linter 2016-01-10 18:00:52 +00:00
Jakob Borg
837fde70ae Add XUnit compatible test output 2016-01-10 00:50:43 +01:00
Jakob Borg
0db80710aa Detect nonstandard hash algo and stop folder (ref #2314) 2016-01-01 20:14:31 +01:00
Jakob Borg
e5b33ce9f6 Regenerate XDR for empty struct types 2015-11-24 20:54:49 +01:00
Jakob Borg
d46f267663 Handle sparse files (fixes #245) 2015-11-21 17:58:09 +01:00
Jakob Borg
e3c0fba34b Must not call hex.Dump in non-debug mode... 2015-10-27 10:27:18 +01:00
Jakob Borg
83131103cf Don't load block list in ...Truncated methods
Speeds up and reduces allocations on those operations, at the price of
having a manually tweaked XDR decoder for FileInfoTruncated.

benchmark                         old ns/op      new ns/op      delta
BenchmarkReplaceAll-8             1868198122     1880206886     +0.64%
BenchmarkUpdateOneChanged-8       231852         172695         -25.51%
BenchmarkUpdateOneUnchanged-8     230624         179341         -22.24%
BenchmarkNeedHalf-8               104601744      109461427      +4.65%
BenchmarkHave-8                   29102480       34105026       +17.19%
BenchmarkGlobal-8                 150547687      172778045      +14.77%
BenchmarkNeedHalfTruncated-8      102471355      76564986       -25.28%
BenchmarkHaveTruncated-8          28758368       14277481       -50.35%
BenchmarkGlobalTruncated-8        151192913      106070136      -29.84%

benchmark                         old allocs     new allocs     delta
BenchmarkReplaceAll-8             555577         557554         +0.36%
BenchmarkUpdateOneChanged-8       1135           587            -48.28%
BenchmarkUpdateOneUnchanged-8     1135           587            -48.28%
BenchmarkNeedHalf-8               374780         374775         -0.00%
BenchmarkHave-8                   151992         152085         +0.06%
BenchmarkGlobal-8                 530033         530135         +0.02%
BenchmarkNeedHalfTruncated-8      374699         22160          -94.09%
BenchmarkHaveTruncated-8          151834         4904           -96.77%
BenchmarkGlobalTruncated-8        530037         30536          -94.24%

benchmark                         old bytes      new bytes      delta
BenchmarkReplaceAll-8             1765116216     1765305376     +0.01%
BenchmarkUpdateOneChanged-8       135085         93043          -31.12%
BenchmarkUpdateOneUnchanged-8     134976         92928          -31.15%
BenchmarkNeedHalf-8               44758752       44751791       -0.02%
BenchmarkHave-8                   11845052       11967172       +1.03%
BenchmarkGlobal-8                 80431136       80431065       -0.00%
BenchmarkNeedHalfTruncated-8      46526459       18243543       -60.79%
BenchmarkHaveTruncated-8          11348357       418998         -96.31%
BenchmarkGlobalTruncated-8        80977672       43116991       -46.75%
2015-10-21 23:49:10 +02:00
Jakob Borg
8c26fe44c3 Actually run protocol tests faster with -short (on Go 1.5...) 2015-10-21 14:45:18 +02:00
Jakob Borg
8c7d9f3dd2 Protocol tests should run faster with -short 2015-10-21 14:35:59 +02:00
Audrius Butkevicius
9f4cd7716e Add more information about the folders to ClusterConfig 2015-10-17 09:46:46 +01:00
Audrius Butkevicius
a323d85d32 Add more information about the device to ClusterConfig 2015-10-16 19:40:12 +01:00
Audrius Butkevicius
491a33de0b Move device name into the protocol messages 2015-10-16 19:40:12 +01:00
Jakob Borg
76af9ba53d Implement facility based logger, debugging via REST API
This implements a new debug/trace infrastructure based on a slightly
hacked up logger. Instead of the traditional "if debug { ... }" I've
rewritten the logger to have no-op Debugln and Debugf, unless debugging
has been enabled for a given "facility". The "facility" is just a
string, typically a package name.

This will be slightly slower than before; but not that much as it's
mostly a function call that returns immediately. For the cases where it
matters (the Debugln takes a hex.Dump() of something for example, and
it's not in a very occasional "if err != nil" branch) there is an
l.ShouldDebug(facility) that is fast enough to be used like the old "if
debug".

The point of all this is that we can now toggle debugging for the
various packages on and off at runtime. There's a new method
/rest/system/debug that can be POSTed a set of facilities to enable and
disable debug for, or GET from to get a list of facilities with
descriptions and their current debug status.

Similarly a /rest/system/log?since=... can grab the latest log entries,
up to 250 of them (hardcoded constant in main.go) plus the initial few.

Not implemented in this commit (but planned) is a simple debug GUI
available on /debug that shows the current log in an easily pasteable
format and has checkboxes to enable the various debug facilities.

The debug instructions to a user then becomes "visit this URL, check
these boxes, reproduce your problem, copy and paste the log". The actual
log viewer on the hypothetical /debug URL can poll regularly for new log
entries and this bypass the 250 line limit.

The existing STTRACE=foo variable is still obeyed and just sets the
start state of the system.
2015-10-03 18:09:53 +02:00
Jakob Borg
e0a16e08dd Relax folder label restrictions 2015-09-25 13:45:58 +02:00
Jakob Borg
fbf911cf7e Unbreak comments 2015-09-22 21:51:05 +02:00
Jakob Borg
7fdfa81fb8 Fix vet and lint complaints 2015-09-22 20:34:24 +02:00
Jakob Borg
4316992d95 Add 'lib/protocol/' from commit 'f91191218b192ace841c878f161832d19c09145a'
git-subtree-dir: lib/protocol
git-subtree-mainline: 5ecb8bdd8a
git-subtree-split: f91191218b
2015-09-22 19:34:29 +02:00