Commit Graph

7583 Commits

Author SHA1 Message Date
Jakob Borg
cfa834177b
build(deps): update dependencies (#9751) 2024-10-02 12:53:14 +00:00
tomasz1986
c454fc8baa
chore(build): use conventional commit title in update script (#9747) 2024-09-30 15:59:14 -05:00
Jakob Borg
dbe7fa9155
Merge branch 'infrastructure'
* infrastructure:
  feat(ursrv): new metrics based approach
2024-09-30 14:17:02 -05:00
Jakob Borg
4d842f7d3b
feat(ursrv): new metrics based approach 2024-09-30 14:16:27 -05:00
Syncthing Release Automation
0e68221c91 gui, man, authors: Update docs, translations, and contributors 2024-09-30 03:48:06 +00:00
Jakob Borg
19f63c7ea3
chore(model): improve tracking sentPrevSeq for index debugging (#9740) 2024-09-29 22:18:24 +00:00
Emil Lundberg
fb939ec496
fix(model): prevent division by zero in numHashers (#9744)
This should prevent the panic that occurred in this test run:
https://github.com/syncthing/syncthing/actions/runs/11095876010/job/30825046810

```
2024-09-29T21:01:53.5425372Z === RUN   TestIssue4357
2024-09-29T21:01:53.5505943Z panic: runtime error: integer divide by zero [recovered]
2024-09-29T21:01:53.5512200Z 	panic: runtime error: integer divide by zero
2024-09-29T21:01:53.5516633Z
2024-09-29T21:01:53.5523018Z goroutine 2655 [running]:
2024-09-29T21:01:53.5524157Z github.com/thejerf/suture/v4.(*Supervisor).runService.func2.2()
2024-09-29T21:01:53.5527176Z 	/home/runner/go/pkg/mod/github.com/thejerf/suture/v4@v4.0.5/supervisor.go:563 +0xd0
2024-09-29T21:01:53.5530556Z panic({0x1080d20?, 0x1851290?})
2024-09-29T21:01:53.5564723Z 	/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.1.linux-amd64/src/runtime/panic.go:785 +0x132
2024-09-29T21:01:53.5566616Z github.com/syncthing/syncthing/lib/model.(*model).numHashers(0xc0006f6180, {0x117dc1a, 0x7})
2024-09-29T21:01:53.5568061Z 	/home/runner/work/syncthing/syncthing/lib/model/model.go:2581 +0x210
2024-09-29T21:01:53.5569912Z github.com/syncthing/syncthing/lib/model.(*folder).scanSubdirsChangedAndNew(0xc00c38c808, {0x0, 0x0, 0x0}, 0xc0003fc060)
2024-09-29T21:01:53.5571612Z 	/home/runner/work/syncthing/syncthing/lib/model/folder.go:653 +0x250
2024-09-29T21:01:53.5573010Z github.com/syncthing/syncthing/lib/model.(*folder).scanSubdirs(0xc00c38c808, {0x0, 0x0, 0x0})
2024-09-29T21:01:53.5574447Z 	/home/runner/work/syncthing/syncthing/lib/model/folder.go:512 +0xd0f
2024-09-29T21:01:53.5576011Z github.com/syncthing/syncthing/lib/model.(*folder).scanTimerFired(0xc00c38c808)
2024-09-29T21:01:53.5577367Z 	/home/runner/work/syncthing/syncthing/lib/model/folder.go:916 +0x46
2024-09-29T21:01:53.5579010Z github.com/syncthing/syncthing/lib/model.(*folder).Serve(0xc00c38c808, {0x1307650, 0xc0006a0910})
2024-09-29T21:01:53.5580428Z 	/home/runner/work/syncthing/syncthing/lib/model/folder.go:205 +0xd7e
2024-09-29T21:01:53.5581624Z github.com/thejerf/suture/v4.(*Supervisor).runService.func2()
2024-09-29T21:01:53.5582978Z 	/home/runner/go/pkg/mod/github.com/thejerf/suture/v4@v4.0.5/supervisor.go:567 +0x249
2024-09-29T21:01:53.5584400Z created by github.com/thejerf/suture/v4.(*Supervisor).runService in goroutine 2651
2024-09-29T21:01:53.5585872Z 	/home/runner/go/pkg/mod/github.com/thejerf/suture/v4@v4.0.5/supervisor.go:541 +0x32a
2024-09-29T21:01:53.5661413Z FAIL	github.com/syncthing/syncthing/lib/model	5.510s
```

### Testing

I have not been able to reproduce the panic throughout a few minutes of
continuously running the test without this fix, but judging by the
traceback it seems to only happen if the test happens to delete the
folder from config at the same time `scanTimerFired` triggers.
2024-09-30 00:01:52 +02:00
Jakob Borg
39df3173d4
chore(model): log sequence anomaly when update appears not to "take" (#9741)
I hope this doesn't fire, but 👻  I'm Seeing Things I Can't Explain 👻
2024-09-29 15:04:06 +00:00
maxice8
429672e0b4
docs(docker): add healthcheck to docker-compose (#9742)
### Purpose

Syncthing had a healthcheck API for a while, and the example Dockerfile
for it has it in the form of:

HEALTHCHECK --interval=1m --timeout=10s \
CMD curl -fkLsS -m 2 127.0.0.1:8384/rest/noauth/health | grep -o
--color=never OK || exit 1

Let's add it to the docker-compose as well

### Testing

I use this docker-compose.yml file to deploy via ansible (using
community.docker.docker_compose_v2) to my machine with success, using
`wait: true` in ansible for it to use `docker compose up --wait`.

```yml
- name: Enable syncthing docker
  community.docker.docker_compose_v2:
    project_src: /srv/syncthing
    wait: true
    wait_timeout: 90
```
2024-09-29 09:53:13 -05:00
Simon Frei
605fd6d726
fix(ignore): ensure normalization of patterns and paths match (fixes #9597) (#9717)
In ignores, normalize the input when parsing it.
When scanning, normalize earlier such that the path is already
normalized when checking ignores. This requires splitting normalization
of the string from normalization of the file, as we don't want to
attempt the latter if the file is ignored.

Closes #9598

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2024-09-28 17:16:44 +02:00
Jakob Borg
3c476542d2
fix(ur): actually send usage report directly when enabled (#9736)
There was a bug that the unique ID was not set when reporting was
enabled, and thus the reports where rejected by the server. The unique
ID got set only on startup, so next time Syncthing restarted.

This makes sure to set the unique ID when blank.
2024-09-28 17:02:05 +02:00
Jakob Borg
31874f3ebb
chore(model): remove GUI/log warning on sequence anomaly (#9738)
I can see already in our Sentry data that there are a fair amount of
these warnings, and mostly the shape of it. Asking users to report them
will likely cause a lot of reporting effort to fairly little additional
value. We can do that when/if we have something more targeted to ask
for.
2024-09-28 16:38:07 +02:00
Jakob Borg
77942747db
Merge branch 'infrastructure'
* infrastructure:
  feat(stupgrades): filter returned releases per compatibility
2024-09-26 10:25:02 +02:00
Jakob Borg
fe01b396ba
feat(stupgrades): filter returned releases per compatibility 2024-09-26 10:22:23 +02:00
Jakob Borg
3583949706
refactor(upgrade): rename insecureGet which is no longer insecure (#9735) 2024-09-25 15:50:22 +00:00
Jakob Borg
23fc22ebc5
chore: add more advanced policy configuration (#9726)
This codifies a review policy which is closer to what I always
envisioned, but which isn't expressible using the normal checks in the
GitHub GUI. It would move the commit approval check from GitHub into the
policy-bot check which is already present to enforce the
conventional-commits standard. Approvals in general would still work the
same -- it's just that the bot picks it up and toggles the status
accordingly. From a GitHub side when this is enabled we'd remove the
requires-review check from there and let the bot decide that part. We
would still require builds and tests to pass of course.

There are a couple of relexations from the current policy, details in
the code but briefly:

- Changes to translations or dependencies by a trusted person don't
require review
- Trivial changes by a trusted person, explicitly marked as such, don't
require review

This enables less bureaucracy for things like adding new translated
languages and updating dependencies, and enables the trivial-change
workflow to a larger audience than, like, me, who could always just
bypass the rules by way of being admin.
2024-09-25 17:41:56 +02:00
Jakob Borg
cba163a1fd
chore: enable TLS client cache for HTTPS where appropriate (#9721)
https://forum.syncthing.net/t/infrastructure-report-discovery-stuff/22819/4
2024-09-24 08:55:04 +02:00
Jakob Borg
a8e2c8edb6
fix(connections): announce PtP links again (fixes #9730) (#9731) 2024-09-23 14:32:19 +02:00
Syncthing Release Automation
3e501d9036 gui, man, authors: Update docs, translations, and contributors 2024-09-23 03:46:01 +00:00
bt90
9ca101756d
chore(ursrv): add Nix detection (#9729)
Classify the builder `nix@nix` as [Nix](https://nixos.org/)

![369684243-172cab09-df6f-449a-a638-1f0a0c080ab3](https://github.com/user-attachments/assets/37a6e0a5-bdcb-4b31-8b36-eaaa42423382)
2024-09-22 14:03:40 +02:00
bt90
a873d12c65
chore(ursrv): extend F-Droid detection (#9728)
Our f-droid apps are currently built using `vagrant@bookworm`:

![grafik](https://github.com/user-attachments/assets/172cab09-df6f-449a-a638-1f0a0c080ab3)
2024-09-22 13:48:38 +02:00
Emil Lundberg
8ff670c564
fix(gui): get version from header when not authenticated (#9724)
### Purpose

Since #8757, the Syncthing GUI now has an unauthenticated state. One
consequence of this is that `$scope.versionBase()` is not initialized
while unauthenticated, which causes the `docsURL` function to truncate
links to just `https://docs.syncthing.net`/, discarding the section
path. This currently affects at least the "Help > Introduction" link
reachable both while logged in and not. The issue is exacerbated in
https://github.com/syncthing/syncthing/pull/9175 where we sometimes want
to show additional contextual help links from the login page to
particular sections of the docs.

I don't think it's any worse to try to preserve the section path even
without an explicit version tag, than to fall back to just the host and
lose all context the link was attempting to provide.

### Testing

- On commit b1ed2802fb (before):
  - Open the GUI, set a username and log out.
- Open the "Help" drop-down. The "Introduction" item links to:
https://docs.syncthing.net/
  - Log in.
- Open the "Help" drop-down. The "Introduction" item links to:
https://docs.syncthing.net/v1.27.10/intro/gui
- On commit 44fef317800ce1d0795b4e2ebfbd5e9deda849ef (after):
  - Open the GUI, set a username and log out.
- Open the "Help" drop-down. The "Introduction" item links to:
https://docs.syncthing.net/intro/gui
  - Log in.
- Open the "Help" drop-down. The "Introduction" item links to:
https://docs.syncthing.net/v1.27.10/intro/gui

### Screenshots

This is a GUI change, but affecting only URLs in the markup with no
visual changes.


### Drawbacks

If a `docsURL` call generates a versionless link to a docs page that
doesn't exist on https://docs.syncthing.net - presumably because
Syncthing is not the latest version and links to a deleted page? - then
this will lead to GitHub's generic 404 page with no link to the
Syncthing docs root. Before, any versionless link would also be a
pathless link, leading to the Syncthing docs root instead of a 404 page.
2024-09-22 09:47:02 +02:00
Jakob Borg
b1ed2802fb
fix(connections): skip point-to-point interfaces when listing LANs (fixes #9719) (#9720)
Point-to-point interfaces are typically VPNs and similar which, for our
purposes, do not qualify as LANs.
2024-09-21 09:27:23 +02:00
Jakob Borg
b70cb580c8
build(deps): update all dependencies (#9723) 2024-09-21 09:25:27 +02:00
Sonu Kumar Saw
28be3ba788
chore(connections): lower log level from INFO to DEBUG for "already connected to this device" messages (fixes #9715) (#9722)
### Purpose

The primary aim of this change is to minimize log clutter in production
environments. There are many lines in the logs coming from an expected
race condition when two devices connect `already connected to this
device`. These messages do not indicate errors and can overwhelm the log
files with unnecessary noise.

By lowering the logging level, we enhance the usability of the logs,
making it easier for users and developers to identify actual issues
without being distracted

### Testing
1. Build syncthing locally
2. Start two Syncthing instances
```bash
./syncthing -no-browser -home=~/.config/syncthing1
./syncthing -no-browser -home=~/.config/syncthing2
```
3. Enable the DEBUG logs from UI for `connections` package
4. Connect the synching instances by adding remote devices from the UI
5. Observe the logs for the message `XXXX already connected to this
device`

### Screenshots


![image](https://github.com/user-attachments/assets/882ccb4c-d39d-463a-8f66-2aad97010700)

## Authorship

Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.
2024-09-21 07:19:58 +00:00
Jakob Borg
d4770ddc77
chore(cmd): clean up commands (#9705)
Move infrastructure related commands to under `cmd/infra` and
development stuff to `cmd/dev`. The default build command builds the
regular user facing binaries: syncthing, stdiscosrv, and strelaysrv.
2024-09-21 09:04:22 +02:00
Simon Frei
cbe1220680
chore(fs): put the caseFS as the outermost layer again (#9716)
Reasoning in comments. The main motivation is to avoid all the case
checks when walking the filesystem.

"again" as we already tried once, but it caused a major issue ragarding
mtimefs layer. The root of this problem has been fixed in the meantime
in ac8b3342a
2024-09-18 20:31:19 +02:00
André Colomb
0b95c5fa76
fix(meta): return read error in forbidden_words_test (#9706)
When reading a file fails, the error is currently swallowed / hidden.
Probably just a typo.
2024-09-18 19:12:24 +02:00
Jakob Borg
0343bca257
Merge branch 'infrastructure'
* infrastructure:
  chore(stdiscosrv): ensure incoming addresses are sorted and unique
  chore(stdiscosrv): use zero-allocation merge in the common case
  chore(stdiscosrv): properly clean out old addresses from memory
  chore(stdiscosrv): calculate IPv6 GUA
2024-09-16 09:33:15 +02:00
Syncthing Release Automation
878016db39 gui, man, authors: Update docs, translations, and contributors 2024-09-16 03:47:01 +00:00
Simon Frei
1f4fde9525
chore(protocol): prioritize closing a connection (#9711)
The read/write loops may keep going for a while on a closing connection
with lots of read/write activity, as it's random which select case is
chosen. And if the connection is slow or even broken, a single
read/write
can take a long time/until timeout. Add initial non-blocking selects
with only the cases relevant to closing, to prioritize those.
2024-09-15 21:13:56 +02:00
Jakob Borg
5b9d8a838f
chore(stdiscosrv): ensure incoming addresses are sorted and unique 2024-09-15 17:01:16 +02:00
Jakob Borg
8b19cb1e11
chore(stdiscosrv): use zero-allocation merge in the common case 2024-09-15 15:26:40 +02:00
Jakob Borg
ce1e259bb4
chore(stdiscosrv): properly clean out old addresses from memory 2024-09-15 14:20:59 +02:00
Jakob Borg
2238a288d9
fix(model): shut down index sender faster (#9704) 2024-09-15 11:37:49 +02:00
Jakob Borg
c8ee2a5cf6
chore(stdiscosrv): calculate IPv6 GUA 2024-09-15 10:48:16 +02:00
Ross Smith II
1704827d04
chore(gui): update HumanDuration.js (#9710)
Relevant changes:

ko: Use correct names for month and hour in Korean (465eaed)
Hide unit count if 2 in Arabic (f90d847)
2024-09-15 10:21:18 +02:00
Jakob Borg
a156e88eef
Merge branch 'infrastructure'
* infrastructure:
  chore(stdiscosrv): hide internal/undocumented flags
  chore(stdiscosrv): remove legacy replication
  chore(stdiscosrv): clean up s3 handling
  chore(stdiscosrv): less garbage in statistics
  chore(stdiscosrv): improve expire, logging
  chore(stdiscosrv): sched in loop
  chore(stdiscosrv): database writing logging
  chore(stdiscosrv): use order-preserving expire
  chore(stdiscosrv): simplify sorting
  chore(stdiscosrv): reduce allocations in cert handling
  chore(stdiscosrv): reduce unnecessary allocations in merge
  feat(stdiscosrv): enable HTTP profiler
  feat(discosrv): in-memory storage with S3 backing
  feat(stdiscosrv): make compression optional (and faster)
2024-09-13 08:50:24 +02:00
Jakob Borg
94d0195b63
chore(stdiscosrv): hide internal/undocumented flags 2024-09-13 08:49:13 +02:00
Jakob Borg
1616edcee3
chore(stdiscosrv): remove legacy replication 2024-09-13 08:49:13 +02:00
Jakob Borg
6505e123bb
chore(stdiscosrv): clean up s3 handling 2024-09-13 08:48:04 +02:00
Jakob Borg
63e4659282
chore(stdiscosrv): less garbage in statistics 2024-09-13 08:48:04 +02:00
Jakob Borg
f3f5557c8e
chore(stdiscosrv): improve expire, logging 2024-09-13 08:48:04 +02:00
Jakob Borg
b794726e1f
chore(stdiscosrv): sched in loop 2024-09-13 08:48:04 +02:00
Jakob Borg
3d59740a0a
chore(stdiscosrv): database writing logging 2024-09-13 08:48:04 +02:00
Jakob Borg
66fb65b01f
chore(stdiscosrv): use order-preserving expire 2024-09-13 08:48:04 +02:00
Jakob Borg
5c2fcbfd19
chore(stdiscosrv): simplify sorting 2024-09-13 08:48:03 +02:00
Jakob Borg
f9b72330a8
chore(stdiscosrv): reduce allocations in cert handling 2024-09-13 08:48:03 +02:00
Jakob Borg
822b6ac36b
chore(stdiscosrv): reduce unnecessary allocations in merge 2024-09-13 08:48:03 +02:00
Jakob Borg
77f7778292
feat(stdiscosrv): enable HTTP profiler 2024-09-13 08:48:03 +02:00