c1ec9a8826
This reduces allocations, in number and in size, while getting extended attributes. This is mostly noticable when there is a large number of new files to scan and we're running with the default scanProgressInterval -- then a queue of files is built in-memory, and this queue includes extended attributes as part of file metadata. (Arguable it shouldn't, but that's a more difficult and involved change.) With 1M files to scan, each with one extended attribute, current peak memory usage looks like this: Showing nodes accounting for 1425.30MB, 98.19% of 1451.64MB total Dropped 1435 nodes (cum <= 7.26MB) Showing top 10 nodes out of 54 flat flat% sum% cum cum% 976.56MB 67.27% 67.27% 976.56MB 67.27% github.com/syncthing/syncthing/lib/fs.getXattr 305.44MB 21.04% 88.31% 305.44MB 21.04% github.com/syncthing/syncthing/lib/scanner.(*walker).walk.func1 45.78MB 3.15% 91.47% 1045.23MB 72.00% github.com/syncthing/syncthing/lib/fs.(*BasicFilesystem).GetXattr 22.89MB 1.58% 93.04% 22.89MB 1.58% github.com/syncthing/syncthing/lib/fs.listXattr 22.89MB 1.58% 94.62% 22.89MB 1.58% github.com/syncthing/syncthing/lib/protocol.(*PlatformData).SetXattrs 16MB 1.10% 95.72% 16.01MB 1.10% github.com/syndtr/goleveldb/leveldb/memdb.New After the change, it's this: Showing nodes accounting for 502.32MB, 95.70% of 524.88MB total Dropped 1400 nodes (cum <= 2.62MB) Showing top 10 nodes out of 91 flat flat% sum% cum cum% 305.43MB 58.19% 58.19% 305.43MB 58.19% github.com/syncthing/syncthing/lib/scanner.(*walker).walk.func1 45.79MB 8.72% 66.91% 68.68MB 13.09% github.com/syncthing/syncthing/lib/fs.(*BasicFilesystem).GetXattr 32MB 6.10% 73.01% 32.01MB 6.10% github.com/syndtr/goleveldb/leveldb/memdb.New 22.89MB 4.36% 77.37% 22.89MB 4.36% github.com/syncthing/syncthing/lib/fs.listXattr 22.89MB 4.36% 81.73% 22.89MB 4.36% github.com/syncthing/syncthing/lib/protocol.(*PlatformData).SetXattrs 15.35MB 2.92% 84.66% 15.36MB 2.93% github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).Get 15.28MB 2.91% 87.57% 15.28MB 2.91% strings.(*Builder).grow (The usage for xattrs is reduced from 976 MB to 68 MB) |
||
|---|---|---|
| .github | ||
| assets | ||
| cmd | ||
| etc | ||
| gui | ||
| lib | ||
| man | ||
| meta | ||
| next-gen-gui | ||
| proto | ||
| script | ||
| test | ||
| .codecov.yml | ||
| .deepsource.toml | ||
| .gitattributes | ||
| .gitignore | ||
| .golangci.yml | ||
| .yamlfmt | ||
| AUTHORS | ||
| build.go | ||
| build.ps1 | ||
| build.sh | ||
| CONDUCT.md | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| Dockerfile.builder | ||
| Dockerfile.stcrashreceiver | ||
| Dockerfile.stdiscosrv | ||
| Dockerfile.strelaypoolsrv | ||
| Dockerfile.strelaysrv | ||
| Dockerfile.stupgrades | ||
| Dockerfile.ursrv | ||
| go.mod | ||
| go.sum | ||
| GOALS.md | ||
| LICENSE | ||
| README-Docker.md | ||
| README.md | ||
| tools.go | ||
Goals
Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals below. The goals are listed in order of importance, the most important ones first. This is the summary version of the goal list - for more commentary, see the full Goals document.
Syncthing should be:
-
Safe From Data Loss
Protecting the user's data is paramount. We take every reasonable precaution to avoid corrupting the user's files.
-
Secure Against Attackers
Again, protecting the user's data is paramount. Regardless of our other goals, we must never allow the user's data to be susceptible to eavesdropping or modification by unauthorized parties.
-
Easy to Use
Syncthing should be approachable, understandable, and inclusive.
-
Automatic
User interaction should be required only when absolutely necessary.
-
Universally Available
Syncthing should run on every common computer. We are mindful that the latest technology is not always available to every individual.
-
For Individuals
Syncthing is primarily about empowering the individual user with safe, secure, and easy to use file synchronization.
-
Everything Else
There are many things we care about that don't make it on to the list. It is fine to optimize for these values, as long as they are not in conflict with the stated goals above.
Getting Started
Take a look at the getting started guide.
There are a few examples for keeping Syncthing running in the background on your system in the etc directory. There are also several GUI implementations for Windows, Mac, and Linux.
Docker
To run Syncthing in Docker, see the Docker README.
Vote on features/bugs
We'd like to encourage you to vote on issues that matter to you. This helps the team understand what are the biggest pain points for our users, and could potentially influence what is being worked on next.
Getting in Touch
The first and best point of contact is the Forum. If you've found something that is clearly a bug, feel free to report it in the GitHub issue tracker.
If you believe that you’ve found a Syncthing-related security vulnerability, please report it by emailing security@syncthing.net. Do not report it in the Forum or issue tracker.
Building
Building Syncthing from source is easy. After extracting the source bundle from
a release or checking out git, you just need to run go run build.go and the
binaries are created in ./bin. There's a guide with more details on the
build process.
Signed Releases
As of v0.10.15 and onwards, release binaries are GPG signed with the key D26E6ED000654A3E, available from https://syncthing.net/security.html and most key servers.
There is also a built-in automatic upgrade mechanism (disabled in some distribution channels) which uses a compiled in ECDSA signature. macOS binaries are also properly code signed.
Documentation
Please see the Syncthing documentation site [source].
All code is licensed under the MPLv2 License.