Simon Frei
158559023e
lib/db: Fix sequence updating for remote invalid items ( #5420 )
...
* lib/db: Fix sequence updating for remote invalid items
* fix for the unit test introduced in the previous commit
* lib/db: Polish blockmap
2019-01-04 20:19:10 +01:00
Jakob Borg
b50d57b7fd
lib/db: Refactor: use a Lowlevel type underneath Instance (ref #5198 ) ( #5212 )
...
This adds a thin type that holds the state associated with the
leveldb.DB, leaving the huge Instance type more or less stateless. Also
moves some keying stuff into the DB package so that other packages need
not know the keying specifics.
(This does not, yet, fix the cmd/stindex program, in order to keep the
diff size down. Hence the keying constants are still exported.)
2018-10-10 11:34:24 +02:00
Audrius Butkevicius
ef0dcea6a4
lib/model: Verify request content against weak (and possibly strong) hash ( #4767 )
2018-05-05 10:24:44 +02:00
Jakob Borg
e9f05d138f
lib/db, lib/model: Always use reasonable sized batches ( fixes #2250 , fixes #4112 )
...
Harmonize how we use batches in the model, using ProtoSize() to judge
the actual weight of the entire batch instead of estimating. Use smaller
batches in the block map - I think we might have though that batch.Len()
in the leveldb was the batch size in bytes, but it's actually number of
operations.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4114
2017-04-22 14:23:33 +00:00
Jakob Borg
f7fc0c1d3e
all: Update license url to https (ref #3976 )
2017-02-09 08:04:16 +01:00
Jakob Borg
47f22ff3e5
build: Enable gometalinter "unconvert" check
2016-12-21 14:53:45 +01:00
Jakob Borg
ac190b2e39
Change DB label format (index folders, devices)
2016-01-03 19:32:40 +01:00
Jakob Borg
2f12d41d9d
Don't dirty blockmap key between lookups ( fixes #2455 )
2015-11-13 15:44:30 +01:00
Jakob Borg
2a4fc28318
We should pass around db.Instance instead of leveldb.DB
...
We're going to need the db.Instance to keep some state, and for that to
work we need the same one passed around everywhere. Hence this moves the
leveldb-specific file opening stuff into the db package and exports the
dbInstance type.
2015-10-31 12:35:30 +01:00
Jakob Borg
918ef4dff8
Use batches in blockmap, speeds up and reduces memory usage on large Replace and Update ops
...
benchmark old ns/op new ns/op delta
BenchmarkReplaceAll-8 2880834572 1868198122 -35.15%
BenchmarkUpdateOneChanged-8 236596 231852 -2.01%
BenchmarkUpdateOneUnchanged-8 227326 230624 +1.45%
BenchmarkNeedHalf-8 105151538 104601744 -0.52%
BenchmarkHave-8 28827492 29102480 +0.95%
BenchmarkGlobal-8 150768724 150547687 -0.15%
BenchmarkNeedHalfTruncated-8 104434216 102471355 -1.88%
BenchmarkHaveTruncated-8 27860093 28758368 +3.22%
BenchmarkGlobalTruncated-8 149972888 151192913 +0.81%
benchmark old allocs new allocs delta
BenchmarkReplaceAll-8 555451 555577 +0.02%
BenchmarkUpdateOneChanged-8 1135 1135 +0.00%
BenchmarkUpdateOneUnchanged-8 1135 1135 +0.00%
BenchmarkNeedHalf-8 374779 374780 +0.00%
BenchmarkHave-8 151996 151992 -0.00%
BenchmarkGlobal-8 530066 530033 -0.01%
BenchmarkNeedHalfTruncated-8 374702 374699 -0.00%
BenchmarkHaveTruncated-8 151834 151834 +0.00%
BenchmarkGlobalTruncated-8 530049 530037 -0.00%
benchmark old bytes new bytes delta
BenchmarkReplaceAll-8 5018351912 1765116216 -64.83%
BenchmarkUpdateOneChanged-8 135085 135085 +0.00%
BenchmarkUpdateOneUnchanged-8 134976 134976 +0.00%
BenchmarkNeedHalf-8 44769400 44758752 -0.02%
BenchmarkHave-8 11930612 11845052 -0.72%
BenchmarkGlobal-8 81523668 80431136 -1.34%
BenchmarkNeedHalfTruncated-8 46692342 46526459 -0.36%
BenchmarkHaveTruncated-8 11348357 11348357 +0.00%
BenchmarkGlobalTruncated-8 81843956 80977672 -1.06%
2015-10-21 23:05:23 +02:00
Jakob Borg
0d9a04c713
Reuse blockkey, speeds up large Update and Replace calls
...
benchmark old ns/op new ns/op delta
BenchmarkReplaceAll-8 2866418930 2880834572 +0.50%
BenchmarkUpdateOneChanged-8 226635 236596 +4.40%
BenchmarkUpdateOneUnchanged-8 229090 227326 -0.77%
BenchmarkNeedHalf-8 104483393 105151538 +0.64%
BenchmarkHave-8 29288220 28827492 -1.57%
BenchmarkGlobal-8 159269126 150768724 -5.34%
BenchmarkNeedHalfTruncated-8 108235000 104434216 -3.51%
BenchmarkHaveTruncated-8 28945489 27860093 -3.75%
BenchmarkGlobalTruncated-8 149355833 149972888 +0.41%
benchmark old allocs new allocs delta
BenchmarkReplaceAll-8 1054944 555451 -47.35%
BenchmarkUpdateOneChanged-8 1135 1135 +0.00%
BenchmarkUpdateOneUnchanged-8 1135 1135 +0.00%
BenchmarkNeedHalf-8 374777 374779 +0.00%
BenchmarkHave-8 151995 151996 +0.00%
BenchmarkGlobal-8 530063 530066 +0.00%
BenchmarkNeedHalfTruncated-8 374699 374702 +0.00%
BenchmarkHaveTruncated-8 151834 151834 +0.00%
BenchmarkGlobalTruncated-8 530021 530049 +0.01%
benchmark old bytes new bytes delta
BenchmarkReplaceAll-8 5074297112 5018351912 -1.10%
BenchmarkUpdateOneChanged-8 135097 135085 -0.01%
BenchmarkUpdateOneUnchanged-8 134976 134976 +0.00%
BenchmarkNeedHalf-8 44759436 44769400 +0.02%
BenchmarkHave-8 11911138 11930612 +0.16%
BenchmarkGlobal-8 81609867 81523668 -0.11%
BenchmarkNeedHalfTruncated-8 46588024 46692342 +0.22%
BenchmarkHaveTruncated-8 11348354 11348357 +0.00%
BenchmarkGlobalTruncated-8 79485168 81843956 +2.97%
2015-10-21 23:05:23 +02:00
Jakob Borg
4581c57478
Fix import paths
2015-09-22 19:38:46 +02:00
Jakob Borg
4a9997e449
lib/db need not depend on lib/config
2015-09-04 12:01:00 +02:00
Jakob Borg
7705a6c1f1
mv internal lib
2015-08-09 09:35:26 +02:00
