Alexander Neumann
|
ea22b2dfb1
|
chunker: move buffer pool to chunker package
|
2015-05-05 00:51:47 +02:00 |
|
Alexander Neumann
|
64a290c8db
|
chunker: Add buffer to New()
|
2015-05-04 23:25:07 +02:00 |
|
Florian Weingarten
|
282c4379db
|
Remove chunker.Reset() and chunker pool
|
2015-05-04 16:49:49 -04:00 |
|
Florian Weingarten
|
ebae478631
|
Prefer mixedCaps over underscore
|
2015-04-24 19:39:32 -04:00 |
|
Alexander Neumann
|
4f4f3c421a
|
Use package "restic/test"
|
2015-04-09 21:15:48 +02:00 |
|
Alexander Neumann
|
3d058929e3
|
Fix chunker tests
|
2015-04-06 00:31:31 +02:00 |
|
Alexander Neumann
|
c969a42e8d
|
chunker: Allow resetting polynomial
|
2015-04-05 22:53:06 +02:00 |
|
Alexander Neumann
|
25e3ac40ee
|
chunker: Cache computations
|
2015-04-05 22:53:06 +02:00 |
|
Alexander Neumann
|
367cc75970
|
chunker: Refactor to take polynomial on creation
|
2015-04-05 22:52:55 +02:00 |
|
Alexander Neumann
|
59cfc4bd03
|
Slightly improve performance for chunker
benchcmp output:
benchmark old ns/op new ns/op delta
BenchmarkChunkerWithSHA256 151752169 148072829 -2.42%
BenchmarkChunkerWithMD5 97166433 93357090 -3.92%
BenchmarkChunker 77265146 73642723 -4.69%
benchmark old MB/s new MB/s speedup
BenchmarkChunkerWithSHA256 69.10 70.81 1.02x
BenchmarkChunkerWithMD5 107.92 112.32 1.04x
BenchmarkChunker 135.71 142.39 1.05x
benchmark old allocs new allocs delta
BenchmarkChunkerWithSHA256 15 13 -13.33%
BenchmarkChunkerWithMD5 14 12 -14.29%
BenchmarkChunker 8 6 -25.00%
benchmark old bytes new bytes delta
BenchmarkChunkerWithSHA256 262924 26821 -89.80%
BenchmarkChunkerWithMD5 262774 13554 -94.84%
BenchmarkChunker 262678 13458 -94.88%
|
2015-02-13 20:03:40 +01:00 |
|
Alexander Neumann
|
3bf66d24fa
|
chunker: Add test, only pass hash.Hash to New()
|
2015-02-11 13:12:36 +01:00 |
|
Alexander Neumann
|
3daeee8e2c
|
Make chunker resettable
|
2015-02-09 23:37:33 +01:00 |
|
Alexander Neumann
|
bda33e612c
|
Implement streaming chunker using io.Reader
|
2015-02-09 00:28:54 +01:00 |
|
Alexander Neumann
|
a5c33d80d8
|
Chunker: remove pool, buf and make bufsize an option
|
2015-02-08 19:32:12 +01:00 |
|
Alexander Neumann
|
bdcdcdea7d
|
Optimize chunker
Skip MinSize bytes at the beginning of each chunk. This increases
throughput about 30% percent.
Before:
$ go test -v -bench . ./chunker
=== RUN TestChunker
--- PASS: TestChunker (0.60s)
=== RUN TestChunkerReuse
--- PASS: TestChunkerReuse (2.18s)
PASS
BenchmarkChunker 10 111229642 ns/op 94.27 MB/s
--- BENCH: BenchmarkChunker
chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
ok github.com/restic/restic/chunker 4.120s
After:
$ go test -v -bench . ./chunker
=== RUN TestChunker
--- PASS: TestChunker (0.48s)
=== RUN TestChunkerReuse
--- PASS: TestChunkerReuse (1.75s)
PASS
BenchmarkChunker 20 81468596 ns/op 128.71 MB/s
--- BENCH: BenchmarkChunker
chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
ok github.com/restic/restic/chunker 4.061s
|
2015-01-14 16:33:41 +01:00 |
|
Alexander Neumann
|
e2fea0d088
|
Rename khepri -> restic
|
2014-12-05 21:45:49 +01:00 |
|
Alexander Neumann
|
9dc0bf6378
|
Fix reusing chunker
|
2014-12-03 23:20:45 +01:00 |
|
Alexander Neumann
|
f95788ed90
|
Limit memory usage, add several sync.Pool
|
2014-11-23 21:26:20 +01:00 |
|
Alexander Neumann
|
83ea81d8c3
|
Fix special case with null byte input
|
2014-10-03 21:44:39 +02:00 |
|
Alexander Neumann
|
878a96e761
|
Fix missing byte
|
2014-09-18 22:34:08 +02:00 |
|
Alexander Neumann
|
8e24a8b811
|
Add more documentation and error handling
|
2014-09-18 22:20:12 +02:00 |
|
Alexander Neumann
|
18131f88bf
|
Add code for content defined chunking (CDC)
|
2014-09-18 21:10:20 +02:00 |
|