octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-11-23 05:12:10 +00:00
Code Issues Releases Wiki Activity
395 Commits 7 Branches 33 Tags 66 MiB
367cc75970
Commit Graph

17 Commits

Author SHA1 Message Date
Alexander Neumann
367cc75970 chunker: Refactor to take polynomial on creation 2015-04-05 22:52:55 +02:00
Alexander Neumann
3cdf3a25b9 chunker: Require a random irreducible polynomial 
This also implements the necessary polynomial arithmetics in F_2[X].
 2015-04-05 22:52:43 +02:00
Alexander Neumann
59cfc4bd03 Slightly improve performance for chunker 
benchcmp output:

benchmark                      old ns/op     new ns/op     delta
BenchmarkChunkerWithSHA256     151752169     148072829     -2.42%
BenchmarkChunkerWithMD5        97166433      93357090      -3.92%
BenchmarkChunker               77265146      73642723      -4.69%

benchmark                      old MB/s     new MB/s     speedup
BenchmarkChunkerWithSHA256     69.10        70.81        1.02x
BenchmarkChunkerWithMD5        107.92       112.32       1.04x
BenchmarkChunker               135.71       142.39       1.05x

benchmark                      old allocs     new allocs     delta
BenchmarkChunkerWithSHA256     15             13             -13.33%
BenchmarkChunkerWithMD5        14             12             -14.29%
BenchmarkChunker               8              6              -25.00%

benchmark                      old bytes     new bytes     delta
BenchmarkChunkerWithSHA256     262924        26821         -89.80%
BenchmarkChunkerWithMD5        262774        13554         -94.84%
BenchmarkChunker               262678        13458         -94.88%
 2015-02-13 20:03:40 +01:00
Alexander Neumann
3bf66d24fa chunker: Add test, only pass hash.Hash to New() 2015-02-11 13:12:36 +01:00
Alexander Neumann
3daeee8e2c Make chunker resettable 2015-02-09 23:37:33 +01:00
Alexander Neumann
bda33e612c Implement streaming chunker using io.Reader 2015-02-09 00:28:54 +01:00
Alexander Neumann
a5c33d80d8 Chunker: remove pool, buf and make bufsize an option 2015-02-08 19:32:12 +01:00
Alexander Neumann
bdcdcdea7d Optimize chunker 
Skip MinSize bytes at the beginning of each chunk. This increases
throughput about 30% percent.

Before:
    $ go test -v -bench . ./chunker
    === RUN TestChunker
    --- PASS: TestChunker (0.60s)
    === RUN TestChunkerReuse
    --- PASS: TestChunkerReuse (2.18s)
    PASS
    BenchmarkChunker          10     111229642 ns/op      94.27 MB/s
    --- BENCH: BenchmarkChunker
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
    ok      github.com/restic/restic/chunker    4.120s

After:
    $ go test -v -bench . ./chunker
    === RUN TestChunker
    --- PASS: TestChunker (0.48s)
    === RUN TestChunkerReuse
    --- PASS: TestChunkerReuse (1.75s)
    PASS
    BenchmarkChunker          20      81468596 ns/op     128.71 MB/s
    --- BENCH: BenchmarkChunker
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
    ok      github.com/restic/restic/chunker    4.061s
 2015-01-14 16:33:41 +01:00
Alexander Neumann
e2fea0d088 Rename khepri -> restic 2014-12-05 21:45:49 +01:00
Alexander Neumann
9dc0bf6378 Fix reusing chunker 2014-12-03 23:20:45 +01:00
Alexander Neumann
f95788ed90 Limit memory usage, add several sync.Pool 2014-11-23 21:26:20 +01:00
Alexander Neumann
d594cd89b7 Chunker: remove unneeded return 2014-11-19 22:56:52 +01:00
Alexander Neumann
83ea81d8c3 Fix special case with null byte input 2014-10-03 21:44:39 +02:00
Alexander Neumann
878a96e761 Fix missing byte 2014-09-18 22:34:08 +02:00
Alexander Neumann
1c03bd7f7b Clarify behaviour of Next() 2014-09-18 22:22:44 +02:00
Alexander Neumann
8e24a8b811 Add more documentation and error handling 2014-09-18 22:20:12 +02:00
Alexander Neumann
18131f88bf Add code for content defined chunking (CDC) 2014-09-18 21:10:20 +02:00