octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-11-23 05:12:10 +00:00
Code Issues Releases Wiki Activity
252 Commits 7 Branches 33 Tags 66 MiB
3daeee8e2c
Commit Graph

13 Commits

Author SHA1 Message Date
Alexander Neumann
3daeee8e2c Make chunker resettable 2015-02-09 23:37:33 +01:00
Alexander Neumann
bda33e612c Implement streaming chunker using io.Reader 2015-02-09 00:28:54 +01:00
Alexander Neumann
a5c33d80d8 Chunker: remove pool, buf and make bufsize an option 2015-02-08 19:32:12 +01:00
Alexander Neumann
bdcdcdea7d Optimize chunker 
Skip MinSize bytes at the beginning of each chunk. This increases
throughput about 30% percent.

Before:
    $ go test -v -bench . ./chunker
    === RUN TestChunker
    --- PASS: TestChunker (0.60s)
    === RUN TestChunkerReuse
    --- PASS: TestChunkerReuse (2.18s)
    PASS
    BenchmarkChunker          10     111229642 ns/op      94.27 MB/s
    --- BENCH: BenchmarkChunker
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
    ok      github.com/restic/restic/chunker    4.120s

After:
    $ go test -v -bench . ./chunker
    === RUN TestChunker
    --- PASS: TestChunker (0.48s)
    === RUN TestChunkerReuse
    --- PASS: TestChunkerReuse (1.75s)
    PASS
    BenchmarkChunker          20      81468596 ns/op     128.71 MB/s
    --- BENCH: BenchmarkChunker
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
    ok      github.com/restic/restic/chunker    4.061s
 2015-01-14 16:33:41 +01:00
Alexander Neumann
e2fea0d088 Rename khepri -> restic 2014-12-05 21:45:49 +01:00
Alexander Neumann
9dc0bf6378 Fix reusing chunker 2014-12-03 23:20:45 +01:00
Alexander Neumann
f95788ed90 Limit memory usage, add several sync.Pool 2014-11-23 21:26:20 +01:00
Alexander Neumann
d594cd89b7 Chunker: remove unneeded return 2014-11-19 22:56:52 +01:00
Alexander Neumann
83ea81d8c3 Fix special case with null byte input 2014-10-03 21:44:39 +02:00
Alexander Neumann
878a96e761 Fix missing byte 2014-09-18 22:34:08 +02:00
Alexander Neumann
1c03bd7f7b Clarify behaviour of Next() 2014-09-18 22:22:44 +02:00
Alexander Neumann
8e24a8b811 Add more documentation and error handling 2014-09-18 22:20:12 +02:00
Alexander Neumann
18131f88bf Add code for content defined chunking (CDC) 2014-09-18 21:10:20 +02:00