Alexander Neumann
|
65aae5f8c1
|
Disable preloading for now
|
2015-02-17 23:38:15 +01:00 |
|
Alexander Neumann
|
d3fe4a7f6d
|
Close Reader after read
|
2015-02-17 23:37:45 +01:00 |
|
Alexander Neumann
|
215fb982c6
|
Do not run archiver tests by default
|
2015-02-17 23:23:12 +01:00 |
|
Alexander Neumann
|
677a5038c5
|
Increase concurrency for loading blobs
|
2015-02-17 23:15:08 +01:00 |
|
Alexander Neumann
|
b6f25aa690
|
Remove Each(), add basic stats
|
2015-02-17 23:11:56 +01:00 |
|
Alexander Neumann
|
4bb7f2f2ed
|
Add Preloader for blobs
|
2015-02-17 22:39:44 +01:00 |
|
Alexander Neumann
|
c631b5c738
|
Add Len() functions for IDSet and Map
|
2015-02-17 22:39:16 +01:00 |
|
Alexander Neumann
|
4d1e7b9f24
|
Change parameters of LoadTree()
|
2015-02-17 22:37:34 +01:00 |
|
Alexander Neumann
|
5a16a66973
|
Add 'plaintext' parameter to key.Decrypt()
% benchcmp /tmp/{before,after}
benchmark old ns/op new ns/op delta
BenchmarkChunkEncrypt 261144414 260252046 -0.34%
BenchmarkChunkEncryptParallel 261201438 261267029 +0.03%
BenchmarkArchiveDirectory 0.00 0.00 +0.00%
BenchmarkEncryptWriter 88297245 87526529 -0.87%
BenchmarkEncrypt 87406446 87917897 +0.59%
BenchmarkDecryptReader 89948630 90042541 +0.10%
BenchmarkEncryptDecryptReader 178374144 179477306 +0.62%
BenchmarkDecrypt 88289705 87937073 -0.40%
BenchmarkSaveJSON 213917 213571 -0.16%
BenchmarkSaveFrom 74881361 75111148 +0.31%
benchmark old MB/s new MB/s speedup
BenchmarkChunkEncrypt 40.15 40.29 1.00x
BenchmarkChunkEncryptParallel 40.14 40.13 1.00x
BenchmarkEncryptWriter 95.00 95.84 1.01x
BenchmarkEncrypt 95.97 95.41 0.99x
BenchmarkDecryptReader 93.26 93.16 1.00x
BenchmarkEncryptDecryptReader 47.03 46.74 0.99x
BenchmarkDecrypt 95.01 95.39 1.00x
BenchmarkSaveFrom 56.01 55.84 1.00x
benchmark old allocs new allocs delta
BenchmarkChunkEncrypt 113 112 -0.88%
BenchmarkChunkEncryptParallel 104 103 -0.96%
BenchmarkArchiveDirectory 0 0 +0.00%
BenchmarkEncryptWriter 20 20 +0.00%
BenchmarkEncrypt 14 14 +0.00%
BenchmarkDecryptReader 18 18 +0.00%
BenchmarkEncryptDecryptReader 40 45 +12.50%
BenchmarkDecrypt 17 16 -5.88%
BenchmarkSaveJSON 125 125 +0.00%
BenchmarkSaveFrom 116 119 +2.59%
benchmark old bytes new bytes delta
BenchmarkChunkEncrypt 8515750 118956 -98.60%
BenchmarkChunkEncryptParallel 8515766 118972 -98.60%
BenchmarkArchiveDirectory 0 0 +0.00%
BenchmarkEncryptWriter 28927 28927 +0.00%
BenchmarkEncrypt 422313 2473 -99.41%
BenchmarkDecryptReader 527827 527827 +0.00%
BenchmarkEncryptDecryptReader 4100824 4100870 +0.00%
BenchmarkDecrypt 8391127 2509 -99.97%
BenchmarkSaveJSON 9208 9208 +0.00%
BenchmarkSaveFrom 39694 40541 +2.13%
|
2015-02-17 21:10:26 +01:00 |
|
Alexander Neumann
|
43ae0b69ab
|
Fix benchmarks for ChunkEncrypt
|
2015-02-17 20:02:43 +01:00 |
|
Alexander Neumann
|
af57fb86d7
|
Reduce memory usage of decryptReader
benchmark old ns/op new ns/op delta
BenchmarkChunkEncrypt 260007360 261144414 +0.44%
BenchmarkChunkEncryptParallel 262839697 261201438 -0.62%
BenchmarkArchiveDirectory 0.00 0.00 +0.00%
BenchmarkEncryptWriter 86994839 88297245 +1.50%
BenchmarkEncrypt 87414849 87406446 -0.01%
BenchmarkDecryptReader 90354651 89948630 -0.45%
BenchmarkEncryptDecryptReader 184533845 178374144 -3.34%
BenchmarkDecrypt 88153894 88289705 +0.15%
BenchmarkSaveJSON 213906 213917 +0.01%
BenchmarkSaveFrom 75263853 74881361 -0.51%
benchmark old MB/s new MB/s speedup
BenchmarkChunkEncrypt 40.33 40.15 1.00x
BenchmarkChunkEncryptParallel 39.89 40.14 1.01x
BenchmarkEncryptWriter 96.43 95.00 0.99x
BenchmarkEncrypt 95.96 95.97 1.00x
BenchmarkDecryptReader 92.84 93.26 1.00x
BenchmarkEncryptDecryptReader 45.46 47.03 1.03x
BenchmarkDecrypt 95.16 95.01 1.00x
BenchmarkSaveFrom 55.73 56.01 1.01x
benchmark old allocs new allocs delta
BenchmarkChunkEncrypt 113 113 +0.00%
BenchmarkChunkEncryptParallel 104 104 +0.00%
BenchmarkArchiveDirectory 0 0 +0.00%
BenchmarkEncryptWriter 20 20 +0.00%
BenchmarkEncrypt 14 14 +0.00%
BenchmarkDecryptReader 18 18 +0.00%
BenchmarkEncryptDecryptReader 55 40 -27.27%
BenchmarkDecrypt 17 17 +0.00%
BenchmarkSaveJSON 125 125 +0.00%
BenchmarkSaveFrom 119 116 -2.52%
benchmark old bytes new bytes delta
BenchmarkChunkEncrypt 8515750 8515750 +0.00%
BenchmarkChunkEncryptParallel 8515766 8515766 +0.00%
BenchmarkArchiveDirectory 0 0 +0.00%
BenchmarkEncryptWriter 28927 28927 +0.00%
BenchmarkEncrypt 422313 422313 +0.00%
BenchmarkDecryptReader 527827 527827 +0.00%
BenchmarkEncryptDecryptReader 35814894 4100824 -88.55%
BenchmarkDecrypt 8391127 8391127 +0.00%
BenchmarkSaveJSON 9208 9208 +0.00%
BenchmarkSaveFrom 40541 39694 -2.09%
|
2015-02-17 18:14:56 +01:00 |
|
Alexander Neumann
|
be03a31bdd
|
Use correct buffer size for benchmark
|
2015-02-17 18:05:58 +01:00 |
|
Alexander Neumann
|
4fd8222da6
|
Remove unneeded buffers from Archiver.SaveFile()
|
2015-02-16 23:47:32 +01:00 |
|
Alexander Neumann
|
d4cfa7b797
|
Remove scanner (not needed any more)
|
2015-02-16 23:47:14 +01:00 |
|
Alexander Neumann
|
0a164ad5e0
|
Add test and benchmark for Scanner
|
2015-02-16 23:13:23 +01:00 |
|
Alexander Neumann
|
a5e13f1280
|
Switch SaveFrom() to Reader/Writer
benchcmp:
benchmark old ns/op new ns/op delta
BenchmarkSaveFrom 76159512 74795081 -1.79%
benchmark old MB/s new MB/s speedup
BenchmarkSaveFrom 55.07 56.08 1.02x
benchmark old allocs new allocs delta
BenchmarkSaveFrom 97 117 +20.62%
benchmark old bytes new bytes delta
BenchmarkSaveFrom 433826 40117 -90.75%
benchmark old ns/op new ns/op delta
BenchmarkArchiveDirectory-4 29539886128 29262480891 -0.94%
benchmark old allocs new allocs delta
BenchmarkArchiveDirectory-4 540048 709662 +31.41%
benchmark old bytes new bytes delta
BenchmarkArchiveDirectory-4 404573416 443098816 +9.52%
|
2015-02-16 22:42:11 +01:00 |
|
Alexander Neumann
|
874b29b91f
|
Improve memory usage (and speed) of crypto reader/writer
benchcmp:
benchmark old ns/op new ns/op delta
BenchmarkChunkEncrypt 260811933 261294510 +0.19%
BenchmarkChunkEncryptParallel 262787948 259394601 -1.29%
BenchmarkEncryptWriter 89769328 87108804 -2.96%
BenchmarkEncrypt 88586645 87489151 -1.24%
BenchmarkDecryptReader 99669425 90019414 -9.68%
BenchmarkEncryptDecryptReader 189263157 185185733 -2.15%
BenchmarkDecrypt 89631109 89172417 -0.51%
benchmark old MB/s new MB/s speedup
BenchmarkChunkEncrypt 40.20 40.13 1.00x
BenchmarkChunkEncryptParallel 39.90 40.42 1.01x
BenchmarkEncryptWriter 93.45 96.30 1.03x
BenchmarkEncrypt 94.69 95.88 1.01x
BenchmarkDecryptReader 84.16 93.19 1.11x
BenchmarkEncryptDecryptReader 44.32 45.30 1.02x
BenchmarkDecrypt 93.59 94.07 1.01x
benchmark old allocs new allocs delta
BenchmarkChunkEncrypt 113 113 +0.00%
BenchmarkChunkEncryptParallel 104 104 +0.00%
BenchmarkEncryptWriter 20 20 +0.00%
BenchmarkEncrypt 14 14 +0.00%
BenchmarkDecryptReader 38 18 -52.63%
BenchmarkEncryptDecryptReader 61 55 -9.84%
BenchmarkDecrypt 17 17 +0.00%
benchmark old bytes new bytes delta
BenchmarkChunkEncrypt 8515750 8515750 +0.00%
BenchmarkChunkEncryptParallel 8515766 8515766 +0.00%
BenchmarkEncryptWriter 8391305 28927 -99.66%
BenchmarkEncrypt 2475 2475 +0.00%
BenchmarkDecryptReader 33563550 527827 -98.43%
BenchmarkEncryptDecryptReader 50348456 35814894 -28.87%
BenchmarkDecrypt 8391127 8391127 +0.00%
|
2015-02-16 22:35:46 +01:00 |
|
Alexander Neumann
|
d94d003165
|
Add test and benchmark for SaveFrom()
|
2015-02-16 22:35:46 +01:00 |
|
Alexander Neumann
|
731e81ef06
|
Re-use zlib writers to reduce memory usage
benchcmp:
benchmark old ns/op new ns/op delta
BenchmarkArchiveDirectory-4 29624960475 29511001504 -0.38%
BenchmarkSaveJSON-4 379833 225609 -40.60%
benchmark old allocs new allocs delta
BenchmarkArchiveDirectory-4 546736 540642 -1.11%
BenchmarkSaveJSON-4 150 126 -16.00%
benchmark old bytes new bytes delta
BenchmarkArchiveDirectory-4 1372476952 438519336 -68.05%
BenchmarkSaveJSON-4 1462773 9087 -99.38%
|
2015-02-16 22:35:26 +01:00 |
|
Alexander Neumann
|
06ed5c12b8
|
Remove Compress() and Uncompress() from backend
|
2015-02-16 00:30:51 +01:00 |
|
Alexander Neumann
|
b762a342d3
|
SaveJSON: use streams
|
2015-02-16 00:24:58 +01:00 |
|
Alexander Neumann
|
fb2cd7b485
|
HashingWriter: Track size
|
2015-02-16 00:24:43 +01:00 |
|
Alexander Neumann
|
b816192d1c
|
backend: Throw error when Close() is called twice
|
2015-02-16 00:24:02 +01:00 |
|
Alexander Neumann
|
259389f7d1
|
EncryptWriter: do not close underlying writer
|
2015-02-16 00:22:46 +01:00 |
|
Alexander Neumann
|
68911ef11d
|
server: use streams for LoadJSON*()
|
2015-02-15 23:49:06 +01:00 |
|
Alexander Neumann
|
907d401e32
|
backend: use HashingWriter
|
2015-02-15 23:46:21 +01:00 |
|
Alexander Neumann
|
3dbe02182b
|
Remove EncryptFrom*() methods, add Tests for EncryptTo()
|
2015-02-15 23:38:45 +01:00 |
|
Alexander Neumann
|
5c724b92b8
|
Add HashingWriter and HashAppendWriter
|
2015-02-15 23:21:35 +01:00 |
|
Alexander Neumann
|
5801d9f42f
|
Add EncryptTo() methods
|
2015-02-15 18:13:42 +01:00 |
|
Alexander Neumann
|
72c951a6ec
|
Simply backend interface
Rename CreateBlob() method to Create(), remove old Create* methods
|
2015-02-15 17:26:20 +01:00 |
|
Alexander Neumann
|
35636a9d92
|
Introduce CreateBlob() method for backend
|
2015-02-15 17:09:49 +01:00 |
|
Alexander Neumann
|
f8f8107d55
|
wip
|
2015-02-15 15:17:24 +01:00 |
|
Alexander Neumann
|
2f1137bac4
|
Add pipe package
|
2015-02-15 15:09:26 +01:00 |
|
Alexander Neumann
|
937c91e1cf
|
Add gofmt test to wercker.yml
|
2015-02-15 15:07:43 +01:00 |
|
Alexander Neumann
|
59cfc4bd03
|
Slightly improve performance for chunker
benchcmp output:
benchmark old ns/op new ns/op delta
BenchmarkChunkerWithSHA256 151752169 148072829 -2.42%
BenchmarkChunkerWithMD5 97166433 93357090 -3.92%
BenchmarkChunker 77265146 73642723 -4.69%
benchmark old MB/s new MB/s speedup
BenchmarkChunkerWithSHA256 69.10 70.81 1.02x
BenchmarkChunkerWithMD5 107.92 112.32 1.04x
BenchmarkChunker 135.71 142.39 1.05x
benchmark old allocs new allocs delta
BenchmarkChunkerWithSHA256 15 13 -13.33%
BenchmarkChunkerWithMD5 14 12 -14.29%
BenchmarkChunker 8 6 -25.00%
benchmark old bytes new bytes delta
BenchmarkChunkerWithSHA256 262924 26821 -89.80%
BenchmarkChunkerWithMD5 262774 13554 -94.84%
BenchmarkChunker 262678 13458 -94.88%
|
2015-02-13 20:03:40 +01:00 |
|
Alexander Neumann
|
0f36b19a4c
|
Update Server to match new Backend interface
|
2015-02-11 22:46:40 +01:00 |
|
Alexander Neumann
|
92e2647505
|
Add tests for sftp backend, split out id tests
|
2015-02-11 20:17:55 +01:00 |
|
Alexander Neumann
|
6c68150e45
|
Add streaming functions to mockBackend
|
2015-02-11 20:01:01 +01:00 |
|
Alexander Neumann
|
89bf88df7a
|
Add streaming methods to interface
|
2015-02-11 19:55:23 +01:00 |
|
Alexander Neumann
|
fdbe2f0c2d
|
Implement streaming methods for sftp backend
|
2015-02-11 19:55:11 +01:00 |
|
Alexander Neumann
|
0a45ec1d3a
|
Implement streaming functions for local backend
|
2015-02-11 19:50:29 +01:00 |
|
Alexander Neumann
|
64f7b4e5e9
|
Rename HashReader -> HashAppendReader, add HashingReader
|
2015-02-11 19:37:48 +01:00 |
|
Alexander Neumann
|
842dea173c
|
Generalize hash in backend
|
2015-02-11 19:25:43 +01:00 |
|
Alexander Neumann
|
5f0cce8546
|
Move HashReader to backend
|
2015-02-11 19:20:45 +01:00 |
|
Alexander Neumann
|
a564a454c4
|
Add stream functions for decrypt
|
2015-02-11 18:36:31 +01:00 |
|
Alexander Neumann
|
c6901af5aa
|
Add streaming encrypt functions
|
2015-02-11 18:05:43 +01:00 |
|
Alexander Neumann
|
93abaf204a
|
Fix get_random for count != 4k, add randomReader()
|
2015-02-11 17:38:51 +01:00 |
|
Alexander Neumann
|
3bf66d24fa
|
chunker: Add test, only pass hash.Hash to New()
|
2015-02-11 13:12:36 +01:00 |
|
Alexander Neumann
|
869ba83c6d
|
Add Benchmarks for Scanner and Archiver
|
2015-02-10 21:59:18 +01:00 |
|
Alexander Neumann
|
9fcd23bd38
|
Add parallel benchmark for chunk+encrypt
|
2015-02-09 23:39:16 +01:00 |
|