5478 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Alexander Neumann	2166367039	Fix comparison for new files with '.'	2015-03-10 14:17:50 +01:00
Alexander Neumann	14298fe232	Cache blobs for each snapshot	2015-03-09 23:50:32 +01:00
Alexander Neumann	b2cce7f051	Remove extra argument	2015-03-08 21:21:31 +01:00
Alexander Neumann	ab8b97c4ba	Remove code duplication ... The top-level tree is now handled by the DirWorkers	2015-03-08 21:06:51 +01:00
Alexander Neumann	6007452fd2	Fix darwin support	2015-03-08 14:32:49 +01:00
Alexander Neumann	b3cdd3ead9	Remove unused code	2015-03-08 14:22:35 +01:00
Alexander Neumann	ba892e1ec2	Refactor backup pipeline	2015-03-08 14:16:01 +01:00
Alexander Neumann	1516cb5996	Bump Go version to 1.3	2015-03-07 22:31:44 +01:00
Alexander Neumann	798ac97a52	Re-enable dedup, add test (closes #95)	2015-03-07 12:05:33 +01:00
Alexander Neumann	03bdbb064f	Allow cancelling walker	2015-03-02 19:58:11 +01:00
Alexander Neumann	f167366be5	Add walk for trees, restructure	2015-03-02 19:58:06 +01:00
Alexander Neumann	c8be54564f	Add local cache	2015-03-02 10:04:01 +01:00
Alexander Neumann	c056382c9c	Remove zlib compression	2015-03-02 10:02:55 +01:00
Alexander Neumann	b4cb75693e	Do not close file if it could not be opened	2015-02-22 00:21:56 +01:00
Alexander Neumann	ef498a987b	Allow configuring temp dir for tests	2015-02-21 16:53:14 +01:00
Alexander Neumann	a59b0ec1f6	Add progress report for loading blobs	2015-02-21 15:32:48 +01:00
Alexander Neumann	58cded6b75	Refactor progress reporting	2015-02-21 14:25:06 +01:00
Alexander Neumann	97b963ef1d	Report files and dirs	2015-02-21 14:05:02 +01:00
Alexander Neumann	2f3aa344af	Make decryptReader fulfill flate.Reader ... benchmark old ns/op new ns/op delta BenchmarkLoadJSONID 50232171 48596972 -3.26% benchmark old allocs new allocs delta BenchmarkLoadJSONID 43643 42884 -1.74% benchmark old bytes new bytes delta BenchmarkLoadJSONID 5773048 3785517 -34.43%	2015-02-18 23:30:59 +01:00
Alexander Neumann	a0fea201d9	Reenable Preloading	2015-02-18 23:10:59 +01:00
Alexander Neumann	f214dce87c	Add tests for Archiver.Preload() and a few more	2015-02-18 22:48:59 +01:00
Alexander Neumann	3bb2aba141	Remove test files	2015-02-18 21:50:59 +01:00
Alexander Neumann	68141c738b	Correct length of plaintext before decryption	2015-02-18 20:44:21 +01:00
Alexander Neumann	2462f93533	Disable Tests when benchdir is unset	2015-02-17 23:40:37 +01:00
Alexander Neumann	f8e47181b0	Add more debug for preloading	2015-02-17 23:38:40 +01:00
Alexander Neumann	65aae5f8c1	Disable preloading for now	2015-02-17 23:38:15 +01:00
Alexander Neumann	d3fe4a7f6d	Close Reader after read	2015-02-17 23:37:45 +01:00
Alexander Neumann	215fb982c6	Do not run archiver tests by default	2015-02-17 23:23:12 +01:00
Alexander Neumann	677a5038c5	Increase concurrency for loading blobs	2015-02-17 23:15:08 +01:00
Alexander Neumann	b6f25aa690	Remove Each(), add basic stats	2015-02-17 23:11:56 +01:00
Alexander Neumann	4bb7f2f2ed	Add Preloader for blobs	2015-02-17 22:39:44 +01:00
Alexander Neumann	c631b5c738	Add Len() functions for IDSet and Map	2015-02-17 22:39:16 +01:00
Alexander Neumann	4d1e7b9f24	Change parameters of LoadTree()	2015-02-17 22:37:34 +01:00
Alexander Neumann	5a16a66973	Add 'plaintext' parameter to key.Decrypt() ... % benchcmp /tmp/{before,after} benchmark old ns/op new ns/op delta BenchmarkChunkEncrypt 261144414 260252046 -0.34% BenchmarkChunkEncryptParallel 261201438 261267029 +0.03% BenchmarkArchiveDirectory 0.00 0.00 +0.00% BenchmarkEncryptWriter 88297245 87526529 -0.87% BenchmarkEncrypt 87406446 87917897 +0.59% BenchmarkDecryptReader 89948630 90042541 +0.10% BenchmarkEncryptDecryptReader 178374144 179477306 +0.62% BenchmarkDecrypt 88289705 87937073 -0.40% BenchmarkSaveJSON 213917 213571 -0.16% BenchmarkSaveFrom 74881361 75111148 +0.31% benchmark old MB/s new MB/s speedup BenchmarkChunkEncrypt 40.15 40.29 1.00x BenchmarkChunkEncryptParallel 40.14 40.13 1.00x BenchmarkEncryptWriter 95.00 95.84 1.01x BenchmarkEncrypt 95.97 95.41 0.99x BenchmarkDecryptReader 93.26 93.16 1.00x BenchmarkEncryptDecryptReader 47.03 46.74 0.99x BenchmarkDecrypt 95.01 95.39 1.00x BenchmarkSaveFrom 56.01 55.84 1.00x benchmark old allocs new allocs delta BenchmarkChunkEncrypt 113 112 -0.88% BenchmarkChunkEncryptParallel 104 103 -0.96% BenchmarkArchiveDirectory 0 0 +0.00% BenchmarkEncryptWriter 20 20 +0.00% BenchmarkEncrypt 14 14 +0.00% BenchmarkDecryptReader 18 18 +0.00% BenchmarkEncryptDecryptReader 40 45 +12.50% BenchmarkDecrypt 17 16 -5.88% BenchmarkSaveJSON 125 125 +0.00% BenchmarkSaveFrom 116 119 +2.59% benchmark old bytes new bytes delta BenchmarkChunkEncrypt 8515750 118956 -98.60% BenchmarkChunkEncryptParallel 8515766 118972 -98.60% BenchmarkArchiveDirectory 0 0 +0.00% BenchmarkEncryptWriter 28927 28927 +0.00% BenchmarkEncrypt 422313 2473 -99.41% BenchmarkDecryptReader 527827 527827 +0.00% BenchmarkEncryptDecryptReader 4100824 4100870 +0.00% BenchmarkDecrypt 8391127 2509 -99.97% BenchmarkSaveJSON 9208 9208 +0.00% BenchmarkSaveFrom 39694 40541 +2.13%	2015-02-17 21:10:26 +01:00
Alexander Neumann	43ae0b69ab	Fix benchmarks for ChunkEncrypt	2015-02-17 20:02:43 +01:00
Alexander Neumann	af57fb86d7	Reduce memory usage of decryptReader ... benchmark old ns/op new ns/op delta BenchmarkChunkEncrypt 260007360 261144414 +0.44% BenchmarkChunkEncryptParallel 262839697 261201438 -0.62% BenchmarkArchiveDirectory 0.00 0.00 +0.00% BenchmarkEncryptWriter 86994839 88297245 +1.50% BenchmarkEncrypt 87414849 87406446 -0.01% BenchmarkDecryptReader 90354651 89948630 -0.45% BenchmarkEncryptDecryptReader 184533845 178374144 -3.34% BenchmarkDecrypt 88153894 88289705 +0.15% BenchmarkSaveJSON 213906 213917 +0.01% BenchmarkSaveFrom 75263853 74881361 -0.51% benchmark old MB/s new MB/s speedup BenchmarkChunkEncrypt 40.33 40.15 1.00x BenchmarkChunkEncryptParallel 39.89 40.14 1.01x BenchmarkEncryptWriter 96.43 95.00 0.99x BenchmarkEncrypt 95.96 95.97 1.00x BenchmarkDecryptReader 92.84 93.26 1.00x BenchmarkEncryptDecryptReader 45.46 47.03 1.03x BenchmarkDecrypt 95.16 95.01 1.00x BenchmarkSaveFrom 55.73 56.01 1.01x benchmark old allocs new allocs delta BenchmarkChunkEncrypt 113 113 +0.00% BenchmarkChunkEncryptParallel 104 104 +0.00% BenchmarkArchiveDirectory 0 0 +0.00% BenchmarkEncryptWriter 20 20 +0.00% BenchmarkEncrypt 14 14 +0.00% BenchmarkDecryptReader 18 18 +0.00% BenchmarkEncryptDecryptReader 55 40 -27.27% BenchmarkDecrypt 17 17 +0.00% BenchmarkSaveJSON 125 125 +0.00% BenchmarkSaveFrom 119 116 -2.52% benchmark old bytes new bytes delta BenchmarkChunkEncrypt 8515750 8515750 +0.00% BenchmarkChunkEncryptParallel 8515766 8515766 +0.00% BenchmarkArchiveDirectory 0 0 +0.00% BenchmarkEncryptWriter 28927 28927 +0.00% BenchmarkEncrypt 422313 422313 +0.00% BenchmarkDecryptReader 527827 527827 +0.00% BenchmarkEncryptDecryptReader 35814894 4100824 -88.55% BenchmarkDecrypt 8391127 8391127 +0.00% BenchmarkSaveJSON 9208 9208 +0.00% BenchmarkSaveFrom 40541 39694 -2.09%	2015-02-17 18:14:56 +01:00
Alexander Neumann	be03a31bdd	Use correct buffer size for benchmark	2015-02-17 18:05:58 +01:00
Alexander Neumann	4fd8222da6	Remove unneeded buffers from Archiver.SaveFile()	2015-02-16 23:47:32 +01:00
Alexander Neumann	d4cfa7b797	Remove scanner (not needed any more)	2015-02-16 23:47:14 +01:00
Alexander Neumann	0a164ad5e0	Add test and benchmark for Scanner	2015-02-16 23:13:23 +01:00
Alexander Neumann	a5e13f1280	Switch SaveFrom() to Reader/Writer ... benchcmp: benchmark old ns/op new ns/op delta BenchmarkSaveFrom 76159512 74795081 -1.79% benchmark old MB/s new MB/s speedup BenchmarkSaveFrom 55.07 56.08 1.02x benchmark old allocs new allocs delta BenchmarkSaveFrom 97 117 +20.62% benchmark old bytes new bytes delta BenchmarkSaveFrom 433826 40117 -90.75% benchmark old ns/op new ns/op delta BenchmarkArchiveDirectory-4 29539886128 29262480891 -0.94% benchmark old allocs new allocs delta BenchmarkArchiveDirectory-4 540048 709662 +31.41% benchmark old bytes new bytes delta BenchmarkArchiveDirectory-4 404573416 443098816 +9.52%	2015-02-16 22:42:11 +01:00
Alexander Neumann	874b29b91f	Improve memory usage (and speed) of crypto reader/writer ... benchcmp: benchmark old ns/op new ns/op delta BenchmarkChunkEncrypt 260811933 261294510 +0.19% BenchmarkChunkEncryptParallel 262787948 259394601 -1.29% BenchmarkEncryptWriter 89769328 87108804 -2.96% BenchmarkEncrypt 88586645 87489151 -1.24% BenchmarkDecryptReader 99669425 90019414 -9.68% BenchmarkEncryptDecryptReader 189263157 185185733 -2.15% BenchmarkDecrypt 89631109 89172417 -0.51% benchmark old MB/s new MB/s speedup BenchmarkChunkEncrypt 40.20 40.13 1.00x BenchmarkChunkEncryptParallel 39.90 40.42 1.01x BenchmarkEncryptWriter 93.45 96.30 1.03x BenchmarkEncrypt 94.69 95.88 1.01x BenchmarkDecryptReader 84.16 93.19 1.11x BenchmarkEncryptDecryptReader 44.32 45.30 1.02x BenchmarkDecrypt 93.59 94.07 1.01x benchmark old allocs new allocs delta BenchmarkChunkEncrypt 113 113 +0.00% BenchmarkChunkEncryptParallel 104 104 +0.00% BenchmarkEncryptWriter 20 20 +0.00% BenchmarkEncrypt 14 14 +0.00% BenchmarkDecryptReader 38 18 -52.63% BenchmarkEncryptDecryptReader 61 55 -9.84% BenchmarkDecrypt 17 17 +0.00% benchmark old bytes new bytes delta BenchmarkChunkEncrypt 8515750 8515750 +0.00% BenchmarkChunkEncryptParallel 8515766 8515766 +0.00% BenchmarkEncryptWriter 8391305 28927 -99.66% BenchmarkEncrypt 2475 2475 +0.00% BenchmarkDecryptReader 33563550 527827 -98.43% BenchmarkEncryptDecryptReader 50348456 35814894 -28.87% BenchmarkDecrypt 8391127 8391127 +0.00%	2015-02-16 22:35:46 +01:00
Alexander Neumann	d94d003165	Add test and benchmark for SaveFrom()	2015-02-16 22:35:46 +01:00
Alexander Neumann	731e81ef06	Re-use zlib writers to reduce memory usage ... benchcmp: benchmark old ns/op new ns/op delta BenchmarkArchiveDirectory-4 29624960475 29511001504 -0.38% BenchmarkSaveJSON-4 379833 225609 -40.60% benchmark old allocs new allocs delta BenchmarkArchiveDirectory-4 546736 540642 -1.11% BenchmarkSaveJSON-4 150 126 -16.00% benchmark old bytes new bytes delta BenchmarkArchiveDirectory-4 1372476952 438519336 -68.05% BenchmarkSaveJSON-4 1462773 9087 -99.38%	2015-02-16 22:35:26 +01:00
Alexander Neumann	06ed5c12b8	Remove Compress() and Uncompress() from backend	2015-02-16 00:30:51 +01:00
Alexander Neumann	b762a342d3	SaveJSON: use streams	2015-02-16 00:24:58 +01:00
Alexander Neumann	fb2cd7b485	HashingWriter: Track size	2015-02-16 00:24:43 +01:00
Alexander Neumann	b816192d1c	backend: Throw error when Close() is called twice	2015-02-16 00:24:02 +01:00
Alexander Neumann	259389f7d1	EncryptWriter: do not close underlying writer	2015-02-16 00:22:46 +01:00
Alexander Neumann	68911ef11d	server: use streams for LoadJSON*()	2015-02-15 23:49:06 +01:00