Alexander Neumann
0662c14fee
Move chunkerBufSize to more appropiate location
2015-05-01 17:40:50 +02:00
Florian Weingarten
0d9360a815
more cleanup
2015-04-29 22:33:34 -04:00
Alexander Neumann
60a0fe8349
Major restructure, bundle blobs
...
This commit also breaks the repository format.
2015-04-29 23:27:32 +02:00
Alexander Neumann
d19b23d4f1
Move Server and Key to new sub-package
2015-04-26 14:46:15 +02:00
Alexander Neumann
434c79354e
Merge pull request #143 from fw42/count_errors
...
Count errors separately and not as files that are "done"
2015-04-26 12:05:09 +02:00
Florian Weingarten
61b1a5b1de
Count errors separately and not as files that are "done"
2015-04-26 01:54:35 +00:00
Florian Weingarten
c947623707
Fix progress off-by-1
2015-04-25 15:40:42 -04:00
Florian Weingarten
262e5a8b4a
O_NOATIME on Linux
2015-04-25 00:45:20 +00:00
Alexander Neumann
b2307cafa2
Seed chunker with random per-repository polynomial
2015-04-06 00:22:19 +02:00
Alexander Neumann
24b14e21cc
Remove pools for nodes and IDs
...
This removes the allocation pools for nodes and IDs. I feel they aren't
really needed. Benchmarks:
benchmark old ns/op new ns/op delta
BenchmarkChunkEncrypt 197890867 198616293 +0.37%
BenchmarkChunkEncryptParallel 196127004 198819818 +1.37%
BenchmarkArchiveDirectory 1098848419 1087237723 -1.06%
BenchmarkPreload 30464455 29910239 -1.82%
BenchmarkLoadTree 3265092 3088543 -5.41%
BenchmarkEncryptWriter 37213511 37134683 -0.21%
BenchmarkEncrypt 36037879 36166546 +0.36%
BenchmarkDecryptReader 38165659 38556734 +1.02%
BenchmarkEncryptDecryptReader 77027044 77194987 +0.22%
BenchmarkDecrypt 36017602 35937888 -0.22%
BenchmarkSaveJSON 47906 50270 +4.93%
BenchmarkSaveFrom 49775973 50520969 +1.50%
BenchmarkLoadJSONID 105290245 107281849 +1.89%
BenchmarkChunkerWithSHA256 151501430 148264078 -2.14%
BenchmarkChunkerWithMD5 93606346 94036392 +0.46%
BenchmarkChunker 74285431 75933882 +2.22%
BenchmarkPipelineWalker 387689 346467 -10.63%
benchmark old MB/s new MB/s speedup
BenchmarkChunkEncrypt 52.99 52.79 1.00x
BenchmarkChunkEncryptParallel 53.46 52.74 0.99x
BenchmarkEncryptWriter 225.42 225.90 1.00x
BenchmarkEncrypt 232.77 231.94 1.00x
BenchmarkDecryptReader 219.79 217.57 0.99x
BenchmarkEncryptDecryptReader 108.90 108.67 1.00x
BenchmarkDecrypt 232.90 233.42 1.00x
BenchmarkSaveFrom 84.26 83.02 0.99x
BenchmarkChunkerWithSHA256 69.21 70.72 1.02x
BenchmarkChunkerWithMD5 112.02 111.51 1.00x
BenchmarkChunker 141.15 138.09 0.98x
benchmark old allocs new allocs delta
BenchmarkChunkEncrypt 110 110 +0.00%
BenchmarkChunkEncryptParallel 100 100 +0.00%
BenchmarkArchiveDirectory 475591 476635 +0.22%
BenchmarkPreload 28059 24182 -13.82%
BenchmarkLoadTree 3124 2889 -7.52%
BenchmarkEncryptWriter 19 19 +0.00%
BenchmarkEncrypt 13 13 +0.00%
BenchmarkDecryptReader 16 15 -6.25%
BenchmarkEncryptDecryptReader 39 39 +0.00%
BenchmarkDecrypt 11 11 +0.00%
BenchmarkSaveJSON 74 74 +0.00%
BenchmarkSaveFrom 109 112 +2.75%
BenchmarkLoadJSONID 103630 97849 -5.58%
BenchmarkChunkerWithSHA256 13 13 +0.00%
BenchmarkChunkerWithMD5 12 12 +0.00%
BenchmarkChunker 6 6 +0.00%
BenchmarkPipelineWalker 212 165 -22.17%
benchmark old bytes new bytes delta
BenchmarkChunkEncrypt 64697 64697 +0.00%
BenchmarkChunkEncryptParallel 64681 64681 +0.00%
BenchmarkArchiveDirectory 193385504 193790864 +0.21%
BenchmarkPreload 4064701 3942000 -3.02%
BenchmarkLoadTree 344954 325396 -5.67%
BenchmarkEncryptWriter 12793 12793 +0.00%
BenchmarkEncrypt 1950 1950 +0.00%
BenchmarkDecryptReader 3120 2774 -11.09%
BenchmarkEncryptDecryptReader 1528036 1528036 +0.00%
BenchmarkDecrypt 1919 1919 +0.00%
BenchmarkSaveJSON 5524 5524 +0.00%
BenchmarkSaveFrom 31353 40804 +30.14%
BenchmarkLoadJSONID 12872020 16010968 +24.39%
BenchmarkChunkerWithSHA256 26821 26821 +0.00%
BenchmarkChunkerWithMD5 13554 13554 +0.00%
BenchmarkChunker 13458 13458 +0.00%
BenchmarkPipelineWalker 58584 55560 -5.16%
2015-03-28 16:51:37 +01:00
Alexander Neumann
f157f775da
Add checks for Server.Load(), use Blob for load
2015-03-28 15:07:08 +01:00
Alexander Neumann
5e69788eac
Refactor backends
2015-03-28 14:36:50 +01:00
Alexander Neumann
cfa2229bc0
Refactor cache refresh for blobs, add progress
2015-03-22 15:09:50 +01:00
Alexander Neumann
aca0692ee6
Add more error reporting
2015-03-21 14:43:33 +01:00
Alexander Neumann
71a57537ef
Report errors, ignore files and continue. Closes #108
2015-03-15 15:48:05 +01:00
Alexander Neumann
3d768e39ea
Add more error reporting
2015-03-15 12:20:30 +01:00
Alexander Neumann
3c92c7e689
Recreate blob cache if missing ( closes #104 )
2015-03-14 18:15:47 +01:00
Alexander Neumann
e1fc17aeb1
Implement per-repository cache
2015-03-14 12:16:28 +01:00
Alexander Neumann
7b54c21726
Store ID in Snapshot
2015-03-10 14:17:52 +01:00
Alexander Neumann
2166367039
Fix comparison for new files with '.'
2015-03-10 14:17:50 +01:00
Alexander Neumann
14298fe232
Cache blobs for each snapshot
2015-03-09 23:50:32 +01:00
Alexander Neumann
ab8b97c4ba
Remove code duplication
...
The top-level tree is now handled by the DirWorkers
2015-03-08 21:06:51 +01:00
Alexander Neumann
b3cdd3ead9
Remove unused code
2015-03-08 14:22:35 +01:00
Alexander Neumann
ba892e1ec2
Refactor backup pipeline
2015-03-08 14:16:01 +01:00
Alexander Neumann
f167366be5
Add walk for trees, restructure
2015-03-02 19:58:06 +01:00
Alexander Neumann
c8be54564f
Add local cache
2015-03-02 10:04:01 +01:00
Alexander Neumann
a59b0ec1f6
Add progress report for loading blobs
2015-02-21 15:32:48 +01:00
Alexander Neumann
58cded6b75
Refactor progress reporting
2015-02-21 14:25:06 +01:00
Alexander Neumann
97b963ef1d
Report files and dirs
2015-02-21 14:05:02 +01:00
Alexander Neumann
a0fea201d9
Reenable Preloading
2015-02-18 23:10:59 +01:00
Alexander Neumann
f8e47181b0
Add more debug for preloading
2015-02-17 23:38:40 +01:00
Alexander Neumann
677a5038c5
Increase concurrency for loading blobs
2015-02-17 23:15:08 +01:00
Alexander Neumann
4bb7f2f2ed
Add Preloader for blobs
2015-02-17 22:39:44 +01:00
Alexander Neumann
4fd8222da6
Remove unneeded buffers from Archiver.SaveFile()
2015-02-16 23:47:32 +01:00
Alexander Neumann
06ed5c12b8
Remove Compress() and Uncompress() from backend
2015-02-16 00:30:51 +01:00
Alexander Neumann
f8f8107d55
wip
2015-02-15 15:17:24 +01:00
Alexander Neumann
bbb3145f11
Add chunker pool
2015-02-09 23:38:50 +01:00
Alexander Neumann
bda33e612c
Implement streaming chunker using io.Reader
2015-02-09 00:28:54 +01:00
Alexander Neumann
25a214809b
Refactor debug into debug module
2015-01-17 16:32:13 +01:00
Alexander Neumann
af43fb3dda
Fix handling vanished files
2015-01-14 17:48:23 +01:00
Alexander Neumann
4e54c6af2c
Fix calls to debug()
2015-01-14 17:48:10 +01:00
Alexander Neumann
48751e1935
Introduce debug tags and debug breaks
...
The environment variable DEBUG_TAGS can be used to control what is
printed on stderr. Example:
DEBUG_TAGS="+all,-Archiver.*" ./restic backup foo
The variable DEBUG_BREAK can be used to set a breakpoint (implemented
using SIGSTOP):
DEBUG_BREAK=Archiver.Snapshot ./restic backup foo
2015-01-14 16:34:30 +01:00
Alexander Neumann
203a911de9
Refactor repository structure
...
Merge Map data type into Tree.
2015-01-14 16:34:30 +01:00
Alexander Neumann
cdf3336e7a
Improve error message for newer files
2015-01-08 20:49:32 +01:00
Alexander Neumann
33bcf31bae
Fix incremental backup
...
Copying blobs (for unchanged files and subtrees) from old BlobList to
new BlobList was missing
2015-01-05 21:40:43 +01:00
Alexander Neumann
6eb969a492
Update modified files, store error message
2015-01-04 23:25:13 +01:00
Alexander Neumann
e543f5926c
Refactor Archiver and backup command
...
Improve incremental backup by first scanning the tree, loading the old
tree and afterwards comparing both trees in memory.
2015-01-04 22:39:30 +01:00
Alexander Neumann
fe231af7fc
Remove error from return value of NewContentHandler
2015-01-04 20:07:30 +01:00
Alexander Neumann
4b70bba588
Split Scanner from Archiver, refactor Progress
2015-01-04 18:23:00 +01:00
Alexander Neumann
a93bc3c991
Rename functions, remove code for compare
...
* Archiver: Rename loadTree -> scan and LoadTree -> Scan
* Archiver: Remove code to compare against an old snapshot/tree,
the current implementation was really slow
2015-01-04 16:13:50 +01:00
