Commit Graph

394 Commits

Author SHA1 Message Date
Alexander Neumann 3cdf3a25b9 chunker: Require a random irreducible polynomial
This also implements the necessary polynomial arithmetics in F_2[X].
2015-04-05 22:52:43 +02:00
Alexander Neumann 094ca7e635 Remove unused key offset from blob struct 2015-03-28 16:59:14 +01:00
Alexander Neumann 24b14e21cc Remove pools for nodes and IDs
This removes the allocation pools for nodes and IDs. I feel they aren't
really needed. Benchmarks:

    benchmark                         old ns/op      new ns/op      delta
    BenchmarkChunkEncrypt             197890867      198616293      +0.37%
    BenchmarkChunkEncryptParallel     196127004      198819818      +1.37%
    BenchmarkArchiveDirectory         1098848419     1087237723     -1.06%
    BenchmarkPreload                  30464455       29910239       -1.82%
    BenchmarkLoadTree                 3265092        3088543        -5.41%
    BenchmarkEncryptWriter            37213511       37134683       -0.21%
    BenchmarkEncrypt                  36037879       36166546       +0.36%
    BenchmarkDecryptReader            38165659       38556734       +1.02%
    BenchmarkEncryptDecryptReader     77027044       77194987       +0.22%
    BenchmarkDecrypt                  36017602       35937888       -0.22%
    BenchmarkSaveJSON                 47906          50270          +4.93%
    BenchmarkSaveFrom                 49775973       50520969       +1.50%
    BenchmarkLoadJSONID               105290245      107281849      +1.89%
    BenchmarkChunkerWithSHA256        151501430      148264078      -2.14%
    BenchmarkChunkerWithMD5           93606346       94036392       +0.46%
    BenchmarkChunker                  74285431       75933882       +2.22%
    BenchmarkPipelineWalker           387689         346467         -10.63%

    benchmark                         old MB/s     new MB/s     speedup
    BenchmarkChunkEncrypt             52.99        52.79        1.00x
    BenchmarkChunkEncryptParallel     53.46        52.74        0.99x
    BenchmarkEncryptWriter            225.42       225.90       1.00x
    BenchmarkEncrypt                  232.77       231.94       1.00x
    BenchmarkDecryptReader            219.79       217.57       0.99x
    BenchmarkEncryptDecryptReader     108.90       108.67       1.00x
    BenchmarkDecrypt                  232.90       233.42       1.00x
    BenchmarkSaveFrom                 84.26        83.02        0.99x
    BenchmarkChunkerWithSHA256        69.21        70.72        1.02x
    BenchmarkChunkerWithMD5           112.02       111.51       1.00x
    BenchmarkChunker                  141.15       138.09       0.98x

    benchmark                         old allocs     new allocs     delta
    BenchmarkChunkEncrypt             110            110            +0.00%
    BenchmarkChunkEncryptParallel     100            100            +0.00%
    BenchmarkArchiveDirectory         475591         476635         +0.22%
    BenchmarkPreload                  28059          24182          -13.82%
    BenchmarkLoadTree                 3124           2889           -7.52%
    BenchmarkEncryptWriter            19             19             +0.00%
    BenchmarkEncrypt                  13             13             +0.00%
    BenchmarkDecryptReader            16             15             -6.25%
    BenchmarkEncryptDecryptReader     39             39             +0.00%
    BenchmarkDecrypt                  11             11             +0.00%
    BenchmarkSaveJSON                 74             74             +0.00%
    BenchmarkSaveFrom                 109            112            +2.75%
    BenchmarkLoadJSONID               103630         97849          -5.58%
    BenchmarkChunkerWithSHA256        13             13             +0.00%
    BenchmarkChunkerWithMD5           12             12             +0.00%
    BenchmarkChunker                  6              6              +0.00%
    BenchmarkPipelineWalker           212            165            -22.17%

    benchmark                         old bytes     new bytes     delta
    BenchmarkChunkEncrypt             64697         64697         +0.00%
    BenchmarkChunkEncryptParallel     64681         64681         +0.00%
    BenchmarkArchiveDirectory         193385504     193790864     +0.21%
    BenchmarkPreload                  4064701       3942000       -3.02%
    BenchmarkLoadTree                 344954        325396        -5.67%
    BenchmarkEncryptWriter            12793         12793         +0.00%
    BenchmarkEncrypt                  1950          1950          +0.00%
    BenchmarkDecryptReader            3120          2774          -11.09%
    BenchmarkEncryptDecryptReader     1528036       1528036       +0.00%
    BenchmarkDecrypt                  1919          1919          +0.00%
    BenchmarkSaveJSON                 5524          5524          +0.00%
    BenchmarkSaveFrom                 31353         40804         +30.14%
    BenchmarkLoadJSONID               12872020      16010968      +24.39%
    BenchmarkChunkerWithSHA256        26821         26821         +0.00%
    BenchmarkChunkerWithMD5           13554         13554         +0.00%
    BenchmarkChunker                  13458         13458         +0.00%
    BenchmarkPipelineWalker           58584         55560         -5.16%
2015-03-28 16:51:37 +01:00
Alexander Neumann 702b6cfc1e Make ID.Str() handle nil ids correctly 2015-03-28 16:46:21 +01:00
Alexander Neumann 2abda75404 Refactor mockBackend for test 2015-03-28 16:42:35 +01:00
Alexander Neumann b86786aec9 Do not archive non-existing items 2015-03-28 16:35:46 +01:00
Alexander Neumann 8eccfc6e79 Add test for backing up non-existing files
restic doesn't terminate in this case, this was described by @rubenv
in #90 https://github.com/restic/restic/issues/90#issuecomment-86901128
2015-03-28 16:16:19 +01:00
Alexander Neumann 4d24f2eab3 Correct plen for snapshots 2015-03-28 16:16:19 +01:00
Alexander Neumann 616170ed2e Run sftp integration test in a subshell 2015-03-28 15:53:33 +01:00
Alexander Neumann 1954680c9e Go1.3: Fix range over channel 2015-03-28 15:52:15 +01:00
Alexander Neumann 2c136381c2 Fix travis test 2015-03-28 15:51:08 +01:00
Alexander Neumann a6596cdb29 Run integration test for sftp backend with travis 2015-03-28 15:48:11 +01:00
Alexander Neumann 223108c6ae Remove unused vars and const 2015-03-28 15:28:08 +01:00
Alexander Neumann f157f775da Add checks for Server.Load(), use Blob for load 2015-03-28 15:07:08 +01:00
Alexander Neumann 5e69788eac Refactor backends 2015-03-28 14:36:50 +01:00
Alexander Neumann f51aba1510 CI: Move gofmt check to end of tests
Suggested by @tyll
2015-03-25 16:17:18 +01:00
Alexander Neumann 53e66a4935 Design.md: Minor fixes 2015-03-22 22:53:29 +01:00
Alexander Neumann 2dad4fba3a Clarify ciphertext mac 2015-03-22 21:29:19 +01:00
Alexander Neumann 02282a4fff Clarify use of poly1305 2015-03-22 21:26:14 +01:00
Alexander Neumann 53ad706c6d Remove unused 'ram' tool 2015-03-22 21:03:25 +01:00
Alexander Neumann 0ed2a066a0 Document crypto and master key JSON struct 2015-03-22 21:03:01 +01:00
Alexander Neumann 65a653693e Rename keys to MasterKeys 2015-03-22 19:19:33 +01:00
Alexander Neumann 664a12c950 Collect all master crypto keys in a single structure 2015-03-22 19:18:34 +01:00
Alexander Neumann 78727c17a3 Update documentation to reflect changed crypto 2015-03-22 19:18:09 +01:00
Alexander Neumann 47d71d23d2 Refactor JSON data structure for master keys 2015-03-22 19:09:30 +01:00
Alexander Neumann 3e246170de Remove cache entries for non-existing snapshots 2015-03-22 16:09:35 +01:00
Alexander Neumann 5257c54585 Correct string for catting data blobs 2015-03-22 15:42:46 +01:00
Alexander Neumann 1c0e76ccd6 Bugfix: Backup changed data in incremental mode 2015-03-22 15:09:54 +01:00
Alexander Neumann cfa2229bc0 Refactor cache refresh for blobs, add progress 2015-03-22 15:09:50 +01:00
Alexander Neumann 702cf3c2ff Progress: Add function that is called on Start() 2015-03-22 14:19:16 +01:00
Alexander Neumann f2df072f48 Progress: Don't panic for not running progress 2015-03-22 14:17:19 +01:00
Florian Daniel 2506fa9bef fixed variable handling, new go ldflags 2015-03-22 13:41:45 +01:00
Alexander Neumann 0d39d46ac6 Fix YAML config (again) 2015-03-22 13:26:41 +01:00
Alexander Neumann c58fc8b6a2 Fix travis again 2015-03-22 13:22:32 +01:00
Alexander Neumann e1acade174 Fix travis build 2015-03-22 13:09:14 +01:00
Alexander Neumann 30ce48dc32 Add '-s' ldflag and output size to travis tests 2015-03-22 12:58:32 +01:00
Alexander Neumann aca0692ee6 Add more error reporting 2015-03-21 14:43:33 +01:00
Alexander Neumann 865d315ad6 Shorten status output if terminal is small
Closes #107
2015-03-16 20:48:29 +01:00
Alexander Neumann 489cc654e8 Fix ETA display
Closes #111
2015-03-16 20:20:53 +01:00
Florian Daniel d9fbf9ad02 archlinux PKGBUILD added. Thanks to the contributors.
Closes #44
Closes #106
2015-03-15 16:32:43 +01:00
Alexander Neumann 5740f202e3 Add "Golang" keyword to README 2015-03-15 16:13:03 +01:00
Alexander Neumann 71a57537ef Report errors, ignore files and continue. Closes #108 2015-03-15 15:48:05 +01:00
Alexander Neumann 1bb82afcf6 Add tests for bug #101 2015-03-15 15:27:22 +01:00
Alexander Neumann c9ee79fc54 DebugIf: only test function when tag is enabled 2015-03-15 14:19:37 +01:00
Alexander Neumann 6aed4035f5 Add BreakIf function 2015-03-15 14:00:28 +01:00
Alexander Neumann 3d768e39ea Add more error reporting 2015-03-15 12:20:30 +01:00
Alexander Neumann 56bca76b74 Fix benchmark for EncryptWriter 2015-03-14 22:42:30 +01:00
Alexander Neumann 12c8522b33 Fix file mode for data blobs 2015-03-14 20:19:52 +01:00
Alexander Neumann e896a33c43 Remove a few Go versions 2015-03-14 20:02:17 +01:00
Alexander Neumann 662e07d17a Refactor crypto layer, switch HMAC for Poyl1305-AES
HMAC-SHA256 calls SHA256() twice which is very expensive. Therefore,
this commit uses Poly1305-AES instead of HMAC-SHA256.

benchcmp:

     benchmark                         old ns/op      new ns/op      delta
     BenchmarkChunkEncrypt             261033772      195114818      -25.25%
     BenchmarkChunkEncryptParallel     260973195      195787368      -24.98%
     BenchmarkArchiveDirectory         1050500651     1002615884     -4.56%
     BenchmarkPreload                  23544286       24994508       +6.16%
     BenchmarkLoadTree                 350065         427665         +22.17%
     BenchmarkEncryptWriter            87789753       31069126       -64.61%
     BenchmarkEncrypt                  88283197       38259043       -56.66%
     BenchmarkDecryptReader            90478843       40714818       -55.00%
     BenchmarkEncryptDecryptReader     179917626      81231730       -54.85%
     BenchmarkDecrypt                  87871591       37784207       -57.00%
     BenchmarkSaveJSON                 52481          56861          +8.35%
     BenchmarkSaveFrom                 75404085       51108596       -32.22%
     BenchmarkLoadJSONID               90545437       82696805       -8.67%

     benchmark                         old MB/s     new MB/s     speedup
     BenchmarkChunkEncrypt             40.17        53.74        1.34x
     BenchmarkChunkEncryptParallel     40.18        53.56        1.33x
     BenchmarkEncryptWriter            95.55        270.00       2.83x
     BenchmarkEncrypt                  95.02        219.26       2.31x
     BenchmarkDecryptReader            92.71        206.03       2.22x
     BenchmarkEncryptDecryptReader     46.62        103.27       2.22x
     BenchmarkDecrypt                  95.46        222.01       2.33x
     BenchmarkSaveFrom                 55.62        82.07        1.48x

     benchmark                         old allocs     new allocs     delta
     BenchmarkChunkEncrypt             112            110            -1.79%
     BenchmarkChunkEncryptParallel     103            100            -2.91%
     BenchmarkArchiveDirectory         383704         392083         +2.18%
     BenchmarkPreload                  21765          21874          +0.50%
     BenchmarkLoadTree                 341            436            +27.86%
     BenchmarkEncryptWriter            20             17             -15.00%
     BenchmarkEncrypt                  14             13             -7.14%
     BenchmarkDecryptReader            18             15             -16.67%
     BenchmarkEncryptDecryptReader     46             39             -15.22%
     BenchmarkDecrypt                  16             12             -25.00%
     BenchmarkSaveJSON                 81             86             +6.17%
     BenchmarkSaveFrom                 117            121            +3.42%
     BenchmarkLoadJSONID               80525          80264          -0.32%

     benchmark                         old bytes     new bytes     delta
     BenchmarkChunkEncrypt             118956        64697         -45.61%
     BenchmarkChunkEncryptParallel     118972        64681         -45.63%
     BenchmarkArchiveDirectory         160236600     177498232     +10.77%
     BenchmarkPreload                  2772488       3302992       +19.13%
     BenchmarkLoadTree                 49102         46484         -5.33%
     BenchmarkEncryptWriter            28927         8388146       +28897.64%
     BenchmarkEncrypt                  2473          1950          -21.15%
     BenchmarkDecryptReader            527827        2774          -99.47%
     BenchmarkEncryptDecryptReader     4100875       1528036       -62.74%
     BenchmarkDecrypt                  2509          2154          -14.15%
     BenchmarkSaveJSON                 4971          5892          +18.53%
     BenchmarkSaveFrom                 40117         31742         -20.88%
     BenchmarkLoadJSONID               9444217       9442106       -0.02%

This closes #102.
2015-03-14 20:00:41 +01:00