octoleo/restic
2
2
Fork 0
mirror of https://github.com/octoleo/restic.git synced 2024-11-02 11:46:36 +00:00
Code Issues Releases Wiki Activity
2,516 Commits 7 Branches 33 Tags 66 MiB
47282abfa4
Commit Graph

2516 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexander Neumann
55ae5dab2b design: Add s3.layout option 2017-05-15 23:37:16 +02:00
Alexander Neumann
fa41183a53 s3: Add s3.layout option and layout auto detection 2017-05-15 23:37:02 +02:00
Alexander Neumann
959aa0f595 Simplify layout detection 2017-05-15 23:36:23 +02:00
Alexander Neumann
069752cb42 Make layout default Dirname() consistent 
Always return a trailing slash now.
 2017-05-15 23:35:52 +02:00
Alexander Neumann
0c537837d9 Rename testdata files 2017-05-15 22:39:50 +02:00
Alexander Neumann
f19852a738 Rename CloudLayout -> RESTLayout 
The REST backend uses a special layout without subdirs below data/. This
layout is just used there and nowhere else, and our REST server
implementation uses the default layout for the on disk storage. So we
remove the REST layout from the auto detection code.
 2017-05-15 22:05:43 +02:00
Alexander Neumann
61cade6222 Rename layout s3 -> s3legacy 2017-05-15 21:59:28 +02:00
Alexander Neumann
a4e3a0dd97 design: Remove cloud layout 2017-05-15 21:51:19 +02:00
Alexander Neumann
22a6cd3a26 Merge pull request #964 from restic/update-minio-go 
Update github.com/minio/minio-go
 2017-05-15 21:15:28 +02:00
Alexander Neumann
7f10828dfa Update github.com/minio/minio-go 2017-05-15 20:47:50 +02:00
Alexander Neumann
04264c6338 Remove doc/PKGBUILD 2017-05-15 19:48:57 +02:00
Alexander Neumann
13393c76dc Merge pull request #957 from middelink/fix-953 
Change backup policy to be inclusive
 2017-05-15 17:19:04 +02:00
Alexander Neumann
13c12ca83d Fix PKGBUILD 
Closes #963
 2017-05-15 17:17:23 +02:00
Pauline Middelink
7fffd408af Do not use singleton counter. 
Revert change for running the numbered tests as subtests.
 2017-05-15 08:53:59 +02:00
Alexander Neumann
089b04c8a9 Merge pull request #962 from restic/improve-s3-partial-download 
Improve s3 backend
 2017-05-14 22:00:16 +02:00
Alexander Neumann
b192dacc13 Add text to the issue template 2017-05-14 21:37:45 +02:00
Alexander Neumann
250a45ab15 s3: Move interfaces to function 2017-05-14 20:36:26 +02:00
Alexander Neumann
3b44b87137 s3: Remove file after usage in test 2017-05-14 20:34:22 +02:00
Alexander Neumann
26c16b9fd3 s3: Correctly return token in Load 2017-05-14 20:19:12 +02:00
Alexander Neumann
0bd40bae6e s3: Prevent net/http client from closing the reader 2017-05-14 20:19:12 +02:00
Alexander Neumann
246ccf09b9 s3: add getRemainingSize 2017-05-14 20:19:12 +02:00
Alexander Neumann
9452f416bf s3: Use low level API for saving files 
benchmark                        old ns/op       new ns/op       delta
    BenchmarkBackendMinio/Save-4     184482294       40663344        -77.96%
    BenchmarkBackendS3/Save-4        35030825568     54475455819     +55.51%

    benchmark                        old MB/s     new MB/s     speedup
    BenchmarkBackendMinio/Save-4     90.95        412.64       4.54x
    BenchmarkBackendS3/Save-4        0.48         0.31         0.65x

    benchmark                        old allocs     new allocs     delta
    BenchmarkBackendMinio/Save-4     631            560            -11.25%
    BenchmarkBackendS3/Save-4        646            584            -9.60%

    benchmark                        old bytes     new bytes     delta
    BenchmarkBackendMinio/Save-4     66818060      50735         -99.92%
    BenchmarkBackendS3/Save-4        66834000      73024         -99.89%
 2017-05-14 20:19:12 +02:00
Alexander Neumann
1e0e6ee573 s3: Use low-level API with a Range header for Load 
benchmark                                         old ns/op      new ns/op      delta
    BenchmarkBackendMinio/LoadFile-4                  9213315        11001787       +19.41%
    BenchmarkBackendMinio/LoadPartialFile-4           4176619        3479707        -16.69%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     4391521        3139214        -28.52%
    BenchmarkBackendS3/LoadFile-4                     2886070905     2505907501     -13.17%
    BenchmarkBackendS3/LoadPartialFile-4              762702722      735694398      -3.54%
    BenchmarkBackendS3/LoadPartialFileOffset-4        789724328      1108989142     +40.43%

    benchmark                                         old MB/s     new MB/s     speedup
    BenchmarkBackendMinio/LoadFile-4                  1821.21      1525.15      0.84x
    BenchmarkBackendMinio/LoadPartialFile-4           1004.49      1205.67      1.20x
    BenchmarkBackendMinio/LoadPartialFileOffset-4     955.34       1336.45      1.40x
    BenchmarkBackendS3/LoadFile-4                     5.81         6.70         1.15x
    BenchmarkBackendS3/LoadPartialFile-4              5.50         5.70         1.04x
    BenchmarkBackendS3/LoadPartialFileOffset-4        5.31         3.78         0.71x

    benchmark                                         old allocs     new allocs     delta
    BenchmarkBackendMinio/LoadFile-4                  406            204            -49.75%
    BenchmarkBackendMinio/LoadPartialFile-4           225            206            -8.44%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     227            207            -8.81%
    BenchmarkBackendS3/LoadFile-4                     600            388            -35.33%
    BenchmarkBackendS3/LoadPartialFile-4              416            302            -27.40%
    BenchmarkBackendS3/LoadPartialFileOffset-4        417            303            -27.34%

    benchmark                                         old bytes     new bytes     delta
    BenchmarkBackendMinio/LoadFile-4                  29475         13904         -52.83%
    BenchmarkBackendMinio/LoadPartialFile-4           4218838       13958         -99.67%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     4219175       14332         -99.66%
    BenchmarkBackendS3/LoadFile-4                     114152        97424         -14.65%
    BenchmarkBackendS3/LoadPartialFile-4              4265416       56212         -98.68%
    BenchmarkBackendS3/LoadPartialFileOffset-4        4266520       56308         -98.68%
 2017-05-14 20:19:12 +02:00
Alexander Neumann
be0e53c07b tests: Add test for backend Save() from file 2017-05-14 19:55:32 +02:00
Alexander Neumann
9dd58196e3 Merge pull request #961 from restic/add-backend-benchmarks 
Add backend benchmarks
 2017-05-14 19:49:33 +02:00
Alexander Neumann
66b4999765 tests: Remove code generation, use reflection 
This simplifies the code.
 2017-05-14 12:59:07 +02:00
Alexander Neumann
403e201e1a tests: Improve robustness of config tests 2017-05-14 12:50:20 +02:00
Alexander Neumann
77a55fbe5c tests: Add documentation 2017-05-14 11:48:30 +02:00
Alexander Neumann
90c1608d88 sftp: Add Benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann
7a51640262 rest: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann
13946e7db7 mem: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann
e009c002ba local: Add benchmark 2017-05-14 11:48:30 +02:00
Alexander Neumann
d24e0cc6cc s3: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann
5b8131e2d3 tests: Add benchmark for Save 2017-05-14 11:48:30 +02:00
Alexander Neumann
8fc25cc567 tests: Add benchmarks for partial file (+offset) 2017-05-14 11:48:30 +02:00
Alexander Neumann
77ebb95d3d tests: Add BenchmarkLoadFile 2017-05-14 11:48:30 +02:00
Alexander Neumann
f142b1c22f tests: Add benchmarks to test suite 2017-05-14 11:48:30 +02:00
Alexander Neumann
1c9159d6a0 Merge pull request #960 from restic/travis-run-tests-against-s3 
Enable CI tests against AWS S3
 2017-05-14 10:26:16 +02:00
Alexander Neumann
82e15dc6dc Allow skipping the test against s3 
If RESTIC_TEST_S3_REPOSITORY is not available it means we're probably
running on Travis for a third-party PR, so the access credentials for S3
are not exposed.
 2017-05-14 00:33:19 +02:00
Alexander Neumann
ee68f9298b tests: Ensure that backend tests cannot be skipped on Travis 2017-05-14 00:33:15 +02:00
Alexander Neumann
5c6ec78789 s3: Remove log line 2017-05-13 19:58:25 +02:00
Alexander Neumann
4ac0d3ad40 backend tests load: Use reader with Size() method 2017-05-13 19:56:11 +02:00
Harshavardhana
c7209ef231 api: Failed call retry with region only when http.StatusBadRequest. 
Fixes https://github.com/minio/minio-go/issues/677
 2017-05-13 19:53:24 +02:00
Alexander Neumann
0249c16b04 Revert "minio: Apply fix" 
This reverts commit f43d34899d.
 2017-05-13 19:41:55 +02:00
Alexander Neumann
f43d34899d minio: Apply fix 
https://github.com/minio/minio-go/issues/677
 2017-05-13 13:18:42 +02:00
Alexander Neumann
19daefd04e Merge pull request #938 from restic/rework-backend-tests 
WIP: rework backend integration tests
 2017-05-12 22:09:57 +02:00
Alexander Neumann
ec5e984ed9 Increase waiting time for network services 2017-05-12 21:45:18 +02:00
Alexander Neumann
88de3cfecc rest: Wait until REST server is started 2017-05-12 21:33:34 +02:00
Alexander Neumann
a654f41ddb Add docstring for package test 2017-05-12 21:19:32 +02:00
Alexander Neumann
fbf2462325 Rename backend test functions 2017-05-12 21:06:49 +02:00