octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-14 16:34:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,342 Commits 1 Branch 23 Tags 13 MiB
80972aa33d
Commit Graph

253 Commits

Author SHA1 Message Date
Andrew Gaul
bacd15714a Use consistent default for readwrite_timeout 
Previously curl threads mutated this shared state without a lock.
Found via ThreadSanitizer.
 2019-07-16 22:03:56 -07:00
Takeshi Nakatani
543aed2a32
Merge pull request #1096 from gaul/pthread/lock-checking 
Expand error checking to all pthread_mutex_t
 2019-07-15 11:43:56 +09:00
Andrew Gaul
43ec064fb9 Expand error checking to all pthread_mutex_t 2019-07-14 18:21:23 -07:00
Andrew Gaul
ffac4c8417 Avoid narrowing time_t and off_t args in logging 
This displays correct results on 32-bit platforms.
 2019-07-14 17:02:36 -07:00
Takeshi Nakatani
e936854493
Merge pull request #1094 from larsch/master 
Fix multipart upload not setting curl options
 2019-07-15 05:13:37 +09:00
Lars Christensen
58037da061 Fix multipart upload not setting curl options 2019-07-14 14:09:37 +02:00
Andrew Gaul
6fd42d9fe4 Prefer modern C headers 
Found and fixed via clang-tidy.
 2019-07-12 03:50:59 -07:00
Takeshi Nakatani
e5e63d6ac3
Merge pull request #1067 from gaul/mismatched-free 
Fix mismatched new[]/free
 2019-07-07 16:55:07 +09:00
Takeshi Nakatani
7a65a414c3
Merge pull request #1080 from gaul/xml/simplification 
Add simple XML parsing wrapper
 2019-07-07 16:44:18 +09:00
Andrew Gaul
944d21cabb Fix mismatched new[]/free 
Found via Valgrind.
 2019-07-07 00:36:48 -07:00
Andrew Gaul
b8ff6a647e Add simple XML parsing wrapper 
Also simplify check_region_error.
 2019-07-05 17:39:09 -07:00
Andrew Gaul
bbfa91141a Configure clang-tidy target 
Also fix nits.
 2019-07-03 14:04:11 -07:00
Takeshi Nakatani
fbebc6fa57
Merge pull request #1048 from gaul/log/curl 
Emit older curl warnings only once
 2019-06-24 00:42:44 +09:00
Andrew Gaul
738eaadcbf Emit older curl warnings only once 
This makes Travis logs actually readable.
 2019-06-22 10:01:20 -07:00
Andrew Gaul
15b7450713 Convert file offsets from size_t to off_t 
The latter is 64-bits on 32-bit platforms when specifying
-D_FILE_OFFSET_BITS=64.  This allows early Raspberry Pis to use files
larger than 2 GB.  It also cleans up some ugly casting.  Fixes #620.
Fixes #656.
 2019-06-15 17:05:37 -07:00
Michal Lula
0d4847596e fix issue with aws session token provided inside ~/.aws/credentials file 2019-04-25 14:29:35 +02:00
Andrew Gaul
0677a096a9 Prefer simple over compound statements in macros 
This prohibits missing semicolons.
 2019-04-17 21:46:24 +09:00
Michal Lula
f9cd43b684 add session token support 2019-04-16 16:53:05 +02:00
Andrew Gaul
6f8ecb28c5 Correctly calculate MD5 hash for SSE-C keys 
Previously s3fs calculated the strlen of a binary input instead of
using the stored length.  This yielded IO errors when interacting with
SSE-encrypted objects.  Fixes #696.
 2019-04-08 21:49:42 +09:00
Andrew Gaul
6f6a67807b Prefer new over malloc 
The former cannot return NULL and will allow use of scoped pointers.
 2019-04-07 16:54:24 +09:00
Takeshi Nakatani
71766039ff Support undefined CURLoption in libcurl library used in build 2019-03-22 10:47:42 +00:00
Andrew Gaul
1e86cc643d Fix lazy typo 2019-03-09 17:46:02 +09:00
Takeshi Nakatani
10d9f75366 Improvement of curl session pool for multipart 
Improvement of curl session pool for multipart(2)

Improvement of curl session pool for multipart(3)
 2019-02-25 14:46:24 +00:00
Takeshi Nakatani
eeb839242b Fixed code for latest cppcheck error on OSX 2019-02-17 13:59:11 +00:00
Takeshi Nakatani
e07cb020cc
Merge pull request #954 from gaul/clear-iter 
Clear containers instead of individual erases
 2019-02-06 21:12:36 +09:00
Andrew Gaul
a1d3ff9766 Automatically abort failed multipart requests 
This can avoid dangling parts.  However, many transfers fail due to
network errors so we still need other mechanisms to handle these
parts.
 2019-02-03 10:29:20 -08:00
Andrew Gaul
4d0bef1e90 Clear containers instead of individual erases 
This has O(n) runtime instead of O(n log n).
 2019-02-02 23:58:43 -08:00
Takeshi Nakatani
c04e8e7a9d
Merge pull request #949 from gaul/503-slow-down 
Implement exponential backoff for 503
 2019-02-03 15:19:28 +09:00
Takeshi Nakatani
d8185a25aa
Merge pull request #946 from gaul/async/completed-tids 
Simplify async request completion code
 2019-02-03 14:08:49 +09:00
Takeshi Nakatani
53337a0a28
Merge pull request #944 from gaul/utility-mode 
Repair utility mode
 2019-02-03 13:59:56 +09:00
Takeshi Nakatani
055ecf6ea7
Merge pull request #940 from gaul/parallel-multipart-copy 
Copy parts in parallel
 2019-02-03 11:44:15 +09:00
Takeshi Nakatani
c603680e02
Merge pull request #939 from gaul/stringstream-specific 
Prefer specific [io]stringstream where possible
 2019-02-03 11:06:57 +09:00
Andrew Gaul
8ff05d8e38 Implement exponential backoff for 503 
Amazon returns SlowDown when overloaded.  Also return ENOTSUP for 501
and immediately return EIO for 500 instead of retrying.  Fixes #603.
 2019-02-02 00:03:50 -08:00
Andrew Gaul
8c527c3616 Simplify async request completion code 
Workers now notify the master thread when they complete, unifying the
Linux and macOS code paths.  This also avoids excessive
pthread_tryjoin_np calls.  Follows on to
88cd8feb05.
 2019-01-31 22:55:23 -08:00
Andrew Gaul
54a074647e Repair utility mode 
This deinitialized S3fsCurl twice and incorrectly calculated V4
signatures.
 2019-01-31 18:45:39 -08:00
Andrew Gaul
c5ebf5d328 Copy parts in parallel 
S3 can copy multipart much faster than single part due to IO
parallelization.  Renaming a 4 GB file reduces from 72 to 20 seconds
with bigger gains with larger files.
 2019-01-31 10:21:39 -08:00
Andrew Gaul
07636c8a8d Prefer specific [io]stringstream where possible 
These better communicate intent and are slightly more efficient.
 2019-01-29 10:44:33 -08:00
Andrew Gaul
35d55ee513 Remove unneeded void parameter 
This is implicit in C++.  Found and fixed via clang-tidy.
 2019-01-28 23:22:27 -08:00
Takeshi Nakatani
a0c1f30ae7
Merge pull request #932 from gaul/autolock 
Prefer AutoLock for synchronization
 2019-01-27 15:59:18 +09:00
Andrew Gaul
92d3114584 Prefer AutoLock for synchronization 
This simplifies the code and fixes an issue with unlocked access.
Also use a recursive lock for StatCache to avoid races between
lock..unlock..lock sequences.
 2019-01-25 15:28:41 -08:00
Andrew Gaul
84b421d6ef Prefer empty over size checks 
Found and fixed via clang-tidy.
 2019-01-23 11:30:28 -08:00
Takeshi Nakatani
010276ceab
Merge pull request #921 from gaul/clang-tidy/redundant-string-init 
Remove redundant string initializations
 2019-01-23 19:44:59 +09:00
Andrew Gaul
1fc25e8c3f Remove redundant string initializations 
Found and fixed via clang-tidy.
 2019-01-22 23:16:37 -08:00
Andrew Gaul
61ecafd426 Remove unnecessary string copies 
Found via clang-tidy.
 2019-01-22 23:09:37 -08:00
Takeshi Nakatani
fada95f58e
Merge pull request #914 from gaul/readdir/head-of-line 
Issue readdir HEAD requests without batching
 2019-01-21 22:10:50 +09:00
Takeshi Nakatani
014b8c5982
Merge pull request #913 from gaul/assert 
Prefer abort over assert(false)
 2019-01-21 21:56:41 +09:00
Andrew Gaul
46d79c5bc2 Issue readdir HEAD requests without batching 
Previously s3fs would issue a batch of HEAD requests and wait for all
to succeed before issuing the next batch.  Now it issues the first
batch and only waits for a single call to succeed before issuing the
next call.  This can improve performance when one call lags due to
network errors.  I measured 25% improvement with the same level of
parallelism.  This commit also reparents parallelism knobs for
consistency.  Follows on to 88cd8feb05.
Fixes #223.
 2019-01-20 18:07:22 -08:00
Andrew Gaul
40ba3b44a1 Prefer abort over assert(false) 
The compiler can remove the latter when compiled with NDEBUG which may
cause unintended control flow.
 2019-01-20 12:30:27 -08:00
Takeshi Nakatani
ac72431195 Added detail error message when HTTP 301/307 status 2019-01-20 10:07:58 +00:00
Takeshi Nakatani
36a4903843 Added an error message when HTTP 301 status 2019-01-20 07:17:40 +00:00