Commit Graph

734 Commits

Author SHA1 Message Date
Takeshi Nakatani
1db94a0b30 Fixed to use copy api in multipart upload 2019-09-25 19:30:58 -07:00
Takeshi Nakatani
bedd648d47 Fixed build error by cppcheck 1.89 2019-09-23 10:49:49 +00:00
Takeshi Nakatani
58b3cce320
Merge pull request #1066 from gaul/sign-requests
Sign request immediately before sending
2019-09-23 15:58:21 +09:00
Takeshi Nakatani
81102a5963
Merge pull request #1149 from ggtakec/fix_retry
Fixed a bug that type and op are not inherited
2019-09-08 20:32:43 +09:00
Takeshi Nakatani
42fb30852b
Merge pull request #1145 from gaul/bool
Fix a few bool types
2019-09-08 20:04:30 +09:00
Takeshi Nakatani
e51361cb94
Merge pull request #1143 from gaul/doc/umask
Document umask flag
2019-09-08 19:49:58 +09:00
Takeshi Nakatani
e2129001eb Fixed a bug that type and op are not inherited 2019-09-08 09:59:04 +00:00
Andrew Gaul
805cc064af Fix a few bool types
Found via clang-tidy.
2019-09-05 10:42:42 -07:00
Andrew Gaul
3c2279db39 Document umask flag 2019-09-04 20:53:58 -07:00
Andrew Gaul
412876ca33 Expose base parameter in s3fs_strtoofft
This fixes a regression from ccf3e7bfa2
which caused the misparsing of octal inputs for the mp_umask and umask
flags.  It also allows some callers to be more precise about their
decimal inputs.
2019-09-04 20:41:47 -07:00
Andrew Gaul
461a346bf4 Sign request immediately before sending
Previously s3fs could create a long list of pre-signed requests which
could take longer than the default S3 clock skew limit of 15 minutes.
This also changes SHA-256 computation from single- to multi-threaded
since this is now computed in the worker threads.  Regression from
88cd8feb05.  Fixes #1019.
2019-08-29 15:35:27 -07:00
liuyongqing
ae4bcd405c use correct content-type when overwriting and copying files 2019-08-28 22:25:09 -07:00
Takeshi Nakatani
0536dc1112 Refixed strict processing of MultiRead method 2019-08-25 20:46:01 -07:00
Takeshi Nakatani
1c3507ede1
Merge pull request #1135 from gaul/curl-timeout
Remove useless CURLE_OPERATION_TIMEDOUT check
2019-08-25 17:53:02 +09:00
Andrew Gaul
01ac815346 Remove useless CURLE_OPERATION_TIMEDOUT check
CheckBucket only returns errno and HTTP status codes.
2019-08-20 18:44:48 -07:00
Andrew Gaul
cc2eed84a5 Promote default ACL to enum
This sanity checks ACLs during initialization and also omits sending
the header when set to PRIVATE which is the default.
2019-08-20 15:05:58 -07:00
Takeshi Nakatani
873e376098 Strict processing of MultiRead method 2019-08-13 15:23:13 +00:00
Takeshi Nakatani
4a0c23258e Fixed MultiRead method to exit after waiting other parts at error 2019-08-12 14:45:06 -07:00
Takeshi Nakatani
5c4a0a862a Fixed multipart copy and its retry handler 2019-08-11 16:21:26 +00:00
Takeshi Nakatani
f5bf41cf11 Fixed simple copy with nocpoyapi and modify flag in fdcache 2019-08-11 07:42:48 +00:00
Takeshi Nakatani
f74c7407db
Merge pull request #1118 from gaul/atoi
Prefer s3fs_strtoofft over atoi and strtol
2019-08-06 21:23:37 +09:00
Takeshi Nakatani
433c04af26
Merge pull request #1116 from gaul/bodydata
Do not heap allocate BodyData
2019-08-06 21:08:02 +09:00
Takeshi Nakatani
2e51a339a9
Merge pull request #1115 from gaul/hash-length
Use length from s3fs_HMAC256 for consistency
2019-08-06 20:49:22 +09:00
Takeshi Nakatani
14d3e12103
Merge pull request #1114 from gaul/flush/return-value
Consume return value from FdEntity::Flush
2019-08-06 14:12:25 +09:00
Andrew Gaul
ccf3e7bfa2 Prefer s3fs_strtoofft over atoi and strtol
The former propagates errors consistently.
2019-08-03 16:13:48 -07:00
Andrew Gaul
7ecfba811e Do not heap allocate BodyData
Callers already manage lifetimes via Clear().
2019-08-03 15:50:46 -07:00
Takeshi Nakatani
1e7330e499
Merge pull request #1113 from gaul/unused
Remove unused field
2019-08-04 01:31:56 +09:00
Takeshi Nakatani
3d42d0515d
Merge pull request #1109 from gaul/lock-already-held
Pass lock_already_held state to DelStat
2019-08-03 22:13:22 +09:00
Takeshi Nakatani
3d70e8966f
Merge pull request #1108 from gaul/lock/diskspace
Always hold lock when accessing free_disk_space
2019-08-03 22:01:31 +09:00
Andrew Gaul
a5c20175a1 Use length from s3fs_HMAC256 for consistency
EVP_MAX_MD_SIZE can range up to 64.  Found by Coverity.
2019-08-02 10:40:55 -07:00
Andrew Gaul
43d1439420 Consume return value from FdEntity::Flush
Found via Coverity.
2019-08-02 10:01:06 -07:00
Andrew Gaul
d8cf26bd50 Remove unused field
Found via clang.
2019-08-01 16:42:38 -07:00
Andrew Gaul
f528a86219 Pass lock_already_held state to DelStat 2019-08-01 11:07:56 -07:00
Andrew Gaul
5b15c7c4e9 Always hold lock when accessing free_disk_space
Slightly reorder locks to avoid double locking.  Found via
ThreadSanitizer.
2019-07-30 08:36:54 -07:00
Andrew Gaul
afd438d363 Access FdEntity metadata while holding fdent_lock
Create copies to avoid thread races.  Found via ThreadSanitizer.
Follows on to ecf13a8cb9.
2019-07-29 13:54:57 -07:00
Andrew Gaul
fb937635f5 Eagerly initialize sysconf variables
Previously s3fs had races updating these shared variables.  Found via
ThreadSanitizer.
2019-07-17 09:08:13 -07:00
Takeshi Nakatani
3ad1c95e86
Merge pull request #1103 from gaul/config/readwrite-timeout
Use consistent default for readwrite_timeout
2019-07-17 22:37:02 +09:00
Andrew Gaul
bacd15714a Use consistent default for readwrite_timeout
Previously curl threads mutated this shared state without a lock.
Found via ThreadSanitizer.
2019-07-16 22:03:56 -07:00
Andrew Gaul
99aace4fc9 Avoid race when using thread-unsafe gmtime
Found via ThreadSanitizer.
2019-07-16 07:28:34 -07:00
Takeshi Nakatani
513f41fddf
Merge pull request #1097 from ggtakec/fix_lock
Add Lock() func parameter in RowFlush()
2019-07-15 11:56:21 +09:00
Takeshi Nakatani
543aed2a32
Merge pull request #1096 from gaul/pthread/lock-checking
Expand error checking to all pthread_mutex_t
2019-07-15 11:43:56 +09:00
Takeshi Nakatani
20ea96328c
Merge pull request #1095 from gaul/intmax_t
Avoid narrowing time_t and off_t args in logging
2019-07-15 11:33:44 +09:00
Takeshi Nakatani
f78bcc5229 Add Lock() func parameter in RowFlush() 2019-07-15 01:52:29 +00:00
Andrew Gaul
43ec064fb9 Expand error checking to all pthread_mutex_t 2019-07-14 18:21:23 -07:00
Andrew Gaul
ffac4c8417 Avoid narrowing time_t and off_t args in logging
This displays correct results on 32-bit platforms.
2019-07-14 17:02:36 -07:00
Andrew Gaul
4adcd4a6c8 Implement s3fs_strtoofft via strtoll
This tightens error checking and aligns s3fs with known good behavior.
2019-07-14 15:15:06 -07:00
Takeshi Nakatani
e936854493
Merge pull request #1094 from larsch/master
Fix multipart upload not setting curl options
2019-07-15 05:13:37 +09:00
Takeshi Nakatani
850a813171
Merge pull request #1092 from liuyongqing/master
fix coredump caused by fd_manager_lock locking
2019-07-15 04:53:10 +09:00
Takeshi Nakatani
5bbcd3b981
Merge pull request #1089 from gaul/null-dereference
Avoid null dereference
2019-07-15 04:17:10 +09:00
Takeshi Nakatani
a337c32840
Merge pull request #1088 from gaul/clang-tidy/misplaced-const
Avoid misplaced const via removing unneeded typedef
2019-07-15 04:08:51 +09:00