Commit Graph

760 Commits

Author SHA1 Message Date
Andrew Gaul
8945e98d8b Support Google Cloud Storage headers
This allows s3fs to interpret objects created by gsutil.
2020-05-03 18:33:13 +09:00
Andrew Gaul
2cc88b933f Warn about missing MIME types instead of exiting
s3fs uses the MIME types file to set Content-Type for uploaded
objects.  Most distribution packages should install this via
recommended (not required) dependencies.  Users compiling from source
may not have this installed and s3fs should not prevent launching
since most users do not care about Content-Type.  Instead warn about
MIME types absence.  Fixes #1270.
2020-04-29 20:03:50 +09:00
Andrew Gaul
005a684600 Fix typos 2020-04-22 21:49:11 +09:00
Takeshi Nakatani
f26a0aa71d Fixed insufficient upload size for mix multipart upload 2020-04-22 09:31:22 +09:00
Andrew Gaul
cf529e0af7 Add handler for HTTP 416
This prevents retries when the server indicates an unsatisfiable MPU
copy part request.  References #1220.
2020-04-21 19:45:10 +09:00
Takeshi Nakatani
4da02d023b Improved strictness of cache file stats(file) 2020-04-21 19:45:03 +09:00
Takeshi Nakatani
811ea0cb85 Fixed the truncation bug of stat file for cache file 2020-04-19 07:08:49 +00:00
Andrew Gaul
48a872e285 Address cppcheck 1.90 warning 2020-04-12 22:20:44 +09:00
Takeshi Nakatani
c44a60f3f5 Fixed a bug of stats cache compression 2020-04-12 18:33:00 +09:00
Takeshi Nakatani
9e01d5b8d1
Merge pull request #1254 from ggtakec/modify_mimetypes
Added mime option for strict checking of mime types file
2020-04-11 14:48:47 +09:00
Takeshi Nakatani
7fbda230f5 Added mime option for strict checking of mime types file 2020-03-30 14:41:18 +00:00
Andrew Gaul
56141557dc Avoid unneeded string copy
Found by clang-tidy 10.
2020-03-28 08:49:49 +09:00
yongqingliu
b72f4b43a4 use correct content-type when complete multipart upload 2020-02-10 16:58:28 +09:00
Andrew Gaul
924eeb3587 Document host and servicepath
The various bits of host, url, and servicepath seem to overlap.
References #1203.
2020-02-04 21:13:29 +09:00
Andrew Gaul
bc9126d774 Set directory MIME type to application/x-directory
Previously s3fs auto-detected the MIME type of directories like
"TOYOTA TRUCK 8.2.2" as application/x-troff-man.  This caused get_mode
to not set S_IFDIR which failed directory creation.  Instead force all
object names ending in / to application/x-directory.  Fixes #1183.
2020-02-04 20:03:21 +09:00
rallister
4e26728cbf
break recursion when calling GetIAMCredentials (#1233)
break recursion when calling GetIAMCredentials
2020-01-31 16:48:37 +09:00
Takeshi Nakatani
018ccb9a11 Suppressed a lot of message output about cache cleanup 2020-01-30 14:42:28 +00:00
yongqingliu
b762a0a85b fix deadlock due to fdmanager_lock and fdent_data_lock 2020-01-29 12:03:53 +08:00
Andrew Gaul
e0712f444d Update source for clang-tidy 9.0.0 2020-01-13 20:56:45 +09:00
Andrei Precup
39102608aa Add intelligent tiering as option for storage class
Resolves Issue #1219
2020-01-13 20:24:35 +09:00
Takeshi Nakatani
23945a0130 In memory cache for softlinks with cache out 2020-01-13 20:23:10 +09:00
Alvaro Huarte
bdfb9ee815 Document requester_pays option 2019-11-25 22:53:50 +09:00
Alvaro Huarte
a24f78f5a4 Adds requester_pays option to enable requests involving Requester Pays buckets 2019-11-18 12:38:16 +01:00
Takeshi Nakatani
a07a5336f6
Merge pull request #1185 from bryceml/master
handle edge case when bucket name and first part of endpoint match
2019-10-31 23:42:47 +09:00
Bryce Larson
0c42a74a8a
handle edge case when bucket name and first part of endpoint match 2019-10-26 00:51:19 -06:00
Takeshi Nakatani
cf3e82d10a Fixed a rename bug when enable_noobj_cache 2019-10-23 12:39:42 +00:00
Takeshi Nakatani
1db94a0b30 Fixed to use copy api in multipart upload 2019-09-25 19:30:58 -07:00
Takeshi Nakatani
bedd648d47 Fixed build error by cppcheck 1.89 2019-09-23 10:49:49 +00:00
Takeshi Nakatani
58b3cce320
Merge pull request #1066 from gaul/sign-requests
Sign request immediately before sending
2019-09-23 15:58:21 +09:00
Takeshi Nakatani
81102a5963
Merge pull request #1149 from ggtakec/fix_retry
Fixed a bug that type and op are not inherited
2019-09-08 20:32:43 +09:00
Takeshi Nakatani
42fb30852b
Merge pull request #1145 from gaul/bool
Fix a few bool types
2019-09-08 20:04:30 +09:00
Takeshi Nakatani
e51361cb94
Merge pull request #1143 from gaul/doc/umask
Document umask flag
2019-09-08 19:49:58 +09:00
Takeshi Nakatani
e2129001eb Fixed a bug that type and op are not inherited 2019-09-08 09:59:04 +00:00
Andrew Gaul
805cc064af Fix a few bool types
Found via clang-tidy.
2019-09-05 10:42:42 -07:00
Andrew Gaul
3c2279db39 Document umask flag 2019-09-04 20:53:58 -07:00
Andrew Gaul
412876ca33 Expose base parameter in s3fs_strtoofft
This fixes a regression from ccf3e7bfa2
which caused the misparsing of octal inputs for the mp_umask and umask
flags.  It also allows some callers to be more precise about their
decimal inputs.
2019-09-04 20:41:47 -07:00
Andrew Gaul
461a346bf4 Sign request immediately before sending
Previously s3fs could create a long list of pre-signed requests which
could take longer than the default S3 clock skew limit of 15 minutes.
This also changes SHA-256 computation from single- to multi-threaded
since this is now computed in the worker threads.  Regression from
88cd8feb05.  Fixes #1019.
2019-08-29 15:35:27 -07:00
liuyongqing
ae4bcd405c use correct content-type when overwriting and copying files 2019-08-28 22:25:09 -07:00
Takeshi Nakatani
0536dc1112 Refixed strict processing of MultiRead method 2019-08-25 20:46:01 -07:00
Takeshi Nakatani
1c3507ede1
Merge pull request #1135 from gaul/curl-timeout
Remove useless CURLE_OPERATION_TIMEDOUT check
2019-08-25 17:53:02 +09:00
Andrew Gaul
01ac815346 Remove useless CURLE_OPERATION_TIMEDOUT check
CheckBucket only returns errno and HTTP status codes.
2019-08-20 18:44:48 -07:00
Andrew Gaul
cc2eed84a5 Promote default ACL to enum
This sanity checks ACLs during initialization and also omits sending
the header when set to PRIVATE which is the default.
2019-08-20 15:05:58 -07:00
Takeshi Nakatani
873e376098 Strict processing of MultiRead method 2019-08-13 15:23:13 +00:00
Takeshi Nakatani
4a0c23258e Fixed MultiRead method to exit after waiting other parts at error 2019-08-12 14:45:06 -07:00
Takeshi Nakatani
5c4a0a862a Fixed multipart copy and its retry handler 2019-08-11 16:21:26 +00:00
Takeshi Nakatani
f5bf41cf11 Fixed simple copy with nocpoyapi and modify flag in fdcache 2019-08-11 07:42:48 +00:00
Takeshi Nakatani
f74c7407db
Merge pull request #1118 from gaul/atoi
Prefer s3fs_strtoofft over atoi and strtol
2019-08-06 21:23:37 +09:00
Takeshi Nakatani
433c04af26
Merge pull request #1116 from gaul/bodydata
Do not heap allocate BodyData
2019-08-06 21:08:02 +09:00
Takeshi Nakatani
2e51a339a9
Merge pull request #1115 from gaul/hash-length
Use length from s3fs_HMAC256 for consistency
2019-08-06 20:49:22 +09:00
Takeshi Nakatani
14d3e12103
Merge pull request #1114 from gaul/flush/return-value
Consume return value from FdEntity::Flush
2019-08-06 14:12:25 +09:00