526700f2de
Updated files for support ALPINE v3.17 ( #2116 )
2023-03-05 13:30:59 +09:00
f8a825e9b9
multipart upload id is converted by url encode ( #2097 )
2023-02-18 11:40:25 +09:00
d1388ff446
Added proxy and proxy_cred_file option
2023-02-08 21:53:34 +09:00
38e8a830c9
Fixed a deadlock bug when specifying iam role(IAMv2)
2023-01-31 20:53:39 +09:00
16bc44948e
Retry BucketCheck containing directory paths
2023-01-09 21:18:47 +09:00
e4f85c1e08
Fixed renaming bug with SSE
2023-01-09 21:17:48 +09:00
222110e153
Fixed a bug about multipart upload with SSE
2023-01-07 09:12:55 +05:30
0ba49518e9
Make some methods and parameters const ( #2078 )
...
This requires making some locks mutable.
2023-01-04 20:23:39 +09:00
aec7efc3af
Updated comments in curl.cpp (related to #1827 ) ( #2052 )
2022-11-05 11:45:28 +09:00
30353f1a83
Remove unnecessary calls to dup ( #1827 )
...
These functions already use pread and keep track of an explicit offset
for the SHA-256 calculations.
2022-11-05 10:24:41 +09:00
c491fbeabc
Replace uses of lock_already_held flag with AutoLock::Type
2022-07-30 16:20:43 +09:00
48e9e51f4f
Remove more unneeded headers identified by IWYU ( #2011 )
2022-07-30 12:06:47 +09:00
01a92476e6
Remove unneeded headers identified by IWYU ( #2007 )
2022-07-28 23:38:38 +09:00
3928a7e359
Remove more uses of const_cast ( #2006 )
...
Follows on to #2004 .
2022-07-28 23:37:15 +09:00
e30a5939d0
Fixed deadlock in S3fsCurl::DestroyCurlHandle
2022-07-28 13:47:27 +09:00
ccfc119e45
Improved to avoid unnecessary head request
2022-07-27 21:51:28 +09:00
d22e1dc018
Add the stream upload which starts uploading parts before Flush
2022-07-17 22:20:45 +09:00
92fd5bc3e1
Fixed errors reported by cppcheck 2.8 ( #1949 )
2022-05-26 01:48:53 +09:00
684ced5a41
Changed handling the credential in S3fsCred more robust
2022-03-02 22:39:15 +09:00
1678803566
Added S3fsCred class and moved Credential related processing in it
2022-02-13 21:38:30 +09:00
6edb6067f3
Remove strcasestr polyfill
2022-01-23 21:49:51 +09:00
b14e39815b
Use polyfills in MSYS2 environment
2022-01-23 21:49:51 +09:00
93d1c30d4d
Use XML parsing with PUT HTTP 200 responses ( #1858 )
...
This works around the missing strcasestr on win32. References #728 .
2022-01-14 16:10:22 +09:00
85ca2a3e45
fix mixupload return EntityTooSmall while a copypart is less than 5MB after split ( #1809 )
...
* fix mixupload return EntityTooSmall while a copypart is less than 5MB after split
* fix possible part exceeds 5GB when multipart_copy_size is set to 5120MB
* Update curl.cpp
Co-authored-by: liubingrun <liubr1@chinatelecom.cn>
2021-11-27 16:53:26 +09:00
3cf00626a2
Add option to allow unsigned payloads ( #1801 )
...
This reduces CPU usage of sigv4. This reduces test run-time by 7
seconds per flag.
2021-11-01 23:33:55 +09:00
2f412804e2
Fixed forgetting to clear the dirty flag for meta information
...
Addressed an error in macos cpp check
2021-10-15 22:54:55 +09:00
b4edad86d6
remove Expect: 100-continue header when requesting an IMDSv2 access token
2021-09-09 08:12:36 +09:00
9d1552a54e
fix IAM role retrieval from IMDSv2
...
AWS IMDSv2 support was added in #1462 , but the implementation did not
cover the addional IMDS access that occurs with the iam_role=auto
configuration. This change implements IMDSv2 support for the IMDS
call to determine the instance's role name.
See also
https://stackoverflow.com/questions/69031023/how-to-make-s3fs-use-imds-v2-when-mounting-s3-buckets-from-ec2-instance
2021-09-03 20:36:34 +09:00
47ebfcc60a
Consume return value from curl_easy_setopt ( #1759 )
...
Found via Coverity.
2021-09-02 08:07:06 +09:00
57b2e4a4f1
Fix 32-bit compilation issues ( #1757 )
2021-08-31 19:36:02 +09:00
48817d849f
Require explicit length in s3fs_decode64 ( #1755 )
...
This is available from std::string::size in callers.
2021-08-31 09:22:10 +09:00
a868c0656e
Changed etaglist_t from string list to new structure etagpairs list
2021-08-16 09:27:12 +09:00
66006ba48d
Add dedicated upper- and lower-case hex functions ( #1734 )
...
This makes the call sites more readable than a boolean parameter.
2021-08-04 07:28:51 +09:00
18e9c62087
Make string constants read-only const ( #1733 )
...
This removes some global constructors. Also use a consistent ALL_CAPS
style.
2021-08-03 00:10:27 +09:00
d3278f4886
Loosen CheckBucket to check only the bucket ( #1728 )
...
Previously it checked if there was an object present, assuming that
this was a directory object normally created for s3fs directories.
However most S3 clients do not create this object for virtual folders.
Fixes #1460 . Fixes #1687 .
2021-07-25 12:29:00 +09:00
77f0b75d2f
Add jitter to avoid thundering herd ( #1723 )
2021-07-25 10:22:19 +09:00
7890989cbb
Address cppcheck 2.5 warnings ( #1720 )
...
This slightly simplifies memory management.
2021-07-14 22:18:09 +09:00
487df27008
Return EREMOTE when reading Glacier objects ( #1705 )
...
Previously s3fs returned EIO which was not helpful to the caller.
Returning a more specific error code allows automatically restoring
the object via RestoreObject in a subsequent commit.
References #1466 .
2021-06-30 09:25:36 +09:00
1965916f7a
Convert storage class values to uppercase ( #1704 )
...
AWS requires uppercase values. Fixes a regression from
bbcccd6e98#1613 .
2021-06-30 09:03:31 +09:00
2154e898bc
Fix typos ( #1700 )
2021-06-27 11:22:33 +09:00
a5c1915772
Set CURLOPT_UNRESTRICTED_AUTH when authenticating ( #1681 )
...
This is necessary for authentication to work with AWS when it responds with 307 to the check bucket request when mounting. This happens to newly created buckets according to https://aws.amazon.com/premiumsupport/knowledge-center/s3-http-307-response/ .
Prior to this, curl would follow the redirect, but would not include the `Authorization` header which would end up with `404`.
Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
2021-06-25 20:26:12 +09:00
20281737b1
Set explicit Content-Length: 0 when initiating MPU ( #1696 )
...
This works around a GCS quirk that incorrectly requires this header.
Fixes #1661 .
2021-06-25 18:36:40 +09:00
b9ce0faee2
Set Content-MD5 header even for empty objects
...
The Content-MD5 header is required for any request to upload an object
with a retention period configured using Amazon S3 Object Lock
2021-06-19 16:10:58 +09:00
a100be9dce
Prefer std::string::empty over length == 0 ( #1679 )
...
This is more concise.
2021-06-13 20:03:10 +09:00
600cee118d
Fix implicit narrowing conversions ( #1672 )
...
These do not appear to be problematic but rather just clean up warnings.
Found via clang -Wshorten-64-to-32.
2021-06-13 12:50:07 +09:00
c2c56d0263
Added info object about multipart uploading for each pseudo fd
...
(and fixed typo about method name)
2021-06-04 22:42:58 +09:00
ac578d188e
Introduced pseudo fd and separated fd for each file opening
2021-06-04 22:42:58 +09:00
53dfd48f59
Correct usage of istringstream ( #1670 )
...
Previously this looped one more time than necessary due to the eof
check:
https://isocpp.org/wiki/faq/input-output#istream-and-eof
Remove now redundant empty check.
2021-06-02 07:14:32 +09:00
bbcccd6e98
Make storage class a string ( #1663 )
...
This allows non-standard storage classes like Google Cloud Storage
Nearline. Fixes #1613 .
2021-05-21 23:34:31 +09:00
bb6d2b1b74
Replace snprintf with string and ostringstream ( #1649 )
...
These uses are probably safe from a buffer overflow perspective but
can cause data race issues in logging due to static buffers.
2021-05-08 02:48:47 +09:00
Takeshi Nakatani
Andrew Gaul
Naoki Ikeguchi
LiuBingrun
Kamil Jakrzewski
Noah Meyerhans
Jan Stastny
Dmitrii Vasilev