octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2025-01-28 16:18:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,780 Commits 1 Branch 23 Tags
Commit Graph

966 Commits

Author SHA1 Message Date
Andrew Gaul
543231c9f2
Address warnings from clang-tidy 11 (#1470) 2020-11-09 21:15:20 +09:00
Noah Meyerhans
d96a08d4ad fixup! Implement AWS IMDSv2 support 2020-11-07 14:48:52 +09:00
Noah Meyerhans
f2f930300a Implement AWS IMDSv2 support 
AWS IMDSv2 is a session oriented method for retrieving instance metadata,
including IAM credentials, in Amazon EC2.  It is enabled by default in
non-enforcing mode in AWS (meaning it retains backwards compatibility with
existing IMDSv1 clients), but can be switched to enforcing mode, in which
clients are required to return API tokens with requests.

With this change, we implement support for IMDSv2 and enable it by default when
IAM roles are our source for authentication credentials.  In the event that
s3fs is running in cloud environment offering an IMDSv1-compatible API, we
support graceful fallback to that mode.  It can also be selected explicitly via
the imdsv1only mount option.

More details on IMDSv2 are available at
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
and
https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/

Signed-off-by: Noah Meyerhans <nmeyerha@amazon.com>
 2020-11-07 14:48:52 +09:00
yoshihiko
81ad3ce0ae Fix directory authority 2020-11-07 14:47:53 +09:00
Takeshi Nakatani
38e1eaa8a3 Added logfile option for non-syslog logging 2020-10-19 20:10:19 +09:00
Andrew Gaul
6aa786b886
Flush dirty data after a writing a number of bytes (#1448) 
This allows s3fs to write large files without consuming a large amount
of temporary local storage but can slow uploads due to server-side
copies.  References #617.  Fixed #1056.  Fixes #1257.
 2020-10-13 22:30:42 +09:00
Takeshi Nakatani
58750cc441 Fixed a bug that symlink could not be read after restarting s3fs 2020-10-11 22:14:36 +09:00
Andrew Gaul
910255745e Allow relative path with use_path 
Fixes #1161.
 2020-10-10 18:13:23 +09:00
Takeshi Nakatani
cf86fa51b0
Merge pull request #1441 from gaul/coverity 
Suppress false positive from Coverity
 2020-10-04 23:44:51 +09:00
Takeshi Nakatani
0492f75197
Merge pull request #1439 from gaul/run-once 
Protect curl_warnings_once with a mutex
 2020-10-04 22:43:56 +09:00
Takeshi Nakatani
493cf20f95
Merge pull request #1435 from gaul/no-exceptions 
Remove exceptions from s3fs_strtoofft
 2020-10-04 22:19:25 +09:00
Andrew Gaul
3553fb65a0 Suppress false positive from Coverity 2020-10-04 14:58:13 +09:00
Takeshi Nakatani
059cc57ba6 Added atime and Corrected atime/mtime/ctime operations 2020-10-04 13:54:01 +09:00
Andrew Gaul
4df4ffe06f Protect curl_warnings_once with a mutex 
This allows ThreadSanitizer to run without warnings.
 2020-10-03 21:09:35 +09:00
Takeshi Nakatani
462347256d Changed the message level in the AutoFdEntity::GetFdEntity 2020-10-03 20:41:17 +09:00
Takeshi Nakatani
133feb67c3
Merge pull request #1434 from gaul/const-string 
Use const std::string& where possible
 2020-10-03 10:02:03 +09:00
Takeshi Nakatani
2cf195741c Fixed signature error due to case of hex string 2020-10-02 18:09:13 +00:00
Takeshi Nakatani
f61baada46
Merge pull request #1433 from gaul/hex 
Further simplify and centralize hex conversion
 2020-10-03 00:34:02 +09:00
Takeshi Nakatani
6e1e0d1d31
Merge pull request #1432 from gaul/v2-or-v4 
Add sigv4 flag and fix v4 regression
 2020-10-03 00:09:55 +09:00
Takeshi Nakatani
66419e7292
Merge pull request #1431 from gaul/string/find_first_of 
Prefer string::find_first_of(char) where possible
 2020-10-02 23:42:17 +09:00
Takeshi Nakatani
9e998877e9
Merge pull request #1430 from gaul/ostringstream 
Simplify use of ostringstream
 2020-10-02 23:26:41 +09:00
Andrew Gaul
3628b9d1e2 Remove exceptions from s3fs_strtoofft 
Explicit return codes are simpler and safer.
 2020-10-02 22:53:12 +09:00
Andrew Gaul
8d2bd874d7 Use const std::string& where possible 
This hints to callers on usage.
 2020-10-02 08:48:43 +09:00
Andrew Gaul
7e27c6cf7d Further simplify and centralize hex conversion 2020-10-01 23:31:06 +09:00
Andrew Gaul
b8ff4ede49 Sign GET requests with V4 signature correctly 
This bug was previously masked by v2 fallback.  Fixes regression from
81805715bdcd50843170fff1c7adf1f19b72addf.
 2020-10-01 20:03:14 +09:00
Andrew Gaul
081d6c1245 Allow setting signature V4-only 
Default to allowing V2 or V4.
 2020-10-01 20:03:14 +09:00
Andrew Gaul
eb8004c355 Prefer string::find_first_of(char) where possible 2020-10-01 18:24:45 +09:00
Andrew Gaul
b3bf9f8f54 Simplify use of ostringstream 
Resetting this does not re-use the underlying buffer so creating a new
ostringstream has similar efficiency.
 2020-10-01 17:55:34 +09:00
Andrew Gaul
503c86bb8a Call is_prefix instead of compare and substr 2020-09-27 22:19:54 +09:00
Takeshi Nakatani
757f4caee8
Merge pull request #1425 from gaul/create-bucket/sse 
Do not send SSE headers during bucket creation
 2020-09-27 11:34:15 +09:00
Takeshi Nakatani
95fabd1f3a
Merge pull request #1423 from gaul/nettle 
Fix MD5 when using gnutls with nettle
 2020-09-27 11:17:55 +09:00
Takeshi Nakatani
0b42e08636
Merge pull request #1422 from gaul/c++/append-assign-at 
Remove calls to append, assign, and at
 2020-09-27 10:59:48 +09:00
Takeshi Nakatani
d5e4f99e72
Merge pull request #1413 from liuyongqing/master 
not call put headers if not exist pending meta
 2020-09-27 10:37:22 +09:00
Andrew Gaul
f35fe850c0 Do not send SSE headers during bucket creation 
This allows integration tests to pass with use_sse.
 2020-09-26 13:25:18 +09:00
Andrew Gaul
9a55c9fd9f Fix MD5 when using gnutls with nettle 
This matches the non-nettle definition of s3fs_md5_fd.
 2020-09-25 23:07:50 +09:00
Neeraj Kumar Kashyap
041b4ec05c enforce exact key match in IBM IAM response 2020-09-25 22:41:19 +09:00
Andrew Gaul
2438066d52 Remove calls to append, assign, and at 
operator+, operator=, and operator[] are more idiomatic and consistent
with the code base.
 2020-09-25 12:19:51 +09:00
yongqingliu
e98827ec6f not call put headers if not exist pending meta 2020-09-21 14:12:07 +08:00
Andrew Gaul
05863a3178 Fix clang-tidy warnings 2020-09-21 07:03:42 +09:00
Takeshi Nakatani
075d161bb1
Merge pull request #1408 from gaul/env 
Ensure environment variable is set when using ECS
 2020-09-20 13:01:17 +09:00
Takeshi Nakatani
6f40503328
Merge pull request #1406 from gaul/strcmp 
Remove unnecessary calls to strcmp(str.c_str())
 2020-09-20 11:28:41 +09:00
Takeshi Nakatani
3440c3348c
Merge pull request #1405 from gaul/is-prefix 
Rename STR2NCMP to is_prefix for clarity
 2020-09-20 10:03:53 +09:00
Takeshi Nakatani
853be26612
Merge pull request #1404 from liuyongqing/master 
fix dead lock in disk insufficient and optimize code
 2020-09-20 01:34:02 +09:00
Andrew Gaul
44d2cc15f7 Ensure environment variable is set when using ECS 
Previously s3fs crashed when AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
was not set.  References #1162.
 2020-09-18 08:14:58 +09:00
Andrew Gaul
81805715bd Remove unnecessary calls to strcmp(str.c_str()) 2020-09-17 20:00:45 +09:00
Andrew Gaul
ebcbb02d16 Rename STR2NCMP to is_prefix for clarity 
This may prevent confusion seen in
39ec8e242e73ca19127c09b6c62a8c9fd7e1266d.
 2020-09-17 19:45:54 +09:00
Takeshi Nakatani
93cf67c65d
Merge pull request #1399 from gaul/md5-sha256 
Use more descriptive names for digest functions
 2020-09-17 17:11:19 +09:00
yongqingliu
211cc0f5f2 fix dead lock in disk insufficient and optimize code 2020-09-16 22:45:28 +08:00
Takeshi Nakatani
c58c91fc4f
Merge pull request #1398 from gaul/memset 
Remove unneeded memset calls
 2020-09-15 22:09:54 +09:00
Andrew Gaul
b7187352e1 Use more descriptive names for digest functions 
Previously they used hex to refer to binary data which is confusing
since other std::string return types are actually hex.  Also remove
unused s3fs_md5sum.
 2020-09-15 21:18:58 +09:00