octoleo
/
s3fs-fuse
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,665 Commits 1 Branch 22 Tags 11 MiB
Commit Graph

1665 Commits

Author SHA1 Message Date
Andrew Gaul a6563211af Update ChangeLog and configure.ac for 1.88 2021-01-11 17:59:28 +09:00
Andrew Gaul 8d66b0e4a8
Updated ChangeLog for release 1.88 (#1514) 
References #1511.
 2021-01-05 16:46:41 +09:00
Takeshi Nakatani 555c1dde3d Use Gtihub Actions instead of Travis CI 2021-01-04 23:46:54 +09:00
Andrew Gaul b04bca37a5
Add configuration for Chaos HTTP Proxy (#1508) 
This can find errors in retry logic.  Chaos HTTP Proxy does not
support SSL bouncestorage/chaos-http-proxy#1 so users must set
s3proxy.endpoint and run via:

CHAOS_HTTP_PROXY=1 S3_URL=http://127.0.0.1:8080 make check -C test

It can also be helpful to increase retries and reduce sleep times.
References #1504.
 2021-01-04 23:32:04 +09:00
fly3366 168e588ac7
fix: Add reset offset (#1503) 2021-01-04 22:57:56 +09:00
Andrew Gaul 7158e50ee2
Quote | and && to allow passing it to retry (#1506) 2021-01-04 22:28:40 +09:00
Andrew Gaul bd0fadbe5f
Remove authorization header when remaking handle (#1505) 
This avoids including Authorization in SignedHeaders.  s3fs will
recreate the Authorization header before sending the request.
 2021-01-04 21:37:34 +09:00
kontrollanten d1c638ab7a fix 404 error message 2020-12-30 14:10:17 +09:00
fly3366 51f65d7b14 fix: miss header when retry 2020-12-23 22:16:07 +09:00
Andrew Gaul a16d00d673
Simply curl_slist_sort_insert (#1494) 
Some good taste from Linus:
https://github.com/mkirchner/linked-list-good-taste
Also avoid an allocation when replacing a value and tighten up tests.
 2020-12-23 20:29:33 +09:00
Takeshi Nakatani 4d0daddad4
Fixed about Github Actions failure(centos8 and macos) (#1498) 2020-12-23 19:34:04 +09:00
Brian Myers d246b9e8bf Update README for change in brew arguments 
* Homebrew changed the arguments required to install casks. Previously,
  `brew cask install <package>` was the correct syntax, but now it's
  `brew install --cask <package>`. This small commit updates the
  instructions for installing `osxfuse` to reflect this change
 2020-12-22 11:14:24 +09:00
Adam Johnson ef3d4e506d Update README note on eventual consistency 
AWS promise read-after-write consistency now, but presumably non-AWS S3-compatible services are still eventually consistent.
 2020-12-15 08:50:53 +09:00
Takeshi Nakatani 462b37b0bb Added Github Actions workflow 2020-12-05 22:57:40 +09:00
Takeshi Nakatani 4341291cc2
Changed a condition of the atime test(fixed #1477) (#1487) 2020-12-03 02:45:05 +09:00
Takeshi Nakatani c589886ba5
Added a condition to the atime test (#1477) 2020-12-02 23:21:25 +09:00
Andrew Gaul d0363b118e
Add tests for curl_util (#1481) 2020-11-24 21:37:09 +09:00
Takeshi Nakatani 533322859d Added comment for nullPointerRedundantCheck/cppcheck2.2 2020-11-21 20:06:14 +09:00
Andrew Gaul f8d5b76edb
Simplify handling of returned ETag (#1479) 
This works around lifetime warnings uncovered by cppcheck.
References #1478.
 2020-11-21 06:56:05 +09:00
Andrew Gaul 834862f8a4
Add S3_ENDPOINT test variable (#1476) 2020-11-15 22:10:26 +09:00
Takeshi Nakatani d9f6469b7b Fixed flushing dirty data and compressed the cache size 2020-11-14 16:45:37 +09:00
Takeshi Nakatani 852e6ee4c6 Fixed .gitignore for src/test_string_util.cpp 2020-11-14 16:22:30 +09:00
Andrew Gaul ecb24c9c26
Disable preprocessor warnings w/o optimizations (#1473) 
This removes some fortify source warnings.  Also disable failing
memory sanitizer.
 2020-11-09 21:48:07 +09:00
Andrew Gaul 543231c9f2
Address warnings from clang-tidy 11 (#1470) 2020-11-09 21:15:20 +09:00
Noah Meyerhans d96a08d4ad fixup! Implement AWS IMDSv2 support 2020-11-07 14:48:52 +09:00
Noah Meyerhans f2f930300a Implement AWS IMDSv2 support 
AWS IMDSv2 is a session oriented method for retrieving instance metadata,
including IAM credentials, in Amazon EC2.  It is enabled by default in
non-enforcing mode in AWS (meaning it retains backwards compatibility with
existing IMDSv1 clients), but can be switched to enforcing mode, in which
clients are required to return API tokens with requests.

With this change, we implement support for IMDSv2 and enable it by default when
IAM roles are our source for authentication credentials.  In the event that
s3fs is running in cloud environment offering an IMDSv1-compatible API, we
support graceful fallback to that mode.  It can also be selected explicitly via
the imdsv1only mount option.

More details on IMDSv2 are available at
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
and
https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/

Signed-off-by: Noah Meyerhans <nmeyerha@amazon.com>
 2020-11-07 14:48:52 +09:00
yoshihiko 81ad3ce0ae Fix directory authority 2020-11-07 14:47:53 +09:00
Takeshi Nakatani 38e1eaa8a3 Added logfile option for non-syslog logging 2020-10-19 20:10:19 +09:00
Andrew Gaul 6aa786b886
Flush dirty data after a writing a number of bytes (#1448) 
This allows s3fs to write large files without consuming a large amount
of temporary local storage but can slow uploads due to server-side
copies.  References #617.  Fixed #1056.  Fixes #1257.
 2020-10-13 22:30:42 +09:00
Takeshi Nakatani 58750cc441 Fixed a bug that symlink could not be read after restarting s3fs 2020-10-11 22:14:36 +09:00
Takeshi Nakatani 2188fb067e
Merge pull request #1446 from gaul/use-cache/relative-path 
Allow relative path with use_path
 2020-10-10 23:13:07 +09:00
Andrew Gaul 910255745e Allow relative path with use_path 
Fixes #1161.
 2020-10-10 18:13:23 +09:00
Takeshi Nakatani cf86fa51b0
Merge pull request #1441 from gaul/coverity 
Suppress false positive from Coverity
 2020-10-04 23:44:51 +09:00
Takeshi Nakatani a4e4ce8aea
Merge pull request #1440 from gaul/test/undefined-behavior-sanitizer 
Add memory and undefined behavior sanitizer
 2020-10-04 23:19:36 +09:00
Andrew Gaul 7f43b7fa53 Add memory and undefined behavior sanitizer 
Remove Valgrind which duplicates the sanitizers and is much slower.
 2020-10-04 22:55:27 +09:00
Takeshi Nakatani 0492f75197
Merge pull request #1439 from gaul/run-once 
Protect curl_warnings_once with a mutex
 2020-10-04 22:43:56 +09:00
Takeshi Nakatani 493cf20f95
Merge pull request #1435 from gaul/no-exceptions 
Remove exceptions from s3fs_strtoofft
 2020-10-04 22:19:25 +09:00
Andrew Gaul 3553fb65a0 Suppress false positive from Coverity 2020-10-04 14:58:13 +09:00
Takeshi Nakatani 059cc57ba6 Added atime and Corrected atime/mtime/ctime operations 2020-10-04 13:54:01 +09:00
Andrew Gaul 4df4ffe06f Protect curl_warnings_once with a mutex 
This allows ThreadSanitizer to run without warnings.
 2020-10-03 21:09:35 +09:00
Takeshi Nakatani 462347256d Changed the message level in the AutoFdEntity::GetFdEntity 2020-10-03 20:41:17 +09:00
Takeshi Nakatani 133feb67c3
Merge pull request #1434 from gaul/const-string 
Use const std::string& where possible
 2020-10-03 10:02:03 +09:00
Takeshi Nakatani be308e9d11
Merge pull request #1436 from ggtakec/fix_code 
Fixed signature error due to case of hex string
 2020-10-03 03:44:49 +09:00
Takeshi Nakatani 2cf195741c Fixed signature error due to case of hex string 2020-10-02 18:09:13 +00:00
Takeshi Nakatani f61baada46
Merge pull request #1433 from gaul/hex 
Further simplify and centralize hex conversion
 2020-10-03 00:34:02 +09:00
Takeshi Nakatani 6e1e0d1d31
Merge pull request #1432 from gaul/v2-or-v4 
Add sigv4 flag and fix v4 regression
 2020-10-03 00:09:55 +09:00
Takeshi Nakatani 66419e7292
Merge pull request #1431 from gaul/string/find_first_of 
Prefer string::find_first_of(char) where possible
 2020-10-02 23:42:17 +09:00
Takeshi Nakatani 9e998877e9
Merge pull request #1430 from gaul/ostringstream 
Simplify use of ostringstream
 2020-10-02 23:26:41 +09:00
Takeshi Nakatani 5ebd4039e6
Merge pull request #1426 from gaul/is_prefix 
Call is_prefix instead of compare and substr
 2020-10-02 23:03:10 +09:00
Andrew Gaul 3628b9d1e2 Remove exceptions from s3fs_strtoofft 
Explicit return codes are simpler and safer.
 2020-10-02 22:53:12 +09:00