octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-16 01:07:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,736 Commits 1 Branch 23 Tags 13 MiB
b5fef788da
Commit Graph

54 Commits

Author SHA1 Message Date
Andrew Gaul
20a6c9d35d
Update to S3Proxy 1.8.0 (#1598) 
Notably this fixes an issue with the filesystem backend which returned
the incorrect Content-Type for some directories.  It also reduces
memory use with GetObject using range requests with large objects.
References #1543.

Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.8.0
 2021-03-28 11:56:10 +09:00
Takeshi Nakatani
a0f347b10f Added no_time_stamp_ms option mainly for testing 2021-02-20 20:24:26 +09:00
Carsten Grohmann
5debf523b0 Allow changing temporary directory used for tests 
Set temporary directory to $TMPDIR if set or use default /var/tmp
 2021-02-19 13:59:23 +09:00
Andrew Gaul
1987bcbea3
Re-fix retry logic (#1533) 
This logic has always been broken but happened to try one time due to
&& operator precedence.  7158e50ee2
broke this further when quoting && since the command was not
evaluated.
 2021-01-25 18:32:40 +09:00
Andrew Gaul
b04bca37a5
Add configuration for Chaos HTTP Proxy (#1508) 
This can find errors in retry logic.  Chaos HTTP Proxy does not
support SSL bouncestorage/chaos-http-proxy#1 so users must set
s3proxy.endpoint and run via:

CHAOS_HTTP_PROXY=1 S3_URL=http://127.0.0.1:8080 make check -C test

It can also be helpful to increase retries and reduce sleep times.
References #1504.
 2021-01-04 23:32:04 +09:00
Andrew Gaul
7158e50ee2
Quote | and && to allow passing it to retry (#1506) 2021-01-04 22:28:40 +09:00
Takeshi Nakatani
4d0daddad4
Fixed about Github Actions failure(centos8 and macos) (#1498) 2020-12-23 19:34:04 +09:00
Andrew Gaul
834862f8a4
Add S3_ENDPOINT test variable (#1476) 2020-11-15 22:10:26 +09:00
Andrew Gaul
89b1c32b24 Rework s3fs pid handling 
Previously S3FS_PID was not set in the correct shell and thus
ps u $S3FS_PID showed all programs.  This caused the flag detection to
fail when users ran other instances of s3fs with different flags.
References #1402.
 2020-09-18 22:05:37 +09:00
Andrew Gaul
8fb70c5e4a Fix expandtab settings for tests 
This matches the main source.
 2020-09-15 22:11:14 +09:00
Takeshi Nakatani
b5ffd419d8 Source file division and set 4 spaces and cleanup 2020-08-26 17:43:50 +09:00
Andrew Gaul
c8e13300e1 Add script to run tests using sanitizers 
These currently show several kinds of errors.
 2020-08-15 17:28:35 +09:00
Andrew Gaul
4154e539ea Test creating a directory with external program 
References #1264.
 2020-07-25 21:22:53 +09:00
Takeshi Nakatani
eb58460175
Merge pull request #1294 from gaul/test/profile 
Allow overriding test credentials with a profile
 2020-05-24 16:14:31 +09:00
Andrew Gaul
73098220bf Allow overriding test credentials with a profile 2020-05-24 12:05:16 +09:00
Andrew Gaul
ca7756fa77 Update to S3Proxy 1.7.1 
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.7.1
 2020-05-23 10:06:44 +09:00
Andrew Gaul
da70cb92a8 Provide AWS CLI with correct test credentials 
This allows tests to pass against real S3 implementations like
Backblaze.  References #272.
 2020-05-22 19:27:18 +09:00
Andrew Gaul
dc62953040 Upgrade to S3Proxy 1.7.0 
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.7.0
 2019-10-28 20:11:01 -07:00
Takeshi Nakatani
d8f1aef7be Added stats cache expire option for test 2019-08-06 10:55:38 +00:00
Takeshi Nakatani
daba563a1c Added auto_cache option for test on osx 2019-08-06 02:44:18 +00:00
Takeshi Nakatani
b79b0b1a92
Merge pull request #1119 from ggtakec/osx_direct_io 
Added direct_io option for test on osx
 2019-08-06 11:14:51 +09:00
Takeshi Nakatani
dc85eda188 Added direct_io option for test on osx 2019-08-05 17:12:49 +00:00
Andrew Gaul
d22acae9a3 Increase test startup retries on Linux 
This matches the number of retries on macOS and allows Valgrind to
work.
 2019-08-03 16:10:42 -07:00
Takeshi Nakatani
6177d7b096
Merge pull request #1073 from gaul/deps/s3proxy 
Upgrade to S3Proxy 1.6.2
 2019-07-07 15:19:00 +09:00
Takeshi Nakatani
3161bf4608
Merge pull request #1070 from gaul/bash/nounset 
Prohibit pipeline failures
 2019-07-07 15:10:53 +09:00
Takeshi Nakatani
2349dafb98
Merge pull request #1069 from gaul/macos/sed 
Use system sed on macOS
 2019-07-07 15:00:48 +09:00
Andrew Gaul
4f23f38583 Individually test multiple s3fs flags 
Remove unneeded comments; single part limits ensure that the tests
exercise multipart code paths even with smaller files.
References #971.
 2019-07-03 21:09:40 -07:00
Andrew Gaul
f439c6382f Upgrade to S3Proxy 1.6.2 
Notably this includes support for List Objects v2 which AWS CLI uses.
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.2
 2019-07-03 10:37:26 -07:00
Andrew Gaul
9472ee4a01 Prohibit pipeline failures 2019-07-03 02:57:40 -07:00
Andrew Gaul
1f1f824da7 Use system sed on macOS 
This requires fewer developer customizations to work on macOS.
Requires some GNU workarounds.
 2019-07-02 22:17:02 -07:00
Andrew Gaul
1cf3d2452e Correct macOS integration test umount 
Previously errexit could prevent this from succeeding.
 2019-06-19 11:01:32 -07:00
Andrew Gaul
0176fc712b Address shellcheck errors 2019-04-09 21:11:31 +09:00
Andrew Gaul
e5785d4690 Prefer HTTPS links in documentation where possible 
This also corrects a test URL which was HTTPS in practice.
 2019-04-05 17:31:06 +09:00
Andrew Gaul
e9297f39ea Upgrade to S3Proxy 1.6.1 
Notably, this improve performance when listing objects using the
filesystem backend, allowing new tests like expanding large tar files.
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.1
 2019-02-06 17:06:43 -08:00
Takeshi Nakatani
c0cf90cf8b Checked and corrected all typo 2019-01-27 12:04:29 +00:00
Andrew Gaul
523fe1e309 Repair xattr tests 
These did not run due to missing use_xattr flag and Travis
misconfiguration.
 2019-01-13 18:54:16 -08:00
Andrew Gaul
1b86e4d414 Upgrade to S3Proxy 1.6.0 
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.0
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.5.5
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.5.4
 2018-05-16 16:38:17 -07:00
Takeshi Nakatani
f4515b5cfa Merge pull request #646 from andrewgaul/s3proxy-pid 
Simplify S3Proxy PID handling
 2017-09-20 05:17:00 +09:00
Andrew Gaul
5014c1827b Simplify S3Proxy PID handling 
Also remove log prefixing since newer S3Proxy versions do this
already.  Finally remove unnecessary wait.
 2017-09-17 20:19:56 -07:00
Andrew Gaul
f531e6aff2 Configure S3Proxy for SSL 
This also demonstrates that SSL certificate checking occurs and the
tests must disable it for S3Proxy's self-signed certificate.
References #640.
 2017-09-17 16:16:18 -07:00
Takeshi Nakatani
96d8e6d823 Merge remote-tracking branch 'upstream/macosx' into macosx 2017-08-11 14:20:12 +00:00
Takeshi Nakatani
62b8084300 Added travis test on osx for #601 2017-08-11 14:09:43 +00:00
Andrew Gaul
98b724391f Upgrade to S3Proxy 1.5.3 
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.3
 2017-07-09 22:41:39 -07:00
Andrew Gaul
25a03c370a Upgrade to S3Proxy 1.5.2 
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.2
 2017-03-02 10:55:24 -08:00
Andrew Gaul
cafe6015e3 Upgrade to S3Proxy 1.5.1 
Enabled previously broken tests and test with default v4 signer.
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.0
https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.1
 2016-11-15 23:09:48 -08:00
Robb Kistler
7bfaa24d25 Integration test summary, continue on error 
Details in README.md and s3fs-integration-test-common.sh

Factor out s3fs-fuse and s3proxy start/stop.  The plan is to make it easier to
add test suites besides small-integration-test.sh that can test with various
s3fs options.

Each test run starts in a uniquely named at the top of the bucket.  This allows
multiple runs against persistent storage without worrying about cleaning
up in error conditions that leave artifiacts behind.

Tests continues if a test case fails.

Results are summarized at the end of the test run

Environment variable to control debug level of s3fs-fuse

Environment variable to enable public bucket (makes it easier to poke
around with tools like curl)

Environment variable to start s3fs-fuse under valgrind

Environment variable that casues script sets up s3fs-fuse and then wait
indefinitely, making it easy to experiment manually with the mount
point.

Additional test case
 2016-02-05 05:40:28 -08:00
Robb Kistler
6a8a2e4800 Allow integration testing against Amazon S3 
Example command line:
S3FS_CREDENTIALS_FILE=keyfile \
TEST_BUCKET_1=somebucket \
S3PROXY_BINARY="" \
URL="http://s3.amazonaws.com" ./small-integration-test.sh
 2015-09-11 14:35:12 -07:00
Andrew Gaul
236aeb9dfd Silence wget 2015-08-20 11:38:27 -07:00
Andrew Gaul
fa5c7ff4df Upgrade to S3Proxy 1.4.0 
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.4.0
 2015-03-29 23:59:39 -07:00
Andrew Gaul
4e18bf0bc2 Use S3Proxy 1.4.0-SNAPSHOT 2015-03-09 18:05:14 -07:00