Commit Graph

66 Commits

Author SHA1 Message Date
Takeshi Nakatani 46014397d8 Added test by a shell script static analysis tool(ShellCheck) 2022-01-22 22:23:08 +09:00
Andrew Gaul 6300859c80
Prefer = over == for older shell compatibility (#1857) 2022-01-14 12:40:55 +09:00
Andrew Gaul 2892d3b755
Call curl --fail to propagate exit code (#1856) 2022-01-14 11:52:52 +09:00
Andrew Gaul 577e2bc987
Generate S3Proxy SSL certificate during tests (#1845)
Also provide CA bundle to AWS CLI to work around CI failures instead
of ignoring errors.  Fixes #1812.
2022-01-09 15:13:36 +09:00
Andrew Gaul adb58af17b
Annotate local variables (#1844)
This prevents collisions with other globals.  Fixes #1843.
2022-01-09 13:03:36 +09:00
Andrew Gaul e734763002
Remove createbucket option (#1841)
AWS CLI can do this.  Fixes #1840.
2022-01-05 01:59:31 +09:00
Andrew Gaul 616db5bf1c
Prefer curl over wget for fetching dependencies (#1836)
The former is lighter-weight and libcurl is already a dependency for
s3fs.
2022-01-05 00:43:36 +09:00
Andrew Gaul 3cf00626a2
Add option to allow unsigned payloads (#1801)
This reduces CPU usage of sigv4.  This reduces test run-time by 7
seconds per flag.
2021-11-01 23:33:55 +09:00
Andrew Gaul 72a9f40f3f
Update to S3Proxy 1.9.0 (#1788)
Notably this fixes an issue with the transient provider reading parts
of large files.

Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.9.0
2021-10-26 23:20:52 +09:00
Carsten Grohmann 0abeec9cae Simpily errexit modifications 2021-10-26 21:47:36 +09:00
Andrew Gaul ea3c21f270
Reduce errexit modifications (#1785)
This is less error prone but requires some magic && ||.
2021-10-25 23:53:45 +09:00
Andrew Gaul 2154e898bc
Fix typos (#1700) 2021-06-27 11:22:33 +09:00
Andrew Gaul 20a6c9d35d
Update to S3Proxy 1.8.0 (#1598)
Notably this fixes an issue with the filesystem backend which returned
the incorrect Content-Type for some directories.  It also reduces
memory use with GetObject using range requests with large objects.
References #1543.

Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.8.0
2021-03-28 11:56:10 +09:00
Takeshi Nakatani a0f347b10f Added no_time_stamp_ms option mainly for testing 2021-02-20 20:24:26 +09:00
Carsten Grohmann 5debf523b0 Allow changing temporary directory used for tests
Set temporary directory to $TMPDIR if set or use default /var/tmp
2021-02-19 13:59:23 +09:00
Andrew Gaul 1987bcbea3
Re-fix retry logic (#1533)
This logic has always been broken but happened to try one time due to
&& operator precedence.  7158e50ee2
broke this further when quoting && since the command was not
evaluated.
2021-01-25 18:32:40 +09:00
Andrew Gaul b04bca37a5
Add configuration for Chaos HTTP Proxy (#1508)
This can find errors in retry logic.  Chaos HTTP Proxy does not
support SSL bouncestorage/chaos-http-proxy#1 so users must set
s3proxy.endpoint and run via:

CHAOS_HTTP_PROXY=1 S3_URL=http://127.0.0.1:8080 make check -C test

It can also be helpful to increase retries and reduce sleep times.
References #1504.
2021-01-04 23:32:04 +09:00
Andrew Gaul 7158e50ee2
Quote | and && to allow passing it to retry (#1506) 2021-01-04 22:28:40 +09:00
Takeshi Nakatani 4d0daddad4
Fixed about Github Actions failure(centos8 and macos) (#1498) 2020-12-23 19:34:04 +09:00
Andrew Gaul 834862f8a4
Add S3_ENDPOINT test variable (#1476) 2020-11-15 22:10:26 +09:00
Andrew Gaul 89b1c32b24 Rework s3fs pid handling
Previously S3FS_PID was not set in the correct shell and thus
ps u $S3FS_PID showed all programs.  This caused the flag detection to
fail when users ran other instances of s3fs with different flags.
References #1402.
2020-09-18 22:05:37 +09:00
Andrew Gaul 8fb70c5e4a Fix expandtab settings for tests
This matches the main source.
2020-09-15 22:11:14 +09:00
Takeshi Nakatani b5ffd419d8 Source file division and set 4 spaces and cleanup 2020-08-26 17:43:50 +09:00
Andrew Gaul c8e13300e1 Add script to run tests using sanitizers
These currently show several kinds of errors.
2020-08-15 17:28:35 +09:00
Andrew Gaul 4154e539ea Test creating a directory with external program
References #1264.
2020-07-25 21:22:53 +09:00
Takeshi Nakatani eb58460175
Merge pull request #1294 from gaul/test/profile
Allow overriding test credentials with a profile
2020-05-24 16:14:31 +09:00
Andrew Gaul 73098220bf Allow overriding test credentials with a profile 2020-05-24 12:05:16 +09:00
Andrew Gaul ca7756fa77 Update to S3Proxy 1.7.1
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.7.1
2020-05-23 10:06:44 +09:00
Andrew Gaul da70cb92a8 Provide AWS CLI with correct test credentials
This allows tests to pass against real S3 implementations like
Backblaze.  References #272.
2020-05-22 19:27:18 +09:00
Andrew Gaul dc62953040 Upgrade to S3Proxy 1.7.0
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.7.0
2019-10-28 20:11:01 -07:00
Takeshi Nakatani d8f1aef7be Added stats cache expire option for test 2019-08-06 10:55:38 +00:00
Takeshi Nakatani daba563a1c Added auto_cache option for test on osx 2019-08-06 02:44:18 +00:00
Takeshi Nakatani b79b0b1a92
Merge pull request #1119 from ggtakec/osx_direct_io
Added direct_io option for test on osx
2019-08-06 11:14:51 +09:00
Takeshi Nakatani dc85eda188 Added direct_io option for test on osx 2019-08-05 17:12:49 +00:00
Andrew Gaul d22acae9a3 Increase test startup retries on Linux
This matches the number of retries on macOS and allows Valgrind to
work.
2019-08-03 16:10:42 -07:00
Takeshi Nakatani 6177d7b096
Merge pull request #1073 from gaul/deps/s3proxy
Upgrade to S3Proxy 1.6.2
2019-07-07 15:19:00 +09:00
Takeshi Nakatani 3161bf4608
Merge pull request #1070 from gaul/bash/nounset
Prohibit pipeline failures
2019-07-07 15:10:53 +09:00
Takeshi Nakatani 2349dafb98
Merge pull request #1069 from gaul/macos/sed
Use system sed on macOS
2019-07-07 15:00:48 +09:00
Andrew Gaul 4f23f38583 Individually test multiple s3fs flags
Remove unneeded comments; single part limits ensure that the tests
exercise multipart code paths even with smaller files.
References #971.
2019-07-03 21:09:40 -07:00
Andrew Gaul f439c6382f Upgrade to S3Proxy 1.6.2
Notably this includes support for List Objects v2 which AWS CLI uses.
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.2
2019-07-03 10:37:26 -07:00
Andrew Gaul 9472ee4a01 Prohibit pipeline failures 2019-07-03 02:57:40 -07:00
Andrew Gaul 1f1f824da7 Use system sed on macOS
This requires fewer developer customizations to work on macOS.
Requires some GNU workarounds.
2019-07-02 22:17:02 -07:00
Andrew Gaul 1cf3d2452e Correct macOS integration test umount
Previously errexit could prevent this from succeeding.
2019-06-19 11:01:32 -07:00
Andrew Gaul 0176fc712b Address shellcheck errors 2019-04-09 21:11:31 +09:00
Andrew Gaul e5785d4690 Prefer HTTPS links in documentation where possible
This also corrects a test URL which was HTTPS in practice.
2019-04-05 17:31:06 +09:00
Andrew Gaul e9297f39ea Upgrade to S3Proxy 1.6.1
Notably, this improve performance when listing objects using the
filesystem backend, allowing new tests like expanding large tar files.
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.1
2019-02-06 17:06:43 -08:00
Takeshi Nakatani c0cf90cf8b Checked and corrected all typo 2019-01-27 12:04:29 +00:00
Andrew Gaul 523fe1e309 Repair xattr tests
These did not run due to missing use_xattr flag and Travis
misconfiguration.
2019-01-13 18:54:16 -08:00
Andrew Gaul 1b86e4d414 Upgrade to S3Proxy 1.6.0
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.0
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.5.5
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.5.4
2018-05-16 16:38:17 -07:00
Takeshi Nakatani f4515b5cfa Merge pull request #646 from andrewgaul/s3proxy-pid
Simplify S3Proxy PID handling
2017-09-20 05:17:00 +09:00