octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-14 16:34:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,692 Commits 1 Branch 23 Tags 13 MiB
1c8aadafd1
Commit Graph

51 Commits

Author SHA1 Message Date
Andrew Gaul
1987bcbea3
Re-fix retry logic (#1533) 
This logic has always been broken but happened to try one time due to
&& operator precedence.  7158e50ee2
broke this further when quoting && since the command was not
evaluated.
 2021-01-25 18:32:40 +09:00
Andrew Gaul
b04bca37a5
Add configuration for Chaos HTTP Proxy (#1508) 
This can find errors in retry logic.  Chaos HTTP Proxy does not
support SSL bouncestorage/chaos-http-proxy#1 so users must set
s3proxy.endpoint and run via:

CHAOS_HTTP_PROXY=1 S3_URL=http://127.0.0.1:8080 make check -C test

It can also be helpful to increase retries and reduce sleep times.
References #1504.
 2021-01-04 23:32:04 +09:00
Andrew Gaul
7158e50ee2
Quote | and && to allow passing it to retry (#1506) 2021-01-04 22:28:40 +09:00
Takeshi Nakatani
4d0daddad4
Fixed about Github Actions failure(centos8 and macos) (#1498) 2020-12-23 19:34:04 +09:00
Andrew Gaul
834862f8a4
Add S3_ENDPOINT test variable (#1476) 2020-11-15 22:10:26 +09:00
Andrew Gaul
89b1c32b24 Rework s3fs pid handling 
Previously S3FS_PID was not set in the correct shell and thus
ps u $S3FS_PID showed all programs.  This caused the flag detection to
fail when users ran other instances of s3fs with different flags.
References #1402.
 2020-09-18 22:05:37 +09:00
Andrew Gaul
8fb70c5e4a Fix expandtab settings for tests 
This matches the main source.
 2020-09-15 22:11:14 +09:00
Takeshi Nakatani
b5ffd419d8 Source file division and set 4 spaces and cleanup 2020-08-26 17:43:50 +09:00
Andrew Gaul
c8e13300e1 Add script to run tests using sanitizers 
These currently show several kinds of errors.
 2020-08-15 17:28:35 +09:00
Andrew Gaul
4154e539ea Test creating a directory with external program 
References #1264.
 2020-07-25 21:22:53 +09:00
Takeshi Nakatani
eb58460175
Merge pull request #1294 from gaul/test/profile 
Allow overriding test credentials with a profile
 2020-05-24 16:14:31 +09:00
Andrew Gaul
73098220bf Allow overriding test credentials with a profile 2020-05-24 12:05:16 +09:00
Andrew Gaul
ca7756fa77 Update to S3Proxy 1.7.1 
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.7.1
 2020-05-23 10:06:44 +09:00
Andrew Gaul
da70cb92a8 Provide AWS CLI with correct test credentials 
This allows tests to pass against real S3 implementations like
Backblaze.  References #272.
 2020-05-22 19:27:18 +09:00
Andrew Gaul
dc62953040 Upgrade to S3Proxy 1.7.0 
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.7.0
 2019-10-28 20:11:01 -07:00
Takeshi Nakatani
d8f1aef7be Added stats cache expire option for test 2019-08-06 10:55:38 +00:00
Takeshi Nakatani
daba563a1c Added auto_cache option for test on osx 2019-08-06 02:44:18 +00:00
Takeshi Nakatani
b79b0b1a92
Merge pull request #1119 from ggtakec/osx_direct_io 
Added direct_io option for test on osx
 2019-08-06 11:14:51 +09:00
Takeshi Nakatani
dc85eda188 Added direct_io option for test on osx 2019-08-05 17:12:49 +00:00
Andrew Gaul
d22acae9a3 Increase test startup retries on Linux 
This matches the number of retries on macOS and allows Valgrind to
work.
 2019-08-03 16:10:42 -07:00
Takeshi Nakatani
6177d7b096
Merge pull request #1073 from gaul/deps/s3proxy 
Upgrade to S3Proxy 1.6.2
 2019-07-07 15:19:00 +09:00
Takeshi Nakatani
3161bf4608
Merge pull request #1070 from gaul/bash/nounset 
Prohibit pipeline failures
 2019-07-07 15:10:53 +09:00
Takeshi Nakatani
2349dafb98
Merge pull request #1069 from gaul/macos/sed 
Use system sed on macOS
 2019-07-07 15:00:48 +09:00
Andrew Gaul
4f23f38583 Individually test multiple s3fs flags 
Remove unneeded comments; single part limits ensure that the tests
exercise multipart code paths even with smaller files.
References #971.
 2019-07-03 21:09:40 -07:00
Andrew Gaul
f439c6382f Upgrade to S3Proxy 1.6.2 
Notably this includes support for List Objects v2 which AWS CLI uses.
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.2
 2019-07-03 10:37:26 -07:00
Andrew Gaul
9472ee4a01 Prohibit pipeline failures 2019-07-03 02:57:40 -07:00
Andrew Gaul
1f1f824da7 Use system sed on macOS 
This requires fewer developer customizations to work on macOS.
Requires some GNU workarounds.
 2019-07-02 22:17:02 -07:00
Andrew Gaul
1cf3d2452e Correct macOS integration test umount 
Previously errexit could prevent this from succeeding.
 2019-06-19 11:01:32 -07:00
Andrew Gaul
0176fc712b Address shellcheck errors 2019-04-09 21:11:31 +09:00
Andrew Gaul
e5785d4690 Prefer HTTPS links in documentation where possible 
This also corrects a test URL which was HTTPS in practice.
 2019-04-05 17:31:06 +09:00
Andrew Gaul
e9297f39ea Upgrade to S3Proxy 1.6.1 
Notably, this improve performance when listing objects using the
filesystem backend, allowing new tests like expanding large tar files.
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.1
 2019-02-06 17:06:43 -08:00
Takeshi Nakatani
c0cf90cf8b Checked and corrected all typo 2019-01-27 12:04:29 +00:00
Andrew Gaul
523fe1e309 Repair xattr tests 
These did not run due to missing use_xattr flag and Travis
misconfiguration.
 2019-01-13 18:54:16 -08:00
Andrew Gaul
1b86e4d414 Upgrade to S3Proxy 1.6.0 
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.0
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.5.5
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.5.4
 2018-05-16 16:38:17 -07:00
Takeshi Nakatani
f4515b5cfa Merge pull request #646 from andrewgaul/s3proxy-pid 
Simplify S3Proxy PID handling
 2017-09-20 05:17:00 +09:00
Andrew Gaul
5014c1827b Simplify S3Proxy PID handling 
Also remove log prefixing since newer S3Proxy versions do this
already.  Finally remove unnecessary wait.
 2017-09-17 20:19:56 -07:00
Andrew Gaul
f531e6aff2 Configure S3Proxy for SSL 
This also demonstrates that SSL certificate checking occurs and the
tests must disable it for S3Proxy's self-signed certificate.
References #640.
 2017-09-17 16:16:18 -07:00
Takeshi Nakatani
96d8e6d823 Merge remote-tracking branch 'upstream/macosx' into macosx 2017-08-11 14:20:12 +00:00
Takeshi Nakatani
62b8084300 Added travis test on osx for #601 2017-08-11 14:09:43 +00:00
Andrew Gaul
98b724391f Upgrade to S3Proxy 1.5.3 
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.3
 2017-07-09 22:41:39 -07:00
Andrew Gaul
25a03c370a Upgrade to S3Proxy 1.5.2 
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.2
 2017-03-02 10:55:24 -08:00
Andrew Gaul
cafe6015e3 Upgrade to S3Proxy 1.5.1 
Enabled previously broken tests and test with default v4 signer.
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.0
https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.1
 2016-11-15 23:09:48 -08:00
Robb Kistler
7bfaa24d25 Integration test summary, continue on error 
Details in README.md and s3fs-integration-test-common.sh

Factor out s3fs-fuse and s3proxy start/stop.  The plan is to make it easier to
add test suites besides small-integration-test.sh that can test with various
s3fs options.

Each test run starts in a uniquely named at the top of the bucket.  This allows
multiple runs against persistent storage without worrying about cleaning
up in error conditions that leave artifiacts behind.

Tests continues if a test case fails.

Results are summarized at the end of the test run

Environment variable to control debug level of s3fs-fuse

Environment variable to enable public bucket (makes it easier to poke
around with tools like curl)

Environment variable to start s3fs-fuse under valgrind

Environment variable that casues script sets up s3fs-fuse and then wait
indefinitely, making it easy to experiment manually with the mount
point.

Additional test case
 2016-02-05 05:40:28 -08:00
Robb Kistler
6a8a2e4800 Allow integration testing against Amazon S3 
Example command line:
S3FS_CREDENTIALS_FILE=keyfile \
TEST_BUCKET_1=somebucket \
S3PROXY_BINARY="" \
URL="http://s3.amazonaws.com" ./small-integration-test.sh
 2015-09-11 14:35:12 -07:00
Andrew Gaul
236aeb9dfd Silence wget 2015-08-20 11:38:27 -07:00
Andrew Gaul
fa5c7ff4df Upgrade to S3Proxy 1.4.0 
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.4.0
 2015-03-29 23:59:39 -07:00
Andrew Gaul
4e18bf0bc2 Use S3Proxy 1.4.0-SNAPSHOT 2015-03-09 18:05:14 -07:00
Andrew Gaul
0f7d77d599 Small fixes to integration tests 
Use S3Proxy pid instead of self pid, ensure correct passwd
permissions, and use fusermount instead of umount so that non-root can
run tests.
 2015-03-03 01:42:03 -08:00
Andrew Gaul
e811ae1104 Use s3proxy to run integration tests 
References #129.
 2015-02-24 12:08:22 -08:00
Ka-Hing Cheung
03d84a07d1 fix rename before close 
nautilus does this when you drag and drop to overwrite a file:

1) create .goutputstream-XXXXXX to write to
2) fsync the fd for .goutputstream-XXXXXX
3) rename .goutputstream-XXXXXX to target file
4) close the fd for .goutputstream-XXXXXX

previously, doing this on s3fs would result in an empty target file
because after the rename, s3fs would not flush the content of
.goutputstream-XXXXXX to target file.

this change moves the FdEntity from the old path to the new path
whenever rename happens. On flush s3fs would now flush the correct
content to the rename target.
 2015-01-12 15:05:54 -08:00