octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-12-23 09:18:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,869 Commits 1 Branch 23 Tags 13 MiB
e452ef3940
Commit Graph

226 Commits

Author SHA1 Message Date
Andrew Gaul
74c11ef226
Check bucket before trying to create it (#1874) 
This makes it easier to run tests against S3 other than S3Proxy.
 2022-01-26 23:42:12 +09:00
Takeshi Nakatani
46014397d8 Added test by a shell script static analysis tool(ShellCheck) 2022-01-22 22:23:08 +09:00
Andrew Gaul
6300859c80
Prefer = over == for older shell compatibility (#1857) 2022-01-14 12:40:55 +09:00
Andrew Gaul
2892d3b755
Call curl --fail to propagate exit code (#1856) 2022-01-14 11:52:52 +09:00
Andrew Gaul
577e2bc987
Generate S3Proxy SSL certificate during tests (#1845) 
Also provide CA bundle to AWS CLI to work around CI failures instead
of ignoring errors.  Fixes #1812.
 2022-01-09 15:13:36 +09:00
Andrew Gaul
adb58af17b
Annotate local variables (#1844) 
This prevents collisions with other globals.  Fixes #1843.
 2022-01-09 13:03:36 +09:00
Andrew Gaul
fc7543fa25
Make ut_test compatible with Python 3 (#1838) 
This may allow removing Python 2 on newer distros.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
 2022-01-09 11:08:36 +09:00
Andrew Gaul
a44ea7c12a
Replace write_multiple_offsets.py with write_multiblock (#1837) 
This reduces the dependency on Python 2.
 2022-01-09 10:51:17 +09:00
Andrew Gaul
e734763002
Remove createbucket option (#1841) 
AWS CLI can do this.  Fixes #1840.
 2022-01-05 01:59:31 +09:00
Andrew Gaul
616db5bf1c
Prefer curl over wget for fetching dependencies (#1836) 
The former is lighter-weight and libcurl is already a dependency for
s3fs.
 2022-01-05 00:43:36 +09:00
Andrew Gaul
07e2e3f72a
Remove sleep 1 from test_update_directory_time (#1803) 
Reduces per-flag test run-time by 5 seconds.
 2021-11-04 08:16:40 +09:00
Andrew Gaul
3cf00626a2
Add option to allow unsigned payloads (#1801) 
This reduces CPU usage of sigv4.  This reduces test run-time by 7
seconds per flag.
 2021-11-01 23:33:55 +09:00
Andrew Gaul
e289915dcb
Remove require-root script (#1800) 
Tests do not require this.
 2021-10-31 10:48:15 +09:00
Andrew Gaul
06dec32965
Use AWS CLI to create explicit times in the past (#1797) 
s3fs can also do this via utimensat but tests should not trust this.
Also break tests into individual functions.  This further reduces test
run-time 8 seconds per flag.
 2021-10-30 10:54:18 +09:00
Andrew Gaul
86317dd185
Replace dd if=/dev/urandom with junk data generator (#1786) 
This reduces test run time for a single flag from 73 to 60 seconds.
 2021-10-28 22:54:25 +09:00
Andrew Gaul
40d2e0d1ad
Reduce sleep time to 1 (#1793) 
This reduces test run-time 15 seconds per flag or 2.5 minutes when
testing all flags.
 2021-10-27 23:47:08 +09:00
Takeshi Nakatani
b6c5069ef7 Fixed the test is multi-block writing by one flush 2021-10-27 08:19:05 +09:00
Andrew Gaul
72a9f40f3f
Update to S3Proxy 1.9.0 (#1788) 
Notably this fixes an issue with the transient provider reading parts
of large files.

Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.9.0
 2021-10-26 23:20:52 +09:00
Andrew Gaul
495d51113c
Remove unneeded sleeps from tests (#1784) 
Also use a unique file name for every test.  This ensures that tests
like test_external_directory_creation and test_external_modification
do not collide.
 2021-10-26 23:19:14 +09:00
Carsten Grohmann
0abeec9cae Simpily errexit modifications 2021-10-26 21:47:36 +09:00
Andrew Gaul
ea3c21f270
Reduce errexit modifications (#1785) 
This is less error prone but requires some magic && ||.
 2021-10-25 23:53:45 +09:00
Takeshi Nakatani
23fe6f4dee Fixed parse_string function in write_multiblock.cc 2021-10-25 17:56:49 +09:00
Takeshi Nakatani
34ea2acd75 Add a test that is multi-block writing by one flush 2021-10-25 17:56:49 +09:00
Takeshi Nakatani
ea64886469 Fixed a bug in test_(zero_)cache_file_stat test function 2021-10-24 18:24:12 +09:00
Takeshi Nakatani
023aaf7dff Fixed wrong stat of cache after new creation file 
And added a test for stat of cache after new creation file
 2021-10-17 16:10:14 +09:00
Takeshi Nakatani
34f89e5936 Added fake_diskfree option to deceive free disk space for test 2021-08-02 11:04:42 +09:00
Andrew Gaul
9cfa177af0
Add flag to run all tests (#1710) 
This allows CI to run the full set of tests while making local
development faster.
 2021-07-02 08:41:47 +09:00
Andrew Gaul
2154e898bc
Fix typos (#1700) 2021-06-27 11:22:33 +09:00
Takeshi Nakatani
f9e80f995d Fixed a bug about rename existing file of different sizes by mpcopy 2021-06-26 00:04:58 +09:00
Andrew Gaul
944c1fd148
Clean up litter in ut_test (#1697) 2021-06-25 19:15:03 +09:00
Takeshi Nakatani
0555c4216e Added test case for utimens is calling during multipart upload 2021-06-21 17:20:24 +09:00
Andrew Gaul
1571379304
Set LC_ALL=en_US.UTF-8 in tests (#1691) 
This allows tests to pass using the S3Proxy filesystem provider.
References #1665.
 2021-06-20 19:27:25 +09:00
Andrew Gaul
f5701fa9ad
Correct typo in undefinedsanitizer config (#1669) 2021-05-29 22:07:33 +09:00
Carsten Grohmann
c0bcb41175
Increase test robustness by adding LC_ALL=C (#1660) 
Use C locale, because some tests check for English expressions

Fixes #1658
 2021-05-20 21:19:50 +09:00
Takeshi Nakatani
9d8f1b00f7 Fixed make_random_string test code being slow on macos 2021-05-08 13:55:39 +09:00
Andrew Gaul
771bbfeac5
Do not create zero-byte object when creating file (#1640) 
Previously s3fs created this object to store metadata and overwrote it
when flushing.  This prevented use with object stores which do not
allow overwrites like HDS.  Instead only create an in-memory
representation which reduces the time to create small files.
Fixes #1013.
 2021-04-30 19:56:33 +09:00
Andrew Gaul
8a51a26819 Allow truncation of open and modified files 
Regression introduced in f5bf41cf11.
Fixes #1575.
 2021-04-25 12:35:35 +09:00
Andrew Gaul
20a6c9d35d
Update to S3Proxy 1.8.0 (#1598) 
Notably this fixes an issue with the filesystem backend which returned
the incorrect Content-Type for some directories.  It also reduces
memory use with GetObject using range requests with large objects.
References #1543.

Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.8.0
 2021-03-28 11:56:10 +09:00
Takeshi Nakatani
a19d223434 Bypass an OSX test(no different from other OS) which takes very time consuming 2021-02-23 20:47:51 +09:00
Takeshi Nakatani
5b5bc3114a Fixed a bug that padded null bytes when changing xattr 2021-02-21 12:23:36 +09:00
Takeshi Nakatani
81e267d421 Fixed the same bug in chmod/chown/utimens and added tests 2021-02-20 23:30:50 +09:00
Takeshi Nakatani
a0f347b10f Added no_time_stamp_ms option mainly for testing 2021-02-20 20:24:26 +09:00
Carsten Grohmann
5debf523b0 Allow changing temporary directory used for tests 
Set temporary directory to $TMPDIR if set or use default /var/tmp
 2021-02-19 13:59:23 +09:00
Carsten Grohmann
1c8aadafd1 Fix typo in an error message of the test suite 2021-02-19 13:59:23 +09:00
Takeshi Nakatani
d33f252404 Fixed forgetting to clean up test files 2021-02-14 00:07:46 +09:00
Andrew Gaul
7f6fbb0021
Correct singlepart_copy_limit unit (#1561) 
This is documented to be in MB not KB.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
 2021-02-11 22:53:50 +09:00
Andrew Gaul
f2c5e38724
Allow integration tests to use larger files (#1548) 
Previously these failed due to dd only copying 32 MB from /dev/urandom
and exhausting the limited space in /tmp.  References #1543.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
 2021-02-07 12:50:02 +09:00
Andrew Gaul
1987bcbea3
Re-fix retry logic (#1533) 
This logic has always been broken but happened to try one time due to
&& operator precedence.  7158e50ee2
broke this further when quoting && since the command was not
evaluated.
 2021-01-25 18:32:40 +09:00
Andrew Gaul
b04bca37a5
Add configuration for Chaos HTTP Proxy (#1508) 
This can find errors in retry logic.  Chaos HTTP Proxy does not
support SSL bouncestorage/chaos-http-proxy#1 so users must set
s3proxy.endpoint and run via:

CHAOS_HTTP_PROXY=1 S3_URL=http://127.0.0.1:8080 make check -C test

It can also be helpful to increase retries and reduce sleep times.
References #1504.
 2021-01-04 23:32:04 +09:00
Andrew Gaul
7158e50ee2
Quote | and && to allow passing it to retry (#1506) 2021-01-04 22:28:40 +09:00