2e77920943
Added support for xattr as POSIX ACL ( #2039 )
2022-10-22 21:48:02 +09:00
4a813aec42
Support the object under no directory object path by compat_dir ( #2023 )
2022-10-22 15:12:00 +09:00
4304ec63bb
Added stat information to the mount point ( #1964 )
2022-10-22 11:46:13 +09:00
4b2f3fecb5
Set mtime/ctime/atime of all objects as nanosecond
2022-07-28 13:47:03 +09:00
11adf11957
Changed the time(a/c/m) acquisition of stat to nanosecond string
2022-07-25 07:21:37 +09:00
136c5ec653
Fixed a bug in the test_external_creation test
2022-07-18 22:34:53 +09:00
d22e1dc018
Add the stream upload which starts uploading parts before Flush
2022-07-17 22:20:45 +09:00
3a0799ec18
Re-re-re-fix propagating the return code ( #1984 )
...
This shell incantation is necessary to have errexit take effect for
test bodies. Previous only the _last_ command's exit code was
propagated to the test runner, masking real failures. Reverts
ea3c21f270
2022-07-17 02:13:06 +09:00
86da2eed3a
Do not call exit in individual test functions ( #1985 )
...
This could prevent the test runner from reporting failures.
2022-07-17 01:33:50 +09:00
4d303caa62
Add Valgrind to sanitizer script ( #1978 )
...
Theoretically msan can do this but practically it requires a custom
libc++.
2022-07-09 16:40:23 +09:00
f11eb7d69b
Fixed a bug that regular files could not be created by mknod
2022-06-29 16:56:19 +09:00
73b49c1038
Fixed a bug that regular files could not be created by mknod
2022-06-29 16:56:19 +09:00
8b90cd6ba1
Enable notsup_compat_dir by default ( #1970 )
...
Few applications create the dir_$folder$ objects and users can enable
compat_dir if required. This commit reduces readdir latency by 33%.
Also remove notsup_compat_dir from tests since these directories are
never created. Fixes #927 . References #1643 .
2022-06-28 07:56:06 +09:00
ac72bf34dd
Run test_external_directory_creation for all flags ( #1969 )
2022-06-28 07:50:24 +09:00
ec7810f08e
Fixed a bug could not change the mode while the file was opened
2022-06-12 13:48:42 +09:00
a30beded1c
Removed unnecessary debug options for aws command
2022-04-22 22:36:06 +09:00
df7bbb28d5
Replace awscli version 1 to 2
2022-04-19 23:11:25 +09:00
dc40f16161
Upgrade to S3Proxy 2.0.0 ( #1929 )
...
Release notes:
https://github.com/gaul/s3proxy/releases/tag/s3proxy-2.0.0
This enables Java 17 compatibility.
2022-04-04 23:03:16 +09:00
5c57e17b77
Enable noobj_cache by default ( #1922 )
...
This should improve performance in many situations. s3fs already
enables the stat cache by default so memorizing noobj makes this more
consistent. Fixes #1901 .
2022-03-12 16:57:31 +09:00
d842d45b2b
Fixed a bug about truncation for shrinking file
2022-03-02 22:41:10 +09:00
afb0897553
Typos
2022-02-24 19:15:00 +09:00
01e24967b6
Add test for external object creation ( #1900 )
...
This test demonstrates the behavior before and after the stat cache
timeout when using noobj_cache.
2022-02-23 23:34:58 +09:00
0842c5718f
Use more new file names for every test ( #1902 )
...
This makes the tests more robust. Also fix filename to end in .txt.
2022-02-23 22:59:21 +09:00
232ff28cc7
Re-re-fix propagating the return code ( #1903 )
...
Previously the integration tests were exiting after the first failed
test instead of running all of them an reporting their statuses.
Follows on to dbf93c0152
2022-02-23 14:27:29 +09:00
305d660e39
Use custom CA bundle instead of ignoring errors ( #1910 )
...
Fixes #1846 .
2022-02-23 10:04:05 +09:00
302150b4f5
Filter mountpoints via mount -t ( #1905 )
...
This is portable between Linux and macOS.
2022-02-20 20:49:35 +09:00
3fa03d4e1e
Pass explicit -p option to ps ( #1904 )
...
This ensures that a pid follows.
2022-02-20 20:40:29 +09:00
94e8e23eef
Fixed test_external_directory_creation test when cache enabled ( #1885 )
2022-02-13 13:32:19 +09:00
dbf93c0152
Propagate return code properly ( #1884 )
...
Previously this did not propagate test failures. A bad rebase
introduced this logic in 495d51113c
2022-02-06 22:45:20 +09:00
30cf7a50bb
Added stat check for subdir created with awscli
2022-01-30 18:31:36 +09:00
74c11ef226
Check bucket before trying to create it ( #1874 )
...
This makes it easier to run tests against S3 other than S3Proxy.
2022-01-26 23:42:12 +09:00
46014397d8
Added test by a shell script static analysis tool(ShellCheck)
2022-01-22 22:23:08 +09:00
6300859c80
Prefer = over == for older shell compatibility ( #1857 )
2022-01-14 12:40:55 +09:00
2892d3b755
Call curl --fail to propagate exit code ( #1856 )
2022-01-14 11:52:52 +09:00
577e2bc987
Generate S3Proxy SSL certificate during tests ( #1845 )
...
Also provide CA bundle to AWS CLI to work around CI failures instead
of ignoring errors. Fixes #1812 .
2022-01-09 15:13:36 +09:00
adb58af17b
Annotate local variables ( #1844 )
...
This prevents collisions with other globals. Fixes #1843 .
2022-01-09 13:03:36 +09:00
fc7543fa25
Make ut_test compatible with Python 3 ( #1838 )
...
This may allow removing Python 2 on newer distros.
Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
2022-01-09 11:08:36 +09:00
a44ea7c12a
Replace write_multiple_offsets.py with write_multiblock ( #1837 )
...
This reduces the dependency on Python 2.
2022-01-09 10:51:17 +09:00
e734763002
Remove createbucket option ( #1841 )
...
AWS CLI can do this. Fixes #1840 .
2022-01-05 01:59:31 +09:00
616db5bf1c
Prefer curl over wget for fetching dependencies ( #1836 )
...
The former is lighter-weight and libcurl is already a dependency for
s3fs.
2022-01-05 00:43:36 +09:00
07e2e3f72a
Remove sleep 1 from test_update_directory_time ( #1803 )
...
Reduces per-flag test run-time by 5 seconds.
2021-11-04 08:16:40 +09:00
3cf00626a2
Add option to allow unsigned payloads ( #1801 )
...
This reduces CPU usage of sigv4. This reduces test run-time by 7
seconds per flag.
2021-11-01 23:33:55 +09:00
e289915dcb
Remove require-root script ( #1800 )
...
Tests do not require this.
2021-10-31 10:48:15 +09:00
06dec32965
Use AWS CLI to create explicit times in the past ( #1797 )
...
s3fs can also do this via utimensat but tests should not trust this.
Also break tests into individual functions. This further reduces test
run-time 8 seconds per flag.
2021-10-30 10:54:18 +09:00
86317dd185
Replace dd if=/dev/urandom with junk data generator ( #1786 )
...
This reduces test run time for a single flag from 73 to 60 seconds.
2021-10-28 22:54:25 +09:00
40d2e0d1ad
Reduce sleep time to 1 ( #1793 )
...
This reduces test run-time 15 seconds per flag or 2.5 minutes when
testing all flags.
2021-10-27 23:47:08 +09:00
b6c5069ef7
Fixed the test is multi-block writing by one flush
2021-10-27 08:19:05 +09:00
72a9f40f3f
Update to S3Proxy 1.9.0 ( #1788 )
...
Notably this fixes an issue with the transient provider reading parts
of large files.
Release notes:
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.9.0
2021-10-26 23:20:52 +09:00
495d51113c
Remove unneeded sleeps from tests ( #1784 )
...
Also use a unique file name for every test. This ensures that tests
like test_external_directory_creation and test_external_modification
do not collide.
2021-10-26 23:19:14 +09:00
0abeec9cae
Simpily errexit modifications
2021-10-26 21:47:36 +09:00
ea3c21f270
Reduce errexit modifications ( #1785 )
...
This is less error prone but requires some magic && ||.
2021-10-25 23:53:45 +09:00
23fe6f4dee
Fixed parse_string function in write_multiblock.cc
2021-10-25 17:56:49 +09:00
34ea2acd75
Add a test that is multi-block writing by one flush
2021-10-25 17:56:49 +09:00
ea64886469
Fixed a bug in test_(zero_)cache_file_stat test function
2021-10-24 18:24:12 +09:00
023aaf7dff
Fixed wrong stat of cache after new creation file
...
And added a test for stat of cache after new creation file
2021-10-17 16:10:14 +09:00
34f89e5936
Added fake_diskfree option to deceive free disk space for test
2021-08-02 11:04:42 +09:00
9cfa177af0
Add flag to run all tests ( #1710 )
...
This allows CI to run the full set of tests while making local
development faster.
2021-07-02 08:41:47 +09:00
2154e898bc
Fix typos ( #1700 )
2021-06-27 11:22:33 +09:00
f9e80f995d
Fixed a bug about rename existing file of different sizes by mpcopy
2021-06-26 00:04:58 +09:00
944c1fd148
Clean up litter in ut_test ( #1697 )
2021-06-25 19:15:03 +09:00
0555c4216e
Added test case for utimens is calling during multipart upload
2021-06-21 17:20:24 +09:00
1571379304
Set LC_ALL=en_US.UTF-8 in tests ( #1691 )
...
This allows tests to pass using the S3Proxy filesystem provider.
References #1665 .
2021-06-20 19:27:25 +09:00
f5701fa9ad
Correct typo in undefinedsanitizer config ( #1669 )
2021-05-29 22:07:33 +09:00
c0bcb41175
Increase test robustness by adding LC_ALL=C ( #1660 )
...
Use C locale, because some tests check for English expressions
Fixes #1658
2021-05-20 21:19:50 +09:00
9d8f1b00f7
Fixed make_random_string test code being slow on macos
2021-05-08 13:55:39 +09:00
771bbfeac5
Do not create zero-byte object when creating file ( #1640 )
...
Previously s3fs created this object to store metadata and overwrote it
when flushing. This prevented use with object stores which do not
allow overwrites like HDS. Instead only create an in-memory
representation which reduces the time to create small files.
Fixes #1013 .
2021-04-30 19:56:33 +09:00
8a51a26819
Allow truncation of open and modified files
...
Regression introduced in f5bf41cf11Fixes #1575 .
2021-04-25 12:35:35 +09:00
20a6c9d35d
Update to S3Proxy 1.8.0 ( #1598 )
...
Notably this fixes an issue with the filesystem backend which returned
the incorrect Content-Type for some directories. It also reduces
memory use with GetObject using range requests with large objects.
References #1543 .
Release notes:
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.8.0
2021-03-28 11:56:10 +09:00
a19d223434
Bypass an OSX test(no different from other OS) which takes very time consuming
2021-02-23 20:47:51 +09:00
5b5bc3114a
Fixed a bug that padded null bytes when changing xattr
2021-02-21 12:23:36 +09:00
81e267d421
Fixed the same bug in chmod/chown/utimens and added tests
2021-02-20 23:30:50 +09:00
a0f347b10f
Added no_time_stamp_ms option mainly for testing
2021-02-20 20:24:26 +09:00
5debf523b0
Allow changing temporary directory used for tests
...
Set temporary directory to $TMPDIR if set or use default /var/tmp
2021-02-19 13:59:23 +09:00
1c8aadafd1
Fix typo in an error message of the test suite
2021-02-19 13:59:23 +09:00
d33f252404
Fixed forgetting to clean up test files
2021-02-14 00:07:46 +09:00
7f6fbb0021
Correct singlepart_copy_limit unit ( #1561 )
...
This is documented to be in MB not KB.
Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
2021-02-11 22:53:50 +09:00
f2c5e38724
Allow integration tests to use larger files ( #1548 )
...
Previously these failed due to dd only copying 32 MB from /dev/urandom
and exhausting the limited space in /tmp. References #1543 .
Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
2021-02-07 12:50:02 +09:00
1987bcbea3
Re-fix retry logic ( #1533 )
...
This logic has always been broken but happened to try one time due to
&& operator precedence. 7158e50ee2
2021-01-25 18:32:40 +09:00
b04bca37a5
Add configuration for Chaos HTTP Proxy ( #1508 )
...
This can find errors in retry logic. Chaos HTTP Proxy does not
support SSL bouncestorage/chaos-http-proxy#1 so users must set
s3proxy.endpoint and run via:
CHAOS_HTTP_PROXY=1 S3_URL=http://127.0.0.1:8080 make check -C test
It can also be helpful to increase retries and reduce sleep times.
References #1504 .
2021-01-04 23:32:04 +09:00
7158e50ee2
Quote | and && to allow passing it to retry ( #1506 )
2021-01-04 22:28:40 +09:00
4d0daddad4
Fixed about Github Actions failure(centos8 and macos) ( #1498 )
2020-12-23 19:34:04 +09:00
4341291cc2
Changed a condition of the atime test( fixed #1477 ) ( #1487 )
2020-12-03 02:45:05 +09:00
c589886ba5
Added a condition to the atime test ( #1477 )
2020-12-02 23:21:25 +09:00
834862f8a4
Add S3_ENDPOINT test variable ( #1476 )
2020-11-15 22:10:26 +09:00
ecb24c9c26
Disable preprocessor warnings w/o optimizations ( #1473 )
...
This removes some fortify source warnings. Also disable failing
memory sanitizer.
2020-11-09 21:48:07 +09:00
a4e4ce8aea
Merge pull request #1440 from gaul/test/undefined-behavior-sanitizer
...
Add memory and undefined behavior sanitizer
2020-10-04 23:19:36 +09:00
7f43b7fa53
Add memory and undefined behavior sanitizer
...
Remove Valgrind which duplicates the sanitizers and is much slower.
2020-10-04 22:55:27 +09:00
059cc57ba6
Added atime and Corrected atime/mtime/ctime operations
2020-10-04 13:54:01 +09:00
081d6c1245
Allow setting signature V4-only
...
Default to allowing V2 or V4.
2020-10-01 20:03:14 +09:00
89b1c32b24
Rework s3fs pid handling
...
Previously S3FS_PID was not set in the correct shell and thus
ps u $S3FS_PID showed all programs. This caused the flag detection to
fail when users ran other instances of s3fs with different flags.
References #1402 .
2020-09-18 22:05:37 +09:00
76d88f2291
Merge pull request #1403 from gaul/vim/expandtab
...
Fix expandtab settings for tests
2020-09-17 21:35:58 +09:00
72340cfbd9
Merge pull request #1401 from gaul/test/hardlink
...
Add test for hardlink
2020-09-17 20:04:07 +09:00
8fb70c5e4a
Fix expandtab settings for tests
...
This matches the main source.
2020-09-15 22:11:14 +09:00
78e2345c19
Add test for hardlink
2020-09-15 22:07:25 +09:00
3bc565b986
Add test for concurrent reads
2020-09-15 21:25:51 +09:00
b5ffd419d8
Source file division and set 4 spaces and cleanup
2020-08-26 17:43:50 +09:00
2feefeec47
Added mv file test when disk space is insufficient
2020-08-16 12:47:29 +00:00
f28e3bd89e
Merge pull request #1362 from gaul/test/sanitizers
...
Add script to run tests using sanitizers
2020-08-16 19:15:05 +09:00
2bb745cdd7
Fixed a bug about move file over limit of ensure space
2020-08-16 17:42:20 +09:00
c8e13300e1
Add script to run tests using sanitizers
...
These currently show several kinds of errors.
2020-08-15 17:28:35 +09:00
Takeshi Nakatani
Andrew Gaul
Carsten Grohmann