octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-12-23 09:18:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,284 Commits 1 Branch 23 Tags 13 MiB
499577c2a9
Commit Graph

170 Commits

Author SHA1 Message Date
Andrew Gaul
6300859c80
Prefer = over == for older shell compatibility (#1857) 2022-01-14 12:40:55 +09:00
Andrew Gaul
adb58af17b
Annotate local variables (#1844) 
This prevents collisions with other globals.  Fixes #1843.
 2022-01-09 13:03:36 +09:00
Andrew Gaul
a44ea7c12a
Replace write_multiple_offsets.py with write_multiblock (#1837) 
This reduces the dependency on Python 2.
 2022-01-09 10:51:17 +09:00
Andrew Gaul
07e2e3f72a
Remove sleep 1 from test_update_directory_time (#1803) 
Reduces per-flag test run-time by 5 seconds.
 2021-11-04 08:16:40 +09:00
Andrew Gaul
06dec32965
Use AWS CLI to create explicit times in the past (#1797) 
s3fs can also do this via utimensat but tests should not trust this.
Also break tests into individual functions.  This further reduces test
run-time 8 seconds per flag.
 2021-10-30 10:54:18 +09:00
Andrew Gaul
86317dd185
Replace dd if=/dev/urandom with junk data generator (#1786) 
This reduces test run time for a single flag from 73 to 60 seconds.
 2021-10-28 22:54:25 +09:00
Andrew Gaul
40d2e0d1ad
Reduce sleep time to 1 (#1793) 
This reduces test run-time 15 seconds per flag or 2.5 minutes when
testing all flags.
 2021-10-27 23:47:08 +09:00
Takeshi Nakatani
b6c5069ef7 Fixed the test is multi-block writing by one flush 2021-10-27 08:19:05 +09:00
Andrew Gaul
495d51113c
Remove unneeded sleeps from tests (#1784) 
Also use a unique file name for every test.  This ensures that tests
like test_external_directory_creation and test_external_modification
do not collide.
 2021-10-26 23:19:14 +09:00
Takeshi Nakatani
34ea2acd75 Add a test that is multi-block writing by one flush 2021-10-25 17:56:49 +09:00
Takeshi Nakatani
ea64886469 Fixed a bug in test_(zero_)cache_file_stat test function 2021-10-24 18:24:12 +09:00
Takeshi Nakatani
023aaf7dff Fixed wrong stat of cache after new creation file 
And added a test for stat of cache after new creation file
 2021-10-17 16:10:14 +09:00
Andrew Gaul
2154e898bc
Fix typos (#1700) 2021-06-27 11:22:33 +09:00
Takeshi Nakatani
f9e80f995d Fixed a bug about rename existing file of different sizes by mpcopy 2021-06-26 00:04:58 +09:00
Takeshi Nakatani
0555c4216e Added test case for utimens is calling during multipart upload 2021-06-21 17:20:24 +09:00
Takeshi Nakatani
9d8f1b00f7 Fixed make_random_string test code being slow on macos 2021-05-08 13:55:39 +09:00
Andrew Gaul
771bbfeac5
Do not create zero-byte object when creating file (#1640) 
Previously s3fs created this object to store metadata and overwrote it
when flushing.  This prevented use with object stores which do not
allow overwrites like HDS.  Instead only create an in-memory
representation which reduces the time to create small files.
Fixes #1013.
 2021-04-30 19:56:33 +09:00
Andrew Gaul
8a51a26819 Allow truncation of open and modified files 
Regression introduced in f5bf41cf11.
Fixes #1575.
 2021-04-25 12:35:35 +09:00
Takeshi Nakatani
a19d223434 Bypass an OSX test(no different from other OS) which takes very time consuming 2021-02-23 20:47:51 +09:00
Takeshi Nakatani
5b5bc3114a Fixed a bug that padded null bytes when changing xattr 2021-02-21 12:23:36 +09:00
Takeshi Nakatani
81e267d421 Fixed the same bug in chmod/chown/utimens and added tests 2021-02-20 23:30:50 +09:00
Takeshi Nakatani
d33f252404 Fixed forgetting to clean up test files 2021-02-14 00:07:46 +09:00
Andrew Gaul
f2c5e38724
Allow integration tests to use larger files (#1548) 
Previously these failed due to dd only copying 32 MB from /dev/urandom
and exhausting the limited space in /tmp.  References #1543.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
 2021-02-07 12:50:02 +09:00
Takeshi Nakatani
4d0daddad4
Fixed about Github Actions failure(centos8 and macos) (#1498) 2020-12-23 19:34:04 +09:00
Takeshi Nakatani
4341291cc2
Changed a condition of the atime test(fixed #1477) (#1487) 2020-12-03 02:45:05 +09:00
Takeshi Nakatani
c589886ba5
Added a condition to the atime test (#1477) 2020-12-02 23:21:25 +09:00
Takeshi Nakatani
059cc57ba6 Added atime and Corrected atime/mtime/ctime operations 2020-10-04 13:54:01 +09:00
Andrew Gaul
89b1c32b24 Rework s3fs pid handling 
Previously S3FS_PID was not set in the correct shell and thus
ps u $S3FS_PID showed all programs.  This caused the flag detection to
fail when users ran other instances of s3fs with different flags.
References #1402.
 2020-09-18 22:05:37 +09:00
Takeshi Nakatani
76d88f2291
Merge pull request #1403 from gaul/vim/expandtab 
Fix expandtab settings for tests
 2020-09-17 21:35:58 +09:00
Takeshi Nakatani
72340cfbd9
Merge pull request #1401 from gaul/test/hardlink 
Add test for hardlink
 2020-09-17 20:04:07 +09:00
Andrew Gaul
8fb70c5e4a Fix expandtab settings for tests 
This matches the main source.
 2020-09-15 22:11:14 +09:00
Andrew Gaul
78e2345c19 Add test for hardlink 2020-09-15 22:07:25 +09:00
Andrew Gaul
3bc565b986 Add test for concurrent reads 2020-09-15 21:25:51 +09:00
Takeshi Nakatani
b5ffd419d8 Source file division and set 4 spaces and cleanup 2020-08-26 17:43:50 +09:00
Takeshi Nakatani
2feefeec47 Added mv file test when disk space is insufficient 2020-08-16 12:47:29 +00:00
Takeshi Nakatani
2bb745cdd7 Fixed a bug about move file over limit of ensure space 2020-08-16 17:42:20 +09:00
Andrew Gaul
0c1bc0f803 Add portability wrapper for stat(1) 
Fixes #1344.
 2020-08-02 23:14:58 +09:00
Andrew Gaul
4154e539ea Test creating a directory with external program 
References #1264.
 2020-07-25 21:22:53 +09:00
Takeshi Nakatani
62c8be85d6 Interim fix for Travis macOS Build 2020-07-10 09:47:18 +09:00
Takeshi Nakatani
1f796d432d Fixed upload error about mixuploading sparse file and truncating file 2020-06-27 22:44:19 +09:00
Takeshi Nakatani
35006e318f Fixed about ParallelMixMultipartUpload 2020-06-24 12:48:55 +09:00
Andrew Gaul
8a7548a9d4 Import ossfs tests 
This turns up a failure in test_rename_file when calling seek(0) on a
renamed file.
 2020-06-01 09:08:25 +09:00
Andrew Gaul
8945e98d8b Support Google Cloud Storage headers 
This allows s3fs to interpret objects created by gsutil.
 2020-05-03 18:33:13 +09:00
Takeshi Nakatani
ce1221c867 Add test for cache file stat content 2020-04-22 15:53:00 +00:00
Andrew Gaul
3af17c3019 Add test for out-of-order writes 
References #1220.  References #1277.
 2020-04-22 21:48:55 +09:00
Takeshi Nakatani
538fbed302
Merge pull request #1266 from gaul/test/cache-eviction 
Test cache eviction
 2020-04-14 22:30:04 +09:00
Andrew Gaul
a44fc1103d Avoid setting bogus group id 
Addresses a symptom on newer macOS in Travis.
 2020-04-12 22:37:22 +09:00
Andrew Gaul
f373df9682 Test cache eviction 2020-04-11 19:00:38 +09:00
Takeshi Nakatani
7fbda230f5 Added mime option for strict checking of mime types file 2020-03-30 14:41:18 +00:00
Takeshi Nakatani
0665d78550 Temporary measure: reduce test time on MacOS 2020-02-04 15:38:19 +00:00
Takeshi Nakatani
55d670f22f Change content-types test for MacOS 2020-02-04 14:51:50 +00:00
Andrew Gaul
bc9126d774 Set directory MIME type to application/x-directory 
Previously s3fs auto-detected the MIME type of directories like
"TOYOTA TRUCK 8.2.2" as application/x-troff-man.  This caused get_mode
to not set S_IFDIR which failed directory creation.  Instead force all
object names ending in / to application/x-directory.  Fixes #1183.
 2020-02-04 20:03:21 +09:00
Andrew Gaul
4df50e7f85 Test that s3fs automatically detects Content-Type 
References #1217.
 2020-02-04 19:47:09 +09:00
Takeshi Nakatani
018ccb9a11 Suppressed a lot of message output about cache cleanup 2020-01-30 14:42:28 +00:00
Takeshi Nakatani
ee1d3a9057
Merge pull request #1151 from liuyongqing/master 
fix deadlock in clean up cache
 2020-01-30 22:17:50 +09:00
yongqingliu
b762a0a85b fix deadlock due to fdmanager_lock and fdent_data_lock 2020-01-29 12:03:53 +08:00
Takeshi Nakatani
010a6b83ef Not trap chown command errors directly 2020-01-28 10:32:55 +00:00
Takeshi Nakatani
9e77650e8c Fixed removing test file in integration-test-main.sh 2020-01-26 13:32:41 +00:00
Takeshi Nakatani
cf3e82d10a Fixed a rename bug when enable_noobj_cache 2019-10-23 12:39:42 +00:00
Takeshi Nakatani
27a5536749 Fixed test_write_multiple_offsets error on osx 2019-10-14 17:29:39 +00:00
Andrew Gaul
80162c126b Add test for writing to multiple offsets 
References #1098.
 2019-09-25 19:31:08 -07:00
Takeshi Nakatani
1db94a0b30 Fixed to use copy api in multipart upload 2019-09-25 19:30:58 -07:00
liuyongqing
ae4bcd405c use correct content-type when overwriting and copying files 2019-08-28 22:25:09 -07:00
Andrew Gaul
2daa1d53d9 Verify file size via metadata as well as data 2019-08-20 18:36:44 -07:00
Takeshi Nakatani
a7d83df3eb Fixed typo in #1129 2019-08-14 12:39:38 +00:00
Takeshi Nakatani
ca7266fb76 Modified the test script a little 2019-08-13 14:21:42 +00:00
Takeshi Nakatani
f5bf41cf11 Fixed simple copy with nocpoyapi and modify flag in fdcache 2019-08-11 07:42:48 +00:00
Takeshi Nakatani
d8f1aef7be Added stats cache expire option for test 2019-08-06 10:55:38 +00:00
Takeshi Nakatani
dc85eda188 Added direct_io option for test on osx 2019-08-05 17:12:49 +00:00
Andrew Gaul
c321c8c23f Add test to read an externally created object 
References #890.
 2019-08-01 16:12:51 -07:00
Andrew Gaul
5cb7a31c09 Add a test for concurrent writes 2019-07-16 21:57:43 -07:00
Takeshi Nakatani
c7f8f61d09 Change test_concurrency test 2019-07-15 15:58:54 -07:00
Andrew Gaul
4a192ffdf9 Run xattr tests on macOS 2019-07-07 00:39:45 -07:00
Takeshi Nakatani
3161bf4608
Merge pull request #1070 from gaul/bash/nounset 
Prohibit pipeline failures
 2019-07-07 15:10:53 +09:00
Takeshi Nakatani
2349dafb98
Merge pull request #1069 from gaul/macos/sed 
Use system sed on macOS
 2019-07-07 15:00:48 +09:00
Andrew Gaul
9472ee4a01 Prohibit pipeline failures 2019-07-03 02:57:40 -07:00
Andrew Gaul
1f1f824da7 Use system sed on macOS 
This requires fewer developer customizations to work on macOS.
Requires some GNU workarounds.
 2019-07-02 22:17:02 -07:00
Andrew Gaul
c596441f58 Flush file when opening second fd 
Previously when s3fs had dirty local data and an application opened a
second fd it would remove the stat cache entry, query S3 for the size
which was still zero, then reinitialize FdEntry with this incorrect
size.  Instead flush local data to S3 so this query works.  It is
possible that a more involved patch could do this with a less
heavyweight approach but this requires changing open.  This does not
completely fix git clone but allows it to proceed further.  Also make
some cache methods const-correct.  References #839.
 2019-07-02 01:12:09 -07:00
Andrew Gaul
048aea1151 Remove litter from test_concurrency 2019-06-26 20:29:27 -07:00
Andrew Gaul
9e5eaad79b Remove cache file when object time differs 
Check the modification times to determine whether an object has
updated.  This relies on low clock skew between s3fs and the S3
server; a more robust approach could use the ETag.  Fixes #1047.
 2019-06-22 19:09:00 -07:00
Andrew Gaul
f53503438c Increase FdEntity reference count when returning 
Previously s3fs had a race condition where one client could delete
FdEntity that another client was using.  Add a simple concurrent test
which previously failed but now succeeds.  Fixes #964.
 2019-03-09 15:55:04 +09:00
Takeshi Nakatani
d487348d21
Merge pull request #919 from gaul/test/mv-nonempty-dir 
Add test for mv non-empty directory
 2019-01-23 19:21:12 +09:00
Andrew Gaul
79bd3441eb Add test for mv non-empty directory 2019-01-22 22:13:17 -08:00
Andrew Gaul
5f5da4b2cb Load tail range during overwrite 
Previously s3fs experienced data loss when writing to the middle of a
file.  Corrupt files would have the expected data from 0..offset+size
but unexpected NUL bytes from offset+size..EOF.  References #808.
 2019-01-22 22:02:40 -08:00
Andrew Gaul
6401b4ae92 Flush file before renaming 
Previously s3fs could copy the zero-byte stub object without including
any pending writes.  Fixes #145.
 2019-01-17 12:05:10 -08:00
Andrew Gaul
2c43b1e12b Store and retrieve file change time 
This introduces a new header with the change time; existing objects
will report modification time.  Fixes #771.
 2019-01-14 10:05:11 -08:00
Takeshi Nakatani
62b8084300 Added travis test on osx for #601 2017-08-11 14:09:43 +00:00
Takeshi Nakatani
7aae4782d9 Merge pull request #504 from andrewgaul/test 
Use describe helper function
 2016-11-20 09:10:16 +09:00
Andrew Gaul
cd0c8599cc Use describe helper function 2016-11-19 15:36:02 -08:00
Andrew Gaul
20878a1618 Add missing call to mtime test 2016-11-19 15:10:41 -08:00
Andrew Gaul
cafe6015e3 Upgrade to S3Proxy 1.5.1 
Enabled previously broken tests and test with default v4 signer.
Release notes:

https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.0
https://github.com/andrewgaul/s3proxy/releases/tag/s3proxy-1.5.1
 2016-11-15 23:09:48 -08:00
Takeshi Nakatani
ec110bb0f3 Added small logic in test script for test_chown 2016-09-11 13:41:50 +00:00
Takeshi Nakatani
232befb52a Added small logic in test script for test_chown 2016-09-11 13:37:53 +00:00
Takeshi Nakatani
f363c21ff5 Added comments in failure message for test_chown 2016-09-11 13:28:40 +00:00
Tianlong Wu
cf23dc78ab Use 'return' instead of 'exit' in test 2016-04-22 16:24:26 +08:00
Tianlong Wu
115bd51f3f Fix a bug of truncating empty file 2016-04-22 14:49:37 +08:00
Robb Kistler
4e583583cd Test for writing after an lseek past end of file 
This is a test to demonstrate Issue #375
 2016-03-23 16:03:38 -07:00
Robb Kistler
7bfaa24d25 Integration test summary, continue on error 
Details in README.md and s3fs-integration-test-common.sh

Factor out s3fs-fuse and s3proxy start/stop.  The plan is to make it easier to
add test suites besides small-integration-test.sh that can test with various
s3fs options.

Each test run starts in a uniquely named at the top of the bucket.  This allows
multiple runs against persistent storage without worrying about cleaning
up in error conditions that leave artifiacts behind.

Tests continues if a test case fails.

Results are summarized at the end of the test run

Environment variable to control debug level of s3fs-fuse

Environment variable to enable public bucket (makes it easier to poke
around with tools like curl)

Environment variable to start s3fs-fuse under valgrind

Environment variable that casues script sets up s3fs-fuse and then wait
indefinitely, making it easy to experiment manually with the mount
point.

Additional test case
 2016-02-05 05:40:28 -08:00
Takeshi Nakatani
4bd5ffb0fa Update integration-test-main.sh as additional change for #300 2015-12-03 13:44:43 +00:00
bazeli
b85bd53336 Update integration-test-main.sh 
new test for mtime preservation copying file with `cp -p`
 2015-11-24 17:29:54 +09:00