Commit Graph

42 Commits

Author SHA1 Message Date
Andrew Gaul e14a2eb94b
Add AWS CLI config for use_sse=custom (#2230) 2023-07-25 23:31:20 +09:00
Takeshi Nakatani e715b77307
Added the function to update mtime/ctime of the parent directory (#2016) 2023-02-12 17:59:40 +09:00
Takeshi Nakatani 2e77920943
Added support for xattr as POSIX ACL (#2039) 2022-10-22 21:48:02 +09:00
Takeshi Nakatani d22e1dc018 Add the stream upload which starts uploading parts before Flush 2022-07-17 22:20:45 +09:00
Andrew Gaul 8b90cd6ba1
Enable notsup_compat_dir by default (#1970)
Few applications create the dir_$folder$ objects and users can enable
compat_dir if required.  This commit reduces readdir latency by 33%.
Also remove notsup_compat_dir from tests since these directories are
never created.  Fixes #927.  References #1643.
2022-06-28 07:56:06 +09:00
Andrew Gaul 5c57e17b77
Enable noobj_cache by default (#1922)
This should improve performance in many situations.  s3fs already
enables the stat cache by default so memorizing noobj makes this more
consistent.  Fixes #1901.
2022-03-12 16:57:31 +09:00
Andrew Gaul 74c11ef226
Check bucket before trying to create it (#1874)
This makes it easier to run tests against S3 other than S3Proxy.
2022-01-26 23:42:12 +09:00
Takeshi Nakatani 46014397d8 Added test by a shell script static analysis tool(ShellCheck) 2022-01-22 22:23:08 +09:00
Andrew Gaul e734763002
Remove createbucket option (#1841)
AWS CLI can do this.  Fixes #1840.
2022-01-05 01:59:31 +09:00
Andrew Gaul e289915dcb
Remove require-root script (#1800)
Tests do not require this.
2021-10-31 10:48:15 +09:00
Takeshi Nakatani 34f89e5936 Added fake_diskfree option to deceive free disk space for test 2021-08-02 11:04:42 +09:00
Andrew Gaul 9cfa177af0
Add flag to run all tests (#1710)
This allows CI to run the full set of tests while making local
development faster.
2021-07-02 08:41:47 +09:00
Andrew Gaul 7f6fbb0021
Correct singlepart_copy_limit unit (#1561)
This is documented to be in MB not KB.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
2021-02-11 22:53:50 +09:00
Andrew Gaul 081d6c1245 Allow setting signature V4-only
Default to allowing V2 or V4.
2020-10-01 20:03:14 +09:00
Andrew Gaul 8fb70c5e4a Fix expandtab settings for tests
This matches the main source.
2020-09-15 22:11:14 +09:00
Takeshi Nakatani b5ffd419d8 Source file division and set 4 spaces and cleanup 2020-08-26 17:43:50 +09:00
Andrew Gaul f373df9682 Test cache eviction 2020-04-11 19:00:38 +09:00
Takeshi Nakatani 32ae0d2c79 Bypassed MacOS ensure_diskfree test 2020-02-04 14:00:47 +00:00
yongqingliu b762a0a85b fix deadlock due to fdmanager_lock and fdent_data_lock 2020-01-29 12:03:53 +08:00
Andrew Gaul aba8e6ccfa Test enable_noobj_cache 2019-08-01 12:39:11 -07:00
Takeshi Nakatani 3161bf4608
Merge pull request #1070 from gaul/bash/nounset
Prohibit pipeline failures
2019-07-07 15:10:53 +09:00
Andrew Gaul 4f23f38583 Individually test multiple s3fs flags
Remove unneeded comments; single part limits ensure that the tests
exercise multipart code paths even with smaller files.
References #971.
2019-07-03 21:09:40 -07:00
Andrew Gaul 9472ee4a01 Prohibit pipeline failures 2019-07-03 02:57:40 -07:00
Robb Kistler 7bfaa24d25 Integration test summary, continue on error
Details in README.md and s3fs-integration-test-common.sh

Factor out s3fs-fuse and s3proxy start/stop.  The plan is to make it easier to
add test suites besides small-integration-test.sh that can test with various
s3fs options.

Each test run starts in a uniquely named at the top of the bucket.  This allows
multiple runs against persistent storage without worrying about cleaning
up in error conditions that leave artifiacts behind.

Tests continues if a test case fails.

Results are summarized at the end of the test run

Environment variable to control debug level of s3fs-fuse

Environment variable to enable public bucket (makes it easier to poke
around with tools like curl)

Environment variable to start s3fs-fuse under valgrind

Environment variable that casues script sets up s3fs-fuse and then wait
indefinitely, making it easy to experiment manually with the mount
point.

Additional test case
2016-02-05 05:40:28 -08:00
Robb Kistler 1927ccfe0a Don't loop on fusermount if mountpoint is gone 2015-12-15 15:07:00 -08:00
Takeshi Nakatani 1b323a6252 Changed debug option to dbglevel in test script. 2015-10-18 17:31:31 +00:00
Robb Kistler 3d5b8a7672 Use 127.0.0.1 not localhost in s3proxy wait loop
localhost doesn't always resolve to 127.0.0.1
2015-09-16 00:06:41 -07:00
Robb Kistler 6a8a2e4800 Allow integration testing against Amazon S3
Example command line:
S3FS_CREDENTIALS_FILE=keyfile \
TEST_BUCKET_1=somebucket \
S3PROXY_BINARY="" \
URL="http://s3.amazonaws.com" ./small-integration-test.sh
2015-09-11 14:35:12 -07:00
Andrew Gaul a157ac59ca Enable Content-MD5 during multipart upload part
This allows retries of multi-part uploads instead of discovering a
fatal error during complete multipart upload.  Also enable Content-MD5
for integration tests and refactor hexadecimal code.
2015-08-18 02:54:00 -07:00
Andrew Gaul ff8a0c2eea Parse ETag from copy multipart correctly
Previously s3fs misparsed this, preventing renames of files larger
than 5 GB.  Integration test disabled until S3Proxy 1.5.0 is released.
2015-08-11 14:43:35 -07:00
Ka-Hing Cheung 7c298e94f5 add exit handler to cleanup on failures
and other changes that make debugging easier
2015-03-09 15:56:38 -07:00
Andrew Gaul 0f7d77d599 Small fixes to integration tests
Use S3Proxy pid instead of self pid, ensure correct passwd
permissions, and use fusermount instead of umount so that non-root can
run tests.
2015-03-03 01:42:03 -08:00
Andrew Gaul e811ae1104 Use s3proxy to run integration tests
References #129.
2015-02-24 12:08:22 -08:00
Ka-Hing Cheung 03d84a07d1 fix rename before close
nautilus does this when you drag and drop to overwrite a file:

1) create .goutputstream-XXXXXX to write to
2) fsync the fd for .goutputstream-XXXXXX
3) rename .goutputstream-XXXXXX to target file
4) close the fd for .goutputstream-XXXXXX

previously, doing this on s3fs would result in an empty target file
because after the rename, s3fs would not flush the content of
.goutputstream-XXXXXX to target file.

this change moves the FdEntity from the old path to the new path
whenever rename happens. On flush s3fs would now flush the correct
content to the rename target.
2015-01-12 15:05:54 -08:00
ben.lemasurier@gmail.com 2a09e0864e Fixed a possible memory leak in the stat cache where
- items with an initial hit count of 0 would not be deleted

Added an additiional integration test



git-svn-id: http://s3fs.googlecode.com/svn/trunk@383 df820570-a93a-0410-bd06-b72b767a4274
2011-09-26 15:20:14 +00:00
mooredan@suncup.net c8d5b35f8f Resolves issue #154
Installed and tested fix for file permissions/cache issue


git-svn-id: http://s3fs.googlecode.com/svn/trunk@311 df820570-a93a-0410-bd06-b72b767a4274
2011-02-11 03:30:02 +00:00
mooredan@suncup.net 6f7e180133 Resolves issue #152
- added move directory test
- fix bug introduced with fixing issue #150



git-svn-id: http://s3fs.googlecode.com/svn/trunk@310 df820570-a93a-0410-bd06-b72b767a4274
2011-02-10 01:07:46 +00:00
mooredan@suncup.net 0f18298886 Fix for issue #145 and additional tests in make check
git-svn-id: http://s3fs.googlecode.com/svn/trunk@301 df820570-a93a-0410-bd06-b72b767a4274
2011-01-19 05:26:01 +00:00
mooredan@suncup.net acc7363433 Checkpoint for implementation of multipart upload
Check issue #142 for details

Code is operational, but not quite ready for
prime time -- needs some clean up


git-svn-id: http://s3fs.googlecode.com/svn/trunk@297 df820570-a93a-0410-bd06-b72b767a4274
2010-12-28 04:15:23 +00:00
mooredan@suncup.net 5c64ff83cf Restructing to take care of the directory rename.
This is a checkpoint. No functional changes in this commit.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@288 df820570-a93a-0410-bd06-b72b767a4274
2010-12-20 05:26:27 +00:00
mooredan@suncup.net 99aa781701 Clean up from last time just in case last time had
an error and left the test file in the bucket


git-svn-id: http://s3fs.googlecode.com/svn/trunk@276 df820570-a93a-0410-bd06-b72b767a4274
2010-12-08 03:08:10 +00:00
mooredan@suncup.net e9b8216d21 In preparation to remove the unnecessary "s3fs"
directory from the trunk directory.

First do a svn cp of all of the source up to
trunk.  This is supposed to preserve change
history -- we'll see.

The source remains untouched until this gets
worked out.

Also in preparation of bringing in the source
collateral for the debian package into the
repository. I expect that the top level will
look like this:

svn/
   s3fs/
      trunk/
      tags/
      branches/
   dpkg/
      trunk/
      tags/
      branches/


So far that's how it is looking.  I'll be
very careful to ensure integrity of the data.
As a result this may be a multistep process.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@236 df820570-a93a-0410-bd06-b72b767a4274
2010-11-13 23:59:23 +00:00