Previously when s3fs had dirty local data and an application opened a
second fd it would remove the stat cache entry, query S3 for the size
which was still zero, then reinitialize FdEntry with this incorrect
size. Instead flush local data to S3 so this query works. It is
possible that a more involved patch could do this with a less
heavyweight approach but this requires changing open. This does not
completely fix git clone but allows it to proceed further. Also make
some cache methods const-correct. References #839.
Check the modification times to determine whether an object has
updated. This relies on low clock skew between s3fs and the S3
server; a more robust approach could use the ETag. Fixes#1047.
Previously s3fs had a race condition where one client could delete
FdEntity that another client was using. Add a simple concurrent test
which previously failed but now succeeds. Fixes#964.
Previously s3fs experienced data loss when writing to the middle of a
file. Corrupt files would have the expected data from 0..offset+size
but unexpected NUL bytes from offset+size..EOF. References #808.
Details in README.md and s3fs-integration-test-common.sh
Factor out s3fs-fuse and s3proxy start/stop. The plan is to make it easier to
add test suites besides small-integration-test.sh that can test with various
s3fs options.
Each test run starts in a uniquely named at the top of the bucket. This allows
multiple runs against persistent storage without worrying about cleaning
up in error conditions that leave artifiacts behind.
Tests continues if a test case fails.
Results are summarized at the end of the test run
Environment variable to control debug level of s3fs-fuse
Environment variable to enable public bucket (makes it easier to poke
around with tools like curl)
Environment variable to start s3fs-fuse under valgrind
Environment variable that casues script sets up s3fs-fuse and then wait
indefinitely, making it easy to experiment manually with the mount
point.
Additional test case
Open the test file once outside of the tests for loop. This helps avoid
object consistency problems when running against S3 providers without
strong consistency (like Amazon). See Issue #263.
This also encodes asterisk and tilde correctly when listing a file
with a V4 auth endpoint. Also add tests for special characters
although s3proxy does not yet support V4 auth.
Fixes#188. Fixes#194.
nautilus does this when you drag and drop to overwrite a file:
1) create .goutputstream-XXXXXX to write to
2) fsync the fd for .goutputstream-XXXXXX
3) rename .goutputstream-XXXXXX to target file
4) close the fd for .goutputstream-XXXXXX
previously, doing this on s3fs would result in an empty target file
because after the rename, s3fs would not flush the content of
.goutputstream-XXXXXX to target file.
this change moves the FdEntity from the old path to the new path
whenever rename happens. On flush s3fs would now flush the correct
content to the rename target.
