Commit Graph

1976 Commits

Author SHA1 Message Date
Andrew Gaul
19abd9ffaf Move IBM information to Non Amazon S3 wiki page
This gives consistency with other providers:

https://github.com/s3fs-fuse/s3fs-fuse/wiki/Non-Amazon-S3
2020-08-01 20:22:30 +09:00
Takeshi Nakatani
cbd925c56f Moved the SIGUSR2 handler to S3fsSignals class 2020-07-28 14:54:35 +09:00
Takeshi Nakatani
63bbb47378
Merge pull request #1341 from gaul/stat-cache/default
Change default stat_cache_expire
2020-07-26 23:54:14 +09:00
Andrew Gaul
0fbd0eac80 Change default stat_cache_expire
Previously s3fs cached files forever which confused users with
creating objects using another client.
2020-07-26 23:04:43 +09:00
Takeshi Nakatani
e5231fa3c7
Merge pull request #1340 from gaul/test/external-directory
Test creating a directory with external program
2020-07-26 22:06:40 +09:00
Takeshi Nakatani
ad1961417d Added SIGUSR1 option for cache file integrity test 2020-07-26 21:04:11 +09:00
Andrew Gaul
4154e539ea Test creating a directory with external program
References #1264.
2020-07-25 21:22:53 +09:00
Takeshi Nakatani
e0a38adaf6
Merge pull request #1333 from ggtakec/similar_processing
Put similar processing together into method GetCacheFileStatTopDir
2020-07-12 20:29:45 +09:00
Takeshi Nakatani
c3e711da58
Merge pull request #1327 from pxeger/master
`s3fs#bucketname ... fuse` -> `bucketname ... fuse.s3fs` (#1321)
2020-07-12 19:14:27 +09:00
Takeshi Nakatani
700e288718 Put similar processing together into method GetCacheFileStatTopDir 2020-07-12 08:12:02 +00:00
Takeshi Nakatani
e72a64785b
Merge pull request #1331 from gaul/travis/simplify
Simplify conditional statements
2020-07-12 12:52:21 +09:00
Andrew Gaul
5ace2b70fc Simplify conditional statements 2020-07-10 19:34:46 +09:00
Takeshi Nakatani
62c8be85d6 Interim fix for Travis macOS Build 2020-07-10 09:47:18 +09:00
Takeshi Nakatani
3f6b5ae6a5
Merge pull request #1329 from gaul/cppcheck-2.1
Fix NULL pointer deference
2020-07-09 23:00:36 +09:00
Takeshi Nakatani
dc365b65a0 Fix NULL pointer deference
Found via cppcheck 2.1.
2020-07-09 21:40:23 +09:00
Takeshi Nakatani
9c88ec2128
Merge pull request #1326 from gaul/test/python2
Use Python 2 for write_multiple_offsets
2020-07-09 18:45:41 +09:00
pxeger
3dd9832f61
s3fs#bucketname ... fuse -> bucketname ... fuse.s3fs (#1321) 2020-07-07 16:06:00 +01:00
Andrew Gaul
4d1f5c899f Use Python 2 for write_multiple_offsets
This aligns with ut_test.py .  Using an older Python also allows
compatibility with the older macOS 10.12 Travis CI.  References #1323.
2020-07-07 21:12:52 +09:00
Takeshi Nakatani
1f796d432d Fixed upload error about mixuploading sparse file and truncating file 2020-06-27 22:44:19 +09:00
Takeshi Nakatani
35006e318f Fixed about ParallelMixMultipartUpload 2020-06-24 12:48:55 +09:00
Eric Vantillard
7d0c66e08a Add support for glacier storage class.
Just a copy of what have been done in PR #271.
2020-06-23 11:23:21 +09:00
Takeshi Nakatani
9dc4148743
Merge pull request #1312 from ggtakec/fix_bug_cache
Fixed a bug about serializing from cache file
2020-06-19 22:54:39 +09:00
Takeshi Nakatani
f324d8e04f Fixed a bug about serializing from cache file 2020-06-19 12:57:27 +00:00
Takeshi Nakatani
f16ee96d7e
Merge pull request #1306 from gaul/http/500
Retry with exponential backoff during 500 error
2020-06-06 15:30:22 +09:00
Takeshi Nakatani
0d849b38c2
Merge pull request #1305 from gaul/alibaba/multipart
Ignore case when comparing ETags
2020-06-06 15:05:39 +09:00
Takeshi Nakatani
8ed020610f
Merge pull request #1296 from gaul/test/oss
Import ossfs tests
2020-06-06 14:40:11 +09:00
Andrew Gaul
d8766b2051 Retry with exponential backoff during 500 error
Amazon suggests retrying on both 500 and 503:

https://aws.amazon.com/premiumsupport/knowledge-center/http-5xx-errors-s3/

Fixes #1251.
2020-06-05 21:01:30 +09:00
Andrew Gaul
9db70bab63 Ignore case when comparing ETags
This allows multipart upload to work with Alibaba OSS.
References #1297.
2020-06-05 18:17:52 +09:00
Andrew Gaul
8a7548a9d4 Import ossfs tests
This turns up a failure in test_rename_file when calling seek(0) on a
renamed file.
2020-06-01 09:08:25 +09:00
Takeshi Nakatani
0cb057dadd
Merge pull request #1303 from gaul/rename/use_cache
Relink cache stats file atomically via rename
2020-06-01 00:10:33 +09:00
Takeshi Nakatani
0f5db0d1bf
Merge pull request #1302 from gaul/rename/nocopy
Fix renames of open files with nocopyapi option
2020-05-31 23:46:46 +09:00
Takeshi Nakatani
94e67c9c58
Merge pull request #1301 from gaul/pthread-result
Check results from pthread mutex calls
2020-05-31 23:11:24 +09:00
Andrew Gaul
274321524c Relink cache stats file atomically via rename
The new file may already exist so link may fail.  Further link/unlink
is not atomic.  Addresses an error when renaming an open with with
use_cache.  References #1296.
2020-05-31 23:09:58 +09:00
Andrew Gaul
40f7007263 Check results from pthread mutex calls
Also remove some unnecessary exception handling.
2020-05-30 16:37:55 +09:00
Andrew Gaul
66597ec5f2 Fix renames of open files with nocopyapi option
References #1296.
2020-05-30 15:45:43 +09:00
Takeshi Nakatani
75e72385cc Added a parameter to output body to curldbg option 2020-05-25 08:49:01 +09:00
Takeshi Nakatani
eb58460175
Merge pull request #1294 from gaul/test/profile
Allow overriding test credentials with a profile
2020-05-24 16:14:31 +09:00
Takeshi Nakatani
0852521a7e
Merge pull request #1293 from gaul/s3proxy
Update to S3Proxy 1.7.1
2020-05-24 15:22:00 +09:00
Takeshi Nakatani
56ed6bb97f
Merge pull request #1292 from gaul/retries
Do not allow zero retries
2020-05-24 15:03:03 +09:00
Andrew Gaul
73098220bf Allow overriding test credentials with a profile 2020-05-24 12:05:16 +09:00
Andrew Gaul
ca7756fa77 Update to S3Proxy 1.7.1
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.7.1
2020-05-23 10:06:44 +09:00
Andrew Gaul
8b15db6dcb Do not allow zero retries
Retries actually means tries, e.g., if the user sets zero, s3fs will
never try an operation at all.
2020-05-23 10:05:23 +09:00
Takeshi Nakatani
0b60aa81eb
Merge pull request #1285 from ggtakec/wrapped_s3fs_strtoofft
Not abort process by exception threw from s3fs_strtoofft
2020-05-22 22:36:34 +09:00
Andrew Gaul
da70cb92a8 Provide AWS CLI with correct test credentials
This allows tests to pass against real S3 implementations like
Backblaze.  References #272.
2020-05-22 19:27:18 +09:00
Andrew Gaul
746a027e98 Expand on random write limitation 2020-05-05 08:12:04 +09:00
Takeshi Nakatani
80c11b6c12 Not abort process by exception threw from s3fs_strtoofft 2020-05-03 13:46:05 +00:00
Takeshi Nakatani
b76226a06d
Merge pull request #1286 from gaul/gcs
Support Google Cloud Storage headers
2020-05-03 22:41:02 +09:00
Andrew Gaul
8945e98d8b Support Google Cloud Storage headers
This allows s3fs to interpret objects created by gsutil.
2020-05-03 18:33:13 +09:00
Takeshi Nakatani
97c249d5b9 Not abort process by exception threw from s3fs_strtoofft 2020-05-03 08:08:28 +00:00
Takeshi Nakatani
6e134a23f9
Merge pull request #1280 from ggtakec/add_test_fdcache
Add test for cache file stat content
2020-05-03 16:33:50 +09:00