Commit Graph

1754 Commits

Author SHA1 Message Date
Takeshi Nakatani
f324d8e04f Fixed a bug about serializing from cache file 2020-06-19 12:57:27 +00:00
Takeshi Nakatani
f16ee96d7e
Merge pull request #1306 from gaul/http/500
Retry with exponential backoff during 500 error
2020-06-06 15:30:22 +09:00
Takeshi Nakatani
0d849b38c2
Merge pull request #1305 from gaul/alibaba/multipart
Ignore case when comparing ETags
2020-06-06 15:05:39 +09:00
Takeshi Nakatani
8ed020610f
Merge pull request #1296 from gaul/test/oss
Import ossfs tests
2020-06-06 14:40:11 +09:00
Andrew Gaul
d8766b2051 Retry with exponential backoff during 500 error
Amazon suggests retrying on both 500 and 503:

https://aws.amazon.com/premiumsupport/knowledge-center/http-5xx-errors-s3/

Fixes #1251.
2020-06-05 21:01:30 +09:00
Andrew Gaul
9db70bab63 Ignore case when comparing ETags
This allows multipart upload to work with Alibaba OSS.
References #1297.
2020-06-05 18:17:52 +09:00
Andrew Gaul
8a7548a9d4 Import ossfs tests
This turns up a failure in test_rename_file when calling seek(0) on a
renamed file.
2020-06-01 09:08:25 +09:00
Takeshi Nakatani
0cb057dadd
Merge pull request #1303 from gaul/rename/use_cache
Relink cache stats file atomically via rename
2020-06-01 00:10:33 +09:00
Takeshi Nakatani
0f5db0d1bf
Merge pull request #1302 from gaul/rename/nocopy
Fix renames of open files with nocopyapi option
2020-05-31 23:46:46 +09:00
Takeshi Nakatani
94e67c9c58
Merge pull request #1301 from gaul/pthread-result
Check results from pthread mutex calls
2020-05-31 23:11:24 +09:00
Andrew Gaul
274321524c Relink cache stats file atomically via rename
The new file may already exist so link may fail.  Further link/unlink
is not atomic.  Addresses an error when renaming an open with with
use_cache.  References #1296.
2020-05-31 23:09:58 +09:00
Andrew Gaul
40f7007263 Check results from pthread mutex calls
Also remove some unnecessary exception handling.
2020-05-30 16:37:55 +09:00
Andrew Gaul
66597ec5f2 Fix renames of open files with nocopyapi option
References #1296.
2020-05-30 15:45:43 +09:00
Takeshi Nakatani
75e72385cc Added a parameter to output body to curldbg option 2020-05-25 08:49:01 +09:00
Takeshi Nakatani
eb58460175
Merge pull request #1294 from gaul/test/profile
Allow overriding test credentials with a profile
2020-05-24 16:14:31 +09:00
Takeshi Nakatani
0852521a7e
Merge pull request #1293 from gaul/s3proxy
Update to S3Proxy 1.7.1
2020-05-24 15:22:00 +09:00
Takeshi Nakatani
56ed6bb97f
Merge pull request #1292 from gaul/retries
Do not allow zero retries
2020-05-24 15:03:03 +09:00
Andrew Gaul
73098220bf Allow overriding test credentials with a profile 2020-05-24 12:05:16 +09:00
Andrew Gaul
ca7756fa77 Update to S3Proxy 1.7.1
Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.7.1
2020-05-23 10:06:44 +09:00
Andrew Gaul
8b15db6dcb Do not allow zero retries
Retries actually means tries, e.g., if the user sets zero, s3fs will
never try an operation at all.
2020-05-23 10:05:23 +09:00
Takeshi Nakatani
0b60aa81eb
Merge pull request #1285 from ggtakec/wrapped_s3fs_strtoofft
Not abort process by exception threw from s3fs_strtoofft
2020-05-22 22:36:34 +09:00
Andrew Gaul
da70cb92a8 Provide AWS CLI with correct test credentials
This allows tests to pass against real S3 implementations like
Backblaze.  References #272.
2020-05-22 19:27:18 +09:00
Andrew Gaul
746a027e98 Expand on random write limitation 2020-05-05 08:12:04 +09:00
Takeshi Nakatani
80c11b6c12 Not abort process by exception threw from s3fs_strtoofft 2020-05-03 13:46:05 +00:00
Takeshi Nakatani
b76226a06d
Merge pull request #1286 from gaul/gcs
Support Google Cloud Storage headers
2020-05-03 22:41:02 +09:00
Andrew Gaul
8945e98d8b Support Google Cloud Storage headers
This allows s3fs to interpret objects created by gsutil.
2020-05-03 18:33:13 +09:00
Takeshi Nakatani
97c249d5b9 Not abort process by exception threw from s3fs_strtoofft 2020-05-03 08:08:28 +00:00
Takeshi Nakatani
6e134a23f9
Merge pull request #1280 from ggtakec/add_test_fdcache
Add test for cache file stat content
2020-05-03 16:33:50 +09:00
Takeshi Nakatani
a4803543a1
Merge pull request #1282 from gaul/mime
Warn about missing MIME types instead of exiting
2020-05-03 15:15:05 +09:00
Andrew Gaul
2cc88b933f Warn about missing MIME types instead of exiting
s3fs uses the MIME types file to set Content-Type for uploaded
objects.  Most distribution packages should install this via
recommended (not required) dependencies.  Users compiling from source
may not have this installed and s3fs should not prevent launching
since most users do not care about Content-Type.  Instead warn about
MIME types absence.  Fixes #1270.
2020-04-29 20:03:50 +09:00
Takeshi Nakatani
ce1221c867 Add test for cache file stat content 2020-04-22 15:53:00 +00:00
Andrew Gaul
005a684600 Fix typos 2020-04-22 21:49:11 +09:00
Andrew Gaul
3af17c3019 Add test for out-of-order writes
References #1220.  References #1277.
2020-04-22 21:48:55 +09:00
Takeshi Nakatani
f26a0aa71d Fixed insufficient upload size for mix multipart upload 2020-04-22 09:31:22 +09:00
Takeshi Nakatani
2b4619842d
Merge pull request #1278 from gaul/http-416
Add handler for HTTP 416
2020-04-21 22:38:54 +09:00
Andrew Gaul
cf529e0af7 Add handler for HTTP 416
This prevents retries when the server indicates an unsatisfiable MPU
copy part request.  References #1220.
2020-04-21 19:45:10 +09:00
Takeshi Nakatani
4da02d023b Improved strictness of cache file stats(file) 2020-04-21 19:45:03 +09:00
Takeshi Nakatani
fe0677651e
Merge pull request #1271 from ggtakec/fix_stat_file
Fixed the truncation bug of stat file for cache file
2020-04-19 16:27:11 +09:00
Takeshi Nakatani
811ea0cb85 Fixed the truncation bug of stat file for cache file 2020-04-19 07:08:49 +00:00
Miguel de Val-Borro
a5f84535f3 Add install instructions for Arch Linux 2020-04-18 19:27:52 +09:00
Andrew Gaul
84bf460f99 Remove deprecated sudo configuration
Addresses warnings of the form:

jobs.include: deprecated key sudo (The key `sudo` has no effect
anymore.)
2020-04-15 10:09:52 +09:00
Takeshi Nakatani
538fbed302
Merge pull request #1266 from gaul/test/cache-eviction
Test cache eviction
2020-04-14 22:30:04 +09:00
Andrew Gaul
feafb44bae Clean up macOS FUSE loading 2020-04-12 23:18:27 +09:00
Andrew Gaul
a44fc1103d Avoid setting bogus group id
Addresses a symptom on newer macOS in Travis.
2020-04-12 22:37:22 +09:00
Andrew Gaul
48a872e285 Address cppcheck 1.90 warning 2020-04-12 22:20:44 +09:00
Takeshi Nakatani
c44a60f3f5 Fixed a bug of stats cache compression 2020-04-12 18:33:00 +09:00
Andrew Gaul
f373df9682 Test cache eviction 2020-04-11 19:00:38 +09:00
Takeshi Nakatani
9e01d5b8d1
Merge pull request #1254 from ggtakec/modify_mimetypes
Added mime option for strict checking of mime types file
2020-04-11 14:48:47 +09:00
Takeshi Nakatani
7fbda230f5 Added mime option for strict checking of mime types file 2020-03-30 14:41:18 +00:00
Andrew Gaul
56141557dc Avoid unneeded string copy
Found by clang-tidy 10.
2020-03-28 08:49:49 +09:00