790 Commits

Author SHA1 Message Date
Andrew Gaul
8a18806a57 Make man page and --help more consistent
Mostly small fixes but also some reordering.  References #929.  Found
via:

diff -u <(man --no-hyphenation doc/man/s3fs.1 | tr -s ' ' '\n' | sed '/^-o$/d' ) <(src/s3fs --help | tr -s ' ' '\n' | sed '/^-$/d')
2019-04-14 14:34:24 +09:00
Takeshi Nakatani
0a99470369
Merge pull request #1002 from gaul/sse-c
Correctly calculate MD5 hash for SSE-C keys
2019-04-09 00:17:12 +09:00
Takeshi Nakatani
cd280d8702
Merge pull request #1001 from gaul/missing-braces
Add missing braces
2019-04-08 23:55:02 +09:00
Andrew Gaul
6f8ecb28c5 Correctly calculate MD5 hash for SSE-C keys
Previously s3fs calculated the strlen of a binary input instead of
using the stored length.  This yielded IO errors when interacting with
SSE-encrypted objects.  Fixes #696.
2019-04-08 21:49:42 +09:00
Andrew Gaul
042332bcec Add missing braces
Found via clang-tidy.  Also fix errant indentation.
2019-04-07 23:12:27 +09:00
Takeshi Nakatani
071cd0f849 Modified to discard _netdev option etc 2019-04-07 11:51:55 +00:00
Andrew Gaul
6f6a67807b Prefer new over malloc
The former cannot return NULL and will allow use of scoped pointers.
2019-04-07 16:54:24 +09:00
Takeshi Nakatani
ea517c80a4 Fixed a bug about overwrite metadata at updating directory stats 2019-03-29 15:30:30 +00:00
Takeshi Nakatani
71766039ff Support undefined CURLoption in libcurl library used in build 2019-03-22 10:47:42 +00:00
Andrew Gaul
058706014b Simplify string comparison
Found via newer cppcheck.
2019-03-14 22:19:33 +09:00
Takeshi Nakatani
62dcda6a56 Fixed ref-count when error occurred. 2019-03-10 06:04:19 +00:00
Takeshi Nakatani
cbf072bc55
Merge pull request #967 from gaul/reference-count
Increase FdEntity reference count when returning
2019-03-10 14:27:10 +09:00
Takeshi Nakatani
b5ca400500
Merge pull request #970 from gaul/nocopyapi
Remove from file from stat cache during rename
2019-03-10 12:56:04 +09:00
Andrew Gaul
08b132ddb9 Remove from file from stat cache during rename
This addresses failures with test_rename_before_close when using
nocopyapi.  Note that test_update_time still fails.
2019-03-09 17:53:35 +09:00
Andrew Gaul
1e86cc643d Fix lazy typo 2019-03-09 17:46:02 +09:00
Andrew Gaul
f53503438c Increase FdEntity reference count when returning
Previously s3fs had a race condition where one client could delete
FdEntity that another client was using.  Add a simple concurrent test
which previously failed but now succeeds.  Fixes #964.
2019-03-09 15:55:04 +09:00
Takeshi Nakatani
0d43d070cc
Merge pull request #965 from ggtakec/fix_multi_x
Improvement of curl session pool for multipart
2019-02-27 22:44:25 +09:00
Takeshi Nakatani
0791fdca2a
Merge pull request #960 from kristjanvalur/wtf8
Wtf8
2019-02-27 21:21:38 +09:00
Kristján Valur Jónsson
6e8678d5e3 remove lines that wer accidentally duplicated 2019-02-25 16:02:01 +00:00
Takeshi Nakatani
10d9f75366 Improvement of curl session pool for multipart
Improvement of curl session pool for multipart(2)

Improvement of curl session pool for multipart(3)
2019-02-25 14:46:24 +00:00
Andrew Gaul
74d8671e54 Work around cppcheck warnings
Follows on to eeb839242ba826707205b0b776c042203226d01b.
2019-02-20 21:55:41 +09:00
Kristján Valur Jónsson
4c41eac29c fix documentation and man page 2019-02-20 11:24:29 +00:00
Kristján Valur Jónsson
3c97c1b251 merged main 2019-02-19 10:53:00 +00:00
Kristján Valur Jónsson
84c671a81a fix indentation 2019-02-19 10:37:43 +00:00
Kristján Valur Jónsson
f336bdebcc add command line flag and documentation 2019-02-19 10:32:37 +00:00
Kristján Valur Jónsson
e5b8377202 fix comments and code 2019-02-19 10:32:10 +00:00
Kristján Valur Jónsson
4f42f4ab0c Enable s3fs encoding and decoding in the fuse interface 2019-02-18 13:36:48 +00:00
Kristján Valur Jónsson
11b385820d more robust wtf8 encoding 2019-02-18 12:27:44 +00:00
Takeshi Nakatani
eeb839242b Fixed code for latest cppcheck error on OSX 2019-02-17 13:59:11 +00:00
Kristján Valur Jónsson
ca2d1d873d Adding utility functions to convert invalid utf8 to wtf8 encoding
This is to deal with windows clients who pass in cp1252 as if it
were utf8
2019-02-15 15:57:03 +00:00
Takeshi Nakatani
231fd001d9 Added a non-interactive option to utility mode
Rebase for resolving conflicts

Deleted interactive mode on utility mode

Fixed minor mistakes
2019-02-10 17:36:18 +00:00
Takeshi Nakatani
314dc5a398
Merge pull request #955 from gaul/clang-tidy/misc
Address miscellaneous clang-tidy warnings
2019-02-06 21:21:54 +09:00
Takeshi Nakatani
e07cb020cc
Merge pull request #954 from gaul/clear-iter
Clear containers instead of individual erases
2019-02-06 21:12:36 +09:00
Andrew Gaul
e87e40b3b4 Address miscellaneous clang-tidy warnings
Also add configuration with suppressions.
2019-02-04 22:58:40 -08:00
Takeshi Nakatani
bd66b57ad3
Merge pull request #952 from gaul/multipart/failed
Automatically abort failed multipart requests
2019-02-04 22:19:27 +09:00
Andrew Gaul
a1d3ff9766 Automatically abort failed multipart requests
This can avoid dangling parts.  However, many transfers fail due to
network errors so we still need other mechanisms to handle these
parts.
2019-02-03 10:29:20 -08:00
Andrew Gaul
4d0bef1e90 Clear containers instead of individual erases
This has O(n) runtime instead of O(n log n).
2019-02-02 23:58:43 -08:00
Takeshi Nakatani
960823fb40 Added S3FS_MALLOC_TRIM build switch 2019-02-03 07:36:17 +00:00
Takeshi Nakatani
c04e8e7a9d
Merge pull request #949 from gaul/503-slow-down
Implement exponential backoff for 503
2019-02-03 15:19:28 +09:00
Takeshi Nakatani
fb6debd986
Merge pull request #948 from gaul/too-many-parts
Add logging for too many parts
2019-02-03 14:55:54 +09:00
Takeshi Nakatani
d8185a25aa
Merge pull request #946 from gaul/async/completed-tids
Simplify async request completion code
2019-02-03 14:08:49 +09:00
Takeshi Nakatani
53337a0a28
Merge pull request #944 from gaul/utility-mode
Repair utility mode
2019-02-03 13:59:56 +09:00
Takeshi Nakatani
ae51556d04
Merge pull request #943 from gaul/hard-link
Return not supported when hard linking
2019-02-03 12:45:21 +09:00
Takeshi Nakatani
055ecf6ea7
Merge pull request #940 from gaul/parallel-multipart-copy
Copy parts in parallel
2019-02-03 11:44:15 +09:00
Takeshi Nakatani
c603680e02
Merge pull request #939 from gaul/stringstream-specific
Prefer specific [io]stringstream where possible
2019-02-03 11:06:57 +09:00
Takeshi Nakatani
814aadd7e3
Merge pull request #938 from gaul/clang-tidy/redundant-void
Remove unneeded void parameter
2019-02-03 10:57:56 +09:00
Andrew Gaul
8ff05d8e38 Implement exponential backoff for 503
Amazon returns SlowDown when overloaded.  Also return ENOTSUP for 501
and immediately return EIO for 500 instead of retrying.  Fixes #603.
2019-02-02 00:03:50 -08:00
Andrew Gaul
dfa84b82a8 Add logging for too many parts
References #610.
2019-02-01 19:34:26 -08:00
Andrew Gaul
8c527c3616 Simplify async request completion code
Workers now notify the master thread when they complete, unifying the
Linux and macOS code paths.  This also avoids excessive
pthread_tryjoin_np calls.  Follows on to
88cd8feb053980c808d67771d63a84ca25f6db8a.
2019-01-31 22:55:23 -08:00
Andrew Gaul
54a074647e Repair utility mode
This deinitialized S3fsCurl twice and incorrectly calculated V4
signatures.
2019-01-31 18:45:39 -08:00