Commit Graph

1153 Commits

Author SHA1 Message Date
Takeshi Nakatani
53337a0a28
Merge pull request #944 from gaul/utility-mode
Repair utility mode
2019-02-03 13:59:56 +09:00
Takeshi Nakatani
ae51556d04
Merge pull request #943 from gaul/hard-link
Return not supported when hard linking
2019-02-03 12:45:21 +09:00
Takeshi Nakatani
b3de9195a7
Merge pull request #942 from gaul/c++03
Ensure s3fs compiles with C++03
2019-02-03 12:35:49 +09:00
Takeshi Nakatani
055ecf6ea7
Merge pull request #940 from gaul/parallel-multipart-copy
Copy parts in parallel
2019-02-03 11:44:15 +09:00
Takeshi Nakatani
c603680e02
Merge pull request #939 from gaul/stringstream-specific
Prefer specific [io]stringstream where possible
2019-02-03 11:06:57 +09:00
Takeshi Nakatani
814aadd7e3
Merge pull request #938 from gaul/clang-tidy/redundant-void
Remove unneeded void parameter
2019-02-03 10:57:56 +09:00
Takeshi Nakatani
dce63d1529
Merge pull request #937 from gaul/malloc-trim
Disable malloc_trim
2019-02-03 10:48:15 +09:00
Andrew Gaul
6ac8618381 Ensure s3fs compiles with C++03 2019-02-01 08:33:59 -08:00
Andrew Gaul
54a074647e Repair utility mode
This deinitialized S3fsCurl twice and incorrectly calculated V4
signatures.
2019-01-31 18:45:39 -08:00
Andrew Gaul
c5ebf5d328 Copy parts in parallel
S3 can copy multipart much faster than single part due to IO
parallelization.  Renaming a 4 GB file reduces from 72 to 20 seconds
with bigger gains with larger files.
2019-01-31 10:21:39 -08:00
Andrew Gaul
43c6ef560e Return not supported when hard linking
This is more correct than permission denied.
2019-01-30 16:43:04 -08:00
Andrew Gaul
3076abc744 Disable malloc_trim
This avoids walking the entire heap multiple times for complex
operations like readdir.  This does not entirely eliminate the
observed performance regression but does dramatically reduce s3fs CPU
usage.  References #935.  Fixes #936.
2019-01-29 15:29:07 -08:00
Andrew Gaul
07636c8a8d Prefer specific [io]stringstream where possible
These better communicate intent and are slightly more efficient.
2019-01-29 10:44:33 -08:00
Andrew Gaul
35d55ee513 Remove unneeded void parameter
This is implicit in C++.  Found and fixed via clang-tidy.
2019-01-28 23:22:27 -08:00
Takeshi Nakatani
a442e843be
Merge pull request #934 from ggtakec/master
Checked and corrected all typo
2019-01-27 21:33:16 +09:00
Takeshi Nakatani
c0cf90cf8b Checked and corrected all typo 2019-01-27 12:04:29 +00:00
Takeshi Nakatani
3b1cc3b197
Merge pull request #933 from gaul/cache/remove-mirror-path
Remove mirror path when deleting cache
2019-01-27 16:15:49 +09:00
Takeshi Nakatani
a0c1f30ae7
Merge pull request #932 from gaul/autolock
Prefer AutoLock for synchronization
2019-01-27 15:59:18 +09:00
Takeshi Nakatani
8822a86709
Merge pull request #931 from gaul/doc/typo
Correct sigv2 typo
2019-01-27 15:48:24 +09:00
Takeshi Nakatani
98f397de0e
Merge pull request #930 from gaul/doc/md5-multipart
Correct enable_content_md5 docs
2019-01-27 15:36:56 +09:00
Takeshi Nakatani
fd4d23f8f7
Merge pull request #926 from kzidane/master
Accept paths with : in them
2019-01-27 15:23:24 +09:00
Takeshi Nakatani
4820f0a42b
Merge pull request #925 from gaul/clang-tidy/delete-null
Remove redundant null checks before delete
2019-01-27 15:15:31 +09:00
Takeshi Nakatani
807a618cf7
Merge pull request #924 from gaul/clang-tidy/empty
Prefer empty over size checks
2019-01-27 15:04:52 +09:00
Andrew Gaul
a93e500b44 Remove mirror path when deleting cache
Fixes #827.
2019-01-25 18:10:03 -08:00
Andrew Gaul
92d3114584 Prefer AutoLock for synchronization
This simplifies the code and fixes an issue with unlocked access.
Also use a recursive lock for StatCache to avoid races between
lock..unlock..lock sequences.
2019-01-25 15:28:41 -08:00
Andrew Gaul
5062d6fbd9 Correct sigv2 typo 2019-01-25 14:30:30 -08:00
Andrew Gaul
7d14ebaf09 Correct enable_content_md5 docs
Both S3fsCurl::PutRequest and S3fsCurl::UploadMultipartPostSetup can
calculate and send Content-MD5 to the server.  Remove spurious comment
about large files and make man page and help consistent.
References #929.
2019-01-25 14:27:52 -08:00
Kareem Zidane
cd794a6985 Accept paths with : in them 2019-01-23 14:56:25 -05:00
Andrew Gaul
84b421d6ef Prefer empty over size checks
Found and fixed via clang-tidy.
2019-01-23 11:30:28 -08:00
Andrew Gaul
8316da5bbe Remove redundant null checks before delete
Found by clang-tidy.
2019-01-23 11:25:25 -08:00
Takeshi Nakatani
fa287aeef7
Merge pull request #923 from ggtakec/master
Reverted automatic region change and changed messages
2019-01-23 23:00:31 +09:00
Takeshi Nakatani
caaf4cac55 Reverted #912(Automatic region change) and added message 2019-01-23 13:23:03 +00:00
Takeshi Nakatani
010276ceab
Merge pull request #921 from gaul/clang-tidy/redundant-string-init
Remove redundant string initializations
2019-01-23 19:44:59 +09:00
Takeshi Nakatani
f219817eb3
Merge pull request #920 from gaul/clang-tidy/string-copy
Remove unnecessary string copies
2019-01-23 19:30:55 +09:00
Takeshi Nakatani
d487348d21
Merge pull request #919 from gaul/test/mv-nonempty-dir
Add test for mv non-empty directory
2019-01-23 19:21:12 +09:00
Takeshi Nakatani
eb0b29708f
Merge pull request #918 from gaul/overwrite-file-range
Load tail range during overwrite
2019-01-23 19:12:23 +09:00
Takeshi Nakatani
877842a720
Merge pull request #917 from gaul/readme/aws-cli
Reference better-known AWS CLI for compatibility
2019-01-23 18:38:08 +09:00
Andrew Gaul
1fc25e8c3f Remove redundant string initializations
Found and fixed via clang-tidy.
2019-01-22 23:16:37 -08:00
Andrew Gaul
61ecafd426 Remove unnecessary string copies
Found via clang-tidy.
2019-01-22 23:09:37 -08:00
Andrew Gaul
79bd3441eb Add test for mv non-empty directory 2019-01-22 22:13:17 -08:00
Andrew Gaul
5f5da4b2cb Load tail range during overwrite
Previously s3fs experienced data loss when writing to the middle of a
file.  Corrupt files would have the expected data from 0..offset+size
but unexpected NUL bytes from offset+size..EOF.  References #808.
2019-01-22 22:02:40 -08:00
Andrew Gaul
dede19d8c0 Reference better-known AWS CLI for compatibility 2019-01-21 17:32:57 -08:00
Takeshi Nakatani
fada95f58e
Merge pull request #914 from gaul/readdir/head-of-line
Issue readdir HEAD requests without batching
2019-01-21 22:10:50 +09:00
Takeshi Nakatani
014b8c5982
Merge pull request #913 from gaul/assert
Prefer abort over assert(false)
2019-01-21 21:56:41 +09:00
Andrew Gaul
46d79c5bc2 Issue readdir HEAD requests without batching
Previously s3fs would issue a batch of HEAD requests and wait for all
to succeed before issuing the next batch.  Now it issues the first
batch and only waits for a single call to succeed before issuing the
next call.  This can improve performance when one call lags due to
network errors.  I measured 25% improvement with the same level of
parallelism.  This commit also reparents parallelism knobs for
consistency.  Follows on to 88cd8feb05.
Fixes #223.
2019-01-20 18:07:22 -08:00
Andrew Gaul
40ba3b44a1 Prefer abort over assert(false)
The compiler can remove the latter when compiled with NDEBUG which may
cause unintended control flow.
2019-01-20 12:30:27 -08:00
Takeshi Nakatani
beadf95975
Merge pull request #912 from ggtakec/master
Automatic region change made possible other than us-east-1(default)
2019-01-20 20:13:07 +09:00
Takeshi Nakatani
2887f8916b Automatic region change made possible other than us-east-1(default) 2019-01-20 10:51:49 +00:00
Takeshi Nakatani
0c9a8932f7
Merge pull request #911 from ggtakec/master
Added detail error message when HTTP 301/307 status
2019-01-20 19:31:04 +09:00
Takeshi Nakatani
ac72431195 Added detail error message when HTTP 301/307 status 2019-01-20 10:07:58 +00:00