1254 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Takeshi Nakatani	b3de9195a7	Merge pull request #942 from gaul/c++03 ... Ensure s3fs compiles with C++03	2019-02-03 12:35:49 +09:00
Takeshi Nakatani	055ecf6ea7	Merge pull request #940 from gaul/parallel-multipart-copy ... Copy parts in parallel	2019-02-03 11:44:15 +09:00
Takeshi Nakatani	c603680e02	Merge pull request #939 from gaul/stringstream-specific ... Prefer specific [io]stringstream where possible	2019-02-03 11:06:57 +09:00
Takeshi Nakatani	814aadd7e3	Merge pull request #938 from gaul/clang-tidy/redundant-void ... Remove unneeded void parameter	2019-02-03 10:57:56 +09:00
Takeshi Nakatani	dce63d1529	Merge pull request #937 from gaul/malloc-trim ... Disable malloc_trim	2019-02-03 10:48:15 +09:00
Andrew Gaul	8ff05d8e38	Implement exponential backoff for 503 ... Amazon returns SlowDown when overloaded. Also return ENOTSUP for 501 and immediately return EIO for 500 instead of retrying. Fixes #603.	2019-02-02 00:03:50 -08:00
Andrew Gaul	dfa84b82a8	Add logging for too many parts ... References #610.	2019-02-01 19:34:26 -08:00
Andrew Gaul	6ac8618381	Ensure s3fs compiles with C++03	2019-02-01 08:33:59 -08:00
Andrew Gaul	8c527c3616	Simplify async request completion code ... Workers now notify the master thread when they complete, unifying the Linux and macOS code paths. This also avoids excessive pthread_tryjoin_np calls. Follows on to 88cd8feb05.	2019-01-31 22:55:23 -08:00
Andrew Gaul	54a074647e	Repair utility mode ... This deinitialized S3fsCurl twice and incorrectly calculated V4 signatures.	2019-01-31 18:45:39 -08:00
Andrew Gaul	c5ebf5d328	Copy parts in parallel ... S3 can copy multipart much faster than single part due to IO parallelization. Renaming a 4 GB file reduces from 72 to 20 seconds with bigger gains with larger files.	2019-01-31 10:21:39 -08:00
Andrew Gaul	43c6ef560e	Return not supported when hard linking ... This is more correct than permission denied.	2019-01-30 16:43:04 -08:00
Andrew Gaul	3076abc744	Disable malloc_trim ... This avoids walking the entire heap multiple times for complex operations like readdir. This does not entirely eliminate the observed performance regression but does dramatically reduce s3fs CPU usage. References #935. Fixes #936.	2019-01-29 15:29:07 -08:00
Andrew Gaul	07636c8a8d	Prefer specific [io]stringstream where possible ... These better communicate intent and are slightly more efficient.	2019-01-29 10:44:33 -08:00
Andrew Gaul	35d55ee513	Remove unneeded void parameter ... This is implicit in C++. Found and fixed via clang-tidy.	2019-01-28 23:22:27 -08:00
Takeshi Nakatani	a442e843be	Merge pull request #934 from ggtakec/master ... Checked and corrected all typo	2019-01-27 21:33:16 +09:00
Takeshi Nakatani	c0cf90cf8b	Checked and corrected all typo	2019-01-27 12:04:29 +00:00
Takeshi Nakatani	3b1cc3b197	Merge pull request #933 from gaul/cache/remove-mirror-path ... Remove mirror path when deleting cache	2019-01-27 16:15:49 +09:00
Takeshi Nakatani	a0c1f30ae7	Merge pull request #932 from gaul/autolock ... Prefer AutoLock for synchronization	2019-01-27 15:59:18 +09:00
Takeshi Nakatani	8822a86709	Merge pull request #931 from gaul/doc/typo ... Correct sigv2 typo	2019-01-27 15:48:24 +09:00
Takeshi Nakatani	98f397de0e	Merge pull request #930 from gaul/doc/md5-multipart ... Correct enable_content_md5 docs	2019-01-27 15:36:56 +09:00
Takeshi Nakatani	fd4d23f8f7	Merge pull request #926 from kzidane/master ... Accept paths with : in them	2019-01-27 15:23:24 +09:00
Takeshi Nakatani	4820f0a42b	Merge pull request #925 from gaul/clang-tidy/delete-null ... Remove redundant null checks before delete	2019-01-27 15:15:31 +09:00
Takeshi Nakatani	807a618cf7	Merge pull request #924 from gaul/clang-tidy/empty ... Prefer empty over size checks	2019-01-27 15:04:52 +09:00
Andrew Gaul	a93e500b44	Remove mirror path when deleting cache ... Fixes #827.	2019-01-25 18:10:03 -08:00
Andrew Gaul	92d3114584	Prefer AutoLock for synchronization ... This simplifies the code and fixes an issue with unlocked access. Also use a recursive lock for StatCache to avoid races between lock..unlock..lock sequences.	2019-01-25 15:28:41 -08:00
Andrew Gaul	5062d6fbd9	Correct sigv2 typo	2019-01-25 14:30:30 -08:00
Andrew Gaul	7d14ebaf09	Correct enable_content_md5 docs ... Both S3fsCurl::PutRequest and S3fsCurl::UploadMultipartPostSetup can calculate and send Content-MD5 to the server. Remove spurious comment about large files and make man page and help consistent. References #929.	2019-01-25 14:27:52 -08:00
Kareem Zidane	cd794a6985	Accept paths with : in them	2019-01-23 14:56:25 -05:00
Andrew Gaul	84b421d6ef	Prefer empty over size checks ... Found and fixed via clang-tidy.	2019-01-23 11:30:28 -08:00
Andrew Gaul	8316da5bbe	Remove redundant null checks before delete ... Found by clang-tidy.	2019-01-23 11:25:25 -08:00
Takeshi Nakatani	fa287aeef7	Merge pull request #923 from ggtakec/master ... Reverted automatic region change and changed messages	2019-01-23 23:00:31 +09:00
Takeshi Nakatani	caaf4cac55	Reverted #912(Automatic region change) and added message	2019-01-23 13:23:03 +00:00
Takeshi Nakatani	010276ceab	Merge pull request #921 from gaul/clang-tidy/redundant-string-init ... Remove redundant string initializations	2019-01-23 19:44:59 +09:00
Takeshi Nakatani	f219817eb3	Merge pull request #920 from gaul/clang-tidy/string-copy ... Remove unnecessary string copies	2019-01-23 19:30:55 +09:00
Takeshi Nakatani	d487348d21	Merge pull request #919 from gaul/test/mv-nonempty-dir ... Add test for mv non-empty directory	2019-01-23 19:21:12 +09:00
Takeshi Nakatani	eb0b29708f	Merge pull request #918 from gaul/overwrite-file-range ... Load tail range during overwrite	2019-01-23 19:12:23 +09:00
Takeshi Nakatani	877842a720	Merge pull request #917 from gaul/readme/aws-cli ... Reference better-known AWS CLI for compatibility	2019-01-23 18:38:08 +09:00
Andrew Gaul	1fc25e8c3f	Remove redundant string initializations ... Found and fixed via clang-tidy.	2019-01-22 23:16:37 -08:00
Andrew Gaul	61ecafd426	Remove unnecessary string copies ... Found via clang-tidy.	2019-01-22 23:09:37 -08:00
Andrew Gaul	79bd3441eb	Add test for mv non-empty directory	2019-01-22 22:13:17 -08:00
Andrew Gaul	5f5da4b2cb	Load tail range during overwrite ... Previously s3fs experienced data loss when writing to the middle of a file. Corrupt files would have the expected data from 0..offset+size but unexpected NUL bytes from offset+size..EOF. References #808.	2019-01-22 22:02:40 -08:00
Andrew Gaul	dede19d8c0	Reference better-known AWS CLI for compatibility	2019-01-21 17:32:57 -08:00
Takeshi Nakatani	fada95f58e	Merge pull request #914 from gaul/readdir/head-of-line ... Issue readdir HEAD requests without batching	2019-01-21 22:10:50 +09:00
Takeshi Nakatani	014b8c5982	Merge pull request #913 from gaul/assert ... Prefer abort over assert(false)	2019-01-21 21:56:41 +09:00
Andrew Gaul	46d79c5bc2	Issue readdir HEAD requests without batching ... Previously s3fs would issue a batch of HEAD requests and wait for all to succeed before issuing the next batch. Now it issues the first batch and only waits for a single call to succeed before issuing the next call. This can improve performance when one call lags due to network errors. I measured 25% improvement with the same level of parallelism. This commit also reparents parallelism knobs for consistency. Follows on to 88cd8feb05. Fixes #223.	2019-01-20 18:07:22 -08:00
Andrew Gaul	40ba3b44a1	Prefer abort over assert(false) ... The compiler can remove the latter when compiled with NDEBUG which may cause unintended control flow.	2019-01-20 12:30:27 -08:00
Takeshi Nakatani	beadf95975	Merge pull request #912 from ggtakec/master ... Automatic region change made possible other than us-east-1(default)	2019-01-20 20:13:07 +09:00
Takeshi Nakatani	2887f8916b	Automatic region change made possible other than us-east-1(default)	2019-01-20 10:51:49 +00:00
Takeshi Nakatani	0c9a8932f7	Merge pull request #911 from ggtakec/master ... Added detail error message when HTTP 301/307 status	2019-01-20 19:31:04 +09:00