1375 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Andrew Gaul	f7760976a5	Document lack of inotify support ... References #45.	2019-02-17 19:45:08 +09:00
Kristján Valur Jónsson	ca2d1d873d	Adding utility functions to convert invalid utf8 to wtf8 encoding ... This is to deal with windows clients who pass in cp1252 as if it were utf8	2019-02-15 15:57:03 +00:00
Takeshi Nakatani	951761ee2c	Merge pull request #951 from ggtakec/master ... Added a non-interactive option to utility mode	2019-02-11 03:03:01 +09:00
Takeshi Nakatani	231fd001d9	Added a non-interactive option to utility mode ... Rebase for resolving conflicts Deleted interactive mode on utility mode Fixed minor mistakes	2019-02-10 17:36:18 +00:00
Takeshi Nakatani	e00afa8128	Merge pull request #957 from gaul/deps/s3proxy-1.6.1 ... Upgrade to S3Proxy 1.6.1	2019-02-10 23:32:13 +09:00
Andrew Gaul	e9297f39ea	Upgrade to S3Proxy 1.6.1 ... Notably, this improve performance when listing objects using the filesystem backend, allowing new tests like expanding large tar files. Release notes: https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.1	2019-02-06 17:06:43 -08:00
Takeshi Nakatani	314dc5a398	Merge pull request #955 from gaul/clang-tidy/misc ... Address miscellaneous clang-tidy warnings	2019-02-06 21:21:54 +09:00
Takeshi Nakatani	e07cb020cc	Merge pull request #954 from gaul/clear-iter ... Clear containers instead of individual erases	2019-02-06 21:12:36 +09:00
Takeshi Nakatani	9f79b9e0da	Merge pull request #953 from gaul/readme/s3ql ... Update s3ql link	2019-02-06 21:03:28 +09:00
Andrew Gaul	e87e40b3b4	Address miscellaneous clang-tidy warnings ... Also add configuration with suppressions.	2019-02-04 22:58:40 -08:00
Andrew Gaul	f0f95478ec	Update s3ql link	2019-02-04 21:03:46 -08:00
Takeshi Nakatani	bd66b57ad3	Merge pull request #952 from gaul/multipart/failed ... Automatically abort failed multipart requests	2019-02-04 22:19:27 +09:00
Andrew Gaul	a1d3ff9766	Automatically abort failed multipart requests ... This can avoid dangling parts. However, many transfers fail due to network errors so we still need other mechanisms to handle these parts.	2019-02-03 10:29:20 -08:00
Takeshi Nakatani	7f61a947c2	Merge pull request #950 from ggtakec/master ... Added S3FS_MALLOC_TRIM build switch	2019-02-03 17:23:09 +09:00
Andrew Gaul	4d0bef1e90	Clear containers instead of individual erases ... This has O(n) runtime instead of O(n log n).	2019-02-02 23:58:43 -08:00
Takeshi Nakatani	960823fb40	Added S3FS_MALLOC_TRIM build switch	2019-02-03 07:36:17 +00:00
Takeshi Nakatani	c04e8e7a9d	Merge pull request #949 from gaul/503-slow-down ... Implement exponential backoff for 503	2019-02-03 15:19:28 +09:00
Takeshi Nakatani	fb6debd986	Merge pull request #948 from gaul/too-many-parts ... Add logging for too many parts	2019-02-03 14:55:54 +09:00
Takeshi Nakatani	d8185a25aa	Merge pull request #946 from gaul/async/completed-tids ... Simplify async request completion code	2019-02-03 14:08:49 +09:00
Takeshi Nakatani	53337a0a28	Merge pull request #944 from gaul/utility-mode ... Repair utility mode	2019-02-03 13:59:56 +09:00
Takeshi Nakatani	ae51556d04	Merge pull request #943 from gaul/hard-link ... Return not supported when hard linking	2019-02-03 12:45:21 +09:00
Takeshi Nakatani	b3de9195a7	Merge pull request #942 from gaul/c++03 ... Ensure s3fs compiles with C++03	2019-02-03 12:35:49 +09:00
Takeshi Nakatani	055ecf6ea7	Merge pull request #940 from gaul/parallel-multipart-copy ... Copy parts in parallel	2019-02-03 11:44:15 +09:00
Takeshi Nakatani	c603680e02	Merge pull request #939 from gaul/stringstream-specific ... Prefer specific [io]stringstream where possible	2019-02-03 11:06:57 +09:00
Takeshi Nakatani	814aadd7e3	Merge pull request #938 from gaul/clang-tidy/redundant-void ... Remove unneeded void parameter	2019-02-03 10:57:56 +09:00
Takeshi Nakatani	dce63d1529	Merge pull request #937 from gaul/malloc-trim ... Disable malloc_trim	2019-02-03 10:48:15 +09:00
Andrew Gaul	8ff05d8e38	Implement exponential backoff for 503 ... Amazon returns SlowDown when overloaded. Also return ENOTSUP for 501 and immediately return EIO for 500 instead of retrying. Fixes #603.	2019-02-02 00:03:50 -08:00
Andrew Gaul	dfa84b82a8	Add logging for too many parts ... References #610.	2019-02-01 19:34:26 -08:00
Andrew Gaul	6ac8618381	Ensure s3fs compiles with C++03	2019-02-01 08:33:59 -08:00
Andrew Gaul	8c527c3616	Simplify async request completion code ... Workers now notify the master thread when they complete, unifying the Linux and macOS code paths. This also avoids excessive pthread_tryjoin_np calls. Follows on to 88cd8feb05.	2019-01-31 22:55:23 -08:00
Andrew Gaul	54a074647e	Repair utility mode ... This deinitialized S3fsCurl twice and incorrectly calculated V4 signatures.	2019-01-31 18:45:39 -08:00
Andrew Gaul	c5ebf5d328	Copy parts in parallel ... S3 can copy multipart much faster than single part due to IO parallelization. Renaming a 4 GB file reduces from 72 to 20 seconds with bigger gains with larger files.	2019-01-31 10:21:39 -08:00
Andrew Gaul	43c6ef560e	Return not supported when hard linking ... This is more correct than permission denied.	2019-01-30 16:43:04 -08:00
Andrew Gaul	3076abc744	Disable malloc_trim ... This avoids walking the entire heap multiple times for complex operations like readdir. This does not entirely eliminate the observed performance regression but does dramatically reduce s3fs CPU usage. References #935. Fixes #936.	2019-01-29 15:29:07 -08:00
Andrew Gaul	07636c8a8d	Prefer specific [io]stringstream where possible ... These better communicate intent and are slightly more efficient.	2019-01-29 10:44:33 -08:00
Andrew Gaul	35d55ee513	Remove unneeded void parameter ... This is implicit in C++. Found and fixed via clang-tidy.	2019-01-28 23:22:27 -08:00
Takeshi Nakatani	a442e843be	Merge pull request #934 from ggtakec/master ... Checked and corrected all typo	2019-01-27 21:33:16 +09:00
Takeshi Nakatani	c0cf90cf8b	Checked and corrected all typo	2019-01-27 12:04:29 +00:00
Takeshi Nakatani	3b1cc3b197	Merge pull request #933 from gaul/cache/remove-mirror-path ... Remove mirror path when deleting cache	2019-01-27 16:15:49 +09:00
Takeshi Nakatani	a0c1f30ae7	Merge pull request #932 from gaul/autolock ... Prefer AutoLock for synchronization	2019-01-27 15:59:18 +09:00
Takeshi Nakatani	8822a86709	Merge pull request #931 from gaul/doc/typo ... Correct sigv2 typo	2019-01-27 15:48:24 +09:00
Takeshi Nakatani	98f397de0e	Merge pull request #930 from gaul/doc/md5-multipart ... Correct enable_content_md5 docs	2019-01-27 15:36:56 +09:00
Takeshi Nakatani	fd4d23f8f7	Merge pull request #926 from kzidane/master ... Accept paths with : in them	2019-01-27 15:23:24 +09:00
Takeshi Nakatani	4820f0a42b	Merge pull request #925 from gaul/clang-tidy/delete-null ... Remove redundant null checks before delete	2019-01-27 15:15:31 +09:00
Takeshi Nakatani	807a618cf7	Merge pull request #924 from gaul/clang-tidy/empty ... Prefer empty over size checks	2019-01-27 15:04:52 +09:00
Andrew Gaul	a93e500b44	Remove mirror path when deleting cache ... Fixes #827.	2019-01-25 18:10:03 -08:00
Andrew Gaul	92d3114584	Prefer AutoLock for synchronization ... This simplifies the code and fixes an issue with unlocked access. Also use a recursive lock for StatCache to avoid races between lock..unlock..lock sequences.	2019-01-25 15:28:41 -08:00
Andrew Gaul	5062d6fbd9	Correct sigv2 typo	2019-01-25 14:30:30 -08:00
Andrew Gaul	7d14ebaf09	Correct enable_content_md5 docs ... Both S3fsCurl::PutRequest and S3fsCurl::UploadMultipartPostSetup can calculate and send Content-MD5 to the server. Remove spurious comment about large files and make man page and help consistent. References #929.	2019-01-25 14:27:52 -08:00
Kareem Zidane	cd794a6985	Accept paths with : in them	2019-01-23 14:56:25 -05:00