833 Commits

Author	SHA1	Message	Date
Lars Christensen	58037da061	Fix multipart upload not setting curl options	2019-07-14 14:09:37 +02:00
Andrew Gaul	1eb266588e	Add a missing lock to SetCtime ... Also lock before log statements which touch member variables.	2019-07-13 17:53:38 -07:00
root	4e351c59e3	fix coredump caused by fd_manager_lock locking	2019-07-13 12:03:27 +08:00
Andrew Gaul	eb597289cb	Avoid null dereference ... Found via clang-tidy.	2019-07-12 18:40:24 -07:00
Andrew Gaul	6fd42d9fe4	Prefer modern C headers ... Found and fixed via clang-tidy.	2019-07-12 03:50:59 -07:00
Andrew Gaul	efff9c01a6	Avoid misplaced const via removing unneeded typedef ... Found via clang-tidy.	2019-07-12 03:41:19 -07:00
Andrew Gaul	a83d5baa90	Remove recursive locking ... Recursive locking is frowned upon and is incompatible with PTHREAD_MUTEX_ERRORCHECK. Also clean up pthread_mutex_lock error checking.	2019-07-10 12:39:00 -07:00
Takeshi Nakatani	50d13255e4	Merge pull request #1081 from gaul/write-blocks-readdir ... Break FdEntity locks into data and metadata	2019-07-11 03:40:26 +09:00
Takeshi Nakatani	e5e63d6ac3	Merge pull request #1067 from gaul/mismatched-free ... Fix mismatched new[]/free	2019-07-07 16:55:07 +09:00
Takeshi Nakatani	7a65a414c3	Merge pull request #1080 from gaul/xml/simplification ... Add simple XML parsing wrapper	2019-07-07 16:44:18 +09:00
Andrew Gaul	944d21cabb	Fix mismatched new[]/free ... Found via Valgrind.	2019-07-07 00:36:48 -07:00
Takeshi Nakatani	d267212289	Merge pull request #1079 from gaul/cppcheck/shadowing ... Avoid shadowing variable in FdEntity::Open	2019-07-07 16:29:42 +09:00
Takeshi Nakatani	58d8e5586a	Merge pull request #1078 from gaul/multipart/2x ... Issue multipart when object size exceeds part size	2019-07-07 16:21:22 +09:00
Takeshi Nakatani	ce803daf4a	Merge pull request #1077 from gaul/openssl/locking ... Annotate OpenSSL locking functions as unused	2019-07-07 16:10:14 +09:00
Takeshi Nakatani	9bf34e2fda	Merge pull request #1076 from gaul/doc/man-help ... Add documentation for use_session_token and use_rrs	2019-07-07 15:45:42 +09:00
Takeshi Nakatani	6bd1a7eac0	Merge pull request #1074 from gaul/clang-tidy ... Configure clang-tidy target	2019-07-07 15:28:08 +09:00
Andrew Gaul	8aa06d621a	Add documentation for use_session_token and use_rrs ... Fixes #929.	2019-07-06 10:04:43 -07:00
Andrew Gaul	ecf13a8cb9	Break FdEntity locks into data and metadata ... Previously long-running data operations like RowFlush would block metadata operations like GetStats and thus user readdir. Fixes #928.	2019-07-05 23:12:24 -07:00
Andrew Gaul	b8ff6a647e	Add simple XML parsing wrapper ... Also simplify check_region_error.	2019-07-05 17:39:09 -07:00
Andrew Gaul	49110c671d	Avoid shadowing variable in FdEntity::Open ... Found via cppcheck 1.88.	2019-07-05 15:06:18 -07:00
Andrew Gaul	febaf6849f	Issue multipart when object size exceeds part size ... Previously s3fs issued multipart uploads when the object size was twice the part size. Conjoining this with the part size was confusing and s3fs should add a separate tunable for this if needed, similar to singlepart_copy_limit. Fixes #1058.	2019-07-05 12:14:56 -07:00
Andrew Gaul	4893174652	Annotate OpenSSL locking functions as unused ... OpenSSL 1.1.0 removed these and generates warnings with newer versions.	2019-07-05 10:45:57 -07:00
Andrew Gaul	4f23f38583	Individually test multiple s3fs flags ... Remove unneeded comments; single part limits ensure that the tests exercise multipart code paths even with smaller files. References #971.	2019-07-03 21:09:40 -07:00
Andrew Gaul	bbfa91141a	Configure clang-tidy target ... Also fix nits.	2019-07-03 14:04:11 -07:00
Takeshi Nakatani	21321a9d96	Merge pull request #1065 from gaul/doc/ssl-verify-hostname ... Document ssl_verify_hostname	2019-07-03 22:15:33 +09:00
Takeshi Nakatani	f03b50fd13	Merge pull request #1063 from gaul/truncate-2nd-open-fd ... Flush file when opening second fd	2019-07-03 21:14:03 +09:00
Andrew Gaul	f02105c346	Document ssl_verify_hostname ... References #1064.	2019-07-02 10:26:29 -07:00
Andrew Gaul	c596441f58	Flush file when opening second fd ... Previously when s3fs had dirty local data and an application opened a second fd it would remove the stat cache entry, query S3 for the size which was still zero, then reinitialize FdEntry with this incorrect size. Instead flush local data to S3 so this query works. It is possible that a more involved patch could do this with a less heavyweight approach but this requires changing open. This does not completely fix git clone but allows it to proceed further. Also make some cache methods const-correct. References #839.	2019-07-02 01:12:09 -07:00
Andrew Gaul	455e29cbea	Make fdpage a value type in fdpage_list_t ... This simplifies memory management.	2019-07-01 09:30:48 -07:00
Takeshi Nakatani	511d223468	Merge pull request #1059 from gaul/casting ... Fix casting warning on 32-bit	2019-07-01 22:26:14 +09:00
Andrew Gaul	d7f77a6282	Fix casting warning on 32-bit	2019-06-29 20:57:42 -07:00
Andrew Gaul	f1ad626b46	Log flags as hexadecimal	2019-06-24 16:13:07 -07:00
Takeshi Nakatani	a78d8d1da4	Merge pull request #1049 from gaul/external-modication ... Remove cache file when object time differs	2019-06-24 00:51:54 +09:00
Takeshi Nakatani	fbebc6fa57	Merge pull request #1048 from gaul/log/curl ... Emit older curl warnings only once	2019-06-24 00:42:44 +09:00
Andrew Gaul	9e5eaad79b	Remove cache file when object time differs ... Check the modification times to determine whether an object has updated. This relies on low clock skew between s3fs and the S3 server; a more robust approach could use the ETag. Fixes #1047.	2019-06-22 19:09:00 -07:00
Andrew Gaul	738eaadcbf	Emit older curl warnings only once ... This makes Travis logs actually readable.	2019-06-22 10:01:20 -07:00
Andrew Gaul	15b7450713	Convert file offsets from size_t to off_t ... The latter is 64-bits on 32-bit platforms when specifying -D_FILE_OFFSET_BITS=64. This allows early Raspberry Pis to use files larger than 2 GB. It also cleans up some ugly casting. Fixes #620. Fixes #656.	2019-06-15 17:05:37 -07:00
Andrew Gaul	059ab1f0f4	Update ctime during nocopyapi operations ... Follows on to 2c43b1e12b14fcbdc80bc45ff71733be05548cd7. Fixes #971.	2019-04-30 18:59:51 +09:00
Michal Lula	0d4847596e	fix issue with aws session token provided inside ~/.aws/credentials file	2019-04-25 14:29:35 +02:00
Takeshi Nakatani	8e86ef1634	Merge pull request #1015 from gaul/c++03 ... Remove uses of std::map::at	2019-04-22 23:01:29 +09:00
Andrew Gaul	9e4f9d4bdc	Remove uses of std::map::at ... This provides compatibility with C++03 and removes duplicate lookups.	2019-04-18 16:06:59 +09:00
Andrew Gaul	0677a096a9	Prefer simple over compound statements in macros ... This prohibits missing semicolons.	2019-04-17 21:46:24 +09:00
Michal Lula	f9cd43b684	add session token support	2019-04-16 16:53:05 +02:00
Andrew Gaul	8a18806a57	Make man page and --help more consistent ... Mostly small fixes but also some reordering. References #929. Found via: diff -u <(man --no-hyphenation doc/man/s3fs.1 | tr -s ' ' '\n' | sed '/^-o$/d' ) <(src/s3fs --help | tr -s ' ' '\n' | sed '/^-$/d')	2019-04-14 14:34:24 +09:00
Takeshi Nakatani	0a99470369	Merge pull request #1002 from gaul/sse-c ... Correctly calculate MD5 hash for SSE-C keys	2019-04-09 00:17:12 +09:00
Takeshi Nakatani	cd280d8702	Merge pull request #1001 from gaul/missing-braces ... Add missing braces	2019-04-08 23:55:02 +09:00
Andrew Gaul	6f8ecb28c5	Correctly calculate MD5 hash for SSE-C keys ... Previously s3fs calculated the strlen of a binary input instead of using the stored length. This yielded IO errors when interacting with SSE-encrypted objects. Fixes #696.	2019-04-08 21:49:42 +09:00
Andrew Gaul	042332bcec	Add missing braces ... Found via clang-tidy. Also fix errant indentation.	2019-04-07 23:12:27 +09:00
Takeshi Nakatani	071cd0f849	Modified to discard _netdev option etc	2019-04-07 11:51:55 +00:00
Andrew Gaul	6f6a67807b	Prefer new over malloc ... The former cannot return NULL and will allow use of scoped pointers.	2019-04-07 16:54:24 +09:00