1109 Commits

Author	SHA1	Message	Date
Takeshi Nakatani	21321a9d96	Merge pull request #1065 from gaul/doc/ssl-verify-hostname ... Document ssl_verify_hostname	2019-07-03 22:15:33 +09:00
Takeshi Nakatani	f03b50fd13	Merge pull request #1063 from gaul/truncate-2nd-open-fd ... Flush file when opening second fd	2019-07-03 21:14:03 +09:00
Andrew Gaul	f02105c346	Document ssl_verify_hostname ... References #1064.	2019-07-02 10:26:29 -07:00
Andrew Gaul	c596441f58	Flush file when opening second fd ... Previously when s3fs had dirty local data and an application opened a second fd it would remove the stat cache entry, query S3 for the size which was still zero, then reinitialize FdEntry with this incorrect size. Instead flush local data to S3 so this query works. It is possible that a more involved patch could do this with a less heavyweight approach but this requires changing open. This does not completely fix git clone but allows it to proceed further. Also make some cache methods const-correct. References #839.	2019-07-02 01:12:09 -07:00
Andrew Gaul	455e29cbea	Make fdpage a value type in fdpage_list_t ... This simplifies memory management.	2019-07-01 09:30:48 -07:00
Takeshi Nakatani	511d223468	Merge pull request #1059 from gaul/casting ... Fix casting warning on 32-bit	2019-07-01 22:26:14 +09:00
Andrew Gaul	d7f77a6282	Fix casting warning on 32-bit	2019-06-29 20:57:42 -07:00
Andrew Gaul	f1ad626b46	Log flags as hexadecimal	2019-06-24 16:13:07 -07:00
Takeshi Nakatani	a78d8d1da4	Merge pull request #1049 from gaul/external-modication ... Remove cache file when object time differs	2019-06-24 00:51:54 +09:00
Takeshi Nakatani	fbebc6fa57	Merge pull request #1048 from gaul/log/curl ... Emit older curl warnings only once	2019-06-24 00:42:44 +09:00
Andrew Gaul	9e5eaad79b	Remove cache file when object time differs ... Check the modification times to determine whether an object has updated. This relies on low clock skew between s3fs and the S3 server; a more robust approach could use the ETag. Fixes #1047.	2019-06-22 19:09:00 -07:00
Andrew Gaul	738eaadcbf	Emit older curl warnings only once ... This makes Travis logs actually readable.	2019-06-22 10:01:20 -07:00
Andrew Gaul	15b7450713	Convert file offsets from size_t to off_t ... The latter is 64-bits on 32-bit platforms when specifying -D_FILE_OFFSET_BITS=64. This allows early Raspberry Pis to use files larger than 2 GB. It also cleans up some ugly casting. Fixes #620. Fixes #656.	2019-06-15 17:05:37 -07:00
Andrew Gaul	059ab1f0f4	Update ctime during nocopyapi operations ... Follows on to 2c43b1e12b. Fixes #971.	2019-04-30 18:59:51 +09:00
Michal Lula	0d4847596e	fix issue with aws session token provided inside ~/.aws/credentials file	2019-04-25 14:29:35 +02:00
Takeshi Nakatani	8e86ef1634	Merge pull request #1015 from gaul/c++03 ... Remove uses of std::map::at	2019-04-22 23:01:29 +09:00
Andrew Gaul	9e4f9d4bdc	Remove uses of std::map::at ... This provides compatibility with C++03 and removes duplicate lookups.	2019-04-18 16:06:59 +09:00
Andrew Gaul	0677a096a9	Prefer simple over compound statements in macros ... This prohibits missing semicolons.	2019-04-17 21:46:24 +09:00
Michal Lula	f9cd43b684	add session token support	2019-04-16 16:53:05 +02:00
Andrew Gaul	8a18806a57	Make man page and --help more consistent ... Mostly small fixes but also some reordering. References #929. Found via: diff -u <(man --no-hyphenation doc/man/s3fs.1 | tr -s ' ' '\n' | sed '/^-o$/d' ) <(src/s3fs --help | tr -s ' ' '\n' | sed '/^-$/d')	2019-04-14 14:34:24 +09:00
Takeshi Nakatani	0a99470369	Merge pull request #1002 from gaul/sse-c ... Correctly calculate MD5 hash for SSE-C keys	2019-04-09 00:17:12 +09:00
Takeshi Nakatani	cd280d8702	Merge pull request #1001 from gaul/missing-braces ... Add missing braces	2019-04-08 23:55:02 +09:00
Andrew Gaul	6f8ecb28c5	Correctly calculate MD5 hash for SSE-C keys ... Previously s3fs calculated the strlen of a binary input instead of using the stored length. This yielded IO errors when interacting with SSE-encrypted objects. Fixes #696.	2019-04-08 21:49:42 +09:00
Andrew Gaul	042332bcec	Add missing braces ... Found via clang-tidy. Also fix errant indentation.	2019-04-07 23:12:27 +09:00
Takeshi Nakatani	071cd0f849	Modified to discard _netdev option etc	2019-04-07 11:51:55 +00:00
Andrew Gaul	6f6a67807b	Prefer new over malloc ... The former cannot return NULL and will allow use of scoped pointers.	2019-04-07 16:54:24 +09:00
Takeshi Nakatani	ea517c80a4	Fixed a bug about overwrite metadata at updating directory stats	2019-03-29 15:30:30 +00:00
Takeshi Nakatani	71766039ff	Support undefined CURLoption in libcurl library used in build	2019-03-22 10:47:42 +00:00
Andrew Gaul	058706014b	Simplify string comparison ... Found via newer cppcheck.	2019-03-14 22:19:33 +09:00
Takeshi Nakatani	62dcda6a56	Fixed ref-count when error occurred.	2019-03-10 06:04:19 +00:00
Takeshi Nakatani	cbf072bc55	Merge pull request #967 from gaul/reference-count ... Increase FdEntity reference count when returning	2019-03-10 14:27:10 +09:00
Takeshi Nakatani	b5ca400500	Merge pull request #970 from gaul/nocopyapi ... Remove from file from stat cache during rename	2019-03-10 12:56:04 +09:00
Andrew Gaul	08b132ddb9	Remove from file from stat cache during rename ... This addresses failures with test_rename_before_close when using nocopyapi. Note that test_update_time still fails.	2019-03-09 17:53:35 +09:00
Andrew Gaul	1e86cc643d	Fix lazy typo	2019-03-09 17:46:02 +09:00
Andrew Gaul	f53503438c	Increase FdEntity reference count when returning ... Previously s3fs had a race condition where one client could delete FdEntity that another client was using. Add a simple concurrent test which previously failed but now succeeds. Fixes #964.	2019-03-09 15:55:04 +09:00
Takeshi Nakatani	0d43d070cc	Merge pull request #965 from ggtakec/fix_multi_x ... Improvement of curl session pool for multipart	2019-02-27 22:44:25 +09:00
Takeshi Nakatani	0791fdca2a	Merge pull request #960 from kristjanvalur/wtf8 ... Wtf8	2019-02-27 21:21:38 +09:00
Kristján Valur Jónsson	6e8678d5e3	remove lines that wer accidentally duplicated	2019-02-25 16:02:01 +00:00
Takeshi Nakatani	10d9f75366	Improvement of curl session pool for multipart ... Improvement of curl session pool for multipart(2) Improvement of curl session pool for multipart(3)	2019-02-25 14:46:24 +00:00
Andrew Gaul	74d8671e54	Work around cppcheck warnings ... Follows on to eeb839242b.	2019-02-20 21:55:41 +09:00
Kristján Valur Jónsson	4c41eac29c	fix documentation and man page	2019-02-20 11:24:29 +00:00
Kristján Valur Jónsson	3c97c1b251	merged main	2019-02-19 10:53:00 +00:00
Kristján Valur Jónsson	84c671a81a	fix indentation	2019-02-19 10:37:43 +00:00
Kristján Valur Jónsson	f336bdebcc	add command line flag and documentation	2019-02-19 10:32:37 +00:00
Kristján Valur Jónsson	e5b8377202	fix comments and code	2019-02-19 10:32:10 +00:00
Kristján Valur Jónsson	4f42f4ab0c	Enable s3fs encoding and decoding in the fuse interface	2019-02-18 13:36:48 +00:00
Kristján Valur Jónsson	11b385820d	more robust wtf8 encoding	2019-02-18 12:27:44 +00:00
Takeshi Nakatani	eeb839242b	Fixed code for latest cppcheck error on OSX	2019-02-17 13:59:11 +00:00
Kristján Valur Jónsson	ca2d1d873d	Adding utility functions to convert invalid utf8 to wtf8 encoding ... This is to deal with windows clients who pass in cp1252 as if it were utf8	2019-02-15 15:57:03 +00:00
Takeshi Nakatani	231fd001d9	Added a non-interactive option to utility mode ... Rebase for resolving conflicts Deleted interactive mode on utility mode Fixed minor mistakes	2019-02-10 17:36:18 +00:00
Takeshi Nakatani	314dc5a398	Merge pull request #955 from gaul/clang-tidy/misc ... Address miscellaneous clang-tidy warnings	2019-02-06 21:21:54 +09:00
Takeshi Nakatani	e07cb020cc	Merge pull request #954 from gaul/clear-iter ... Clear containers instead of individual erases	2019-02-06 21:12:36 +09:00
Andrew Gaul	e87e40b3b4	Address miscellaneous clang-tidy warnings ... Also add configuration with suppressions.	2019-02-04 22:58:40 -08:00
Takeshi Nakatani	bd66b57ad3	Merge pull request #952 from gaul/multipart/failed ... Automatically abort failed multipart requests	2019-02-04 22:19:27 +09:00
Andrew Gaul	a1d3ff9766	Automatically abort failed multipart requests ... This can avoid dangling parts. However, many transfers fail due to network errors so we still need other mechanisms to handle these parts.	2019-02-03 10:29:20 -08:00
Andrew Gaul	4d0bef1e90	Clear containers instead of individual erases ... This has O(n) runtime instead of O(n log n).	2019-02-02 23:58:43 -08:00
Takeshi Nakatani	960823fb40	Added S3FS_MALLOC_TRIM build switch	2019-02-03 07:36:17 +00:00
Takeshi Nakatani	c04e8e7a9d	Merge pull request #949 from gaul/503-slow-down ... Implement exponential backoff for 503	2019-02-03 15:19:28 +09:00
Takeshi Nakatani	fb6debd986	Merge pull request #948 from gaul/too-many-parts ... Add logging for too many parts	2019-02-03 14:55:54 +09:00
Takeshi Nakatani	d8185a25aa	Merge pull request #946 from gaul/async/completed-tids ... Simplify async request completion code	2019-02-03 14:08:49 +09:00
Takeshi Nakatani	53337a0a28	Merge pull request #944 from gaul/utility-mode ... Repair utility mode	2019-02-03 13:59:56 +09:00
Takeshi Nakatani	ae51556d04	Merge pull request #943 from gaul/hard-link ... Return not supported when hard linking	2019-02-03 12:45:21 +09:00
Takeshi Nakatani	055ecf6ea7	Merge pull request #940 from gaul/parallel-multipart-copy ... Copy parts in parallel	2019-02-03 11:44:15 +09:00
Takeshi Nakatani	c603680e02	Merge pull request #939 from gaul/stringstream-specific ... Prefer specific [io]stringstream where possible	2019-02-03 11:06:57 +09:00
Takeshi Nakatani	814aadd7e3	Merge pull request #938 from gaul/clang-tidy/redundant-void ... Remove unneeded void parameter	2019-02-03 10:57:56 +09:00
Andrew Gaul	8ff05d8e38	Implement exponential backoff for 503 ... Amazon returns SlowDown when overloaded. Also return ENOTSUP for 501 and immediately return EIO for 500 instead of retrying. Fixes #603.	2019-02-02 00:03:50 -08:00
Andrew Gaul	dfa84b82a8	Add logging for too many parts ... References #610.	2019-02-01 19:34:26 -08:00
Andrew Gaul	8c527c3616	Simplify async request completion code ... Workers now notify the master thread when they complete, unifying the Linux and macOS code paths. This also avoids excessive pthread_tryjoin_np calls. Follows on to 88cd8feb05.	2019-01-31 22:55:23 -08:00
Andrew Gaul	54a074647e	Repair utility mode ... This deinitialized S3fsCurl twice and incorrectly calculated V4 signatures.	2019-01-31 18:45:39 -08:00
Andrew Gaul	c5ebf5d328	Copy parts in parallel ... S3 can copy multipart much faster than single part due to IO parallelization. Renaming a 4 GB file reduces from 72 to 20 seconds with bigger gains with larger files.	2019-01-31 10:21:39 -08:00
Andrew Gaul	43c6ef560e	Return not supported when hard linking ... This is more correct than permission denied.	2019-01-30 16:43:04 -08:00
Andrew Gaul	3076abc744	Disable malloc_trim ... This avoids walking the entire heap multiple times for complex operations like readdir. This does not entirely eliminate the observed performance regression but does dramatically reduce s3fs CPU usage. References #935. Fixes #936.	2019-01-29 15:29:07 -08:00
Andrew Gaul	07636c8a8d	Prefer specific [io]stringstream where possible ... These better communicate intent and are slightly more efficient.	2019-01-29 10:44:33 -08:00
Andrew Gaul	35d55ee513	Remove unneeded void parameter ... This is implicit in C++. Found and fixed via clang-tidy.	2019-01-28 23:22:27 -08:00
Takeshi Nakatani	c0cf90cf8b	Checked and corrected all typo	2019-01-27 12:04:29 +00:00
Takeshi Nakatani	3b1cc3b197	Merge pull request #933 from gaul/cache/remove-mirror-path ... Remove mirror path when deleting cache	2019-01-27 16:15:49 +09:00
Takeshi Nakatani	a0c1f30ae7	Merge pull request #932 from gaul/autolock ... Prefer AutoLock for synchronization	2019-01-27 15:59:18 +09:00
Takeshi Nakatani	8822a86709	Merge pull request #931 from gaul/doc/typo ... Correct sigv2 typo	2019-01-27 15:48:24 +09:00
Takeshi Nakatani	98f397de0e	Merge pull request #930 from gaul/doc/md5-multipart ... Correct enable_content_md5 docs	2019-01-27 15:36:56 +09:00
Takeshi Nakatani	fd4d23f8f7	Merge pull request #926 from kzidane/master ... Accept paths with : in them	2019-01-27 15:23:24 +09:00
Takeshi Nakatani	4820f0a42b	Merge pull request #925 from gaul/clang-tidy/delete-null ... Remove redundant null checks before delete	2019-01-27 15:15:31 +09:00
Andrew Gaul	a93e500b44	Remove mirror path when deleting cache ... Fixes #827.	2019-01-25 18:10:03 -08:00
Andrew Gaul	92d3114584	Prefer AutoLock for synchronization ... This simplifies the code and fixes an issue with unlocked access. Also use a recursive lock for StatCache to avoid races between lock..unlock..lock sequences.	2019-01-25 15:28:41 -08:00
Andrew Gaul	5062d6fbd9	Correct sigv2 typo	2019-01-25 14:30:30 -08:00
Andrew Gaul	7d14ebaf09	Correct enable_content_md5 docs ... Both S3fsCurl::PutRequest and S3fsCurl::UploadMultipartPostSetup can calculate and send Content-MD5 to the server. Remove spurious comment about large files and make man page and help consistent. References #929.	2019-01-25 14:27:52 -08:00
Kareem Zidane	cd794a6985	Accept paths with : in them	2019-01-23 14:56:25 -05:00
Andrew Gaul	84b421d6ef	Prefer empty over size checks ... Found and fixed via clang-tidy.	2019-01-23 11:30:28 -08:00
Andrew Gaul	8316da5bbe	Remove redundant null checks before delete ... Found by clang-tidy.	2019-01-23 11:25:25 -08:00
Takeshi Nakatani	caaf4cac55	Reverted #912(Automatic region change) and added message	2019-01-23 13:23:03 +00:00
Takeshi Nakatani	010276ceab	Merge pull request #921 from gaul/clang-tidy/redundant-string-init ... Remove redundant string initializations	2019-01-23 19:44:59 +09:00
Takeshi Nakatani	f219817eb3	Merge pull request #920 from gaul/clang-tidy/string-copy ... Remove unnecessary string copies	2019-01-23 19:30:55 +09:00
Andrew Gaul	1fc25e8c3f	Remove redundant string initializations ... Found and fixed via clang-tidy.	2019-01-22 23:16:37 -08:00
Andrew Gaul	61ecafd426	Remove unnecessary string copies ... Found via clang-tidy.	2019-01-22 23:09:37 -08:00
Andrew Gaul	5f5da4b2cb	Load tail range during overwrite ... Previously s3fs experienced data loss when writing to the middle of a file. Corrupt files would have the expected data from 0..offset+size but unexpected NUL bytes from offset+size..EOF. References #808.	2019-01-22 22:02:40 -08:00
Takeshi Nakatani	fada95f58e	Merge pull request #914 from gaul/readdir/head-of-line ... Issue readdir HEAD requests without batching	2019-01-21 22:10:50 +09:00
Takeshi Nakatani	014b8c5982	Merge pull request #913 from gaul/assert ... Prefer abort over assert(false)	2019-01-21 21:56:41 +09:00
Andrew Gaul	46d79c5bc2	Issue readdir HEAD requests without batching ... Previously s3fs would issue a batch of HEAD requests and wait for all to succeed before issuing the next batch. Now it issues the first batch and only waits for a single call to succeed before issuing the next call. This can improve performance when one call lags due to network errors. I measured 25% improvement with the same level of parallelism. This commit also reparents parallelism knobs for consistency. Follows on to 88cd8feb05. Fixes #223.	2019-01-20 18:07:22 -08:00
Andrew Gaul	40ba3b44a1	Prefer abort over assert(false) ... The compiler can remove the latter when compiled with NDEBUG which may cause unintended control flow.	2019-01-20 12:30:27 -08:00
Takeshi Nakatani	2887f8916b	Automatic region change made possible other than us-east-1(default)	2019-01-20 10:51:49 +00:00
Takeshi Nakatani	ac72431195	Added detail error message when HTTP 301/307 status	2019-01-20 10:07:58 +00:00
Takeshi Nakatani	f2184e34dd	Ignore after period character of floating point in x-amz-meta-mtime	2019-01-20 08:28:06 +00:00
Takeshi Nakatani	36a4903843	Added an error message when HTTP 301 status	2019-01-20 07:17:40 +00:00
Takeshi Nakatani	c83a3e67c9	Merge pull request #885 from LutzFinsterle2019/master ... Update s3fs_util.cpp for correspondence of Nextcloud contype	2019-01-20 15:32:58 +09:00
Takeshi Nakatani	05014c49c8	Merge pull request #906 from gaul/doc/https ... Prefer HTTPS links where possible	2019-01-20 15:10:51 +09:00
Takeshi Nakatani	aa69107165	Merge pull request #905 from gaul/clang-tidy/redundant ... Fix comparison in s3fs_strtoofft	2019-01-20 15:01:40 +09:00
Takeshi Nakatani	d373b0eca3	Merge pull request #904 from gaul/clang-tidy/c-str ... Remove unnecessary calls to std::string::c_str	2019-01-20 14:13:46 +09:00
Takeshi Nakatani	6aa40b2747	Merge pull request #903 from gaul/clang-tidy/find_char ... Prefer find(char) over find(const char *)	2019-01-20 14:05:06 +09:00
Takeshi Nakatani	34c3bfe408	Merge pull request #902 from gaul/clang-tidy/pass-by-value ... Avoid pass-by-value when not necessary	2019-01-20 13:45:44 +09:00
Takeshi Nakatani	6ac56e722d	Merge pull request #901 from gaul/clang-tidy/leaks ... Plug memory leaks	2019-01-20 12:41:08 +09:00
Takeshi Nakatani	61dc7f0a70	Merge pull request #900 from gaul/leak ... Plug memory leak	2019-01-20 12:30:31 +09:00
Takeshi Nakatani	9f000957dd	Merge pull request #899 from gaul/response-code ... Tighten up HTTP response code check	2019-01-20 11:16:23 +09:00
Andrew Gaul	aa9bd1fa3c	Prefer HTTPS links where possible ... Fix a few stale links as well.	2019-01-18 11:09:08 -08:00
Andrew Gaul	5a2dc03a1c	Fix comparison in s3fs_strtoofft ... Also backfill unit tests. Document limitations. Found via clang-tidy.	2019-01-17 22:59:25 -08:00
Andrew Gaul	508fafbe62	Remove unnecessary calls to std::string::c_str ... Found via clang-tidy.	2019-01-17 22:05:16 -08:00
Andrew Gaul	e29548178b	Prefer find(char) over find(const char *) ... The former can be faster. Found via clang-tidy.	2019-01-17 20:24:24 -08:00
Andrew Gaul	ab2f36f202	Plug memory leaks ... Found via clang-tidy.	2019-01-17 18:54:34 -08:00
Andrew Gaul	b8c9fcfd70	Avoid pass-by-value when not necessary ... This requires unnecessary memcpy. Found via clang-tidy.	2019-01-17 18:22:11 -08:00
Andrew Gaul	58ce544e83	Plug memory leak ... Previously this appended to an empty list, zeroed the list, then appended to the list. Instead zero the list first and then append. This also enables sending Content-MD5 which can eagerly detect transmission errors. Found via Valgrind.	2019-01-17 16:24:34 -08:00
Andrew Gaul	e98ce36301	Tighten up HTTP response code check ... Previously s3fs considered 3xx codes as successful. When writing an object to a misconfigured region, s3fs did not propagate the 301 as a write error to the client. References #693.	2019-01-17 15:04:59 -08:00
Andrew Gaul	6401b4ae92	Flush file before renaming ... Previously s3fs could copy the zero-byte stub object without including any pending writes. Fixes #145.	2019-01-17 12:05:10 -08:00
Takeshi Nakatani	25b49e1a2e	Merge pull request #894 from gaul/default-mode ... Default uid/gid/mode when object lacks permissions	2019-01-16 18:48:54 +09:00
Takeshi Nakatani	c7def35b54	Merge pull request #895 from gaul/bucket-name-dot ... Emit more friendly error for buckets with dots	2019-01-16 18:39:47 +09:00
Andrew Gaul	c512516e14	Emit more friendly error for buckets with dots ... These fail SSL certificate checks due to the *.s3.amazon.com wildcard. Fixes #284.	2019-01-14 18:47:36 -08:00
Andrew Gaul	2c43b1e12b	Store and retrieve file change time ... This introduces a new header with the change time; existing objects will report modification time. Fixes #771.	2019-01-14 10:05:11 -08:00
Andrew Gaul	e2d5641d99	Default uid/gid/mode when object lacks permissions ... This addresses a common use case when interacting with objects from both s3fs and other S3 tools. Fixes #890.	2019-01-13 21:57:23 -08:00
LutzFinsterle2019	c985b5e4d0	Corrected Comment to C++ style	2019-01-12 10:19:48 +01:00
LutzFinsterle2019	18cb2e2662	Update s3fs_util.cpp ... Sorry for answering late, have been busy lately. The comment is: "Nextcloud stores Directory objects with this mime type when mounting a Bucket as external Storage"	2019-01-10 07:44:36 +01:00
LutzFinsterle2019	743c706b0a	Update s3fs_util.cpp ... Nextcloud Compatibility in directory mime-types	2019-01-07 07:41:27 +01:00
Takeshi Nakatani	4ed0e5f35a	Merge pull request #882 from earlchew/issue-817 ... [curl] Assume long encryption keys are base64 encoded	2019-01-06 17:29:04 +09:00
Takeshi Nakatani	fd6b37d3da	Merge pull request #877 from gaul/aws/credentials ... Check arguments and environment before .aws/creds	2019-01-06 17:03:11 +09:00
Takeshi Nakatani	56e24de0d4	Merge pull request #870 from gaul/typos ... Correct typos in command-line parsing	2019-01-06 16:21:46 +09:00
Earl Chew	ed5795eead	[curl] Assume long encryption keys are base64 encoded ... Correct tabs and whitespace. Signed-off-by: Earl Chew <earl_chew@yahoo.com>	2019-01-05 21:08:41 +00:00
Andrew Gaul	3d225163f8	Correct typos in command-line parsing ... Also fix stray macOS references. References #869.	2019-01-02 14:40:57 -08:00
Andrew Gaul	0569cec3ea	Check arguments and environment before .aws/creds ... Fixes #857. Fixes #864.	2018-12-20 15:51:32 -08:00
Andrew Gaul	a2f8ac535e	Address cppcheck 1.86 errors ... Lifetime, shadowing, and unused variables. Found via the Travis macOS builder.	2018-12-20 14:56:31 -08:00
Takeshi Nakatani	d9e89deef6	Merge pull request #865 from orozery/multihead_warning_check ... fix multihead warning check	2018-11-29 20:53:47 +09:00
Or Ozeri	b91fc5409e	fix multihead warning check	2018-11-28 09:54:02 +02:00
Michael Chen	3c970646d1	Add 'profile' option to command line help.	2018-11-19 08:26:23 -08:00
Andrew Gaul	88cd8feb05	Upload S3 parts without batching ... Previously s3fs would issue a batch of uploads and wait for all to succeed before issuing the next batch. Now it issues the first batch and only waits for a single part to succeed before uploading the next part. This can improve performance when one part lags due to network errors. Fixes #183.	2018-11-16 18:32:38 -08:00
Takeshi Nakatani	5fba542a29	Merge pull request #852 from gaul/aws-credentials-file ... Allow credentials from ${HOME}/.aws/credentials	2018-11-11 11:37:38 +09:00
Andrew Gaul	9e530c86ae	Allow credentials from ${HOME}/.aws/credentials ... This matches the configuration from popular tools like AWS CLI and allows multiple profile names via -o profile=name. The existing credential mechanisms continue to work. Fixes #822.	2018-11-04 17:47:07 -08:00
Andrew Gaul	664f910083	Correctly compare list_object_max_keys ... Previously this did not allow an argument. Fixes #843. References #793.	2018-11-04 10:49:35 -08:00
Takeshi Nakatani	6fe92d5ed6	Merge pull request #832 from gaul/hex ... Simplify hex conversion	2018-10-08 23:03:42 +09:00
Andrew Gaul	c703fa15c0	Simplify hex conversion ... Addresses GCC 8 warning: common_auth.cpp: In function ‘std::__cxx11::string s3fs_sha256sum(int, off_t, ssize_t)’: common_auth.cpp:84:12: warning: ‘char* strncat(char*, const char*, size_t)’ output may be truncated copying 2 bytes from a string of length 2 [-Wstringop-truncation] strncat(sha256, hexbuf, 2);	2018-10-01 19:08:27 -07:00
Jonatan Nilsson	d9c106cfde	Add support for storage class ONEZONE_IA.	2018-09-30 14:27:31 +02:00
Takeshi Nakatani	c5af62b023	Merge pull request #820 from gaul/big-writes ... Enable big writes if capable	2018-09-17 17:24:06 +09:00
Earl Chew	41c23adb0e	[curl] Assume long encryption keys are base64 encoded ... Amazon SSE-C https://tinyurl.com/ychug4cg writes: > Use this header to provide the 256-bit, base64-encoded encryption key > for Amazon S3 to use to encrypt or decrypt your data. It seems likely that future keys would be as long, or longer, so this change assumes that text longer than 32 characters are keys that are base64 encoded. Signed-off-by: Earl Chew <earl_chew@yahoo.com>	2018-09-16 21:23:22 +00:00
Andrew Gaul	a85183d42c	Enable big writes if capable ... Fixes #813.	2018-09-10 22:21:51 -07:00
Mattia Rizzolo	c376efdd28	Fix typo s/mutliple/multiple/ ... Signed-off-by: Mattia Rizzolo <mattia@mapreri.org>	2018-08-19 00:14:57 +02:00
Takeshi Nakatani	7e0c53dfe9	Added list_object_max_keys option based on #783 PR	2018-07-08 03:49:10 +00:00
root	35d3fce7a0	Review comment: Include the error code being returned	2018-07-06 05:14:32 -04:00
root	4177d8bd3b	Review comment: Include the error code being returned	2018-07-06 03:03:57 -04:00
root	ad5349a488	Changes as per review comments	2018-07-05 05:02:04 -04:00
root	92a4034c5e	Log messages for 5xx and 4xx HTTP response code	2018-07-04 03:50:45 -04:00
Takeshi Nakatani	3e4002df0d	Merge pull request #780 from wjt/initialize-libgcry ... gnutls_auth: initialize libgcrypt	2018-06-24 12:48:08 +09:00
Takeshi Nakatani	1b9ec7f4fc	Merge pull request #774 from nkkashyap/master ... Option for IAM authentication endpoint	2018-06-24 12:36:23 +09:00
Takeshi Nakatani	0d3fb0658a	Fixed a error by cppcheck on OSX	2018-06-24 02:38:59 +00:00
Will Thompson	73cf2ba95d	gnutls_auth: initialize libgcrypt ... Without this change, the following warning appears in the syslog/journal during startup: Libgcrypt warning: missing initialization - please fix the application From the [documentation][0]: > The function `gcry_check_version` initializes some subsystems used by > Libgcrypt and must be invoked before any other function in the > library. Fixes #524, which says: > gnutls is initialized by gnutls_global_init() function and > gcry_check_version() function for initializing libgcry is called from > this gnutls_global_init(). I checked the gnutls source and it hasn't contained a call to gcry_check_version() since the libgcrypt backend was removed in 2011 (commit 8116cdc8f131edd586dad3128ae35dd744cfc32f). In any case, the gcry_check_version() documentation continues: > It is important that these initialization steps are not done by a > library but by the actual application. so it would be incorrect for a library used by s3fs to initialize libgcrypt. [0]: https://www.gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html	2018-06-21 20:55:00 +01:00
Neeraj Kumar Kashyap	5a481e6a01	Option for IBM IAM auth endpoint added return	2018-06-04 16:44:14 +05:30
Neeraj Kumar Kashyap	d8e12839af	Option for IBM IAM auth endpoint	2018-05-31 16:02:48 +05:30
Or Ozeri	d4e86a17d1	Revert "enable FUSE read_sync by default" ... This reverts commit 86b0921ac4. Conflicts: src/s3fs.cpp	2018-05-28 13:49:54 +03:00
Takeshi Nakatani	ae9d8eb734	Fixed memory leak	2018-05-27 10:48:03 +00:00
Takeshi Nakatani	57b2a60172	Merge pull request #764 from orozery/remove_false_multihead_warnings ... Remove false multihead warnings	2018-05-23 22:38:35 +09:00
Takeshi Nakatani	212bbbbdf0	Merge pull request #763 from orozery/cleanup_share_after_handles ... cleanup curl handles before curl share	2018-05-23 22:30:36 +09:00
Takeshi Nakatani	da95afba8a	Merge pull request #756 from orozery/optimize_defaults ... Optimize defaults	2018-05-23 22:05:00 +09:00
Or Ozeri	0bd875eb9e	remove false readdir_multi_head warnings	2018-05-22 17:10:50 +03:00
Or Ozeri	af63a42773	cleanup curl handles before curl share	2018-05-21 13:20:09 +03:00
Or Ozeri	86b0921ac4	enable FUSE read_sync by default	2018-05-06 16:10:36 +03:00
Or Ozeri	4a72b60707	increase default stat cache size from 1000 to 100000	2018-05-06 15:31:07 +03:00
Takeshi Nakatani	e3de6ea458	Added reset curl handle when returning to handle pool	2018-05-06 12:11:53 +00:00
Neeraj Kumar Kashyap	25375a6b48	Validate the URL fixed inefficient usage of find	2018-05-04 11:24:32 +05:30
Neeraj Kumar Kashyap	ca87df7d44	Validate the URL format for http/https	2018-05-03 22:08:28 +05:30
Takeshi Nakatani	d052dc0b9d	Merge pull request #753 from cfz/master ... fix xpath selector in bucket listing	2018-05-02 12:04:12 +09:00
陈方舟	04493de767	fix xpath selector in bucket listing ... the original implementation in get_base_exp() depends on the order of xml return from the server. patriotically, when listing a directory with sub directory(s), the xml document response contains more than 2 <Prefix> nodes(some of them are in <CommonPrefixes> node). the source code arbitrarily select the first one in the documents (nodes->nodeTab[0]->xmlChildrenNode). some s3 compatible service return the list-bucket result in different result, leading the s3fs to a wrong behavior	2018-04-23 15:11:29 +08:00
Or Ozeri	4fdab46617	don't fail mkdir when directory exists	2018-04-08 11:13:47 +03:00
Takeshi Nakatani	1a23b880d5	Merge pull request #739 from orozery/cleanup_failing_curl_handles ... cleanup curl handle state on retries	2018-04-01 22:45:04 +09:00
Or Ozeri	adcf5754ae	cleanup failing curl handles on retries	2018-03-29 13:56:08 +03:00
Tristen Horton	0863672e27	add a more helpful error message for when there are multiple entries for the same bucket in the passwd file	2018-03-13 14:37:34 -07:00
Takeshi Nakatani	0f503ced25	Merge pull request #729 from dmgk/master ... FreeBSD build fixes	2018-03-04 16:36:31 +09:00
Takeshi Nakatani	987a166bf4	Merge pull request #726 from orozery/instance_name_logging ... add an instance_name option for logging	2018-03-04 15:41:12 +09:00
Takeshi Nakatani	57b6f0eeaf	Merge pull request #724 from orozery/dont_fail_multirequest ... don't fail multirequest on single thread error	2018-03-04 15:35:29 +09:00
Takeshi Nakatani	f71a28f9b9	Merge pull request #714 from orozery/reduce_lock_contention ... reduce lock contention on file open	2018-03-04 13:36:08 +09:00
Dmitri Goutnik	c9f4312588	FreeBSD build fixes	2018-03-02 15:58:52 -05:00
Or Ozeri	8b657eee41	add disk space reservation	2018-02-28 19:20:23 +02:00
Takeshi Nakatani	be9d407fa0	Fixed cppcheck error on osx	2018-02-28 12:29:58 +00:00
Takeshi Nakatani	c494e54320	Fixed cppcheck error on osx	2018-02-28 12:06:06 +00:00
Or Ozeri	b52b6f3fc5	add an instance_name option for logging	2018-02-28 09:51:35 +02:00
Or Ozeri	82c9733101	don't fail multirequest on single thread error	2018-02-26 12:06:08 +02:00
Takeshi Nakatani	a45ff6cdaa	Fixed cppcheck error and clean ^M code	2018-02-25 13:08:41 +00:00
Takeshi Nakatani	960d45c853	Fixed cppcheck error on osx	2018-02-25 08:51:19 +00:00
Or Ozeri	0edf056e95	reduce lock contention on file open	2018-02-04 17:13:58 +02:00
Takeshi Nakatani	6b58220009	Merge pull request #697 from pwulff/master ... Fixing race condition in FdEntity::GetStats	2017-12-17 15:46:48 +09:00
Paul Wulff	ee6abea956	Race condition in FdManager::Rename because no mutex is used.	2017-12-15 15:27:51 +01:00
Paul Wulff	cea7d44717	Fixing race condition in FdEntity::GetStats	2017-12-13 10:49:00 +01:00
Or Ozeri	0da87e75fe	fix condition for parallel download	2017-12-04 16:07:33 +02:00
Or Ozeri	bd4bc0e7f1	add support for IBM IAM authentication	2017-11-23 12:01:52 +02:00
Takeshi Nakatani	026260e7a1	Improved use of temporary files	2017-11-23 09:18:11 +00:00
Takeshi Nakatani	99fe93b7f1	Merge pull request #684 from gaul/signedness ... Correct signedness warning	2017-11-23 17:33:46 +09:00
Or Ozeri	11bd7128d2	remove use of jsoncpp	2017-11-22 13:36:27 +02:00
Andrew Gaul	7cda32664b	Correct signedness warning ... Fixes regression from 0418e53b3c.	2017-11-19 11:14:37 -08:00
Takeshi Nakatani	97fc845a6a	Changed functions about reading passwd file.	2017-11-19 11:49:11 +00:00
Takeshi Nakatani	7d9ac0163b	Changed functions about reading passwd file.	2017-11-19 11:38:12 +00:00
Takeshi Nakatani	e1928288fe	Merge pull request #676 from gaul/sstream ... Move str definition from header to implementation	2017-11-19 16:34:15 +09:00
Andrew Gaul	0418e53b3c	Reduce use of preprocessor ... This provides type-safety and avoids token expansion side effects.	2017-11-18 22:40:06 -08:00
Takeshi Nakatani	bad48ab59a	Merge pull request #671 from psyvision/master ... Add support for ECS metadata endpoint	2017-11-19 14:43:06 +09:00
Andrew Gaul	bbad76bb71	Move str definition from header to implementation ... Also clean up some char * to std::string conversions.	2017-11-18 11:34:34 -08:00
Richard Caunt	b95e4acaeb	Remove debug logging statements	2017-11-08 15:52:35 +00:00
Richard Caunt	c238701d09	Corrected ECS headers	2017-11-08 15:21:49 +00:00
Richard Caunt	60d2ac3c7a	Adding x-amz-security-token header	2017-11-08 15:09:59 +00:00
Richard Caunt	967ef4d56b	Corrected fat finger mistakes	2017-11-08 13:14:49 +00:00
Richard Caunt	ad57bdda6c	Corrected keycount check	2017-11-08 13:06:22 +00:00
Richard Caunt	a0b69d1d3d	Corrected keyval[].c_str()	2017-11-08 13:01:52 +00:00
Richard Caunt	5df94d7e33	Add debug messages	2017-11-08 09:50:39 +00:00
Richard Caunt	1cbe9fb7a3	Gotta pass that cppcheckgit add .	2017-11-07 21:41:51 +00:00
Richard Caunt	8660abaea2	Use jsoncpp to parse AWS JSON	2017-11-07 21:20:02 +00:00
Richard Caunt	366f0705a0	ECS credentials bug fixes	2017-11-06 21:45:58 +00:00
Richard Caunt	ccea87ca68	Added check for is_ecs during get_access_keys ... We need to presume that if `is_ecs` we are deferring access key loading	2017-11-06 11:02:27 +00:00
Richard Caunt	5d54883e2f	Remove commented out code	2017-11-05 19:25:34 +00:00
Richard Caunt	662f65c3c8	Add support for ECS metadata endpoint	2017-11-05 19:24:02 +00:00
Takeshi Nakatani	5db550a298	Fixed a bug in S3fsCurl::LocateBundle	2017-11-05 11:26:05 +00:00
Takeshi Nakatani	e3c77d2906	Merge pull request #664 from orozery/auth_refactor ... auth headers insertion refactoring	2017-11-05 15:14:40 +09:00
Or Ozeri	384b4cbafa	auth headers insertion refactoring	2017-10-30 11:52:58 +02:00
Andrew Gaul	40501a7a73	Lock FdEntity when mutating orgmeta ... References #654.	2017-10-23 22:41:42 -07:00
Takeshi Nakatani	ab89b4cd4a	Merge pull request #659 from ggtakec/master ... Do not fail updating directory when removing old-style object(ref #658)	2017-10-15 21:56:35 +09:00
Takeshi Nakatani	1eba27a50a	Refixed s3fs_init message(ref #652)	2017-10-15 06:45:19 +00:00
Takeshi Nakatani	41206fa0e2	Do not fail updating directory when removing old-style object(ref #658)	2017-10-15 05:03:44 +00:00
jurafxp	ae91b6f673	Fix s3fs_init message	2017-10-01 00:08:00 +02:00
Takeshi Nakatani	5957d9ead0	Fixed with unnecessary equal in POST uploads url argment - #643	2017-09-17 10:52:28 +00:00
Takeshi Nakatani	00bc9142c4	Fixed potential atomic violation in S3fsCurl::AddUserAgent - #633	2017-09-17 09:16:05 +00:00
Scott Talbert	20da0e4dd3	Fix intermittent upload failures on macOS ... There were multiple problems with the FdManager::GetFreeDiskSpace() function on macOS: 1) When calling statvfs(), f_frsize should be used instead of f_bsize when converting available blocks to bytes. This was causing the free space calculation to be incorrect. 2) On macOS, fsblkcnt_t is a 32-bit integer. Thus, when calculating available disk space, there were frequently overflows. This caused s3fs to incorrectly determine that the cache location was out of space in the middle of a transfer which caused uploads to fail. Changing this to a uint64_t resolves the problem.	2017-09-08 15:23:10 -04:00
Takeshi Nakatani	96d8e6d823	Merge remote-tracking branch 'upstream/macosx' into macosx	2017-08-11 14:20:12 +00:00
Takeshi Nakatani	62b8084300	Added travis test on osx for #601	2017-08-11 14:09:43 +00:00
Takeshi Nakatani	0c6a3882a2	Fixed clock_gettime build failure on macOS 10.12 Sierra - #600	2017-05-28 10:04:25 +00:00
Takeshi Nakatani	a08880ae15	Merge pull request #608 from tlevi/chown_nocopy ... Fix chown_nocopy losing existing uid/gid if unspecified	2017-05-28 18:36:08 +09:00
Tony Levi	cc94e1da26	Fix chown_nocopy losing existing uid/gid if unspecified	2017-05-25 16:53:08 +09:30
Tony Levi	2b7ea5813c	Expand buffer for group information if too small and retry	2017-05-23 10:42:43 +09:30
Takeshi Nakatani	337da59368	Not fallback to HTTP - #596	2017-05-13 15:47:39 +00:00
Takeshi Nakatani	d2ae14d8b7	Check bucket at public bucket and add nocopyapi option automatically	2017-05-13 07:48:50 +00:00
Takeshi Nakatani	7115835834	Check bucket at public bucket and add nocopyapi option automatically	2017-05-13 07:35:55 +00:00
Takeshi Nakatani	24df69f688	Backward compatible for changing default transport to HTTPS	2017-05-13 06:47:51 +00:00
Takeshi Nakatani	465c15ef40	Merge pull request #588 from andrewgaul/https ... Default transport to HTTPS	2017-05-09 23:04:57 +09:00
Takeshi Nakatani	a22675bafd	Merge pull request #567 from andrewgaul/default-acl ... Do not send ACL unless overridden	2017-05-09 23:03:27 +09:00
Andrew Gaul	0e0ae38f6d	Default transport to HTTPS ... This protects private data when used over the public Internet. Users can opt-in to unencrypted HTTP if they need additional performance on a local network. Fixes #282.	2017-05-07 10:59:54 -07:00
Andrew Gaul	7b30d5d15b	Do not send canned ACL header when empty string ... Some providers such as StorageGRID do not support canned ACLs. Setting to empty allows callers to omit the header. References #125.	2017-05-07 10:52:31 -07:00
Takeshi Nakatani	9d10a5aa70	Changed copyright year format for debian pkg	2017-05-07 11:24:17 +00:00
Takeshi Nakatani	a12e0d5ec4	Fixed failure to upload/copy with SSE_C and SSE_KMS	2017-05-07 09:29:08 +00:00
Takeshi Nakatani	d07c3f38b7	Check errors returned in 200 OK responses for put header request	2017-05-06 02:15:53 +00:00
Takeshi Nakatani	1fe0334c08	Enhanced bucket/path parameter check	2017-05-05 19:55:24 +00:00
Takeshi Nakatani	3ac39d61f8	Added notsup_compat_dir option	2017-05-05 17:28:29 +00:00
Takeshi Nakatani	864e20e1f2	Refactored the get_object_attribute function	2017-05-05 10:02:21 +00:00
Takeshi Nakatani	51b3183cba	Refactored the check_object_access function	2017-05-05 09:51:30 +00:00
Takeshi Nakatani	758b92e823	Added option for complementing lack of stat mode	2017-05-04 03:41:24 +00:00
Takeshi Nakatani	df0ff3a2fd	Merge pull request #556 from orozery/fix_nocache_multipart_upload ... fix multipart upload handling without cache	2017-04-16 19:22:15 +09:00
Takeshi Nakatani	edcf4c6218	Merge pull request #555 from orozery/dont_sign_empty_headers ... don't sign empty headers (as they are discarded by libcurl)	2017-04-16 19:16:47 +09:00
Takeshi Nakatani	28efff5986	Merge pull request #554 from orozery/cache_cleanup ... cleanup cache directory when running out of disk space	2017-04-16 19:13:11 +09:00
Takeshi Nakatani	efba9bcbc1	Merge pull request #553 from orozery/custom_cipher_suite ... add TLS cipher suites customization	2017-04-16 19:09:27 +09:00
Takeshi Nakatani	6bd179c92b	Merge pull request #552 from orozery/foreground_threads ... switch S3fsMultiCurl to use foreground threads	2017-04-16 19:05:16 +09:00
Or Ozeri	96764b7410	switch S3fsMultiCurl to use foreground threads	2017-04-09 16:56:49 +03:00
Takeshi Nakatani	ff3eb1971f	Merge branch 'master' into fix_nocache_multipart_upload	2017-04-09 22:13:33 +09:00
Takeshi Nakatani	b4c90d6957	Fixed a bug about multipart uploading at no disk free space related to #509	2017-04-09 04:37:20 +00:00
Or Ozeri	75b59a7c16	switch S3fsMultiCurl to use foreground threads	2017-04-04 15:32:53 +03:00
Or Ozeri	3bcca75a88	don't sign empty headers (as they are discarded by libcurl)	2017-04-04 15:24:20 +03:00
Takeshi Nakatani	f0f61b3b55	Fixed a bug in logic about truncating stat cache	2017-04-02 11:51:58 +00:00
Takeshi Nakatani	fef3fbc225	Added check_cache_dir_exist option(refixed #347) - #538	2017-04-02 08:10:16 +00:00
Or Ozeri	8ee95ff7ab	fix multipart upload handling without cache	2017-04-02 10:27:43 +03:00
Or Ozeri	95578cad43	cleanup cache directory when running out of disk space	2017-04-02 10:22:12 +03:00
Takeshi Nakatani	465bbd3729	Updated stat_cache_expire option description - #545	2017-04-02 07:19:16 +00:00
Takeshi Nakatani	980ba398bc	Fixed double initialization of SSL library - #524	2017-03-19 17:11:18 +00:00
Takeshi Nakatani	523043a2aa	Changed base cached time of stat_cache_expire option - #523	2017-03-19 15:19:04 +00:00
Andrew Gaul	03217baa99	Address cppcheck 1.77 warnings	2017-03-06 12:41:08 -08:00
Takeshi Nakatani	6affefff5b	Merge pull request #509 from andrewgaul/mpu ... Use server-provided ETag during complete upload	2017-03-06 21:54:51 +09:00
Takeshi Nakatani	6fc972972f	Changed clock_gettime func to s3fs_clock_gettime for homebrew - #468	2016-12-04 10:31:41 +00:00
Takeshi Nakatani	7b307601b5	Merge pull request #511 from s3fs-fuse/issue#435 ... Fixed a bug about uploading NULL to some part of the file contents	2016-12-04 17:20:50 +09:00
Andrew Gaul	915a1321c7	Use server-provided ETag during complete upload ... This avoids calculating the MD5 locally and enables use with object stores which do not use MD5 as ETag.	2016-11-23 18:48:57 -08:00
Andrew Gaul	d375bca0d0	Correct typos	2016-11-19 15:57:41 -08:00
Takeshi Nakatani	edd0a11fb5	Merge pull request #494 from mapreri/typo ... Fix typo s/destroied/destroyed/	2016-11-20 05:53:23 +09:00
Takeshi Nakatani	67a836223a	Merge pull request #495 from driskell/fix_sse_copy ... Fix invalid V4 signature on multipart copy requests	2016-11-20 05:43:10 +09:00
Takeshi Nakatani	81e209bdd1	Fixed issue#435 codes	2016-11-19 20:09:35 +00:00
Jason Woods	6f688770fd	Fix invalid V4 signature on multipart copy requests	2016-11-13 13:22:00 +00:00
Mattia Rizzolo	8c0b1d9c5b	Fix typo s/destroied/destroyed/	2016-11-11 23:27:17 +00:00
Takeshi Nakatani	efde0ec9de	Merge pull request #489 from ggtakec/master ... Changed headers_t map using nocase compare function - #488	2016-10-23 23:23:31 +09:00
Takeshi Nakatani	632495374b	Chnaged headers_t map using nocase compare function - #488	2016-10-23 14:14:19 +00:00
Takeshi Nakatani	15b797f3ee	Merge pull request #488 from ggtakec/master ... Fixed searching Content-Length without case sensitive - #480	2016-10-23 22:42:52 +09:00
Takeshi Nakatani	a7a64d954a	Fixed searching Content-Length without case sensitive - #480	2016-10-23 13:27:01 +00:00
Takeshi Nakatani	cca217f613	Merge pull request #487 from driskell/debugging ... Split header debugging onto multiple lines for easier reading	2016-10-23 21:51:38 +09:00
Takeshi Nakatani	1a9cf6f66d	Fixed searching Content-Length without case sensitive - #480	2016-10-23 12:40:51 +00:00
Jason Woods	02d7296210	Split header debugging onto multiple lines for easier reading	2016-10-22 15:11:18 +01:00
Takeshi Nakatani	a688df813e	Fixed a bug at read symlink	2016-10-11 13:32:08 +00:00
Takeshi Nakatani	164424bc89	Merge branch 'master' into issue#435	2016-10-11 12:13:03 +00:00
Takeshi Nakatani	7fabd18b1f	Trim symbolic link original path in file.	2016-10-11 10:22:30 +00:00
Takeshi Nakatani	5db369d67e	Trim symbolic link original path in file.	2016-10-11 10:17:46 +00:00
Takeshi Nakatani	dba32fdf78	Trim symbolic link original path in file.	2016-10-11 10:06:21 +00:00
Takeshi Nakatani	716baada22	Testing patch codes for issue#435	2016-10-10 12:16:09 +00:00
Takeshi Nakatani	9fd1368611	OS-specific correspondence of the extended attribute header	2016-10-02 07:23:19 +00:00
Takeshi Nakatani	65d52506c4	Added use_xattr option for #467 and #460	2016-09-19 04:28:01 +00:00
Takeshi Nakatani	1a96f40a10	Fixed a bug about could not copy file mode from org file	2016-09-11 13:09:23 +00:00
Takeshi Nakatani	1ddc14d59d	Changed for accepting mount options compatible with mtab - #449	2016-07-24 08:17:58 +00:00
Peter Watkins	a19206cf0f	Accept mount arguments compatible with mtab ... Using "mount -a" fails for already-mounted s3fs directories, because s3fs mount arguments don't match the form in /etc/mtab. Calling "mount -a" should quietly succeed when a directory is already mounted. To fix this, accept mount commands of the form: s3fs s3fs /srv/object-store -o bucket=mybucket or in /etc/fstab form: s3fs /srv/object-store fuse.s3fs bucket=mybucket 0 0 This matches the form in /etc/mtab and allows "mount -a" to work properly.	2016-07-13 17:23:33 -04:00
Takeshi Nakatani	0f9428ad5a	Add mirror file logic for removing cache file	2016-07-13 17:23:32 -04:00
Tony Levi	ab4b92074c	Fix cppcheck complaint	2016-06-30 11:51:25 +09:30
Tony Levi	d57c12d3c3	Possible fix for leaks during entry expiry / truncation (#340)	2016-06-28 15:01:14 +09:30
Takeshi Nakatani	6005929a96	Handled all curl error without exiting process - #437	2016-06-27 10:38:49 +00:00
Takeshi Nakatani	49ffaa1d94	Merge pull request #432 from andrewgaul/create-bucket-location ... Include location constraint when creating bucket	2016-06-26 09:39:47 +09:00
Andrew Gaul	320b8e1171	Include location constraint when creating bucket ... This allows creating buckets in non-default regions. Also improve do_create_bucket error handling.	2016-06-13 10:35:37 -07:00
Andrew Gaul	95cb5d201f	Correct search and replace typo	2016-06-13 10:25:33 -07:00
Nathaniel W. Turner	584ea488bf	Use role name instead of profile name when iam_role=auto ... When using an instance with an IAM Role, transient credentials can be found in http://169.254.169.254/latest/meta-data/ at iam/security-credentials/role-name and s3fs tries to do this. However, it is using the profile-name where role-name is needed. In many cases the role and profile name are the same, but they are not always. The simplest way to find the role name appears to be to GET http://169.254.169.254/latest/meta-data/iam/security-credentials/ itself, which returns a listing of the role names for which temporary credentials exist. (I think there will probably only be one, but we probably want to split on newlines and take the first one here in case that assumption is not valid). This is the approach the AWS SDK appears to use (based on WireShark analysis). Bug: https://github.com/s3fs-fuse/s3fs-fuse/issues/421 Signed-off-by: Nathaniel W. Turner <nate@houseofnate.net>	2016-05-24 13:34:19 -04:00
Nathaniel W. Turner	594c9ca7d2	Skip early credential checks when iam_role=auto ... If user specifies iam_role=auto (or just iam_role), credentials will not be loaded during early phase, so skip credential checks there. Signed-off-by: Nathaniel W. Turner <nate@houseofnate.net>	2016-05-20 12:49:02 -04:00
Takeshi Nakatani	34b604cdfe	Fixed a bug about stat_cache_expire - #382	2016-05-14 09:03:52 +00:00
Takeshi Nakatani	50f1ad51c8	loading IAM role name automatically(iam_role option) - #387	2016-05-06 04:37:32 +00:00
Takeshi Nakatani	6cc30eea44	Allow duplicate key in ahbe_conf - #386	2016-05-06 01:08:39 +00:00
Takeshi Nakatani	1ddbd4d6bb	Fixed 'load_sse_c' option not working - #388	2016-05-06 00:36:54 +00:00
Takeshi Nakatani	845fdb43f2	Merge pull request #404 from rockuw/keep-alive ... Add curl handler pool to reuse connections	2016-04-26 23:40:45 +09:00
Tianlong Wu	b78adb4bb0	Add curl handler pool to reuse connections	2016-04-22 14:57:31 +08:00
Tianlong Wu	115bd51f3f	Fix a bug of truncating empty file	2016-04-22 14:49:37 +08:00
Takeshi Nakatani	10589a9497	Supported User-Agent header - #383	2016-04-17 07:44:03 +00:00
Takeshi Nakatani	090c37a1c1	Fixed writing sparsed file - #375,#379,#394	2016-04-12 18:24:36 +00:00
Takeshi Nakatani	fff40bbff3	Revert "Fixed a bug about writing sparsed file - #375"	2016-04-13 01:24:24 +09:00
Mattia Rizzolo	136ec654c2	fix typo in curl.cpp: s/returing/returning/	2016-04-02 15:19:06 +00:00
Takeshi Nakatani	ded4faf2e4	Fixed a bug about writing sparsed file - #375	2016-03-22 05:44:14 +00:00
Takeshi Nakatani	98d55582eb	Chnaged about constructor(destructor) in cache.h	2016-03-13 09:47:37 +00:00
Takeshi Nakatani	84bdd51021	Fixed a bug about etag comparison in stats cache.	2016-03-13 09:29:06 +00:00
Takeshi Nakatani	67efc11d94	Always set stats cache for opened file	2016-03-13 05:43:28 +00:00
Takeshi Nakatani	4c65c09f4d	Checked content-type by no case-sensitivity - #363	2016-02-13 05:58:59 +00:00
yurykats	230991782b	Update s3fs_util.cpp	2016-02-08 16:39:56 -05:00
Takeshi Nakatani	f81e6103cb	Fixed codes about clock_gettime for osx(3)	2016-02-07 08:27:02 +00:00
Takeshi Nakatani	0755c6f60c	Fixed codes about clock_gettime for osx(2)	2016-02-07 08:10:23 +00:00
Takeshi Nakatani	e01ded9e27	Fixed codes about clock_gettime for osx	2016-02-07 07:40:55 +00:00
Takeshi Nakatani	1af7aaeccb	Fixed addhead.cpp for cppcheck	2016-02-07 05:53:56 +00:00
Takeshi Nakatani	c7cf86c2ef	Sepalated AdditionalHeader class from curl.*	2016-02-07 05:41:56 +00:00
Takeshi Nakatani	6472eedddc	Supported regex type for additional header format.	2016-02-07 05:08:52 +00:00
Takeshi Nakatani	150b83f61e	Remove stat file cache dir if specified del_cache - #337	2016-02-06 18:59:13 +00:00
Takeshi Nakatani	c5a94cfc0c	Check cache dirctory path and attributes - #347	2016-02-06 13:38:48 +00:00
Takeshi Nakatani	203df6b58a	Merge pull request #346 from RobbKistler/empty-dir ... Fix empty directory check against AWS S3	2016-02-06 18:59:45 +09:00
Takeshi Nakatani	b90b51f2c5	Changed cache out logic for stat - #340	2016-02-06 09:09:17 +00:00
Takeshi Nakatani	8b457133da	Merge pull request #341 from hryang/master ... Fix the memory leak issue in fdcache.	2016-02-06 14:41:28 +09:00
Robb Kistler	4eff6b4dd1	Fix empty directory check against AWS S3 ... For ListBucketResult on an empty directory, AWS S3 and S3Proxy 1.4 differ. AWS will match the directory name, S3Proxy does not. Changing max-keys=1 to max-keys-2 works for both implementations. append_objects_from_xml() will swallow the directory key. The log level of this message is changed from ERROR to DBG. Fixes #345	2016-02-04 23:13:00 -08:00
haoran.yanghr	e3765ad497	Tune the code indent.	2016-01-28 11:16:06 +08:00
haoran.yanghr	dd9f3aed36	Fix the memory leak issue in fdcache. See issue #340	2016-01-28 11:11:53 +08:00
Takeshi Nakatani	83937700dd	Fixed a bug about IAMCRED type could not be retried.	2016-01-24 05:01:50 +00:00
Takeshi Nakatani	e932583309	Merge pull request #334 from andrewgaul/bucket-host ... Bucket host should include port and not path	2016-01-17 14:46:40 +09:00
Andrew Gaul	88a4f04217	Bucket host should include port and not path ... This resolves issues when using v4 signing with path-style requests.	2016-01-16 15:58:54 -08:00
Andrew Gaul	ff607e1a2d	Correct multiple issues with ListBucketRequest ... * provide correct path * sign query string * URL encode query string	2016-01-16 10:17:20 -08:00
Takeshi Nakatani	4bfbfa3621	Merge pull request #331 from andrewgaul/clang ... Address various clang warnings	2016-01-16 16:14:50 +09:00
Takeshi Nakatani	43b91d3235	Merge pull request #330 from andrewgaul/pass-by-reference ... Pass by const reference where possible	2016-01-16 16:14:31 +09:00
Andrew Gaul	e003732f18	Address various clang warnings ... Found with: -Wc++11-extensions -Wc++11-extra-semi -Wmissing-variable-declarations -Wundef	2016-01-11 00:52:24 -08:00
Andrew Gaul	b946b59522	Pass by const reference where possible	2016-01-10 16:58:24 -08:00
Andrew Gaul	ea6b287d1a	Fix v4 signature with use_path_request_style ... Previously s3fs omitted the bucket name when using path request style causing SignatureDoesNotMatch with v4 signatures.	2016-01-10 13:41:56 -08:00
Takeshi Nakatani	ea151a70c4	Merge pull request #321 from mcellis33/320 ... 320: delete stat cache entry in s3fs_fsync so st_size is refreshed	2015-12-20 15:05:28 +09:00
Mark Ellis	163daa5de1	320: delete stat cache entry in s3fs_fsync so st_size is refreshed	2015-12-18 15:39:25 -08:00
Robb Kistler	b581290c30	Cleanly exit fuse loop on error in s3fs_init ... This allows FUSE to clean the mount point up, preventing "Transport endpoint not connected" errors on subsequent access to the mount.	2015-12-15 15:25:56 -08:00
Takeshi Nakatani	2b3ece467b	Merge pull request #311 from RobbKistler/dbg-message ... Change error log to debug log in s3fs_read()	2015-12-08 00:03:48 +09:00
Robb Kistler	8e688816d4	Change error log to debug log in s3fs_read()	2015-12-03 21:25:27 -08:00
Takeshi Nakatani	759b44135a	Check pthread prtability in configure as additional change for #307	2015-12-03 07:47:17 +00:00
Takeshi Nakatani	8b53e0d931	Merge pull request #307 from rockuw/master ... Fix pthread portability problem	2015-12-03 16:35:30 +09:00
Tianlong Wu	3e655bad3b	PTHREAD_MUTEX_RECURSIVE_NP is a enum not macro	2015-12-03 13:44:11 +08:00
Takeshi Nakatani	5e97cb0f48	Changed ensure free disk space as additional change for #306	2015-12-03 05:40:26 +00:00
Tianlong Wu	f44b61c403	Fix pthread portability problem	2015-12-03 10:44:38 +08:00
Guy	6067af6ef1	Fix read concurrency to work in parallel count ... When the prefetch size is limited to the multipart size, the entire parallel logic of the read flow does not have an opportunity to use parallel get. This fix increases the read performance significantly over our own s3 on-premise solution.	2015-11-30 18:38:15 +02:00
Takeshi Nakatani	7b62de80f6	Fixed a bug about mtime - #299	2015-11-29 15:53:53 +00:00
Robb Kistler	fff2952d5f	Fix syslog level used by S3FS_PRN_EXIT() ... Without converting from s3fs log levels to syslog levels, the syslog ends up being LOG_EMERG which can cause a broadcast message to all users.	2015-11-25 13:53:08 -08:00
Takeshi Nakatani	c673d9d935	File opened with O_TRUNC is not flushed - changed #291	2015-11-08 04:55:17 +00:00
Takeshi Nakatani	0fdda61fb5	Merge pull request #293 from SnakeHunt2012/master ... Fix a small spelling issue.	2015-11-08 13:26:01 +09:00
Takeshi Nakatani	331b8456a0	Merge pull request #291 from RobbKistler/truncate ... Issue #290: File opened with O_TRUNC is not flushed	2015-11-08 13:23:06 +09:00
SnakeHunt2012	c04bcce206	Fix a small spelling issue.	2015-11-06 16:49:37 +08:00
Robb Kistler	dd7d9268f2	Force flush in s3fs_open() if file is truncated.	2015-11-03 22:06:25 -08:00
Robb Kistler	e4da5c59b6	Print source file in log messages	2015-11-03 08:34:02 -08:00
Takeshi Nakatani	001206f7c1	Fixed a bug about head request(copy) for SSE - issue#286	2015-11-01 14:05:47 +00:00
Takeshi Nakatani	2ef7f497f6	Fixed a bug about head request(copy) for SSE - issue#286	2015-11-01 13:54:47 +00:00
Takeshi Nakatani	8dd234dd8f	Fixed bugs about cppcheck error	2015-10-20 15:47:07 +00:00
Takeshi Nakatani	83d46ef8c6	Fixed bugs about a object larger than free disk space	2015-10-20 15:19:04 +00:00
Takeshi Nakatani	d102eb752d	Supported a object which is larger than free disk space	2015-10-18 17:03:41 +00:00
Takeshi Nakatani	f51ad1f33e	Supported for SSE KMS	2015-10-06 14:46:14 +00:00
Takeshi Nakatani	92e52dadd4	Changed and cleaned the logic for debug message.	2015-09-30 19:41:27 +00:00
Andrew Gaul	785ed642ba	Add support for standard_ia storage class ... This enables storage with lower at-rest prices, higher request prices, and lower availability. Also rework existing reduced redundancy parsing into a more generic storage class. More background on standard_ia: https://aws.amazon.com/blogs/aws/aws-storage-update-new-lower-cost-s3-storage-option-glacier-price-reduction/	2015-09-17 13:35:25 -07:00
Takeshi Nakatani	ead346c6d3	Merge pull request #261 from andrewgaul/help-timeouts ... Correct help timeouts	2015-09-13 15:49:15 +09:00
Andrew Gaul	0358908910	Correct help timeouts ... Follow-on to #167.	2015-09-10 11:45:05 -07:00
Andrew Gaul	32ce1a7267	Wrap help text at 80 characters	2015-09-10 11:43:09 -07:00
Takeshi Nakatani	97b8b34aab	Added chacking cache dir perms at starting(2).	2015-08-23 04:14:57 +00:00
Takeshi Nakatani	ce66430fac	Added chacking cache dir perms at starting.	2015-08-23 03:57:34 +00:00
Oliver Friedmann	d7d96907cf	This fixes an issue with caching when the creation of a subdirectory within the cache is aborted because a common cached parent directory already exists.	2015-08-21 19:30:04 -04:00
Takeshi Nakatani	751c868769	s3fs can print version with short commit hash - #228	2015-08-21 16:19:31 +00:00
Takeshi Nakatani	632578f328	Merge pull request #247 from andrewgaul/base64 ... Base64 cleanup	2015-08-22 00:45:26 +09:00
Andrew Gaul	bcfadbe1a8	Unlock during early return in TruncateCache ... Found via Coverity. Regression from dfa63345ed.	2015-08-19 13:54:14 -07:00
Andrew Gaul	b5c027f15d	Add unit tests for base64 encoding and decoding	2015-08-19 13:49:10 -07:00
Andrew Gaul	15db80b459	NUL terminate decoded base64 string ... For consistency with encoded strings.	2015-08-19 13:48:07 -07:00
Andrew Gaul	76c0ef86e4	Move base64 and hex functions to string_util	2015-08-19 13:47:26 -07:00
Takeshi Nakatani	a3e820e733	Merge pull request #245 from andrewgaul/map-duplicate-lookups ... Elide duplicate lookups of std::map via iterators	2015-08-20 01:22:06 +09:00
Takeshi Nakatani	4ad57bdea5	Merge pull request #240 from andrewgaul/md5 ... Enable Content-MD5 during multipart upload part	2015-08-20 01:19:01 +09:00
Takeshi Nakatani	085733d7c9	Merge pull request #239 from andrewgaul/google-code ... Update stale Google Code reference in --help	2015-08-20 01:08:00 +09:00
Takeshi Nakatani	fcb58aec3c	Merge pull request #238 from andrewgaul/cppcheck ... Enable all cppcheck rules	2015-08-20 01:06:50 +09:00
Takeshi Nakatani	026a9f2bdc	Merge pull request #235 from andrewgaul/complete-mpu-leak ... Plug leak during complete multipart upload	2015-08-20 00:40:00 +09:00
Takeshi Nakatani	fd04b9a437	Merge pull request #233 from andrewgaul/remove-inttostr ... Remove IntToStr	2015-08-20 00:34:21 +09:00
Takeshi Nakatani	ea99603b58	Merge pull request #232 from andrewgaul/stat-cache-locking ... Always hold stat_cache_lock when using stat_cache	2015-08-20 00:28:57 +09:00
Andrew Gaul	67d1576dfb	Elide duplicate lookups of std::map via iterators ... Also remove use of C++11 std::map::at.	2015-08-18 14:00:42 -07:00
Andrew Gaul	a157ac59ca	Enable Content-MD5 during multipart upload part ... This allows retries of multi-part uploads instead of discovering a fatal error during complete multipart upload. Also enable Content-MD5 for integration tests and refactor hexadecimal code.	2015-08-18 02:54:00 -07:00
Andrew Gaul	32520fd1fb	Update stale Google Code reference in --help	2015-08-16 23:30:41 -07:00
Andrew Gaul	c0b21d8808	Enable all cppcheck rules	2015-08-16 17:13:24 -07:00
Andrew Gaul	9c5bf0bb66	Plug leak during complete multipart upload	2015-08-15 22:38:24 -07:00
Andrew Gaul	dfa63345ed	Always hold stat_cache_lock when using stat_cache ... We could further improve this code by holding stat_cache_lock before calls to DelStat instead of unlocking then relocking it.	2015-08-14 20:14:12 -07:00
Andrew Gaul	3f59b8da01	Rewrite AutoLock ... Previously AutoLock::Lock allowed subsequent callers to proceed without the lock. Further is_locked was not always protected by auto_mutex. Finally AutoLock eagerly released auto_mutex when recursively unlocking. s3fs does not need recursive locks so we rewrite and simplify AutoLock. Partially surfaced by Coverity.	2015-08-14 20:00:56 -07:00
Andrew Gaul	0ea88a73c7	Remove IntToStr ... str duplicates this functionality. Also add unit test.	2015-08-12 08:25:09 -07:00
Takeshi Nakatani	756d1e5e81	Configure cppcheck #224	2015-08-12 15:04:16 +00:00
Takeshi Nakatani	2482aada43	Merge pull request #222 from andrewgaul/explicit ... Annotate constructors as explicit	2015-08-12 23:42:03 +09:00
Takeshi Nakatani	64146f69a4	Merge pull request #221 from andrewgaul/compare ... Compare idiomatically	2015-08-12 23:41:24 +09:00
Takeshi Nakatani	49e32967ec	Merge pull request #219 from andrewgaul/coverity ... Address Coverity errors	2015-08-12 23:40:47 +09:00
Takeshi Nakatani	09dff484e1	Merge pull request #215 from RobbKistler/memleak ... Fix mem leak in openssl_auth.cpp:s3fs_sha256hexsum	2015-08-12 23:38:51 +09:00
Andrew Gaul	ff8a0c2eea	Parse ETag from copy multipart correctly ... Previously s3fs misparsed this, preventing renames of files larger than 5 GB. Integration test disabled until S3Proxy 1.5.0 is released.	2015-08-11 14:43:35 -07:00
Robb Kistler	fcb55c2109	Fix mem leaks in openssl_auth.cpp, nss_auth.cpp ... Fix memory leaks in openssl_auth.cpp:s3fs_sha256hexsum and nss_auth.cpp:s3fs_sha256hexsum. Leaks occur every time a file is created.	2015-08-06 12:45:40 -07:00
Andrew Gaul	b6fa2deb9f	Annotate constructors as explicit ... This prevents implicit conversions.	2015-08-05 23:41:53 -07:00
Andrew Gaul	801ca0c2d3	Compare idiomatically	2015-08-05 23:35:08 -07:00
Andrew Gaul	8ee71caabb	Address Coverity errors ... Fixed an uninitialized member, misordered NULL check, resource leak, and unconsumed return value.	2015-08-05 23:28:06 -07:00
Jamie Alessio	912bc58df0	Fixed a few small spelling issues.	2015-07-10 11:50:40 -07:00
Nate Rosenblum	9b3c87ec97	Specialize {set,get}xattr for OS X ... These system calls take an extra 'position' parameter on OS X. A non-zero position value is only valid for resource forks (the Darwin VFS layer will reject anything else with EINVAL); this patch simply adds and ignores the parameter on Apple platforms. Allows building against OSXFUSE.	2015-06-25 12:56:15 -07:00
Takeshi Nakatani	966d229787	fixed fallback to sigv2 for bucket create and GCS	2015-06-20 04:34:32 +00:00
Takeshi Nakatani	4d49ace06b	Merge pull request #192 from andrewgaul/special-characters ... Simplify URL encoding	2015-06-20 11:47:22 +09:00
Takeshi Nakatani	ad8c64104e	Merge pull request #199 from s3fs-fuse/xattr ... Supported extended attributes(retry)	2015-06-20 11:46:47 +09:00
Takeshi Nakatani	fe3abed9f0	Chaged codes about iterator etc	2015-06-13 03:27:07 +00:00
Takeshi Nakatani	0ecf4aa6b4	Chaged codes about iterator	2015-06-13 03:08:56 +00:00
Andrew Gaul	4e03acf17a	Simplify URL encoding ... This also encodes asterisk and tilde correctly when listing a file with a V4 auth endpoint. Also add tests for special characters although s3proxy does not yet support V4 auth. Fixes #188. Fixes #194.	2015-06-10 13:15:58 -07:00
Takeshi Nakatani	84fb3d83d8	Fixed xattr for binary value	2015-06-06 16:39:39 +00:00
Bartlomiej Palmowski	3522e5eda3	Add no_check_certificate option which allows to ignore issues with self signed certs.	2015-05-20 17:32:36 +02:00
Takeshi Nakatani	3056644969	Merge pull request #185 from andrewgaul/typos ... Correct obvious typos in usage and README	2015-05-06 22:37:22 +09:00
Andrew Gaul	8a73d9fff0	Correct obvious typos in usage and README	2015-05-04 16:25:05 -07:00
Andrew Gaul	28ee9f27b9	Add usage information for multipart_size ... Also improve error message.	2015-05-04 16:21:58 -07:00
Dan Moore	5b11ac0f4c	Moved __APPLE__ #endif to correct position	2015-04-27 12:14:09 -07:00
Dan Moore	adb5a35097	configure.ac: detect target, if target is darwin (OSX), then ... change the minimum version of fuse required. Change the checkers to use a variable for the minimum fuse version instead of it being hardcoded in four different places. src/s3fs.cpp: Use __APPLE__ define around fuse code that is offensive to osxfuse. Not including the code doesn't seem to matter.	2015-04-25 17:13:20 -07:00
Takeshi Nakatani	aac92bd6c0	Fixed wrong owner checking and return codes	2015-04-21 16:18:05 +00:00
Takeshi Nakatani	f258a14070	Supported extended attributes, initial commit	2015-04-20 17:24:57 +00:00
Peter A. Bigot	92fcee824b	curl: use pathrequeststyle option when constructing Host endpoint ... Buckets with mixed-case names can't be accessed with the virtual-hosted style API due to DNS limitations. S3FS has an option for pathrequeststyle which is used for the URL, but it was not applied when building the endpoint passed through the Host header. Fix this, and relax the validation on bucket names when using this style. See: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro Signed-off-by: Peter A. Bigot <pab@pabigot.com>	2015-04-19 08:31:40 -05:00
Takeshi Nakatani	490ed8f689	Reviewed and fixed response codes print in curl.cpp - #157	2015-04-18 13:32:04 +00:00
Ka-Hing Cheung	70097709b2	switch to use region specific endpoints to compute correct v4 signature ... fix #133	2015-04-14 16:25:17 -07:00
Takeshi Nakatani	bd27294ab0	Increased default connecting/reading/writing timeout value	2015-04-12 02:04:13 +00:00
Ka-Hing Cheung	6e0a302f7d	refactor sigv4 to reduce code duplication	2015-04-09 15:11:59 -07:00
Ka-Hing Cheung	98af055d8b	send the correct Host header when using -o url ... fixes #161	2015-04-09 13:53:50 -07:00
Takeshi Nakatani	0f13c8fe97	Fixed a bug about ssl session sharing with libcurl older 7.23.0 - issue/#126	2015-03-21 07:04:20 +00:00
Takeshi Nakatani	2fc3a4e91e	Fixed a bug: unable to mount bucket subdirectory	2015-03-21 04:31:59 +00:00
Takeshi Nakatani	a04bec85b2	Fixed url-encoding for ampersand etc on sigv4 - Improvement/#149	2015-03-21 02:11:55 +00:00
Takeshi Nakatani	26453c4874	Fixed a bug not handling fsync.	2015-03-10 16:18:03 +00:00
Takeshi Nakatani	d06b6d7d41	Fixed a bug no use_cache case about fixed #138 - issue#141	2015-03-08 16:41:14 +00:00
Takeshi Nakatani	114966e7c0	Fixed bugs, not turn use_cache off and ty to load to end - issue#97	2015-03-04 08:48:37 +00:00
Takeshi Nakatani	2f8ad7ace8	Merge pull request #135 from andrewgaul/mpu-v4 ... Correct V4 signature for initiate multipart upload	2015-03-01 22:57:10 +09:00
Andrew Gaul	a07e804f57	Include Content-Type in complete MPU V2 signature ... Previously this failed with SignatureDoesNotMatch since the headers included it but the signature did not. Fixes #125.	2015-02-28 18:03:21 -08:00
Andrew Gaul	e9656810e3	Correct V4 signature for initiate multipart upload ... Query parameters need a trailing = for V4 signatures. Send correct content-sha256 although Amazon does not seem to enforce this for zero-length bodies. Finally remove a stale comment. Fixes #133.	2015-02-28 17:50:06 -08:00
Takeshi Nakatani	53083202ba	Merge pull request #132 from andrewgaul/s3proxy-integration-test ... Use S3Proxy to run integration tests	2015-02-27 00:17:46 +09:00
Andrew Gaul	e811ae1104	Use s3proxy to run integration tests ... References #129.	2015-02-24 12:08:22 -08:00
Timur Alperovich	be5735edb8	Fallback to v2 signatures correctly. ... Missing parameter to SetSignatureV4() call in the fallback code path results in not actually falling back.	2015-02-16 17:35:09 -08:00
Takeshi Nakatani	cf2b0cca22	Added new mp_umask option about issue#107, pr#110	2015-02-07 17:16:45 +00:00
Takeshi Nakatani	1424f87754	Supported signature version 4 for GnuTLS/NSS and automatically set endpoint/sigv2	2015-02-02 16:36:08 +00:00
Takeshi Nakatani	4f953f9bd7	Clean codes for signature v4 and added new sigv2 option	2015-01-28 17:13:11 +00:00
Takeshi Nakatani	0d2f3e2dc4	Fixed bugs, segfault and signature error at listing.	2015-01-24 16:36:30 +00:00
Takeshi Nakatani	bb1f1d3faa	Merged manually from caxapniy/s3fs-fuse/tree/1.77v4merge for signature v4 - #102	2015-01-20 16:31:36 +00:00
Takeshi Nakatani	98daf16681	Merge pull request #104 from kahing/rename_before_close ... fix rename before close	2015-01-14 00:40:41 +09:00
Takeshi Nakatani	939ba2b4b3	Merge pull request #101 from adobos/directory_empty_optimization ... Optimized function "bool directory_empty()"	2015-01-14 00:21:47 +09:00
Takeshi Nakatani	d0b82428d5	Merge pull request #100 from adobos/dns_ssl_switch_bugfix ... CURL handles not properly initialized to use DNS or SSL session caching.	2015-01-14 00:11:46 +09:00
Takeshi Nakatani	902911765e	Merge pull request #93 from andrewgaul/unit-test ... Add simple unit tests for trim functions	2015-01-14 00:07:01 +09:00
Ka-Hing Cheung	03d84a07d1	fix rename before close ... nautilus does this when you drag and drop to overwrite a file: 1) create .goutputstream-XXXXXX to write to 2) fsync the fd for .goutputstream-XXXXXX 3) rename .goutputstream-XXXXXX to target file 4) close the fd for .goutputstream-XXXXXX previously, doing this on s3fs would result in an empty target file because after the rename, s3fs would not flush the content of .goutputstream-XXXXXX to target file. this change moves the FdEntity from the old path to the new path whenever rename happens. On flush s3fs would now flush the correct content to the rename target.	2015-01-12 15:05:54 -08:00
Andrej Dobos	045f1e7906	CURL handles were not properly initialized to use DNS caching, or SSL session caching.	2014-12-23 22:31:54 -08:00
Andrej Dobos	69ef7fbefb	Optimized function directory_empty: check for at most one entry when evaluating whether a directory is empty or not (as opposed to doing full directory listing)	2014-12-23 22:29:13 -08:00
Andrew Gaul	a56b8db410	Add simple unit tests for trim functions ... Subsequent commits will use this infrastructure. Also reparent prepare_url which relies on unrelated bucket, foreground2, and pathrequeststyle symbols.	2014-12-06 18:07:14 -08:00
Takeshi Nakatani	082eb24c12	Merge pull request #83 from tmwong2003/develop ... Changed option processing to use strtol() to get a umask	2014-11-16 23:49:24 +09:00
Theodore Wong	f04b659f5e	Changed option processing to use strtol() to get a umask ... get_mode()/s3fs_strtoofft() does not handle octal umask values, which results in unexpected behavior when trying to set a world-readable umask value.	2014-11-12 23:29:41 +00:00
bupt_tengteng	b31ec5c4af	Update curl.cpp ... The space causes signature mismatch when using "ahbe_conf" file to add additional headers.When s3 use the" x-amaz" header to calculates the signature, the format is as follow: PUT application/octet-stream Wed, 05 Nov 2014 03:05:08 GMT x-amz-acl:private x-amz-meta-gid:0 x-amz-meta-mode:33188 x-amz-meta-mtime:1415156708 x-amz-meta-uid:0 There is no space after colon.	2014-11-05 11:28:33 +08:00
Takeshi Nakatani	651e8c3158	Merge pull request #64 from andrewgaul/failed-read-eio ... Return EIO on failed read	2014-11-03 01:03:32 +09:00
Vincent Bernat	7212072ff0	url: handle scheme omission ... When the scheme is omitted in URL overriding (for example `example.com` instead of `https://example.com`), s3fs is modifying the URL by inserting `s3.` in the middle of the name (`examples3..com`). This can be a bit difficult to troubleshoot and curl seems to handle schema-less requests just fine. So, just handle this case correctly.	2014-10-23 10:25:17 +02:00
Takeshi Nakatani	28d82c9ccd	Fixed #68(FreeBSD issue)	2014-10-22 14:21:01 +00:00
Andrew Gaul	50137fe026	Address clang always true warnings	2014-10-16 23:34:12 -07:00
Takeshi Nakatani	9237d07226	Merge pull request #63 from jollyroger/spelling ... Fix spelling errors	2014-10-13 11:38:13 +09:00
Takeshi Nakatani	8c2be4aa85	Merge pull request #62 from jollyroger/fix-stray-chars ... Remove stray chars from source files	2014-10-13 11:34:52 +09:00
Andrew Gaul	a1ca8b7124	Return EIO on failed read ... Previously S3fsMultiCurl::MultiRead did not report read errors since it did not treat failed callback setup as a fatal operation error. Failed callback setups usually result from exceeding the number of allowed retries. Previously cp did not report an error during a network outage but now does: $ cp ~/s3-path/s3-file . cp: error reading ‘/home/gaul/s3-path/s3-file’: Input/output error cp: failed to extend ‘./s3-file’: Input/output error	2014-10-03 21:30:11 -07:00
Andriy Senkovych	6633366218	Fix spelling errors	2014-10-01 13:42:39 +03:00
Andriy Senkovych	22ea65f02c	Remove stray chars from source files	2014-10-01 13:20:29 +03:00
Andrew Gaul	3d69ee0c30	Emit response on failed CheckBucket requests ... This allows callers to diagnose errors like InvalidAccessKeyId and RequestTimeTooSkewed.	2014-09-28 16:12:53 -07:00
Andrew Gaul	c88a5f38be	Disable CURLOPT_FAILONERROR for CheckBucket ... curl will not consume the body of a response when CURLOPT_FAILONERROR is set. This prevents logging of responses for failed requests.	2014-09-28 16:12:43 -07:00
Takeshi Nakatani	7a7c7572ea	Cleaned up codes for next packaging.	2014-09-07 15:08:27 +00:00
Takeshi Nakatani	ae4ae88b6d	Merge pull request #53 from s3fs-fuse/sse-c ... Support for SSE-C #39	2014-09-07 22:31:33 +09:00
Takeshi Nakatani	f0c33f8ef2	clean codes	2014-08-27 00:59:49 +00:00
Takeshi Nakatani	20b1c207be	fixed issue #39	2014-08-26 17:11:10 +00:00
YAMASAKI Masahide	f1ca5d0340	:set nobomb	2014-08-25 19:18:34 +09:00
Takeshi Nakatani	cbec8da9a3	fixed a bug issue #49	2014-08-14 15:58:06 +00:00
Takeshi Nakatani	7a55eab399	Support for SSE-C, issue #39	2014-07-19 19:02:55 +00:00
Takeshi Nakatani	c1a6d76fc3	Fixed a bug issue #40	2014-06-28 17:36:35 +00:00
Takeshi Nakatani	ba34ba181a	Fixed a bug issue #41	2014-06-28 17:24:25 +00:00
Andrew Dunn	601482eff5	Added support for path API request style. ... Rather than using virtual host style requests, path style requests can be used instead. i.e. rather than bucketname.s3.amazon.com/... the s3fs will be able to request from s3.amazon.com/bucketname/... This is useful for S3 compatible APIs which don't support the virtual host style request. It is enabled with the new option, `use_path_style_request`. Example: /usr/bin/s3fs data ~/netcdf -o url="https://swift.rc.nectar.org.au:8888/" -o use_path_request_style -o allow_other -o uid=500 -o gid=500	2014-06-04 00:03:49 +10:00
Takeshi Nakatani	61020370d5	Changed codes for CR code in passwd file(googlecode issue#417).	2014-06-02 16:12:55 +00:00
Takeshi Nakatani	160196798b	Changed initializing logic for nss lib/openssl lib/s3fs own.	2014-06-01 03:54:02 +00:00
Takeshi Nakatani	cd27f0aa54	Supported another crypt libraries as GnuTLS and NSS, and added configure options	2014-05-06 14:23:05 +00:00
Takeshi Nakatani	8bba566774	Retry to send request at CURLE_SSL_CONNECT_ERROR	2014-04-04 16:23:56 +00:00
Takeshi Nakatani	157612e7e7	Merge pull request #28 from andrewgaul/fdcache-init-signedness-warning ... Address signedness warning in FdCache::Init	2014-04-05 01:02:13 +09:00
Andrew Gaul	d475e22774	Address signedness warning in FdCache::Init ... This commit allows GCC 4.8 to compile s3fs withing warnings.	2014-04-01 23:50:08 -07:00
Takeshi Nakatani	4762e53b5d	Added multipart_size option for #16	2014-03-30 07:53:41 +00:00
Takeshi Nakatani	d7563309a2	Fixed a bug #18(losing check retry error)	2014-03-30 06:40:49 +00:00
Mason Malone	74fb29d9fb	Fix typos in docs for max_stat_cache_size (the default is actually 1000)	2014-03-06 12:44:10 -05:00
Takeshi Nakatani	16487c4e26	Fixed a bug: issue #14, s3fs -u should return 0 if there are no lost multiparts	2014-03-03 17:00:20 +00:00
Takeshi Nakatani	52d56d15e4	Fixed a bug(googlecode issue 405), enable_content_md5 Input/output error	2014-03-03 16:19:08 +00:00
Takeshi Nakatani	5fd33405af	Fixed a bug - issue#5	2014-01-16 16:04:41 +00:00
Pascal Deschênes	db3bd7c366	Fix compilation error on MacOSX with missing const	2014-01-04 21:24:27 -05:00
worpet	33ec3739e2	Update s3fs_util.cpp ... Fixed local time was incorrectly being applied to IAM and Last-Modified dates.	2013-12-30 09:35:39 -06:00
root	d5626fe595	Changed url encoding of space character to use %20 instead of the plus (+) sign.	2013-12-17 15:12:03 +01:00
ggtakec@gmail.com	74db6748dd	Changes codes ... 1) Changed condition for retrying multipart error Changes condition that 404 is not retrying, other case is retrying, when multipart request failed. 2) file type wrong fdcache.h file type is wrong, so that fixed it. git-svn-id: http://s3fs.googlecode.com/svn/trunk@498 df820570-a93a-0410-bd06-b72b767a4274	2013-11-19 01:48:53 +00:00
ggtakec@gmail.com	8acbaf7199	Fixed a bug. ... * Fixed a bug Fixes a bug that is dead loop when s3fs listed in a directory with directory objects which was not object(no information). This bug made by r493, and reported by issue 389. git-svn-id: http://s3fs.googlecode.com/svn/trunk@497 df820570-a93a-0410-bd06-b72b767a4274	2013-11-18 02:29:41 +00:00