octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-12-27 02:22:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,174 Commits 1 Branch 23 Tags 13 MiB
eeb839242b
Commit Graph

611 Commits

Author SHA1 Message Date
Takeshi Nakatani
2887f8916b Automatic region change made possible other than us-east-1(default) 2019-01-20 10:51:49 +00:00
Takeshi Nakatani
ac72431195 Added detail error message when HTTP 301/307 status 2019-01-20 10:07:58 +00:00
Takeshi Nakatani
f2184e34dd Ignore after period character of floating point in x-amz-meta-mtime 2019-01-20 08:28:06 +00:00
Takeshi Nakatani
36a4903843 Added an error message when HTTP 301 status 2019-01-20 07:17:40 +00:00
Takeshi Nakatani
c83a3e67c9
Merge pull request #885 from LutzFinsterle2019/master 
Update s3fs_util.cpp for correspondence of Nextcloud contype
 2019-01-20 15:32:58 +09:00
Takeshi Nakatani
05014c49c8
Merge pull request #906 from gaul/doc/https 
Prefer HTTPS links where possible
 2019-01-20 15:10:51 +09:00
Takeshi Nakatani
aa69107165
Merge pull request #905 from gaul/clang-tidy/redundant 
Fix comparison in s3fs_strtoofft
 2019-01-20 15:01:40 +09:00
Takeshi Nakatani
d373b0eca3
Merge pull request #904 from gaul/clang-tidy/c-str 
Remove unnecessary calls to std::string::c_str
 2019-01-20 14:13:46 +09:00
Takeshi Nakatani
6aa40b2747
Merge pull request #903 from gaul/clang-tidy/find_char 
Prefer find(char) over find(const char *)
 2019-01-20 14:05:06 +09:00
Takeshi Nakatani
34c3bfe408
Merge pull request #902 from gaul/clang-tidy/pass-by-value 
Avoid pass-by-value when not necessary
 2019-01-20 13:45:44 +09:00
Takeshi Nakatani
6ac56e722d
Merge pull request #901 from gaul/clang-tidy/leaks 
Plug memory leaks
 2019-01-20 12:41:08 +09:00
Takeshi Nakatani
61dc7f0a70
Merge pull request #900 from gaul/leak 
Plug memory leak
 2019-01-20 12:30:31 +09:00
Takeshi Nakatani
9f000957dd
Merge pull request #899 from gaul/response-code 
Tighten up HTTP response code check
 2019-01-20 11:16:23 +09:00
Andrew Gaul
aa9bd1fa3c Prefer HTTPS links where possible 
Fix a few stale links as well.
 2019-01-18 11:09:08 -08:00
Andrew Gaul
5a2dc03a1c Fix comparison in s3fs_strtoofft 
Also backfill unit tests.  Document limitations.  Found via
clang-tidy.
 2019-01-17 22:59:25 -08:00
Andrew Gaul
508fafbe62 Remove unnecessary calls to std::string::c_str 
Found via clang-tidy.
 2019-01-17 22:05:16 -08:00
Andrew Gaul
e29548178b Prefer find(char) over find(const char *) 
The former can be faster.  Found via clang-tidy.
 2019-01-17 20:24:24 -08:00
Andrew Gaul
ab2f36f202 Plug memory leaks 
Found via clang-tidy.
 2019-01-17 18:54:34 -08:00
Andrew Gaul
b8c9fcfd70 Avoid pass-by-value when not necessary 
This requires unnecessary memcpy.  Found via clang-tidy.
 2019-01-17 18:22:11 -08:00
Andrew Gaul
58ce544e83 Plug memory leak 
Previously this appended to an empty list, zeroed the list, then
appended to the list.  Instead zero the list first and then append.
This also enables sending Content-MD5 which can eagerly detect
transmission errors.  Found via Valgrind.
 2019-01-17 16:24:34 -08:00
Andrew Gaul
e98ce36301 Tighten up HTTP response code check 
Previously s3fs considered 3xx codes as successful.  When writing an
object to a misconfigured region, s3fs did not propagate the 301 as a
write error to the client.  References #693.
 2019-01-17 15:04:59 -08:00
Andrew Gaul
6401b4ae92 Flush file before renaming 
Previously s3fs could copy the zero-byte stub object without including
any pending writes.  Fixes #145.
 2019-01-17 12:05:10 -08:00
Takeshi Nakatani
25b49e1a2e
Merge pull request #894 from gaul/default-mode 
Default uid/gid/mode when object lacks permissions
 2019-01-16 18:48:54 +09:00
Takeshi Nakatani
c7def35b54
Merge pull request #895 from gaul/bucket-name-dot 
Emit more friendly error for buckets with dots
 2019-01-16 18:39:47 +09:00
Andrew Gaul
c512516e14 Emit more friendly error for buckets with dots 
These fail SSL certificate checks due to the *.s3.amazon.com wildcard.
Fixes #284.
 2019-01-14 18:47:36 -08:00
Andrew Gaul
2c43b1e12b Store and retrieve file change time 
This introduces a new header with the change time; existing objects
will report modification time.  Fixes #771.
 2019-01-14 10:05:11 -08:00
Andrew Gaul
e2d5641d99 Default uid/gid/mode when object lacks permissions 
This addresses a common use case when interacting with objects from
both s3fs and other S3 tools.  Fixes #890.
 2019-01-13 21:57:23 -08:00
LutzFinsterle2019
c985b5e4d0
Corrected Comment to C++ style 2019-01-12 10:19:48 +01:00
LutzFinsterle2019
18cb2e2662
Update s3fs_util.cpp 
Sorry for answering late, have been busy lately.
The comment is: "Nextcloud stores Directory objects with this mime type when mounting a Bucket as external Storage"
 2019-01-10 07:44:36 +01:00
LutzFinsterle2019
743c706b0a
Update s3fs_util.cpp 
Nextcloud Compatibility in directory mime-types
 2019-01-07 07:41:27 +01:00
Takeshi Nakatani
4ed0e5f35a
Merge pull request #882 from earlchew/issue-817 
[curl] Assume long encryption keys are base64 encoded
 2019-01-06 17:29:04 +09:00
Takeshi Nakatani
fd6b37d3da
Merge pull request #877 from gaul/aws/credentials 
Check arguments and environment before .aws/creds
 2019-01-06 17:03:11 +09:00
Takeshi Nakatani
56e24de0d4
Merge pull request #870 from gaul/typos 
Correct typos in command-line parsing
 2019-01-06 16:21:46 +09:00
Earl Chew
ed5795eead [curl] Assume long encryption keys are base64 encoded 
Correct tabs and whitespace.

Signed-off-by: Earl Chew <earl_chew@yahoo.com>
 2019-01-05 21:08:41 +00:00
Andrew Gaul
3d225163f8 Correct typos in command-line parsing 
Also fix stray macOS references.  References #869.
 2019-01-02 14:40:57 -08:00
Andrew Gaul
0569cec3ea Check arguments and environment before .aws/creds 
Fixes #857.  Fixes #864.
 2018-12-20 15:51:32 -08:00
Andrew Gaul
a2f8ac535e Address cppcheck 1.86 errors 
Lifetime, shadowing, and unused variables.  Found via the Travis macOS
builder.
 2018-12-20 14:56:31 -08:00
Takeshi Nakatani
d9e89deef6
Merge pull request #865 from orozery/multihead_warning_check 
fix multihead warning check
 2018-11-29 20:53:47 +09:00
Or Ozeri
b91fc5409e fix multihead warning check 2018-11-28 09:54:02 +02:00
Michael Chen
3c970646d1
Add 'profile' option to command line help. 2018-11-19 08:26:23 -08:00
Andrew Gaul
88cd8feb05 Upload S3 parts without batching 
Previously s3fs would issue a batch of uploads and wait for all to
succeed before issuing the next batch.  Now it issues the first batch
and only waits for a single part to succeed before uploading the next
part.  This can improve performance when one part lags due to network
errors.  Fixes #183.
 2018-11-16 18:32:38 -08:00
Takeshi Nakatani
5fba542a29
Merge pull request #852 from gaul/aws-credentials-file 
Allow credentials from ${HOME}/.aws/credentials
 2018-11-11 11:37:38 +09:00
Andrew Gaul
9e530c86ae Allow credentials from ${HOME}/.aws/credentials 
This matches the configuration from popular tools like AWS CLI and
allows multiple profile names via -o profile=name.  The existing
credential mechanisms continue to work.  Fixes #822.
 2018-11-04 17:47:07 -08:00
Andrew Gaul
664f910083 Correctly compare list_object_max_keys 
Previously this did not allow an argument.  Fixes #843.
References #793.
 2018-11-04 10:49:35 -08:00
Takeshi Nakatani
6fe92d5ed6
Merge pull request #832 from gaul/hex 
Simplify hex conversion
 2018-10-08 23:03:42 +09:00
Andrew Gaul
c703fa15c0 Simplify hex conversion 
Addresses GCC 8 warning:

common_auth.cpp: In function ‘std::__cxx11::string s3fs_sha256sum(int, off_t, ssize_t)’:
common_auth.cpp:84:12: warning: ‘char* strncat(char*, const char*, size_t)’ output may be truncated copying 2 bytes from a string of length 2 [-Wstringop-truncation]
    strncat(sha256, hexbuf, 2);
 2018-10-01 19:08:27 -07:00
Jonatan Nilsson
d9c106cfde Add support for storage class ONEZONE_IA. 2018-09-30 14:27:31 +02:00
Takeshi Nakatani
c5af62b023
Merge pull request #820 from gaul/big-writes 
Enable big writes if capable
 2018-09-17 17:24:06 +09:00
Earl Chew
41c23adb0e [curl] Assume long encryption keys are base64 encoded 
Amazon SSE-C https://tinyurl.com/ychug4cg writes:

> Use this header to provide the 256-bit, base64-encoded encryption key
> for Amazon S3 to use to encrypt or decrypt your data.

It seems likely that future keys would be as long, or longer, so
this change assumes that text longer than 32 characters are keys
that are base64 encoded.

Signed-off-by: Earl Chew <earl_chew@yahoo.com>
 2018-09-16 21:23:22 +00:00
Andrew Gaul
a85183d42c Enable big writes if capable 
Fixes #813.
 2018-09-10 22:21:51 -07:00
Mattia Rizzolo
c376efdd28
Fix typo s/mutliple/multiple/ 
Signed-off-by: Mattia Rizzolo <mattia@mapreri.org>
 2018-08-19 00:14:57 +02:00
Takeshi Nakatani
7e0c53dfe9 Added list_object_max_keys option based on #783 PR 2018-07-08 03:49:10 +00:00
root
35d3fce7a0 Review comment: Include the error code being returned 2018-07-06 05:14:32 -04:00
root
4177d8bd3b Review comment: Include the error code being returned 2018-07-06 03:03:57 -04:00
root
ad5349a488 Changes as per review comments 2018-07-05 05:02:04 -04:00
root
92a4034c5e Log messages for 5xx and 4xx HTTP response code 2018-07-04 03:50:45 -04:00
Takeshi Nakatani
3e4002df0d
Merge pull request #780 from wjt/initialize-libgcry 
gnutls_auth: initialize libgcrypt
 2018-06-24 12:48:08 +09:00
Takeshi Nakatani
1b9ec7f4fc
Merge pull request #774 from nkkashyap/master 
Option for IAM authentication endpoint
 2018-06-24 12:36:23 +09:00
Takeshi Nakatani
0d3fb0658a Fixed a error by cppcheck on OSX 2018-06-24 02:38:59 +00:00
Will Thompson
73cf2ba95d
gnutls_auth: initialize libgcrypt 
Without this change, the following warning appears in the syslog/journal
during startup:

  Libgcrypt warning: missing initialization - please fix the application

From the [documentation][0]:

> The function `gcry_check_version` initializes some subsystems used by
> Libgcrypt and must be invoked before any other function in the
> library.

Fixes #524, which says:

> gnutls is initialized by gnutls_global_init() function and
> gcry_check_version() function for initializing libgcry is called from
> this gnutls_global_init().

I checked the gnutls source and it hasn't contained a call to
gcry_check_version() since the libgcrypt backend was removed in 2011
(commit 8116cdc8f131edd586dad3128ae35dd744cfc32f). In any case, the
gcry_check_version() documentation continues:

> It is important that these initialization steps are not done by a
> library but by the actual application.

so it would be incorrect for a library used by s3fs to initialize
libgcrypt.

[0]: https://www.gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
 2018-06-21 20:55:00 +01:00
Neeraj Kumar Kashyap
5a481e6a01 Option for IBM IAM auth endpoint added return 2018-06-04 16:44:14 +05:30
Neeraj Kumar Kashyap
d8e12839af Option for IBM IAM auth endpoint 2018-05-31 16:02:48 +05:30
Or Ozeri
d4e86a17d1 Revert "enable FUSE read_sync by default" 
This reverts commit 86b0921ac4.

Conflicts:
	src/s3fs.cpp
 2018-05-28 13:49:54 +03:00
Takeshi Nakatani
ae9d8eb734 Fixed memory leak 2018-05-27 10:48:03 +00:00
Takeshi Nakatani
57b2a60172
Merge pull request #764 from orozery/remove_false_multihead_warnings 
Remove false multihead warnings
 2018-05-23 22:38:35 +09:00
Takeshi Nakatani
212bbbbdf0
Merge pull request #763 from orozery/cleanup_share_after_handles 
cleanup curl handles before curl share
 2018-05-23 22:30:36 +09:00
Takeshi Nakatani
da95afba8a
Merge pull request #756 from orozery/optimize_defaults 
Optimize defaults
 2018-05-23 22:05:00 +09:00
Or Ozeri
0bd875eb9e remove false readdir_multi_head warnings 2018-05-22 17:10:50 +03:00
Or Ozeri
af63a42773 cleanup curl handles before curl share 2018-05-21 13:20:09 +03:00
Or Ozeri
86b0921ac4 enable FUSE read_sync by default 2018-05-06 16:10:36 +03:00
Or Ozeri
4a72b60707 increase default stat cache size from 1000 to 100000 2018-05-06 15:31:07 +03:00
Takeshi Nakatani
e3de6ea458 Added reset curl handle when returning to handle pool 2018-05-06 12:11:53 +00:00
Neeraj Kumar Kashyap
25375a6b48 Validate the URL fixed inefficient usage of find 2018-05-04 11:24:32 +05:30
Neeraj Kumar Kashyap
ca87df7d44 Validate the URL format for http/https 2018-05-03 22:08:28 +05:30
Takeshi Nakatani
d052dc0b9d
Merge pull request #753 from cfz/master 
fix xpath selector in bucket listing
 2018-05-02 12:04:12 +09:00
陈方舟
04493de767
fix xpath selector in bucket listing 
the original implementation in get_base_exp() depends on the order of xml return from the server.
patriotically, when listing a directory with sub directory(s), the xml document response contains more than 2 <Prefix> nodes(some of them are in <CommonPrefixes> node).
the source code arbitrarily select the first one in the documents (nodes->nodeTab[0]->xmlChildrenNode).
some s3 compatible service return the list-bucket result in different result, leading the s3fs to a wrong behavior
 2018-04-23 15:11:29 +08:00
Or Ozeri
4fdab46617 don't fail mkdir when directory exists 2018-04-08 11:13:47 +03:00
Takeshi Nakatani
1a23b880d5
Merge pull request #739 from orozery/cleanup_failing_curl_handles 
cleanup curl handle state on retries
 2018-04-01 22:45:04 +09:00
Or Ozeri
adcf5754ae cleanup failing curl handles on retries 2018-03-29 13:56:08 +03:00
Tristen Horton
0863672e27
add a more helpful error message for when there are multiple entries for the same bucket in the passwd file 2018-03-13 14:37:34 -07:00
Takeshi Nakatani
0f503ced25
Merge pull request #729 from dmgk/master 
FreeBSD build fixes
 2018-03-04 16:36:31 +09:00
Takeshi Nakatani
987a166bf4
Merge pull request #726 from orozery/instance_name_logging 
add an instance_name option for logging
 2018-03-04 15:41:12 +09:00
Takeshi Nakatani
57b6f0eeaf
Merge pull request #724 from orozery/dont_fail_multirequest 
don't fail multirequest on single thread error
 2018-03-04 15:35:29 +09:00
Takeshi Nakatani
f71a28f9b9
Merge pull request #714 from orozery/reduce_lock_contention 
reduce lock contention on file open
 2018-03-04 13:36:08 +09:00
Dmitri Goutnik
c9f4312588 FreeBSD build fixes 2018-03-02 15:58:52 -05:00
Or Ozeri
8b657eee41 add disk space reservation 2018-02-28 19:20:23 +02:00
Takeshi Nakatani
be9d407fa0 Fixed cppcheck error on osx 2018-02-28 12:29:58 +00:00
Takeshi Nakatani
c494e54320 Fixed cppcheck error on osx 2018-02-28 12:06:06 +00:00
Or Ozeri
b52b6f3fc5 add an instance_name option for logging 2018-02-28 09:51:35 +02:00
Or Ozeri
82c9733101 don't fail multirequest on single thread error 2018-02-26 12:06:08 +02:00
Takeshi Nakatani
a45ff6cdaa Fixed cppcheck error and clean ^M code 2018-02-25 13:08:41 +00:00
Takeshi Nakatani
960d45c853 Fixed cppcheck error on osx 2018-02-25 08:51:19 +00:00
Or Ozeri
0edf056e95 reduce lock contention on file open 2018-02-04 17:13:58 +02:00
Takeshi Nakatani
6b58220009
Merge pull request #697 from pwulff/master 
Fixing race condition in FdEntity::GetStats
 2017-12-17 15:46:48 +09:00
Paul Wulff
ee6abea956 Race condition in FdManager::Rename because no mutex is used. 2017-12-15 15:27:51 +01:00
Paul Wulff
cea7d44717 Fixing race condition in FdEntity::GetStats 2017-12-13 10:49:00 +01:00
Or Ozeri
0da87e75fe fix condition for parallel download 2017-12-04 16:07:33 +02:00
Or Ozeri
bd4bc0e7f1 add support for IBM IAM authentication 2017-11-23 12:01:52 +02:00
Takeshi Nakatani
026260e7a1 Improved use of temporary files 2017-11-23 09:18:11 +00:00
Takeshi Nakatani
99fe93b7f1
Merge pull request #684 from gaul/signedness 
Correct signedness warning
 2017-11-23 17:33:46 +09:00