868 Commits

Author SHA1 Message Date
Andrew Gaul
664f910083 Correctly compare list_object_max_keys
Previously this did not allow an argument.  Fixes #843.
References #793.
2018-11-04 10:49:35 -08:00
Takeshi Nakatani
6fe92d5ed6
Merge pull request #832 from gaul/hex
Simplify hex conversion
2018-10-08 23:03:42 +09:00
Andrew Gaul
c703fa15c0 Simplify hex conversion
Addresses GCC 8 warning:

common_auth.cpp: In function ‘std::__cxx11::string s3fs_sha256sum(int, off_t, ssize_t)’:
common_auth.cpp:84:12: warning: ‘char* strncat(char*, const char*, size_t)’ output may be truncated copying 2 bytes from a string of length 2 [-Wstringop-truncation]
    strncat(sha256, hexbuf, 2);
2018-10-01 19:08:27 -07:00
Jonatan Nilsson
d9c106cfde Add support for storage class ONEZONE_IA. 2018-09-30 14:27:31 +02:00
Takeshi Nakatani
c5af62b023
Merge pull request #820 from gaul/big-writes
Enable big writes if capable
2018-09-17 17:24:06 +09:00
Earl Chew
41c23adb0e [curl] Assume long encryption keys are base64 encoded
Amazon SSE-C https://tinyurl.com/ychug4cg writes:

> Use this header to provide the 256-bit, base64-encoded encryption key
> for Amazon S3 to use to encrypt or decrypt your data.

It seems likely that future keys would be as long, or longer, so
this change assumes that text longer than 32 characters are keys
that are base64 encoded.

Signed-off-by: Earl Chew <earl_chew@yahoo.com>
2018-09-16 21:23:22 +00:00
Andrew Gaul
a85183d42c Enable big writes if capable
Fixes #813.
2018-09-10 22:21:51 -07:00
Mattia Rizzolo
c376efdd28
Fix typo s/mutliple/multiple/
Signed-off-by: Mattia Rizzolo <mattia@mapreri.org>
2018-08-19 00:14:57 +02:00
Takeshi Nakatani
7e0c53dfe9 Added list_object_max_keys option based on #783 PR 2018-07-08 03:49:10 +00:00
root
35d3fce7a0 Review comment: Include the error code being returned 2018-07-06 05:14:32 -04:00
root
4177d8bd3b Review comment: Include the error code being returned 2018-07-06 03:03:57 -04:00
root
ad5349a488 Changes as per review comments 2018-07-05 05:02:04 -04:00
root
92a4034c5e Log messages for 5xx and 4xx HTTP response code 2018-07-04 03:50:45 -04:00
Takeshi Nakatani
3e4002df0d
Merge pull request #780 from wjt/initialize-libgcry
gnutls_auth: initialize libgcrypt
2018-06-24 12:48:08 +09:00
Takeshi Nakatani
1b9ec7f4fc
Merge pull request #774 from nkkashyap/master
Option for IAM authentication endpoint
2018-06-24 12:36:23 +09:00
Takeshi Nakatani
0d3fb0658a Fixed a error by cppcheck on OSX 2018-06-24 02:38:59 +00:00
Will Thompson
73cf2ba95d
gnutls_auth: initialize libgcrypt
Without this change, the following warning appears in the syslog/journal
during startup:

  Libgcrypt warning: missing initialization - please fix the application

From the [documentation][0]:

> The function `gcry_check_version` initializes some subsystems used by
> Libgcrypt and must be invoked before any other function in the
> library.

Fixes #524, which says:

> gnutls is initialized by gnutls_global_init() function and
> gcry_check_version() function for initializing libgcry is called from
> this gnutls_global_init().

I checked the gnutls source and it hasn't contained a call to
gcry_check_version() since the libgcrypt backend was removed in 2011
(commit 8116cdc8f131edd586dad3128ae35dd744cfc32f). In any case, the
gcry_check_version() documentation continues:

> It is important that these initialization steps are not done by a
> library but by the actual application.

so it would be incorrect for a library used by s3fs to initialize
libgcrypt.

[0]: https://www.gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
2018-06-21 20:55:00 +01:00
Neeraj Kumar Kashyap
5a481e6a01 Option for IBM IAM auth endpoint added return 2018-06-04 16:44:14 +05:30
Neeraj Kumar Kashyap
d8e12839af Option for IBM IAM auth endpoint 2018-05-31 16:02:48 +05:30
Or Ozeri
d4e86a17d1 Revert "enable FUSE read_sync by default"
This reverts commit 86b0921ac40ad0a34cd4202901efc2f4f774777b.

Conflicts:
	src/s3fs.cpp
2018-05-28 13:49:54 +03:00
Takeshi Nakatani
ae9d8eb734 Fixed memory leak 2018-05-27 10:48:03 +00:00
Takeshi Nakatani
57b2a60172
Merge pull request #764 from orozery/remove_false_multihead_warnings
Remove false multihead warnings
2018-05-23 22:38:35 +09:00
Takeshi Nakatani
212bbbbdf0
Merge pull request #763 from orozery/cleanup_share_after_handles
cleanup curl handles before curl share
2018-05-23 22:30:36 +09:00
Takeshi Nakatani
da95afba8a
Merge pull request #756 from orozery/optimize_defaults
Optimize defaults
2018-05-23 22:05:00 +09:00
Or Ozeri
0bd875eb9e remove false readdir_multi_head warnings 2018-05-22 17:10:50 +03:00
Or Ozeri
af63a42773 cleanup curl handles before curl share 2018-05-21 13:20:09 +03:00
Or Ozeri
86b0921ac4 enable FUSE read_sync by default 2018-05-06 16:10:36 +03:00
Or Ozeri
4a72b60707 increase default stat cache size from 1000 to 100000 2018-05-06 15:31:07 +03:00
Takeshi Nakatani
e3de6ea458 Added reset curl handle when returning to handle pool 2018-05-06 12:11:53 +00:00
Neeraj Kumar Kashyap
25375a6b48 Validate the URL fixed inefficient usage of find 2018-05-04 11:24:32 +05:30
Neeraj Kumar Kashyap
ca87df7d44 Validate the URL format for http/https 2018-05-03 22:08:28 +05:30
Takeshi Nakatani
d052dc0b9d
Merge pull request #753 from cfz/master
fix xpath selector in bucket listing
2018-05-02 12:04:12 +09:00
陈方舟
04493de767
fix xpath selector in bucket listing
the original implementation in get_base_exp() depends on the order of xml return from the server.
patriotically, when listing a directory with sub directory(s), the xml document response contains more than 2 <Prefix> nodes(some of them are in <CommonPrefixes> node).
the source code arbitrarily select the first one in the documents (nodes->nodeTab[0]->xmlChildrenNode).
some s3 compatible service return the list-bucket result in different result, leading the s3fs to a wrong behavior
2018-04-23 15:11:29 +08:00
Or Ozeri
4fdab46617 don't fail mkdir when directory exists 2018-04-08 11:13:47 +03:00
Takeshi Nakatani
1a23b880d5
Merge pull request #739 from orozery/cleanup_failing_curl_handles
cleanup curl handle state on retries
2018-04-01 22:45:04 +09:00
Or Ozeri
adcf5754ae cleanup failing curl handles on retries 2018-03-29 13:56:08 +03:00
Tristen Horton
0863672e27
add a more helpful error message for when there are multiple entries for the same bucket in the passwd file 2018-03-13 14:37:34 -07:00
Takeshi Nakatani
0f503ced25
Merge pull request #729 from dmgk/master
FreeBSD build fixes
2018-03-04 16:36:31 +09:00
Takeshi Nakatani
987a166bf4
Merge pull request #726 from orozery/instance_name_logging
add an instance_name option for logging
2018-03-04 15:41:12 +09:00
Takeshi Nakatani
57b6f0eeaf
Merge pull request #724 from orozery/dont_fail_multirequest
don't fail multirequest on single thread error
2018-03-04 15:35:29 +09:00
Takeshi Nakatani
f71a28f9b9
Merge pull request #714 from orozery/reduce_lock_contention
reduce lock contention on file open
2018-03-04 13:36:08 +09:00
Dmitri Goutnik
c9f4312588 FreeBSD build fixes 2018-03-02 15:58:52 -05:00
Or Ozeri
8b657eee41 add disk space reservation 2018-02-28 19:20:23 +02:00
Takeshi Nakatani
be9d407fa0 Fixed cppcheck error on osx 2018-02-28 12:29:58 +00:00
Takeshi Nakatani
c494e54320 Fixed cppcheck error on osx 2018-02-28 12:06:06 +00:00
Or Ozeri
b52b6f3fc5 add an instance_name option for logging 2018-02-28 09:51:35 +02:00
Or Ozeri
82c9733101 don't fail multirequest on single thread error 2018-02-26 12:06:08 +02:00
Takeshi Nakatani
a45ff6cdaa Fixed cppcheck error and clean ^M code 2018-02-25 13:08:41 +00:00
Takeshi Nakatani
960d45c853 Fixed cppcheck error on osx 2018-02-25 08:51:19 +00:00
Or Ozeri
0edf056e95 reduce lock contention on file open 2018-02-04 17:13:58 +02:00