833 Commits

Author SHA1 Message Date
LutzFinsterle2019
18cb2e2662
Update s3fs_util.cpp
Sorry for answering late, have been busy lately.
The comment is: "Nextcloud stores Directory objects with this mime type when mounting a Bucket as external Storage"
2019-01-10 07:44:36 +01:00
LutzFinsterle2019
743c706b0a
Update s3fs_util.cpp
Nextcloud Compatibility in directory mime-types
2019-01-07 07:41:27 +01:00
Takeshi Nakatani
4ed0e5f35a
Merge pull request #882 from earlchew/issue-817
[curl] Assume long encryption keys are base64 encoded
2019-01-06 17:29:04 +09:00
Takeshi Nakatani
fd6b37d3da
Merge pull request #877 from gaul/aws/credentials
Check arguments and environment before .aws/creds
2019-01-06 17:03:11 +09:00
Takeshi Nakatani
56e24de0d4
Merge pull request #870 from gaul/typos
Correct typos in command-line parsing
2019-01-06 16:21:46 +09:00
Earl Chew
ed5795eead [curl] Assume long encryption keys are base64 encoded
Correct tabs and whitespace.

Signed-off-by: Earl Chew <earl_chew@yahoo.com>
2019-01-05 21:08:41 +00:00
Andrew Gaul
3d225163f8 Correct typos in command-line parsing
Also fix stray macOS references.  References #869.
2019-01-02 14:40:57 -08:00
Andrew Gaul
0569cec3ea Check arguments and environment before .aws/creds
Fixes #857.  Fixes #864.
2018-12-20 15:51:32 -08:00
Andrew Gaul
a2f8ac535e Address cppcheck 1.86 errors
Lifetime, shadowing, and unused variables.  Found via the Travis macOS
builder.
2018-12-20 14:56:31 -08:00
Takeshi Nakatani
d9e89deef6
Merge pull request #865 from orozery/multihead_warning_check
fix multihead warning check
2018-11-29 20:53:47 +09:00
Or Ozeri
b91fc5409e fix multihead warning check 2018-11-28 09:54:02 +02:00
Michael Chen
3c970646d1
Add 'profile' option to command line help. 2018-11-19 08:26:23 -08:00
Andrew Gaul
88cd8feb05 Upload S3 parts without batching
Previously s3fs would issue a batch of uploads and wait for all to
succeed before issuing the next batch.  Now it issues the first batch
and only waits for a single part to succeed before uploading the next
part.  This can improve performance when one part lags due to network
errors.  Fixes #183.
2018-11-16 18:32:38 -08:00
Takeshi Nakatani
5fba542a29
Merge pull request #852 from gaul/aws-credentials-file
Allow credentials from ${HOME}/.aws/credentials
2018-11-11 11:37:38 +09:00
Andrew Gaul
9e530c86ae Allow credentials from ${HOME}/.aws/credentials
This matches the configuration from popular tools like AWS CLI and
allows multiple profile names via -o profile=name.  The existing
credential mechanisms continue to work.  Fixes #822.
2018-11-04 17:47:07 -08:00
Andrew Gaul
664f910083 Correctly compare list_object_max_keys
Previously this did not allow an argument.  Fixes #843.
References #793.
2018-11-04 10:49:35 -08:00
Takeshi Nakatani
6fe92d5ed6
Merge pull request #832 from gaul/hex
Simplify hex conversion
2018-10-08 23:03:42 +09:00
Andrew Gaul
c703fa15c0 Simplify hex conversion
Addresses GCC 8 warning:

common_auth.cpp: In function ‘std::__cxx11::string s3fs_sha256sum(int, off_t, ssize_t)’:
common_auth.cpp:84:12: warning: ‘char* strncat(char*, const char*, size_t)’ output may be truncated copying 2 bytes from a string of length 2 [-Wstringop-truncation]
    strncat(sha256, hexbuf, 2);
2018-10-01 19:08:27 -07:00
Jonatan Nilsson
d9c106cfde Add support for storage class ONEZONE_IA. 2018-09-30 14:27:31 +02:00
Takeshi Nakatani
c5af62b023
Merge pull request #820 from gaul/big-writes
Enable big writes if capable
2018-09-17 17:24:06 +09:00
Earl Chew
41c23adb0e [curl] Assume long encryption keys are base64 encoded
Amazon SSE-C https://tinyurl.com/ychug4cg writes:

> Use this header to provide the 256-bit, base64-encoded encryption key
> for Amazon S3 to use to encrypt or decrypt your data.

It seems likely that future keys would be as long, or longer, so
this change assumes that text longer than 32 characters are keys
that are base64 encoded.

Signed-off-by: Earl Chew <earl_chew@yahoo.com>
2018-09-16 21:23:22 +00:00
Andrew Gaul
a85183d42c Enable big writes if capable
Fixes #813.
2018-09-10 22:21:51 -07:00
Mattia Rizzolo
c376efdd28
Fix typo s/mutliple/multiple/
Signed-off-by: Mattia Rizzolo <mattia@mapreri.org>
2018-08-19 00:14:57 +02:00
Takeshi Nakatani
7e0c53dfe9 Added list_object_max_keys option based on #783 PR 2018-07-08 03:49:10 +00:00
root
35d3fce7a0 Review comment: Include the error code being returned 2018-07-06 05:14:32 -04:00
root
4177d8bd3b Review comment: Include the error code being returned 2018-07-06 03:03:57 -04:00
root
ad5349a488 Changes as per review comments 2018-07-05 05:02:04 -04:00
root
92a4034c5e Log messages for 5xx and 4xx HTTP response code 2018-07-04 03:50:45 -04:00
Takeshi Nakatani
3e4002df0d
Merge pull request #780 from wjt/initialize-libgcry
gnutls_auth: initialize libgcrypt
2018-06-24 12:48:08 +09:00
Takeshi Nakatani
1b9ec7f4fc
Merge pull request #774 from nkkashyap/master
Option for IAM authentication endpoint
2018-06-24 12:36:23 +09:00
Takeshi Nakatani
0d3fb0658a Fixed a error by cppcheck on OSX 2018-06-24 02:38:59 +00:00
Will Thompson
73cf2ba95d
gnutls_auth: initialize libgcrypt
Without this change, the following warning appears in the syslog/journal
during startup:

  Libgcrypt warning: missing initialization - please fix the application

From the [documentation][0]:

> The function `gcry_check_version` initializes some subsystems used by
> Libgcrypt and must be invoked before any other function in the
> library.

Fixes #524, which says:

> gnutls is initialized by gnutls_global_init() function and
> gcry_check_version() function for initializing libgcry is called from
> this gnutls_global_init().

I checked the gnutls source and it hasn't contained a call to
gcry_check_version() since the libgcrypt backend was removed in 2011
(commit 8116cdc8f131edd586dad3128ae35dd744cfc32f). In any case, the
gcry_check_version() documentation continues:

> It is important that these initialization steps are not done by a
> library but by the actual application.

so it would be incorrect for a library used by s3fs to initialize
libgcrypt.

[0]: https://www.gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
2018-06-21 20:55:00 +01:00
Neeraj Kumar Kashyap
5a481e6a01 Option for IBM IAM auth endpoint added return 2018-06-04 16:44:14 +05:30
Neeraj Kumar Kashyap
d8e12839af Option for IBM IAM auth endpoint 2018-05-31 16:02:48 +05:30
Or Ozeri
d4e86a17d1 Revert "enable FUSE read_sync by default"
This reverts commit 86b0921ac40ad0a34cd4202901efc2f4f774777b.

Conflicts:
	src/s3fs.cpp
2018-05-28 13:49:54 +03:00
Takeshi Nakatani
ae9d8eb734 Fixed memory leak 2018-05-27 10:48:03 +00:00
Takeshi Nakatani
57b2a60172
Merge pull request #764 from orozery/remove_false_multihead_warnings
Remove false multihead warnings
2018-05-23 22:38:35 +09:00
Takeshi Nakatani
212bbbbdf0
Merge pull request #763 from orozery/cleanup_share_after_handles
cleanup curl handles before curl share
2018-05-23 22:30:36 +09:00
Takeshi Nakatani
da95afba8a
Merge pull request #756 from orozery/optimize_defaults
Optimize defaults
2018-05-23 22:05:00 +09:00
Or Ozeri
0bd875eb9e remove false readdir_multi_head warnings 2018-05-22 17:10:50 +03:00
Or Ozeri
af63a42773 cleanup curl handles before curl share 2018-05-21 13:20:09 +03:00
Or Ozeri
86b0921ac4 enable FUSE read_sync by default 2018-05-06 16:10:36 +03:00
Or Ozeri
4a72b60707 increase default stat cache size from 1000 to 100000 2018-05-06 15:31:07 +03:00
Takeshi Nakatani
e3de6ea458 Added reset curl handle when returning to handle pool 2018-05-06 12:11:53 +00:00
Neeraj Kumar Kashyap
25375a6b48 Validate the URL fixed inefficient usage of find 2018-05-04 11:24:32 +05:30
Neeraj Kumar Kashyap
ca87df7d44 Validate the URL format for http/https 2018-05-03 22:08:28 +05:30
Takeshi Nakatani
d052dc0b9d
Merge pull request #753 from cfz/master
fix xpath selector in bucket listing
2018-05-02 12:04:12 +09:00
陈方舟
04493de767
fix xpath selector in bucket listing
the original implementation in get_base_exp() depends on the order of xml return from the server.
patriotically, when listing a directory with sub directory(s), the xml document response contains more than 2 <Prefix> nodes(some of them are in <CommonPrefixes> node).
the source code arbitrarily select the first one in the documents (nodes->nodeTab[0]->xmlChildrenNode).
some s3 compatible service return the list-bucket result in different result, leading the s3fs to a wrong behavior
2018-04-23 15:11:29 +08:00
Or Ozeri
4fdab46617 don't fail mkdir when directory exists 2018-04-08 11:13:47 +03:00
Takeshi Nakatani
1a23b880d5
Merge pull request #739 from orozery/cleanup_failing_curl_handles
cleanup curl handle state on retries
2018-04-01 22:45:04 +09:00