Mattia Rizzolo
c376efdd28
Fix typo s/mutliple/multiple/
...
Signed-off-by: Mattia Rizzolo <mattia@mapreri.org>
2018-08-19 00:14:57 +02:00
Takeshi Nakatani
7e0c53dfe9
Added list_object_max_keys option based on #783 PR
2018-07-08 03:49:10 +00:00
root
35d3fce7a0
Review comment: Include the error code being returned
2018-07-06 05:14:32 -04:00
root
4177d8bd3b
Review comment: Include the error code being returned
2018-07-06 03:03:57 -04:00
root
ad5349a488
Changes as per review comments
2018-07-05 05:02:04 -04:00
root
92a4034c5e
Log messages for 5xx and 4xx HTTP response code
2018-07-04 03:50:45 -04:00
Takeshi Nakatani
3e4002df0d
Merge pull request #780 from wjt/initialize-libgcry
...
gnutls_auth: initialize libgcrypt
2018-06-24 12:48:08 +09:00
Takeshi Nakatani
1b9ec7f4fc
Merge pull request #774 from nkkashyap/master
...
Option for IAM authentication endpoint
2018-06-24 12:36:23 +09:00
Takeshi Nakatani
0d3fb0658a
Fixed a error by cppcheck on OSX
2018-06-24 02:38:59 +00:00
Will Thompson
73cf2ba95d
gnutls_auth: initialize libgcrypt
...
Without this change, the following warning appears in the syslog/journal
during startup:
Libgcrypt warning: missing initialization - please fix the application
From the [documentation][0]:
> The function `gcry_check_version` initializes some subsystems used by
> Libgcrypt and must be invoked before any other function in the
> library.
Fixes #524 , which says:
> gnutls is initialized by gnutls_global_init() function and
> gcry_check_version() function for initializing libgcry is called from
> this gnutls_global_init().
I checked the gnutls source and it hasn't contained a call to
gcry_check_version() since the libgcrypt backend was removed in 2011
(commit 8116cdc8f131edd586dad3128ae35dd744cfc32f). In any case, the
gcry_check_version() documentation continues:
> It is important that these initialization steps are not done by a
> library but by the actual application.
so it would be incorrect for a library used by s3fs to initialize
libgcrypt.
[0]: https://www.gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
2018-06-21 20:55:00 +01:00
Neeraj Kumar Kashyap
5a481e6a01
Option for IBM IAM auth endpoint added return
2018-06-04 16:44:14 +05:30
Neeraj Kumar Kashyap
d8e12839af
Option for IBM IAM auth endpoint
2018-05-31 16:02:48 +05:30
Or Ozeri
d4e86a17d1
Revert "enable FUSE read_sync by default"
...
This reverts commit 86b0921ac4
2018-05-28 13:49:54 +03:00
Takeshi Nakatani
ae9d8eb734
Fixed memory leak
2018-05-27 10:48:03 +00:00
Takeshi Nakatani
57b2a60172
Merge pull request #764 from orozery/remove_false_multihead_warnings
...
Remove false multihead warnings
2018-05-23 22:38:35 +09:00
Takeshi Nakatani
212bbbbdf0
Merge pull request #763 from orozery/cleanup_share_after_handles
...
cleanup curl handles before curl share
2018-05-23 22:30:36 +09:00
Takeshi Nakatani
da95afba8a
Merge pull request #756 from orozery/optimize_defaults
...
Optimize defaults
2018-05-23 22:05:00 +09:00
Or Ozeri
0bd875eb9e
remove false readdir_multi_head warnings
2018-05-22 17:10:50 +03:00
Or Ozeri
af63a42773
cleanup curl handles before curl share
2018-05-21 13:20:09 +03:00
Or Ozeri
86b0921ac4
enable FUSE read_sync by default
2018-05-06 16:10:36 +03:00
Or Ozeri
4a72b60707
increase default stat cache size from 1000 to 100000
2018-05-06 15:31:07 +03:00
Takeshi Nakatani
e3de6ea458
Added reset curl handle when returning to handle pool
2018-05-06 12:11:53 +00:00
Neeraj Kumar Kashyap
25375a6b48
Validate the URL fixed inefficient usage of find
2018-05-04 11:24:32 +05:30
Neeraj Kumar Kashyap
ca87df7d44
Validate the URL format for http/https
2018-05-03 22:08:28 +05:30
Takeshi Nakatani
d052dc0b9d
Merge pull request #753 from cfz/master
...
fix xpath selector in bucket listing
2018-05-02 12:04:12 +09:00
陈方舟
04493de767
fix xpath selector in bucket listing
...
the original implementation in get_base_exp() depends on the order of xml return from the server.
patriotically, when listing a directory with sub directory(s), the xml document response contains more than 2 <Prefix> nodes(some of them are in <CommonPrefixes> node).
the source code arbitrarily select the first one in the documents (nodes->nodeTab[0]->xmlChildrenNode).
some s3 compatible service return the list-bucket result in different result, leading the s3fs to a wrong behavior
2018-04-23 15:11:29 +08:00
Or Ozeri
4fdab46617
don't fail mkdir when directory exists
2018-04-08 11:13:47 +03:00
Takeshi Nakatani
1a23b880d5
Merge pull request #739 from orozery/cleanup_failing_curl_handles
...
cleanup curl handle state on retries
2018-04-01 22:45:04 +09:00
Or Ozeri
adcf5754ae
cleanup failing curl handles on retries
2018-03-29 13:56:08 +03:00
Tristen Horton
0863672e27
add a more helpful error message for when there are multiple entries for the same bucket in the passwd file
2018-03-13 14:37:34 -07:00
Takeshi Nakatani
0f503ced25
Merge pull request #729 from dmgk/master
...
FreeBSD build fixes
2018-03-04 16:36:31 +09:00
Takeshi Nakatani
987a166bf4
Merge pull request #726 from orozery/instance_name_logging
...
add an instance_name option for logging
2018-03-04 15:41:12 +09:00
Takeshi Nakatani
57b6f0eeaf
Merge pull request #724 from orozery/dont_fail_multirequest
...
don't fail multirequest on single thread error
2018-03-04 15:35:29 +09:00
Takeshi Nakatani
f71a28f9b9
Merge pull request #714 from orozery/reduce_lock_contention
...
reduce lock contention on file open
2018-03-04 13:36:08 +09:00
Dmitri Goutnik
c9f4312588
FreeBSD build fixes
2018-03-02 15:58:52 -05:00
Or Ozeri
8b657eee41
add disk space reservation
2018-02-28 19:20:23 +02:00
Takeshi Nakatani
be9d407fa0
Fixed cppcheck error on osx
2018-02-28 12:29:58 +00:00
Takeshi Nakatani
c494e54320
Fixed cppcheck error on osx
2018-02-28 12:06:06 +00:00
Or Ozeri
b52b6f3fc5
add an instance_name option for logging
2018-02-28 09:51:35 +02:00
Or Ozeri
82c9733101
don't fail multirequest on single thread error
2018-02-26 12:06:08 +02:00
Takeshi Nakatani
a45ff6cdaa
Fixed cppcheck error and clean ^M code
2018-02-25 13:08:41 +00:00
Takeshi Nakatani
960d45c853
Fixed cppcheck error on osx
2018-02-25 08:51:19 +00:00
Or Ozeri
0edf056e95
reduce lock contention on file open
2018-02-04 17:13:58 +02:00
Takeshi Nakatani
6b58220009
Merge pull request #697 from pwulff/master
...
Fixing race condition in FdEntity::GetStats
2017-12-17 15:46:48 +09:00
Paul Wulff
ee6abea956
Race condition in FdManager::Rename because no mutex is used.
2017-12-15 15:27:51 +01:00
Paul Wulff
cea7d44717
Fixing race condition in FdEntity::GetStats
2017-12-13 10:49:00 +01:00
Or Ozeri
0da87e75fe
fix condition for parallel download
2017-12-04 16:07:33 +02:00
Or Ozeri
bd4bc0e7f1
add support for IBM IAM authentication
2017-11-23 12:01:52 +02:00
Takeshi Nakatani
026260e7a1
Improved use of temporary files
2017-11-23 09:18:11 +00:00
Takeshi Nakatani
99fe93b7f1
Merge pull request #684 from gaul/signedness
...
Correct signedness warning
2017-11-23 17:33:46 +09:00
Or Ozeri
11bd7128d2
remove use of jsoncpp
2017-11-22 13:36:27 +02:00
Andrew Gaul
7cda32664b
Correct signedness warning
...
Fixes regression from 0418e53b3c
2017-11-19 11:14:37 -08:00
Takeshi Nakatani
97fc845a6a
Changed functions about reading passwd file.
2017-11-19 11:49:11 +00:00
Takeshi Nakatani
7d9ac0163b
Changed functions about reading passwd file.
2017-11-19 11:38:12 +00:00
Takeshi Nakatani
e1928288fe
Merge pull request #676 from gaul/sstream
...
Move str definition from header to implementation
2017-11-19 16:34:15 +09:00
Andrew Gaul
0418e53b3c
Reduce use of preprocessor
...
This provides type-safety and avoids token expansion side effects.
2017-11-18 22:40:06 -08:00
Takeshi Nakatani
bad48ab59a
Merge pull request #671 from psyvision/master
...
Add support for ECS metadata endpoint
2017-11-19 14:43:06 +09:00
Andrew Gaul
bbad76bb71
Move str definition from header to implementation
...
Also clean up some char * to std::string conversions.
2017-11-18 11:34:34 -08:00
Richard Caunt
b95e4acaeb
Remove debug logging statements
2017-11-08 15:52:35 +00:00
Richard Caunt
c238701d09
Corrected ECS headers
2017-11-08 15:21:49 +00:00
Richard Caunt
60d2ac3c7a
Adding x-amz-security-token header
2017-11-08 15:09:59 +00:00
Richard Caunt
967ef4d56b
Corrected fat finger mistakes
2017-11-08 13:14:49 +00:00
Richard Caunt
ad57bdda6c
Corrected keycount check
2017-11-08 13:06:22 +00:00
Richard Caunt
a0b69d1d3d
Corrected keyval[].c_str()
2017-11-08 13:01:52 +00:00
Richard Caunt
5df94d7e33
Add debug messages
2017-11-08 09:50:39 +00:00
Richard Caunt
1cbe9fb7a3
Gotta pass that cppcheckgit add .
2017-11-07 21:41:51 +00:00
Richard Caunt
8660abaea2
Use jsoncpp to parse AWS JSON
2017-11-07 21:20:02 +00:00
Richard Caunt
366f0705a0
ECS credentials bug fixes
2017-11-06 21:45:58 +00:00
Richard Caunt
ccea87ca68
Added check for is_ecs during get_access_keys
...
We need to presume that if `is_ecs` we are deferring access key loading
2017-11-06 11:02:27 +00:00
Richard Caunt
5d54883e2f
Remove commented out code
2017-11-05 19:25:34 +00:00
Richard Caunt
662f65c3c8
Add support for ECS metadata endpoint
2017-11-05 19:24:02 +00:00
Takeshi Nakatani
5db550a298
Fixed a bug in S3fsCurl::LocateBundle
2017-11-05 11:26:05 +00:00
Takeshi Nakatani
e3c77d2906
Merge pull request #664 from orozery/auth_refactor
...
auth headers insertion refactoring
2017-11-05 15:14:40 +09:00
Or Ozeri
384b4cbafa
auth headers insertion refactoring
2017-10-30 11:52:58 +02:00
Andrew Gaul
40501a7a73
Lock FdEntity when mutating orgmeta
...
References #654 .
2017-10-23 22:41:42 -07:00
Takeshi Nakatani
ab89b4cd4a
Merge pull request #659 from ggtakec/master
...
Do not fail updating directory when removing old-style object(ref #658 )
2017-10-15 21:56:35 +09:00
Takeshi Nakatani
1eba27a50a
Refixed s3fs_init message(ref #652 )
2017-10-15 06:45:19 +00:00
Takeshi Nakatani
41206fa0e2
Do not fail updating directory when removing old-style object(ref #658 )
2017-10-15 05:03:44 +00:00
jurafxp
ae91b6f673
Fix s3fs_init message
2017-10-01 00:08:00 +02:00
Takeshi Nakatani
5957d9ead0
Fixed with unnecessary equal in POST uploads url argment - #643
2017-09-17 10:52:28 +00:00
Takeshi Nakatani
00bc9142c4
Fixed potential atomic violation in S3fsCurl::AddUserAgent - #633
2017-09-17 09:16:05 +00:00
Scott Talbert
20da0e4dd3
Fix intermittent upload failures on macOS
...
There were multiple problems with the FdManager::GetFreeDiskSpace() function
on macOS:
1) When calling statvfs(), f_frsize should be used instead of f_bsize when
converting available blocks to bytes. This was causing the free space
calculation to be incorrect.
2) On macOS, fsblkcnt_t is a 32-bit integer. Thus, when calculating available
disk space, there were frequently overflows. This caused s3fs to incorrectly
determine that the cache location was out of space in the middle of a transfer
which caused uploads to fail. Changing this to a uint64_t resolves the
problem.
2017-09-08 15:23:10 -04:00
Takeshi Nakatani
96d8e6d823
Merge remote-tracking branch 'upstream/macosx' into macosx
2017-08-11 14:20:12 +00:00
Takeshi Nakatani
62b8084300
Added travis test on osx for #601
2017-08-11 14:09:43 +00:00
Takeshi Nakatani
0c6a3882a2
Fixed clock_gettime build failure on macOS 10.12 Sierra - #600
2017-05-28 10:04:25 +00:00
Takeshi Nakatani
a08880ae15
Merge pull request #608 from tlevi/chown_nocopy
...
Fix chown_nocopy losing existing uid/gid if unspecified
2017-05-28 18:36:08 +09:00
Tony Levi
cc94e1da26
Fix chown_nocopy losing existing uid/gid if unspecified
2017-05-25 16:53:08 +09:30
Tony Levi
2b7ea5813c
Expand buffer for group information if too small and retry
2017-05-23 10:42:43 +09:30
Takeshi Nakatani
337da59368
Not fallback to HTTP - #596
2017-05-13 15:47:39 +00:00
Takeshi Nakatani
d2ae14d8b7
Check bucket at public bucket and add nocopyapi option automatically
2017-05-13 07:48:50 +00:00
Takeshi Nakatani
7115835834
Check bucket at public bucket and add nocopyapi option automatically
2017-05-13 07:35:55 +00:00
Takeshi Nakatani
24df69f688
Backward compatible for changing default transport to HTTPS
2017-05-13 06:47:51 +00:00
Takeshi Nakatani
465c15ef40
Merge pull request #588 from andrewgaul/https
...
Default transport to HTTPS
2017-05-09 23:04:57 +09:00
Takeshi Nakatani
a22675bafd
Merge pull request #567 from andrewgaul/default-acl
...
Do not send ACL unless overridden
2017-05-09 23:03:27 +09:00
Andrew Gaul
0e0ae38f6d
Default transport to HTTPS
...
This protects private data when used over the public Internet. Users
can opt-in to unencrypted HTTP if they need additional performance on
a local network. Fixes #282 .
2017-05-07 10:59:54 -07:00
Andrew Gaul
7b30d5d15b
Do not send canned ACL header when empty string
...
Some providers such as StorageGRID do not support canned ACLs.
Setting to empty allows callers to omit the header. References #125 .
2017-05-07 10:52:31 -07:00
Takeshi Nakatani
9d10a5aa70
Changed copyright year format for debian pkg
2017-05-07 11:24:17 +00:00
Takeshi Nakatani
a12e0d5ec4
Fixed failure to upload/copy with SSE_C and SSE_KMS
2017-05-07 09:29:08 +00:00
Takeshi Nakatani
d07c3f38b7
Check errors returned in 200 OK responses for put header request
2017-05-06 02:15:53 +00:00
Takeshi Nakatani
1fe0334c08
Enhanced bucket/path parameter check
2017-05-05 19:55:24 +00:00
Takeshi Nakatani
3ac39d61f8
Added notsup_compat_dir option
2017-05-05 17:28:29 +00:00
Takeshi Nakatani
864e20e1f2
Refactored the get_object_attribute function
2017-05-05 10:02:21 +00:00
Takeshi Nakatani
51b3183cba
Refactored the check_object_access function
2017-05-05 09:51:30 +00:00
Takeshi Nakatani
758b92e823
Added option for complementing lack of stat mode
2017-05-04 03:41:24 +00:00
Takeshi Nakatani
df0ff3a2fd
Merge pull request #556 from orozery/fix_nocache_multipart_upload
...
fix multipart upload handling without cache
2017-04-16 19:22:15 +09:00
Takeshi Nakatani
edcf4c6218
Merge pull request #555 from orozery/dont_sign_empty_headers
...
don't sign empty headers (as they are discarded by libcurl)
2017-04-16 19:16:47 +09:00
Takeshi Nakatani
28efff5986
Merge pull request #554 from orozery/cache_cleanup
...
cleanup cache directory when running out of disk space
2017-04-16 19:13:11 +09:00
Takeshi Nakatani
efba9bcbc1
Merge pull request #553 from orozery/custom_cipher_suite
...
add TLS cipher suites customization
2017-04-16 19:09:27 +09:00
Takeshi Nakatani
6bd179c92b
Merge pull request #552 from orozery/foreground_threads
...
switch S3fsMultiCurl to use foreground threads
2017-04-16 19:05:16 +09:00
Or Ozeri
96764b7410
switch S3fsMultiCurl to use foreground threads
2017-04-09 16:56:49 +03:00
Takeshi Nakatani
ff3eb1971f
Merge branch 'master' into fix_nocache_multipart_upload
2017-04-09 22:13:33 +09:00
Takeshi Nakatani
b4c90d6957
Fixed a bug about multipart uploading at no disk free space related to #509
2017-04-09 04:37:20 +00:00
Or Ozeri
75b59a7c16
switch S3fsMultiCurl to use foreground threads
2017-04-04 15:32:53 +03:00
Or Ozeri
3bcca75a88
don't sign empty headers (as they are discarded by libcurl)
2017-04-04 15:24:20 +03:00
Takeshi Nakatani
f0f61b3b55
Fixed a bug in logic about truncating stat cache
2017-04-02 11:51:58 +00:00
Takeshi Nakatani
fef3fbc225
Added check_cache_dir_exist option(refixed #347 ) - #538
2017-04-02 08:10:16 +00:00
Or Ozeri
8ee95ff7ab
fix multipart upload handling without cache
2017-04-02 10:27:43 +03:00
Or Ozeri
95578cad43
cleanup cache directory when running out of disk space
2017-04-02 10:22:12 +03:00
Takeshi Nakatani
465bbd3729
Updated stat_cache_expire option description - #545
2017-04-02 07:19:16 +00:00
Takeshi Nakatani
980ba398bc
Fixed double initialization of SSL library - #524
2017-03-19 17:11:18 +00:00
Takeshi Nakatani
523043a2aa
Changed base cached time of stat_cache_expire option - #523
2017-03-19 15:19:04 +00:00
Andrew Gaul
03217baa99
Address cppcheck 1.77 warnings
2017-03-06 12:41:08 -08:00
Takeshi Nakatani
6affefff5b
Merge pull request #509 from andrewgaul/mpu
...
Use server-provided ETag during complete upload
2017-03-06 21:54:51 +09:00
Takeshi Nakatani
6fc972972f
Changed clock_gettime func to s3fs_clock_gettime for homebrew - #468
2016-12-04 10:31:41 +00:00
Takeshi Nakatani
7b307601b5
Merge pull request #511 from s3fs-fuse/issue#435
...
Fixed a bug about uploading NULL to some part of the file contents
2016-12-04 17:20:50 +09:00
Andrew Gaul
915a1321c7
Use server-provided ETag during complete upload
...
This avoids calculating the MD5 locally and enables use with object
stores which do not use MD5 as ETag.
2016-11-23 18:48:57 -08:00
Andrew Gaul
d375bca0d0
Correct typos
2016-11-19 15:57:41 -08:00
Takeshi Nakatani
edd0a11fb5
Merge pull request #494 from mapreri/typo
...
Fix typo s/destroied/destroyed/
2016-11-20 05:53:23 +09:00
Takeshi Nakatani
67a836223a
Merge pull request #495 from driskell/fix_sse_copy
...
Fix invalid V4 signature on multipart copy requests
2016-11-20 05:43:10 +09:00
Takeshi Nakatani
81e209bdd1
Fixed issue#435 codes
2016-11-19 20:09:35 +00:00
Jason Woods
6f688770fd
Fix invalid V4 signature on multipart copy requests
2016-11-13 13:22:00 +00:00
Mattia Rizzolo
8c0b1d9c5b
Fix typo s/destroied/destroyed/
2016-11-11 23:27:17 +00:00
Takeshi Nakatani
efde0ec9de
Merge pull request #489 from ggtakec/master
...
Changed headers_t map using nocase compare function - #488
2016-10-23 23:23:31 +09:00
Takeshi Nakatani
632495374b
Chnaged headers_t map using nocase compare function - #488
2016-10-23 14:14:19 +00:00
Takeshi Nakatani
15b797f3ee
Merge pull request #488 from ggtakec/master
...
Fixed searching Content-Length without case sensitive - #480
2016-10-23 22:42:52 +09:00
Takeshi Nakatani
a7a64d954a
Fixed searching Content-Length without case sensitive - #480
2016-10-23 13:27:01 +00:00
Takeshi Nakatani
cca217f613
Merge pull request #487 from driskell/debugging
...
Split header debugging onto multiple lines for easier reading
2016-10-23 21:51:38 +09:00
Takeshi Nakatani
1a9cf6f66d
Fixed searching Content-Length without case sensitive - #480
2016-10-23 12:40:51 +00:00
Jason Woods
02d7296210
Split header debugging onto multiple lines for easier reading
2016-10-22 15:11:18 +01:00
Takeshi Nakatani
a688df813e
Fixed a bug at read symlink
2016-10-11 13:32:08 +00:00
Takeshi Nakatani
164424bc89
Merge branch 'master' into issue#435
2016-10-11 12:13:03 +00:00
Takeshi Nakatani
7fabd18b1f
Trim symbolic link original path in file.
2016-10-11 10:22:30 +00:00
Takeshi Nakatani
5db369d67e
Trim symbolic link original path in file.
2016-10-11 10:17:46 +00:00
Takeshi Nakatani
dba32fdf78
Trim symbolic link original path in file.
2016-10-11 10:06:21 +00:00
Takeshi Nakatani
716baada22
Testing patch codes for issue#435
2016-10-10 12:16:09 +00:00
Takeshi Nakatani
9fd1368611
OS-specific correspondence of the extended attribute header
2016-10-02 07:23:19 +00:00
Takeshi Nakatani
65d52506c4
Added use_xattr option for #467 and #460
2016-09-19 04:28:01 +00:00
Takeshi Nakatani
1a96f40a10
Fixed a bug about could not copy file mode from org file
2016-09-11 13:09:23 +00:00
Takeshi Nakatani
1ddc14d59d
Changed for accepting mount options compatible with mtab - #449
2016-07-24 08:17:58 +00:00
Peter Watkins
a19206cf0f
Accept mount arguments compatible with mtab
...
Using "mount -a" fails for already-mounted s3fs directories,
because s3fs mount arguments don't match the form in /etc/mtab.
Calling "mount -a" should quietly succeed when a directory is
already mounted.
To fix this, accept mount commands of the form:
s3fs s3fs /srv/object-store -o bucket=mybucket
or in /etc/fstab form:
s3fs /srv/object-store fuse.s3fs bucket=mybucket 0 0
This matches the form in /etc/mtab and allows "mount -a" to
work properly.
2016-07-13 17:23:33 -04:00
