octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-12-27 02:22:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
883 Commits 1 Branch 23 Tags 13 MiB
1eba27a50a
Commit Graph

434 Commits

Author SHA1 Message Date
Takeshi Nakatani
edd0a11fb5 Merge pull request #494 from mapreri/typo 
Fix typo s/destroied/destroyed/
 2016-11-20 05:53:23 +09:00
Takeshi Nakatani
67a836223a Merge pull request #495 from driskell/fix_sse_copy 
Fix invalid V4 signature on multipart copy requests
 2016-11-20 05:43:10 +09:00
Takeshi Nakatani
81e209bdd1 Fixed issue#435 codes 2016-11-19 20:09:35 +00:00
Jason Woods
6f688770fd Fix invalid V4 signature on multipart copy requests 2016-11-13 13:22:00 +00:00
Mattia Rizzolo
8c0b1d9c5b
Fix typo s/destroied/destroyed/ 2016-11-11 23:27:17 +00:00
Takeshi Nakatani
efde0ec9de Merge pull request #489 from ggtakec/master 
Changed headers_t map using nocase compare function - #488
 2016-10-23 23:23:31 +09:00
Takeshi Nakatani
632495374b Chnaged headers_t map using nocase compare function - #488 2016-10-23 14:14:19 +00:00
Takeshi Nakatani
15b797f3ee Merge pull request #488 from ggtakec/master 
Fixed searching Content-Length without case sensitive - #480
 2016-10-23 22:42:52 +09:00
Takeshi Nakatani
a7a64d954a Fixed searching Content-Length without case sensitive - #480 2016-10-23 13:27:01 +00:00
Takeshi Nakatani
cca217f613 Merge pull request #487 from driskell/debugging 
Split header debugging onto multiple lines for easier reading
 2016-10-23 21:51:38 +09:00
Takeshi Nakatani
1a9cf6f66d Fixed searching Content-Length without case sensitive - #480 2016-10-23 12:40:51 +00:00
Jason Woods
02d7296210 Split header debugging onto multiple lines for easier reading 2016-10-22 15:11:18 +01:00
Takeshi Nakatani
a688df813e Fixed a bug at read symlink 2016-10-11 13:32:08 +00:00
Takeshi Nakatani
164424bc89 Merge branch 'master' into issue#435 2016-10-11 12:13:03 +00:00
Takeshi Nakatani
7fabd18b1f Trim symbolic link original path in file. 2016-10-11 10:22:30 +00:00
Takeshi Nakatani
5db369d67e Trim symbolic link original path in file. 2016-10-11 10:17:46 +00:00
Takeshi Nakatani
dba32fdf78 Trim symbolic link original path in file. 2016-10-11 10:06:21 +00:00
Takeshi Nakatani
716baada22 Testing patch codes for issue#435 2016-10-10 12:16:09 +00:00
Takeshi Nakatani
9fd1368611 OS-specific correspondence of the extended attribute header 2016-10-02 07:23:19 +00:00
Takeshi Nakatani
65d52506c4 Added use_xattr option for #467 and #460 2016-09-19 04:28:01 +00:00
Takeshi Nakatani
1a96f40a10 Fixed a bug about could not copy file mode from org file 2016-09-11 13:09:23 +00:00
Takeshi Nakatani
1ddc14d59d Changed for accepting mount options compatible with mtab - #449 2016-07-24 08:17:58 +00:00
Peter Watkins
a19206cf0f Accept mount arguments compatible with mtab 
Using "mount -a" fails for already-mounted s3fs directories,
because s3fs mount arguments don't match the form in /etc/mtab.
Calling "mount -a" should quietly succeed when a directory is
already mounted.

To fix this, accept mount commands of the form:

s3fs s3fs /srv/object-store -o bucket=mybucket

or in /etc/fstab form:

s3fs  /srv/object-store  fuse.s3fs  bucket=mybucket 0 0

This matches the form in /etc/mtab and allows "mount -a" to
work properly.
 2016-07-13 17:23:33 -04:00
Takeshi Nakatani
0f9428ad5a Add mirror file logic for removing cache file 2016-07-13 17:23:32 -04:00
Tony Levi
ab4b92074c Fix cppcheck complaint 2016-06-30 11:51:25 +09:30
Tony Levi
d57c12d3c3 Possible fix for leaks during entry expiry / truncation (#340) 2016-06-28 15:01:14 +09:30
Takeshi Nakatani
6005929a96 Handled all curl error without exiting process - #437 2016-06-27 10:38:49 +00:00
Takeshi Nakatani
49ffaa1d94 Merge pull request #432 from andrewgaul/create-bucket-location 
Include location constraint when creating bucket
 2016-06-26 09:39:47 +09:00
Andrew Gaul
320b8e1171 Include location constraint when creating bucket 
This allows creating buckets in non-default regions.  Also improve
do_create_bucket error handling.
 2016-06-13 10:35:37 -07:00
Andrew Gaul
95cb5d201f Correct search and replace typo 2016-06-13 10:25:33 -07:00
Nathaniel W. Turner
584ea488bf Use role name instead of profile name when iam_role=auto 
When using an instance with an IAM Role, transient credentials can be
found in http://169.254.169.254/latest/meta-data/ at
iam/security-credentials/role-name and s3fs tries to do this. However,
it is using the profile-name where role-name is needed. In many cases
the role and profile name are the same, but they are not always.

The simplest way to find the role name appears to be to GET
http://169.254.169.254/latest/meta-data/iam/security-credentials/
itself, which returns a listing of the role names for which temporary
credentials exist. (I think there will probably only be one, but we
probably want to split on newlines and take the first one here in case
that assumption is not valid). This is the approach the AWS SDK appears
to use (based on WireShark analysis).

Bug: https://github.com/s3fs-fuse/s3fs-fuse/issues/421
Signed-off-by: Nathaniel W. Turner <nate@houseofnate.net>
 2016-05-24 13:34:19 -04:00
Nathaniel W. Turner
594c9ca7d2 Skip early credential checks when iam_role=auto 
If user specifies iam_role=auto (or just iam_role), credentials will not
be loaded during early phase, so skip credential checks there.

Signed-off-by: Nathaniel W. Turner <nate@houseofnate.net>
 2016-05-20 12:49:02 -04:00
Takeshi Nakatani
34b604cdfe Fixed a bug about stat_cache_expire - #382 2016-05-14 09:03:52 +00:00
Takeshi Nakatani
50f1ad51c8 loading IAM role name automatically(iam_role option) - #387 2016-05-06 04:37:32 +00:00
Takeshi Nakatani
6cc30eea44 Allow duplicate key in ahbe_conf - #386 2016-05-06 01:08:39 +00:00
Takeshi Nakatani
1ddbd4d6bb Fixed 'load_sse_c' option not working - #388 2016-05-06 00:36:54 +00:00
Takeshi Nakatani
845fdb43f2 Merge pull request #404 from rockuw/keep-alive 
Add curl handler pool to reuse connections
 2016-04-26 23:40:45 +09:00
Tianlong Wu
b78adb4bb0 Add curl handler pool to reuse connections 2016-04-22 14:57:31 +08:00
Tianlong Wu
115bd51f3f Fix a bug of truncating empty file 2016-04-22 14:49:37 +08:00
Takeshi Nakatani
10589a9497 Supported User-Agent header - #383 2016-04-17 07:44:03 +00:00
Takeshi Nakatani
090c37a1c1 Fixed writing sparsed file - #375,#379,#394 2016-04-12 18:24:36 +00:00
Takeshi Nakatani
fff40bbff3 Revert "Fixed a bug about writing sparsed file - #375" 2016-04-13 01:24:24 +09:00
Mattia Rizzolo
136ec654c2
fix typo in curl.cpp: s/returing/returning/ 2016-04-02 15:19:06 +00:00
Takeshi Nakatani
ded4faf2e4 Fixed a bug about writing sparsed file - #375 2016-03-22 05:44:14 +00:00
Takeshi Nakatani
98d55582eb Chnaged about constructor(destructor) in cache.h 2016-03-13 09:47:37 +00:00
Takeshi Nakatani
84bdd51021 Fixed a bug about etag comparison in stats cache. 2016-03-13 09:29:06 +00:00
Takeshi Nakatani
67efc11d94 Always set stats cache for opened file 2016-03-13 05:43:28 +00:00
Takeshi Nakatani
4c65c09f4d Checked content-type by no case-sensitivity - #363 2016-02-13 05:58:59 +00:00
yurykats
230991782b Update s3fs_util.cpp 2016-02-08 16:39:56 -05:00
Takeshi Nakatani
f81e6103cb Fixed codes about clock_gettime for osx(3) 2016-02-07 08:27:02 +00:00
Takeshi Nakatani
0755c6f60c Fixed codes about clock_gettime for osx(2) 2016-02-07 08:10:23 +00:00
Takeshi Nakatani
e01ded9e27 Fixed codes about clock_gettime for osx 2016-02-07 07:40:55 +00:00
Takeshi Nakatani
1af7aaeccb Fixed addhead.cpp for cppcheck 2016-02-07 05:53:56 +00:00
Takeshi Nakatani
c7cf86c2ef Sepalated AdditionalHeader class from curl.* 2016-02-07 05:41:56 +00:00
Takeshi Nakatani
6472eedddc Supported regex type for additional header format. 2016-02-07 05:08:52 +00:00
Takeshi Nakatani
150b83f61e Remove stat file cache dir if specified del_cache - #337 2016-02-06 18:59:13 +00:00
Takeshi Nakatani
c5a94cfc0c Check cache dirctory path and attributes - #347 2016-02-06 13:38:48 +00:00
Takeshi Nakatani
203df6b58a Merge pull request #346 from RobbKistler/empty-dir 
Fix empty directory check against AWS S3
 2016-02-06 18:59:45 +09:00
Takeshi Nakatani
b90b51f2c5 Changed cache out logic for stat - #340 2016-02-06 09:09:17 +00:00
Takeshi Nakatani
8b457133da Merge pull request #341 from hryang/master 
Fix the memory leak issue in fdcache.
 2016-02-06 14:41:28 +09:00
Robb Kistler
4eff6b4dd1 Fix empty directory check against AWS S3 
For ListBucketResult on an empty directory, AWS S3 and S3Proxy 1.4
differ.  AWS will match the directory name, S3Proxy does not.

Changing max-keys=1 to max-keys-2 works for both implementations.
append_objects_from_xml() will swallow the directory key.  The log
level of this message is changed from ERROR to DBG.

Fixes #345
 2016-02-04 23:13:00 -08:00
haoran.yanghr
e3765ad497 Tune the code indent. 2016-01-28 11:16:06 +08:00
haoran.yanghr
dd9f3aed36 Fix the memory leak issue in fdcache. See issue #340 2016-01-28 11:11:53 +08:00
Takeshi Nakatani
83937700dd Fixed a bug about IAMCRED type could not be retried. 2016-01-24 05:01:50 +00:00
Takeshi Nakatani
e932583309 Merge pull request #334 from andrewgaul/bucket-host 
Bucket host should include port and not path
 2016-01-17 14:46:40 +09:00
Andrew Gaul
88a4f04217 Bucket host should include port and not path 
This resolves issues when using v4 signing with path-style requests.
 2016-01-16 15:58:54 -08:00
Andrew Gaul
ff607e1a2d Correct multiple issues with ListBucketRequest 
* provide correct path
* sign query string
* URL encode query string
 2016-01-16 10:17:20 -08:00
Takeshi Nakatani
4bfbfa3621 Merge pull request #331 from andrewgaul/clang 
Address various clang warnings
 2016-01-16 16:14:50 +09:00
Takeshi Nakatani
43b91d3235 Merge pull request #330 from andrewgaul/pass-by-reference 
Pass by const reference where possible
 2016-01-16 16:14:31 +09:00
Andrew Gaul
e003732f18 Address various clang warnings 
Found with:

-Wc++11-extensions
-Wc++11-extra-semi
-Wmissing-variable-declarations
-Wundef
 2016-01-11 00:52:24 -08:00
Andrew Gaul
b946b59522 Pass by const reference where possible 2016-01-10 16:58:24 -08:00
Andrew Gaul
ea6b287d1a Fix v4 signature with use_path_request_style 
Previously s3fs omitted the bucket name when using path request style
causing SignatureDoesNotMatch with v4 signatures.
 2016-01-10 13:41:56 -08:00
Takeshi Nakatani
ea151a70c4 Merge pull request #321 from mcellis33/320 
320: delete stat cache entry in s3fs_fsync so st_size is refreshed
 2015-12-20 15:05:28 +09:00
Mark Ellis
163daa5de1 320: delete stat cache entry in s3fs_fsync so st_size is refreshed 2015-12-18 15:39:25 -08:00
Robb Kistler
b581290c30 Cleanly exit fuse loop on error in s3fs_init 
This allows FUSE to clean the mount point up, preventing
"Transport endpoint not connected" errors on subsequent
access to the mount.
 2015-12-15 15:25:56 -08:00
Takeshi Nakatani
2b3ece467b Merge pull request #311 from RobbKistler/dbg-message 
Change error log to debug log in s3fs_read()
 2015-12-08 00:03:48 +09:00
Robb Kistler
8e688816d4 Change error log to debug log in s3fs_read() 2015-12-03 21:25:27 -08:00
Takeshi Nakatani
759b44135a Check pthread prtability in configure as additional change for #307 2015-12-03 07:47:17 +00:00
Takeshi Nakatani
8b53e0d931 Merge pull request #307 from rockuw/master 
Fix pthread portability problem
 2015-12-03 16:35:30 +09:00
Tianlong Wu
3e655bad3b PTHREAD_MUTEX_RECURSIVE_NP is a enum not macro 2015-12-03 13:44:11 +08:00
Takeshi Nakatani
5e97cb0f48 Changed ensure free disk space as additional change for #306 2015-12-03 05:40:26 +00:00
Tianlong Wu
f44b61c403 Fix pthread portability problem 2015-12-03 10:44:38 +08:00
Guy
6067af6ef1 Fix read concurrency to work in parallel count 
When the prefetch size is limited to the multipart size, the entire parallel logic of the read flow does not have an opportunity to use parallel get.
This fix increases the read performance significantly over our own s3 on-premise solution.
 2015-11-30 18:38:15 +02:00
Takeshi Nakatani
7b62de80f6 Fixed a bug about mtime - #299 2015-11-29 15:53:53 +00:00
Robb Kistler
fff2952d5f Fix syslog level used by S3FS_PRN_EXIT() 
Without converting from s3fs log levels to syslog levels, the syslog
ends up being LOG_EMERG which can cause a broadcast message to all
users.
 2015-11-25 13:53:08 -08:00
Takeshi Nakatani
c673d9d935 File opened with O_TRUNC is not flushed - changed #291 2015-11-08 04:55:17 +00:00
Takeshi Nakatani
0fdda61fb5 Merge pull request #293 from SnakeHunt2012/master 
Fix a small spelling issue.
 2015-11-08 13:26:01 +09:00
Takeshi Nakatani
331b8456a0 Merge pull request #291 from RobbKistler/truncate 
Issue #290: File opened with O_TRUNC is not flushed
 2015-11-08 13:23:06 +09:00
SnakeHunt2012
c04bcce206 Fix a small spelling issue. 2015-11-06 16:49:37 +08:00
Robb Kistler
dd7d9268f2 Force flush in s3fs_open() if file is truncated. 2015-11-03 22:06:25 -08:00
Robb Kistler
e4da5c59b6 Print source file in log messages 2015-11-03 08:34:02 -08:00
Takeshi Nakatani
001206f7c1 Fixed a bug about head request(copy) for SSE - issue#286 2015-11-01 14:05:47 +00:00
Takeshi Nakatani
2ef7f497f6 Fixed a bug about head request(copy) for SSE - issue#286 2015-11-01 13:54:47 +00:00
Takeshi Nakatani
8dd234dd8f Fixed bugs about cppcheck error 2015-10-20 15:47:07 +00:00
Takeshi Nakatani
83d46ef8c6 Fixed bugs about a object larger than free disk space 2015-10-20 15:19:04 +00:00
Takeshi Nakatani
d102eb752d Supported a object which is larger than free disk space 2015-10-18 17:03:41 +00:00
Takeshi Nakatani
f51ad1f33e Supported for SSE KMS 2015-10-06 14:46:14 +00:00
Takeshi Nakatani
92e52dadd4 Changed and cleaned the logic for debug message. 2015-09-30 19:41:27 +00:00
Andrew Gaul
785ed642ba Add support for standard_ia storage class 
This enables storage with lower at-rest prices, higher request prices,
and lower availability.  Also rework existing reduced redundancy
parsing into a more generic storage class.  More background on
standard_ia:

https://aws.amazon.com/blogs/aws/aws-storage-update-new-lower-cost-s3-storage-option-glacier-price-reduction/
 2015-09-17 13:35:25 -07:00
Takeshi Nakatani
ead346c6d3 Merge pull request #261 from andrewgaul/help-timeouts 
Correct help timeouts
 2015-09-13 15:49:15 +09:00