octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-14 08:24:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
679 Commits 1 Branch 23 Tags 12 MiB
b979d40778
Commit Graph

679 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
ben.lemasurier@gmail.com
f29986f4c6 Simplified s3fs_check_service 
simply using strstr to see if the bucket is available.
removed the check to see if the service has any buckets at all.


git-svn-id: http://s3fs.googlecode.com/svn/trunk@360 df820570-a93a-0410-bd06-b72b767a4274
 2011-08-25 16:34:10 +00:00
ben.lemasurier@gmail.com
5a6ef54b36 trivial change to fix some valgrind warnings regarding unititialized values. 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@359 df820570-a93a-0410-bd06-b72b767a4274
 2011-08-02 15:08:28 +00:00
ben.lemasurier@gmail.com
b41f982d59 Fixes issue #205 
when s3fs is linked against libcurl < 7.20.0 we'd receive CURLM_CALL_MULTI_PERFORM



git-svn-id: http://s3fs.googlecode.com/svn/trunk@358 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-29 20:03:38 +00:00
ben.lemasurier@gmail.com
07baba972a Handle curl send and recv errors a little more gracefully 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@357 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-29 15:48:15 +00:00
ben.lemasurier@gmail.com
824431712e s3fs-1.59 release 
- fixes issue #206



git-svn-id: http://s3fs.googlecode.com/svn/trunk@356 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-28 19:46:14 +00:00
ben.lemasurier@gmail.com
40fde1e0ea Possible fix for issue #206 
- handle bad xml data from Amazon



git-svn-id: http://s3fs.googlecode.com/svn/trunk@355 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-28 15:43:52 +00:00
mooredan@suncup.net
b54b2ddfda Fixed issue #210 
Applied user supplied patch


git-svn-id: http://s3fs.googlecode.com/svn/trunk@353 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-19 19:52:38 +00:00
ben.lemasurier@gmail.com
f484ea00ae Fixes issue #172 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@352 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-07 22:09:40 +00:00
ben.lemasurier@gmail.com
a16617a5d5 bump version for bugfix (#205) 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@351 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-07 16:26:07 +00:00
ben.lemasurier@gmail.com
913fee101e fix for issue #205 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@350 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-07 16:25:04 +00:00
ben.lemasurier@gmail.com
ee1915ff93 missed this on the last commit 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@349 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-02 18:52:44 +00:00
ben.lemasurier@gmail.com
2eafa487d7 Massive speed improvements for readdir operations 
complete s3fs_readdir() refactor
    - multi interface now batches HTTP requests
      - proper HTTP KeepAlive sessions are back! (CURLOPT_FORBID_REUSE is no longer required)
    - use xpath to quickly grab xml nodes
    - lots of cleanup
    - fixes some strange stat cache behavior
    - huge readdir performance benefits (8-14x in my case) on large directories



git-svn-id: http://s3fs.googlecode.com/svn/trunk@348 df820570-a93a-0410-bd06-b72b767a4274
 2011-07-02 02:11:54 +00:00
ben.lemasurier@gmail.com
1a6885359c Ability to disable multipart uploads 
- Added 'nomultipart' option to disable multipart uploads
  - see issue #181


git-svn-id: http://s3fs.googlecode.com/svn/trunk@347 df820570-a93a-0410-bd06-b72b767a4274
 2011-06-27 02:21:38 +00:00
ben.lemasurier@gmail.com
6db8dafca4 bump version number 
- removed debugging line


git-svn-id: http://s3fs.googlecode.com/svn/trunk@346 df820570-a93a-0410-bd06-b72b767a4274
 2011-06-26 00:42:45 +00:00
ben.lemasurier@gmail.com
1597dfe659 Support for mounting a remote directory (issue #7). 
- fixed a bug in the file cache, it was attempting to set the mtime
    on symlinks
  - general code cleanup; moved some string functions to string_util.cpp


git-svn-id: http://s3fs.googlecode.com/svn/trunk@345 df820570-a93a-0410-bd06-b72b767a4274
 2011-06-26 00:37:52 +00:00
ben.lemasurier@gmail.com
94d72bdaf3 trivial compiler warning fix. 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@344 df820570-a93a-0410-bd06-b72b767a4274
 2011-06-22 22:32:08 +00:00
mooredan@suncup.net
30abdff53f resolves issue #155 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@340 df820570-a93a-0410-bd06-b72b767a4274
 2011-04-06 15:06:28 +00:00
ben.lemasurier@gmail.com
2e09e5201b fixes issue #23 and issue #160. validate the file cache by comparing the local/remote size/mtime values instead of an md5 sum 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@338 df820570-a93a-0410-bd06-b72b767a4274
 2011-03-10 00:11:55 +00:00
mooredan@suncup.net
ebdde932a6 Strip the trailing '/' characters off of the url= option's 
argument

Resolves issue #171


git-svn-id: http://s3fs.googlecode.com/svn/trunk@336 df820570-a93a-0410-bd06-b72b767a4274
 2011-03-09 20:55:31 +00:00
ben.lemasurier@gmail.com
6cd9e9e65d moved generic curl routines to their own file 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@332 df820570-a93a-0410-bd06-b72b767a4274
 2011-03-01 19:35:55 +00:00
ben.lemasurier@gmail.com
60f96aa8fb header file inclusion required for make dist-gzip/zip/etc 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@331 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-28 20:10:40 +00:00
mooredan@suncup.net
c16925bb10 Added check to check for illegal characters in bucket name 
Resolves issue #163



git-svn-id: http://s3fs.googlecode.com/svn/trunk@330 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-26 14:48:02 +00:00
mooredan@suncup.net
2fe1abc66b First attempt to resolve issue 161 -- added handler for 
curl error code 23 - CURLE_WRITE_ERROR

When encountered, it does a retry.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@329 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-26 14:11:46 +00:00
ben.lemasurier@gmail.com
48d1a73e06 cleanup only; s3fs.cpp is getting huge, split caching to its own file 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@328 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-25 17:35:12 +00:00
ben.lemasurier@gmail.com
ec4c5b3627 reduced the number of keys returned to see if a directory is empty, we only need to know if 1 exists. Minor cleanups. 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@327 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-23 17:16:12 +00:00
ben.lemasurier@gmail.com
6c89527aed minor cleanup, no functional changes. 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@326 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-23 16:03:08 +00:00
ben.lemasurier@gmail.com
c07e27eff1 only delete stat cache entries when file could have been modified. 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@324 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-22 23:01:42 +00:00
ben.lemasurier@gmail.com
9b5f694497 update to r322, this only works when local file cache is turned on. Initial 'make check' erroneously passed 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@323 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-22 22:28:57 +00:00
ben.lemasurier@gmail.com
0fb4427444 Fixes issues #134, "double upload". This _should_ result in a large performance improvment. 
- s3fs_flush() now checks to see whether the file on the remote end is the same as the local copy.
  - md5sum() now requires a file descriptor instead of a path.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@322 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-22 21:28:01 +00:00
apetresc@gmail.com
fe47e37a98 Adding a dist-hook to remove .svn directories from things in EXTRA_DIST. 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@321 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-21 19:56:00 +00:00
ben.lemasurier@gmail.com
1496f6a81e Further multipart cleanup/error checking, in preparation for multi-threaded uploads. 
"Last-Modified" is now returned with get_headers() data


git-svn-id: http://s3fs.googlecode.com/svn/trunk@320 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-17 17:31:43 +00:00
ben.lemasurier@gmail.com
cfa0fd2992 clean up get_local_fd() to use md5sum() 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@319 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-16 16:52:45 +00:00
ben.lemasurier@gmail.com
00bde54d0a A large amount of cleanup for multipart uploads; preparation work for upcoming multi-threaded upload support. 
Functional changes are limited to the multipart upload process. Each uploaded part is now verified against a local md5sum.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@318 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-15 23:32:27 +00:00
ben.lemasurier@gmail.com
1a79d451c5 Fixes an issue reintroduced in r315: s3fs_readdir was not populating the file stat cache 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@317 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-14 18:54:30 +00:00
mooredan@suncup.net
ecaaf4d324 Implemented max_stat_cache_size as an option 
Resolves issue #157



git-svn-id: http://s3fs.googlecode.com/svn/trunk@316 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-12 16:48:23 +00:00
mooredan@suncup.net
6a3a68b01c Bound the size of stat_cache as described in issue #157 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@315 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-12 15:02:44 +00:00
apetresc
04217e2cff Committing Ben's man page with some minor fixes and edits. Made sure to include it in the distribution tarball. 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@313 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-11 20:57:44 +00:00
mooredan@suncup.net
2c0456680e Resolves issue #156 
s3fs_readdir() now looks at stat_cache


git-svn-id: http://s3fs.googlecode.com/svn/trunk@312 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-11 03:52:31 +00:00
mooredan@suncup.net
c8d5b35f8f Resolves issue #154 
Installed and tested fix for file permissions/cache issue


git-svn-id: http://s3fs.googlecode.com/svn/trunk@311 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-11 03:30:02 +00:00
mooredan@suncup.net
6f7e180133 Resolves issue #152 
- added move directory test
- fix bug introduced with fixing issue #150



git-svn-id: http://s3fs.googlecode.com/svn/trunk@310 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-10 01:07:46 +00:00
apetresc
fd2a6c120a Committing patch contributed by Ben LeMasurier to cache directory attributes. Details and tests on Issue 150. 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@309 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-08 18:23:38 +00:00
mooredan@suncup.net
0a233011a5 potential fix for issue 148 
- increase max_keys in readdir from 50 to 500
- handle the curle_couldnt_resolve_host error better
- add the curl forbid reuse option



git-svn-id: http://s3fs.googlecode.com/svn/trunk@308 df820570-a93a-0410-bd06-b72b767a4274
 2011-02-05 01:35:18 +00:00
mooredan@suncup.net
850d7b7d47 Increment version number 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@305 df820570-a93a-0410-bd06-b72b767a4274
 2011-01-21 17:11:19 +00:00
mooredan@suncup.net
b3726191b2 Clean up a couple of compile warnings seen in CentOS 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@304 df820570-a93a-0410-bd06-b72b767a4274
 2011-01-21 01:12:09 +00:00
mooredan@suncup.net
a5e7745589 Another checkpoint 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@303 df820570-a93a-0410-bd06-b72b767a4274
 2011-01-21 00:50:55 +00:00
mooredan@suncup.net
8e4c89fdec Checkpoint for large file (> 2GB) upload capability 
needs more testing.


git-svn-id: http://s3fs.googlecode.com/svn/trunk@302 df820570-a93a-0410-bd06-b72b767a4274
 2011-01-20 22:40:59 +00:00
mooredan@suncup.net
0f18298886 Fix for issue #145 and additional tests in make check 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@301 df820570-a93a-0410-bd06-b72b767a4274
 2011-01-19 05:26:01 +00:00
mooredan@suncup.net
3d9c255ba2 Release of 1.33 
git-svn-id: http://s3fs.googlecode.com/svn/trunk@300 df820570-a93a-0410-bd06-b72b767a4274
 2010-12-30 19:17:18 +00:00
mooredan@suncup.net
f94bbd70f9 Cleaned up compile time warnings as reported by -Wall 
Beginning of s3fs "utility" mode - initially -u option
just reports in progress multipart uploads for the
bucket. Eventually this mode could be used for
other S3 tasks not accessible through typical
file system operations

For multipart upload, use safer mkstemp() instead
of tmpnam() for temporary file

Increased the curl connect and readwrite timeouts
to 10 and 30 seconds respectively.

Autodetect when a big file is being uploaded,
increase the readwrite timeout to 120 seconds. This
was found through experimentation.  When uploading
a big file, it is suspected that time is needed
for S3 to assemble the file before it is available
for access. It was found that when a large file
was uploaded via rsync, the final mtime and
chmod modifications were timing out, even though
the upload itself was successful.


Multipart upload is ready for use. A couple of
error checks are still needed in the function and
some cleanup.  Need some feedback on how it
is working though.




git-svn-id: http://s3fs.googlecode.com/svn/trunk@298 df820570-a93a-0410-bd06-b72b767a4274
 2010-12-30 03:13:21 +00:00
mooredan@suncup.net
acc7363433 Checkpoint for implementation of multipart upload 
Check issue #142 for details

Code is operational, but not quite ready for
prime time -- needs some clean up


git-svn-id: http://s3fs.googlecode.com/svn/trunk@297 df820570-a93a-0410-bd06-b72b767a4274
 2010-12-28 04:15:23 +00:00