Commit Graph

74 Commits

Author SHA1 Message Date
mooredan@suncup.net
850d7b7d47 Increment version number
git-svn-id: http://s3fs.googlecode.com/svn/trunk@305 df820570-a93a-0410-bd06-b72b767a4274
2011-01-21 17:11:19 +00:00
mooredan@suncup.net
8e4c89fdec Checkpoint for large file (> 2GB) upload capability
needs more testing.


git-svn-id: http://s3fs.googlecode.com/svn/trunk@302 df820570-a93a-0410-bd06-b72b767a4274
2011-01-20 22:40:59 +00:00
mooredan@suncup.net
3d9c255ba2 Release of 1.33
git-svn-id: http://s3fs.googlecode.com/svn/trunk@300 df820570-a93a-0410-bd06-b72b767a4274
2010-12-30 19:17:18 +00:00
mooredan@suncup.net
f94bbd70f9 Cleaned up compile time warnings as reported by -Wall
Beginning of s3fs "utility" mode - initially -u option
just reports in progress multipart uploads for the
bucket. Eventually this mode could be used for
other S3 tasks not accessible through typical
file system operations

For multipart upload, use safer mkstemp() instead
of tmpnam() for temporary file

Increased the curl connect and readwrite timeouts
to 10 and 30 seconds respectively.

Autodetect when a big file is being uploaded,
increase the readwrite timeout to 120 seconds. This
was found through experimentation.  When uploading
a big file, it is suspected that time is needed
for S3 to assemble the file before it is available
for access. It was found that when a large file
was uploaded via rsync, the final mtime and
chmod modifications were timing out, even though
the upload itself was successful.


Multipart upload is ready for use. A couple of
error checks are still needed in the function and
some cleanup.  Need some feedback on how it
is working though.




git-svn-id: http://s3fs.googlecode.com/svn/trunk@298 df820570-a93a-0410-bd06-b72b767a4274
2010-12-30 03:13:21 +00:00
mooredan@suncup.net
acc7363433 Checkpoint for implementation of multipart upload
Check issue #142 for details

Code is operational, but not quite ready for
prime time -- needs some clean up


git-svn-id: http://s3fs.googlecode.com/svn/trunk@297 df820570-a93a-0410-bd06-b72b767a4274
2010-12-28 04:15:23 +00:00
mooredan@suncup.net
784d51d805 separated out a common function to mknod and create
git-svn-id: http://s3fs.googlecode.com/svn/trunk@290 df820570-a93a-0410-bd06-b72b767a4274
2010-12-22 17:19:52 +00:00
mooredan@suncup.net
5c64ff83cf Restructing to take care of the directory rename.
This is a checkpoint. No functional changes in this commit.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@288 df820570-a93a-0410-bd06-b72b767a4274
2010-12-20 05:26:27 +00:00
mooredan@suncup.net
68774b5960 Added code to address the potential short write
associated with BIO_write

Resolves issue #6


git-svn-id: http://s3fs.googlecode.com/svn/trunk@287 df820570-a93a-0410-bd06-b72b767a4274
2010-12-20 00:06:56 +00:00
mooredan@suncup.net
90ee6b8f9b Some more unwinding of the C++ classes, should make
refactoring easier and the code easier to understand (for me anyway)

Opened up the VERIFY macro so that memory cleanup can be done
before returning from a function.

Make the file descriptor function calls a bit more robust,
check the return codes.

Current code tested on Debian sid, CentOS (with FUSE 2.84) and Ubuntu 10.10




git-svn-id: http://s3fs.googlecode.com/svn/trunk@286 df820570-a93a-0410-bd06-b72b767a4274
2010-12-19 22:27:56 +00:00
mooredan@suncup.net
f56b95f11e Fixed memory leak issues as outlined in issue #104
No tarball until further testing on other platforms.
Extensively tested on Debian sid 64bit

Resolves issue #104


git-svn-id: http://s3fs.googlecode.com/svn/trunk@285 df820570-a93a-0410-bd06-b72b767a4274
2010-12-19 01:34:27 +00:00
mooredan@suncup.net
147dd86215 Minimum FUSE version is now 2.8.4 !
re-wrote the curl write-to-memory callback function
(this helped eliminate a memory leak)

eliminated another memory leak

more debug messages



git-svn-id: http://s3fs.googlecode.com/svn/trunk@281 df820570-a93a-0410-bd06-b72b767a4274
2010-12-17 04:40:15 +00:00
mooredan@suncup.net
dfd6d6c1b6 Turn off CURL_FAILONERROR and parse the HTTP return
code with CURLE_OK is returned.

clean up a few debug/stdout messages

This commit doesn't really fix anything, but eliminates
the suspicous HTTP 404 errors from the syslog -- these are
usually normal



git-svn-id: http://s3fs.googlecode.com/svn/trunk@280 df820570-a93a-0410-bd06-b72b767a4274
2010-12-11 04:42:52 +00:00
mooredan@suncup.net
9c6d671fec Service another curl error appropriately
do not output via cout unless we are in "foreground" mode


git-svn-id: http://s3fs.googlecode.com/svn/trunk@279 df820570-a93a-0410-bd06-b72b767a4274
2010-12-09 20:56:29 +00:00
mooredan@suncup.net
d3c42255b9 Handle a couple of more specific curl errors.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@278 df820570-a93a-0410-bd06-b72b767a4274
2010-12-09 02:59:49 +00:00
mooredan@suncup.net
7358b3512e Now servicing the CURLE_COULDNT_CONNECT error better.
Rather than erroring out, it is treated like the
CURLE_OPERATION_TIMEOUT in that it doesn't exit the
timeout loop while the retry count is > 0.  I
also added a short duration sleep to not retry
immediately.

Resolves issue #132


git-svn-id: http://s3fs.googlecode.com/svn/trunk@277 df820570-a93a-0410-bd06-b72b767a4274
2010-12-08 04:52:19 +00:00
mooredan@suncup.net
412afb6953 During the check_service function, parse a unsuccessful HTTP
return for more specific information as to why the communication
failed. Most common reasons are the "time too skewed" or
credentials failure.

This could be extended to the curl routine that is used
during normal operation. However, the check_service routine
is a good first pass.


Resolves issue #133


git-svn-id: http://s3fs.googlecode.com/svn/trunk@275 df820570-a93a-0410-bd06-b72b767a4274
2010-12-08 02:39:13 +00:00
mooredan@suncup.net
8b10de5559 Added an additional check in check_service to
expose the curl compiled with openssl vs. nss issue.

If the issue is seen, emit an informational message
and give the user an option to over-ride checking of
the hostname -- it's recommended not to use bucket
names with periods and https

As implied, added an option ssl_verify_hostname=[0|1]

Tested on fedora 14. Will check on Ubuntu/Debian/CentOS
after check in.

Resolves issue #128



git-svn-id: http://s3fs.googlecode.com/svn/trunk@270 df820570-a93a-0410-bd06-b72b767a4274
2010-11-26 22:11:48 +00:00
mooredan@suncup.net
49c3687a52 This is a fix for Issue #125. Add a more robust way of
trapping the CURLE_SSL_CACERT error and trying to automatically
correct it. 

Tested on CentOS 5.5 and ensured that Debian/Ubuntu doesn't break
because of this. This only applies to the https usage.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@269 df820570-a93a-0410-bd06-b72b767a4274
2010-11-26 03:13:53 +00:00
mooredan@suncup.net
ded2dd527d Added check for left square bracket character at the
beginning of the line in the password file.

Resolves issue #126


git-svn-id: http://s3fs.googlecode.com/svn/trunk@263 df820570-a93a-0410-bd06-b72b767a4274
2010-11-24 02:44:15 +00:00
mooredan@suncup.net
4187f03089 Don't try to read from /etc/mime.types if it
is not present/readable.

Don't process blank lines in mime.types

There wasn't an issue seen with this, just
cleaning up the code a bit.


git-svn-id: http://s3fs.googlecode.com/svn/trunk@245 df820570-a93a-0410-bd06-b72b767a4274
2010-11-22 18:28:07 +00:00
mooredan@suncup.net
7429227922 Added a couple of casts to take care of compile warnings
Resolves issue #88

Specifying credentials on the command line is no longer
supported. There are several other ways to do this now.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@243 df820570-a93a-0410-bd06-b72b767a4274
2010-11-21 01:30:25 +00:00
mooredan@suncup.net
d239780a29 Added check for preliminary checking of credentials
and if the bucket exists on the amazon server.

Resolves issue #121


git-svn-id: http://s3fs.googlecode.com/svn/trunk@241 df820570-a93a-0410-bd06-b72b767a4274
2010-11-20 17:55:15 +00:00
mooredan@suncup.net
65e0a2ff84 Re-enable the -f option for FUSE. (This got disabled
with the conversion to get_opt

-f is passed along to FUSE. This makes FUSE run
in foreground (non-daemon) mode and some debugging
messages now appear on STDOUT



git-svn-id: http://s3fs.googlecode.com/svn/trunk@240 df820570-a93a-0410-bd06-b72b767a4274
2010-11-19 22:23:38 +00:00
mooredan@suncup.net
e9b8216d21 In preparation to remove the unnecessary "s3fs"
directory from the trunk directory.

First do a svn cp of all of the source up to
trunk.  This is supposed to preserve change
history -- we'll see.

The source remains untouched until this gets
worked out.

Also in preparation of bringing in the source
collateral for the debian package into the
repository. I expect that the top level will
look like this:

svn/
   s3fs/
      trunk/
      tags/
      branches/
   dpkg/
      trunk/
      tags/
      branches/


So far that's how it is looking.  I'll be
very careful to ensure integrity of the data.
As a result this may be a multistep process.



git-svn-id: http://s3fs.googlecode.com/svn/trunk@236 df820570-a93a-0410-bd06-b72b767a4274
2010-11-13 23:59:23 +00:00