Beginning of s3fs "utility" mode - initially -u option
just reports in progress multipart uploads for the
bucket. Eventually this mode could be used for
other S3 tasks not accessible through typical
file system operations
For multipart upload, use safer mkstemp() instead
of tmpnam() for temporary file
Increased the curl connect and readwrite timeouts
to 10 and 30 seconds respectively.
Autodetect when a big file is being uploaded,
increase the readwrite timeout to 120 seconds. This
was found through experimentation. When uploading
a big file, it is suspected that time is needed
for S3 to assemble the file before it is available
for access. It was found that when a large file
was uploaded via rsync, the final mtime and
chmod modifications were timing out, even though
the upload itself was successful.
Multipart upload is ready for use. A couple of
error checks are still needed in the function and
some cleanup. Need some feedback on how it
is working though.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@298 df820570-a93a-0410-bd06-b72b767a4274
Check issue #142 for details
Code is operational, but not quite ready for
prime time -- needs some clean up
git-svn-id: http://s3fs.googlecode.com/svn/trunk@297 df820570-a93a-0410-bd06-b72b767a4274
Implemented create() function - Resolves issue #18
Issues will be updated with more detail.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@289 df820570-a93a-0410-bd06-b72b767a4274
refactoring easier and the code easier to understand (for me anyway)
Opened up the VERIFY macro so that memory cleanup can be done
before returning from a function.
Make the file descriptor function calls a bit more robust,
check the return codes.
Current code tested on Debian sid, CentOS (with FUSE 2.84) and Ubuntu 10.10
git-svn-id: http://s3fs.googlecode.com/svn/trunk@286 df820570-a93a-0410-bd06-b72b767a4274
No tarball until further testing on other platforms.
Extensively tested on Debian sid 64bit
Resolves issue #104
git-svn-id: http://s3fs.googlecode.com/svn/trunk@285 df820570-a93a-0410-bd06-b72b767a4274
re-wrote the curl write-to-memory callback function
(this helped eliminate a memory leak)
eliminated another memory leak
more debug messages
git-svn-id: http://s3fs.googlecode.com/svn/trunk@281 df820570-a93a-0410-bd06-b72b767a4274
code with CURLE_OK is returned.
clean up a few debug/stdout messages
This commit doesn't really fix anything, but eliminates
the suspicous HTTP 404 errors from the syslog -- these are
usually normal
git-svn-id: http://s3fs.googlecode.com/svn/trunk@280 df820570-a93a-0410-bd06-b72b767a4274
Rather than erroring out, it is treated like the
CURLE_OPERATION_TIMEOUT in that it doesn't exit the
timeout loop while the retry count is > 0. I
also added a short duration sleep to not retry
immediately.
Resolves issue #132
git-svn-id: http://s3fs.googlecode.com/svn/trunk@277 df820570-a93a-0410-bd06-b72b767a4274
return for more specific information as to why the communication
failed. Most common reasons are the "time too skewed" or
credentials failure.
This could be extended to the curl routine that is used
during normal operation. However, the check_service routine
is a good first pass.
Resolves issue #133
git-svn-id: http://s3fs.googlecode.com/svn/trunk@275 df820570-a93a-0410-bd06-b72b767a4274
sure of the usefulness or purpose of this option, but
am including it for completeness
Re-categorized SYS_INFO messages ot SYS_DEBUG messages
to reduce the amount of spew into syslog -- these can
be turned back on with the -d option
Resolves issue #120
git-svn-id: http://s3fs.googlecode.com/svn/trunk@274 df820570-a93a-0410-bd06-b72b767a4274
defined. Added #defines around appropriate code.
Latest revision now compiles and works on CentOS
git-svn-id: http://s3fs.googlecode.com/svn/trunk@271 df820570-a93a-0410-bd06-b72b767a4274
expose the curl compiled with openssl vs. nss issue.
If the issue is seen, emit an informational message
and give the user an option to over-ride checking of
the hostname -- it's recommended not to use bucket
names with periods and https
As implied, added an option ssl_verify_hostname=[0|1]
Tested on fedora 14. Will check on Ubuntu/Debian/CentOS
after check in.
Resolves issue #128
git-svn-id: http://s3fs.googlecode.com/svn/trunk@270 df820570-a93a-0410-bd06-b72b767a4274
trapping the CURLE_SSL_CACERT error and trying to automatically
correct it.
Tested on CentOS 5.5 and ensured that Debian/Ubuntu doesn't break
because of this. This only applies to the https usage.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@269 df820570-a93a-0410-bd06-b72b767a4274
During the service_check() function, if curl returns
a CURLE_SSL_CACERT error then report it and do not
start the s3fs service.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@265 df820570-a93a-0410-bd06-b72b767a4274
rename (mv) a directory, its contents "disappear". To
get the contents back, just re-create the directory.
This issue has led to more than one support issues.
This patch disallows renaming of directories and
returns a operation not supported message, e.g.
% mv dir1 dir2
mv: cannot move `dir1' to `dir2': Operation not supported
Another observation is that when a directory was renamed
it loses its directory status and becomes a normal file.
This is a first step to allow for supporting directory
renames.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@260 df820570-a93a-0410-bd06-b72b767a4274
is not present/readable.
Don't process blank lines in mime.types
There wasn't an issue seen with this, just
cleaning up the code a bit.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@245 df820570-a93a-0410-bd06-b72b767a4274
Resolves issue #88
Specifying credentials on the command line is no longer
supported. There are several other ways to do this now.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@243 df820570-a93a-0410-bd06-b72b767a4274
with the conversion to get_opt
-f is passed along to FUSE. This makes FUSE run
in foreground (non-daemon) mode and some debugging
messages now appear on STDOUT
git-svn-id: http://s3fs.googlecode.com/svn/trunk@240 df820570-a93a-0410-bd06-b72b767a4274
all work from the copy of source files up on directory
...and the history is still there.
Not deleting the directories yet because need to
preserve svn properties -- not sure how to copy
vs. recreate -- need to look it up
On the todo list: remove s3fs, s3fs/src and s3fs/test
git-svn-id: http://s3fs.googlecode.com/svn/trunk@237 df820570-a93a-0410-bd06-b72b767a4274
directory from the trunk directory.
First do a svn cp of all of the source up to
trunk. This is supposed to preserve change
history -- we'll see.
The source remains untouched until this gets
worked out.
Also in preparation of bringing in the source
collateral for the debian package into the
repository. I expect that the top level will
look like this:
svn/
s3fs/
trunk/
tags/
branches/
dpkg/
trunk/
tags/
branches/
So far that's how it is looking. I'll be
very careful to ensure integrity of the data.
As a result this may be a multistep process.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@236 df820570-a93a-0410-bd06-b72b767a4274
Sending DEBUG messages to syslog is off by default.
-d (or --debug) enables DEBUG messages to go to
syslog
Additionally, if an additional -d command line option
is given, then the -d option is passed to FUSE, upon
which FUSE outputs debug messages to STDOUT
resolves issue #120
git-svn-id: http://s3fs.googlecode.com/svn/trunk@235 df820570-a93a-0410-bd06-b72b767a4274
- The script was looking for /root/.passwd-s3fs on my debian
machine (it worked correctly on Ubuntu). Fixed the common.sh
file to always look in the SUDO_USER's home directory
- Added the passwd_file option to the s3fs command. Since the
test runs as root, the $SUDO_USER's password file wasn't
being looked for by the program itself. Instead the normal
precedence was being followed and the /etc/passwd-s3fs file
was being used.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@234 df820570-a93a-0410-bd06-b72b767a4274
If any password file is used, regardless if it is specified
on the command line, ~/.passwd-s3fs or /etc/passwd-s3fs it
is checked for appropriate permissions.
No password file is allowed to have any others permissions
Only the /etc/passwd-s3fs file is allowed to have any
group permissions, all others are not allowed to have
any group permissions.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@233 df820570-a93a-0410-bd06-b72b767a4274
The correct way to solve this is to consistently faily when mounting. Also offer a flag to forcefully create the bucket if it doesn't yet exist.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@231 df820570-a93a-0410-bd06-b72b767a4274
To run integration tests, just use: sudo make check
This will give you a report of all passes and failures.
To add an integration test, just add it to the TESTS variable in s3fs/test/Makefile.am
Make sure your test sources integration-test-common.sh and require-root.sh
Having many of these for every feature will help us avoid regressions and develop with confidence.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@230 df820570-a93a-0410-bd06-b72b767a4274
Add support for mulitple access id/secret access keys in passwd-s3fs file
This change applies to any passwd-s3fs files.
The format of the file is more robust, error checked and extended:
- as before, any line beginning with # is ignored
- any empty line is ignored
- any non-ignored line which contains a space or tab is an error
- any non-ignored line which does not contain a : separator is an error
The format of the file is:
[bucket:]AccessKeyId:SecretAccessKey
The bucket can now be specified to allow for multiple credentials.
A default entry is as before:
AccessKeyId:SecretAccessKey
Only one default entry is allowed, if more than one default entry
is found, that is an error. A default entry is not required, if the
bucket that is being mounted has its own entry.
If the user's .passwd-s3fs file is present but credentials cannot
be determined from it, then the system-wide /etc/passwd-s3fs will
be consulted (if readable by the current user).
This change is completely backward compatable with the existing
scheme and has been well tested.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@229 df820570-a93a-0410-bd06-b72b767a4274
Issue #102 --help command line
Issue #107 command line checking and other error checking
Issue #112 uses getopt_long now, but FUSE's own parser is still there
some command line checking is split between main() and the
helper function
Issue #117 command line option for credentials file
Issue #118 use of environment variables for credentials
Issue #119 support for a local passwd-s3fs file
Big checkin, many new lines of code.
Comand line option checking is now more robust and does quite
a bit of error checking before handing the mount off to FUSE.
Cleaned up the code and made several new functions from
existing code
Many ways to supply credentials now and there is a precedance
order followed.
--help is helpful
git-svn-id: http://s3fs.googlecode.com/svn/trunk@227 df820570-a93a-0410-bd06-b72b767a4274
If the "public_bucket=1" option is used, then
the command line options for Access Key ID and
Secret Access Key are ignored as well as the
/etc/passwd-s3fs file
Internally, the "Authorization: AWS ..." header
line is not included in the header.
Tested on a public bucket and it appears to work.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@221 df820570-a93a-0410-bd06-b72b767a4274
=========================
Removed extra space in the message emitted by --version
M s3fs/src/Makefile.am
==========================
Added the .h files to the s3fs_SOURCES. These files
were not being included in the .tar.gz file when
"make dist" was being executed. This was sourced
as an error with the "make distcheck" target.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@217 df820570-a93a-0410-bd06-b72b767a4274
