mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2024-06-07 03:20:50 +00:00
Fixed Issue 229 and Changes codes
1) Set metadata "Content-Encoding" automatically(Issue 292)
For this issue, s3fs is added new option "ahbe_conf".
New option means the configuration file path, and this file specifies
additional HTTP header by file(object) extension.
Thus you can specify any HTTP header for each object by extension.
* ahbe_conf file format:
-----------
line = [file suffix] HTTP-header [HTTP-header-values]
file suffix = file(object) suffix, if this field is empty,
it means "*"(all object).
HTTP-header = additional HTTP header name
HTTP-header-values = additional HTTP header value
-----------
* Example:
-----------
.gz Content-Encoding gzip
.Z Content-Encoding compress
X-S3FS-MYHTTPHEAD myvalue
-----------
A sample configuration file is uploaded in "test" directory.
If ahbe_conf parameter is specified, s3fs loads it's configuration
and compares extension(suffix) of object(file) when uploading
(PUT/POST) it. If the extension is same, s3fs adds/sends specified
HTTP header and value.
A case of sample configuration file, if a object(it's extension is
".gz") which already has Content-Encoding HTTP header is renamed
to ".txt" extension, s3fs does not set Content-Encoding. Because
".txt" is not match any line in configuration file.
So, s3fs matches the extension by each PUT/POST action.
* Please take care about "Content-Encoding".
This new option allows setting ANY HTTP header by object extension.
For example, you can specify "Content-Encoding" for ".gz"/etc
extension in configuration. But this means that S3 always returns
"Content-Encoding: gzip" when a client requests with other
"Accept-Encoding:" header. It SHOULD NOT be good.
Please see RFC 2616.
2) Changes about allow_other/uid/gid option for mount point
I reviewed about mount point permission and allow_other/uid/gid
options, and found bugs about these.
s3fs is fixed bugs and changed to the following specifications.
* s3fs only allows uid(gid) options as 0(root), when the effective
user is zero(root).
* A mount point(directory) must have a permission to allow
accessing by effective user/group.
* If allow_other option is specified, the mount point permission
is set 0777(all users allow all access).
In another case, the mount point is set 0700(only allows
effective user).
* When uid/gid option is specified, the mount point owner/group
is set uid/gid option value.
If uid/gid is not set, it is set effective user/group id.
This changes maybe fixes some issue(321, 338).
3) Changes a logic about (Issue 229)
The chmod command returns -EIO when changing the mount point.
It is correct, s3fs can not changed owner/group/mtime for the
mount point, but s3fs sends a request for changing the bucket.
This revision does not send the request, and returns EIO as
soon as possible.
git-svn-id: http://s3fs.googlecode.com/svn/trunk@465 df820570-a93a-0410-bd06-b72b767a4274
This commit is contained in:
ggtakec@gmail.com
parent
2361c83f69
commit
d7689151ab
|
|
@ -75,6 +75,24 @@ this option can not be specified with use_rrs.
|
|||
\fB\-o\fR passwd_file (default="")
|
||||
specify the path to the password file, which which takes precedence over the password in $HOME/.passwd-s3fs and /etc/passwd-s3fs
|
||||
.TP
|
||||
\fB\-o\fR ahbe_conf (default="" which means disabled)
|
||||
This option specifies the configuration file path which file is the additional HTTP header by file(object) extension.
|
||||
The configuration file format is below:
|
||||
-----------
|
||||
line = [file suffix] HTTP-header [HTTP-values]
|
||||
file suffix = file(object) suffix, if this field is empty, it means "*"(all object).
|
||||
HTTP-header = additional HTTP header name
|
||||
HTTP-values = additional HTTP header value
|
||||
-----------
|
||||
Sample:
|
||||
-----------
|
||||
.gz Content-Encoding gzip
|
||||
.Z Content-Encoding compress
|
||||
X-S3FS-MYHTTPHEAD myvalue
|
||||
-----------
|
||||
A sample configuration file is uploaded in "test" directory.
|
||||
If you specify this option for set "Content-Encoding" HTTP header, please take care for RFC 2616.
|
||||
.TP
|
||||
\fB\-o\fR public_bucket (default="" which means disabled)
|
||||
anonymously mount a public bucket when set to 1, ignores the $HOME/.passwd-s3fs and /etc/passwd-s3fs files.
|
||||
.TP
|
||||
|
|
|
|||
221
src/curl.cpp
221
src/curl.cpp
|
|
@ -28,6 +28,7 @@
|
|||
#include <errno.h>
|
||||
#include <syslog.h>
|
||||
#include <pthread.h>
|
||||
#include <assert.h>
|
||||
#include <curl/curl.h>
|
||||
#include <openssl/bio.h>
|
||||
#include <openssl/buffer.h>
|
||||
|
|
@ -700,7 +701,7 @@ S3fsCurl* S3fsCurl::UploadMultipartPostRetryCallback(S3fsCurl* s3fscurl)
|
|||
part_num = atoi(part_num_str.c_str());
|
||||
|
||||
// duplicate request
|
||||
S3fsCurl* newcurl = new S3fsCurl();
|
||||
S3fsCurl* newcurl = new S3fsCurl(s3fscurl->IsUseAhbe());
|
||||
newcurl->partdata.etaglist = s3fscurl->partdata.etaglist;
|
||||
newcurl->partdata.etagpos = s3fscurl->partdata.etagpos;
|
||||
newcurl->partdata.fd = s3fscurl->partdata.fd;
|
||||
|
|
@ -726,7 +727,7 @@ int S3fsCurl::ParallelMultipartUploadRequest(const char* tpath, headers_t& meta,
|
|||
etaglist_t list;
|
||||
off_t remaining_bytes;
|
||||
unsigned char* buf;
|
||||
S3fsCurl s3fscurl;
|
||||
S3fsCurl s3fscurl(true);
|
||||
|
||||
FPRNNN("[tpath=%s][fd=%d]", SAFESTRPTR(tpath), fd);
|
||||
|
||||
|
|
@ -775,7 +776,7 @@ int S3fsCurl::ParallelMultipartUploadRequest(const char* tpath, headers_t& meta,
|
|||
chunk = remaining_bytes > MULTIPART_SIZE ? MULTIPART_SIZE : remaining_bytes;
|
||||
|
||||
// s3fscurl sub object
|
||||
S3fsCurl* s3fscurl_para = new S3fsCurl();
|
||||
S3fsCurl* s3fscurl_para = new S3fsCurl(true);
|
||||
s3fscurl_para->partdata.fd = fd2;
|
||||
s3fscurl_para->partdata.startpos = st.st_size - remaining_bytes;
|
||||
s3fscurl_para->partdata.size = chunk;
|
||||
|
|
@ -826,7 +827,7 @@ S3fsCurl* S3fsCurl::ParallelGetObjectRetryCallback(S3fsCurl* s3fscurl)
|
|||
return NULL;
|
||||
}
|
||||
// duplicate request(setup new curl object)
|
||||
S3fsCurl* newcurl = new S3fsCurl();
|
||||
S3fsCurl* newcurl = new S3fsCurl(s3fscurl->IsUseAhbe());
|
||||
if(0 != (result = newcurl->PreGetObjectRequest(
|
||||
s3fscurl->path.c_str(), s3fscurl->partdata.fd, s3fscurl->partdata.startpos, s3fscurl->partdata.size))){
|
||||
DPRN("failed downloading part setup(%d)", result);
|
||||
|
|
@ -889,9 +890,9 @@ int S3fsCurl::ParallelGetObjectRequest(const char* tpath, int fd, off_t start, s
|
|||
//-------------------------------------------------------------------
|
||||
// Methods for S3fsCurl
|
||||
//-------------------------------------------------------------------
|
||||
S3fsCurl::S3fsCurl() :
|
||||
hCurl(NULL), path(""), base_path(""), saved_path(""), url(""), requestHeaders(NULL),
|
||||
bodydata(NULL), headdata(NULL), LastResponseCode(-1), postdata(NULL), postdata_remaining(0)
|
||||
S3fsCurl::S3fsCurl(bool ahbe) :
|
||||
hCurl(NULL), path(""), base_path(""), saved_path(""), url(""), requestHeaders(NULL), bodydata(NULL),
|
||||
headdata(NULL), LastResponseCode(-1), postdata(NULL), postdata_remaining(0), is_use_ahbe(ahbe)
|
||||
{
|
||||
}
|
||||
|
||||
|
|
@ -993,6 +994,13 @@ bool S3fsCurl::ClearInternalData(void)
|
|||
return true;
|
||||
}
|
||||
|
||||
bool S3fsCurl::SetUseAhbe(bool ahbe)
|
||||
{
|
||||
bool old = is_use_ahbe;
|
||||
is_use_ahbe = ahbe;
|
||||
return old;
|
||||
}
|
||||
|
||||
bool S3fsCurl::GetResponseCode(long& responseCode)
|
||||
{
|
||||
if(!hCurl){
|
||||
|
|
@ -1505,6 +1513,10 @@ int S3fsCurl::PutHeadRequest(const char* tpath, headers_t& meta, bool ow_sse_flg
|
|||
if(ow_sse_flg && S3fsCurl::is_use_sse){
|
||||
requestHeaders = curl_slist_sort_insert(requestHeaders, "x-amz-server-side-encryption:AES256");
|
||||
}
|
||||
if(is_use_ahbe){
|
||||
// set additional header by ahbe conf
|
||||
requestHeaders = AdditionalHeader::get()->AddHeader(requestHeaders, tpath);
|
||||
}
|
||||
if(!S3fsCurl::IsPublicBucket()){
|
||||
requestHeaders = curl_slist_sort_insert(
|
||||
requestHeaders,
|
||||
|
|
@ -1601,6 +1613,10 @@ int S3fsCurl::PutRequest(const char* tpath, headers_t& meta, int fd, bool ow_sse
|
|||
if(ow_sse_flg && S3fsCurl::is_use_sse){
|
||||
requestHeaders = curl_slist_sort_insert(requestHeaders, "x-amz-server-side-encryption:AES256");
|
||||
}
|
||||
if(is_use_ahbe){
|
||||
// set additional header by ahbe conf
|
||||
requestHeaders = AdditionalHeader::get()->AddHeader(requestHeaders, tpath);
|
||||
}
|
||||
if(!S3fsCurl::IsPublicBucket()){
|
||||
requestHeaders = curl_slist_sort_insert(
|
||||
requestHeaders,
|
||||
|
|
@ -1852,6 +1868,10 @@ int S3fsCurl::PreMultipartPostRequest(const char* tpath, headers_t& meta, string
|
|||
if(ow_sse_flg && S3fsCurl::is_use_sse){
|
||||
requestHeaders = curl_slist_sort_insert(requestHeaders, "x-amz-server-side-encryption:AES256");
|
||||
}
|
||||
if(is_use_ahbe){
|
||||
// set additional header by ahbe conf
|
||||
requestHeaders = AdditionalHeader::get()->AddHeader(requestHeaders, tpath);
|
||||
}
|
||||
if(!S3fsCurl::IsPublicBucket()){
|
||||
requestHeaders = curl_slist_sort_insert(
|
||||
requestHeaders,
|
||||
|
|
@ -2167,6 +2187,10 @@ int S3fsCurl::CopyMultipartPostRequest(const char* from, const char* to, int par
|
|||
if(ow_sse_flg && S3fsCurl::is_use_sse){
|
||||
requestHeaders = curl_slist_sort_insert(requestHeaders, "x-amz-server-side-encryption:AES256");
|
||||
}
|
||||
if(is_use_ahbe){
|
||||
// set additional header by ahbe conf
|
||||
requestHeaders = AdditionalHeader::get()->AddHeader(requestHeaders, to);
|
||||
}
|
||||
if(!S3fsCurl::IsPublicBucket()){
|
||||
requestHeaders = curl_slist_sort_insert(
|
||||
requestHeaders,
|
||||
|
|
@ -2599,6 +2623,189 @@ int S3fsMultiCurl::Request(void)
|
|||
return 0;
|
||||
}
|
||||
|
||||
//-------------------------------------------------------------------
|
||||
// Class AdditionalHeader
|
||||
//-------------------------------------------------------------------
|
||||
AdditionalHeader AdditionalHeader::singleton;
|
||||
|
||||
//-------------------------------------------------------------------
|
||||
// Class AdditionalHeader method
|
||||
//-------------------------------------------------------------------
|
||||
AdditionalHeader::AdditionalHeader()
|
||||
{
|
||||
if(this == AdditionalHeader::get()){
|
||||
is_enable = false;
|
||||
}else{
|
||||
assert(false);
|
||||
}
|
||||
}
|
||||
|
||||
AdditionalHeader::~AdditionalHeader()
|
||||
{
|
||||
if(this == AdditionalHeader::get()){
|
||||
Unload();
|
||||
}else{
|
||||
assert(false);
|
||||
}
|
||||
}
|
||||
|
||||
bool AdditionalHeader::Load(const char* file)
|
||||
{
|
||||
if(!file){
|
||||
DPRNNN("file is NULL.");
|
||||
return false;
|
||||
}
|
||||
Unload();
|
||||
|
||||
ifstream AH(file);
|
||||
if(!AH.good()){
|
||||
DPRNNN("Could not open file(%s).", file);
|
||||
return false;
|
||||
}
|
||||
|
||||
// read file
|
||||
string line;
|
||||
while(getline(AH, line)){
|
||||
if('#' == line[0]){
|
||||
continue;
|
||||
}
|
||||
if(0 == line.size()){
|
||||
continue;
|
||||
}
|
||||
// load a line
|
||||
stringstream ss(line);
|
||||
string key(""); // suffix(key)
|
||||
string head; // additional HTTP header
|
||||
string value; // header value
|
||||
if(0 == isblank(line[0])){
|
||||
ss >> key;
|
||||
}
|
||||
if(ss){
|
||||
ss >> head;
|
||||
if(ss && static_cast<size_t>(ss.tellg()) < line.size()){
|
||||
value = line.substr(static_cast<int>(ss.tellg()) + 1);
|
||||
}
|
||||
}
|
||||
|
||||
// check it
|
||||
if(0 == head.size()){
|
||||
if(0 == key.size()){
|
||||
continue;
|
||||
}
|
||||
DPRNNN("file format error: %s key(suffix) is no HTTP header value.", key.c_str());
|
||||
Unload();
|
||||
return false;
|
||||
}
|
||||
|
||||
// set charcntlist
|
||||
int keylen = key.size();
|
||||
charcnt_list_t::iterator iter;
|
||||
for(iter = charcntlist.begin(); iter != charcntlist.end(); ++iter){
|
||||
if(keylen == (*iter)){
|
||||
break;
|
||||
}
|
||||
}
|
||||
if(iter == charcntlist.end()){
|
||||
charcntlist.push_back(keylen);
|
||||
}
|
||||
// set addheader
|
||||
if(addheader.end() == addheader.find(key)){
|
||||
headerpair_t hpair;
|
||||
hpair[head] = value;
|
||||
addheader[key] = hpair;
|
||||
}else{
|
||||
(addheader[key])[head] = value;
|
||||
}
|
||||
// set flag
|
||||
if(!is_enable){
|
||||
is_enable = true;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
void AdditionalHeader::Unload(void)
|
||||
{
|
||||
is_enable = false;
|
||||
charcntlist.clear();
|
||||
addheader.clear();
|
||||
}
|
||||
|
||||
bool AdditionalHeader::AddHeader(headers_t& meta, const char* path) const
|
||||
{
|
||||
if(!is_enable){
|
||||
return true;
|
||||
}
|
||||
if(!path){
|
||||
DPRNNN("path is NULL.");
|
||||
return false;
|
||||
}
|
||||
int nPathLen = strlen(path);
|
||||
for(charcnt_list_t::const_iterator iter = charcntlist.begin(); iter != charcntlist.end(); ++iter){
|
||||
// get target charactor count
|
||||
if(nPathLen < (*iter)){
|
||||
continue;
|
||||
}
|
||||
// make target suffix(same charactor count) & find
|
||||
string suffix(&path[nPathLen - (*iter)]);
|
||||
if(addheader.end() == addheader.find(suffix)){
|
||||
continue;
|
||||
}
|
||||
for(headerpair_t::const_iterator piter = addheader.at(suffix).begin(); piter != addheader.at(suffix).end(); ++piter){
|
||||
// Adding header
|
||||
meta[(*piter).first] = (*piter).second;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
struct curl_slist* AdditionalHeader::AddHeader(struct curl_slist* list, const char* path) const
|
||||
{
|
||||
headers_t meta;
|
||||
|
||||
if(!AddHeader(meta, path)){
|
||||
return list;
|
||||
}
|
||||
for(headers_t::iterator iter = meta.begin(); iter != meta.end(); ++iter){
|
||||
string slistval = (*iter).first + ": " + (*iter).second;
|
||||
// Adding header
|
||||
list = curl_slist_sort_insert(list, slistval.c_str());
|
||||
}
|
||||
return list;
|
||||
}
|
||||
|
||||
bool AdditionalHeader::Dump(void) const
|
||||
{
|
||||
if(!foreground2){
|
||||
return true;
|
||||
}
|
||||
// charactor count list
|
||||
stringstream ssdbg;
|
||||
ssdbg << "Charactor count list[" << charcntlist.size() << "] = {";
|
||||
for(charcnt_list_t::const_iterator citer = charcntlist.begin(); citer != charcntlist.end(); ++citer){
|
||||
ssdbg << " " << (*citer);
|
||||
}
|
||||
ssdbg << " }\n";
|
||||
|
||||
// additional header
|
||||
ssdbg << "Additional Header list[" << addheader.size() << "] = {\n";
|
||||
for(addheader_t::const_iterator aiter = addheader.begin(); aiter != addheader.end(); ++aiter){
|
||||
string key = (*aiter).first;
|
||||
if(0 == key.size()){
|
||||
key = "*";
|
||||
}
|
||||
for(headerpair_t::const_iterator piter = (*aiter).second.begin(); piter != (*aiter).second.end(); ++piter){
|
||||
ssdbg << " " << key << "\t--->\t" << (*piter).first << ": " << (*piter).second << "\n";
|
||||
}
|
||||
}
|
||||
ssdbg << "}";
|
||||
|
||||
// print all
|
||||
FPRNINFO("%s", ssdbg.str().c_str());
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
//-------------------------------------------------------------------
|
||||
// Utility functions
|
||||
//-------------------------------------------------------------------
|
||||
|
|
|
|||
37
src/curl.h
37
src/curl.h
|
|
@ -144,10 +144,11 @@ class S3fsCurl
|
|||
const unsigned char* postdata; // use by post method and read callback function.
|
||||
int postdata_remaining; // use by post method and read callback function.
|
||||
filepart partdata; // use by multipart upload/get object callback
|
||||
bool is_use_ahbe; // additional header by extension
|
||||
|
||||
public:
|
||||
// constructor/destructor
|
||||
S3fsCurl();
|
||||
S3fsCurl(bool ahbe = false);
|
||||
~S3fsCurl();
|
||||
|
||||
private:
|
||||
|
|
@ -244,6 +245,10 @@ class S3fsCurl
|
|||
BodyData* GetBodyData(void) const { return bodydata; }
|
||||
BodyData* GetHeadData(void) const { return headdata; }
|
||||
long GetLastResponseCode(void) const { return LastResponseCode; }
|
||||
bool SetUseAhbe(bool ahbe);
|
||||
bool EnableUseAhbe(void) { return SetUseAhbe(true); }
|
||||
bool DisableUseAhbe(void) { return SetUseAhbe(false); }
|
||||
bool IsUseAhbe(void) const { return is_use_ahbe; }
|
||||
};
|
||||
|
||||
//----------------------------------------------
|
||||
|
|
@ -285,6 +290,36 @@ class S3fsMultiCurl
|
|||
int Request(void);
|
||||
};
|
||||
|
||||
//----------------------------------------------
|
||||
// class AdditionalHeader
|
||||
//----------------------------------------------
|
||||
typedef std::list<int> charcnt_list_t;
|
||||
typedef std::map<std::string, std::string> headerpair_t;
|
||||
typedef std::map<std::string, headerpair_t> addheader_t;
|
||||
|
||||
class AdditionalHeader
|
||||
{
|
||||
private:
|
||||
static AdditionalHeader singleton;
|
||||
bool is_enable;
|
||||
charcnt_list_t charcntlist;
|
||||
addheader_t addheader;
|
||||
|
||||
public:
|
||||
// Reference singleton
|
||||
static AdditionalHeader* get(void) { return &singleton; }
|
||||
|
||||
AdditionalHeader();
|
||||
~AdditionalHeader();
|
||||
|
||||
bool Load(const char* file);
|
||||
void Unload(void);
|
||||
|
||||
bool AddHeader(headers_t& meta, const char* path) const;
|
||||
struct curl_slist* AddHeader(struct curl_slist* list, const char* path) const;
|
||||
bool Dump(void) const;
|
||||
};
|
||||
|
||||
//----------------------------------------------
|
||||
// Utility Functions
|
||||
//----------------------------------------------
|
||||
|
|
|
|||
|
|
@ -906,7 +906,7 @@ int FdEntity::RowFlush(const char* tpath, headers_t& meta, bool ow_sse_flg, bool
|
|||
S3fsCurl::SetReadwriteTimeout(backup);
|
||||
}
|
||||
}else{
|
||||
S3fsCurl s3fscurl;
|
||||
S3fsCurl s3fscurl(true);
|
||||
result = s3fscurl.PutRequest(tpath ? tpath : path.c_str(), meta, fd, ow_sse_flg);
|
||||
}
|
||||
|
||||
|
|
|
|||
131
src/s3fs.cpp
131
src/s3fs.cpp
|
|
@ -25,6 +25,7 @@
|
|||
#include <dirent.h>
|
||||
#include <errno.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <libxml/xpath.h>
|
||||
#include <libxml/xpathInternals.h>
|
||||
#include <libxml/tree.h>
|
||||
|
|
@ -78,7 +79,9 @@ std::string bucket = "";
|
|||
//-------------------------------------------------------------------
|
||||
// Static valiables
|
||||
//-------------------------------------------------------------------
|
||||
static mode_t root_mode = 0;
|
||||
static uid_t mp_uid = 0; // owner of mount point(only not specified uid opt)
|
||||
static gid_t mp_gid = 0; // group of mount point(only not specified gid opt)
|
||||
static mode_t mp_mode = 0; // mode of mount point
|
||||
static std::string mountpoint;
|
||||
static std::string passwd_file = "";
|
||||
static bool utility_mode = false;
|
||||
|
|
@ -87,10 +90,12 @@ static bool nocopyapi = false;
|
|||
static bool norenameapi = false;
|
||||
static bool nonempty = false;
|
||||
static bool allow_other = false;
|
||||
static uid_t s3fs_uid = 0; // default = root.
|
||||
static gid_t s3fs_gid = 0; // default = root.
|
||||
static bool is_s3fs_umask = false;// default does not set.
|
||||
static uid_t s3fs_uid = 0;
|
||||
static gid_t s3fs_gid = 0;
|
||||
static mode_t s3fs_umask = 0;
|
||||
static bool is_s3fs_uid = false;// default does not set.
|
||||
static bool is_s3fs_gid = false;// default does not set.
|
||||
static bool is_s3fs_umask = false;// default does not set.
|
||||
static bool is_remove_cache = false;
|
||||
|
||||
// mutex
|
||||
|
|
@ -136,6 +141,7 @@ static int check_for_aws_format(void);
|
|||
static int check_passwd_file_perms(void);
|
||||
static int read_passwd_file(void);
|
||||
static int get_access_keys(void);
|
||||
static int set_moutpoint_attribute(struct stat& mpst);
|
||||
static int my_fuse_opt_proc(void* data, const char* arg, int key, struct fuse_args* outargs);
|
||||
|
||||
// fuse interface functions
|
||||
|
|
@ -314,7 +320,9 @@ static int get_object_attribute(const char* path, struct stat* pstbuf, headers_t
|
|||
memset(pstat, 0, sizeof(struct stat));
|
||||
if(0 == strcmp(path, "/") || 0 == strcmp(path, ".")){
|
||||
pstat->st_nlink = 1; // see fuse faq
|
||||
pstat->st_mode = root_mode;
|
||||
pstat->st_mode = mp_mode;
|
||||
pstat->st_uid = is_s3fs_uid ? s3fs_uid : mp_uid;
|
||||
pstat->st_gid = is_s3fs_gid ? s3fs_gid : mp_gid;
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -462,7 +470,7 @@ static int check_object_access(const char* path, int mask, struct stat* pstbuf)
|
|||
// root is allowed all accessing.
|
||||
return 0;
|
||||
}
|
||||
if(0 != s3fs_uid && s3fs_uid == pcxt->uid){
|
||||
if(is_s3fs_uid && s3fs_uid == pcxt->uid){
|
||||
// "uid" user is allowed all accessing.
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -472,8 +480,8 @@ static int check_object_access(const char* path, int mask, struct stat* pstbuf)
|
|||
}
|
||||
|
||||
// for "uid", "gid" option
|
||||
uid_t obj_uid = (0 != s3fs_uid ? s3fs_uid : pst->st_uid);
|
||||
gid_t obj_gid = (0 != s3fs_gid ? s3fs_gid : pst->st_gid);
|
||||
uid_t obj_uid = (is_s3fs_uid ? s3fs_uid : pst->st_uid);
|
||||
gid_t obj_gid = (is_s3fs_gid ? s3fs_gid : pst->st_gid);
|
||||
|
||||
// compare file mode and uid/gid + mask.
|
||||
mode_t mode = pst->st_mode;
|
||||
|
|
@ -535,11 +543,11 @@ static int check_object_owner(const char* path, struct stat* pstbuf)
|
|||
// root is allowed all accessing.
|
||||
return 0;
|
||||
}
|
||||
if(0 != s3fs_uid && s3fs_uid == pcxt->uid){
|
||||
if(is_s3fs_uid && s3fs_uid == pcxt->uid){
|
||||
// "uid" user is allowed all accessing.
|
||||
return 0;
|
||||
}
|
||||
if(pcxt->uid == (0 != s3fs_uid ? s3fs_uid : pst->st_uid)){
|
||||
if(pcxt->uid == pst->st_uid){
|
||||
return 0;
|
||||
}
|
||||
return -EPERM;
|
||||
|
|
@ -622,7 +630,7 @@ static FdEntity* get_local_fent(const char* path, bool is_load)
|
|||
static int put_headers(const char* path, headers_t& meta, bool ow_sse_flg)
|
||||
{
|
||||
int result;
|
||||
S3fsCurl s3fscurl;
|
||||
S3fsCurl s3fscurl(true);
|
||||
struct stat buf;
|
||||
|
||||
FPRNNN("[path=%s]", path);
|
||||
|
|
@ -726,7 +734,7 @@ static int s3fs_readlink(const char* path, char* buf, size_t size)
|
|||
// common function for creation of a plain object
|
||||
static int create_file_object(const char* path, mode_t mode, uid_t uid, gid_t gid)
|
||||
{
|
||||
FPRNNN("[path=%s][mode=%d]", path, mode);
|
||||
FPRNNN("[path=%s][mode=%04o]", path, mode);
|
||||
|
||||
headers_t meta;
|
||||
meta["Content-Type"] = S3fsCurl::LookupMimeType(string(path));
|
||||
|
|
@ -735,7 +743,7 @@ static int create_file_object(const char* path, mode_t mode, uid_t uid, gid_t gi
|
|||
meta["x-amz-meta-mode"] = str(mode);
|
||||
meta["x-amz-meta-mtime"] = str(time(NULL));
|
||||
|
||||
S3fsCurl s3fscurl;
|
||||
S3fsCurl s3fscurl(true);
|
||||
return s3fscurl.PutRequest(path, meta, -1, false); // fd=-1 means for creating zero byte object.
|
||||
}
|
||||
|
||||
|
|
@ -745,7 +753,7 @@ static int s3fs_mknod(const char *path, mode_t mode, dev_t rdev)
|
|||
headers_t meta;
|
||||
struct fuse_context* pcxt;
|
||||
|
||||
FPRN("[path=%s][mode=0%o][dev=%lu]", path, mode, rdev);
|
||||
FPRN("[path=%s][mode=%04o][dev=%lu]", path, mode, rdev);
|
||||
|
||||
if(NULL == (pcxt = fuse_get_context())){
|
||||
return -EIO;
|
||||
|
|
@ -766,7 +774,7 @@ static int s3fs_create(const char* path, mode_t mode, struct fuse_file_info* fi)
|
|||
headers_t meta;
|
||||
struct fuse_context* pcxt;
|
||||
|
||||
FPRN("[path=%s][mode=%d][flags=%d]", path, mode, fi->flags);
|
||||
FPRN("[path=%s][mode=%04o][flags=%d]", path, mode, fi->flags);
|
||||
|
||||
if(NULL == (pcxt = fuse_get_context())){
|
||||
return -EIO;
|
||||
|
|
@ -801,7 +809,7 @@ static int s3fs_create(const char* path, mode_t mode, struct fuse_file_info* fi)
|
|||
|
||||
static int create_directory_object(const char* path, mode_t mode, time_t time, uid_t uid, gid_t gid)
|
||||
{
|
||||
FPRNN("[path=%s][mode=%d][time=%lu][uid=%d][gid=%d]", path, mode, time, uid, gid);
|
||||
FPRNN("[path=%s][mode=%04o][time=%lu][uid=%d][gid=%d]", path, mode, time, uid, gid);
|
||||
|
||||
if(!path || '\0' == path[0]){
|
||||
return -1;
|
||||
|
|
@ -827,7 +835,7 @@ static int s3fs_mkdir(const char* path, mode_t mode)
|
|||
int result;
|
||||
struct fuse_context* pcxt;
|
||||
|
||||
FPRN("[path=%s][mode=%d]", path, mode);
|
||||
FPRN("[path=%s][mode=%04o]", path, mode);
|
||||
|
||||
if(NULL == (pcxt = fuse_get_context())){
|
||||
return -EIO;
|
||||
|
|
@ -1089,7 +1097,7 @@ static int rename_large_object(const char* from, const char* to)
|
|||
return result;
|
||||
}
|
||||
|
||||
S3fsCurl s3fscurl;
|
||||
S3fsCurl s3fscurl(true);
|
||||
if(0 != (result = s3fscurl.MultipartRenameRequest(from, to, meta, buf.st_size))){
|
||||
return result;
|
||||
}
|
||||
|
|
@ -1303,8 +1311,12 @@ static int s3fs_chmod(const char* path, mode_t mode)
|
|||
struct stat stbuf;
|
||||
int nDirType = DIRTYPE_UNKNOWN;
|
||||
|
||||
FPRN("[path=%s][mode=%d]", path, mode);
|
||||
FPRN("[path=%s][mode=%04o]", path, mode);
|
||||
|
||||
if(0 == strcmp(path, "/")){
|
||||
DPRNNN("Could not change mode for maount point.");
|
||||
return -EIO;
|
||||
}
|
||||
if(0 != (result = check_parent_object_access(path, X_OK))){
|
||||
return result;
|
||||
}
|
||||
|
|
@ -1365,8 +1377,12 @@ static int s3fs_chmod_nocopy(const char* path, mode_t mode)
|
|||
struct stat stbuf;
|
||||
int nDirType = DIRTYPE_UNKNOWN;
|
||||
|
||||
FPRNN("[path=%s][mode=%d]", path, mode);
|
||||
FPRNN("[path=%s][mode=%04o]", path, mode);
|
||||
|
||||
if(0 == strcmp(path, "/")){
|
||||
DPRNNN("Could not change mode for maount point.");
|
||||
return -EIO;
|
||||
}
|
||||
if(0 != (result = check_parent_object_access(path, X_OK))){
|
||||
return result;
|
||||
}
|
||||
|
|
@ -1441,6 +1457,10 @@ static int s3fs_chown(const char* path, uid_t uid, gid_t gid)
|
|||
|
||||
FPRN("[path=%s][uid=%d][gid=%d]", path, uid, gid);
|
||||
|
||||
if(0 == strcmp(path, "/")){
|
||||
DPRNNN("Could not change owner for maount point.");
|
||||
return -EIO;
|
||||
}
|
||||
if(0 != (result = check_parent_object_access(path, X_OK))){
|
||||
return result;
|
||||
}
|
||||
|
|
@ -1518,6 +1538,10 @@ static int s3fs_chown_nocopy(const char* path, uid_t uid, gid_t gid)
|
|||
|
||||
FPRNN("[path=%s][uid=%d][gid=%d]", path, uid, gid);
|
||||
|
||||
if(0 == strcmp(path, "/")){
|
||||
DPRNNN("Could not change owner for maount point.");
|
||||
return -EIO;
|
||||
}
|
||||
if(0 != (result = check_parent_object_access(path, X_OK))){
|
||||
return result;
|
||||
}
|
||||
|
|
@ -1602,6 +1626,10 @@ static int s3fs_utimens(const char* path, const struct timespec ts[2])
|
|||
|
||||
FPRN("[path=%s][mtime=%zd]", path, ts[1].tv_sec);
|
||||
|
||||
if(0 == strcmp(path, "/")){
|
||||
DPRNNN("Could not change mtime for maount point.");
|
||||
return -EIO;
|
||||
}
|
||||
if(0 != (result = check_parent_object_access(path, X_OK))){
|
||||
return result;
|
||||
}
|
||||
|
|
@ -1663,6 +1691,10 @@ static int s3fs_utimens_nocopy(const char* path, const struct timespec ts[2])
|
|||
|
||||
FPRNN("[path=%s][mtime=%s]", path, str(ts[1].tv_sec).c_str());
|
||||
|
||||
if(0 == strcmp(path, "/")){
|
||||
DPRNNN("Could not change mtime for maount point.");
|
||||
return -EIO;
|
||||
}
|
||||
if(0 != (result = check_parent_object_access(path, X_OK))){
|
||||
return result;
|
||||
}
|
||||
|
|
@ -1981,7 +2013,7 @@ static S3fsCurl* multi_head_retry_callback(S3fsCurl* s3fscurl)
|
|||
if(!s3fscurl){
|
||||
return NULL;
|
||||
}
|
||||
S3fsCurl* newcurl = new S3fsCurl();
|
||||
S3fsCurl* newcurl = new S3fsCurl(s3fscurl->IsUseAhbe());
|
||||
string path = s3fscurl->GetPath();
|
||||
string base_path = s3fscurl->GetBasePath();
|
||||
string saved_path = s3fscurl->GetSpacialSavedPath();
|
||||
|
|
@ -2881,6 +2913,34 @@ static int get_access_keys(void)
|
|||
return EXIT_FAILURE;
|
||||
}
|
||||
|
||||
//
|
||||
// Check & Set attributes for mount point.
|
||||
//
|
||||
static int set_moutpoint_attribute(struct stat& mpst)
|
||||
{
|
||||
mp_uid = geteuid();
|
||||
mp_gid = getegid();
|
||||
mp_mode = S_IFDIR | (allow_other ? (S_IRWXU | S_IRWXG | S_IRWXO) : S_IRWXU);
|
||||
|
||||
FPRNNN("PROC(uid=%d, gid=%d) - MountPoint(uid=%d, gid=%d, mode=%04o)", mp_uid, mp_gid, mpst.st_uid, mpst.st_gid, mpst.st_mode);
|
||||
|
||||
// check owner
|
||||
if(0 == mp_uid || mpst.st_uid == mp_uid){
|
||||
return true;
|
||||
}
|
||||
// check group permission
|
||||
if(mpst.st_gid == mp_gid || 1 == is_uid_inculde_group(mp_uid, mpst.st_gid)){
|
||||
if(S_IRWXG == (mpst.st_mode & S_IRWXG)){
|
||||
return true;
|
||||
}
|
||||
}
|
||||
// check other permission
|
||||
if(S_IRWXO == (mpst.st_mode & S_IRWXO)){
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// This is repeatedly called by the fuse option parser
|
||||
// if the key is equal to FUSE_OPT_KEY_OPT, it's an option passed in prefixed by
|
||||
// '-' or '--' e.g.: -f -d -ousecache=/tmp
|
||||
|
|
@ -2928,11 +2988,10 @@ static int my_fuse_opt_proc(void* data, const char* arg, int key, struct fuse_ar
|
|||
program_name.c_str(), mountpoint.c_str());
|
||||
return -1;
|
||||
}
|
||||
root_mode = stbuf.st_mode; // save mode for later usage
|
||||
if(allow_other){
|
||||
root_mode |= (S_IXUSR | S_IXGRP | S_IXOTH | S_IFDIR);
|
||||
}else{
|
||||
root_mode |= S_IFDIR;
|
||||
if(!set_moutpoint_attribute(stbuf)){
|
||||
fprintf(stderr, "%s: MOUNTPOINT: %s permission denied.\n",
|
||||
program_name.c_str(), mountpoint.c_str());
|
||||
return -1;
|
||||
}
|
||||
|
||||
if(!nonempty){
|
||||
|
|
@ -2957,10 +3016,20 @@ static int my_fuse_opt_proc(void* data, const char* arg, int key, struct fuse_ar
|
|||
}else if(key == FUSE_OPT_KEY_OPT){
|
||||
if(strstr(arg, "uid=") != 0){
|
||||
s3fs_uid = strtoul(strchr(arg, '=') + sizeof(char), 0, 10);
|
||||
if(0 != geteuid() && 0 == s3fs_uid){
|
||||
fprintf(stderr, "%s: root user can only specify uid=0.\n", program_name.c_str());
|
||||
return -1;
|
||||
}
|
||||
is_s3fs_uid = true;
|
||||
return 1; // continue for fuse option
|
||||
}
|
||||
if(strstr(arg, "gid=") != 0){
|
||||
s3fs_gid = strtoul(strchr(arg, '=') + sizeof(char), 0, 10);
|
||||
if(0 != getegid() && 0 == s3fs_gid){
|
||||
fprintf(stderr, "%s: root user can only specify gid=0.\n", program_name.c_str());
|
||||
return -1;
|
||||
}
|
||||
is_s3fs_gid = true;
|
||||
return 1; // continue for fuse option
|
||||
}
|
||||
if(strstr(arg, "umask=") != 0){
|
||||
|
|
@ -3125,6 +3194,16 @@ static int my_fuse_opt_proc(void* data, const char* arg, int key, struct fuse_ar
|
|||
FdManager::SetPageSize(pagesize);
|
||||
return 0;
|
||||
}
|
||||
if(strstr(arg, "ahbe_conf=") != 0){
|
||||
string ahbe_conf = strchr(arg, '=') + sizeof(char);
|
||||
if(!AdditionalHeader::get()->Load(ahbe_conf.c_str())){
|
||||
fprintf(stderr, "%s: failed to load ahbe_conf file(%s).\n",
|
||||
program_name.c_str(), ahbe_conf.c_str());
|
||||
return -1;
|
||||
}
|
||||
AdditionalHeader::get()->Dump();
|
||||
return 0;
|
||||
}
|
||||
if(strstr(arg, "noxmlns") != 0){
|
||||
noxmlns = true;
|
||||
return 0;
|
||||
|
|
|
|||
|
|
@ -825,6 +825,27 @@ void show_help (void)
|
|||
" passwd_file (default=\"\")\n"
|
||||
" - specify which s3fs password file to use\n"
|
||||
"\n"
|
||||
" ahbe_conf (default=\"\" which means disabled)\n"
|
||||
" - This option specifies the configuration file path which\n"
|
||||
" file is the additional HTTP header by file(object) extension.\n"
|
||||
" The configuration file format is below:\n"
|
||||
" -----------\n"
|
||||
" line = [file suffix] HTTP-header [HTTP-values]\n"
|
||||
" file suffix = file(object) suffix, if this field is empty,\n"
|
||||
" it means \"*\"(all object).\n"
|
||||
" HTTP-header = additional HTTP header name\n"
|
||||
" HTTP-values = additional HTTP header value\n"
|
||||
" -----------\n"
|
||||
" Sample:\n"
|
||||
" -----------\n"
|
||||
" .gz Content-Encoding gzip\n"
|
||||
" .Z Content-Encoding compress\n"
|
||||
" X-S3FS-MYHTTPHEAD myvalue\n"
|
||||
" -----------\n"
|
||||
" A sample configuration file is uploaded in \"test\" directory.\n"
|
||||
" If you specify this option for set \"Content-Encoding\" HTTP \n"
|
||||
" header, please take care for RFC 2616.\n"
|
||||
"\n"
|
||||
" connect_timeout (default=\"10\" seconds)\n"
|
||||
" - time to wait for connection before giving up\n"
|
||||
"\n"
|
||||
|
|
|
|||
|
|
@ -4,5 +4,6 @@ EXTRA_DIST = \
|
|||
integration-test-common.sh \
|
||||
require-root.sh \
|
||||
small-integration-test.sh \
|
||||
mergedir.sh
|
||||
mergedir.sh \
|
||||
sample_ahbe.conf
|
||||
|
||||
|
|
|
|||
41
test/sample_ahbe.conf
Normal file
41
test/sample_ahbe.conf
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
# S3FS: Samlpe ahbe_conf parameter file.
|
||||
#
|
||||
# This file is configuration file for additional header by extension(ahbe).
|
||||
# s3fs loads this file at starting.
|
||||
#
|
||||
# Format:
|
||||
# line = [file suffix] HTTP-header [HTTP-header-values]
|
||||
# file suffix = file(object) suffix, if this field is empty,
|
||||
# it means "*"(all object).
|
||||
# HTTP-header = additional HTTP header name
|
||||
# HTTP-header-values = additional HTTP header value
|
||||
#
|
||||
# <suffix(extension)> <HTTP header> <HTTP header values>
|
||||
#
|
||||
# Example:
|
||||
# " Content-Encoding gzip" --> all object
|
||||
# ".gz Content-Encoding gzip" --> only ".gz" extension file
|
||||
#
|
||||
# Notice:
|
||||
# If you need to set all object, you can specify without "suffix".
|
||||
# Then all of object(file) is added additional header.
|
||||
# If you have this configuration file for Content-Encoding, you should
|
||||
# know about RFC 2616.
|
||||
#
|
||||
# "The default (identity) encoding; the use of no transformation
|
||||
# whatsoever. This content-coding is used only in the Accept-
|
||||
# Encoding header, and SHOULD NOT be used in the Content-Encoding
|
||||
# header."
|
||||
#
|
||||
.gz Content-Encoding gzip
|
||||
.Z Content-Encoding compress
|
||||
.bz2 Content-Encoding bzip2
|
||||
.svgz Content-Encoding gzip
|
||||
.svg.gz Content-Encoding gzip
|
||||
.tgz Content-Encoding gzip
|
||||
.tar.gz Content-Encoding gzip
|
||||
.taz Content-Encoding gzip
|
||||
.tz Content-Encoding gzip
|
||||
.tbz2 Content-Encoding gzip
|
||||
gz.js Content-Encoding gzip
|
||||
|
||||
Loading…
Reference in New Issue
Block a user