From d7689151abc6726f5365a48e650f8726118018ba Mon Sep 17 00:00:00 2001 From: "ggtakec@gmail.com" Date: Fri, 16 Aug 2013 19:24:01 +0000 Subject: [PATCH] Fixed Issue 229 and Changes codes 1) Set metadata "Content-Encoding" automatically(Issue 292) For this issue, s3fs is added new option "ahbe_conf". New option means the configuration file path, and this file specifies additional HTTP header by file(object) extension. Thus you can specify any HTTP header for each object by extension. * ahbe_conf file format: ----------- line = [file suffix] HTTP-header [HTTP-header-values] file suffix = file(object) suffix, if this field is empty, it means "*"(all object). HTTP-header = additional HTTP header name HTTP-header-values = additional HTTP header value ----------- * Example: ----------- .gz Content-Encoding gzip .Z Content-Encoding compress X-S3FS-MYHTTPHEAD myvalue ----------- A sample configuration file is uploaded in "test" directory. If ahbe_conf parameter is specified, s3fs loads it's configuration and compares extension(suffix) of object(file) when uploading (PUT/POST) it. If the extension is same, s3fs adds/sends specified HTTP header and value. A case of sample configuration file, if a object(it's extension is ".gz") which already has Content-Encoding HTTP header is renamed to ".txt" extension, s3fs does not set Content-Encoding. Because ".txt" is not match any line in configuration file. So, s3fs matches the extension by each PUT/POST action. * Please take care about "Content-Encoding". This new option allows setting ANY HTTP header by object extension. For example, you can specify "Content-Encoding" for ".gz"/etc extension in configuration. But this means that S3 always returns "Content-Encoding: gzip" when a client requests with other "Accept-Encoding:" header. It SHOULD NOT be good. Please see RFC 2616. 2) Changes about allow_other/uid/gid option for mount point I reviewed about mount point permission and allow_other/uid/gid options, and found bugs about these. s3fs is fixed bugs and changed to the following specifications. * s3fs only allows uid(gid) options as 0(root), when the effective user is zero(root). * A mount point(directory) must have a permission to allow accessing by effective user/group. * If allow_other option is specified, the mount point permission is set 0777(all users allow all access). In another case, the mount point is set 0700(only allows effective user). * When uid/gid option is specified, the mount point owner/group is set uid/gid option value. If uid/gid is not set, it is set effective user/group id. This changes maybe fixes some issue(321, 338). 3) Changes a logic about (Issue 229) The chmod command returns -EIO when changing the mount point. It is correct, s3fs can not changed owner/group/mtime for the mount point, but s3fs sends a request for changing the bucket. This revision does not send the request, and returns EIO as soon as possible. git-svn-id: http://s3fs.googlecode.com/svn/trunk@465 df820570-a93a-0410-bd06-b72b767a4274 --- doc/man/s3fs.1 | 18 ++++ src/curl.cpp | 221 ++++++++++++++++++++++++++++++++++++++++-- src/curl.h | 37 ++++++- src/fdcache.cpp | 2 +- src/s3fs.cpp | 131 ++++++++++++++++++++----- src/s3fs_util.cpp | 21 ++++ test/Makefile.am | 3 +- test/sample_ahbe.conf | 41 ++++++++ 8 files changed, 438 insertions(+), 36 deletions(-) create mode 100644 test/sample_ahbe.conf diff --git a/doc/man/s3fs.1 b/doc/man/s3fs.1 index fc3f7e1..d75cea3 100644 --- a/doc/man/s3fs.1 +++ b/doc/man/s3fs.1 @@ -75,6 +75,24 @@ this option can not be specified with use_rrs. \fB\-o\fR passwd_file (default="") specify the path to the password file, which which takes precedence over the password in $HOME/.passwd-s3fs and /etc/passwd-s3fs .TP +\fB\-o\fR ahbe_conf (default="" which means disabled) +This option specifies the configuration file path which file is the additional HTTP header by file(object) extension. + The configuration file format is below: + ----------- + line = [file suffix] HTTP-header [HTTP-values] + file suffix = file(object) suffix, if this field is empty, it means "*"(all object). + HTTP-header = additional HTTP header name + HTTP-values = additional HTTP header value + ----------- + Sample: + ----------- + .gz Content-Encoding gzip + .Z Content-Encoding compress + X-S3FS-MYHTTPHEAD myvalue + ----------- + A sample configuration file is uploaded in "test" directory. +If you specify this option for set "Content-Encoding" HTTP header, please take care for RFC 2616. +.TP \fB\-o\fR public_bucket (default="" which means disabled) anonymously mount a public bucket when set to 1, ignores the $HOME/.passwd-s3fs and /etc/passwd-s3fs files. .TP diff --git a/src/curl.cpp b/src/curl.cpp index 2d8ce51..d8f55ac 100644 --- a/src/curl.cpp +++ b/src/curl.cpp @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -700,7 +701,7 @@ S3fsCurl* S3fsCurl::UploadMultipartPostRetryCallback(S3fsCurl* s3fscurl) part_num = atoi(part_num_str.c_str()); // duplicate request - S3fsCurl* newcurl = new S3fsCurl(); + S3fsCurl* newcurl = new S3fsCurl(s3fscurl->IsUseAhbe()); newcurl->partdata.etaglist = s3fscurl->partdata.etaglist; newcurl->partdata.etagpos = s3fscurl->partdata.etagpos; newcurl->partdata.fd = s3fscurl->partdata.fd; @@ -726,7 +727,7 @@ int S3fsCurl::ParallelMultipartUploadRequest(const char* tpath, headers_t& meta, etaglist_t list; off_t remaining_bytes; unsigned char* buf; - S3fsCurl s3fscurl; + S3fsCurl s3fscurl(true); FPRNNN("[tpath=%s][fd=%d]", SAFESTRPTR(tpath), fd); @@ -775,7 +776,7 @@ int S3fsCurl::ParallelMultipartUploadRequest(const char* tpath, headers_t& meta, chunk = remaining_bytes > MULTIPART_SIZE ? MULTIPART_SIZE : remaining_bytes; // s3fscurl sub object - S3fsCurl* s3fscurl_para = new S3fsCurl(); + S3fsCurl* s3fscurl_para = new S3fsCurl(true); s3fscurl_para->partdata.fd = fd2; s3fscurl_para->partdata.startpos = st.st_size - remaining_bytes; s3fscurl_para->partdata.size = chunk; @@ -826,7 +827,7 @@ S3fsCurl* S3fsCurl::ParallelGetObjectRetryCallback(S3fsCurl* s3fscurl) return NULL; } // duplicate request(setup new curl object) - S3fsCurl* newcurl = new S3fsCurl(); + S3fsCurl* newcurl = new S3fsCurl(s3fscurl->IsUseAhbe()); if(0 != (result = newcurl->PreGetObjectRequest( s3fscurl->path.c_str(), s3fscurl->partdata.fd, s3fscurl->partdata.startpos, s3fscurl->partdata.size))){ DPRN("failed downloading part setup(%d)", result); @@ -889,9 +890,9 @@ int S3fsCurl::ParallelGetObjectRequest(const char* tpath, int fd, off_t start, s //------------------------------------------------------------------- // Methods for S3fsCurl //------------------------------------------------------------------- -S3fsCurl::S3fsCurl() : - hCurl(NULL), path(""), base_path(""), saved_path(""), url(""), requestHeaders(NULL), - bodydata(NULL), headdata(NULL), LastResponseCode(-1), postdata(NULL), postdata_remaining(0) +S3fsCurl::S3fsCurl(bool ahbe) : + hCurl(NULL), path(""), base_path(""), saved_path(""), url(""), requestHeaders(NULL), bodydata(NULL), + headdata(NULL), LastResponseCode(-1), postdata(NULL), postdata_remaining(0), is_use_ahbe(ahbe) { } @@ -993,6 +994,13 @@ bool S3fsCurl::ClearInternalData(void) return true; } +bool S3fsCurl::SetUseAhbe(bool ahbe) +{ + bool old = is_use_ahbe; + is_use_ahbe = ahbe; + return old; +} + bool S3fsCurl::GetResponseCode(long& responseCode) { if(!hCurl){ @@ -1505,6 +1513,10 @@ int S3fsCurl::PutHeadRequest(const char* tpath, headers_t& meta, bool ow_sse_flg if(ow_sse_flg && S3fsCurl::is_use_sse){ requestHeaders = curl_slist_sort_insert(requestHeaders, "x-amz-server-side-encryption:AES256"); } + if(is_use_ahbe){ + // set additional header by ahbe conf + requestHeaders = AdditionalHeader::get()->AddHeader(requestHeaders, tpath); + } if(!S3fsCurl::IsPublicBucket()){ requestHeaders = curl_slist_sort_insert( requestHeaders, @@ -1601,6 +1613,10 @@ int S3fsCurl::PutRequest(const char* tpath, headers_t& meta, int fd, bool ow_sse if(ow_sse_flg && S3fsCurl::is_use_sse){ requestHeaders = curl_slist_sort_insert(requestHeaders, "x-amz-server-side-encryption:AES256"); } + if(is_use_ahbe){ + // set additional header by ahbe conf + requestHeaders = AdditionalHeader::get()->AddHeader(requestHeaders, tpath); + } if(!S3fsCurl::IsPublicBucket()){ requestHeaders = curl_slist_sort_insert( requestHeaders, @@ -1852,6 +1868,10 @@ int S3fsCurl::PreMultipartPostRequest(const char* tpath, headers_t& meta, string if(ow_sse_flg && S3fsCurl::is_use_sse){ requestHeaders = curl_slist_sort_insert(requestHeaders, "x-amz-server-side-encryption:AES256"); } + if(is_use_ahbe){ + // set additional header by ahbe conf + requestHeaders = AdditionalHeader::get()->AddHeader(requestHeaders, tpath); + } if(!S3fsCurl::IsPublicBucket()){ requestHeaders = curl_slist_sort_insert( requestHeaders, @@ -2167,6 +2187,10 @@ int S3fsCurl::CopyMultipartPostRequest(const char* from, const char* to, int par if(ow_sse_flg && S3fsCurl::is_use_sse){ requestHeaders = curl_slist_sort_insert(requestHeaders, "x-amz-server-side-encryption:AES256"); } + if(is_use_ahbe){ + // set additional header by ahbe conf + requestHeaders = AdditionalHeader::get()->AddHeader(requestHeaders, to); + } if(!S3fsCurl::IsPublicBucket()){ requestHeaders = curl_slist_sort_insert( requestHeaders, @@ -2599,6 +2623,189 @@ int S3fsMultiCurl::Request(void) return 0; } +//------------------------------------------------------------------- +// Class AdditionalHeader +//------------------------------------------------------------------- +AdditionalHeader AdditionalHeader::singleton; + +//------------------------------------------------------------------- +// Class AdditionalHeader method +//------------------------------------------------------------------- +AdditionalHeader::AdditionalHeader() +{ + if(this == AdditionalHeader::get()){ + is_enable = false; + }else{ + assert(false); + } +} + +AdditionalHeader::~AdditionalHeader() +{ + if(this == AdditionalHeader::get()){ + Unload(); + }else{ + assert(false); + } +} + +bool AdditionalHeader::Load(const char* file) +{ + if(!file){ + DPRNNN("file is NULL."); + return false; + } + Unload(); + + ifstream AH(file); + if(!AH.good()){ + DPRNNN("Could not open file(%s).", file); + return false; + } + + // read file + string line; + while(getline(AH, line)){ + if('#' == line[0]){ + continue; + } + if(0 == line.size()){ + continue; + } + // load a line + stringstream ss(line); + string key(""); // suffix(key) + string head; // additional HTTP header + string value; // header value + if(0 == isblank(line[0])){ + ss >> key; + } + if(ss){ + ss >> head; + if(ss && static_cast(ss.tellg()) < line.size()){ + value = line.substr(static_cast(ss.tellg()) + 1); + } + } + + // check it + if(0 == head.size()){ + if(0 == key.size()){ + continue; + } + DPRNNN("file format error: %s key(suffix) is no HTTP header value.", key.c_str()); + Unload(); + return false; + } + + // set charcntlist + int keylen = key.size(); + charcnt_list_t::iterator iter; + for(iter = charcntlist.begin(); iter != charcntlist.end(); ++iter){ + if(keylen == (*iter)){ + break; + } + } + if(iter == charcntlist.end()){ + charcntlist.push_back(keylen); + } + // set addheader + if(addheader.end() == addheader.find(key)){ + headerpair_t hpair; + hpair[head] = value; + addheader[key] = hpair; + }else{ + (addheader[key])[head] = value; + } + // set flag + if(!is_enable){ + is_enable = true; + } + } + return true; +} + +void AdditionalHeader::Unload(void) +{ + is_enable = false; + charcntlist.clear(); + addheader.clear(); +} + +bool AdditionalHeader::AddHeader(headers_t& meta, const char* path) const +{ + if(!is_enable){ + return true; + } + if(!path){ + DPRNNN("path is NULL."); + return false; + } + int nPathLen = strlen(path); + for(charcnt_list_t::const_iterator iter = charcntlist.begin(); iter != charcntlist.end(); ++iter){ + // get target charactor count + if(nPathLen < (*iter)){ + continue; + } + // make target suffix(same charactor count) & find + string suffix(&path[nPathLen - (*iter)]); + if(addheader.end() == addheader.find(suffix)){ + continue; + } + for(headerpair_t::const_iterator piter = addheader.at(suffix).begin(); piter != addheader.at(suffix).end(); ++piter){ + // Adding header + meta[(*piter).first] = (*piter).second; + } + } + return true; +} + +struct curl_slist* AdditionalHeader::AddHeader(struct curl_slist* list, const char* path) const +{ + headers_t meta; + + if(!AddHeader(meta, path)){ + return list; + } + for(headers_t::iterator iter = meta.begin(); iter != meta.end(); ++iter){ + string slistval = (*iter).first + ": " + (*iter).second; + // Adding header + list = curl_slist_sort_insert(list, slistval.c_str()); + } + return list; +} + +bool AdditionalHeader::Dump(void) const +{ + if(!foreground2){ + return true; + } + // charactor count list + stringstream ssdbg; + ssdbg << "Charactor count list[" << charcntlist.size() << "] = {"; + for(charcnt_list_t::const_iterator citer = charcntlist.begin(); citer != charcntlist.end(); ++citer){ + ssdbg << " " << (*citer); + } + ssdbg << " }\n"; + + // additional header + ssdbg << "Additional Header list[" << addheader.size() << "] = {\n"; + for(addheader_t::const_iterator aiter = addheader.begin(); aiter != addheader.end(); ++aiter){ + string key = (*aiter).first; + if(0 == key.size()){ + key = "*"; + } + for(headerpair_t::const_iterator piter = (*aiter).second.begin(); piter != (*aiter).second.end(); ++piter){ + ssdbg << " " << key << "\t--->\t" << (*piter).first << ": " << (*piter).second << "\n"; + } + } + ssdbg << "}"; + + // print all + FPRNINFO("%s", ssdbg.str().c_str()); + + return true; +} + //------------------------------------------------------------------- // Utility functions //------------------------------------------------------------------- diff --git a/src/curl.h b/src/curl.h index e304255..9dd7c05 100644 --- a/src/curl.h +++ b/src/curl.h @@ -144,10 +144,11 @@ class S3fsCurl const unsigned char* postdata; // use by post method and read callback function. int postdata_remaining; // use by post method and read callback function. filepart partdata; // use by multipart upload/get object callback + bool is_use_ahbe; // additional header by extension public: // constructor/destructor - S3fsCurl(); + S3fsCurl(bool ahbe = false); ~S3fsCurl(); private: @@ -244,6 +245,10 @@ class S3fsCurl BodyData* GetBodyData(void) const { return bodydata; } BodyData* GetHeadData(void) const { return headdata; } long GetLastResponseCode(void) const { return LastResponseCode; } + bool SetUseAhbe(bool ahbe); + bool EnableUseAhbe(void) { return SetUseAhbe(true); } + bool DisableUseAhbe(void) { return SetUseAhbe(false); } + bool IsUseAhbe(void) const { return is_use_ahbe; } }; //---------------------------------------------- @@ -285,6 +290,36 @@ class S3fsMultiCurl int Request(void); }; +//---------------------------------------------- +// class AdditionalHeader +//---------------------------------------------- +typedef std::list charcnt_list_t; +typedef std::map headerpair_t; +typedef std::map addheader_t; + +class AdditionalHeader +{ + private: + static AdditionalHeader singleton; + bool is_enable; + charcnt_list_t charcntlist; + addheader_t addheader; + + public: + // Reference singleton + static AdditionalHeader* get(void) { return &singleton; } + + AdditionalHeader(); + ~AdditionalHeader(); + + bool Load(const char* file); + void Unload(void); + + bool AddHeader(headers_t& meta, const char* path) const; + struct curl_slist* AddHeader(struct curl_slist* list, const char* path) const; + bool Dump(void) const; +}; + //---------------------------------------------- // Utility Functions //---------------------------------------------- diff --git a/src/fdcache.cpp b/src/fdcache.cpp index ccf915a..604f314 100644 --- a/src/fdcache.cpp +++ b/src/fdcache.cpp @@ -906,7 +906,7 @@ int FdEntity::RowFlush(const char* tpath, headers_t& meta, bool ow_sse_flg, bool S3fsCurl::SetReadwriteTimeout(backup); } }else{ - S3fsCurl s3fscurl; + S3fsCurl s3fscurl(true); result = s3fscurl.PutRequest(tpath ? tpath : path.c_str(), meta, fd, ow_sse_flg); } diff --git a/src/s3fs.cpp b/src/s3fs.cpp index c227a8e..36ede39 100644 --- a/src/s3fs.cpp +++ b/src/s3fs.cpp @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -78,7 +79,9 @@ std::string bucket = ""; //------------------------------------------------------------------- // Static valiables //------------------------------------------------------------------- -static mode_t root_mode = 0; +static uid_t mp_uid = 0; // owner of mount point(only not specified uid opt) +static gid_t mp_gid = 0; // group of mount point(only not specified gid opt) +static mode_t mp_mode = 0; // mode of mount point static std::string mountpoint; static std::string passwd_file = ""; static bool utility_mode = false; @@ -87,10 +90,12 @@ static bool nocopyapi = false; static bool norenameapi = false; static bool nonempty = false; static bool allow_other = false; -static uid_t s3fs_uid = 0; // default = root. -static gid_t s3fs_gid = 0; // default = root. -static bool is_s3fs_umask = false;// default does not set. +static uid_t s3fs_uid = 0; +static gid_t s3fs_gid = 0; static mode_t s3fs_umask = 0; +static bool is_s3fs_uid = false;// default does not set. +static bool is_s3fs_gid = false;// default does not set. +static bool is_s3fs_umask = false;// default does not set. static bool is_remove_cache = false; // mutex @@ -136,6 +141,7 @@ static int check_for_aws_format(void); static int check_passwd_file_perms(void); static int read_passwd_file(void); static int get_access_keys(void); +static int set_moutpoint_attribute(struct stat& mpst); static int my_fuse_opt_proc(void* data, const char* arg, int key, struct fuse_args* outargs); // fuse interface functions @@ -314,7 +320,9 @@ static int get_object_attribute(const char* path, struct stat* pstbuf, headers_t memset(pstat, 0, sizeof(struct stat)); if(0 == strcmp(path, "/") || 0 == strcmp(path, ".")){ pstat->st_nlink = 1; // see fuse faq - pstat->st_mode = root_mode; + pstat->st_mode = mp_mode; + pstat->st_uid = is_s3fs_uid ? s3fs_uid : mp_uid; + pstat->st_gid = is_s3fs_gid ? s3fs_gid : mp_gid; return 0; } @@ -462,7 +470,7 @@ static int check_object_access(const char* path, int mask, struct stat* pstbuf) // root is allowed all accessing. return 0; } - if(0 != s3fs_uid && s3fs_uid == pcxt->uid){ + if(is_s3fs_uid && s3fs_uid == pcxt->uid){ // "uid" user is allowed all accessing. return 0; } @@ -472,8 +480,8 @@ static int check_object_access(const char* path, int mask, struct stat* pstbuf) } // for "uid", "gid" option - uid_t obj_uid = (0 != s3fs_uid ? s3fs_uid : pst->st_uid); - gid_t obj_gid = (0 != s3fs_gid ? s3fs_gid : pst->st_gid); + uid_t obj_uid = (is_s3fs_uid ? s3fs_uid : pst->st_uid); + gid_t obj_gid = (is_s3fs_gid ? s3fs_gid : pst->st_gid); // compare file mode and uid/gid + mask. mode_t mode = pst->st_mode; @@ -535,11 +543,11 @@ static int check_object_owner(const char* path, struct stat* pstbuf) // root is allowed all accessing. return 0; } - if(0 != s3fs_uid && s3fs_uid == pcxt->uid){ + if(is_s3fs_uid && s3fs_uid == pcxt->uid){ // "uid" user is allowed all accessing. return 0; } - if(pcxt->uid == (0 != s3fs_uid ? s3fs_uid : pst->st_uid)){ + if(pcxt->uid == pst->st_uid){ return 0; } return -EPERM; @@ -622,7 +630,7 @@ static FdEntity* get_local_fent(const char* path, bool is_load) static int put_headers(const char* path, headers_t& meta, bool ow_sse_flg) { int result; - S3fsCurl s3fscurl; + S3fsCurl s3fscurl(true); struct stat buf; FPRNNN("[path=%s]", path); @@ -726,7 +734,7 @@ static int s3fs_readlink(const char* path, char* buf, size_t size) // common function for creation of a plain object static int create_file_object(const char* path, mode_t mode, uid_t uid, gid_t gid) { - FPRNNN("[path=%s][mode=%d]", path, mode); + FPRNNN("[path=%s][mode=%04o]", path, mode); headers_t meta; meta["Content-Type"] = S3fsCurl::LookupMimeType(string(path)); @@ -735,7 +743,7 @@ static int create_file_object(const char* path, mode_t mode, uid_t uid, gid_t gi meta["x-amz-meta-mode"] = str(mode); meta["x-amz-meta-mtime"] = str(time(NULL)); - S3fsCurl s3fscurl; + S3fsCurl s3fscurl(true); return s3fscurl.PutRequest(path, meta, -1, false); // fd=-1 means for creating zero byte object. } @@ -745,7 +753,7 @@ static int s3fs_mknod(const char *path, mode_t mode, dev_t rdev) headers_t meta; struct fuse_context* pcxt; - FPRN("[path=%s][mode=0%o][dev=%lu]", path, mode, rdev); + FPRN("[path=%s][mode=%04o][dev=%lu]", path, mode, rdev); if(NULL == (pcxt = fuse_get_context())){ return -EIO; @@ -766,7 +774,7 @@ static int s3fs_create(const char* path, mode_t mode, struct fuse_file_info* fi) headers_t meta; struct fuse_context* pcxt; - FPRN("[path=%s][mode=%d][flags=%d]", path, mode, fi->flags); + FPRN("[path=%s][mode=%04o][flags=%d]", path, mode, fi->flags); if(NULL == (pcxt = fuse_get_context())){ return -EIO; @@ -801,7 +809,7 @@ static int s3fs_create(const char* path, mode_t mode, struct fuse_file_info* fi) static int create_directory_object(const char* path, mode_t mode, time_t time, uid_t uid, gid_t gid) { - FPRNN("[path=%s][mode=%d][time=%lu][uid=%d][gid=%d]", path, mode, time, uid, gid); + FPRNN("[path=%s][mode=%04o][time=%lu][uid=%d][gid=%d]", path, mode, time, uid, gid); if(!path || '\0' == path[0]){ return -1; @@ -827,7 +835,7 @@ static int s3fs_mkdir(const char* path, mode_t mode) int result; struct fuse_context* pcxt; - FPRN("[path=%s][mode=%d]", path, mode); + FPRN("[path=%s][mode=%04o]", path, mode); if(NULL == (pcxt = fuse_get_context())){ return -EIO; @@ -1089,7 +1097,7 @@ static int rename_large_object(const char* from, const char* to) return result; } - S3fsCurl s3fscurl; + S3fsCurl s3fscurl(true); if(0 != (result = s3fscurl.MultipartRenameRequest(from, to, meta, buf.st_size))){ return result; } @@ -1303,8 +1311,12 @@ static int s3fs_chmod(const char* path, mode_t mode) struct stat stbuf; int nDirType = DIRTYPE_UNKNOWN; - FPRN("[path=%s][mode=%d]", path, mode); + FPRN("[path=%s][mode=%04o]", path, mode); + if(0 == strcmp(path, "/")){ + DPRNNN("Could not change mode for maount point."); + return -EIO; + } if(0 != (result = check_parent_object_access(path, X_OK))){ return result; } @@ -1365,8 +1377,12 @@ static int s3fs_chmod_nocopy(const char* path, mode_t mode) struct stat stbuf; int nDirType = DIRTYPE_UNKNOWN; - FPRNN("[path=%s][mode=%d]", path, mode); + FPRNN("[path=%s][mode=%04o]", path, mode); + if(0 == strcmp(path, "/")){ + DPRNNN("Could not change mode for maount point."); + return -EIO; + } if(0 != (result = check_parent_object_access(path, X_OK))){ return result; } @@ -1441,6 +1457,10 @@ static int s3fs_chown(const char* path, uid_t uid, gid_t gid) FPRN("[path=%s][uid=%d][gid=%d]", path, uid, gid); + if(0 == strcmp(path, "/")){ + DPRNNN("Could not change owner for maount point."); + return -EIO; + } if(0 != (result = check_parent_object_access(path, X_OK))){ return result; } @@ -1518,6 +1538,10 @@ static int s3fs_chown_nocopy(const char* path, uid_t uid, gid_t gid) FPRNN("[path=%s][uid=%d][gid=%d]", path, uid, gid); + if(0 == strcmp(path, "/")){ + DPRNNN("Could not change owner for maount point."); + return -EIO; + } if(0 != (result = check_parent_object_access(path, X_OK))){ return result; } @@ -1602,6 +1626,10 @@ static int s3fs_utimens(const char* path, const struct timespec ts[2]) FPRN("[path=%s][mtime=%zd]", path, ts[1].tv_sec); + if(0 == strcmp(path, "/")){ + DPRNNN("Could not change mtime for maount point."); + return -EIO; + } if(0 != (result = check_parent_object_access(path, X_OK))){ return result; } @@ -1663,6 +1691,10 @@ static int s3fs_utimens_nocopy(const char* path, const struct timespec ts[2]) FPRNN("[path=%s][mtime=%s]", path, str(ts[1].tv_sec).c_str()); + if(0 == strcmp(path, "/")){ + DPRNNN("Could not change mtime for maount point."); + return -EIO; + } if(0 != (result = check_parent_object_access(path, X_OK))){ return result; } @@ -1981,7 +2013,7 @@ static S3fsCurl* multi_head_retry_callback(S3fsCurl* s3fscurl) if(!s3fscurl){ return NULL; } - S3fsCurl* newcurl = new S3fsCurl(); + S3fsCurl* newcurl = new S3fsCurl(s3fscurl->IsUseAhbe()); string path = s3fscurl->GetPath(); string base_path = s3fscurl->GetBasePath(); string saved_path = s3fscurl->GetSpacialSavedPath(); @@ -2881,6 +2913,34 @@ static int get_access_keys(void) return EXIT_FAILURE; } +// +// Check & Set attributes for mount point. +// +static int set_moutpoint_attribute(struct stat& mpst) +{ + mp_uid = geteuid(); + mp_gid = getegid(); + mp_mode = S_IFDIR | (allow_other ? (S_IRWXU | S_IRWXG | S_IRWXO) : S_IRWXU); + + FPRNNN("PROC(uid=%d, gid=%d) - MountPoint(uid=%d, gid=%d, mode=%04o)", mp_uid, mp_gid, mpst.st_uid, mpst.st_gid, mpst.st_mode); + + // check owner + if(0 == mp_uid || mpst.st_uid == mp_uid){ + return true; + } + // check group permission + if(mpst.st_gid == mp_gid || 1 == is_uid_inculde_group(mp_uid, mpst.st_gid)){ + if(S_IRWXG == (mpst.st_mode & S_IRWXG)){ + return true; + } + } + // check other permission + if(S_IRWXO == (mpst.st_mode & S_IRWXO)){ + return true; + } + return false; +} + // This is repeatedly called by the fuse option parser // if the key is equal to FUSE_OPT_KEY_OPT, it's an option passed in prefixed by // '-' or '--' e.g.: -f -d -ousecache=/tmp @@ -2928,11 +2988,10 @@ static int my_fuse_opt_proc(void* data, const char* arg, int key, struct fuse_ar program_name.c_str(), mountpoint.c_str()); return -1; } - root_mode = stbuf.st_mode; // save mode for later usage - if(allow_other){ - root_mode |= (S_IXUSR | S_IXGRP | S_IXOTH | S_IFDIR); - }else{ - root_mode |= S_IFDIR; + if(!set_moutpoint_attribute(stbuf)){ + fprintf(stderr, "%s: MOUNTPOINT: %s permission denied.\n", + program_name.c_str(), mountpoint.c_str()); + return -1; } if(!nonempty){ @@ -2957,10 +3016,20 @@ static int my_fuse_opt_proc(void* data, const char* arg, int key, struct fuse_ar }else if(key == FUSE_OPT_KEY_OPT){ if(strstr(arg, "uid=") != 0){ s3fs_uid = strtoul(strchr(arg, '=') + sizeof(char), 0, 10); + if(0 != geteuid() && 0 == s3fs_uid){ + fprintf(stderr, "%s: root user can only specify uid=0.\n", program_name.c_str()); + return -1; + } + is_s3fs_uid = true; return 1; // continue for fuse option } if(strstr(arg, "gid=") != 0){ s3fs_gid = strtoul(strchr(arg, '=') + sizeof(char), 0, 10); + if(0 != getegid() && 0 == s3fs_gid){ + fprintf(stderr, "%s: root user can only specify gid=0.\n", program_name.c_str()); + return -1; + } + is_s3fs_gid = true; return 1; // continue for fuse option } if(strstr(arg, "umask=") != 0){ @@ -3125,6 +3194,16 @@ static int my_fuse_opt_proc(void* data, const char* arg, int key, struct fuse_ar FdManager::SetPageSize(pagesize); return 0; } + if(strstr(arg, "ahbe_conf=") != 0){ + string ahbe_conf = strchr(arg, '=') + sizeof(char); + if(!AdditionalHeader::get()->Load(ahbe_conf.c_str())){ + fprintf(stderr, "%s: failed to load ahbe_conf file(%s).\n", + program_name.c_str(), ahbe_conf.c_str()); + return -1; + } + AdditionalHeader::get()->Dump(); + return 0; + } if(strstr(arg, "noxmlns") != 0){ noxmlns = true; return 0; diff --git a/src/s3fs_util.cpp b/src/s3fs_util.cpp index 80a60d1..2d88d78 100644 --- a/src/s3fs_util.cpp +++ b/src/s3fs_util.cpp @@ -825,6 +825,27 @@ void show_help (void) " passwd_file (default=\"\")\n" " - specify which s3fs password file to use\n" "\n" + " ahbe_conf (default=\"\" which means disabled)\n" + " - This option specifies the configuration file path which\n" + " file is the additional HTTP header by file(object) extension.\n" + " The configuration file format is below:\n" + " -----------\n" + " line = [file suffix] HTTP-header [HTTP-values]\n" + " file suffix = file(object) suffix, if this field is empty,\n" + " it means \"*\"(all object).\n" + " HTTP-header = additional HTTP header name\n" + " HTTP-values = additional HTTP header value\n" + " -----------\n" + " Sample:\n" + " -----------\n" + " .gz Content-Encoding gzip\n" + " .Z Content-Encoding compress\n" + " X-S3FS-MYHTTPHEAD myvalue\n" + " -----------\n" + " A sample configuration file is uploaded in \"test\" directory.\n" + " If you specify this option for set \"Content-Encoding\" HTTP \n" + " header, please take care for RFC 2616.\n" + "\n" " connect_timeout (default=\"10\" seconds)\n" " - time to wait for connection before giving up\n" "\n" diff --git a/test/Makefile.am b/test/Makefile.am index 389eff3..0b7d9c6 100644 --- a/test/Makefile.am +++ b/test/Makefile.am @@ -4,5 +4,6 @@ EXTRA_DIST = \ integration-test-common.sh \ require-root.sh \ small-integration-test.sh \ - mergedir.sh + mergedir.sh \ + sample_ahbe.conf diff --git a/test/sample_ahbe.conf b/test/sample_ahbe.conf new file mode 100644 index 0000000..a93e0a5 --- /dev/null +++ b/test/sample_ahbe.conf @@ -0,0 +1,41 @@ +# S3FS: Samlpe ahbe_conf parameter file. +# +# This file is configuration file for additional header by extension(ahbe). +# s3fs loads this file at starting. +# +# Format: +# line = [file suffix] HTTP-header [HTTP-header-values] +# file suffix = file(object) suffix, if this field is empty, +# it means "*"(all object). +# HTTP-header = additional HTTP header name +# HTTP-header-values = additional HTTP header value +# +# +# +# Example: +# " Content-Encoding gzip" --> all object +# ".gz Content-Encoding gzip" --> only ".gz" extension file +# +# Notice: +# If you need to set all object, you can specify without "suffix". +# Then all of object(file) is added additional header. +# If you have this configuration file for Content-Encoding, you should +# know about RFC 2616. +# +# "The default (identity) encoding; the use of no transformation +# whatsoever. This content-coding is used only in the Accept- +# Encoding header, and SHOULD NOT be used in the Content-Encoding +# header." +# +.gz Content-Encoding gzip +.Z Content-Encoding compress +.bz2 Content-Encoding bzip2 +.svgz Content-Encoding gzip +.svg.gz Content-Encoding gzip +.tgz Content-Encoding gzip +.tar.gz Content-Encoding gzip +.taz Content-Encoding gzip +.tz Content-Encoding gzip +.tbz2 Content-Encoding gzip +gz.js Content-Encoding gzip +