octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2025-01-09 16:06:46 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,179 Commits 1 Branch 23 Tags 12 MiB
a4f694c345
Commit Graph

1219 Commits

Author SHA1 Message Date
Takeshi Nakatani
01b3caa38c Fixed errors of cppcheck 2.12.0 2023-09-24 19:55:02 +09:00
Andrew Gaul
01189e99fc
Store mvnode in vector instead of manual linked list (#2312) 
This simplifies code and avoids manual memory management.  References #2261.
 2023-09-13 22:32:15 +09:00
Andrew Gaul
3d73d5a687
Delete unneeded constructors and assignment operators (#2309) 2023-09-06 23:52:10 +09:00
Andrew Gaul
fa3a472c6b
Remove several calls to free (#2308) 2023-09-06 23:50:33 +09:00
Andrew Gaul
4d5632912a
Initialize variable before use (#2302) 
clang-analyzer found a path where this could be used without
initialization.
 2023-09-06 23:32:49 +09:00
Takeshi Nakatani
a74034a012 Fixed a bug with setting the statvfs value 2023-09-05 09:03:11 -07:00
AdamQQQ
68c45ce791
s3fs: print unmounting hint when the mount point is stale (#2295) 
When the error code returned by the stat information of the mount point
is ENOTCONN, print unmount command hint for user to fix.

Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
 2023-09-03 10:50:09 +09:00
Andrew Gaul
e8cb6d6d34
Abort after failed MPU (#2298) 
This reclaims storage after a failed MPU which caused OutOfMemory
issues in #2291.
 2023-08-29 23:29:16 +09:00
Andrew Gaul
7bb9609827
Return errors from AutoFdEntity::Open (#2296) 
Found via pjdfstest which creates a PATH_MAX path that should return
NAMETOOLONG.
 2023-08-29 22:57:30 +09:00
Andrew Gaul
82107f4b6c
Skip is_uid_include_group when GID available (#2292) 
This can avoid an expensive computation which is 20% of test runtime.
 2023-08-27 15:24:33 +09:00
Eryu Guan
ee49ca4abf
s3fs: print fuse context in s3fs fuse operations (#2274) 
Print fuse context like pid in fuse operations, so we know which process
is triggering this operation.

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-08-27 14:00:57 +09:00
Andrew Gaul
48548f0896
Remove unused functions (#2289) 2023-08-22 23:17:43 +09:00
Andrew Gaul
981e97ee76
Use default move constructor (#2288) 
This is identical to the explicit one.
 2023-08-22 23:14:09 +09:00
Andrew Gaul
a568aa70fd
Replace uses of emplace with operator=(&&) (#2287) 
emplace does not overwrite the value if the key already exists.  C++17 has
insert_or_assign but C++11 only has operator= which invokes the default
constructor.  Follows on to 6781ef5bd1.
 2023-08-22 23:12:12 +09:00
Takeshi Nakatani
218adcb29b
Fixed errors in cache.cpp from cppcheck 2.11.1 (#2286) 2023-08-20 19:00:20 +09:00
Andrew Gaul
6c55bcfdd8
Own values in add_header (#2285) 
Also fix up indentation.
 2023-08-20 18:59:18 +09:00
Andrew Gaul
8d04ee3e01
Own values in stat_cache and symlink_cache (#2284) 
This removes an unnecessary use of unique_ptr.
 2023-08-20 12:10:47 +09:00
Takeshi Nakatani
6781ef5bd1 Reverted to direct array access instead of std::map emplace 2023-08-20 09:44:38 +09:00
Andrew Gaul
7e94b64ae7
Use unique_ptr in SSL functions (#2282) 
References #2261.
 2023-08-19 23:29:00 +09:00
Andrew Gaul
50f6c38c84
Replace xattr_value with std::string (#2280) 2023-08-19 11:12:43 +09:00
Takeshi Nakatani
280ed5d706
Additional fix for #2276(Convert BodyData to std::string) (#2278) 2023-08-18 00:35:50 +09:00
Andrew Gaul
2518ff3568
Convert BodyData to std::string (#2276) 
This is simpler and avoids some copies.
 2023-08-17 22:49:41 +09:00
Andrew Gaul
c65ce8a42c
Add clang-tidy to CI (#2270) 2023-08-17 22:42:11 +09:00
Andrew Gaul
b2bb12fd2c
Remove unneeded explicit std::string constructors (#2273) 
std::string(const char*) implicitly constructs these.  The remaining call sites
requires string literals from C++14.
 2023-08-17 22:12:28 +09:00
Andrew Gaul
7f30353fb9
Return std::unique_ptr from S3fsCurl callbacks (#2272) 
References #2261.
 2023-08-17 22:08:56 +09:00
Andrew Gaul
67e6b9e495
Simplify xattr_value with owned values (#2262) 
References #2261.
 2023-08-15 22:54:46 +09:00
Andrew Gaul
6823c5a7ec
Enable clang-tidy cppcoreguidelines (#2269) 2023-08-15 22:12:33 +09:00
Andrew Gaul
d1272d296a
Tighten up CLI argument handling (#2268) 
This ensures that each option is only handled once.
 2023-08-15 21:45:38 +09:00
Andrew Gaul
d120e54284
Improve illegal bucket name error message (#2263) 
This may help users debug situations like:

https://stackoverflow.com/questions/76359564/why-does-mounting-s3fs-bucket-on-centos-7-using-fstab-fail-but-mount-a-works
 2023-08-15 21:31:05 +09:00
Andrew Gaul
e157d811cb
Use std::string::compare and operator== where possible (#2256) 2023-08-15 21:22:36 +09:00
Andrew Gaul
56a4e67009
Replace more raw pointers with std::unique_ptr (#2255) 2023-08-14 00:03:10 +09:00
Takeshi Nakatani
5b93765802 Fixed a warning for compiling C/C++ codes 2023-08-13 20:49:39 +09:00
Takeshi Nakatani
acea1d33f9 Fixed string test for s3fs_base64 2023-08-13 20:48:38 +09:00
Andrew Gaul
528a61718d
Convert manual memory allocations to std::unique_ptr (#2253) 2023-08-11 23:26:07 +09:00
Andrew Gaul
c5a75a1fb2
Delete copy constructors and assignment operators (#2257) 
One of these was buggy and others had the wrong parameters and return
types.
 2023-08-11 13:12:03 +09:00
Andrew Gaul
3790a0f8b4
Calculate MD5 without using a temporary file (#2252) 
This mirrors the SHA256 code.
 2023-08-07 00:17:15 +09:00
Andrew Gaul
779afe5d62
Make help more consistent (#2251) 2023-08-06 22:25:10 +09:00
Andrew Gaul
26b5658d70
Wrap ps3fscred with std::unique_ptr (#2250) 
This removes many manual memory deallocations.
 2023-08-06 22:23:25 +09:00
Andrew Gaul
c568a69452
Return std::string from base64 encoding function (#2248) 
This is avoids manual memory allocations.
 2023-08-06 22:22:02 +09:00
Andrew Gaul
13ad53eef7
Convert most std::list to std::vector (#2247) 
This tends to be more efficient due to fewer allocations.  Also fix std::sort
comparator which should be strictly less than.
 2023-08-05 10:05:32 +09:00
Andrew Gaul
b5c3fc0a08
Convert fixed-size allocations to C++11 std::array (#2242) 
This is safer and more efficient.
 2023-08-05 09:36:22 +09:00
Andrew Gaul
b29f8d0f2b
Use C++ enum class for most enums (#2241) 
This promotes type-safety.
 2023-07-30 22:53:17 +09:00
Andrew Gaul
5699875e30
Use C++11 emplace where possible (#2240) 
This is more concise and sometimes more efficient.
 2023-07-30 22:51:20 +09:00
Andrew Gaul
a7b38a6940
Address stray warnings (#2237) 2023-07-29 09:19:18 +09:00
Andrew Gaul
1f04165a33
Convert most str callers to C++11 std::to_string (#2238) 
Remaining ones handle timespec.
 2023-07-28 18:21:55 +09:00
Andrew Gaul
36db898d01
Use C++11 std::map::erase return value (#2236) 2023-07-27 23:34:43 +09:00
Andrew Gaul
a4a2841c05
Use C++11 nullptr instead of 0 or NULL (#2234) 
This improves type-safety.
 2023-07-27 21:56:58 +09:00
Andrew Gaul
0ece204393 Fix -Wshorten-64-to-32 warnings 2023-07-27 12:23:26 +09:00
Andrew Gaul
6344d74ae3
Replace some raw pointers with std::unique_ptr (#2195) 
This simplifies code paths and makes memory leaks less likely.  It
also makes memory ownership more explicit by requiring std::move.
This commit requires C++11.  References #2179.
 2023-07-27 09:12:28 +09:00
Takeshi Nakatani
faec0d9d15 Refixed for cppcheck 2.1x 2023-07-26 07:55:33 +09:00
Takeshi Nakatani
cb3dc28e6e Supported cppcheck 2.10(and changed std from c++03 to c++11 for RHEL7) 2023-07-25 08:08:06 +09:00
Andrew Gaul
2405706643
Insert SSE headers when appropriate (#2228) 
References #2218.  References #2227.
 2023-07-23 16:17:34 +09:00
Eryu Guan
7978395083 Use smart pointer to manage pcfstat object 
Previously pcfstat points to a raw pointer, and it may be leaked if
function returned before deleting it.

So use smart pointer to automatically release the object.

Note that currently s3fs only uses c++03, so we use auto_ptr here, not
unique_ptr, which requires c++11.

Fixes: 6ca5a24a7f ("Fix two inconsistency issues between stat cache and cache file (#2152)")
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-07-19 20:15:55 +09:00
Takeshi Nakatani
d0a944fcaa Fixed data race about fuse_fill_dir_t function and data pointer 2023-07-14 22:32:12 +09:00
Andrew Gaul
537384e9b5
Guard filler calls with filled check (#2215) 
Follows on to e650d8c55c.
 2023-07-13 22:46:19 +09:00
Andrew Gaul
e650d8c55c
Explicitly handle CommonPrefixes with nocompat_dir (#2212) 
Previously the test missed listing implicit directories and another
test was incorrect.  This fixes a regression from 1.91.
 2023-07-13 21:15:34 +09:00
Takeshi Nakatani
9663215bb4 Fixed data race at OPENSSL_sk_dup/free in libcurl 2023-07-12 22:51:57 +09:00
Takeshi Nakatani
b2537052ef Fixed data race in threads found thread sanitizer 2023-07-09 20:53:27 +09:00
Eryu Guan
2e51908bec Check FdEntity::Open() status correctly 
FdEntity::Open() returns -errno on error, but FdManager::Open() only
checks if its ret is -1. This may lead to use '-errno' as pseudo fd
in next read or write, which would fail due to '-errno' is not in
fent map.

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-07-05 08:50:12 +09:00
LiuBingrun
a8edbd8622
fix streamuplod not working (#2204) 
fuse_main will do fork to daemonize. if ThreadPool is created before
this, worker thread will not be placed in child process.

Move ThreadPool Init to s3fs_init and destory to s3fs_destory.

Signed-off-by: liubingrun <liubr1@chinatelecom.cn>
 2023-07-02 10:56:59 +09:00
Alex Fishman
03066ea29a
Fix extended attribute support when using FUSE-T (#2201) 
* Add support for FUSE-T on macos

Signed-off-by: Alex Fishman <alex@fuse-t.org>
Signed-off-by: alex <alex@alex-NUC10.lan>

* Ignore value pointer when size is zero on setattrx

Signed-off-by: Alex Fishman <alex@fuse-t.org>

---------

Signed-off-by: Alex Fishman <alex@fuse-t.org>
Signed-off-by: alex <alex@alex-NUC10.lan>
 2023-06-26 22:05:37 +09:00
Andrew Gaul
6448c8f1a8
Protect FdEntity::physical_fd with fdent_lock (#2194) 
* Protect FdEntity::physical_fd with fdent_lock
Found via ThreadSanitizer.
 2023-06-25 16:43:15 +09:00
Andrew Gaul
3b6688253f
Address cppcheck 2.10 warnings (#2163) 
Disable newer cppcheck until we can diagnose this further.
References #2162.
 2023-06-25 16:04:16 +09:00
Takeshi Nakatani
580775b47c Removed unnecessary debug output 2023-06-07 20:48:50 -07:00
Eryu Guan
6ca5a24a7f
Fix two inconsistency issues between stat cache and cache file (#2152) 
* Fix inconsistency between stat cache file and cache file

We unlock stat cache file too early in FdEntity::Open(), and would
truncate cache file and update stat cache file, so there's a window that
stat cache doesn't reflect cache file status.

Suggested-by: Takeshi Nakatani <ggtakec@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

* Mark pagelist as unloaded if cache file has been truncated

If cache file size doesn't match object size, the cache file might be
corrupted, so invalidate it and save new cache stat file.

Suggested-by: Takeshi Nakatani <ggtakec@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

---------

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-05-30 18:39:50 +09:00
Takeshi Nakatani
0d6b02090e
Revert "Update curl.cpp: reduce memory cache use (#2157)" (#2170) 
This reverts commit 5b487f651a.
 2023-05-27 20:33:43 +09:00
Tan Guofu
5b487f651a
Update curl.cpp: reduce memory cache use (#2157) 
flush the file and clean the page cache when filepart download done
 2023-05-27 19:15:47 +09:00
Takeshi Nakatani
1fc0e52dc3 Fixed mount point stat flag to be thread safe 2023-05-21 13:46:06 +09:00
Andrew Gaul
59c3b26655
Fix clang-tidy 16 nits (#2158) 2023-05-14 01:59:26 +09:00
Takeshi Nakatani
8296fe32cb
Directly and simplify requests in mount point checks (#2155) 2023-05-14 01:29:24 +09:00
Takeshi Nakatani
ca9a257eec
Allow mount points without directory objects by compat_dir (#2153) 2023-05-07 09:15:27 +09:00
Takeshi Nakatani
6d4bb59865
Corresponded to upload in case of calling release without flush (#2150) 2023-04-23 22:59:04 +09:00
Ottavia Balducci
9b75abfbe6
New option: bucket_size (#2148) 
* Added bucket_size option
 2023-04-23 14:04:38 +09:00
Takeshi Nakatani
c4f95f14cb
Fixed a bug about attributes of mount point (#2147) 2023-04-15 14:34:39 +09:00
Takeshi Nakatani
9c74014443
Fixed a bug in handling file names containing CR(0x1D) (#2136) 2023-03-26 13:19:16 +09:00
Takeshi Nakatani
a25cb9e07a
Fixed a bug when reading a reduced file without flushing (#2133) 2023-03-26 11:45:21 +09:00
Takeshi Nakatani
9648eba5bb
Changed to rename cache files when renaming large files (#2135) 2023-03-21 14:15:34 +09:00
Takeshi Nakatani
18495c44aa
Added cache clearing on upload error (#2127) 2023-03-13 21:22:13 +09:00
Leon
4f22354aae
Fix an error to calculate no data page bytes (#2122) 
Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
 2023-03-11 17:23:56 +09:00
Garen Chan
9d00b8d4a8
Fixed segmentation fault caused by file write failure (#2123) 2023-03-11 16:45:56 +09:00
Takeshi Nakatani
526700f2de
Updated files for support ALPINE v3.17 (#2116) 2023-03-05 13:30:59 +09:00
Takeshi Nakatani
a25a9450a6
Fixed to fail when the mount point directory is not existed (#2114) 2023-03-05 13:03:48 +09:00
Takeshi Nakatani
f8a825e9b9
multipart upload id is converted by url encode (#2097) 2023-02-18 11:40:25 +09:00
Takeshi Nakatani
dbddd762a9
Fixed token update process using ext-cred library (#2101) 2023-02-18 09:49:05 +09:00
Takeshi Nakatani
e715b77307
Added the function to update mtime/ctime of the parent directory (#2016) 2023-02-12 17:59:40 +09:00
Takeshi Nakatani
d1388ff446 Added proxy and proxy_cred_file option 2023-02-08 21:53:34 +09:00
Takeshi Nakatani
38e8a830c9 Fixed a deadlock bug when specifying iam role(IAMv2) 2023-01-31 20:53:39 +09:00
Takeshi Nakatani
7605c2e8fb Updated help contents and man pages for streamupload and etc 2023-01-31 20:53:20 +09:00
Takeshi Nakatani
16bc44948e Retry BucketCheck containing directory paths 2023-01-09 21:18:47 +09:00
Takeshi Nakatani
e4f85c1e08 Fixed renaming bug with SSE 2023-01-09 21:17:48 +09:00
Takeshi Nakatani
222110e153 Fixed a bug about multipart upload with SSE 2023-01-07 09:12:55 +05:30
Andrew Gaul
0ba49518e9
Make some methods and parameters const (#2078) 
This requires making some locks mutable.
 2023-01-04 20:23:39 +09:00
Takeshi Nakatani
23a8124c51 Compatible with OpenSSL 3.0 2022-11-27 21:46:08 +09:00
Takeshi Nakatani
d8e4e34b74 Fixed a warning by compiling on fedora 37 2022-11-23 21:57:31 +09:00
Andrew Gaul
956eb77369
Updates for clang-tidy 15 (#2058) 2022-11-21 23:29:35 +09:00
Takeshi Nakatani
465986e397
Fixed a bug when end of path for mount point is multi slash (#2057) 2022-11-19 15:07:27 +09:00
Takeshi Nakatani
aec7efc3af
Updated comments in curl.cpp (related to #1827) (#2052) 2022-11-05 11:45:28 +09:00
Andrew Gaul
30353f1a83
Remove unnecessary calls to dup (#1827) 
These functions already use pread and keep track of an explicit offset
for the SHA-256 calculations.
 2022-11-05 10:24:41 +09:00
Takeshi Nakatani
2e77920943
Added support for xattr as POSIX ACL (#2039) 2022-10-22 21:48:02 +09:00
Takeshi Nakatani
4a813aec42
Support the object under no directory object path by compat_dir (#2023) 2022-10-22 15:12:00 +09:00