Commit Graph

81 Commits

Author SHA1 Message Date
Andrew Gaul c97f7a2a13
Address clang-tidy 18 warnings (#2428) 2024-03-07 01:04:22 +09:00
Takeshi Nakatani 517574c40c Fixed a bug in fdatasync(fsync) 2024-02-06 14:11:37 +09:00
Andrew Gaul 68bbfee8ea
Address clang-tidy modernize-deprecated-headers (#2370) 2023-11-19 10:00:16 +09:00
Andrew Gaul 4845831f93
Convert some const to constexpr (#2342)
This guarantees that the function or value will resolve at compile-time.
2023-11-14 22:15:17 +09:00
Takeshi Nakatani f9d3941d9d Fixed a bug in the re-upload part of Streamupload 2023-11-10 10:41:26 +09:00
Takeshi Nakatani d0c4b5c763 Fixed a bug in exclusive control of pagelist in FdEntity class 2023-09-26 21:26:17 +09:00
Andrew Gaul 61df7bf42c
Use std::unique_ptr for fclose (#2318)
References #2261.
2023-09-25 23:55:11 +09:00
Andrew Gaul 4d5632912a
Initialize variable before use (#2302)
clang-analyzer found a path where this could be used without
initialization.
2023-09-06 23:32:49 +09:00
Andrew Gaul e8cb6d6d34
Abort after failed MPU (#2298)
This reclaims storage after a failed MPU which caused OutOfMemory
issues in #2291.
2023-08-29 23:29:16 +09:00
Andrew Gaul 7bb9609827
Return errors from AutoFdEntity::Open (#2296)
Found via pjdfstest which creates a PATH_MAX path that should return
NAMETOOLONG.
2023-08-29 22:57:30 +09:00
Andrew Gaul b2bb12fd2c
Remove unneeded explicit std::string constructors (#2273)
std::string(const char*) implicitly constructs these.  The remaining call sites
requires string literals from C++14.
2023-08-17 22:12:28 +09:00
Takeshi Nakatani 5b93765802 Fixed a warning for compiling C/C++ codes 2023-08-13 20:49:39 +09:00
Andrew Gaul b29f8d0f2b
Use C++ enum class for most enums (#2241)
This promotes type-safety.
2023-07-30 22:53:17 +09:00
Andrew Gaul a7b38a6940
Address stray warnings (#2237) 2023-07-29 09:19:18 +09:00
Andrew Gaul 1f04165a33
Convert most str callers to C++11 std::to_string (#2238)
Remaining ones handle timespec.
2023-07-28 18:21:55 +09:00
Andrew Gaul 36db898d01
Use C++11 std::map::erase return value (#2236) 2023-07-27 23:34:43 +09:00
Andrew Gaul a4a2841c05
Use C++11 nullptr instead of 0 or NULL (#2234)
This improves type-safety.
2023-07-27 21:56:58 +09:00
Andrew Gaul 0ece204393 Fix -Wshorten-64-to-32 warnings 2023-07-27 12:23:26 +09:00
Andrew Gaul 6344d74ae3
Replace some raw pointers with std::unique_ptr (#2195)
This simplifies code paths and makes memory leaks less likely.  It
also makes memory ownership more explicit by requiring std::move.
This commit requires C++11.  References #2179.
2023-07-27 09:12:28 +09:00
Takeshi Nakatani faec0d9d15 Refixed for cppcheck 2.1x 2023-07-26 07:55:33 +09:00
Eryu Guan 7978395083 Use smart pointer to manage pcfstat object
Previously pcfstat points to a raw pointer, and it may be leaked if
function returned before deleting it.

So use smart pointer to automatically release the object.

Note that currently s3fs only uses c++03, so we use auto_ptr here, not
unique_ptr, which requires c++11.

Fixes: 6ca5a24a7f ("Fix two inconsistency issues between stat cache and cache file (#2152)")
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
2023-07-19 20:15:55 +09:00
Takeshi Nakatani b2537052ef Fixed data race in threads found thread sanitizer 2023-07-09 20:53:27 +09:00
Andrew Gaul 6448c8f1a8
Protect FdEntity::physical_fd with fdent_lock (#2194)
* Protect FdEntity::physical_fd with fdent_lock
Found via ThreadSanitizer.
2023-06-25 16:43:15 +09:00
Eryu Guan 6ca5a24a7f
Fix two inconsistency issues between stat cache and cache file (#2152)
* Fix inconsistency between stat cache file and cache file

We unlock stat cache file too early in FdEntity::Open(), and would
truncate cache file and update stat cache file, so there's a window that
stat cache doesn't reflect cache file status.

Suggested-by: Takeshi Nakatani <ggtakec@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

* Mark pagelist as unloaded if cache file has been truncated

If cache file size doesn't match object size, the cache file might be
corrupted, so invalidate it and save new cache stat file.

Suggested-by: Takeshi Nakatani <ggtakec@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

---------

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
2023-05-30 18:39:50 +09:00
Takeshi Nakatani 18495c44aa
Added cache clearing on upload error (#2127) 2023-03-13 21:22:13 +09:00
Takeshi Nakatani f8a825e9b9
multipart upload id is converted by url encode (#2097) 2023-02-18 11:40:25 +09:00
Andrew Gaul 0ba49518e9
Make some methods and parameters const (#2078)
This requires making some locks mutable.
2023-01-04 20:23:39 +09:00
Takeshi Nakatani c491fbeabc Replace uses of lock_already_held flag with AutoLock::Type 2022-07-30 16:20:43 +09:00
Andrew Gaul 48e9e51f4f
Remove more unneeded headers identified by IWYU (#2011) 2022-07-30 12:06:47 +09:00
Andrew Gaul 3928a7e359
Remove more uses of const_cast (#2006)
Follows on to #2004.
2022-07-28 23:37:15 +09:00
Takeshi Nakatani 4b2f3fecb5 Set mtime/ctime/atime of all objects as nanosecond 2022-07-28 13:47:03 +09:00
Andrew Gaul e0655008b3
Protect pending_status in UploadPending (#1992)
This requires avoiding double-locking in RowFlush.  References #1991.
2022-07-22 23:30:04 +09:00
Takeshi Nakatani 22f2392fca Fixed bugs about stream upload 2022-07-19 21:29:56 +09:00
Takeshi Nakatani faddb4900f Merged the code corresponding to the mknod fix(f11eb7d) 2022-07-17 22:20:45 +09:00
Takeshi Nakatani b0eeaa6679 Reflected the result of the review in the code 2022-07-17 22:20:45 +09:00
Takeshi Nakatani d22e1dc018 Add the stream upload which starts uploading parts before Flush 2022-07-17 22:20:45 +09:00
Takeshi Nakatani f11eb7d69b Fixed a bug that regular files could not be created by mknod 2022-06-29 16:56:19 +09:00
Takeshi Nakatani 73b49c1038 Fixed a bug that regular files could not be created by mknod 2022-06-29 16:56:19 +09:00
Andrew Gaul 8a5c4306f5
Preserve sub-second precision where possible (#1915) 2022-02-23 23:58:51 +09:00
Andrew Gaul 08adffd2fe
Fix typos (#1916) 2022-02-23 23:31:52 +09:00
Takeshi Nakatani 839a33de49
Fixed not to call Flush even if the file size is increased (#1887)
Changed s3fs_truncate function.
This change reduces the number of file uploads if the file size is changed.

On macOS, I have found that the truncate call when "size=0" cannot reflect the file size.(This reason is not understood...)
To avoid this, only when "size=0", the flush method is called as before.

Other than that, I found a bug in FdEntity::Open() and fixed it.

Fixes #1875.
2022-02-15 21:29:07 +09:00
Takeshi Nakatani 1678803566 Added S3fsCred class and moved Credential related processing in it 2022-02-13 21:38:30 +09:00
Andrew Gaul f6ed972926
Always flush open files with O_CREAT flag (#1879)
Previously s3fs only created files that had dirty data and not those
with zero-bytes.  Regression from
771bbfeac5.  References #1013.  Found
via pjdfstest.  References #1589.
2022-01-30 22:02:37 +09:00
Takeshi Nakatani 023aaf7dff Fixed wrong stat of cache after new creation file
And added a test for stat of cache after new creation file
2021-10-17 16:10:14 +09:00
Takeshi Nakatani 2f412804e2 Fixed forgetting to clear the dirty flag for meta information
Addressed an error in macos cpp check
2021-10-15 22:54:55 +09:00
VVoidV d9f2d17040
1. fix RowFlush can not upload last part smaller than 5MB using NoCacheMultipartPost; (#1753)
2. fix deadlock in UploadPendingMeta
2021-08-31 00:41:47 +09:00
Takeshi Nakatani a868c0656e Changed etaglist_t from string list to new structure etagpairs list 2021-08-16 09:27:12 +09:00
Takeshi Nakatani 199b3d4709 Fixed a bug in disk free space calculation 2021-07-18 14:52:11 +09:00
Takeshi Nakatani 945cc2ac54 Added UntreatedParts class instead of untreated upload info members in PseudoFdInfo 2021-07-12 21:45:29 +09:00
Takeshi Nakatani c30acbbf90 Splited some methods of FdEntity class by uploading mode
- Splited FdEntity::Write method by uploading mode
- Splited FdEntity::RowFlush method by uplading mode
2021-07-10 17:24:34 +09:00