ccdcccd44c
Fix DeadLock in FdManager::ChangeEntityToTempPath ( #2455 )
...
commit e3b50ad
2024-05-11 11:29:40 +09:00
54aa278df0
Fixed errors reported by cppcheck 2.13.0 ( #2400 )
2024-01-25 00:46:45 +09:00
e3b50ad3e1
Convert FdEntity to std::unique_ptr ( #2383 )
2023-12-07 23:56:35 +09:00
4845831f93
Convert some const to constexpr ( #2342 )
...
This guarantees that the function or value will resolve at compile-time.
2023-11-14 22:15:17 +09:00
3856637cd2
s3fs: add option free_space_ratio to control cache size ( #2351 )
...
* Try to cleanup cache directory when initing without enough disk space
Also optimize log messages to print detailed errors to the user.
Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
* s3fs: add option free_space_ratio to control cache size
Since the ensure_diskfree option is not convenient enough, we have added
a new option "-o free_space_ratio" to control the space used by the s3fs
cache based on the current disk size.
The value of this option can be between 0 and 100. It will control the
size of the cache according to this ratio to ensure that the idle ratio
of the disk is greater than this value.
For example, when the value is 10 and the disk space is 50GB, it will
ensure that the disk will reserve at least 50GB * 10% = 5GB of remaining
space.
Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
---------
Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
2023-10-20 18:11:47 +09:00
361e10d09c
Add scope_guard for ad-hoc resource management ( #2313 )
...
References #2261 . Suggested by:
https://stackoverflow.com/questions/10270328/the-simplest-and-neatest-c11-scopeguard
2023-09-26 07:52:55 +09:00
61df7bf42c
Use std::unique_ptr for fclose ( #2318 )
...
References #2261 .
2023-09-25 23:55:11 +09:00
7bb9609827
Return errors from AutoFdEntity::Open ( #2296 )
...
Found via pjdfstest which creates a PATH_MAX path that should return
NAMETOOLONG.
2023-08-29 22:57:30 +09:00
b2bb12fd2c
Remove unneeded explicit std::string constructors ( #2273 )
...
std::string(const char*) implicitly constructs these. The remaining call sites
requires string literals from C++14.
2023-08-17 22:12:28 +09:00
e157d811cb
Use std::string::compare and operator== where possible ( #2256 )
2023-08-15 21:22:36 +09:00
5b93765802
Fixed a warning for compiling C/C++ codes
2023-08-13 20:49:39 +09:00
36db898d01
Use C++11 std::map::erase return value ( #2236 )
2023-07-27 23:34:43 +09:00
a4a2841c05
Use C++11 nullptr instead of 0 or NULL ( #2234 )
...
This improves type-safety.
2023-07-27 21:56:58 +09:00
faec0d9d15
Refixed for cppcheck 2.1x
2023-07-26 07:55:33 +09:00
cb3dc28e6e
Supported cppcheck 2.10(and changed std from c++03 to c++11 for RHEL7)
2023-07-25 08:08:06 +09:00
2e51908bec
Check FdEntity::Open() status correctly
...
FdEntity::Open() returns -errno on error, but FdManager::Open() only
checks if its ret is -1. This may lead to use '-errno' as pseudo fd
in next read or write, which would fail due to '-errno' is not in
fent map.
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
2023-07-05 08:50:12 +09:00
3b6688253f
Address cppcheck 2.10 warnings ( #2163 )
...
Disable newer cppcheck until we can diagnose this further.
References #2162 .
2023-06-25 16:04:16 +09:00
c491fbeabc
Replace uses of lock_already_held flag with AutoLock::Type
2022-07-30 16:20:43 +09:00
48e9e51f4f
Remove more unneeded headers identified by IWYU ( #2011 )
2022-07-30 12:06:47 +09:00
4b2f3fecb5
Set mtime/ctime/atime of all objects as nanosecond
2022-07-28 13:47:03 +09:00
d75c6d6538
Changed the message level(ERR to INFO) of skip cleanup cache ( #1946 )
2022-05-22 14:36:56 +09:00
d842d45b2b
Fixed a bug about truncation for shrinking file
2022-03-02 22:41:10 +09:00
839a33de49
Fixed not to call Flush even if the file size is increased ( #1887 )
...
Changed s3fs_truncate function.
This change reduces the number of file uploads if the file size is changed.
On macOS, I have found that the truncate call when "size=0" cannot reflect the file size.(This reason is not understood...)
To avoid this, only when "size=0", the flush method is called as before.
Other than that, I found a bug in FdEntity::Open() and fixed it.
Fixes #1875 .
2022-02-15 21:29:07 +09:00
4dfe2bfdd7
Include climits to support musl libc
...
PATH_MAX constant is not visible from any of currently included header
files in system with musl libc, where compilation fails with an error
below. The constant is defined in limits.h which is directly include via
climits header file.
fdcache.cpp: In static member function 'static FILE* FdManager::MakeTempFile()':
fdcache.cpp:381:14: error: 'PATH_MAX' was not declared in this scope
381 | char cfn[PATH_MAX];
| ^~~~~~~~
Fixes: d67b83e671
2022-02-14 09:19:30 +09:00
1678803566
Added S3fsCred class and moved Credential related processing in it
2022-02-13 21:38:30 +09:00
beecf32dff
fclose(FILE*) instead of close(fileno(FILE*)) ( #1758 )
...
This is the same thing but confuses Coverity.
2021-09-01 19:41:55 +09:00
34f89e5936
Added fake_diskfree option to deceive free disk space for test
2021-08-02 11:04:42 +09:00
f505c8224e
Explicitly specify lock type ( #1701 )
...
This makes it more clear and type-safe if the caller already has the
lock. Follows on to 84174c560d
2021-06-27 15:15:48 +09:00
2154e898bc
Fix typos ( #1700 )
2021-06-27 11:22:33 +09:00
6edf3d6427
Updated to clearly output message about the file descriptor type(pseudo/physical)
2021-06-25 19:18:08 +09:00
d67b83e671
Allow configuration for temporary files directory
2021-06-16 21:29:58 +09:00
c2c56d0263
Added info object about multipart uploading for each pseudo fd
...
(and fixed typo about method name)
2021-06-04 22:42:58 +09:00
ac578d188e
Introduced pseudo fd and separated fd for each file opening
2021-06-04 22:42:58 +09:00
84174c560d
Fix data races caused by incorrect locking ( #1668 )
...
Found via Threadsanitizer. Fixes #1471 .
2021-05-29 00:11:55 +09:00
bb6d2b1b74
Replace snprintf with string and ostringstream ( #1649 )
...
These uses are probably safe from a buffer overflow perspective but
can cause data race issues in logging due to static buffers.
2021-05-08 02:48:47 +09:00
8a51a26819
Allow truncation of open and modified files
...
Regression introduced in f5bf41cf11Fixes #1575 .
2021-04-25 12:35:35 +09:00
4c6690f5f0
Add timestamps to debug log and cache integrity check ( #1540 )
...
* Add UTC timestamp to debug output
for easier correlation with external events
Old output:
$ s3fs mybucket /bucket/ -o curldbg,dbglevel=debug -d -d -f
[CRT] s3fs_logger.cpp:LowSetLogLevel(201): change debug level from...
[INF] s3fs.cpp:set_mountpoint_attribute(3989): PROC(uid=0, ...
New output:
$ s3fs mybucket /bucket/ -o curldbg,dbglevel=debug -d -d -f
2021-01-28T21:09:16.264Z [CRT] s3fs_logger.cpp:LowSetLogLevel(202):...
2021-01-28T21:09:16.264Z [INF] s3fs.cpp:set_mountpoint_attrib...
* Add UTC timestamp to cache integrity check
for easier correlation with external events.
$ s3fs mybucket /mybucket -oset_check_cache_sigusr1=/tmp/check.cache
Old output:
$ kill -s SIGUSR1 $(pgrep s3fs)
$ cat /tmp/check.cache
------------------------------------------------------------
Check cache file and its stats file consistency
------------------------------------------------------------
------------------------------------------------------------
Summary - Total files: 0
Detected error files: 0
Detected error directories: 0
------------------------------------------------------------
New output:
$ kill -s SIGUSR1 $(pgrep s3fs)
$ cat /tmp/check.cache
---------------------------------------------------------------------------
Check cache file and its stats file consistency at 2021-01-30T13:04:14.111Z
---------------------------------------------------------------------------
---------------------------------------------------------------------------
Summary - Total files: 0
Detected error files: 0
Detected error directories: 0
---------------------------------------------------------------------------
* Fix indentation of S3fsLog::GetCurrentTime()
2021-02-04 10:41:29 +09:00
b0e8758b63
Use result instead of res for consistency ( #1530 )
2021-01-25 07:56:10 +09:00
05863a3178
Fix clang-tidy warnings
2020-09-21 07:03:42 +09:00
3958450c05
Added a class for automating fdentity reference counts
2020-09-14 22:02:28 +09:00
1043e08dfa
Remove uses of implicit namespace std
...
Fixed via:
sed -i '/using namespace std/{N;d}' src/*.cpp
sed -i 's/ string/ std::string/g' src/*.cpp
sed -i 's/(string/(std::string/g' src/*.cpp
sed -i 's/\[string/\[std::string/g' src/*.cpp
sed -i 's/^string/std::string/g' src/*.cpp
sed -i 's/ ifstream/ std::ifstream/g' src/*.cpp
sed -i 's/ istringstream/ std::istringstream/g' src/*.cpp
sed -i 's/ ostringstream/ std::ostringstream/g' src/*.cpp
sed -i 's/ max(/ std::max(/g' src/*.cpp
sed -i 's/ min(/ std::min(/g' src/*.cpp
sed -i 's/ endl/ std::endl/g' src/*.cpp
2020-09-13 11:57:20 +09:00
b5ffd419d8
Source file division and set 4 spaces and cleanup
2020-08-26 17:43:50 +09:00
a5186c73c2
Merge pull request #1365 from ggtakec/check_lseek
...
Dynamically determine whether lseek extended options are supported
2020-08-20 22:36:56 +09:00
958ad83a4b
Correct vim modeline to 2-space indentation
2020-08-19 21:03:46 +09:00
55cb8920d5
Dynamically determine whether lseek extended options are supported
2020-08-16 12:37:11 +00:00
46acbf10ba
Merge pull request #1363 from gaul/is-modified-race
...
Lock fdent_data_lock before accessing pagelist
2020-08-16 20:25:24 +09:00
2c0408b95a
Merge pull request #1357 from gaul/readdir
...
Call readdir instead of readdir_r
2020-08-16 18:50:38 +09:00
057da86d87
Merge pull request #1356 from gaul/printf-size_t
...
Prefer %zd and %zu for ssize_t and size_t
2020-08-16 18:16:59 +09:00
8de6cb3504
Lock fdent_data_lock before accessing pagelist
...
Found via ThreadSanitizer. References #1353 . References #1362 .
2020-08-16 17:44:03 +09:00
2bb745cdd7
Fixed a bug about move file over limit of ensure space
2020-08-16 17:42:20 +09:00
LiuBingrun
Takeshi Nakatani
Andrew Gaul
AdamQQQ
Eryu Guan
Petr Vaněk
Carsten Grohmann