octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-16 17:25:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,775 Commits 1 Branch 23 Tags 13 MiB
4cc210c5ab
Commit Graph

157 Commits

Author SHA1 Message Date
Andrew Gaul
f505c8224e
Explicitly specify lock type (#1701) 
This makes it more clear and type-safe if the caller already has the
lock.  Follows on to 84174c560d.
 2021-06-27 15:15:48 +09:00
Andrew Gaul
2154e898bc
Fix typos (#1700) 2021-06-27 11:22:33 +09:00
Takeshi Nakatani
6edf3d6427 Updated to clearly output message about the file descriptor type(pseudo/physical) 2021-06-25 19:18:08 +09:00
Carsten Grohmann
d67b83e671 Allow configuration for temporary files directory 2021-06-16 21:29:58 +09:00
Takeshi Nakatani
c2c56d0263 Added info object about multipart uploading for each pseudo fd 
(and fixed typo about method name)
 2021-06-04 22:42:58 +09:00
Takeshi Nakatani
ac578d188e Introduced pseudo fd and separated fd for each file opening 2021-06-04 22:42:58 +09:00
Andrew Gaul
84174c560d
Fix data races caused by incorrect locking (#1668) 
Found via Threadsanitizer.  Fixes #1471.
 2021-05-29 00:11:55 +09:00
Andrew Gaul
bb6d2b1b74
Replace snprintf with string and ostringstream (#1649) 
These uses are probably safe from a buffer overflow perspective but
can cause data race issues in logging due to static buffers.
 2021-05-08 02:48:47 +09:00
Andrew Gaul
8a51a26819 Allow truncation of open and modified files 
Regression introduced in f5bf41cf11.
Fixes #1575.
 2021-04-25 12:35:35 +09:00
Carsten Grohmann
4c6690f5f0
Add timestamps to debug log and cache integrity check (#1540) 
* Add UTC timestamp to debug output

for easier correlation with external events

Old output:
$ s3fs mybucket /bucket/  -o curldbg,dbglevel=debug -d -d -f
[CRT] s3fs_logger.cpp:LowSetLogLevel(201): change debug level from...
[INF]     s3fs.cpp:set_mountpoint_attribute(3989): PROC(uid=0, ...

New output:
$ s3fs mybucket /bucket/  -o curldbg,dbglevel=debug -d -d -f
2021-01-28T21:09:16.264Z [CRT] s3fs_logger.cpp:LowSetLogLevel(202):...
2021-01-28T21:09:16.264Z [INF]     s3fs.cpp:set_mountpoint_attrib...

* Add UTC timestamp to cache integrity check

for easier correlation with external events.

$ s3fs mybucket /mybucket  -oset_check_cache_sigusr1=/tmp/check.cache

Old output:
$ kill -s SIGUSR1 $(pgrep s3fs)
$ cat /tmp/check.cache
------------------------------------------------------------
Check cache file and its stats file consistency
------------------------------------------------------------
------------------------------------------------------------
Summary - Total files:                0
          Detected error files:       0
          Detected error directories: 0
------------------------------------------------------------

New output:
$ kill -s SIGUSR1 $(pgrep s3fs)
$ cat /tmp/check.cache
---------------------------------------------------------------------------
Check cache file and its stats file consistency at 2021-01-30T13:04:14.111Z
---------------------------------------------------------------------------
---------------------------------------------------------------------------
Summary - Total files:                0
          Detected error files:       0
          Detected error directories: 0
---------------------------------------------------------------------------

* Fix indentation of S3fsLog::GetCurrentTime()
 2021-02-04 10:41:29 +09:00
Andrew Gaul
b0e8758b63
Use result instead of res for consistency (#1530) 2021-01-25 07:56:10 +09:00
Andrew Gaul
05863a3178 Fix clang-tidy warnings 2020-09-21 07:03:42 +09:00
Takeshi Nakatani
3958450c05 Added a class for automating fdentity reference counts 2020-09-14 22:02:28 +09:00
Andrew Gaul
1043e08dfa Remove uses of implicit namespace std 
Fixed via:

sed -i '/using namespace std/{N;d}' src/*.cpp
sed -i 's/ string/ std::string/g' src/*.cpp
sed -i 's/(string/(std::string/g' src/*.cpp
sed -i 's/\[string/\[std::string/g' src/*.cpp
sed -i 's/^string/std::string/g' src/*.cpp
sed -i 's/ ifstream/ std::ifstream/g' src/*.cpp
sed -i 's/ istringstream/ std::istringstream/g' src/*.cpp
sed -i 's/ ostringstream/ std::ostringstream/g' src/*.cpp
sed -i 's/ max(/ std::max(/g' src/*.cpp
sed -i 's/ min(/ std::min(/g' src/*.cpp
sed -i 's/ endl/ std::endl/g' src/*.cpp
 2020-09-13 11:57:20 +09:00
Takeshi Nakatani
b5ffd419d8 Source file division and set 4 spaces and cleanup 2020-08-26 17:43:50 +09:00
Takeshi Nakatani
a5186c73c2
Merge pull request #1365 from ggtakec/check_lseek 
Dynamically determine whether lseek extended options are supported
 2020-08-20 22:36:56 +09:00
Andrew Gaul
958ad83a4b Correct vim modeline to 2-space indentation 2020-08-19 21:03:46 +09:00
Takeshi Nakatani
55cb8920d5 Dynamically determine whether lseek extended options are supported 2020-08-16 12:37:11 +00:00
Takeshi Nakatani
46acbf10ba
Merge pull request #1363 from gaul/is-modified-race 
Lock fdent_data_lock before accessing pagelist
 2020-08-16 20:25:24 +09:00
Takeshi Nakatani
2c0408b95a
Merge pull request #1357 from gaul/readdir 
Call readdir instead of readdir_r
 2020-08-16 18:50:38 +09:00
Takeshi Nakatani
057da86d87
Merge pull request #1356 from gaul/printf-size_t 
Prefer %zd and %zu for ssize_t and size_t
 2020-08-16 18:16:59 +09:00
Andrew Gaul
8de6cb3504 Lock fdent_data_lock before accessing pagelist 
Found via ThreadSanitizer.  References #1353.  References #1362.
 2020-08-16 17:44:03 +09:00
Takeshi Nakatani
2bb745cdd7 Fixed a bug about move file over limit of ensure space 2020-08-16 17:42:20 +09:00
Andrew Gaul
35090ba4d5 Call readdir instead of readdir_r 
Only a single thread uses this directory stream.  Further, modern
implementations are thread-safe by default and deprecated this call:

https://man7.org/linux/man-pages/man3/readdir_r.3.html
 2020-08-16 13:33:23 +09:00
Andrew Gaul
132a1bebbb Prefer %zd and %zu for ssize_t and size_t 
This removes unnecessary casting.
 2020-08-15 17:30:23 +09:00
Takeshi Nakatani
ad1961417d Added SIGUSR1 option for cache file integrity test 2020-07-26 21:04:11 +09:00
Takeshi Nakatani
700e288718 Put similar processing together into method GetCacheFileStatTopDir 2020-07-12 08:12:02 +00:00
Takeshi Nakatani
1f796d432d Fixed upload error about mixuploading sparse file and truncating file 2020-06-27 22:44:19 +09:00
Takeshi Nakatani
35006e318f Fixed about ParallelMixMultipartUpload 2020-06-24 12:48:55 +09:00
Takeshi Nakatani
f324d8e04f Fixed a bug about serializing from cache file 2020-06-19 12:57:27 +00:00
Takeshi Nakatani
0cb057dadd
Merge pull request #1303 from gaul/rename/use_cache 
Relink cache stats file atomically via rename
 2020-06-01 00:10:33 +09:00
Takeshi Nakatani
0f5db0d1bf
Merge pull request #1302 from gaul/rename/nocopy 
Fix renames of open files with nocopyapi option
 2020-05-31 23:46:46 +09:00
Andrew Gaul
274321524c Relink cache stats file atomically via rename 
The new file may already exist so link may fail.  Further link/unlink
is not atomic.  Addresses an error when renaming an open with with
use_cache.  References #1296.
 2020-05-31 23:09:58 +09:00
Andrew Gaul
40f7007263 Check results from pthread mutex calls 
Also remove some unnecessary exception handling.
 2020-05-30 16:37:55 +09:00
Andrew Gaul
66597ec5f2 Fix renames of open files with nocopyapi option 
References #1296.
 2020-05-30 15:45:43 +09:00
Takeshi Nakatani
97c249d5b9 Not abort process by exception threw from s3fs_strtoofft 2020-05-03 08:08:28 +00:00
Takeshi Nakatani
f26a0aa71d Fixed insufficient upload size for mix multipart upload 2020-04-22 09:31:22 +09:00
Takeshi Nakatani
4da02d023b Improved strictness of cache file stats(file) 2020-04-21 19:45:03 +09:00
Takeshi Nakatani
811ea0cb85 Fixed the truncation bug of stat file for cache file 2020-04-19 07:08:49 +00:00
Andrew Gaul
48a872e285 Address cppcheck 1.90 warning 2020-04-12 22:20:44 +09:00
Takeshi Nakatani
018ccb9a11 Suppressed a lot of message output about cache cleanup 2020-01-30 14:42:28 +00:00
yongqingliu
b762a0a85b fix deadlock due to fdmanager_lock and fdent_data_lock 2020-01-29 12:03:53 +08:00
Andrew Gaul
e0712f444d Update source for clang-tidy 9.0.0 2020-01-13 20:56:45 +09:00
Takeshi Nakatani
cf3e82d10a Fixed a rename bug when enable_noobj_cache 2019-10-23 12:39:42 +00:00
Takeshi Nakatani
1db94a0b30 Fixed to use copy api in multipart upload 2019-09-25 19:30:58 -07:00
Takeshi Nakatani
f5bf41cf11 Fixed simple copy with nocpoyapi and modify flag in fdcache 2019-08-11 07:42:48 +00:00
Takeshi Nakatani
3d70e8966f
Merge pull request #1108 from gaul/lock/diskspace 
Always hold lock when accessing free_disk_space
 2019-08-03 22:01:31 +09:00
Andrew Gaul
5b15c7c4e9 Always hold lock when accessing free_disk_space 
Slightly reorder locks to avoid double locking.  Found via
ThreadSanitizer.
 2019-07-30 08:36:54 -07:00
Andrew Gaul
afd438d363 Access FdEntity metadata while holding fdent_lock 
Create copies to avoid thread races.  Found via ThreadSanitizer.
Follows on to ecf13a8cb9.
 2019-07-29 13:54:57 -07:00
Andrew Gaul
bacd15714a Use consistent default for readwrite_timeout 
Previously curl threads mutated this shared state without a lock.
Found via ThreadSanitizer.
 2019-07-16 22:03:56 -07:00