octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-15 08:54:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,880 Commits 1 Branch 23 Tags 13 MiB
b9e2be5c21
Commit Graph

1017 Commits

Author SHA1 Message Date
Takeshi Nakatani
839a33de49
Fixed not to call Flush even if the file size is increased (#1887) 
Changed s3fs_truncate function.
This change reduces the number of file uploads if the file size is changed.

On macOS, I have found that the truncate call when "size=0" cannot reflect the file size.(This reason is not understood...)
To avoid this, only when "size=0", the flush method is called as before.

Other than that, I found a bug in FdEntity::Open() and fixed it.

Fixes #1875.
 2022-02-15 21:29:07 +09:00
Petr Vaněk
4dfe2bfdd7 Include climits to support musl libc 
PATH_MAX constant is not visible from any of currently included header
files in system with musl libc, where compilation fails with an error
below. The constant is defined in limits.h which is directly include via
climits header file.

fdcache.cpp: In static member function 'static FILE* FdManager::MakeTempFile()':
fdcache.cpp:381:14: error: 'PATH_MAX' was not declared in this scope
  381 |     char cfn[PATH_MAX];
      |              ^~~~~~~~

Fixes: d67b83e671 ("Allow configuration for temporary files directory")
 2022-02-14 09:19:30 +09:00
Takeshi Nakatani
1678803566 Added S3fsCred class and moved Credential related processing in it 2022-02-13 21:38:30 +09:00
Takeshi Nakatani
d7e929e0a8
Fixed some Github Actions errors. (#1886) 
- Fix knownConditionTrueFalse cppcheck(2.7) error on MacOS
- Fixed package installing failure of appstream download on centos8
 2022-02-13 14:23:35 +09:00
Andrew Gaul
9224f792f0
Use CLOCK_REALTIME for UTIME_NOW (#1881) 
Previously s3fs_utimens used CLOCK_MONOTONIC_COARSE which was not
1970-based.  Found via pjdfstest.  References #1589.
 2022-01-30 22:19:15 +09:00
Andrew Gaul
f6ed972926
Always flush open files with O_CREAT flag (#1879) 
Previously s3fs only created files that had dirty data and not those
with zero-bytes.  Regression from
771bbfeac5.  References #1013.  Found
via pjdfstest.  References #1589.
 2022-01-30 22:02:37 +09:00
Andrew Gaul
0c75a63184
Preserve sub-second precision with utimens (#1880) 
Found via pjdfstest.  References #1589.
 2022-01-30 21:45:51 +09:00
Takeshi Nakatani
e452ef3940 Fixed the fault tolerance when time stamp getting fails 2022-01-30 18:31:36 +09:00
Andrew Gaul
cd5a69b9eb
Handle UTIME_NOW and UTIME_OMIT special values (#1868) 
FUSE 3 will require this behavior.  References #1159.
 2022-01-29 11:35:37 +09:00
Andrew Gaul
662882d2f0
Always call clock_gettime(2) (#1871) 
e01ded9e27 introduced this compatibility
shim but macOS 10.12 (2016) added this:
https://stackoverflow.com/a/39801564 .  Also remove fallback to
time(3) which loses precision.
 2022-01-25 08:36:27 +09:00
Andrew Gaul
de0c87c801
Convert S3FS_LOW_LOGPRN from a macro to a function (#1869) 
This shrinks the binary size from 770 to 540 KB and reduces compile
times.
 2022-01-23 23:10:09 +09:00
Naoki Ikeguchi
41aaa4184f Avoid double setting values in statfs 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
581f5c0356 Move strptime polyfill to string_util 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
e5f6f112db Fix typo 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
b3cef944b2 Fix test_page_list_SOURCES has no if MSYS clause 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
6edb6067f3 Remove strcasestr polyfill 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
b2c659c0a6 Disable compiling polyfills in not MSYS2 env 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
3ac9f571f5 Use std::get_time instead in strptime polyfill 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
19303a546e Fix the statfs issue, using f_frsize instead 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
6370e150dd Disable features that causes problems on Windows 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
b14e39815b Use polyfills in MSYS2 environment 2022-01-23 21:49:51 +09:00
Naoki Ikeguchi
6aaf9433a5 Add polyfills for MSYS2 environment 2022-01-23 21:49:51 +09:00
Andrew Gaul
93d1c30d4d
Use XML parsing with PUT HTTP 200 responses (#1858) 
This works around the missing strcasestr on win32.  References #728.
 2022-01-14 16:10:22 +09:00
Andrew Gaul
25012f3839 Fix typo in -o enable_unsigned_payload 2022-01-12 22:50:49 +09:00
Andrew Gaul
e734763002
Remove createbucket option (#1841) 
AWS CLI can do this.  Fixes #1840.
 2022-01-05 01:59:31 +09:00
LiuBingrun
85ca2a3e45
fix mixupload return EntityTooSmall while a copypart is less than 5MB after split (#1809) 
* fix  mixupload return EntityTooSmall while a copypart is less than 5MB after split
* fix possible part exceeds 5GB when multipart_copy_size is set to 5120MB
* Update curl.cpp
Co-authored-by: liubingrun <liubr1@chinatelecom.cn>
 2021-11-27 16:53:26 +09:00
Andrew Gaul
3cf00626a2
Add option to allow unsigned payloads (#1801) 
This reduces CPU usage of sigv4.  This reduces test run-time by 7
seconds per flag.
 2021-11-01 23:33:55 +09:00
Takeshi Nakatani
7273d561f5 Added exclusive control of statc variables in s3fs xml parser 2021-10-27 08:18:19 +09:00
Takeshi Nakatani
78126aea0b Added exclusive control of statc variables in s3fs xml parser 2021-10-27 08:18:19 +09:00
Takeshi Nakatani
7892eee207 Fixed a bug that copied without considering the length of xmlChar 2021-10-27 08:18:19 +09:00
Takeshi Nakatani
023aaf7dff Fixed wrong stat of cache after new creation file 
And added a test for stat of cache after new creation file
 2021-10-17 16:10:14 +09:00
Takeshi Nakatani
2f412804e2 Fixed forgetting to clear the dirty flag for meta information 
Addressed an error in macos cpp check
 2021-10-15 22:54:55 +09:00
Carsten Grohmann
be0b17329a
Fix wrong function name in log message (#1774) 2021-10-10 11:08:32 +09:00
Kamil Jakrzewski
b4edad86d6 remove Expect: 100-continue header when requesting an IMDSv2 access token 2021-09-09 08:12:36 +09:00
Noah Meyerhans
9d1552a54e fix IAM role retrieval from IMDSv2 
AWS IMDSv2 support was added in #1462, but the implementation did not
cover the addional IMDS access that occurs with the iam_role=auto
configuration.  This change implements IMDSv2 support for the IMDS
call to determine the instance's role name.

See also
https://stackoverflow.com/questions/69031023/how-to-make-s3fs-use-imds-v2-when-mounting-s3-buckets-from-ec2-instance
 2021-09-03 20:36:34 +09:00
Andrew Gaul
47ebfcc60a
Consume return value from curl_easy_setopt (#1759) 
Found via Coverity.
 2021-09-02 08:07:06 +09:00
Andrew Gaul
beecf32dff
fclose(FILE*) instead of close(fileno(FILE*)) (#1758) 
This is the same thing but confuses Coverity.
 2021-09-01 19:41:55 +09:00
Andrew Gaul
57b2e4a4f1
Fix 32-bit compilation issues (#1757) 2021-08-31 19:36:02 +09:00
Andrew Gaul
48817d849f
Require explicit length in s3fs_decode64 (#1755) 
This is available from std::string::size in callers.
 2021-08-31 09:22:10 +09:00
VVoidV
d9f2d17040
1. fix RowFlush can not upload last part smaller than 5MB using NoCacheMultipartPost; (#1753) 
2. fix deadlock in UploadPendingMeta
 2021-08-31 00:41:47 +09:00
Andrew Gaul
cd98afdd7b
Do not NUL terminate base64 decoded output (#1752) 
This is binary data and must use the explicit length.
 2021-08-31 00:15:47 +09:00
Andrew Gaul
dac6885fb0
Don't over-allocate in base64 encoding and decoding (#1751) 2021-08-30 00:03:10 +09:00
VVoidV
fcd180891b
fix misuse of IsUploading (#1747) 
Co-authored-by: liubingrun <liubr1@chinatelecom.cn>
 2021-08-29 23:41:02 +09:00
Takeshi Nakatani
a868c0656e Changed etaglist_t from string list to new structure etagpairs list 2021-08-16 09:27:12 +09:00
Andrew Gaul
15e89b78de
Add a partial page_list unit test (#1735) 2021-08-04 07:36:32 +09:00
Andrew Gaul
66006ba48d
Add dedicated upper- and lower-case hex functions (#1734) 
This makes the call sites more readable than a boolean parameter.
 2021-08-04 07:28:51 +09:00
Andrew Gaul
18e9c62087
Make string constants read-only const (#1733) 
This removes some global constructors.  Also use a consistent ALL_CAPS
style.
 2021-08-03 00:10:27 +09:00
Takeshi Nakatani
34f89e5936 Added fake_diskfree option to deceive free disk space for test 2021-08-02 11:04:42 +09:00
Carsten Grohmann
e1f3b9d8c1
Add support for AWS-style environment variables (#1729) 
Support AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN
in addition to the AWSACCESSKEYID, AWSSECRETACCESSKEY and
AWSSESSIONTOKEN.

The old environment variables are still supported, but they are
deprecated and no longer documented.

Close #1708
 2021-07-26 23:29:45 +09:00
Andrew Gaul
d3278f4886
Loosen CheckBucket to check only the bucket (#1728) 
Previously it checked if there was an object present, assuming that
this was a directory object normally created for s3fs directories.
However most S3 clients do not create this object for virtual folders.
Fixes #1460.  Fixes #1687.
 2021-07-25 12:29:00 +09:00