Andrew Gaul
beecf32dff
fclose(FILE*) instead of close(fileno(FILE*)) ( #1758 )
...
This is the same thing but confuses Coverity.
2021-09-01 19:41:55 +09:00
Andrew Gaul
57b2e4a4f1
Fix 32-bit compilation issues ( #1757 )
2021-08-31 19:36:02 +09:00
Andrew Gaul
48817d849f
Require explicit length in s3fs_decode64 ( #1755 )
...
This is available from std::string::size in callers.
2021-08-31 09:22:10 +09:00
VVoidV
d9f2d17040
1. fix RowFlush can not upload last part smaller than 5MB using NoCacheMultipartPost; ( #1753 )
...
2. fix deadlock in UploadPendingMeta
2021-08-31 00:41:47 +09:00
Andrew Gaul
cd98afdd7b
Do not NUL terminate base64 decoded output ( #1752 )
...
This is binary data and must use the explicit length.
2021-08-31 00:15:47 +09:00
Andrew Gaul
dac6885fb0
Don't over-allocate in base64 encoding and decoding ( #1751 )
2021-08-30 00:03:10 +09:00
VVoidV
fcd180891b
fix misuse of IsUploading ( #1747 )
...
Co-authored-by: liubingrun <liubr1@chinatelecom.cn>
2021-08-29 23:41:02 +09:00
Vincent Milum Jr
d5d541c7f7
Adding FreeBSD example to README
...
S3FS has existing on FreeBSD since 2009, and should be reflected here that it is well supported.
2021-08-26 10:18:56 +09:00
Takeshi Nakatani
a868c0656e
Changed etaglist_t from string list to new structure etagpairs list
2021-08-16 09:27:12 +09:00
Andrew Gaul
cd466ebdd4
Update ChangeLog and configure.ac for 1.90 ( #1739 )
...
Fixes #1737 .
2021-08-08 14:01:19 +09:00
Andrew Gaul
15e89b78de
Add a partial page_list unit test ( #1735 )
2021-08-04 07:36:32 +09:00
Andrew Gaul
66006ba48d
Add dedicated upper- and lower-case hex functions ( #1734 )
...
This makes the call sites more readable than a boolean parameter.
2021-08-04 07:28:51 +09:00
Andrew Gaul
18e9c62087
Make string constants read-only const ( #1733 )
...
This removes some global constructors. Also use a consistent ALL_CAPS
style.
2021-08-03 00:10:27 +09:00
Takeshi Nakatani
34f89e5936
Added fake_diskfree option to deceive free disk space for test
2021-08-02 11:04:42 +09:00
Carsten Grohmann
e1f3b9d8c1
Add support for AWS-style environment variables ( #1729 )
...
Support AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN
in addition to the AWSACCESSKEYID, AWSSECRETACCESSKEY and
AWSSESSIONTOKEN.
The old environment variables are still supported, but they are
deprecated and no longer documented.
Close #1708
2021-07-26 23:29:45 +09:00
Andrew Gaul
d3278f4886
Loosen CheckBucket to check only the bucket ( #1728 )
...
Previously it checked if there was an object present, assuming that
this was a directory object normally created for s3fs directories.
However most S3 clients do not create this object for virtual folders.
Fixes #1460 . Fixes #1687 .
2021-07-25 12:29:00 +09:00
Andrew Gaul
77f0b75d2f
Add jitter to avoid thundering herd ( #1723 )
2021-07-25 10:22:19 +09:00
Andrew Gaul
012e6dd8a2
Fix README nit ( #1726 )
2021-07-22 13:54:20 +09:00
Takeshi Nakatani
199b3d4709
Fixed a bug in disk free space calculation
2021-07-18 14:52:11 +09:00
Andrew Gaul
7890989cbb
Address cppcheck 2.5 warnings ( #1720 )
...
This slightly simplifies memory management.
2021-07-14 22:18:09 +09:00
Takeshi Nakatani
945cc2ac54
Added UntreatedParts class instead of untreated upload info members in PseudoFdInfo
2021-07-12 21:45:29 +09:00
Takeshi Nakatani
c30acbbf90
Splited some methods of FdEntity class by uploading mode
...
- Splited FdEntity::Write method by uploading mode
- Splited FdEntity::RowFlush method by uplading mode
2021-07-10 17:24:34 +09:00
Martynov Maxim
881025cc9e
Add initiator pid, uid and gid to a debug log ( #1716 )
2021-07-10 16:23:20 +09:00
Takeshi Nakatani
4cc210c5ab
Prevents the etag string buffer from being destroyed during use
...
and absorbed MPPART_INFO structure into filepart structure
2021-07-03 22:42:18 +09:00
Takeshi Nakatani
2d5316a334
Fixed 1710 - Added a flag for check in Linux
2021-07-02 11:34:51 +09:00
Andrew Gaul
9cfa177af0
Add flag to run all tests ( #1710 )
...
This allows CI to run the full set of tests while making local
development faster.
2021-07-02 08:41:47 +09:00
Andrew Gaul
fe44355d25
Report s3fs space as 18.4 EB ( #1709 )
...
256 TB probably seemed large in 2007 but in 2021 it is small enough to
confuse some users.
2021-07-02 07:56:05 +09:00
Andrew Gaul
487df27008
Return EREMOTE when reading Glacier objects ( #1705 )
...
Previously s3fs returned EIO which was not helpful to the caller.
Returning a more specific error code allows automatically restoring
the object via RestoreObject in a subsequent commit.
References #1466 .
2021-06-30 09:25:36 +09:00
Andrew Gaul
1965916f7a
Convert storage class values to uppercase ( #1704 )
...
AWS requires uppercase values. Fixes a regression from
bbcccd6e98
. References #1613 .
2021-06-30 09:03:31 +09:00
Andrew Gaul
8948eded09
Address clang-tidy warnings ( #1703 )
2021-06-30 08:42:44 +09:00
Andrew Gaul
2f59cb5a0a
Remove unneeded volatile qualifiers ( #1702 )
...
We should use proper locking instead. Specifically for
is_meta_pending this does not do what was intended due to the
read-modify-write of the member.
2021-06-30 08:20:44 +09:00
Andrew Gaul
f505c8224e
Explicitly specify lock type ( #1701 )
...
This makes it more clear and type-safe if the caller already has the
lock. Follows on to 84174c560d
.
2021-06-27 15:15:48 +09:00
Andrew Gaul
2154e898bc
Fix typos ( #1700 )
2021-06-27 11:22:33 +09:00
Takeshi Nakatani
f9e80f995d
Fixed a bug about rename existing file of different sizes by mpcopy
2021-06-26 00:04:58 +09:00
Jan Stastny
a5c1915772
Set CURLOPT_UNRESTRICTED_AUTH when authenticating ( #1681 )
...
This is necessary for authentication to work with AWS when it responds with 307 to the check bucket request when mounting. This happens to newly created buckets according to https://aws.amazon.com/premiumsupport/knowledge-center/s3-http-307-response/ .
Prior to this, curl would follow the redirect, but would not include the `Authorization` header which would end up with `404`.
Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
2021-06-25 20:26:12 +09:00
Takeshi Nakatani
6edf3d6427
Updated to clearly output message about the file descriptor type(pseudo/physical)
2021-06-25 19:18:08 +09:00
Andrew Gaul
944c1fd148
Clean up litter in ut_test ( #1697 )
2021-06-25 19:15:03 +09:00
Andrew Gaul
20281737b1
Set explicit Content-Length: 0 when initiating MPU ( #1696 )
...
This works around a GCS quirk that incorrectly requires this header.
Fixes #1661 .
2021-06-25 18:36:40 +09:00
Takeshi Nakatani
0555c4216e
Added test case for utimens is calling during multipart upload
2021-06-21 17:20:24 +09:00
Takeshi Nakatani
5b6684ca19
Fixed a bug utimens is calling before flush
2021-06-21 17:20:24 +09:00
Carsten Grohmann
c4ac923b4c
Ensure NUL-terminated result after strncpy ( #1694 )
...
Long symlinks may cause that the result buffer is filled and not proper
terminated with a null byte.
2021-06-21 08:08:56 +09:00
Takeshi Nakatani
cc022a68f4
Fixed a bug in s3fs_write
2021-06-20 23:13:44 +09:00
Andrew Gaul
1571379304
Set LC_ALL=en_US.UTF-8 in tests ( #1691 )
...
This allows tests to pass using the S3Proxy filesystem provider.
References #1665 .
2021-06-20 19:27:25 +09:00
Takeshi Nakatani
858562ed53
Fixed comparison warning for size_t variable
2021-06-20 16:53:25 +09:00
Andrew Gaul
b8724425d3
Require explicit base parameter in cvt_strtoofft ( #1683 )
...
Also convert most callers of cvt_strtoofft to base 10 which avoid the
magical behavior of interpreting a leading 0 as octal.
References #1682 .
2021-06-20 11:00:15 +09:00
Dmitrii Vasilev
b9ce0faee2
Set Content-MD5 header even for empty objects
...
The Content-MD5 header is required for any request to upload an object
with a retention period configured using Amazon S3 Object Lock
2021-06-19 16:10:58 +09:00
Carsten Grohmann
1e0e2752bf
Improve warning for terminated threads
...
with non-zero return code.
2021-06-18 07:54:38 +09:00
Carsten Grohmann
7a488b93d0
Set decimal base for converting subseconds to int
...
The leading 0 triggers an automatic conversion as an octal value. This
fails because it is a decimal value.
Setting the base to 10 prevents this automatism and treats the value
as a decimal value.
Fixes: [WAN] string_util.cpp:cvt_strtoofft(96): something error is occurred in convert std::string(017080564) to off_t, thus return 0 as default.
Related to #1676
2021-06-17 06:27:02 +09:00
Carsten Grohmann
d67b83e671
Allow configuration for temporary files directory
2021-06-16 21:29:58 +09:00
Andrew Gaul
a100be9dce
Prefer std::string::empty over length == 0 ( #1679 )
...
This is more concise.
2021-06-13 20:03:10 +09:00