octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-16 01:07:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,954 Commits 1 Branch 23 Tags 13 MiB
5b95a0fcb6
Commit Graph

1954 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Andrew Gaul
b8724425d3
Require explicit base parameter in cvt_strtoofft (#1683) 
Also convert most callers of cvt_strtoofft to base 10 which avoid the
magical behavior of interpreting a leading 0 as octal.
References #1682.
 2021-06-20 11:00:15 +09:00
Dmitrii Vasilev
b9ce0faee2 Set Content-MD5 header even for empty objects 
The Content-MD5 header is required for any request to upload an object
with a retention period configured using Amazon S3 Object Lock
 2021-06-19 16:10:58 +09:00
Carsten Grohmann
1e0e2752bf Improve warning for terminated threads 
with non-zero return code.
 2021-06-18 07:54:38 +09:00
Carsten Grohmann
7a488b93d0 Set decimal base for converting subseconds to int 
The leading 0 triggers an automatic conversion as an octal value. This
fails because it is a decimal value.

Setting the base to 10 prevents this automatism and treats the value
as a decimal value.

Fixes: [WAN] string_util.cpp:cvt_strtoofft(96): something error is occurred in convert std::string(017080564) to off_t, thus return 0 as default.

Related to #1676
 2021-06-17 06:27:02 +09:00
Carsten Grohmann
d67b83e671 Allow configuration for temporary files directory 2021-06-16 21:29:58 +09:00
Andrew Gaul
a100be9dce
Prefer std::string::empty over length == 0 (#1679) 
This is more concise.
 2021-06-13 20:03:10 +09:00
Andrew Gaul
4d39ea887e
Protect orgmeta with fdent_lock (#1678) 
Fixes a crash seen with the CentOS 7 builder and when running with
AddressSanitizer.  Regression introduced by
ac578d188e.  Fixes #1677.
 2021-06-13 16:14:24 +09:00
Andrew Gaul
7638b5b3e3
Prefer std::string::rbegin over operator[] (#1673) 
This is more concise and safer due to not repeating the variable name.
We cannot use std::string::back since it is not available in C++03.
 2021-06-13 13:26:38 +09:00
Andrew Gaul
600cee118d
Fix implicit narrowing conversions (#1672) 
These do not appear to be problematic but rather just clean up warnings.
Found via clang -Wshorten-64-to-32.
 2021-06-13 12:50:07 +09:00
Takeshi Nakatani
c2c56d0263 Added info object about multipart uploading for each pseudo fd 
(and fixed typo about method name)
 2021-06-04 22:42:58 +09:00
Takeshi Nakatani
ac578d188e Introduced pseudo fd and separated fd for each file opening 2021-06-04 22:42:58 +09:00
Andrew Gaul
53dfd48f59
Correct usage of istringstream (#1670) 
Previously this looped one more time than necessary due to the eof
check:

https://isocpp.org/wiki/faq/input-output#istream-and-eof

Remove now redundant empty check.
 2021-06-02 07:14:32 +09:00
Andrew Gaul
f5701fa9ad
Correct typo in undefinedsanitizer config (#1669) 2021-05-29 22:07:33 +09:00
Andrew Gaul
84174c560d
Fix data races caused by incorrect locking (#1668) 
Found via Threadsanitizer.  Fixes #1471.
 2021-05-29 00:11:55 +09:00
Carsten Grohmann
9bf525ee7a Ensuring multipart size even when storage is low 
When the temporary storage was full, the old implementation started an
upload even if there was not enough data to completely fill the minimum
multipart size or fill the user-selected multipart size.

The new implementation ensures the minimum multipart size by forcing
the user-selected multipart size.

Fixes #1591
 2021-05-27 22:55:52 +09:00
Takeshi Nakatani
4b69d4b1bb Fixed a bug when the disk capacity was insufficient in RowFlush 2021-05-22 23:11:46 +09:00
Andrew Gaul
bbcccd6e98
Make storage class a string (#1663) 
This allows non-standard storage classes like Google Cloud Storage
Nearline.  Fixes #1613.
 2021-05-21 23:34:31 +09:00
Carsten Grohmann
c0bcb41175
Increase test robustness by adding LC_ALL=C (#1660) 
Use C locale, because some tests check for English expressions

Fixes #1658
 2021-05-20 21:19:50 +09:00
Takeshi Nakatani
b5fef788da Additional bug fixing for not creating zero-byte object 2021-05-09 17:33:53 +09:00
Andrew Gaul
42f5965d8a
Fix off_t to int narrowing in PageList::GetSparseFilePages (#1654) 
Found via clang-tidy.
 2021-05-09 14:35:00 +09:00
Andrew Gaul
d904d91252
Prefer std::map::insert over find and operator[] (#1653) 
This avoids a duplicate lookups.
 2021-05-09 14:11:35 +09:00
Andrew Gaul
4d81a4bf68
Use same time when creating directory objects (#1652) 
This avoids mismatched times and is slightly faster.
 2021-05-09 13:44:59 +09:00
Andrew Gaul
9abe3fa662
Use same time when creating directory objects (#1652) 
This avoids mismatched times and is slightly faster.
 2021-05-09 13:44:12 +09:00
Takeshi Nakatani
9d8f1b00f7 Fixed make_random_string test code being slow on macos 2021-05-08 13:55:39 +09:00
Andrew Gaul
bb6d2b1b74
Replace snprintf with string and ostringstream (#1649) 
These uses are probably safe from a buffer overflow perspective but
can cause data race issues in logging due to static buffers.
 2021-05-08 02:48:47 +09:00
Andrew Gaul
096a230b70
Allow arbitrary size AWS secret keys (#1648) 
Previously s3fs limited these to 123 characters.  Fixes #1626.
 2021-05-06 22:24:38 +09:00
Andrew Gaul
8ef01d37a9
Fix a few nits (#1645) 
Make some strings more const, initialize members, and abort if lock
initialization fails.  Partially found via clang-tidy.
 2021-05-06 19:40:35 +09:00
Andrew Gaul
cb9148f6cd
Update for clang-tidy 12 (#1644) 2021-05-03 09:43:18 +09:00
Carsten Grohmann
c4fa53ec8b
Add section about storage consumption to manpage (#1639) 
References s3fs-fuse#1591. References s3fs-fuse#1595.
 2021-05-01 15:09:29 +09:00
Andrew Gaul
1e2df406ee
Add Ubuntu 21.04 to CI (#1642) 2021-04-30 23:51:55 +09:00
Andrew Gaul
1dabfbe1da
Update CI to latest Fedora version (#1641) 2021-04-30 20:29:28 +09:00
Andrew Gaul
771bbfeac5
Do not create zero-byte object when creating file (#1640) 
Previously s3fs created this object to store metadata and overwrote it
when flushing.  This prevented use with object stores which do not
allow overwrites like HDS.  Instead only create an in-memory
representation which reduces the time to create small files.
Fixes #1013.
 2021-04-30 19:56:33 +09:00
Andrew Gaul
3694786112
Propagate errno instead of EIO (#1638) 
This improves error fidelity.  Follows on to
b70f8db037.  References #1523.
 2021-04-30 07:09:00 +09:00
Andrew Gaul
e477d7d186
Add note about Homebrew and FUSE (#1637) 2021-04-25 21:35:08 +09:00
Andrew Gaul
fbf3c83019
Consider S3 errors in HTTP PUT 200 responses (#1635) 
S3 can emit these in unusual situations.  Fixes #1317.
 2021-04-25 13:18:11 +09:00
Andrew Gaul
8a51a26819 Allow truncation of open and modified files 
Regression introduced in f5bf41cf11.
Fixes #1575.
 2021-04-25 12:35:35 +09:00
Takeshi Nakatani
1838f52e19 Declare undefined symbols for fallocate function 2021-04-25 10:12:25 +09:00
Andrew Gaul
e9eb248f2f
Fail CheckBucket when S3 returns PermanentRedirect (#1630) 
Previously s3fs allowed mounting but all operations returned EIO.
References #693.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
 2021-04-24 18:46:24 +09:00
Ambika Nair
77581eda59 Change in ibm iam endpoint 2021-04-21 21:42:55 +09:00
Andrew Gaul
7f3e423bbe
Preserve sub-second time precision (#1624) 
Found via pjdfstests.  References #897.  References #1589.
 2021-04-18 13:11:12 +09:00
Andrew Gaul
706e3cbebd
Update ctime and mtime when opening with O_TRUNC (#1623) 
Found via pjdfstests.  References #1589.
 2021-04-15 07:08:49 +09:00
Takeshi Nakatani
2effffd8e2 Fixed codes for cppcheck 2.4.1 2021-04-13 21:12:35 +09:00
Takeshi Nakatani
2908878988 Fixed codes for cppcheck 2.4.1 2021-04-13 21:12:35 +09:00
Andrew Gaul
01a26a9011
Do not allow open with O_RDONLY and O_TRUNC (#1620) 
Found via pjdfstests.  References #1589.
 2021-04-13 07:28:10 +09:00
Andrew Gaul
23e1fbf7b9
Update ctime when truncating file (#1619) 
Found via pjdfstests.  References #1589.
 2021-04-13 07:08:06 +09:00
Andrew Gaul
ed2e877bb6
Return EACESS when lacking required executable bit (#1617) 
This makes the check consistent with read and write.  Found via
pjdfstests.  References #1551.  References #1589.
 2021-04-11 22:39:27 +09:00
Andrew Gaul
3663082a01
Do not allow renaming to a non-empty directory (#1604) 
Found via pjdfstests.  References #1589.
 2021-03-28 19:10:52 +09:00
Andrew Gaul
9645d57c05
Translate KeyTooLongError to ENAMETOOLONG (#1602) 
AWS does not support keys longer than 1024 characters.  Add special
handling for HeadObject which does not have a response body.
Found via pjdfstests.  References #1589.
 2021-03-28 13:17:41 +09:00
Andrew Gaul
f6fbd75320
Return EFBIG when file exceeds multipart limit (#1600) 
Found via pjdfstests.  References #1589.
 2021-03-28 12:33:01 +09:00
Andrew Gaul
20a6c9d35d
Update to S3Proxy 1.8.0 (#1598) 
Notably this fixes an issue with the filesystem backend which returned
the incorrect Content-Type for some directories.  It also reduces
memory use with GetObject using range requests with large objects.
References #1543.

Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.8.0
 2021-03-28 11:56:10 +09:00