octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-10 14:50:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,838 Commits 1 Branch 23 Tags 13 MiB
75b16c72aa
Commit Graph

1838 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Andrew Gaul
75b16c72aa
Build s3fs in parallel (#1848) 
GitHub runners provide 2 Linux CPUs or 3 macOS CPUs:

https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources
 2022-01-09 20:22:49 +09:00
Andrew Gaul
577e2bc987
Generate S3Proxy SSL certificate during tests (#1845) 
Also provide CA bundle to AWS CLI to work around CI failures instead
of ignoring errors.  Fixes #1812.
 2022-01-09 15:13:36 +09:00
Andrew Gaul
adb58af17b
Annotate local variables (#1844) 
This prevents collisions with other globals.  Fixes #1843.
 2022-01-09 13:03:36 +09:00
Andrew Gaul
dd11de3a50
Add Debian Bullseye to CI (#1842) 
Stretch is supported until June 2022:

https://wiki.debian.org/LTS
 2022-01-09 12:11:00 +09:00
Andrew Gaul
fc7543fa25
Make ut_test compatible with Python 3 (#1838) 
This may allow removing Python 2 on newer distros.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
 2022-01-09 11:08:36 +09:00
Andrew Gaul
a44ea7c12a
Replace write_multiple_offsets.py with write_multiblock (#1837) 
This reduces the dependency on Python 2.
 2022-01-09 10:51:17 +09:00
Andrew Gaul
e734763002
Remove createbucket option (#1841) 
AWS CLI can do this.  Fixes #1840.
 2022-01-05 01:59:31 +09:00
Andrew Gaul
37af08bacf
Use JRE instead of JDK for Debian-based distros (#1839) 
This reduces the dependencies installed.  It does not appear that
Fedora-based distros support this.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
 2022-01-05 01:29:06 +09:00
Andrew Gaul
616db5bf1c
Prefer curl over wget for fetching dependencies (#1836) 
The former is lighter-weight and libcurl is already a dependency for
s3fs.
 2022-01-05 00:43:36 +09:00
Serhii Polishchuk
9ad9c382f4 Update README.md 
Fixed install error
 2022-01-04 22:44:49 +09:00
Andrew Gaul
1d090aa7a3 Install default-jdk-headless instead of default-jdk 
This reduces the dependencies installed.
 2022-01-04 18:18:24 +09:00
Andrew Gaul
61e9029be4 Fix typos in CI scripts 2022-01-03 19:50:13 +09:00
Andrew Gaul
5de92e9788
Bump CI to Fedora 35 (#1806) 2021-12-02 23:45:19 +09:00
LiuBingrun
85ca2a3e45
fix mixupload return EntityTooSmall while a copypart is less than 5MB after split (#1809) 
* fix  mixupload return EntityTooSmall while a copypart is less than 5MB after split
* fix possible part exceeds 5GB when multipart_copy_size is set to 5120MB
* Update curl.cpp
Co-authored-by: liubingrun <liubr1@chinatelecom.cn>
 2021-11-27 16:53:26 +09:00
Andrew Gaul
07e2e3f72a
Remove sleep 1 from test_update_directory_time (#1803) 
Reduces per-flag test run-time by 5 seconds.
 2021-11-04 08:16:40 +09:00
Andrew Gaul
3cf00626a2
Add option to allow unsigned payloads (#1801) 
This reduces CPU usage of sigv4.  This reduces test run-time by 7
seconds per flag.
 2021-11-01 23:33:55 +09:00
Andrew Gaul
e289915dcb
Remove require-root script (#1800) 
Tests do not require this.
 2021-10-31 10:48:15 +09:00
Andrew Gaul
06dec32965
Use AWS CLI to create explicit times in the past (#1797) 
s3fs can also do this via utimensat but tests should not trust this.
Also break tests into individual functions.  This further reduces test
run-time 8 seconds per flag.
 2021-10-30 10:54:18 +09:00
Andrew Gaul
86317dd185
Replace dd if=/dev/urandom with junk data generator (#1786) 
This reduces test run time for a single flag from 73 to 60 seconds.
 2021-10-28 22:54:25 +09:00
Andrew Gaul
473da56abf
Use default JDK instead of forcing Java 8 (#1796) 
S3Proxy requires Java 8 or later, not 8 specifically.
 2021-10-28 22:27:48 +09:00
Andrew Gaul
162ab14517
Bump Ubuntu CI to latest non-LTS version (#1794) 2021-10-28 22:10:20 +09:00
Andrew Gaul
40d2e0d1ad
Reduce sleep time to 1 (#1793) 
This reduces test run-time 15 seconds per flag or 2.5 minutes when
testing all flags.
 2021-10-27 23:47:08 +09:00
Takeshi Nakatani
b6c5069ef7 Fixed the test is multi-block writing by one flush 2021-10-27 08:19:05 +09:00
Takeshi Nakatani
7273d561f5 Added exclusive control of statc variables in s3fs xml parser 2021-10-27 08:18:19 +09:00
Takeshi Nakatani
78126aea0b Added exclusive control of statc variables in s3fs xml parser 2021-10-27 08:18:19 +09:00
Takeshi Nakatani
7892eee207 Fixed a bug that copied without considering the length of xmlChar 2021-10-27 08:18:19 +09:00
Andrew Gaul
72a9f40f3f
Update to S3Proxy 1.9.0 (#1788) 
Notably this fixes an issue with the transient provider reading parts
of large files.

Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.9.0
 2021-10-26 23:20:52 +09:00
Andrew Gaul
495d51113c
Remove unneeded sleeps from tests (#1784) 
Also use a unique file name for every test.  This ensures that tests
like test_external_directory_creation and test_external_modification
do not collide.
 2021-10-26 23:19:14 +09:00
Carsten Grohmann
0abeec9cae Simpily errexit modifications 2021-10-26 21:47:36 +09:00
Andrew Gaul
ea3c21f270
Reduce errexit modifications (#1785) 
This is less error prone but requires some magic && ||.
 2021-10-25 23:53:45 +09:00
Takeshi Nakatani
23fe6f4dee Fixed parse_string function in write_multiblock.cc 2021-10-25 17:56:49 +09:00
Takeshi Nakatani
34ea2acd75 Add a test that is multi-block writing by one flush 2021-10-25 17:56:49 +09:00
Takeshi Nakatani
ea64886469 Fixed a bug in test_(zero_)cache_file_stat test function 2021-10-24 18:24:12 +09:00
Takeshi Nakatani
023aaf7dff Fixed wrong stat of cache after new creation file 
And added a test for stat of cache after new creation file
 2021-10-17 16:10:14 +09:00
Takeshi Nakatani
2f412804e2 Fixed forgetting to clear the dirty flag for meta information 
Addressed an error in macos cpp check
 2021-10-15 22:54:55 +09:00
Takeshi Nakatani
d6ffd389da Excluded ubuntu 16.04 from the CI build execution environment 2021-10-15 08:54:13 +09:00
Carsten Grohmann
be0b17329a
Fix wrong function name in log message (#1774) 2021-10-10 11:08:32 +09:00
Kamil Jakrzewski
b4edad86d6 remove Expect: 100-continue header when requesting an IMDSv2 access token 2021-09-09 08:12:36 +09:00
Noah Meyerhans
9d1552a54e fix IAM role retrieval from IMDSv2 
AWS IMDSv2 support was added in #1462, but the implementation did not
cover the addional IMDS access that occurs with the iam_role=auto
configuration.  This change implements IMDSv2 support for the IMDS
call to determine the instance's role name.

See also
https://stackoverflow.com/questions/69031023/how-to-make-s3fs-use-imds-v2-when-mounting-s3-buckets-from-ec2-instance
 2021-09-03 20:36:34 +09:00
Andrew Gaul
47ebfcc60a
Consume return value from curl_easy_setopt (#1759) 
Found via Coverity.
 2021-09-02 08:07:06 +09:00
Andrew Gaul
beecf32dff
fclose(FILE*) instead of close(fileno(FILE*)) (#1758) 
This is the same thing but confuses Coverity.
 2021-09-01 19:41:55 +09:00
Andrew Gaul
57b2e4a4f1
Fix 32-bit compilation issues (#1757) 2021-08-31 19:36:02 +09:00
Andrew Gaul
48817d849f
Require explicit length in s3fs_decode64 (#1755) 
This is available from std::string::size in callers.
 2021-08-31 09:22:10 +09:00
VVoidV
d9f2d17040
1. fix RowFlush can not upload last part smaller than 5MB using NoCacheMultipartPost; (#1753) 
2. fix deadlock in UploadPendingMeta
 2021-08-31 00:41:47 +09:00
Andrew Gaul
cd98afdd7b
Do not NUL terminate base64 decoded output (#1752) 
This is binary data and must use the explicit length.
 2021-08-31 00:15:47 +09:00
Andrew Gaul
dac6885fb0
Don't over-allocate in base64 encoding and decoding (#1751) 2021-08-30 00:03:10 +09:00
VVoidV
fcd180891b
fix misuse of IsUploading (#1747) 
Co-authored-by: liubingrun <liubr1@chinatelecom.cn>
 2021-08-29 23:41:02 +09:00
Vincent Milum Jr
d5d541c7f7 Adding FreeBSD example to README 
S3FS has existing on FreeBSD since 2009, and should be reflected here that it is well supported.
 2021-08-26 10:18:56 +09:00
Takeshi Nakatani
a868c0656e Changed etaglist_t from string list to new structure etagpairs list 2021-08-16 09:27:12 +09:00
Andrew Gaul
cd466ebdd4
Update ChangeLog and configure.ac for 1.90 (#1739) 
Fixes #1737.
 2021-08-08 14:01:19 +09:00