1
0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2025-01-14 17:20:01 +00:00
Commit Graph

1837 Commits

Author SHA1 Message Date
Andrew Gaul
577e2bc987
Generate S3Proxy SSL certificate during tests ()
Also provide CA bundle to AWS CLI to work around CI failures instead
of ignoring errors.  Fixes .
2022-01-09 15:13:36 +09:00
Andrew Gaul
adb58af17b
Annotate local variables ()
This prevents collisions with other globals.  Fixes .
2022-01-09 13:03:36 +09:00
Andrew Gaul
dd11de3a50
Add Debian Bullseye to CI ()
Stretch is supported until June 2022:

https://wiki.debian.org/LTS
2022-01-09 12:11:00 +09:00
Andrew Gaul
fc7543fa25
Make ut_test compatible with Python 3 ()
This may allow removing Python 2 on newer distros.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
2022-01-09 11:08:36 +09:00
Andrew Gaul
a44ea7c12a
Replace write_multiple_offsets.py with write_multiblock ()
This reduces the dependency on Python 2.
2022-01-09 10:51:17 +09:00
Andrew Gaul
e734763002
Remove createbucket option ()
AWS CLI can do this.  Fixes .
2022-01-05 01:59:31 +09:00
Andrew Gaul
37af08bacf
Use JRE instead of JDK for Debian-based distros ()
This reduces the dependencies installed.  It does not appear that
Fedora-based distros support this.

Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>
2022-01-05 01:29:06 +09:00
Andrew Gaul
616db5bf1c
Prefer curl over wget for fetching dependencies ()
The former is lighter-weight and libcurl is already a dependency for
s3fs.
2022-01-05 00:43:36 +09:00
Serhii Polishchuk
9ad9c382f4 Update README.md
Fixed install error
2022-01-04 22:44:49 +09:00
Andrew Gaul
1d090aa7a3 Install default-jdk-headless instead of default-jdk
This reduces the dependencies installed.
2022-01-04 18:18:24 +09:00
Andrew Gaul
61e9029be4 Fix typos in CI scripts 2022-01-03 19:50:13 +09:00
Andrew Gaul
5de92e9788
Bump CI to Fedora 35 () 2021-12-02 23:45:19 +09:00
LiuBingrun
85ca2a3e45
fix mixupload return EntityTooSmall while a copypart is less than 5MB after split ()
* fix  mixupload return EntityTooSmall while a copypart is less than 5MB after split
* fix possible part exceeds 5GB when multipart_copy_size is set to 5120MB
* Update curl.cpp
Co-authored-by: liubingrun <liubr1@chinatelecom.cn>
2021-11-27 16:53:26 +09:00
Andrew Gaul
07e2e3f72a
Remove sleep 1 from test_update_directory_time ()
Reduces per-flag test run-time by 5 seconds.
2021-11-04 08:16:40 +09:00
Andrew Gaul
3cf00626a2
Add option to allow unsigned payloads ()
This reduces CPU usage of sigv4.  This reduces test run-time by 7
seconds per flag.
2021-11-01 23:33:55 +09:00
Andrew Gaul
e289915dcb
Remove require-root script ()
Tests do not require this.
2021-10-31 10:48:15 +09:00
Andrew Gaul
06dec32965
Use AWS CLI to create explicit times in the past ()
s3fs can also do this via utimensat but tests should not trust this.
Also break tests into individual functions.  This further reduces test
run-time 8 seconds per flag.
2021-10-30 10:54:18 +09:00
Andrew Gaul
86317dd185
Replace dd if=/dev/urandom with junk data generator ()
This reduces test run time for a single flag from 73 to 60 seconds.
2021-10-28 22:54:25 +09:00
Andrew Gaul
473da56abf
Use default JDK instead of forcing Java 8 ()
S3Proxy requires Java 8 or later, not 8 specifically.
2021-10-28 22:27:48 +09:00
Andrew Gaul
162ab14517
Bump Ubuntu CI to latest non-LTS version () 2021-10-28 22:10:20 +09:00
Andrew Gaul
40d2e0d1ad
Reduce sleep time to 1 ()
This reduces test run-time 15 seconds per flag or 2.5 minutes when
testing all flags.
2021-10-27 23:47:08 +09:00
Takeshi Nakatani
b6c5069ef7 Fixed the test is multi-block writing by one flush 2021-10-27 08:19:05 +09:00
Takeshi Nakatani
7273d561f5 Added exclusive control of statc variables in s3fs xml parser 2021-10-27 08:18:19 +09:00
Takeshi Nakatani
78126aea0b Added exclusive control of statc variables in s3fs xml parser 2021-10-27 08:18:19 +09:00
Takeshi Nakatani
7892eee207 Fixed a bug that copied without considering the length of xmlChar 2021-10-27 08:18:19 +09:00
Andrew Gaul
72a9f40f3f
Update to S3Proxy 1.9.0 ()
Notably this fixes an issue with the transient provider reading parts
of large files.

Release notes:

https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.9.0
2021-10-26 23:20:52 +09:00
Andrew Gaul
495d51113c
Remove unneeded sleeps from tests ()
Also use a unique file name for every test.  This ensures that tests
like test_external_directory_creation and test_external_modification
do not collide.
2021-10-26 23:19:14 +09:00
Carsten Grohmann
0abeec9cae Simpily errexit modifications 2021-10-26 21:47:36 +09:00
Andrew Gaul
ea3c21f270
Reduce errexit modifications ()
This is less error prone but requires some magic && ||.
2021-10-25 23:53:45 +09:00
Takeshi Nakatani
23fe6f4dee Fixed parse_string function in write_multiblock.cc 2021-10-25 17:56:49 +09:00
Takeshi Nakatani
34ea2acd75 Add a test that is multi-block writing by one flush 2021-10-25 17:56:49 +09:00
Takeshi Nakatani
ea64886469 Fixed a bug in test_(zero_)cache_file_stat test function 2021-10-24 18:24:12 +09:00
Takeshi Nakatani
023aaf7dff Fixed wrong stat of cache after new creation file
And added a test for stat of cache after new creation file
2021-10-17 16:10:14 +09:00
Takeshi Nakatani
2f412804e2 Fixed forgetting to clear the dirty flag for meta information
Addressed an error in macos cpp check
2021-10-15 22:54:55 +09:00
Takeshi Nakatani
d6ffd389da Excluded ubuntu 16.04 from the CI build execution environment 2021-10-15 08:54:13 +09:00
Carsten Grohmann
be0b17329a
Fix wrong function name in log message () 2021-10-10 11:08:32 +09:00
Kamil Jakrzewski
b4edad86d6 remove Expect: 100-continue header when requesting an IMDSv2 access token 2021-09-09 08:12:36 +09:00
Noah Meyerhans
9d1552a54e fix IAM role retrieval from IMDSv2
AWS IMDSv2 support was added in , but the implementation did not
cover the addional IMDS access that occurs with the iam_role=auto
configuration.  This change implements IMDSv2 support for the IMDS
call to determine the instance's role name.

See also
https://stackoverflow.com/questions/69031023/how-to-make-s3fs-use-imds-v2-when-mounting-s3-buckets-from-ec2-instance
2021-09-03 20:36:34 +09:00
Andrew Gaul
47ebfcc60a
Consume return value from curl_easy_setopt ()
Found via Coverity.
2021-09-02 08:07:06 +09:00
Andrew Gaul
beecf32dff
fclose(FILE*) instead of close(fileno(FILE*)) ()
This is the same thing but confuses Coverity.
2021-09-01 19:41:55 +09:00
Andrew Gaul
57b2e4a4f1
Fix 32-bit compilation issues () 2021-08-31 19:36:02 +09:00
Andrew Gaul
48817d849f
Require explicit length in s3fs_decode64 ()
This is available from std::string::size in callers.
2021-08-31 09:22:10 +09:00
VVoidV
d9f2d17040
1. fix RowFlush can not upload last part smaller than 5MB using NoCacheMultipartPost; ()
2. fix deadlock in UploadPendingMeta
2021-08-31 00:41:47 +09:00
Andrew Gaul
cd98afdd7b
Do not NUL terminate base64 decoded output ()
This is binary data and must use the explicit length.
2021-08-31 00:15:47 +09:00
Andrew Gaul
dac6885fb0
Don't over-allocate in base64 encoding and decoding () 2021-08-30 00:03:10 +09:00
VVoidV
fcd180891b
fix misuse of IsUploading ()
Co-authored-by: liubingrun <liubr1@chinatelecom.cn>
2021-08-29 23:41:02 +09:00
Vincent Milum Jr
d5d541c7f7 Adding FreeBSD example to README
S3FS has existing on FreeBSD since 2009, and should be reflected here that it is well supported.
2021-08-26 10:18:56 +09:00
Takeshi Nakatani
a868c0656e Changed etaglist_t from string list to new structure etagpairs list 2021-08-16 09:27:12 +09:00
Andrew Gaul
cd466ebdd4
Update ChangeLog and configure.ac for 1.90 ()
Fixes .
2021-08-08 14:01:19 +09:00
Andrew Gaul
15e89b78de
Add a partial page_list unit test () 2021-08-04 07:36:32 +09:00