octoleo/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2024-11-11 07:10:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,418 Commits 1 Branch 23 Tags 12 MiB
a24f78f5a4
Commit Graph

89 Commits

Author SHA1 Message Date
Alvaro Huarte
a24f78f5a4 Adds requester_pays option to enable requests involving Requester Pays buckets 2019-11-18 12:38:16 +01:00
Takeshi Nakatani
1db94a0b30 Fixed to use copy api in multipart upload 2019-09-25 19:30:58 -07:00
Takeshi Nakatani
bedd648d47 Fixed build error by cppcheck 1.89 2019-09-23 10:49:49 +00:00
Takeshi Nakatani
0536dc1112 Refixed strict processing of MultiRead method 2019-08-25 20:46:01 -07:00
Andrew Gaul
cc2eed84a5 Promote default ACL to enum 
This sanity checks ACLs during initialization and also omits sending
the header when set to PRIVATE which is the default.
 2019-08-20 15:05:58 -07:00
Takeshi Nakatani
5c4a0a862a Fixed multipart copy and its retry handler 2019-08-11 16:21:26 +00:00
Andrew Gaul
7ecfba811e Do not heap allocate BodyData 
Callers already manage lifetimes via Clear().
 2019-08-03 15:50:46 -07:00
Andrew Gaul
b8ff6a647e Add simple XML parsing wrapper 
Also simplify check_region_error.
 2019-07-05 17:39:09 -07:00
Andrew Gaul
15b7450713 Convert file offsets from size_t to off_t 
The latter is 64-bits on 32-bit platforms when specifying
-D_FILE_OFFSET_BITS=64.  This allows early Raspberry Pis to use files
larger than 2 GB.  It also cleans up some ugly casting.  Fixes #620.
Fixes #656.
 2019-06-15 17:05:37 -07:00
Michal Lula
0d4847596e fix issue with aws session token provided inside ~/.aws/credentials file 2019-04-25 14:29:35 +02:00
Michal Lula
f9cd43b684 add session token support 2019-04-16 16:53:05 +02:00
Takeshi Nakatani
71766039ff Support undefined CURLoption in libcurl library used in build 2019-03-22 10:47:42 +00:00
Andrew Gaul
1e86cc643d Fix lazy typo 2019-03-09 17:46:02 +09:00
Takeshi Nakatani
10d9f75366 Improvement of curl session pool for multipart 
Improvement of curl session pool for multipart(2)

Improvement of curl session pool for multipart(3)
 2019-02-25 14:46:24 +00:00
Takeshi Nakatani
d8185a25aa
Merge pull request #946 from gaul/async/completed-tids 
Simplify async request completion code
 2019-02-03 14:08:49 +09:00
Andrew Gaul
8c527c3616 Simplify async request completion code 
Workers now notify the master thread when they complete, unifying the
Linux and macOS code paths.  This also avoids excessive
pthread_tryjoin_np calls.  Follows on to
88cd8feb05.
 2019-01-31 22:55:23 -08:00
Andrew Gaul
c5ebf5d328 Copy parts in parallel 
S3 can copy multipart much faster than single part due to IO
parallelization.  Renaming a 4 GB file reduces from 72 to 20 seconds
with bigger gains with larger files.
 2019-01-31 10:21:39 -08:00
Andrew Gaul
46d79c5bc2 Issue readdir HEAD requests without batching 
Previously s3fs would issue a batch of HEAD requests and wait for all
to succeed before issuing the next batch.  Now it issues the first
batch and only waits for a single call to succeed before issuing the
next call.  This can improve performance when one call lags due to
network errors.  I measured 25% improvement with the same level of
parallelism.  This commit also reparents parallelism knobs for
consistency.  Follows on to 88cd8feb05.
Fixes #223.
 2019-01-20 18:07:22 -08:00
Andrew Gaul
88cd8feb05 Upload S3 parts without batching 
Previously s3fs would issue a batch of uploads and wait for all to
succeed before issuing the next batch.  Now it issues the first batch
and only waits for a single part to succeed before uploading the next
part.  This can improve performance when one part lags due to network
errors.  Fixes #183.
 2018-11-16 18:32:38 -08:00
Jonatan Nilsson
d9c106cfde Add support for storage class ONEZONE_IA. 2018-09-30 14:27:31 +02:00
Takeshi Nakatani
ae9d8eb734 Fixed memory leak 2018-05-27 10:48:03 +00:00
Or Ozeri
0bd875eb9e remove false readdir_multi_head warnings 2018-05-22 17:10:50 +03:00
Or Ozeri
bd4bc0e7f1 add support for IBM IAM authentication 2017-11-23 12:01:52 +02:00
Andrew Gaul
0418e53b3c Reduce use of preprocessor 
This provides type-safety and avoids token expansion side effects.
 2017-11-18 22:40:06 -08:00
Takeshi Nakatani
bad48ab59a
Merge pull request #671 from psyvision/master 
Add support for ECS metadata endpoint
 2017-11-19 14:43:06 +09:00
Richard Caunt
1cbe9fb7a3 Gotta pass that cppcheckgit add . 2017-11-07 21:41:51 +00:00
Richard Caunt
8660abaea2 Use jsoncpp to parse AWS JSON 2017-11-07 21:20:02 +00:00
Richard Caunt
662f65c3c8 Add support for ECS metadata endpoint 2017-11-05 19:24:02 +00:00
Or Ozeri
384b4cbafa auth headers insertion refactoring 2017-10-30 11:52:58 +02:00
Takeshi Nakatani
00bc9142c4 Fixed potential atomic violation in S3fsCurl::AddUserAgent - #633 2017-09-17 09:16:05 +00:00
Takeshi Nakatani
9d10a5aa70 Changed copyright year format for debian pkg 2017-05-07 11:24:17 +00:00
Takeshi Nakatani
df0ff3a2fd Merge pull request #556 from orozery/fix_nocache_multipart_upload 
fix multipart upload handling without cache
 2017-04-16 19:22:15 +09:00
Or Ozeri
96764b7410 switch S3fsMultiCurl to use foreground threads 2017-04-09 16:56:49 +03:00
Or Ozeri
8ee95ff7ab fix multipart upload handling without cache 2017-04-02 10:27:43 +03:00
Andrew Gaul
03217baa99 Address cppcheck 1.77 warnings 2017-03-06 12:41:08 -08:00
Andrew Gaul
d375bca0d0 Correct typos 2016-11-19 15:57:41 -08:00
Takeshi Nakatani
50f1ad51c8 loading IAM role name automatically(iam_role option) - #387 2016-05-06 04:37:32 +00:00
Tianlong Wu
b78adb4bb0 Add curl handler pool to reuse connections 2016-04-22 14:57:31 +08:00
Takeshi Nakatani
10589a9497 Supported User-Agent header - #383 2016-04-17 07:44:03 +00:00
Takeshi Nakatani
c7cf86c2ef Sepalated AdditionalHeader class from curl.* 2016-02-07 05:41:56 +00:00
Takeshi Nakatani
6472eedddc Supported regex type for additional header format. 2016-02-07 05:08:52 +00:00
Takeshi Nakatani
4bfbfa3621 Merge pull request #331 from andrewgaul/clang 
Address various clang warnings
 2016-01-16 16:14:50 +09:00
Andrew Gaul
e003732f18 Address various clang warnings 
Found with:

-Wc++11-extensions
-Wc++11-extra-semi
-Wmissing-variable-declarations
-Wundef
 2016-01-11 00:52:24 -08:00
Andrew Gaul
b946b59522 Pass by const reference where possible 2016-01-10 16:58:24 -08:00
Takeshi Nakatani
83d46ef8c6 Fixed bugs about a object larger than free disk space 2015-10-20 15:19:04 +00:00
Takeshi Nakatani
d102eb752d Supported a object which is larger than free disk space 2015-10-18 17:03:41 +00:00
Takeshi Nakatani
f51ad1f33e Supported for SSE KMS 2015-10-06 14:46:14 +00:00
Takeshi Nakatani
92e52dadd4 Changed and cleaned the logic for debug message. 2015-09-30 19:41:27 +00:00
Andrew Gaul
785ed642ba Add support for standard_ia storage class 
This enables storage with lower at-rest prices, higher request prices,
and lower availability.  Also rework existing reduced redundancy
parsing into a more generic storage class.  More background on
standard_ia:

https://aws.amazon.com/blogs/aws/aws-storage-update-new-lower-cost-s3-storage-option-glacier-price-reduction/
 2015-09-17 13:35:25 -07:00
Andrew Gaul
b6fa2deb9f Annotate constructors as explicit 
This prevents implicit conversions.
 2015-08-05 23:41:53 -07:00