Previously s3fs would issue a batch of HEAD requests and wait for all
to succeed before issuing the next batch. Now it issues the first
batch and only waits for a single call to succeed before issuing the
next call. This can improve performance when one call lags due to
network errors. I measured 25% improvement with the same level of
parallelism. This commit also reparents parallelism knobs for
consistency. Follows on to 88cd8feb05.
Fixes#223.
Previously s3fs would issue a batch of uploads and wait for all to
succeed before issuing the next batch. Now it issues the first batch
and only waits for a single part to succeed before uploading the next
part. This can improve performance when one part lags due to network
errors. Fixes#183.
