Skip to content
Snippets Groups Projects
Commit 507f7cb7 authored by Fons Rademakers's avatar Fons Rademakers
Browse files

From Fabio Hernandez:

The enclosed patch includes a new class and some modifications to existing
classes. The new class TS3WebFile is an extension of TWebFile and belongs
to the net module. I deliberately did not modify the existing TAS3File.
I think that the name TS3WebFile reflects better the fact that this solution
is intended to be generic to several S3 servers and not limited to Amazon's,
in addition to the fact that it actually extends the capabilities of TWebFile.

Compared to the current support of S3 in ROOT (basically the class TAS3File),
the modifications in this patch include the improvements below:

 - add support for using HTTPS : you can use different schemas for specifying
   the underlying transport protocol to use "s3:", "s3http:", "s3https:" 
   ["s3" uses HTTPS]. The current schema, namely "as3:", is supported for
   backwards compatibility.
 - extend support for other S3 service providers that do not offer the
   virtual hosting functionality (currently only Amazon offers this)
 - support the possibility of specifying user credentials on a per-file
   basis or for all S3 files via environment variables
 - honor the "NOPROXY" option when specified in the constructor
 - exploit the capability of the S3 file server to provide partial
   content responses to multi-range HTTP requests

Here are some examples of usages from the end user perspective:

   TFile* f = TFile::Open("s3://s3.amazonaws.com/mybucket/path/to/my/file", "AUTH=<accessKey>:<secretKey> NOPROXY")
   TFile* f = TFile::Open("s3://s3.amazonaws.com/mybucket/path/to/my/file")   // Uses environmental variables for retrieving credentials

Limitations:

 - we cannot efficiently detect that a S3 server is able to respond to
   multi-range HTTP GET requests. Some servers, such as Amazon's, respond
   to such kind of requests with the whole file contents. Other servers,
   such as Huawei's, respond with the exact partial content requested.
   Therefore, I added the possibility of configuring the behavior via the
   ROOT configuration file: the identity of the servers known to correctly
   support multi-range requests is configurable. If the server is known to
   support this feature, ROOT will send multi-range requests, otherwise it
   will issue multiple single-range GET requests, which is also the default
   behavior. 
 - currently the virtual host syntax
      "s3://mybucket.s3.amazonaws.com/path/to/my/file"
   is not supported but can be added if this is considered useful


git-svn-id: http://root.cern.ch/svn/root/trunk@48397 27541ba8-7e3a-0410-8455-c3a389f83636
parent 9e18172b
Branches
Tags
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment