Extending SimpleS3 - Genbox/SimpleS3 GitHub Wiki

Introduction

SimpleS3 is designed to be easy to extend with new functions. If you want to contribute support for a new function, these are the places you should change:

• Request & a request marshaller
• Response & a response marshaller
• Operations
• Client

In order to contribute, create a fork of this library and make the changes in your own version. Once you are finished, simply create a pull request so that it can be reviewed. For more information on how GitHub collaboration works, see their collaboration guidelines.

SimpleS3.Utility.S3Template

This is a utility that helps you generate the boilerplate code required to extend SimpleS3. You simply open Api.txt in the project, enter the name and type of the S3 function and run the utility. By default, it outputs the finished code files to C:\Temp

The Api.txt file looks like this:

GetBucketAccelerateConfiguration,bucket
GetObject,object

The S3 function name comes from the Amazon S3 documentation and the type can either be bucket or object.

Once you have generated files, you can simply move them into the SimpleS3.Core project. Move the whole folder tree and the files are moved to the right place. The folder structure should look something like this

./Internals/Marshallers/Requests/Objects/GetObjectRequestMarshal.cs
./Internals/Marshallers/Responses/Objects/GetObjectResponseMarshal.cs
./Network/Requests/Objects/GetObjectRequest.cs
./Network/Responses/Objects/GetObjectResponse.cs

Requests

A request is just a simple class that holds all the user input. Amazon S3 takes in values in their requests, so you need a property for each value that S3 accepts.

Here is an example of a request:

public class GetObjectRequest : BaseRequest, IHasRequestPayer
{
    internal GetObjectRequest() : base(HttpMethod.GET)
    {
    }

    public GetObjectRequest(string bucketName, string objectKey) : this()
    {
        BucketName = bucketName;
        ObjectKey = objectKey;
    }

    public Payer RequestPayer { get; set; }
}

Notice how it inherits from BaseRequest and IHasRequestPayer. All requests need to inherit from BaseRequest, but the interesting part is the IHasRequestPayer interface. SimpleS3 comes with a lot of built-in interfaces that help with keeping a standard name for common properties across requests, so if your request has support for RequestPayer, simply inherit from the IHasRequestPayer interface and you are good to go. A really nice feature is that SimpleS3 automatically detect that you inherit from IHasRequestPayer and maps it, so you don't need to create a marshaller. However, if your request has a custom property, then you need a request marshaller.

Request Marshaller

A request marshaller translates your request to an Amazon S3 HTTP request. HTTP works with headers and parameters, so we need to map your request to those.

Note: If your request only inherits from one of the IHasXXXXX interfaces and have no other properties, you don't need to create a custom request marshal.

Here is an example of a request marshaller:

internal class GetObjectRequestMarshal : IRequestMarshal<GetObjectRequest>
{
    public Stream MarshalRequest(GetObjectRequest request, IConfig config)
    {
        request.SetHeader("x-amz-request-charged", request.RequestPayer == Payer.Requester ? "requester" : null);
        return null;
    }
}

As you can see, it sets the header x-amz-request-charged to "requester" if the RequestPayer property is set to Payer.Requester. We return null since we do not have a stream on the request that we need to send. If your request has stream content, return that instead.

Response

We need a response that contains the data that comes back from Amazon S3 after we have sent a request. Like requests, we can also inherit from IHasXXXXX interfaces in responses, and if we don't have any custom properties, we don't need to make a response marshaller as well.

Here is an example of a response:

public class GetObjectResponse : BaseResponse, IHasContent, IHasRequestCharged
{
    public ContentReader Content { get; internal set; }
    public bool RequestCharged { get; internal set; }
}

All responses need to inherit from BaseResponse. Notice the IHasContent and IHasRequestCharged interfaces. The IHasContent makes sure we have the Content property that contains a stream, and the IHasRequestCharged makes sure we have the RequestCharged property, that is a boolean to indicate if the request was charged to the requester or not (this is a feature of S3).

In order to map from HTTP to the request above, we need a response marshaller.

Response marshaller

A response marshaller maps parameters, headers and body content to an S3 response.

Here is an example of a response marshaller:

internal class GetObjectResponseMarshal : IResponseMarshal<GetObjectRequest, GetObjectResponse>
{
    public void MarshalResponse(IConfig config, GetObjectRequest request, GetObjectResponse response, IDictionary<string, string> headers, Stream responseStream)
    {
        response.RequestCharged = headers.ContainsKey("x-amz-request-charged");
        response.Content = new ContentReader(responseStream);
    }
}

In this response marshaller, we set the RequestCharged property if the HTTP response contains a header called "x-amz-request-charged" and we set the Content property to the body response steam.

Operations

Once you have a request + marshaller and a response + marshaller, then we need to add the function to the SimpleS3 API. If your request is object based, then you add it to ObjectOperations. If it is bucket based, then you add it to BucketOperations. Operations are the lower API layer in SimpleS3 and all functions must be mapped on this level.

Here we have added the GetObjectAclAsync method to ObjectOperations:

public class ObjectOperations : IObjectOperations
{
    public Task<GetObjectAclResponse> GetObjectAsync(GetObjectRequest request, CancellationToken token = default)
    {
        return _requestHandler.SendRequestAsync<GetObjectRequest, GetObjectResponse>(request, token);
    }
}

It simply takes in the request and gives back a response. The _requestHandler field is already present in the class when you extend it. Remember to take in a CancellationToken in your method as well.

Client

The clients are part of the higher-level APIs and make it easier for users to send a request without a lot of boilerplate code. If your request/response is related to objects, then add your method to S3ObjectClient. If they are related to buckets, add your method to the S3BucketClient instead.

Here we have added the GetObjectAsync method to S3ObjectClient:

public class S3ObjectClient : IObjectClient
{
    public Task<GetObjectResponse> GetObjectAsync(string bucketName, string objectKey, Action<GetObjectRequest> config = null, CancellationToken token = default)
    {
        GetObjectRequest req = new GetObjectRequest(bucketName, objectKey);
        config?.Invoke(req);

        return ObjectOperations.GetObjectAsync(req, token);
    }
}

Finishing up

That's it, you have now extended SimpleS3 with a new S3 function.