Working with self signed certificates and MinIO with DotNetSDK - minio/wiki GitHub Wiki

This document aims at listing steps for easy deployment of MinIO using self signed certificates and accessing the buckets and objects from Windows boxes using DotNetSDK.

Credits

A big thanks to Ersan Bozduman for DotNet SDK client and guidance through figuring out ways to solve it! 👏

Deploy MinIO instance using self signed certificates

Follow the below steps to start a HTTPS enabled instance of MinIO

Step-1: Generate server certificates using certgen tool

certgen --host "127.0.0.1,localhost,YOU-PUBLIC-IP"

Step-2: Generate client certificates using the certgen tool

certgen --client --host "127.0.0.1,localhost,YOUR-PUBLIC-IP" --common-name "minio.local"

Make sure to use CN using the option -common-name for the client certificates

Step-3: Copy the server (public.crt) and client (client.crt) to /usr/share/ca-certificates. This is required to trust the self signed certificates

sudo cp public.crt  /usr/share/ca-certificates
sudo cp client.crt  /usr/share/ca-certificates

Step-4: Use below command to include both server's public and client certificates as trusted

sudo dpkg-reconfigure ca-certificates

You may need to install ca-certificates on your server if not available already

sudo apt-get install ca-certificates

Step-5: Start the MinIO instance using the server certificates

cp public.crt ~/.minio/certs
cp private.key ~/.minio/certs
MINIO_IDENTITY_TLS_ENABLE=on minio start ./data

Create a policy for the bucket to be accessed from outside

Step-1: Create a JSON with name CN.json where CN is common name used above

cat minio.local.json 
{
   "Version" : "2012-10-17",
   "Statement" : [
      {
         "Action" : [
            "s3:GetBucketLocation",
            "s3:ListBucket",
            "s3:ListBucketMultipartUploads"
         ],
         "Resource" : [
            "arn:aws:s3:::test1"
         ],
         "Principal" : {
            "AWS" : [
               "*"
            ]
         },
         "Effect" : "Allow"
      },
      {
         "Action" : [
            "s3:AbortMultipartUpload",
            "s3:DeleteObject",
            "s3:GetObject",
            "s3:ListMultipartUploadParts",
            "s3:PutObject"
         ],
         "Effect" : "Allow",
         "Principal" : {
            "AWS" : [
               "*"
            ]
         },
         "Resource" : [
            "arn:aws:s3:::test1/*"
         ]
      }
   ]
}

Note test1 is the bucket which would be used

Step-2: Create policy

mc alias set m1 https://YOUR-PUBLIC-IP:9000 minioadmin minioadmin
mc admin policy create m1 minio.local ./minio.local.json

Step-3: Verify that you are able to create STS credentials using this instance now

curl -XPOST --key ./client.key --cert ./client.crt "https://YOUR-PUBLIC-IP:9000?Action=AssumeRoleWithCertificate&Version=2011-06-15&DurationSeconds=3600"

You should be able to get temporary STS credentials details as below

<?xml version="1.0" encoding="UTF-8"?>                                                                     
<AssumeRoleWithCertificateResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/"><AssumeRoleWithCertificateResult><Credentials><AccessKeyId>BUU9HSGXTTVPTC1VWIVU</AccessKeyId><SecretAccessKey>sbjlg55YjB8Ch9sffIzO7GYojov9OuNI3ZvEwkk7</SecretAccessKey><SessionToken>eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJCVVU5SFNHWFRUVlBUQzFWV0lWVSIsImF1ZCI6WyJDZXJ0Z2VuIERldmVsb3BtZW50Il0sImV4cCI6MTY4MTEzNjQyNywiaXNzIjoibWluaW8ubG9jYWwiLCJwYXJlbnQiOiJ0bHM6bWluaW8ubG9jYWwiLCJzdWIiOiJtaW5pby5sb2NhbCJ9.w9RlWYXB4H0-kecisL2zQw2Oz_jhktHax8e7VtKgcBgkeOx5ho7n3ceMcP4AaS2YgQ03t58wnksX9LVWl3-GOQ</SessionToken><Expiration>2023-04-10T14:20:27Z</Expiration></Credentials></AssumeRoleWithCertificateResult><ResponseMetadata><RequestId>17549566383E2E12</RequestId></ResponseMetadata></AssumeRoleWithCertificateResponse>

Trust the certificates on Windows box now. Use certutil tool available with DotNet framework

Step-1: Execute below command to add client and server certificates to trusted list

certutil -addstore root c:\PATH\TO\client.crt
certutil -addstore root c:\PATH\TO\public.crt

Step-2: Create client.pfx on your server and move to Windows machine. In case of older versions of Windows (e.g. Windows 2016) you may need to create .pfx file using older versions of openssl. For example for Windows 2016, we need to use openssl 1.1.1

openssl pkcs12 -export -out client.pfx -inkey client.key -in client.crt

Use export password manage on prompt as its hard-coded in below sample code.

Step-3: Clean and trust dev-certs (optional)

dotnet dev-certs https --clean
dotnet dev-certs https --trust

Execute MinIO DotNet SDK based client to load and download an object to the above mentioned instance and bucket

Step-1: Sample client

using System.Security.Cryptography.X509Certificates;
using Minio;
using Minio.DataModel;
using Minio.Credentials;
using System.Security.Authentication;
using System.Threading.Tasks;

namespace myProject
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello, World!");
            CertificateProvider_Test();
        }

        internal static void CertificateProvider_Test()
        {
            var endPoint = "https://YOUR-PUBLIC-IP:9000/";
            var objectName = "obj";
            var bucketName = "test1";
            var uploadFile = "file1.txt";
            var downloadFile = "Copy-file1.txt";
            var remoteAddress = "YOUR-PUBLIC-IP";
            var Port = 9000;
            try
            {
                using (var cert = new X509Certificate2("client.pfx", "manage"))
                {
                    var provider = new CertificateIdentityProvider()
                        .WithStsEndpoint(endPoint)
                        .WithCertificate(cert).Build();

                    var handler = new HttpClientHandler();
                    handler.ClientCertificateOptions = ClientCertificateOption.Manual;
                    handler.SslProtocols = SslProtocols.None;
                    MinioClient minioClient = new MinioClient()
                        .WithHttpClient(new HttpClient(handler))
                        .WithEndpoint(remoteAddress + ":" + Port).WithSSL()
                        .WithCredentialsProvider(provider).Build();

                    Console.Write("Uploading ...");
                    Task.Run(async() => await uploadObject(minioClient, bucketName, objectName, uploadFile, null)).Wait();
                    Console.WriteLine("DONE\nCertificateIdentityProvider Upload test PASSed\n");


                    Console.Write("Downloading ...");
                    Task.Run(async() => await downloadObject(minioClient, bucketName, objectName, downloadFile, null)).Wait();
                    Console.WriteLine("DONE\nCertificateIdentityProvider Download test PASSed\n");
                }
            }
            catch (Exception e)
            {
                Console.WriteLine($"\nCertificateIdentityProvider test exception: {e}\n");
                throw; 
            }
        }

        static async Task uploadObject(MinioClient minio, string bucketName, string objectName, string fileName,
        ServerSideEncryption sse = null)
        {
            try
            {
                var bs = File.ReadAllBytes(fileName);
                Console.WriteLine($"Read {bs.Length} many Bytes");

                using (var filestream = new MemoryStream(bs))
                {
                    var metaData = new Dictionary<string, string>
                    {
                        { "Test-Metadata", "Test  Test" }
                    };
                    var args = new PutObjectArgs()
                        .WithBucket(bucketName)
                        .WithObject(objectName)
                        .WithStreamData(filestream)
                        .WithObjectSize(filestream.Length)
                        .WithContentType("application/octet-stream")
                        .WithHeaders(metaData);
                    await minio.PutObjectAsync(args).ConfigureAwait(false);
                }
                Console.WriteLine($"Uploaded object {objectName} to bucket {bucketName}");
            }
            catch (Exception e)
            {
                Console.WriteLine($"[Bucket]  Exception: {e}");
                throw;
            }
        }

        static async Task downloadObject(MinioClient minio, string bucketName, string objectName,
        string downloadFile, ServerSideEncryption sse = null)
        {
            try
            {
                var args = new GetObjectArgs()
                    .WithBucket(bucketName)
                    .WithObject(objectName)
                    .WithFile(downloadFile);
                var stat = await minio.GetObjectAsync(args);
                Console.WriteLine($"Downloaded the file {downloadFile} from bucket {bucketName}");
                Console.WriteLine($"Stat details of object {objectName} in bucket {bucketName}\n" + stat);
                Console.WriteLine();
            }
            catch (Exception e)
            {
                Console.WriteLine($"[Bucket]  Exception: {e}");
                throw;
            }
        }
    }
}

Thanks Ersan Bozduman for contributing this sample code. Make sure you have a file file1.txt with some sample data.

Step-2: Use below command to execute the client

dotnet run --framework net7.0

You need to have a proper DotNet framework deployed and dotnet commands available.

Note: You would need MinIO SDK package to be downloaded and available. Follow the below steps to assure the same.

Step-1: Update your .csproj as below

<PropertyGroup>
  <RestoreSources>$(RestoreSources);PATH/TO/DOWNLOADED/MINIO/NUPKG/FILE;https://api.nuget.org/v3/index.json</RestoreSources>
</PropertyGroup>

The minio.nupkg can be downloaded from https://www.nuget.org/packages/Minio

Step-2: Execute below command to add package Minio

dotnet add package Minio
⚠️ **GitHub.com Fallback** ⚠️