Certificates for development and testing – Part 1 (Background) - arcdev/engram404 GitHub Wiki

originally posted 2016-06-26 at https://engram404.net/certificates-for-development-and-testing/

(feel free to skip to Part 2 – HOWTO

–

Ah, certificates – those precious little ingredients in the dark art of encryption that keeps all of our transactions safe and private. They're just little devils to work with.

The Problem

When we start writing software, especially web sites, we as developers tend to forget that our app is going to be secured with TLS (sometimes still referred to as SSL, though SSL is considered deprecated/obsolete as of 2015). Once we start deploying to other environments it inevitably bites us.
Let's solve that problem sooner.

I guess we should review certificates a bit.

When a browser (or other client – I'll just call it a ‘browser' going forward) receives a certificate and tells you that your connection is authenticated and secure there's a whole conversation going on between your browser and the web server. I'm not going to cover that. (You can Google it.) The part we're going to focus on here is the trust relationship. The browser requests that the server provide some form of authentication to prove that the server is who it says it is. In TLS, the server sends a certificate to the browser. That's all well can good, but how does the browser trust that certificate? It looks a who signed that certificate. (We'll get into signing later, just bear with me.) The signatory is called a Certificate Authority (usually referred to as CA) and is a third-party organization (like VeriSign, Comodo, CAcert, and DigiSign, among others) that browsers have agreed to trust. When the browser gets the certificate from the server, it inspects the certificate to see who signed it. If that signer (CA) is trusted by the browser, then the browser treats the server as valid/trusted.

Let's simplify that a bit. We have 2 certificates in play here:

1. CA certificate (a public key)
2. server certificate (signed by the CA)

We should probably take a quick detour on keys.

In our case, we're talking about asymmetric encryption. There are public keys and private keys.
A private key is, well, private. It should never, ever, ever be exposed to the world. If it is, it should immediately be revoked and all certificates signed with that key are now invalid. (This is were a Certificate Revocation List, or CRL, comes into play.)
A public key is, conversely and likely obviously, public. Anyone and everyone can have a copy of it.
The private key can be used to sign something (a certificate in our case).
The public key can be used to verify that the private key signed something.
The public key can be used to encrypt something – but only the private key can decrypt that something. (This is the heart of the asymmetry.)
If you think this through, it's absolutely fantastic. Everyone (with the public key) can prove that the thing they got was truly from the guy with the private key. But they can't pretend to be the guy with the private key. Further, if the one with the public key wants to send something securely to the one with the private key, that can be done without any worry that someone else might be able to decrypt it. I'll stop now on that bit before I go completely over the edge…

Back to the certificates.

The CA certificate is basically a public key. It's used to verify that the certificate provided by the web server was truly signed by the CA.
The CA certificate must be trusted by the operating system and/or browser. Usually, the operating system manufacturer handles this in some form without you even knowing. (Firefox is the one exception I'm aware of because they use a certificate store that is managed outside of the operating system.)
Keep in mind that we, as users, have the option to trust any certificate. (This is what we're going to do a bit later.)

Since getting a certificate is generally rather costly (minimum of $100 per year per server and normally quite a bit more), we tend to simply use a self-signed certificate in non-production environments. (This is one where the CA certificate and the server certificate are one-and-the-same.) A self-signed certificate is great when you're working within a consistent, controlled environment usually within the same operating system family (e.g. Windows). Once you start spanning operating systems, a self-signed certificate can become challenging.

It's too bad we can't just be our own Certificate Authority (CA).

Well, we can – sorta.
If we were to create our own private-key-public-key pair and trust our public key (among non-production devices), then we could use our private key to sign as many certificates we wanted – and all of our devices (that trust our public key) will trust these certificates. (It's close to a self-signed certificate, but provides a bit more separation and a bit closer simulation to the real-world.)

Here are the high-level steps for generating the certificates:

Pretending to be the CA

  1. Create a private key (we'll use this to sign the request later)
  2. Create a public key certificate (this is what we'll tell our devices to trust)

Pretending to be the client organization

  1. Create a private key (that represents the client organization)
  2. Create a certificate request

Pretending to be the CA

  1. Sign the certificate request

Steps 1 & 2 only ever really need to be done once (as long as you don't lose them).
The remaining steps will be done for every server (or developer machine) that needs to be trusted.

The public key certificate (from step 2) gets trusted by your server and by any other device that needs to trust you.
The signed certificate (from step 6) gets configured in your web server as the certificate identifying the server.

So how do we actually accomplish all of this goodness?

Stay tuned.