Note

The content if this page is copied from here with edits and removals of some parts not applicable to our infrastructure. This copy was performed just to the information is not lost and simplify the use.

The certregen module can painlessly refresh a certificate authority (CA) that's about to expire. It can also revive a CA that has already expired.

This module is for regenerating and redistributing Puppet CA certificates and refreshing CRLs, without invalidating certificates signed by the original CA.

A Puppet deployment's CA certificate is only valid for a limited time (usually five years), after which it expires. When a CA expires, Puppet's services will no longer accept any certificates signed by that CA, and your Puppet infrastructure will immediately stop working.

If your Puppet infrastructure has been in place for almost five years, you should:

• Check to see if your CA is expiring soon.

If your CA is expiring soon (or it's already expired and Puppet has stopped working), you should:

• Generate a new CA certificate using the existing CA keypair. This will also automatically update the expiration date of the certificate revocation list (CRL).
• Distribute the new CA cert and CRL to every node in your Puppet infrastructure.

The certregen module can help you with all of these tasks.

Note: This module is NOT currently designed for the following tasks:

• Re-keying and replacing a CA that has become untrustworthy (by a private key compromise or by a vulnerability like Heartbleed or the old certdnsnames bug).
• Refreshing normal node certificates that are nearing expiration.

For the time being, you're on your own for these.

To install the module run:

sudo puppet module install puppetlabs-certregen

The certregen module can help you with four main tasks:

• Check whether any certificates (including the CA) are expired or nearing their expiration date.
• Refresh and redistribute a CA that hasn't yet expired, in a working Puppet deployment.
• Refresh and redistribute a CRL that hasn't yet expired, in a working Puppet deployment.
• Revive and redistribute an expired CA and CRL, in a Puppet deployment that has stopped working.

The healthcheck action can show you which certificates are expiring soon, as well as any that have already expired. The most important certificate is your CA cert --- if it is almost expired, you must refresh it soon.

1. On your Puppet CA server, run sudo puppet certregen healthcheck.

   This finds any certificates with less than 10% of their lifetime remaining (plus any that have already expired), and lists their certname, fingerprint, remaining lifetime, and expiration date. (If no certs are near expiration, the output is blank.)

   [root@pe-201621-master vagrant]# puppet certregen healthcheck
   "foo.com" (SHA256) 07:2F:61:B4:FB:B3:05:75:9D:45:D1:A8:B1:69:0F:D0:EB:C9:27:03:4E:F8:DD:4A:59:AE:DF:EF:8E:11:74:69
     Status: expiring
     Expiration date: 2016-11-02 19:52:59 UTC
     Expires in: 4 minutes, 19 seconds
   

2. Search the healthcheck output for a cert named "ca", which is your CA. If "ca" is expiring or expired, you must refresh it or revive it as soon as possible. See the tasks below for more details.

To refresh an expiring CA, you must:

• Regenerate the CA certificate
• Distribute the new CA with the certregen::client class.

Before you begin, check to make sure your CA really needs to be refreshed (see above).

1. On your Puppet CA server, run sudo puppet certregen ca.

   This will result in an error, which is normal: since this action replaces your CA cert, it has an extra layer of protection. The error should look something like this:

   Error: The serial number of the CA certificate to rotate must be provided. If you are sure that you want to rotate the CA certificate, rerun this command with --ca_serial 03
   

2. In the error message, find and remember the --ca_serial <NUMBER> option.

3. Run sudo puppet certregen ca --ca_serial <NUMBER>, using the number from the error message.

   By default, this gives the new CA a lifetime of five years. If you wish to set a non-default lifetime, you can add --ca_ttl <DURATION>. See the docs on the ca_ttl setting for details.

   When you regenerate the CA certificate, the CRL will be refreshed at the same time with a new expiration of five years as well.

   At this point:

   • The CA certificate on your CA server has been replaced with a new one. The new CA uses the same keypair, subject, issuer, and subject key identifier as the old one; in practical terms, this means it is a seamless replacement for the old CA.
   • The CRL on your CA server has been updated with a new expiration date, but is otherwise unchanged.
   • Your Puppet nodes are still using the old CA and CRL.

4. In your main manifest directory, add a new manifest file called ca.pp. In that file, add this line:

   include certregen::client

   Note: You must do this in every active environment, so that every node receives this class in its catalog. You must also ensure that the certregen module is installed in every active environment.

   If you have a prohibitively large number of environments... contact us, because we're still developing this module and soliciting feedback on it.

5. If your deployment has multiple compile masters, make sure each one completes a Puppet run against the CA server.

6. Ensure that Puppet runs at least once on every other node in your deployment.

7. Restart the Puppet server in the puppet master:

sudo systemctl restart puppetserver.service

At this point, the new CA and CRL is fully distributed and you're good for another five years.

If your CA has expired, Puppet has already stopped working, and recovering will take some extra work. You must:

• Regenerate the CA certificate
• Manually distribute the new CA to the nodes.

Before you begin:

• Check to make sure your CA has really expired (see above).

1. On your Puppet CA server, run sudo puppet certregen ca.

   This will result in an error, which is normal: since this action replaces your CA cert, it has an extra layer of protection. The error should look something like this:

   Error: The serial number of the CA certificate to rotate must be provided. If you are sure that you want to rotate the CA certificate, rerun this command with --ca_serial 03
   

2. In the error message, find and remember the --ca_serial <NUMBER> option.

3. Run sudo puppet certregen ca --ca_serial <NUMBER>, using the number from the error message.

By default, this gives the new CA a lifetime of five years. If you wish to set a non-default lifetime, you can add --ca_ttl <DURATION>. See the docs on the ca_ttl setting for details.

When you regenerate the CA certificate, the CRL will be refreshed at the same time with a new expiration of five years as well.

4. Apply changes and force the regeneration of the puppet master node certificate

When running the following commands have in mind that the puppet master is a node itself. The commands involve both roles and assume that the puppet master hostname is "puppet":

NODE_HOSTNAME="puppet" # EDIT if needed
sudo puppet cert clean ${NODE_HOSTNAME} # master role
sudo systemctl restart puppetserver.service # master role
sudo puppet agent -t # node role

At this point:

• The CA certificate on your CA server has been replaced with a new one. The new CA uses the same keypair, subject, issuer, and subject key identifier as the old one; in practical terms, this means it is a seamless replacement for the old CA.
• The CRL on your CA server_ has been updated with a new expiration date, but is otherwise unchanged.
• The connection of your puppet master to itself as a node has been reset and is functional
• Your Puppet nodes still have the old CA, and Puppet is still non-functional.

5. Manually copy the updated CA and CRL to all the nodes:

In the puppet master copy the ca and crl files to your home:

PUPPET_MASTER="puppet.domain.tld" # EDIT
ssh ${PUPPET_MASTER}

sudo cp /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem ~
sudo cp /etc/puppetlabs/puppet/ssl/ca/ca_crl.pem ~

Download them in your computer:

scp ${PUPPET_MASTER}:ca_crt.pem ~
scp ${PUPPET_MASTER}:ca_crl.pem ~

and copy them to the nodes in question:

PUPPET_NODE="mynode.domain.tld" # EDIT
scp ca_crt.pem ${PUPPET_NODE}:
scp ca_crl.pem ${PUPPET_NODE}:

In each node, move them to its final destination:

ssh ${PUPPET_NODE}
sudo mv ca_crt.pem /var/lib/puppet/ssl/certs/ca.pem
sudo mv ca_crl.pem /var/lib/puppet/ssl/crl.pem

6. Run the agent at the node:

sudo puppet agent -t

7. Repeat the steps 5) and 6) for every node.

At this point, the new CA and CRL is fully distributed and you're good for another five years.

We can check the general status of certificates that are expired or about to expire with:

sudo puppet certregen healthcheck

To refresh this certificate we have to clean it in the puppet master with:

NODE_NAME='mynode' # EDIT
sudo puppet cert clean $NODE_NAME

Then ssh into the node affected and do:

HOSTNAME=`hostname -s`
sudo find /var/lib/puppet/ssl -name ${HOSTNAME}.pem -delete
sudo puppet agent -t

After that, the node should have generated another certificate which you should sign in the puppet master:

sudo puppet cert sign ${NODE_NAME}

If the node in question happens to be the puppet master itself, you should restart the puppet service with

sudo systemctl restart puppetserver.service