Using JMX over SSL with Java Mission Control - tjayr/remote-jmx-monitoring GitHub Wiki

Remote JMX has always been a little awkward, most tutorials on the subject tend to avoid securing the connection as it makes it even more challenging. This is a set of steps to enable file based authentication and SSL for remote JMX connections using self signed certificates.

Keystore Explorer - https://keystore-explorer.org/

JDK Mission Control - https://www.oracle.com/java/technologies/jdk-mission-control.html

The sample application is configured to use the key stores, trust stores and certificates mentioned in this document. See the readme for build and run instructions.

To enable authentication for JMX, 2 files need to be created on the server running the application

The password file contains username and password pairs:

jmxremote.password

jmc_user   <password>

The access file contains roles for each user:

jmxremote.access

jmc_user    readwrite \
                create javax.management.monitor.*,javax.management.timer.* \
                unregister

The jmxremote.password file needs to be readable only by the user running Java:

$ chmod 400 jmxremote.password

Add the following JVM properties to the server application to enable authentication:

-Dcom.sun.management.jmxremote.authenticate=true
-Dcom.sun.management.jmxremote.access.file=/path/to/jmxremote.access
-Dcom.sun.management.jmxremote.password.file=/path/to/jmxremote.password

When connecting to a target application with Mission Control, enter the credentials in the connection properties window

At this point, we have an authenticate connection, but the connection is not encrypted and the credentials will be transmitted in the clear. To secure this connection SSL needs to be configured.

This is where things get a bit more involved. we need to create keystores, truststores, generate private keys, certificates and then exchange the certificates between the client and server in order to establish trust. The key pairs and certificates are created using Keystore Explorer (they could also be created with Java keytool).

Much of the difficulty lies in these steps, as if a mistake is made the connection will fail but it won't give any useful error messages to aid with debugging (It can further complicated if the connection is going through firewalls).

We'll do the following steps in order:

1. Create a keystore and key pair for Java Mission Control
2. Create a trust store for Java Mission Control
3. Create a keystore and key pair for the application to be monitored
4. Create a trust store for the application to be monitored
5. Add the certificate from Java Mission Control keystore to the applications trust store
6. Add the certificate from the application to the Java Mission Control trust store

In Keystore Explorer select:

File > New > PKCS #12

File > Save As > File name as mission-control-keystore

When prompted for a password enter changeit

For simplicity, the same password will be used for all subsequent keystores, keys and truststores

Next, generate a key pair:

Tools > Generate Key Pair

For the Algorithm Selection: RSA and Key size 2048

You can change the validtity period to something longer than 1 year if required:

A certificate name is required, click the edit icon and enter the details:

When prompted enter an alias name e.g. java-mission-control

When prompted, use changeit as the key password

Repeat the process to create a truststore file

File > New > PKCS #12

File > Save As > File name as mission-control-truststore

When prompted for a password enter changeit

File > New > PKCS #12

File > Save As > File name as monitored-app-keystore

When prompted for a password enter changeit

Next, generate a key pair:

Tools > Generate Key Pair

For the Algorithm Selection: RSA and Key size 2048

You can change the validtity period to something longer than 1 year if required:

A certificate name is required, click the edit icon and enter the details:

When prompted enter an alias name e.g. monitored-application

When prompted, use changeit as the key password

The server applications trust store is a little different. The JVM has a built in truststore, which is a file called cacerts located in <JVM-HOME>/lib/security. This file contains the certificates of public known certificate authority entities, as the application can have only 1 trust store its a good to have all of these certificates in our trust store, we'll make a copy of this file to use it as the basis for the applications trust store - monitored-app-truststore.

Why make a copy, why not add our certs to cacerts directly?

Yes, this is an option, however if the JVM is upgraded the cacerts file will be overwritten and our imported certificates will be lost. By making a copy we can preserve our imported certificates through upgrades. There will still be maintenance of the cacerts required eventually as after a JVM upgrade we won't have any new CA entries in our copy of the truststore - however our monitoring applications will still work and won't accidentally break due to a JVM upgrade.

The next step is to export a certificate from the key pair.

In the mission-control-keystore, right click on the java-mission-control alias created in the previously step and select:

Export > Export Certificate Chain

Export length is Head only and the format is X.509

Select a path and enter a filename e.g. java-mission-control.cer

Import the certificate to the application truststore

Open the monitored-app-truststore

Choose Tools > Import Trusted Certificate

Select the certificate just exported e.g.g java-mission-control.cer

Save the changes

Now the process is repeated to the application certificate to the mission control trust store

Open the monitored-app-keystore in Keystore Explorer monitored-app-keystore, right click on the monitored-application alias created previously step and select:

Export > Export Certificate Chain

Export length is Head only and the format is X.509

Select a path and enter a filename e.g. monitored-application.cer

Import the certificate to the mission control truststore

Open the mission-control-truststore

Tools > Import Trusted Certificate

Select the certificate just exported e.g. monitored-application.cer

Save the changes.

Window > Preferences > JDK Mission Control > JMXRMI

Enter the path to the mission-control-keystore and mission-control-truststore and set the passwords.

Restart Mission control for the changes to take effect.

The following properties need to be added to the applcation to enable SSL:

-Dcom.sun.management.jmxremote.ssl=true
-Dcom.sun.management.jmxremote.ssl.need.client.auth=true
-Dcom.sun.management.jmxremote.registry.ssl=true
-Djavax.net.ssl.keyStore=/var/tmp/keys/monitored-app-keystore
-Djavax.net.ssl.keyStorePassword=changeit
-Djavax.net.ssl.trustStore=/var/tmp/keys/monitored-app-truststore
-Djavax.net.ssl.trustStorePassword=changeit

The complete set of configuration for SSL with authentication for the application (applied via docker-compose.yml):

java -Djava.rmi.server.hostname=0.0.0.0
        -Djava.net.preferIPv4Stack=true
        -Dcom.sun.management.jmxremote
        -Dcom.sun.management.jmxremote.port=7091
        -Dcom.sun.management.jmxremote.rmi.port=7091
        -Dcom.sun.management.jmxremote.local.only=false
        -Dcom.sun.management.jmxremote.authenticate=true
        -Dcom.sun.management.jmxremote.access.file=/var/tmp/auth/jmxremote.access
        -Dcom.sun.management.jmxremote.password.file=/var/tmp/auth/jmxremote.password
        -Dcom.sun.management.jmxremote.ssl=true
        -Dcom.sun.management.jmxremote.ssl.need.client.auth=true
        -Dcom.sun.management.jmxremote.registry.ssl=true
        -Djavax.net.ssl.keyStore=/var/tmp/keys/monitored-app-keystore
        -Djavax.net.ssl.keyStorePassword=changeit
        -Djavax.net.ssl.trustStore=/var/tmp/keys/monitored-app-truststore
        -Djavax.net.ssl.trustStorePassword=changeit
    -jar /opt/monitored-app/monitored-app-1.0.1.jar