Enable NiFi service for SSL - stanislawbartkowski/hdpwiredencryption GitHub Wiki
Activating NiFi UI for SSL requires creating keystore and trustore, configuring NiFi using Ambari Console and defining the admin user. Before configuring, the NiFi service should be stopped.
CA-signed keystore and truststore.
There is an additional service, "NIFI CERTIFICATE AUTHORITY" providing certificates for SSL based communication. The certificates are signed by root certificate located in /etc/nifi/conf/nifi-certificate-authority-keystore.jks on "NIFI CERTIFICATE NODE".
There is a procedure to inject externally CA-signed intermediate certificate.
The description below is related only to self-signed certificate NiFi UI SSL communication.
Important: If you want to use CA-signed certificate or certificate created outside "NIFI CERTIFICATE AUTHORITY", make sure that "Subject Alternative Names" certificate extension contains FQDN hostname.
The default locations for keystore and trustore are /etc/nifi/conf/keystore.jks and /etc/nifi/conf/truststore.jks. The keystone can be removed and during NiFi restart, the keystore and truststore are recreated using certificates generated by "NIFI CERTIFICATE AUTHORITY" service.
After keystore is created, identify the DN of the certificate.
keytool -list -keystore /etc/nifi/conf/keystore.jks -v
.......
Alias name: nifi-key
Creation date: 2020-10-15
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=hdm2.sb.com, OU=NIFI
Issuer: CN=hdw2.sb.com, OU=NIFI
Serial number: 1752cdd7ffc00000000
Valid from: Thu Oct 15 17:25:15 CEST 2020 until: Sun Oct 15 17:25:15 CEST 2023
.......
DN from this example: CN=hdm2.sb.com, OU=NIFI. This DN is later used in the configuration, it is case sensitive and be sure to preserve all spaces inside the DN, here the space after the comma (,).
NiFi UI requires client certificate authentication. Use the CA-signed certificate or self-signed certificate.
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365
openssl pkcs12 -export -in cert.pem -inkey key.pem -out keystore.p12 -name nifi-key
Values for -t, -p and -c parameters can be read from /etc/nifi/conf/nifi-certificate-authority.json configuration files.
{
........
"keyStore" : "/usr/hdf/current/nifi/conf/nifi-certificate-authority-keystore.jks",
"keyStoreType" : "jks",
"keyStorePassword" : "tQ549j4BoJMnA8wQirWQjJORWVDP5kLJbhsqAtmuakY",
"keyPassword" : "tQ549j4BoJMnA8wQirWQjJORWVDP5kLJbhsqAtmuakY",
"token" : "JfbQ4MwZ83Dgcytw",
"caHostname" : "hdw2.sb.com",
"port" : 10443,
...........
}
/usr/hdf/3.5.1.0-17/nifi-toolkit/bin/tls-toolkit.sh client -D "CN=admin, OU=NIFI" -t JfbQ4MwZ83Dgcytw -p 10443 -c hdw2.sb.com
The tool creates a local config.json file. Particularly, the file contains a password for newly created keystore.jks.
vi config.json
{
....
"keyStorePassword" : "1Y9iEo86Bqcy1HKbd0yACb4L4eL8RnOb/k1Jy2Sy5yU",
"keyPassword" : "1Y9iEo86Bqcy1HKbd0yACb4L4eL8RnOb/k1Jy2Sy5yU",
...
}
The password can be changed to more appropriate if necessary.
keytool -storepasswd -keystore keystore.jks
Transform jks keystore to pkc12 truststore
keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.p12 -deststoretype PKCS12 -srcalias nifi-key
Export certificate
keytool -export -keystore keystore.jks -alias nifi-key -file cert.cer
keytool -list -keystore keystore.p12 -alias nifi-key -v
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=SB, OU=EE, O=DD, L=CC, ST=BB, C=AA
Issuer: CN=SB, OU=EE, O=DD, L=CC, ST=BB, C=AA
DN in this example: CN=SB, OU=EE, O=DD, L=CC, ST=BB, C=AA. This DN is later used for configuration, it is case sensitive and be sure to preserve all spaces after commas (,).
Move client certificate to NiFi node and import into NiFi truststore.
cd /etc/nifi/conf
keytool -import -file /root/cert.pem -alias adminclient -keystore truststore.jks
Any time the truststore is updated, the NiFi service should be restarted.
The client private certificate should be imported into Web browser. The method depends on the browser used. While launching the NiFi UI as secure HTTP, select this certificate for connection.
Ambari->NiFi->Configs->Advanced
Advanced nifi-ambari-ssl-config
| Property | Description | Sample value |
|---|---|---|
| Initial Admin Identity | The admin user, DN of the client certificate, make sure to preserve case and all spaces | CN=SB, OU=EE, O=DD, L=CC, ST=BB, C=AA |
| Enable SSL? | On | |
| Key password | Password of the private key, usually the same as keystore password | |
| Keystore path | Location of keystore | Default: {{nifi_config_dir}}/keystore.jks |
| Keystore password | Keystore password, if keystore does not exist used for keystore creation | secret |
| Truststore path | Location of truststore | Default: {{nifi_config_dir}}/truststore.jks |
| Truststore password | secret | |
| Node Identities | DN of NiFi node certificate, make sure to preserve case and all spaces | CN=hdm2.sb.com, OU=NIFI |
Important: any time Initial Admin Identity or Node Identities is modified, the following procedure should be applied.
- Stop NiFi service
- Remove the following files:
rm /var/lib/nifi/conf/users.xml
rm /var/lib/nifi/conf/authorizations.xml<br
- Restart NiFi service users.xml and *authorizations.xml" files will be recreated reflecting the changes.
Launch NiFi UI on a secure port. The Initial Admin Identity will be the initial administrator, further user management can be maintained through NiFi UI panels.
Important: Although I was able to integrate Ambari and Nifi that way, it didn't seem to make much sense. The user in SSL client authentication is identified through certificate, for example: "CN=SB, OU=EE, O=DD, L=CC, ST=BB, C=AA". In Ambari, the user is identified through user name, for instance, Linux account or AD account and both cannot match each other. Requires further investigation.
Very similar steps like HDFS plugin by tailored for NiFi service
On Nifi node, prepare Ranger plugin keystore. If more than NiFi nodes are used, share the keystore between the nodes.
cd /etc/nifi/conf/
Important: keep in mind CN name, do not use FQDN hostname, the CN name should be different than all other plugins CN names.
keytool -genkey -keyalg RSA -alias rangerNifiAgent -keystore ranger-plugin-keystore.jks -validity 360 -keysize 2048
Enter keystore password:
What is your first and last name?
[Unknown]: rangernifiplugin
What is the name of your organizational unit?
[Unknown]: AA
What is the name of your organization?
[Unknown]: BB
Create Knox Nifi Plugin truststore, import Ranger Admin certificate
keytool -import -file /root/ranger-admin-trust.cer -alias rangeradmintrust -keystore ranger-plugin-truststore.jks
Secure stores
chown nifi:nifi *.jks
chmod 400 *jks
Export Ranger Plugin certificate
keytool -export -keystore ranger-plugin-keystore.jks -alias rangerNifiAgent -file ranger-nifiAgent-trust.cer
On Ranger Admin node, import certificate into Ranger Admin truststore, use a different alias name.
cd /etc/ranger/admin/conf
keytool -import -file /root/ranger-nifiAgent-trust.cer -alias rangerNifiAgentTrust -keystore ranger-admin-truststore.jks
Nifi->Configs->Advanced->Advanced ranger-nifi-policymgr-ssl
| Property | Sample value |
|---|---|
| owner.for.certificate | rangernifiplugin (DN of the certificate) |
| xasecure.policymgr.clientssl.keystore | /etc/nifi/conf/ranger-plugin-keystore.jks |
| xasecure.policymgr.clientssl.keystore.password | secret |
| xasecure.policymgr.clientssl.truststore | /etc/nifi/conf/ranger-plugin-truststore.jks |
| xasecure.policymgr.clientssl.truststore.password | secret |
Restart Nifi and Ranger
Ranger Admin UI-> Access Manager -> <cluster_name>_nifi
Provide DN of Ranger Knox Plugin certificate to Common Name for Certificate field (rangernifiplugin).
Audit-> Plugins
<cluster>_nifi should declare HTTP Response Code 200 with fresh date.