dn: cn=schema
changetype: modify
add: attributeTypes
attributeTypes: ( classId-oid
  NAME 'classId'
  DESC 'CMS defined attribute'
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.15
  X-ORIGIN 'user defined' )

dn: cn=schema
changetype: modify
add: attributeTypes
attributeTypes: ( certProfileConfig-oid
  NAME 'certProfileConfig'
  DESC 'CMS defined attribute'
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.40
  X-ORIGIN 'user defined' )

dn: cn=schema
changetype: modify
add: objectClasses
objectClasses: ( certProfile-oid
  NAME 'certProfile'
  DESC 'CMS defined class'
  SUP top
  STRUCTURAL MUST cn MAY ( classId $ certProfileConfig )
  X-ORIGIN 'user defined' )

dn: ou=certificateProfiles,ou=ca,{rootSuffix}
objectClass: top
objectClass: organizationalUnit
ou: certificateProfiles

dn: cn=<certProfileId>,ou=certificateProfiles,ou=ca,{rootSuffix}
objectClass: top
objectClass: certProfile
cn: <certProfileId>
classId: <classId>
certProfileConfig: <octet string>

Currently, profiles are read at startup. This means that we need some mechanism to trigger the refreshing of the profiles (without restart) when changes made on other clones are replicated to the local database.

A persistent LDAP search will be executed in its own thread to monitor for changes that occur beneath the certificateProfiles OU and update the ProfileSubsystem accordingly. The entire implementation is contained within the ProfileSubsystem class. Specifically, the presistent query is executed by the run() method of the Monitor inner class, which extends Thread.

The pki ca-profile-edit <profile-id> command will be added. With due consideration for authentication and authorisation, the behaviour of this command will be:

1. Retrieve the current profile content (in the existing key-value format used for file-based profiles, rather than LDIF, JSON or other.)

2. Save the content to a temporary file.

3. Invoke an editor on the file. Respect the <code>EDITOR</code> environment if set, otherwise invoke vi(1). The user makes changes, saves the file and quits the editor.

4. If changes were made to the profile, store the updated profile in the database (the change will be automatically replicated to clones). If no changes were made, report that no changes to the profile were detected.

5. Remove the temporary file.

Other useful operations that could be implement as subcommands of pki ca-profile include:

• Showing a diff between a profile and the system/default version of that profile (if it exists).

• Creating a copy of a profile, under a different name (most likely for subsequent editing.)

Updates to profiles via the CLI tool shall not require a restart of the pki-tomcatd service.

Existing access controls shall remain. That is:

• Update of an existing profile - agent disables the profile; admin then is allowed to update; agent reviews the profile and enables it.

• Adding a new profile - admin creates the profile; agent approves it.

One of the two initially-proposed solutions was a hybrid LDAP/files solution, where system profiles continued to be stored on the filesystem, but modifications could be stored in LDAP, and all custom profiles would be stored in LDAP:

• Profile creation will store the new profile in LDAP, so that it will be replicated.

• Modification of a file-based profile will result in the modified profile being stored in LDAP, so that it will be replicated. Consequently, the LDAP profile storage must take precedence over file-based profile storage in the profile lookup process.

• Because LDAP and file-based versions of a single profile may now exist at the same time (the LDAP version being the active version), the behaviour of the delete profile operation needs to be clarified. Because System profiles proposes using the shared system profiles (which an instance will not be able to delete), I propose that Dogtag prohibit the deletion of profiles that have a file-based version (whether or not there is also an LDAP version).

• If there is a use case for restoring a profile to the default version distributed or installed by Dogtag (where it exists), a new restore profile operation can be implemented. This operation would remove the (modified) profile from the LDAP directory. The file-based version will then become the active version. Attempting to restore a profile that exists only in LDAP would be an error.

The main motivation for this proposed solution was to simplify application of updates to default profiles:

• When upgrading to LDAP-based profiles, upgrade scripts must detect added or modified profiles and move these into the LDAP profile storage. Added profiles will then be removed from the CA subsystem profiles directory, and modified profiles will be restored to a pristine state, which will ensure:

  • updates to default profiles can always be written to the corresponding file-based profiles without conflict;

  • a smooth changeover to a System profiles directory will be possible, if this proposal is implemented.

alee had reservations:

• I understand why you have profiles in both LDAP and file format. However, I think this makes things complicated. My preference would be to have all new systems maintain their profiles solely in LDAP, rather than some admixture.

• There is a precedent for moving data that was formerly in files to LDAP - and that was the data in the security domain. Originally, this data was in files. At some point, we changed the servlets that update the security domain to use LDAP instead, and used a parameter in CS.cfg to determine whether the data was in LDAP or files.

edewata proposed a variation where only custom profiles would be stored in LDAP, and default profiles would continue to be managed on the filesystem, as they currently are.

• I think all system/default profiles should remain file-based and all custom profiles should be LDAP-based. It will make a clean separation: system profiles are owned by us (Dogtag developers), custom profiles are owned by the admin.

• The system profiles will be read-only. This way we will be able to update the system profiles without writing any upgrade scripts because the files will be updated automatically by RPM. Just one requirement, all server instances must be upgraded to the same version.

• If the admin wants to change a system profile, they can clone it into a custom profile and make the changes there. The custom profiles cannot have the same names as the system profiles, so there’s won’t be any conflict/confusion, and no need to support a "restore" command. In general we won’t need to write upgrade scripts for custom profiles except if we change the LDAP schema.

One significant point in favour of edewata's variation is that administrators can continue to manage profiles in the way they are used to, i.e. editing them directly. The pki ca-profile-edit CLI is deemed to be a sufficient mitigation.

Due to the rejection of automatic updates to default profiles (see below), which was the primary motivation for the files/LDAP hybrid solution, and in consideration of the increased complexity, the hybrid solution was rejected.

The original proposal for LDAP-only profiles was to automatically effect behavioural changes to default profiles as part of the upgrade process:

• There is currently a 10.3 ticket to create a database upgrade framework. Once this framework is in place, it can be used to perform a migration from files to LDAP, as well as modify default profiles when the default profile is being used.

This was rejected, although tools will still be provided for an administrator to perform the update at their discretion. alee explains:

• There is another problem, and that is that it is not clear that we want updates to the default profiles to be propagated to existing instances. I have looked at the profiles and there have been only a handful of changes over the last 7 years. Those changes include things like updating the default signing algorithms or the default validity. More likely than not, admins would prefer that we not change the behavior of profiles in existing instances underneath them.

• The changes that I have found are all behavioral - and therefore things that admin can opt out of — or would prefer to do on their own schedule. There have been no structural changes.

• If there are structural changes, then we need to (and can) provide an upgrade script which would run with the automatic upgrade. An example of this would be a schema upgrade as we sort out how to represent profiles in LDAP.

edewata proposed a fine-grained storage of profile data, instead of simply storing the current profile data as a single bytestring (in the same way that all the profile data is currently stored in a single file):

• I suppose we want to have something that resembles the actual Profile data structure (see ProfileData Java class). There should be an LDAP attribute for each single-valued Java attribute (e.g. name, description, enabled, visible). This way the profile is more manageable and can be queried based on these attributes. For collection attributes (e.g. inputs, outputs, policySets) we can use child LDAP entries to represent them.

• About the REST interface & CLI, since this will be the primary way to edit profiles, we might want to have more granular commands to modify parts of the profile. Right now with ca-profile-mod command you need to send the entire profile in a file. It would be nice to be able to specify some parameters to change certain attributes only, or use separate commands to manage the inputs/outputs.

• We’ll also need an interface to find existing cert records that use a certain profile and bulk modify them to use a different profile. This will be useful when you create a clone to change the system profile.

There are obvious benefits to this proposal but it is more work (the existing machinery for reading and modifying file-based profiles would no longer be useful for LDAP profiles), and not necessary to maintain the current behaviour and meet the basic goals concerning replication. It is therefore deferred.

edewata proposed a mechanism whereby profiles can inherit from other profiles:

• Basically each LDAP profile will have an optional parent. The parent can be the file-based system/default profile, or another LDAP profile. A sub-profile will inherit all attributes, except when it’s explicitly declared in the sub-profile. This mechanism allows us to create just a proxy/alias, a full clone, or anything in between. For example, a proxy profile might only have a few attributes:

This proposal was deemed to be out of scope with respect to current requirements but fundamentally compatible with this proposal, and was therefore deferred.

$ grep subsystem.1.class /var/lib/pki/pki-tomcat/ca/conf/CS.cfg

subsystem.1.class=com.netscape.cmscore.profile.LDAPProfileSubsystem

$ ldapsearch -D "cn=Directory Manager" -w $DM_PASSWORD \
  -b "ou=certificateProfiles,ou=ca,$BASEDN" -s one cn

# extended LDIF
#
# LDAPv3
# base <ou=certificateProfiles,ou=ca,o=ipaca> with scope oneLevel
# filter: (objectclass=*)
# requesting: cn
#

# search result
search: 2
result: 32 No such object
matchedDN: ou=ca,o=ipaca

# numResponses: 1

# extended LDIF
#
# LDAPv3
# base <ou=certificateProfiles,ou=ca,o=ipaca> with scope oneLevel
# filter: (objectclass=*)
# requesting: cn
#

# search result
search: 2
result: 0 Success

# numResponses: 1

subsystem.1.class=com.netscape.cmscore.profile.LDAPProfileSubsystem

subsystem.1.class=com.netscape.cmscore.profile.ProfileSubsystem