title: The Magic of ldifdiff.pl: Make one directory like another link: https://onemoretech.wordpress.com/2007/10/12/the-magic-of-ldifdiffpl-make-one-directory-like-another/ author: lembobro description: post_id: 621 created: 2007/10/12 16:28:55 created_gmt: 2007/10/12 16:28:55 comment_status: open post_name: the-magic-of-ldifdiffpl-make-one-directory-like-another status: publish post_type: post

The Magic of ldifdiff.pl: Make one directory like another

The Net::LDAP (or “perl-ldap”) module for Perl is a wonderful thing. Just as wonderful are a couple of utilities from HP’s Kartik Subbarao: ldifsort.pl and ldifdiff.pl, which are found in the “contrib” section of the module distribution (Kartik is the guy who led HP’s initiative to replace all their aging Sun and Novell directories with OpenLDAP, for both internal and external applications).

ldifsort.pl sorts LDIF files generated by ldapsearch and other tools from data in an LDAP directory, while ldifdiff.pl finds the differences between two sorted LDIF files and outputs the LDIF code that will make the target like the source. The resulting LDIF can be piped to a file that can be used by ldapmodify and other tools that are used to modify LDAP entry data. This can be a life-saver when you have to reconcile entry data between two different directories.

Let’s say, for example, you want to user passwords on LDAP directory target.example.com to match those on LDAP directory source.example.com.

Before beginning make sure that your LDAP utilities, including ldifsort.pl and ldifdiff.pl, are in your $PATH and that Net::LDAP has been installed into your Perl environment.

First you need to get the existing data from these two directories. Using the OpenLDAP version of ldapsearch, you’d use the following commands:

ldapsearch -x -LLL -h source.example.com -D "cn=orcladmin" -w secret -b cn=users,dc=example,dc=com" -s sub "objectclass=person" cn userpassword >source.ldif

ldapsearch -x -LLL -h target.example.com -D "cn=orcladmin" -w secret -b "cn=users,dc=example,dc=com" -s sub "objectclass=person" cn userpassword >target.ldif

Some notes on the switches or options used with the command in these examples:

The -x switch is unique to the OpenLDAP version of the LDAP command line tools. It is used to tell the utility to make a simple bind to the directory. Most other vendor’s tools, like those from Sun or Oracle, default to simple bind. The -LLL switch is also an OpenLDAP-specific directive, instructing the tool to output clean LDIF without comments or fancy formatting. Other vendors use different switches for this, do an ldapsearch --help to find out what your distro uses.

The -h is of course for the directory host name, -D for the user entry to bind with (usually fully qualified like “cn=myentry,cn=users,dc=example,dc=com”), with some directories you don’t use a fully qualfied string for the root account like “cn=directory manager” or “cn=orcladmin”. -w is for the cleartext password, -b is for the basedn (here I’ve anchored it to the cn=users container to just return entries in there), -s is the scope of the search (sub, one or base, sub searches down through mutliple levels of the directory tree).

Your search filter comes right after the scope directive, and it must be in quotes. Here I’ve restricted it to all entries with an objectclass of “person”. More complex filters need to be enclosed in quotes and parens, like “(&(objectclass=person)(sn=smith))”.

Next, I list the specific attributes I want to compare. I could grab all the attributes in the entry, but if all I want to do is sync passwords I’ll only request that attribute and one other “anchor” attribute I know is in every entry in case some don’t have a password. Usually I use uid or cn, depending upon which is the “naming attribute” for an entry (if the dn is “cn=myuser1,cn=users,dc=example,dc=com”, the naming attribute will be “cn”, if it’s “uid=myuser1,ou=people,dc=example,dc=com”, then it will be “uid”). Another way I could have done this was test for the presence of userpassword in entries by including it in my filter, like “(&(objectlass=person)(userpassword=*))”. Depending on your directory server this might not work because “userpassword” is generally not indexed for search.

Finally, I pipe the output to a file.

Now that we have our two data files, we need to sort them by dn so that they can be properly compared. To do this we use the ldifsort.pl utility. OpenLDAP’s ldapsearch does have an option for it’s ldapsearch to sort results, but I’ve found that the Perl utility to give more consistent results.

ldifsort.pl -k dn source.ldif >source.sorted

ldifsort.pl -k dn target.ldif >target.sorted

What I’ve done here is direct the utility to sort these two files by entry dn (designating dn after -k for “key attribute”) and piped the result to a file for each directory.

Finally, we run the ldifdiff.pl tool against these two sorted LDIF files, thus:

ldifdiff.pl -k dn source.ldif target.ldif >target.diff

To see all the options available when using ldifdiff.pl, do a perldoc ldifdiff.pl. In this example I’ve simply set it to do the compare using dn as the key attribute and made the file called source.ldif (holding our “source”, or authoritative, directory data) the one that the output will cause the other directory’s data to conform to. Comparisons are done case-sensitively, unless attributes to be treated case-insensitively are listed after a -c switch (for example:
-c "cn,objectclass,uid,sn,givenname,mail", make sure you include the quotes). In this case userpassword is a hash value that we definitely want to do a case-sensitive compare on, so I won’t qualify it.

The resulting code, which I’m piping to the file called “target.diff”, since it will be used on the target to make it like the source, will look something like:

`

dn: cn=myuser1,cn=users,dc=example,dc=com
changetype: modify
replace: userpassword
userpassword:  {SHA}5V4MTmPSh5K1KaTvMQp4Cwmc4BU=
	
dn: cn=myuser5,,cn=users,dc=example,dc=com
changetype: modify
replace: userpassword
userpassword:  {SHA}8Y8FfqRKlFoIOgDm/MEWN9GGBC0=

`

One thing to look out for is the … unexpected. Sometimes, no matter how much you think you know about your data, obvious (and sometimes not so obvious) results come out of this kind of compare operation. Always double check your resulting diff file to make sure the changes it will make are what you really want to happen on the target server. For example, in the real life exercise this example was taken from a couple of entries that were on the source but not on the target showed up as “changetype: add” in the output. Because there were only 2 like this, I just deleted them from the file. Had I left them in they wouldn’t have been created on the target anyway because the entry code lacked some key attributes needed to create an entry, like objectclass. Needless to say, You Have Been Warned ™.

When you run ldapmodify against the target directory using this file, it will modify the entries on the target accordingly.

ldapmodify -x -h target.example.com -D "cn=orcladmin"-w secret -c -f target.diff

I always use the -c switch so that the utility won’t stop on any error it encouters. Using the UNIX script command in conjunction with this kind of operation will allow you to capture the console output to examine any errors that may come up. The -f switch obviously indicates the file name.

Output should look like this:

`

modifying entry "cn=myuser1,cn=users,dc=example,dc=com"
modifying entry "cn=myuser5,cn=users,dc=example,dc=com"

Copyright 2004-2019 Phil Lembo