title: Net::LDAPS Fail after POODLE fix link: https://onemoretech.wordpress.com/2014/10/22/netldaps-fail-after-poodle-fix/ author: phil2nc description: post_id: 8710 created: 2014/10/22 13:45:05 created_gmt: 2014/10/22 17:45:05 comment_status: closed post_name: netldaps-fail-after-poodle-fix status: publish post_type: post

Net::LDAPS Fail after POODLE fix

After re-configuring my LDAP servers to only accept TLS connections (the POODLE fix), I found that all my scripts that use Net::LDAPS were throwing a protocol error. The punch line is that there isn't anything wrong with Net::LDAPS or the recent fixes for POODLE. Analysis follows. Let me start by saying, I really love the whole Net::LDAP suite of perl modules. As I've said before, they literally saved my career as a sysadmin. Net::LDAP is what saw me through my first directory services deployment, and continues to help me make hard things easy (and impossible things possible). Yesterday I got a call from a colleague who noticed that a mass user entry update he'd done on our master directory cluster (which he inherited from me) wasn't showing up in an app I administer that is back-ended by an OpenDJ directory. A script on that OpenDJ server server has been reliably importing changes from the master directory every two hours for the last year. I had actually tested it only last week after making a minor change to accommodate additional attributes, so I knew it was working. Except it wasn't. When I went to check the logs I found that the last successful import had been on October 16. Yes, that's right: the day I had implemented the fix for POODLE on OpenDJ. It turns out that my script was using Net::LDAPS to establish an LDAP over SSL connection to my server, and that connection was failing with a protocol error. I was able to replicate this using a test script, which gave the same error:

IO::Socket::SSL: connect: Connection refused at ./sslconnect.pl line 14,  line 522.

The machine the script was running on was the directory server itself, a Red Hat Enterprise 6.5 system with perl 5.10.1 and IO::Socket::SSL 1.31, which I thought was fully updated to the latest versions of all packages. Further testing showed that this error didn't occur when the same script was run on my Fedora 20 workstation, which had perl 5.18.2 and IO::Socket::SSL 1.955 (all updated to the latest rev). But those differences were just a distraction, it turned out. The real problem was that openssl on my Red Hat Enterprise servers was at least one revision out of date. Without updating any of my perl modules I was able to get things working again by updating openssl from 1.0.1e-16.el6 to 1.0.1e-30.el6 (on Fedora 20 the latest package is openssl-1.0.1e-30.fc20). Sad to say, I didn't come to this conclusion through clever diagnostics. Instead I succumbed to the instinctive, but obviously misplaced, logic that there must have been something wrong with my perl setup and updated all the related modules on my dev box first. My "control" in this case was my production machine. After seeing the openssl update make the difference on dev, I applied only that update to prod -- scientifically proving it was sufficient to fix the problem. IO::Socket::SSL, along with dependencies like Net::SSLeay, leverage the system openssl libraries rather than implementing SSL functions in pure perl the way modules like Net::SSH::Perl did (compiling Math::Pari for Net::SSH::Perl was long a right of passage for novice sysadmins -- even more "fun" on Solaris than it later turned out to be on Linux). As a result these modules, and modules that depend on them (Net::LDAP, Net::LDAPS, etc.), can be sensitive to changes in those libraries. This made a difference because there have been some dramatic changes in both the openssl libraries and our handling of the underlying SSL protocols, over the last year. That has created a perfect storm of variations which were bound to result in some application breakage. But this is a good thing, because it means we're finally at a point where we're willing to power through our natural laziness (and fear) for the sake of better security. At least I am.

Copyright 2004-2019 Phil Lembo