polling LDAP for changes

June 9, 2005 7:56 PM

LDAPFilter: I'm writing an application that stores its configuration in an LDAP server. I'd like to have it poll the server and reload itself when the configuration changes. What's the best way to do this?

My product will run on eDirectory, but I'd like to make it as compatible with other servers (primarily OpenLDAP) as possible. If I were just worried about eDirectory then I'd simply use the Revision attribute because it is incremented each time there is a change. Then I noticed that both OpenLDAP and eDirectory include the operational attribute modifyTimestamp which seems like exactly what I needed... at first glance.

The problem is with eDirectory, partitions, and replicas. The server I connect to will only hold a sub-reference of a partition with multiple read-write replicas. Thus, if I modify an object and then immediately read the modifyTimestamp back then I'm not guaranteed that I'll hit the same replica for both operations (and I might get the old timestamp instead of the new one).

Couple solutions I've come up with: (1) just fudge the value in my software by 30 seconds, but then I'm relying on the fact that my machine has a synced time with the tree. (2) Sleep in my software for 30 seconds and then read the value out, but I hate to add extra delay, especially since it may be perceptable to the user. (3) Loop over a search until I get the updated timestamp. (4) Force the administrator to point my software at a single LDAP server holding a read-write replica, but then I lose redundancy.

I just wish OpenLDAP had an equivalent to the Counter syntax. Any developers out there have a good solution to this? Or should I just hack in one of 1-4 above? I want to write robust software, but I'm torn on what to do.

posted by sbutler to computers & internet (4 comments total)

Perhaps you don't need to detect changes at all. Maybe just read all the configuration data, whether it's changed or not, and set it up to execute.

Another possibility is to use some sort of hash function over the subsystems to figure out if something's changed. This again might be expensive because you have to read all the configuration data back.

Googling around reveals OpenLDAP does have a system for returning the modication date of an entry but it's probably not a standard shared by eDirectory. In this case you may just have to write code to handle both servers specifically. It's not clear that the standard LDAP protocol defines a mechanism for querying metadata like entry modification or creation date.

(BTW, this seems like a bad idea to me. I'd hate the idea of a production system just up and reconfiguring itself whenever it wanted. Mayhaps consider an option to turn it off.)
posted by nixerman at 8:07 PM on June 9, 2005

It seems like there's only one case where this matters -- when the application that made the change quits almost immediately and is then restarted. The configuration it reads back from the LDAP server won't necessarily be the last configuration it saved due to the redundancy features.

In all other cases, there's either time for the config changes to propagate (and other running copies of the application will reload when they notice the change), or the changes will (presumably) be held in memory for the running application (it would be wasteful to have to check the LDAP server for every config attribute every time, IMHO, when you can cache locally).

So, what to do about the quick quit-restart (hopefully not crash-restart) case? You could perhaps store a config version counter in local temporary permanent storage (/tmp for a unix-like system, c:\temp for a windows, etc.) and check it against the config version returned by the LDAP server queried. If they're different, inform the user and pause for 30 seconds before checking again.

Or, you could cache the entire configuration locally on disk, though I suspect you don't want to do that at all, hence the LDAP server.

On preview, nixerman's comment on the production application reconfiguring it underneath you also rings true for me. Of course, it depends on the application and what effects such a reconfiguration may have on it, but even a popup "config has changed -- reload?" every 30 seconds if someone else was changing things hither and thither would shit me to tears in a very short time indeed.
posted by 5MeoCMP at 8:30 PM on June 9, 2005

Are you using JNDI to connect to the LDAP server? I'm guessing not, but JNDI provides an interface for event notification, which totally rocks. There should be a similar interface at the product level, but I'm not familiar with the LDAP products you're using.
posted by knave at 10:22 PM on June 9, 2005

(BTW, this seems like a bad idea to me. I'd hate the idea of a production system just up and reconfiguring itself whenever it wanted. Mayhaps consider an option to turn it off.)

You know, I've already coded a way to allow remote reloads. I think I'll just force the admin to do a reload if they want to accept the new configuration changes.

It's weird: sometimes I get so stuck in a particular mode that I forget about another option that's available. Thanks!

Are you using JNDI to connect to the LDAP server? I'm guessing not, but JNDI provides an interface for event notification, which totally rocks. There should be a similar interface at the product level, but I'm not familiar with the LDAP products you're using.

Nope. Used JNDI + NDAP for my last project and loved it. No event notifications, but I still had a lot of fun. This time though I'm in Perl and using Net::LDAP.
posted by sbutler at 7:05 AM on June 10, 2005