December 2007 Archives

I talked about this one last year during some benchmarking. I just had a classic case crop up today.

It has long been consensus in the support forums that, (paraphrased) "If you have hyperthreading turned on and get an I/O thread stuck on a logical process, woe be unto you."

I have a server that I've been backing up for a fellow admin in another department. This particular server has 525GB of storage to back up, so it's going to take some time. It has been vexing figuring this one out. Until today, when I finally twigged to the fact that this server has HT turned on. I turn HT off as almost the first thing I do when setting up a server, so I don't think about it when troubleshooting.

Between 1000 and 1215 today, the backup got 882MB of data. Yeah, very crappy.

At 1215 I turned off the logical processors. This is a handy feature NetWare has, and I used it in the article I linked above.

At 1222 when I checked back the backup was up to 4.0GB.

At 1417 it is now up to 71GB backed up.

The only thing that changed was me turning off the logical processors. That's it. At that rate, this server should be backed up in around 15 hours, which is a far cry from the 30+ hours it was doing before.

Turn Hyperthreading off on your NetWare servers. Just do it.

The new eDir 8.8 has introduced some changes into my environment, and from the looks of it some of them were there before I did the upgrade. Specifically to the CA in the tree. In googling around, I found this excerpt from the CA documentation:

With Certificate Server 3.2 and later, in order to completely backup the Certificate Authority, it is necessary to back up the CRL database and the Issued Certificates database. On NetWare, these files are located in the sys:system\certserv directory.

For other platforms, both of these databases are located in the same directory as the eDirectory dib files. The defaults for these locations are as follows:

• Windows: c:\novell\nds\dibfiles

• Linux/AIX/Solaris: /var/opt/novell/edirectory/data/dib

These defaults can be changed at the time that eDirectory is installed.

The files to back up for the CRL database are crl.db, crl.lck, crl.01 and the crl.rfl directory. The files to back up for the Issue Certificates database are cert.db, cert.lck, cert.01, and the cert.rfl directory.

I didn't know about that directory. I also didn't know that the CA is publishing a certificate-revocation-list to sys:apache2\htdocs\crl\. Time to twiddle the backup jobs.

Here in the US we have four National Holidays that are date-locked. The rest are subject to the Monday Holiday Law. The four date-locked ones are:

• Independence Day, July 4th
• Veterans Day, November 11th
• Christmas, December 25th
• New Years Day, January 1st

Whenever such a holiday lands on either a Tuesday or a Thursday, the Monday and Friday respectively mean that as many people as possible take the day off. Such as today! This being a 'family holiday' people like to get Christmas Eve off. And boy have they. I'm the only Technical Services person here today, though I have a stack of cell numbers in case anything goes pear-shaped. ADMCS is pretty light, and I think there is one (1) person at the ADMCS ops helpdesk. ATUS is probably similarly staffed.

An old coworker of mine used to say that she loved days like these. She could get SO MUCH DONE without interruption. I know what she means.

My turn comes next week. I'm taking New Years Eve off.

One of the things that grabbed by attention with 8.8 is 'priority sync'. The documentation has an overview of it:

6.0 Priority Sync

Priority Sync is a new feature in Novell® eDirectory 8.8™ that is complimentary to the current synchronization process in eDirectory. Through Priority Sync, you can synchronize the modified critical data, such as passwords, immediately.

You can sync your critical data through Priority Sync when you cannot wait for normal synchronization. The Priority Sync process is faster than the normal synchronization process. Priority Sync is supported only between two or more eDirectory 8.8 or later servers hosting the same partition.

6.1 Need for Priority Sync

Normal synchronization can take some time, during which the modified data would not be available on other servers. For example, suppose that in your setup you have different applications talking to the directory. You change your password on Server1. With normal synchronization, it is some time before this change is synchronized with Server2. Therefore, a user would still be able to authenticate to the directory through an application talking to Server2, using the old password.

In large deployments, when the critical data of an object is modified, changes need to be synchronized immediately. The Priority Sync process resolves this issue.

Which sounds spiffy. Instant sync of passwords? I'm all for that. Then I remembered, 'wasn't that happening already? That's right, that's the "SYNC_IMMEDIATE" flag in schema.' And that's what's described in this older CoolSolutions article.

Looking at iMonitor I see this:

As 90-95% of our user objects are in either the root container or the students container, those are the statistics I'm interested in. The "maximum ring delta" number is very, very rarely over 30 seconds for these two partitions. With it being intersession right now, we're seeing some higher numbers than usual right now but it is still kept in close sync. As we have 24 hour computer labs, and a simple login causes several user-object attributes to update, we have a continual flow of directory changes. In our case, using Priority Sync would buy us a few seconds at most. We're not under any sort of regulatory mandate to do things 'instantly' so that isn't an issue, and our password-change process is well known to our end users for taking "up to 5 minutes".

Still, I like the idea even if it isn't a good fit for us.

And as far as upgrades go, it was pretty much a non-event.

Whenever you do upgrades like this you always wonder if those balls you're juggling are tennis-balls or grenades. It took about a half hour per server and didn't have any significant hitches. The one problem that did surface is that the OES1-linux server's LDAP server had its certificate change from the one it was using to SSL CertificateDNS. This was not good, as that certificate doesn't have the subject-name we need and this caused some S/LDAP binds to fail due to SSL validation problems. That was an easy fix. The LDAP servers on the NetWare boxes didn't change.

This was a tennis-ball upgrade. So far.

We haven't turned on case-sensitive LDAP binds yet, but soon. Soon.

One unexpected side-effect of getting all three eDir servers upgraded to 8.8 like this, is that the Change Cache is now cleaned of those permanent residents we've had for ages. Woo!

Wow, last post was the 30th? Jeez. I was on vacation all last week, which accounts for some of it. And it's looking like I'll be out sick for at least a pair of days with a crud I got while wandering about. Not sharing that with work, nosir.

On my list of things to do during the winter inter-session is to get eDir 8.8 deployed in the production tree. I just need to have ALL the servers in the tree (all, not just replica holders due to backlink updates) up and talking when I do the first one, and that could take some scheduling. This is the first step to OES2, which will be deployed on the eDir servers first.

As soon as I get some new hardware, since they're getting old.