October 2005 Archives

Poking at some stats for the last week.

Top 5 search-terms for myweb-students:

1. skate
2. skate videos
3. skate video
4. washington state bird
5. steriods in baseball

Top 5 search-terms for myweb-facstaff:

1. mark vanderven
2. wasserman
3. daphnia magna
4. athletics
5. ryan wasserman

And it comes as no surprise that the top three downloaded files on myweb-students are skate videos. At least they're clearly home-shot, rather than ripped from DVD somewhere. No RIAA take-down notices to worry about.

Amazing what someone who has been working with Apache w-a-y longer than me can come up with once he puts his mind to it. Our resident Apache geek, who helped me get the myweb service up, by the way, came up with a way to provide 'myweb-like' service out of the shared volumes if we so wish. The main problem he had was translating his apache-on-solaris knowledge into apache-on-netware, which is different. To get things to work the way we wanted to took all of one line:

AliasMatch ^/deptweb/([^/]*)(/.*)?$ "WUF-FACSHARE/FACSHARE:/$1/deptweb/$2"

And that worked. Unfortunately 'deptweb' was running afoul of the bit of mod-rewrite magic I do to make URLS like http://myweb.yadda.wwu.edu/riedesg into http://myweb.yadda.wwu.edu/~riedesg in the back so mod_edir can pick it up. The magic of which can be found here. The fix for this was to add another RewriteCond/RewriteRule pair for "deptweb". Once that went in, the AliasMatch actually matched and all seems well with the world. We'll see if/when this service gets announced.

It turns out that iPrint depends on uniqueID, same as mod_edir did. Unlike mod_edir, there is no simple switch to flip it to use cn instead. Happily, you CAN edit the search URL, which is what I ended up doing.

This is a little annoying. I had thought that we had been populating uniqueID as part of our account creation automation, but apparently we're not. I'll have to see about getting that fixed.

The decision has been made to get one of us geeks trained up in Linux. We already have 1.75 Solaris guys around, one of whom is already support some SLES. But since it is pretty clear that NetWare will have maybe one more major release before finding software for it becomes hard, we need to get used to a Linux world.

Part of that is looking for training, and that's proving to be harder than expected. I'd like to take the 3037 course, but finding it is proving to be hard. It isn't offered anywhere within 200 miles of here, nor is it offered near the one cheap out-of-state location (near family, so Hotel and Food are not required, saving nearly a third of the total cost if I went elsewhere). The 3038 course is offered locally, and seems to be more common out in the world as well. All things considered, I'd like to get the course out of the way before BrainShare, so I can understand more of what's going on there.

So here I am, trying to get SLES installed into a VMWare session. So far it is going pretty good. I've actually managed to get SP2 installed, and tell the system to check the SP2 package list for updates when installing software.

But I do have to say that the SLES experience is a w-a-y different world than NetWare. It is flagrantly clear that the SLES info on novell.com was written by a different company than the ole Novell we all know. Figuring out how to even install the service-pack required mounting the ISO and checking the README file. NetWare service-packs generally have that info posted on the TID associated with the SP.

I even managed to get a printer installed! Yay Yast! I could have done it command-line but it would have taken about seven times longer as I muddle through MAN-pages. My first CUPS printer. Awwww. Thank goodness that particular printer speaks postscript!

The next step is to create a new partition, add a bunch of users, and play Permission Bingo.

In the classic Microsoft way, I just discovered a scarily powerful Exchange tool. It is called the Exchange Server Information Store Viewer (MDBVU32). I had been lamenting the lack of an equivalent to GWCHECK, but this come kinda close. While it won't do a sanity check on a specific mailbox, you can do things like delete the Calendar folder from a mailbox... something regular tools don't permit. I just had a user with two of each of the root folders come up (side-effect of the Exchange fun last week), and this tool was the only one I could find to fix it.

Interesting stuff.

This week's log dump has shown that we have a user who has placed their "My Music" folder into their myweb folder. Including a rather popular song by Jason Miraz. The RIAA hasn't noticed yet that I've heard of. But we're still dispatching the bat-squad.

It looks like Novell didn't get it all with the revised SP. For a list of what's fixed and what's not, go here. This does complicate the SP install, but not as bad that SP4 was. Still, a heads up. Mostly the 'missed' fixes only impact you if you've already applied some SP3 patches.

And in other news, I've been working up a Wiki entry in the CoolWiki they've set up. This is a list of how to install the SP based on information from Novell and the Support Forums. All the usual disclaimers apply.

http://wiki.novell.com/index.php/Oesnwsp1

A work in progess like all Wiki, and probably missing key details. But it is a start.

I learned something today. From this thread I learned that there is a side-effect to the PCI-X architecture that I wasn't aware of. Apparently the BIOS reserves a portion of RAM for its use. If your RAM is close to or at the 4GB line, you can get memory in Extended Memory even though you are not actually crossing the 4GB barrier. So I go check our servers.

FACSRV1:memory
Total server memory: 3,669,577 Kilobytes
Total Extended Server memory: 458,748 Kilobytes

The "Extended Server memory" line is the key one. Extended server memory is a lot slower than regular "server memory" due to the 32-bit limitation in RAM. To use extended, memory has to be paged from extended, down below the 4GB line, manipulated, then paged back. Hardly efficient. And as I remember from Brainshare, all 32-bit operating systems have this problem. NetWare can support RAM over 4GB... but you'll need a lot of it to get much performance benefit from it. Same for Windows and Linux. Obviously 64-bit operating systems don't have this problem.

It could be worse. Apparently Dell BIOS reserves the top 1GB!

We have 'em. We know.

Short version:

The transaction logs for the two EVS systems attempted to cohabitate, resulting in log-file pollution. Cleaning it is taking a ruddy long time.

You can find it here.

The FCC recently posted an order that is causing concern. You can find it here. While the exact specifics of what it means are still being argued, the consensus here is that we have to provide the feds the capability of performing wire-taps (sniffs) on our data network after the appropriate paperwork has been filed. Not just VOIP, but other forms of data as well. This may come as a, 'well, duh,' thing, but the order specifically states, "facilities-based broadband Internet access providers," need to comply, and we figure we fit that definition even though we have no VOIP (that we know of) yet.

We won't be performing the tapping, just spanning the appropriate switch-ports and allowing them to plug their own equipment into our equipment. Since we are a University of a certain size, police presence in our network infrastructure is not completely unknown historically. But we must provide access.

They've revamped their point system! It seems that AppNotes are a little more confusing. The AppNote page. At the top:

AppNotes include technical information about designing, implementing, administering, and programming for computer systems based on Novell products. Cool Solutions awards Novell rewards points (up to 500 points) for each AppNote published, depending on content.

Which is nice. The rewards are worth real things. Tips used to be worth a T-Shirt in the olden days (I have one). These days, a T-Shirt is worth 75 points, and they have a scale up to 20,000 points for a cruise. No one has gotten the cruise yet. But later on down the AppNote page:

5. Article published, $100 awarded.

The AppNote will be published in HTML and PDF and advertised on the magazine's home page and in the community's "What's New" newsletter, as well as added to the AppNotes by Date and AppNotes by Title pages. An e-mail gift certificate for $100 will be forwarded to the author from either Amazon.com or Globogift.

So actual cash-like substance. Interesting. I wonder which it is?

I will be going! Horay!

Catch this! N65NSS4B has released. This replaces the critical patch N65NSS4A (now withdrawn) that was put in place since pool-rebuilds could cause data-loss. What does this new patch supply?

NSS Issues fixed after the release of N65NSS4A.EXE:
1. Fixed an abend in NSS when doing a file salvage operation. This happens on pools where quotas or user space restrictions are in force and the file has either a data strema (typical for MAC files) or an EA (extended attribute).
2. Fixed a latching issue in NSS if the pool has restrictions during a file purge operating.
3. Fixed traditional emulation for salvage and purge APIs to return a successful status if they are successful. There were some cases where the API completed successfully but the return was not recognized as successful. This was typically seen by customers running Novell Remote Manager and doing a purge.

CIFS Issues fixed after the release of N65NSS4A.EXE:
None.

AFP Issues fixed after the release of N65NSS4A.EXE:
1. Fixed an abend in COMN.NSS when MACs are dealing with files that have #nnn in the name.

The main thing in there is Issue #1, the Salvage issue. If you have a mac and purge something, Bad Things can happen. Since we're a .edu, we have macs. And most directories have quotas on them. Therefore, abends. Bad stuff.

Novell has a TID out detailing various files they, oops!, forgot to put into SP4, or just plain mixed up. Aie. This is why we wait when SPs come out. As I said before, this particular SP just plain looks like a troublesome one.

If our current problems relating to memory management on our DS servers continue much longer, we may end up throwing on SP4 (OES SP1) before I'm really ready for it. SP4 is the first service pack after the main release of OES, so it counts as a first service pack and all that entails. There is already a critical NSS patch that you need to apply after SP4. And the support forums are reporting issues regarding existing iManager 2.5 installs.

One of the new gotchas for SP application these days involves Java. For a while now Java has had to be unloaded before the SP could go in, since most SPs update Java. Unfortunately, Java is Java so it can complain about unloading. And if Java refuses to unload gracefully during SP application, your SP application will freeze. Whee! So the support forums are recommending that you reboot your server before you apply the SP, unload java manually, and then doing the SP.

I just spent more time than I care to think about compiling a staticly-linked kernel for the one Linux server I manage. It's a server that does one and only one thing, so I can afford to crank it down pretty hard. This step should make root-kitting it a little harder.

But it took a l-o-n-g time to compile a kernel that'd work. I thought I could get away with getting a dynamic kernel that showed no modules in 'lsmod', and then flinking the 'use modules' switch. But that just changed everything listed as "m" to "y" in the .config file, and that, as you might expect, didn't work out so good. I ended up with a kernel that was about 4.5megs, and it complained, "Kernel is too big, consider using modules or bzImage". And since modules was out of the question and I was already using bzImage, I had to see what I could whack out.

Round two worked better, but took a lot of tweaking. I took the config file that worked for the modules-none-loaded build, and did a find and replace on "=m" with "=n", then set it to not use loadable modules. It wouldn't compile, since there were dependancies in crypto and a few other areas.

About 15 compiles later I now have a kernel that works. The big problem I had to figure out was why eth0 kept giving me a SIDIOINUSE or something like that. Turned out that a touchpad driver was attempting to load on the IRQ for eth0. Removed the touchpad driver from the .config, and now I have both ethernet cards working. Yay!

Still took too long.

So far, MyWeb has behaved on the student half. But then, it's gone a week or three without breaking before, so we're not out of the woods yet. But there is always hope.