April 2005 Archives

DOS boot disks for netware, on IP!

This particular thing has been around for a while, but we hadn't looked at it in a year. Things have changed. There have been IP boot-disks at coolsolutions for a while, so we looked to see if they were more mature.

http://www.veder.com/nwdsk/

Yep. I downloaded this particular one today. It isn't a pre-packaged boot-disk, more of a collection of scripts that generate a boot-disk. You can pick your boot OS, from OpenDOS, FreeDOS, Bart's PE, MS-DOS 6.22, or Win98. It also has a pretty wide selection of NIC drivers as well, which came in handy.

We were looking for a way to image a DL380 with Ghost on an IP-only network. I had to copy the B57.CAB directly to the disk to get the right drivers on, but I was able to run Ghost from our imaging server alright. Very exciting. Lab-imaging was one of our last true IPX dependancies, and now the lab-geeks are going to be trying this disk out. Or so I'm told.

The idea behind the disk is pretty simple. A boot-disk that has memory management built in, and deploys a ramdrive. Then expands the rest of the OS on the ramdrive to finish the boot. This way you can get around the 1.44MB limit of your standard floppy. From the looks of it you might be able to create bootable CD's with this script-set, though I haven't done it yet.

Size queen

| 1 Comment
I'm sure I'm not the only one salivating at the idea of a dual, 64-bit processor workstation. So what if its a mac. It's a dual, 64-bit processor workstation! Off the shelf!

Ahem.

Now that's out of the way. It wouldn't be a good idea for a work desktop, since several of my Novell apps are still solidly in Win32-land. Also, connecting to our very own Novell environment is dodgy at best from a Mac, especially the bits I need to get at in my role as local deity. So no, no G5 for me at work.

Home though... except for that ^!$%^! budget, I'd be tempted.

Novell Auditing

Auditing has come a l-o-n-g way since the old Autitcon days. A long way. They now support dumping the event-log to a wide selection of repositories, including flat-file, JDBC->Oracle, JDBC->MSSql, MySQL, and SMTP for some reason. Lots of repositories.

Can we use this? Probably. Will we use it? Probably not. Logs like that are generally used when a lawsuit hits or we need to trace down precicely what happened. And thumbing through potentially terabytes of data for just those events is not the best use of limited resources.

All this came up with a vendor troubleshoot. This particular package is behaving badly when its files are on the Netware cluster, and the vendor doesn't know why. It is clear we're dealing with MS-trained techs out of their depth in a NetWare world. They've asked us for our "novell logs" for the directory in question, and access logs for the specific time period. Um... don't have it. We could get it in, but give us a month to get the system set up correctly.

Want Gmail?

But somehow managed to not get around to bumming an invite?

http://isnoop.net/gmail/

Go grab your own. Almost a million invites left!

..aaaaand breathe

Juggling high-explosives went well this weekend. The HP techs were able to get the new SAN enclosures in, get everything with updated firmware, and upgraded the management appliance into a non-broken state. No data loss! Though the extreme care exhibited by the techs, and having to rebuild the management appliance, ment we didn't get everything up until 40 minutes after our announced outage window. But it all got done! Yay!

Sadly, due to the time overrun, we were unable to check out the exchange bad-cluster thing.

Discovering disk-space

We found out where the extra space-usage was on the Exchange servers. Turns out the EXCHSRVR directories for the two halves of the exchange cluster are effectively identical.

We also found out that there was something like 35 GB going missing that wasn't turning up in the directory lists, but was showing as used at the disk-level. I hammered that thing, but couldn't dig it up. Until I reran checkdisk and actually paid attention to the drive stats, and noticed that BAD CLUSTERS was up around 32GB of space.

Whaaaaa? This is a SAN. That sort of thing shouldn't happen. We have something like three layers of error correction between the physical platter and the OS. Clearly the bad stuff wasn't in regular data our we'da heard about it by now. Because of how the striping happens, bad clusters like that should have completely shot the SAN to a smouldering glow. But the only signs of space like this is on this one Exchange server. My theory is that these got marked bad through some OS-level mistake, rather than actual bad stuff.

This will get closer inspection when we bring things up after the SAN update this Sunday. Wozers.
..without resorting to PXE. A practical guide.

Since TUT291 at Brainshare was all about installing OES-Linux onto HP blades, those of us who have to do it with a NetWare kernel are somewhat left out in the dark. This is a guide to how I managed to get it to work. The process will give you an image you can use to set up Netware from Deployment Manager.

Needed:
  • ILO with Advanced Pack. HP has cunningly selected the feature set of ILO in the blades such that you can't manage multiple blades without the Advanced Pack, so you probably already have this if you have blades. The Advanced Pack is, of course, an additional license you need to pay. All the vendors are doing this, so this isn't a surprise.
  • A workstation on the same LAN (i.e. no WAN links) as the blade you are installing to.
  • A small, knoppix-based ISO-Linux on CD. I like Insert.
The preparation phase of this is pretty minor, but needed.
  1. Create a pair of .zip (or better yet, .tgz) archives that contain all of the PROD and OS disks. If you have the DVD, this makes it easier. Just copy the whole DVD to the archive.
  2. Have your ISO-Linux on CD.
  3. Have the OS-disk from the NetWare install on CD/DVD.
Now to get going.
  1. Start with a blank blade.
  2. Go into the blade's ILO
  3. Insert your NW65OS disk into your own CD Rom drive
  4. Go to the ILO's Virtual Media tab, and attach your CD-Rom drive to the Blade.
  5. Start up the blade
  6. Go through the first steps of setting up a Netware server, but cancel out when it asks for a PSM to load.
    1. You could theoretically do the entire install from here, but you'd get more grey hair than you'd want. This procedure will save you time if you have to do more than one Netware install.
    2. This gives you a DOS partition to install from, and one that Novell knows works.
  7. Detach the CD-Rom from ILO, and reboot the blade
  8. In DOS, go into FDISK. Create a D: drive sized about 1.5GB
  9. Reboot, format D:
  10. Reboot. Put the ISO-Linux CD in your drive, and attach it by way of ILO. This should cause it to boot from CD. Go into text-mode if you can, it'll save time.
  11. I haven't found a CD distro that'll auto-detect the SmartArray, so you'll have to make the device yourself, and mount it.
    1. cat /proc/partitions
    2. One of the lines should read something like "cciss/c0dop5", which corresponds with a theoretical partition-5 on a SCSI drive. This is the D: drive you created.
    3. mknod /dev/hphome b 104 5
      1. This creates the /dev record needed to mount. 104 is the major number, and 5 is the minor number, both of which are given in the "/proc/partitions" output
    4. mount /dev/hphome /mnt/hd
      1. This mounts the drive where we can use it
  12. Retrieve the OS and PROD archives, and expand them onto /mnt/hd.
    1. example: "unzip -o /ramdisk/nw65prod.zip"
    2. example: "tar -xvvzf /ramdisk/nw65prod.tar.gz"
    3. What this does is copy the contents of the archive to the D: drive. Sneaky, eh?
  13. Once both archives are copied, reboot, detach the CD-Rom, and go into DR-DOS to verify things look right. If everything worked, you should have a D:\ with a complete NetWare 6.5 source directory in it.
  14. If everything looks right, image the server from Deployment Manager and use it to install other NetWare servers.

It could be worse

Remember that.

What is the biggest single mail account you have in your environment not including archives.

At our office we have an accounts lady who has a 11gig yes gig GW mailbox

At our office in the US they have one who is......

Wait for it .......

30 gig

Lance
Pardon me while my brain explodes. At oldjob we didn't let postoffices get that big, much less whole mailboxes. Here we have a fine olde tradition of nazi mail quotas, so the folk around here consider 50meg generous.

Information Security, the magazine

After reading the past few issues, I've come to the realization that I'm not getting much out of it any more. It isn't because it doesn't appeal to me, its largely because they're hawking widgets I have no hope of affording or getting past the privacy hawks. Even the case-studies aren't all that interesting, as they're deploying some spiffy new technology we'll never get, and how they overcame their obstacles. Working in higher ed does have its differences, and one of those is a more hostile IT environment.

In a sense we have a more immediate need for cranking things down, yet ironically we are unable to do just that. Our servers require very stringent patch-schedules since the time between patch-release and exploit release is now measurable in hours these days. Plus we've had at least one compromise that could be attributable to a zero-day (i.e. undisclosed vulnerability, for which no patch exists yet) exploit.

In a sense, our defenses are better than those at a private corp since we don't have the safety-blanket of a firewall to tuck us in at night and allow us to sleep well. Put up a vulnerable version of PHPBB anywhere in our network, and it'll get hacked within a day or two. Because of this, our 'soft interior' is a bit crunchier than your average corp. On the plus side, we haven't had an enterprise-wide worm nail us since I got here.

Yet more code reuse

Back in August I dusted off a script from my old job that performed disk-space monitoring. It has been a while since then, and the trending data that the script is providing has already been useful in managing our storage. I miss Crystal Reports, but Excel can do most of what I want it to do in this case. The pretty pictures are much appreaciated by those who go begging for money.

Yesterday I got the OK to try and set the same thing up for our Exchange servers. Much to my surprise, modifying the windows-section of that script took all of 30 minutes. I had already solved the problem of how to dump to CSV instead of Oracle, and 80% of the problem of how to monitor only specific volumes. Most of the conversion time was taken in figuring out the logic to only grab the volumes I need, but with a Microsoft twist instead of Novell. I had to turn SNMP on on the exchange-cluster nodes, but that was locked down pretty simply. Give me another couple of months and I'll be able to perform trending data on that too!

And we've already learned of a potential issue. Node1 has 25% free, and Node2 has 47% free. Both have nearly identical numbers of users (within 2% variance), but somehow the users on Node1 are using it more. See? This is useful information to have! Especially when combined with the decision to up mail quotas a few months back.

Classroom use of MyWeb

| 1 Comment
I've known for some time that some instructers have been using MyWeb for classroom stuff. But this morning I noticed a pretty sizable spike in traffic to the system and I've trace it to a single page:

http://www.ac.wwu.edu/~newmedia/bonnie/354/354_student.html

This contains a list of links to student directories in MyWeb. This concerns me since I do not consider MyWeb to be a 'reliable' service. It is a 'best-try' service, which may experience unannounced, significant, and prolonged downtime. Since this has the potential of impacting classroom experience, I feel the need to make this known.

Intermapper probe

I managed to get a custom Intermapper probe worked up! This will check the available cache-buffers for a netware server, and set a warning/alarm threshold for it. Nifty! Not much production use quite yet since it is new, but it does seem to work as advertised.

<header>
type = "custom-snmp"
package = "edu.wwu.ts.netware.cb"
probe_name = "snmp.custom.netware.cachememory"
human_name = "Netware cache-memory monitoring"
version = 0.3
address_type = "IP,AT"
port_number = "161"
</header>

<parameters>
"CBCacheWarn" = "128000"
"CBCacheAlert" = "64000"
</parameters>


<snmp-device-variables>
memCacheMemoryFree, 1.3.6.1.4.1.23.2.79.1.3.0, INTEGER, "Available cache buffers"
</snmp-device-variables>

<snmp-device-thresholds>
alarm: ${memCacheMemoryFree} < ${CBCacheAlert} "Cache Buffers critically short"
warning: ${memCacheMemoryFree} < ${CBCacheWarn} "Cache Buffers getting short"
</snmp-device-thresholds>

<snmp-device-display>

\B5\NetWare Memory Thresholds\OP \4\Memory in Cache Buffers:\O\ ${memCacheMemoryFree} 4kb buffers

</snmp-device-display>

Nifty VMWare thingy

The VMWare license from Brainshare showed up. So I installed it. One thing I noticed after poking around is that, unlike MyVPC from MS, they have 'tools' for NetWare. This is really freakin cool. The big problem with NetWare is that it NOOPs the CPU during idle instead of HALTing it. This means that the Virtualization Software's CPU usage is 100% during that time. That ain't good. The 'tools' force it to HALT instead, which is really nifty. If I had two Netware servers running at the same time, the NOOP thing would force both to behave very eratically. I should be able to virtualize clusters with this stuff!

More poking later.

Update lite

It has been a busy couple of day out of work, so I haven't been here all that much. Thus, fewer updates.

In other news, Novell has shipped us field-test code for NDPSM. The new module has some improvements, but we've found that it is capable of blowing the stack. Unfortunately, we haven't isolated where this happens. When it fails, zeros get scribbled all over hither and yon and handily overwrites the area code was when things went south.

Unannounced SP3 changes

The abend.log format has changed! They're now including register dumps and a stack-trace. Hopefully this will mean fewer core-dumps need taking.

The output from the 'cluster resources' command is now sorted alphabeticly. Previously it was sorted by date of addition. A cosmetic change, but a good one for readability.