Well, it's been quite a while since I did anything to the site, but the archives are coming along nicely (most of the objects are coded and I just have to get the postgres queries written and tested, and we should have online archives). Launched a new site design yesterday and fixed it up some today. It's all DIVs and CSS, which was liberating: not having to worry about tables is a blessing. And it seems to work in Mozilla, IE5 and NS6, and doesn't look that bad in NS4. So there you go.

Hacked together a poppassd with APOP support using Alan Brown's poppassd hack. Installed all of the manpages upon realizing that I had neglected to install a necessary C function call manpages, to which I had reason to refer while hacking aforementioned daemon. I've been busy over the past week getting all of the h.com employee sites moved over as well. Just have 3guns.com left to go, then hesketh.com itself will move. First, though, I'll migrate the [wd] list over to webdesign-l.com.

Installed and configured Qualcomm's popper. Now I just need to find a poppassd client that supports both shadow passwords and APOP. Or write one.

Installed Date::Calc for copious free time.

OK, had a busy day - set up the new webalizer, but haven't configured anything yet. Need to make sure I take the changes into account. Installed GD and LibPNG as well. All this stuff to support hesketh.com personal Web projects, including all of mine (a jaundiced eye, copious free time, and more) -- not to mention the Web Standards Project, which we're hosting now. But more on that front later...

Just a quick test. Configuring ftp.

Launched the site last night, after much wrestling with mod_perl (doesn't seem to want to run as a DSO, maybe we can fix it, maybe not, either way, not that big a deal). Nothing much on the site yet, just the recycled content from the old /lists/webdesign-l/ directory on hesketh.com, which we also launched with a new look and feel and great new content. Headed out to SF tomorrow, so no progress for the next few days (unless I just can't sleep at night).

Looks like we'll be delaying the launch, probably good given how much is left to do. Still, made good progress tonight, have the archiver parsing mbox format into text files and stripping unwanted headers, and I have the code for txt-to-xml, I just need to plug it in. Then it's just a matter of syncing the messages with the db and doing consistency checks.

Date: Wed, 25 Oct 2000 21:51:42 -0400
From: Steven Champeon 
To: thelist@webdesign-l.com
Subject: [WD]: test


OK, here's a quick rundown of what's been done over the past couple of days. Getting habanero into the co-lo went fine, if you ignore the initial heart-stopping, stomach-clenching fear when it didn't recognize the SCSI drives on first boot. Note to self: always check all of your cabling, internal and external, before trying to boot a recently moved computer, even if it was running just fine fifteen minutes ago. We didn't have the right hardware (read: big washers) to install the slide rails in the cabinet, so habanero sits happily at the bottom of a half cabinet, with wasabi humming away on a shelf right above.

Installed NTP to keep everything in sync. Configured BIND as a slave for all of our personal domains, webdesign-l.com, webdesignl.com (for people who can't type) and hesketh.com. Upgraded sendmail to 8.11.1, then recompiled a few times for practice (and until I remembered to add a line for MAP_REGEX to the site.config.m4). Configured sendmail. Installed majordomo. Installed bulk_mailer, which is getting harder to find. I found it here.

Back on non-beta blogger.

Tonight, we take habanero into the co-lo, so I'm running backups.

Installed Damien Conway's Text::Autoformat, on Hao's suggestion. Hopefully, this will be able to deal with all the [wd] members who prefer not to wrap their text.

It's been a busy last few days. I spent most of the last week getting the old mbox-format archives in shape, making up for several versions of Eudora mangling them, and taking copious notes regarding the things I may be expected to find broken in live messages. Utterly grueling work, because not interesting enough to keep my attention, and not so much that writing a one-off script would have helped.

Just testing the blogger beta.

Added support for another idea: favorites. The gist is that you should be able to save your favorite posts on the site for reference and have them there when you go back. This is like bookmarks, but more like the way that MSDN saves your last five queries -- by respecting the context-sensitive nature of the information, it comes to the forefront instead of being hidden in a bookmarks file somewhere. It also allows us to see who likes what posts, on what topics, and by which authors, so we can cull from the favorites to construct a FAQ, either dynamically via database calls, or by hand using the favorites information as a guide.

Spent a couple hours today looking at whether it's going to be possible/feasible to extend majordomo to use DBI for its config and list access stuff. The verdict: I'd have to be nuts. I could write my own mailer in a tenth of the time it would take me to hack DBI support into Majordomo (and probably will, after we launch). I'd say about 90% of the code in Majordomo is there to support its "remote configuration by email" functionality, and it's some of the worst written code I've ever seen (though I have to give it some credit for being Perl4 compatible, that doesn't really matter now.)

Finished up the preliminary database schema rewrite for Postgres tonight. Need to look into whether or not I can constrain an array of integers to a foreign key (using INTEGER[16] REFERENCES email). This is so I can keep multiple email address references in each member record. First array offset should be current active email, others should be either disabled or alternates.

OK, in order to provide the shortest URLs I can think of, I'm using Math::BaseCalc to convert timestamps on posts, adjusted for our epoch (April 1997), to Base 60 strings. The idea is that a very short string consisting of numbers, lowercase and uppercase letters can uniquely stand for any time between 4/1/1997, down to the second. I'm using a suffix to distinguish between any posts that may have been received within a one-second period. So, a post sent right now might have the ID 'AbGor.00', while a post sent sixty seconds after midnight on April 1st would have the ID '10.00'. As far as UNIX is concerned, time will end on '2li3G7.00', which if you adjust for our epoch gives us '1fNHG7.00'. That being the case (trust me) I'd only have to pad out six spaces in the LHS (e.g., '0AbGor.00'). Oh, and it's guaranteed to sort properly by date, using the built-in Perl and Postgres sorting functions. Fun stuff. :)

And what is it with Postgres' identity crisis? You've got Postgres, Postgres95, PostgresSQL; it installs the client app psql, other apps that being with pg_, others still that being with pgsq, all in /usr/local/pgsql. I've having to symlink everything to postgres just to keep things consistent.

Postgres' psql client works a lot better now that I've installed the readline-devel RPMs. (Thanks to Bruce Momjian's online book, available as 389 pages of PDF in only 2.3MB :)

Well, after an annoying series of minor mishaps, I decided to strip Perl off the system entirely (I had 5.00503 in /usr/bin/perl, and 5.6 in /usr/local/bin/perl) and rebuild from scratch with 5.6. So that should solve some of the more annoying aspects of using -MCPAN, at least. Also has problems with some of the CPAN modules (particularly the XML::DOM stuff) so patched perl according to this patch. Had to install the Zlib-devel RPMs as well, for AxKit.

  • Bundle::CPAN
  • Bundle::LWP
  • Bundle::XML - finally gave up on XML::DOM; it doesn't seem to like perl 5.6 (if it seems important, I'll go back to 5.005)
  • Bundle::Apache
  • Bundle::MySQL
  • HTML::Mason
  • PgSQL
  • AxKit

Working on finishing up all the config stuff today; also installing AxKit, Matt Sergeant's excellent XML app server for mod_perl and Apache.

Worked on proftpd config, still not done. Probably a difference between PAM on rh6.2 and rh4.2. Need to look at the PAM docs in /usr/doc/pam*/ It's working now, but I suspect it's only because I'm using the system passwd/shadow/group files (even though I'm using the AuthPAMAuthoritative off directive).

Had some trouble getting CVS to run as pserver out of xinetd, but a quick google search solved the problem.

Note to self: read the CVS FAQ.

Cool patch to CVS to provide chroot().

Hey, this is cool: a list of Ethernet number assignments by manufacturer. Turns out the Tulip is really an Intel Pro/100+. Heh. Lots of other fun stuff here, too.

A good laugh, courtesy of the Postgres INSTALL document:
  5. If you are not upgrading an existing system, skip to step 7.
     If you are running 7.*, skip to step 6.
Heh. And I managed to get it installed regardless.

Other stuff installed: xinetd, Apache, mod_perl (plus a slew of CPAN modules and bundles) and Mason. Upgraded Perl so I now have both 5.005 and 5.6.
Other stuff left to go: MySQL (some people like having two databases. :) and various utilities. Still need to configure the CVS repository. Need to configure xinetd for CVS, too. Need to setup ipchains to keep prying eyes out. Various other config, including proftpd. Need to install majordomo and check it into CVS (I'll be hacking that poor piece of software into little, tiny, specialized bits for use with [wd]) and install bulk_mailer, too. Need to config sendmail and BIND, too. Need to install/configure sudo, and compile dnw/ipw/whois/etc.

Postgres is taking a little longer than I thought it would; it seems that I need to install g++ (actually, egcs-c++) in order to compile it. Darn fancy new programming languages...

Redhat really needs to update their mirrors list. Over half of the sites I tried are no longer mirroring the redhat site. Some of the sites haven't worked in a year (Interpath, for example). The best site is probably Compaq's opensource support site, though it only has the updates directories (as opposed to the full installs). But that's all I usually need, anyway. Heh. It might explain why the other sites are getting hammered and this one isn't :)

Well, that settles that. We may yet put habanero into a cabinet (depending on pricing, etc.) but the box definitely works in a standard rack. It's sitting in the rack in the wiring closet right now, and seems stable enough. Next up: upgrading all the syslogd/klogd packages and any other security fixes needed, installing Postgres, mod_perl, Mason, and getting the actual archives copied over to the machine.

Now that we've got the box itself up, running, configured, and assembled, we are grappling with the rack mounts. It seems they're designed for use with a cabinet (not a regular rack) so we won't be able to throw it into the rack we have here; we'll likely have to wait until we take it over to the co-lo facility at Interlan.

Nice. Booted up the first time. I got an ATAPI incompatibility warning for the Secondary Master, but booted through it. Later, found it was an issue with legacy configuration; the BIOS thought the IDE hard drive was supposed to be a CD-ROM drive. Auto-detect saved the day.

We finally got the rack in today. It's an Enlight EN-8950 and it looks like it'll hold an industrial toaster inside once we've got all the drives installed. We're moving the parts out of a VALinux tower box we got a few months ago, but never got around to finishing. We're running with an ATX Tyan S1832DL Tiger 100 (MP-capable) motherboard (jumper settings), PIII/500, 256MB SDRAM, upgradeable to 1GB, two 9GB Quantum Atlas IV UltraSCSI drives, (and a SymBIOS SCSI host adapter), Intel PRO/100+ ethernet and a 30GB Onstream ADR unit for backups (even though the driver is a bit weird).

I've also decided to throw in an old Quantum Fireball SE 4.3GB IDE drive, for scratch and home directory abuse. (I yanked it out of my iMac when I upgraded to an MP3-compatible 30GB disk, and it's been sitting around unused for a few months. Nobody seems to want it. Funny--I paid $400 for a 2GB four years ago, and thought I'd never fill it. Now I can't even give a 4GB IDE drive away.)

The server is running a specially-tweaked RedHat install, version 6.2, with kernel version 2.2.16. I've got to update the kernel once we get everything else ready.
Beats hell out of the old P166 we are running everything on now. I'll post pictures soon.

OK, I'm going to be keeping track of progress related to the new webdesign-L server (and Web site, complete with archives and so forth) here. I'm still not sure if I'll make this public, even in a limited fashion, just yet. I mostly just want to be able to look back on this stuff when we're live.