Rule number one in server-administration: never ever fix something that is not broken.
I spent 8 hours today on fixing one of my two servers, the one that is hosting all my personal sites, because I thought it sounded like a good idea to update a few libraries on my system.
As a matter of fact, my server crashed very badly, and I had to rebuild all the modules with the right options and so on... so that's why it was unavailable for the last 8 hours.
So, instead of producing or updating the podcast, I had to type thousands of Unix-commands and find out by searching on google why it didn't work... to eventually come to a conclusion that my homemade suEXEC wrapper for Apache 1.3 didn't appreciate the glibc update, I managed to isolate the line that was causing suEXEC to segfault, but I couldn't find any bug in that line...
So I decided it would be cleaner to install suPHP that is made for that... but that was to find out two hours later that suPHP wasn't fully supporting Apache 1.3. Yipee. After installing Apache 2.0, reconfiguring it, recompiling PHP and its modules, I finally managed to get this server back online.
I don't even remember why I wanted to update glibc... I guess this had something to do with DomainKeys, postfix and so on... but I'll find that later on.
When I think I wanted to spend this day on music... :(
Update Friday 1st of february: I noticed the sets download script doesn't work anymore, I'll work on fixing this tomorrow, sorry guys!
Update 1st of march: There still are some issues from time to time, the server load rises up for unknown reasons. I am investigating this, and I hope I'll be able to find out what is happening. Sorry for all the bad experience this causes, I'm sure it will soon be fixed and that I would learn a lot of that :)
