For the sake of learning from past mistakes, a quick note about my recent server troubles…
I am masochistic enough to run my own web and email server on my residential broadband connection. It is also my firewall and router – yes I run all this on a shoestring. I’ve kept it running for almost a decade now, and I learned early on that splurging on NIC’s and a good power supply is a good idea. I use two nice Netgear NIC’s in it, and they get constant abuse. They’ve held up for years, but one of them often does not power up on reboot until I pull it and stuff it back in and cross my fingers. But on Monday night, my server started flaking out, with BIOS not posting at all on reboot. I had to play around, pulling memory, removing extra hard drives, unplugging fans, etc., and finally got it to come back to life. But on Tuesday morning I awoke to a dead NIC in my server.
I played around more, swapping out 3 different NIC’s, and testing with 5 different OS’es and live CD’s. But no matter what, my server would not get a response from Roadrunner’s DNS server. Everything worked fine if I plugging in another machine, regardless of whether it was running linux or Windows. Eventually, using a brand new NIC and fully powering down the cable modem and PC got me a DNS response (and a new IP, *sigh*). Remember, you have to actually physically unplug the PC from the wall to power down the NIC!
So now I am using a crappy new $10 100Mbit TP-LINK NIC for WAN traffic and the one good Netgear NIC for LAN. To get them set up properly was a pita, networks don’t like IP-MAC address changes, and I had to change the good Netgear NIC from eth1 to eth0. To get it working took a few steps. I had to bake support for the TP-LINK NIC into the kernel, it uses a Realtek 8139 C+ chipset and that requires the 8139too module. Then I researched how MAC addresses get assigned to eth# network slots, and was shocked to learn that traditionally, the kernel assigns them based on the order in which it loads modules. What if you have two of the same cards?! I wondered how many elite geeks bought different brands of NIC just so they could control the assignments! As in many situations, udev to the rescue. I already use udev, and I found that it was already using a rule for the MAC<->eth# associations, in [/etc/udev/rules.d/70-persistent-net.rules]. Totally sweet. I swapped things around so that the new NIC was eth0, and the previously-eth1 NIC was eth0. Total win, yay. As the last steps, I had to rerun my firewall script and resave my iptables rules with [/etc/init.d/iptables save], and… like every geek struggling to run a server on crappy Roadrunner residential service… I had to repropagate my DNS records with my new IP. Thanks 1and1.com for making me click a thousand times to do it. In 1and1’s defense, they said “next time just call customer service and we’ll do the clicking”, haha cool!
If you’re reading this, then everything worked out! :>
[…] working by the end of Tuesday – it was a long day – read about the details on my tech blog. Makes me wonder how much of an idiot I looked like as people got error messages when they went to […]