I landed a nice gig writing a cross-platform C++ server to perform outage analysis for a smartgrid software package, using web services to communicate with the other components.  It started out completely cross-platform, and had me at leveraging all the goodies of the latest boost libraries to do a lot of the cross-platform sweatshop work.  But like many other projects, the idealism of the clean-slate design soon gave way to focusing on getting it done, and the project became dependent on Windows-only components.  Bleh.

Unfortunately, this left me using primarily a Windows development environment on my desktop during that push of the project.  Now I’m focusing on a mobile stock-trading app, with the first two components being an Android app and a corresponding cross-platform C++ server.  Bottom line: it’s time to bump up my desktop gentoo installation and get it humming again! (continued…)

Technology in the early 21st century is here to kill us. It stresses us out and shortens our lives and makes us feel like failed Neanderthals every single day of our lives.

We are drowning in data yet no one controls theirs at all. At best it happens to be properly backed up at someone else’s cloud location for a short period of time before that fails or the company goes bankrupt or inevitably tries to extort money from you.

The solution to taking control of your own data must involve massive local redundant backed-up storage (MALREBS). There is no other alternative. And this is a technically impossible task for 99.99% of the human race. In this day and age, everything precious to you is stored either by someone else who really doesn’t give a shit about your data, on ridiculously flaky steel platters that randomly fail while you are sleeping, or solid state drives that wear out from the second you start using them. But here we are so let’s struggle on to get it done.

One required component of MALREBS is raid. This turns those flaky hard drives from 100% chance of total failure to perhaps something less than 90%. This is very very important.

There is only one raid solution to rely on. Linux software raid is the only solution that is both free (in all senses) and reliable, in that you can take your hard drive data with you from a very old (possibly non-existent) system to a brand new system and have a reasonable expectation that the data will still be accessible.

But here’s the rub: mdraid has not received the polish it needs to Just Work. It has serious flaws that after hours of learning, still leave you unsure and hanging and most likely bailing out of the entire process. But it is the best thing we have on the planet, so let’s distill it down to the essentials.

  1. check S.M.A.R.T. data of drives – run tests and make sure they are completely healthy!
  2. clean raid drives of superblock and partition data
    mdadm --misc --zero-superblock /dev/sdd && dd if=/dev/zero of=/dev/sdd bs=1M count=100 && mdadm --examine /dev/sdd
     mdadm: No md superblock detected on /dev/sdd.
  3. use whole drives (not drive partitions) in a newly created raid
    mdadm --create --verbose /dev/md0 --level=mirror --raid-devices=2 /dev/sdd /dev/sde
     mdadm: size set to 3906887488K
     mdadm: automatically enabling write-intent bitmap on large array
     Continue creating array? yes
     mdadm: Defaulting to version 1.2 metadata
     mdadm: array /dev/md0 started.
    watch -n 1 cat /proc/mdstat
    # wait 400 FUCKING MINUTES for a GODDAMNED EMPTY 4TB DRIVE to sync with ANOTHER empty 4TB drive, FUCKSAKE
  4. save and reboot and make sure the mdraid service restores the raid
    mdadm --detail --scan >>/etc/mdadm.conf
     rc-update add mdraid boot
     # start then stop then start the /etc/init.d/mdraid service, make sure this works to restore your raid (check /proc/mdstat)
     # format /dev/md0 as ext4 and set up an auto mount point in /etc/fstab
     # reboot and pray
  5. hopefully much later, upon failure, to restore a single drive, set it up as a raid:
    madadm -A /dev/sdd # I THINK! it's all very iffy. Which SUCKS.

Simultaneously bumping server, desktop, media center, laptop to kernel 3.10.7 et al… (continued…)

The stupid Windoze gaming sirens had me out of linux for a bit on my desktop. Back with a vengeance, and a huge emerge world. Hours of fun! The nice thing is I’ve gotten into gentoo enough that I no longer feel the need to document every massive emerge. For the most part, just keep using these strategies until it works itself out – while keeping your brain engaged of course:

  • emerge -DavuN @world
  • emerge -DavuN –with-bdeps=y @world
  • emerge -DavuN @world –autounmask-write
  • emerge –resume –skipfirst
  • emerge –unmerge (i unmerge the SHIT out of all conflicting non-world packages! keep it clean! :O)
  • emerge -1v (anything that’s missing!)
  • gcc-config (as early as possible)
  • eselect (boost|java-vm|php|python|etc….)
  • use eix and equery to answer any package questions
  • get the latest gentoo-sources and configure that kernel!
  • don’t forget to emerge nvidia-drivers or any other kernel-specific package
  • google for help – gentoo support information is AMAZING
  • dispatch-config
  • emerge -av –depclean
  • revdep-rebuild
  • lafilefixer –justfixit
  • python-updater
  • perl-cleaner –all
  • targeted rebuild, for those nasty upgrades: revdep-rebuild –library libpng14.so.14
  • find broken autotools stuff: find /usr/ -name ‘*.la’ -exec grep png14 -c {} +|grep \:1
  • rinse and repeat!

I had a few new wrinkles…

  • considered emerging gnome-base/gnome (new) to get GNOME 3 but it is soft-masked and who knows how much trouble would follow
  • stupid libpng upgrades always cause everyone all kinds of headaches – see this awesome post
    revdep-rebuild --library '/usr/lib64/libpng14.so.14' -- --keep-going
    emerge -1av --keep-going $(find /usr \( -name "*.la" -o -name "*.pc" -o -name "*-config" -o -name "*.pm" \) -exec grep -H png14 {} \; | cut -d : -f 1 | xargs qfile -CSq | sort | uniq)
  • removed nsplugin USE flag from picasa and acroread since it thought that firefox (not firefox-bin) was a requirement, stooopid things
  • Qt needed a bump, which should happen all at once, which is impossible 🙂
  • of course i had to add a FEW more bells and whistles!
  • emerge -DavuN xfce4-meta xfce4-verve-plugin xfce4-mixer xfce4-taskmanager xfwm4-themes thunar thunar-volman tumbler thunar-archive-plugin google-chrome
  • etc

The Digital Machine hive has moved, after massive disturbance to the nest. To chronicle the story… (continued…)