I installed it on case in ~/development/simon:
sudo apt-get install simon build-essential libx11-dev libasound2-dev
Then there's some bs thing you have to get behind a EULA, see development/simon/HTK. I stopped short of builing it. Some instructions.
And it has this system to download plugins, but the Model downloads seem to be stuffed with what... spam?? Hard to find a real model pack.
I did manual training but never got a word recognized afterwards. It seemed to be listening... it threw errors about missing SphinxTrainer... not sure.