| flash drives
|
- A [Cara] 32GB Windows 10 multi-version boot disk (25GB free); dinosaur+...; she wants argo
- B RetroPi! only works on Pi2 atm
- C Ubuntu 15.10 desktop boot
- E blank
- F blank
- G blue/white 32GB - GoPro!
- H blank
- I Crucial M500 firmware flasher
- J Jackfruit Pi
- K kodi (openelec) (unlabeled, in bedroom Pi driving Seiki)
- L Lime Pi (was OS X Lion boot)
- red-on-black 1GB old msdos
- blue-on-black 4GB old photos
Non-MicroSSD...
- D 2GB msdos found stuff
- P ABB 125MB empty
- Q OpenELEC boot 9/2015
- R 2GB homecoming 2014 playlist
- S toshiba msdos
- T tinywave 32GB lots of backups, star wars uncut
- U 8GB Ubuntu 15.10 Server 64-bit installer
- X 4GB empty
- Y 2GB old photos
Old/gone...
- (returned) mellow mushroom GoT wolf 4GB need to add a 720p disney film!!
- W BOOM, no more, was Raspberry Pi 2 (fully dedicated partitions)
- PNY 16GB from bill, get it back to him!!
|
| master keyboard shortcuts map
|
| See .i3/config. Typically you can press ctrl-shift-K to open it in editor.
|
| Ubuntu repo management
|
To see what repos Ubuntu is currently using:
cat /etc/apt/sources.list
|
| Ubuntu upgrade from command line
|
To update ubuntu from an i3 terminal:
sudo apt-get update # Fetches the list of available updates
sudo apt-get upgrade # Strictly upgrades the current package
sudo apt-get dist-upgrade # Installs updates (new ones)
|
| script favs
|
| COMMANDS WORTH REMEMBERING
My scripts:
sc show most-common quick-scripts
greplogs blah search all my log files (named ToDo.txt) for "blah"
greplogs blah -C 3 search all my log files (named ToDo.txt), and show 3 surrounding lines of context
grepcode blah -C 3 same, but search code files underneath the current dir (h|hpp|cpp|pl|php)
restart_process_as_needed.pl name=#path/file#
checks for process named 'file'; if not found, starts path/file
pk substring prompt to kill all processes with 'substring'
General-purpose:
cp -axv drive drive_backup clone an unmounted linux system
ps ax|grep blah show running processes with "blah" in them
lsof -c cp show open files for any process starting with "cp"
lspci list all the hardware using the PCI bus (video cards, etc)
watch #command# repeat the command every 2 secs (default), showing output
tail -f #logfile# display the tail end of the logfile, in realtime
shutdown -rF now reboot, forcing an fsck on startup (or, put a [forcefsck] file in drive root)
fuser -m /dev/hdb1 find open files on a drive, then processes that have the file(s) open
/dev/hdb1: 538
ps auxw|grep 538
donncha 538 0.4 2.7 219212 56792 ? SLl Feb11 11:25 rhythmbox
wc -l file count lines - can pipe to it too
find / -name "libjpeg*" brute-force find (in this case, all libraries starting with jpeg...)
iostat -dx 5 report on drive % utilization etc ("emerge systat" to get it)
iotop top for io, of course
hdparm -t /dev/sda tests actual speed of hard drive
Ubuntu-specific:
xrandr --output DVI-I-2 --off && xrandr --output DVI-I-2 --auto --right-of DVI-I-1 fixes lost monitor, also try arandr
sudo software-center [XXX.deb] to go to software center; specify a deb file to install (steam etc).
sudo unity-control-center all settings; also a pathway to upgrading and selecting nvidia driver
sudo lsb_release -a display the ubuntu version
Gentoo-specific (see blog for USE flag notes):
elogviewer X app to list installed packages including installation notes
genlop -lu show the history of merged and unmerged packages
eix #search# search for all packages (available AND installed)
eix --world -c condensed list of all packages in your [world]
emerge -Davu blah emerge package blah (if needed), confirming first
dispatch-conf handles updating of config files after emerge, see /etc/dispatch-conf.conf
equery uses #package# show USE flags of package
equery depends #pkg# show packages that depend on package
equery belongs #file# show the package to which the file belongs
equery hasuse #flag# shows packages that have the USE flag
equery depgraph --depth=# #pkg# shows the entire dependency tree for pkg
eix-sync grabs latest package list, updates portage and eix (usually called via cron)
emerge --emptytree --update world CAREFUL, this reinstalls EVERYTHING :>
Apps:
alsamixer set 7.1-channel HD audio volume levels
alsa-config test, set optimal audio levels
mythfilldatabase fill TV listings database
For development:
git rebase --hard HEAD throw away your local changes that have not yet been committed
git rebase --hard master force the current branch to match master (losing ALL commits!)
cvs history -c [file] get list of revisions for [file]
cvs diff -r 1.2 [file] compare past revision to current
cvs diff -r 1.3 -r 1.2 [file] compare two revisions
Run sc to get the latest list:
-----------------------------------------------
/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\
-----------------------------------------------
useful commands and scripts
_______________________________________________
\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/^\v/
---------------------------------------------
i3[scriptname] run an i3-specific script
equery files pkg show package files, use |less
elogv[iewer] browse recent emerges and comments
edit_fluxbox_menu edit and apply fb menu changes
edit_firewall edit, restart iptables w/screen
mdb-[] access db tools
gtdm;gw;gd;gwa go to tdm|wimpy|dune|wally via ssh
greplogs [term] search logs for term [-C 3]
grepcode [term] search code in curr dir for term
ds [dir] get directory sizes
pk [proc_regex] kill matches - type ALL for -9
lsof -c [proc] show open files for process
which|whereis app show app location|paths
alsamixer adjust sound levels
eq finally an eq in linux, yay!
watch [command] repeatedly run command
rs restore screen (w/detach)
st;gt search/get torrents
post_torrent run from m@wimpy:rrip/flac (or mp3)
srd [#] show most recent downloads
move_and_resymlink symlink destdir
resymlink symlink destdir (if file is already there)
rm_torrent.pl torrent cleanup; run for usage
extract_tordir extract dir from .torrent
ls_broken_symlinks uses curr dir
ls_nonsymlinks uses curr dir
rm_broken_symlinks uses curr dir, WILL rm THEM
td;etd show/edit active todo w/cvs
top top resource-consuming procs
apachetop -f log site-specific apache requests
ex;em emacs with/without X
uc;cm cvs update/commit (no message)
sp;ep search/edit private data
edit_spam_[rules|settings] what it says
edit_sieve_rules edit mailfilter rules
tree -fid -L 1 great tree display of dir & files
edit_torrent_[] shows (common) or rss (rare)
search_xbmc_keys search key file for term
edit_world edit portage master pkg list
xscreensaver-demo then select Disable to turn off
recordmydesktop does what it says
bandwidth monitor current bw usage
edit_rsync_jobs edit bp<->tdm nightly rsync
trac-tda $1 $2 trac admin tool, RUN AS ROOT
erase_trac_ticket_block 17 20 blah (run as root)
update_ampache_tda update music catalog, RUN AS ROOT
edit_gentoo_grub_cfg set up gentoo kernels on 3-boot
htdj/*_scripts/push_release(.bat)
get_ampache_official changes
publish_hangthedj_ampache_module
get_ampache_official changes
cdh;huc;hcm;hpush scripts to manage hangthedj repos
also see scripts/git_* helpers
ec2-ubuntu-ssh ssh to amazon ec2 server
mp3search search (and optionally copy mp3s
|
| Disabling xscreensaver
|
emacs ~/.xscreensaver
timeout: 12:00:00 # the max timeout is 12 hours
crontab -e
# So... I have .xscreensaver set to max delay (12 hours)
# and I call this every ten hours to restart the idle timer.
* */10 * * * /usr/bin/xscreensaver-command --deactivate >/dev/null
|
| linux software raid [mdraid]
|
mdraid has not received the polish it needs to Just Work. It has serious flaws that after hours of learning, still leave you unsure and hanging and most likely bailing out of the entire process. But it is the best thing we have on the planet, so let’s distill it down to the essentials.
- check S.M.A.R.T. data of drives – run tests and make sure they are completely healthy!
- clean raid drives of superblock and partition data
mdadm --misc --zero-superblock /dev/sdd && dd if=/dev/zero of=/dev/sdd bs=1M count=100 && mdadm --examine /dev/sdd
mdadm: No md superblock detected on /dev/sdd.
- use whole drives (not drive partitions) in a newly created raid
mdadm --create --verbose /dev/md0 --level=mirror --raid-devices=2 /dev/sdd /dev/sde
mdadm: size set to 3906887488K
mdadm: automatically enabling write-intent bitmap on large array
Continue creating array? yes
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
watch -n 1 cat /proc/mdstat
# wait 400 FUCKING MINUTES for a GODDAMNED EMPTY 4TB DRIVE to sync with ANOTHER empty 4TB drive, FUCKSAKE
# IF YOU REBOOT BEFORE THAT, IT WILL BE AS IF YOU NEVER SET UP A RAID
- save and reboot and make sure the mdraid service restores the raid
mdadm --detail --scan >>/etc/mdadm.conf
rc-update add mdraid boot
# start then stop then start the /etc/init.d/mdraid service, make sure this works to restore your raid (check /proc/mdstat)
# format /dev/md0 as ext4 and set up an auto mount point in /etc/fstab
# reboot and pray
- hopefully much later, upon failure, to restore a single drive, set it up as a raid:
madadm -A /dev/sdd # I THINK! it's all very iffy. Which SUCKS.
|
| Test your drives - DO THIS OFTEN
|
Look at available tests:
smartctl -c /dev/sdf # to peek at available tests and how long they typically take to run
Short test:
smartctl -t short /dev/sdf
watch -n 3 smartctl -l selftest /dev/sdf # you won't see anything until it finishes, takes about 1 minute
Long test:
smartctl -t long /dev/sdf
watch -n 3 smartctl -l selftest /dev/sdf # takes about 5 minutes
|
| Adding a new drive (with testing)
|
To add a new drive:
- ls /dev/sd* # and make note of what you have
- stuff the new drive in the SATA tower or plug it into a SATA socket
- ls /dev/sd* # you should now see something new, like [/dev/sdf]
- or do: cat /var/log/messages|grep sd # to see what it used
- fdisk looks good again...
fdisk /dev/sdg
p # to show what is there already, hopefully nothing
(d if needed to delete existing partitions)
n # new
1 # partition 1
<enter> # for default of 2048
<enter> # for default, end of drive
w # write and quit
- format: mkfs.ext4 -m 0 /dev/sdf1 # the [-m 0] is very important; without it, goofy ext4 wastes 5% "saved for root" wtf
- mount /dev/sdf1 /somewhere # make sure it looks good!
# install smartctl tools
smartctl -c /dev/sdf # to peek at available tests and how long they typically take to run
smartctl -t short /dev/sdf
watch -n 3 smartctl -l selftest /dev/sdf # until success, takes about 1 minute
# if you're feeling ambitious, rerun with [-t long]
- umount # we are NOT going to be manually mounting! we need to label the drive and put it in /etc/fstab
- e2label /dev/sdb1 box-type-sizename # what a stupid f'in name for that util
- emacs /etc/fstab - and put in an entry like this:
/dev/disk/by-label/box-type-sizename /my-mount-location ext4 noatime 0 2
- mount /my-mount-location # now it will be there when you reboot! but do this to get it NOW :-)
- Note that if you mount the new drive under /spiceflow, samba will already be serving it up, whoop!
|
| Adding a user
|
Ubuntu
- sudo adduser m # this will prompt you for everything
- sudo useradd -d /home/m -s /bin/bash -m m # this one-liner works well too
- sudo visudo # add a NOPASSWD entry for the user to the END of the file (or NOPASSWD won't stick)
- cd .ssh # and scp two files from bitpost (make sure they're rw only to user): id_rsa and authorized_keys
Gentoo server
su -
useradd #newuser# -m -G wheel(,etc)
# Other possibilities (likely not needed):
## only needed if you want them to have their own group (otherwise use "users" group)
#groupadd #newuser#
## the additional groups here are optional
#useradd #newuser# -m -g #newuser# -G users,wheel,audio,video,games,dvd,usb -s /bin/bash
## Only needed if you didn't use [-m] option
#mkdir /home/#newuser#
#chown #newuser#:users /home/#newuser#
passwd #newuser#
# grant ssh access by adding to "AllowUsers"
em /etc/ssh/sshd_config
/etc/init.d/sshd restart
# grant samba access to home folder
# it's already set up in /etc/samba/smb.conf
# but the user must be manually added to samba
# use same pwd as before to sync them
smbpasswd -a #newuser#
/etc/init.d/samba restart
# YOU PROBABLY HAVE TO RESTART ANY STUPID WINDOWS BOX before it will see things correctly! crazy
# you can try this in Windows but it didn't work for me:
# net use
# net session \\samba.server.ip.address /delete
|
| Check disk speed
|
To check SATA capability:
bitpost ~ # hdparm -I /dev/disk/by-label/reservoir-3g-mel |grep -i speed
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
To test read speed:
bitpost ~ # hdparm -tT /dev/disk/by-label/reservoir-3g-mel
/dev/disk/by-label/reservoir-3g-mel:
Timing cached reads: 33760 MB in 2.00 seconds = 16900.70 MB/sec
Timing buffered disk reads: 274 MB in 3.00 seconds = 91.28 MB/sec
You can use dd to check speed, too:
dd conv=fdatasync if=/dev/zero of=/spiceflow/newraid-4tb-bp/dd_test_output.dat bs=8k count=256k
=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 4.57219 s, 470 MB/s
# don't forget to clean up..
rm /spiceflow/newraid-4tb-bp/dd_test_output.dat
|
| Back up a linux system to a second bootable drive
|
Steps:
- We want to copy all files from the root drive to a new drive.
- We also want to update the boot menu to boot off the new drive.
- We also want to change /etc/fstab on the second drive to use the new root path.
These are the steps for backup of the dune box to the /spiceflow/2.0tb-newmovies/ drive.
When we're done, we'll boot from that drive to prove we have a working standby system.
rsync does an excellent job of copying just what we need. Test it with this:
# a (archive mode -rlptgoD) v (verbose) x (don't cross filesystems) h (human-readable) n (dry run)
rsync -avxhn --progress / /spiceflow/2.0tb-newmovies/
Do the job with this:
rsync -avxh / /spiceflow/2.0tb-newmovies/
sent 28.10G bytes received 8.64M bytes 15.63M bytes/sec
total size is 39.57G speedup is 1.41
There was def some old stuff in there worth cleaning up:
/home/m/development/svn/mythtv...
/var/tmp
xbmc log files
etc
Next we fix /etc/fstab on the new drive to use the drive as root. Original config:
/dev/disk/by-label/d-g2-root / ext3 noatime 0 1
/dev/disk/by-label/d-sp-20newmovies /spiceflow/2.0tb-newmovies ext4 noatime 0 2
New config (remember tho, we lose the 2gb drive):
/dev/disk/by-label/d-g2-root /root-hot-drive ext3 noatime 0 1
/dev/disk/by-label/d-sp-20newmovies / ext4 noatime 0 2
Note: this never worked out for me due to my system's mix of ext3 and ext4.
I'll retry once I migrate everything to ext4.
Next, grub:
mount /boot
em /boot/grub/menu.lst
Orig entry:
title ----- linux-2.6.34-gentoo-r11_withext4
root (hd0,0)
kernel /boot/linux-2.6.34-gentoo-r11_withext4 root=/dev/sda4
Add a new "standby" entry under it - do a "df" to find the root:
title ----- ==STANDBY== linux-2.6.34-gentoo-r11_withext4
root (hd0,0)
kernel /boot/linux-2.6.34-gentoo-r11_withext4 root=/dev/hda1
Then reboot to standby and see what we have... (no "latest" video library etc., but otherwise the same?)
|
| Mount an IMG file
|
You have to peek at the file with fdisk:
fdisk -lu /home/m/bitpost/download/torrents/OpenELEC-Generic.x86_64-5.0.8-efi.img
From that you can tell the "units" and "Start" offsets:
Device Start End Sectors Size Type
/home/m/bitpost/download/torrents/OpenELEC-Generic.x86_64-5.0.8-efi.img1 2048 526336 524289 256M Microsoft basic data
/home/m/bitpost/download/torrents/OpenELEC-Generic.x86_64-5.0.8-efi.img2 528384 593920 65537 32M Linux filesystem
From there you can mount with a little math. Multiply the units by Start to get offset (512x2048=1048576), and use it to mount that partition:
mount -t auto -o loop,offset=1048576 /home/m/bitpost/download/torrents/OpenELEC-Generic.x86_64-5.0.8-efi.img /media/m/iso
aifol
|
| Chroot into gentoo from (say) ubuntu
|
# These may already be done
mount /mnt/gentoo
cp -L /etc/resolv.conf /mnt/gentoo/etc/
mount -t proc none /mnt/gentoo/proc
mount --rbind /dev /mnt/gentoo/dev
chroot /mnt/gentoo /bin/bash
# ** NOW IN CHROOT **
env-update
source /etc/profile
export PS1="(chroot) $PS1"
# GET TO IT! :-)
|
| Windows symbolic links (symlinks - needed for config files, svn overlay)
|
i'm setting up hard links from sublime install folder to shared config settings:
mklink [/H /D /J] link_path Target_file/folder_path
We had to use a soft link, hard won't go across drives. And you have to run the cmd window as admin. Don't specify type, and it will create a soft file link:
mklink "C:\Users\Michael\AppData\Roaming\Sublime Text 3\Packages\User\Default (Windows).sublime-keymap" "D:\Michael's Data\development\config\common\home\m\.config\sublime-text-3\Packages\User\Default (Windows).sublime-keymap"
mklink "C:\Users\Michael\AppData\Roaming\Sublime Text 3\Packages\User\Preferences.sublime-settings" "D:\Michael's Data\development\config\common\home\m\.config\sublime-text-3\Packages\User\Preferences.sublime-settings"
Here is another example, setting up a link in an SVN repo folder to point to a git-controlled folder on another drive. SVN is OK with this, cool.
(open ADMIN cmd prompt)
cd C:\CODE\QA\ContinuousIntegration\Trunk
mklink /J scripts "D:\Michael's Data\development\power_analytics\scripts"
Junction created for scripts <<===>> D:\Michael's Data\development\power_analytics\scripts
|
| Route traffic to avoid VPN
|
| You can use "route" to assign paths to specific traffic.
So bitpost.com requests go through eth0 instead of VPN, eg...
C:\WINDOWS\system32>route add 174.109.14.112 192.168.50.1
Note that you can do this with [route] in linux too, syntax is a bit diff.
|
|
