From Bitpost wiki
Jump to navigation Jump to search

Running ProxMox and many VMs.


Hive - AbtDev1 - Matcha - Bandit - Positronic - Morosoph - Glam - Matryoshka - Hoard - Tornados

Add a new VM to melange - Clone a melange VM

VM Backups

Backups of all VMs (except hive data) are here:

Datacenter > Storage > bitpost-backup

That is a directly-configured samba link to's softraid. It has enough raided storage to fit cascading backups for all VMs there. Whoooop.

NOTE: NEVER MOUNT /spiceflow/bitpost, as it may conflict with Proxmox samba link. The Proxmox samba link should come up on melange reboot, and do everything it needs, itself.


Copy media

On melange, you can plug microSSDs into the melange hub, and you can also reach samba shares.

So direct copy from shares to microSSD is possible without 1Gbit network bottleneck.

sudo su -
mount /dev/sdh1 /media
mount /spiceflow/safe
rsync --progress /media/... /spiceflow/safe/...

Turns out that's not much of a bottleneck yet... my best cards were only getting 100MB/s write speed... test more cards maybe! In the meantime, just use Nautilus on cast.

Reading should def be faster though, if you need to copy from microSSD to shares.


Start order

Each VM has a "Start/Shutdown order" Option. We want some VMs to start first. To do that, we set them to a specific order number, with a delay, which will slot them to start before the default VMs. You can also specify an up value, allowing for a pause before starting the VMs in the next order. Nice.

We want hive to start first, so NAS drives can be connected on startup of other VMs.

We might want positronic to start before AbtDev1.... But to do that right, we need to set ALL other VMs to order 2, and AbtDev1 to order 3. It's not even really needed though - AbtDev1 won't be connecting to positronic until I go there and run a development build, and by then positronic should be well under way. So I'll go the much-easier route and leave all VMs other than hive to the default.

NOTE that bitpost is on bare metal, and needs positronic, so manage that manually on any restart of melange!

Hive: order=1,up=60
All other VMs: order=Any (the default)


Minor updates

Minor updates are so easy, eg for 7.3.6.

  • (optional) You don't NEED to, but it's a good time to upgrade and shutdown any VMs. It also went fine to do this while leaving all VMs up though!
  • Click:
Datacenter > melange > (melange bar) > Updates > Upgrade
  • Reboot

Reboot after upgrade

Melange VMs all come back nicely, because they come up staged well. See the order fo the VMs for details, particularly hive:

VM > Options > Start/shutdown order: order=1,up=90


1 TB M.2 nvme 
/dev/mapper/pve-root / # /dev/nvme0n1p3
/boot/efi /dev/nvme0n1p2
  • GPU: Nvidia GK208 - GeForce GT 640
sudo lspci|grep VGA
VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 640 Rev. 2] (rev a1)
  • 4 SAS connections on mobo, with a SAS-to-SATA cable; make sure to turn on "SATA mode" for U.2 in BIOS; all passed through to hive
  • 4 SATA ssd connections on mobo, all passed through to hive


32GB x 4 sticks (maxed out)

128GB total, distributed as follows:

  • 34GB AbtDev1
  • 24GB Hive
  • 16GB Positronic
  • 12GB Matcha
  • 12GB Glam
  • 06GB Matryoshka
  • 06GB Bandit
  • 03GB Morosoph
  • 04GB Hoard (stopped)
  • 04GB Tornados (stopped)

Historic update after 64 > 128 upgrade Had ~83GB distributed, distribute +37GB, leaving ~8GB (OS provides ~125/128GB, ie it needs ~3GB)

  • 20GB AbtDev1 + 14 = 34816MB
  • 19GB Hive + 5 = 24576MB
  • 10GB Positronic +6 = 16384MB
  • 08GB Glam +4 = 12288MB
  • 08GB Matcha +4 = 12288MB
  • 02GB Bandit +4


We pass the LSI PCI-E board (and its 8 ssd drives), the 4 SATA SSD drives, and the 4 U.2 SSD drives to Hive, for storage management.

LSI PCI-E board passthru

TODO (I don't remember)

SSD drives passthru

  • Identify drives on melange
 🌐 m@melange  [~] sudo lsblk |awk 'NR==1{print $0" DEVICE-ID(S)"}NR>1{dev=$1;printf $0" ";system("find /dev/disk/by-id -lname \"*"dev"\" -printf \" %p\"");print "";}'|grep -v -E 'part|lvm'
sdh                            8:112  0 931.5G  0 disk   /dev/disk/by-id/wwn-0x500a0751e5a2fb01 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A2FB01
sdi                            8:128  0 931.5G  0 disk   /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A2FD56 /dev/disk/by-id/wwn-0x500a0751e5a2fd56
sdj                            8:144  1 931.5G  0 disk   /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A313D2 /dev/disk/by-id/wwn-0x500a0751e5a313d2
sdk                            8:160  1 931.5G  0 disk   /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A313D6 /dev/disk/by-id/wwn-0x500a0751e5a313d6
sdl                            8:176  1 931.5G  0 disk   /dev/disk/by-id/ata-CT1000MX500SSD1_2117E59AAE1B /dev/disk/by-id/wwn-0x500a0751e59aae1b
sdm                            8:192  1 931.5G  0 disk   /dev/disk/by-id/wwn-0x500a0751e5a2e131 /dev/disk/by-id/ata-CT1000MX500SSD1_2121E5A2E131
sdn                            8:208  1 931.5G  0 disk   /dev/disk/by-id/wwn-0x500a0751e5a3009a /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A3009A
nvme0n1                      259:0    0 931.5G  0 disk   /dev/disk/by-id/nvme-eui.002538510141169d /dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_1TB_S4EWNJ0N107994E
  • Use qm set to pass them all to hive (VM 104)
sudo qm set 104 -scsi11 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A2FD56
sudo qm set 104 -scsi12 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A2FB01
sudo qm set 104 -scsi13 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A313D2
sudo qm set 104 -scsi14 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A313D6
sudo qm set 104 -scsi15 /dev/disk/by-id/ata-CT1000MX500SSD1_2117E59AAE1B
sudo qm set 104 -scsi16 /dev/disk/by-id/ata-CT1000MX500SSD1_2121E5A2E131
sudo qm set 104 -scsi17 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A3009A

# check
sudo cat /etc/pve/qemu-server/104.conf
(restart hive)


Plug something into melange and see what it sees:

Bus 004 Device 002: ID 05ac:12a8 Apple, Inc. iPhone 5/5C/5S/6/SE

Now we will pass that port through:

proxmox UI > Datacenter > Nodes > VMs > 111 (matcha) > Hardware > Add > USB > it will actually show up in a pick list!

Hell yeah so easy.