Melange
Running ProxMox and many VMs.
VMs
Hive - AbtDev1 - Bandit - Positronic - Morosoph - Glam - Matryoshka - Hoard
Copy media
On melange, you can plug microSSDs into the melange hub, and you can also reach samba shares.
So direct copy from shares to microSSD is possible without 1Gbit network bottleneck. Mount /media, and use rsync --progress as root.
Turns out that's not much of a bottleneck yet... my best cards were only getting 100MB/s write speed... test more cards maybe!
Reading should def be faster though, if you need to copy from microSSD to shares.
Hardware
- LSI Broadcom SAS 9300-8i 8-port 12Gb/s SATA+SAS PCI-Express 3.0 Low Profile Host Bus Adapter: Pass through of 8 drive connections to hive to use for its storage
- 4 SAS connections on mobo, with a SAS-to-SATA cable; make sure to turn on "SATA mode" for U.2 in BIOS; all passed through to hive
- 4 SATA ssd connections on mobo; 3 are passed through to hive
- 1 TB M.2 nvme
/dev/mapper/pve-root / # /dev/nvme0n1p3 /boot/efi /dev/nvme0n1p2
Passthroughs
We pass the LSI PCI-E board (and its 8 ssd drives), the 4 SATA SSD drives, and the 4 U.2 SSD drives to Hive, for storage management.
LSI PCI-E board passthru
TODO (I don't remember)
SSD drives passthru
- Identify drives on melange
🌐 m@melange [~] sudo lsblk |awk 'NR==1{print $0" DEVICE-ID(S)"}NR>1{dev=$1;printf $0" ";system("find /dev/disk/by-id -lname \"*"dev"\" -printf \" %p\"");print "";}'|grep -v -E 'part|lvm'
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT DEVICE-ID(S)
sdh 8:112 0 931.5G 0 disk /dev/disk/by-id/wwn-0x500a0751e5a2fb01 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A2FB01
sdi 8:128 0 931.5G 0 disk /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A2FD56 /dev/disk/by-id/wwn-0x500a0751e5a2fd56
sdj 8:144 1 931.5G 0 disk /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A313D2 /dev/disk/by-id/wwn-0x500a0751e5a313d2
sdk 8:160 1 931.5G 0 disk /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A313D6 /dev/disk/by-id/wwn-0x500a0751e5a313d6
sdl 8:176 1 931.5G 0 disk /dev/disk/by-id/ata-CT1000MX500SSD1_2117E59AAE1B /dev/disk/by-id/wwn-0x500a0751e59aae1b
sdm 8:192 1 931.5G 0 disk /dev/disk/by-id/wwn-0x500a0751e5a2e131 /dev/disk/by-id/ata-CT1000MX500SSD1_2121E5A2E131
sdn 8:208 1 931.5G 0 disk /dev/disk/by-id/wwn-0x500a0751e5a3009a /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A3009A
nvme0n1 259:0 0 931.5G 0 disk /dev/disk/by-id/nvme-eui.002538510141169d /dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_1TB_S4EWNJ0N107994E
- Use qm set to pass them all to hive (VM 104)
sudo qm set 104 -scsi11 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A2FD56 sudo qm set 104 -scsi12 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A2FB01 sudo qm set 104 -scsi13 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A313D2 sudo qm set 104 -scsi14 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A313D6 sudo qm set 104 -scsi15 /dev/disk/by-id/ata-CT1000MX500SSD1_2117E59AAE1B sudo qm set 104 -scsi16 /dev/disk/by-id/ata-CT1000MX500SSD1_2121E5A2E131 sudo qm set 104 -scsi17 /dev/disk/by-id/ata-CT1000MX500SSD1_2122E5A3009A # check sudo cat /etc/pve/qemu-server/104.conf
(restart hive)
Hardware history
Installation
sudo emacs /etc/fstab # and paste samba stanza from another machine sudo emacs /root/samba_credentials sudo mkdir /spiceflow && sudo chmod 777 /spiceflow 🌐 m@melange [~] mkdir /spiceflow/bitpost 🌐 m@melange [~] mkdir /spiceflow/grim 🌐 m@melange [~] mkdir /spiceflow/mack 🌐 m@melange [~] mkdir /spiceflow/reservoir 🌐 m@melange [~] mkdir /spiceflow/sassy 🌐 m@melange [~] mkdir /spiceflow/safe
Now you can mount em up and hang em high!