Proxmox

From Bitpost wiki

Available VM Types

  • Place ISOs in /var/lib/vz/template/iso
  • To upload via Proxmox web ui:
Storage View > Datacenter > melange > local > (you might have to hit refresh now!) > ISOs > Upload

VM Installation

  • VM install from ISO
    • When you first boot an ubuntu iso, it will behave like an installation thumb drive.
    • Install to the only available drive of the VM (/dev/sda). Proxmox is smart enough to allow this. The install ISO ends up as a DVD drive.
    • Once VM install is completed, remove the DVD:
      • sudo umount /dev/sf0 (or whatever)
      • Proxmox > (VM) > hardware > DVD > remove
      • Proxmox > (VM) > shutdown, then start

Proxmox update

See melange upgrade notes.

Creating and cloning VMs

Melange has specific instructions here:

Add a new VM to melange - Clone a melange VM

Changing VM disk size

Grow

Increase is fairly easy:

  • (VM) > Hardware > Hard Disk > Disk Action > Resize > Add number of GB to increase
  • reboot the VM
  • Update the OS

Ubuntu

  • Update the ubuntu filesystem (if you didn't use LVM (which is basically useless))
sudo parted /dev/sda # use the right one, obv!
  print # to get partition list
  resizepart 2 # use the right one, obv!
  End?  100%
sudo resize2fs /dev/sda#

If you used LVM in ubuntu (stop doing that!), you have to do all this: 

sudo su -
parted -s -a opt /dev/sda "print free" # see existing partitions
parted -s -a opt /dev/sda "print free" "resizepart 3 100%" "print free" # resize partition to fill space
pvresize /dev/sda3 # to get the LVM resized to the partition
lvdisplay
lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv # to resize the logical volume
resize2fs /dev/ubuntu-vg/ubuntu-lv

Mac

For a macOS VM, thenickdude has notes.

  • In proxmox, stop the VM, find the hardware disk and do a Disk operation to increase the size, and restart the VM
  • Fix it up in an OSX terminal:
diskutil list
# fix the partition
diskutil repairDisk disk0
# actually resize the contents
diskutil apfs resizeContainer disk1 0

Windows

  • Use diskpart to remove the restore partition, if it is in the way of expanding the C:\ drive.
  • Use disk manager to expand C:\

Shrink

I have not yet found a way to shrink a VM's disk. I had to significantly shrink bandit's drive. I ended up rebuilding it from scratch on a smaller starting drive.

Obv. ridiculous. Not for lack of trying. Read on for some notes from failed attempts.

Clonezilla MAY be able to shrink partitions, then clone them, then let you restore them to same-sized partitions on a smaller drive. I never got it to work successfully. You would have to get pretty good at precisely sizing partitions... Some basic steps follow...

  • Create a new vm with the new disk size you want (and a diff hostname).
  • Old vm > Add CD > Use the clonezilla iso > change boot order > boot into clonezilla
  • Select the advanced mode > for target, select samba > use m, /spiceflow/softraid/backup, 192.168.21.1
  • It should prompt for password and verify the drive size, sweet!!
  • Clone the PARTITIONS not the DRIVE or it won't shrink and it won't fit on new drive
  • Go through all the prompts using defaults to send /dev/sda to a backup file on softraid
  • Shut down the old vm, move the clonezilla iso to the new vm, boot clonezilla
  • Use advanced mode, restore from smb image (similar to savepart), but make sure to select -icds (skip partition size check), i also had it create a new partition table (was that good? i got an error)
  • fix the hostname in proxmox, and dnsmasq MAC address on bitpost

More horror that never worked...

Shrinking is VERY painful and VERY hard to find instructions, a lot of trial and horror... and no luck yet

  • Add CD > choose gparted live cd > boot with it, shrink partition
  • Stop the VM, remove the CD
  • ssh melange # and massage the disk there
sudo su -
lvdisplay | grep "LV Path\|LV Size"
qemu-img info /dev/pve/vm-105-disk-0
# DO NOT RESIZE DIRECTLY, it fucks the disk: qemu-img resize -f raw vm-105-disk-0 -210G
# FYI THIS ALSO FUCKED THE DRIVE: lvreduce -L 40G /dev/pve/vm-105-disk-0
# i think that is ONLY FOR LVM DISKS
mount /spiceflow/bitpost/
qemu-img convert vm-105-disk-0 /spiceflow/bitpost/softraid/backup/vm-105-disk-0.raw
qemu-img resize -f raw /spiceflow/bitpost/softraid/backup/vm-105-disk-0.raw --shrink -200G
  Image resized.
# DO NOT DO THIS it will FUCK THE DISK: # qemu-img convert -p -O qcow2 /spiceflow/bitpost/softraid/backup/vm-105-disk-0.raw vm-105-disk-0
# now just copy it back over itself!  weird but it works (no it doesn't, when you restore)
cp /spiceflow/bitpost/softraid/backup/vm-105-disk-0.raw --shrink -200G
  • edit 105.conf and reduce disk to the proper size (not sure if that's just so the GUI looks correct?)
emacs /etc/pve/nodes/melange/qemu-server/105.conf

Changing VM CPU allocation

  • Navigate to (VM) > Hardware > Processors
  • Adjust core count. Overallocation is a very good idea. CPU cores will be used only as threads need them. I've read that some shops successfully overallocate by a factor of 20:1. With our 12 Ryzen 9 cores, that means we would allocate 120 total cores to our VMs(!). Don't do that, ha.
  • Stick with 1 socket (that's just for sizing to match any paid licensing)

VM remote desktop display

I'm using SPICE for a full responsive 4k UI on other thin(ner) clients. BUT... use the VirtIO-GPU Display driver! (NOT SPICE)

SPICE

NOTE that the qmu driver CRASHES even after fixing the memory allocated (see below). Instead, use the VirtIO-GPU Display driver, which works fine with SPICE,

When using qmu driver, using vscode on abtdev1 on a second desktop would crash the VM with SPICE memory set to 36(MB). I bumped it to 80 and it lasted a bit longer. It apparently needs more when you use more desktops, in both Windows and linux, according to this. But ... jsut don't use qmu/SPICE display!

Proxmox config

There is a LOT of configuration. After getting it all set, I copied it all out, following some forum suggestion. Try to keep it updated!

Config is here: 

development/config/proxmox/melange

Manage it from here: 

🌐 m@melange  [~/development/config/proxmox/melange] cat backup.txt

For example, to back up a VM config after making changes: 

🌐 m@melange  [~/development/config/proxmox/melange/etc/pve/qemu-server] git pull
git mv 111.conf 111.conf-old-montery
sudo cp /etc/pve/qemu-server/111.conf .
sudo chown m:m 111.conf
git add 111.conf
git commit -a -m "Updated matcha 111 config now that Ventura is working" && git push

Storage

CIFS

To mount a samba share as proxmox storage: 

proxmox UI > Server view > Datacenter > Storage > Add > CIFS > enter fields

Remember to set Content Type to include the things you will place there.

Backups

  • I am backing up to softraid/backup on bitpost. See the bitpost-backup SMB share under the melange tree in the lefthand bar.
  • I am backing up EVERY VM. In order to back up the hive VM (104), which includes TB of storage that we cannot back up, you must mark all drives other than the OS drive with ",no backup", here:
/etc/pve/nodes/melange/qemu-server/104.conf
  • Proxmox puts files here:
/spiceflow/softraid/backup/dump/vzdump-qemu-103-2022_03_24-09_15_59.vma.zst
  • I am currently keeping 1 backup and 1 "weekly" backup. Odd, but you configure that in Storage:
proxmox UI > Server view > Datacenter > Storage > bitpost-backup > Edit > Backup Retention tab

It's a bit confusing, this tool helps a lot!

Configuration:

  • I set up a dedicated CIFS share for that on bitpost, named backup.
  • I added a Storage that uses it, and set 'VZDump backup file' as the Content Type.
  • From there, it's easy to configure VM backups.
proxmox UI > Server view > Datacenter > Backup > Add

I don't want to try to back up my massive FreeNAS data! So select "Exclude VMs" and check hive/104.

  • If something goes wrong, you can kill the backup task from a terminal:
 🌐 m@melange  [~] ps ax|grep vz
 🌐 m@melange  [~] sudo kill -9 #pid#
# this may cause "VM is locked (backup)" - to fix that:
sudo qm unlock 101
  • You can watch the backup job logging in the proxmox system log:
sudo tail -f /var/log/syslog 
...
Mar 24 09:23:01 melange pvedaemon[21327]: INFO: Finished Backup of VM 103 (00:07:02)
Mar 24 09:23:01 melange pvedaemon[21327]: INFO: Starting Backup of VM 105 (qemu)
...

debian

I had debian apt-get stop working due to distro changes. Run this to fix it: 

ssh melange
sudo apt-get --allow-releaseinfo-change update

Troubleshooting

Samba

Samba is supposed to work so you don't have to touch it, but when it goes down it's not easy to restart. I'm still trying to figure out how to restart samba, short of restarting the box.

I think bitpost.com losing its DNS may cause a real problem for proxmox. Prolly good to follow this advice. I use "bitpost" with the LAN IP in /etc/hosts:

  • From the manual: To avoid DNS lookup delays, it is usually preferable to use an IP address instead of a DNS name - unless you have a very reliable DNS server, or list the server in the local /etc/hosts file.

Proxmox Installation

  • In BIOS, enable SVM (cpu virtualization) (you need a modern AMD or Intel chipset)
  • Get the latest proxmox release ISO, dd it to a thumb drive (use Flash_Drives SAM 64 EVO)
  • Boot and install onto the primary drive over any existing OSes
  • During install, use ext4
    • I deemed ZFS too fancy, it's basically software raid, and troublesome according to some
  • Create a user and use ssh key (NOTE you might want to keep some root terminals open so you don't screw up and get locked out!):
apt install sudo
adduser m
visudo # and allow m to sudo
nano /etc/ssh/sshd_config # and turn off password login, root login
su - m
# set up ssh keys
# in another terminal, test:
ssh melange
  • Fix the fucking default proxmox apt repo from "enterprise" to "no-license". BTW, they say it's not well tested, and you better pay for a subscription and get a key for the "better" repo. Cmon you all that is utter bullshit...
# Either delete /etc/apt/sources.list.d/pve-enterprise.list file or comment all lines in this file with #.
emacs -nw /etc/apt/sources.list.d/pve-no-subscription.list
deb http://download.proxmox.com/debian/pve buster pve-no-subscription
Retrieved from "https://bitpost.com/w/index.php?title=Proxmox&oldid=7578"