Hotplug (qemu disk,nic,cpu,memory)
Proxmox support hotplugging of Qemu devices:
- Disks (hotplug/unplug)
- NICs (hotplug/unplug)
- CPU (hotplug/unplug)
- Memory (hotplug/unplug)
Linux Guests Notes
Kernel Modules Needed
You need to have 2 modules loaded in your Linux guest:
#modprobe acpiphp #modprobe pci_hotplug
CPU and Memory Hotplug
You need a recent kernel (> 3.10) and a udev rules file, to auto-enable CPU or memory hotplugging.
You can add the file:
/lib/udev/rules.d/80-hotplug-cpu-mem.rules
SUBSYSTEM=="cpu", ACTION=="add", TEST=="online", ATTR{online}=="0", ATTR{online}="1" SUBSYSTEM=="memory", ACTION=="add", TEST=="state", ATTR{state}=="offline", ATTR{state}="online"
For Linux kernel >= 4.7, you don't need the udev rule for memory hotplug; you only need to add this kernel parameter at boot:
memhp_default_state=online
For Linux kernel >=5.15 (e.g. Ubuntu 22.04) you only need the UDEV rule for cpu hotplug:
/lib/udev/rules.d/80-hotplug-cpu.rules
SUBSYSTEM=="cpu", ACTION=="add", TEST=="online", ATTR{online}=="0", ATTR{online}="1"
Windows Guests Notes
- Disk and NIC hotplug work out of the box for all Windows versions >2003
- CPU and memory hotplug work on > 2008 enterprise/datacenter, 2012 > standard
Disk and NIC Hotplug/unplug
Edit your VM configuration file:
/etc/pve/qemuserver/<VMID>.conf
and add:
hotplug: network,disk,....
Note: Since Proxmox 3.3, the hotplug option is available in the GUI.
Then add/remove your disks or NICs as usual.
For disk unplug, verify that your disk is not mounted in your Linux guest or enabled in Windows Disk Management.
CPU Hotplug
- In the VM Options, edit the "Hotplug" Value and activate "CPU Hotplug".
- In the Hardware Section of your VM, specify the maximum number of Sockets/Cores.
- You can now use the "CPU Options" dialog to change the number of vCPUs, while the machine is running.
maxcpus = sockets * cores.
Note : Windows CPU hotplug has been working since qemu 2.6 and CPU hot unplug has been working since qemu 2.7.
Memory Hotplug
Memory hotplug is available in proxmox 3.4.
You need to enable NUMA and hotplug to get it working:
numa: 1 hotplug: memory,...
Then you can use the memory GUI options as usual.
Memory Hot Unplug
- Memory unplug currently does not work on Windows (<=win10 currently)
- Memory unplug can be unstable on Linux (<= kernel 4.10 currently)
For Linux memory unplug, you need to have movable zone enabled, in the kernel config (not enabled by default on Debian/Ubuntu):
CONFIG_MOVABLE_NODE=YES
and the "movable_node" boot kernel parameter enabled.
Here are some future kernel patches to improve unplug : http://www.spinics.net/lists/linux-mm/msg125359.html
Update max_mem_regions when hotplug is for more than 64GB
For Error : a used vhost backend has no free memory slots left
Kernel sets how many memory dimms per vhost it supports, default 64. You can increase the limit by creating a file /etc/modprobe.d/vhost.conf
Containing options vhost max_mem_regions=128 (default is 64, you can try increasing it further if necessary) and then rebooting the Proxmox machine.
echo "options vhost max_mem_regions=128" > /etc/modprobe.d/vhost.conf rmmod vhost_net rmmod vhost modprobe vhost_net cat /sys/module/vhost/parameters/max_mem_regions # 128
Ref Forum link : https://forum.proxmox.com/threads/problem-with-hotplug-and-64gb-ram.73621/page-3#post-600595