Dynamic Memory Management: Difference between revisions
(→Requirements for Windows VM: add notice about blnsvr.exe bug on 32bit systems with RAM more than 2GB) |
(Added some notes about VFIO and OOM) |
||
| (11 intermediate revisions by 8 users not shown) | |||
| Line 1: | Line 1: | ||
= Introduction | == Introduction == | ||
Optimized and effective memory management is a key factor in virtualization environments. KSM and Auto-Ballooning enables sophisticated and economic configurations for physical RAM utilization. | Optimized and effective memory management is a key factor in virtualization environments. KSM and Auto-Ballooning enables sophisticated and economic configurations for physical RAM utilization. | ||
= KSM | == KSM == | ||
KSM (Kernel Samepage Merging) is running in the Linux kernel and scans the memory of all the virtual machines running on a single host, looking for duplication and consolidating. With KSM we're able to improve virtual machine density by as much as 300% without impacting performance. One of the great benefits of using Linux as the hypervisor means KSM is not limited to KVM and virtual machines, but can also reduce memory pressure with normal Linux applications. | KSM (Kernel Samepage Merging) is running in the Linux kernel and scans the memory of all the virtual machines running on a single host, looking for duplication and consolidating. With KSM we're able to improve virtual machine density by as much as 300% without impacting performance. One of the great benefits of using Linux as the hypervisor means KSM is not limited to KVM and virtual machines, but can also reduce memory pressure with normal Linux applications. | ||
It was integrated in PVE since version 1.5, and is implemented with the package "ksm-control-daemon" (check your version with the cli command "pveversion -v"). | |||
Just install several KVM virtual machines with the same OS (using at least | === KSM in action === | ||
Just install several KVM virtual machines with the same OS (using at least 80% of your physical memory on the host) and wait a few minutes. You will notice higher CPU activities on the host (ksm daemon) and the used memory on the host will be lowered significantly (see start page showing the overall memory usage). | |||
Howto verify that KSM is working (how many pages are being shared between your KVM guests): | Howto verify that KSM is working (how many pages are being shared between your KVM guests): | ||
| Line 14: | Line 16: | ||
Note: a page is 4096 bytes | Note: a page is 4096 bytes | ||
= Ballooning | The file /etc/ksmtuned.conf allows for some customization of its behaviour. | ||
See also | |||
* http://www.linux-kvm.org/page/KSM | |||
* http://www.linux-kvm.com/content/using-ksm-kernel-samepage-merging-kvm | |||
* http://en.wikipedia.org/wiki/Kernel_SamePage_Merging_%28KSM%29 | |||
=== Disable KSM === | |||
If you don't care about memory optimization but care about save CPU overhead produced by KSM, in Proxmox >= 4.x you can disable it with: | |||
<pre>systemctl disable ksmtuned</pre> | |||
and reboot | |||
== Ballooning == | |||
Memory ballooning (KVM only) allows you to have your guest dynamically change it’s memory usage by evicting unused memory during run time. It reduces the impact your guest can have on memory usage of your host by giving up unused memory back to the host. | Memory ballooning (KVM only) allows you to have your guest dynamically change it’s memory usage by evicting unused memory during run time. It reduces the impact your guest can have on memory usage of your host by giving up unused memory back to the host. | ||
The Proxmox VE host can loan ballooned memory to a busy VM. The VM decides which processes or cache pages to swap out to free up memory for the balloon. The VM (Windows or Linux) knows best which memory regions it can give up without impacting performance of the VM. | The Proxmox VE host can loan ballooned memory to a busy VM. The VM decides which processes or cache pages to swap out to free up memory for the balloon. The VM (Windows or Linux) knows best which memory regions it can give up without impacting performance of the VM. | ||
== Requirements for Windows VM | Note that the Guest cannot ask for the memory back until the host has 'deflated' this balloon. If set too aggressively, some applications in the guest may run out of memory as a result and your guest's OOM killer may activate, killing important processes - https://www.kernel.org/doc/gorman/html/understand/understand016.html | ||
Note that if you are passing through a physical PCI(e) device or a Virtual Function I/O (VFIO) Mediated device (MDEV) such as a vGPU, then ballooning will not work since these devices are mapped to fixed memory addresses in the host and in the guest. If you do enable ballooning, the KVM process will not release the memory back to the host but the guest may report a reduced memory availability which can be manually recovered using the KVM monitor using the balloon command. | |||
=== Manually ballooning (testing) === | |||
In the monitor tab of your guest, type balloon <target_memory_in_mb> - so balloon 16384 will instruct your guest to target 16GB of memory. This is a 'target' which the driver on the VM side will try to reach by trying to free available memory. | |||
=== Requirements for Windows VM === | |||
==== Installation ==== | |||
*You need to install the VirtIO Balloon Driver and the management service (blnsvr.exe -i). | *You need to install the VirtIO Balloon Driver and the management service (blnsvr.exe -i). | ||
*Notice: there is a small blnsvr.exe bug on 32bit systems with RAM more than 2GB https://github.com/YanVugenfirer/kvm-guest-drivers-windows/issues/11 | *Notice: there is a small blnsvr.exe bug on 32bit systems with RAM more than 2GB: https://github.com/YanVugenfirer/kvm-guest-drivers-windows/issues/11 | ||
See [[Windows VirtIO Drivers]] to get info about | |||
* downloading VirtIO drivers | |||
* changelog and guest OS compatibility | |||
* other kind of guest devices supported | |||
=== Enable Auto-Ballooning on Windows 2008r2 | ==== Download ==== | ||
Download the latest drivers (ISO) as suggested by the page [[Windows_VirtIO_Drivers]] to your desktop. | |||
Then upload the ISO to your Proxmox VE server: | |||
* logon to the Proxmox VE web interface | |||
* select a ISO-enabled storage (see [[Storage_Model#Storage_type_Content]]) | |||
* switch to "content" tab | |||
* just use the "upload" button on the menu bar. | |||
==== Choose the right driver ==== | |||
* Follow the link: [[Windows_VirtIO_Drivers#Choose_the_right_driver]] | |||
==== Enable Auto-Ballooning on Windows Server 2012 / Windows 8.1 or newer ==== | |||
Starting with [https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/ virtio-win-0.1.173-2] the driver ISO provides an installer located at "<cddrive>:\virtio-win-gt-x64.msi" that can, besides other things, install the ballooning service. | |||
If it does not work you can follow the manual way which is described below for Windows 2008r2. | |||
==== Enable Auto-Ballooning on Windows 2008r2 ==== | |||
#Set the VM memory to "Automatically allocate memory within this range" - I choose 4096/2048 as example (see screenshot) | #Set the VM memory to "Automatically allocate memory within this range" - I choose 4096/2048 as example (see screenshot) | ||
| Line 47: | Line 90: | ||
</gallery> | </gallery> | ||
=== Enable Auto-Ballooning on Windows 2003 / Windows Xp | ==== Enable Auto-Ballooning on Windows 2003 / Windows Xp ==== | ||
You need to install the balloon service with BLNSVR, like for win2008. | You need to install the balloon service with BLNSVR, like for win2008. | ||
| Line 62: | Line 105: | ||
</pre> | </pre> | ||
== Requirements for Linux VM | |||
=== See also === | |||
* [[Windows_VirtIO_Drivers]] | |||
* [[Paravirtualized Block Drivers for Windows]] | |||
* [[Paravirtualized Network Drivers for Windows]] | |||
=== Requirements for Linux VM === | |||
Modern Linux Kernels does include the Balloon drivers by default. It works out of the box, and you only need to set the VM to "Automatically allocate memory within this range" | Modern Linux Kernels does include the Balloon drivers by default. It works out of the box, and you only need to set the VM to "Automatically allocate memory within this range" | ||
=External Links= | == External Links == | ||
tbd. | tbd. | ||
= Video Tutorials | |||
== Video Tutorials == | |||
tbd: [http://www.youtube.com/user/ProxmoxVE Proxmox VE Youtube channel] | tbd: [http://www.youtube.com/user/ProxmoxVE Proxmox VE Youtube channel] | ||
[[Category:HOWTO]] | |||
Latest revision as of 21:17, 6 October 2023
Introduction
Optimized and effective memory management is a key factor in virtualization environments. KSM and Auto-Ballooning enables sophisticated and economic configurations for physical RAM utilization.
KSM
KSM (Kernel Samepage Merging) is running in the Linux kernel and scans the memory of all the virtual machines running on a single host, looking for duplication and consolidating. With KSM we're able to improve virtual machine density by as much as 300% without impacting performance. One of the great benefits of using Linux as the hypervisor means KSM is not limited to KVM and virtual machines, but can also reduce memory pressure with normal Linux applications.
It was integrated in PVE since version 1.5, and is implemented with the package "ksm-control-daemon" (check your version with the cli command "pveversion -v").
KSM in action
Just install several KVM virtual machines with the same OS (using at least 80% of your physical memory on the host) and wait a few minutes. You will notice higher CPU activities on the host (ksm daemon) and the used memory on the host will be lowered significantly (see start page showing the overall memory usage).
Howto verify that KSM is working (how many pages are being shared between your KVM guests):
watch cat /sys/kernel/mm/ksm/pages_sharing
Note: a page is 4096 bytes
The file /etc/ksmtuned.conf allows for some customization of its behaviour.
See also
- http://www.linux-kvm.org/page/KSM
- http://www.linux-kvm.com/content/using-ksm-kernel-samepage-merging-kvm
- http://en.wikipedia.org/wiki/Kernel_SamePage_Merging_%28KSM%29
Disable KSM
If you don't care about memory optimization but care about save CPU overhead produced by KSM, in Proxmox >= 4.x you can disable it with:
systemctl disable ksmtuned
and reboot
Ballooning
Memory ballooning (KVM only) allows you to have your guest dynamically change it’s memory usage by evicting unused memory during run time. It reduces the impact your guest can have on memory usage of your host by giving up unused memory back to the host.
The Proxmox VE host can loan ballooned memory to a busy VM. The VM decides which processes or cache pages to swap out to free up memory for the balloon. The VM (Windows or Linux) knows best which memory regions it can give up without impacting performance of the VM.
Note that the Guest cannot ask for the memory back until the host has 'deflated' this balloon. If set too aggressively, some applications in the guest may run out of memory as a result and your guest's OOM killer may activate, killing important processes - https://www.kernel.org/doc/gorman/html/understand/understand016.html
Note that if you are passing through a physical PCI(e) device or a Virtual Function I/O (VFIO) Mediated device (MDEV) such as a vGPU, then ballooning will not work since these devices are mapped to fixed memory addresses in the host and in the guest. If you do enable ballooning, the KVM process will not release the memory back to the host but the guest may report a reduced memory availability which can be manually recovered using the KVM monitor using the balloon command.
Manually ballooning (testing)
In the monitor tab of your guest, type balloon <target_memory_in_mb> - so balloon 16384 will instruct your guest to target 16GB of memory. This is a 'target' which the driver on the VM side will try to reach by trying to free available memory.
Requirements for Windows VM
Installation
- You need to install the VirtIO Balloon Driver and the management service (blnsvr.exe -i).
- Notice: there is a small blnsvr.exe bug on 32bit systems with RAM more than 2GB: https://github.com/YanVugenfirer/kvm-guest-drivers-windows/issues/11
See Windows VirtIO Drivers to get info about
- downloading VirtIO drivers
- changelog and guest OS compatibility
- other kind of guest devices supported
Download
Download the latest drivers (ISO) as suggested by the page Windows_VirtIO_Drivers to your desktop.
Then upload the ISO to your Proxmox VE server:
- logon to the Proxmox VE web interface
- select a ISO-enabled storage (see Storage_Model#Storage_type_Content)
- switch to "content" tab
- just use the "upload" button on the menu bar.
Choose the right driver
- Follow the link: Windows_VirtIO_Drivers#Choose_the_right_driver
Enable Auto-Ballooning on Windows Server 2012 / Windows 8.1 or newer
Starting with virtio-win-0.1.173-2 the driver ISO provides an installer located at "<cddrive>:\virtio-win-gt-x64.msi" that can, besides other things, install the ballooning service.
If it does not work you can follow the manual way which is described below for Windows 2008r2.
Enable Auto-Ballooning on Windows 2008r2
- Set the VM memory to "Automatically allocate memory within this range" - I choose 4096/2048 as example (see screenshot)
- Start the VM and install all virtio drivers, including the balloon driver (see screenshot)
- Copy and rename as Administrator the WIN7\AMD64 directory from the virtio.iso to "c:/Program files/Balloon"
- Open a CMD as Administrator and cd into "c:/Program Files/Balloon"
- Install the BLNSVR with "BLNSVR.exe -i"
As soon as the service is started, also the memory information displayed on the Proxmox VE GUI is identical to the value shown in the windows task manager (see screenshot).
If you need details about ballooning stats for this VM, go to the KVM monitor and enter 'info balloon'
VirtIO drivers: KVM project wiki downloads
Set Memory to Dynamic
VirtIO Balloon Driver
BLNSVR Service
Memory Usage
Enable Auto-Ballooning on Windows 2003 / Windows Xp
You need to install the balloon service with BLNSVR, like for win2008.
You also need to install the driver manually
Download the "devcon" software on microsoft website
http://support.microsoft.com/kb/311272
Then install the balloon driver with
devcon install BALLOON.inf "PCI\VEN_1AF4&DEV_1002&SUBSYS_00051AF4&REV_00"
See also
- Windows_VirtIO_Drivers
- Paravirtualized Block Drivers for Windows
- Paravirtualized Network Drivers for Windows
Requirements for Linux VM
Modern Linux Kernels does include the Balloon drivers by default. It works out of the box, and you only need to set the VM to "Automatically allocate memory within this range"
External Links
tbd.