Proxmox VE inside VirtualBox: Difference between revisions

From Proxmox VE
Jump to navigation Jump to search
m (Typos.)
 
(43 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= How to install Proxmox PVE inside Oracle VirtualBox =
Running '''Proxmox VE inside VirtualBox''' is possible but is not recommended for production use. Virtualizing the Proxmox VE hypervisor inside VirtualBox is usually done for development and testing purposes. For example, wanting to use VirtualBox as a host for Proxmox on a laptop is a [http://forum.proxmox.com/google.php?cx=partner-pub-1240956337013298%3A0281164794&cof=FORID%3A9&ie=ISO-8859-1&q=virtualbox&sa.x=-1192&sa.y=-47&siteurl=forum.proxmox.com%2F&ref=&ss=1399j312179j10 common wish] but can be tricky to get right. This page covers special considerations necessary for Proxmox VE to function as expected when it is being hosted by the VirtualBox hypervisor.


* WARNING: ''Article in development''
Unless otherwise stated, this page assumes you are using:
** For general setup, please comment on http://forum.proxmox.com/threads/19966-Proxmox-under-Virtualbox
** For network issues, http://forum.proxmox.com/threads/20054-Proxmox-under-Virtualbox-no-outbound-networking
* Instructions assume VirtualBox 4.3.16 or higher
* Much credit goes to the authors of the article on orestad-linux.se


== Motivation ==
* VirtualBox 4.3.16 or higher.
For development and testing purposes, it can be useful to install PVE inside a virtual machine on e.g. a laptop. Wanting to use Virtualbox as a host for Proxmox on a laptop is a [http://forum.proxmox.com/google.php?cx=partner-pub-1240956337013298%3A0281164794&cof=FORID%3A9&ie=ISO-8859-1&q=virtualbox&sa.x=-1192&sa.y=-47&siteurl=forum.proxmox.com%2F&ref=&ss=1399j312179j10 common wish] but can be tricky to get right.
* Proxmox VE 4.x or higher (i.e. for lxc containers only, OpenVz is not supported any more)


== Goal ==
= Prerequisites =


For accessing the internet use NAT - an address would be assigned by VirtualBox's DHCP service, usually something like 192.168.x.x or 10.0.x.x - but this would keep changing and completely disappear when WLAN loses connection. Instead, to have it under control we won't use DHCP from the WLAN but rather manually set up a separate subnet and manually assign a static IP address for the VM containing PVE
For Proxmox VE inside VirtualBox to work well, it's strongly recommended that your VirtualBox host (e.g., your laptop) meets the minimum hardware requirements and software configuration. Otherwise, Proxmox VE may still work but it can be extremely slow.


== Prerequisites ==
* VirtualBox should support hardware virtualization acceleration. This means the feature is activated in EFI/BIOS. If you don't see this option in your EFI/BIOS screens, you should double-check to see if your CPU supports it.
For this to work it's strongly recommended (otherwise it may work but can be extremely slow) that
* Further, if you plan to host any guests within Proxmox and are running VirtualBox 6.1 or later, you should enable nested hardware virtualization acceleration in the VirtualBox settings for the Proxmox VE virtual machine itself. The setting is in the [https://www.virtualbox.org/manual/UserManual.html#settings-processor VirtualBox Machine Processor Tab] (System → Processor → '''Enable Nested VT-x/AMD-V''').
* your laptop needs to support HW acceleration (to be activated in BIOS, but check it your CPU supports it)
* When creating the VirtualBox VM to host Proxmox VE, you should set its [https://www.virtualbox.org/manual/UserManual.html#settings-basic OS type] to be run as '''Linux: Debian (64-bit)''' guest.
* in virtual machine's settings the above is activated by System > Acceleration > Hardware Virtualization: Enable VT-x/AMD-V  
It has to be run as Linux Debian (64-bit) guest.
* The assumption of this article is that the PVE installation needs to be accessible only from the laptop.
* These instructions suppose the reader has minimal knowledge of networking, advanced configuration options that assume more knowledge are at [[Network Model]].  


== Containers as target ==
= Nested Containers and simple Virtual Machines =


You can run Containers (OpenVZ) in PVE without any problems. It's possible to configure KVM (Virtual Machines), you can run them too if they don't have any graphical interface and don't consume to much CPU resources (tested e.g. with [http://slaxrouter.sourceforge.net/slax-router slax-router]); be careful: in all other cases the Virtual Machine will freeze and can block your guest system too!
You can run LXC Containers in the virtualized Proxmox VE without any problems. It's also possible to create and configure fully nested virtual machines, so long as you enable VirtualBox's supported for nested VT-X/AMD-V hardware instructions, noted above. Nested virtual machines will run fairly well if they don't have any graphical interface and don't consume too many CPU resources. (This was tested with, e.g., [http://slaxrouter.sourceforge.net/slax-router slax-router].) Do be careful, though. If your nested VMs require too many resources, they will freeze up and can block your guest system too!


== Install PVE ==
Also, note that some VirtIO drivers are known to make problems if nested in a VirtualBox environment, although this seems to have gotten more reliable with later version of VirtualBox (specifically 6.1 and higher). For this reason, it's recommend to '''use <tt>e1000</tt>''' for the nested VM's network adapters if you can, especially since the improved performance offered by the ''virtio'' driver is not typically required in a nested scenario such as this.
1. Create a new Virtualbox Virtual Machine for Proxmox


Sample settings:
Further, when using earlier VirtualBox versions, KVM hardware virtualization does not work and must not be set! The reason is that with these VirtualBox versions, nested hardware virtualization support is not possible. [https://www.virtualbox.org/manual/UserManual.html#nested-virt Nested virtualization became possible starting in VirtualBox version 6.1].
* Name: Proxmox-vm
* OS Type: Linux
* Version: Debian (64 bit)
* Memory: 6GB
* Disk: Use a SSD if possible. Preallocated might provide faster access.


2. Edit the settings
= Network Considerations =
* Audio: disable
* Storage:
** IDE - Select Empty - click IDE Secondary - select the disk symbol, pick ISO for Proxmox
* Network:
** Adapter 1: NAT (attention: '''NOT''' NAT-network!!)
** Adapter 2: Host-only Adapter, vboxnet0; recommended leave the advanced settings as they are.


Laptops use usually (only) WLAN connections which can be tricky because their network address continually change, and sometimes disappear, whereas desktops can assign a static ip address on a stable network. In any case it's recommended to use for access ''to'' PVE VirtualBox's "Host-only Network" via "Host-only Adapter". For access to outside (internet) ''from'' PVE use VirtualBox's "NAT". '''Do not use''' (it's not reliable!)
Most [https://www.virtualbox.org/manual/UserManual.html#networkingmodes VirtualBox networking modes] will work as expected when running Proxmox VE as a guest within VirtualBox. However, there are a few things to consider specially.
* "NAT network" (note the additional word "network"!), nor
* "Bridged Adapter" (to WLAN)


== Start the newly created Virtual Machine ==
== Internet access ==


# This will install Proxmox from the DVD; pick the drive you created
The easiest way to give your Proxmox VE node access to the Internet when it is running as a guest within VirtualBox is to use VirtualBox's ''NAT'' networking mode. (This is not the same as the ''NAT network'' networking mode!) For example, configure the VirtualBox VM's "Adapter 1" to use the NAT networking mode and boot the Proxmox VE guest.
# At the last step in PVE set up you will be asked for an IP address. Choose a static address in the IPv4 Network.
[[File:Proxmox_in_Virtualbox_Network_Setup.png]]
# Once the installer has finished, eject the ISO from the virtual DVD drive
# Reboot


== Start the installed Proxmox Virtual Machine ==
When Proxmox starts, it will recognize its physical NIC and receive an IP address from VirtualBox's internal DHCP service. Usually, this is something like <tt>10.0.x.x</tt> (for example: <code>10.0.2.15</code>). Your Proxmox VE node is now on its own, totally independent network, separated from VirtualBox host's network access. From here, you can configure any number of Proxmox VE guests as you normally would within Proxmox VE itself.


Now the virtual machine containing proxmox has started we need to reconfigure networking based on the static network independent of the WLAN settings.
However, since your Proxmox VE node is now behind a NAT router, you will not be able to access the  Proxmox VE Web GUI nor the SSH port unless you also configure the VirtualBox NAT service to forward traffic for these services to the correct ports. For details, see [https://www.virtualbox.org/manual/UserManual.html#natforward §6.3.1, "Configuring Port Forwarding with NAT", of the VirtualBox User Manual].


== Configure the Network ==
For example, if you forward host post <tt>8006</tt> to guest port <tt>8006</tt>, you can access the Proxmox VE Web GUI via a URL such as <tt>https://localhost:8006</tt> in a browser running on your VirtualBox host.


=== Create Host-Only Network in Virtualbox===
== VirtualBox guest to Proxmox guest communication requires VirtualBox NIC in promiscuous mode ==


This network is to permit traffic from the laptop to the PVE in Virtualbox.
If you plan to have Proxmox VE host its own (nested) virtual machines or containers that are attached to a VirtualBox virtual network (as opposed to running entirely within Proxmox VE's own internal bridged networking) and would like those guests to be able to communicate with the VirtualBox host or with other VirtualBox-hosted (as opposed to Proxmox VE-hosted) virtual machines, you should [https://forum.proxmox.com/threads/why-is-this-simple-bridge-network-not-receiving-arp-replies-correctly.75695/#post-337550 ensure that the relevant VirtualBox virtual network adapter is permitted to use promiscuous mode].


In Virtualbox, there may be a Host-Only network already configured. This is set up at the Preferences > Network > Host-only Networks tab.
This is not the default setting! Rather, by default, VirtualBox denies its guest VMs from effectively enabling promiscuous mode. (Its default setting is ''Deny''.) The setting is available from the Network tab in a VirtualBox VM Machine Settings window (Network &rarr; Adapter N [where N is the adapter number] &rarr; Advanced &rarr; Promiscuous Mode) or can be changed via command line with:
Each adapter has an IPv4 address + a IPv4 Network Mask, addresses can be served by a VirtualBox DHCP server, Proxmox PVE is best set up with a static address on the Host-Only network.


For example
<pre>
[[File:Virtualbox_Host-Only_network.png]]
vboxmanage modifyvm <uuid|vm_name> --nicpromiscN <deny|allow-vms|allow-all>
</pre>


Without this option changed away from ''Deny'', communication from a VirtualBox-hosted VM to a (nested) Proxmox VE-hosted VM or container will fail.


Check from the host machine:
If communication from a VirtualBox-hosted guest to a nested Proxmox VE-hosted guest still fails after making the above change, try restarting the virtualized Proxmox VE node itself.
  $ ifconfig
        vboxnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        ether 0a:00:27:00:00:00
        inet 172.241.0.100 netmask 0xffffff80 broadcast 172.241.0.127


== Changing the Default Network ==
= Install and Run Proxmox VE =


Proxmox default network is set up at the last stage of configuration to e.g. IP address 172.241.0.101. The WEB GUI then runs on port 8006 answering to https.
The following instructions describe how to configure an example, simple, single virtualized Proxmox VE node within VirtualBox that is known to work reliably:


During installation PVE's Default Network is set to eth0 on bridge adapter vmbr0. You will need to change this so that the NAT network will be connected to the first virtual NIC.  
1. Create a new VirtualBox Virtual Machine for Proxmox


The result you want is to use eth1 / vmbr0. You can this adapt in WEB GUI after installation. Here's how:
Sample settings:
* Name: Proxmox-vm
* OS Type: Linux
* Version: Debian (64 bit)
* Memory: 6GB
* Disk: At least 8GB in size


We are going to change things around so that instead of all traffic being routed over the WLAN bridge (i.e. containers IP addresses appear on the WLAN),  internet is accessed from Proxmox and CTs via NAT, Proxmox to your laptop and vice versa goes by "Host only" network. Access from the laptop to CTs also goes via "Host Only", but gets an additional hop in virtual Proxmox host.  
2. Edit the settings
* Audio: disable
* Storage:
** IDE - Select Empty - click IDE Secondary - select the disk symbol, pick ISO for Proxmox
* Network:
** Adapter 1: Host-only Adapter, vboxnet0; recommended leave the advanced settings as they are, unless you plan to have VirtualBox-hosted VMs in communication with Proxmox VE-hosted guests, in which case be sure to allow Promiscuous Mode. (Paravirtualized Network (virtio-net)).
** Adapter 2: NAT (again, this is not the same as ''NAT network''!)


=== On your laptop (the host running Virtual Box), Define the Subnet for NAT ===
Notably, the following networking modes caused trouble historically, but you may have better luck these days:


On command line. (Assuming your machine's name is "proxmox-vm")
* "NAT network" (note the additional word "network"!), nor
* Create the Nat Subnet:
* "Bridged Adapter" (to WLAN)
 
VBoxManage modifyvm "proxmox-vm" --natnet1 "192.168.11.0/24"
 
 
In VirtualBox' Machine Settings:
* Set Adapter 1 to "NAT"
 
=== Give the laptop a route to the CTs ===
 
route add -net 192.168.9.0/24 gw 192.168.4.2
 
=== Configure Proxmox to talk to both networks ===
 
==== In PVE, define a new Network Linux Bridge ====
 
This is going to bridge (send & receive) traffic that arrives on eth0 so that it looks like it originates from the NAT Subnet address (in this case 192.168.11.15) and sent to the default gateway (in this case 192.168.11.2)
 
[[File:Create_Linux_Bridge_for_Virtual_Box_Private_Network.png]]
 
Assuming the NIC connected to "NAT" is eth0.
 
* Make a bridge called vmbr1
 
* Bridge eth0 to it
 
* Assign an address from the NAT subnet to it, e.g. 192.168.11.15
 
* Set default gateway to "2" in the NAT subnet, e.g. 192.168.11.2
 
In summary, the PVE virtual machine In Virtual Box:
* eth0 = "Adapter 1" connected with NAT - address 192.168.11.15/24, gateway 192.168.11.2, DNS 192.168.11.3
* eth1 = "Adapter 2" connected with "Host only" - address 192.168.4.2/24
* vmbr1 (not bridged to any NIC in virtual Proxmox host) - address 192.168.9.1/24
 
=== In the OpenVZ Containers ===
* venet0 address 192.168.11.16, 17 etc.
* veth NIC bridged to vmbr1 address 192.168.9.2,3 etc
* route to physical PC with
route add -net 192.168.4.0/24 gw 192.168.9.1
 
 
 
=== Network in Containers or KVMs ===
 
In PVE's virtual machines you can achieve access to both networks by bridging your NAT subnet address to either vmbr0 (for Internal Network) or to direct IP addressing (venet0 ; for Containers only, don't set both networks with venet0!).
 
 
 
 
== Try it! ==
 
[[File:PVE_started_in_Virtualbox.png]]
 
https://172.241.0.101:8006/#v1:0:18:4::::::
 
== Test access ==
 
# From the laptop you should be able to ping the static ip address you assigned
 
$ ping 172.241.0.101
PING 172.241.0.101 (172.241.0.101): 56 data bytes
64 bytes from 172.241.0.101: icmp_seq=0 ttl=64 time=0.579 ms
64 bytes from 172.241.0.101: icmp_seq=1 ttl=64 time=0.710 ms
64 bytes from 172.241.0.101: icmp_seq=2 ttl=64 time=0.411 ms
 
 
# From the PVE machine you should be able to see the bridge
 
[[File:Network_from_within_PVE.png]]
 
# But because the network is host-only, only the laptop can see the VM containing PVE


== References ==
Then start the newly created Virtual Machine normally. You should see the Proxmox VE installer and can continue with installing Proxmox VE inside VirtualBox as you like.
* http://orestad-linux.se/proxmox/
* https://wiki.debian.org/NetworkConfiguration
* https://wiki.debian.org/BridgeNetworkConnections
* http://www.microhowto.info/howto/bridge_traffic_between_two_or_more_ethernet_interfaces_on_linux.html
* http://forum.proxmox.com/threads/5722-open-VZ-container-has-trouble-reaching-the-internet
* http://archive.gregk.me/2010/working-on-vm-server-while-offline/
* http://askubuntu.com/questions/293816/in-virtualbox-how-do-i-set-up-host-only-virtual-machines-that-can-access-the-in
* http://technology.amis.nl/2014/01/27/a-short-guide-to-networking-in-virtual-box-with-oracle-linux-inside/
* http://translate.google.com/translate?hl=en&sl=de&u=https://forum.hosteurope.de/viewtopic.php%3Ff%3D64%26t%3D12125&prev=/search%3Fq%3D%2522route%2B-A%2Binet%2522%2Bproxmox%26safe%3Doff%26client%3Dsafari%26rls%3Den%26biw%3D1177%26bih%3D761
* http://forum.proxmox.com/threads/3440-Venet-Interface-some-problem
* http://www.rackspace.com/blog/vms-vlans-and-bridges-oh-my-part-1/

Latest revision as of 02:02, 15 September 2020

Running Proxmox VE inside VirtualBox is possible but is not recommended for production use. Virtualizing the Proxmox VE hypervisor inside VirtualBox is usually done for development and testing purposes. For example, wanting to use VirtualBox as a host for Proxmox on a laptop is a common wish but can be tricky to get right. This page covers special considerations necessary for Proxmox VE to function as expected when it is being hosted by the VirtualBox hypervisor.

Unless otherwise stated, this page assumes you are using:

  • VirtualBox 4.3.16 or higher.
  • Proxmox VE 4.x or higher (i.e. for lxc containers only, OpenVz is not supported any more)

Prerequisites

For Proxmox VE inside VirtualBox to work well, it's strongly recommended that your VirtualBox host (e.g., your laptop) meets the minimum hardware requirements and software configuration. Otherwise, Proxmox VE may still work but it can be extremely slow.

  • VirtualBox should support hardware virtualization acceleration. This means the feature is activated in EFI/BIOS. If you don't see this option in your EFI/BIOS screens, you should double-check to see if your CPU supports it.
  • Further, if you plan to host any guests within Proxmox and are running VirtualBox 6.1 or later, you should enable nested hardware virtualization acceleration in the VirtualBox settings for the Proxmox VE virtual machine itself. The setting is in the VirtualBox Machine Processor Tab (System → Processor → Enable Nested VT-x/AMD-V).
  • When creating the VirtualBox VM to host Proxmox VE, you should set its OS type to be run as Linux: Debian (64-bit) guest.

Nested Containers and simple Virtual Machines

You can run LXC Containers in the virtualized Proxmox VE without any problems. It's also possible to create and configure fully nested virtual machines, so long as you enable VirtualBox's supported for nested VT-X/AMD-V hardware instructions, noted above. Nested virtual machines will run fairly well if they don't have any graphical interface and don't consume too many CPU resources. (This was tested with, e.g., slax-router.) Do be careful, though. If your nested VMs require too many resources, they will freeze up and can block your guest system too!

Also, note that some VirtIO drivers are known to make problems if nested in a VirtualBox environment, although this seems to have gotten more reliable with later version of VirtualBox (specifically 6.1 and higher). For this reason, it's recommend to use e1000 for the nested VM's network adapters if you can, especially since the improved performance offered by the virtio driver is not typically required in a nested scenario such as this.

Further, when using earlier VirtualBox versions, KVM hardware virtualization does not work and must not be set! The reason is that with these VirtualBox versions, nested hardware virtualization support is not possible. Nested virtualization became possible starting in VirtualBox version 6.1.

Network Considerations

Most VirtualBox networking modes will work as expected when running Proxmox VE as a guest within VirtualBox. However, there are a few things to consider specially.

Internet access

The easiest way to give your Proxmox VE node access to the Internet when it is running as a guest within VirtualBox is to use VirtualBox's NAT networking mode. (This is not the same as the NAT network networking mode!) For example, configure the VirtualBox VM's "Adapter 1" to use the NAT networking mode and boot the Proxmox VE guest.

When Proxmox starts, it will recognize its physical NIC and receive an IP address from VirtualBox's internal DHCP service. Usually, this is something like 10.0.x.x (for example: 10.0.2.15). Your Proxmox VE node is now on its own, totally independent network, separated from VirtualBox host's network access. From here, you can configure any number of Proxmox VE guests as you normally would within Proxmox VE itself.

However, since your Proxmox VE node is now behind a NAT router, you will not be able to access the Proxmox VE Web GUI nor the SSH port unless you also configure the VirtualBox NAT service to forward traffic for these services to the correct ports. For details, see §6.3.1, "Configuring Port Forwarding with NAT", of the VirtualBox User Manual.

For example, if you forward host post 8006 to guest port 8006, you can access the Proxmox VE Web GUI via a URL such as https://localhost:8006 in a browser running on your VirtualBox host.

VirtualBox guest to Proxmox guest communication requires VirtualBox NIC in promiscuous mode

If you plan to have Proxmox VE host its own (nested) virtual machines or containers that are attached to a VirtualBox virtual network (as opposed to running entirely within Proxmox VE's own internal bridged networking) and would like those guests to be able to communicate with the VirtualBox host or with other VirtualBox-hosted (as opposed to Proxmox VE-hosted) virtual machines, you should ensure that the relevant VirtualBox virtual network adapter is permitted to use promiscuous mode.

This is not the default setting! Rather, by default, VirtualBox denies its guest VMs from effectively enabling promiscuous mode. (Its default setting is Deny.) The setting is available from the Network tab in a VirtualBox VM Machine Settings window (Network → Adapter N [where N is the adapter number] → Advanced → Promiscuous Mode) or can be changed via command line with: 

vboxmanage modifyvm <uuid|vm_name> --nicpromiscN <deny|allow-vms|allow-all>

Without this option changed away from Deny, communication from a VirtualBox-hosted VM to a (nested) Proxmox VE-hosted VM or container will fail.

If communication from a VirtualBox-hosted guest to a nested Proxmox VE-hosted guest still fails after making the above change, try restarting the virtualized Proxmox VE node itself.

Install and Run Proxmox VE

The following instructions describe how to configure an example, simple, single virtualized Proxmox VE node within VirtualBox that is known to work reliably:

1. Create a new VirtualBox Virtual Machine for Proxmox

Sample settings:

  • Name: Proxmox-vm
  • OS Type: Linux
  • Version: Debian (64 bit)
  • Memory: 6GB
  • Disk: At least 8GB in size

2. Edit the settings

  • Audio: disable
  • Storage:
    • IDE - Select Empty - click IDE Secondary - select the disk symbol, pick ISO for Proxmox
  • Network:
    • Adapter 1: Host-only Adapter, vboxnet0; recommended leave the advanced settings as they are, unless you plan to have VirtualBox-hosted VMs in communication with Proxmox VE-hosted guests, in which case be sure to allow Promiscuous Mode. (Paravirtualized Network (virtio-net)).
    • Adapter 2: NAT (again, this is not the same as NAT network!)

Notably, the following networking modes caused trouble historically, but you may have better luck these days:

  • "NAT network" (note the additional word "network"!), nor
  • "Bridged Adapter" (to WLAN)

Then start the newly created Virtual Machine normally. You should see the Proxmox VE installer and can continue with installing Proxmox VE inside VirtualBox as you like.

Retrieved from "https://pve.proxmox.com/mediawiki/index.php?title=Proxmox_VE_inside_VirtualBox&oldid=10832"