Cloud-Init is the defacto multi-distribution package that handles early initialization of a virtual machine instance. Using Cloud-Init, configuration of network devices and ssh keys on the hypervisor side is possible. When the VM starts for the first time, the Cloud-Init software inside the VM will apply those settings.
Many Linux distributions provide ready-to-use Cloud-Init images, mostly designed for OpenStack. These images will also work with Proxmox VE. While it may seem convenient to get such ready-to-use images, we usually recommended to prepare the images by yourself. The advantage is that you will know exactly what you have installed, and this helps you later to easily customize the image for your needs.
Once you have created such a Cloud-Init image we recommend to convert it into a VM template. From a VM template you can quickly create linked clones, so this is a fast method to roll out new VM instances. You just need to configure the network (and maybe the ssh keys) before you start the new VM.
We recommend using SSH key-based authentication to login to the VMs provisioned by Cloud-Init. It is also possible to set a password, but this is not as safe as using SSH key-based authentication because Proxmox VE needs to store an encrypted version of that password inside the Cloud-Init data.
Proxmox VE generates an ISO image to pass the Cloud-Init data to the VM. For that purpose all Cloud-Init VMs need to have an assigned CDROM drive. Also many Cloud-Init images assume to have a serial console, so it is recommended to add a serial console and use it as display for those VMs.
Preparing Cloud-Init Templates
The first step is to prepare your VM. Basically you can use any VM. Simply install the Cloud-Init packages inside the VM that you want to prepare. On Debian/Ubuntu based systems this is as simple as:
apt-get install cloud-init
Already many distributions provide ready-to-use Cloud-Init images (provided as .qcow2 files), so alternatively you can simply download and import such images. For the following example, we will use the cloud image provided by Ubuntu at https://cloud-images.ubuntu.com.
# download the image wget https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img # create a new VM qm create 9000 --memory 2048 --net0 virtio,bridge=vmbr0 # import the downloaded disk to local-lvm storage qm importdisk 9000 bionic-server-cloudimg-amd64.img local-lvm # finally attach the new disk to the VM as scsi drive qm set 9000 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-9000-disk-1
|Ubuntu Cloud-Init images require the virtio-scsi-pci controller type for SCSI drives.|
Add Cloud-Init CDROM drive
The next step is to configure a CDROM drive which will be used to pass the Cloud-Init data to the VM.
qm set 9000 --ide2 local-lvm:cloudinit
To be able to boot directly from the Cloud-Init image, set the bootdisk parameter to scsi0, and restrict BIOS to boot from disk only. This will speed up booting, because VM BIOS skips the testing for a bootable CDROM.
qm set 9000 --boot c --bootdisk scsi0
Also configure a serial console and use it as a display. Many Cloud-Init images rely on this, as it is an requirement for OpenStack images.
qm set 9000 --serial0 socket --vga serial0
In a last step, it is helpful to convert the VM into a template. From this template you can then quickly create linked clones. The deployment from VM templates is much faster than creating a full clone (copy).
qm template 9000
Deploying Cloud-Init Templates
qm clone 9000 123 --name ubuntu2
Then configure the SSH public key used for authentication, and configure the IP setup:
qm set 123 --sshkey ~/.ssh/id_rsa.pub qm set 123 --ipconfig0 ip=10.0.10.123/24,gw=10.0.10.1
You can also configure all the Cloud-Init options using a single command only. We have simply splitted the above example to separate the commands for reducing the line length. Also make sure to adopt the IP setup for your specific environment.
Custom Cloud-Init Configuration
The Cloud-Init integration also allows custom config files to be used instead of the automatically generated configs. This is done via the cicustom option on the command line:
qm set 9000 --cicustom "user=<volume>,network=<volume>,meta=<volume>"
The custom config files have to be on a storage that supports snippets and have to be available on all nodes the VM is going to be migrated to. Otherwise the VM won’t be able to start. For example:
qm set 9000 --cicustom "user=local:snippets/userconfig.yaml"
There are three kinds of configs for Cloud-Init. The first one is the user config as seen in the example above. The second is the network config and the third the meta config. They can all be specified together or mixed and matched however needed. The automatically generated config will be used for any that don’t have a custom config file specified.
The generated config can be dumped to serve as a base for custom configs:
qm cloudinit dump 9000 user
The same command exists for network and meta.
Cloud-Init specific Options
- cicustom: [meta=<volume>] [,network=<volume>] [,user=<volume>]
Specify custom files to replace the automatically generated ones at start.
Specify a custom file containing all meta data passed to the VM via cloud-init. This is provider specific meaning configdrive2 and nocloud differ.
Specify a custom file containing all network data passed to the VM via cloud-init.
Specify a custom file containing all user data passed to the VM via cloud-init.
- cipassword: <string>
Password to assign the user. Using this is generally not recommended. Use ssh keys instead. Also note that older cloud-init versions do not support hashed passwords.
- citype: <configdrive2 | nocloud>
Specifies the cloud-init configuration format. The default depends on the configured operating system type (ostype. We use the nocloud format for Linux, and configdrive2 for windows.
- ciuser: <string>
User name to change ssh keys and password for instead of the image’s configured default user.
- ipconfig[n]: [gw=<GatewayIPv4>] [,gw6=<GatewayIPv6>] [,ip=<IPv4Format/CIDR>] [,ip6=<IPv6Format/CIDR>]
Specify IP addresses and gateways for the corresponding interface.
IP addresses use CIDR notation, gateways are optional but need an IP of the same type specified.
The special string dhcp can be used for IP addresses to use DHCP, in which case no explicit gateway should be provided. For IPv6 the special string auto can be used to use stateless autoconfiguration.
If cloud-init is enabled and neither an IPv4 nor an IPv6 address is specified, it defaults to using dhcp on IPv4.
Default gateway for IPv4 traffic.
Requires option(s): ip
Default gateway for IPv6 traffic.
Requires option(s): ip6
- ip=<IPv4Format/CIDR> (default = dhcp)
IPv4 address in CIDR format.
- ip6=<IPv6Format/CIDR> (default = dhcp)
IPv6 address in CIDR format.
- nameserver: <string>
Sets DNS server IP address for a container. Create will automatically use the setting from the host if neither searchdomain nor nameserver are set.
- searchdomain: <string>
Sets DNS search domains for a container. Create will automatically use the setting from the host if neither searchdomain nor nameserver are set.
- sshkeys: <string>
Setup public SSH keys (one key per line, OpenSSH format).