Cloud-Init FAQ

What is cloud-init?

Cloud-init is basically a program which runs in a guest machine on boot, which search for configuration data to apply to the guest system during initialization.

The idea is to have a defined OS independent configuration format for options common to many systems (such as the hostname and networking configuration). In practice this doesn't always work very well, so there are a few quirks and issues you should be aware of, as we will not be adding OS and cloud-init-version specific workarounds, as that would defeat the whole point of this system.

See the Cloud-Init Support page of the reference documentation for detailed command line examples.

Usage in PVE

In PVE cloud-init images are attached as ISO images via a virtual CDROM drive. In order to start using the cloud-init tab in the Web UI, you'll have to add such a drive. This can be done in the Hardware tab, by clicking the Add button and selecting CloudInit Drive. Afterwards, the options in the cloud-init tab become usable.

Options

User

Cloud Images are often setup to setup a user's password and/or ssh keys. This is usually done for a default user often named after the distribution running. This option may be used to change which user to setup. Using this should create a user of the specified name and apply the password and ssh key settings.

Password

The password option should be straightforward. Take note however, that before cloud-init version 17.1, only plaintext passwords were supported. Our API by default pre-hashes passwords sent as plaintext, so to set a password on older cloud images, the plaintext passwords would need to be manually written into the VM config in /etc/pve/qemu-server/${vmid}.conf

SSH public key

This can be one or multiple SSH public keys setup for the above configured user. When loading them from a file, however, note that PVE currently will not accept comment lines.

DNS domain and DNS server

These work similar to their container counterparts, and similarly multiple DNS servers can be specified as a whitespace separated list.

IP Config (ethX)

Much like with containers, this will configure the guest's IP address. There should be one such entry per network interface. Note that the way the network configuration is passed to the guest can be quite different depending on what type of cloud-init image is being used. This can be changed via the citype option. See below.

citype (CLI only)

Currently PVE supports 2 types of cloud-init images: nocloud v1 (the default and recommended type), and configdrive v2. Most images of major distributions should fork fine with the default. In the future, nocloud v2 may be supported as well which should have a more robust way to configure network devices as it does not depend on specific device names (see the Networking section below).

The configdrive v2 variant may work for windows guests using cloudbase-init, but this is not officially supported.

Quirks and Requirements for cloud init images.

Ideally the cloud image should be running an up-to-date cloud-init version, at least 18.2 or newer. There are several issues with older versions mentioned below.

Networking

For legacy reasons the network device names of cloud images are required to use the "old" network device names: (eth0, eth1, ...). This is because older cloud-init configuration formats made the same mistake as everyone else before, matching interfaces by name with no other option. Newer cloud-init versions support netplan-like configurations where you can match by mac address or driver, but these aren't widely available in the current cloud-init images. (At the time of writing, only the ubuntu bionic images supported this.).

The newer nocloud v2 config format will be supported in the future, where this requirement is lifted.

Note that (at the time of writing) the cloud-init version shipped with centos 7 images does cannot handle CIDR notation for IPv4 addresses which we use in our PVE 5.2 release version.

Hostname

Before cloud-init version 18.2, the host name was not set before DHCP requests, meaning the image's default host name is being used at that point, leading to hostname clashes.

Creating a custom cloud image

Step 1: Install the base system

This can be any system for which you can install a working cloud-init package. Note that if you create this image yourself, it would be a good idea to try to install a more recent version of cloud-init than what you find in various upstream distribution repositories.

Step 2: System preparation

Rename network devices

For best compatibility, specifically when using older cloud-init versions, you'll want to setup udev rules / systemd-networkd configuration/link files etc. to revert from predictable device names to kernel names (eth0 and so on). See the Predictable Network Interface Names <https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/> chapter on freedesktop.org.

Add a default user

Typically when using cloud-init you'll log into a predefined user other than root, although this is optional. The user name can be configured in the cloud-init package, and overridden by PVE's cloud-init configuration options.

Install standard packages

Since you usually won't be logging in as root and get a preconfigured user setup, you'll want to install sudo in order to be able to perform administrative tasks later on. Besides setting a user password, you can also provide ssh keys deployed by cloud-init on boot, so you most likely also want to setup an ssh server and perform some initial configuration on it (such as choosing an alternate port, disabling root access, limiting ciphers, etc.)

Setup a serial terminal

Many cloud images expect a serial line to be available. Kernel messages and a loging prompt are often configured for ttyS0.

If you're going to boot with grub, you can use the following configuration:

# /etc/default/grub
(...)
GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0"
(...)

After changing this, you'll have to recreate your grub configuration (eg. update-grub).

Alternatively you can configure your inittab appropriately if you have one, or run the equivalent of systemctl enable getty@ttyS0.service.

Clean up the image

After setting up the default user and sudo, you may want to remove the root password (and any others you don't want to include in the template) to not have some default root password in a template deployed in various locations.

Step 3: Install and configure cloud-init

Once everything is prepared, the last step is to install and configure the cloud-init package. (apt install cloud-init, yum install cloud-init and so on.).

Configure data sources

PVE currently supports the nocloud-v1 and configdrive-v2 data sources. If you're only using the template with PVE you can use the following configuration snipppet as a boot-time optimization:

# /etc/cloud/cloud.cfg.d/99_pve.cfg
datasource_list: [ NoCloud, ConfigDrive ]

Set the default user name

The default user is usually found in /etc/cloud/cloud.cfg's system_info section under default_user. You can change its name property. Distributions often use the distribution's name in lower case as the default user name there. Eg. on Ubuntu you'll find the following section:

# /etc/cloud/cloud.cfg
(...)
system_info:
    default_user:
        name: ubuntu
(...)

There are various additional options you can add here (such as what kind of line should be added to the sudoers file). For more detailed information see the upstream cloud-init documentation.

Troubleshooting

Once your template is ready, you'll want to test it. If you chose to cleanup the login data and the user setup doesn't work for some reason, you may have just created a system you cannot log into. You can use all the usual ways to try to debug this, but here are some useful reminders:

• For a systemd based distribution you can use grub's 'edit' mode to add systemd.debug-shell=1 to the kernel command line, giving you a root shell on Ctrl+Alt+F9 early on.
• Attaching a live iso to the VM allows you to chroot into it and read the syslogs / journal of the previous boot attempts (journalctl -b-1).
• Since cloud-init is often setup to disable root login, a temporary debug user with sudo access can be added before setting up cloud-init which you can remove or disable later on.

See Also

• Cloud-Init Support