Automated Installation

From Proxmox VE
Revision as of 11:54, 21 November 2024 by C.heiss (talk | contribs) (sync 8.3 changes)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This article documents the automated installation process of Proxmox projects like Proxmox VE.

Introduction

The automated installation method allows installing a Proxmox solution in an unattended manner. This enables you to fully automate the setup process on bare-metal. Once the installation is complete and the host has booted up, automation tools like Ansible can be used to further configure the installation.

The necessary options for the installer must be provided in an answer file. This file allows using filter rules to determine which disks and network cards should be used.

To use the automated installation, it is first necessary to choose a source from which the answer file is fetched from and then prepare an installation ISO with that choice.

Once the ISO is prepared, its initial boot menu will show a new boot entry named "Automated Installation" which gets automatically selected after a 10-second timeout.

Overview

Assistant Tool

The proxmox-auto-install-assistant tool provides the prepare-iso sub-command which can be used to prepare a new enough, but otherwise standard ISO of a Proxmox project for automated installation. You will have to install it first: 

apt install proxmox-auto-install-assistant
Yellowpin.svg Note: The xorriso binary is required for preparing an ISO. On Debian-based systems, you can install it using apt install xorriso.

See the help of the sub-command has more details: proxmox-auto-install-assistant prepare-iso --help.

Answer File Format

The answer file is a TOML-formatted configuration file that provides the basic configuration for a system, like the root password, the network configuration and the target root disk.

To allow the installation of systems with (for example) many disks and network devices, you can use filters to match on device properties, such as serial numbers, vendor, or other unique details like the MAC address of a network interface.

For more details about the schema and the possible filters, see the Answer File Format section.

Answer File Source

There are several locations where the automatic installer can fetch an answer file from:

  • From the ISO directly
  • Read from a separate partition, identified by the partition label
  • Fetched through HTTP from the network
    • The source URL can be pre-determined or queried from DNS or DHCP

For now, you must choose exactly one source.

Prepare an Installation ISO

See the following for a more detailed description of the possible sources and examples on how to prepare a ISO with the different fetch-from modes.

Answer included in the ISO

It is possible to prepare an ISO so that it includes an answer.toml directly, allowing you to have a unified medium for rolling out automatic installation. 

proxmox-auto-install-assistant prepare-iso /path/to/source.iso --fetch-from iso --answer-file /path/to/answer.toml

Answer on Separate Partition

The ISO can also be prepared to search for an answer file called answer.toml on a separate partition/file system (for example, on a USB flash drive). In this mode, the automatic installer searches for a partition with a specific name, by default either proxmox-ais or PROXMOX-AIS (Automated Installation Source). 

proxmox-auto-install-assistant prepare-iso /path/to/source.iso --fetch-from partition

The partition label for which the automatic installer should search for can be customized by passing the --partition-label parameter to proxmox-auto-install-assistant. 

proxmox-auto-install-assistant prepare-iso /path/to/source.iso --fetch-from partition --partition-label YOURLABEL

Afterward, prepare the USB flash drive, for example on /dev/sdX1. You may then adapt and run the following commands as root: 

# format as new vfat file system (DESTRUCTIVE!)
mkfs.vfat /dev/sdX1

# set the label
fatlabel /dev/sdX1 "PROXMOX-AIS" # or "YOURLABEL"

# mount the USB pen drive 
mkdir /mnt/usb
mount /dev/sdX1 /mnt/usb

# copy over the answer file as "answer.toml"
cp my-prepared-answer.toml /mnt/usb/answer.toml

# unmount the USB pen drive
sync
umount /mnt/usb

Answer Fetched via HTTP

To fetch the answer file via HTTP(S), the installer needs to know the URL. The URL may be provided via the following ways:

  • URL defined in the ISO
  • as DHCP option (250)
  • via a DNS TXT record
    • The DNS TXT record needs to be located at proxmox-auto-installer.{search domain}, where {search domain} is the search domain provided by the DHCP server.
Yellowpin.svg Note: Fetching the fingerprint via DHCP or DNS records is only done if the same method is used to retrieve the URL!

The HTTP(S) POST request sends JSON data that can help to identify the physical machine and use that information to generate a custom answer file. A full example of the JSON data can be found under the code listing at System information POST data.

Certificate Fingerprint Matching

It is possible to provide the 'SHA256' certificate fingerprint of the TLS certificate. This is useful in the following situations:

  • the URL is using an IP address instead of a FQDN
  • a self-signed certificate is used which the installer does not trust
  • added security by pinning the known certificate of the server

There are three ways to provide the 'SHA256' fingerprint to the installer:

  • Fingerprint defined in the ISO
  • as a DHCP option (251)
  • via a DNS TXT record
    • The DNS TXT record must be located at proxmox-auto-installer-cert-fingerprint.{search domain}, where {search domain} is the search domain provided by the DHCP server.
Yellowpin.svg Note: Fetching the fingerprint via DHCP or DNS records is only done if the same method is used to retrieve the URL!

Example

Similarly, it is possible to specify the URL from which the installer should fetch an answer file as well as the TLS certificate fingerprint. For example, to specify both: 

proxmox-auto-install-assistant prepare-iso /path/to/source.iso --fetch-from http --url "https://10.0.0.100/get_answer/" --cert-fingerprint "04:42:97:27:F6:29:2F:9F:3D:7F:13:11:C8:E2:F5:5F:84:03:95:D9:F5:14:72:7C:9E:90:47:03:D2:96:2B:EC"

Answer File Format

The answer file is expected in TOML format.

The following example shows an answer file with all possible sections. It uses the DHCP-provided network settings and will use ZFS in a RAID-1 on disks sda and sdb. It also has a post installation webhook and first-boot hook configured: 

[global]
keyboard = "de"
country = "at"
fqdn = "pveauto.testinstall"
mailto = "mail@no.invalid"
timezone = "Europe/Vienna"
root_password = "123456"
root_ssh_keys = [
    "ssh-ed25519 AAAA..."
]

[network]
source = "from-dhcp"

[disk-setup]
filesystem = "zfs"
zfs.raid = "raid1"
disk_list = ["sda", "sdb"]

[post-installation-webhook]
url = "https://my.endpoint.local/postinst"
cert_fingerprint = "AA:E8:CB:95:B1:..."

[first-boot]
source = "from-iso"
ordering = "before-network"

Global Section

This section contains the following keys:

  • keyboard -- The keyboard layout with the following possible options:
 de de-ch dk en-gb en-us es fi fr fr-be fr-ca fr-ch hu is it jp lt mk nl no pl pt pt-br se si tr
  • country -- The country code in the two letter variant. For example, at, us or fr.
  • fqdn -- The fully qualified domain name of the host. The domain part will be used as the search domain.
  • mailto -- The default email address for the user root.
  • timezone -- The timezone in tzdata format. For example, Europe/Vienna or America/New_York.
  • root_password -- The password for the root user.
  • root_password_hashed -- The pre-hashed password for the root user, which will be written verbatim to /etc/passwd. May be used instead of root_password and can be generated using the mkpasswd tool, for example.
  • root_ssh_keys -- Optional. SSH public keys to add to the authorized_keys file of the root user after the installation.
  • reboot_on_error -- Optional. If set to true, the installer will reboot automatically when an error is encountered. Defaults to false, giving the administrator a chance to investigate why an installation failed.

Either root_password or root_password_hashed must be set. Setting none or both will result in a validation error.

Network Section

This section contains the following keys:

  • source -- Where to source the static network configuration from. This can be from-dhcp or from-answer.
    If set to from-dhcp the other network options must not be set. The installer will then use the active NIC and the DHCP settings received during installation to write out a static network configuration.
  • cidr -- The IP address in CIDR notation. For example, 192.168.1.10/24
  • dns -- The IP address of the DNS server.
  • gateway -- The IP address of the default gateway.
  • filter -- Filter against the UDEV properties to select the network card. See filters.

Disk Setup Section

This section contains the following keys:

  • filesystem -- One of the following options: ext4, xfs, zfs, or btrfs. btrfs is only available for Proxmox VE installations.
  • disk_list -- List of disks to use. Useful if you are sure about the disk names. For example: disk_list = ["sda", "sdb"]
  • filter -- Filter against UDEV properties to select the disks for the installation. See filters.
Yellowpin.svg Note: Use either disk_list or filter. Defining both is not allowed.
  • filter_match -- Can be "any" or "all". Decides if a match of any filter is enough of if all filters need to match for a disk to be selected. The default is "any".
  • zfs -- Defines ZFS-specific properties. See ZFS Advanced Options of our documentation. The properties are:
    •  raid -- The RAID level that should be used. Options are raid0, raid1, raid10, raidz-1, raidz-2, or raidz-3.
    •  ashift -- Optional. Specifies the ashift property of the created zpool.
    •  arc_max -- Optional. Specifies the maximum amount of memory in MiB ZFS may use for its ARC in. See also Limit ZFS Memory Usage.
    • checksum -- Optional. Specifies the checksumming algorithm. Options are on (default), fletcher4 and sha256.
    •  compress -- Optional. Specifies whether compression is enabled and if yes, what algorithm to use. Options are on (default), off, lzjb, lz4, zle, gzip and zstd.
    •  copies -- Optional. Specifies the copies>property of the created zpool. See zfsprops.7 for more information.
    •  hdsize -- Optional. Defines the total hard disk size to be used in GB. Only honored for bootable disks, that is only the first disk or mirror for RAID0, RAID1 or RAID10, and all disks in RAID-Z[123].
  • lvm -- Advanced properties that can be used with the ext4 or xfs file system. See LVM Advanced Options. The properties are:
    •  hdsize -- Optional. Specifies the total hard disk size to be used in GB. It can be used to reserve free space on the hard disk for further partitioning after the installation.
    •  swapsize -- Optional. Specifies the size of the swap volume in GB. Default is the size of installed memory, clamped to between 4 GB and 8 GB.
    •  maxroot -- Optional. Specifies the maximum size of the root volume in GB. Maximum is hdsize / 4.
    •  maxvz -- Optional. Specifies the maximum size of the data volume in GB.
    •  minfree -- Optional. Specifies the amount of free space that should be left in the LVM volume group.
  • btrfs -- Defines BTRFS specific options.
Yellowpin.svg Note: Only available on Proxmox VE installations. When using the HTTP method, it can also be dynamically determined from the system information data using the "product"."enable_btrfs" key.
    •  raid -- The RAID level that should be used. Options are raid0, raid1, and raid10.
    •  hdsize -- Optional. Specifies the total hard disk size to be used in GB.
    • compress -- The compression type to use. Possible options are on, off, zlib, lzo and zstd. Defaults to off. See also the btrfs(5) manpage.

Post Installation Webhook Section

Optional. It can be used to configure a webhook to be called after a successful installation. See Post-installation notification for more information.

This section contains the following keys:

  • url -- The URL the information about the installed system should be sent to as HTTP POST request.
  • cert_fingerprint -- Optional. SHA256 certificate fingerprint if certificate pinning should be used.

First Boot Hook Section

Optional. It can be used to configure a script to run on the first boot of the new system after a successful installation.

If configured, this installs an additional package named proxmox-first-boot. After booting the new system for the first time, this package can safely be removed using apt purge proxmox-first-boot.

This section contains the following keys:

  • source -- Where to source the executable for running at first boot from. It can either be from-iso or from-url.
    When using from-iso, the executable must be specified when preparing the ISO using proxmox-auto-install-assistant prepare-iso --on-first-boot path/to/file.sh.
  • ordering -- Optional. At what stage of the boot to run the hook. It can be one of before-network, network-online or fully-up.
    - before-network -- Runs before any networking devices are set up.
    - network-online -- Runs after network connectivity has been established. See also network-online.target.
    - fully-up -- Default. The system is fully booted and ready for normal operation. Project-dependent APIs will also be available. For example, on Proxmox VE tools like qm, pct and pvesh can be used.
  • url -- Required when source = "from-url". The URL of the executable file to download.
  • cert-fingerprint -- Optional. SHA256 certificate fingerprint if certificate pinning should be used for the download of the executable file.
Yellowpin.svg Note: The maximum executable file size is 1 MiB, for both integrating it into the ISO and fetching it from a URL.

Answer File Validation

The proxmox-auto-install-assistant tool can also be used to validate the syntax of an answer file and display the identifying information that will be sent to the HTTP(s) server when fetching the answer file.

For example, to validate an answer file: 

$ proxmox-auto-install-assistant validate-answer answer.toml 
The file was parsed successfully, no syntax errors found!

Filters

Filters allow you to match against device properties exposed by udevadm.

The proxmox-auto-install-assistant utility can display these properties and allows you to test filters in advance. The utility is available in the installer environment (via its debug mode) and on already existing Proxmox Virtual Environment installation.

For example, to fetch information about the available disks: 

$ proxmox-auto-install-assistant device-info -t disk
[…]
    "nvme1n1": {
      "CURRENT_TAGS": ":systemd:",
      "DEVLINKS": "/dev/disk/by-id/nvme-KIOXIA_KCMYXVUG1T60_9DUXXXXXXXX_1 /dev/disk/by-path/pci-0000:e2:00.0-nvme-1 /dev/disk/by-diskseq/12 /dev/disk/by-id/nvme-eui.01000000000000008ce38ee300708529 /dev/disk/by-id/nvme-KIOXIA_KCMYXVUG1T60_9DUXXXXXXXX",
      "DEVNAME": "/dev/nvme1n1",
      "DEVPATH": "/devices/virtual/nvme-subsystem/nvme-subsys1/nvme1n1",
      "DEVTYPE": "disk",
      "DISKSEQ": "12",
      "ID_MODEL": "KIOXIA KCMYXVUG1T60",
      "ID_NSID": "1",
      "ID_PART_TABLE_TYPE": "gpt",
      "ID_PART_TABLE_UUID": "539fabeb-aecd-6643-94a9-f28a68cfa12d",
      "ID_PATH": "pci-0000:e2:00.0-nvme-1",
      "ID_PATH_TAG": "pci-0000_e2_00_0-nvme-1",
      "ID_REVISION": "1UETE103",
      "ID_SERIAL": "KIOXIA_KCMYXVUG1T60_9DUXXXXXXXX_1",
      "ID_SERIAL_SHORT": "9DU0A02D0L33",
      "ID_WWN": "eui.01000000000000008ce38ee300708529",
      "MAJOR": "259",
      "MINOR": "5",
      "SUBSYSTEM": "block",
      "TAGS": ":systemd:",
      "USEC_INITIALIZED": "3169050"
    },
[…]

The key of the filter decides on which property it should be applied to. For example, in order to match against the vendor and model number of the disk, the filter in the answer file could look like this: 

filter.ID_SERIAL = "KIOXIA_KCMYXVUG1T60*"
Yellowpin.svg Note: The * globbing symbol at the end is used to match anything after the defined filter!

This will match for all Kioxia disks with that model number. You may verify which disks will be found by the filter by running the following command: 

$ proxmox-auto-install-assistant device-match disk ID_SERIAL='KIOXIA_KCMYXVUG1T60*'
[
  "nvme0n1",
  "nvme1n1",
  "nvme4n1",
  "nvme5n1"
]

The filter_match parameter controls whether all filters must apply, or if it is enough if any of the filters match. This makes it possible to use different disk models for the installation by using different properties.

For example: 

filter.ID_SERIAL = "KIOXIA*"
filter.ID_MODEL = "ATP*"
Yellowpin.svg Note: For network cards, only the first match will be used as the installer requires only one network card.

More complex network setups can be configured after the installation. Using properties with unique identifiers will result in the most predictable behavior (for example, the MAC address).

Filter Syntax

The following special characters can be used in filters:

  • ? -- matches any single character
  • * -- matches any number of characters, can be none
  • [a], [abc], [0-9] -- matches any single character inside the brackets, ranges are possible
  • [!a] -- negate the filter, any single character but the ones specified

Useful Properties

For network cards, the following properties can be useful:

  • ID_NET_NAME
  • ID_NET_NAME_MAC
  • ID_VENDOR_FROM_DATABASE
  • ID_MODEL_FROM_DATABASE

For disks, these properties can be useful:

  • DEVNAME
  • ID_SERIAL_SHORT
  • ID_WWN
  • ID_MODEL
  • ID_SERIAL

Post-installation notification

Optionally, a webhook can be configured to receive detailed information as JSON data about the new system after a successful installation. This includes disks, network interfaces, machine-id and SSH host keys, to uniquely identify machines. It can also be used to trigger additional automated setup deployment tools.

A full example of what such JSON data looks like can be found under the code listing at Post-installation webhook JSON example.

Meta schema information

The "$schema" object carries meta-information about the JSON document itself.

Currently, only one field is defined:

  • version -- Describes the version and thus structure of the document. Follows the format "<major>.<minor>" and applies semantic versioning meaning for both the major and minor number
    The major number will be increased on breaking changes, such as removing an existing field. The minor number will be increased for backwards-compatible changes, for example the addition of new fields.

Helper Tool

The proxmox-auto-install-assistant tool can be used to validate the syntax of an answer file and display the identifying information that will be sent to the HTTP(s) server when fetching the answer file.

For example, to validate an answer file: 

$ proxmox-auto-install-assistant validate-answer answer.toml 
The file was parsed successfully, no syntax errors found!

If the syntax of the answer file has errors, it will let us know: 

$ proxmox-auto-install-assistant validate-answer answer.toml 
Error parsing answer file: TOML parse error at line 12, column 1
   |
12 | [disk-setup]
   | ^^^^^^^^^^^^
Need either 'disk_list' or 'filter' set

To display the identifying information of the current machine: 

$ proxmox-auto-install-assistant system-info
{
  "product": "Not available. Would be one of the following: pve, pmg, pbs",
  "system": {
    "serial": "900030XXXX",
    "name": "RS700-E11-RS12U 1HE Intel Dual-CPU RI2112-ASXSN Server",
    "sku": "SKU",
    "uuid": "6fb36fd4-77a1-57e6-13ae-XXXXXXcb48d7"
  },
  "baseboard": {
    "name": "Z13PP-D32 Series",
    "asset_tag": "To be filled by O.E.M.",
    "serial": "23041870000XXXX"
  },
  "chassis": {
    "serial": "I02308XXXX",
    "asset_tag": "To be filled by O.E.M."
  },
  "mac_addresses": [
    "14:23:f2:14:08:20",
    "a0:36:bc:cb:7b:77",
    "14:23:f2:14:08:21",
    "e4:3d:1a:fa:37:98",
    "14:23:f2:14:08:22",
    "e4:3d:1a:fa:37:99",
    "14:23:f2:14:08:23",
    "a0:36:bc:cb:7b:78",
    "00:62:0b:d2:d7:50",
    "e4:3d:1a:fa:37:9a",
    "00:62:0b:d2:d7:51",
    "e4:3d:1a:fa:37:9b",
    "3e:7c:a7:f5:fb:ee",
    "e4:3d:1a:fa:37:9b",
    "2e:11:ed:d0:e3:98"
  ]
}

How useful the identifying information is depends mainly on the hardware vendor and how much of the information is configured correctly. In this example, you can see that some fields are still configured with placeholder values.


Examples

Answer Files

ZFS Mirror on Samsung Disks

We assume that there are only two Samsung disks present. These should be used for the OS in a ZFS Mirror (RAID-1). Additionally, we want to use only 150 GiB of the disks' capacities and leave the rest empty for further customization. 

[global]
keyboard = "de"
country = "at"
fqdn = "pveauto.testinstall"
mailto = "mail@no.invalid"
timezone = "Europe/Vienna"
root_password = "123456"

[network]
source = "from-dhcp"

[disk-setup]
filesystem = "zfs"
zfs.raid = "raid1"
zfs.hdsize = 150
filter.ID_MODEL = "Samsung*"

Ext4 With No Swap and Data LV on /dev/sda

Install on /dev/sda and define swap and data LVs with size 0. 

[global]
keyboard = "de"
country = "at"
fqdn = "pveauto.testinstall"
mailto = "mail@no.invalid"
timezone = "Europe/Vienna"
root_password = "123456"

[network]
source = "from-dhcp"

[disk-setup]
filesystem = "ext4"
lvm.swapsize = 0
lvm.maxvz = 0
disk_list = ['sda']

ZFS Mirror With Manual Network Config

[global]
keyboard = "de"
country = "at"
fqdn = "pveauto.testinstall"
mailto = "mail@no.invalid"
timezone = "Europe/Vienna"
root_password = "123456"

[network]
source = "from-answer"
cidr = "10.10.10.10/24"
dns = "10.10.10.1"
gateway = "10.10.10.1"
filter.ID_NET_NAME_MAC = "*e43d1afa379a"

[disk-setup]
filesystem = "zfs"
zfs.raid = "raid1"
filter.ID_MODEL = "Samsung*"
Yellowpin.svg Note: There is a * in the filter for the network card. This is necessary because that property usually has enx prefixed.

When displaying this property with proxmox-auto-install-assistant device-info -t network, its full value looks like this: 

"ID_NET_NAME_MAC": "enxe43d1afa379a",

Serving Answer Files via HTTP

Yellowpin.svg Note: These are merely examples! Please ensure that your connection is secured via TLS or that your network is trusted.

Setting up a reverse proxy in front that provides TLS for your server is a good idea.


Serving a Static Answer File via netcat

This variant is doing the bare minimum to provide a single answer file.

Ensure that you have netcat-traditional installed: 

apt update
apt install netcat-traditional

Create a directory in which the server will run and give it adequate permissions, for example: 

mkdir -p /srv/proxmox/auto-install-server
chmod 700 /srv/proxmox/auto-install-server

Place your answer.toml file in that directory.

You may then serve the file on https://[HOST IP]:8000 like this: 

cd /srv/proxmox/auto-install-server
while true; do cat <(printf "HTTP/1.1 200 OK\n\n") answer.toml | nc -l -q 0 -p 8000; done &

The process will run in the background.

Yellowpin.svg Note: Please ensure that your connection is secured by TLS through something like a reverse proxy.

To terminate the process, you can kill it via its job ID or its PID. For such background jobs, those can be listed via jobs -l.

For example, if the process's job ID is 1 and its PID is 371012, you can terminate it by running either kill %1 or kill 371012.


Serving a Static Answer File via Python

This option showcases how to use Python to serve a single answer file. It can be a starting point for your solution.

Ensure that you have python3-aiohttp installed: 

apt update
apt install python3-aiohttp

Create a directory in which the server will run and give it adequate permissions, for example: 

mkdir -p /srv/proxmox/auto-install-server
chmod 700 /srv/proxmox/auto-install-server

Place your answer.toml file in that directory.

Then save the following script as server.py in the server's directory as well: 

import logging

from aiohttp import web

routes = web.RouteTableDef()


@routes.post("/answer")
async def answer(request: web.Request):
    logging.info(f"Received request from peer '{request.remote}'")

    file_contents = app.get("answer_file", None)

    if file_contents is None:
        return web.Response(status=404, text="not found")

    return web.Response(text=file_contents)


if __name__ == "__main__":
    app = web.Application()

    with open("answer.toml") as answer_file:
        file_contents = answer_file.read()

    app["answer_file"] = file_contents

    logging.basicConfig(level=logging.INFO)

    app.add_routes(routes)
    web.run_app(app, host="0.0.0.0", port=8000)

The server's directory should now look like this: 

# tree -p /srv/proxmox/auto-install-server
[drwx------]  /srv/proxmox/auto-install-server
├── [-rw-r--r--]  answer.toml
└── [-rw-r--r--]  server.py

1 directory, 2 files

You may now serve the answer file on https://[HOST IP]:8000/answer by starting the server with Python: 

cd /srv/proxmox/auto-install-server
python3 server.py

The server will run in the foreground and can be terminated by hitting CTRL+C in the console.

Yellowpin.svg Note: Please ensure that your connection is secured by TLS through something like a reverse proxy.

Serving Answer Files Depending on MAC Address via Python

This is a slightly more advanced example that can dynamically serve answer files depending on the MAC address of the host that made the request. This is achieved by reading the JSON data from the HTTP POST request.

Ensure that you have python3-aiohttp and python3-tomlkit installed: 

apt update
apt install python3-aiohttp python3-tomlkit

Create a directory in which the server will run and give it adequate permissions, for example: 

mkdir -p /srv/proxmox/auto-install-server
chmod 700 /srv/proxmox/auto-install-server

Before you can run the server below, you must set up the following:

  • Place a file named default.toml in the server's directory.
This is the fallback answer file that will be used if no MAC address match was found.
  • Create a directory named answers in the server's directory.
This directory will contain the answer files associated with each MAC address.

You may then add as many answer files to the answers directory as you want.

  • Each file must be named after the MAC address it is associated with and also end with .toml.
For example: BC:24:11:AB:12:21.toml

Then save the following script as server.py in the server's directory as well: 

import logging
import json
import pathlib

try:
    import tomlkit
    from aiohttp import web
except ImportError as e:
    import sys

    message = """Could not import required packages.
Please ensure you've installed all necessary packages first!

On Debian-based distributions, you should be able to install them via:

\tapt update
\tapt install python3-aiohttp python3-tomlkit"""

    print(message, file=sys.stderr)

    raise e

DEFAULT_ANSWER_FILE_PATH = pathlib.Path("./default.toml")
ANSWER_FILE_DIR = pathlib.Path("./answers/")

routes = web.RouteTableDef()


@routes.post("/answer")
async def answer(request: web.Request):
    try:
        request_data = json.loads(await request.text())
    except json.JSONDecodeError as e:
        return web.Response(
            status=500,
            text=f"Internal Server Error: failed to parse request contents: {e}",
        )

    logging.info(
        f"Request data for peer '{request.remote}':\n"
        f"{json.dumps(request_data, indent=1)}"
    )

    try:
        answer = create_answer(request_data)

        logging.info(f"Answer file for peer '{request.remote}':\n{answer}")

        return web.Response(text=answer)
    except Exception as e:
        logging.exception(f"failed to create answer: {e}")
        return web.Response(status=500, text=f"Internal Server Error: {e}")


def create_answer(request_data: dict) -> str:
    with open(DEFAULT_ANSWER_FILE_PATH) as file:
        answer = tomlkit.parse(file.read())

    for nic in request_data.get("network_interfaces", []):
        if "mac" not in nic:
            continue

        answer_mac = lookup_answer_for_mac(nic["mac"])
        if answer_mac is not None:
            answer = answer_mac

    return tomlkit.dumps(answer)


def lookup_answer_for_mac(mac: str) -> tomlkit.TOMLDocument | None:
    mac = mac.lower()

    for filename in ANSWER_FILE_DIR.glob("*.toml"):
        if filename.name.lower().startswith(mac):
            with open(filename) as mac_file:
                return tomlkit.parse(mac_file.read())


def assert_default_answer_file_exists():
    if not DEFAULT_ANSWER_FILE_PATH.exists():
        raise RuntimeError(
            f"Default answer file '{DEFAULT_ANSWER_FILE_PATH}' does not exist"
        )


def assert_default_answer_file_parseable():
    with open(DEFAULT_ANSWER_FILE_PATH) as file:
        try:
            tomlkit.parse(file.read())
        except Exception as e:
            raise RuntimeError(
                "Could not parse default answer file "
                f"'{DEFAULT_ANSWER_FILE_PATH}':\n{e}"
            )


def assert_answer_dir_exists():
    if not ANSWER_FILE_DIR.exists():
        raise RuntimeError(f"Answer file directory '{ANSWER_FILE_DIR}' does not exist")


if __name__ == "__main__":
    assert_default_answer_file_exists()
    assert_answer_dir_exists()
    assert_default_answer_file_parseable()

    app = web.Application()

    logging.basicConfig(level=logging.INFO)

    app.add_routes(routes)
    web.run_app(app, host="0.0.0.0", port=8000)

The server's directory should now look like this: 

# tree -p /srv/proxmox/auto-install-server
[drwx------]  /srv/proxmox/auto-install-server
├── [drwxr-xr-x]  answers
│   ├── [-rw-r--r--]  BC:24:11:AB:12:21.toml
│   ├── [-rw-r--r--]  BC:24:11:BE:F2:A2.toml
│   └── [-rw-r--r--]  BC:24:11:DC:CD:21.toml
├── [-rw-r--r--]  default.toml
└── [-rw-r--r--]  server.py

2 directories, 5 files

You may now serve your answer files on https://[HOST IP]:8000/answer by starting the server with Python: 

cd /srv/proxmox/auto-install-server
python3 server.py

The server will run in the foreground and can be terminated by hitting CTRL+C in the console.

Yellowpin.svg Note: Please ensure that your connection is secured by TLS through something like a reverse proxy.

Third party tools

The following third-party tools around the Proxmox Automated Installation might be useful:

Troubleshooting

If the installation fails for some reason, it will drop into a shell (unless the reboot_on_error option in the answer file is set to true). This gives you the chance to troubleshoot what went wrong.

The log files of interest will most likely be:

  • /tmp/fetch_answer.log — the steps to retrieve an answer file
  • /tmp/auto_installer — parsing of the answer file, matching of hardware to use
  • /tmp/install-low-level-start-session.log — the actual installation process

Code listings

System information POST data

The actual contents of the DMI information ("dmi" key) might vary wildly, depending on the system. The keys itself are guaranteed to be present, but the values depend on the manufacturer and firmware of the system. 

{
  "$schema": {
    "version": "1.0"
  },
  "product": {
    "fullname": "Proxmox VE",
    "product": "pve",
    "enable_btrfs": true
  },
  "iso": {
    "release": "8.2",
    "isorelease": "2"
  },
  "dmi": {
    "system": {
      "serial": "900030XXXX",
      "name": "RS700-E11-RS12U 1HE Intel Dual-CPU RI2112-ASXSN Server",
      "sku": "SKU",
      "uuid": "6fb36fd4-77a1-57e6-13ae-XXXXXXcb48d7"
    },
    "baseboard": {
      "name": "Z13PP-D32 Series",
      "asset_tag": "To be filled by O.E.M.",
      "serial": "23041870000XXXX"
    },
    "chassis": {
      "asset_tag": "To be filled by O.E.M.",
      "serial": "I02308XXXX"
    }
  },
  "network_interfaces": [
    {
      "link": "enp6s18",
      "mac": "00:10:20:30:40:50"
    },
    {
      "link": "enp6s19",
      "mac": "11:22:33:44:55:66"
    }
  ]
}

Post-installation webhook JSON example

The system has multiple disks and network interfaces, was installed using ext4 as a file system on the first disk and the first network interface is used as management interface. 

{
  "$schema": {
    "version": "1.0"
  },
  "debian-version": "12.5",
  "product": {
    "fullname": "Proxmox VE",
    "short": "pve",
    "version": "8.2.2"
  },
  "iso": {
    "release": "8.2",
    "isorelease": "1"
  },
  "kernel-version": {
    "sysname": "Linux",
    "release": "6.8.4-2-pve",
    "version": "#1 SMP PREEMPT_DYNAMIC PMX 6.8.4-2 (2024-04-10T17:36Z)",
    "machine": "x86_64"
  },
  "boot-info": {
    "mode": "efi",
    "secureboot": true
  },
  "cpu-info": {
    "cores": 4,
    "cpus": 4,
    "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat npt lbrv nrip_save tsc_scale vmcb_clean flushbyasid pausefilter pfthreshold v_vmsave_vmload vgif umip rdpid arch_capabilities",
    "hvm": true,
    "model": "AMD Ryzen 7 3700X 8-Core Processor",
    "sockets": 1
  },
  "dmi": {
    "system": {
      "serial": "",
      "sku": "",
      "uuid": "b2fd1aa2-dc6e-4d8f-ad67-6dcc31984938",
      "name": "Standard PC (Q35 + ICH9, 2009)"
    },
    "baseboard": {},
    "chassis": {
      "asset_tag": "",
      "serial": ""
    }
  },
  "filesystem": "ext4",
  "fqdn": "host.domain",
  "machine-id": "b8737afea804482697ffe04db69c73d1",
  "disks": [
    {
      "size": 8589934592,
      "is-bootdisk": true,
      "udev-properties": {
        "CURRENT_TAGS": ":systemd:",
        "DEVLINKS": "/dev/disk/by-path/virtio-pci-0000:06:0a.0 /dev/disk/by-diskseq/9 /dev/disk/by-path/pci-0000:06:0a.0",
        "DEVNAME": "/dev/vda",
        "DEVPATH": "/devices/pci0000:00/0000:00:1e.0/0000:05:01.0/0000:06:0a.0/virtio0/block/vda",
        "DEVTYPE": "disk",
        "DISKSEQ": "9",
        "ID_PATH": "pci-0000:06:0a.0",
        "ID_PATH_TAG": "pci-0000_06_0a_0",
        "MAJOR": "253",
        "MINOR": "0",
        "SUBSYSTEM": "block",
        "TAGS": ":systemd:",
        "USEC_INITIALIZED": "3291667"
      }
    },
    {
      "size": 8589934592,
      "udev-properties": {
        "DEVNAME": "/dev/vdb",
        [..]
      }
    }
  ],
  "network-interfaces": [
    {
      "mac": "00:10:20:30:40:50",
      "address": "10.0.0.2/24",
      "is-management": true,
      "udev-properties": {
        "CURRENT_TAGS": ":systemd:",
        "DEVPATH": "/devices/pci0000:00/0000:00:1e.0/0000:05:01.0/0000:06:12.0/virtio6/net/enp6s18",
        "ID_BUS": "pci",
        "ID_MODEL_FROM_DATABASE": "Virtio network device",
        "ID_MODEL_ID": "0x1000",
        "ID_NET_DRIVER": "virtio_net",
        "ID_NET_LINK_FILE": "/usr/lib/systemd/network/99-default.link",
        "ID_NET_NAME": "enp6s18",
        "ID_NET_NAME_MAC": "enxdeadffc2635e",
        "ID_NET_NAME_PATH": "enp6s18",
        "ID_NET_NAMING_SCHEME": "v252",
        "ID_PATH": "pci-0000:06:12.0",
        "ID_PATH_TAG": "pci-0000_06_12_0",
        "ID_PCI_CLASS_FROM_DATABASE": "Network controller",
        "ID_PCI_SUBCLASS_FROM_DATABASE": "Ethernet controller",
        "ID_VENDOR_FROM_DATABASE": "Red Hat, Inc.",
        "ID_VENDOR_ID": "0x1af4",
        "IFINDEX": "2",
        "INTERFACE": "enp6s18",
        "SUBSYSTEM": "net",
        "SYSTEMD_ALIAS": "/sys/subsystem/net/devices/enp6s18",
        "TAGS": ":systemd:",
        "USEC_INITIALIZED": "3315145"
      }
    },
    {
      "mac": "11:22:33:44:55:66",
      "udev-properties": {
        "INTERFACE": "enp6s19",
        [..]
      }
    }
  ],
  "ssh-public-host-keys": {
    "ecdsa": "ecdsa-sha2-nistp256 [..] root@host.domain",
    "ed25519": "ssh-ed25519 [..] root@host.domain",
    "rsa": "ssh-rsa [..] root@host.domain"
  }
}
Retrieved from "https://pve.proxmox.com/mediawiki/index.php?title=Automated_Installation&oldid=12065"