From Proxmox VE

Introduction

Intel Modular Server (IMS) is a powerful server platform with integrated SAN. This how to describes the installation and configuration of Proxmox VE on IMS - with the support of live migration (KVM guests and containers) between compute modules.

A big thanks to Intel, sponsoring the hardware to the Proxmox VE test lab.

Before you start

Make sure you got the latest firmware on the IMS - at the time of these tests we got the firmware V5.5.

Depending on the workload, you need to clarify among all other things the following in advance:

Number of KVM guests, including storage requirements
Number of OpenVZ containers, including storage requirements
Backup considerations

Hardware configuration

Storage

The storage drive bay supports six 3.5" SAS/SATA or fourteen 2.5" SAS hard disk drives. We prefer the model with 14 drives as it provides more flexibility.

Note:

If you need more storage (more than 14 disks), you can connect external storage via mini SAS port, or you can connect your existing iSCSI or NFS storage.

Storage is the most important part for overall performance. So if your budget is limited, use cheaper CPU´s but never cheap and slow hard drives.

Pool1: Proxmox VE installation disk (and container storage)

Create a storage pool for all installation disks (for best performance, use 4 disks)
Create virtual drives (for best performance, use Raid10), and assign each one to a compute module as the boot drive (0)

Keep in mind, that all OpenVZ containers are also stored on these disks.

Example: Four 300GB disks and three compute modules

As we use Raid10, we have a total capacity of 600 GB, so we can assign 200 GB to each compute module for the installation and the local storage (mainly used to store containers).

Pool2: Shared LVM storage for KVM guests

By using the shared LUN feature (you need to order an activation key), you can assign a virtual disk to multiple compute modules enabling live migration of KVM guests between compute modules.

This is one of the main advantages of the IMS, as you do not need an extra SAN box. No need for expensive fiber channel (FC) infrastructure or the complexity of configuration of an extra storage network for iSCSI.

Create a storage pool for LVM storage (for best performance, use 4 disks)
Create one virtual drive (for best performance, you can also use Raid10) and assign the disk to ALL compute modules as drive 1.

Example: Four 300GB disks

As we use Raid10, we have a total capacity of 600 GB, so we can use 600 GB for our KVM virtual machines.

Pool3: Backup and ISO images - NFS

Create a storage pool, big enough for storing the backup data and ISO images
Create one big virtual drive and assign it to one single compute module as the third drive (2)

Example: Three 300GB disks

We use Raid5, so we have 600 GB for backups and ISO images.

Hot spare drive

We recommend you add at least one global hot spare disk. We suggest to use identical disks on all slots.

Compute Modules

You can use up to six compute modules each one with 2 Intel Xeon processors 5500 series´s and up to 96 GB of memory.

Our test setup:

Three MFS5520VI
Each module has two Xeon E5540
and 12 GB memory (6 x 2 GB)

Network

Nothing special needed.

Installation of Proxmox VE

Bare-metal ISO install

By using the integrated remote console, installation can be done by mapping the ISO images to the compute modules. See also Installation After reboot, install the latest packages on all nodes:

aptitude update && aptitude safe-upgrade

Proxmox VE package list:

pveversion -v

pve-manager: 1.5-8 (pve-manager/1.5/4674)
running kernel: 2.6.18-2-pve
proxmox-ve-2.6.18: 1.5-5
pve-kernel-2.6.18-2-pve: 2.6.18-5
qemu-server: 1.1-11
pve-firmware: 1.0-3
libpve-storage-perl: 1.0-10
vncterm: 0.9-2
vzctl: 3.0.23-1pve8
vzdump: 1.2-5
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm-2.6.18: 0.9.1-5

Create Cluster

Promote one server to the master and join the other nodes, see Proxmox_VE_Cluster

Setup the LVM disk for KVM guest

Login to the master via SSH and partition the disk:

Create a big single partition (sdb1)

fdisk /dev/sdb

Create the physical volume:

pvcreate /dev/sdb1
 Physical volume "/dev/sdb1" successfully created

Create the volume group 'vmdisks' (just choose a unique name):

vgcreate vmdisks /dev/sdb1
  Volume group "vmdisks" successfully created

And finally: Add the LVM Group to the storage list via the web interface.

Optional: Setup a NFS server on one Proxmox VE node

For small setups its quite useful to install a NFS server just on Proxmox VE (If you already have a NFS server in your network, just use the existing one).

Login to the server where the third hard disk is assigned via SSH and partition the disk:

Create a big single partition (sdc1)

fdisk /dev/sdc

Format the partition with ext3

mkfs.ext3 /dev/sdc1

query the UUID of the partition

blkid

Add the partition to your /etc/fstab (replace the UUID with yours)

nano /etc/fstab

UUID=2f1fe410-293f-4f1d-bdfb-8b45f1d3b4aa /srv ext3 defaults 0 2

create dir´s:

mkdir /srv
cd /srv
mkdir iso
mkdir backup

install nfs-server package (use the packages from lenny-backports, nfs-kernel-server >= 1.2.2-1)

aptitude -t lenny-backports install nfs-kernel-server

Configure nfs exports (just an example, adapt this to your network setup):

nano /etc/exports
/srv/backup     192.168.0.0/255.255.240.0(rw,sync,no_subtree_check,no_root_squash)
/srv/iso        192.168.0.0/255.255.240.0(rw,sync,no_subtree_check,no_root_squash)

And finally, restart NFS server:

/etc/init.d/nfs-kernel-server restart

NFS-ISO Store

Now, you can add the NFS-ISO Store via web interface.

Backup Store

Add the NFS-Backup Store via web interface.

Install Virtual Machines

Get templates

In order to create a container, you need to get templates first - Either download them via web interface or just upload via web interface (if you got them already). You can also transfer templates directly via scp to /var/lib/vz/template/cache/ on the master server (these templates will be replicated to all cluster nodes).

Upload ISO images to the NFS-ISO Store

In order to install KVM guests, you need a installation media - we use ISO images for maximum convenience. Upload them via web interface (DVD´s cannot be uploaded, size limit) or transfer them via scp to /mnt/pve/NFS-ISO/.

Links

Intel Modular Server

Personal tools