Monitoring with Icinga

From Proxmox VE
Jump to: navigation, search


This wikipage describes the installation of icinga (with pnp for graphical output) on a debian-box.

Icinga is a fork of Nagios - the config is the same. If you need further information search also for nagios.

It's take some time to install icinga but if you follow this guide step by step you (hopefully) will get an working installation.

To get to the goal faster - make a preconfig-vm for proxmox (based on devil-linux), but not yet...

Normally the monitoring-host should be a standalone stable machine, for test it's ok in a vm.

Some definitions

  • Call the debian-box monitoring and the ip in this example is
  • The proxmox-host is called proxmox and has the ip
  • The VM which should be monitored has the ip

Put into practice

Install on monitoring using debian-504-i386-netinst.iso - select only standard

Login as root

We need some more stuff

apt-get update
apt-get install openssh-server sudo lynx apache2 build-essential \
       libgd2-xpm-dev librrd-dev rrdtool librrds-perl librrdp-perl \
       libnet-snmp-perl php5 php5-gd unzip

In this example we use as working-user support (defined during the installation process)

ssh support@

Enable support for sudo

visudo -f /etc/sudoers

uncomment the last line:

addgroup support sudo

Create a user for icinga

/usr/sbin/useradd -m icinga
passwd icinga
exit # again support
exit # logoff
ssh support@ # to activate changes in group
mkdir ~/software
cd ~/software

Build and install icinga

Current Version - 1.5.1 (Circa Sep 2011)

tar xvzf icinga-1.2.1.tar.gz
cd icinga-1.2.1
make all
sudo make fullinstall
cd /etc/rc2.d/
sudo ln -s ../init.d/icinga S92icinga
cd ../rc0.d
sudo ln -s ../init.d/icinga K10icinga
sudo vi /usr/local/icinga/etc/objects/contacts.cfg # change the email-address of icingaadmin
sudo htpasswd -c /usr/local/icinga/etc/htpasswd.users icingaadmin # this account/password is for the webfrontend
sudo ln -s /usr/local/icinga/etc /etc/icinga # to access the config below /etc
sudo addgroup www-data icinga

Build and install the nagios-plugins

Version is current as of Sep 2011.

cd ~/software
tar xvzf nagios-plugins-1.4.15.tar.gz
cd nagios-plugins-1.4.15
./configure --prefix=/usr/local/icinga --with-nagios-user=icinga --with-cgiurl=/icinga/cgi-bin
sudo make install

Install snmp-plugin

cd ~/software
chmod +x
sudo chown icinga
sudo mv /usr/local/icinga/libexec/

Build and install pnp

cd ~/software

Download the kohana framework and scp it to support@monitoring:software/ - or with lynx directly on monitoring. Current versions are 3.2.0 and 3.1.4.

cd ~/software 
sudo mv kohana /var/www/
sudo chown -R www-data:www-data /var/www/kohana
sudo mv /var/www/kohana/install.php /var/www/kohana/

PHP4Nagois Current Version is 0.6.15

tar xvzf pnp4nagios-0.6.7.tar.gz
cd pnp4nagios-0.6.7/
./configure --with-nagios-user=$USER --with-nagios-group=$GROUP --sysconfdir=$PREFIX/etc/pnp \
           --prefix=$PREFIX --datarootdir=$PREFIX/share/pnp --with-rrdtool=/usr/bin/rrdtool \
           --sysconfdir=$PREFIX/etc/pnp --with-perfdata-dir=$PREFIX/share/perfdata \
           --with-kohana_system=/var/www/kohana/system --exec-prefix=$PREFIX \
           --with-perfdata-logfile=$PREFIX/var/perfdata.log \
make all
sudo make install
sudo make install-webconf
sudo make install-config
sudo ln -s /usr/local/icinga/lib/kohana/system /usr/local/icinga/share/pnp/system
sudo vi /usr/local/icinga/share/pnp/index.php
# change "$kohana_system = '/var/www/kohana/system';" to
$kohana_system = '/usr/local/icinga/lib/kohana/system';


cd /etc/apache2/mods-enabled
sudo ln -s ../mods-available/rewrite.load .
cd /etc/icinga
su icinga
ssh-keygen # we need for further plugins the ssh-key - not in this example
mv /etc/icinga/pnp/process_perfdata.cfg-sample /etc/icinga/pnp/process_perfdata.cfg
mkdir proxmox
mkdir check_commands
# add two lines to icinga.cfg:

change in icinga.cfg "process_performance_data=0" to


and uncomment the following two lines:


Comment out the last two blocks in /etc/icinga/objects/commands.cfg (perfdata), or delete the line (but make a copy before) afterwards append the following configuration with:

cat << EOF >> /etc/icinga/objects/commands.cfg
define command {
      command_name    process-service-perfdata
      command_line    /usr/local/icinga/libexec/

define command {
       command_name    process-host-perfdata
       command_line    /usr/local/icinga/libexec/ -d HOSTPERFDATA
mv /etc/icinga/pnp/rra.cfg-sample /etc/icinga/pnp/rra.cfg

add the snmp-password in /etc/icinga/resource.cfg:

cat << EOF > /etc/icinga/check_commands/check_snmp_int.cfg
define command {
       command_name    check_snmp_int
       command_line    \$USER1\$/  -H \$HOSTADDRESS\$ -C \$USER4\$ -n "\$ARG1\$" "\$ARG2\$" "\$ARG3\$" -f

define command {
       command_name    check_snmp_int_pve
       command_line    \$USER1\$/  -H \$ARG1\$ -C \$USER4\$ -n "\$ARG2\$" "\$ARG3\$" "\$ARG4\$" -f

cat << EOF >> /etc/icinga/objects/templates.cfg

# PVE host definition template - This is NOT a real host, just a template!
define host{
       name                    pve            ; The name of this host template
       use                     generic-host   ; This template inherits other values from the generic-host template
       check_period            24x7           ; By default, Linux hosts are checked round the clock
       check_interval          5              ; Actively check the host every 5 minutes
       retry_interval          1              ; Schedule host check retries at 1 minute intervals
       max_check_attempts      5              ; Check each Linux host 10 times (max)
       check_command         check-host-alive ; Default command to check Linux hosts
       notification_period     workhours      ; Linux admins hate to be woken up, so we only notify during the day
                                              ; Note that the notification_period variable is being overridden from
                                              ; the value that is inherited from the generic-host template!
       notification_interval   120            ; Resend notifications every 2 hours
       notification_options    d,u,r          ; Only send notifications for specific host states
       contact_groups          pve-admins     ; Notifications get sent to the admins by default
       register                0              ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL HOST, JUST A TEMPLATE!
cat << EOF >> /etc/icinga/objects/contacts.cfg

define contactgroup{
       contactgroup_name       pve-admins
       alias                   Proxmox Administrators
       members                 icingaadmin
exit # to be support
sudo vi /etc/apache2/conf.d/pnp4nagios.conf # to change /usr/local/nagios/etc/htpasswd.users in:
<Directory "/usr/local/icinga/share/pnp">
        AllowOverride None
        Order allow,deny
        Allow from all
        # Use the same value as defined in nagios.conf
        AuthName "Incinga Access"
        AuthType Basic
        AuthUserFile /usr/local/icinga/etc/htpasswd.users

sudo vi /etc/php5/apache2/php.ini

magic_quotes_gpc = Off

Now it's time for a first test!

sudo /etc/init.d/apache2 restart
sudo /etc/init.d/icinga start

connect with a web browser to
and to
Use as account icingaadmin with the selected password
The "PNP4Nagios Environment Tests" should be passed. Ok! Further...

cd /usr/local/icinga/share/pnp/
sudo mv install.php

Now, it's time to monitor one client

Some stuff is to be done on the proxmox-host:

ssh root@proxmox
apt-get update
apt-get install snmpd # we monitor the interfaces via snmp

change in /etc/default/snmpd the listen address from

SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/'
vi /etc/snmp/snmpd.conf
#  source          community
#com2sec paranoid  default         public
com2sec readonly  pvesecret
#com2sec readwrite default         private

restart snmpd:

/etc/init.d/snmpd restart

Place the script to generate icinga-configs:

mkdir /usr/local/scripts
cd /usr/local/scripts
mkdir pvekvm2icinga.d

create the script with following content:

# (c) Udo Lembke 2/2010 v. 0.1 gplv3
# this script need the vmid as parameter


proxmox_host=`/sbin/ifconfig vmbr0 | grep "inet addr:" | awk '{print $2}' | tr -d "addr:"`

if [[ $# -ne 1 ]] || [[ $1 -le 100 ]]; then
   echo "call \""$0" VMID\" VMID is 101-999"
   exit 1

echo "create "$1".conf for icinga"

if [[ ! -f /etc/qemu-server/$1.conf ]]; then
   echo "VM-config not found";
   exit 1;
if [[ ! -f $host_preconf/$1.conf ]]; then
   echo "$host_preconf/$1.conf not accessible"
   echo "nessasary for VM-ip-address"
   exit 1;

vm_name=`grep "name:" /etc/qemu-server/$1.conf | awk '{print $2}'`
vm_vlans_conf=`grep "vlan[0-9]" /etc/qemu-server/$1.conf | awk -F: '{print $1}'| tr -d "vlan"`
vm_vlans=`/sbin/ifconfig | grep "vmtab"$1 | awk '{print $1}' | awk -Fi '{print $2}'`
if [[ `echo $vm_vlans| wc -w` -ne `echo $vm_vlans_conf | wc -w` ]]; then echo "Attention, number of active interfaces are not equal to the config-file"; fi

echo "define host{" > /tmp/pvekvm2icinga_$1
echo "        use                     pve" >> /tmp/pvekvm2icinga_$1
echo "        host_name               $vm_name" >> /tmp/pvekvm2icinga_$1

alias_line=`grep "alias" $host_preconf/$1.conf`
address_line=`grep "address" $host_preconf/$1.conf`

if [[ -n $alias_line ]]; then
    echo $alias_line | awk '{print "        "$1"                   "$2,$3,$4,$5}'>> /tmp/pvekvm2icinga_$1
else echo "        alias                   $vm_name; VM$1 on $proxmox_hostname" >> /tmp/pvekvm2icinga_$1
if [[ -n $address_line ]]; then
    echo $address_line | awk '{print "        "$1"                 "$2}' >> /tmp/pvekvm2icinga_$1
else echo "        address       " >> /tmp/pvekvm2icinga_$1

echo "        }" >> /tmp/pvekvm2icinga_$1
echo "" >> /tmp/pvekvm2icinga_$1
echo "define service{" >> /tmp/pvekvm2icinga_$1
echo "        use                             generic-service" >> /tmp/pvekvm2icinga_$1
echo "        host_name                       $vm_name" >> /tmp/pvekvm2icinga_$1
echo "        service_description             PING" >> /tmp/pvekvm2icinga_$1
echo "        is_volatile                     0" >> /tmp/pvekvm2icinga_$1
echo "        check_period                    24x7" >> /tmp/pvekvm2icinga_$1
echo "        max_check_attempts              4" >> /tmp/pvekvm2icinga_$1
echo "        normal_check_interval           1" >> /tmp/pvekvm2icinga_$1
echo "        retry_check_interval            1" >> /tmp/pvekvm2icinga_$1
echo "        contact_groups                  pve-admins" >> /tmp/pvekvm2icinga_$1
echo "        notification_options            w,u,c,r" >> /tmp/pvekvm2icinga_$1
echo "        notification_interval           120" >> /tmp/pvekvm2icinga_$1
echo "        notification_period             24x7" >> /tmp/pvekvm2icinga_$1
echo "        check_command                   check_ping!200.0,40%!500.0,80%" >> /tmp/pvekvm2icinga_$1
echo "        }" >> /tmp/pvekvm2icinga_$1
echo "" >> /tmp/pvekvm2icinga_$1

for vlanid in $vm_vlans_conf
    echo "define service{" >> /tmp/pvekvm2icinga_$1
    echo "        use                             generic-service" >> /tmp/pvekvm2icinga_$1 
    echo "        host_name                       $vm_name" >> /tmp/pvekvm2icinga_$1
    echo "        service_description             ${vlan[$vlanid]}_TRAFFIC" >> /tmp/pvekvm2icinga_$1
    echo "        is_volatile                     0" >> /tmp/pvekvm2icinga_$1
    echo "        check_period                    24x7" >> /tmp/pvekvm2icinga_$1
    echo "        max_check_attempts              4" >> /tmp/pvekvm2icinga_$1
    echo "        normal_check_interval           1" >> /tmp/pvekvm2icinga_$1
    echo "        retry_check_interval            1" >> /tmp/pvekvm2icinga_$1
    echo "        contact_groups                  pve-admins" >> /tmp/pvekvm2icinga_$1
    echo "        notification_options            w,u,c,r" >> /tmp/pvekvm2icinga_$1
    echo "        notification_interval           120" >> /tmp/pvekvm2icinga_$1
    echo "        notification_period             24x7" >> /tmp/pvekvm2icinga_$1
    echo "        check_command                   check_snmp_int_pve!$proxmox_host!vmtab$1i$vlanid!!" >> /tmp/pvekvm2icinga_$1
    echo "        }" >> /tmp/pvekvm2icinga_$1
    echo "" >> /tmp/pvekvm2icinga_$1
    echo "define serviceextinfo{" >> /tmp/pvekvm2icinga_$1 
    echo "        host_name                       $vm_name" >> /tmp/pvekvm2icinga_$1
    echo "        service_description             ${vlan[$vlanid]}_TRAFFIC" >> /tmp/pvekvm2icinga_$1
    echo "        action_url                      /pnp4nagios/graph?host=$vm_name&srv=${vlan[$vlanid]}_TRAFFIC" >> /tmp/pvekvm2icinga_$1
    echo "        #notes_url                       /wiki/index.php/$vm_name#${vlan[$vlanid]}_TRAFFIC" >> /tmp/pvekvm2icinga_$1
    echo "}" >> /tmp/pvekvm2icinga_$1
    echo "" >> /tmp/pvekvm2icinga_$1


scp /tmp/pvekvm2icinga_$1 $icinga_user@$icinga_host:/etc/icinga/proxmox/$1.cfg
ssh $icinga_user@$icinga_host /etc/init.d/icinga reload
# rm /tmp/pvekvm2icinga_$1
chmod +x

Modify the names for the networks for your usage
To make your life easier, copy your public key to monitoring

scp ~/.ssh/ icinga@monitoring:~/.ssh/authorized_keys

To run the script with an VMID of a kvm-guest, which have bridged network we must first create a configfile in /usr/local/scripts/pvekvm2icinga.d like this:

cat 101.conf
alias Lubuntu test machine

alias is "can have" address is "must have".

generate the config of VMID 101 (with transfer to monitor)

/usr/local/scripts/ 101

Now you should see in icinga->Service Detail the vm with the service ping and one for each network-interface. If you click on the sign near Status, you get the network-chart in an extra browser-tab (or window).

That's the beginning! You can do a lot more with incinga.

If you have trouble with the icinga-config check them with (as user icinga or root):

/usr/local/icinga/bin/icinga -v /etc/icinga/icinga.cfg