Debian Appliance Builder: Difference between revisions

From Proxmox VE
Jump to navigation Jump to search
(Update to latest dab release - Attention: needs more updates due, work in progress ...)
Line 1: Line 1:
=Introduction=
= Introduction =
Creating high quality appliances is a difficult task and requires deep knowledge of the underlying operating system. So we created the 'Debian Appliance Builder' to simplify that task.
'dab' is a script to automate the creation of OpenVZ appliances. It is basically a rewrite of debootstrap in perl, but uses OpenVZ instead of chroot and generates OpenVZ templates. Another difference is that it supports multi-stage building of templates. That way you can execute arbitrary scripts between package installation steps to accomplish what you want.


Furthermore, some common tasks are fully automated - like setting up a database server (mysql or postgres). To accomplish minimal template creation time, packages are cached to a local directory, so you do not need a local Debian/Ubuntu mirror (although this would speed up the first run).
Creating high quality appliances is a difficult task and requires deep knowledge of the underlying operating system. So we created the 'Debian Appliance Builder' to simplify that task. 'dab' is a script to automate the creation of OpenVZ appliances. It is basically a rewrite of debootstrap in perl, but uses OpenVZ instead of chroot and generates OpenVZ templates. Another difference is that it supports multi-stage building of templates. That way you can execute arbitrary scripts between package installation steps to accomplish what you want.  
All generated templates includes an appliance description file. Those can be used to build appliance repositories.


'dab' is licensed under GPLv2 (Open source).
Furthermore, some common tasks are fully automated - like setting up a database server (mysql or postgres). To accomplish minimal template creation time, packages are cached to a local directory, so you do not need a local Debian mirror (although this would speed up the first run). All generated templates includes an appliance description file. Those can be used to build appliance repositories.
==Main features==
 
*Fully automated build process with standard 'make' scripts
'dab' is licensed under GPLv2 (Open source).  
*Local package cache
 
*Multi-stage build process
== Main features ==
*Creates and includes appliance description files
 
*Simplifies various tasks like database installation
*Fully automated build process with standard 'make' scripts  
*Appliances can be certified and integrated into the Proxmox VE download directory
*Local package cache  
*Supports Debian and Ubuntu based appliances
*Multi-stage build process  
*Generates 32 or 64 bit appliances
*Creates and includes appliance description files  
*Simplifies various tasks like database installation  
*Appliances can be certified and integrated into the Proxmox VE download directory  
*Supports Debian 6 (Squeeze) based appliances  
*Generates 32 or 64 bit appliances  
*Automatic dependency resolution
*Automatic dependency resolution


==Target audience==
== Target audience ==
'dab' is an easy way for software vendors and open source project to package their product into a "ready-to-run" appliance. Appliances can be published into the Proxmox VE appliance download directory to reach the target customer. See [[Download_templates_(Video)]].


In order to work with 'dab', basic Linux knowledge is needed.
'dab' is an easy way for software vendors and open source project to package their product into a "ready-to-run" appliance. Appliances can be published into the Proxmox VE appliance download directory to reach the target customer. See [[Download templates (Video)]].  


=Installation=
In order to work with 'dab', basic Linux knowledge is needed.
The script needs to be run as root, so it is not recommended to start it on a production machine with running containers.
 
==System requirements==
= Installation =
Prepare your operating system - you can install 'dab' on the following operating systems:
 
*Proxmox VE (fastest)
The script needs to be run as root, so it is not recommended to start it on a production machine with running containers.  
 
== System requirements ==
 
Prepare your operating system - you can install 'dab' on the following operating systems:  
 
*Proxmox VE (fastest)  
*Proxmox VE inside Proxmox VE (KVM) - (most comfortable)  
*Proxmox VE inside Proxmox VE (KVM) - (most comfortable)  
*Proxmox VE inside VMWare/XEN 64bit virtual machine
*Proxmox VE inside VMWare/XEN 64bit virtual machine  
*Debian 64/Ubuntu 64 with OpenVZ Kernel (not tested)
*Debian 64/Ubuntu 64 with OpenVZ Kernel (not tested)


=='dab' installation==
== 'dab' installation ==
Log into Proxmox VE, and install 'dab' using apt:
<pre>apt-get install dab</pre>


Or download and install it manually:
Log into Proxmox VE, and install 'dab' using apt:
<pre>apt-get install dab</pre>
Or download and install it manually:  
<pre>wget http://download.proxmox.com/debian/dists/lenny/pve/binary-amd64/dab_1.1-14_all.deb
dpkg -i dab_1.1-14_all.deb</pre>
== 'dab' sample templates  ==


<pre>wget http://download.proxmox.com/debian/dists/lenny/pve/binary-amd64/dab_1.1-9_all.deb
A set of examples is also available. Simply download them:  
dpkg -i dab_1.1-9_all.deb</pre>


A sample install on Proxmox VE 1.6 will output the following:
Check for current examples at ftp://download.proxmox.com/sources/
<pre>
<pre>wget ftp://download.proxmox.com/sources/dab-pve-appliances_2010-05-25.tar.gz</pre>
# apt-get install dab
Unpack with tar:  
<pre>tar -zxvf dab-pve-appliances_2010-05-25.tar.gz</pre>  
== 'dab' manual pages  ==


Reading package lists... Done
The package includes a manual page:  
Building dependency tree
<pre>man dab</pre> <pre>dab(1)               User Contributed Perl Documentation              dab(1)
Reading state information... Done
The following extra packages will be installed:
  binutils make
Suggested packages:
  binutils-doc make-doc
The following NEW packages will be installed:
  binutils dab make
0 upgraded, 3 newly installed, 0 to remove and 35 not upgraded.
Need to get 3158kB of archives.
After this operation, 9351kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://download.proxmox.com lenny/pve dab 1.1-9 [27.5kB]
Get:2 http://ftp.debian.org lenny/main binutils 2.18.1~cvs20080103-7 [2730kB]
Get:3 http://ftp.debian.org lenny/main make 3.81-5 [401kB]
Fetched 3158kB in 27s (113kB/s)
Selecting previously deselected package binutils.
(Reading database ... 25914 files and directories currently installed.)
Unpacking binutils (from .../binutils_2.18.1~cvs20080103-7_amd64.deb) ...
Selecting previously deselected package make.
Unpacking make (from .../archives/make_3.81-5_amd64.deb) ...
Selecting previously deselected package dab.
Unpacking dab (from .../apt/archives/dab_1.1-9_all.deb) ...
Processing triggers for man-db ...
Setting up binutils (2.18.1~cvs20080103-7) ...
Setting up make (3.81-5) ...
Setting up dab (1.1-9) ...
</pre>


=='dab' sample templates==
A set of examples is also available. Simply download them:


Check for current examples at ftp://download.proxmox.com/sources/


<pre>wget ftp://download.proxmox.com/sources/dab-pve-appliances_2010-05-25.tar.gz</pre>


Unpack with tar:


<pre>tar -zxvf dab-pve-appliances_2010-05-25.tar.gz</pre>


=='dab' manual pages==
The package includes a manual page:
<pre>man dab</pre>


<pre>
 
dab(1)                User Contributed Perl Documentation              dab(1)
 
NAME
 
 
 
 






NAME
       dab - Debian OpenVZ Appliance Builder
       dab - Debian OpenVZ Appliance Builder


SYNOPSIS
SYNOPSIS
       dab command [OPTIONS]
       dab command [OPTIONS]
      dab init
    dab init
          Downloads the package descriptions form the repository. Also
        Downloads the package descriptions form the repository. Also
          truncates the "logfile".
        truncates the "logfile".
 
 
 
 
 
 


       dab bootstrap
       dab bootstrap
          Bootstrap a debian system and allocate a temporary container (we
        Bootstrap a debian system and allocate a temporary container (we
          use IDs 90000 and above).
        use IDs 90000 and above).
 
 
 
 
 
 


           --exim
           --exim
              Use exim as MTA (we use postfix by default)
            Use exim as MTA (we use postfix by default)
 
 
 
 
 
 


           --minimal
           --minimal
              Do not install standard packages.
            Do not install standard packages.
 
 
 
 
 
 


       dab veid
       dab veid
          Print used container ID.
        Print used container ID.
 
 
 
 
 
 


       dab basedir
       dab basedir
          Print container private directory.
        Print container private directory.
 
 
 
 
 
 


       dab packagefile
       dab packagefile
          Print the appliance file name.
        Print the appliance file name.
 
 
 
 
 
 


       dab install pkg ...
       dab install pkg ...
          Install one or more packages. pkg can also refer to a file named
        Install one or more packages. pkg can also refer to a file named
          "xyz.pkglist" which contains a list of packages. All dependencies
        "xyz.pkglist" which contains a list of packages. All dependencies
          are automatically installed.
        are automatically installed.
 
 
 
 
 
 


       dab unpack pkg ...
       dab unpack pkg ...
          Unpack one or more packages. pkg can also refer to a file named
        Unpack one or more packages. pkg can also refer to a file named
          "xyz.pkglist" which contains a list of packages. All dependencies
        "xyz.pkglist" which contains a list of packages. All dependencies
          are automatically unpacked.
        are automatically unpacked.
 
 
 
 
 
 


       dab exec CMD ARGS
       dab exec CMD ARGS
          Executes command CMD inside the container.
        Executes command CMD inside the container.
 
 
 
 
 
 


       dab enter
       dab enter
          Calls "vzctl enter CTID" - this is for debugging only.
        Calls "vzctl enter CTID" - this is for debugging only.
 
 
 
 
 
 


       dab task mysql
       dab task mysql
          Install a mysql database server. During appliance generation we use
        Install a mysql database server. During appliance generation we use
          "admin" as mysql root password (also stored in /root/.my.cnf).
        "admin" as mysql root password (also stored in /root/.my.cnf).
 
 
 
 
 
 


           --password=XXX
           --password=XXX
              Specify the mysql root password. The special value "random" can
            Specify the mysql root password. The special value "random" can
              be use to generate a random root password when the appliance is
            be use to generate a random root password when the appliance is
              started first time (stored in /root/.my.cnf)
            started first time (stored in /root/.my.cnf)
 
 
 
 
 
 


           --start
           --start
              Start the mysql server (if you want to execute sql commands
            Start the mysql server (if you want to execute sql commands
              during appliance generation).
            during appliance generation).
 
 
 
 
 
 


       dab task postgres
       dab task postgres
          Install a postgres database server.
        Install a postgres database server.
 
 
 
 
 
 


           --version=XXX
           --version=XXX
              Select Postgres version. Posible values are 7.4, 8.1 and 8.3
            Select Postgres version. Posible values are 7.4, 8.1 and 8.3
              (depends on the selected suite).
            (depends on the selected suite).
 
 
 
 
 
 


           --start
           --start
              Start the postgres server (if you want to execute sql commands
            Start the postgres server (if you want to execute sql commands
              during appliance generation).
            during appliance generation).
 
 
 
 
 
 


       dab task php
       dab task php
          Install php5.
        Install php5.
 
 
 
 
 
 


           --memlimit=i
           --memlimit=i
              Set the php memory_limit.
            Set the php memory_limit.
 
 
 
 
 
 


       dab finalize
       dab finalize
          Cleanup everything inside the container and generate the final
        Cleanup everything inside the container and generate the final
          appliance package.
        appliance package.
 
 
 
 
 
 


           --keepmycnf
           --keepmycnf
              Do not delete file "/root/.my.cfg" (mysql).
            Do not delete file "/root/.my.cfg" (mysql).
 
 
 
 
 
 


       dab list
       dab list
          List installed packages.
        List installed packages.
 
 
 
 
 
 


           --verbose
           --verbose
              Also print package versions.
            Also print package versions.
 
 
 
 
 
 


       dab clean
       dab clean
          Remove all temporary file and destroy the used OpenVZ container.
        Remove all temporary file and destroy the used OpenVZ container.
 
 
 
 
 
 


       dab dist-clean
       dab dist-clean
          Like clean, but also removes the package cache (except when you
        Like clean, but also removes the package cache (except when you
          specified your own cache directory in the config file)
        specified your own cache directory in the config file)
 
 
 


DESCRIPTION
DESCRIPTION
       dab is a script to automate the creation of OpenVZ appliances. It is
       dab is a script to automate the creation of OpenVZ appliances. It is
      basically a rewrite of debootstrap in perl, but uses OpenVZ instead of
    basically a rewrite of debootstrap in perl, but uses OpenVZ instead of
      chroot and generates OpenVZ templates. Another difference is that it
    chroot and generates OpenVZ templates. Another difference is that it
      supports multi-stage building of templates. That way you can execute
    supports multi-stage building of templates. That way you can execute
      arbitrary scripts between to accomplish what you want.
    arbitrary scripts between to accomplish what you want.
 
 
 
 
 
 


       Furthermore some common tasks are fully automated, like setting up a
       Furthermore some common tasks are fully automated, like setting up a
      database server (mysql or postgres).
    database server (mysql or postgres).
 
 
 
 
 
 


       To accomplish minimal template creation time, packages are cached to a
       To accomplish minimal template creation time, packages are cached to a
      local directory, so you do not need a local debian mirror (although
    local directory, so you do not need a local debian mirror (although
      this would speed up the first run).
    this would speed up the first run).
 
 
 
 
 
 


       See http://pve.proxmox.com/wiki/Debian_Appliance_Builder for examples.
       See http://pve.proxmox.com/wiki/Debian_Appliance_Builder for examples.


       This script need to be run as root, so it is not recommended to start
       This script need to be run as root, so it is not recommended to start
      it on a production machine with running container. Proxmox VE
    it on a production machine with running container. Proxmox VE
      (http://pve.proxmox.com) is the preferred environment, because it is
    (http://pve.proxmox.com) is the preferred environment, because it is
      able to log the console output when a container starts. You wont be
    able to log the console output when a container starts. You wont be
      able to detect errors during container startup when running on standard
    able to detect errors during container startup when running on standard
      OpenVZ. So many people run Proxmox VE inside a KVM or VMWare 64bit
    OpenVZ. So many people run Proxmox VE inside a KVM or VMWare 64bit
      virtual machine to build appliances.
    virtual machine to build appliances.
 
 
 
 
 
 


       All generated templates includes an appliance description file. Those
       All generated templates includes an appliance description file. Those
      can be used to build appliance repositories.
    can be used to build appliance repositories.
 
 
 


CONFIGURATION
CONFIGURATION
       Configuration is read from the file "dab.conf" inside the current
       Configuration is read from the file "dab.conf" inside the current
      working directory. The files contains key value pairs, separated by
    working directory. The files contains key value pairs, separated by
      colon.
    colon.
 
 
 
 
 
 


       Suite: etch|lenny|squeeze|hardy|intrepid|jaunty
       Suite: etch|lenny|squeeze|hardy|intrepid|jaunty
        The Debian or Ubuntu suite.
      The Debian or Ubuntu suite.
 
 
 
 
 
 


       Source: URL [components]
       Source: URL [components]
        Defines a source location. By default we use the following for
      Defines a source location. By default we use the following for
        debian:
      debian:
 
 
 
 
 
 


           Source: http://ftp.debian.org/debian SUITE main contrib
           Source: http://ftp.debian.org/debian SUITE main contrib
          Source: http://security.debian.org SUITE/updates main contrib
        Source: http://security.debian.org SUITE/updates main contrib
 
 
 
 
 
 


         Note: SUITE is a variable and will be substituted.
         Note: SUITE is a variable and will be substituted.


         There are also reasonable defaults for Ubuntu. If you do not specify
         There are also reasonable defaults for Ubuntu. If you do not specify
        any source the defaults are used.
      any source the defaults are used.
 
 
 
 
 
 


       Depends: dependencies
       Depends: dependencies
        Debian like package dependencies. This can be used to make sure that
      Debian like package dependencies. This can be used to make sure that
        speific package versions are available.
      speific package versions are available.
 
 
 
 
 
 


       CacheDir: path
       CacheDir: path
        Allows you to specify the directory where downloaded packages are
      Allows you to specify the directory where downloaded packages are
        cached.
      cached.
 
 
 
 
 
 
 
      Mirror: SRCURL =&gt; DSTURL
      Define a mirror location. for example:
 
 
 
 
 
 
 
          Mirror: http://ftp.debian.org/debian =&gt; ftp://mirror/debian
 
 
 
 


      Mirror: SRCURL => DSTURL
        Define a mirror location. for example:


          Mirror: http://ftp.debian.org/debian => ftp://mirror/debian


       All other settings in this files are also included into the appliance
       All other settings in this files are also included into the appliance
      description file.
    description file.
 
 
 
 
 
 


       Name: name
       Name: name
        The name of the appliance.
      The name of the appliance.
 
 
 
 
 
 


         Appliance names must consist only of lower case letters (a-z), digits
         Appliance names must consist only of lower case letters (a-z), digits
        (0-9), plus (+) and minus (-) signs, and periods (.). They must be at
      (0-9), plus (+) and minus (-) signs, and periods (.). They must be at
        least two characters long and must start with an alphanumeric
      least two characters long and must start with an alphanumeric
        character.
      character.
 
 
 
 
 
 


       Architecture: i386|amd64
       Architecture: i386|amd64
        Target architecture.
      Target architecture.
 
 
 
 
 
 


       Version: upstream_version[-build_revision]
       Version: upstream_version[-build_revision]
        The version number of an appliance.
      The version number of an appliance.
 
 
 
 
 
 


         =item: Section: section
         =item: Section: section


         This field specifies an application area into which the appliance has
         This field specifies an application area into which the appliance has
        been classified. Currently we use the following section names:
      been classified. Currently we use the following section names:
        system, admin, www
      system, admin, www
 
 
 
 
 
 
 
      Maintainer: name &lt;email&gt;
      The appliance maintainer's name and email address. The name should
      come first, then the email address inside angle brackets &lt;&gt; (in
      RFC822 format).
 
 
 
 
 


      Maintainer: name <email>
        The appliance maintainer's name and email address. The name should
        come first, then the email address inside angle brackets <> (in
        RFC822 format).


       Infopage: URL
       Infopage: URL
        Link to web page containing more informations about this appliance.
      Link to web page containing more informations about this appliance.
 
 
 
 
 
 


       Description: single line synopsis
       Description: single line synopsis
          extended description over several lines (indended by space) may follow.
        extended description over several lines (indended by space) may follow.
 
 
 


Appliance description file
Appliance description file
       All generated templates includes an appliance description file called
       All generated templates includes an appliance description file called


         /etc/appliance.info
         /etc/appliance.info


       this is the first file inside the tar archive. That way it can be
       this is the first file inside the tar archive. That way it can be
      easily exctracted without scanning the whole archive. The file itself
    easily exctracted without scanning the whole archive. The file itself
      contains informations like a debian "control" file. It can be used to
    contains informations like a debian "control" file. It can be used to
      build appliance repositories.
    build appliance repositories.
 
 
 
 
 
 


       Most fields are directly copied from the configuration file "dab.conf".
       Most fields are directly copied from the configuration file "dab.conf".


       Additionally there are some auto-generated files:
       Additionally there are some auto-generated files:


       Installed-Size: bytes
       Installed-Size: bytes
          It gives the total amount of disk space required to install the
        It gives the total amount of disk space required to install the
          named appliance. The disk space is represented in megabytes as a
        named appliance. The disk space is represented in megabytes as a
          simple decimal number.
        simple decimal number.
 
 
 
 
 
 


       Type: type
       Type: type
          This is always "openvz".
        This is always "openvz".
 
 
 
 
 
 


       OS: [debian-4.0|debian-5.0|ubuntu-8.0]
       OS: [debian-4.0|debian-5.0|ubuntu-8.0]
          Operation system.
        Operation system.
 
 
 
 
 
 


       Appliance repositories usually add additional fields:
       Appliance repositories usually add additional fields:


       md5sum: md5sum
       md5sum: md5sum
          MD5 checksum
        MD5 checksum
 
 
 


FILES
FILES
       The following files are created inside your working directory:
       The following files are created inside your working directory:


         dab.conf          appliance configuration file
         dab.conf          appliance configuration file


         logfile          contains installation logs
         logfile          contains installation logs


         .veid            stores the used container ID
         .veid            stores the used container ID


         cache/*          default package cache directory
         cache/*          default package cache directory


         info/*            package information cache
         info/*            package information cache


AUTHOR
AUTHOR
       Dietmar Maurer <dietmar@proxmox.com>
 
 
 
 
 
 
 
       Dietmar Maurer &lt;dietmar@proxmox.com&gt;
 
 
 
 
 
 


       Many thanks to Proxmox Server Solutions (www.proxmox.com) for
       Many thanks to Proxmox Server Solutions (www.proxmox.com) for
      sponsoring this work.
    sponsoring this work.
 
 
 


COPYRIGHT AND DISCLAIMER
COPYRIGHT AND DISCLAIMER
       Copyright (C) 2007-2009 Proxmox Server Solutions GmbH
       Copyright (C) 2007-2009 Proxmox Server Solutions GmbH


       Copyright: dab is under GNU GPL, the GNU General Public License.
       Copyright: dab is under GNU GPL, the GNU General Public License.


       This program is free software; you can redistribute it and/or modify it
       This program is free software; you can redistribute it and/or modify it
      under the terms of the GNU General Public License as published by the
    under the terms of the GNU General Public License as published by the
      Free Software Foundation; version 2 dated June, 1991.
    Free Software Foundation; version 2 dated June, 1991.
 
 
 
 
 
 


       This program is distributed in the hope that it will be useful, but
       This program is distributed in the hope that it will be useful, but
      WITHOUT ANY WARRANTY; without even the implied warranty of
    WITHOUT ANY WARRANTY; without even the implied warranty of
      MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
      General Public License for more details.
    General Public License for more details.
 
 
 
 
 
 


       You should have received a copy of the GNU General Public License along
       You should have received a copy of the GNU General Public License along
      with this program; if not, write to the Free Software Foundation, Inc.,
    with this program; if not, write to the Free Software Foundation, Inc.,
      51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
    51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
 
 
 


POD ERRORS
POD ERRORS
       Hey! The above document had some coding errors, which are explained
       Hey! The above document had some coding errors, which are explained
      below:
    below:
 
 
 
 
 
 


       Around line 382:
       Around line 382:
          =back without =over
        =back without =over
 
 
 
 
 
 






1.1                              2010-10-04                            dab(1)
</pre>


=Build Virtual Appliances=
We published several examples to show how to work with 'dab'. To start, just build a first OS template based on our examples. If you understand the basics, you can start customizing your own appliance (based on a standard OS template).


Basic recommendations:
*Choose the best suitable base OS (Debian Etch or Lenny, Ubuntu Hardy)
*Choose the right architecture (32 or 64 bit): unless your appliance needs a lot of RAM choose 32 bit
*Always try to use Debian/Ubuntu packages for applications (instead of tar.gz files)
*Pre-configure applications as much as possible (PHP, Apache, MySql, Postgresql, etc.) - based on the recommendations of the software
*Use unique keys and passwords (auto generate them during first start-up)
*Define the update procedure for users running the appliance in production: if you use Debian packages, update is easy (apt-get update)
*Contact the Proxmox support team for help [http://www.proxmox.com/forum/ Proxmox Support Forum]
*And finally: Get certified! See [[Certification Overview]]


==Debian 5 Standard (Lenny, 32bit)==
1.1                              2010-10-04                            dab(1)
This example shows the basic steps when creating appliances with 'dab'. First step is to create a temporary working directory and cd into that directory (or just cd into the previously un-tared examples):
<pre>
mkdir debian-5.0-standard
cd debian-5.0-standard
</pre>


Now you need to create/edit the configuration file called 'dab.conf':


<pre>
Suite: lenny
Architecture: i386
Name: debian-5.0-standard
Version: 5.0-1
Section: system
Maintainer: Proxmox Support Team <support@proxmox.com>
Infopage: http://pve.proxmox.com/wiki/Debian_5.0_Standard
Description: Debian 5.0 (standard)
A small Debian Lenny system including all standard packages.
</pre>


Next step: Download the available package lists:
<pre>dab init</pre>


Start the creation process with:


<pre>dab bootstrap</pre>


Finally build the final appliance package with:
</pre>  
<pre>dab finalize</pre>
= Build Virtual Appliances  =


Thats all. Simply upload the final template to your Proxmox VE server (/var/lib/vz/template/cache/) and test what you created. Sure, always check if everything went well before releasing a template. For this purpose all output is logged to a file called 'logfile'.
We published several examples to show how to work with 'dab'. To start, just build a first OS template based on our examples. If you understand the basics, you can start customizing your own appliance (based on a standard OS template).  


When finished you may want to cleanup your working directory with:
Basic recommendations:  
<pre>dab clean</pre>


or use 'dist-clean' if you also want to erase the package cache directory:
*Choose the best suitable base OS (Debian Etch or Lenny, Ubuntu Hardy)
<pre>dab dist-clean</pre>
*Choose the right architecture (32 or 64 bit): unless your appliance needs a lot of RAM choose 32 bit
*Always try to use Debian/Ubuntu packages for applications (instead of tar.gz files)
*Pre-configure applications as much as possible (PHP, Apache, MySql, Postgresql, etc.) - based on the recommendations of the software
*Use unique keys and passwords (auto generate them during first start-up)
*Define the update procedure for users running the appliance in production: if you use Debian packages, update is easy (apt-get update)
*Contact the Proxmox support team for help [http://www.proxmox.com/forum/ Proxmox Support Forum]
*And finally: Get certified! See [[Certification Overview]]


== Debian 5 Standard (Lenny, 32bit)  ==


You can automate above steps using make. Simply create a file called 'Makefile' (here is an example):
This example shows the basic steps when creating appliances with 'dab'. First step is to create a temporary working directory and cd into that directory (or just cd into the previously un-tared examples):
<pre>mkdir debian-5.0-standard
cd debian-5.0-standard
</pre>
Now you need to create/edit the configuration file called 'dab.conf':
<pre>Suite: lenny
Architecture: i386
Name: debian-5.0-standard
Version: 5.0-1
Section: system
Maintainer: Proxmox Support Team &lt;support@proxmox.com&gt;
Infopage: http://pve.proxmox.com/wiki/Debian_5.0_Standard
Description: Debian 5.0 (standard)
A small Debian Lenny system including all standard packages.
</pre>
Next step: Download the available package lists:
<pre>dab init</pre>
Start the creation process with:
<pre>dab bootstrap</pre>
Finally build the final appliance package with:  
<pre>dab finalize</pre>
Thats all. Simply upload the final template to your Proxmox VE server (/var/lib/vz/template/cache/) and test what you created. Sure, always check if everything went well before releasing a template. For this purpose all output is logged to a file called 'logfile'.


<pre>
When finished you may want to cleanup your working directory with:
BASEDIR:=$(shell dab basedir)
<pre>dab clean</pre>
or use 'dist-clean' if you also want to erase the package cache directory:
<pre>dab dist-clean</pre>  
<br> You can automate above steps using make. Simply create a file called 'Makefile' (here is an example):
<pre>BASEDIR:=$(shell dab basedir)


all: info/init_ok
all: info/init_ok
Line 437: Line 906:
         dab dist-clean
         dab dist-clean
         rm -f *~
         rm -f *~
</pre>
</pre>  
To start template creation you can now simply type:
<pre>make</pre>
and
<pre>make clean</pre>
to cleanup the directory.


To start template creation you can now simply type:
The '''[[Build your first DAB Appliance Template]]''' page shows how to practically setup '''DAB''' and get along building templates.
<pre>make</pre>


and
== Other OS sample Appliances  ==


<pre>make clean</pre>
These example OS templates are included in the [ftp://download.proxmox.com/sources/dab-pve-appliances_2010-02-04.tar.gz dab-pve-appliances] (42.5 MB)


to cleanup the directory.
*[[Debian 4.0 Standard]] (Etch, 32bit)
*Debian 4 Standard (Etch, 64bit)
*[[Debian 5.0 Standard]] (Lenny, 64bit)
*[[Ubuntu Hardy Standard]] (Hardy, 32bit)
*Ubuntu 8.04 Standard (Hardy, 64bit)


The '''[[Build your first DAB Appliance Template]]''' page shows how to practically setup '''DAB''' and get along building templates.
== Sample Application Appliances  ==


==Other OS sample Appliances==
These example application templates are included in the [ftp://download.proxmox.com/sources/dab-pve-appliances_2010-02-04.tar.gz dab-pve-appliances] (42.5 MB)  
These example OS templates are included in the [ftp://download.proxmox.com/sources/dab-pve-appliances_2010-02-04.tar.gz dab-pve-appliances] (42.5 MB)
*[[Debian 4.0 Standard]] (Etch, 32bit)
*Debian 4 Standard (Etch, 64bit)
*[[Debian 5.0 Standard]] (Lenny, 64bit)
*[[Ubuntu Hardy Standard]] (Hardy, 32bit)
*Ubuntu 8.04 Standard (Hardy, 64bit)


==Sample Application Appliances==
*[[Acquia Drupal]] (Debian Etch, 32 bit)  
These example application templates are included in the [ftp://download.proxmox.com/sources/dab-pve-appliances_2010-02-04.tar.gz dab-pve-appliances] (42.5 MB)
*[[Drupal]] (Debian Lenny, 32 bit)  
*[[Acquia Drupal]] (Debian Etch, 32 bit)
*[[MediaWiki]] (Debian Lenny, 32 bit)  
*[[Drupal]] (Debian Lenny, 32 bit)
*[[Joomla!]] (Debian Lenny, 32 bit)  
*[[MediaWiki]] (Debian Lenny, 32 bit)
*[[Wordpress]] (Debian Lenny, 32 bit)  
*[[Joomla!]] (Debian Lenny, 32 bit)
*[[RT Request Tracker]] (Debian Lenny, 32 bit)  
*[[Wordpress]] (Debian Lenny, 32 bit)
*[[RT Request Tracker]] (Debian Lenny, 32 bit)
*[[X11 Terminal Server]] (with GNOME, Debian Lenny 32 bit)
*[[X11 Terminal Server]] (with GNOME, Debian Lenny 32 bit)


=Howto certify a Virtual Appliance=
= Howto certify a Virtual Appliance =
See [[Certification_Overview]]
 
See [[Certification Overview]]  
 
= Get support  =


=Get support=
[[Get support]] including commercial support offering concerning 'dab'.  
[[Get_support]] including commercial support offering concerning 'dab'.


[[Category: Virtual Appliances]][[Category: HOWTO]]
[[Category:Virtual_Appliances]] [[Category:HOWTO]]

Revision as of 11:37, 9 May 2011

Introduction

Creating high quality appliances is a difficult task and requires deep knowledge of the underlying operating system. So we created the 'Debian Appliance Builder' to simplify that task. 'dab' is a script to automate the creation of OpenVZ appliances. It is basically a rewrite of debootstrap in perl, but uses OpenVZ instead of chroot and generates OpenVZ templates. Another difference is that it supports multi-stage building of templates. That way you can execute arbitrary scripts between package installation steps to accomplish what you want.

Furthermore, some common tasks are fully automated - like setting up a database server (mysql or postgres). To accomplish minimal template creation time, packages are cached to a local directory, so you do not need a local Debian mirror (although this would speed up the first run). All generated templates includes an appliance description file. Those can be used to build appliance repositories.

'dab' is licensed under GPLv2 (Open source).

Main features

  • Fully automated build process with standard 'make' scripts
  • Local package cache
  • Multi-stage build process
  • Creates and includes appliance description files
  • Simplifies various tasks like database installation
  • Appliances can be certified and integrated into the Proxmox VE download directory
  • Supports Debian 6 (Squeeze) based appliances
  • Generates 32 or 64 bit appliances
  • Automatic dependency resolution

Target audience

'dab' is an easy way for software vendors and open source project to package their product into a "ready-to-run" appliance. Appliances can be published into the Proxmox VE appliance download directory to reach the target customer. See Download templates (Video).

In order to work with 'dab', basic Linux knowledge is needed.

Installation

The script needs to be run as root, so it is not recommended to start it on a production machine with running containers.

System requirements

Prepare your operating system - you can install 'dab' on the following operating systems:

  • Proxmox VE (fastest)
  • Proxmox VE inside Proxmox VE (KVM) - (most comfortable)
  • Proxmox VE inside VMWare/XEN 64bit virtual machine
  • Debian 64/Ubuntu 64 with OpenVZ Kernel (not tested)

'dab' installation

Log into Proxmox VE, and install 'dab' using apt:

apt-get install dab

Or download and install it manually:

wget http://download.proxmox.com/debian/dists/lenny/pve/binary-amd64/dab_1.1-14_all.deb
dpkg -i dab_1.1-14_all.deb

'dab' sample templates

A set of examples is also available. Simply download them:

Check for current examples at ftp://download.proxmox.com/sources/

wget ftp://download.proxmox.com/sources/dab-pve-appliances_2010-05-25.tar.gz

Unpack with tar:

tar -zxvf dab-pve-appliances_2010-05-25.tar.gz

'dab' manual pages

The package includes a manual page:

man dab
dab(1)                User Contributed Perl Documentation               dab(1)





NAME




      dab - Debian OpenVZ Appliance Builder



SYNOPSIS




      dab command [OPTIONS]
    dab init
        Downloads the package descriptions form the repository. Also
        truncates the "logfile".




      dab bootstrap
        Bootstrap a debian system and allocate a temporary container (we
        use IDs 90000 and above).




          --exim
            Use exim as MTA (we use postfix by default)




          --minimal
            Do not install standard packages.




      dab veid
        Print used container ID.




      dab basedir
        Print container private directory.




      dab packagefile
        Print the appliance file name.




      dab install pkg ...
        Install one or more packages. pkg can also refer to a file named
        "xyz.pkglist" which contains a list of packages. All dependencies
        are automatically installed.




      dab unpack pkg ...
        Unpack one or more packages. pkg can also refer to a file named
        "xyz.pkglist" which contains a list of packages. All dependencies
        are automatically unpacked.




      dab exec CMD ARGS
        Executes command CMD inside the container.




      dab enter
        Calls "vzctl enter CTID" - this is for debugging only.




      dab task mysql
        Install a mysql database server. During appliance generation we use
        "admin" as mysql root password (also stored in /root/.my.cnf).




          --password=XXX
            Specify the mysql root password. The special value "random" can
            be use to generate a random root password when the appliance is
            started first time (stored in /root/.my.cnf)




          --start
            Start the mysql server (if you want to execute sql commands
            during appliance generation).




      dab task postgres
        Install a postgres database server.




          --version=XXX
            Select Postgres version. Posible values are 7.4, 8.1 and 8.3
            (depends on the selected suite).




          --start
            Start the postgres server (if you want to execute sql commands
            during appliance generation).




      dab task php
        Install php5.




          --memlimit=i
            Set the php memory_limit.




      dab finalize
        Cleanup everything inside the container and generate the final
        appliance package.




          --keepmycnf
            Do not delete file "/root/.my.cfg" (mysql).




      dab list
        List installed packages.




          --verbose
            Also print package versions.




      dab clean
        Remove all temporary file and destroy the used OpenVZ container.




      dab dist-clean
        Like clean, but also removes the package cache (except when you
        specified your own cache directory in the config file)



DESCRIPTION




      dab is a script to automate the creation of OpenVZ appliances. It is
    basically a rewrite of debootstrap in perl, but uses OpenVZ instead of
    chroot and generates OpenVZ templates. Another difference is that it
    supports multi-stage building of templates. That way you can execute
    arbitrary scripts between to accomplish what you want.




      Furthermore some common tasks are fully automated, like setting up a
    database server (mysql or postgres).




      To accomplish minimal template creation time, packages are cached to a
    local directory, so you do not need a local debian mirror (although
    this would speed up the first run).




      See http://pve.proxmox.com/wiki/Debian_Appliance_Builder for examples.




      This script need to be run as root, so it is not recommended to start
    it on a production machine with running container. Proxmox VE
    (http://pve.proxmox.com) is the preferred environment, because it is
    able to log the console output when a container starts. You wont be
    able to detect errors during container startup when running on standard
    OpenVZ. So many people run Proxmox VE inside a KVM or VMWare 64bit
    virtual machine to build appliances.




      All generated templates includes an appliance description file. Those
    can be used to build appliance repositories.



CONFIGURATION




      Configuration is read from the file "dab.conf" inside the current
    working directory. The files contains key value pairs, separated by
    colon.




      Suite: etch|lenny|squeeze|hardy|intrepid|jaunty
      The Debian or Ubuntu suite.




      Source: URL [components]
      Defines a source location. By default we use the following for
      debian:




         Source: http://ftp.debian.org/debian SUITE main contrib
       Source: http://security.debian.org SUITE/updates main contrib




        Note: SUITE is a variable and will be substituted.




        There are also reasonable defaults for Ubuntu. If you do not specify
      any source the defaults are used.




      Depends: dependencies
      Debian like package dependencies. This can be used to make sure that
      speific package versions are available.




      CacheDir: path
      Allows you to specify the directory where downloaded packages are
      cached.




      Mirror: SRCURL => DSTURL
      Define a mirror location. for example:




         Mirror: http://ftp.debian.org/debian => ftp://mirror/debian




      All other settings in this files are also included into the appliance
    description file.




      Name: name
      The name of the appliance.




        Appliance names must consist only of lower case letters (a-z), digits
      (0-9), plus (+) and minus (-) signs, and periods (.). They must be at
      least two characters long and must start with an alphanumeric
      character.




      Architecture: i386|amd64
      Target architecture.




      Version: upstream_version[-build_revision]
      The version number of an appliance.




        =item: Section: section




        This field specifies an application area into which the appliance has
      been classified. Currently we use the following section names:
      system, admin, www




      Maintainer: name <email>
      The appliance maintainer's name and email address. The name should
      come first, then the email address inside angle brackets <> (in
      RFC822 format).




      Infopage: URL
      Link to web page containing more informations about this appliance.




      Description: single line synopsis
       extended description over several lines (indended by space) may follow.



Appliance description file




      All generated templates includes an appliance description file called




       /etc/appliance.info




      this is the first file inside the tar archive. That way it can be
    easily exctracted without scanning the whole archive. The file itself
    contains informations like a debian "control" file. It can be used to
    build appliance repositories.




      Most fields are directly copied from the configuration file "dab.conf".




      Additionally there are some auto-generated files:




      Installed-Size: bytes
        It gives the total amount of disk space required to install the
        named appliance. The disk space is represented in megabytes as a
        simple decimal number.




      Type: type
        This is always "openvz".




      OS: [debian-4.0|debian-5.0|ubuntu-8.0]
        Operation system.




      Appliance repositories usually add additional fields:




      md5sum: md5sum
        MD5 checksum



FILES




      The following files are created inside your working directory:




       dab.conf          appliance configuration file




       logfile           contains installation logs




       .veid             stores the used container ID




       cache/*           default package cache directory




       info/*            package information cache



AUTHOR




      Dietmar Maurer <dietmar@proxmox.com>




      Many thanks to Proxmox Server Solutions (www.proxmox.com) for
    sponsoring this work.



COPYRIGHT AND DISCLAIMER




      Copyright (C) 2007-2009 Proxmox Server Solutions GmbH




      Copyright: dab is under GNU GPL, the GNU General Public License.




      This program is free software; you can redistribute it and/or modify it
    under the terms of the GNU General Public License as published by the
    Free Software Foundation; version 2 dated June, 1991.




      This program is distributed in the hope that it will be useful, but
    WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
    General Public License for more details.




      You should have received a copy of the GNU General Public License along
    with this program; if not, write to the Free Software Foundation, Inc.,
    51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.



POD ERRORS




      Hey! The above document had some coding errors, which are explained
    below:




      Around line 382:
        =back without =over







1.1 2010-10-04 dab(1)




Build Virtual Appliances

We published several examples to show how to work with 'dab'. To start, just build a first OS template based on our examples. If you understand the basics, you can start customizing your own appliance (based on a standard OS template).

Basic recommendations:

  • Choose the best suitable base OS (Debian Etch or Lenny, Ubuntu Hardy)
  • Choose the right architecture (32 or 64 bit): unless your appliance needs a lot of RAM choose 32 bit
  • Always try to use Debian/Ubuntu packages for applications (instead of tar.gz files)
  • Pre-configure applications as much as possible (PHP, Apache, MySql, Postgresql, etc.) - based on the recommendations of the software
  • Use unique keys and passwords (auto generate them during first start-up)
  • Define the update procedure for users running the appliance in production: if you use Debian packages, update is easy (apt-get update)
  • Contact the Proxmox support team for help Proxmox Support Forum
  • And finally: Get certified! See Certification Overview

Debian 5 Standard (Lenny, 32bit)

This example shows the basic steps when creating appliances with 'dab'. First step is to create a temporary working directory and cd into that directory (or just cd into the previously un-tared examples):

mkdir debian-5.0-standard
cd debian-5.0-standard

Now you need to create/edit the configuration file called 'dab.conf':

Suite: lenny
Architecture: i386
Name: debian-5.0-standard
Version: 5.0-1
Section: system
Maintainer: Proxmox Support Team <support@proxmox.com>
Infopage: http://pve.proxmox.com/wiki/Debian_5.0_Standard
Description: Debian 5.0 (standard)
 A small Debian Lenny system including all standard packages.

Next step: Download the available package lists:

dab init

Start the creation process with:

dab bootstrap

Finally build the final appliance package with:

dab finalize

Thats all. Simply upload the final template to your Proxmox VE server (/var/lib/vz/template/cache/) and test what you created. Sure, always check if everything went well before releasing a template. For this purpose all output is logged to a file called 'logfile'.

When finished you may want to cleanup your working directory with:

dab clean

or use 'dist-clean' if you also want to erase the package cache directory:

dab dist-clean


You can automate above steps using make. Simply create a file called 'Makefile' (here is an example):

BASEDIR:=$(shell dab basedir)

all: info/init_ok
        dab bootstrap
        dab finalize

info/init_ok: dab.conf
        dab init
        touch $@

.PHONY: clean
clean:
        dab clean
        rm -f *~

.PHONY: dist-clean
dist-clean:
        dab dist-clean
        rm -f *~

To start template creation you can now simply type:

make

and

make clean

to cleanup the directory.

The Build your first DAB Appliance Template page shows how to practically setup DAB and get along building templates.

Other OS sample Appliances

These example OS templates are included in the dab-pve-appliances (42.5 MB)

Sample Application Appliances

These example application templates are included in the dab-pve-appliances (42.5 MB)

Howto certify a Virtual Appliance

See Certification Overview

Get support

Get support including commercial support offering concerning 'dab'.