https://pve.proxmox.com/mediawiki/api.php?action=feedcontributions&user=Peter712&feedformat=atomProxmox VE - User contributions [en]2024-03-28T20:34:02ZUser contributionsMediaWiki 1.35.11https://pve.proxmox.com/mediawiki/index.php?title=Proxmox_Cluster_file_system_(pmxcfs)&diff=4046Proxmox Cluster file system (pmxcfs)2012-03-03T22:12:55Z<p>Peter712: </p>
<hr />
<div>{{Note|Article about Proxmox VE 2.0 beta}}<br />
=Introduction=<br />
<br />
Proxmox Cluster file system (pmxcfs) is a database-driven file system for storing configuration files, replicated in real time on all nodes using corosync. We use this to store all PVE related configuration files. Although the file system stores all data inside a persistent database on disk, a copy of the data resides in RAM. That imposes restriction on the maximal size, which is currently 30MB. This is still enough to store the configuration of several thousand virtual machines.<br />
<br />
====Advantages====<br />
<br />
* seamless replication of all configuration to all nodes in real time<br />
* provides strong consistency checks to avoid duplicate VM IDs<br />
* read-only when a node looses quorum<br />
* automatic updates of the corosync cluster configuration to all nodes.<br />
<br />
====POSIX Compatibility====<br />
<br />
The file system is based on fuse, so the behavior is POSIX like. But<br />
many feature are simply not implemented, because we do not need them:<br />
<br />
* you can just generate normal files and directories, but no symbolic links, ...<br />
* you can't rename non-empty directories (because this makes it easier to guarantee that VMIDs are unique).<br />
* you can't change file permissions (permissions are based on path)<br />
* O_EXCL creates were not atomic (like old NFS)<br />
* O_TRUNC creates are not atomic (fuse restriction)<br />
* ...<br />
<br />
====File access rights====<br />
<br />
All files/dirs are owned by user 'root' and have group<br />
'www-data'. Only root has write permissions, but group 'www-data' can<br />
read most files. Files below the following paths:<br />
<br />
/etc/pve/priv/<br />
/etc/pve/nodes/${NAME}/priv/<br />
<br />
are only accessible by root.<br />
<br />
=Technologie=<br />
<br />
We use the [http://www.corosync.org Corosync Cluster Engine] for cluster communication, and [http://www.sqlite.org SQlite] for the database file. The filesystem is implemented in user space using [http://fuse.sourceforge.net/ FUSE].<br />
<br />
<br />
=File system layout=<br />
<br />
The file system is mounted at:<br />
<br />
/etc/pve<br />
<br />
====Files====<br />
<br />
cluster.conf => corosync/cman cluster configuration file<br />
storage.cfg => PVE storage configuration<br />
user.cfg => PVE access control configuration (users/groups/...)<br />
domains.cfg => PVE Authentication domains <br />
authkey.pub => public key used by ticket system<br />
<br />
priv/shadow.cfg => shadow password file<br />
priv/authkey.key => private key used by ticket system<br />
<br />
nodes/${NAME}/pve-ssl.pem => public ssl key fo apache<br />
nodes/${NAME}/priv/pve-ssl.key => private ssl key<br />
nodes/${NAME}/qemu-server/${VMID}.conf => VM configuration data for KVM VMs<br />
nodes/${NAME}/openvz/${VMID}.conf => VM configuratin data for OpenVZ containers<br />
<br />
====Symbolic links====<br />
<br />
local => nodes/${LOCALNAME}<br />
qemu-server => nodes/${LOCALNAME}/qemu-server/<br />
openvz => nodes/${LOCALNAME}/openvz/<br />
<br />
====Special status files for debugging (JSON)====<br />
<br />
.version => file versions (to detect file modifications)<br />
.members => Info about cluster members<br />
.vmlist => List of all VMs<br />
.clusterlog => Cluster log (last 50 entries)<br />
.rrd => RRD data (most recent entries)<br />
<br />
<br />
====Enable/Disable debugging====<br />
<br />
# enable verbose syslog messages<br />
echo "1" >/etc/pve/.debug <br />
<br />
# disable verbose syslog messages<br />
echo "0" >/etc/pve/.debug <br />
<br />
=Source code=<br />
<br />
The source code is available [https://git.proxmox.com/?p=pve-cluster.git;a=summary here].<br />
<br />
[[Category: Proxmox VE 2.0]]</div>Peter712https://pve.proxmox.com/mediawiki/index.php?title=Debian_Appliance_Builder&diff=2052Debian Appliance Builder2010-06-19T16:58:11Z<p>Peter712: /* 'dab' sample templates */</p>
<hr />
<div>=Introduction=<br />
Creating high quality appliances is a difficult task and requires deep knowledge of the underlying operating system. So we created the 'Debian Appliance Builder' to simplify that task. <br />
'dab' is a script to automate the creation of OpenVZ appliances. It is basically a rewrite of debootstrap in perl, but uses OpenVZ instead of chroot and generates OpenVZ templates. Another difference is that it supports multi-stage building of templates. That way you can execute arbitrary scripts between package installation steps to accomplish what you want.<br />
<br />
Furthermore, some common tasks are fully automated - like setting up a database server (mysql or postgres). To accomplish minimal template creation time, packages are cached to a local directory, so you do not need a local Debian/Ubuntu mirror (although this would speed up the first run).<br />
All generated templates includes an appliance description file. Those can be used to build appliance repositories.<br />
<br />
'dab' is licensed under GPLv2 (Open source).<br />
==Main features==<br />
*Fully automated build process with standard 'make' scripts<br />
*Local package cache<br />
*Multi-stage build process<br />
*Creates and includes appliance description files<br />
*Simplifies various tasks like database installation<br />
*Appliances can be certified and integrated into the Proxmox VE download directory<br />
*Supports Debian and Ubuntu based appliances<br />
*Generates 32 or 64 bit appliances<br />
*Automatic dependency resolution<br />
<br />
==Target audience==<br />
'dab' is an easy way for software vendors and open source project to package their product into a "ready-to-run" appliance. Appliances can be published into the Proxmox VE appliance download directory to reach the target customer. See [[Download_templates_(Video)]].<br />
<br />
In order to work with 'dab', basic Linux knowledge is needed.<br />
<br />
=Installation=<br />
The script needs to be run as root, so it is not recommended to start it on a production machine with running containers.<br />
==System requirements==<br />
Prepare your operating system - you can install 'dab' on the following operating systems:<br />
*Proxmox VE (fastest)<br />
*Proxmox VE inside Proxmox VE (KVM) - (most comfortable) <br />
*Proxmox VE inside VMWare/XEN 64bit virtual machine<br />
*Debian 64/Ubuntu 64 with OpenVZ Kernel (not tested)<br />
<br />
=='dab' installation==<br />
Log into Proxmox VE, and install 'dab' using apt:<br />
<pre>apt-get install dab</pre><br />
<br />
Or download and install it manually from:<br />
<br />
<pre>wget http://download.proxmox.com/debian/dists/lenny/pve/binary-amd64/dab_1.1-3_all.deb</pre><br />
<br />
A sample install on Proxmox VE 1.3 will output the following:<br />
<pre><br />
apt-get install dab<br />
<br />
Reading package lists... Done<br />
Building dependency tree<br />
Reading state information... Done<br />
The following extra packages will be installed:<br />
binutils make<br />
Suggested packages:<br />
binutils-doc make-doc<br />
The following NEW packages will be installed:<br />
binutils dab make<br />
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.<br />
Need to get 3157kB of archives.<br />
After this operation, 9351kB of additional disk space will be used.<br />
Do you want to continue [Y/n]? y<br />
Get:1 http://download.proxmox.com lenny/pve dab 1.1-3 [26.7kB]<br />
Get:2 http://ftp.debian.org lenny/main binutils 2.18.1~cvs20080103-7 [2730kB]<br />
Get:3 http://ftp.debian.org lenny/main make 3.81-5 [401kB]<br />
Fetched 3157kB in 19s (160kB/s)<br />
Selecting previously deselected package binutils.<br />
(Reading database ... 24838 files and directories currently installed.)<br />
Unpacking binutils (from .../binutils_2.18.1~cvs20080103-7_amd64.deb) ...<br />
Selecting previously deselected package make.<br />
Unpacking make (from .../archives/make_3.81-5_amd64.deb) ...<br />
Selecting previously deselected package dab.<br />
Unpacking dab (from .../apt/archives/dab_1.1-3_all.deb) ...<br />
Processing triggers for man-db ...<br />
Setting up binutils (2.18.1~cvs20080103-7) ...<br />
Setting up make (3.81-5) ...<br />
Setting up dab (1.1-3) ...<br />
</pre><br />
<br />
=='dab' sample templates==<br />
A set of examples is also available. Simply download them:<br />
<br />
Check for current examples at ftp://download.proxmox.com/sources/<br />
<br />
<pre>wget ftp://download.proxmox.com/sources/dab-pve-appliances_2010-05-25.tar.gz</pre><br />
<br />
Unpack with tar:<br />
<br />
<pre>tar -zxvf dab-pve-appliances_2010-05-25.tar.gz</pre><br />
<br />
=='dab' manual pages==<br />
The package includes a manual page:<br />
<pre>man dab</pre><br />
<br />
<pre><br />
dab(1) User Contributed Perl Documentation dab(1)<br />
<br />
NAME<br />
dab - Debian OpenVZ Appliance Builder<br />
<br />
SYNOPSIS<br />
dab command [OPTIONS]<br />
dab init<br />
Downloads the package descriptions form the repository. Also<br />
truncates the "logfile".<br />
<br />
dab bootstrap<br />
Bootstrap a debian system and allocate a temporary container (we<br />
use IDs 90000 and above).<br />
<br />
--exim<br />
Use exim as MTA (we use postfix by default)<br />
<br />
--minimal<br />
Do not install standard packages.<br />
<br />
dab veid<br />
Print used container ID.<br />
<br />
dab basedir<br />
Print container private directory.<br />
<br />
dab packagefile<br />
Print the appliance file name.<br />
<br />
dab install pkg ...<br />
Install one or more packages. pkg can also refer to a file named<br />
"xyz.pkglist" which contains a list of packages. All dependencies<br />
are automatically installed.<br />
<br />
dab unpack pkg ...<br />
Unpack one or more packages. pkg can also refer to a file named<br />
"xyz.pkglist" which contains a list of packages. All dependencies<br />
are automatically unpacked.<br />
<br />
dab exec CMD ARGS<br />
Executes command CMD inside the container.<br />
<br />
dab enter<br />
Calls "vzctl enter CTID" - this is for debugging only.<br />
<br />
dab task mysql<br />
Install a mysql database server. During appliance generation we use<br />
"admin" as mysql root password (also stored in /root/.my.cnf).<br />
<br />
--password=XXX<br />
Specify the mysql root password. The special value "random" can<br />
be use to generate a random root password when the appliance is<br />
started first time (stored in /root/.my.cnf)<br />
<br />
--start<br />
Start the mysql server (if you want to execute sql commands<br />
during appliance generation).<br />
<br />
dab task postgres<br />
Install a postgres database server.<br />
<br />
--version=XXX<br />
Select Postgres version. Posible values are 7.4, 8.1 and 8.3<br />
(depends on the selected suite).<br />
<br />
--start<br />
Start the postgres server (if you want to execute sql commands<br />
during appliance generation).<br />
<br />
dab task php<br />
Install php5.<br />
<br />
--memlimit=i<br />
Set the php memory_limit.<br />
<br />
dab finalize<br />
Cleanup everything inside the container and generate the final<br />
appliance package.<br />
<br />
--keepmycnf<br />
Do not delete file "/root/.my.cfg" (mysql).<br />
<br />
dab list<br />
List installed packages.<br />
<br />
--verbose<br />
Also print package versions.<br />
<br />
dab clean<br />
Remove all temporary file and destroy the used OpenVZ container.<br />
<br />
dab dist-clean<br />
Like clean, but also removes the package cache (except when you<br />
specified your own cache directory in the config file)<br />
<br />
DESCRIPTION<br />
dab is a script to automate the creation of OpenVZ appliances. It is<br />
basically a rewrite of debootstrap in perl, but uses OpenVZ instead of<br />
chroot and generates OpenVZ templates. Another difference is that it<br />
supports multi-stage building of templates. That way you can execute<br />
arbitrary scripts between to accomplish what you want.<br />
<br />
Furthermore some common tasks are fully automated, like setting up a<br />
database server (mysql or postgres).<br />
<br />
To accomplish minimal template creation time, packages are cached to a<br />
local directory, so you do not need a local debian mirror (although<br />
this would speed up the first run).<br />
<br />
See http://pve.proxmox.com/wiki/Debian_Appliance_Builder for examples.<br />
<br />
This script need to be run as root, so it is not recommended to start<br />
it on a production machine with running container. Proxmox VE<br />
(http://pve.proxmox.com) is the preferred environment, because it is<br />
able to log the console output when a container starts. You wont be<br />
able to detect errors during container startup when running on standard<br />
OpenVZ. So many people run Proxmox VE inside a KVM or VMWare 64bit<br />
virtual machine to build appliances.<br />
<br />
All generated templates includes an appliance description file. Those<br />
can be used to build appliance repositories.<br />
<br />
CONFIGURATION<br />
Configuration is read from the file "dab.conf" inside the current<br />
working directory. The files contains key value pairs, separated by<br />
colon.<br />
<br />
Suite: etch|lenny|hardy|intrepid|jaunty<br />
The Debian or Ubuntu suite.<br />
<br />
Source: URL [components]<br />
Defines a source location. By default we use the following for<br />
debian:<br />
<br />
Source: http://ftp.debian.org/debian SUITE main contrib<br />
Source: http://security.debian.org SUITE/updates main contrib<br />
Note: SUITE is a variable and will be substituted.<br />
<br />
There are also reasonable defaults for Ubuntu. If you do not specify<br />
any source the defaults are used.<br />
<br />
Depends: dependencies<br />
Debian like package dependencies. This can be used to make sure that<br />
speific package versions are available.<br />
<br />
CacheDir: path<br />
Allows you to specify the directory where downloaded packages are<br />
cached.<br />
<br />
Mirror: SRCURL => DSTURL<br />
Define a mirror location. for example:<br />
<br />
Mirror: http://ftp.debian.org/debian => ftp://mirror/debian<br />
<br />
All other settings in this files are also included into the appliance<br />
description file.<br />
<br />
Name: name<br />
The name of the appliance.<br />
<br />
Appliance names must consist only of lower case letters (a-z), digits<br />
(0-9), plus (+) and minus (-) signs, and periods (.). They must be at<br />
least two characters long and must start with an alphanumeric<br />
character.<br />
<br />
Architecture: i386|amd64<br />
Target architecture.<br />
<br />
Version: upstream_version[-build_revision]<br />
The version number of an appliance.<br />
<br />
=item: Section: section<br />
<br />
This field specifies an application area into which the appliance has<br />
been classified. Currently we use the following section names:<br />
system, admin, www<br />
<br />
Maintainer: name <email><br />
The appliance maintainer's name and email address. The name should<br />
come first, then the email address inside angle brackets <> (in<br />
RFC822 format).<br />
<br />
Infopage: URL<br />
Link to web page containing more informations about this appliance.<br />
<br />
Description: single line synopsis<br />
extended description over several lines (indended by space) may follow.<br />
<br />
Appliance description file<br />
All generated templates includes an appliance description file called<br />
<br />
/etc/appliance.info<br />
<br />
this is the first file inside the tar archive. That way it can be<br />
easily exctracted without scanning the whole archive. The file itself<br />
contains informations like a debian "control" file. It can be used to<br />
build appliance repositories.<br />
<br />
Most fields are directly copied from the configuration file "dab.conf".<br />
<br />
Additionally there are some auto-generated files:<br />
<br />
Installed-Size: bytes<br />
It gives the total amount of disk space required to install the<br />
named appliance. The disk space is represented in megabytes as a<br />
simple decimal number.<br />
<br />
Type: type<br />
This is always "openvz".<br />
<br />
OS: [debian-4.0|debian-5.0|ubuntu-8.0]<br />
Operation system.<br />
<br />
Appliance repositories usually add additional fields:<br />
<br />
md5sum: md5sum<br />
MD5 checksum<br />
<br />
FILES<br />
The following files are created inside your working directory:<br />
<br />
dab.conf appliance configuration file<br />
<br />
logfile contains installation logs<br />
<br />
.veid stores the used container ID<br />
<br />
cache/* default package cache directory<br />
<br />
info/* package information cache<br />
<br />
AUTHOR<br />
Dietmar Maurer <dietmar@proxmox.com><br />
<br />
Many thanks to Proxmox Server Solutions (www.proxmox.com) for<br />
sponsoring this work.<br />
<br />
COPYRIGHT AND DISCLAIMER<br />
Copyright (C) 2007-2009 Proxmox Server Solutions GmbH<br />
<br />
Copyright: dab is under GNU GPL, the GNU General Public License.<br />
<br />
This program is free software; you can redistribute it and/or modify it<br />
under the terms of the GNU General Public License as published by the<br />
Free Software Foundation; version 2 dated June, 1991.<br />
<br />
This program is distributed in the hope that it will be useful, but<br />
WITHOUT ANY WARRANTY; without even the implied warranty of<br />
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU<br />
General Public License for more details.<br />
<br />
You should have received a copy of the GNU General Public License along<br />
with this program; if not, write to the Free Software Foundation, Inc.,<br />
51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.<br />
<br />
POD ERRORS<br />
Hey! The above document had some coding errors, which are explained<br />
below:<br />
<br />
Around line 382:<br />
=back without =over<br />
<br />
1.1 2009-05-28 dab(1)<br />
</pre><br />
<br />
=Build Virtual Appliances=<br />
We published several examples to show how to work with 'dab'. To start, just build a first OS template based on our examples. If you understand the basics, you can start customizing your own appliance (based on a standard OS template).<br />
<br />
Basic recommendations:<br />
*Choose the best suitable base OS (Debian Etch or Lenny, Ubuntu Hardy)<br />
*Choose the right architecture (32 or 64 bit): unless your appliance needs a lot of RAM choose 32 bit<br />
*Always try to use Debian/Ubuntu packages for applications (instead of tar.gz files)<br />
*Pre-configure applications as much as possible (PHP, Apache, MySql, Postgresql, etc.) - based on the recommendations of the software<br />
*Use unique keys and passwords (auto generate them during first start-up)<br />
*Define the update procedure for users running the appliance in production: if you use Debian packages, update is easy (apt-get update)<br />
*Contact the Proxmox support team for help [http://www.proxmox.com/forum/ Proxmox Support Forum]<br />
*And finally: Get certified! See [[Certification Overview]]<br />
<br />
==Debian 5 Standard (Lenny, 32bit)==<br />
This example shows the basic steps when creating appliances with 'dab'. First step is to create a temporary working directory and cd into that directory (or just cd into the previously un-tared examples):<br />
<pre><br />
mkdir debian-5.0-standard<br />
cd debian-5.0-standard<br />
</pre><br />
<br />
Now you need to create/edit the configuration file called 'dab.conf':<br />
<br />
<pre><br />
Suite: lenny<br />
Architecture: i386<br />
Name: debian-5.0-standard<br />
Version: 5.0-1<br />
Section: system<br />
Maintainer: Proxmox Support Team <support@proxmox.com><br />
Infopage: http://pve.proxmox.com/wiki/Debian_5.0_Standard<br />
Description: Debian 5.0 (standard)<br />
A small Debian Lenny system including all standard packages.<br />
</pre><br />
<br />
Next step: Download the available package lists:<br />
<pre>dab init</pre><br />
<br />
Start the creation process with:<br />
<br />
<pre>dab bootstrap</pre><br />
<br />
Finally build the final appliance package with:<br />
<pre>dab finalize</pre><br />
<br />
Thats all. Simply upload the final template to your Proxmox VE server (/var/lib/vz/template/cache/) and test what you created. Sure, always check if everything went well before releasing a template. For this purpose all output is logged to a file called 'logfile'.<br />
<br />
When finished you may want to cleanup your working directory with:<br />
<pre>dab clean</pre><br />
<br />
or use 'dist-clean' if you also want to erase the package cache directory:<br />
<pre>dab dist-clean</pre><br />
<br />
<br />
You can automate above steps using make. Simply create a file called 'Makefile' (here is an example):<br />
<br />
<pre><br />
BASEDIR:=$(shell dab basedir)<br />
<br />
all: info/init_ok<br />
dab bootstrap<br />
dab finalize<br />
<br />
info/init_ok: dab.conf<br />
dab init<br />
touch $@<br />
<br />
.PHONY: clean<br />
clean:<br />
dab clean<br />
rm -f *~<br />
<br />
.PHONY: dist-clean<br />
dist-clean:<br />
dab dist-clean<br />
rm -f *~<br />
</pre><br />
<br />
To start template creation you can now simply type:<br />
<pre>make</pre><br />
<br />
and<br />
<br />
<pre>make clean</pre><br />
<br />
to cleanup the directory.<br />
<br />
==Other OS sample Appliances==<br />
These example OS templates are included in the [ftp://download.proxmox.com/sources/dab-pve-appliances_2010-02-04.tar.gz dab-pve-appliances] (42.5 MB)<br />
*[[Debian 4.0 Standard]] (Etch, 32bit)<br />
*Debian 4 Standard (Etch, 64bit)<br />
*[[Debian 5.0 Standard]] (Lenny, 64bit)<br />
*[[Ubuntu Hardy Standard]] (Hardy, 32bit)<br />
*Ubuntu 8.04 Standard (Hardy, 64bit)<br />
<br />
==Sample Application Appliances==<br />
These example application templates are included in the [ftp://download.proxmox.com/sources/dab-pve-appliances_2010-02-04.tar.gz dab-pve-appliances] (42.5 MB)<br />
*[[Acquia Drupal]] (Debian Etch, 32 bit)<br />
*[[Drupal]] (Debian Lenny, 32 bit)<br />
*[[MediaWiki]] (Debian Lenny, 32 bit)<br />
*[[Joomla!]] (Debian Lenny, 32 bit)<br />
*[[Wordpress]] (Debian Lenny, 32 bit)<br />
*[[RT Request Tracker]] (Debian Lenny, 32 bit)<br />
*[[X11 Terminal Server]] (with GNOME, Debian Lenny 32 bit)<br />
<br />
=Howto certify a Virtual Appliance=<br />
See [[Certification_Overview]]<br />
<br />
=Get support=<br />
[[Get_support]] including commercial support offering concerning 'dab'.<br />
<br />
[[Category: Virtual Appliances]][[Category: HOWTO]]</div>Peter712