Storage: ZFS
Introduction
tbd. (running zfs on Proxmox VE is not officially supported)
Native ZFS for Linux on Proxmox
2013-03-27: 0.6.1 ZFSOnLinux (ZoL) is now ready for wide scale deployment on everything from desktops to super computers. See announcement
check http://zfsonlinux.org/ See the Documentation and Community Resources for more information and help.
using Debian Wheezy packages from zfsonlinux
For PVE on Wheezy [ 3.0+ ].
2013-05-29 - Debian Packages DKMS style packages for Debian are available from the zfsonlinux.org repository. These packages track the latest official upstream tag and are refreshed as new releases are made available from http://zfsonlinux.org/debian.html .
also check http://pthree.org/2012/04/17/install-zfs-on-debian-gnulinux/
- If you are upgrading from the Ubuntu PPA method remove ubuntu packages first.
aptitude remove ubuntu-zfs
- If this is a new install or you are repeating a previously failed attempt
dpkg --purge zfsonlinux zfsutils libzfs1 zfs-dkms dkms
- move the old zfs.list from /etc/apt/sources.list.d
- make sure pve headers are installed. if not :
aptitude install pve-headers-$(uname -r)
- I think the ln is still needed.
ln -s /lib/modules/$(uname -r)/build /lib/modules/$(uname -r)/source
- then follow instructions from http://zfsonlinux.org/debian.html , i copied here , but check link in case these changed:
su - wget http://archive.zfsonlinux.org/debian/pool/main/z/zfsonlinux/zfsonlinux_1%7Ewheezy_all.deb dpkg -i zfsonlinux_1~wheezy_all.deb apt-get update apt-get install debian-zfs
you should see zfs and spl module builds... check if OK
# ls -l /lib/modules/$(uname -r)/updates/dkms/ total 3128 -rw-r--r-- 1 root root 340944 May 29 10:25 splat.ko -rw-r--r-- 1 root root 302104 May 29 10:25 spl.ko -rw-r--r-- 1 root root 13392 May 29 10:27 zavl.ko -rw-r--r-- 1 root root 71232 May 29 10:27 zcommon.ko -rw-r--r-- 1 root root 1935120 May 29 10:27 zfs.ko -rw-r--r-- 1 root root 130408 May 29 10:27 znvpair.ko -rw-r--r-- 1 root root 40424 May 29 10:27 zpios.ko -rw-r--r-- 1 root root 330368 May 29 10:27 zunicode.ko
TBD:
- test the above again and update the instructions as necessary.
- does dkms auto build modules when pve kernel is upgraded? for that to happen headers would need to be installed and the ln may be needed.
kernel upgrade
currently this needs to be done when a kernel upgrade occurs.
- before reboot:
- install pve-headers
- make a symlink
- make spl and zfs modules for new kernel.
this is an example, replace the xx with new pver kernel number. and there is probably a grep/sed or some way to put the kernel version in the following..
aptitude install pve-headers-2.6.32-99-pve ln -s /lib/modules/2.6.32-99-pve/build /lib/modules/2.6.32-99-pve/source
- this will force modules to be built ( there is probably a better way to do this , as all kernels get their zfs and spl modules rebuilt, and we just need the new kernel modules... ).
aptitude reinstall spl-dkms zfs-dkms
zfs links and docs
and this has some very important information to know before implementing zfs on a production system.
check these well written manual pages:
man zfs man zpool
How to install ZFS-fuse under Proxmox
(Tested with Kernel 2.6.32)
apt-get install build-essential libaio-dev libattr1-dev libacl1-dev libz-dev libz-dev libfuse-dev libfuse2 scons libssl-dev wget -nd http://zfs-fuse.net/releases/0.6.0/zfs-fuse-0.6.0.tar.bz2 tar jxvf zfs-fuse-0.6.0.tar.bz2 cd zfs-fuse-0.6.0 cd src scons scond install
Note: zfs-fuse-0.6.0 is now (Dec 2012) deprecated. Use zfs-fuse-0.7.0.tar.bz2 instead.
Fire up ZFS-fuse daemon
/usr/local/sbin/zfs-fuse
Create Zpool
zpool create -m /var/lib/vz/images2 /dev/sdb
Create ZFS
zfs create images2/109
ZFS clone a vm 109 to vm 110
zfs snapshot images2/109@master zfs clone images2/109@master images2/110
zfs mounting workaround
The default zfs mount -a script runs too late in the boot process for most system scripts. The following make zfs mounts start on time.
2014-01-22 the info below came from this excellent wiki page: http://wiki.complete.org/ConvertingToZFS
- Edit /etc/default/zfs and set ZFS_MOUNT='yes'
- edit /etc/insserv.conf,
- and at the end of the $local_fs line,
- add zfs-mount (without a plus).
edit /etc/init.d/zfs-mount and find three lines near the top, changing them like this:
# Required-Start: # Required-Stop: # Default-Start: S
note remove the Required-Start and -Stop entries.
- Activating init.d changes Then run:
insserv -v -d zfs-mount
I had an issue with pve storage on zfs, before pve would start before zfs and create directories at the zfs mount point. to fix that start single user mode and remove the directories [ make sure they are empty.... ].