Storage: ZFS

zfs notes

Native ZFS for Linux on Proxmox 2.0

• 2012-02-25. check  http://zfsonlinux.org/ . looks like a solid project.

I had tried 3 other operating systems and for each was able to recover all the zfs file systems from previous installs. today I set up zfsonlinux using the following info on 2 prox 2.0 systems

• install these

aptitude install build-essential gawk alien fakeroot zlib1g-dev uuid uuid-dev libssl-dev parted  pve-headers-$(uname -r) 

• get make and install spl

cd /usr/src
git clone https://github.com/zfsonlinux/spl.git
cd spl
./configure
make deb
sudo dpkg -i *.deb
modprobe spl

• get make and install zfs

cd /usr/src
git clone https://github.com/zfsonlinux/zfs.git
cd zfs
./configure
make deb
dpkg -i *.deb

modprobe zfs

if you do not have and zfs pools made then put this to /etc/modules

spl
zfs

reboot to make sure modules load at boot. I found on one system with limited memory [ compared to what was running] that zfs module would not load .

• then import preexisting pool

zpool  import -f tank

df

Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/mapper/pve-root
              ext3     92G  1.1G   86G   2% /
tmpfs        tmpfs    2.0G     0  2.0G   0% /lib/init/rw
udev         tmpfs    2.0G  284K  2.0G   1% /dev
tmpfs        tmpfs    2.0G  3.1M  2.0G   1% /dev/shm
/dev/mapper/pve-data
              ext3    256G  188M  255G   1% /var/lib/vz
/dev/sda1     ext3    495M   34M  436M   8% /boot
/dev/fuse     fuse     30M   12K   30M   1% /etc/pve
tank/bkup      zfs    3.0T  237G  2.7T   8% /bkup
tank/home      zfs    2.7T     0  2.7T   0% /export/home
tank/ht        zfs    2.9T  216G  2.7T   8% /export/ht
tank/data/home-fbc
               zfs    2.7T  3.5G  2.7T   1% /home-fbc
tank           zfs    2.7T     0  2.7T   0% /tank
tank/data      zfs    2.7T     0  2.7T   0% /tank/data
tank/data/u    zfs    2.7T  801M  2.7T   1% /u

Create filesystems, and mount points

zfs create tank/home
zfs set mountpoint=/export/home tank/home

zfs create tank/bkup
zfs set mountpoint=/bkup tank/bkup

zfs create tank/ht
zfs set mountpoint=/export/ht tank/ht

====Create home directories for several users===
*Note: automatically mounted at /export/home/{ahrens,bonwick,billm} thanks to inheritance
<pre>
# zfs create tank/home/ahrens
# zfs create tank/home/bonwick
# zfs create tank/home/billm

zfs links

there is a lot of well written information on zfs. here are some links:

check this first

• http://hub.opensolaris.org/bin/download/Community+Group+zfs/docs/zfslast.pdf

and this has some very important information to know before implementing zfs on a production system.

• http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide

How to install ZFS-fuse under Proxmox

(Tested with Kernel 2.6.32)

 apt-get install build-essential libaio-dev libattr1-dev libacl1-dev libz-dev libz-dev libfuse-dev libfuse2 scons libssl-dev
 wget -nd http://zfs-fuse.net/releases/0.6.0/zfs-fuse-0.6.0.tar.bz2
 tar jxvf zfs-fuse-0.6.0.tar.bz2
 cd zfs-fuse-0.6.0
 cd src
 scons
 scond install

Fire up ZFS-fuse daemon

 /usr/local/sbin/zfs-fuse

Create Zpool

 zpool create -m /var/lib/vz/images2 /dev/sdb

Create ZFS

 zfs create images2/109

ZFS clone a vm 109 to vm 110

 zfs snapshot images2/109@master
 zfs clone images2/109@master images2/110

Debian GNU/kFreeBSD

As of 2012-02 the squeeze version does not have NFS, so AFAIK can not be used for Proxmox storage. When Debsd has nfs support then I'd use this for storage of iso's and backups to start. In the mean time this works as a samba server..

zfs works much better using Debian/GNU kFreeBSD then fuse.

after installing the system to kvm or hardware:

• add some disks . in KVM i was only able to get ide to work.

• the scsi disks look like:

ls /dev/ad*
/dev/ad10  /dev/ad10s1  /dev/ad10s2  /dev/ad11  /dev/ad12  /dev/ad13

• per http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide - if I read it right - use entire disk not slices for zfs (I assume slices = partitions).

• if the disks have partition tables, then clear them. just not the operating system disk! I used fdisk to clear the tables. This may not be necessary , if it is not then please delete this part.

fdisk /dev/ad11
o
w

then same for ad12 and ad13

• install this

aptitude install zfsutils

• make a pool

zpool create -f  zfs-pool /dev/ad11 /dev/ad12  /dev/ad13

df | grep zfs-pool
zfs-pool       zfs    1.4T   18K  1.4T   1% /zfs-pool

links

• http://wiki.debian.org/Debian_GNU/kFreeBSD