Difference between revisions of "Storage: ZFS"

From Proxmox VE
Jump to navigation Jump to search
 
(198 intermediate revisions by 16 users not shown)
Line 1: Line 1:
=zfs notes=
+
<!--PVE_IMPORT_START_MARKER-->
 
+
<!-- Do not edit - this is autogenerated content -->
==Native ZFS for Linux on Proxmox 2.0==
+
{{#pvedocs:pve-storage-zfspool-plain.html}}
*2012-02-25.  check  http://zfsonlinux.org/ .  looks like a solid project.
+
[[Category:Reference Documentation]]
 
+
<pvehide>
I had tried 3 other operating systems and for each was able to recover all the zfs  file systems from previous installs.  today I set up zfsonlinux using the following info on 2 prox 2.0 systems
+
Storage pool type: zfspool
 
+
This backend allows you to access local ZFS pools (or ZFS file systems
*install these
+
inside such pools).
aptitude install build-essential gawk alien fakeroot zlib1g-dev uuid uuid-dev libssl-dev parted  pve-headers-$(uname -r)
+
Configuration
*get make and install spl
+
The backend supports the common storage properties content, nodes,
<pre>
+
disable, and the following ZFS specific properties:
cd /usr/src
+
pool
git clone https://github.com/zfsonlinux/spl.git
+
Select the ZFS pool/filesystem. All allocations are done within that
cd spl
+
pool.
./configure
+
blocksize
make deb
+
Set ZFS blocksize parameter.
sudo dpkg -i *.deb
+
sparse
modprobe spl
+
Use ZFS thin-provisioning. A sparse volume is a volume whose
</pre>
+
reservation is not equal to the volume size.
*get make and install zfs
+
mountpoint
<pre>
+
The mount point of the ZFS pool/filesystem. Changing this does not
cd /usr/src
+
affect the mountpoint property of the dataset seen by zfs.
git clone https://github.com/zfsonlinux/zfs.git
+
Defaults to /&lt;pool&gt;.
cd zfs
+
Configuration Example (/etc/pve/storage.cfg)
./configure
+
zfspool: vmdata
make deb
+
        pool tank/vmdata
dpkg -i *.deb
+
        content rootdir,images
</pre>
+
        sparse
 
+
File naming conventions
modprobe zfs
+
The backend uses the following naming scheme for VM images:
 
+
vm-&lt;VMID&gt;-&lt;NAME&gt;      // normal VM images
if you do not have and zfs pools made then put this to /etc/modules
+
base-&lt;VMID&gt;-&lt;NAME&gt;    // template VM image (read-only)
spl
+
subvol-&lt;VMID&gt;-&lt;NAME&gt; // subvolumes (ZFS filesystem for containers)
zfs
+
&lt;VMID&gt;
 
+
This specifies the owner VM.
reboot to make sure modules load at boot.  I found on one system with limited memory [ compared to what was running] that zfs module would not load .
+
&lt;NAME&gt;
 
+
This can be an arbitrary name (ascii) without white space. The
=Creating Pools and Filesystems=
+
backend uses disk[N] as default, where [N] is replaced by an
'''use raidz2 or mirror in main data systems.'''
+
integer to make the name unique.
 
+
Storage Features
==Create a simple pool named “tank”==
+
ZFS is probably the most advanced storage type regarding snapshot and
zpool create  tank  /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf
+
cloning. The backend uses ZFS datasets for both VM images (format
 
+
raw) and container data (format subvol). ZFS properties are
==Create filesystems, and mount points==
+
inherited from the parent dataset, so you can simply set defaults
<pre>
+
on the parent dataset.
zfs create tank/home
+
Table 1. Storage features for backend zfs
zfs set mountpoint=/export/home tank/home
+
Content types
 
+
Image formats
zfs create tank/bkup
+
Shared
zfs set mountpoint=/bkup tank/bkup
+
Snapshots
 
+
Clones
zfs create tank/ht
+
images rootdir
zfs set mountpoint=/export/ht tank/ht
+
raw subvol
 
+
no
==Create home directories for several users==
+
yes
*Note: automatically mounted at /export/home/{ahrens,bonwick,billm} thanks to inheritance
+
yes
<pre>
+
Examples
# zfs create tank/home/ahrens
+
It is recommended to create an extra ZFS file system to store your VM images:
# zfs create tank/home/bonwick
+
# zfs create tank/vmdata
# zfs create tank/home/billm
+
To enable compression on that newly allocated file system:
</pre>
+
# zfs set compression=on tank/vmdata
 
+
You can get a list of available ZFS filesystems with:
==Add more space to the pool==
+
# pvesm zfsscan
zpool add tank mirror c4d0 c5d0
+
See Also
 
+
Storage
==destroy==
+
ZFS on Linux
to start over
+
</pvehide>
zpool destroy zfs-pool
+
<!--PVE_IMPORT_END_MARKER-->
 
 
===Create home directories for several users===
 
*Note: automatically mounted at /export/home/{joe,bill,tom} thanks to inheritance
 
<pre>
 
zfs create tank/home/joe
 
zfs create tank/home/bill
 
zfs create tank/home/tom
 
</pre>
 
 
 
===Setting Properties===
 
*Automatically NFS-export all home directories  [ I do not know how to use zfs nfs yet , or if it works yet in linux...].
 
zfs set sharenfs=rw tank/home
 
*check sharenfs 
 
<pre>
 
zfs get sharenfs tank/home
 
NAME     PROPERTY VALUE    SOURCE
 
tank/home  sharenfs  rw        local
 
</pre>
 
*Turn on compression for everything in the pool
 
zfs set compression=on tank
 
 
 
==zfs links==
 
there is a lot of well written information on zfs. here are some links:
 
 
 
check this first
 
*http://hub.opensolaris.org/bin/download/Community+Group+zfs/docs/zfslast.pdf
 
 
 
and this has some very important information to know before implementing zfs on a production  system.
 
*http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
 
 
 
= How to install ZFS-fuse under Proxmox =
 
 
 
(Tested with Kernel 2.6.32)
 
 
 
  apt-get install build-essential libaio-dev libattr1-dev libacl1-dev libz-dev libz-dev libfuse-dev libfuse2 scons libssl-dev
 
  wget -nd http://zfs-fuse.net/releases/0.6.0/zfs-fuse-0.6.0.tar.bz2
 
  tar jxvf zfs-fuse-0.6.0.tar.bz2
 
  cd zfs-fuse-0.6.0
 
  cd src
 
  scons
 
  scond install
 
 
 
Fire up ZFS-fuse daemon
 
  /usr/local/sbin/zfs-fuse
 
 
 
Create Zpool
 
  zpool create -m /var/lib/vz/images2 /dev/sdb
 
 
 
Create ZFS
 
  zfs create images2/109
 
 
 
ZFS clone a vm 109 to vm 110
 
  zfs snapshot images2/109@master
 
  zfs clone images2/109@master images2/110
 
 
 
== Debian GNU/kFreeBSD ==
 
'''As of 2012-02 the squeeze version does not have NFS, so AFAIK can not be used for Proxmox storage.''' When  Debsd has nfs support then I'd use this for storage of iso's and backups to start.  In the mean time this works as a samba server..
 
 
 
zfs works much better using  Debian/GNU kFreeBSD  then fuse.
 
 
 
after installing the system to kvm or hardware:
 
 
 
*add some disks .  in KVM i was only able to get ide to work.
 
 
 
*the scsi disks look like:
 
<pre>
 
ls /dev/ad*
 
/dev/ad10  /dev/ad10s1  /dev/ad10s2  /dev/ad11  /dev/ad12  /dev/ad13
 
</pre>
 
 
 
*  per http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide  - if I read it right - use entire disk not slices for zfs (I assume slices = partitions).
 
 
 
*if the disks have partition tables, then clear them. just not the operating system disk!   I used fdisk to clear the tables. This may not be necessary , if it is not then please delete this part.
 
<pre>
 
fdisk /dev/ad11
 
o
 
w
 
</pre>
 
then same for ad12 and ad13
 
 
 
 
*install this
 
aptitude install zfsutils
 
 
 
*make a pool
 
zpool create -f  zfs-pool /dev/ad11 /dev/ad12  /dev/ad13
 
 
 
df | grep zfs-pool
 
zfs-pool      zfs    1.4T  18K  1.4T  1% /zfs-pool
 
 
 
===links===
 
*http://wiki.debian.org/Debian_GNU/kFreeBSD
 

Latest revision as of 08:08, 6 December 2019

Storage pool type: zfspool

This backend allows you to access local ZFS pools (or ZFS file systems inside such pools).

Configuration

The backend supports the common storage properties content, nodes, disable, and the following ZFS specific properties:

pool

Select the ZFS pool/filesystem. All allocations are done within that pool.

blocksize

Set ZFS blocksize parameter.

sparse

Use ZFS thin-provisioning. A sparse volume is a volume whose reservation is not equal to the volume size.

mountpoint

The mount point of the ZFS pool/filesystem. Changing this does not affect the mountpoint property of the dataset seen by zfs. Defaults to /<pool>.

Configuration Example (/etc/pve/storage.cfg)
zfspool: vmdata
        pool tank/vmdata
        content rootdir,images
        sparse

File naming conventions

The backend uses the following naming scheme for VM images:

vm-<VMID>-<NAME>      // normal VM images
base-<VMID>-<NAME>    // template VM image (read-only)
subvol-<VMID>-<NAME>  // subvolumes (ZFS filesystem for containers)
<VMID>

This specifies the owner VM.

<NAME>

This can be an arbitrary name (ascii) without white space. The backend uses disk[N] as default, where [N] is replaced by an integer to make the name unique.

Storage Features

ZFS is probably the most advanced storage type regarding snapshot and cloning. The backend uses ZFS datasets for both VM images (format raw) and container data (format subvol). ZFS properties are inherited from the parent dataset, so you can simply set defaults on the parent dataset.

Table 1. Storage features for backend zfs
Content types Image formats Shared Snapshots Clones

images rootdir

raw subvol

no

yes

yes

Examples

It is recommended to create an extra ZFS file system to store your VM images:

# zfs create tank/vmdata

To enable compression on that newly allocated file system:

# zfs set compression=on tank/vmdata

You can get a list of available ZFS filesystems with:

# pvesm zfsscan