Storage: Sheepdog

From Proxmox VE
Jump to: navigation, search

Build your sheepdog cluster

Proxmox have a special sheepdog package for you :)

Note: Sheepdog is still not stable, so please do not use it in production environments.


on each proxmox server you want to use for your sheepdog cluster

#apt-get install pve-sheepdog


It'll create a /var/lib/sheepdog/ directory.

If you want use default proxmox root partition, you need to add user_xattr option to the mountpoint,

edit /etc/fstab:

/dev/pve/root / ext3 errors=remount-ro,user_xattr,noatime 0 1


Proxmox > 3.0 (sheepdog >= 0.6.0)

  • Meta-datas && datas in the same default directoy

Default sheepdog directory is /var/lib/sheepdog/, best practice is to have raid volume for this.

  • Datas in dedicated disks

New feature from sheepdog 0.6, is to be able to manage single disks with no raid, and handle failure, unplug, hotplug.

you can create discN subdirectory and mount disks directly.

mkdir /var/lib/sheepdog/disc0
mkdir /var/lib/sheepdog/disc1
mkdir /var/lib/sheepdog/disc2
mkdir /var/lib/sheepdog/discN

/etc/fstab

/dev/sdb1 /var/lib/sheepdog/disc0/ ext3 errors=remount-ro,user_xattr,noatime 0 1
/dev/sdc1 /var/lib/sheepdog/disc1/ ext3 errors=remount-ro,user_xattr,noatime 0 1
/dev/sdd1 /var/lib/sheepdog/disc2/ ext3 errors=remount-ro,user_xattr,noatime 0 1
/dev/sdX1 /var/lib/sheepdog/discN/ ext3 errors=remount-ro,user_xattr,noatime 0 1
  • Use Journal on fast ssd for fast writes

You can use a fast ssd dedicated (raid volume recommended) disk, to handle the sheepdog journal

create a subdirectory

mkdir /var/lib/sheepdog/journal/

/etc/fstab

/dev/sde1 /var/lib/sheepdog/journal/ ext3 errors=remount-ro,user_xattr 0 1
  • Tuning

You can change default sheepdog directory and journal size in

/etc/default/sheepdog

ROOTDIR=/var/lib/sheepdog/
JOURNALSIZE=256


and restart sheepdog daemon

#/etc/init.d/sheepdog restart

Note that sheepdog will not run until corosync is running, so you may want to complete PVE HA setup first.

Proxmox 3.0 (sheepdog 0.56)

Recommendation is to mount a dedicated disk (or raidvolume), in /var/lib/sheepdog/disc0/. You can use xfs as filesystem.

now, create a startup file in /var/lib/sheepdog/disc0

#touch /var/lib/sheepdog/disc0/startup

and start sheepdog deamon

#/etc/init.d/sheepdog start

On 1 server

Now, you can create your sheepdog cluster

#collie cluster format --copies=2

"--copies" specifies the number of default data redundancy. In this case, the replicated data will be stored on 2 machines.

Copies is also the minimum number of sheepdog server you need to have always online to have a running sheepdog cluster.


Your cluster is ready !

Configure Proxmox to use the sheepdog cluster

edit your /etc/pve/storage.cfg and add the configuration (gui creation is not available yet)

sheepdog: mysheepdogclustername
        portal 127.0.0.1:7000
        content images

That's all !

you can now create your disk on the sheepdog storage :)