Sheepdog cluster install

From Proxmox VE
Jump to: navigation, search


In this howTo we will setup a Sheepdog Cluster, using an interactive script for the installation/configuration.

NOTE: although is experimental and not officially supported on Proxmox VE, sheepdog has now a commercial support.

Download the script

These scripts will help in the installation and in the configuration of the sheepdog node, we need git to download them.

EDIT : link changed

#git clone

Install zookeeper and sheep

Launch the script:

cd sheepdog-utils/

So the script will print out:

If you already installed the standard qemu-kvm package it will be removed.
If /usr/src/qemu or /usr/src/sheepdog exist, they will be replaced after trying
to uninstalling them.

Do you whant do proceed?

Chosse what you need to install

1) sheepdog
2) sheepdog + qemu
3) sheepdog + qemu + zookeeper
4) help


Now you can choose:

  • 1 it will install only sheepdog.
  • 2 it will install sheepdog and qemu.
  • 3 it will install sheepdog, qemu and zookeeper.

Zookeeper is the alternative daemon to corosync. For sheepdog, it way work better and you need to install it only on 3 nodes(*) for managing a cluster of hundreds of sheepdog nodes. Obviously, for bigger clusters we appeal to the common sense, 3 zookeeper node is a bit dangerous if you lose one of them.

(*) Zookeeper need at least 3 node, it's possible to create a 2 node cluster, but in case of the loss of one node, you'll experience the split brain problem.

So we follow the more complex choice, installing sheepdog+qemu+zookeeper

After downloading all the necessary it will ask:

More than one ip has beed deteceted:
Type the ip that will be used by zookeeper.

Here you have to put the ip address that zookeper should use, then:

Type the other zookeeper nodes ip, separated by a single space.
If this is the only zookeeper node, just press enter.

Here you have to list all the ip of the nodes that will have zookeper installed. Keep in mind that you need at least 3 zookeper nodes. You can add the ip address later on editing the zookeper configuration file. ( /etc/zookeeper/conf/zoo.cfg )

Next step it will set the right fd limit ( max number of open files ) and ask for a reboot. After this you should run:


This script will configure sheepdog to work with the disks and partitions you like.

So the script now will ask as follow:

Before running this script:
- configure all your network cards
- edit your fstab and mount all the required devices

Press ctrl+c sto stop it at anytime.


In the fstab you should need to prepare something like this:

/dev/sda1 /mnt/sheep/disk01 xfs defaults,noatime        0       0
/dev/sda1 /mnt/sheep/disk02 xfs defaults,noatime        0       0
/dev/sda1 /mnt/sheep/disk03 xfs defaults,noatime        0       0

We use xfs but you can choose any filesystem that could support xattr. Press yes if your setting are done.

Is this node going to be a gateway only?                                                                                                       
(A gateway only node is able to run guests, but it doesn't store any data.                                                                     
The cluster size is not going to be affected).                                                                                                 

Here if you want to use the node as gateway only or as storage. We choose No.

Remember that each device must have extended attributes enabled.                                                                               
Check your /etc/fstab if in doubt.                                                                                                             
The following mount point has been detected:                                                                                                   
Type the mount point(s) used by sheepdog, one per line.                                                                                        
(Leave blank once done)                                                                                                                        

So put the /mnt/sheep/disk01  and press enter
Do it for all the partition you like to use in the cluster.

The selected mount point are: /mnt/sheep/disk01 /mnt/sheep/disk02 /mnt/sheep/disk03                                        
This node has been recognized as a Zookeeper node.
These servers' ip will be used:

Are these the right zookeeper' ip(s)?


Type yes.

Gateway only:   no
Devices:        /mnt/old_boot
Dedicate nic:
Zookeeper:      /mnt/sheep/disk01 /mnt/sheep/disk02 /mnt/sheep/disk03

If everythin is correct, press 'y' or 'n' to exit



Now you should have a correctly configured sheepdog node. Just repeat the and for all the node you need.

Contributor and support

If you need support, contact Extensys S.R.L.