Sheepdog cluster install
In this howTo we will setup a Sheepdog Cluster, using an interactive script for the installation/configuration.
NOTE: although is experimental and not officially supported on Proxmox VE, sheepdog has now a commercial support.
Download the script
These scripts will help in the installation and in the configuration of the sheepdog node, we need git to download them.
EDIT : link changed
#git clone https://github.com/sheepdog/sheepdog-utils
Install zookeeper and sheep
Launch the script:
cd sheepdog-utils/ ./install.sh
So the script will print out:
If you already installed the standard qemu-kvm package it will be removed. If /usr/src/qemu or /usr/src/sheepdog exist, they will be replaced after trying to uninstalling them. Do you whant do proceed? Chosse what you need to install 1) sheepdog 2) sheepdog + qemu 3) sheepdog + qemu + zookeeper 4) help Number:
Now you can choose:
- 1 it will install only sheepdog.
- 2 it will install sheepdog and qemu.
- 3 it will install sheepdog, qemu and zookeeper.
Zookeeper is the alternative daemon to corosync. For sheepdog, it way work better and you need to install it only on 3 nodes(*) for managing a cluster of hundreds of sheepdog nodes. Obviously, for bigger clusters we appeal to the common sense, 3 zookeeper node is a bit dangerous if you lose one of them.
(*) Zookeeper need at least 3 node, it's possible to create a 2 node cluster, but in case of the loss of one node, you'll experience the split brain problem.
So we follow the more complex choice, installing sheepdog+qemu+zookeeper
After downloading all the necessary it will ask:
More than one ip has beed deteceted: 192.168.2.59 192.168.70.100 Type the ip that will be used by zookeeper. ip:
Here you have to put the ip address that zookeper should use, then:
Type the other zookeeper nodes ip, separated by a single space. If this is the only zookeeper node, just press enter. ip:
Here you have to list all the ip of the nodes that will have zookeper installed. Keep in mind that you need at least 3 zookeper nodes. You can add the ip address later on editing the zookeper configuration file. ( /etc/zookeeper/conf/zoo.cfg )
Next step it will set the right fd limit ( max number of open files ) and ask for a reboot. After this you should run:
This script will configure sheepdog to work with the disks and partitions you like.
So the script now will ask as follow:
Before running this script: - configure all your network cards - edit your fstab and mount all the required devices Press ctrl+c sto stop it at anytime. Continue? (y/n):
In the fstab you should need to prepare something like this:
/dev/sda1 /mnt/sheep/disk01 xfs defaults,noatime 0 0 /dev/sda1 /mnt/sheep/disk02 xfs defaults,noatime 0 0 /dev/sda1 /mnt/sheep/disk03 xfs defaults,noatime 0 0
We use xfs but you can choose any filesystem that could support xattr. Press yes if your setting are done.
Is this node going to be a gateway only? (A gateway only node is able to run guests, but it doesn't store any data. The cluster size is not going to be affected). (y/n):
Here if you want to use the node as gateway only or as storage. We choose No.
Remember that each device must have extended attributes enabled. Check your /etc/fstab if in doubt. The following mount point has been detected: /mnt/sheep/disk01 /mnt/sheep/disk02 /mnt/sheep/disk03 Type the mount point(s) used by sheepdog, one per line. (Leave blank once done) path: So put the /mnt/sheep/disk01 and press enter Do it for all the partition you like to use in the cluster. The selected mount point are: /mnt/sheep/disk01 /mnt/sheep/disk02 /mnt/sheep/disk03 This node has been recognized as a Zookeeper node. These servers' ip will be used: 192.168.2.59 192.168.2.60 Are these the right zookeeper' ip(s)? (y/n):
SUMMARY ======= Gateway only: no Devices: /mnt/old_boot Dedicate nic: Zookeeper: /mnt/sheep/disk01 /mnt/sheep/disk02 /mnt/sheep/disk03 ======= If everythin is correct, press 'y' or 'n' to exit (y/n): Yes
Now you should have a correctly configured sheepdog node. Just repeat the install.sh and assistant.sh for all the node you need.
Contributor and support
If you need support, contact Extensys S.R.L.