[PVE-User] Ceph or Gluster

[Lindsay Mathieson]

On 22/04/2016 11:31 PM, Brian :: wrote:
> 10Gbps or faster at a minimum or you will have pain. Even using 4
> nodes with 4 spinner disks in each node and you will be maxing out
> 1Gbps network.


Can't say I saw that on our cluster.

- 3 Nodes
- 3 OSD's per Node
- SSD journals for each OSD.
- 2*1G Eth in LACP Bond dedicated to Ceph
- 1G Admin Net
- Size = 3 (Replica 3)

Never came close to maxing out 1 1Gbps connection, write throughput and 
IOPS are terrible. Read is pretty good though.

Currently trialing Gluster 3.7.11 on ZFS Bricks, Replica 3 also. Triple 
the throughput ans IOPS I was getting with Ceph, maxes out the 2*1G 
connection, also seems to deal with VM I/O spikes better too, not 
letting other VM's be stalled.

Not convinced its as robust as Ceph yet, give it a few more weeks. It 
does cope very well with failover and brick heals (using 64MB shards).

-- 
Lindsay Mathieson