[PVE-User] Ceph and journal on a file...

[Marco Gaiarin]

I'm doing some test with ceph, and i'm trying to put journal on a file
(TMPFS); probably is a bad idea, but i'm only doing some test...
Also, i'm putting OSD on partitions, not disks, so i'm forced to use
commandline. No trouble at all.

So i've done:

 root at capitanamerica:~# ceph-disk -v prepare --cluster-uuid 5f5ecc90-9664-4f95-8bea-9274f4eac005 --data-dev --journal-file /dev/sda5 /mnt/osdcache/sda5
 INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=None --name=osd. --lookup osd_mkfs_type
 2016-09-13 15:50:47.401894 7f8d21210780 -1 did not load config file, using default settings.
 [...]
 DEBUG:ceph-disk:Creating journal file /mnt/osdcache/sda5 with size 0 (ceph-osd will resize and allocate)
 DEBUG:ceph-disk:Journal is file /mnt/osdcache/sda5
 WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same device as the osd data
 DEBUG:ceph-disk:OSD data device /dev/sda5 is a partition
 DEBUG:ceph-disk:Creating xfs fs on /dev/sda5
 INFO:ceph-disk:Running command: /sbin/mkfs -t xfs -f -i size=2048 -- /dev/sda5
 [...]
 DEBUG:ceph-disk:Mounting /dev/sda5 on /var/lib/ceph/tmp/mnt.CiR4H9 with options noatime,inode64
 INFO:ceph-disk:Running command: /bin/mount -t xfs -o noatime,inode64 -- /dev/sda5 /var/lib/ceph/tmp/mnt.CiR4H9
 DEBUG:ceph-disk:Preparing osd data dir /var/lib/ceph/tmp/mnt.CiR4H9
 DEBUG:ceph-disk:Creating symlink /var/lib/ceph/tmp/mnt.CiR4H9/journal -> /mnt/osdcache/sda5
 DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.CiR4H9
 INFO:ceph-disk:Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.CiR4H9
 DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sda5
 INFO:ceph-disk:Running command: /sbin/partprobe /dev/sda5

but when i try to activate it:

 root at capitanamerica:~# ceph-disk -v activate /dev/sda5
 INFO:ceph-disk:Running command: /sbin/blkid -p -s TYPE -ovalue -- /dev/sda5
 INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
 INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
 DEBUG:ceph-disk:Mounting /dev/sda5 on /var/lib/ceph/tmp/mnt.DHYch9 with options noatime,inode64
 INFO:ceph-disk:Running command: /bin/mount -t xfs -o noatime,inode64 -- /dev/sda5 /var/lib/ceph/tmp/mnt.DHYch9
 DEBUG:ceph-disk:Cluster uuid is 5f5ecc90-9664-4f95-8bea-9274f4eac005
 INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
 DEBUG:ceph-disk:Cluster name is ceph
 DEBUG:ceph-disk:OSD uuid is a947135c-c4c3-445f-b5a3-2ada38085bf8
 DEBUG:ceph-disk:Allocating OSD id...
 INFO:ceph-disk:Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd create --concise a947135c-c4c3-445f-b5a3-2ada38085bf8
 DEBUG:ceph-disk:OSD id is 6
 DEBUG:ceph-disk:Initializing OSD...
 INFO:ceph-disk:Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/tmp/mnt.DHYch9/activate.monmap
 got monmap epoch 2
 INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster ceph --mkfs --mkkey -i 6 --monmap /var/lib/ceph/tmp/mnt.DHYch9/activate.monmap --osd-data /var/lib/ceph/tmp/mnt.DHYch9 --osd-journal /var/lib/ceph/tmp/mnt.DHYch9/journal --osd-uuid a947135c-c4c3-445f-b5a3-2ada38085bf8 --keyring /var/lib/ceph/tmp/mnt.DHYch9/keyring
 2016-09-13 15:54:19.577249 7fb9ec266880 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
 2016-09-13 15:54:19.577275 7fb9ec266880 -1 journal check: ondisk fsid 00000000-0000-0000-0000-000000000000 doesn't match expected a947135c-c4c3-445f-b5a3-2ada38085bf8, invalid (someone else's?) journal
 2016-09-13 15:54:19.577305 7fb9ec266880 -1 filestore(/var/lib/ceph/tmp/mnt.DHYch9) mkjournal error creating journal on /var/lib/ceph/tmp/mnt.DHYch9/journal: (22) Invalid argument
 2016-09-13 15:54:19.577333 7fb9ec266880 -1 OSD::mkfs: ObjectStore::mkfs failed with error -22
 2016-09-13 15:54:19.577383 7fb9ec266880 -1  ** ERROR: error creating empty object store in /var/lib/ceph/tmp/mnt.DHYch9: (22) Invalid argument
 ERROR:ceph-disk:Failed to activate


I've tried to google aroud a bit, but found nothing useful.

So i'm asking here. Thanks.

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
    http://www.lanostrafamiglia.it/25/index.php/component/k2/item/123
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)