[PVE-User] Support for Ceph Nautilus?

[Thomas Lamprecht]

On 5/17/19 9:53 AM, Christian Balzer wrote:
> On Fri, 17 May 2019 08:05:21 +0200 Thomas Lamprecht wrote:
>> On 5/17/19 4:27 AM, Christian Balzer wrote:
>>> is there anything that's stopping the current PVE to work with an
>>> externally configured Ceph Nautilus cluster?  
>>
>>
>> Short: rather not, you need to try out to be sure though.

Dominik reminded me that with the new ceph release scheme you always
should be able to use clients with one stable version older or newer
than the server version just fine, so as Mimic was no stable release
you should be able to use Luminous clients with Nautilus server without
issues.

>>
> Whenever Ceph Nautilus packages drop for either Stretch or Buster, I'm
> quite aware of the issues you mention below.

Ah, I did not understand your question this way, sorry, and as said
they'll drop when PVE 6 drops and that'll highly probably be after
buster :-)

>  
>> You probably cannot use the kernel RBD as it's support may be to old
>> for Nautilus images.
>>
>> The userspace libraries we use else would need to be updated to Nautilus
>> to ensure working fine with everything changed in Nautilus.
>>
>> So why don't we, the Proxmox development community, don't just update to
>> Nautilus?
>>
>> That ceph version started to employ programming language features available
>> in only relative recent compiler versions, sadly the one which _every_ binary
>> in Stretch is build with (gcc 6.3) does not supports those features - we
>> thought about workarounds, but felt very uneasy of all of them - the compiler
>> and it's used libc is such a fundamental base of a Linux Distro that we cannot
>> change that for a single application without hurting stability and bringing up
>> lots of problems in any way.
>>
>> So Nautilus will come first with Proxmox VE 6.0 based upon Debian Buster,
>> compiled with gcc 8, which supports all those new shiny used features.
>>
> Any timeline (he asks innocently, knowing the usual Debian release delays)?
> 

For that it can make sense to ask the Debian release team, as at the moment
no public information is available, one can only speculate on the count and
types of release blocking bugs, with that and past releases in mind one can
probably roughly extrapolate the timeline and be correct for ± a month or so.

>>>
>>> No matter how many bugfixes and backports have been done for Luminous, it
>>> still feels like a very weak first attempt with regards to making
>>> Bluestore the default storage and I'd rather not deploy anything based on
>>> it.  
>>
>> We use Bluestore in our own Infrastructure without issues, and lot's of PVE user
>> do also - if that make you feelings shift a bit to the better.
>>
> 
> The use case I have is one where Bluestore on the same HW would perform
> noticeably worse than filestore and with cache tiers being in limbo
> (neither recommended nor replaced with something else) 
> 
> At this point in time I would like to deploy Nautilus and then forget
> about it, that installation will not be upgraded ever and retired before
> the usual 5 years are up.

sound like a security nightmare if not heavily contained, but I guess you know
that having that on the internet is not the best idea and that Proxmox release
cycles are normally supported for about ~3 years from initial .0 release, at
least in the past, and that not only means security update but any issue or
question which will come up in >3 years (you all prob. know that, and can deal
with it yourself, I do no question that, just as a reminder for anybody reading
this)

cheers,
Thomas