[pve-devel] rfc : qemu fw_cfg as alternative to cloudinit

[Thomas Lamprecht]

On 10/04/2017 10:31 AM, Alexandre DERUMIER wrote:
>>> I saw this pop up on qemu-devel and edk2 (OVMF) devel here and there 
>>> but never looked really specific at that time. 
>>> It could be promising, IMO. To allow it to work we need more current 
>>> OVMF images, but those are already underway :) 
> 
> great :)  
> 

With a bit of luck we get them into PVE 5.1. 

> 
>>> Side question out of interest, whats your specific use for kubernetes/rancher? 
> 
> I'll be to deploy php application (stateless) in production for our company.
> Not yet databases or statefull (I'm still afraid of docker ;).
> So we'll have a mix of database vm + vms hosting dockers in same vlans.
> Security will be done by vlan.  (Each website/application have his own vlan with databases vms + docker vms)
> 

Ah OK, thanks for the explanation.


>>> Anything that could also be integrated in our ha-manager? 
> 
> mmm, if we could have something magic, like auto create a new vm, if we detect that we don't have enough ressource on others vms.
> (maybe inside a pool, like we have a pool of 2vms, and we allow auto create up to 5vms  when theses 2vms are overloaded).
> But it's a complex task.
> 

Yes it is. How to calculate a load level can hardly be done
in a general way as there are so many factors and causes.

> 
>>> There are plans for basic (at first) load distribution of services with 
>>> the intend to make the ha-manager a general resource manager where 
>>> service can be ha-managed but also not. That this is not already done 
>>> is mostly my fault, duh - other things got in the way (as always). 
> 
> Well, some kind of auto balancing of vms from ressources usage (at least cpu) could be wonderfull !
> We have talked about this since years :)  
> 

I will try to finally bite this bullet and at least try
some initial simple version.

> 
> ----- Mail original -----
> De: "Thomas Lamprecht" <t.lamprecht at proxmox.com>
> À: "pve-devel" <pve-devel at pve.proxmox.com>, "aderumier" <aderumier at odiso.com>
> Envoyé: Mercredi 4 Octobre 2017 08:47:40
> Objet: Re: [pve-devel] rfc : qemu fw_cfg as alternative to cloudinit
> 
> Hi, 
> 
> On 10/04/2017 08:14 AM, Alexandre DERUMIER wrote: 
>> Hi, 
>>
>> I just discover a new qemu feature : qemu fw_cfg 
>>
>> http://www.contrib.andrew.cmu.edu/~somlo/QEMU_fw_cfg/ 
>>
> 
> Here the up-to-date specification: 
> <https://git.qemu.org/?p=qemu.git;a=blob_plain;f=docs/specs/fw_cfg.txt;h=08c00bdf44a2cca1064356470af2d5438d44e3c9;hb=HEAD> 
> 
>>
>> It's allow to add a custom firmware, from a host file 
>>
>> like "-fw_cfg name=opt/GuestInfo,file=/etc/pve/qemu-server/100.conf" , for example 
>>
>>
>> Then it's possible to read this file in guest in 
>>
>> "/sys/firmware/qemu_fw_cfg/by_name/opt/GuestInfo/raw" 
>>
>> This need a new module include in guest kernel 4.6 (CONFIG_FW_CFG_SYSFS), 
>> but the module is also available out of kernel tree here: 
>> http://www.contrib.andrew.cmu.edu/~somlo/QEMU_fw_cfg/qemu_fw_cfg_kmod.tgz 
>>
>> (really small, it's simply extend standard firmware , so it's work with old kernel too) 
>>
>> for windows, seem that they are working on it (https://bugzilla.redhat.com/show_bug.cgi?id=1375444) 
>>
>>
>> Advantage vs cloudinit: 
>>
>> - no need to pass cdrom, no extra pci device.(it's simply a firmware) 
>> - you can use it in initramfs. (format disk, do mount point,...) 
>>
>>
>> disavantage vs cloudinit: 
>>
>> - read only (you can change it after boot, but it's really for vm init) 
> 
> The writing part got reintroduced with QEMU 2.9, in another form, 
> your Article from above mentions that it was written during QEMU 2.6 
> release time so maybe even that could be done in some way (did not 
> tried it out) 
> 
>> - no daemon or standard to parse the config, you need to do it yourself. 
>> But coreos for example, have a projet "ignition", 
>> https://coreos.com/ignition/docs/latest/ 
>> which can use it (it's working in initramfs) 
>>
>>
>>
>> I think it should be possible to provide qemu linux distro template like lxc, format the disk in initramfs, do the mount point, deploy source for example. 
>> Maybe also provide initramfs to qemu with -initrd, with an integrated custom config parser,to format the disk, do the mountpoint, deploy the template,set ssh keys, .... 
>>
>> What do you think about it ? 
>>
> 
> I saw this pop up on qemu-devel and edk2 (OVMF) devel here and there 
> but never looked really specific at that time. 
> It could be promising, IMO. To allow it to work we need more current 
> OVMF images, but those are already underway :) 
> 
>>
>> ( 
>> I'm currently working on a terraform proxmox provider, to deploy vm, as we are looking to deploy Rancher next year (a kubernetes clusters managers), 
>> and I need to be able to create vm from outside proxmox, with ip configuration,hostname,...) 
>>
> 
> Side question out of interest, whats your specific use for kubernetes/rancher? 
> Anything that could also be integrated in our ha-manager? 
> There are plans for basic (at first) load distribution of services with 
> the intend to make the ha-manager a general resource manager where 
> service can be ha-managed but also not. That this is not already done 
> is mostly my fault, duh - other things got in the way (as always). 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>