[pve-devel] RFC: qemu-server : add cloudinit support

[Wolfgang Bumiller]

On Thu, Jun 11, 2015 at 12:51:42PM +0200, Alexandre DERUMIER wrote:
> >>For one, it seems to be started on several points during the init 
> >>process. There seem to be 5 different systemd units specified for it 
> >>even. 
> yes, I'm seeing this 4 units.
> 
> I'm think they are related to different cloud-init stage (init,config,final, see /etc/cloud/cloud.cfg).

Yes, it's just not immediately clear when each of them happen and what
they're _exactly_ doing.

> If I remember, init stage is done only at first boot.
> cloud_init_modules:
>  - migrator
>  - bootcmd
>  - write-files
>  - resizefs
>  - set_hostname
>  - update_hostname
>  - update_etc_hosts
>  - ca-certs
>  - rsyslog
>  - users-groups
>  - ssh

This part is configured in /etc apparently. So there the list of enabled
hooks is configured AFAIU.

> I'm not sure about other stage.

Yeah, I really wish there was a detailed design document somewhere ;-)

> >>Then there seems to be a system-wide configuration in /etc/cloud that 
> >>defines not just datasources but also default configurations, like a 
> >>default user, and whether root may login with a password, etc.
> 
> users,keys,password are not defined in /etc/cloud/cloud.cfg but in configdrive
> They are some system default thing, like admin account for debian for example (but no password)

They _can_ be, but I haven't figured out if/how/when they're added if
you actually do it in there. At least my test user never came to live on
my arch test vm.

> but the main thing is the the datasource provider (I'm using configdrive like openstack|ovirt)
> other things are list of modules we can call from configdrive

Yes, that much is clear to me by now.

> >>At this 
> >>point I'm wondering whether this configuration will be re-applied on 
> >>every boot. (The documentation doesn't mention any of this actually, I 
> >>found this part on the ArchWiki[1]...) 
> 
> >>And does that mean cloud-init keeps overwriting the systems's regular
> >>configuration files?
> 
> If you don't provide the configdrive (no datasource), it'll do nothing.
> That's why I remove the configdrive after first init.

It doing nothing is what I'm afraid of. I've tested an ubuntu
installation just now, and after removing the config drive the
cloud-init init-process just sat there, not letting me boot all the way
through to the login screen. IOW the entire boot process was hanging
around doing nothing. This isn't really ideal.

OTOH it may be desirable to keep the config datasource attached. That
way the WEB UI's configuration is the "permanent" one, and changes made
in the running VM to /etc/ (eg. /etc/network/interfaces) are only
temporary.

Otherwise the options in the UI would have to be deleted/greyed out with
it to reflect the new situation, while if the config is kept around
there'd be some permanently-available settings in the UI.

And with the init process hanging dead in my tests (for around 10
minutes, and afterwards errors appear like using sudo shows "unable to
resolve host <somename>.localdomain", which tells me cloud-init decided
to screw something up...) it means that the only safe way to remove the
datasource is to also have it uninstall the cloud-init package on the OS
- unless the packaged cloud-init is smarter. (I tried an ubuntu-server
14.10 image)

> >>The documentation only mentions a directory structure in /var. [2] 
> >>Now I'm left to wonder... does the cloud init process somehow 
> >>overwride/interfere with the regular initialization process? Does it 
> >>actually perform physical merging of configuration files from the 
> >>datasource into /var? (The documentation also mentions merging[3]) 
> 
> I don't see nothing special in /var/lib/cloud, no config cache or something like that

Yes, this section of the documentation still seems somewhat weird to me.

> >>Finally: OS support. It seems to only ship with support linux (few 
> >>specific distros even) and freebsd [4]. For that purpose it seems a 
> >>little overengineered. 
> 
> linux distro : centos,rhel,fedora,debian,ubuntu,suse,archlinux,gentoo
> bsd: freebsd
> windows
> http://thornelabs.net/2014/06/01/where-to-find-openstack-cloud-images.html
> 
> don't seem too bad ?

According to this cloud-init doesn't actually run on windows directly,
but there's a "port" named cloudbased-init. It supports a config-disk
too, but I doubt it uses the same layout and syntax everywhere. (After
all, network interfaces for instance aren't exactly called 'eth0' and
'eth1' on windows ;-) )