[pve-devel] qemu memory hotplug limited to 64 dimm

[Dietmar Maurer]

> On December 13, 2016 at 4:12 PM Alexandre DERUMIER <aderumier at odiso.com>
> wrote:
> 
> 
> I have asked to user to modprobe vhost module with region at 509, seem to
> works fine.
> 
> maybe can we force it in kernel directly ?

yes, seems to be easy:

drivers/vhost/vhost.c:static ushort max_mem_regions = 64;

What value should we set exactly?

Any drawbacks?

> 
> 
> ----- Mail original -----
> De: "aderumier" <aderumier at odiso.com>
> À: "pve-devel" <pve-devel at pve.proxmox.com>
> Envoyé: Lundi 12 Décembre 2016 19:46:56
> Objet: Re: [pve-devel] qemu memory hotplug limited to 64 dimm
> 
> also here: 
> https://patchwork.kernel.org/patch/5825741/ 
> 
> 
> Seem that increasing regions decrease performance, because of extra lookups. 
> (I think with more regions, it's some kind of tree, where your need to lookup
> first the parent region, then the child region) 
> 
> 
> ----- Mail original ----- 
> De: "aderumier" <aderumier at odiso.com> 
> À: "pve-devel" <pve-devel at pve.proxmox.com> 
> Cc: "dietmar" <dietmar at proxmox.com> 
> Envoyé: Lundi 12 Décembre 2016 19:42:03 
> Objet: Re: [pve-devel] qemu memory hotplug limited to 64 dimm 
> 
> I found some kernel patch to bump it to 509 (seem that another kvm fixed was
> needed) 
> 
> http://www.spinics.net/lists/kvm/msg117654.html 
> 
> But I think it was never commited to the kernel. 
> 
> I'll try follow the discuss. 
> 
> ----- Mail original ----- 
> De: "aderumier" <aderumier at odiso.com> 
> À: "dietmar" <dietmar at proxmox.com> 
> Cc: "pve-devel" <pve-devel at pve.proxmox.com> 
> Envoyé: Lundi 12 Décembre 2016 19:29:54 
> Objet: Re: [pve-devel] qemu memory hotplug limited to 64 dimm 
> 
> >>Can't we simply increase that max_mem_regions default value? 
> 
> I'm trying to see why they have limited it to 64. 
> 
> maybe a qemu bug with previous qemu version? don't known. 
> 
> I have some big spare servers to test with 370G ram, i'll try to do test with
> them. 
> 
> 
> ----- Mail original ----- 
> De: "dietmar" <dietmar at proxmox.com> 
> À: "aderumier" <aderumier at odiso.com>, "pve-devel" <pve-devel at pve.proxmox.com> 
> Envoyé: Lundi 12 Décembre 2016 17:09:41 
> Objet: Re: [pve-devel] qemu memory hotplug limited to 64 dimm 
> 
> Can't we simply increase that max_mem_regions default value? 
> 
> 
> > On December 12, 2016 at 4:54 PM Alexandre DERUMIER <aderumier at odiso.com> 
> > wrote: 
> > 
> > 
> > vhost 64 limit has been introduced here, because of qemu crash with bigger 
> > value 
> > 
> > https://patchwork.kernel.org/patch/6709021/ 
> > 
> > ----- Mail original ----- 
> > De: "aderumier" <aderumier at odiso.com> 
> > À: "pve-devel" <pve-devel at pve.proxmox.com> 
> > Envoyé: Lundi 12 Décembre 2016 16:51:11 
> > Objet: [pve-devel] qemu memory hotplug limited to 64 dimm 
> > 
> > Hi, 
> > 
> > a proxmox user has reported problem to hotplug memory with high values (48G)
> > 
> > 
> > https://forum.proxmox.com/threads/apt-get-upgrade-to-update-proxmox-server.30992/
> > 
> > 
> > 
> > 
> > Seem that qemu check about a vhost value 
> > 
> > https://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg05998.html 
> > 
> > cat /sys/module/vhost/parameters/max_mem_regions > 64 (verified on my 
> > servers). 
> > 
> > 
> > I think this limit was not present when we have implemented hotplug. 
> > 
> > Not sure was we can do, but I think our method don't scale anymore :( 
> > 
> > Maybe should we introduce a new fixed "dimm size" option, to allow hotplug
> > of 
> > more memory ? 
> > 
> > 
> > 
> > _______________________________________________ 
> > pve-devel mailing list 
> > pve-devel at pve.proxmox.com 
> > http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> > 
> > _______________________________________________ 
> > pve-devel mailing list 
> > pve-devel at pve.proxmox.com 
> > http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel