[pve-devel] Error in PVE with win2008r2 and 256GB RAM

Alexandre DERUMIER aderumier at odiso.com
Thu Dec 18 08:11:47 CET 2014


>>Many thanks for your answer, but i am not sure if it is a good idea due to 
>>that i don't understand the advantage since that i can disable the huge 
>>pages of this mode: 

They are 2 modes for hugepages,

the transparent hugepage mode, managed by the kernel 

But also, old way, manual hugepages (aka hugetlbfs, mounted in /dev/hugepage..)

from:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-transhuge.html

" However, transparent hugepage mode is not recommended for database workloads."


So, hugepages are always usefull for big memory vms, because it's reduce cpu usage on memory access.
But transparent hugepage sometimes don't work good with some workloads like database.

I think than manually defined hugepages can give good an extra boost vs disable TBL.



>>Moreover, i have a serious problem with the PVE cluster communication in two 
>>of eight PVE nodes, and with a VM, if you can help me, i will be extremely 
>>grateful, please see this link: 
>>http://forum.proxmox.com/threads/20523-Quorum-problems-with-PVE-2-3-and-3-3?p=104995#post104995 

Do you mix 2.6.32 and 3.10 kernel in your cluster ?
(I have had some strange problems with mixed kernel, never find the problem).

Also, this could be a multicast snooping problem.

What are your hardware switches ?
Myself, I disable snooping on linux vmbr, enable snooping on physical swiches + igmp querier on physical switches.



----- Mail original -----
De: "Cesar Peschiera" <brain at click.com.py>
À: "pve-devel" <pve-devel at pve.proxmox.com>, "aderumier" <aderumier at odiso.com>
Envoyé: Jeudi 18 Décembre 2014 07:09:11
Objet: Fw: [pve-devel] Error in PVE with win2008r2 and 256GB RAM

Hi Alexandre 

Many thanks for your answer, but i am not sure if it is a good idea due to 
that i don't understand the advantage since that i can disable the huge 
pages of this mode: 

shell> vim /etc/default/grub: 
GRUB_CMDLINE_LINUX_DEFAULT="...transparent_hugepage=never" 

shell> update-grub 

Moreover, i have a serious problem with the PVE cluster communication in two 
of eight PVE nodes, and with a VM, if you can help me, i will be extremely 
grateful, please see this link: 
http://forum.proxmox.com/threads/20523-Quorum-problems-with-PVE-2-3-and-3-3?p=104995#post104995 

Best regards 
Cesar 

> ----- Original Message ----- 
> From: "Alexandre DERUMIER" <aderumier at odiso.com> 
> To: "Cesar Peschiera" <brain at click.com.py> 
> Cc: "pve-devel" <pve-devel at pve.proxmox.com> 
> Sent: Thursday, December 18, 2014 2:24 AM 
> Subject: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 
> 
> 
> Note that currently, 
> 
> hugepages are managed with transparent hugepage mecanism. 
> 
> But it's seem that we can defined manually hugepages by numa nodes 
> 
> -object 
> memory-backend-file,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu,size=1024M,id=ram-node0 
> -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> -object 
> memory-backend-file,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu,size=1024M,id=ram-node1 
> -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> 
> I'll try to make a patch if you want to test. 
> 
> 
> ----- Mail original ----- 
> De: "aderumier" <aderumier at odiso.com> 
> À: "Cesar Peschiera" <brain at click.com.py> 
> Cc: "pve-devel" <pve-devel at pve.proxmox.com> 
> Envoyé: Jeudi 18 Décembre 2014 06:15:44 
> Objet: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 
> 
>>>Moreover, i guess that i have problems of Hugapages. 
> 
> I have found interesting blog: 
> 
> http://developerblog.redhat.com/2014/03/10/examining-huge-pages-or-transparent-huge-pages-performance/ 
> 
> It's explain how to see if hugepage impact performance or not. 
> 
> 
> 
> ----- Mail original ----- 
> De: "Cesar Peschiera" <brain at click.com.py> 
> À: "aderumier" <aderumier at odiso.com>, "pve-devel" 
> <pve-devel at pve.proxmox.com> 
> Envoyé: Jeudi 18 Décembre 2014 03:44:33 
> Objet: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 
> 
> Hi Alexandre 
> 
> I have installed your patches and with some test of MS-SQL-Server, i see a 
> better behavior in terms of speed (soon i will give the comparisons). 
> 
> Moreover, i guess that i have problems of Hugapages. 
> Please see this link, and answer me if you can: 
> http://forum.proxmox.com/threads/20449-Win2008R2-exaggeratedly-slow-with-256GB-RAM-and-strange-behaviours-in-PVE?p=104996#post104996 
> 
> 
> ----- Original Message ----- 
> From: "Alexandre DERUMIER" <aderumier at odiso.com> 
> To: "Cesar Peschiera" <brain at click.com.py> 
> Cc: <pve-devel at pve.proxmox.com> 
> Sent: Tuesday, December 02, 2014 9:50 AM 
> Subject: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 
> 
> 
> Hi, 
> can you test this: 
> 
> http://odisoweb1.odiso.net/pve-qemu-kvm_2.2-2_amd64.deb 
> http://odisoweb1.odiso.net/qemu-server_3.3-5_amd64.deb 
> 
> 
> then edit your vm config file: 
> 
> 
> sockets: 2 
> cores: 4 
> memory: 262144 
> numa0: memory=131072,policy=bind 
> numa1: memory=131072,policy=bind 
> 
> 
> (you need 1 numa by socket, total numa memory must be equal to vm memory). 
> 
> you can change cores number if you want. 
> 
> 
> and start the vm ? 
> 
> 
> ----- Mail original ----- 
> 
> De: "Alexandre DERUMIER" <aderumier at odiso.com> 
> À: "Cesar Peschiera" <brain at click.com.py> 
> Cc: pve-devel at pve.proxmox.com 
> Envoyé: Mardi 2 Décembre 2014 12:40:29 
> Objet: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 
> 
> Hi, 
> 
> some news. 
> 
> It's seem that current proxmox qemu build don't have numa support enable. 
> 
> So, previous command line don't work. 
> 
> 
> I'll send a patch for pve-qemu-kvm and also to add numa options to vm 
> config 
> file. 
> 
> 
> 
> ----- Mail original ----- 
> 
> De: "Alexandre DERUMIER" <aderumier at odiso.com> 
> À: "Cesar Peschiera" <brain at click.com.py> 
> Cc: pve-devel at pve.proxmox.com 
> Envoyé: Mardi 2 Décembre 2014 07:05:47 
> Objet: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 
> 
>>>at i would like to ask you if you can give me your suggestions in 
>>>practical terms, besides the brief theoretical explanation, this is due 
>>>to 
>>>that i am not a developer and i don't understand as apply it in my PVE. 
> 
> About the command line, each vm is a kvm process. 
> 
> So start your vm with current config, do a "ps -aux" , copy the big 
> "kvm -id 
> ... " command line for your vm, 
> 
> stop the vm. 
> 
> then, 
> 
> add my specials lines about numa, 
> 
> and paste the command line to start the vm ! 
> 
> 
> (kvm is so simple ;) 
> 
> 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 



More information about the pve-devel mailing list