https://pve.proxmox.com/mediawiki/api.php?action=feedcontributions&user=Supme&feedformat=atomProxmox VE - User contributions [en]2024-03-29T11:18:03ZUser contributionsMediaWiki 1.35.11https://pve.proxmox.com/mediawiki/index.php?title=High_Availability_Cluster_4.x&diff=7324High Availability Cluster 4.x2015-07-06T10:44:28Z<p>Supme: /* HA Goups */</p>
<hr />
<div>=Introduction= <br />
<br />
<b>BETA NOT FOR PRODUCTION</b><br />
<br />
Proxmox VE High Availability Cluster (Proxmox VE HA Cluster) enables the definition of high available virtual machines. In simple words, if a virtual machine (VM) is configured as HA and the physical host fails, the VM is automatically restarted on one of the remaining Proxmox VE Cluster nodes.<br />
<br />
The Proxmox VE HA Cluster is based on the Proxmox VE HA Manager (pve-ha-manager) - using watchdog fencing.<br />
Major benefit of Linux softdog or hardware watchdog is zero configuration - it just works out of the box.<br />
<br />
In order to learn more about functionality of the new Proxmox VE HA manager, install the HA simulator. <br />
<br />
==Update to the latest version==<br />
Before you start, make sure you have installed the latest packages, just run the following on all nodes:<br />
<br />
apt-get update && apt-get dist-upgrade<br />
<br />
= System requirements =<br />
<br />
If you run HA, high end server hardware with no single point of failure is required. This includes redundant disks, redundant power supply, UPS systems, and network bonding. <br />
<br />
*Proxmox VE 4.0 comes with Self-Fencing with hardware watchdog or Software watchdog.<br />
*Fully configured [[Proxmox_VE_4.x_Cluster]] (version 4.0 and later), with at least 3 nodes (maximum supported configuration: currently 32 nodes per cluster).<br />
*Shared storage (SAN, NAS/NFS, Ceph, DRBD9, ... for virtual disk images) <br />
*Reliable, redundant network, suitable configured which supports multicast<br />
*An extra network for cluster communication, one network for VM traffic and one network for storage traffic. <br />
<br />
It's essential that you use redundant network connections for the cluster communication (bonding). If not, a simple switch reboot (or power loss on the switch) can fence all cluster nodes if it takes longer than 60 sec.<br />
<br />
=HA Configuration=<br />
<br />
Adding and managing VM´s or containers for HA can be done via GUI or CLI.<br />
<br />
==Fencing==<br />
Proxmox VE Cluster 4.0 or greater comes with watchdog fencing.<br />
This works out of the box, no configuration is required.<br />
<br />
How Watchdog fencing works:<br />
<br />
If the node has connection with the cluster and has quorum, the watchdog will be reset. If quorum is lost, the node is not able to reset the watchdog. This will trigger a reboot after 60 seconds.<br />
<br />
If your hardware has a hardware watchdog, this one will be automatically detected and used. Otherwise, ha-manager just uses the Linux softdog. Therefore testing Proxmox VE HA inside a virtual environment is possible.<br />
<br />
==HA Groups==<br />
The Proxmox VE HA Cluster is using Groups for vm to node mapping. <br />
<br />
For example: If a "vm100" is in the group "ONE" and group "ONE" has members "pve1,pve2" and "vm100" is running on pve1.<br />
<br />
Therefore "pve1" is crashing. "vm100" can only migrated to "pve2".<br />
<br />
The Groups has two option restricted and nofailback.<br />
<br />
*restricted: VM's bound to the group may only run on cluster members which are also members of the group. If no members of the group are available, the service is placed in the stopped state.<br />
<br />
*nofailback: Enabling this option for a group will prevent automated fail-back after a more-preferred node rejoins the cluster.<br />
<br />
==Enable a VM/CT for HA==<br />
On the CLI, you can use ha-manger to achieve this task.<br />
<br />
<b>IMPORTANT:</b><br />
<br />
If you enable HA it's not possible to turnoff the VM inside the VM. Also, if it is disabled the VM will be stopped.<br />
<br />
If you add a VM/CT, its instantly 'ha-managed'. <br />
<br />
ha-manager add vm:100<br />
<br />
To add a VM/CT on GUI.<br />
<br />
[[Image:Screen-Add-HA-4-managed_VM-CT.png|Screen-Add-HA-4-managed_VM-CT|thumb]]<br />
<br />
==Disable a VM/CT for HA==<br />
<br />
If you want to disable a ha-managed VM/CT (e.g. for shutdown) via CLI:<br />
<br />
ha-manager disable vm:100<br />
<br />
If you want to re-enable a ha-managed VM/CT:<br />
<br />
ha-manager enable vm:100<br />
<br />
==HA Cluster Maintenance (node reboots)==<br />
If you need to reboot a node, e.g. because of a kernel update, you need to migrate all VM/CT to another node or disable them. By disabling them, all resources are stopped. All VM guests will get an ACPI shutdown request (if this won't work due to VM internal settings, they'll just get a 'stop').<br />
<br />
The command will take some time for execution, monitor the "tasks" and the VM´s and CT´s on the GUI. As soon as the VM/CT are either stopped or migrated, you can reboot your node. As soon as the node is up again, continue with the next node and so on.<br />
<br />
=HA Simulator=<br />
[[Image:Screen-HA-4-simulator.png|Screen-HA-4-simulator|thumb]]<br />
<br />
By using the HA simulator you can test and learn all functionalities of the Proxmox VE HA solutions.<br />
<br />
The simulator allows you to watch and test the behaviour of a real-world 3 node cluster with 6 VM's.<br />
<br />
You do not have to setup or configure a real cluster, the HA simulator runs out of the box on the current code base.<br />
<br />
Install with apt:<br />
<br />
apt-get install pve-ha-simulator<br />
<br />
To start the simulator you must have a X11 redirection to your current system.<br />
<br />
If you are on a Linux machine you can use:<br />
<br />
ssh root@<IPofPVE4> -Y<br />
<br />
On Windows it is working with [http://mobaxterm.mobatek.net/ mobaxterm].<br />
<br />
After starting the simulator create a working directory:<br />
<br />
mkdir working<br />
<br />
To start the simulator type<br />
<br />
pve-ha-simulator working/<br />
<br />
=Video Tutorials=<br />
[http://www.youtube.com/user/ProxmoxVE Proxmox VE Youtube channel]<br />
<br />
=Testing=<br />
Before going into production it is highly recommended to do as many tests as possible. Then, do some more.<br />
==Useful command line tools==<br />
Here is a list of useful CLI tools:<br />
*ha-manger - to manage the ha stack of the cluster<br />
*pvecm - to manage the cluster-manager <br />
*corosync* - to manipulate the corosync <br />
<br />
[[Category: HOWTO]][[Category: Installation]]</div>Supmehttps://pve.proxmox.com/mediawiki/index.php?title=Mainboards&diff=2456Mainboards2010-09-13T10:28:47Z<p>Supme: /* SUPERMICRO */</p>
<hr />
<div>=Introduction=<br />
This Mainboard / Motherboard compatibility list for Proxmox VE is work in progress and is never complete. Everybody is encouraged to add working configurations - do not forget to add the used Proxmox VE kernel version (2.6.18/2.6.24/2.6.32).<br />
<br />
==Intel==<br />
* S3000 (2.6.18/2.6.24/2.6.32) - [http://www.intel.com/support/motherboards/server/s3000ah/sb/CS-029798.htm Product Discontinuation Notice]<br />
* [http://www.intel.com/Products/Server/Motherboards/Entry-S3200SH/Entry-S3200SH-overview.htm S3200] (2.6.18/2.6.24/2.6.32)<br />
* S3420GPLC (2.6.32) see: [http://forum.proxmox.com/threads/3699-Hardware-Success-Story?highlight=hardware+success+story Hardware Success Story]<br />
* [http://www.intel.com/products/server/motherboards/s5000vsa/s5000vsa-overview.htm S5000Vsa] (2.6.32)<br />
* [http://www.intel.com/p/en_US/support/highlights/dsktpboards/d975xbx Intel D975BX] works fine with pve-manager/1.5/4822 and kernel: 2.6.18-2-pve<br />
<br />
==ASUS==<br />
* M4A78T-E (2.6.24) onboard-Nic don't work<br />
* M3A78 AMI ACPI BIOS 1606, CPU AMD Athlon 64 X2 Dual Core Processor 4200+ works with 2.6.24-2-pve but no with more recent (even with 2.6.24-5-pve hangs at boot time)<br />
* M3A78-EM, Phenom II X4 920 Processor, 1.2 and 1.3 hangs at boot time, not tried 1.4<br />
* M3A, AMD Phenom 9550 processor, works with 1.3 (not tried with 1.4, don't remember about previous versions of Proxmox) <br />
* [http://www.asus.com/product.aspx?P_ID=yZD4yFdLw1l3gZ35 P7P55D Deluxe] works fine<br />
* [http://www.asus.com/product.aspx?P_ID=j02KziJq95KbCQNm P7P55D Pro] works fine with pve-manager/1.5/4822 and kernel: 2.6.18-2-pve<br />
<br />
==Gigabyte==<br />
* [http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=2692 GA-G33M-S2L] (2.6.24/2.6.32), everything works out of the box.<br />
* [http://www.gigabyte.com.tw/products/product-page.aspx?pid=3285&dl=1#ov GA-EP45-UD3L] - Use on BIOS [http://download.gigabyte.asia/FileList/BIOS/motherboard_bios_ga-ep45-ud3l_f4.exe vF4] - The BIOS vF7 causes ACPI errors and does not boot<br />
* [http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=3302 GA-MA770 UD3 rev 2.0] - BIOS FG causes kernel panic on copy large files (> 5GB) to server (AMD Athlon 5050e, 160GB SATA, 8GB RAM)<br />
<br />
==SUPERMICRO==<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8STE.cfm X8STE] (2.6.18), all components working ("cluster-proof")<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8ST3-F.cfm X8ST3-F] (2.6.32), all components working (Latest [http://www.supermicro.com/about/policies/disclaimer.cfm?url=/support/resources/getfile.aspx?ID=733 BIOS])<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/3210/X7SBE.cfm X7SBE] (2.6.32-3-pve), all components working (claster, DRBD storage, software RAID0)<br />
[[Category: HOWTO]] [[Category: Hardware]]</div>Supmehttps://pve.proxmox.com/mediawiki/index.php?title=Mainboards&diff=2455Mainboards2010-09-13T10:27:29Z<p>Supme: /* SUPERMICRO */</p>
<hr />
<div>=Introduction=<br />
This Mainboard / Motherboard compatibility list for Proxmox VE is work in progress and is never complete. Everybody is encouraged to add working configurations - do not forget to add the used Proxmox VE kernel version (2.6.18/2.6.24/2.6.32).<br />
<br />
==Intel==<br />
* S3000 (2.6.18/2.6.24/2.6.32) - [http://www.intel.com/support/motherboards/server/s3000ah/sb/CS-029798.htm Product Discontinuation Notice]<br />
* [http://www.intel.com/Products/Server/Motherboards/Entry-S3200SH/Entry-S3200SH-overview.htm S3200] (2.6.18/2.6.24/2.6.32)<br />
* S3420GPLC (2.6.32) see: [http://forum.proxmox.com/threads/3699-Hardware-Success-Story?highlight=hardware+success+story Hardware Success Story]<br />
* [http://www.intel.com/products/server/motherboards/s5000vsa/s5000vsa-overview.htm S5000Vsa] (2.6.32)<br />
* [http://www.intel.com/p/en_US/support/highlights/dsktpboards/d975xbx Intel D975BX] works fine with pve-manager/1.5/4822 and kernel: 2.6.18-2-pve<br />
<br />
==ASUS==<br />
* M4A78T-E (2.6.24) onboard-Nic don't work<br />
* M3A78 AMI ACPI BIOS 1606, CPU AMD Athlon 64 X2 Dual Core Processor 4200+ works with 2.6.24-2-pve but no with more recent (even with 2.6.24-5-pve hangs at boot time)<br />
* M3A78-EM, Phenom II X4 920 Processor, 1.2 and 1.3 hangs at boot time, not tried 1.4<br />
* M3A, AMD Phenom 9550 processor, works with 1.3 (not tried with 1.4, don't remember about previous versions of Proxmox) <br />
* [http://www.asus.com/product.aspx?P_ID=yZD4yFdLw1l3gZ35 P7P55D Deluxe] works fine<br />
* [http://www.asus.com/product.aspx?P_ID=j02KziJq95KbCQNm P7P55D Pro] works fine with pve-manager/1.5/4822 and kernel: 2.6.18-2-pve<br />
<br />
==Gigabyte==<br />
* [http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=2692 GA-G33M-S2L] (2.6.24/2.6.32), everything works out of the box.<br />
* [http://www.gigabyte.com.tw/products/product-page.aspx?pid=3285&dl=1#ov GA-EP45-UD3L] - Use on BIOS [http://download.gigabyte.asia/FileList/BIOS/motherboard_bios_ga-ep45-ud3l_f4.exe vF4] - The BIOS vF7 causes ACPI errors and does not boot<br />
* [http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=3302 GA-MA770 UD3 rev 2.0] - BIOS FG causes kernel panic on copy large files (> 5GB) to server (AMD Athlon 5050e, 160GB SATA, 8GB RAM)<br />
<br />
==SUPERMICRO==<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8STE.cfm X8STE] (2.6.18), all components working ("cluster-proof")<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8ST3-F.cfm X8ST3-F] (2.6.32), all components working (Latest [http://www.supermicro.com/about/policies/disclaimer.cfm?url=/support/resources/getfile.aspx?ID=733 BIOS])<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/3210/X7SBE.cfm X7SBE] (2.6.32-3-pve), all components working ("cluster-proof")<br />
[[Category: HOWTO]] [[Category: Hardware]]</div>Supmehttps://pve.proxmox.com/mediawiki/index.php?title=Mainboards&diff=2454Mainboards2010-09-13T10:26:30Z<p>Supme: /* SUPERMICRO */</p>
<hr />
<div>=Introduction=<br />
This Mainboard / Motherboard compatibility list for Proxmox VE is work in progress and is never complete. Everybody is encouraged to add working configurations - do not forget to add the used Proxmox VE kernel version (2.6.18/2.6.24/2.6.32).<br />
<br />
==Intel==<br />
* S3000 (2.6.18/2.6.24/2.6.32) - [http://www.intel.com/support/motherboards/server/s3000ah/sb/CS-029798.htm Product Discontinuation Notice]<br />
* [http://www.intel.com/Products/Server/Motherboards/Entry-S3200SH/Entry-S3200SH-overview.htm S3200] (2.6.18/2.6.24/2.6.32)<br />
* S3420GPLC (2.6.32) see: [http://forum.proxmox.com/threads/3699-Hardware-Success-Story?highlight=hardware+success+story Hardware Success Story]<br />
* [http://www.intel.com/products/server/motherboards/s5000vsa/s5000vsa-overview.htm S5000Vsa] (2.6.32)<br />
* [http://www.intel.com/p/en_US/support/highlights/dsktpboards/d975xbx Intel D975BX] works fine with pve-manager/1.5/4822 and kernel: 2.6.18-2-pve<br />
<br />
==ASUS==<br />
* M4A78T-E (2.6.24) onboard-Nic don't work<br />
* M3A78 AMI ACPI BIOS 1606, CPU AMD Athlon 64 X2 Dual Core Processor 4200+ works with 2.6.24-2-pve but no with more recent (even with 2.6.24-5-pve hangs at boot time)<br />
* M3A78-EM, Phenom II X4 920 Processor, 1.2 and 1.3 hangs at boot time, not tried 1.4<br />
* M3A, AMD Phenom 9550 processor, works with 1.3 (not tried with 1.4, don't remember about previous versions of Proxmox) <br />
* [http://www.asus.com/product.aspx?P_ID=yZD4yFdLw1l3gZ35 P7P55D Deluxe] works fine<br />
* [http://www.asus.com/product.aspx?P_ID=j02KziJq95KbCQNm P7P55D Pro] works fine with pve-manager/1.5/4822 and kernel: 2.6.18-2-pve<br />
<br />
==Gigabyte==<br />
* [http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=2692 GA-G33M-S2L] (2.6.24/2.6.32), everything works out of the box.<br />
* [http://www.gigabyte.com.tw/products/product-page.aspx?pid=3285&dl=1#ov GA-EP45-UD3L] - Use on BIOS [http://download.gigabyte.asia/FileList/BIOS/motherboard_bios_ga-ep45-ud3l_f4.exe vF4] - The BIOS vF7 causes ACPI errors and does not boot<br />
* [http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=3302 GA-MA770 UD3 rev 2.0] - BIOS FG causes kernel panic on copy large files (> 5GB) to server (AMD Athlon 5050e, 160GB SATA, 8GB RAM)<br />
<br />
==SUPERMICRO==<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8STE.cfm X8STE] (2.6.18), all components working ("cluster-proof")<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8ST3-F.cfm X8ST3-F] (2.6.32), all components working (Latest [http://www.supermicro.com/about/policies/disclaimer.cfm?url=/support/resources/getfile.aspx?ID=733 BIOS])<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/3210/X7SBE.cfm X7SBE] (2.6.32), all components working ("cluster-proof")<br />
[[Category: HOWTO]] [[Category: Hardware]]</div>Supmehttps://pve.proxmox.com/mediawiki/index.php?title=Mainboards&diff=2453Mainboards2010-09-13T10:26:08Z<p>Supme: /* SUPERMICRO */</p>
<hr />
<div>=Introduction=<br />
This Mainboard / Motherboard compatibility list for Proxmox VE is work in progress and is never complete. Everybody is encouraged to add working configurations - do not forget to add the used Proxmox VE kernel version (2.6.18/2.6.24/2.6.32).<br />
<br />
==Intel==<br />
* S3000 (2.6.18/2.6.24/2.6.32) - [http://www.intel.com/support/motherboards/server/s3000ah/sb/CS-029798.htm Product Discontinuation Notice]<br />
* [http://www.intel.com/Products/Server/Motherboards/Entry-S3200SH/Entry-S3200SH-overview.htm S3200] (2.6.18/2.6.24/2.6.32)<br />
* S3420GPLC (2.6.32) see: [http://forum.proxmox.com/threads/3699-Hardware-Success-Story?highlight=hardware+success+story Hardware Success Story]<br />
* [http://www.intel.com/products/server/motherboards/s5000vsa/s5000vsa-overview.htm S5000Vsa] (2.6.32)<br />
* [http://www.intel.com/p/en_US/support/highlights/dsktpboards/d975xbx Intel D975BX] works fine with pve-manager/1.5/4822 and kernel: 2.6.18-2-pve<br />
<br />
==ASUS==<br />
* M4A78T-E (2.6.24) onboard-Nic don't work<br />
* M3A78 AMI ACPI BIOS 1606, CPU AMD Athlon 64 X2 Dual Core Processor 4200+ works with 2.6.24-2-pve but no with more recent (even with 2.6.24-5-pve hangs at boot time)<br />
* M3A78-EM, Phenom II X4 920 Processor, 1.2 and 1.3 hangs at boot time, not tried 1.4<br />
* M3A, AMD Phenom 9550 processor, works with 1.3 (not tried with 1.4, don't remember about previous versions of Proxmox) <br />
* [http://www.asus.com/product.aspx?P_ID=yZD4yFdLw1l3gZ35 P7P55D Deluxe] works fine<br />
* [http://www.asus.com/product.aspx?P_ID=j02KziJq95KbCQNm P7P55D Pro] works fine with pve-manager/1.5/4822 and kernel: 2.6.18-2-pve<br />
<br />
==Gigabyte==<br />
* [http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=2692 GA-G33M-S2L] (2.6.24/2.6.32), everything works out of the box.<br />
* [http://www.gigabyte.com.tw/products/product-page.aspx?pid=3285&dl=1#ov GA-EP45-UD3L] - Use on BIOS [http://download.gigabyte.asia/FileList/BIOS/motherboard_bios_ga-ep45-ud3l_f4.exe vF4] - The BIOS vF7 causes ACPI errors and does not boot<br />
* [http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=3302 GA-MA770 UD3 rev 2.0] - BIOS FG causes kernel panic on copy large files (> 5GB) to server (AMD Athlon 5050e, 160GB SATA, 8GB RAM)<br />
<br />
==SUPERMICRO==<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8STE.cfm X8STE] (2.6.18), all components working ("cluster-proof")<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8ST3-F.cfm X8ST3-F] (2.6.32), all components working (Latest [http://www.supermicro.com/about/policies/disclaimer.cfm?url=/support/resources/getfile.aspx?ID=733 BIOS])<br />
* [http://www.supermicro.com/products/motherboard/Xeon3000/3210/X7SBE.cfm X7SBE] (2.6.18), all components working ("cluster-proof")<br />
[[Category: HOWTO]] [[Category: Hardware]]</div>Supme