[PVE-User] bond0 Issues

[Andrew Niemantsverdriet]

After searching online today I found a possible solution so that
changes to the switch itself do not have to be made.

It appears that if you give the bond0 an IP address and leave the
bridge with out an IP that may fix the problem. I tried it and found
that Proxmox does not allow for a blank address on vmbr0. I tried
manually to edit the /etc/network/interfaces file and was still unable
to get this to work. In fact the machine was unresponsive when I did
that, so I may screwed something up. I am not opposed to getting the
802.3ad link going. I am just exploring the possibilities.

On Fri, Jun 12, 2009 at 9:24 AM, Jeff Saxe<jsaxe at briworks.com> wrote:
> Ah, I just looked into it a bit more. 802.3ad is dynamic link aggregation,
> and in absence of a vendor-proprietary protocol like Cisco's PAgP, this
> pretty much has to be the standard Link Aggregation Control Protocol, LACP.
> A data sheet In found on the ProCurve 4200vl switch says it does support
> that method. So go ahead and make your bond0 interface on Proxmox use
> 802.3ad, and on the switch set the four ports into one channel group or LAG
> group or something, and turn LACP to "active" mode on those four ports --
> this means the switch is actively seeking to form a group, and the Linux
> server on the other side just needs to respond affirmatively to the special
> LACP frames. Assuming it works, this will be safer, and you'll be able to
> monitor it from the switch side -- in the GUI or CLI or even SNMP queries,
> you can see which links are part of the bundle, how long they have been in
> the bundle, etc. And it does provide distribution of frames across the
> links, in a manner very similar to the "balance-xor" mode.
>
> http://www.mjmwired.net/kernel/Documentation/networking/bonding.txt
>
> I'm afraid I have never configured a ProCurve switch, despite my networking
> engineering experience, so I can't follow through on my promise to help you.
> Sorry! Look in the menus for options about "channel bonding", "LAG groups",
> "LACP active mode", etc.
>
> -- JeffS
>
>
> On Jun 12, 2009, at 10:34 AM, Andrew Niemantsverdriet wrote:
>
>> Jeff,
>>
>> It is my understanding that you DO NOT need special switch
>> configuration for using balance-rr, balance-xor, active-backup,
>> balance-tlb or balance-alb. You DO need special switch configuration
>> for 802.3ad and broadcast. Am I wrong? I could very well be just
>> started looking into bonding 2 days ago when I migrated a server and
>> caused issues.
>>
>> The switch I am using is a HP Procurve 4200vl
>>
>> Thanks,
>> _
>> /-\ ndrew
>>
>> On Fri, Jun 12, 2009 at 6:22 AM, Jeff Saxe<jsaxe at briworks.com> wrote:
>>>
>>> I think the more pressing question is: Did you configure the Ethernet
>>> switch on the other side of your four bonded NIC cables to create a link
>>> aggregate, i.e., "port channel" or "EtherChannel"? I can certainly see why
>>> one would bond 4 NICs together -- it gives 4 times the bandwidth, as long as
>>> a lot of conversations are happening over it so that the layer 2
>>> load-balancing rules can take effect. And you can throw them all in the same
>>> bridge, and all the IP addresses just magically float between all four
>>> links, and the failover time when one link fails (i.e., one switch module in
>>> a multi-module switch fails) is extremely fast, about a second or so. But
>>> the devices on both sides of the aggregate have to agree that it is going to
>>> happen, otherwise bridge frame-flooding behavior and Spanning Tree protocols
>>> will get very confused!
>>>
>>> I have not tried the NIC bonding in Proxmox yet (the test servers I am
>>> using happen to have only one NIC), but I know how it works in general. If
>>> your switch defaults to using a link aggregation protocol, like LACP or
>>> PAgP, you will have to turn it off for these four ports, since I don't think
>>> the Debian Linux side of it will perform LACP. Just set those four ports on
>>> the Ethernet switch to the same channel group number and to "channeling mode
>>> ON", i.e., always on, not negotiating whether to channel or not. And be sure
>>> not to accidentally plug in any other servers into any of those four ports
>>> later until you take them out of the aggregate, or again you will knock one
>>> or both servers off the network... mode "ON" is easier, but LACP is safer,
>>> since it verifies that the devices on both sides of the cables agree on
>>> who's plugged into the channeling ports before forming the aggregate.
>>>
>>> Good luck, Andrew. If you tell me what model of switch you have, I can
>>> probably hack up the config statements for you.
>>>
>>> -- Jeff Saxe
>>> Network Engineer, Blue Ridge InternetWorks
>>> Charlottesville, Virginia
>>>
>
>



-- 
 _
/-\ ndrew Niemantsverdriet
Academic Computing
(406) 238-7360
Rocky Mountain College
1511 Poly Dr.
Billings MT, 59102