[PVE-User] Confusing about Bond 802.3ad

[Mark Adams]

That is it, as I understand it Josh. you basically need to turn your switch
in to X seperate switches so each nodes nic, is running on a "seperate"
network.

if you were to do the same thing physically without any config, with 3
nodes, you would need to have as many seperate switches as you wanted nics
in the balance-rr.

I understand mikrotik support balance-rr, but tbh I don't even count them
as a normal switch manufacturer. Their game is routers.... I don't know any
other switches which have support for balance-rr?

as for the 3Gbps limit I mentioned earlier with balance-rr (no matter how
many nics you have)... I don't know if that was just an issue of the day as
cheap 10Gbps came along and the need evaporated for me. I would love to
know if anyone has a test setup to try it though.

Cheers

On Sat, 25 Aug 2018, 00:15 Josh Knight, <josh at noobbox.com> wrote:

> Just guessing here, if the switch doesn't support rr on its port channels,
> then using separate VLANs instead of bundles on the switch is essentially
> wiring nodeA to nodeB. That way you don't hit the port channel hashing on
> the switch and you keep the rr as-is from A to B.
>
> I would also try using UDP mode on iperf to see if it's TCP retransmission
> that's preventing you from getting closer to 4Gbps. Another useful tool is
> maisezahn for traffic generation, though it is more complex to run.
>
> On Fri, Aug 24, 2018, 4:59 PM Gilberto Nunes <gilberto.nunes32 at gmail.com>
> wrote:
>
> > I can get 3 gbps. At least 1.3 gbps.
> > Don't know why!
> > Em 24/08/2018 17:36, "mj" <lists at merit.unu.edu> escreveu:
> >
> > > Hi Mark,
> > >
> > > On 08/24/2018 06:20 PM, Mark Adams wrote:
> > >
> > >> also, balance-rr through a switch requires each nic to be on a
> seperate
> > >> vlan. You probably need to remove your lacp config also but this
> depends
> > >> on
> > >> switch model and configuration. so safest idea is remove it.
> > >>
> > >
> > > <snip>
> > >
> > > then I belive your iperf test will return ~3Gbps... i seem to remember
> > >> performance doesnt get much better than this but I cant remember why.
> > >>
> > >> Also can't say if this is a good setup for ceph performance..
> > >>
> > >
> > > This is really interesting info, i did not know this. Someone has tried
> > > this with ceph? Any experiences to share..?
> > >
> > > Strange that performence turns out to be ~3Gbps, instead of the
> expected
> > > 4...
> > >
> > > Anyone with more information    on this subject?
> > >
> > > Have a nice weekend all!
> > >
> > > MJ
> > > _______________________________________________
> > > pve-user mailing list
> > > pve-user at pve.proxmox.com
> > > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> > >
> > _______________________________________________
> > pve-user mailing list
> > pve-user at pve.proxmox.com
> > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> >
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>