Multi-vendor networks look beautiful on paper. In practice, they love to test our patience.
Recently at ITcare, we built a LAB for a PoC with Cisco vXR, Juniper Networks, Arista Networks vEOS and Nokia vSR routers. We wanted one simple backbone. A single OSPF area, MPLS with LDP everywhere, and iBGP sessions with 2 virtual route reflectors.
Even the first step reminded us that theory and practice are never the same. Forming OSPF adjacencies across different vendors always turns into a small puzzle. Everyone treats MTU differently. Some consider the Ethernet MTU, others the IP MTU. Nokia surprised us in a good way. Their network interfaces come with a very large default MTU, somewhere close to 9000. It makes sense, because you usually want jumbo frame support on backbone links, but it become tricky when you want to establish eBGP sessions due to tcp-mss. Still, we had to align MTU values almost on every interconnect, because no two vendors matched each other from the start. Once that was sorted out, OSPF came up fine.
Then we enabled MPLS and LDP. Full Internet routing tables injected. Everything looked perfect.
Until someone looked deeper into label allocation.
All vendors had a different logic, which is expected. But Cisco vXR was the most generous. It allocated labels for every prefix in LSDB. Juniper did not. Arista did not. Nokia did not. Even Mikrotik did not :).
So we disabled LDP between Arista nl-ams01-br01 and Cisco de-fra02-br01. That is when things became very interesting. Even though Cisco no longer had an LDP session with Arista, it continued to allocate a label for any FEC from LSDB and advertised it downstream. Juniper received the label and installed it under inet.3 convinced that a label switched path existed end to end. In reality, the packet left Cisco with no label. The control plane happily said everything was fine while the data plane quietly hide some details.
Nothing was technically wrong. Each vendor behaved exactly as designed. Yet in a multi vendor MPLS network this can become a real headache when you troubleshoot services like L3VPN or L2VPN. You think the path is established. You think the labels are in place. Only the packet knows the truth.
This lab reminded us again how important it is to understand the individual behavior of each platform. The more vendors you mix, the more creative the network becomes.
