Exploring ASICs: Turning the Simple into a Challenge

When setting up a new infrastructure, we’ve all been there – carefully comparing switches, with eyes mostly on port density and the price per port. Especially when these decisions are made in sync with management and financial teams. But here’s the thing, we often overlook a silent player in this game – the ASIC model.

In our experience, neglecting ASIC details has led to unexpected challenges, like drops on interfaces that may or may not have been an actual problem.

Drops on switch interfaces due to the packet-buffer usually become noticeable in two circumstances, either a congestion or mixed-speed setups, moving from 100G to 40G or 10G interfaces. The culprit? Switches with small packet buffers – or perhaps blame it on bursty traffic?

What happens usually next, simply picture the moment, an entire team of engineers transforms into wizards, casting magical spells (or something like that) around Quality of Service (QoS), desperately seeking ways to gracefully navigate through the drops and creatively manage a cascade of packets, incoming at an alarmingly high rate, into the limited space of a modest packet-buffer sized switch. The root cause traces back to a seemingly innocent decision made in the early stages – selecting the right switches and crafting the right design.

While drops aren’t always a deal-breaker, they’re essential for TCP to find the best speed for different operations or sessions. And the biggest challenge is to explain the drop reason and it’s positive/negative impact to management, explaining why switches behave this way, and why it was a good decision to move forward with that specific make/model. On the other hand, switches with larger buffers can prevent drops but might bring higher latency and fewer features.

It is well known that nowadays Broadcom leads the industry in offering highly complex, customized system-on-a-chip ASICs to meet customer requirements for deeply differentiated systems. Let’s review what options are most widely used in most Mid-to-Large sized infrastructures:

1. Trident Series. With performance scale, Trident leads the pack in terms of raw performance, making it a go-to choice for high-density data centers and complex networking infrastructures. Usually Trident is the ASIC that is designed to be run at leaf/ToR layer providing performance, scalability and cutting-edge technologies required in modern Data Centers like EVPN-VxLAN include different futures like anycast-gateway.
   
   • Trident+(9MB buffer): An early pioneer, offering a solid foundation for various networking applications. Well-suited for scenarios where cost-effectiveness and reliability are paramount.
     • Juniper QFX3500
     • Cisco N3K-C3064
     • Arista DCS-7050S/7050T/7050Q
   • Trident2(12MB buffer): Building upon its predecessor, Trident2 enhances scalability and performance. Ideal for environments demanding higher throughput and advanced features.
     • Juniper QFX5100
     • Cisco N3K-C3172PQ
     • Arista DCS-7050SX/7050TX
   • Trident2+(16MB buffer): An incremental improvement, focusing on refining power efficiency and adding support for emerging technologies. Suitable for organizations with an eye on energy consumption and future-proofing.
     • Juniper QFX5110
     • Cisco N3K-C31108PC
     • Arista DCS-7050SX2
   • Trident3(32MB buffer): The latest iteration, designed for the most demanding networking environments. Boasts enhanced capabilities in terms of performance, programmability, and support for cutting-edge features.
     • Juniper QFX5120
     • Cisco N3K-C3132C
     • Arista DCS-7050SX3
       
       
2. Tomahawk Series. With performance dominance, Tomahawk edges ahead in terms of performance, throughput and latency, making it the preferred choice for environments where speed is paramount. Usually Tomahawk based switches, are oriented towards Spine layer or low-latency infrastructures where the main goal, is to quickly move packets east-west / north-south, with less focus on features.
   
   • Tomahawk(16MB buffer): A game-changer in the realm of data centers, providing high-density, low-latency solutions. Suited for scenarios where data throughput and efficiency are critical.
     • Juniper QFX5200
     • Cisco N3K-C3264
     • Arista DCS-7060CX
   • Tomahawk2(42MB buffer): The next evolution, focusing on increased speed and flexibility. Ideal for organizations aiming to stay ahead in the competitive landscape of cloud computing and modern data centers.
     • Juniper QFX5210
     • Cisco N3K-C3264C
     • Dell Z9264F-ON
       
       
3. Jericho Series. Among the first silicons that offered huge buffers based on new technologies that sacrificing latency and processing time, is offering advanced features like network slicing and telemetry. It was designed for exceptional programmability and scalability, they meet the demands of modern networks seamlessly. With this type of ASICs you can forget about drops, mixed port-speed infrastructures and benefit from huge tables that can expand routing capabilities inside your infrastructure.
   
   • Jericho(4GB-32GB buffer): A breakthrough in innovation, featuring advanced capabilities such as network slicing and telemetry. Tailored for networks requiring unprecedented programmability and scalability.
     • Arista DCS-7280R
   • Jericho+(8GB-24GB buffer): The enhanced version, emphasizing even greater programmability and support for evolving technologies. Perfect for organizations with a forward-looking approach to networking.
     • Arista DCS-7280R2
     • Cisco Nexus 3600
   • Jericho2(4GB-16GB buffers): The latest evolution, building on the success of Jericho. Boasting enhanced features and performance, Jericho2 is designed to meet the growing demands of modern networking environments.
     • Juniper ACX7100
     • Cisco NCS 5700
     • Arista DCS-7280R3

Above is just a list of ASICs that are widely used, but, network switch silicon market is constantly developing and nowadays we have Trident4, Tomahawk5, Jericho3-AI and many others. There are many options, but we need to go through an exercies and select the right one for the infrastructure we want to build. Let’s check on how to make the right decision and focus on a specific platform:

1. Port Density and Flexibility:
   • Assess how the ASIC aligns with your network’s port density requirements and its flexibility to accommodate diverse configurations. Keep in mind that any network grows, and you will need to increase the capacities.
2. Advanced Features:
   • Prioritize power-efficient ASICs to contribute to sustainability goals and reduce operational costs over the long term. Foresight what you are planning to do in your infrastructure, check on what features and technologies are you planning to deploy long-term, and look into the right model.
3. Latency Considerations:
   • Evaluate the latency characteristics of the ASIC to ensure they match the specific needs of your network, particularly in applications sensitive to low latency.
4. Scalability:
   • Verify how well the ASIC supports scalability to accommodate future growth seamlessly without compromising performance. Check port density, and features that are supported by different ASICs that will help you scale, without the requirements for additional investment/hardware replacement.
5. Buffer Management:
   • Understand the buffering capabilities of the ASIC, considering the trade-offs between small and large buffers and their impact on network performance. Make sure to avoid mixed port speeds in your infrastructure when moving forward with low packet buffer switches
6. Cost-Effectiveness:
   • Evaluate the overall cost-effectiveness of the chosen ASIC, balancing performance and capabilities with the associated expenses.
7. Vendor Ecosystem:
   • Consider the broader vendor ecosystem around the ASIC, including available software and tools that complement its capabilities, ensuring a cohesive network environment. Trying to streamline your architecture by opting for a single vendor is beneficial for your team to streamline operations, simplify management, and enhance interoperability

The most important thing to keep in mind, is that selecting the right ASIC requires a careful examination of the unique demands of your network. The “rule of thumb” says that usually Trident based switches are used as ToR/Leaf and Tomahawk switches are used as Spine, but this is not always the case. Choose wisely, keeping in mind the specific requirements and aspirations of your organization, avoid haste, as impulsive decisions may result in paying double or triple the cost and effort.

#Switches #ASICs #Broadcom #Technology #Innovation