Month: July 2012

Ultra-low-latency ToR Switches For Building Scalable Leaf-Spine Fabrics

When building scalable Leaf-Spine fabrics, network architects look for low-latency, high-density switches to use at the leaf layer. There are many fixed switches that can be used for Top-0f-rack (ToR) at the leaf layer to provide connectivity upstream to the spine layer. What I’m about to compare are 3 ultra-low-latency ToR switches based on merchant silicon available in the market today for that purpose.

Cisco Nexus 3064 

The 3064 is 1 RU heigh and has a low latency and low power consumption per port. It has (48) 1/10GbE ports and (4) 40 GbE uplinks which can be each used in native 40 GbE or split into four 10GbE ports. It runs the same Nx-OS as the Nexus 7000 and 5000 series.

The Nexus 3064 is Cisco’s first switch in the Nexus family to use merchant silicon (Broadcom Trident+ chip). I’m curious to see whether Cisco will continue to use merchant silicon in future products or stick to their propreitery Nuova ASIC of the 7000 and 5000 series.

 

Arista 7050S-64

Arista 7050S-64 is very similar to the Cisco Nexus 3064 in terms of latency, interface types, and switching capacity. Its power consumption is less than the Nexus 3064 though. Arista’s fixed switches are known for their low power consumption and the 7050S-64 is no exception. Its power consumption is under 2W per port. You really cannot beat that!

 

Dell Force10 S4810

The Dell Force10 S4810 is another great ToR switch that can be used to build leaf-spine fabrics. It offers the same interface types as the Nexus 3064 and Arista 7050s-64; and similar form factor. It does however have slightly higher power consumption per port.

 

Ultra-low-latency 10/40 GbE Top-of-Rack Switches

Cisco Nexus 3064Arista 7050S-64Dell Force10 S4810
Ports48 x 1/10GbE SPF+ and 4 x 40GbE QSFP+48 x 1/10GbE SPF+ and 4 x 40GbE QSFP+48 x 1/10GbE SPF+ and 4 x 40GbE QSFP+
Packet Latency (64 bytes)824ns800ns700ns
OSNx-OSArista EOSFTOS
Form Factor1 RU1 RU1 RU
Switching Capacity1.28 Tbps1.28 Tbps1.28 Tbps
Power Supply2 Redundant & Hot swappable power supplies2 Redundant & Hot swappable power supplies2 Redundant & Hot swappable power supplies
Typical Operating Power177W103W220W
Full Data SheetData SheetData SheetData Sheet


Share This:
Facebooktwitterredditpinterestlinkedintumblrmail

Cisco UCS Supported IOM Connectivity Options

When connecting the UCS chassis to the Fabric interconnect, it’s important to follow the design rules or otherwise you may end up with unexpected behavior. UCS supports up to two fabric extenders (2100/2200 series) per chassis and two fabric interconnects (6100/6200 series) per cluster. To have a fully redundant system you will need two fabric extenders and two fabric interconnects connected as shown below the first picture (topology in the top right and top left).

Here are some UCS design rules to keep in mind:

  • Direct one-to-one relationship between the FEX and fabric interconnect. Meaning each Fabric Extender (FEX) can be connected only to a single fabric interconnect. You cannot dual-home a FEX. Similarly a fabric interconnect cannot connect to more than one FEX.
  • If you choose to have only one FEX in the chassis you must place that FEX into the left bay (as viewed from the rear of the enclosure)
  • When using two fabric interconnects for redundancy you must establish a cluster link between them by connecting the L1/L2 ports on the first fabric interconnect to the L1/L2 pots on the second one.

Correct IOM connectivity options:

The picture below shows the supported IOM connectivity options for UCS

Cisco UCS Supported Connectivity Options

 

Incorrect IOM connectivity options:

The picture below shows some unsupported IOM connectivity options for UCS

Cisco UCS unSupported Connectivity Options

The first topology (upper left) is not supported because the links between the fabric interconnects are missing

The second topology (lower left) is not supported because both FEX’s are uplinked to the same fabric interconnect.

The third topology (upper right) is not supported because the FEX is dual-homed to the fabric interconnects.

The fourth topology (lower right) is not supported because there is only one FEX in the chassis so that FEX should be placed into the left bay.

 


Share This:
Facebooktwitterredditpinterestlinkedintumblrmail