FCoE Quickie: Using Standard Ethernet Switches

I happened to come across a tweet on Twitter that asked the question:

“Can FCoE ‘just work’ on regular Ethernet switches?”

The answer to the question, while leading the asker to the right answer (no, you can’t use any old Ethernet switch) was nevertheless inaccurate. 

In relation to the question at hand, FCoE requires two things to run reliably:

  • 10Gb Ethernet, and
  • a lossless environment

Nearly all modern data center switches available on the market are 10G or higher, and of these nearly all of them allow for a lossless environment using Priority Flow Control (PFC). For the more geeky amongst you, look for IEEE 802.1Qbb in the spec sheet.

The person who responded to the question leapt the rails in his final paragraph, however:

Regarding your update question [about running FCoE on a standard Ethernet switch], it’s not an ‘ill-advised’, it’s a ‘packets won’t form so will not work’ thing. The specific error you’d see would vary depending on your CNA and its drivers but essentially there’s no successful L2/3 connection so that’s the kind of errors you get.

This is simply inaccurate.

Packets are encapsulated (i.e., “formed”) for FCoE at the initiator and target, not at the switch. Wether an initiator is using a hardware Converged Network Adapter (CNA) or a software-based initiator (á la open-fcoe) makes no difference.

The author is likely assuming that the initial FCoE fabric login would never happen without a switch capable of doing so, and as far as that is concerned he is correct. However, that only matters if one cares about using a FC name server for connectivity. VN2VN FCoE does not require a switch to have FCoE capability – only PFC (i.e., lossless, or “no drop”) capabilities.

The correct answer to the requestor’s “update” question is that it is extremely ill-advised to run FCoE on a non-lossless, standard Ethernet switch.

The issue is what happens when congestion occurs – the protocol assumption is that frames are not going t be dropped due to congestion. Instead, the FCoE protocol assumes that the traffic will be paused in instances of congestion so as not to lose data. Standard Ethernet switches, when faced with congestion, will simply drop packets and expect the upper layer protocols (TCP, usually) to sort out the retransmission. With FCoE there is no TCP, so Bad Things™ can happen when overloaded.

One final point: the last sentence about “L3 connection errors” makes no sense, as FCoE is, from an Ethernet standpoint, a Layer 3 protocol, as opposed to IP. FCoE is not routed, nor does it cross L2 boundaries in an Ethernet environment.

I highly recommend this book on Fibre Channel over Ethernet (FCoE) or this one on I/O Consolidation in the Data Center for 100% accurate information.

There was no way to insert a comment or make a correction to the original website, but it seemed appropriate to write it here.