NX-OS Fundamentals and Leaf-Spine Architecture
Cisco Nexus switches run NX-OS, which differs from IOS in several ways: features must be explicitly enabled (feature ospf, feature bgp, feature vpc), VDC (Virtual Device Context) allows logical partitioning on high-end platforms, and vPC (virtual Port Channel) eliminates STP blocked ports by allowing active-active dual uplinks. vPC components: vPC domain ID (must match on both peers), vPC peer-link (carries VLAN traffic for orphan ports and SVI traffic), vPC peer-keepalive (out-of-band heartbeat for split-brain detection), and vPC member ports (the actual LAG to downstream devices). Peer-link failure does not cause failover — keepalive failure does.
VXLAN BGP EVPN Fabric
VXLAN (RFC 7348) encapsulates L2 frames in UDP port 4789, extending L2 domains across L3 boundaries. BGP EVPN (RFC 7432) is the control plane that distributes MAC and IP reachability (type-2 routes), host mobility (type-2 with sequence number), prefix routes (type-5), and multicast group membership (type-6/7). In a leaf-spine fabric: leafs are VTEP (VXLAN Tunnel Endpoints), spines are route reflectors for BGP EVPN. Underlay uses IS-IS or OSPF for loopback reachability. Anycast gateway (same gateway MAC and IP on all leafs for a VNI) eliminates default gateway bouncing. Distributed Anycast Gateway is the standard design.
ACI: APIC, Tenants, and Contracts
ACI (Application Centric Infrastructure) uses a policy model based on the logical construct hierarchy: Tenant > VRF > Bridge Domain > EPG (Endpoint Group). Communication between EPGs requires a Contract (provider EPG advertises, consumer EPG uses, subject defines the filters/protocols). APIC is the SDN controller — it programs all leaf and spine policies via OpFlex. ACI fabric uses IS-IS as the underlay and VXLAN+ARP gleaning for the overlay (no BGP EVPN unless L3Out is configured). Microsegmentation: EPGs can contain endpoints by attribute (IP, VM tag, domain) for zero-trust within the fabric.
Storage Networking: FC, FCoE, and NVMe-oF
Fibre Channel concepts: World Wide Port Name (WWPN) identifies HBAs, World Wide Node Name (WWNN) identifies arrays. Zoning controls which initiators can see which targets — hard zoning by port, soft zoning by WWN (less secure). VSANs virtualise the FC fabric similar to VLANs. FCoE (Fibre Channel over Ethernet) maps FC frames to Ethernet using the FCoE protocol (EtherType 0x8906). Requires lossless Ethernet: PFC (Priority Flow Control) per-priority pausing, DCBx (Data Center Bridging Exchange) to negotiate lossless settings. FIP (FCoE Initialisation Protocol) handles FCF discovery and VF_Port login. NVMe-oF (NVMe over Fabrics) is the modern successor: NVMe commands transmitted over RDMA (RoCE v2 for Ethernet) or FC (FC-NVMe). Latency is dramatically lower than FCoE because the protocol stack is thinner.