Cisco Catalyst 4000 Series Switches

Document ID: 13414

Contents

Introduction
Prerequisites
      Requirements
      Components Used
      Conventions
      Background Information
Basic Configuration
      Catalyst Control Plane Protocols
      VLAN Trunking Protocol
      Extended VLAN and MAC Address Reduction
      Autonegotiation
      Gigabit Ethernet
      Dynamic Trunking Protocol
      Spanning Tree Protocol
      EtherChannel
      Unidirectional Link Detection
      Jumbo Frame
Management Configuration
      Network Diagrams
      In-Band Management
      Out-of-Band Management
      System Tests
      System and Hardware Error Detection
      EtherChannel/Link Errors Handling
      Catalyst 6500/6000 Packet Buffer Diagnostics
      System Logging
      Simple Network Management Protocol
      Remote Monitoring
      Network Time Protocol
      Cisco Discovery Protocol
Security Configuration
      Basic Security Features
      Terminal Access Controller Access Control System
Configuration Checklist
NetPro Discussion Forums - Featured Conversations
Related Information

Introduction

This document discusses the implementation of Cisco Catalyst series switches in your network, specifically the Catalyst 4500/4000, 5500/5000, and 6500/6000 platforms. Configurations and commands are discussed under the assumption that you are running Catalyst OS (CatOS) General Deployment software 6.4(3) or later. Although some design considerations are presented, this document does not cover overall campus design.

Prerequisites

Requirements

This document assumes familiarity with the Catalyst 6500 Series Command Reference, 7.6.

Although references to public online material for further reading are provided throughout the document, these are other foundational and educational references:

• Cisco ISP Essentials— Essential IOS Features Every ISP Should Consider.

• Cisco Network Monitoring and Event Correlation Guidelines

• Gigabit Campus Network Design—Principles and Architecture

• Cisco SAFE: A Security Blueprint for Enterprise Networks

Components Used

This document is not restricted to specific software and hardware versions.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Background Information

These solutions represent years of field experience from Cisco engineers working with many of our largest customers and complex networks. Consequently, this document emphasizes real-world configurations that make networks successful. This paper offers these solutions:

• Solutions that have statistically the broadest field exposure, and thus the lowest risk.

• Solutions that are simple, trading some flexibility for deterministic results.

• Solutions that are easy to manage and configured by network operations teams.

• Solutions that promote high availability and high stability.

This document is divided into these four sections:

• Basic Configuration— features used by a majority of networks such as Spanning Tree Protocol (STP) and trunking.

• Management Configuration— design considerations along with system and event monitoring using Simple Network Management Protocol (SNMP), Remote Monitoring (RMON), Syslog, Cisco Discovery Protocol (CDP), and Network Time Protocol (NTP).

• Security Configuration— passwords, port security, physical security, and authentication using TACACS+.

• Configuration Checklist— summary of suggested configuration templates.

Basic Configuration

Features deployed with the majority of Catalyst networks are discussed in this section.

Catalyst Control Plane Protocols

This section introduces the protocols that run between switches under normal operation. A basic understanding of these protocols is helpful in tackling each section.

Supervisor Traffic

Most features enabled in a Catalyst network require two or more switches to cooperate, so there must be a controlled exchange of keepalive messages, configuration parameters, and management changes. Whether these protocols are Cisco proprietary, like CDP, or standards-based, like IEEE 802.1d (STP), all have certain elements in common when implemented on the Catalyst series.

In basic frame forwarding, user data frames originate from end systems, and their source address and destination address are not changed throughout Layer 2 (L2) switched domains. Content Addressable Memory (CAM) lookup-tables on each switch Supervisor Engine are populated by a source address learning process and indicate which egress port must forward each frame received. If the address learning process is incomplete (the destination is unknown or the frame is destined to a broadcast or multicast address), it is forwarded (flooded) out all ports in that VLAN.

The switch must also recognize which frames are to be switched through the system and which must be directed to the switch CPU itself (also known as the Network Management Processor [NMP]).

The Catalyst control plane is created using special entries in the CAM table called system entries in order to receive and direct traffic to the NMP on an internal switch port. Thus, by using protocols with well-known destination MAC addresses, control plane traffic can be separated from the data traffic. Issue show CAM system command on a switch to confirm this, as shown:

>show cam system

* = Static Entry. + = Permanent Entry. # = System Entry. R = Router Entry.
X = Port Security Entry
VLAN  Dest MAC/Route Des    [CoS]  Destination Ports or VCs / [Protocol Type]
----  ------------------    -----  -------------------------------------------
1     00-d0-ff-88-cb-ff  #          1/3

!--- NMP internal port.

1     01-00-0c-cc-cc-cc  #          1/3

!--- CDP and so on.

1     01-00-0c-cc-cc-cd  #          1/3

!--- Cisco STP.

1     01-80-c2-00-00-00  #          1/3

!--- IEEE STP.

1     01-80-c2-00-00-01  #          1/3

!--- IEEE flow control.

1     00-03-6b-51-e1-82 R#          15/1

!--- Multilayer Switch Feature Card (MSFC) router.

...

Cisco has a reserved range of Ethernet MAC and protocol addresses, as shown. Each one is covered later in this document. However, a summary is presented in this table for convenience.

The majority of Cisco control protocols use an IEEE 802.3 SNAP encapsulation, including LLC 0xAAAA03, OUI 0x00000C, which can be seen on a LAN analyzer trace. Other common properties of these protocols include:

• These protocols assume point-to-point connectivity. Note that the deliberate use of multicast destination addresses enables two Catalysts to transparently communicate over non-Cisco switches, as devices that do not understand and intercept the frames simply flood them. However, point-to-multipoint connections through multi-vendor environments can result in inconsistent behavior and must generally be avoided.

• These protocols terminate at Layer 3 (L3) routers; they function only within a switch domain.

• These protocols receive prioritization over user data by ingress application-specific integrated circuit (ASIC) processing and scheduling.

After the introduction of the control protocol destination addresses, the source address must also be described for completeness. Switch protocols use a MAC address taken from a bank of available addresses provided by an EPROM on the chassis. Issue the show module command in order to display the address ranges available to each module when it sources traffic such as STP bridge protocol data units (BPDUs) or ISL frames.

>show module

...
Mod MAC-Address(es)                        Hw     Fw         Sw
--- -------------------------------------- ------ ---------- -----------------
1   00-01-c9-da-0c-1e to 00-01-c9-da-0c-1f 2.2    6.1(3)     6.1(1d)
    00-01-c9-da-0c-1c to 00-01-c9-da-0c-1
    00-d0-ff-88-c8-00 to 00-d0-ff-88-cb-ff

!--- MACs for sourcing traffic.

...
VLAN 1

VLAN 1

VLAN 1 has a special significance in Catalyst networks.

The Catalyst Supervisor Engine always uses the default VLAN, VLAN 1, to tag a number of control and management protocols when trunking, such as CDP, VTP and PAgP. All ports, including the internal sc0 interface, are configured by default to be members of VLAN 1. All trunks carry VLAN 1 by default, and in CatOS software versions earlier than 5.4, it was not possible to block user data in VLAN 1.

These definitions are needed in order to help clarify some well-used terms in Catalyst networking:

• The management VLAN is where sc0 resides; this VLAN can be changed.

• The native VLAN is defined as the VLAN to which a port returns when not trunking, and is the untagged VLAN on an 802.1Q trunk. By default, VLAN 1 is the native VLAN.

• In order to change the native VLAN, issue the set vlan vlan-id mod/port command.

  Note: Create the VLAN before you set it as the native VLAN of the trunk.

These are several good reasons to tune a network and alter the behavior of ports in VLAN 1:

• When the diameter of VLAN 1, like any other VLAN, gets large enough to be a risk to stability (particularly from an STP perspective) it needs to be pruned back. This is discussed in more detail in the In-Band Management section of this document.

• Control plane data on VLAN 1 must be kept separate from the user data in order to simplify troubleshooting and maximize available CPU cycles.

• L2 loops in VLAN 1 must be avoided when multilayer-campus networks are designed without STP, and trunking is still required to the access layer if there are multiple VLANs and IP subnets. To do this, manually clear VLAN 1 from trunk ports.

In summary, note this information about trunks:

• CDP, VTP, and PAgP updates are always forwarded on trunks with a VLAN 1 tag. This is the case even if VLAN 1 is cleared from the trunks and is not the native VLAN. If VLAN 1 is cleared for user data, these is no impact on control plane traffic that is still sent using VLAN 1.

• On an ISL trunk, DTP packets are sent on VLAN1. This is the case even if VLAN 1 is cleared from the trunk and is no longer the native VLAN. On an 802.1Q trunk, DTP packets are sent on the native VLAN. This is the case even if the native VLAN is cleared from the trunk.

• In PVST+, the 802.1Q IEEE BPDUs are forwarded untagged on the common Spanning Tree VLAN 1 for interoperability with other vendors, unless VLAN 1 is cleared from the trunk. This is the case regardless of the native VLAN configuration. Cisco PVST+ BPDUs are sent and tagged for all other VLANs. Refer to the Spanning Tree Protocol section in this document for more details.

• 802.1s Multiple Spanning Tree (MST) BPDUs are always sent on VLAN 1 on both ISL and 802.1Q trunks. This applies even when VLAN 1 is cleared from the trunks.

• Do not clear or disable VLAN 1 on trunks between MST bridges and PVST+ bridges. But, in the case that VLAN 1 is disabled, the MST bridge must become root in order for all VLANs to avoid the MST bridge putting its boundary ports in the root-inconsistent state. Refer to Understanding Multiple Spanning Tree Protocol (802.1s) for details.

Recommendations

In order to keep a VLAN in an up/up state with no clients or hosts connected in that VLAN , you need to have at least one physical device connected in that VLAN. Otherwise, the VLAN has an up/down state. Currently, there is no command to put a VLAN interface up/up when there are no active ports in the switch for that VLAN.

If you do not want to connect a device, connect a loopback plug in any port for that VLAN. As an alternative, try a crossover cable that connects two ports in that VLAN on the same switch. This method forces the port up. Refer to the Loopback Plug section of Loopback Tests for T1/56K Lines for more information.

When a network is multihomed to service providers, the network acts as a transit network between two service providers. If the VLAN number received in a packet needs to be translated or changed when passed from one service provider to another service provider, it is advisable to use the QinQ feature in order to translate the VLAN number.

VLAN Trunking Protocol

Before you create VLANs, determine the VTP mode to be used in the network. VTP enables VLAN configuration changes to be made centrally on one or more switches. Those changes automatically propagate to all other switches in the domain.

Operational Overview

VTP is a L2 messaging protocol that maintains VLAN configuration consistency. VTP manages the addition, deletion, and renaming of VLANs on a network-wide basis. VTP minimizes misconfigurations and configuration inconsistencies that can cause a number of problems, such as duplicate VLAN names, incorrect VLAN-type specifications, and security violations. The VLAN database is a binary file and is stored in NVRAM on VTP servers separately from the configuration file.

The VTP protocol communicates between switches using an Ethernet destination multicast MAC address (01-00-0c-cc-cc-cc) and SNAP HDLC protocol type Ox2003. It does not work over non-trunk ports (VTP is a payload of ISL or 802.1Q), so messages cannot be sent until DTP has brought the trunk online.

Message types include summary advertisements every five minutes, subset advertisements and request advertisements when there are changes, and joins when VTP pruning is enabled. The VTP configuration revision number is incremented by one with every change on a server, which then propagates the new table across the domain.

If a VLAN is deleted, ports that were once a member of that VLAN are placed in an inactive state. Similarly, if a switch in client mode is unable to receive the VTP VLAN table at boot-up (either from a VTP server or another VTP client), all ports in VLANs other than the default VLAN 1 are deactivated.

This table provides a feature comparison summary for various VTP modes:

In VTP transparent mode, VTP updates are ignored (the VTP multicast MAC address is removed from the system CAM that is normally used to pick up control frames and direct them to the supervisor engine). As the protocol uses a multicast address, a switch in transparent mode (or another vendor switch) simply floods the frame to other Cisco switches in the domain.

¹ CatOS software release 7.1 introduces the option to disable VTP with use of the off mode. In VTP off mode, the switch behaves in a way that is very similar to the VTP transparent mode, except that off mode also suppresses the forwarding of VTP updates.

This table provides a summary of the initial configuration:

VTP version 2 (VTPv2) includes this functional flexibility. However, it is not interoperable with VTP version 1 (VTPv1):

• Token Ring support

• Unrecognized VTP information support; switches now propagate values they cannot parse.

• Version-dependent transparent mode; transparent mode no longer checks domain name. This enables support of more than one domain across a transparent domain.

• Version number propagation; if VTPv2 is possible on all switches, all can be enabled through the configuration of a single switch.

Refer to Understanding and Configuring VLAN Trunk Protocol (VTP) for more information.

VTP Version 3

CatOS software release 8.1 introduces support for VTP version 3 (VTPv3). VTPv3 provides enhancements over the existing versions. These enhancements allow for:

• Support for extended VLANs

• Support for the creation and advertisement of private VLANs

• Support for VLAN instances and MST mapping propagation instances (which are supported in CatOS release 8.3)

• Improved server authentication

• Protection from accidental insertion of the "wrong" database into a VTP domain

• Interaction with VTPv1 and VTPv2

• The ability to be configured on a per-port basis

One of the major differences between VTPv3 implementation and the earlier version is the introduction of a VTP primary server. Ideally, there must be only one primary server in a VTPv3 domain, if the domain is not partitioned. Any changes that you make to the VTP domain must be executed on the VTP primary server in order to be propagated to the VTP domain. There can be multiple servers within a VTPv3 domain, which are also known as secondary servers. When a switch is configured to be a server, the switch becomes a secondary server by default. The secondary server can store the configuration of the domain but cannot modify the configuration. A secondary server can become the primary server with a successful takeover from the switch.

Switches that run VTPv3 only accept a VTP database with a higher revision number than the current primary server. This process differs significantly from VTPv1 and VTPv2, in which a switch always accepts a superior configuration from a neighbor in the same domain. This change with VTPv3 provides protection. A new switch that is introduced into the network with a higher VTP revision number cannot overwrite the VLAN configuration of the entire domain.

The VTPv3 also introduces an enhancement to how the VTP handles passwords. If you use the hidden password configuration option in order to configure a password as "hidden", these items occur:

• The password does not appear in plain text in the configuration. The secret hexadecimal format of the password is saved in the configuration.

• If you try to configure the switch as a primary server, you are prompted for the password. If your password matches the secret password, the switch becomes a primary server, which allows you to configure the domain.

Note: It is important to note that the primary server is only necessary when you need to modify the VTP configuration for any instance. A VTP domain can operate with no active primary server because the secondary servers ensure persistence of the configuration over reloads. The primary server state is exited for these reasons:

• A switch reload

• A high-availability switchover between the active and redundant supervisor engines

• A takeover from another server

• A change in the mode configuration

• Any VTP domain configuration change, such as a change in:

  • Version

  • Domain name

  • Domain password

VTPv3 also allows the switches to participate in multiple instances of VTP. In this case, the same switch can be the VTP server for one instance and a client for another instance because the VTP modes are specific to different VTP instances. For example, a switch can operate in transparent mode for an MST instance while the switch is configured in server mode for a VLAN instance.

In terms of interaction with VTPv1 and VTPv2, the default behavior in all versions of VTP has been that the earlier versions of VTP simply drop the new version updates. Unless the VTPv1 and VTPv2 switches are in transparent mode, all VTPv3 updates are dropped. On the other hand, after VTPv3 switches receive a legacy VTPv1 or VTPv2 frame on a trunk, the switches pass a scaled-down version of their database update to the VTPv1 and VTPv2 switches. However, this information exchange is unidirectional in that no updates from VTPv1 and VTPv2 switches are accepted by the VTPv3 switches. On trunk connections, VTPv3 switches continue to send out scaled-down updates as well as full-fledged VTPv3 updates in order to cater to the existence of VTPv2 and VTPv3 neighbors across the trunk ports.

In order to provide VTPv3 support for extended VLANs, the format of the VLAN database, in which the VTP assigns 70 bytes per VLAN, is changed. The change allows for the coding of non-default values only, instead of the carrying of unmodified fields for the legacy protocols. Because of this change, 4K VLAN support is the size of the resulting VLAN database.

Recommendation

There is no specific recommendation on whether to use VTP client/server modes or VTP transparent mode. Some customers prefer the ease of management of VTP client/server mode despite some considerations noted later. The recommendation is to have two server mode switches in each domain for redundancy, typically the two distribution-layer switches. The rest of the switches in the domain must be set to client mode. When you implement client/server mode with the use of VTPv2, be mindful that a higher revision number is always accepted in the same VTP domain. If a switch that is configured in either VTP client or server mode is introduced into the VTP domain and has a higher revision number than the existing VTP servers, this overwrites the VLAN database within the VTP domain. If the configuration change is unintentional and VLANs are deleted, the overwrite can cause a major outage in the network. In order to ensure that client or server switches always have a configuration revision number that is lower than that of the server, change the client VTP domain name to something other than the standard name. Then revert back to the standard. This action sets the configuration revision on the client to 0.

There are pros and cons to the VTP ability to make changes easily on a network. Many enterprises prefer the cautious approach of VTP transparent mode for these reasons:

• It encourages good change control practice, as the requirement in order to modify a VLAN on a switch or trunk port has to be considered one switch at a time.

• It limits the risk of an administrator error that impacts the entire domain, such as the deletion of a VLAN by accident..

• There is no risk that a new switch introduced into the network with a higher VTP revision number can overwrite the entire domain VLAN configuration.

• It encourages VLANs to be pruned from trunks running to switches that do not have ports in that VLAN. This makes frame flooding more bandwidth-efficient. Manual pruning is also beneficial because it reduces the spanning tree diameter (see the DTP section of this document). Before pruning unused VLANs on port channel trunks, ensure that any ports connected to IP phones are configured as access ports with voice VLAN.

• The extended VLAN range in CatOS 6.x and CatOS 7.x, numbers 1025 through 4094, can only be configured in this way. For more information, see the Extended VLAN and MAC Address Reduction section of this document.

• VTP transparent mode is supported in Campus Manager 3.1, part of Cisco Works 2000. The old restriction that required at least one server in a VTP domain has been removed.

Note: Specifying VLANs with the set command only adds VLANs, and does not clear them. For example, the set trunk x/y 1-10 command does not set the allowed list to just VLANs 1-10. Issue the clear trunk x/y 11-1005 command in order to achieve the desired result.

Although token ring switching is outside the scope of this document, note that VTP transparent mode is not recommended for TR-ISL networks. The basis for token ring switching is that the whole domain forms a single distributed multi-port bridge, so every switch must have the same VLAN information.

Other Options

VTPv2 is a requirement in token ring environments, where client/server mode is highly recommended.

VTPv3 provides the ability to implement tighter authentication and configuration revision control. VTPv3 essentially provides the same level of functionality, but with more enhanced security, as VTPv1/VTPv2 transparent mode offers. In addition, VTPv3 is partially compatible with the legacy VTP versions.

The benefits of pruning VLANs to reduce unnecessary frame flooding are advocated in this document. Theset vtp pruning enable command prunes VLANs automatically, which stops the inefficient flooding of frames where they are not needed. Unlike manual VLAN pruning, automatic pruning does not limit the Spanning Tree diameter.

From CatOS 5.1, the Catalyst switches can map 802.1Q VLAN numbers greater than 1000 to ISL VLAN numbers. In CatOS 6.x, Catalyst 6500/6000 switches support 4096 VLANs in accordance with the IEEE 802.1Q standard. These VLANs are organized into these three ranges, only some of which are propagated to other switches in the network with VTP:

• normal-range VLANs: 1–1001

• extended-range VLANs: 1025–4094 (can only be propagated by VTPv3)

• reserved-range VLANs: 0, 1002—1024, 4095

The IEEE has produced a standards-based architecture in order to accomplish similar results as VTP. As a member of the 802.1Q Generic Attribute Registration Protocol (GARP), the Generic VLAN Registration Protocol (GVRP) allows VLAN management interoperability between vendors, but is outside the scope of this document.

Note: CatOS 7.x introduces the option to set VTP to off mode, a mode very similar to transparent. However, the switch does not forward VTP frames. This can be useful in some designs when trunking to switches outside of your administrative control.

Extended VLAN and MAC Address Reduction

The MAC address reduction feature enables extended-range VLAN identification. The enablement of MAC address reduction disables the pool of MAC addresses that are used for the VLAN spanning tree and leaves a single MAC address. This MAC address identifies the switch. CatOS software release 6.1(1) introduces MAC address reduction support for Catalyst 6500/6000 and Catalyst 4500/4000 switches to support 4096 VLANs in compliance with the IEEE 802.1Q standard.

Operation Overview

Switch protocols use a MAC address that is taken from a bank of available addresses that an EPROM on the chassis provides as part of the bridge identifiers for VLANs that run under PVST+. Catalyst 6500/6000 and Catalyst 4500/4000 switches support either 1024 or 64 MAC addresses, which depends on the chassis type.

Catalyst switches with 1024 MAC addresses do not enable MAC address reduction by default. MAC addresses are allocated sequentially. The first MAC address in the range is assigned to VLAN 1. The second MAC address in the range is assigned to VLAN 2, and so on. This enables the switches to support 1024 VLANs with each VLAN using a unique bridge identifier.

¹ MAC address reduction is enabled by default for switches that have 64 MAC addresses, and the feature cannot be disabled.

For Catalyst series switches with 1024 MAC addresses, an enablement of MAC address reduction allows support of 4096 VLANs that run under PVST+ or 16 Multiple Instance STP (MISTP) instances to have unique identifiers without an increase in the number of MAC addresses that are required on the switch. MAC address reduction reduces the number of MAC addresses that are required by the STP from one per VLAN or MISTP instance to one per switch.

This figure shows that the bridge identifier MAC address reduction is not enabled. The bridge identifier consists of a 2-byte bridge priority and a 6-byte MAC address:

MAC address reduction modifies the STP bridge identifier portion of the BPDU. The original 2-byte priority field is split into two fields. This split results in a 4-bit bridge priority field and a 12-bit system ID extension that allows for VLAN numbering of 0 through 4095.

When you have MAC address reduction enabled on Catalyst switches in order to leverage extended range VLANs, enable MAC address reduction on all switches within the same STP domain. This step is necessary in order to keep the STP root calculations on all switches consistent. After you enable MAC address reduction, the root bridge priority becomes a multiple of 4096 plus the VLAN ID. The switches without MAC address reduction can claim root inadvertently because these switches have a finer granularity in the selection of the bridge ID.

Configuration Guidelines

You must follow certain guidelines when you configure extended VLAN range. The switch can allocate a block of VLANs from the extended range for internal purposes. For example, the switch can allocate the VLANs for the routed ports or Flex WAN modules. Allocation of the block of VLANs always starts from VLAN 1006 and goes up. If you have any VLANs within the range that the Flex WAN module requires, all the required VLANs are not allocated because the VLANs are never allocated from the user VLAN area. Issue the show vlan command or the show vlan summary command on a switch in order to display both the user-assigned and internal VLANs.

>show vlan summary

Current Internal Vlan Allocation Policy - Ascending

Vlan status    Count  Vlans
-------------  -----  ------------------------------------------
VTP Active         7   1,17,174,1002-1005

Internal           7   1006-1011,1016

!--- These are internal VLANs.

>show vlan
---- -------------------------------- --------- ------- --------

1    default                          active    7       4/1-48



!--- Output suppressed.


1006 Online Diagnostic Vlan1          active    0       internal
1007 Online Diagnostic Vlan2          active    0       internal
1008 Online Diagnostic Vlan3          active    0       internal
1009 Voice Internal Vlan              active    0       internal
1010 Dtp Vlan                         active    0       internal
1011 Private Vlan Internal Vlan       suspend   0       internal
1016 Online SP-RP Ping Vlan           active    0       internal

!--- These are internal VLANs.

Additionally, before you use the extended-range VLANs, you must delete any existing 802.1Q-to-ISL mappings. Also, in versions earlier than VTPv3, you must statically configure the extended VLAN on each switch with the use of VTP transparent mode. Refer to the Extended-Range VLAN Configuration Guidelines section of Configuring VLANs for more information.

Note: In software that is earlier than software release 8.1(1), you cannot configure the VLAN name for extended-range VLANs. This capability is independent of any VTP version or mode.

Recommendation

Try to maintain a consistent MAC address reduction configuration within the same STP domain. However, the enforcement of MAC address reduction on all network devices can be impractical when new chassis with 64 MAC addresses are introduced to the STP domain. MAC address reduction is enabled by default for switches that have 64 MAC addresses, and the feature cannot be disabled. Understand that, when two systems are configured with the same spanning-tree priority, the system without MAC address reduction has a better spanning-tree priority. Issue this command in order to enable or disable MAC address reduction:

set spantree macreduction enable | disable

Allocation of the internal VLANs is in ascending order and starts at VLAN 1006. Assign the user VLANs as close to VLAN 4094 as possible in order to avoid conflicts between the user VLANs and the internal VLANs. With Catalyst 6500 switches that run Cisco IOS® system software, you can configure the internal VLAN allocation in descending order. The Command-Line Interface (CLI) equivalent for CatOS software is not officially supported.

Autonegotiation

Ethernet/Fast Ethernet

Autonegotiation is an optional function of the IEEE Fast Ethernet (FE) standard (802.3u) that enables devices to automatically exchange information over a link about speed and duplex abilities. Autonegotiation operates at Layer 1 (L1), and targets access layer ports where transient users such as PCs connect to the network.

Operational Overview

The most common cause of performance issues on 10/100 Mbps Ethernet links occur when one port on the link operates at half-duplex while the other is at full-duplex. This occasionally happens when one or both ports on a link are reset and the autonegotiation process does not cause both link partners to have the same configuration. It also happens when administrators reconfigure one side of a link and forget to reconfigure the other side. The typical symptoms of this are increasing frame check sequence (FCS), cyclic redundancy check (CRC), alignment, or runt counters on the switch.

Autonegotiation is discussed in detail in these documents. These documents include explanations of how autonegotiation works and configuration options.

• Configuring and Troubleshooting Ethernet 10/100Mb Half/Full Duplex Auto-Negotiation

• Troubleshooting Cisco Catalyst Switches to Network Interface Card (NIC) Compatibility Issues

A common misconception about autonegotiation is that it is possible to manually configure one link partner for 100 Mbps full-duplex and autonegotiate to full-duplex with the other link partner. In fact, an attempt to do this results in a duplex mismatch. This is a consequence of one link partner autonegotiating, not seeing any autonegotiation parameters from the other link partner, and defaulting to half-duplex.

Most Catalyst Ethernet modules support 10/100 Mbps and half/full-duplex, but the show port capabilities mod/port command confirms this.

FEFI

Far end fault indication (FEFI) protects 100BASE-FX (fiber) and Gigabit interfaces, while autonegotiation protects 100BASE-TX (copper) against physical-layer/signaling related faults.

A far end fault is an error in the link that one station can detect while the other cannot, such as a disconnected TX-wire. In this example, the sending station could still receive valid data and detect that the link is good through the link-integrity-monitor. It does not detect that its transmission is not being received by the other station. A 100BASE-FX station that detects such a remote fault can modify its transmitted IDLE stream to send a special bit-pattern (referred to as the FEFI IDLE pattern) to inform the neighbor of the remote fault; the FEFI-IDLE pattern subsequently triggers a shutdown of the remote port (errdisable). Refer to the UDLD section of this document for more information on fault protection.

FEFI is supported by this hardware and these modules:

• Catalyst 5500/5000: WS-X5201R, WS-X5305, WS-X5236, WS-X5237, WS-U5538, and WS-U5539

• Catalyst 6500/6000 and 4500/4000: All 100BASE-FX modules and GE modules

Recommendation

Whether to configure autonegotiation on 10/100 links or to hard code speed and duplex ultimately depends on the type of link partner or end device you have connected to a Catalyst switch port. Autonegotiation between end devices and Catalyst switches generally works well, and Catalyst switches are compliant with the IEEE 802.3u specification. However, problems can result when NIC or vendor switches do not conform exactly. Hardware incompatibility and other issues can also exist as a result of vendor-specific advanced features, such as auto-polarity or cabling integrity, that are not described in the IEEE 802.3u specification for 10/100 Mbps autonegotiation. Refer to Field Notice: Performance Issue with Intel Pro/1000T NICs connecting to CAT4K/6K for an example of this.

Anticipate that there will be some situations that require host, port speed, and duplex to be set. In general, follow these basic troubleshooting steps:

• Make sure that either autonegotiation is configured on both sides of the link or hard coding is configured on both sides.

• Check the CatOS release notes for common caveats.

• Verify the version of NIC driver or operating system you are running, as the latest driver or patch is often required.

As a rule, try to use autonegotiation first for any type of link partner. There are obvious benefits to configuring autonegotiation for transient devices like laptops. Ideally, autonegotiation also works well with non-transient devices such as servers and fixed workstations or from switch-to-switch and switch-to-router. For some of the reasons mentioned, negotiation issues can arise. In these cases, follow the basic troubleshooting steps outlined in the TAC links provided.

If the port speed is set to auto on a 10/100 Mbps Ethernet port, both speed and duplex are autonegotiated. Issue this command in order to set the port to auto:

set port speed port range auto

!--- This is the default.

If hard coding the port, issue these configuration commands:

set port speed port range 10 | 100 
set port duplex port range full | half

In CatOS 8.3 and later, Cisco has introduced the optional auto-10-100 keyword. Use the auto-10-100 keyword on ports that support speeds of 10/100/1000 Mbps but where autonegotiation to 1000 Mbps is undesirable. Use of the auto-10-100 keyword makes the port behave in the same way as a 10/100-Mbps port that has the speed set to auto. The speed and duplex are negotiated for 10/100-Mbps ports only, and the 1000-Mbps speed does not take part in the negotiation.

set port speed port_range auto-10-100

Other Options

When no autonegotiation is used between switches, L1 fault indication can also be lost for certain problems. It is helpful to use L2 protocols to augment failure detection, such as aggressive UDLD.

Gigabit Ethernet

Gigabit Ethernet (GE) has an autonegotiation procedure (IEEE 802.3z) that is more extensive than that for 10/100 Mbps Ethernet and is used to exchange flow-control parameters, remote fault information, and duplex information (even though Catalyst series GE ports only support full-duplex mode).

Note: 802.3z has been superseded by IEEE 802.3:2000 specs. Refer to IEEE Standards On Line LAN/MAN Standards Subscription: Archives for more information.

Operational Overview

GE port negotiation is enabled by default, and the ports on both ends of a GE link must have the same setting. Unlike FE, the GE link does not come up if the autonegotiation setting differs on the ports at each end of the link. However, the only condition that is required for an autonegotiation-disabled port to link up is a valid Gigabit signal from the far end. This behavior is independent of the autonegotiation configuration of the far end. For example, assume that there are two devices, A and B. Each device can have autonegotiation enabled or disabled. This table is a list of possible configurations and respective link states:

In GE, synchronization and autonegotiation (if they are enabled) are performed upon link startup through the use of a special sequence of reserved link code words.

Note: There is a dictionary of valid words and not all possible words are valid in GE.

The life of a GE connection can be characterized in this way:

A loss of synchronization means that the MAC detects a link down. Loss of synchronization applies whether autonegotiation is enabled or disabled. Synchronization is lost under certain failed conditions, such as the receipt of three invalid words in succession. If this condition persists for 10 ms, a "sync fail" condition is asserted and the link is changed to the link_down state. After synchronization is lost, another three consecutive valid idles are necessary in order to resynchronize. Other catastrophic events, such as a loss of receive (Rx) signal, causes a link-down event.

Autonegotiation is a part of the linkup process. When the link is up, autonegotiation is over. However, the switch still monitors the status of the link. If autonegotiation is disabled on a port, the "autoneg" phase is no longer an option.

The GE copper specification (1000BASE-T) does support autonegotiation through a Next Page Exchange. Next Page Exchange allows autonegotiation for 10/100/1000-Mbps speeds on copper ports.

Note: The GE fiber specification only makes provisions for the negotiation of duplex, flow control, and remote fault detection. GE fiber ports do not negotiate port speed. Refer to sections 28 and 37 of the IEEE 802.3-2002 specification for more information on autonegotiation.

Synchronization restart delay is a software feature that controls the total autonegotiation time. If autonegotiation is not successful within this time, the firmware restarts autonegotiation in case there is a deadlock. The set port sync-restart-delay command only has an effect when autonegotiation is set to enable.

Recommendation

Enabling autonegotiation is much more critical in a GE environment than in a 10/100 environment. In fact, autonegotiation must only be disabled on switch ports that attach to devices not capable of supporting negotiation or where connectivity issues arise from interoperability issues. Cisco recommends that Gigabit negotiation be enabled (default) on all switch-to-switch links and generally all GE devices. Issue this command in order to enable autonegotiation:

set port negotiation port range enable

!--- This is the default.

One known exception is when there is a connection to a Gigabit Switch Router (GSR) running Cisco IOS Software earlier than release 12.0(10)S, the release that added flow control and autonegotiation. In this case, turn off those two features, or the switch port reports not connected, and the GSR reports errors. This is a sample command sequence:

set port flowcontrol receive port range off
set port flowcontrol send port range off
set port negotiation port range disable

Switch-to-server connections must be looked at on a case-by-case basis. Cisco customers have encountered issues with Gigabit negotiation on Sun, HP, and IBM servers.

Other Options

Flow control is an optional part of the 802.3x specification and must be negotiated if used. Devices can or cannot be capable of sending and/or responding to a PAUSE frame (well known MAC 01-80-C2-00-00-00 0F). Also, they can not agree to the flow-control request of the far-end neighbor. A port with an input buffer that is filling up sends a PAUSE frame to its link partner, which stops the transmission, and holds any additional frames in the link partner output buffers. This does not solve any steady-state over-subscription problem, but effectively makes the input buffer larger by some fraction of the partner output buffer during bursts.

This feature is best used on links between access-ports and end hosts, where the host output buffer is potentially as large as their virtual memory. Switch-to-switch use has limited benefits.

Issue these commands in order to control this on the switch ports:

set port flowcontrol mod/port receive | send off |on | desired


>show port flowcontrol

 Port  Send FlowControl    Receive FlowControl   RxPause TxPause
       admin    oper       admin    oper
-----  -------- --------   -------- --------     ------- -------
 6/1   off      off        on       on           0       0
 6/2   off      off        on       on           0       0
 6/3   off      off        on       on           0       0

Note: All Catalyst modules respond to a PAUSE frame if negotiated. Some modules (for example, WS-X5410, WS-X4306) never send PAUSE frames even if they negotiate to do so, as they are non-blocking.

Dynamic Trunking Protocol

Encapsulation Type

Trunks extend VLANs between devices by temporarily identifying and tagging (link-local) the original Ethernet frames, thus they enable them to be multiplexed over a single link. This also ensures the separate VLAN broadcast and security domains are maintained between switches. CAM tables maintain the frame-to-VLAN mapping inside the switches.

Trunking is supported on several types of L2 media, including ATM LANE, FDDI 802.10, and Ethernet, although only the latter is be presented here.

ISL Operational Overview

Cisco proprietary identification or tagging scheme, ISL, has been in use for many years. The 802.1Q IEEE standard is also available.

By totally encapsulating the original frame in a two-level tagging scheme, ISL is effectively a tunneling protocol and has the additional benefit of carrying non-Ethernet frames. It adds a 26-byte header and 4-byte FCS to the standard Ethernet frame - the larger Ethernet frames are expected and handled by ports configured to be trunks. ISL supports 1024 VLANs.

ISL Frame Format

Refer to InterSwitch Link and IEEE 802.1Q Frame Format for more information.

802.1Q Operational Overview

The IEEE 802.1Q standard specifies much more than encapsulation types, including Spanning Tree enhancements, GARP (see the VTP section of this document), and 802.1p Quality of Service (QoS) tagging.

The 802.1Q frame format preserves the original Ethernet source address and destination address, yet switches must now expect baby-giant frames to be received, even on access ports where hosts can use tagging in order to express 802.1p user priority for QoS signaling. The tag is 4 bytes, so 802.1Q Ethernet v2 frames are 1522 bytes, an IEEE 802.3ac working group achievement. 802.1Q also supports numbering space for 4096 VLANs.

All data frames transmitted and received are 802.1Q-tagged except for those on the native VLAN (there is an implicit tag based on the ingress switch port configuration). Frames on the native VLAN are always transmitted untagged and normally received untagged. However, they can also be received tagged.

Refer to VLAN Standardization via IEEE 802.10 and Get IEEE 802 for more details.

802.1Q/801.1p Frame Format

Recommendation

As all newer hardware supports 802.1Q (and some only supports 802.1Q, such as the Catalyst 4500/4000 series and CSS 11000), Cisco recommends that all new implementations follow the IEEE 802.1Q standard and older networks gradually migrate from ISL.

The IEEE standard allows vendor interoperability. This is advantageous in all Cisco environments as new host 802.1p capable NICs and devices become available. Although both ISL and 802.1Q implementations are mature, the IEEE standard will ultimately have greater field exposure and greater third party support, such as network analyzer support. The lower encapsulation overhead of 802.1Q compared to ISL is a minor point in favor of 802.1Q as well.

As the encapsulation type is negotiated between switches using DTP, with ISL chosen as the winner by default if both ends support it, it is necessary to issue this command in order to specify dot1q:

set trunk mod/port mode dot1q

If VLAN 1 is cleared from a trunk, as discussed in the In-Band Management section of this document, although no user data is transmitted or received, the NMP continues to pass control protocols such as CDP and VTP on VLAN 1.

Also, as discussed in the VLAN 1 section of this document, CDP, VTP, and PAgP packets are always sent on VLAN 1 when trunking. When using dot1q encapsulation, these control frames are tagged with VLAN 1 if the native VLAN of the switch is changed. If dot1q trunking to a router is enabled and the native VLAN is changed on the switch, a sub-interface in VLAN 1 is needed to receive the tagged CDP frames and provide CDP neighbor visibility on the router.

Note: There is a potential security consideration with dot1q caused by the implicit tagging of the native VLAN, as it can be possible to send frames from one VLAN to another without a router. Refer to Are there Vulnerabilities in VLAN Implementations? for further details. The workaround is to use a VLAN ID for the native VLAN of the trunk that is not used for end user access. The majority of Cisco customers leave VLAN 1 as the native VLAN on a trunk and assign access ports to VLANs other than VLAN 1 in order to achieve this simply.

Trunking Mode

DTP is the second generation of Dynamic ISL (DISL), and exists in order to ensure that the different parameters involved in sending ISL or 802.1Q frames, such as the configured encapsulation type, native VLAN, and hardware capability, are agreed upon by the switches at either end of a trunk. This also helps protect against non-trunk ports flooding tagged frames, a potentially serious security risk, by ensuring that ports and their neighbors are in consistent states.

Operational Overview

DTP is a L2 protocol that negotiates configuration parameters between a switch port and its neighbor. It uses another multicast MAC address (01-00-0c-cc-cc-cc) and a SNAP protocol type of 0x2004. This table is a summary of the configuration modes:

These are some highlights of the protocol:

• DTP assumes a point-to-point connection, and Cisco devices only support 802.1Q trunk ports that are point-to-point.

• During DTP negotiation, the ports do not participate in STP. Only after the port becomes one of the three DTP types (access, ISL, or 802.1Q) does the port be added to STP. Otherwise PAgP, if configured, is the next process to run before the port participates in STP.

• If the port is trunking in ISL mode, DTP packets are sent out on VLAN 1, otherwise (for 802.1Q trunking or non-trunking ports) they are sent out on the native VLAN.

• In desirable mode, DTP packets transfer theVTP domain name (which must match for a negotiated trunk to come up), plus trunk configuration and admin status.

• Messages are sent every second during negotiation, and every 30 seconds after that.

• Be sure to understand that modes on, nonegotiate, and off explicitly specify in which state the port ends up. A bad configuration can lead to a dangerous/inconsistent state where one side is trunking and the other is not.

• A port in on, auto, or desirable mode sends DTP frames periodically. If a port in auto or desirable mode does not see a DTP packet in five minutes, it is set to non-trunk.

Refer to Configuring ISL Trunking on Catalyst 5500/5000 and 6500/6000 Family Switches for more ISL details. Refer to Trunking Between Catalyst 4500/4000, 5500/5000, and 6500/6000 Series Switches Using 802.1Q Encapsulation with Cisco CatOS System Software for more 802.1Q details.

Recommendation

Cisco recommends an explicit trunk configuration of desirable at both ends. In this mode, network operators can trust syslog and command line status messages that a port is up and trunking, unlike on mode, which can make a port appear up even though the neighbor is misconfigured. In addition, desirable mode trunk provides stability in situations where one side of the link cannot become a trunk or drops trunk state. Issue this command in order to set desirable mode:

set trunk mod/port desirable ISL | dot1q

Note: Set trunk to off on all non-trunk ports. This helps eliminate wasted negotiation time when bringing host ports up. This command is also executed when the set port host command is used; refer to the STP section for more information. Issue this command in order to disable a trunk on a range of ports:

set trunk port range off

!--- Ports are not trunking; part of the set port host command.

Other Options

Another common customer configuration uses desirable mode only at the distribution layer and the simplest default configuration (auto mode) at the access layer.

Some switches, such as a Catalyst 2900XL, Cisco IOS routers, or other vendor devices, do not currently support trunk negotiation through DTP. You can use nonegotiate mode on Catalyst 4500/4000, 5500/5000, and 6500/6000 switches in order to set a port to trunk unconditionally with these devices, which can help standardize on a common setting across the campus. Also, you can implement nonegotiate mode in order to reduce the "overall" link initialization time.

Note: Factors such as the channel mode and STP configuration can also affect the initialization time.

Issue this command in order to set nonegotiate mode:

set trunk mod/port nonegotiate ISL | dot1q

Cisco recommends nonegotiate when there ia a connection to a Cisco IOS router because when bridging is performed, some DTP frames received from on mode can get back into the trunk port. Upon reception of the DTP frame, the switch port tries to renegotiate (orbring the trunk down and up) unnecessarily. If nonegotiate is enabled, the switch does not send DTP frames.

Spanning Tree Protocol

Basic Considerations

Spanning Tree Protocol (STP) maintains a loop-free L2 environment in redundant switched and bridged networks. Without STP, frames loop and/or multiply indefinitely, which causes a network meltdown as all devices in the broadcast domain are interrupted continuously by high traffic.

Although in some respects STP is a mature protocol initially developed for slow software-based bridge specifications (IEEE 802.1d), it can be complex to implement well in large switched networks with many VLANs, many switches in a domain, multi-vendor support, and newer IEEE enhancements.

For future reference, CatOS 6.x continues to take on new STP development, such as MISTP, loop-guard, root-guards, and BPDU arrival time skew detection. In addition, further standardized protocols are available in CatOS 7.x, such as IEEE 802.1s shared Spanning Tree and IEEE 802.1w rapid convergence Spanning Tree.

Operational Overview

The root bridge election per VLAN is won by the switch with the lowest root Bridge Identifier (BID). The BID is the bridge priority combined with the switch MAC address.

Initially, BPDUs are sent from all switches, containing the BID of each switch and the path cost to reach that switch. This enables the root bridge and the lowest-cost path to the root to be determined. Additional configuration parameters carried in BPDUs from the root override those that are locally configured so that the whole network uses consistent timers.

The topology then converges through these steps:

1. A single root bridge is elected for the entire Spanning Tree domain.

2. One root port (facing the root bridge) is elected on every non-root bridge.

3. A designated port is elected for BPDU forwarding on every segment.

4. Non-designated ports become blocking.

Refer to Configuring Spanning Tree for more information.

The two types of BPDUs in STP are configuration BPDUs and Topology Change Notification (TCN) BPDUs.

Configuration BPDU Flow

Configuration BPDUs are sourced every hello-interval from every port on the root bridge and subsequently flow to all leaf switches in order to maintain the state of the Spanning Tree. In steady state, the BPDU flow is unidirectional: root ports and blocking ports only receive configuration BPDUs, while designated ports only send configuration BPDUs.

For every BPDU received by a switch from the root, a new one is processed by the Catalyst central NMP and sent out containing the root information. In other words, if the root bridge is lost or all paths to the root bridge are lost, BPDUs stop being received (until the maxage timer starts re-election).

TCN BPDU Flow

TCN BPDUs are sourced from leaf switches and flow towards the root bridge when a topology change is detected in the spanning tree. Root ports only send TCNs, and designated ports only receive TCNs.

The TCN BPDU travels toward the root ridge and is acknowledged at each step, so this is a reliable mechanism. Once it arrives at the root bridge, the root bridge alerts the entire domain that a change has occurred by sourcing Configuration BPDUs with the TCN flag set for maxage + fwddelay time (35 seconds by default). This causes all switches to change their normal CAM aging time from five minutes (by default) to the interval specified by fwddelay (15 seconds by default). Refer to Understanding Spanning Tree Protocol Topology Changes for more details.

Spanning Tree Modes

There are three different ways to correlate VLANs with Spanning Tree:

• A single Spanning Tree for all VLANs, or mono Spanning Tree Protocol, such as IEEE 802.1Q

• A Spanning Tree per VLAN, or shared Spanning Tree, such as Cisco PVST

• A Spanning Tree per set of VLANs, or multiple Spanning Tree, such as Cisco MISTP and IEEE 802.1s

A mono Spanning Tree for all VLANs allows only one active topology and therefore no load balancing. An STP blocked port blocks for all VLANs and carries no data.

One Spanning Tree per VLAN allows load balancing but requires more BPDU CPU processing as the number of VLANs increases. The CatOS release notes provide guidance on the number of logical ports recommended in the Spanning Tree per switch. For example, the Catalyst 6500/6000 Supervisor Engine 1 formula is as such:

number of ports + (number of trunks * number of VLANs on trunks) < 4000

Cisco MISTP and the new 802.1s standard allow the definition of only two active STP instances/topologies, and the mapping of all VLANs to either of these two trees. This technique allows STP to scale to many thousands of VLANs while load balancing is enabled.

BPDU Formats

In order to support the IEEE 802.1Q standard, the existing Cisco STP implementation was extended to become PVST+ by adding support for tunneling across an IEEE 802.1Q mono Spanning Tree region. PVST+ is therefore compatible with both IEEE 802.1Q MST and Cisco PVST protocols and does not require extra commands or configuration. In addition, PVST+ adds verification mechanisms in order to ensure that there is no configuration inconsistency of port trunking and VLAN IDs across switches.

These are some operational highlights of the PVST+ protocol:

• PVST+ interoperates with 802.1Q mono Spanning Tree through the so-called Common Spanning Tree (CST) over an 802.1Q trunk. The CST is always on VLAN 1, so this VLAN needs to be enabled on the trunk to interoperate with other vendors. CST BPDUs are transmitted, always untagged, to the IEEE Standard Bridge-Group (MAC Address 01-80-c2-00-00-00, DSAP 42, SSAP 42). For completeness of description, a parallel set of BPDUs are also transmitted to the Cisco shared Spanning Tree MAC address for VLAN 1.

• PVST+ tunnels PVST BPDUs across 802.1Q VLAN regions as multicast data. Cisco shared Spanning Tree BPDUs are transmitted to MAC address 01-00-0c-cc-cc-cd (SNAP HDLC protocol type 0x010b) for each VLAN on a trunk. BPDUs are untagged on the native VLAN and tagged for all other VLANs.

• PVST+ checks port and VLAN inconsistencies. PVST+ blocks those ports that receive inconsistent BPDUs in order to prevent forwarding loops. It also notifies users through syslog messages about any configuration mismatch.

• PVST+ is backward-compatible with existing Cisco switches running PVST on ISL trunks. ISL-encapsulated BPDUs are still transmitted or received using the IEEE MAC address. In other words, each BPDU type is link-local; there are no translation issues.

Recommendation

All Catalyst switches have STP enabled by default. This is recommended even if a design is chosen that does not include L2 loops so that STP is not enabled in the sense that it is actively maintaining a blocked port.

set spantree enable all

!--- This is the default.

Cisco recommends that STP is left enabled for these reasons:

• If there is a loop (induced by mispatching, bad cable, and so on.), STP prevents detrimental effects to the network caused by multicast and broadcast data.

• Protection against an EtherChannel breaking down.

• Most networks are configured with STP, which gives it maximum field exposure. More exposure generally equates to stable code.

• Protection against dual attached NICs misbehaving (or bridging enabled on servers).

• The software for many protocols (such as PAgP, IGMP snooping, and trunking) is closely related to STP. Running without STP can lead to undesirable results.

Do not change timers, as this can adversely affect stability. The majority of networks deployed are not tuned. The simple STP timers accessible through the command line, such as hello-interval and Maxage, are themselves comprised of a complex set of other assumed and intrinsic timers, so it is difficult to tune timers and consider all the ramifications. Moreover, there is the danger of undermining UDLD protection.

Ideally, keep user traffic off the management VLAN. Especially with older Catalyst switch processors, it is best to avoid problems with STP by keeping the management VLAN separate from user data. One end station that misbehaves could potentially keep the supervisor engine processor so busy with broadcast packets that it can miss one or more BPDUs. However, newer switches with more powerful CPUs and throttling controls relieve this consideration.. See the In-Band Management section of this document for more details.

Do not over-design redundancy. This can lead to a troubleshooting nightmare - too many blocking ports adversely affect long-term stability. Keep the total SPT diameter under seven hops. Try to design to the Cisco multilayer model, with its smaller switched domains, STP triangles, and deterministic blocked ports (as explained in Gigabit Campus Network Design—Principles and Architecture) wherever possible.

Influence and know where Root functionality and blocked ports reside, and document them on the topology diagram. The blocked ports are where STP troubleshooting begins - what made them change from blocking to forwarding is often the key part of root cause analysis. Choose the distribution and core layers as the location of root/secondary Root, since these are considered the most stable parts of the network. Check for optimal L3 and HSRP overlay with L2 data-forwarding paths. This command is a macro that configures the bridge priority; root sets it much lower than the default (32768), while root secondary sets it reasonably lower than the default:

set spantree root secondary vlan range

Note: This macro sets the root priority to be either 8192 (by default), the current root priority minus 1 (if another root bridge is known), or the current root priority (if its MAC address is lower then the current root).

Prune unnecessary VLANs off trunk-ports (a bidirectional exercise). This limits the diameter of STP and NMP processing overhead on portions of the network where certain VLANs are not required. VTP automatic pruning does not remove STP from a trunk. Refer to the VTP section of this document for more information. The default VLAN 1 can also be removed from trunks using CatOS 5.4 and later.

Refer to Spanning Tree Protocol Problems and Related Design Considerations for additional information.

Other Options

Cisco has another STP known asVLAN-bridge. This protocol operates using a destination MAC address of 01-00-0c-cd-cd-ce and protocol type of 0x010c.

This is most useful if there is a need to bridge non-routable or legacy protocols between VLANs without interfering with the IEEE Spanning Tree instance(s) running on those VLANs. If VLAN interfaces for non-bridged traffic become blocked for L2 traffic (and this could easily happen if they participated in the same STP as IP VLANs), the overlaying L3 traffic gets inadvertently pruned off as well - an unwanted side-effect. VLAN-bridge is therefore a separate instance of STP for bridged protocols, which provides a separate topology that can be manipulated without affecting IP traffic.

The Cisco recommendation is to run VLAN-bridge if bridging is required between VLANs on Cisco routers such as the MSFC.

PortFast

PortFast is used to bypass normal Spanning Tree operation on access ports to speed up connectivity between end-stations and the services they need to connect to after link initialization. On some protocols, such as IPX/SPX, it is important to see the access port in forwarding mode immediately after the link state has gone up in order to avoid GNS problems.

Refer to Using Portfast and Other Commands to Fix Workstation Startup Connectivity Delays for more information.

Operational Overview

PortFast skips the normal listening and learning states of STP by moving a port directly from blocking to forwarding mode after the link is known to be running. If this feature is not enabled, STP discards all user data until it decides that the port is ready to be moved to forwarding mode. This could take up to twice the ForwardDelay time (a total of 30 seconds by default).

PortFast mode also prevents an STP TCN from being generated each time a port state changes from learning to forwarding. TCNs are not a problem by themselves, but if a wave of TCNs hit the root bridge (typically in the morning when people turn on their PCs), it could extend convergence time unnecessarily.

STP PortFast is particularly important in both multicast CGMP and Catalyst 5500/5000 MLS networks. TCNs in these environments can cause the static CGMP CAM table entries to be aged out, which results in multicast packet loss until the next IGMP report, and/or flush MLS cache entries that then need to be rebuilt and could result in a router CPU spike, depending on the size of the cache. (Catalyst 6500/6000 MLS implementations and multicast entries learned from IGMP snooping are not affected.)

Recommendation

Cisco recommends that STP PortFast be enabled for all active host ports and disabled for switch-switch links and ports not in use.

Trunking and channeling must also be disabled for all host ports. Each access port is enabled by default for trunking and channeling, yet switch neighbors are not expected by design on host ports. If these protocols are left to negotiate, the subsequent delay in port activation can lead to undesirable situations in which initial packets from workstations, such as DHCP requests, are not forwarded.

CatOS 5.2 introduced a macro command, set port host port range that implements this configuration for access ports and helps autonegotiation and connection performance significantly:

set port host port range


!--- Macro command for these commands:

set spantree portfast port range enable
set trunk port range off
set port channel port range mode off

Note: PortFast does not mean that Spanning Tree is not run at all on those ports. BPDUs are still sent, received, and processed.

Other Options

PortFast BPDU-guard provides a way to prevent loops by moving a non-trunking port into an errdisable state when a BPDU is received on that port.

A BPDU packet must never be received on an access port configured for PortFast, since host ports must not be attached to switches. If a BPDU is observed, it indicates an invalid and possibly dangerous configuration that needs administrative action. When the BPDU-guard feature is enabled, Spanning Tree shuts down PortFast-configured interfaces that receive BPDUs instead of putting them into the STP blocking state.

The command works on a per-switch basis, not per-port, as shown:

set spantree portfast bpdu-guard enable

The network manager is notified by an SNMP trap or syslog message if the port goes down. It is also possible to configure an automatic recovery time for errdisabled ports. Refer to the UDLD section of this document for more details. For more information, refer to Spanning Tree Portfast BPDU Guard Enhancement.

Note: PortFast for trunk ports was introduced in CatOS 7.x and has no effect on trunk ports in earlier releases. PortFast for trunk ports is designed to increase convergence times for L3 networks. To complement this feature, CatOS 7.x also introduced the possibility of the configuration of PortFast BPDU-guard on a per-port basis.

UplinkFast

UplinkFast provides fast STP convergence after a direct link failure in the network access layer. It does not modify STP, and its purpose is to speed up convergence time in a specific circumstance to less than three seconds, rather than the typical 30-second delay. Refer to Understanding and Configuring the Cisco Uplink Fast Feature for more information.

Operational Overview

Using the Cisco multilayer design model at the access layer, if the forwarding uplink is lost, the blocking uplink is immediately moved to a forwarding state without waiting for listening and learning states.

An uplink group is a set of ports per VLAN that can be thought of as a root port and backup root port. Under normal conditions, the root port(s) are assuring connectivity from the access toward the root. If this primary root-connection fails for any reason, the backup root link kicks in immediately without having to go through typical 30 seconds of convergence delay.

Because this effectively bypasses the normal STP topology change-handling process (listening and learning), an alternate topology correction mechanism is needed in order to update switches in the domain that local end stations are reachable through an alternate path. The access layer switch running UplinkFast also generates frames for each MAC address in its CAM to a multicast MAC address (01-00-0c-cd-cd-cd, HDLC protocol 0x200a) to update the CAM table in all switches in the domain with the new topology.

Recommendation

Cisco recommends that UplinkFast be enabled for switches with blocked ports, typically at the access layer. Do not use on switches without the implied topology knowledge of a backup root link - typically distribution and core switches in the Cisco multilayer design. It can be added without disruption to a production network. Issue this command in order to enable UplinkFast:

set spantree uplinkfast enable 

This command also sets the bridge priority high in order to minimize the risk of this becoming a root bridge and the port priority high to minimize becoming a designated port, which breaks the functionality. When you restore a switch that had UplinkFast enabled, the feature has to be disabled, the uplink database cleared with "clear uplink," and the bridge priorities restored manually.

Note: The all protocols keyword for the UplinkFast command is needed when the protocol filtering feature is enabled. As the CAM records the protocol type as well as MAC and VLAN information when protocol filtering is enabled, an UplinkFast frame needs to be generated for each protocol on each MAC address. The rate keyword indicates the packets per second of the uplinkfast topology update frames. The default is recommended. You do not need to configure BackboneFast with Rapid STP (RSTP) or IEEE 802.1w because the mechanism is natively included and automatically enabled in RSTP.

BackboneFast

BackboneFast provides rapid convergence from indirect link failures. With the added functionality to STP, convergence times can typically be reduced from the default of 50 seconds to 30 seconds.

Operational Overview

The mechanism is initiated when a root port or blocked port on a switch receives inferior BPDUs from its designated bridge. This can happen when a downstream switch has lost its connection to the root and starts to send its own BPDUs in order to elect a new root. An inferior BPDU identifies a switch as both the root bridge and the designated bridge.

Under normal Spanning Tree rules, the receiving switch ignores inferior BPDUs for the configured maximum aging time, 20 seconds by default. However, with BackboneFast, the switch sees the inferior BPDU as a signal that the topology could have changed, and tries to determine whether it has an alternate path to the root bridge using Root Link Query (RLQ) BPDUs. This protocol addition allows a switch to check whether the root is still available, moves a blocked port to forwarding in less time, and notifies the isolated switch that sent the inferior BPDU that the root is still there.

These are some highlights of the protocol operation:

• A switch transmits the RLQ packet out the root port only (that is, towards the root bridge).

• A switch that receives a RLQ can reply either if it is the root switch, or if it knows it has lost connection with the root. If it does not know these facts, it must forward the query out its root port.

• If a switch has lost connection to the root, it must reply in the negative to this query.

• The reply must be sent out only the port from which the query came.

• The root switch must always respond to this query with a positive reply.

• If the reply is received on a non-root port, it is discarded.

STP convergence times can therefore be reduced by up to 20 seconds, as maxage does not need to expire.

Refer to Understanding and Configuring Backbone Fast on Catalyst Switches for more information.

Recommendation

The Cisco recommendation is to enable BackboneFast on all switches running STP. It can be added without disruption to a production network. Issue this command in order to enable BackboneFast:

set spantree backbonefast enable

Note: This global level command needs to be configured on all switches in a domain as it adds functionality to the STP protocol that all switches need to understand.

Other Options

BackboneFast is not supported on 2900XLs and 3500s. It must not be enabled if the switch domain contains these switches in addition to Catalyst 4500/4000, 5500/5000, and 6500/6000 switches.

You do not need to configure BackboneFast with RSTP or IEEE 802.1w because the mechanism is natively included and automatically enabled in RSTP.

Spanning Tree Loop Guard

Loop guard is a Cisco proprietary optimization for STP. Loop guard protects L2 networks from loops that are caused by:

• Network interfaces that malfunction

• Busy CPUs

• Anything that prevents the normal forwarding of BPDUs

An STP loop occurs when a blocking port in a redundant topology erroneously transitions to the forwarding state. This transition usually happens because one of the ports in a physically redundant topology (not necessarily the blocking port) ceases to receive BPDUs.

Loop guard is only useful in switched networks where switches are connected by point-to-point links. Most modern campus and data center networks are these types of networks. On a point-to-point link, a designated bridge cannot disappear unless it sends an inferior BPDU or brings the link down. The ST