Cisco ONS 15400 Series

The Ohio Academic Resources Network (OARnet) is the networking arm of Ohio's high-performance computing and networking center, also known as the Ohio Supercomputer Center (OSC). OARnet has evolved to become one of the state's largest regional Internet service providers (ISPs) connecting more than 90 colleges and universities throughout the state of Ohio. The organization's new statewide optical DWDM network, known as The Third Frontier Network (TFN), will link the state's colleges and universities, community colleges, branch campuses, K-12 regional service agencies, and eventually district locations to each other and to the Internet.

As the main provider of Internet and high-speed data connectivity to Ohio's many excellent schools of higher education, among them the University of Cincinnati, Ohio State University, Cleveland State University, Case Western Reserve University, and Ohio University, OARnet faced a growing challenge. Its member customers were deploying increasingly powerful, bandwidth-demanding applications. OARnet was struggling to supply bandwidth across its slower-speed, difficult-to-scale DS-3 and OC-48 ATM network backbone.

According to OARnet, it was seeing aggregate growth in higher education traffic of 30 percent a year. At the same time, its network costs were increasing approximately 25 percent each year as it was becoming more costly and difficult to procure larger circuits between sites-DS-3s, OC-3s, OC-12s, and OC-48s. In some areas, OARnet couldn't purchase the high-speed circuits it needed to meet customer demand.

Demand for bandwidth was increasing because Ohio's major educational institutions were rapidly adopting advanced applications. These included distributed classroom environments, shared instrumentation applications, and research collaboration and advanced high-performance computing. Distributed classroom environments were enabling professors at one institution to teach students at other institutions hundreds of miles away. They were also enabling multiple professors at different schools to team-teach a class, bringing together unparalleled expertise that would be otherwise unavailable.

But distributed classrooms are demanding. They require high-quality and low-latency audio and video links, shared workspaces, electronic whiteboards, archive and playback of multiple streams from remote servers, and advanced audio/video technologies such as 360-degree cameras. These capabilities were quickly consuming DS-3 or OC-3 links, limiting the usefulness of this experience.

Another important application suffering under the slow-speed ATM network was shared instrumentation. Ohio maintains expensive research instruments such as remote telescopes on distant mountains or real-time visualizing supercomputers at different locales. The Ohio State University Hospital in Columbus, Ohio maintains the largest Magnetic Resonance Imaging (MRI) facility in the world. Medical researchers throughout Ohio would greatly benefit from sharing this resource. Using an optical fiber network and with access through Internet2, researchers would be able to steer a remote telescope and display high-resolution MRIs on their local systems. The advancements enabled by the new network would be incalculable.

To ensure that its member institutions would have the most advanced networking technologies available to pursue leading-edge research and collaborative sharing applications, OARnet chose to build a regional dense wavelength-division multiplexing (DWDM) fiber optic network. Calling it the Third Frontier Network (TFN), OARnet built a 1600-mile backbone infrastructure by purchasing either unused or dark fiber from several telecommunications companies and service providers throughout the state. To light the fiber, it deployed more than 50 Cisco® ONS 15454 Multiservice Transport Platforms (MSTPs). These next-generation Cisco multiservice optical platforms integrate critical DWDM transponding and muxponding functions on one platform and also provide support for Gigabit and 10-Gigabit Ethernet connectivity.

"The Third Frontier Network is one of the most important education and research projects in Ohio's proud history, and serves as an example to the nation of how the public and private sectors can collaborate for the greater good of the people," says OSC Executive Director Stan Ahalt. "TFN will generate many new opportunities, not only for advances in education and research, but also for advances in the economy. It will enable us to attract more companies to locate in Ohio and to stimulate the creation of new companies that create new jobs."

By deploying DWDM, OARnet dramatically increased the scalability of its network. The Cisco ONS 15454 MSTP currently supports up to 32 wavelengths of 10-Gbps bandwidth for a total capacity of 320 Gbps per fiber. Beyond capacity alone, however, OARnet was impressed with many of the other capabilities of the Cisco ONS 15454 MSTP, including automated power adjustment, reconfigurable optical add/drop multiplexing (ROADM) and, in particular, transmission capability from tens to hundreds of kilometers. This was enabled through the use of advanced amplification, dispersion compensation, and Enhanced Forward Error Correction (FEC) technologies.

According to OARnet, it required 10-Gbps speeds in its regional network. In the 10-Gbps market space, OARnet found that Cisco Systems® offered the latest advances in optics, including Enhanced FEC and high-dispersion tolerance. These optical advances enable the wavelengths to go much longer distances before the amount of noise corrupts the signal and requires an optical-to-electrical-to-optical regeneration--a very expensive process required on a per-wavelength basis.

While in service, the intelligent DWDM features of the Cisco ONS 15454 MSTP continually adjust to changing network conditions. The firmware embedded in each network element continuously monitors the quality and the strength of the optical signals, adjusting the optical attenuator (device that lowers optical power levels) and other configurable parameters such as launched power and amplifier gain in response to changing power and signal conditions within the network. This ability to monitor power levels ensures power equalization across all wavelengths and minimizes data bit errors. On competing platforms, monitoring and adjusting must be done manually, requiring highly specialized and skilled technicians to visit individual sites, test for power levels and noise, and manually "readjust" and maintain signal stability through the network.

The dynamic adaptation of the network elements in a Cisco DWDM network to changing conditions can compensate for both long-term variations, such as optical component aging, and short-term sudden variations, such as the loss of one or more channels in the DWDM composite signal.

Another valuable tool during the deployment of the new optical network was Cisco Transport Controller, which allowed OARnet to manage all of its optical devices end-to-end per optical ring from one built-in graphical interface. This made it easy for the OARnet team in the field to quickly assimilate the new devices into their deployment, provisioning, and operations management regimen. OARnet has also used Cisco Transport Manager as the centralized overall network management platform.

In OARnet's initial service deployment, wavelengths transporting data for the higher education sites terminate at Cisco ONS 15454 MSTPs into Cisco 12410 routers at the major point of presence (POP) locations in Cincinnati, Dayton, Columbus, Akron, Cleveland, and Toledo. Data is routed to Gigabit Ethernet ports on the Cisco 12410 routers and onto Cisco 3750 Layer 3 switches that can send multiple Gigabit Ethernet services to one or more institutions or can break up a single circuit into smaller increments across multiple institutions.

OARnet says that, before deploying the Cisco ONS 15454 MSTP-based optical network, the cost of acquiring higher-speed circuits from telecom providers was increasing at a rate of about 25 percent a year--and sometimes OARnet couldn't acquire the high-speed circuits it needed. OARnet now estimates that, once it pays off the loan needed to purchase the fiber, equipment, and network interconnections, its recurring costs will drop by approximately US$300,000 a year.

In addition, OARnet has a highly scalable and easy-to-provision optical network. With 32 wavelengths per fiber currently available to OARnet and with features such as automated power provisioning and the use of ROADMs, described earlier, OARnet can rapidly adjust the network with software, adding or dropping off wavelengths with high-speed services at various remote nodes. OARnet can also take advantage of the advanced provisioning tools pioneered on the Cisco 15454 Multiservice Provisioning Platform for SONET circuits and now applicable to DWDM networks. These include features such as A-to-Z network-based wavelength service provisioning, and graphical software wizards to simplify and speed user operations for such tasks as initial ring turn-up, service provisioning, and network node and bandwidth upgrades. Also, if it chooses to add SONET services to the DWDM network at a later date, OARnet will be able to manage those services with the same management tools, Cisco Transport Controller and Cisco Transport Manager, which it uses to manage the DWDM wavelengths.

After it has brought its higher education customers onto the regional DWDM network, OARnet, in partnership with the Ohio state government through the Office of Information Technology, intends to move into the next phase of its plan: to aggregate all of the state agencies and bring them as well onto the Third Frontier Network.