Page1    Page34                                                                        Page36  Page177

between the US Tier2 centers and the European Tier1s over US LHCNet and GEANT2, which is

required in the CMS Computing Model for example, are under discussion.

The US Tier1 centers and US LHCNet's Points of Presence in New York and Chicago are linked

through the metropolitan network (MAN) rings operated by ESnet (Figure 12). ESnet's LIMAN

(Long Island Metropolitan Area Network) has provided dedicated 10 Gbps wavelengths

(lambdas) to BNL since May 2006. In Chicago, ESnet took over the operation of the 10 Gbps

lambdas which connect the FNAL campus over a dark fiber to Starlight for production

networking, using its newly-built West Chicago MAN. A direct connection between Fermilab and

US LHCNet is maintained for network development. ESnet and Caltech have co-designed the

transatlantic portion of the LHC Optical Private Network, to directly connect FNAL and BNL to

CERN via virtual circuits. This provides the same level of redundant and resilient service end-to-

end as exists between the CERN Tier0 and the Tier1s in Europe.

Figure 12: The ESnet and US LHCNet Networks

During 2006-7 we decided to adopt the "circuit-oriented services" paradigm, also adopted by

most of the research and education networks around the world. We also found, on several

occasions, that the Force10 architecture would not allow stable reconfiguration and re-routing of

traffic when a link "flaps" (goes up and down repeatedly). We therefore turned to the Ciena Core

Director CD/CI multiplexers that provide stable fallback in case of link outages at Layer 1 (the

optical layer), and full support for the GFP/VCAT/LCAS protocol suite.

We started deploying the new devices in March 2007 in Chicago, New York City and later in

Geneva. The fourth Ciena CD/CI was installed in Amsterdam in August 2007. These devices

provide robust fallback at the optical layer among the links if any of them fails. The Core

Directors also will provide circuit-oriented services with robust bandwidth guarantees to support

and guarantee rapid delivery of high-priority network traffic flows. The allocation of bandwidth

channels will be dynamic, and policy-driven, in order to optimally match the allocation of the

available bandwidth across the Atlantic to the experiments' needs. These technical capabilities

will be matched to the bandwidth provisioning services being developed in the DICE group

(formed by Dante, Internet2, ESnet and Canarie) as well as campus solutions like the Terapaths

(BNL) and LambdaStation (FNAL) projects, to provide dynamically provisioned end-to-end

circuits, with dynamically adjustable bandwidth provisioning based on the application's needs

35