10GBase-T
Xmultiple's
Engineering Department
The
IEEE 802.3an, 10-Gigabit Ethernet over twisted pair standard,
also known as 10GBase-T is now emerging on a compliant of
switches and servers in data centers products. Advances
in semiconductor designs, and sophisticated algorithms intended
to increase electromagnetic interference (EMI) immunity
and lower operating power have provided networking equipment
manufacturers the technology to bring to market these high
speed new switches and servers. It is apparent 10GBase-T
is becoming the connectivity technology of choice. Data
centers can reduce overall costs and improve flexibility
by implementing 10GBase-T. The operation of a 10GBase-T
transceiver and the inherent advantages of 10GBase-T technology
has many advantages over the alternatives, such as optical
fiber and coaxial copper. Twisted pair is in most homes
and businesses today and it replacing and installing fiber
or coaxial cable would be very costly. Power-reducing algorithms
such as Energy Efficient Ethernet (EEE) and Wake-on-LAN
(WoL) are two elements of these new products which will
impact the installations of 10GBase-T.
The
main benefit of 10GBase-T is for data-center, businesses
and home who want high speed connectivity. 10GBase-T provides
the most flexible, economical, backward-compatible, and
user-friendly 10G Ethernet connectivity option available.
It delivers the ability to interoperate with legacy slower
technologies, the use of ubiquitous and inexpensive cabling
and connectors, the flexibility of full, structured wiring
reach, and the ease of Cat6A cabling deployment. 10GBase-T¡¦s
cost- and power-saving benefits are certain to enable designers
to develop state-of-the-art 10G system.
The
growing importance of cloud computing and increasing utilization
of unified data/storage connectivity are reasons for the
installation of 10Gbps Ethernet. While several connectivity
options are available for 10Gbps Ethernet, both over optical
fiber and copper cables, 10GBase-T, is arguably the most
flexible, economical, backward-compatible, and user-friendly
connectivity option available. 10GBase-T is designed to
operate with the unshielded twisted pair cabling technology,
which is already used for 10/100Mbps and 1G Ethernet. 10GBase-T
has the advantage also of interoperating directly with 1G
Ethernet. It operates over a existing cabling type which
is used worldwide today. It operates up to 100 meters and
thereby reaches 99 percent of the distances required in
data centers and enterprise environments.
10GBase-T
is the fourth generation of IEEE standardized Base-T technologies
which all use RJ45 connectors and unshielded twisted pair
cabling to provide 10Mbps, 100Mbps, 1Gbps, and 10Gbps data
transmission. All these generations are backward-compatible
with prior generations. Because Base-T devices have used
an auto-negotiation protocol defined by IEEE to determine
the capabilities supported by the other end of the link,
this backward-compatibility means upgrading to 10GBase-T
is easy.
The
10GBase-T transceiver uses full duplex transmission with
echo cancellation on each of the four twisted pairs available
in standard Ethernet cables; thereby transmitting an effective
2.5Gbps on each pair. These bits are transformed into a
bandwidth-reducing line code called 128-DSQ (for double
square), which limits the analog bandwidth utilization of
the 10GBase-T modem to 400MHz. High-performance line equalization
countermands the low-pass filter effects of the transmission
channel, and additional digital signal processing (DSP)
functions cancel the crosstalk and echo impairments present
in the cabling. Additionally, powerful Low-Density Parity
Check, or LDPC, forward error correction coding rounds out
some of the DSP functions and allows nearly error-free detection
at close to fundamental limits in signal-to-noise ratio.
The major DSP blocks responsible for line equalization,
LDPC forward error correction, and analog line code data
transformation.
EMI Issues and Concerns
The
128-DSQ line code, used by 10GBase-T systems increases the
number of bits per symbol when compared to prior Base-T
standards. Therefore an external signal with EMI which couples
to the cable¡¦s common mode and gets converted to a differential
signal may cause errors on a 100-meter 10GBase-T link.
Common
EMI tests, such as those mandated by the Telcordia GR1089
standard, call for testing with field strengths of 8.5V/m.
Measurements using Cat5 and Cat6 unshielded twisted pair
cabling in 8.5V/m EM fields indicate that differential pickup
can easily reach 60mV, thereby exceeding the voltage margin
at the receiver of a 100-meter 10GBase-T link. To contend
with such EMI events, particularly in an unshielded cabling
system, 10GBase-T transceivers support adaptive interference
cancellation. Such an algorithm requires three key steps:
detecting the interferer, identifying it, and then removing
it.
Detection
can be based on analysis of the differential signal available
at each receiver. This detection must operate in the presence
of the desired signal being sent from the transmitter on
the other side, thus requiring sophisticated signal processing.
A Fourier transform or other techniques applied to the received
differential signal can reveal an interferer that would
otherwise go undetected because its voltage levels are lower
than that of the transmitter signal.
Once
the EMI signal is detected, adaptive filtering techniques
can learn the frequency and relative amplitude of the interference
and filter it out. The latest generation of 10GBase-T equipment
can recover from large EMI events (10V/m fields) in less
than 10£gsec, and simultaneously cancel multiple EM interferers
and operate error-free after adaptation to the EMI. Test
chambers are used with 10GBase-T equipment to make sure
it meets the Telcordia GR1089 standard for electromagnetic
Interference. This assures users of such equipment in a
data center that EMI will not be a problem.
10GBase-T
Technology Advantages
When
compared to other 10Gbps connectivity solutions, one of
the most important advantages of 10GBase-T is the ability
to communicate and interoperate with legacy, often-slower
Base-T systems. Most commercially available 10GBase-T transceivers
are perfectly capable of reversion to both 1000Base-T (1Gbps)
and 100BAase-TX (100Mbps) protocols. Data centers, therefore,
can ¡§future proof¡¨ their switching architectures. A 10GBase-T
switch purchased today can communicate effectively with
all legacy 1G and 100M servers, while providing the infrastructure
to upgrade to 10G switching when commensurate speed servers
are introduced. This also means that data center expenditures
can grow incrementally. Rather than a wholesale conversion
of all servers and switches to 10G speeds, which would be
required with a non-compatible technology such as SFP+ Direct
Attach, 10GBase-T switching systems can convert only those
links that truly need upgrades to 10G speeds, while maintaining
1G speed on legacy servers that don¡¦t require such data
rates.
Unlike
direct-attach twin-ax cabling systems, which constrain full-performance-supported
distances to seven meters depending on cable thickness,
10GBase-T allows cable spans to reach to the full 100-meter
length permitted by structured cabling rules. This extra
reach affords data center managers the flexibility of locating
switches away from server racks and opens up the data center
to architectures that may be more amenable to accommodating
legacy configurations, which rely on more centralized switching.
Heretofore, the lack of economical cabling options for 10G
Ethernet beyond a single or adjacent rack has led to the
popularity of Top-of-Rack (ToR) architectures, in which
a stack of rack mounted servers are connected with short
cables to a fixed configuration switch in close proximity
-- typically on top of the server rack. However, such architecture
has the drawback of increased management domains, with each
rack switch being a unique control plane instance that must
be managed and updated. A more centralized switching architecture
known as End-of-Row (EoR), in which server ports are routed
to a larger switch servicing several racks of servers, can
have the benefit of a singular entity for management with
commensurate reduction in maintenance costs. Also, because
larger switches amortize the cost of common elements such
as power supplies and cooling fans, the per-port cost of
a larger EoR switch may be lower than the equivalent number
of ports in a collection of ToR switches.
10GBase-T
enables uniform transmission media. The alternative in use
today relies on a hodgepodge of cabling types, lengths and
connectors: Cat6 for 1000BAase-T, twin-ax with SFP+ connectors
for short rungs of 10G, optical modules and multimode fiber
for longer runs of 10G. By standardizing on 10GBase-T the
data center manager can focus on only one cabling system
for all speeds and all distances. And, as luck would have
it, that cabling system can be inexpensive Cat6A with RJ45
connectors that are familiar, cheap, and easy to use and
install.
This,
of course, leads to another great advantage of 10GBase-T
technology -- its ability to use ubiquitous and inexpensive
cabling and, in many cases, the existing installed base
of cabling already in the data center supporting 1000Base-T
systems. Even if a data center does not currently have Cat6
or Cat6A cabling as part of its existing cabling plant,
the purchase price of unshielded twisted pair UTP cable
is at least a factor of three less expensive than connecting
twin-ax cable of the same length and up to factor of 10
less expensive than fiber solutions, when the necessary
optical modules are factored in. 10GBase-T has advantages
over SFP+ DA cables.
Since
twisted-pair cabling and RJ45 connectors have been a part
of the data center infrastructure for many years. Techniques
and tools for terminating (attaching connectors to) Cat6
and Cat6A cables on the data center floor exist and are
used widely. Such tools give mangers the flexibility of
cutting spooled cable to needed lengths rather than ordering
and keeping an inventory of pre-defined lengths of terminated
cable, as would be the case for either optical or twin-ax
counterparts. Another advantage of UTP cable is that, unlike
optical fiber requires tight control of bend radius, deployment
rules for bending and twisting UTP are significantly relaxed,
allowing easy installation in even the tightest places.
Power-Issues
for 10GBase-T
One
of the arguments against 10GBase-T has been power dissipation,
but this perspective is rooted mostly in early implementations
of the technology. Recent advances in semiconductor lithography
have allowed 10GBase-T transceivers to enjoy a dramatic
reduction in the power they dissipate during normal operation.
From a per-port power of over 6W just a few years ago (interestingly
enough, the same power per port that 1000Base-T initially
shipped at), the new 40-nanometer (nm) devices are capable
of sub-4W performance today. And due to continuing shrinkage
of chip feature sizes and the famous ¡§Moore¡¦s Law,¡¨ the
28nm devices that will become available in the 2012 time
frame promise to bring power dissipation down further to
about 2.5W per port when operating over a 100-meter line.
For shorter length lines, most modern transceivers allow
a tradeoff between power dissipation and reach. In a 30-meter
mode, for example, a 28nm device¡¦s power dissipation is
expected to be in the 1.5W range. Figure 4 depicts the power
dissipation of 10GBase-T transceivers as semiconductor lithography
has improved.
10GBase-T
Transceiver power per port. The reductions in per-port power
demonstrated over three prior generations are expected to
continue in future lithography generations.
In
addition to the reductions afforded by advances in semiconductor
technology, Base-T systems in general, and 10GBase-T systems
in particular, are able to take advantage of some unique
and standards-based algorithms, which exploit the nature
of computer traffic to further reduce power dissipation.
WoL
is a standard in which a network element, such as a server,
is put to sleep until awakened by a special signal called
a ¡§magic packet.¡¨ The server¡¦s network interface card
reverts to a very low power dissipation mode during the
sleep period but remains alert and waiting for the magic
packet. Once it arrives, the server is awakened and normal
operation is resumed. Since the wakeup time associated with
WoL is typically tens of seconds, it is designed for long
periods of time when servers are idle, such as at night
or during other lengthy periods of inactivity.
WoL
is designed for lengthy idle periods, whereas another technology,
EEE, is specifically designed to take advantage of the burst
nature of computer traffic. Typical Ethernet traffic contains
many gaps, which can range in duration from microseconds
to milliseconds. Until recently, these gaps have been filled
with so-called ¡§idle patterns,¡¨ in which no real computer
information exchange takes place but whose waveform transitions
can be used for maintaining clock synchronization between
transceivers. The EEE algorithm exchanges those idle patterns
for a Low Power Idle (LPI) mode, in which very little power
is dissipated.
EEE
was developed by the IEEE 802.3az task force and issued
as a completed standard in November 2010. The LPI mode used
during idle periods requires a new signaling scheme composed
of alerts over the line, and to and from station management.
During the LPI mode, a Refresh signal is used to keep current
such receiver parameters as timing lock, equalizer coefficients,
and canceller coefficients. These are also critical to enable
fast transitions from LPI to Active modes. Typical transition
times from Active to LPI mode and back are in the three-microsecond
range. The bottom line is that transceiver power savings
utilizing the EEE algorithm can range between 50 to 90 percent,
depending on actual data patterns.
To
put all this in quantitative terms, a 28nm 10GBase-T transceiver
with a typical Active power dissipation of 1.5W for 30-meter
reach will dissipate only 750mW when utilizing the EEE algorithm
with typical computer data patterns. System-level optimizations
in switches and Ethernet controller silicon are expected
to take advantage of EEE¡¦s low-power idle signaling, and
save far more power than the transceiver, since they can
leverage the consumption of the entire switch or server,
which is more than double the power per port of even the
previous generation of transceivers.
