Jaycee's Networking

July 1, 2009

End-of-Row or Top-of-Rack for Server Networking in DC

Filed under: Information, IOS, Routing Design — Tags: , — Jaycee @ 2:05 pm

There are 3 primary approaches for server networking in dc environment:

1. End-of-Row:

a. When aggregating servers larger than 1U or servers with a mixed amount of interface types and densities, Catalyst 6500 Series switches are used to support one or more racks.

b. Advantage:

1) cost effective – delivering the highest level of switch and port utilization, especially when coupled with the rich set of network visualization services available in the Catalyst 6500 Series. (6500 supports a wide variety of service modules, simplifies pushing security and application networking service into the access layer.)

2) server-independent – provides maximum felxibilty to support a borad range of servers.

3) performance advantage – 2 servers exchange large volumes of information cab be placed on the same line card as opposed to card-to-card or switch-to-switch, which will be slower.

c. Disadvantage:

1) cable/patch panel cost – physical volume of the cables and the waste of the valuable rack space

2. Top-of-Rack:

a. When stacking  40 1U servers in a rack one or two, 1U Rack Switches (like the Catalyst 4948-10G) are often used to aggregate all of these servers with Gigabit Ethernet and then run a couple 10GbE links back to the aggregation switches. (In some cases, 2x 4948 switches are used for HA purpose.) (Catalyst 4948is optimized for the dc environment.)

b. Advangate:

1) simplified cable management

2) avoid rack space and cooling issues

3) avoid cooling issues of end-of-rack switching

4) fast port-to-port switching for servers within the rack

5) predictable oversubscription of the uplink ans smaller switching domains (one per rack) to aid in fault isolatio and containment

c. Disadvantage:

1) Not enough servers to fill the switch in one rack – solution: put one top-of-rack switch server in an adjacent rack to preserve the advantages of the top-of-rack switch wile increasing port utilization.

3. Integrated:

a. When using blade servers, blade switches would be deployed. Cisco Catalyst Blade Switch 3000 Series support the visualization, segmentation, and management tools needed to properly support this environment.

b. When server virtualization is in use, it can rapidly increase the complexity of the network (the number of MAC addresses, complexity of spanning tree, data pathways, etc.)

c. In some larger dc, using the pass-thru module or the balde switches where it’s aggregated into a series of rack switches.

*Most people like dual top-of-rack because servers have dual production uplinks. But they can’t really fit in 40 1U servers due to power limitation or heating problem. So they end up 3 racks using the top-of-rack switch in the middle rack and cables are going between cabinets. End-of-rack is actually designed for this situation. But placing 6500 in the middle rack would cause overheating problem. 6500 switches thus shall be placed at the end of the row.

Advertisements

Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: