Back in 1966, CIA used promise of cars (and stuff from the Sears catalog) to entice indigenous Montagnard (Degar) people to help American war effort in Vietnam, by spying on Viet Cong and North Vietnamese army troops and facilities in their midst. The program was apparently quite successful, and quite a few cars and other items were delivered. Now, driving a vehicle deep inside the ancient rainforests of Vietnam or Laos where Montagnards lived was all-but-impossible due to a complete lack of roads. (But it did not matter; the cars were status symbols.)

I am personally reminded of those cars every time – and it happens surprisingly often – when I encounter a network where a bunch of rather expensive workstations and servers are connected to a bunch of basic switches, usually one under every desk. (And cables are usually what someone found in some drawer.) “Our internet access is slow. Can you please help?” For the record, it is not just the small companies that do this – I have walked into more than one Fortune 1000 facility with cheap secondary switches galore.

Why is that bad? To explain, let us get back to the title question, “why cables and switches matter?” Let us start with the switches:

The purpose of the switches (to oversimplify) is to create a direct and optimal connection between the sender and the receiver; switches determine the optimal path by talking to each other, in a ‘language’ called Spanning-Tree Protocol (and its successors), and then maintain a list of optimal paths. They do it by electing a root bridge, which is the central reference point for the network; it directs how data travels and selectively disables redundant paths to prevent network loops. This election process can take up to 50 seconds when a topology changes; little to no traffic flows during the STP election. Once this process Is complete, the network is in ‘converged’ state. The process is also more CPU intensive than bandwidth intensive. More about this later.

Obviously, the more switches there are on the network, the more complex STP election is. Worse yet, every small switch starts by assuming it is the root. Even worse yet, while STP is reasonably good at determining optimal paths, it is less so when it comes to electing a root bridge. I have seen more than one unmanaged network in which a four-port under-the-desk (i.e. the very slowest of the bunch) ends up being the root.

Why does this matter? All traffic flows are based on the paths calculated relative to the root bridge. For optimal performance, you want your most powerful, centrally located, and well-connected switch elected as the root bridge. Four-port under the desk becoming the root is an instant major bottleneck, since its CPU is designed to handle four ports, not the STP calculations for network of any size.

Now back to cables: what do you think is the very worst thing you can do in a network with a lot of small switches? Use subpar cables, that will have difficulty maintaining network connection at wire speed. Every cable connection failure will trigger a topology change and STP reconvergence, with its 50 second timer and unpredictable root bridge election… you get the idea. I have seen networks with a lot of secondary switches that take hours to converge; those networks are about as fast as driving a car through aforementioned ancient rainforests of Vietnam or Laos.

What to do?

I will share a detailed discussion of a standard layered network design in a later and more technical blog, but here are a couple of simple but less-than-obvious rules.

• Get rid of secondary under-the-desk switches: They are the bane of unmanaged networks. I personally follow a very simple rule: the less switches the network has, the faster and more stable it is.
• Invest in proper cabling: They are your core roads (switches are intersections). The better the roads, the faster the traffic. As easy as plugging a secondary switch in the wall is, it carries a hefty price (it is an equivalent of an intersection with a traffic light). As expensive as proper and future-proof cabling is, it pays in the long run.
• If you have managed switches, design topology with STP/RSTP in mind: As mentioned above, for optimal performance, you want your most powerful, centrally located, and well-connected switch elected as the root bridge. This is one election that should be manipulated.
• Power Over Ethernet (PoE): Many switches can send both data and electrical power down a single Ethernet cable. This allows you to easily install and power devices like security cameras, VoIP phones, and wireless access points without needing nearby electrical outlets. However, such switches also have power ratings – and ensuring that power budget does not exceed the rating is a significant and sometimes neglected part of switching design.