People Make Mistakes. Then the Network Goes Down

human error

There’s a growing demand for connectivity. The customers are dependent on the network. Who knew one human being could bring a network down. And take your reputation down with it.

Human error. Most times it’s accidental, occasionally malicious. Either way, when there’s a service disruption, data centers pay the same price. So do customers who run their businesses on the network. While the industry touts 99% uptime, the reality is not so rosy.

“The vast majority of data center failures are caused by human error. Some industry experts report numbers as high as 75%, but the Uptime Institute generally reports it at about 70%…”

Complexity Feeds Human Error

There are plenty of reasons why people make mistakes. It happens but there are ways to minimize them. The infrastructure of most datacenters – hyperscale, cloud, and enterprise – started out orderly. But as demand expanded, the response was not always as neat. Everything got complicated very quickly at the same time budgets got tighter.

To keep costs in line, technicians and contractors get changed out regularly. The pandemic hit the supply chain hard. Data center owners started sourcing materials wherever they could get them. Quality varied but they needed the cables, or panels, or optics. Pretty soon, the infrastructure was a tangle of add-ons and the configuration documentation was haphazard.

People are the ones making the mistakes, but there are contributing factors that datacenter owners can mitigate.

Minimizing Human Error

Network engineers are running at 150% in most data centers. Critical connectivity is everything. Sometimes details that appear to be mundane get lost in the day-to-day. Unfortunately, some of them contribute to major mistakes down the road. The organization needs to address human error proactively. It’s not sustainable to constantly pull engineering professionals away from core functions to fix mistakes.

1. Accessible SOPs

The organization needs to have standard operating procedures for infrastructure service, repair, and maintenance. Datacenters can’t rely on a discretionary response by techs and contractors. Tasks should be performed consistently, following a step-by-step procedure. For a task with a potentially high impact, the procedure should include review and inspection from a supervisor before going live.

2. Quality Cables & equipment

Cabling is the foundation of the entire network. Find a reliable supplier that consistently provides high-quality, high-performance products. We do our manufacturing in-house to maintain quality control. It’s important to get it right every time, not just once in a while.

End-to-end labeling is essential. Without that clarity, no one should be surprised that mistakes are being made. Our cable identification system includes the serial number, the part number, and the product name on all factory terminated cables.

(Concerned about contaminates? Read The Dangers of Dust)

3. Infrastructure visualization

As complexity grows, the ability to understand the infrastructure layout is drastically reduced. Too often, technicians and equipment are sent to the wrong location. When the infrastructure map isn’t current, it’s the starting point for mistakes. The costs pile up, unnecessary truck rolls, travel burdens on employees.

When there’s no map, technicians have to rummage around, trying to find and fix the problem. Left to their own devices, digging through a tangle of cables, are we really surprised they make mistakes?

4. Worker Protection

On top of everything else, communication is extremely difficult in most data centers. The noise level is extremely high. Areas around the servers run up to 92dB(A) and in racks up to up to 96dB(A). It’s in the danger range of noise-induced hearing loss.

Technicians need to be protected but they also need to communicate with each other to get the job done. There are Bluetooth-enabled headsets that protect hearing but also connect to a smartphone. They cancel out the background noise during conversations when technicians talk to each other. Workers won’t need to step away to communicate. Protect employee health and improve productivity.

Simplicity As a Solution

People tend to think of innovation as disruptive, emerging technologies that transform service delivery or business models. For us, innovation means simplicity. Complexity is hurting the industry and the customer who rely on our networks. We’ve working to reduce it since our founding this company in 2003.

Business Process

Innovation isn’t always a giant leap. It can be incremental changes in the business process. For example, we try to invoice orders under a single part number or SKU. It reduces paperwork and still provides a line-item packing sheet for everything in your order.

Sales Process

Our sales process is unique to the industry. We see no value in selling products without confirming what you need. We send a team of design engineers to your site for a walk-through with your team. Then we draft a visualization of the network for your review. When it’s approved, we send a quote. Once the invoice is received, we send your build to one of our manufacturing facilities.

Membership Services

This is one of those big leaps we mentioned earlier. We’ve created a membership program to offer customers product discounts, live technical consultations, and unlimited email and chat support. The upper tiers of the program offer 2-day or same-day shipping. We provide inventory management and for the top tier – there’s a 50/50 price share. Learn more.

Share on facebook
Share on twitter
Share on linkedin
Share on pinterest

Leave a Reply

Your email address will not be published.

Follow our social media
Subscribe weekly news

Get the latest industry news and updates on our new CaaS membership program.

Nullam quis risus eget urna mollis ornare vel eu leo. Aenean lacinia bibendum nulla sed 

Get started with us today!