I was recently browsing rural accommodation options for a future vacation, and came across a ranch in the mountains in a good location. That’s a good start, so I started looking at reviews for this place. The average rating was 4 stars (out of 5), which is pretty good, but I also noticed quite a few strongly negative reviews – which was intriguing.
The favorable reviewers used words like beautiful, relaxing, peaceful, wonderful and friendly. Whereas the negative reviewers used words like terrible, rude, filthy and dangerous.
What’s going on? How are these people all describing the same place?
Common ground
Some of the reviews had a response from the owner, and in reading the reviews and the responses, I started to see some patterns.
- Most reviewers agreed that the scenery was beautiful and the food was good.
- Most reviewers agreed that the ranch was “rustic” (some would say run-down) and that they were a little short-staffed.
Some people experienced this and had a great time. Others thought it was a disaster. How can that be?
The system is fragile
It seems like the difference between a great vacation and a horror-story would start with something pretty small.
- The generator broke.
- A staff member got sick.
- One of the horses needed new shoes.
- Guests exaggerated their riding skill-level.
In any business you’re going to encounter unexpected issues. But in this case, it sounds like the whole operation is running so close to the bone (understaffed, without backup plans / spares, without needed maintenance) that not only are unexpected problems more likely to happen, but when they do, everything falls apart.
So what ought to be a small problem becomes a large one.
- No power for a week.
- Rooms full of mouse-droppings.
- Trips to the hospital.
- Staff who are so overworked that they start grumbling openly about management, drinking on the job, and (allegedly) cooking meth in their rooms!
So if you got lucky, and visited in a week when nothing went wrong, you may have a great time. But as soon as things start to go off the rails, you’re in for a pretty unpleasant vacation.
And this relates to telecoms… how?
You may not run a ranch (or maybe you do?), but many of you are responsible for operating a voice communications network. If you operate that network the way this ranch is run, your subscribers will experience regular outages.
If you ignore regular maintenance, if you don’t have backup routes, or spare hardware, or redundant routers… you can be sure that occasionally something will go wrong, and you will be totally ill-equipped and ill-prepared to deal with it.
You can’t build a network for the best case scenario. You need to build a resilient network that’s designed with the expectation that things will go wrong – and continues to provide service regardless.
The ranch owner often responded to negative reviews by explaining what went wrong. She’d explain that it was just bad luck on that particular day, without taking any responsibility.
I don’t mean to be particularly judgmental here – I absolutely understand why she thinks that way. It’s easy to give ourselves a pass – to think that this particular service interruption was a freak incident, and we just got unlucky. If we were unlucky then not only can we avoid blame, but we don’t even have to take any actions to fix the situation.
In telecoms we can’t afford to blame bad luck. We need to take responsibility for problems, figure out what went wrong, and figure out how to improve the quality of our service in the future. If you’re having issues, and need an external consultant to be the bad guy and ask all the tough questions, let us know.
