Sometimes the most useful features are the ones hiding in plain sight.
We were recently running a little training session for one of our clients when we demonstrated a small trick in SAS. The people on the call had been working with Metaswitch for years, so this wasn’t an introduction to SAS or basic troubleshooting.
But when we showed them this particular technique, the reaction was essentially: “Wait, you can do that?!”
They immediately saw how useful it could be – and so I figured maybe we should share this tip more widely.
The problem with searching a SAS Detailed Timeline
If you work with a Metaswitch deployment, you’ve probably spent plenty of time in the Service Assurance Server (SAS).
SAS is one of the most useful troubleshooting tools in the Metaswitch ecosystem. You can search for a particular call, open its Detailed Timeline, and then see the events that occurred as that call made its way through the network.
The timeline gives you a summary of each event, and clicking an event displays the detailed information associated with it… and that’s great when you’re following a call sequentially, but sometimes you aren’t only interested in what happened next – you’re hunting for one particular piece of information.
Perhaps you know that an IP address is involved in the problem. Or you’re looking for a particular phone number. Or perhaps you’ve spotted something unusual in a SIP message and want to know everywhere that same value appears during the call.
The information may be somewhere in the detailed events, but those details aren’t all displayed at once. You select an event, examine its details, select another event, examine those details, and so on.
For a call with lots of events, that’s a little like searching for a particular sentence in a book by turning to every page individually. If only there was a search button!
Well… as it happens, there is, but it requires a little effort.
The surprisingly useful “Print” button
While viewing the Detailed Timeline, look at the toolbar at the top of the SAS window.
You’ll see a Print button.
“Print” doesn’t immediately sound like an exciting troubleshooting feature, but it actually unlocks the ability to export the entire call as a searchable text file.
When you click on Print you get this dialog box.
In the dialog, select:
- Detailed events (or even “Engineering level events”, if you’re really keen) for the level of event you want to see.
- All Systems if you want information from all of the systems (e.g. CFS, Perimeta, EAS, etc) involved in the call.
- Then click Print Details.
SAS will produce a printer-friendly output containing not just the event summaries, but the detailed information for the events.
You can then save that output as a text file.
And suddenly, instead of having dozens of separate events whose details you need to inspect individually, you’ve got the detailed information for the whole call in one place.
Now you can search everything
And now you can use a magic tool called… regular old text search. You can search across the detailed information from the entire call for a specific detail like:
- a particular IP address;
- a calling or called telephone number;
- a SIP header;
- part of a SIP URI;
- a particular error or response;
- or some other distinctive piece of text you’ve already identified while troubleshooting.
For example, imagine you’re troubleshooting a call and you’ve identified an unexpected IP address in one of the SIP messages.
You want to know where else that address appears. Searching the normal Detailed Timeline isn’t particularly helpful because the address may be buried inside the details of events that aren’t currently selected. But if you export the Detailed events using Print Details, you can simply search the resulting text for that IP address. You can then jump straight to each occurrence and see the surrounding information.
Did we really just write an article about a Print button?
Yes, because sometimes it’s the simple things that can save a lot of time and help make your job easier.
We’ve written before about the difference between core skills and rare skills. We use this all the time – it’s a core skill for us – and it felt so simple that we didn’t even realize that other people might not have seen it.
Our client had been using Metaswitch for years and hadn’t come across this technique. Their reaction when we demonstrated it convinced us that plenty of other experienced Metaswitch engineers probably haven’t either.
So now you know. Sometimes the best troubleshooting tool is Ctrl+F.
