How to manage SIP phones at scale – in depth

In my previous article I gave an overview of how you can configure the settings on SIP phones – particularly as your deployment scales to hundreds, thousands or tens of thousands of devices.

Today we’ll conclude that article by looking at how this is handled by the Metaswitch SIP Provisioning Server.

As you think about the task of managing the configuration of thousands of SIP devices, deployed at a multitude of different customer sites, there are a few key problems that are immediately apparent.

1. How can we initialize the configuration of a specific phone when it is first deployed?
2. How can we minimize the amount of manual labor involved in setting up the phones in a business?
3. How could we roll-out a change across a large number of devices (e.g. new firmware, or even a new SBC contact IP)?
4. How do we ensure the security of our network – to avoid theft of service by a malicious hacker?

The Metaswitch SIP Provisioning Server handles all of these, and we’ll walk through each of them in turn.

Initial configuration

When building a business group line in the MetaView Web provisioning interface you have the option to Add one or more SIP Phones to the line.

When you add a phone you have to list the MAC address, set the phone model to be “Identified by Endpoint Pack” and then set an authentication window for when you expect the phone to be connected to the network (more on that later).

That’s the server-side configuration, but you also need to tell the phone to pull its configuration from the server – and there are a few different options for this.

• You can manually enter the provisioning URL into the phone (through the keyboard or web UI), which is the simplest option when you’re first testing a phone and don’t have any infrastructure in place.
• If you are drop-shipping phones directly from the manufacturer to the business (or if you want to avoid opening the phones before sending them to your customer), you can use Zero Touch Provisioning (if supported by the manufacturer).
• The third possibility is to use DHCP option 66, where you configure your DHCP server to supply the provisioning URL to the phone as part of the DHCP response. This can be a good method for a large business user that is regularly installing new phones, but it also works well for service providers who want to do a little configuration work on the phones in their lab before shipping to the customer. For example you might power on the phones, use option 66 so they can pull their config, upgrade their firmware, and then ship them off to the end user.

How does Zero Touch Provisioning work?

Good question – it’s a pretty cool system. For this example, I’ll assume you’re using Poly VVX phones.

When a Poly phone is first connected to the IP network, based on its factory settings it will first use DHCP to pick up an IP address, and then it will connect to ztp.polycom.com to request a configuration.

The Poly-administered ZTP server receives the request, and uses the MAC address of the phone to identify the service provider who purchased the phone (you used a Poly-authorized dealer, right?).

As an authorized service provider you have access to an administrative interface where you can program the URL of your own provisioning server. This will typically be something like https://<commportal>.<mydomain>.com/sip-ps/.

The phone then reaches out to your SIP Provisioning Server, which again matches on the MAC address, identifies the specific line that uses this phone, and then sends down a configuration file to the phone.

Using inheritance to minimize manual labor

At this point you may be saying “Okay, fair enough. But how is that configuration file defined? It sounds like a lot of work to build one for each phone.”

This is where the beauty of the inheritance model used by Metaswitch comes to the fore. At the top level are Persistent Profiles, which contain a set of supported phone models (these Endpoint Packs can be found on the Metaswitch Communities website and uploaded into your system), and each Endpoint Pack has a set of default settings that will be true for every phone.

You would configure some global information for all phones in the Phone Profile that’s part of your default Business Group Persistent Profile. This would typically include the fully-qualified domain names (FQDNs) or IP addresses of your Perimeta Session Border Controller / SIP registrar, and a rule to use Line 1 of the phone to be the directory number of the line that owns it.

Since the Metaswitch SIP-PS knows which line you’ve associated with this phone’s MAC address, it can automatically create a configuration for the phone – based solely on these default settings plus the MAC address – that allows the phone to instantly register and be ready to make calls.

But it doesn’t stop there. These top-level settings are associated with the Persistent Profile, but below that a business group administrator (or service provider, on their behalf) can create a set of default settings for phones within that business group, and then at the bottom of the pile, each individual user can update settings for their specific phone to customize the phone to their preference.

When the phone downloads a configuration file, this file will contain any settings customized by the end-user (at the phone level) plus any settings customized for the business group, and then the remainder will use the defaults set up in the Persistent Profile.

You’ll note from the diagram that you can have multiple persistent profiles which are assigned to different business groups and phones. Generally I’d recommend that you minimize the number of profiles – perhaps even to the extent of only having two persistent profiles “Deployment” and “Lab” – since each additional profile increases the work involved in maintaining your deployment (e.g. upgrading phone firmware levels).

Rolling out a configuration change

Now let’s imagine you’re managing a network of phones that are already deployed, and you want to make a configuration change. How does that work?

Perhaps I should start with the simplest case – where a user wants to configure a button on their phone for a particular purpose.

Each end-user can access their phone configuration using the “Manage phones” button from their CommPortal web-interface. Let’s imagine I’ve done that, and now I want to program one of the soft keys on my Yealink T46S to toggle the Do Not Disturb (DND) feature.

I would see something similar to the above screenshot, and I could open up the menus to see the soft keys, Key 1, and then choose from the available features for that soft key action.

Once done, I use the button at the bottom of the screen to save my changes.

What happens next depends on the phone model and the level of firmware it’s running.

• Traditionally, on older models, the configuration would get saved, and then during the night the phone would reach out to the provisioning server to check for updates. The SIP provisioning server would then provide the updated configuration and the change would take effect.
• A user could short-cut this process by rebooting the phone, which would force it to check for a new configuration file as part of the reboot process.
• As you can see in the above screenshot, some phone models now offer the ability for the SIP Provisioning Server to proactively inform the phone about the change (by sending a SIP message), thus causing the phone to pick up the new configuration at its earliest convenience.

That takes care of the simplest case. What if you have a configuration change that will impact an entire business group or even your whole deployment?

That’s where the inheritance model comes into play again. Let’s say you’re making some grand updates to your IP network (don’t do this) and you need to update the SIP domain across all phones in your network.

In this case (using the Service Profile user in MetaView Web) you would manage phone profiles at the Persistent Profile level, and select the phone model you want to change.

You can then find the SIP domain setting and update it. Assuming no-one has overridden this setting at the business group or phone level (and I would recommend that certain critical settings are locked so they can’t be changed by users) then this change will impact every subscriber using that persistent profile.

So that night, when the phones in your deployment check for configuration updates, they will all switch over to using the new SIP domain – saving you a ton of time compared to updating each phone individually.

Security of phone configuration

If you centralize the configuration management for these phones, you need to make very sure that the system is secure – because you don’t want anyone to steal service by downloading the configuration file for a particular phone that they don’t own.

So how does this work?

You’ll notice that in everything we’ve discussed thus far, the MAC address has been the key identifier of the phone. The phone identifies itself to the Zero Touch Provisioning redirect server using its MAC address, and then the MAC address is used to identify the appropriate configuration file to send to the phone.

The problem with this approach is that it’s possible for hackers to spoof a MAC address. If a hacker sends a request with a fake MAC address to the SIP Provisioning Server then the hacker’s phone could receive the configuration file, including SIP domain, SIP username and SIP password.

And that would be bad.

Except that the system is smarter than that. I don’t want to go into all the technical details because (a) that might help the hackers and (b) you might get bored. Yes, that’s right, after 2000 words of technical content I’m suddenly worrying about boring you.

So keeping this at a high level, how do we avoid MAC address spoofing?

• First, by limiting the time window when the MAC address is any use. You may have noticed way back up at the top of this article that when we added a phone we gave a start and end time of the authentication window. This allows us to open up the server to accept a new phone only at a specific time – and ideally we would have a pretty good idea of the day or even hour when the phones are likely to be connected.
• After a phone has connected to the server for the first time using its MAC address, the MAC address is never used again. Instead the phone is provided a secure token which it uses to authenticate in the future.

This strategy makes it highly unlikely that a hacker would be able to spoof the MAC address of just the right phone at just the right moment to steal service, particularly if you are setting up the phones in your lab immediately after configuring the lines in MetaView Web.

Conclusion

When we help a client launch hosted PBX, we invest time up-front in building persistent profiles with the appropriate Endpoint Packs and a good set of default settings to make it easier for our clients to build new business groups and manage the phones in their deployment going forwards.

Hopefully this article has given you a good idea of how it all fits together so that you can do the same. If you have any questions, or would like to see how we could help with your Hosted PBX product, check out our HPBX services page.