ShoreTel Database Replication and Manipulation of MAXDBQUERIES!

It really doesn’t matter what VoIP system you installed they all generally have one architectural characteristic in common; the configuration database.  Depending on the system, you might find a database engine that ranges in complexity from an Access Database to a full blown SQL database.  The database will store configuration information, status information and often, call detail records that document phone system activities.   The characteristic of the database that is consistent across all architectures is the fact that there can only be one “read/write” copy of that database!

Some phone systems distribute the database across multiple servers.  ShoreTel, for example, distributes components of the database to application servers and distributed voice mail servers that characterize the single image architecture in a multi-site environment.   We need to better how a change to the database effects the operation of the system, the bandwidth of the WAN links and the demand on the database engine.  First, what constitutes a change to the database?   Well clearly any configuration change that is made to the system.  For example, adding or deleting a User are clearly going to cause a database update!  Lets take a more subtle example, however. Lets consider what happens to a Agent in a Workgroup, located at a remote site, behind a distributed voice mail server.  Each change that Agent makes on their Call Manager represents a database change.   Logging into the system, and Logging out of the system are database changes.  How about, accepting a call being offered to the Agent by a Workgroup?

Each of these Changes is communicated to the database.  The change is first made on the “read/write” database and then replicated to the remote database copy.   ShoreTel system processes use MS Distributed Component Object Model (DCOM) objects to share information from the configuration database among themselves and to write configuration  information to the database.  User configuration options are written to the database from Personal Call Manager, and the telephone interface.  Each ShoreTel service on a distributed server caches it own copy of the configuration database.  When a distributed server loses connection to the HQ server (read “read/write” database) any changes made are no longer received by the distributed server.   If a DVM restarts without a connection the HQ database, services are started but are not functional.  When the network connection is restored, the configuration is retrieved and again cached by each service as the services become functional.

If there is a flap in the WAN we note that the DVM will in fact reload a copy of the database.  This movement of the database between the HQ server and the DVM servers clearly uses bandwidth and also makes additional demands on the database engine.   In ShoreTel the database engine, is now MySQL.   The question becomes how many simultaneous database access (read, modify, write) transactions can the MySQL server handle at one time?  What happens if the transaction can not be completed?  Does it queue and retry?   In a large system of say 700 workgroup agents at a site, is it possible to overload the MySQL database with state change requests?   If you dig down through the Server registry you will find the MAXDBQUERIES is set by default to 100.  It has been our experience that, defending on the size of the system, it is sometimes necessary to dial this number back to eliminate overloading the database.   This adjustment should be made only on the DVM’s in the configuration and not on the HQ server.   You will need to reboot your DVM servers after making this change.  You should also note the difference between HEX and Decimal!

The following SILENT file clip demonstrates how to make this adjustment.

What is a ShoreTel DVM and why do I need one?

What exactly is the value of a Distributed Voice Mail Server (e.g. DVM)?   What are the pro’s and con’s of installing one?  Does it have any impact on resiliency (not redundancy) as it relates to business continuity in the event of server failures?  ShoreTel has a distributed architecture but like all other VoIP solutions there is only one “read/write” database and that is a component of the ShoreTel architecture aptly named the HQ server.   IF this server goes down and the R/W database is unavailable configuration changes can not be made throughout the “single image” installation.

Installing a DVM at the same level, or in the same site as the HQ server, provides a high degree of resiliency at comparatively low cost.     At the HQ site, put all your HQ users on a DVM.   If the DVM goes down, the HQ will pick up the heavy lifting for the Users on the DVM.  If the HQ goes down, the DVM users will still have VM and AA services.  As of today, there are three services, however,  that are NOT distributed in the ShoreTel architecture. These services are known as Workgroups, Route Points; and Account codes.   If you lose the HQ server, you will lose these services for all sites, even if they have a DVM installed at that site!

As it relates to low cost business continuity options, we like to install a DVM at the HQ site, but we want all switches at all sites to be managed by the HQ server.   This usually provokes a heady discussion, but here is our reasoning.   The real value of a DVM is to keep VM and AA media streams off the very expensive WAN connections. Remember that a DVM can fail up, which means the HQ server can take over Voice Mail and AA processing for the users at a site that has a failed DVM.   It makes sense to put the users at a remote site on the DVM at that site, but does it really make sense to have the switches at that site managed by the DVM at that site?

We think not.   Lets separate the issue of Users and Voice Mail from issues like TAPI, Workgroups and Personal Call Managers.   We need to remember that if a server goes down, the switches managed by that server will lose all the TAPI information for the phones that it controls.  This means you will have no functioning Workgroup Agents and not ability to monitor those Agents. Additionally, the Personal Call Managers will not work for any extensions on switches managed by the down server.

Given that Workgroups is not a distributed service, if the HQ server goes down, you will not have Workgroups anyway.   If the DVM at a remote site goes down, the HQ server will proxy for that sites Voice Mail and Automated Attendants.  Given that the HQ server was managing the switches at that remote site, you will not lose any of the PCM functionality highlighted above.  It occurs to us that this is a better place to be.   Let the HQ manage all switches and use the DVM’s for Voice Mail services for the users at remote sites!  Use a DVM at HQ for additional resiliency.