Required Subscription: ShoreTel ECC
Required Permission Level: Administrator
Occasionally Mitel MIVOICE Connect Contact Center (MVCCC, but formerly ShoreTel ECC) will stop logging detailed call events to the CCIR database. The CCIR database will be running, and reporting tools like Brightmetrics will be able to connect to it, but no new data will show up, as if no calls are coming through Mitel MVCCC.
When the Brightmetrics agent attempts to update the Mitel MVCCC group data, it will check to make sure that the Mitel MVCCC summary tables and the CCIR detail events agree on whether there is no call activity. If the Mitel MVCCC summary tables indicate that calls are coming through Mitel MVCCC, but there is no corresponding data in the CCIR database, the "Group Data" data source will fail to update. So if you get an error indicating that 1 data source in the Mitel MVCCC data set is not updating, and that data source is the group data, it is very likely that Mitel MVCCC has stopped logging to CCIR. You can check with Brightmetrics support to confirm if in doubt.
If that happens, these are the things you can try, in order of severity, to get Mitel MVCCC logging to CCIR again. If one of them works, you don't need to do anything else.
1) Try clicking pause/unpause in the CCIR section of Contact Center Director.
2) Kill the "c2g_process.exe" process on the Mitel MVCCC server using Windows Task Manager.
3) Reboot the CCIR server.
4) Reboot the Mitel MVCCC server (or just restart the Mitel MVCCC services on the Mitel MVCCC.
The first and second options will not have any impact on the system. In the first case, Pause / Unpause is a supported operation intended for when expected maintenance is being performed on the CCIR server. In normal circumstances Pause causes Mitel MVCCC to disconnect from CCIR and buffer the call events it would send, and Unpause re-establishes the connection and sends the buffered call events. It does not often resolve the problem documented here, but is the safest option and therefore the first recommendation.
In the second case killing the "c2g_process.exe" process via Task Manager causes the Mitel MVCCC process supervisor to automatically restart it, which does not have any impact on other Mitel MVCCC (service processes and the fresh instance of c2g_process.exe will establish connections to CCIR. This is most often the step that resolves the lost connection problem.
The third option will not have any impact beyond the CCIR server, so if the server is dedicated to CCIR rebooting it will not have any impact. If it is running any services additional to CCIR, the impact should be evaluated accordingly. If CCIR is installed locally on the Mitel MVCCC server itself (which is no longer supported by Mitel), this option is redundant with the fourth.
The fourth option has the highest impact since Mitel MVCCC will be temporarily unavailable while the server or services restart.
If Mitel MVCCC's diagnostics show a red light for CCIR, you can use that to determine if it is working again, but often it shows a green light even though it isn't working. The Brightmetrics agent will keep retrying every 15 minutes or so, so you can wait after each step to see if it starts updating. Alternatively, if you have MySQL database utilities you can check to see if new data is appearing in the "events" table of the "c2g" database on the CCIR server.
Unfortunately, when such a situation occurs, there is no way to recover the lost window of time. The only place that Mitel MVCCC logs detailed call events is CCIR, so if it fails to create the log of activity as it happens, it does not exist anywhere else from which to recover it.
**Mitel MVCCC DOES support redundant CCIR servers, in which case if Mitel MVCCC can not log to the primary CCIR server it will log to the secondary CCIR server instead. When the primary server comes back online the data logged to the secondary in the interim will be replicated back to the primary. There may still be manual recovery steps necessary to process that data for Brightmetrics reporting, but at least it will be possible to do so. Take note, however, that some types of CCIR logging problems are internal to Mitel MVCCC, and it may not know that it is not writing to the primary CCIR server, or it may not be able to fail over to the secondary for the same reason that it is unable to write to the primary, so redundant servers is not a panacea for lost CCIR data, but it is one tool available.**
If none of the above steps resolve the issue, and this is a new install or there have been any changes recently to network topology or server software, there are 3 other possible issues that could be preventing Mitel MVCCC from writing to CCIR.
The first is either a network firewall or Windows Firewall blocking connectivity to TCP port 6306 on the CCIR server from the Mitel MVCCC server. Resolving that is outside the scope of this document, but your network IT staff should be able to verify that the Mitel MVCCC server can make a connection to TCP port 6306 on the CCIR server and troubleshoot appropriately if not.
The second is that if the IP address of the Mitel MVCCC server or of the CCIR server has been changed, the old IP addresses may be stuck in its configuration. It is easy to check and correct the IP address of the CCIR server as configured on the Mitel MVCCC (server in Contact Center Director, but there are also settings on the CCIR server that can only be changed by editing a configuration file. Starting in the folder "C:\Program Files\Mitel\Mitel CCIR Server" (that will be "Program Files (x86)" on 64-bit servers), look for a file named user_registry.ini. If it is not in that folder, look for it in the "Bin" subfolder instead. If neither shows up, open the "registry.ini" file instead. Look for any "IpAddress" values that match either the old Mitel MVCCC server address or the old CCIR server address and change them to be the new addresses of the respective server, then reboot the CCIR server. Make sure to make backup copies of any files before you change them!
The third is if you have recently done an upgrade(we have currently only seen this occur after an upgrade) of ECC there is a time zone setting that may have been altered during that upgrade. The issue is that the way we detect that CCIR is not updating is by comparing the latest data in the ECC database to the latest data in the CCIR database. And the problem is that ECC is logging data in the wrong timezone, so it looks like CCIR is perpetually not current.
There is an additional setting that is often set in ECC and can tend to be different. While we do not have visibility into your own specific ECC settings we have been told by many customers that this setting is in ECC under System Parameters > Client Preferences. Once it is adjusted, the reports should then match up to the ECC and Core time zone settings.
Related Links: Running BrightMetrics Agent Diagnostics
Article is closed for comments.