CallTower Status
All Systems Operational

About This Site

Welcome to CallTower's home for real-time system status updates. Here you will find a current and historical data on system performance.

Cisco Unified Communications Operational
Cisco Voice Operational
Cisco Jabber ? Operational
VoiceCube Operational
Attendant Console Operational
Cisco Webex Calling Operational
CT Text Operational
CT Cloud Operational
CT Cloud SIP ? Operational
CT Cloud Voice ? Operational
Attendant Console Operational
CT Cloud Meeting Operational
Microsoft Operational
Microsoft 365 Operational
Microsoft Teams - Direct Routing Operational
Microsoft Operator Connect Operational
CT Text Operational
Contact Center Operational
Clarity Connect Operational
Five9 Operational
Intermedia Operational
Cisco Webex Contact Center Operational
Solgari Operational
Global SIP Trunk Operational
Hosted Cloud Based PBX Operational
Zoom Operational
CallTower Connect Operational
Fax Operational
CT Cloud Fax Operational
Concord E-Fax Operational
Concord ATA Fax Operational
eComFax Operational
Skype for Business Operational
Skype for Business - Voice ? Operational
Instant Messenger and Presence Operational
Attendant Console Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Scheduled Maintenance
Scheduled Maintenance Oct 18, 2024 22:00 - Oct 19, 2024 04:00 MDT
Purpose: Patching and preventative maintenance on multiple systems.
Systems Affected:

MasterFax & CT Cloud Fax:
-MasterFax & CT Cloud Fax - General Server Maintenance
***Possible Impact: Customers will experience interruptions to both MasterFax and CT Cloud Fax services during the Maintenance Window.

Cisco Voice:
-HCS Device Pack upgrade to 12.5(1.22212-1)
***Possible Impact: Calltower will be loading a new Device Pack from Cisco to HCS/CUCM. This includes new Firmware on select models. A reboot of the Cisco Unified Communications Manager is required. Your phone will re-register to the next server in the Server list. Calls in progress 'should' remain up until the call has ended. Phone will most likely re register multiple times during the MW time. This is expected behavior.

CT Cloud Voice:
-CT Cloud Voice - Algo Paging Adapter EPP Update
***Possible Impact: Businesses may experience a short period where their Algo Paging Adapters (8180G2, 8186, 8188, 8301) become unreachable as they update their firmware. It is important adapters are not rebooted during their firmware upgrade.
-CT Cloud NOAM - Internal migration
***Possible Impact:
*CT Cloud SIP - Customers can expect CommPortal to be unavailable during migration.
*CT Cloud Voice - Customers can expect CommPortal, Voicemail and SIP provisioning for new phones to be unavailable during migration.
Total downtime: 2-3 hours

Posted on Oct 11, 2024 - 16:20 MDT
Past Incidents
Oct 15, 2024

No incidents reported today.

Oct 14, 2024

No incidents reported.

Oct 13, 2024

No incidents reported.

Oct 12, 2024

No incidents reported.

Oct 11, 2024

No incidents reported.

Oct 10, 2024
Resolved - Microsoft is reporting that the issue affecting inbound calls to Auto Attendants and Call Queues has been resolved. CallTower has confirmed the resolution in our testing. Microsoft’s incident TM907891 is included below.

"Details
Title: Users' calls routed through Auto attendants and Call queues within Microsoft Teams may fail

User impact: Users' calls routed through Auto attendants and Call queues within Microsoft Teams may have failed.

Final status: We've routed the affected call requests to a healthy portion of service infrastructure and confirmed through monitoring service telemetry that impact has been remediated.

Scope of impact: Any user who attempted to make calls routed through Auto attendants and Call queues within Microsoft Teams may have been impacted.

Start time: Thursday, October 10, 2024, at 3:51 PM UTC
End time: Thursday, October 10, 2024, at 5:50 PM UTC

Root cause: Auto attendant and Call queue call routed call requests were being routed to an unhealthy portion of service infrastructure, resulting in impact.

Next steps:
- We're continuing to monitor the affected infrastructure closely to gain further insight into the source of the underlying issue and better prevent similar impact in the future.
- We’re assessing our automated recovery mechanisms in conjunction with our findings into the underlying problem to identify areas for improvement to reduce the time to mitigate similar future issues.

This is the final update for the event."


CallTower will be closing this master case and all associated cases.

Oct 10, 12:33 MDT
Identified - CallTower continues to work with Microsoft on the issue affecting inbound calls to Teams Auto Attendants and Call Queues. Microsoft is working on rerouting traffic to fix this issue now and will provide further updates shortly. CallTower is seeing some successful calls in our internal testing and testing reported by customers.
Oct 10, 12:00 MDT
Update - CallTower continues to urgently investigate the issues affecting inbound calls for Microsoft Teams Operator Connect and Direct routing Auto Attendants and Call Queues. Microsoft has acknowledged a global outage at this time and is working to resolve it. If you need assistance in forwarding numbers, please reach out to support@calltower.com. Microsoft’s most recent update is provided below.

Issue ID: TM907891
Affected services: Microsoft Teams
Status: Investigating
Issue type: Advisory
Start time: Oct 10, 2024, 12:14 PM CDT

Current status
Oct 10, 2024, 12:14 PM CDT
We're investigating a potential issue with users' Auto Attendant and Call Queue calls failing within Microsoft Teams, and we're checking for impact to your organization. We'll provide an update within 30 minutes.

Oct 10, 11:32 MDT
Investigating - CallTower is investigating reports of inbound calls ringing busy.

CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to support@calltower.com. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

Oct 10, 10:49 MDT
Oct 9, 2024

No incidents reported.

Oct 8, 2024

No incidents reported.

Oct 7, 2024
Resolved - CallTower Engineers have continued to monitor the issue and have not seen any further issues. CallTower believes the issue has been resolved.

CallTower will be closing this master case and all associated cases. Within three business days, an Official Root Cause Analysis (RCA) or a Preliminary RCA will be available and provided to customers upon request.

Oct 7, 14:35 MDT
Monitoring - CallTower Engineers have made a change to route around the issue. Our testing shows that calls are now completing as expected. CallTower will continue to monitor. If you are still experiencing an issue please provide Support with call examples to investigate.

CallTower has opened a master case for this issue as it is affecting more than one CallTower customer. This issue may or may not directly affect your services. If you are affected, please report the issue to support@calltower.com. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise.

Oct 7, 12:36 MDT
Identified - CallTower Engineers are aware of an issue causing call failures for our Teams Operator Connect and are working to resolve it.

CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to support@calltower.com. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

Oct 7, 12:07 MDT
Resolved - CallTower has continued to monitor the issue. Our testing and the testing of some of our customers have continued to show successful results. CallTower believes the issue is resolved.

CallTower will be closing this master case and all associated cases. Within three business days, an Official Root Cause Analysis (RCA) or a Preliminary RCA will be available and provided to customers upon request.

Oct 7, 10:53 MDT
Monitoring - CallTower Engineers have identified an issue and implemented a change. We have confirmed through testing that the issue with duplicate calls have been resolved. Please reach out to CallTower Support if you are still experiencing issues. CallTower will continue to monitor.
Oct 7, 10:21 MDT
Update - CallTower Engineers continue to investigate the issue of duplicate incoming calls for our Teams Operator Connect and Intermedia customers. Please continue to send in specific call examples to help us with our investigation.

CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to support@calltower.com. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

Oct 7, 09:52 MDT
Identified - CallTower support has identified multiple reports of customer receiving inbound calls on the Teams Operator Connect platform.

CallTower has opened a master case for this issue as it is affecting more than one CallTower customer. This issue may or may not directly affect your services. If you are affected, please report the issue to support@calltower.com. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise.

Oct 7, 09:18 MDT
Investigating - CallTower is investigating reports of customers receiving duplicate incoming calls on Intermedia.

CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to support@calltower.com. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

Oct 7, 08:40 MDT
Oct 6, 2024

No incidents reported.

Oct 5, 2024

No incidents reported.

Oct 4, 2024

No incidents reported.

Oct 3, 2024

No incidents reported.

Oct 2, 2024

No incidents reported.

Oct 1, 2024

No incidents reported.