360dialog - Issue with On-Premise API Message Delivery – Incident details

All systems operational

Issue with On-Premise API Message Delivery

Resolved
Degraded performance
Started 9 months agoLasted 8 days

Affected

Meta Infrastructure

Degraded performance from 10:35 AM to 8:55 AM

WhatsApp On-Premise API (Message Delivery)

Degraded performance from 10:35 AM to 8:55 AM

Updates
  • Resolved
    Resolved

    The issue is resolved:


    Issue description and impact:
    On 17.01.2024 we received reports from some customers that message and status webhooks were being delayed.

    Cause of the issue:
    We identified an issue with an internal component (callbacker) that was having issues with the latest deployment resulting in the delays.

    Mitigation Steps / Summary:
    We reverted the deployment until we are able to fix underlying issues.

    Prevention of recurrence:
    We have completed an internal review with the relevant teams and work is underway to further strengthen our callback service.

  • Monitoring
    Monitoring

    We are recovering from an issue that affected On-Premise API message delivery from ~6:20am to ~12:20pm UTC on Jan 17th. The situation has stabilized and we are continuing to monitor.

  • Investigating
    Investigating

    We are receiving reports of delays in messaging for On-Premise setups. Our engineering team are currently investigating it.