All systems operational

Reports of messages & notifications not arriving for Cloud API numbers

Resolved
Degraded performance
Started about 2 months ago Lasted about 3 hours

Affected

Meta Infrastructure
WhatsApp Cloud API (Message Delivery)
Updates
  • Resolved
    Resolved

    The issue is now resolved.

    ---

    Post Mortem:

    Issue description and impact:

    On 14.03.2024 at ~16:30 we received reports from some Cloud API customers that messages and status webhooks were not arriving.

    Cause of the issue:

    We identified the cause of the issue as a recent deployment of a new Meta feature (phone number webhook) that was not working as described.

    Mitigation Steps / Summary:

    We rolled back the deployment reverting all numbers to a functioning state. This rollback was completed by ~18:40

    Prevention of recurrence & suggestions:

    We have completed an internal review with the relevant teams and

    made changes / improvements to how we test and deploy new features from Meta.

  • Monitoring
    Monitoring
    A mitigation to restore functionality has been deployed. We are continuing to monitor the situation.
  • Identified
    Identified
    Our engineering team have identified the issue and work is in progress to mitigate it. Note: the issue only affects a small % of Cloud API numbers that are currently using '/waba_webhook' (webhook per WABA) to receive inbound notifications.
  • Investigating
    Investigating
    We are currently investigating reports about messages & notification delivery issues on Cloud API. Our engineering team is currently investigating it. We do not currently have a resolution ETA but we'll update again as soon as possible.