Modern Treasury - Partial API disruption – Incident details

Partial API disruption

Resolved40 minutes

Affected

Platform/API

Operational from 4:22 PM to 5:03 PM

Ledgers API

Operational from 4:22 PM to 5:03 PM

Push to Warehouse

Operational from 4:22 PM to 5:03 PM

Marketing Website

Operational from 4:22 PM to 5:03 PM

  • ResolvedUTC

    Starting at 9:03 am PT we experienced errors affecting ~3% of APIs and Web requests. This issue was caused by a manual invocation of a batch of jobs which overwhelmed our database. By 9:42 am PT, we had removed the jobs and service had returned to normal. The issue has been resolved and the service is operating normally.

  • IdentifiedUTC

    We can confirm errors affecting APIs and Web. We have identified a potential root cause of the errors and are attempting mitigations. Early signs are this mitigation is reducing error rates. We continue to work towards a full root case and recovery.

  • InvestigatingUTC

    We are currently investigating an incident that's impacting a percentage of requests to APIs starting from 9:03 am PT.