Resolved
Degraded Availability on APIs

Started
July 25, 2022 at 6:40 PM
Status
Resolved after 23 minutes
Affected
Platform/API
Ledgers API

Impact

Major outage
  • Resolved
    July 25, 2022 at 7:03 PM

    At around 11:40 AM (PT) on July 25, a faulty change to our tracing and metrics caused extended downtime on our API. For about 23 minutes the Modern Treasury API was in a degraded state, with 100% unavailability for 13 minutes within that period. Going forward, we commit to improving our CI/CD processes to ensure that similar faults are caught earlier in our pipelines and prevented from deployment.

  • Investigating
    July 25, 2022 at 6:40 PM

    We are experiencing a high number of http status code 500s on many of the APIs. We've identified the issue and working on restoring service.