Modern Treasury - Degraded Performance for 40 minutes in us001 – Incident details

All systems operational

Degraded Performance for 40 minutes in us001

Resolved
Degraded performance
Started over 2 years agoLasted 26 minutes

Affected

Platform/API

Degraded performance from 8:13 PM to 8:40 PM

Updates
  • Resolved
    UTC
    Resolved

    From 11:40am to 12:20pm PT, the us001 database was under high load due to data processing jobs.

    During this time, the p95 latency for API requests were elevated, peaking at 1.6seconds. We did not see any API errors during this time period. There were around eleven total 500s due to timeouts from our web UI in the same time period.

    We are currently making changes to our data processing jobs to tackle the root cause.

  • Monitoring
    UTC
    Monitoring

    We implemented a fix and are currently monitoring the result.

  • Investigating
    UTC
    Investigating

    We're experiencing an elevated level of API errors and are currently looking into the issue.