Modern Treasury - Notice history

Payments: Partner Bank under maintenance

Platform/API - Operational

Ledgers API - Operational

Push to Warehouse - Operational

Marketing Website - Operational

Notice history

Dec 2022

API and Application Outage
  • Resolved
    Resolved

    We identified and resolved a misconfiguration with our network settings. We just resolved the issue!

  • Investigating
    Investigating

    The Modern Treasury API and Application may be experiencing degraded performance. We are investigating.

Scheduled Maintenance: Software Upgrade
  • Completed
    December 11, 2022 at 9:09 AM
    Completed
    December 11, 2022 at 9:09 AM

    Maintenance for Redis has completed successfully. Actual downtime was 21 seconds.

  • In progress
    December 11, 2022 at 8:15 AM
    In progress
    December 11, 2022 at 8:15 AM

    Maintenance is now in progress

  • Planned
    December 11, 2022 at 8:15 AM
    Planned
    December 11, 2022 at 8:15 AM

    This is due to a scheduled upgrade of a Redis cluster to a newer software version. Such planned upgrades are part of our maintenance routine as we stay ahead of EOL schedules and ensure that we are always up-to-date on security patches.

    During this time, the platform will be inaccessible. All operations will resume immediately after the maintenance window.

An automated up-time check has failed. (False positive)
  • Resolved
    Resolved

    The issue was caused by a disruption in a downstream provider, but this only affected our uptime check. No customer was impacted.

  • Investigating
    Investigating

    The Modern Treasury application and API may be experiencing degraded performance. We are investigating.

Nov 2022

API and Application Errors.
  • Monitoring
    Monitoring

    We implemented a fix and currently monitoring the result.

  • Resolved
    Resolved

    API error rates and latencies have been mitigated.

  • Monitoring
    Monitoring

    We believe we have mitigated the cause of the errors and will continue to monitor.

  • Investigating
    Investigating

    The Modern Treasury Ledger API may be experiencing degraded performance. We are investigating.

Most Ledger Transaction updates are failing
  • Resolved
    Resolved

    We have resolved the issue.

  • Monitoring
    Monitoring

    We implemented a fix and currently monitoring the result.

  • Identified
    Identified

    We've identified the fix for this issue.

  • Investigating
    Investigating

    We are currently investigating this incident.

Scheduled Maintenance: Hardware Upgrade
  • Completed
    November 20, 2022 at 9:13 AM
    Completed
    November 20, 2022 at 9:13 AM

    Maintenance has been completed successfully. Actual API downtime was 77 seconds.

  • In progress
    November 20, 2022 at 8:15 AM
    In progress
    November 20, 2022 at 8:15 AM

    Maintenance is now in progress

  • Planned
    November 20, 2022 at 8:15 AM
    Planned
    November 20, 2022 at 8:15 AM

    This is due to an upcoming hardware retirement scheduled by our cloud provider for a Redis cluster. During maintenance time, the platform will be inaccessible. All operations will resume immediately after the maintenance window.

<0.2% of API calls returned 502s and 504s
  • Resolved
    Resolved

    We have recovered and now operating normally.

  • Monitoring
    Monitoring

    We have put the fix in place and should be recovering.

  • Identified
    Identified

    We have identified the cause and working on mitigation strategy.

  • Investigating
    Investigating

    We are observing increased error rates on some API calls. Our team is investigating on this issue.

Partial outage of API calls
  • Resolved
    Resolved

    The Modern Treasury Ledger API has recovered.

  • Investigating
    Investigating

    The Modern Treasury Ledger API may be experiencing degraded performance. We are investigating.

Oct 2022

Elevated Error Rates
  • Resolved
    Resolved

    This issue was resolved.

  • Investigating
    Investigating

    We're experiencing an elevated level of API errors and are currently looking into the issue.

High API and application latencies
  • Resolved
    Resolved

    The issue is resolved and we've solved the underlying cause of the contention.

  • Monitoring
    Monitoring

    The issue is resolved and we've solved the underlying cause of the contention.

  • Identified
    Identified

    We have identified the cause of the contention. Working to mitigate and ultimately resolve this issue.

  • Investigating
    Investigating

    The Modern Treasury Ledger API may be experiencing degraded performance. We are investigating.

We are experiencing API and UI issues due to database contention.
  • Resolved
    Resolved

    We have addressed the underlying database contention and will follow-up with further improvements to the system to prevent reoccurrence.

  • Identified
    Identified

    We have identified the issue inside our database.

  • Investigating
    Investigating

    We are experiencing API and UI issues due to database contention.

Modern Treasury UI temporarily unavailable
  • Resolved
    Resolved

    All services are operational and healthy.

  • Identified
    Identified

    We had deployed a bad commit which fails to render UI. This downtime did not affect API calls.

Oct 2022 to Dec 2022

Next