All systems operational

Platform/API

Operational

Ledgers API

Operational

Compliance API

Operational

Push to Warehouse

Operational

Marketing Website

Operational

Notice history

Dec 2023

Partial API/UI Outage (Create Payment Order)
  • Resolved
    Resolved

    This incident has been resolved, and only one Payment Order was affected.

    At 9:35 AM PST, API requests to the Payment Order Create endpoint for wire transfers to one of our banking partners would return a 500. This was due to a bug in a recent change to our routing number lookup service. We noticed the elevated 500 rate due to the retrying Payment Order at 9:40 AM PST and began our investigation. By 10:35 AM PST we deployed a fix for this bug and confirmed that the affected Payment Order was successfully created.

  • Monitoring
    Monitoring

    We have deployed a fix and are currently monitoring the result.

  • Identified
    Identified

    Creating Payment Orders via the UI or API is currently failing for less than 1% of requests for a small subset of customers and may result in a 500. We have identified the cause of the issue and are currently working on a fix.

Nov 2023

Elevated Error Rates in us0
  • Resolved
    Resolved

    Starting from 11/27 9:07 PM ledgers api experienced elevated API timeout errors which affected a single customer in cell us000. This issue was caused by outdated statistics of specific tables in the database that resulted in incorrect query plans. By 11/29 1:21 PM, we had updated and fixed the statistics and query plan and service had returned to normal. The issue has been resolved and the service is operating normally.

  • Monitoring
    Monitoring

    We implemented a fix and will monitor the results for a few hours.

  • Identified
    Identified

    We're experiencing an elevated level of API timeout errors for a single customer in one cell and are currently working on a fix to mitigate the database-related issue. We will update in 30 minutes.

Scheduled Maintenance: Database Upgrades on us0, us5
Partner Scheduled Maintenance: Wells Fargo Gateway API
  • Completed
    November 26, 2023 at 7:00 PM
    Completed
    November 26, 2023 at 7:00 PM

    Maintenance has completed successfully

  • In progress
    November 26, 2023 at 5:00 AM
    In progress
    November 26, 2023 at 5:00 AM

    Maintenance is now in progress

  • Planned
    November 26, 2023 at 5:00 AM
    Planned
    November 26, 2023 at 5:00 AM

    [Planned]​ Modern Treasury has been notified that the Wells Fargo Gateway API will be undergoing scheduled maintenance on ​Sunday, November 26th​ from ​12:00 AM ET​ to ​02:00 PM ET​ (14 hrs).
    The following Wells Fargo APIs will be unavailable:

    • Wire Payments

    Wire payments originated from Wells Fargo accounts during that window may be delayed. Modern Treasury will automatically hold and retry payments when the API becomes available. Balances and transactions will not be affected.

    Customers using file based transmission at Wells Fargo will not be affected. Accounts at other banking partners will not be affected.

Scheduled Maintenance: Push to Warehouse
Partner Scheduled Maintenance: Wells Fargo Gateway API
  • Completed
    November 19, 2023 at 11:00 AM
    Completed
    November 19, 2023 at 11:00 AM

    Maintenance has completed successfully

  • In progress
    November 19, 2023 at 7:00 AM
    In progress
    November 19, 2023 at 7:00 AM

    Maintenance is now in progress

  • Planned
    November 19, 2023 at 7:00 AM
    Planned
    November 19, 2023 at 7:00 AM

    Modern Treasury has been notified that the Wells Fargo Gateway API will be undergoing scheduled maintenance on Sunday, November 19th from 02:00 ET to 06:00 ET (4hrs).

    The following Wells Fargo APIs will be unavailable:

    • ACH Payments
    • Instant Payments
    • RTP Payments
    • Wire Payments

    ACH, RTP, and Wire payments originated from Wells Fargo accounts during that window may be delayed. Modern Treasury will automatically hold and retry payments when the API becomes available. Balances and transactions will not be affected.

    Customers using file based transmission at Wells Fargo will not be affected. Accounts at other banking partners will not be affected.

Oct 2023

Intermittent authentication errors
  • Resolved
    Resolved

    auth0 has confirmed the incident is resolved. The Modern Treasury web app is operational.

  • Monitoring
    Update

    auth0 has confirmed an issue causing increased error rates. We are continuing to monitor the situation, and will update every 60 minutes or as we have new information.

  • Monitoring
    Monitoring

    We have confirmed a connectivity issue with auth0. We are continuing to monitor the situation, and will update every 30 minutes or as we have new information.

  • Investigating
    Investigating

    We are seeing intermittent issues with authentication to the web app. The APIs are still operational. We are currently investigating.

Oct 2023 to Dec 2023