Mainnet API Outage

Incident Report for Hiro Systems

Postmortem

Our API experienced an outage due to excessive load on the database caused by scraping activity. This led to multiple PostgreSQL queries taking several minutes to finish or not completing at all, resulting in increased response times and intermittent service interruptions for users. To mitigate the issue, we implemented measures to manage the load and restore normal service shortly after.

Posted Jul 24, 2024 - 21:21 CDT

Resolved

This incident has been resolved.

Posted Jul 24, 2024 - 20:53 CDT

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Jul 24, 2024 - 20:42 CDT

Identified

The issue has been identified and a fix is being implemented.

Posted Jul 24, 2024 - 20:26 CDT

Investigating

We're experiencing degraded performance in the Hiro Mainnet API and are currently looking into the issue.

Posted Jul 24, 2024 - 20:04 CDT

This incident affected: Services for Mainnet (Stacks Blockchain API) and Sites (explorer.hiro.so).