API Outage
Incident Report for Hiro Systems
Postmortem

Impact

All clients that relied on the mainnet Hiro API. This includes the Explorer, Hiro desktop and web wallets, the stacks CLI, and others. Requests on all endpoints to the mainnet API were unable to be served.

Root cause

A regular maintenance activity on the API's database (Postgres vacuum) took longer than expected. The I/O resources this activity used caused queries to timeout, causing responses to fail.

Action Items

  • We are investigating updating our Postgres vacuuming tasks to have a lesser impact on production queries.
Posted Jul 05, 2022 - 16:27 UTC

Resolved
This incident has been resolved.
Posted Jul 04, 2022 - 02:26 UTC