API Outage

Incident Report for Hiro Systems

Postmortem

Impact

All clients that relied on the mainnet Hiro API. This includes the Explorer, Hiro desktop and web wallets, the stacks CLI, and others. Requests on all endpoints to the mainnet API were unable to be served.

Root cause

A regular maintenance activity on the API's database (Postgres vacuum) took longer than expected. The I/O resources this activity used caused queries to timeout, causing responses to fail.

Action Items

  • We are investigating updating our Postgres vacuuming tasks to have a lesser impact on production queries.
Posted Jul 05, 2022 - 12:27 EDT

Resolved

This incident has been resolved.
Posted Jul 03, 2022 - 22:26 EDT