Impact

All clients that relied on the mainnet Hiro API. This includes the Explorer, Stacks desktop and web wallets, the stacks CLI, and others. Requests on all endpoints to the mainnet API were unable to be served.

Root cause

After scouring our infrastructure, events, metrics, and third party services, we believe brief GCP network instability is what caused this short outage based on general issues on their status page around the affected time: https://status.cloud.google.com/incidents/7BkNzwMsik35A2V5j6vM#FxgErUaBEZS6pEeX6yiz

https://status.cloud.google.com/incidents/qv2VvxMoAzY8xgH4bsLQ

Action Items

We’ve updated our internal alerting to notify us of cloud-provider issues

Posted Nov 21, 2022 - 11:12 CST

Resolved

All services are back to operational , we are continuing to investigate the reason behind the outage and will post an update once we have more details

Posted Nov 01, 2022 - 09:00 CDT

Investigating

We're experiencing a major outage in the Hiro mainnet API and are currently looking into the issue.

Posted Nov 01, 2022 - 08:45 CDT

This incident affected: Services for Mainnet (Stacks Blockchain API), Sites (explorer.hiro.so), and Apps (Leather Desktop Wallet, Leather Web Wallet).