Stacks blockchain testnet slow block progression
Incident Report for Hiro Systems
Postmortem

Impact

All clients that relied on the testnet Hiro API. This includes the Explorer, Stacks desktop and web wallets, the stacks CLI, and others. Requests on all endpoints to the testnet API were still being served, but we showing the same block and transactions for some time.

Root cause

This was determined to be caused by a Testnet Bitcoin block storm. See a writeup on what these are here:

https://blog.lopp.net/the-block-storms-of-bitcoins-testnet/

After the storm subsided, testnet mining resumed and blocks started progressing as normal.

Action Items

Posted Nov 21, 2022 - 17:16 UTC

Resolved
This incident has been resolved, testnet Stacks blocks are progressing as expected.
Posted Nov 01, 2022 - 02:45 UTC
Monitoring
The Stacks blockchain testnet is progressing at a decreased rate due to a testnet Bitcoin block storm. Once the storm passes, testnet Stacks block progression should increase soon after.

You can follow the testnet Bitcoin block progression here: https://www.blockchain.com/explorer/blocks/btc-testnet
Posted Nov 01, 2022 - 02:36 UTC
This incident affected: Sites (explorer.stacks.co), Apps (Hiro Desktop Wallet, Hiro Web Wallet), and Services for Testnet (Stacks Blockchain API).