Explorer Intermittent Failure
Incident Report for Hiro Systems


Between Jan 27, 2023 10:50 UTC and  Jan 29, 2023 9:16 UTC, all requests to explorer.stacks.co intermittently displayed a blank screen.

Root Cause

It was discovered JS bundles were missing in a number of replica pods resulting in failed requests.  Tracking back to the root cause, a version of the Explorer utilizing new JS bundles was inadvertently deployed into production.  The Explorer recovered soon after the correct version was re-deployed.

Action Items

  • Update CI/CD to identify and prevent deploying non-production ready versions of the Explorer
  • Update monitoring tool to more accurately detect request failures.
Posted Feb 09, 2023 - 10:44 CST

The Hiro Explorer is experiencing intermittent failures.
Posted Jan 31, 2023 - 14:49 CST