Chainstack's Breach Started With a Default Grafana Password. The 30-Day Rotation Window Is the Bigger Problem

How does a node provider serving customers across a dozen chains get breached by a login screen?

Ask Chainstack. The company first disclosed a security incident on November 3, 2026. Seven days later it published a technical section dated November 10 that answers the question in uncomfortable detail. No software vulnerability was found. What got through was configuration.

Here's the relevant code. Or rather, here's the code that was never run.

Three failures, one path

The entry point was a development Grafana instance running with default administrator credentials still enabled. Grafana ships with a first-login prompt that forces you to set a real admin password. Someone skipped it, and the instance sat there reachable.

From there, the access path widened. An unauthenticated tool server with broad Kubernetes permissions formed the second link. Machine credentials connected development to production, which is the third link and the one that should scare operators most.

Chainstack's report lists what may have been exposed: account names, organization details, email addresses, and password hashes. On node credentials, the wording is careful. The company says it found no evidence they were read, then treats them as exposed anyway. That's the right call. Assume compromise, verify later.

The ask to customers is a three-part job. Change passwords. Rotate node credentials. Regenerate platform API keys where they're in use. Three different credential types, three different lifetimes, three different failure modes.

The dev box that ate production

Every team says it separates dev from prod. Almost none of them can prove it on demand.

Think about what a shared service account actually is. A single token with a broad role, checked into a pipeline months ago, reused by four jobs nobody documented. The environments look separate on a diagram. The credential says otherwise.

Grafana's own security documentation is unusually blunt about this. It warns that services sitting on the same local network as Grafana can be reached through its data-source proxy. The mitigation is network, not auth. Limit permitted destinations. Put firewall rules in front of sensitive internal services.

So there are three layers doing three different jobs. Authentication answers who gets in. Network boundaries answer what a tool can talk to. Permission limits answer what an account can do once it's talking. Chainstack's incident is what happens when all three are thin at the same time, in a place nobody was watching because it was labeled dev.

And that label is the real problem. A dev Grafana instance with default creds plus a Kubernetes-permissioned tool server isn't a dev environment. It's production with worse monitoring.

Rotation has a finish line almost nobody crosses

Now the part that matters for every operator reading this.

Chainstack's node-management docs describe a staged rotation. Generate new credentials. The old ones keep working. Move your applications over. Then, at some point, deactivate the originals.

Both sets stay live until that final step. There's no automatic expiry timer. I checked the description twice because that's the detail that turns a security fix into a security debt.

The overlap exists for a good reason. Yanking credentials out from under a running indexer at 2am is how you turn an incident into an outage. But the grace period only works if someone closes it.

Here's a sanity check worth running today:

curl -s -X POST https://api.chainstack.com/v1/nodes/YOUR_NODE_ID/credentials \
-H "Authorization: Bearer $OLD_KEY" \
-H "Content-Type: application/json" \
-d '{"action":"rotate"}'

If that returns 200, your old key is still alive. That's your completion record, or the lack of one.

One more gotcha buried in the docs. A node rotation doesn't change a multichain authentication token. So a team that rotates every node credential and calls it done may still be holding a working token from the incident window. That single sentence probably caught more than a few operators mid-remediation.

My take: any rotation flow without a hard expiry is a policy decision disguised as a convenience feature. Vendors should ask you to pick a TTL when you generate. Twenty-four hours, seventy-two hours, a week. Something the calendar can enforce when the on-call engineer forgets.

Second take: "no software vulnerability was exploited" is technically true and rhetorically useless. Default admin credentials on an internet-reachable monitoring instance isn't an unlucky break. It's a missing line in a config file. Ship it to testnet first, but also change the password on your dashboards.

What to watch now

Three concrete things over the next ninety days.

First, password hashes. Chainstack says hashes may have been exposed. Modern hashing makes cracking expensive, not impossible, and reused passwords are the cheap path. If your team reused a Chainstack-era password on any other service, that's the actual exposure, not the hash.

Second, ask your provider for a credential-type breakdown. Not "change your password." A list. Which tokens are live, which are revoked, which have an expiry. A vague advisory leaves developers guessing about automated jobs and service integrations, and guessing is where regressions come from.

Third, watch whether the revised controls actually get verified. The interesting question isn't whether Chainstack patched the Grafana instance. Of course it did. The question is whether one compromised service can still reach far beyond its intended role six months from now.

Clone the repo. Run the test. Then form an opinion. The test here's whether your old credentials still answer a request. If they do, you haven't finished the incident. You've just stopped reading about it.