Move Fast, Fix It In Prod: A Full Breach Trace of the Startup Archetype

Archetype C is the Series B or Series C technology company. Fifty deploys a day is unremarkable. Infrastructure is declarative and lives in version control; a GitOps controller reconciles the cluster to whatever the repository says. Identity comes from a software certificate authority, credentials are short-lived tokens, and there is no security operations centre: you build it, you run it, you secure it.

It does real things better than the enterprise it gets compared against. The team is fused rather than layered, so there is no handoff between the person who notices something and the person who can fix it. Remediation is a commit — in the trace below, about three minutes from recognition to full rollback, with nobody logging into a firewall or killing pods by hand.

The cost is stated plainly in the architecture. Trust is established once, on first use: code that passes CI/CD is trusted at runtime indefinitely. Dashboards are treated as ground truth. Availability wins every tie. None of these choices is stupid; each is correct for the failure mode the team was worried about, which was downtime.

Scored against the Octagon's eight axioms (pass, partial or fail), Archetype C passes one (Axiom 2), fails five outright (Axioms 4, 5, 6, 7, 8) and meets two only partially (Axioms 1 and 3), the same score as the overview's violation matrix. It passes verifiable policy (2), because GitOps makes every policy change replayable from Git history. It fails continuous verification (4), bounded authority (5), Byzantine fault tolerance (6), epistemic integrity (7) and bilateral symmetry (8), because the resource never proves its state to the requester. It meets no intrinsic trust (1) only partially, because pods authenticate with ServiceAccount tokens but share one cluster network, and mediation (3), because the gateway mediates but nothing does pod to pod.

List what the pipeline holds. Production secrets: database passwords, API keys, signing material. The ability to modify Terraform state, Kubernetes manifests and IAM policy. Network reachability into production. The ability to merge on a green build without human review. Compromising it is equivalent to compromising the infrastructure.

Now list the scrutiny it gets. Production is monitored, alerted on and reviewed. The pipeline is internal tooling, configured once and then left alone because it works. That asymmetry is not an oversight: every control placed on the pipeline is friction on the metric the company runs on. The result is verify-once: exactly one verification gate, at build time, with nothing downstream re-checking it. Whoever gets past it is inside with the authority of whatever they replaced.

A developer needs a small utility library and installs one whose name differs from a well-known package by a single character, a typosquat, fictional here. Plausible documentation, believable version history, a few hundred downloads. The more common 2025 route is worse: a hijacked maintainer account pushes a malicious version of a package the team already trusts, as in the Shai-Hulud npm worm. Either way the package carries two payloads. Its install hook runs at install time, on the developer's laptop and again on the CI runner. The library code itself does nothing under test and activates only when it sees production configuration.

The pull request opens and CI installs dependencies. The install hook reads the runner's environment and sends out what the pipeline holds: the registry publishing token and the cloud deploy key. Nothing alarms, because the build is green. This is the step real attacks made the main event in 2025: the compromise of tj-actions/changed-files, an action used by more than 23,000 repositories, dumped CI secrets into workflow logs, and Shai-Hulud used an install hook to steal tokens and republish itself. The test suite passes because the library code is inert under test, and the merge lands. The GitOps controller deploys the new containers with no attestation at admission: the only verification that ever happened is that CI built it, and CI was not what was subverted. The provenance record is accurate and useless.

The pod boots and requests an identity. The API server issues a valid bound ServiceAccount token, nominally one hour (and often extended for legacy compatibility). There is no workload attestation and no code-hash check, so nothing in the issuing path can tell this pod from the legitimate one. It claims to be the billing service and gets exactly the billing service permissions, because authority was scoped to the service, not to the code.

The code activates and scrapes the internal user database over thousands of API calls. The gateway checks the token on each one: valid, unexpired, passed. Past the gateway there is no pod-to-pod mediation, so the same identity is good for lateral movement. Nothing here is a bypass.

The mechanism is not that speed makes people careless. It is that commit-to-production is a single automated channel with no second evaluation point, and the channel does not care what it carries. Automation amplifies risk by removing the intermediate states where a different decision could have been made.

The properties form a loop. Velocity forbids friction at the build gate, which forces verify-once, which is what makes supply-chain injection viable. Fused teams and GitOps then deliver excellent detection and recovery, both of which begin after an injection they cannot prevent.

The degradation policy compounds it. When the burst trips the circuit breaker, the system rate-limits rather than terminating the pod, because terminating might affect a paying customer. Exfiltration continues at reduced rate for about eight minutes. Graceful degradation is right when the threat is load. When the threat is theft, slowing the attacker is a discount, not containment.

Nothing was missing from the telemetry. The platform recorded the traffic spike, the breaker trip and the rate-limit plateau, and graphed all three accurately. The instrumentation was faithfully reporting what the compromised system said about itself.

Nobody was reading it, because the dashboards answer an availability question. They are alerted around latency, error rate and saturation, and the alert that fired was a performance signal. No alert existed for the signal that mattered: a service querying a datastore that is not in its dependency graph, visible from the first second.

A person detected the attack. The alert pulled the on-call engineer, an SRE and the engineering manager into one channel, and within about three minutes one of them asked why the new billing pod was querying the user-profile database instead of the ledger. That is the only detection mechanism in this architecture, and it depends on the right person being awake.

Start from the constraint. A team at this stage will not adopt an enterprise control set, and any control that inserts a human approval gate into the deploy path will be quietly disabled within a quarter by someone under release pressure. The question is which controls buy runtime verification without buying deployment friction.

The missing gate has to sit on the dependency itself, and that part is cheap. Image signing stops images that never came through CI; it does not stop a malicious dependency CI built in, so on its own it would have passed this attack. What the dependency gate needs: a committed lockfile installed with npm ci, install scripts off by default (pnpm 10 does this, with an allowlist for packages that need them), a minimum release age before new versions are installable, and provenance or trusted-publisher checks where the registry offers them. In the pipeline, use GitHub Actions OIDC instead of stored cloud keys and pin third-party actions to a commit SHA, which is what the tj-actions victims lacked.

A second fix is structural. A pipeline runner that can publish packages, change IAM policy and deploy to production on one token is a single credential that owns the company. Split that authority by purpose and expire it in minutes, so the token the install hook steals is scoped and already dead. Cryptographic workload identity for the runner is the durable version, but it establishes who the runner is, not what the runner is running.

Then add runtime verification and containment that actually removes the bad image. Runtime anomaly detection as an eBPF DaemonSet needs no code changes and watches syscall and network behaviour independently of what the application reports. When it says a workload is outside its profile, isolate the pod with a deny-all network-policy label, scale the deployment to zero and revert the image digest in Git; deleting the pod alone just lets the ReplicaSet start the same image again. Network policy, staged first (Cilium's policy audit mode or Calico's staged policies, since upstream Kubernetes NetworkPolicy has no audit mode), removes lateral movement without a cutover that breaks production. Finally, rotate every secret the pipeline held, because the install hook already has them.

Count the verification gates between a developer typing a package name and that package executing with production authority. If the answer is one, and it sits at build time, the blast radius of getting past it is everything downstream: all of production, holding a valid identity, until somebody looks at the right graph.

Fast detection and recovery begin after the injection has already succeeded. The fix is not slower deployments; it is a second verification on the dependency and a third at runtime, in a trust domain the deploying pipeline does not control. If an architecture cannot say when a given trust decision is re-evaluated, that decision is the attack surface.