• +44(0)7855748256
  • bolaogun9@gmail.com
  • London

Engineers knew the Sampoong building was going to fall. Management kept it open.

On 29 June 1995, at 5:52 PM, the Sampoong Department Store in Seoul collapsed in under 20 seconds. 502 people died. 937 were injured. South Korean prosecutors launched a criminal investigation. What they found wasn’t a building that gave no warning. It was a building that sent warnings for three months and had every one of them ignored.

Platform engineering teams are running the same risk right now. The structure isn’t concrete. The warnings look like alerts.


The building that gave three months of warnings

Sampoong was built with serious structural compromises from the start. The design was originally for an office building. Midway through construction, the owner converted it to a department store without redesigning the load-bearing structure. Then he added a rooftop restaurant. Then he had 4 heavy air conditioning units moved into position by dragging them across the roof rather than craning them over the side. Each drag pass cracked the concrete a little more.

By April 1995, the fifth-floor ceiling was visibly cracking. Engineers flagged it. Management ordered cosmetic repairs and kept the store open.

By June, the cracks had widened. Structural engineers told management the building was at risk of collapse. Management disagreed and kept trading.

On the morning of 29 June, the overnight shift noticed the cracks had grown. The store’s facilities managers walked the fifth floor, observed the damage, conferred, and concluded the building was safe to operate. At 5:52 PM, the fifth-floor columns failed. The roof fell through the floor beneath it, then the next, then the next. The building pancaked in under 20 seconds.

The engineers who had been raising the warnings were not in the building when it collapsed. They’d already left.

The action here isn’t a remediation task. It’s an audit. Pull every open alert or finding in your environment older than 30 days. For each one, identify who last reviewed it, when, and what the documented reason for deferral was. You’re looking for a pattern of dismissal, not bad alerts.


Warning decay: how repeated alerts lose their authority

The Sampoong collapse belongs to a family of disasters with one shared characteristic: the warning that eventually proves fatal is rarely the first one raised. It’s the fifth. The twelfth. The one that’s been on the register so long nobody on the current team remembers who raised it.

Diane Vaughan studied this mechanism in her post-mortem of the Challenger disaster. O-ring erosion had been observed and documented in multiple previous launches. Engineers raised concerns. Each time, managers decided the risk was acceptable. Over time, flying with compromised O-rings became a known feature of the programme rather than an active warning sign. Vaughan called this normalisation of deviance: repeated exposure to a warning, without consequence, trains an organisation to treat the warning as meaningless.

Cloud operations runs this exact process. Every platform team carries a class of alerts mentally filed as background noise. They keep firing. Someone keeps closing them. Nobody is actively deciding they’re safe; the team is just busy and the alert has been there forever.

The IaC parallel is concrete. Terraform drift findings sit in CSPM dashboards for months. Each sprint, someone looks at the list and defers remediation. Nothing bad happens. The finding gets buried under newer ones. By month four, the drift is significant, the finding is invisible, and nobody on the current team knows it exists. The drift didn’t stop growing because nobody fixed it. It grew.

The action: implement a decay escalation on unactioned findings. Configure your tooling so a finding unactioned for 14 days escalates to the next severity tier. Unactioned for 30 days, it goes to a named owner with mandatory review. The alert doesn’t get quieter with age. It gets louder.


The walk-through problem

The Sampoong managers didn’t ignore the building. They inspected it. That’s what makes this case worth studying.

They physically walked the fifth floor. They looked at the cracks. They discussed what they saw. And they concluded the building was safe to open. This wasn’t negligence in the conventional sense; it was a process that was architecturally incapable of reaching the correct answer. There were no defined pass/fail criteria. There was no independent reviewer. There was a powerful commercial incentive to find the building acceptable. The inspection confirmed what the managers needed it to confirm.

Platform engineering runs this risk constantly. When a senior SRE does a quick check on an alert category and decides it’s noise, that’s a walk-through. It feels like an informed decision. It’s a pattern-match against past experience, applied without current data. The SRE has seen this alert before. Last time it resolved itself. The conclusion is therefore that it will resolve itself again.

IBM’s 2023 Cost of a Data Breach Report found the average time between a breach occurring and its detection was 204 days. Those weren’t silent breaches. The signals were there. They looked acceptable to whoever reviewed them.

Here’s the uncomfortable part: the teams most at risk are the most experienced ones. Senior engineers pattern-match faster. Their triage is quicker. And quick triage on a familiar-looking alert is often experienced dismissal dressed as assessment.

The action: define pass/fail criteria before any alert review, not during it. Before your team reviews a persistent finding, commit in writing to what observable condition would change the conclusion. One sentence: “We will escalate this if [specific threshold].” Write it before you look at the data. Committing to criteria in advance breaks the confirmation bias loop.


More signal, faster decay

The current generation of AI-assisted cloud security tools is genuinely good at finding things. Every anomalous API call, every configuration deviation, every unusual authentication pattern, every exposed credential, every compliance gap. They generate findings at a rate no platform team can fully action.

This creates a problem that most teams don’t name clearly. More signal should improve security posture. In practice, it accelerates normalisation of deviance. When finding volume triples but team headcount stays flat, triage speed increases and review depth decreases. Your AI tool is filing crack reports every morning. After 3 weeks of unactioned reports, the team stops reading them carefully.

There’s a specific configuration failure baked into most AI monitoring tools by default: findings that aren’t actioned simply age in the backlog. A critical CSPM finding raised in January that hasn’t been remediated by March doesn’t escalate. It sits in the list. By May, it’s buried 400 findings deep. An unactioned finding should escalate in severity, not decay into irrelevance.

The action: audit the ageing behaviour of every AI monitoring tool in your stack. For each one, answer this question: what happens to a critical finding that isn’t actioned in 30 days? If the answer is “nothing,” the tool’s default configuration is working against you. Change it. Most enterprise tools support escalation rules on finding age. Most teams haven’t turned them on.


When nobody owns the warning

Post-collapse investigators found something beyond structural failure at Sampoong: an organisational design failure. The engineers who raised the warnings reported into a different part of the business from the managers who dismissed them. The engineers had no authority to close the building. The managers had no obligation to consult the engineers. The structure made a fatal decision organisationally straightforward.

Platform teams have this gap by default. Security tooling sits with the security team. Alerts route to a SIEM or ticketing system. Platform and product teams own remediation but rarely own the alert in any meaningful sense. When a finding is dismissed, who carries that decision? In most organisations, nobody does. The ticket is closed. The decision is invisible in any audit trail.

A warning nobody owns is a warning nobody is accountable for ignoring.

The action: every critical or high-severity finding in your environment should have a named owner. Not a team. A person. Their name is on the finding. When it’s closed without remediation, their name is on the risk acceptance. This single change makes the invisible visible. It doesn’t prevent deferral; it makes deferral a named human decision rather than a process outcome.


The commercial pressure test

The Sampoong managers had real commercial reasons to keep trading. The building generated revenue every hour it was open. Closure meant lost sales, reputational damage with loyal customers, and the difficult task of explaining why a major department store was suddenly shuttered. Concrete, quantifiable costs. The cost of collapse was probable but not yet certain. The economics looked defensible right up until they weren’t.

Every engineering team runs this calculation. A critical alert requires remediation time. Remediation time displaces feature work. Feature work has a committed deadline. The alert has been there for weeks and nothing bad has happened yet. Deferral is economically rational at the sprint level, using exactly the same logic that kept Sampoong open at the daily operations level.

Both calculations ignore the cost structure of the eventual failure. The economic logic of deferral only holds if the deferred risk doesn’t materialise. The moment it does, the calculation flips entirely.

The action: require a written risk acceptance for every deferred critical alert. Three mandatory fields: the specific risk being accepted, the business reason for acceptance, and the date or condition at which the decision must be reviewed. If the person deferring won’t write those 3 sentences, they’re not managing risk. Build the process so that hope isn’t a sufficient substitute for a decision.


The Warning Decay Framework

6 principles for managing persistent warnings in cloud and platform engineering environments, drawn from what Sampoong and comparable structural failures reveal about how organisations lose the ability to hear warnings they’re actively receiving.

1. Timestamp every dismissal. Record when an alert was first raised, when it was reviewed, and who made the call to defer. A human-readable decision log, not an automatic ticket timestamp. The format: “On [date], [name] reviewed this finding and deferred it because [reason]. Next mandatory review: [date].” If the reviewer can’t complete those fields in plain language, they haven’t made a decision.

2. Separate triage from dismissal. Triage: I’ve assessed this risk, I understand it, and I’m consciously prioritising other work. Dismissal: this alert is annoying and I want it gone. Most teams conflate these and produce the same outcome: a closed ticket. Build a process that requires the assessor to explicitly state which one they’re doing. The language forces clarity that silence doesn’t.

3. Give engineers veto rights. The Sampoong structural engineers had no authority to close the building. Your platform and security engineers need a formal escalation path for warnings they believe are being dismissed rather than triaged. That path must reach someone with genuine closure authority. And using it must carry zero career risk. If using it feels like insubordination, nobody uses it.

4. Define pass/fail criteria before the review. Any alert review without pre-stated criteria defaults to confirmation bias. Before reviewing a persistent finding, commit in writing to what observable condition would change the conclusion: “We will escalate this if [specific threshold].” One sentence. Written before you look at the data. The Sampoong managers walked the building; they just never defined in advance what would make them close it.

5. Escalate unactioned findings, don’t expire them. A critical finding not actioned in 30 days should be harder to ignore, louder, more visible. Configure your monitoring tools so that unactioned high and critical findings escalate in severity over time. Most enterprise tools support this. Most teams haven’t turned it on because the default configuration was built for throughput, not safety.

6. Make the commercial logic explicit. When a critical alert is deferred for business reasons, document the reasons. “We are accepting the risk of [specific outcome] because [specific business reason], and this decision stands until [specific date or condition].” If the person making the call won’t write that sentence, they’re not making a business decision. They’re hoping. Build the process so hoping isn’t enough.


The only question that matters

Sampoong’s structural engineers knew the building was going to collapse. They said so. They were overruled by managers who faced a clear, immediate cost for acting on the warning and an uncertain future cost for ignoring it. The building didn’t fail because nobody noticed. It failed because the people who noticed were overruled by people who needed the warnings to be wrong.

Your monitoring stack has findings that have been open since January. Some of them have been reviewed, deferred, and forgotten. The alert is still firing. The risk is still accumulating. The store is still open.

At what point does your team’s triage process become a walk-through?

The Warning Decay Framework and supporting templates are published in full at blog.ogunlana.net. If your platform team is running AI-assisted monitoring without a formal warning lifecycle process, that’s where to start.

Leave a Reply

Your email address will not be published. Required fields are marked *