Skip to content
Hallway
0:00 / 0:00

#richardcook#resilienceengineering#normalizationofdeviance

Velocity NY 2013: Richard Cook, "Resilience In Complex Adaptive Systems"

O'Reilly

202K subscribers • 24 videos on Hallway

YouTube
  • Recommended by 5 people: 4 on Hacker News and 1 on Lobsters.
  • A classic. People kept recommending it from 2017 to 2021.
  • Presented by Richard Cook
  • May be out of date. Uploaded 13 years ago, in an area where tools change fast.
  • An opinion. It argues one side of a question engineers disagree on.

Intermediate. Assumes working knowledge.

47K views13 years ago5 recommendations

Cloud and DevOpsTools and practiceSRE and incidentsCode review and teamworkDeveloper productivity

Richard Cook explains how complex systems operate near the edge of failure and how human operators provide the resilience that keeps them running.

velocityconf.com/velocityny2013/public/schedule/detail/31784 Resilience In Complex Adaptive Systems: Operating At The Edge Of Failure Systems seem to run at the very edge of failure much of the time. The combination of high workload, limited resources, pressure for additional features and capability, and inherent software, hardware, and network fragility is a noxious kettle of stuff always about to boil over in the form of outages, degraded response, or functional breakdowns. For insiders the surprising thing about our systems is not that they fail so often but that they fail so rarely! This good performance in the face of adverse conditions is called resilience. An important conclusion from resilience studies is that it depends critically on human operators and their ability to anticipate and monitor the system, react to threats, and sacrifice some goals to protect others. This talk will introduce resilience and a model of system dynamics useful in analyzing failed and successful event management and offer an explanation for why our systems run at the edge of failure.

Why it's here

5 people recommend this

What people say. Highly recommended as a foundational talk on system safety, resilience engineering, and the abstract mechanisms behind the normalization of deviance.

sunils34on Hacker News • Recommends

On “How Complex Systems Fail (1998) [pdf]”

Richard Cook's writing and talks on systems have hugely shaped my thinking on systems. For those interested in more, he gave a highly entertaining talk on system resiliency here and introduces a helpful model. link to this video I also wrote a post summarizing how I think about Rasmussen’s model in startups and web applications. link
View on Hacker News

pdkl95on Hacker News • Recommends

On “Safety lapses hobble Los Alamos National Lab’s work on U.S. nuclear warheads”

> the staff had become “de-sensitized” to the risk of a serious accident. Yet again serious problems happen due to the Normalization Of Deviance. We really need to find a way to create working conditions that encourage correcting problematic situations immediately before the behavior becomes normalized. Regarding the management/other problems... I encourage everyone in ever industry that is ever involved with safety (i.e. most industries) to see Richard Cook's short talk about "Resilience in Complex Adaptive Systems"1. 1 link to this video
View on Hacker News

pdkl95on Hacker News • Recommends

On “Bedford and the Normalization of Deviance”

One of the most important talks I've ever seen is Richard Cook's "Resilience In Complex Adaptive Systems"1, in which he explains the fundamental abstract mechanism that causes the normalization of deviance. 1 link to this video
View on Hacker News

pdkl95on Hacker News • Recommends

On “Sydney Opal Tower: Thousands evacuated after 'crack'”

> But if you stop, they may continue increasing the dilution. There’s a normalization of deviance. For a very good explanation of the mechanism that causes this kind of normalization of deviance, I recommend Richard Cook's outstanding talk "Resilience In Complex Adaptive Systems"1. 1 link to this video
View on Hacker News

pdkl95on Hacker News • Recommends

On “Google Is Eating Our Mail”

> Indifference that does not cause a business problem remains indifference. More generally, indifference to small "trivial" problems eventually normalizes the deviant behavior1. Evil behavior doesn't happen in one step, it's end result of regularly being indifferent to "minor" problems2. For a very good explanation of why this happens, I recommend Richard Cook's excellent talk, "Resilience In Complex Adaptive Systems"3. 1 link 2 link 3 link to this video
View on Hacker News

pdkl95on Hacker News • Recommends

On “Facebook executives shut down efforts to make the site less divisive”

…jor - sometimes shocking1 - consequences that nobody would have approved if asked directly. Most of the time, nobody realizes just how deviant their situation had become. For a good explanation of the mechanism underlying the normalization of deviance (as an abstract model), I strongly recommend this2 short talk by Richard Cook. 1 link 2 link to this video ("Resilience In Complex Adaptive Systems")…
View on Hacker News

pdkl95on Hacker News • Recommends

On “When the Judge Distrusts Your Lawyers”

…concerns never goes away, so unless there is a proactive, explicit counter force, way to push back against that force, the system will inevitably be pushed toward failure. Therefor it's important to stop problems early when they are small. The magnitude of the counter force increases rapidly as behavior becomes increasingly deviant. 1 link 2 link to this video…
View on Hacker News

pdkl95on Hacker News • Recommends

On “Facebook to change user terms, limiting effect of EU privacy law”

Aka "Normalization of Deviance"1. Fix the minor problems now , or they become the new normal, eventually accumulating until a huge deviance from "normal" behavior isn't even noticed. The stories2 about airline pilots completely ignoring checklists and unusual alarms while trying to take off with the gust lock still engaged are a shocking example of strong this effect can be. For a very good explanation of how this happens, see Richard Cook's short talk "Resilience in Complex Adaptive Systems"3. 1 link 2 link 3 link to this video
View on Hacker News

di4naon Hacker News • Recommends

On “Everything is broken, and it’s okay”

…on quick sands with our leaky abstractions that try to fit into constantly changing reality) Nearly all the references i point to will tell you that this leakiness and this constant changing reality is exactly what makes them work in the real world. Now onto references. A lot of them can be found at link I particularly advise to start at link link to this video For more in depth, i particularly like link And this is basically a whole meta study of all the research and ideas that came to this field since WW2. Beware it is long link For an example of how things go wrong when things are correct but interact well together, i advise these two papers on space systems. (I link to Adrian blog on them…
View on Hacker News

pdkl95on Hacker News • Recommends

On “New technology has enabled cyber-crime on an industrial scale”

It's just a variation of the Normalization of Deviance. See this1 short talk by Richard Cook for a very good explanation of the mechanism that causes the transition from "robust" to "superfluous". 1 link to this video
View on Hacker News

______-on Hacker News • Recommends

On “Try This One Weird Trick Russian Hackers Hate”

Good comment about this very thing here: link > It's just a variation of the Normalization of Deviance. See this1 short talk by Richard Cook for a very good explanation of the mechanism that causes the transition from "robust" to "superfluous". 1 link to this video
View on Hacker News

catilacon Lobsters • Recommends

Resilience In Complex Adaptive Systems (Velocity NY 2013)
1View on Lobsters

pdkl95on Hacker News • Mentions

On “Bitcoin Exchange Had Too Many Bitcoins”

I should have used more specific language; I'm not trying to argue that short selling is "bad". The benefits - such as dampening a crashing market - are large enough to accommodate some risk. I'm primarily addressing the claim that, "There's absolutely nothing wrong with short selling". Ignoring small, acceptable risks is normalizing deviance1. 1 link to this video
View on Hacker News

pdkl95on Hacker News • Mentions

On “Theranos, CEO Holmes, and Former President Balwani Charged with Fraud”

…ore they grow into bigger, more damaging problems2. > rationalizations for unethical behavior Often, it doesn't even need to be rationalized. When deviant behavior is slowly normalized in small steps, the human mind tends to only see the "small, insignificant problems", and often skips3 over big changes that should be "obvious". 1 link 2 link to this video 3 link…
View on Hacker News

pdkl95on Hacker News • Mentions

On “Zuckerberg Takes Steps to Calm Facebook Employees”

> evil Hollywood et al popularize the misconception that evil is fantastic and done with intent . Most of the time evil is banal 1. The larger problems happen when the unremarkable, small deviances from acceptable behavior becomes normalized 23. 1 link 2 link 3 link to this video
View on Hacker News

pdkl95on Hacker News • Mentions

On “How a Pentagon Contract Became an Identity Crisis for Google”

…question is how small problems are handled? Compromising when the problem isn't a "big enough" ethical problem inevitably2 pushes that boundary back in a cycle of normalized small, easily avoided deviances. Instead, if a hard line is taken against small problems when they are still small , a lot of larger problems can be avoided. 1 link 2 link to this video…
View on Hacker News