More than a decade ago, Netflix made a decision that looked completely irrational from the outside: it started deliberately breaking its own production infrastructure.

It didn’t do this because it was suffering constant outages or because it wanted to test its engineers. It did it because it understood a reality that almost every major cloud platform shares today: failures are inevitable.

The question was never whether a server would stop working, but what would happen when it did.

That idea gave birth to Chaos Monkey, probably the most influential reliability engineering project in modern computing. Fifteen years later, its principles are more alive than ever, and they have gone on to shape managed services at AWS, Azure, and Google Cloud, as well as how distributed applications are designed today.

Chaos Engineering in 20 seconds

• Netflix created Chaos Monkey to randomly shut down production servers.
• The goal was to find weaknesses before users did.
• The initiative evolved into platforms capable of simulating full-region outages, latency spikes, or network problems.
• Today, AWS, Kubernetes, and many companies build these techniques into their everyday reliability engineering.
• The rise of AI and autonomous systems will make testing controlled failures even more important.

For years, high availability meant building infrastructure “that would never fail.”

Today we know that premise is impossible.

Disks fail.

Switches stop responding.

Databases get locked.

Cloud regions can run into trouble.

Even perfectly written software contains bugs that only show up under specific conditions.

The difference between a robust architecture and a fragile one isn’t about avoiding all of those problems.

It’s about how the system responds when they happen.

What is Chaos Engineering, really?

Chaos Engineering is a discipline built around introducing controlled failures into a system to verify that it keeps working correctly.

It isn’t about breaking servers for fun.

Every experiment starts from a hypothesis.

For example:

“If we lose a Kubernetes node, users shouldn’t notice any disruption.”

Or:

“If latency increases between two microservices, the application should keep responding thanks to timeout and retry mechanisms.”

Then that failure is deliberately triggered.

If the system keeps working, the hypothesis was correct.

If it doesn’t, you’ve just found a problem… before your customers did.

That’s precisely the enormous value of Chaos Engineering.

How Chaos Monkey was born

It all started after Netflix migrated from its traditional infrastructure to Amazon Web Services (AWS).

The architecture had gone from a handful of servers to thousands of distributed instances.

That new scenario brought a completely different problem.

It was no longer realistic to assume every server would always be available.

Every day, something would fail.

The solution was extraordinarily simple.

Build a program that randomly picked virtual machines and shut them down during business hours.

Why specifically during the day?

Because all the engineers were at work.

If something broke, they could see it immediately.

If the application stopped responding, the problem wasn’t Chaos Monkey.

It was the architecture.

From one monkey to a whole army

Netflix soon discovered that shutting down servers was just one of many possible types of failure.

That’s how the Simian Army was born.

Each tool simulated a different problem.

Chaos Monkey

The best known of the bunch.

It terminated EC2 instances at random to check whether the application could survive without them.

Latency Monkey

It introduced artificial delays between services.

It was especially useful in microservices architectures, where a few extra tens of milliseconds can multiply quickly.

Chaos Gorilla

It simulated the complete loss of an AWS Availability Zone.

In other words, an entire physical zone of the data center would stop existing.

The application had to keep running from another zone.

Chaos Kong

It took the experiment to the extreme.

It simulated the complete disappearance of an entire cloud region.

It was a critical test for validating multi-region strategies.

Other “monkeys”

Netflix later built tools designed to detect:

• underused resources;
• misconfigurations;
• security issues;
• unexpected dependencies;
• orphaned services.

Over time, the company stopped even talking about “monkeys.”

The goal was no longer to cause failures.

It was to run scientific experiments on how the system behaves.

Modern Chaos Engineering: ChAP and FIT

Netflix gradually replaced the Simian Army with far more sophisticated platforms.

Among them:

• ChAP (Chaos Automation Platform).
• FIT (Failure Injection Testing).

Instead of launching random attacks, they let engineers define very specific experiments.

For example:

“What happens if the authentication service responds two seconds later?”

Or:

“How does losing 25% of a database’s traffic affect the system?”

Every experiment has:

• a hypothesis;
• metrics;
• success criteria;
• automatic mechanisms to stop the test if unexpected effects appear.

It’s exactly the same scientific method applied to infrastructure.

AWS turned the philosophy into a service

Netflix’s ideas ended up directly shaping the cloud industry.

Today Amazon offers AWS Fault Injection Service (AWS FIS), a managed service that lets teams run these kinds of experiments without building their own tooling.

With AWS FIS, teams can simulate scenarios such as:

• shutting down EC2 instances;
• loss of connectivity;
• increased latency;
• extreme CPU usage;
• storage failures;
• disruptions on Amazon ECS;
• incidents in Amazon EKS clusters;
• database degradations.

The difference compared with 2011 is enormous.

Netflix had to build the entire platform from scratch.

Today, any organization can get started within a few hours.

AWS itself is not immune to the failures Chaos Engineering tries to catch ahead of time. A single race condition in an internal DNS automation system was enough to take down the us-east-1 region for hours last October, an incident later dissected in detail in AWS’s own post-mortem, and one that reads like a real-world argument for exactly the kind of controlled failure testing described here.

Kubernetes has made this philosophy even more important

In a virtual-machine-based environment, components changed relatively little.

With Kubernetes, it’s the exact opposite.

Pods appear and disappear constantly.

Containers get recreated.

Nodes join and leave the cluster.

Load balancers constantly shift traffic around.

The infrastructure is dynamic by design.

That makes Chaos Engineering an even more valuable tool.

It’s no longer enough to check that an application works.

You have to prove it keeps working while the environment keeps changing.

AI adds a new layer of complexity

The next big leap will come with AI agents.

More and more platforms will hand decisions over to autonomous systems.

Load balancing.

Auto-scaling.

Resource allocation.

Incident response.

Energy optimization.

In that context, it won’t be enough to validate just the software anymore.

Teams will also need to check how the AI itself responds when it receives incomplete, contradictory, or wrong information.

Over the next few years, we’ll likely see a new generation of Chaos Engineering tools built specifically to validate intelligent agents.

How to get started with Chaos Engineering

You don’t need to be Netflix.

A good Chaos Engineering program usually starts with very simple experiments.

Some examples:

What matters is moving forward step by step.

Never start with extreme scenarios.

And measure absolutely everything.

Resilience doesn’t show up when things go wrong

There’s an idea that sums up this entire discipline perfectly.

Resilience isn’t proven in architecture diagrams.

It’s proven when the infrastructure starts to break.

Chaos Engineering completely changed how we think about high availability.

Instead of asking:

“Will it work?”

The question became:

“What will happen when it stops working?”

And that’s probably still the best question any infrastructure team can ask itself in the era of cloud, Kubernetes, and artificial intelligence.

Frequently Asked Questions

Is Chaos Engineering just about breaking servers?

No. It’s about running controlled experiments to see how a system responds to predefined failures.

Is it only necessary for large companies?

No. Tools like AWS Fault Injection Service, LitmusChaos, or Chaos Mesh now make it possible to get started even on small infrastructures.

Are these experiments run in production?

Yes, though always in a controlled, limited way, with automatic mechanisms to stop the experiment if unexpected risks appear.

Why will it matter even more with AI?

Because autonomous systems will be making decisions across complex infrastructure. Teams will need to verify not just that the software works, but that those agents react correctly when real failures occur.