Scalable Human Blog

TL;DR:
Load shedding is a last-resort strategy for keeping systems alive under extreme stress. When your system is overwhelmed, it’s better to serve fewer users well than to crash trying to serve everyone.

When your software system starts to sink under heavy load, what do you do? Just like sailors tossing cargo overboard to stay afloat, engineers sometimes have to make hard calls to drop low-priority work. This technique is called load shedding, and while it’s not glamorous, it’s often the difference between graceful degradation and total failure.

In this post, we’ll explore what load shedding is, how it differs from other overload protection strategies, and how companies like Netflix and Google use it to keep their systems resilient under pressure.

What Is Load Shedding?

At its core, load shedding is about intentionally dropping some requests to preserve the health of your system. It’s the emergency brake you pull when all else fails—when rate limiting and backpressure just aren’t enough.

Unlike proactive strategies like rate limiting (which caps traffic before it hits your system) or backpressure (which slows down the senders), load shedding kicks in when your system is already in distress. It’s a reactive measure that helps you avoid cascading failures and total collapse.

As Google’s SRE book explains, cascading failures often begin when one component becomes overloaded and slows down, causing other components to back up and eventually fail. Load shedding helps prevent this domino effect by failing fast and selectively.

How Load Shedding Works in Practice

Load shedding is all about making hard choices quickly. You need to identify which requests matter most and drop the rest—without hesitation.

For example, in a Java-based system, you might do something like this:

if (systemLoadMetric.getCpuUsage() > 90 || activeConnections.get() > MAX_SAFE_CONNECTIONS) {
    if (!request.isPriority()) {
        return ResponseEntity.status(503).body("System overloaded");
    }
}
// Process only priority requests when overloaded

Or at the queue level:

if (taskQueue.remainingCapacity() < EMERGENCY_THRESHOLD) {
    taskQueue.poll(); // Drop oldest low-priority task
    logger.warn("Load shedding: dropped task due to overload");
}

This approach ensures that critical traffic—like paying customers or essential API calls—gets through, while less important work is sacrificed.

Netflix has taken this concept even further. In their engineering blog on prioritized load shedding, they describe how they built a system that evaluates every incoming request based on its priority. High-value traffic (e.g., playback starts) is preserved, while lower-priority services (e.g., logging or recommendations) are throttled or dropped entirely during overload.

This level of granularity allows Netflix to maintain a seamless experience for users, even when parts of their infrastructure are under extreme stress.

Accepting Imperfection

Perhaps the hardest part of load shedding isn’t technical—it’s psychological. As engineers, we want to build systems that are always available. But the truth is, perfect availability is a myth.

Trying to serve 100% of traffic during a crisis often leads to serving 0%. Load shedding forces us to accept that it’s better to serve 80% of users well than to fail everyone.

The key is to plan ahead. You need:

• Robust monitoring to know when you’re in trouble
• Clear business rules to decide what gets dropped
• Fast failure paths to avoid clogging your system with doomed requests

When done right, users may not even notice that your system was on the brink.

Key Takeaways

• Load shedding is a reactive strategy that kicks in when your system is already overwhelmed.
• It differs from rate limiting and backpressure by acting as a last line of defense.
• Selective failure is crucial: prioritize critical traffic and let go of the rest.
• Real-world systems like Netflix use prioritized load shedding to maintain reliability under pressure.
• Accepting imperfection is part of the strategy: better to degrade gracefully than crash completely.

Conclusion

Load shedding isn’t about giving up—it’s about surviving. When your system is in crisis, letting go of the least important work can be the smartest move you make. It’s a strategy rooted in realism, not defeat.

So the next time your system starts to sink, remember: you don’t have to save every request to stay afloat. Just the right ones.

Have you implemented load shedding in your systems? Share your experiences or questions in the comments—we’d love to hear how you’ve kept your ship above water.

📚 Further Reading & Related Topics
If you’re exploring load shedding in software systems, these related articles will provide deeper insights:
• How to Relieve Hotspots with Skewed Workloads – This article discusses techniques to mitigate uneven load distribution across systems, a key challenge addressed by load shedding strategies.
• What Is Consistent Hashing? – Learn how consistent hashing helps distribute requests more evenly across nodes, minimizing overload and reducing the need for load shedding.
• Horizontal Scaling vs Vertical Scaling – Understanding these scaling strategies provides context for when and why load shedding becomes necessary in high-traffic software systems.