Scalable Human Blog

TL;DR:
Backpressure and rate limiting are essential tools for building resilient systems. They help your application stay responsive under load by slowing things down before things break.

When systems get overwhelmed, bad things happen—timeouts, crashes, angry users. Imagine a highway where cars are entering faster than they can exit. Eventually, traffic grinds to a halt. Software systems face the same problem when they accept more work than they can process. The solution? Teach your system when to say “slow down.”

In this post, we’ll explore how backpressure and rate limiting keep your system healthy by controlling the flow of requests. You’ll learn how they differ, when to use them, and how to implement them effectively.

What Is Backpressure?

Backpressure is your system’s way of saying, “I’m full, please stop sending me more work for now.” It’s a feedback mechanism that helps upstream components adjust their pace based on the system’s current capacity.

Think of it like a restaurant kitchen. If orders come in too fast, the chef can’t keep up. Instead of letting the order queue grow infinitely (and customers wait forever), the kitchen signals the front of house to pause taking new orders until it catches up.

In code, this might look like:

if (taskQueue.size() > MAX_QUEUE_SIZE) {
    return ResponseEntity.status(503).body("System overloaded, try again later");
}
taskQueue.offer(newTask);

This pattern prevents your system from spiraling into failure due to unbounded queues. As Martin Thompson explains, applying backpressure is about “preserving the integrity of the system” by forcing producers to slow down when consumers can’t keep up.

Backpressure works best when clients can retry later or when you have control over both ends of the communication. It’s commonly used in streaming systems, message queues, and reactive programming.

What Is Rate Limiting?

Rate limiting is like the bouncer at the club door. It doesn’t care how many people want in—it only lets in a certain number per minute. This proactive approach protects your system by capping the number of incoming requests.

Let’s say your API can safely handle 100 requests per second. Using Resilience4j’s RateLimiter, you can enforce that limit:

RateLimiter rateLimiter = RateLimiter.of("apiLimiter", 
    RateLimiterConfig.custom()
        .limitForPeriod(100)
        .limitRefreshPeriod(Duration.ofSeconds(1))
        .timeoutDuration(Duration.ofMillis(0))
        .build());

if (!rateLimiter.acquirePermission()) {
    return ResponseEntity.status(429).body("Rate limit exceeded");
}

Or with Guava:

RateLimiter rateLimiter = RateLimiter.create(100.0); // 100 permits per second
if (!rateLimiter.tryAcquire()) {
    return ResponseEntity.status(429).body("Rate limit exceeded");
}

Rate limiting is especially useful at system boundaries—like APIs—where you want to protect internal resources from being overwhelmed by external clients. It’s also a great defense against abuse and denial-of-service attacks.

Backpressure vs. Rate Limiting: What’s the Difference?

While both techniques aim to prevent overload, they operate differently:

• Backpressure is reactive. It kicks in when the system is already under strain.
• Rate limiting is proactive. It enforces a fixed ceiling on incoming traffic.

Backpressure is great for internal flows where you can propagate the “slow down” signal upstream. Rate limiting is ideal at the edges of your system, where you want to control how much traffic gets in to begin with.

Used together, they form a powerful duo: rate limiting keeps the floodgates from opening too wide, and backpressure ensures that internal components don’t get overwhelmed.

Key Takeaways

• Backpressure helps your system say “not now” when it’s overwhelmed, avoiding infinite queues and crashes.
• Rate limiting enforces a hard cap on incoming traffic, protecting your system before it gets overloaded.
• Backpressure works best when clients can retry or when you control both producer and consumer.
• Rate limiting shines at system boundaries, like APIs or external services.
• Graceful degradation beats catastrophic failure. Saying “no” to some requests keeps your system healthy enough to say “yes” to others.

Conclusion

Backpressure and rate limiting aren’t just performance optimizations—they’re survival strategies. When used wisely, they help your system stay responsive, scalable, and sane under pressure.

So the next time your system starts feeling like a packed highway at rush hour, remember: it’s okay to say “slow down.”

Want to dive deeper? Check out Martin Thompson’s blog on backpressure and the Resilience4j RateLimiter documentation for more implementation details.

Have you implemented backpressure or rate limiting in your systems? Share your experience in the comments—we’d love to hear how you keep your traffic flowing smoothly.

📚 Further Reading & Related Topics
If you’re exploring mastering backpressure and rate limiting for optimal system performance, these related articles will provide deeper insights:
• Latency Optimization Techniques – This article explores advanced techniques to reduce latency using lock-free programming, memory barriers, and efficient data structures, which complements the performance goals of backpressure and rate limiting.
• Reactive Programming with Spring Boot and Project Reactor – Understand how reactive programming models help manage backpressure natively, making it a natural fit for building responsive and resilient systems.
• Load Balancing Algorithms Every Developer Should Know – This guide introduces essential load balancing strategies that work in tandem with rate limiting to ensure system stability and optimal resource utilization.