Every Spring Boot service you've ever written sits on top of a decision you probably never made consciously: does your server hand each request its own thread, or does it multiplex thousands of requests over a handful of threads?

Get this wrong for your workload, and you'll see it under load - thread pool exhaustion in one model, event-loop stalls in the other. Get it right, and you'll barely think about it again.

This post builds the mental model from the ground up: what blocking I/O actually costs you at the OS level, how Java's NIO makes non-blocking I/O possible, and how that plays out differently in Tomcat's thread-per-request design versus Netty's event loop. Part 2 takes this model and applies it to Spring MVC, WebFlux, and Micronaut - including the single most common mistake teams make when they switch to a "non-blocking" stack.

Table of contents

1. Blocking vs Non-blocking I/O

2. Java NIO & Selector

3. Tomcat architecture

4. Netty architecture

5. Tomcat vs Netty thread model

Blocking vs Non-blocking I/O

At the OS level, I/O - reading a socket, querying a database, calling another service - means asking the kernel to do something and waiting for it to finish.

Blocking I/O means the calling thread parks itself until that operation completes. It can't do anything else. If you have one thread per request and that request is waiting on a slow downstream call, that thread is dead weight for the duration - allocated, scheduled, consuming stack memory, and doing absolutely nothing useful.

Non-blocking I/O means the thread issues the request and immediately moves on. It doesn't wait. When the operation completes, something (an event loop, a callback, a selector) notifies the system, and the result gets picked up later - often by a different thread, or the same thread on its next pass through a loop.

Think of it like a restaurant. A blocking model is a waiter who takes one table's order, stands at the kitchen window until the food is ready, delivers it, and only then moves to the next table. A non-blocking model is a waiter who takes an order, moves on to the next table immediately, and gets pinged when any order is ready to deliver - one waiter, many tables, no idle standing around.

This single distinction - does the thread wait, or does it move on - is the root of nearly every architectural difference between Tomcat and Netty, and between Spring MVC and WebFlux. Everything else in this post is a consequence of that choice.

Java NIO & Selector

Java's original I/O (java.io) is blocking by default - a InputStream.read() call parks the thread. Java NIO (java.nio), introduced to support non-blocking I/O, is built on three core pieces:

• Channels: bidirectional conduits for data (a SocketChannel, for instance), replacing the one-directional streams of java.io

• Buffers: fixed-size containers you read from and write into, since NIO deals in chunks rather than continuous streams

• Selector: the piece that makes non-blocking I/O actually useful. A Selector lets a single thread monitor many channels at once, and only wakes up when one of them is actually ready (readable, writable, or has a new connection)

The Selector is the multiplexing mechanism: instead of one thread per connection, you get one thread watching thousands of connections, springing into action only when there's real work to do. This is the low-level primitive that Netty and every non-blocking server is built directly on top of. You rarely touch Selector yourself in application code, but understanding that it exists explains why an event loop can handle so many connections with so few threads.

Tomcat architecture

Tomcat, the default embedded server in Spring Boot, uses a thread-per-request model (technically thread-per-connection, pooled). When a request comes in:

1. A thread is pulled from Tomcat's worker thread pool

2. That thread handles the entire request lifecycle- reading input, running your application code, writing the response

3. If your code makes a blocking call (a DB query, an HTTP call to another service), the thread sits there, blocked, until it returns

4. Only then is the thread returned to the pool for the next request

The scaling lever here is thread pool size. Increase it, and you can handle more concurrent in-flight requests up to a point. Each thread costs memory (default JVM thread stacks are ~1MB), and past a few hundred to a couple thousand threads, context-switching overhead and memory pressure start working against you. Under a traffic spike where downstream calls slow down, every thread ends up blocked waiting, the pool exhausts, and new requests queue or get rejected even though the CPU itself may be nearly idle.

This model is simple to reason about one thread, one request, top to bottom - which is exactly why it's been the default for so long.

Netty architecture

Netty takes the opposite approach: a small, fixed number of threads (typically close to CPU core count) organized into an event loop group, each running an event loop built on NIO's Selector.

Each event loop thread can handle thousands of connections simultaneously, because it's never blocked waiting on any single one. It processes whatever's ready- a readable socket here, a writable one there - cycles through, and moves on. A "request" doesn't own a thread for its lifetime; it occupies a thread only for the brief moments it has actual CPU work to do.

This is why Netty-based servers can sustain enormous numbers of concurrent connections with a handful of threads, where Tomcat would need a thread per in-flight request. The tradeoff: your code has to cooperate. If anything you run inside an event loop blocks, it doesn't just stall one request - it stalls every connection multiplexed on that thread. That failure mode is the entire subject of Part 2.

Tomcat vs Netty thread model

The core insight: Tomcat trades memory/thread overhead for simplicity and fault isolation (one slow request doesn't take down others). Netty trades that isolation for massive connection scalability - but only if you hold up your end of the bargain and never block an event loop thread.

Coming up in Part 2

You now know why Tomcat and Netty behave differently under load. But the frameworks built on top of them - Spring MVC, WebFlux, Micronaut are where this decision actually gets made in practice, and where one very common mistake quietly undoes the entire point of switching to a non-blocking stack.

Part 2 covers:

• Spring MVC's request lifecycle (and why blocking calls are completely natural there)

• Spring WebFlux + Netty and the one condition that has to hold for it to actually pay off

• Micronaut's compile-time DI what it buys you over Spring's runtime reflection

• The blocking-call trap: how a single synchronous JDBC call inside a reactive handler can silently stall dozens of unrelated requests

• A decision framework for when each stack is actually worth it

This post breaks down what each one actually does, where it sits in your architecture, and - most importantly - when to use which.

The Core Problem: They All "Sit in Front" of Something

At a glance, all three:

• Receive incoming traffic

• Sit between the client and your backend services

• Can do TLS termination

• Can route requests somewhere

Because of this surface-level similarity, engineers often treat them as interchangeable. They're not. Each was built to solve a different problem, and understanding why each one exists is the key to telling them apart.

Reverse Proxy: The Traffic Doorman

A reverse proxy sits in front of one or more backend servers and forwards client requests to them. The client only ever talks to the proxy - it has no idea what's happening behind it.

Primary job: Hide backend infrastructure, forward requests, and optionally cache or compress responses.

Typical responsibilities:

• TLS termination (decrypting HTTPS so backend servers don't have to)

• Basic request routing (e.g., /images → image server, /api → app server)

• Caching static content

• Compression (gzip/brotli)

• Hiding internal server details and IP addresses

• Basic protection (rate limiting, blocking bad IPs)

Common tools: Nginx, HAProxy (in proxy mode), Apache HTTP Server, Envoy

Mental model: Think of a hotel concierge. Guests (clients) never directly call a specific room — they talk to the concierge, who quietly figures out where to send the request.

Load Balancer: The Traffic Distributor

A load balancer is a specialized type of reverse proxy whose main job is distributing incoming traffic across multiple identical backend instances to prevent any single server from being overwhelmed.

Primary job: Even out traffic across a pool of servers for scalability and fault tolerance.

Typical responsibilities:

• Distributing requests using algorithms like round-robin, least connections, or weighted routing

• Health checks — detecting and routing around unhealthy instances

• Failover when a server goes down

• Operating at Layer 4 (TCP/UDP) or Layer 7 (HTTP)

Common tools: AWS ELB/ALB/NLB, Nginx, HAProxy, Google Cloud Load Balancer, F5

Mental model: Think of an airport check-in area with multiple counters and a staff member directing you: "Counter 4 is free, go there." The staff member doesn't care what's in your request - they just care that no single counter gets overloaded.

Key distinction from Reverse Proxy: Every load balancer is technically a reverse proxy, but not every reverse proxy is a load balancer. A reverse proxy might forward all traffic to a single server; a load balancer's entire purpose is spreading traffic across many.

API Gateway: The Traffic Manager with Business Logic

An API Gateway is a reverse proxy on steroids, purpose-built for managing APIs - particularly in microservices architectures. It doesn't just forward requests; it understands and acts on the content of the API request itself.

Primary job: Be the single entry point for clients to interact with potentially dozens of backend microservices, while handling cross-cutting API concerns.

Typical responsibilities:

• Authentication and authorization (validating API keys, JWTs, OAuth tokens)

• Rate limiting and throttling per client/API key

• Request/response transformation (e.g., XML to JSON)

• Routing requests to the correct microservice based on path, headers, or version

• API versioning

• Aggregating responses from multiple services into one

• Analytics, logging, and monitoring of API usage

• Sometimes load balancing too - as an added feature

Common tools: Kong, AWS API Gateway, Apigee, Zuul, Amazon Kong, Tyk, Traefik (partially)

Mental model: Think of a corporate receptionist who doesn't just point you to a room - they check your ID, verify you have an appointment, log your visit, decide which department actually handles your request, and sometimes even collect information from three departments before giving you a single combined answer.

Side-by-Side Comparison

Where They Actually Overlap (Why the Confusion Exists)

• A load balancer is a reverse proxy - just with a narrower, traffic-distribution-focused purpose.

• An API Gateway often includes reverse proxy and load balancing capabilities.

• Tools like Nginx, Envoy, and Traefik can be configured to act as any of the three, which blurs the lines in practice.

This is the real source of confusion: it's not that these are unrelated concepts, it's that modern tools are flexible enough to wear multiple hats. The distinction lives in intent and primary responsibility, not necessarily in the tool itself.

A Simplified Way to Remember It

• Reverse Proxy → "I forward your request somewhere and hide what's back there."

• Load Balancer → "I make sure no single server gets crushed by traffic."

• API Gateway → "I understand your API request, check who you are, and decide what happens to it."

A Real-World Architecture Example

Imagine a typical microservices-based e-commerce app:

1. Client sends a request to api.mystore.com/orders

2. API Gateway authenticates the request, checks rate limits, and routes it to the orders microservice

3. Behind the gateway, a Load Balancer distributes that request across 5 instances of the orders service running for redundancy

4. Each instance might sit behind a Reverse Proxy (like Nginx) that terminates TLS and forwards to the actual application process

All three components can coexist in the same request path, each solving a different problem at a different layer.

Final Takeaway

These three aren't competing technologies - they're complementary layers that often work together. The confusion comes from overlapping capabilities in modern tools, not from the concepts themselves being similar. Once you anchor on primary intent - hiding backend, distributing load, or managing API policy - the fog clears fast.

Next time someone says "just load balance it" when they mean "add an API gateway," you'll know exactly why that matters.