Tomcat vs Netty: The Thread Model Decision Most Engineers Get Backwards
(Part 1 of 2 - Backend I/O Internals Series)
Every Spring Boot service you've ever written sits on top of a decision you probably never made consciously: does your server hand each request its own thread, or does it multiplex thousands of requests over a handful of threads?
Get this wrong for your workload, and you'll see it under load - thread pool exhaustion in one model, event-loop stalls in the other. Get it right, and you'll barely think about it again.
This post builds the mental model from the ground up: what blocking I/O actually costs you at the OS level, how Java's NIO makes non-blocking I/O possible, and how that plays out differently in Tomcat's thread-per-request design versus Netty's event loop. Part 2 takes this model and applies it to Spring MVC, WebFlux, and Micronaut - including the single most common mistake teams make when they switch to a "non-blocking" stack.
Table of contents
Blocking vs Non-blocking I/O
At the OS level, I/O - reading a socket, querying a database, calling another service - means asking the kernel to do something and waiting for it to finish.
Blocking I/O means the calling thread parks itself until that operation completes. It can't do anything else. If you have one thread per request and that request is waiting on a slow downstream call, that thread is dead weight for the duration - allocated, scheduled, consuming stack memory, and doing absolutely nothing useful.
Non-blocking I/O means the thread issues the request and immediately moves on. It doesn't wait. When the operation completes, something (an event loop, a callback, a selector) notifies the system, and the result gets picked up later - often by a different thread, or the same thread on its next pass through a loop.
Think of it like a restaurant. A blocking model is a waiter who takes one table's order, stands at the kitchen window until the food is ready, delivers it, and only then moves to the next table. A non-blocking model is a waiter who takes an order, moves on to the next table immediately, and gets pinged when any order is ready to deliver - one waiter, many tables, no idle standing around.
This single distinction - does the thread wait, or does it move on - is the root of nearly every architectural difference between Tomcat and Netty, and between Spring MVC and WebFlux. Everything else in this post is a consequence of that choice.
Java NIO & Selector
Java's original I/O (java.io) is blocking by default - a InputStream.read() call parks the thread. Java NIO (java.nio), introduced to support non-blocking I/O, is built on three core pieces:
Channels: bidirectional conduits for data (a
SocketChannel, for instance), replacing the one-directional streams ofjava.ioBuffers: fixed-size containers you read from and write into, since NIO deals in chunks rather than continuous streams
Selector: the piece that makes non-blocking I/O actually useful. A
Selectorlets a single thread monitor many channels at once, and only wakes up when one of them is actually ready (readable, writable, or has a new connection)
The Selector is the multiplexing mechanism: instead of one thread per connection, you get one thread watching thousands of connections, springing into action only when there's real work to do. This is the low-level primitive that Netty and every non-blocking server is built directly on top of. You rarely touch Selector yourself in application code, but understanding that it exists explains why an event loop can handle so many connections with so few threads.
Tomcat architecture
Tomcat, the default embedded server in Spring Boot, uses a thread-per-request model (technically thread-per-connection, pooled). When a request comes in:
A thread is pulled from Tomcat's worker thread pool
That thread handles the entire request lifecycle- reading input, running your application code, writing the response
If your code makes a blocking call (a DB query, an HTTP call to another service), the thread sits there, blocked, until it returns
Only then is the thread returned to the pool for the next request
The scaling lever here is thread pool size. Increase it, and you can handle more concurrent in-flight requests up to a point. Each thread costs memory (default JVM thread stacks are ~1MB), and past a few hundred to a couple thousand threads, context-switching overhead and memory pressure start working against you. Under a traffic spike where downstream calls slow down, every thread ends up blocked waiting, the pool exhausts, and new requests queue or get rejected even though the CPU itself may be nearly idle.
This model is simple to reason about one thread, one request, top to bottom - which is exactly why it's been the default for so long.
Netty architecture
Netty takes the opposite approach: a small, fixed number of threads (typically close to CPU core count) organized into an event loop group, each running an event loop built on NIO's Selector.
Each event loop thread can handle thousands of connections simultaneously, because it's never blocked waiting on any single one. It processes whatever's ready- a readable socket here, a writable one there - cycles through, and moves on. A "request" doesn't own a thread for its lifetime; it occupies a thread only for the brief moments it has actual CPU work to do.
This is why Netty-based servers can sustain enormous numbers of concurrent connections with a handful of threads, where Tomcat would need a thread per in-flight request. The tradeoff: your code has to cooperate. If anything you run inside an event loop blocks, it doesn't just stall one request - it stalls every connection multiplexed on that thread. That failure mode is the entire subject of Part 2.
Tomcat vs Netty thread model
| Tomcat (thread-per-request) | Netty (event loop) | |
|---|---|---|
| Threads under load | Grows with concurrent requests (bounded by pool size) | Stays flat - fixed small number |
| Blocked thread cost | One request stalled, one thread wasted | One blocked call can stall every request on that event loop thread |
| Memory footprint at scale | High (thread stacks scale with concurrency) | Low (few threads regardless of connection count) |
| Failure mode under spike | Thread pool exhaustion, request queuing/rejection | Event loop starvation if blocking code leaks in |
| Mental model | Simple - linear, one thread per request | Requires discipline - must stay non-blocking end-to-end |
The core insight: Tomcat trades memory/thread overhead for simplicity and fault isolation (one slow request doesn't take down others). Netty trades that isolation for massive connection scalability - but only if you hold up your end of the bargain and never block an event loop thread.
Coming up in Part 2
You now know why Tomcat and Netty behave differently under load. But the frameworks built on top of them - Spring MVC, WebFlux, Micronaut are where this decision actually gets made in practice, and where one very common mistake quietly undoes the entire point of switching to a non-blocking stack.
Part 2 covers:
Spring MVC's request lifecycle (and why blocking calls are completely natural there)
Spring WebFlux + Netty and the one condition that has to hold for it to actually pay off
Micronaut's compile-time DI what it buys you over Spring's runtime reflection
The blocking-call trap: how a single synchronous JDBC call inside a reactive handler can silently stall dozens of unrelated requests
A decision framework for when each stack is actually worth it