Virtual threads

Every thread you have started so far is backed by a thread of the operating system, which gives it a block of memory and keeps track of it. That makes threads expensive, and a program can only have a few thousand of them. Java 21 added virtual threads: threads so cheap that a program can start one for every task, a hundred thousand at once if it needs to. This lesson shows what they cost, how to start them, and the kind of work where they help.

What a platform thread costs#

Every thread you have made so far, with new Thread, with Thread.ofPlatform() or inside a pool, is a platform thread. A platform thread is a thin wrapper around one thread of the operating system, and it keeps that operating system thread for its whole life. That makes each one costly in a few ways:

• Memory. Each platform thread gets its own call stack, the memory that holds the method calls it is in the middle of and their local variables. The JVM sets the whole block aside when the thread starts: 2 MB on a Mac with an Apple chip, like the one used for this page, and 1 MB on a typical Linux PC. The operating system hands over real memory only for the parts the thread touches, but the whole block stays reserved.
• Time. Each one is a request to the operating system. On the Mac used for this page, starting 2,000 platform threads took about 75 milliseconds.
• A limit. The operating system caps how many threads one program may have.

Here is a program that tries to start 10,000 platform threads that each sleep for five seconds. It prints a line after every 500.

started 500 platform threads
started 1000 platform threads
started 1500 platform threads
started 2000 platform threads
[3.132s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 2048k, guardsize: 16k, detached.
[3.132s][warning][os,thread] Failed to start the native thread for java.lang.Thread "Thread-2019"
Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
	at java.base/java.lang.Thread.start0(Native Method)
	at java.base/java.lang.Thread.start(Thread.java:1459)
	at java.base/java.lang.ThreadBuilders$PlatformThreadBuilder.start(ThreadBuilders.java:196)
	at Main.main(Main.java:4)

It never reached 10,000. The two lines that start with a time in square brackets are warnings the JVM printed when the operating system said no. The second one names the thread that could not start, Thread-2019, and threads are numbered from Thread-0, so 2,019 had started. On that Mac one program may have at most 2,048 threads, and the JVM was already using some for its own work. Java reported the refusal as an OutOfMemoryError. The same run gave the same count three times. Other systems have other limits, often higher on Linux, but every system has one.

The memory adds up too. Measured on the same Mac, a program holding 2,000 sleeping platform threads used about 137 MB of memory, against 21 MB for the same program with no extra threads. That is roughly 58 KB of real memory for each thread that does nothing but sleep, on top of the 2 MB block each one reserves.

Key idea

A platform thread is tied to one operating system thread for its whole life. Each one costs memory and a place under a fixed limit, so a program can have a few thousand of them, not a hundred thousand.

Waiting wastes a platform thread#

Most programs that need many threads are not busy computing. They are waiting. A web server reads a request, asks a database for some rows and waits, then calls a payment service and waits again. Reading from the network, a disk or a database is called I/O, short for input and output. A method that does not return until its data has arrived is a blocking call: the thread that made it can do nothing else until then.

A platform thread stuck in a blocking call uses no processor time, but it keeps everything it was given: its operating system thread, its stack and its place under the limit. So a server that gives each request a thread from a pool of 200 can have at most 200 requests in progress. Request number 201 waits in the pool's queue, even when the processor has nothing to do.

This program works out when each task starts, for a pool of 3 threads and 7 tasks that each wait 2 seconds. The tasks start in rounds: the first 3 take the 3 threads, the next 3 start when those finish, and so on. Step through it and watch round and start.

It prints a start time for each task, then all done after 6s. Tasks 3, 4 and 5 sat in the queue for 2 seconds and task 6 for 4 seconds, while the threads that could have served them were only waiting. The line (tasks + threads - 1) / threads is integer division that rounds up: 7 tasks need 3 rounds, though the last round holds only one task. Change threads to 7 and run it again. Every task starts at 0s, and everything is done after 2s.

It prints task 9 starts at 2s and all done after 3s. Task 9 is in round 9 / 4, which is 2 in integer division, so it waits for 2 rounds of one second. Ten tasks need 3 rounds, because 4 threads can take only 4 tasks a round.

1public class Main {
2    public static void main(String[] args) {
3        System.out.println("200 threads: " + secondsFor(10_000, 200, 1) + " s");
4        System.out.println("10000 threads: " + secondsFor(10_000, 10_000, 1) + " s");
5        System.out.println("3 threads: " + secondsFor(7, 3, 2) + " s");
6    }
7 
8    static int secondsFor(int tasks, int threads, int secondsEach) {
9        return 0; // TODO: the number of rounds, times the seconds each round takes
10    }
11}

The first line is the problem in one number. Ten thousand tasks that each wait one second take 50 seconds on a pool of 200 threads, and one second if every task has a thread of its own. You saw what happens when you try to give 10,000 tasks a platform thread each.

Key idea

With platform threads, the number of tasks that can wait at the same time is the number of threads you can afford. Tasks beyond that queue up, even while the processor sits idle.

What a virtual thread is#

A virtual thread is a Thread object that the JVM schedules itself, instead of leaving it to the operating system. It runs on an ordinary platform thread, but only while it has work to do. The platform thread it runs on is called its carrier.

When a virtual thread makes a blocking call, such as Thread.sleep or a read from the network, the JVM takes it off its carrier. Its stack, the method calls it is in the middle of, is saved in the heap like any other object, and the carrier is free to run another virtual thread. When the data arrives or the sleep ends, the virtual thread joins a queue, and the next free carrier picks it up and carries on from where it stopped. Putting a virtual thread on a carrier is called mounting it, and taking it off is unmounting.

Think of a restaurant. With platform threads, each table has its own waiter, who takes the order to the kitchen and then stands at the kitchen door until the food is ready. Two hundred waiters serve two hundred tables, and the restaurant cannot afford more waiters. With virtual threads, each order is a ticket. A few waiters carry tickets to the kitchen. While a dish cooks, its ticket is pinned to a board and the waiter goes to take the next order. When the dish is ready, whichever waiter is free picks the ticket up. The tickets are the virtual threads, the waiters are the carriers, and the board is the heap.

Hello from VirtualThread[#30]/runnable@ForkJoinPool-1-worker-1
Is worker virtual? true
Is main virtual? false

Thread.ofVirtual() is the sibling of the Thread.ofPlatform() builder, and start starts the new thread at once. The printed thread has three parts. VirtualThread[#30] says what it is, with its id number. A virtual thread has no name unless you give it one, so nothing follows the id. runnable is its state: it is running. Everything after the @ is its carrier, ForkJoinPool-1-worker-1: worker 1 of the pool of carriers that the JVM keeps for virtual threads. isVirtual() tells the two kinds apart, and the main thread is a platform thread. When you run it, the id and the worker number may differ.

A virtual thread can come back on another carrier#

This program starts three named virtual threads. Each one prints itself, sleeps for half a second, and prints itself again.

before: VirtualThread[#30,task-3]/runnable@ForkJoinPool-1-worker-3
before: VirtualThread[#29,task-2]/runnable@ForkJoinPool-1-worker-2
before: VirtualThread[#27,task-1]/runnable@ForkJoinPool-1-worker-1
after:  VirtualThread[#27,task-1]/runnable@ForkJoinPool-1-worker-4
after:  VirtualThread[#30,task-3]/runnable@ForkJoinPool-1-worker-2
after:  VirtualThread[#29,task-2]/runnable@ForkJoinPool-1-worker-3

Look at the worker after each @. task-1 went to sleep on worker 1 and woke up on worker 4. In this run all three came back on a different carrier from the one they started on; in another run, task-2 came back on the same one. The sleep unmounted each thread, and any free carrier could pick it up again. The job itself never notices: Thread.currentThread() always returns the virtual thread, never its carrier. The order of the lines changes from run to run as well.

By default the JVM uses as many carriers as the computer has cores. When 10,000 sleeping virtual threads ran on the 8-core Mac used for this page, all of them were carried by the same 8 platform threads, worker-1 to worker-8, in each of three runs.

A sleeping virtual thread is small: its saved stack and a few objects in the heap. On the same Mac, a program holding 100,000 sleeping virtual threads used between 139 and 155 MB of memory, about what 2,000 platform threads used. That is roughly 1.2 to 1.3 KB per virtual thread. A thread that sleeps in the middle of deep method calls has a bigger stack to save, so real tasks use more, but still far less than a platform thread.

Key idea

A virtual thread borrows a carrier only while it has work to do. When it waits, the JVM saves its stack in the heap and the carrier runs another virtual thread, so waiting costs a little heap memory instead of a whole operating system thread.

Three ways to start a virtual thread#

You have already used the first way. Here are all three, side by side:

• Thread.ofVirtual().start(job) uses the builder. Add .name("...") before start to name the thread.
• Thread.startVirtualThread(job) is a shortcut for Thread.ofVirtual().start(job).
• Executors.newVirtualThreadPerTaskExecutor() is an ExecutorService that starts a new virtual thread for every task you submit.

hello from VirtualThread[#30,first]/runnable@ForkJoinPool-1-worker-1
hello from VirtualThread[#33]/runnable@ForkJoinPool-1-worker-1
hello from VirtualThread[#34]/runnable@ForkJoinPool-1-worker-1
hello from VirtualThread[#35]/runnable@ForkJoinPool-1-worker-2
all done

Both Thread methods return the started thread, so join()works as it always has. The executor is opened in a try-with-resources statement. When the block ends, Java calls the executor's close(), which stops it from taking new tasks and then waits until every task already submitted has finished. That is why all done always comes last. In two of four runs, the two submitted tasks printed in the other order, as two threads running at once may.

The trap: Java does not wait for virtual threads#

This program starts a virtual thread and lets main end without a join().

main: done
virtual thread: hello

Run it a few times and the second line is sometimes missing. In 50 runs with java Main.java on the Mac used for this page it was missing 6 times, and in 100 runs of the compiled program, 23 times. Virtual threads are always daemon threads, the kind Java does not wait for before it ends a program (you met them in the introduction to threads). Once main returns, nothing keeps the program alive, so it ends whether or not the virtual thread has printed yet. The same program with Thread.ofPlatform() printed both lines in 100 runs out of 100.

Ten thousand tasks that wait#

Here is the experiment that shows what virtual threads are for. Ten thousand tasks each wait one second, with Thread.sleep standing in for a slow reply from a server, and the program times the whole batch.

10000 tasks took 1057 ms

Each task is a lambda that sleeps and then returns a value, so it is a Callable, and a Callable may throw InterruptedException without a try and catch. All 10,000 virtual threads sleep at the same time, and the batch takes just over a second: between 1,046 and 1,081 ms in five runs. Your times will differ a little, but almost all of it is the one-second wait.

Now change one line so that the tasks share a pool of 200 platform threads: Executors.newFixedThreadPool(200). The output becomes 10000 tasks took 50242 ms, and 50,244 ms in a second run. That is the arithmetic from the timetable: 10,000 tasks in rounds of 200 is 50 rounds of one second each.

A third try gives every task its own platform thread with Executors.newCachedThreadPool(), which starts a new platform thread whenever none is free. On the Mac used for this page it crashed after starting 2,017 threads, with the same OutOfMemoryError as TooMany, and never printed a time.

Key idea

Virtual threads do not make a task faster: each task still waits its full second. They let all 10,000 tasks wait at the same time, so the batch finishes in one second instead of fifty.

When virtual threads do not help#

A virtual thread runs code no faster than a platform thread. It is the same Java code on the same processor; what changed is how cheaply a thread can wait. So the question to ask about a task is whether it spends its time waiting or computing. Here are 16 tasks that never wait. Each one counts the prime numbers below 3,000,000, which keeps a core busy the whole time.

16 busy tasks took 4204 ms

That run took 4,204 ms. Swapping the executor gave the times below. The laptop was busy with other programs while they ran, so they jump around from round to round:

Neither kind of thread came out reliably ahead. Every task needs a core for its whole run, the laptop has 8 cores, and the virtual threads run on 8 carriers, so at most 8 tasks compute at any moment however many threads you start. More threads only take turns on the same cores. The single thread was slowest because it used one core while the other seven sat idle.

Pinning, and what changed in Java 24#

Older articles about virtual threads warn about pinning: a virtual thread that cannot be unmounted while it waits, so its carrier waits with it and runs nothing else. In Java 21 to 23 the usual cause was waiting inside a synchronized method or block, and the advice was to replace synchronized with a lock class. Java 24 removed that cause (JEP 491). A virtual thread can now unmount while it is inside a synchronized method or block, and while it waits to enter one. So on Java 25 you can use synchronized with virtual threads exactly as you learned it.

What still pins a virtual thread is rare in everyday code: running native code, such as a method written in C, and a few moments while a class is being loaded and set up. Pinning never makes a program wrong. It only means one carrier sits idle for as long as that wait lasts.

Structured concurrency, still a preview#

Starting tasks is easy now. Keeping track of them is harder. Suppose a page needs a customer's details and their orders, and you fetch both at once in two virtual threads. If one fetch fails, the other should be cancelled, and the error should reach the code that asked for both. Structured concurrency is an API for exactly that. It treats a group of tasks started together as one unit of work: they start inside one block of code, they have all finished before that block ends, and if one fails, the others are cancelled.

In Java 25 it is a preview for the fifth time (JEP 505), so you have to switch it on with --enable-preview, as modern Java syntax explains, and its classes can still change. They did change in Java 25, which replaced the constructors with factory methods, and Java 26 previews it a sixth time (JEP 525). This course will teach the code once it is final. Until then, an executor in try-with-resources gives you part of the shape: every task you submit inside the block has finished when the block ends.

Further reading#

• JEP 444: Virtual Threads, the proposal that made them final in Java 21, with the reasoning behind their design
• Virtual Threads in Oracle's Java 25 core libraries guide, on when to use them and why not to pool them
• The Thread class in the Java SE 25 API documentation, with ofVirtual, startVirtualThread and isVirtual
• JEP 491: Synchronize Virtual Threads without Pinning, the Java 24 change to synchronized