I have read that NodeJs is able to support hundreds and thousand of concurrent calls because the worker threads do not wait on the IO calls to complete. They use event driven IO. Similarly in .NET we have Async keyword which allows the worker thread to be returned to perform other tasks while the "IO" call is executing. How does that all work. There has to be some thread monitoring for the IO to complete and notify when its ready?
I don't know NodeJs or .NET, and they would be off-topic here anyway, but the underlying concepts are common among programming frameworks and programming models, so I'll answer in general terms.
The word thread can mean several things.
- There is the conceptual meaning: one thing happening after another.
- There is the way this conceptual meaning is represented in a programming language. In almost all programming languages, most ways to combine program fragments lead to those fragments being executed one after the other, thus in a single thread, e.g.
a; bis a very common syntax for “execute
b”. A few keywords, operators or library functions allow code to be executed in a way other than in sequence, e.g.
forkin Unix or
Asyncin .NET. These create conceptual threads.
There are ways to implement this concept at the operating system level. These fall mainly in two categories (refinements are possible but I won't get into those):
- Kernel threads are managed by the OS kernel. The threads live independently. If one thread in a program is blocked on an IO call, that doesn't affect the other threads.
- User threads (also called green threads) are managed by the program. From the perspective of the kernel, there is a single thread, so if the program blocks on an IO call, none of its threads can run. To allow non-blocked threads to keep running, the programming library must ensure that it makes only non-blocking system calls (it may expose blocking functions in the programming interface, however).
There isn't always a direct match between programming language threads and operating system threads. There are several potential reasons for such a mismatch:
- The language was designed with a certain model in mind, but it's implemented on an operating system which doesn't have primitives that map directly to that model.
- The language induces programmers to use threads a lot, but kernel threads are somewhat expensive. So many programming language threads are executed in the same kernel thread. Instead of using blocking IO, the library functions uses non-blocking IO exclusively, and calls a function such as
pollwhich returns as soon as one pending I/O is unblocked.
I don't know NodeJS, but “able to support hundreds and thousand of concurrent calls because the worker threads do not wait on the IO calls to complete” sounds to me like this is what's happening.
- The language discourages programmers from explicitly using threads, possibly because threads with shared memory is extremely hard to get right (the potential for race conditions is huge). Instead, it favors higher-level abstractions where parallel threads are mostly independent. Under the hood, those abstractions may be implemented with operating system threads.
I don't know .NET, but
awaitsounds to me like it's implementing a rendezvous abstraction. The
asynckeyword creates a thread which lives its own life until
awaitjoins it with its parent.