How does asynchronous operations save time in I/O

Question

I'm reading a book CLR via C# which says how Windows does asynchronous I/O:

To read data from the file, ReadAsync internally allocates a Task object to represent the pending completion of the read operation. Then, ReadAsync calls Win32's ReadFile function (#1). ReadFile allocates its IRP (I/O Request Packet) and then passes it down to the Windows kernel (#3). Windows adds the IRP to the hard disk drive’s IRP queue (#4), but now, instead of blocking your thread, your thread is allowed to return to your code; your thread immediately returns from its call to ReadAsync (#5, #6, and #7). Now, of course, the IRP has not necessarily been processed yet, so you cannot have code after ReadAsync that attempts to access the bytes in the passed-in Byte[]. Using the task object, you can call ContinueWith to register a callback method that should execute when the task completes and then process the data in this callback method. Or, alternatively, you can use C#'s asynchronous function feature to simplify your code by allowing you to write it sequentially.

The author also describes a scenario of how asynchronous operations save time in I/O:

let's say that your application wants to download 10 images from various websites and that it takes 5 seconds to download each image. If you perform this work synchronously (downloading one image after another), then it takes you 50 seconds to get the 10 images. However, if you use just one thread to initiate 10 asynchronous download operations, then all 10 are being performed concurrently and all 10 images will come back in just 5 seconds! That is, when performing multiple synchronous I/O operations, the time it takes to get all the results is the sum of the times required for each individual result. However, when performing multiple asynchronous I/O operations, the time it takes to get all the results is the time required to get the single worst-performing operation.

I don't understand how async operations can reduce the time from 50 seconds to 5 seconds. Let's change the NTFS Disk Driever to network adapter driver and change the read file async code to download file async code.

My understanding is, it is true that asynchronous operations allow fewer worker threads to handle incoming client's I/O requests, contrasted by synchronous operations, every I/O request need a worker thread to handle, and when the synchronous operation takes time to complete, the worker thread gets blocked too and therefore it cannot return to the thread pool to serve other requests. But for the network adapter driver, can still only complete one I/O operation one at a time.

For example, let's say the network adapter driver is handling the first I/O request (represented as an IRP queue) to download the first picture. Even though you use asynchronous operations, the worker thread that serves the first I/O request did return to the thread pool to serve the second I/O request, but the network adapter driver is still processing the first I/O request to download the first picture, so the second I/O request as an IRP is still queuing in the network adapter driver. The network adapter driver only handles the second IRP after the first one is completed, so the network adapter driver still serves the second I/O request after 5 seconds, and so on, so it still needs 50 seconds to download 10 pages. In other words, it is not worker threads blocking, it is the network adapter driver that's blocking.

so how can how async operations can reduce the time from 50 seconds to 5 seconds? Unless the network adapter driver itself is a small powerful computer that has multiple CPUs that can serve requests concurrently, but this is obviously not true, drivers like network adapter drivers are cheap devices, it can't be having expensive components like CPUs?

Answer 1

Issuing the ten requests in parallel is unlikely to give you a tenfold speedup, but will usually give you some speedup. It depends on where the bottleneck is. If the images are 5 MB each and your Ethernet/WiFi connection can only transfer data at 1 MB/s, then your network card will essentially be tied up for 5 seconds per image, and issuing the requests in parallel probably won't help. If the images are 5 kB each and you have a gigabit fiber Internet connection, and the 5-second delay is due to the servers being very slow to respond, then your network card will be idle almost all of the time, and you may actually get all of the images back in 5 seconds total if you issue the requests in parallel.

The network card only sends and receives small isolated data packets, typically of about 1500 bytes. It doesn't understand the concept of a TCP or HTTP connection. When it's not sending or receiving packets related to one connection, it can send or receive packets related to another, even if the first connection is still open.

The book is wrong to assume that the 10 requests are perfectly parallelizable; usually that isn't the case. Also, the parallelism it's talking about has nothing to do with asynchronous I/O as such. You'd get the same speedup by starting 10 threads and issuing a blocking I/O call in each one. That might be a bit slower because of thread-creation overhead, but might also be faster because it uses more CPU cores if you have them. For the highest performance you have to create one thread per CPU core and use asynchronous I/O in all of them. NT has built-in support for that model through I/O completion ports.