# When are a controller's registers loaded and ready to inform an I/O Operation?

Below, I appended a fragmented paragraph from Operating Systems Concepts on interrupts.

To start an I/O operation, the device driver loads the appropriate registers in the device controller.

The device controller, in turn, examines the contents of these registers to determine what action to take (such as “read a character from the keyboard”).

The controller starts the transfer of data from the device to its local buffer.

Once the transfer of data is complete, the device controller informs the device driver that it has finished its operation.

The device driver then gives control to other parts of the operating system, possibly returning the data or a pointer to the data if the operation was a read.

For other operations, the device driver returns status information such as “write completed successfully” or “device busy”.

But how does the controller inform the device driver that it has finished its operation? This is accomplished via an interrupt.

My question has to do with the first and second sentences. Let's say we have an USB controller for a keyboard. When does the device driver load the controller's registers with the action they are supposed to handle? Does it happen on system startup, and the controller is just waiting to handle the subsequent data input? Or does it happen everytime the user hits a key?

## Summary

On x86-64, an eXtensible Host Controller (xHC) is responsible to interact with USB from the operating-system. The operating-system writes at conventional positions in RAM to read and write the registers of the xHC. What these registers do is explained in the xHCI specification. The keyboard is polled by the xHC at a certain interval using the Periodic Transfer Ring. When a transfer occurs, the xHC triggers an interrupt. The operating-system will then look in RAM for any change. If any change is detected, the operating-system puts a message on the message queue of the application which currently has focus.

I don't know much about ARM and how it interfaces to USB. I've been in the process of writing an x86-64 kernel which interacts with USB keyboards so I'll answer for x86-64. By the way, x86-64 is the CPU architecture you find in most desktop computers today.

x86 CPUs can't interact with USB directly. They need a PCI compliant device of which some registers are mapped into main memory (RAM). This PCI device is called today the eXtensible Host Controller (xHC). There once was other names but they were all provided as a way for x86 CPUs to interact with USB. The xHCI specification describes the functioning of the xHC (see https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/extensible-host-controler-interface-usb-xhci.pdf).

When the computer boots, there is some firmware on the motherboard which probably questions the different PCI devices for the position of their memory mapped registers (I don't have more information on how). It then informs the operating-system (OS) of where these registers will land using the ACPI tables which is a conventional representation of hardware. The position of these tables is stated in the ACPI convention.

When the computer boots, firmware hands over control to the OS. The OS will look at the ACPI tables to find the base address of the PCI configuration space for the computer. Most often it will be always the same address. It doesn't even need to be actual physical memory. It can be anywhere even outside memory. On QEMU virtual machines (with 128MB of memory by default), the PCI config space is often found above the actual physical memory.

The PCI config space is then scanned with a conventional approach to device addressing. Basically, each function of each device gets a config space. There can be several buses onto which can be up to 32 devices onto which can be up to 8 functions (see https://wiki.osdev.org/PCI). To find the base address of one function you need to use a basic arithmetic formula specified in the PCI convention as such:

UINT64 configSpacePhysAddr = pciConfigBaseAddr + ((bus - first_bus) << 20 | device << 15 | function << 12);


The value pciConfigBaseAddr is often 0x80000000. It cannot be relied on so it must be retrieved in the MCFG ACPI table which is located using other tables. The first table of the ACPI table chain is the RSDP table (see https://wiki.osdev.org/RSDP). The RSDP table has a conventional location and it allows to find all the other tables.

The function is specific to one protocol. For example, on a PCI compliant device which is an WIFI card and a Bluetooth card at the same time, you'll find 2 functions. Each function does more or less a specific thing (a specific protocol). Another example is the xHC from Intel. The xHC will be a device with only one function (to interface with USB).

Every PCI function gets several IDs which allow to identify the device and what it does (or to load a proper driver for it).

The xHC will present a complex interface to software with memory mapped registers. The OS will write to these registers to set up the xHC. The xHC is compatible with MSI-X. MSI-X is a new mechanism which allows to get interrupts directly bypassing the IOAPIC of x86 computers. The interrupt acts similarly to an inter-processor interrupt (also used to bootstrap other processors from core 0 or to control other cores in general).

The xHC has event rings, transfer rings and one command ring. All rings lie in RAM. The event rings are used by the xHC to tell software something happened. Most events will be accompanied with an interrupt. The vector that will be triggered is chosen by the OS using the MSI-X table. There can be up to 2048 different interrupt vectors when using the MSI-X capability of the xHC. The transfer rings are used by software to transfer data to or from USB devices. The command ring is used by software to send commands to the xHC. The transfer ring is the only ring which allows to transfer data to or from USB devices. Every xHC has several doorbell registers which are tied to one USB device. The doorbell registers are used to "ring the bell" to tell the xHC it has pending operations on a certain transfer ring for that specific USB device. Each ring is made of Tranfer Request Blocks (TRBs) which have several formats. For the event rings there are the Event TRBs. For the transfer rings there are the Transfer TRBs. For the command ring there are the Command TRBs. The TRB informs either software or the xHC (depending on the ring) of what happened or what to do.

There can be several event rings or, sometimes, only one. It depends on the implementation (read the xHCI specification for more information). There is one transfer ring for each endpoint of each device. Each USB device has several endpoints. To communicate with these endpoints, the OS places Transfer TRBs on the transfer ring of that endpoint and writes a certain number to the doorbell register of the associated device. That number is the number of the endpoint. This is sometimes termed "ringing the bell" to the xHC.

There exists a periodic transfer ring. The periodic transfer ring is a transfer ring which allows to "program" a Transfer TRB to be executed every time an interval in milliseconds has passed. Since USB is a polled bus, USB devices never initiate any transfer. You are thus left to do some polling of the keyboard (or let the xHC do it using the periodic transfer ring). When a transfer occurs, the xHC will trigger an interrupt on the vector you specified and place a Transfer Event TRB on the event ring. The OS will thus look at the event ring for the USB endpoint for which a transfer has occurred and look at the data. For a keyboard, if the data has changed from the last poll, it means that a key was released or pressed. The OS will thus read the change and send a message to the application which has focus. The transfer will be done to a specific address which is going to be specified in the Transfer TRB placed on the transfer ring of that specific device. The software will thus read that address for the data and will act accordingly.

For a keyboard, the periodic transfer ring is quite useful. The keyboard will be polled and software will probably just put a message on the message queue of the application which currently has focus stating that a key was pressed along with the key. It is up to applications to do something depending on the key.

Every time that the computer wants to interact with the peripheral device. "every time the user presses a key" sounds about right, but I don't know enough about keyboards specifically to vouch that this is exactly how it works.

• So "every time the user presses a key" an I/O request by the device driver is initiated? Jul 1 '21 at 22:35
• @mjfneto, I don't know enough about keyboards to know for sure exactly how that works. There are multiple forms of I/O.
– D.W.
Jul 2 '21 at 2:34