IOC device API
A) General principles
This device allows the kernel to access external mass storage peripherals such as a magnetic hard disk, or a SD card, that can store blocks of data in a linear array of sectors indexed by a simple lba (logic block address).
The block size is supposed to be 512 bytes.
The "kernel" API contains the functions used by the system calls, and defines four operation types : the IOC_READ, IOC_WRITE, IOC_SYNC_READ, and IOC_SYNC_WRITE operations move a given number of contiguous blocks between the block device and a kernel memory buffer. This API is detailed in section C below.
The asynchronous IOC_READ and IOC_WRITE operations are not directly executed by the client thread. The requests are registered in the waiting queue rooted in the IOC chdev descriptor. These requests are actually handled by a dedicated server thread running in the cluster containing the chdev descriptor, that calls the generic ioc_driver_cmd() function for each registered request.
The synchronous IOC_SYNC_READ and IOC_SYNC_WRITE operations does not use the waiting queue, and does not use the server thread. The client thread calls directly the ioc_driver_cmd() function.
Most IOC device hardware implementations have a DMA capability, and the data transfer is not done by the software, but is actually done by the hardware device. In this case, the IOC_IRQ signaling the transfer completion is routed to the core executing the server thread, and handled by the ioc_driver_isr() function (ISR stand for Interrupt Service Routine).
An exception is the IOC-RDK implementation, where the block device is actually implemented as physical memory. This IOC-RDK implementation does not use DMA, and does not use an IRQ. The data transfer is directly executed by the ioc_driver_cmd() software function.
To access the various drivers, the IOC device defines a lower-level "driver" API, that is detailed in section D below.
All IOC device structures and access functions are defined in the dev_ioc.c et dev_ioc.h files.
B) Initialisation
The dev_ioc_init() function makes the following initializations :
- It selects a core in cluster containing the IOC chdev to execute the server thread.
- it links the IOC IRQ to the core executing the server thread.
- it initialises the IOC specific fields of the chdev descriptor.
- it initialises the implementation specific IOC hardware device,
- it initializes the specific software data structures required by the hardware implementation.
It must be called by a local thread.
C) The "kernel" API
These four I/O operations are blocking and return only when the transfer is completed, but the blocking policy depends on the operation type.
C.1) Asynchronous operations
- The dev_ioc_read( xptr_t buffer_xp , uint32_t lba , uint32_t count ) blocking function moves <count> contiguous blocks from the block device, starting from block defined by the <lba> argument, to a kernel buffer defined by the <buffer_xp> argument. It register the request in the IOC device waiting queue. Then it blocks and deschedules.
- The dev_ioc_write( xptr_t buffer_xp , uint32_t lba , uint32_t count ) blocking function moves <count> contiguous blocks from a kernel buffer defined by the <buffer_xp> argument to the block device, starting from block defined by the <lba> argument. It register the request in the IOC device waiting queue. Then it blocks and deschedules.
Almos-mkh uses the asynchronous, READ & WRITE operations, for most data transfers between the file system on the IOC, and the file system cache in memory. The detailed scenario is the following :
- When a client thread request an I/O operation, the request is registered in the ioc_command_t structure embedded in the client thread descriptor, and the client thread registers itself in the waiting queue rooted in the IOC chdev. Then the client thread blocks on the THREAD_BLOCKED_IO condition, and deschedules.
- The DEV server thread attached to the IOC device descriptor handles all commands registered in the IOC device waiting queue. For each pending request, it calls the ioc_driver_cmd() function that is itself a blocking function, returning only when the transfer is completed.
- When the ioc_driver_cmd() function returns, the server thread reactivates the client thread, and handle the next request in the TXT waiting queue, or deschedules if the queue is empty.
Note : According to the scheduler policy, the DEV threads have an higher priority than the USR threads, and a DEV thread keep blocked when the associated waiting queue is empty.
C.2) Synchronous operations
- The dev_ioc_sync_read( xptr_t buffer_xp , uint32_t lba , uint32_t count ) blocking function moves <count> contiguous blocks from the block device, starting from block defined by the <lba> argument, to a kernel buffer defined by the <buffer_xp> argument. It calls directly the IOC driver without rescheduling, and without using the IOC device waiting queue and the server thread.
- The dev_ioc_sync_write( xptr_t buffer_xp , uint32_t lba , uint32_t count ) blocking function moves <count> contiguous blocks from a kernel buffer defined by the <buffer_xp> argument to the block device, starting from block defined by the <lba> argument. It calls directly the IOC driver without rescheduling, and without using the IOC device waiting queue and the server thread.
Almost-mkh uses the synchronous SYNC_READ and SYNC_WRITE operations in the kernel_init() function, or to synchronously update the FAT (both the FAT mapper in memory, and the FAT on IOC device), or the directory files on IOC device.
These synchronous operations use neither the IOC device waiting queue, nor the DEV server thread. The client thread does not deschedules : it registers the arguments in the IOC command structure embedded in the client thread descriptor, and calls directly the blocking ioc_driver_cmd() function, that executes the command and returns only when the transfer is completed. For a synchronous request, and even if the the hardware IOC controller has a DMA capability, the ioc_driver_cmd() function is supposed to use a polling strategy to wait the transfer completion, without using the IOC_IRQ.
D) The "driver" API
All IOC drivers must define three functions :
- void ioc_driver_init( chdev_t *ioc_chdev )
- void ioc_driver_cmd( xptr_t thread_xp )
- void ioc_driver_isr( chdev_t * ioc_chdev )
The ioc_driver_cmd() function arguments are actually defined in the ioc_command_t structure embedded in the client thread descriptor. One command contains four informations:
- type : operation type (defined below)
- count : number of contiguous blocks to be moved.
- buffer_xp : extended pointer on kernel buffer.
- lba : logic block address (index of block in hardware device).
The four operation types for the IOC driver(s) are the following:
- IOC_WRITE : move blocks from a kernel buffer to the hardware device, with a descheduling policy.
- IOC_READ : move blocks from the hardware device to a kernel buffer, with a descheduling policy.
- IOC_SYNC_WRITE : move blocks from a kernel buffer to the hardware device, with a polling policy.
- IOC_SYNC_READ : move blocks from the hardware device to a kernel buffer, with a polling policy.
For asynchronous operations, the ioc_driver_cmd() function is called by the server thread:
- When the hardware block device has a DMA capability, for an asynchronous request, the ioc_driver_cmd() function launches the DMA transfer. Then it blocks on the THREAD_BLOCKED_ISR condition, and deschedules.
- When the I/O operation completes, the hardware rises the IOC_IRQ, and the the ioc_driver_isr() function reports the I/O operation status, and reactivates the server thread.
For synchronous operations, the ioc_driver_cmd() function is called by the client thread:
- It masks the IOC_IRQ, and polls the IOC hardware status register until I/O transfer completion,
- When the I/O operation completes, it reports the I/O operation status in the command, and returns.
- The ioc_driver_isr() function and the IOC_IRQ are not involved.
