WTI mailbox attribution politic

Une version française de cette page peut être trouvée ​ici

The architecture ​TSAR has a component ​Iopic. This component permits an external peripheral to write to a specific address.

This peripheral is principally used to write to certain register of the ​Xicu component. Those registers are mailboxes, when one of them is written then the XICU emits an interrupt so the concerned core will execute the associated ISR.

We call those interruptions the Write Triggered Interruptions (WTI) because they are triggered by a write operation in the Xicu's mailboxes registers.

Mailboxes' attribution

Each cluster possess an XICU which has 16 mailboxes, 4 of them are reserved for the inter process interrupts (IPI). The fifth mailbox of the cluster 0's Xicu is also reserved, it is the DEV_NULL mailbox, its use will be explained later.

External peripherals can use 12 of this mailboxes to signal their I/O operation's end and that the ISR associated to the peripheral has to be executed.

We choose an attribution politic called "polluter-payer".

When a thread executes an I/O operation, it will be the core which executes the thread which will execute the ISR associated to that operation. For this we have to :

1. get a mailbox,
2. reconfigure the Xicu's mask,
3. link the peripheral's ISR to the mailbox

When the ISR will execute, we have to :

1. release the mailbox,
2. reconfigure the Xicu's mask to mask the interrupt,
3. unlink the peripheral's ISR to the mailbox

External peripheral's I/O operation progress

Each external peripheral is protected by a global lock, to execute an I/O operation with one of this peripheral we firstly have to get the lock.

We decided that before to take the peripheral's lock the core should assure that it got a mailbox. Indeed, it is easier to get a mailbox because there are 12 by cluster. Moreover, they are shared only by the cores of the same cluster. A contrario, the peripheral are unique so all the architecture's cores compete for their access.

Once the mailbox and the lock got we have to configure the Iopic

After the interrupt's reception the peripheral's lock and the mailbox will be released.

Moreover, the Iopic will be un-configured, that is if the peripheral sends an interrupt it will be received in a special mailbox : DEV_NULL.

The DEV_NULL mailbox is the first free mailbox of the cluster 0's Xicu, its ISR is executed by the core 0 of this cluster.

The ISR linked to this specific mailbox only prints a error message which points that an unwanted interrupt was received. This mechanism was implemented to protect the operation system against peripheral's potential dysfunction.

In the first figure, the core P1 begins an I/O operation with the disk. In addition to configure the disk, the core also configures the IOPIC so that the disk can write in the local Xicu's fifth mailbox.

When the disk will end its I/O operation, it will order the IOPIC to write in the fifth mailbox of the local Xicu so the core P1 will execute the disk's ISR.

Mailboxes' attribution API

To set up this mechanism we wrote an API, it is composed of a structure and 3 functions.

The two main functions of this API are get_configure_mailbox and put_mailbox. Indeed, they can be used for future drivers' writing.

The mbox_manager structure

This structure can be found in each cluster, it has two fields :

• a spinlock for mutual exclusion
• an array of mailboxes' state

Each entry represents the mailbox's state, those different states are :

• IPI : The box is used for IPI, it can not be used for WTI. The n first mailboxes are reserved for IPI (n is the cluster's number of core).
• FREE : The box is free an can be used for WTI.
• The id of the proprietary core.

void mbox_m_init(struct mbox_manager *mbox_m)

This function is called during the cluster's initialization, it marks the first mailboxes as reserved for IPI whereas the others are marked as free.

void get_configure_mailbox(struct mbox_manager *mbox_m, struct device_s *dev)

This function allocate a mailbox of the mbox_manager mbox_m to the device dev, then it configures the Xicu's mask and link the dev's ISR to the allocated mailbox. This functions blocks while no box was found.

int put_mailbox(uint_t wti_index, struct mbox_manager *mbox_m)

This function releases the mailbox which index is wti_index in the mbox_manager mbox_m, it also masks the interrupt and unlink the ISR to the mailbox. This function returns 0 when success or an error number otherwise.

API's advantages and disadvantages

The use of this API permits a certain flexibility. Indeed, thanks to this API it will be possible to use all the peripherals even if we have an Xicu with only one mailbox (of course the different cores will have to shared the mailbox so there will be sequentiality).

Thanks to "polluter-payer" politic the cores' caches are not spoiled by the ISR of a neighbour core.

However, the use of this API makes the I/O operation longer. Indeed, it seems that the execution of a simple program "open; read; close;" takes 40 000 cycles more compared to an I/O operation without the use of the API.