In this section, we describe the scheduling mechanisms used by ALMOS-MKH.

A. General principles

There is one private scheduler per core, and ALMOS-MKH does not support thread migration: threads are assigned to a given core at thread creation, and will never be executed by another core, until thread destruction. ALMOS-MKH implement a preemptive policy for time sharing between all threads assigned to given core. The <cpu_context> and <fpu_context> fields in the thread descriptor define the storage required to save (or initialize) the core registers values when the thread is not running. A fixed number of TICK periods is allocated to a running thread. The value of the TICK period (in milli-second) is defined by the CONFIG_SCHED_TICK_MS_PERIOD parameter. The number of TICKS is defined by the CONFIG_SCHED_TICK_MS_PERIOD configuration parameter. Finally, the max number of threads assigned to a given core is defined by the CONFIG_SCHED_MAX_THREADS_NR configuration parameter.

B) Thread states

ALMOS-MKH defines two types of threads:

• USER threads are POSIX compliant threads, defined in a given user process. A main thread is always created for an user process. Other threads are created by the pthread_create() syscall.
• KERNEL threads implement kernels services, such as RPC threads executing the Remote Procedure Calls, the DEV threads implementing IO channels operations, or the IDLE thread that is a default (low-power) thread.

From the scheduler point of view, any thread (KERNEL or USER) can be in three states:

• RUNNING : the thread is running on the core. The scheduler <current> field contains a pointer on the running thread.
• READY : the thread is ready to execute, but is not currently running.
• BLOCKED : The thread is blocked on a given condition, and cannot be selected for execution.

The thread state is implemented as the <blocked> bit-vector field in the thread descriptor. A thread generally enter in the BLOCKED state, when a given resource is not available, by calling the thread_block() function that set the relevant bit in the <blocked> bit-vector. It returns to the READY state when another thread releases the blocking resource, and call the thread_unblock() function, that reset the relevant bit. The thread_unblock() function can be called by any thread running in any cluster.

This simple blocking / unblocking mechanism is well suited to the Multi-Kernel-Hybrid architecture, as it is uses simple remote_write accesses.

C) Scheduling policy

Each scheduler maintains two separate, circular, lists of threads: one list of KERNEL threads, and one list of USER threads. A KERNEL threads have a higher priority than the USER threads, and each list is handled with a round-robin priority. When the sched_yield() function is called to perform a context switch for a given core, it implement the following policy:

1. It scan the KERNEL list to find a READY thread. It executes this KERNEL thread if found.
2. If no KERNEL thread is found, it scan the USER list to fin a READY thread. It executes this USER thread if found.
3. If there is no KERNEL thread and no USER thread (other than the calling thread), the calling thread continues execution. if possible.
4. If there is no READY thread, it executes the IDLE thread.

The kernel has the possibility to force the selection of a given thread, identified by the sched_yield() function argument. This is used to reduce the latency of the RPCs.

D. Delayed Context Switches

Context switches can have two causes: The RUNNING thread can explicitly ask to be descheduled. When such a thing happens, it is guaranteed that the current thread does not hold any kernel locks. But the scheduling policy being preemptive, the RUNNING thread can hold one (or several) kernel lock(s) when receiving a TICK interrupt.

In this situation, ALMOS-MKH implements the following delayed context switch mechanism:

1. The RUNNING thread holding locks continues execution, but the THREAD_FLAG_SCHED is set in the running thread descriptor.
2. All kernel functions used to release kernel locks check this THREAD_FLAG_SCHED. When this flag is set, and the last lock is released by the calling thread, the sched_yield() is immediately executed.

More generally, ALMOS-MKH supports descheduling of an USER thread that is currently in kernel mode - as a result of a syscall. In other words, sys calls can be interrupted by interrupts signaling the completion of an I/O operations, or by the TICK interrupt, requiring a context switch.

E) Floating Point Unit

To be Documented.