Memory allocator API change

[Nicolas Pouillon]

Introduction

SOTA

Currently, there is:

• mem_alloc API
  • mem_region pools (optional)
    • memory_allocator API

mem_alloc only takes a reachability constraint, which is the access scope. Optimization scope is not in the API and is implicit. If region are activated, a memory_allocator is chosen depending on the scope and the current CPU. Hack was added to get a foreign scope context: mem_alloc_cpu, making an indirection through a CLS.

Here we see the scope serves two purposes:

• A reachability constraint ("the allocated memory should be accessible from X")
• A proximity (optimization) constraint (implicit, or CLS-based)

This is broken

Proposed evolution

mem_alloc API should take two parameters:

• A reachability constraint
  • "The allocated memory *may* be accessed from X"
• A proxymity scope
  • "The allocated memory should preferably be close to X"
• Better error reporting. Returning NULL is not enough

Some validities:

• SYS: The whole system may access the data
  • Stacks (shm with stacked objects)
  • schedulers (anyone may push a task in it)
  • system globals
  • CLS
  • user tasks data
  • (most variables)
• DMA: This is a subset of SYS, telling devices should be able to DMA there
  • Not all memory map supports DMA
  • Some processors reserve zones for DMA (with different cacheability)
• CPU: Zone accessible solely from a CPU (scratchpad-like)
  • Can only put purely private data in it
  • No support for migration
  • Probably of no use in the kernel (maybe useful in tasks)
• CLUSTER: Zone accessible solely from a cluster
  • Like a shared scratchpad, this will probably exist in ADAM

Some proximities:

• Once per cluster (each time a cost metric changes)
• A global one
  • when allocating a truly shared data, there is no good placement choice therefore we could use a "global" proximity, taking a random memory bank

APIs

Kernel-wise API

struct mem_proximity_s
{
    error_t (*alloc)(
        size_t size,
        void *priv,
        enum mem_reachability_s valid,
        void **addr);
    void *priv;
};

enum mem_reachability_s
{
    MEM_VALID_SYS,
    MEM_VALID_DMA,
    MEM_VALID_CLUSTER,
    MEM_VALID_CPU,
};

error_t mem_alloc(
    size_t size,
    struct mem_proximity_s *prox,
    enum mem_reachability_s valid,
    void **addr);

void mem_free(void *addr);

Then we can have:

• One mem_proximity_s per cluster, taking from allocators with a preference scheme
• One global mem_proximity_s, taking one random allocator

On µC and other small designs, mem_alloc may just ignore prox and/or valid arguments.

Memory allocator API

Mostly unchanged. Just need error reporting.

We should just replace the allocating call with:

/** @this allocate a new memory block in given region */
error_t memory_allocator_pop(
    struct memory_allocator_region_s *region,
    size_t size,
    void **addr);

[Nicolas Pouillon]

There was also an idea about factorization of low-level allocator APIs:

• fine-grained memory_allocator api
• physical page allocator api

This could lead to:

Generic part

struct alloc_pool_s;

/** @this allocates a new block in given pool */
typedef error_t alloc_pool_pop_func_t(
    struct alloc_pool_s *pool,
    size_t size,
    uintptr_t *retptr);

/** @this frees allocated memory block, pool must be able to find itself */
typedef void alloc_pool_push_func_t(
    uintptr_t address);

/** @this returns the size of given memory block */
typedef size_t alloc_pool_getsize_func_t(
    uintptr_t ptr);

struct alloc_pool_model_s
{
    alloc_pool_pop_func_t *pop;
    alloc_pool_push_func_t *push;
    alloc_pool_getsize_func_t *getsize;
};

struct alloc_pool_s
{
    const struct alloc_pool_model_s *model;

    // specific implementation can supersede this structure.
};

Specific APIs

Fine-grained memory allocator

Together with the generic parts, they can add the usual other functions:

struct memory_pool_s
{
    struct alloc_pool_s alloc;
    // header chained list
    // ...
}

/**
 @this initializes a memory pool.

 if @tt *pool is NULL, it will be allocated inside the returned pool and pointer to pool set in *pool,
 if @tt *pool is non-NULL, pointed memory will be used as pool memory
 */
error_t memory_pool_init(
    void *start,
    void *end,
    struct memory_pool_s **pool);

/** @this extends an existing memory pool with a new memory range */
error_t memory_pool_extend(
    struct memory_pool_s *pool,
    void *start,
    void *end);

/** @this resizes the given memory block */
error_t memory_pool_resize(
    void *old,
    size_t size,
    void **new_block);

/** @this reserves a memory space in given region */
error_t memory_pool_reserve(
    struct memory_pool_s *pool,
    void *start,
    size_t size);

Physical page allocator

struct page_pool_s
{
    struct alloc_pool_s alloc;
    // bitmap or whatever
    // ...
}


/**
 @this initializes a page pool.
 */
typedef error_t page_pool_init_func_t(
    struct page_pool_s *pool,
    paddr_t start,
    paddr_t end, /// end is excluded
    );

/** @this extends an existing page pool with a new page range */
typedef error_t page_pool_extend_func_t(
    struct page_pool_s *pool,
    paddr_t start,
    paddr_t end, /// end is excluded
    );

Upper-level factorization

Proximity API could then be used either for page allocators or fine-grained allocators

[Nicolas Pouillon]

s/validity/reachability/ s/superset/subset/