ticket summary component version milestone type owner status created _changetime _description _reporter 57 Rework device API: one device in many classes drivers Topology handling task Nicolas Pouillon new 2010-10-31T10:14:37+01:00 2010-10-31T11:42:53+01:00 "We currently have a problem with devices that are in more than one driver class at the same time. We have to split them in subdevices, even if it is incorrect. Some examples: * CPU nodes are also ICUs * soclib Xicu is timer and ICU There once was a discussion about allowing devices to implement more than one class at the same time. How to do this ?" Nicolas Pouillon 58 Memory allocator API change mutek Topology handling enhancement becoulet new 2010-10-31T10:54:54+01:00 2010-10-31T17:34:02+01:00 "= Introduction = == SOTA == Currently, there is: * mem_alloc API * mem_region pools (optional) * memory_allocator API `mem_alloc` only takes a reachability constraint, which is the access scope. Optimization scope is not in the API and is implicit. If region are activated, a `memory_allocator` is chosen depending on the scope and the current CPU. Hack was added to get a foreign scope context: mem_alloc_cpu, making an indirection through a CLS. Here we see the scope serves two purposes: * A reachability constraint (""the allocated memory should be accessible from X"") * A proximity (optimization) constraint (implicit, or CLS-based) This is broken == Proposed evolution == mem_alloc API should take two parameters: * A reachability constraint * ""The allocated memory *may* be accessed from X"" * A proxymity scope * ""The allocated memory should preferably be close to X"" * Better error reporting. Returning NULL is not enough Some validities: * SYS: The whole system may access the data * Stacks (shm with stacked objects) * schedulers (anyone may push a task in it) * system globals * CLS * user tasks data * (most variables) * DMA: This is a subset of SYS, telling devices should be able to DMA there * Not all memory map supports DMA * Some processors reserve zones for DMA (with different cacheability) * CPU: Zone accessible solely from a CPU (scratchpad-like) * Can only put purely private data in it * No support for migration * Probably of no use in the kernel (maybe useful in tasks) * CLUSTER: Zone accessible solely from a cluster * Like a shared scratchpad, this will probably exist in ADAM Some proximities: * Once per cluster (each time a cost metric changes) * A global one * when allocating a truly shared data, there is no good placement choice therefore we could use a ""global"" proximity, taking a random memory bank = APIs = == Kernel-wise API == {{{ #!c struct mem_proximity_s { error_t (*alloc)( size_t size, void *priv, enum mem_reachability_s valid, void **addr); void *priv; }; enum mem_reachability_s { MEM_VALID_SYS, MEM_VALID_DMA, MEM_VALID_CLUSTER, MEM_VALID_CPU, }; error_t mem_alloc( size_t size, struct mem_proximity_s *prox, enum mem_reachability_s valid, void **addr); void mem_free(void *addr); }}} Then we can have: * One mem_proximity_s per cluster, taking from allocators with a preference scheme * One global mem_proximity_s, taking one random allocator On µC and other small designs, mem_alloc may just ignore prox and/or valid arguments. == Memory allocator API == Mostly unchanged. Just need error reporting. We should just replace the allocating call with: {{{ #!c /** @this allocate a new memory block in given region */ error_t memory_allocator_pop( struct memory_allocator_region_s *region, size_t size, void **addr); }}} " Nicolas Pouillon 8 Make libelf more generic libelf enhancement Joël Porquet new 2009-11-09T13:30:56+01:00 2009-11-15T14:38:30+01:00 [wiki:ToDo/ElfLoading] Joël Porquet 16 Add an absolute time reference in hexo hexo enhancement becoulet new 2009-11-18T11:03:45+01:00 2010-08-03T11:22:25+02:00 "We need an abstracted method to retrieve the architecture's frequency. This can be useful to: * know about the real time elapsed * implement real data rates (some devices use a subdivision of the main clock for generating baudrates, that means we have to know the main clock freq) This could be declared in hexo, implemented in arch. Depending on the arch, it could be: * a calibrating loop on ibmpc * in the fdt for soclib * depdendant on the arch for simple Some problems are non-trivial to solve but may be addressed later: * CPU frequency scaling * GALS " Nicolas Pouillon 38 Add a generic synchronous command to all drivers drivers enhancement Nicolas Pouillon new 2010-07-02T16:07:43+02:00 2010-07-02T16:07:43+02:00 "This could be used to: * do a synchronous write in char devs (without IRQs) * do a configuration modification This could be seen as an in-kernel ioctl(), but ioctl is not a good name." Nicolas Pouillon 60 Le cache de fs ne compile pas drivers defect Nicolas Pouillon new 2010-12-23T17:09:35+01:00 2010-12-23T17:09:35+01:00 "En essayant de compiler MutekH avec l'option permettant d'activer le cache de blocs, j'obtiens le message suivant : /Users/alain/soc/mutekh/drivers/block/cache/block-cache.c: In function 'lba_insert_ascend': /Users/alain/soc/mutekh/drivers/block/cache/block-cache.c:38: warning: unused variable 'res' /Users/alain/soc/mutekh/drivers/block/cache/block-cache.c: In function 'lba_insert_descend': /Users/alain/soc/mutekh/drivers/block/cache/block-cache.c:38: warning: unused variable 'res' //alain/soc/mutekh/drivers/block/cache/block-cache.c: In function 'block_cache_newent': /Users/alain/soc/mutekh/drivers/block/cache/block-cache.c:71: error: 'bs' Users undeclared (first use in this function) /Users/alain/soc/mutekh/drivers/block/cache/block-cache.c:71: error: (Each undeclared identifier is reported only once /Users/alain/soc/mutekh/drivers/block/cache/block-cache.c:71: error: for each function it appears in.) " alain 69 block-file-emu fails in emu SMP arch/emu defect becoulet new 2011-07-29T11:35:16+02:00 2011-07-29T11:35:16+02:00 "Block-file-emu emulates a block device behavior by accessing a file on the host system. Processors in emu SMP mode are emulated by creating processes, done by calling '''fork()'''. But with '''fork()''', file descriptors are not passed to child processes, and thus accesses to the block device from other processors fail. On Linux hosts, the only ones on which the SMP mode is supported, we should call '''clone(CLONE_FILES)''' to create others processors in order to share all file descriptors." gut 48 Add error return codes libnetwork Libnetwork rework enhancement Nicolas Pouillon new 2010-07-06T10:25:35+02:00 2010-07-06T10:25:35+02:00 "Most calls return void. This is bad. There are plenty error causes that should be reported: * preparepkt could say desired payload is too big * pushpack could abandon the packet, or tell the interface is not configured We should see whether to use hexo error codes or libnetwork private ones" Nicolas Pouillon 43 Remove lidudp / libtcp GPCT objects libnetwork Libnetwork rework task Nicolas Pouillon new 2010-07-06T01:38:26+02:00 2010-07-06T01:38:26+02:00 We dont need GPCT objects for these Nicolas Pouillon 44 Dont pass interfaces all the way around libnetwork Libnetwork rework task Nicolas Pouillon new 2010-07-06T01:41:17+02:00 2010-07-06T01:41:17+02:00 "Transport and session layers need not to know what interfaces are used to send the packet. preparepkt and sendpkt functions could not receive the interface parameter. Moreover, the IP descriptor (for instance) is associated to an interface. Thus the interface seems redundant." Nicolas Pouillon 45 Use enums where possible libnetwork Libnetwork rework task Nicolas Pouillon new 2010-07-06T01:42:17+02:00 2010-07-06T01:42:17+02:00 Error types, flags, operations are passed through integer types and #defines. These should be enums where possible. Nicolas Pouillon 46 Always pass structures, never implicit / looked-up references libnetwork Libnetwork rework task Nicolas Pouillon new 2010-07-06T01:44:46+02:00 2010-07-06T01:44:46+02:00 "Some calls take interfaces, others interface names, others interface numbers. All calls should take pointer to interface structures. Lookup should be handled separately. Interface numbering is useless in the libnetwork, this should be a libOS thing." Nicolas Pouillon 47 Scoping of declarations and header cleaning libnetwork Libnetwork rework task Nicolas Pouillon new 2010-07-06T01:46:01+02:00 2010-07-06T01:46:01+02:00 Some headers contain definitions, constants or prototypes that should not be exported, move them to implementation files. Nicolas Pouillon 49 Move libsocket to libc libc Libnetwork rework task becoulet new 2010-07-06T14:56:10+02:00 2010-07-06T14:56:10+02:00 "Socket API is a libc thing. Nothing to do in libnetwork. This would let us merge with file descriptor things" Nicolas Pouillon 50 Move nfs to its own library libnetwork Libnetwork rework task Nicolas Pouillon new 2010-07-06T15:00:57+02:00 2010-07-06T15:00:57+02:00 "NFS (lightweight) library currently in libnetwork should move to its own library either: * at top level, * in libnetwork, * in file systems." Nicolas Pouillon 34 Handle fpu registers in ctx switch hexo Preemptive scheduler usage defect becoulet new 2010-03-24T16:11:56+01:00 2010-07-05T11:00:16+02:00 Save fpu registers on context switch for all cpu types. Use exceptions to enable fpu reg switch on a per context basis. becoulet 3 Rework boot sequence hexo Scattered initialization process task Nicolas Pouillon assigned 2009-11-04T13:16:11+01:00 2010-07-05T11:12:36+02:00 "See [wiki:ToDo/ConstantMemoryBooting] We need this to be homogenized across archs" Nicolas Pouillon 26 Handle .init section mutek Scattered initialization process task somebody new 2010-03-06T17:51:09+01:00 2010-07-05T11:12:30+02:00 Execute all functions referenced in the .init section at some point during system startup. becoulet 29 Device tree en device enumeration improvements drivers Topology handling enhancement coredev new 2010-03-12T17:29:07+01:00 2010-07-05T11:14:59+02:00 "* add a cpu device class * replace addr/irq fields in device_s by resource table with type/size/value. * use device tree to find cpu icu for ipis " becoulet 59 Devices irqs handling and irqs topology drivers Topology handling enhancement Nicolas Pouillon new 2010-11-16T00:03:55+01:00 2010-11-17T10:18:20+01:00 "Change devices IRQ handling and use an IRQ lines topology graph. API proposal in pseudo C: {{{ #!c enum irq_ep_type_e { IRQ_EP_SOURCE, IRQ_EP_SINK, IRQ_EP_MIXED }; struct irq_ep_s { /** endpoint type, may only be usefull for checking */ enum irq_ep_type_e type; /** source ep: link to device which may raise irq, sink ep: link to device which can handle irq */ struct device_s *dev; /** source ep: list of sink ep which can recieve the irq signal, sink ep: list of source ep which can relay this irq */ ep_list_root_t links; CLIST list_entry; }; /* FIXME both endpoints have the same structure, should we merge?*/ /********** devices drivers API */ /** @this is handling function of device node. @param src end point which relayed the irq @param id local identifier of irq line for relaying device, must be updated to next local id by icu devs. Must be changed to -1 when no irq were pending. Non-icu devices only set to -1 or 0. @return pointer to next icu device or NULL The next sink endpoint may be found from icu registers or passed id. On some systems the icu passes the decoded vector id to the processor in hardware and we need a way to pass the value back to icu handler from processor handler. */ #define IRQ_EVENT(n) struct irq_ep_s (*n)(struct irq_ep_s *src, \ intptr_t *id) typedef IRQ_EVENT(irq_event_t); /** irq ack function for *icu devs* only */ #define IRQ_ACK(n) void (*n)(struct irq_ep_s *src, \ intptr_t id) typedef IRQ_ACK(irq_ack_t); /********** global irq processing code */ /** @return true if irq were handled */ bool_t irq_src_process(struct irq_ep_s *src, intptr_t id) { assert(src->type != IRQ_EP_SINK); /* process irq and get next sink */ intptr_t next_id = id; struct irq_ep_s *sink = src->dev->irq_event(src, &next_id); assert(sink->type != IRQ_EP_SOURCE); if (sink == NULL) return next_id == 0; /* not an icu device */ bool_t done = irq_sink_process(sink, next_id); /* for icu devs only */ src->dev->irq_ack(src, id); return done; } bool_t irq_sink_process(struct irq_ep_s *sink, intptr_t id) { bool_t done = 0; assert(sink->type != IRQ_EP_SOURCE); /* for all source connected to icu sink, process */ FOREACH(src, sink->links) done |= irq_src_process(src, id); return done; } /********** cpu irq handling code */ static irq_ep_s cpu_irq_lines[CPU_LINES_COUNT]; CPU_INTERRUPT_HANDLER(irq_handler) { irq_sink_process(cpu_irq_lines + irq, irq); /* may check for lost irqs here */ } }}} == x86 == [[Image(irqs_x86.png)]] == Soclib == [[Image(irqs_soclib.png)]] " becoulet 40 Add a topology description API drivers Topology handling task Nicolas Pouillon new 2010-07-05T11:14:45+02:00 2010-10-31T10:15:31+01:00 "= Overview = == Goals == * Abstract the low-level description APIs * [wiki:FlattenedDeviceTree FlattenedDeviceTrees] * ACPI * OpenFrimware * UEFI ? * Enumerate the platform * Component connection topology * IRQ routing * Memory configuration (Cacheability, latency, …) * Deduce the correct OS mapping * For memory allocators (region API) * Scheduler lists repartition * Shortest-path IRQ routing * SRL tasks/resources mapping == APIs == * Explore the topology * Extract journey costs * Query the topology: Get nodes by a selector * Proximity ({{{get all rams that are at less than 3 hops than cpu XX}}}) * Properties ({{{get all cpus of arch XX}}}) * Properties ({{{get all devices of type XX}}}) -- maybe redundant with hexo's device_s tree = Optional features discussion = * Do we need to handle loads accounting (NoC load, CPU, memory, …) here ? (i.e. something like ""Ok, for now on I use an avg 1 MBps of NoC path from X to Y, then report to others) * NoC is tightly coupled to topology, this could be interesting * CPU is somewhat highly dynamic, even if we can sometimes predict, is it obvious here ? * Memory has its own allocators, do not duplicate * If we do not, we probably need a separate accounting library * Maybe a-la mem_alloc(), per resource * So we must duplicate topology for NoC accounting * We can also put this is model-specific calls = libTopology data model = == Conceptual == It is a graph of nodes. Nodes are associated to an element of the architecture. Some node examples: * A routing element (DSPIN NoC (as a whole)), a local interconnect * A CPU+cache * A memory bank * An ICU * A device Each node can be associated to a device -- There is at most 1 device per node, some nodes have no associated device (e.g. NoC has no dev) == Structures == {{{ #!c struct topo_model_s; // forward decl. see below struct topo_node_s { const struct topo_model_s *model; /// model handling this node void *priv; /// model's private data struct device_s *dev; /// associated device, if any }; /** The only creteria telling two ports are different is private must be different. For models not using private, an useless but different value must be set. */ struct topo_port_s { const struct topo_node_s *node; // node owning the port void *private; // port's private. }; }}} = Needed APIs = == Device to node bijection == {{{ #!c struct device_s *topo_node_to_device(const struct topo_node_s *node); struct topo_node_s *topo_device_to_node(const struct device_s *dev); }}} == Topology exploration == There must be a high-level call to explore the path from a node to another. We will assume: * There is one such path * We do not cross address spaces during the exploration (route a data packet, cant transform it in an IRQ) * There is one address type for destination High level call which does the routing and accumulates '''round-trip''' metrics: {{{ #!c error_t topo_journey_metrics( const struct topo_node_s *start, const struct topo_addr_s *dest, struct topo_metrics_s *result); }}} (round-trip is sufficient as there is roughly the same energy involved in reads and writes, and also gives the model the choice of what a round-trip involves (totally different for a noc and a bus)) An address is a destination valid for a type of routing exploration, most of the time it will be a bus address, but may be an IRQ number, an USB device no, … {{{ #!c enum topo_addr_type_e { TOPO_ADDR_BUS; TOPO_ADDR_IRQ; }; struct topo_addr_s { enum topo_addr_type_e type; union { struct { uintptr_t address; } bus; struct { size_t line_no; } irq; }; }; }}} Accumulated metrics describe the features of the walked path: {{{ #!c struct topo_metrics_s { size_t hop_count; uint32_t latency; /// cycles ? µs ? uint32_t max_byte_per_sec; /// capped max on path uint32_t power_per_byte; /// overall consumption estimation }; }}} == Selection function == {{{ #!c enum topo_crit_val_e { TOPO_CRIT_DROP, /// Node is not interesting, stop recursion here in the graph TOPO_CRIT_RECURS, /// Node is not selected, but we can recurs through it TOPO_CRIT_SELECT, /// Node is selected, and may be recursed down }; enum topo_cmp_e { TOPO_CMP_LEFT = -1, /// left better than right TOPO_CMP_EQ = 0, /// left equals right TOPO_CMP_RIGHT = 1, /// right better than left }; struct topo_select_item_s { const struct topo_node_s *node; struct topo_metrics_s metrics; // not a pointer ! }; /** Selector function @param item Node/metrics to filter @param priv Selector private data @returns whether the node is interesting */ typedef enum topo_crit_val_e topo_crit_func_t( const struct topo_select_item_s *item, const void *priv); /** Sort function @param left Node/metrics to compare @param right Node/metrics to compare @param priv Sort private data @returns comparaison result */ typedef enum topo_cmp_e topo_sort_func_t( const struct topo_select_item_s *left, const struct topo_select_item_s *right, const void *priv); error_t topo_select( const struct topo_node_s *start, /// Start node topo_crit_func_t *crit, const void *crit_priv, /// Criterium function, telling whether a node is eligible topo_sort_func_t *sort, const void *sort_priv, /// Sorting function, telling order of nodes size_t max_items, /// Maximal number of nodes to return struct topo_select_item_s *items); /// Output nodes+metrics. Table must be allocated by caller and be able to contain @tt max_items }}} == Needed calls in model-specific functions == {{{ #!c /** Initialize a topology node. */ typedef void topo_model_init_func_t(struct topo_node_s *node, const void *param); /** Destroy a topology node. Reclaim all associated memory */ typedef void topo_model_cleanup_func_t(struct topo_node_s *node); /** Compute routing and cost from this node to the next hop, routing towards a given destination @param node Node of our model where we are coming from @param input Input port in node. May be NULL on start node @param dest Destination address for the current address space @param metrics Metrics to update on the go @param peer_input Input port of the next hop */ typedef error_t topo_model_journey_next_func_t( const struct topo_node_s *node, const struct topo_port_s *input, const struct topo_addr_s *dest, struct topo_metrics_s *metrics, struct topo_port_s *peer_input); /** Explore the topology graph. Each call to this function must return next valid port. @param node Current node @param input Port where we came from, exploration should not ""go back"" to this edge. May be NULL on recursion start @param metrics Metrics up to this node @param item Returned item for next recursion @param state Pointer to a void* where select_next can store its internal state. @tt *state is @tt NULL on first call for each node. Successive calls to select_next for a given node in a given exploration use the same @tt state value. @note As there is no warning for end of iteration, there is no way of calling @tt free(), so @tt *state should be a pointer to preallocated internal data. */ typedef error_t topo_model_select_next_func_t( const struct topo_node_s *node, const struct topo_port_s *input, const struct topo_metrics_s *input_metrics, struct topo_select_item_s *item, void **state); }}} = topo_model_s struct = {{{ #!c struct topo_model_s { topo_model_init_func_t *init; topo_model_cleanup_func_t *cleanup; topo_model_journey_next_func_t *journey_next; topo_model_select_next_func_t *select_next; }; }}} " Nicolas Pouillon 55 Add a clock driver API drivers Topology handling task Nicolas Pouillon new 2010-08-03T11:22:00+02:00 2010-08-03T11:22:00+02:00 "Microcontrollers have clock-gating features, we also have DVFS platforms in ADAM, therefore, we should think of an API that can handle clock modification for parts of the system. This is probably linked to topology" Nicolas Pouillon 5 Add float support to the math library libm enhancement Joël Porquet new 2009-11-04T14:05:47+01:00 2009-11-04T14:05:47+01:00 "In usual libc, there are at least two ways of doing this: * providing float wrappers which will only cast the arguments and the result: it changes nothing to the memory consumption though (double are still used inside math functions) * providing real float functions (NetBSD seems to have files we can reuse)" Joël Porquet 19 Change X86 memory discovering cpu/x86 enhancement becoulet new 2009-12-23T17:19:30+01:00 2009-12-23T17:19:30+01:00 Use the e820 table provided by the BIOS for discovering the available memory. refauvel 51 Implement uITRON os library global enhancement becoulet new 2010-07-11T11:58:03+02:00 2010-07-11T11:58:03+02:00 "Add libuitron. http://www.ertl.jp/ITRON/spec-e.html " becoulet 53 API to attach priorities to irq lines drivers enhancement Nicolas Pouillon new 2010-07-30T18:44:10+02:00 2010-07-30T18:44:10+02:00 We could modify the ICU driver API to include priority setting to an irq. RT support needs this modification. gut 56 Add kernel objects registry mutek enhancement becoulet new 2010-08-31T16:04:13+02:00 2010-08-31T16:04:13+02:00 Some kernel objects (like slab) may need global registry. Basic functions are list on this wiki page https://www.mutekh.org/trac/mutekh/wiki/KernelObjectRegistry refauvel 54 Memory allocator cleanup mutek task becoulet new 2010-07-31T00:45:31+02:00 2010-07-31T00:45:31+02:00 " * remove memory_allocator_reserve function * remove experimental (files + tokens) * comment complex if tree in push/resize " becoulet 22 Implement per-CPU IRQ stacks for faster IRQ handling hexo Preemptive scheduler usage enhancement becoulet new 2010-03-01T02:59:29+01:00 2010-07-11T12:00:46+02:00 That would be the extension of #11 Nicolas Pouillon 37 Add a Worker thread API mutek Preemptive scheduler usage enhancement becoulet new 2010-06-16T00:39:18+02:00 2010-08-02T13:56:42+02:00 "We sometimes use a worker-thread-like service. Each time it is recreated from scratch. Let's add an API for this. Here is such a proposal: worker.h header: {{{ #!c /** @this is an item pushable to a worker queue. */ typedef CONTAINER_ENTRY_TYPE(CLIST) worker_entry_t; /** @this is a worker function prototype macro. @csee worker_func_t */ #define WORKER_FUNC(n) void (n)(struct worker_thread_s *worker, \ worker_entry_t *entry, \ void *priv) /** @this is a worker function type definition. Function must return once it handled the reason of wakeup. @param worker Worker thread context, may be used to stop self @param entry Item to work on @param priv Private data passed on init */ typedef WORKER_FUNC(worker_func_t); void worker_thread_init(struct worker_thread_s*, void *stack, size_t stack_size, worker_func_t *func, void *func_priv); /** @this runs a context, i.e. this makes the worker ready for wake up. This waits until a work is pushed. */ void worker_thread_start(struct worker_thread_s*); /** @this kills the context, but waits for completion of all actions pending. */ void worker_thread_stop(struct worker_thread_s*); /** @this makes the worker function to be called once. Multiple calls to this function yields multiple consecutive calls to the worker function. @param item The item to pass to the woken-up function @returns -EAGAIN if the thread is not started yet, -EINVAL if the thread already got killed, 0 if OK. */ error_t worker_thread_wakeup(struct worker_thread_s*, worker_entry_t *entry); }}} worker.c file content {{{ #!c struct worker_base_s { worker_entry_t entry; }; CONTAINER_TYPE(worker_queue, CLIST, struct worker_base_s, entry); CONTAINER_FUNC(...) struct worker_thread_s { worker_queue_root_t queue; struct sched_context_s context; bool_t killed; lock_t lock; worker_func_t *func; void *func_priv; }; ... { worker_queue_push(&wt.queue, (worker_queue_item_t*)entry) } }}} Usage example: {{{ #!c struct myjob_s { ... my stuff ... worker_entry_t entry; }; CONTAINER_TYPE(myworker_queue, CLIST, struct myjob_s, entry); void foo() { static struct myjob_s job; worker_thread_wakeup(worker, &job.entry); } static WORKER_FUNC(myfunc) { struct myjob_s *job = myworker_queue_item(entry); } }}} " anonymous