Add a topology description API

[Nicolas Pouillon]

Overview

Goals

• Abstract the low-level description APIs
  • FlattenedDeviceTrees
  • ACPI
• Enumerate the platform
  • Component connection topology
  • IRQ routing
  • Memory configuration (Cacheability, latency, …)
• Deduce the correct OS mapping
  • For memory allocators (region API)
  • Scheduler lists repartition
  • Shortest-path IRQ routing
  • SRL tasks/resources mapping

APIs

• Explore the topology
  • Extract journey costs
• Query the topology: Get nodes by a selector
  • Proximity (get all rams that are at less than 3 hops than cpu XX)
  • Properties (get all cpus of arch XX)
  • Properties (get all devices of type XX) -- maybe redundant with hexo's device_s tree

Optional features discussion

• Do we need to handle loads accounting (NoC load, CPU, memory, …) here ? (i.e. something like "Ok, for now on I use an avg 1 MBps of NoC path from X to Y, then report to others)
  • NoC is tightly coupled to topology, this could be interesting
  • CPU is somewhat highly dynamic, even if we can sometimes predict, is it obvious here ?
  • Memory has its own allocators, do not duplicate
• If we do not, we probably need a separate accounting library
  • Maybe a-la mem_alloc(), per resource
  • So we must duplicate topology for NoC accounting
• We can also put this is model-specific calls

libTopology data model

Conceptual

It is a graph of nodes. Nodes are associated to an element of the architecture. Some node examples:

• A routing element (DSPIN NoC (as a whole)), a local interconnect
• A CPU+cache
• A memory bank
• An ICU
• A device

Each node can be associated to a device -- There is at most 1 device per node, some nodes have no associated device (e.g. NoC has no dev)

Structures

struct topo_model_s
{
  void (*init)(struct topo_node_s *node, void *param);
  void (*cleanup)(struct topo_node_s *node);

  // A bunch of function pointers for operations follows, see below
};

struct topo_node_s
{
  struct topo_model_s *model; /// model handling this node
  void *priv;                 /// model's private data
  struct device_s *dev;       /// associated device, if any
};

Needed APIs

Device to node bijection

struct device_s *topo_node_to_device(struct topo_node_s *node);

struct topo_node_s *topo_device_to_node(struct device_s *dev);

Topology exploration

There must be a high-level call to explore the path from a node to another. We will assume:

• There is one such path
• We do not cross address spaces during the exploration (route a data packet, cant transform it in an IRQ)
• There is one address type for destination

High level call which does the routing and accumulates round-trip metrics:

error_t topo_journey_metrics(
   struct topo_metrics_s *result,
   struct topo_node_s *start,
   struct topo_addr_s *dest);

(round-trip is sufficient as there is roughly the same energy involved in reads and writes, and also gives the model the choice of what a round-trip involves (totally different for a noc and a bus))

An address is a destination valid for a type of routing exploration, most of the time it will be a bus address, but may be an IRQ number, an USB device no, …

enum topo_addr_type_e
{
  TOPO_ADDR_BUS;
};

struct topo_addr_s
{
  enum topo_addr_type_e type;
  union {
    struct {
      uintptr_t address;
    } bus;
  };
};

Accumulated metrics describe the features of the walked path:

struct topo_metrics_s
{
  uint32_t hop_count;
  uint32_t latency;          /// cycles ? us ?
  uint32_t max_byte_per_sec; /// capped max on path
  uint32_t power_per_byte;   /// overall consumption estimation
};

Needed calls in model-specific functions

error_t next(
    struct topo_node_s *node,        /// node of our model where we are coming from
    const struct topo_addr_s *dest,  /// destination address for the current address space
    struct topo_metrics_s *metrics,  /// Metrics to update on the go
    struct topo_node_s **next_node); /// Next hop

Open questions

• Are edges of graph internal to node's private data or are explicit ?
• How to handle multiple output connexions of the same addressing type ?
  • Maybe add an "output" index with source node ? (after source node, there should be no hesitation possible)

[Nicolas Pouillon]

Bringing the discussion here