source: trunk/kernel/doc/40bits

Last change on this file was 1, checked in by alain, 9 years ago

First import
File size: 21.8 KB
Line 
11-Event Manager:
2The even manager has been changed in the fallowing ways :
3 - the API has been changed to avoid given a pointer to the second argumern to event_send we give only the
4 the gpid of the target processor.
5 - local event handling is gloablly the same
6 - remote event has been changed. Must be carrefully handled with physical access:
7 - the kfifo of the remote cpu is physically accessed
8 - the event_notifier must notify after copying the event structure (on the stack ?).
9 !!! This does not aplying that the pointer arguments are copied !!!
10 => modify event handler
11
123-Processes/Thread Management
13
14 3.1) Task Management
15
16 There's one gloabal table that index all process by pid.
17 Replicaing the kernel will also replicate this structure...
18 The fatest way to corrdinate the replica :
19 - Consiste of using the table of cluster 0 as the main table. Each table will contain
20 a pointer to the task structure if the task is local, other ways the table entry is
21 either marked and contains the cluster id wich handle the task or a NULL value.
22 A NULL value at the main table mean that the entry has not been allocated.
23 A NULL value in other table must be confirmed by cheching the main table.
24
25 This imply a change in pid allocate, wich must allocate from the main table by setting
26 the cluster id and marking the entry with a remote flag (0x1). // TODO more on synch
27
28 The lookup need to check for the task in local, if we find a task address we return' it.
29 If we find a remote entry we go to the indicate cluster. If the remote cluster does not contain
30 the address of the task ...
31
32
33 Excuse me! the fastest way is to directly use the main table as the only table and ignore other tables!!!
34 This will require putting a physical addresse on the table to point to task structures and synchronised
35 with a lock.
36
37 3.2)Task descriptor
38
39 This structure contain the following structures that are shared between threads:
40 - vmm (Virtual Memory Manager) : see 2-.
41 - pointer to root/cwd file, fdinfo and binary file: use physical addresses
42 - pointer to task parent: use a parent cluster id field, or directly a physical addresse ?
43 - 2 list : one for the children and one for the siblings
44 - a signal manager (NULL par default ?)
45 - thread handling structures :
46 - a table containing a reference to all the threads (and a pointer to it's page struct)
47 - a list entry root that englobe all thread: not really used, can be deleted
48
49 How to manipulate these fields ? => by message passing ?
50 for thread's pointer we directly placed the ppn: thread's structures are aligned on a page.
51 Other pointers fd_info, bin are to locals strutures.
52 vmm handling is local if a single threaded task, otherwise the handling is done by message passing.
53
54 3.3) Thread Management
55 In the second stage...
56
57
58
592-Memory Management:
60 vmm: message passing to the processor that contain the main structure.
61 There is no need to acess the thread data except for statistique pupose!
62 There is a part that could be done by the thread with passing a message... look in the page table part
63
64 To think : page fault at the same addresses how to avoid them (merge the messages of page fault) ?
65
66 kmem: all remote allocation are forbiden except for user pages (at least for now).
67 (user remote allocation have been done in the second stage.)
68
69
703-Cluster Manager:
71 One cluster manager for each cluster: no need for the global table...no need for the replication of cluster addresses
72 ...no needs for procs base addresses: the cpu_s are directly put into the cluster structures with the cluster base
73 address the compiler will offset to the good addresse.
74
75
763-DQDT:
77 initialisation has been modified to not require the remote allocation API. We set a table of two dqdt in the
78
79
80
814-Drivers
82
83
84
85
865-Boot:
87 A) The main proc of the boot cluster enter the boot loader, initiliase the stack bases for
88 all other procs in the boot_tbl and wait for all other proc to copy the needed info (kernel, boot,
89 boot info) from the boot cluster.
90
91 B) All other procs enter the boot_loader allocate a stack a the end of the cluster ram and
92 wait for their cluster to be initilised by their main proc. (except for those of the boot cluster ?)
93
94 C) The main procs of other clusters enter the boot loader, set the sp register to point to
95 the stack at the end of the current cluster ram.
96 (Enter the C code and) Copy the kernel/boot/boot_info from the main cluster to their cluster
97 increment a value and sleep back (wait instruction).
98
99 D) All procs sleep after incrementing a counter in the in the original BI (boot cluster).
100
101 E) Once the counter reach the number of proc, the main proc of the boot cluster can then
102 copy the kernel/boot/boot_info in the same cluster but at the correct adresses. This will erase the
103 code of the preloader. (the sp pointer of the boot cluster is good, the preloader should set it to
104 point
105
106 !!! moving the boot code is dangerous, since we are currently executing it ... and the
107 addreses in it are fixed ?!!! If we really want this space, we need to give up at this stage on the
108 boot code, BI can be kept.
109
110 ***********************************************************************************************************
111 Initital memory content at boot (TODO):
112
113 ------------------ 0x0000 0000
114 Preloader
115 ------------------ ENDPRELOADER
116 Kernel
117 ------------------ END kernel paddr (important here, the charging addresse (paddr) is differente than vaddr)
118 BI
119 ------------------
120 Boot code
121 ------------------ End of boot
122 .....
123 ------------------ end of cluster space - 4 boot stacks
124 Boot stacks (x nbproc)
125 ----------------- End of cluster space
126
127
128 Boot code role is to set:
129 1 - the kernel and boot info: deplace it to the address 0
130 2 - the stacks for each proc
131 3 - initialise the info structure (allocated on the stack): don't forget to put the BI info address
132 that resulted from displacing it.
133 4 - initialise the page table and the mmu if we are booting in virtual mode
134
135 ***** Phase 1: boot proc of the boot cluster
136 - after the preloader, there's only the boot_proc who has a stack and all other proc are waiting (an WTI).
137 - jump to C code
138 - wakeup all proc (possible improvement: set a distributed wakeup)
139
140 ***** Phase 2: all proc (the boot cluster proc)
141
142 - start by setting a stack at the end of current cluster
143 - jump to C code
144 - set info structure
145 - copy the kernel and BI info (only one proc per cluster) at the start of the current cluster
146 (- set the page table and the mmu)
147 - jump to the kernel (of the cuurrent cluster) and wait in it ? yes wait for the
148 kernel image of cluster 0 to be set (since we will probably synchronise with it)
149
150
151 ? how to signal to the current cluster proc to jump to kernel ?
152
153 --> the boot code is now used by only the boot cluster
154 --> the preloader code is used by only the boot cluster other procs
155
156 ***** Phase 3:
157 - put a barrier to be sure that all procs have entred the boot code
158 - all proc set their stack: to put after the boot proc stack
159 (- set the page table and the mmu)
160
161 --> the preloader code is no longer in use
162
163 - copy the kernel and BI of the boot cluster (by the boot proc)
164 - all proc of the boot cluster can now jump to the kernel and wait
165 - the boot proc enters the kernels and wake up a proc per cluster (except for the current
166 boot cluster)
167
168 --> the boot code is no longer in use
169
170
171 !! be careful the stack of all the procs are still in use !!
172 put the in the reserved zone ?
173
174 N.B.: all addresses of the boot cluster are found using compiled in variables
175
176
177 ****** It's up to the kernel now...
178
179Kernel boot:
180 ***** Phase 1: only the boot proc
181 wake up one proc per cluster
182 !!! Must be sure that they are sleeping (waiting) ? !!!
183
184 ***** Phase 2: A proc per cluster
185 Tasks: '->' indicate dependance
186
187 1) initialise task bootstap and task manager and set idle arg
188 reported: devfs/sysfs roots
189
190 2) initialise arch_memory : (initialise boot_tbl: done in the boot phase); initialise
191 clusters_tbl ?
192 initialise arch: initialise current cluster: initialise ppm (physical page memory) (can
193 alloc now!), cpu_init : event listener!
194 initialise devices (printk) (put all in cluster devices lists, root all
195 interruptions locally (to the proc handling them));
196 and register them (!!!to be reported!!!) -> depend on message api
197
198
199 3) --Barrier--: here we should wait that everybody has set there event listener.
200
201 only main proc of I/O cluster: initialise fs cache locally(?);
202 initialise devfs/sysfs roots and set devices.
203 ----Barier ?------
204
205
206 4) task_bootstarp_finalize : should not be needed all should be done in task_init (TODO)
207 kdmsg_init : a set of lock (isr, exception, printk)_lock and printk_sync
208
209 5) DQDT init: arch_dqdt_init and dqdt_init (TODO)
210
211 6) Forget about bootstrap_replicate_task and cluster_init_table
212 a local cluster_init_cores is needed ?
213
214 register devices in fs ?
215
216 **** Phase 3: all procs (thread idle)
217
218 set thread_idle for each proc (parrallèle ?)
219
220 load thread_idle: free reserved page (the boot loader reserved pages...), create the event
221 thread event manager, kvfsd only in the cluster_IO.
222
223
224 *** Phase 4: kvfsd (only main cpu of cluster IO)
225
226 enable all irq.
227
228 initialise fs -> need __sys_blk
229 task load init require devfs file system
230 if failed set kminishell
231
232
233
234
235
236
237
238
239
240Kernel Runing Modifications:
241
242 *** DQDT (Ressource handling): Must be set using physical addresses ?
243
244 *** FS support: Open, read, write ... all physical access or just when reaching the device ?
245
246 *** Memory support: when there is no multithreaded application (only kthread) the memory
247 management is all done localy. Cross cluster messaging for remote creation.
248
249 *** Task support: a replicated taskmanager, pids are now composed of XYN, X and Y are the
250 cluster offset, N is the offset in the current cluster. Remote creation can be done
251 only using message passing.
252
253 *** Device support:
254
255 *** Exception, Interruptions (not syscalls): all device interruptions are rooted and handled
256 locally, as configured at boot time.
257
258 Add cluster span to the arch config or in the cluster_entry. Yes it must be defined by the
259 boot code since this the code that decide the offset when using virtual memory.
260
261 *** User space support:
262 - Syscall: switch adressing mode in case we come or go from a syscall; save the coproc2 extend register
263 - copy from/to user must be modified
264
265
266Developement process:
267 ********** Phase one *******************
268 Consiste of boting the kernel with no user space support and with a dummy dqdt suport
269
270 *** Stage 0:
271 Rewrite the kernel in the view of passing to 40 bits:
272 Handle gloabal variables:
273 1) remove inused gloabl variables
274 2) simplifie the message passing API
275 3) do a simplified DQDT : replace dqdt files by a dummy : next_cluster++
276 4) task and thread creation only using message passing
277 5) do a true copy_from_usr and copy_to_usr !
278 6) delete the assumption that all address superior to KERNEL_OFFSET belongs
279 to the kernel(genarally used to verify if an address belongd to user space)
280 replace by checking that the address is between USR_OFFSET and _LIMIT
281 (see libk/elf.c) for an example.
282
283
284 *** Stage 1:
285 Develloping boot code: require setting the disk, develloping virtual memory for
286 the perspective of a dooble mode of booting, replicate the kernel and put it
287 at address zero.
288 Develloping kernel boot code (reaching kern_init and printing a message ?), setting boot_dmsg
289
290 *** Stage 2: (TODO: synchronise with stage 0)
291 Rewrite of most of the kernel:
292 ******* GOALS:
293 1) delete (replace) reference to (unused) global variables:
294 - cluster_tbl : no such a thing as cluster init...
295 reference to cluster struct should be removed to keep only cids: this will affect allot of code!
296 No more a access to remote listener ptr (event.c), nor cpu struct, not cluster, nor ppm of remote cluster ...
297
298 - devfs_db replaced by the root dentry/inode of the devfs file system and since
299 there is only one kernel that use it => allocate it dynamically
300 - similar for sysfs_root_entry ?
301 - devfs_n_op/devfs_f_op : are const
302 - (dma,fb,icu,iopic,sda,timer,xicu,tty)_count are per cluster the
303 new naming scheme is device_cidn
304 - rt_timer: Not used => removet it
305 - (exception, isr, printk, boot)_lock: are all used to lock the printing
306 - kexcept_tty, klog_tty, tty_tbl, kisr_tty, kboot_tty ???
307
308 - soclib_(any device)_driver are RO: can we mark them as const ?
309 - boot_stage_done: not used
310
311 2) Event Manager to use physical adresses.
312 3) fs
313 4) all interruption (produced by device and WTI) are handled locally
314 5) all exception are handled locally (there is no remote page fault, since
315 there is no user space => no multithreaded user space application)
316 6) Complete the kernel boot phase.
317 7) Context switch: adding the saving of the coproc registers: DATA_extend register and Virtual
318 mode register
319 ?cpu_gid2ptr: should be modified to fetch only local cluster, if a remote cluster is called, it should return an error
320
321 *** Stage N: user space support
322 $27 is now used by the kerntry code! user space should no longer used it
323 Switch to user space need to modify to set off the MMU and reset when going back...
324 set in init context the mmu register value (0xF)
325
326***Structures modif (with study only struct fields)****
327
328** NOTATIONS:
329-PB: problem,
330-HD: handled in the corresponding structure. This is for structure that are
331directly embeded in the structure and wich contain a non local pointer
332-NPB: no problem
333-LPTR: local pointer
334-RPTR: remote pointer
335-RPTR-S: RPTR that is static => point to a structure that cannot be moved across clusters
336-RPTR-D: RPTR that is dynmic=> point to a structure that can be moved across clusters
337(which mean that the ptr need to be updated!)
338
339
340** Structures that need to accessed remotly (and does they need to be dynamic ?)
341struct task_s;
342struct thread_s;
343
344
345** Structures details
346
347 + thread structure (thread.h)
348 struct thread_s
349 {
350 struct cpu_uzone_s uzone;//NOPB
351 spinlock_t lock;//PB:HD
352 thread_state_t state;//NOPB
353 ...//NOPB
354 struct cpu_s *lcpu;//NOPB: LPTR /*! pointer to the local CPU description structure */
355 struct sched_s *local_sched; //NOPB LPTR /*! pointer to the local scheduler structure */
356
357* struct task_s *task; //PB: RPTR
358
359 thread_attr_t type;//NOPB /*! 3 types : usr (PTHREAD), kernel (KTHREAD) or idle (TH_IDLE) */
360 struct cpu_context_s pws;//NOPB /*! processor work state (register saved zone) */
361
362* struct list_entry list;//PB /*! next/pred threads at the same state */
363* struct list_entry rope;//OB /*! next/pred threads in the __rope list of thread */
364 struct thread_info info;//HD /*! (exit value, statistics, ...) */
365
366 uint_t signature;//NOPB
367 };
368 struct thread_info
369 {
370 ...//NOPB
371 struct cpu_s *ocpu;//PB: RPTR-S
372 uint_t wakeup_date;//NOPB /*! wakeup date in seconds */
373 void *exit_value;//NOPB /*! exit value returned by thread or joined one */
374* struct thread_s *join;//PB:RPTR-D /*! points to waiting thread in join case */
375* struct thread_s *waker;//RPTR-D
376 struct wait_queue_s wait_queue;//PB:HD
377* struct wait_queue_s *queue;//PB
378 struct cpu_context_s pss;//NPB /*! Processor Saved State */
379 pthread_attr_t attr;//HD
380 void *kstack_addr;//NPB:LPTR (always moved with the thread)
381 uint_t kstack_size;//OK
382 struct event_s *e_info;//never used!!!!!!!!!!!
383 struct page_s *page;//LTPR ?
384 };
385 typedef struct
386 {
387 ...//NOPB
388 void *stack_addr;//user space ?
389 size_t stack_size;
390 void *entry_func;//USP /* mandatory */
391 void *exit_func;//USP /* mandatory */
392 void *arg1;//?
393 void *arg2;//?
394 void *sigreturn_func;//?
395 void *sigstack_addr;//USP todo
396 size_t sigstack_size;//?
397 struct sched_param sched_param;//PPB
398 ...//NOPB /* mandatory */
399 } pthread_attr_t;
400
401 + task structure
402 struct task_s
403 {
404 /* Various Locks */
405 mcs_lock_t block;//HD
406 spinlock_t lock;//HD
407 spinlock_t th_lock;//HD
408 struct rwlock_s cwd_lock;//HD
409 spinlock_t tm_lock;//local:HD
410
411 /* Memory Management */
412 struct vmm_s vmm;//HD
413
414 /* Placement Info */
415 struct cluster_s *cluster;//local
416 struct cpu_s *cpu;//local
417
418 /* File system *
419 //for cwd and root they should always point to the same
420 //file struct (not node) to avoid ensurring the coherence
421 //if one of them change
422 struct vfs_inode_s *vfs_root;//
423 struct vfs_inode_s *vfs_cwd;//
424 struct fd_info_s *fd_info;//embed it
425 struct vfs_file_s *bin;//Static: embed it
426
427 /* Task management */
428 pid_t pid;//NPB
429 uid_t uid;//NPB
430 gid_t gid;//NPB
431 uint_t state;//NPB
432 atomic_t childs_nr;//NPB
433 uint16_t childs_limit;//NPB
434
435 //make them of type faddr
436 struct task_s *parent;//PB
437 struct list_entry children;//PB
438 struct list_entry list;//PB
439
440 /* Threads */
441 uint_t threads_count;
442 uint_t threads_nr;
443 uint_t threads_limit;
444 uint_t next_order;
445 uint_t max_order;
446
447 //add a function to register after we creat it ?
448 BITMAP_DECLARE(bitmap, (CONFIG_PTHREAD_THREADS_MAX >> 3));
449 struct thread_s **th_tbl;//PB: TODO: local table of pointer to avoid the page below?
450 struct list_entry th_root;
451 struct page_s *th_tbl_pg;//NPB: local (make it local)
452
453 /* Signal management */
454 struct sig_mgr_s sig_mgr;
455
456 #if CONFIG_FORK_LOCAL_ALLOC
457 struct cluster_s *current_clstr;
458 #endif
459 };
460
461
462
463
464
465File per file modif:
466 *** kern dir files:
467*PKB - kern_init.c (see up)
468 - atomic.c: handle atomic_t and refcount_t objects (try to keep them local ? It will
469 be hard (fs, event handler?))
470 - barrier.c: used only by sys_barrier.c
471 - blkio.c: this layer allow an fs to send request to block device, all request pass
472 here.(manipulate a device struct. With this it's best to replicate the notion
473 of device ? NO). Use blkio async to send an IPI blokio request.
474 - cluster.c : some init functions, cid2ptr (to delete ?), manager, key_op (never used)
475 - cond_var.c: used only by user space (at least for now)
476 - cpu.c : the function cpu_gid2ptr access the cluster structure! see the structure for more
477 - do_exec : use copy_uspace!
478 - do_interupt : remove thread migration handling ?
479 - do_syscall : (N.B. the context save is for the fork syscall) remove thread migration ?
480 - event : modify to support the ...
481 - (keysdb/kfifo/mwmr/mcs_sync/radix/rwlock/semaphore/spinlock).c: their modification depend on the scope in which their are used
482 - kthread_create: local
483 - rr-sched: should be local
484 - scheduler: (thread_local_cpu) should be local
485 - task.c: should get simplified => removing replicate calls (they should may be used to see how the migration goes)
486 - thread_create.c: use of cpu_gid2ptr, should be local
487 - destroy|dup|idle.c: should be local
488 - thread migrate.c: need to be modified
489 SYSCALLS
490 Miscelanious:
491 - sys_alarm.c: manipulte local data (And thread migration is initiated by each thread indepndantly...)
492*PKB - sys_barier.c: not local, to complexe for no reason except performance, we should pospone this functionnality?
493 I think we need such a struc for the boot. This mean that posponing is not possible, but we can simplify it
494 - sys_clock.c: local
495 - sys_cond_var.c:
496 - sys_dma_memcpy.c:
497 - sys_rwlock.c
498 - sys_sem.c
499 Task:
500 - sys_exec.c
501 - sys_fork.c
502 - sys_getpid.c
503 - sys_ps.c
504 - sys_signal.c
505 - sys_thread_create.c
506 - sys_thread_detach.c
507 - sys_thread_exit.c
508 - sys_thread_getattr.c
509 - sys_thread_join.c
510 - sys_thread_migrate.c
511 - sys_thread_sleep.c
512 - sys_thread_wakeup.c
513 - sys_thread_yield.c
514 Task Mem:
515 - sys_madvise.c
516 - sys_mcntl.c
517 - sys_mmap.c
518 - sys_sbrk.c
519 - sys_utls.c
520 FS:
521 - sys_chdir.c:
522 - sys_close.c:
523 - sys_closedir.c:
524 - sys_creat.c:
525 - sys_getcwd.c
526 - sys_lseek.c
527 - sys_mkdir.c
528 - sys_mkfifo.c
529 - sys_open.c
530 - sys_opendir.c
531 - sys_pipe.c
532 - sys_read.c
533 - sys_readdir.c
534 - sys_stat.c
535 - sys_unlink.c
536 - sys_write.c
537 mm/:
538 Task related:
539 vmm.c: munmap (region:detach, split, resize); mmap (shared, private | file, anomymous);
540 skbrk(resize heap region TODO: use the resize func), madvise_migrate, vmm_auto_migrate,
541 (modify page table attribute and set migrate flag), madvise_will_need;
542 vm_region.c:...
543 Other:
544 ppm.c: handle page descriptors. We find some use of cluster ptr that should be removed ?
545 page.c: to make local
546
547
548
549Question:
5501) How is the time handled in almos. Does we use a unique timer to keep the time seamleese across node ?
5512) Copy to/from userspace
5523) Cross cluster lists: thread lists
5534) Only the kentry is mapped when entring the kernel from user space... This is going to be a complexe point.
554Three new things need to be done:
5551- if from user space (or simply if we were in the vitual mode was actif), switch to physical mode of local cluster
5562- if from kernel save DATA_EXT register. Also, if we do cpy_uspace by switchin to the virtulle space, we need to
557do what we do in one.
558
559
560Remark:
561cross cluster access also keep in mind that other access are possible, like page based access (temporarely map a page).
562
563
564
565
566Task and thread subsystems:
567Per cluster vmm or per thread ?
568
569Per cluster: 1) interest: less physical space cosummed and less coherence to handle
570 2) Problems: "resharing" of the stack zone could complex (make it impossible to migrate the stack of thread and thus the thread) ?
571
572Per thread: 1) interest: virtualise the virtual mapping between thread become easy (for stack and TLS)
573 and temporary mappping are allot easier...
574 (and we could free the $27 register!)
575 2) Problems: to much use of physical space (cache) + require more coherence message
576
577
578
579Files need to accessible across cluster, since they could be share by tasks!
580
581
582
583TODO:
584Check that we have mapped enough of the kernel, for the kentry ?
585
586
587
588VMM replicated region (mapper) require that the insertion to the page cache be atomic!
589
590
591DEveloppement results:
592event_fifo.c
593remote operation.c
594arch_init.c:ongoing for devices
595cluster_init.c
596kentry.c:k1 is now used by the kernel! No, it's that we k1 only when we were in
597kernel mode... Otherwise when coming in usert mode we don't need it and save it?
Note: See TracBrowser for help on using the repository browser.