RTAI schedulers and hard real time in kernel/user space.

RTAI provides symmetric hard real time services inter/intra user/kernel space.

Such a support comes through two schedulers, at the moment called rtai_lxrt and rtai_ksched, that can operate in both user and kernel land, differing only in relation to the objects they can schedule. So rtai_lxrt is simply a Linux coscheduler, i.e it supports hard real time for all Linux schedulable objects, i.e. processes/threads/kthreads. Rtai_ksched instead supports not only hard real time for all Linux schedulable objects, i.e. processes/threads/kthreads, but also for RTAI own kernel tasks, i.e. very light kernel space only schedulable objects proper to RTAI. So while in user space there is no choice but scheduling Linux processes and threads one could ask her/himsel what is worth the redundancy of having different scheduling objects in kernel space, as at a first glance they will afford much the same functionalities. Let's say immediately that the big pro of RTAI own light kernel tasks is their fast switching time with respect to Linux kernel threads, while their big con is that they operate outside any Linux environment so they require some special care if one needs to interoperate with Linux. On the contrary Linux kernel threads offer the advantage of being fully integrated in the Linux environment and ease any needed RTAI Linux interaction.

That said let's see first how any of the 2 available RTAI schedulers can become a coscheduler of Linux schedulable objects. To such an aim we assume to be within a process, you know how to build it for sure, a user thread, created either by pthread_create or clone, a kernel thread, created by kernel_thread or the newer support functions found in 2.6.xx. Then to use RTAI you need to extend the Linux task structure, by linking it to RTAI. That comes by using:

• rt_task_init_schmod (rt_task_init being a simpler but somewhat limited schortcut) in user space only;
• set_rtext in kernel space. After such calls your Linux schedulable object can use both Linux and RTAI APIs, without any constraint, but you'll not be in hard real time. At most you might be in Linux user space soft real time by setting SCHED_FIFO/SCHED_RR in rt_task_init_schmod or, in kernel space, by calling rtai_set_linux_task_priority to obtain the same result.

At this point if you want to operate in hard real time you have to call: rtai_make_hard_real_time. In user space you should anticipate such a call with a cautious stack extension plus a preallocation of all the needed memory followed by memory locking, see "man mlockall", to staticalise all of your needed stack and heap. The macro "rt_grow_and_lock_stack" might be your helper. That is needed to avoid breaking hard real time through dynamic virtual memory paging. In kernel space there is no such a need for the stack, beware you'll have available just the small stack the kernel makes available for its kthreads, but you need to do much the same with k/vmalloc, i.e. call them for all of what is needed before going hard real time anyhow, albeit without the need for any further memory locking call.

Now you are in hard real time and to keep so you cannot use any Linux syscall or kernel space service as that will:

• send you back into Linux hands
• execute what you asked to Linux, possibly being rescheduled by it
• return to RTAI hard real time when Linux has served you. RTAI can bear such things but it is suggested that you'll be fully aware of it by mating you hard real time task to a Linux buddy, to which you'll ask for the Linux service you need, thus being fully aware that you are giving up hard real time till your buddy has served you. If you'll find it a bit annoying consider using the support of Linux server RTAI makes available through: rt_linux_syscall_server_create; If that is still too annying then keep mixing Linux and RTAI even for hardened task but be aware that experience shows that even experienced users have found them bitten already by too much confidence in doing it "we know what we are doing" way. With too much a confidence Linux services are often placed, inadvertently of course, in the wrong place and you'll get slapped, better say you'll slap against end run stoppers, may be it is better to call them crashes of course. Notice that RTAI does such transitions as effectively as possible. So as hard real time capabilities improves in Linux natively (PREEMPT_RT) such pains might be lifted a bit and a higher level real time imteroperabilty might become feasible. Finally when a task is exiting or hard real time is not needed any more one should call rtai_make_soft_real_time following it with:
• rt_task_delete in user space, to detach RTAI from the Linux task structure;
• clr_rtext in kernel space to do the same for kernel threads. Latest RTAI versions should be capable of doing that for you, if you'll forget it.

The above generic scheme is mostly used for user space, a worth to note exception being found in netrpc.c. In fact in kernel space the standard usage legacy is to work in hard real time mode directly and there is where the difference between rtai_lxrt and rtai_ksched comes into the play. To have it clear one should recall that RTAI was first in promoting hard real time in user space and the symmetric kernel/user space usage was achieved by using RTAI own light kernel space tasks only. When the need was felt of supporting a symmetric scheduling in user/kernel space based on a coscheduler approach only emphasis was shifted to user space mainly with some kernel space only. So to avoid the burden of making Linux kthreads hard real time, as explained above, a support for the direct creation of hardened Linux kthreads was created, by setting up functions "rt_kthread_init" and "rt_kthread_init_cpuid", mirroring the functions "rt_task_init" and "rt_task_init_cpuid" used for RTAI own light kernel tasks. However RTAI users were most used to the latter function so, to allow them to not care of what was behind the curtains, there was the need to have "rt_task_init" and "rt_task_init_cpuid" create hardened Linux kthreads. In such a view rtai_lxrt must be used so thats kthread_inits and task_inits end in being the same things. Thus users wanting to use RTAI own light kernel threads only or both them and hardened Linux kthreads should use rtai_ksched, caring of using kthread_init/task_inits according to their needs.

So let's get rid of badly inheredited legacy concepts that drifted inito a long series of urban legends, not to say FUD, and state clearly what has been true for years now:

THERE IS JUST ONE WAY OF SCHEDULING IN RTAI AND IS AN ORIGINAL RTAI FIRST, IT HAS AN UBIQUITOUS APPEARENCE IN TWO FORMS BECAUSE OF WHAT EXPLAINED ABOVE.

How does hard real time comes for Linux schedulable objects?

Hardened real time schedulable objects are just normal Linux processes/threads/kthreds, by calling rt_make_hard_real_time a task suspends itself so that another schedulable Linux objects is switch in. As soon as the new task is switched in the RTAI task switching is called, without even exiting the just called Linux "schedule" function and the RTAI tasks is resumed in real time hardened mode. When it has nothing to do it will call RTAI "rt_schedule" and such a function will schedule a Linux object again. Notice that full interoperability of Linux/RTAI context switches is assured by a a common context switch function available in Linux sched.c. This is believed to be a truly original, albeit trivial idea, within Linux environment that appear to have been blatantly copied by others. Doing the trick the other way around, i.e. going back into Linux hands requires some care. In fact once hardened a schedulable object can fully preempt Linux anywhere without it being aware it happened. The solution is as trivial as going hardened thogh. So for the hardened task suspend itself and put itself on a circular buffer list, there is one for each CPU, waiting to be awaken by a call to Linxu "wake_up_process" when there is no other hard real time to be scheduled by the RTAI scheduler, so a full return to the Linux environment is ensured. Thanks to the commonality of context switches when the task is resumed by Linux it finds itself softened again. However it will not be at the end of Linux "schedule" but of RTAI "rt_schedule" instead. So RTAI must care of carring out the same epilogue found in schedule and all the black magic will do no harm. The actual function doing all of it are: steal_from_linux and give_back_to_linux.

Such functions are used, without an explicit user request, when a hardened real time schedulable objects issues a linux syscalls. For that RTAI has to intercept all of the Linux syscalls and if any is done in hard real time mode the hardened object is given back to Linux, then depending on the architecture the Linux syscall is either executed immediately (i.e. internally to the RTAI scheduler) and the object stolen from Linux just after returning from the syscall (e.g. i386) or the syscall is executed along the standard Linux syscall path, so that the recovery of hard real time will happen just at the next RTAI call (e.g. x86_64, at the moment).

---