Frequently Asked Questions
This page contains a list of frequently asked questions about the real-time support in the Linux Kernel.
What is real-time?
Real-time applications have operational deadlines between some triggering event and the application's response to that event. To meet these operational deadlines, programmers use real-time operating systems (RTOS) on which make maximum on which the maximum response time can be calculated or measured realiable for the given application and environment.
A typical RTOS use priorities. The highest priority task wanting the CPU always get the CPU within a fixed amount of time after the event waking the task has taken place. On such an RTOS the latency of a task only depends on the tasks running at equal or higher priorities, all other tasks can be ignored. On a normal OS (such as normal Linux) the latencies depends on everything running on the system, which of course makes it much harder to be convinced that the deadlines will be meet every time on just a reasonably complicated system. This is because preemption can be switched off for a unknown amount of time. The high priority task wanting to run can thus be delayed for an unknown amount of time by low priority tasks running with preemption switched off.
Where can I read more about real-time?
Have a look to Publications page of this wiki.
What are real-time capabilities of the stock 2.6 linux kernel?
Traditionally, the Linux kernel will only allow one process to preempt another only under certain circumstances:
- When the CPU is running user-mode code
- When kernel code returns from a system call or an interrupt back to user space
- When kernel code code blocks on a mutex, or explicitly yields control to another process
If kernel code is executing when some event takes place that requires a high priority thread to start executing, the high priority thread can not preempt the running kernel code, until the kernel code explicitly yields control. In the worst case, the latency could potentially be hundreds milliseconds or more.
The Linux 2.6 configuration option CONFIG_PREEMPT_VOLUNTARY introduces checks to the most common causes of long latencies, so that the kernel can voluntarily yield control to a higher priority task waiting to execute. This can be helpful, but while it reduces the occurences of long latencies (hundreds of milliseconds to potentially seconds or more), it does not eliminate them. However unlike CONFIG_PREEMPT (discussed below), CONFIG_PREEMPT_VOLUTARY has a much lower impact on the overall throughput of the system. (As always, there is a classical tradeoff between throughput --- the overall efficiency of the system --- and latency. With the faster CPU's of modern-day systems, it often makes sense to trade off throughput for lower latencies, but server class systems that do not need minimum latency guarantees may very well chose to use either CONFIG_PREEMPT_VOLUNTARY, or to stick with the traditional non-preemptible kernel design.)
The 2.6 Linux kernel has an additional configuration option, CONFIG_PREEMPT, which causes all kernel code outside of spinlock-protected regions and interrupt handlers to be eligible for non-voluntary preemption by higher priority kernel threads. With this option, worst case latency drops to (around) single digit milliseconds, although some device drivers can have interrupt handlers that will introduce latency much worse than that. If a real-time Linux application requires latencies smaller than single-digit milliseconds, use of the CONFIG_PREEMPT_RT patch is highly recommended.
How does the CONFIG_PREEMPT_RT patch work?
The RT-Preempt patch converts Linux into a fully preemptible kernel. The magic is done with:
- Making in-kernel locking-primitives (using spinlocks) preemptible though reimplementation with rtmutexes:
- Critical sections protected by i.e. spinlock_t and rwlock_t are now preemptible. The creation of non-preemptible sections (in kernel) is still possible with raw_spinlock_t (same APIs like spinlock_t)
- Implementing priority inheritance for in-kernel spinlocks and semaphores. For more information on priority inversion and priority inheritance please consult Introduction to Priority Inversion
- Converting interrupt handlers into preemptible kernel threads: The RT-Preempt patch treats soft interrupt handlers in kernel thread context, which is represented by a task_struct like a common user space process. However it is also possible to register an IRQ in kernel context.
- Converting the old Linux timer API into separate infrastructures for high resolution kernel timers plus one for timeouts, leading to user space POSIX timers with high resolution.
Who are the Maintainers of CONFIG_PREEMPT_RT patch?
How can I start using the CONFIG_PREEMPT_RT patch?
Please see the RT PREEMPT HOWTO for a detailed description. The short version is to get the latest -rt patch from http://www.kernel.org/pub/linux/kernel/projects/rt/, then get the related vanilla kernel from http://kernel.org, apply the patch to the kernel, configure, and compile the kernel.
What mailing lists are available to discuss the CONFIG_PREEMPT_RT patch?
Please see the mailing lists page in the rt wiki.
How to enable/disable realtime support?
Change realtime support run time or statically?
Required parameters for configuring realtime kernel?
Optional parameters for configuring realtime kernel, but they effect realtime performance?
Which Architecture are supporting realtime and who is maintainer?
Note 1: With ARM, -rt support depends on -rt support for each different system a ARM core is used in. E.g. while ARM system from company A may be work quite well with -rt because it is well maintained, an other ARM system from company B may not work because no support for -rt is developed yet. However, basic -rt support is available for ARM, so it depends on specific ARM system if -rt works or not on that specific system.
See platforms tested and in use with CONFIG_PREEMT_RT section in this wiki as well.
What is the procedure to support realtime in new architecture?
What effects by realtime support and how much and samples to test it and who maintains these samples?
|Area||Effected by Realtime(%)||Samples||Contact(s)|
|cyclic handlers||Cyclictest||Thomas Gleixner|
Real-time Application Programming
List of realtime APIs?
Is any API available which allows me to switch from Task A to Task B and/or Thread A to Thread B?
How to write realtime applications?
Which programming languages are suitable for writing realtime applications?
It depends on your latency requirements, but usually these are advisable:
- C is preferred;
- C++ is possible if some constraints are taken into account (see HOWTO: Build an RT-application)
What are important things to keep in mind while writing realtime applications?
Taking care of the following during the initial startup phase:
- Call directly from the main() entry the mlockall() call.
- Create all threads at startup time of the application, and touch each page of the entire stack of each thread. Never start threads dynamically during RT show time, this will ruin RT behavior.
- Never use system calls that are known to generate page faults, such as fopen(). (Opening of files does the mmap() system call, which generates a page-fault).
- Do not use 'compile time static arrays' without initializing them directly after startup, before RT show time.
more information: HOWTO: Build an RT-application
Do I need to recompile my applications to get realtime performance?
Recompile is not necessary. Userland does not notice any difference if it is running on RT or not, except for a much better RT-responsiveness, only changing the kernel is enough.
But, usually, locking bugs in userland applications become much more visible due the more dynamic and fine grained scheduling of the kernel. These are not new bugs, but bugs already existing in the applications, which only become visible on RT (Notice that these bugs are also likely to occur on SMP systems).
- A bad designed application on non-RT will never behave realtime on RT. See also HOWTO: Build an RT-application.
- Never protect shared/global data in userland processes with semaphores (yes, many people do that...), but use PTHREAD_PRIO_INHERIT mutexes instead. (PTHREAD_PRIO_INHERIT mutexes are only supported in Glibc 2.5 and higher. uClibc based file systems do not support this)
Authors of this FAQ include: