CPU shielding using /proc and /dev/cpuset

From RTwiki
Revision as of 05:27, 8 September 2009 by Leemgs (Talk | contribs)

Jump to: navigation, search


Interrupt shielding

Kernel Space

In order to shield CPUs from individual interrupts being serviced on them you have to make sure that the following kernel configuration parameter is set:


User Space

Then make sure that the interrupts are not automatically balanced by the irqbalance daemon. This daemon is started from the irqbalance init script. To disable once do:

# /etc/init.d/irqbalance stop

To disable after next reboot do:

# chkconfig irqbalance off

After this you can change the CPU affinity mask of each interrupt by doing:

# echo hex_mask > /proc/irq/<irq_number>/smp_affinity

To check that the affinity mask has been set you can check the contents of the smp_affinity file.
The mask is updated the next time an interrupt is serviced. So you may not see the change immediately.

More information can be found in /usr/src/kernel/Documentation/IRQ-affinity.txt.

Process shielding

The kernel has an cpuset feature that allows you to create cpusets for real-time purposes. The kernel interface is proc filesystem based. It is described in /usr/src/kernel/Documentation/cpusets.txt.

Each cpuset is represented by a directory in the cgroup file system containing (on top of the standard cgroup files) the following files describing that cpuset:

  - cpus: list of CPUs in that cpuset
  - mems: list of Memory Nodes in that cpuset
  - memory_migrate flag: if set, move pages to cpusets nodes
  - cpu_exclusive flag: is cpu placement exclusive?
  - mem_exclusive flag: is memory placement exclusive?
  - mem_hardwall flag:  is memory allocation hardwalled
  - memory_pressure: measure of how much paging pressure in cpuset
  - memory_spread_page flag: if set, spread page cache evenly on allowed nodes
  - memory_spread_slab flag: if set, spread slab cache evenly on allowed nodes
  - sched_load_balance flag: if set, load balance within CPUs on that cpuset
  - sched_relax_domain_level: the searching range when migrating tasks

In addition, the root cpuset only has the following file:

  - memory_pressure_enabled flag: compute memory_pressure?

Here is a quick example of how to use cpuset to reserve one cpu for your real-time process on a 4 cpu machine:

# mkdir /dev/cpuset/rt0

# echo 0 > /dev/cpuset/rt0/cpus

# echo 0 > /dev/cpuset/rt0/mems

# echo 1 > /dev/cpuset/rt0/cpu_exclusive

# echo $RT_PROC_PID > /dev/cpuset/rt0/tasks

# mkdir /dev/cpuset/system

# echo 1-3 > /dev/cpuset/system/cpus

# echo 0 > /dev/cpuset/system/mems

# echo 1 > /dev/cpuset/system/cpu_exclusive

# for pid in $(cat /dev/cpuset/tasks); do /bin/echo $pid > /dev/cpuset/system/tasks; done

Personal tools