CPU shielding using /proc and /dev/cpuset

From RTwiki
(Difference between revisions)
Jump to: navigation, search
(Process shielding)
m (Protected "CPU shielding using /proc and /dev/cpuset": This page has been migrated to the Linux Foundation RT wiki. (‎[edit=sysop] (indefinite) ‎[move=sysop] (indefinite)))

Revision as of 09:51, 21 December 2017


Interrupt shielding

Kernel Space

In order to shield CPUs from individual interrupts being serviced on them you have to make sure that the following kernel configuration parameter is set:


User Space

Then make sure that the interrupts are not automatically balanced by the irqbalance daemon. This daemon is started from the irqbalance init script. To disable once do:

# /etc/init.d/irqbalance stop

To disable after next reboot do:

# chkconfig irqbalance off

After this you can change the CPU affinity mask of each interrupt by doing:

# echo hex_mask > /proc/irq/<irq_number>/smp_affinity

To check that the affinity mask has been set you can check the contents of the smp_affinity file.
The mask is updated the next time an interrupt is serviced. So you may not see the change immediately.

More information can be found in /usr/src/kernel/Documentation/IRQ-affinity.txt.

Process shielding

The kernel has an cpuset feature that allows you to create cpusets for real-time purposes. The kernel interface is proc filesystem based. It is described in /usr/src/kernel/Documentation/cgroups/cpusets.txt.

Each cpuset is represented by a directory in the cgroup file system containing (on top of the standard cgroup files) the following files describing that cpuset:

  - cpus: list of CPUs in that cpuset
  - mems: list of Memory Nodes in that cpuset
  - memory_migrate flag: if set, move pages to cpusets nodes
  - cpu_exclusive flag: is cpu placement exclusive?
  - mem_exclusive flag: is memory placement exclusive?
  - mem_hardwall flag:  is memory allocation hardwalled
  - memory_pressure: measure of how much paging pressure in cpuset
  - memory_spread_page flag: if set, spread page cache evenly on allowed nodes
  - memory_spread_slab flag: if set, spread slab cache evenly on allowed nodes
  - sched_load_balance flag: if set, load balance within CPUs on that cpuset
  - sched_relax_domain_level: the searching range when migrating tasks

In addition, the root cpuset only has the following file:

  - memory_pressure_enabled flag: compute memory_pressure?

Here is a quick example of how to use cpuset to reserve one cpu for your real-time process on a 4 cpu machine:

# mkdir /dev/cpuset/rt0

# echo 0 > /dev/cpuset/rt0/cpus

# echo 0 > /dev/cpuset/rt0/mems

# echo 1 > /dev/cpuset/rt0/cpu_exclusive

# echo $RT_PROC_PID > /dev/cpuset/rt0/tasks

# mkdir /dev/cpuset/system

# echo 1-3 > /dev/cpuset/system/cpus

# echo 0 > /dev/cpuset/system/mems

# echo 1 > /dev/cpuset/system/cpu_exclusive

# for pid in $(cat /dev/cpuset/tasks); do /bin/echo $pid > /dev/cpuset/system/tasks; done

Personal tools