Cpuset Management Utility/tutorial

From RTwiki
(Difference between revisions)
Jump to: navigation, search
 
(30 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Cpuset (cset) Tutorial
+
[[Cpuset Management Utility | Back up to cpuset page.]]
######################
+
 
 +
<font size="+2">Cpuset (cset) Tutorial</font> <br>
  
 
Alex Tsariounov <alext@novell.com> <br>
 
Alex Tsariounov <alext@novell.com> <br>
Line 12: Line 13:
 
more details.  Additionally, the '''cset''' command has online help.
 
more details.  Additionally, the '''cset''' command has online help.
  
 
+
=Introduction=
Introduction
+
============
+
 
In the Linux kernel, the cpuset facility provides a mechanism for creating
 
In the Linux kernel, the cpuset facility provides a mechanism for creating
 
logical entities called "cpusets" that encompass definitions of CPUs and NUMA
 
logical entities called "cpusets" that encompass definitions of CPUs and NUMA
Line 45: Line 44:
 
extra <tt>set</tt> and <tt>proc</tt> subcommands.
 
extra <tt>set</tt> and <tt>proc</tt> subcommands.
  
 +
==Obtaining Online Help==
 +
For a full list of '''cset''' subcommands:
 +
    # cset help
 +
For in-depth help on individual subcommands:
 +
    # cset help <subcommand>
 +
For options of individual subcommands:
 +
    # cset <subcommand> (-h | --help)
  
Obtaining Online Help
+
=The Basic Shielding Model=
---------------------
+
For a full list of '''cset''' subcommands::
+
    +# cset help+
+
 
+
For in-depth help on individual subcommands::
+
    +# cset help <subcommand>+
+
 
+
For options of individual subcommands::
+
    +# cset <subcommand> (-h | --help)+
+
 
+
 
+
The Basic Shielding Model
+
=========================
+
 
Although any set up of cpusets can really be described as "shielding," there
 
Although any set up of cpusets can really be described as "shielding," there
 
is one prevalent shielding model in use that is so common that '''cset''' has a
 
is one prevalent shielding model in use that is so common that '''cset''' has a
Line 94: Line 87:
 
described in a later section.
 
described in a later section.
  
 
+
==A Simple Shielding Example==
A Simple Shielding Example
+
--------------------------
+
 
Assume that we have a 4-way machine that is not NUMA.  This means there are 4
 
Assume that we have a 4-way machine that is not NUMA.  This means there are 4
 
CPUs at our disposal and there is only one Memory Node available.  On such
 
CPUs at our disposal and there is only one Memory Node available.  On such
Line 109: Line 100:
 
tasks that are running on the system.
 
tasks that are running on the system.
  
NOTE: I use the word "task" to represent either a process or a thread that is
+
{{NOTE|I use the word "task" to represent either a process or a thread that is
running on the system.
+
running on the system.}}
  
 
+
===Setup and  Teardown of the Shield===
Setup and  Teardown of the Shield
+
22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)[[User:Alext|Alext]]
+
 
To set up a shield of 3 CPUs with 1 CPU left for low priority system
 
To set up a shield of 3 CPUs with 1 CPU left for low priority system
 
processing, issue the following command.
 
processing, issue the following command.
  
-------------------------------------------------------------------
+
[zuul:cpuset-trunk]# cset shield -c 1-3
[zuul:cpuset-trunk]# cset shield -c 1-3
+
cset: --> activating shielding:
cset: --> activating shielding:
+
cset: moving 176 tasks from root into system cpuset...
cset: moving 176 tasks from root into system cpuset...
+
[==================================================]%
[==================================================]%
+
cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
+
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+
-------------------------------------------------------------------
+
  
 
This command does a number of things.  First, a ''user'' cpuset is created with
 
This command does a number of things.  First, a ''user'' cpuset is created with
what's called a CPUSPEC (CPU specification) from the <tt>-c/\--cpu</tt> option.  This
+
what's called a CPUSPEC (CPU specification) from the <tt>-c/--cpu</tt> option.  This
 
CPUSPEC specifies to use CPUs 1 through 3 inclusively.  Next, the command
 
CPUSPEC specifies to use CPUs 1 through 3 inclusively.  Next, the command
 
creates a ''system'' cpuset with a CPUSPEC that is the inverse of the <tt>-c</tt>
 
creates a ''system'' cpuset with a CPUSPEC that is the inverse of the <tt>-c</tt>
Line 140: Line 127:
 
''root'' cpuset to the ''system'' cpuset.  This is because you may want these
 
''root'' cpuset to the ''system'' cpuset.  This is because you may want these
 
kernel threads to use all available CPUs.  If you do not, the you can use the
 
kernel threads to use all available CPUs.  If you do not, the you can use the
<tt>-k/\--kthread</tt> option as described below.
+
<tt>-k/--kthread</tt> option as described below.
  
 
The shield setup command above outputs the information of which cpusets were
 
The shield setup command above outputs the information of which cpusets were
Line 146: Line 133:
 
current status of the shield again, issue this command:
 
current status of the shield again, issue this command:
  
-------------------------------------------------------------------
+
[zuul:cpuset-trunk]# cset shield
[zuul:cpuset-trunk]# cset shield
+
cset: --> shielding system active with
cset: --> shielding system active with
+
cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
+
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+
-------------------------------------------------------------------
+
  
 
Which shows us that the shield is set up and that 176 tasks are running in the
 
Which shows us that the shield is set up and that 176 tasks are running in the
Line 175: Line 160:
 
following command.
 
following command.
  
--------------------------------------------------------------------
+
[zuul:cpuset-trunk]# cset shield -k on
[zuul:cpuset-trunk]# cset shield -k on
+
cset: --> activating kthread shielding
cset: --> activating kthread shielding
+
cset: kthread shield activated, moving 70 tasks into system cpuset...
cset: kthread shield activated, moving 70 tasks into system cpuset...
+
[==================================================]%
[==================================================]%
+
cset: done
cset: done
+
--------------------------------------------------------------------
+
  
 
You can see that this moved an additional 70 tasks to the unshielded ''system''
 
You can see that this moved an additional 70 tasks to the unshielded ''system''
cpuset.  Note that the <tt>-k/\--kthread on</tt> parameter can be given at the shield
+
cpuset.  Note that the <tt>-k/--kthread on</tt> parameter can be given at the shield
 
creation time as well and you do not need to perform these two steps
 
creation time as well and you do not need to perform these two steps
 
separately if you know that you will want kernel thread shielding as well.
 
separately if you know that you will want kernel thread shielding as well.
 
Executing '''cset shield''' again shows us the current state of the shield.
 
Executing '''cset shield''' again shows us the current state of the shield.
  
--------------------------------------------------------------------
+
[zuul:cpuset-trunk]# cset shield
[zuul:cpuset-trunk]# cset shield
+
cset: --> shielding system active with
cset: --> shielding system active with
+
cset: "system" cpuset of CPUSPEC(0) with 246 tasks running
cset: "system" cpuset of CPUSPEC(0) with 246 tasks running
+
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+
--------------------------------------------------------------------
+
  
 
You can get a detailed listing of what is running in the shield by specifying
 
You can get a detailed listing of what is running in the shield by specifying
either <tt>-s/\--shield</tt> or <tt>-u/\--unshield</tt> to the <tt>shield</tt> subcommand and using
+
either <tt>-s/--shield</tt> or <tt>-u/--unshield</tt> to the <tt>shield</tt> subcommand and using
 
the verbose flag.  You will get output similar to the following.
 
the verbose flag.  You will get output similar to the following.
  
--------------------------------------------------------------------
+
[zuul:cpuset-trunk]# cset shield --unshield -v
[zuul:cpuset-trunk]# cset shield --unshield -v
+
cset: "system" cpuset of CPUSPEC(0) with 251 tasks running
cset: "system" cpuset of CPUSPEC(0) with 251 tasks running
+
 
   USER      PID  PPID SPPr TASK NAME
 
   USER      PID  PPID SPPr TASK NAME
 
   -------- ----- ----- ---- ---------
 
   -------- ----- ----- ---- ---------
Line 208: Line 188:
 
   root        2    0 Soth [kthreadd]
 
   root        2    0 Soth [kthreadd]
 
   root        84    2 Sf50 [IRQ-9]
 
   root        84    2 Sf50 [IRQ-9]
...
+
...
 
   alext    31796 31789 Soth less
 
   alext    31796 31789 Soth less
 
   root    32653 25222 Roth python ./cset shield --unshield -v
 
   root    32653 25222 Roth python ./cset shield --unshield -v
--------------------------------------------------------------------
 
  
 
Note that I abbreviated the listing; we do have 251 tasks running in the
 
Note that I abbreviated the listing; we do have 251 tasks running in the
Line 220: Line 199:
 
but marked at real time FIFO policy with a priority of 50.  The last task in
 
but marked at real time FIFO policy with a priority of 50.  The last task in
 
the listing is the '''cset''' command itself and is marked as running.  Also note
 
the listing is the '''cset''' command itself and is marked as running.  Also note
that adding a second <tt>-v/\--verbose</tt> option will not restrict the output to
+
that adding a second <tt>-v/--verbose</tt> option will not restrict the output to
 
fit into an 80 character screen.
 
fit into an 80 character screen.
  
 
Tear down of the shield, stopping the shield in other words, is done with the
 
Tear down of the shield, stopping the shield in other words, is done with the
<tt>-r/\--reset</tt> option to the <tt>shield</tt> subcommand.  When this command is issued,
+
<tt>-r/--reset</tt> option to the <tt>shield</tt> subcommand.  When this command is issued,
 
both the ''system'' and ''user'' cpusets are deleted and any tasks that are
 
both the ''system'' and ''user'' cpusets are deleted and any tasks that are
 
running in both of those cpusets are moved to the ''root'' cpuset.  Once so
 
running in both of those cpusets are moved to the ''root'' cpuset.  Once so
Line 230: Line 209:
 
example:
 
example:
  
--------------------------------------------------------------------
+
[zuul:cpuset-trunk]# cset shield --reset
[zuul:cpuset-trunk]# cset shield --reset
+
cset: --> deactivating/reseting shielding
cset: --> deactivating/reseting shielding
+
cset: moving 0 tasks from "/user" user set to root set...
cset: moving 0 tasks from "/user" user set to root set...
+
cset: moving 250 tasks from "/system" system set to root set...
cset: moving 250 tasks from "/system" system set to root set...
+
[==================================================]%
[==================================================]%
+
cset: deleting "/user" and "/system" sets
cset: deleting "/user" and "/system" sets
+
cset: done
cset: done
+
--------------------------------------------------------------------
+
  
 
+
===Moving Interesting Tasks Into and Out of the Shield===
Moving Interesting Tasks Into and Out of the Shield
+
22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)~
+
 
Now that we have a shield running, the objective is to run our "important"
 
Now that we have a shield running, the objective is to run our "important"
 
processes in that shield.  These processes can be anything, but usually they
 
processes in that shield.  These processes can be anything, but usually they
Line 248: Line 223:
 
tasks in the shield:
 
tasks in the shield:
  
. Exec a process into the shield
+
# Exec a process into the shield
. Move an already running task into the shield
+
# Move an already running task into the shield
 
+
  
Execing a Process into the Shield
+
====Execing a Process into the Shield====
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
Running a new process in the shield can be done with the <tt>-e/--exec</tt> option
Running a new process in the shield can be done with the <tt>-e/\--exec</tt> option
+
 
to the <tt>shield</tt> subcommand.  This is the simplest way to get a task to run in
 
to the <tt>shield</tt> subcommand.  This is the simplest way to get a task to run in
 
the shield.  For this example, let's exec a new bash shell into the shield
 
the shield.  For this example, let's exec a new bash shell into the shield
 
with the following commands.
 
with the following commands.
  
--------------------------------------------------------------------
+
<pre>
[zuul:cpuset-trunk]# cset shield -s
+
[zuul:cpuset-trunk]# cset shield -s
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: done
+
cset: done
  
[zuul:cpuset-trunk]# cset shield -e bash
+
[zuul:cpuset-trunk]# cset shield -e bash
cset: --> last message, executed args into cpuset "/user", new pid is: 13300
+
cset: --> last message, executed args into cpuset "/user", new pid is: 13300
  
[zuul:cpuset-trunk]# cset shield -s -v
+
[zuul:cpuset-trunk]# cset shield -s -v
cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
+
cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
 
   USER      PID  PPID SPPr TASK NAME
 
   USER      PID  PPID SPPr TASK NAME
 
   -------- ----- ----- ---- ---------
 
   -------- ----- ----- ---- ---------
Line 274: Line 247:
 
   root    13329 13300 Roth python ./cset shield -s -v
 
   root    13329 13300 Roth python ./cset shield -s -v
  
[zuul:cpuset-trunk]# exit
+
[zuul:cpuset-trunk]# exit
  
[zuul:cpuset-trunk]# cset shield -s
+
[zuul:cpuset-trunk]# cset shield -s
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: done
+
cset: done
--------------------------------------------------------------------
+
</pre>
  
 
The first command above lists the status of the shield.  We see that the
 
The first command above lists the status of the shield.  We see that the
Line 288: Line 261:
 
option.  The last message of cset lists the PID of the new process.
 
option.  The last message of cset lists the PID of the new process.
  
NOTE: '''cset''' follows the tradition of separating the tool options from the
+
{{NOTE|'''cset''' follows the tradition of separating the tool options from the command to be execed options with a double dash (<tt>--</tt>).  This is not shown in this simple example, but if the command you want to exec also takes options,
command to be execed options with a double dash (<tt>\--</tt>).  This is not shown in
+
separate them with the double dash like so: <tt># cset shield -e mycommand -- -v</tt> The <tt>-v</tt> will be passed to <tt>mycommand</tt>, and not to '''cset'''.}}
this simple example, but if the command you want to exec also takes options,
+
separate them with the double dash like so: +# cset shield -e mycommand \-- -v+
+
The <tt>-v</tt> will be passed to <tt>mycommand</tt>, and not to '''cset'''.
+
  
 
The next command lists the status of the shield again.  You will note that
 
The next command lists the status of the shield again.  You will note that
Line 300: Line 270:
 
child, which is the status command, also ran in the shield.
 
child, which is the status command, also ran in the shield.
  
TIP: Execing a shell into the shield is a useful way to experiment with
+
{{HINT:Execing a shell into the shield is a useful way to experiment with
 
running tasks in the shield since all children of the shell will also run in
 
running tasks in the shield since all children of the shell will also run in
the shield.
+
the shield.}}
  
 
The last command exits the shell after which we request a shield status again
 
The last command exits the shell after which we request a shield status again
Line 310: Line 280:
 
status command are running as the root user.  This is because '''cset''' needs to
 
status command are running as the root user.  This is because '''cset''' needs to
 
run as root and so all it's children will also run as root.  If you need to
 
run as root and so all it's children will also run as root.  If you need to
run a process under a different user and or group, you may use the <tt>\--user</tt>
+
run a process under a different user and or group, you may use the <tt>--user</tt>
and <tt>\--group</tt> options for <tt>exec</tt> as follows.
+
and <tt>--group</tt> options for <tt>exec</tt> as follows.
  
--------------------------------------------------------------------
+
<pre>
[zuul:cpuset-trunk]# cset shield --user=alext --group=users -e bash
+
[zuul:cpuset-trunk]# cset shield --user=alext --group=users -e bash
cset: --> last message, executed args into cpuset "/user", new pid is: 14212
+
cset: --> last message, executed args into cpuset "/user", new pid is: 14212
  
alext@zuul> cset shield -s -v
+
alext@zuul> cset shield -s -v
cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
+
cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
 
   USER      PID  PPID SPPr TASK NAME
 
   USER      PID  PPID SPPr TASK NAME
 
   -------- ----- ----- ---- ---------
 
   -------- ----- ----- ---- ---------
 
   alext    14212  8583 Soth bash
 
   alext    14212  8583 Soth bash
 
   alext    14241 14212 Roth python ./cset shield -s -v
 
   alext    14241 14212 Roth python ./cset shield -s -v
--------------------------------------------------------------------
+
</pre>
 
+
  
Moving a Running Task into and out of the Shield
+
====Moving a Running Task into and out of the Shield====
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
While execing a process into the shield is undoubtedly useful, most of the
While execing a process into the shield is undoubtably useful, most of the
+
 
time, you'll want to move already running tasks into and out of the shield.
 
time, you'll want to move already running tasks into and out of the shield.
 
The '''cset''' shield subcommand includes two options for doing this:
 
The '''cset''' shield subcommand includes two options for doing this:
<tt>-s/\--shield</tt> and <tt>-u/\--unshield</tt>.  These options require what's called a
+
<tt>-s/--shield</tt> and <tt>-u/--unshield</tt>.  These options require what's called a
PIDSPEC (process specification) to also be specified with the <tt>-p/\--pid</tt>
+
PIDSPEC (process specification) to also be specified with the <tt>-p/--pid</tt>
 
option.  The PIDSPEC defines which tasks get operated on.  The PIDSPEC can be
 
option.  The PIDSPEC defines which tasks get operated on.  The PIDSPEC can be
 
a single process ID, a list of process IDs separated by commas, and a list of
 
a single process ID, a list of process IDs separated by commas, and a list of
Line 338: Line 306:
 
commas.  For example:
 
commas.  For example:
  
+\--shield --pid 1234+::
+
;--shield --pid 1234
This PIDSPEC argument specifies that PID 1234 be shielded.
+
:This PIDSPEC argument specifies that PID 1234 be shielded.
  
+\--shield --pid 1234,42,1934,15000,15001,15002+::
+
;--shield --pid 1234,42,1934,15000,15001,15002
This PIDSPEC argument specifies that this list of PIDs only be moved into the
+
:This PIDSPEC argument specifies that this list of PIDs only be moved into the shield.
shield.
+
  
+\--unshield -p 5000,5100,6010-7000,9232+::
+
;--unshield -p 5000,5100,6010-7000,9232
This PIDSPEC argument specifies that PIDs 5000,5100 and 9232 be unshielded
+
:This PIDSPEC argument specifies that PIDs 5000,5100 and 9232 be unshielded (moved out of the shield) along with any existing PID that is in the range 6010 through 7000 inclusive.
(moved out of the shield) along with any existing PID that is in the range
+
6010 through 7000 inclusive.
+
  
NOTE: A range in a PIDSPEC does not have to have tasks running for every
+
{{NOTE|A range in a PIDSPEC does not have to have tasks running for every number in that range.  In fact, it is not even an error if there are no tasks running in that range; none will be moved in that case.  The range simply specifies to act on any tasks that have a PID or TID that is within that range.}}
number in that range.  In fact, it is not even an error if there are no tasks
+
running in that range; none will be moved in that case.  The range simply
+
specifies to act on any tasks that have a PID or TID that is within that
+
range.
+
  
 
Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of
 
Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of
 
tasks into and out of the shield.  Additionally, there is one more option that
 
tasks into and out of the shield.  Additionally, there is one more option that
can help with multi-threaded processes, and that is the <tt>\--threads</tt> flag.  If
+
can help with multi-threaded processes, and that is the <tt>--threads</tt> flag.  If
 
this flag is present in a <tt>shield</tt> or <tt>unshield</tt> command with a PIDSPEC and if
 
this flag is present in a <tt>shield</tt> or <tt>unshield</tt> command with a PIDSPEC and if
 
any of the task IDs in the PIDSPEC belong to a thread in a process container,
 
any of the task IDs in the PIDSPEC belong to a thread in a process container,
Line 368: Line 329:
 
range PIDSPEC and back out with the bash variable for the current PID.
 
range PIDSPEC and back out with the bash variable for the current PID.
  
-----------------------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# echo $$
 
[zuul:cpuset-trunk]# echo $$
 
22018
 
22018
Line 391: Line 352:
 
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
 
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
 
cset: done
 
cset: done
-----------------------------------------------------------------------
+
</pre>
  
NOTE: Ordinarily, the <tt>shield</tt> option will shield a PIDSPEC only if it is
+
{{NOTE|Ordinarily, the <tt>shield</tt> option will shield a PIDSPEC only if it is currently running in the ''system'' set--the unshielded set.  The <tt>unshield</tt> option will unshield a PIDSPEC only if it is currently running in the ''user'' set--the shielded set.  If you want to <tt>shield/unshield</tt> a process that happens to be running in the ''root'' set (not common), then use the <tt>--force</tt> option for these commands.}}
currently running in the ''system'' set--the unshielded set.  The <tt>unshield</tt>
+
option will unshield a PIDSPEC only if it is currently running in the ''user''
+
set--the shielded set.  If you want to <tt>shield/unshield</tt> a process that
+
happens to be running in the ''root'' set (not common), then use the <tt>\--force</tt>
+
option for these commands.
+
  
Full Featured Cpuset Manipulation Commands
+
=Full Featured Cpuset Manipulation Commands=
==========================================
+
 
While basic shielding as described above is useful and a common use model for
 
While basic shielding as described above is useful and a common use model for
 
'''cset''', there comes a time when more functionality will be desired to
 
'''cset''', there comes a time when more functionality will be desired to
Line 408: Line 363:
 
manipulate processes within those cpusets.
 
manipulate processes within those cpusets.
  
 
+
==The Set Subcommand==
The Set Subcommand
+
------------------
+
 
In order to do anything with cpusets, you must be able to create, adjust,
 
In order to do anything with cpusets, you must be able to create, adjust,
 
rename, move and destroy them.  The <tt>set</tt> subcommand allows the management of
 
rename, move and destroy them.  The <tt>set</tt> subcommand allows the management of
 
cpusets in such a manner.
 
cpusets in such a manner.
  
 
+
===Creating and Destroying Cpusets with Set===
Creating and Destroying Cpusets with Set
+
22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)
+
 
The basic syntax of <tt>set</tt> for cpuset creation is:
 
The basic syntax of <tt>set</tt> for cpuset creation is:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -c 1-3 -s my_cpuset1
 
[zuul:cpuset-trunk]# cset set -c 1-3 -s my_cpuset1
 
cset: --> created cpuset "my_cpuset1"
 
cset: --> created cpuset "my_cpuset1"
----------------------------------------------------------
+
</pre>
  
 
This creates a cpuset named "my_cpuset1" with a CPUSPEC of CPU1, CPU2 and
 
This creates a cpuset named "my_cpuset1" with a CPUSPEC of CPU1, CPU2 and
 
CPU3.  The CPUSPEC is the same concept as described in the "Setup and
 
CPU3.  The CPUSPEC is the same concept as described in the "Setup and
 
Teardown of the Shield" section above.  The <tt>set</tt> subcommand also takes a
 
Teardown of the Shield" section above.  The <tt>set</tt> subcommand also takes a
<tt>-m/\--mem</tt> option that lets you specify the memory nodes the set will use as
+
<tt>-m/--mem</tt> option that lets you specify the memory nodes the set will use as
 
well as flags to make the CPUs and MEMs exclusive to the cpuset.  If you are
 
well as flags to make the CPUs and MEMs exclusive to the cpuset.  If you are
 
on a non-NUMA machine, just leave the <tt>-m</tt> option out and the default memory
 
on a non-NUMA machine, just leave the <tt>-m</tt> option out and the default memory
Line 437: Line 388:
 
to only use CPUs 1 and 3 (and omit CPU2), then issue the following command.
 
to only use CPUs 1 and 3 (and omit CPU2), then issue the following command.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -c 1,3 -s my_cpuset1
 
[zuul:cpuset-trunk]# cset set -c 1,3 -s my_cpuset1
 
cset: --> modified cpuset "my_cpuset
 
cset: --> modified cpuset "my_cpuset
----------------------------------------------------------
+
</pre>
  
 
'''cset''' will then adjust the CPUs that are assigned to the "my_cpuset1" set to
 
'''cset''' will then adjust the CPUs that are assigned to the "my_cpuset1" set to
only use CPU1 and CPU3.
+
only use CPU1 and CPU3.
  
To rename a cpuset, use the <tt>-n/\--newname</tt> option.  For example:
+
To rename a cpuset, use the <tt>-n/--newname</tt> option.  For example:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -s my_cpuset1 -n super_set
 
[zuul:cpuset-trunk]# cset set -s my_cpuset1 -n super_set
 
cset: --> renaming "/cpusets/my_cpuset1" to "super_set"
 
cset: --> renaming "/cpusets/my_cpuset1" to "super_set"
----------------------------------------------------------
+
</pre>
  
 
Renames the cpuset called "my_cpuset1" to "super_set".
 
Renames the cpuset called "my_cpuset1" to "super_set".
  
To destroy a cpuset, use the <tt>-d/\--destroy</tt> option as follows.
+
To destroy a cpuset, use the <tt>-d/--destroy</tt> option as follows.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -d super_set
 
[zuul:cpuset-trunk]# cset set -d super_set
 
cset: --> processing cpuset "super_set", moving 0 tasks to parent "/"...
 
cset: --> processing cpuset "super_set", moving 0 tasks to parent "/"...
 
cset: --> deleting cpuset "/super_set"
 
cset: --> deleting cpuset "/super_set"
 
cset: done
 
cset: done
----------------------------------------------------------
+
</pre>
  
 
This command destroys the newly created cpuset called "super_set".  When a
 
This command destroys the newly created cpuset called "super_set".  When a
 
cpuset is destroyed, all the tasks running in it are moved to the parent
 
cpuset is destroyed, all the tasks running in it are moved to the parent
 
cpuset.  The ''root'' cpuset, which always exists and always contains all CPUs,
 
cpuset.  The ''root'' cpuset, which always exists and always contains all CPUs,
can not be destroyed.  You may also give the <tt>\--destroy</tt> option a list of
+
can not be destroyed.  You may also give the <tt>--destroy</tt> option a list of
 
cpusets to destroy.
 
cpusets to destroy.
  
NOTE: The '''cset''' subcommand creates the cpusets based on a mounted cpuset
+
{{NOTE|The '''cset''' subcommand creates the cpusets based on a mounted cpuset filesystem.  You do not need to know where that filesystem is mounted, although it is easy to figure out (by default it''s on ''/cpusets').  When you give the <tt>set</tt> subcommand a name for a new cpuset, it is created wherever the cpuset filesystem is mounted at.}}
filesystem.  You do not need to know where that filesystem is mounted,
+
although it is easy to figure out (by default it''s on ''/cpusets').  When you
+
give the <tt>set</tt> subcommand a name for a new cpuset, it is created wherever the
+
cpuset filesystem is mounted at.
+
  
 
If you want to create a cpuset hierarchy, then you must give a path to the
 
If you want to create a cpuset hierarchy, then you must give a path to the
Line 479: Line 426:
 
for which the path is '/'.  For example.
 
for which the path is '/'.  For example.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -c 1,3 -s top_set
 
[zuul:cpuset-trunk]# cset set -c 1,3 -s top_set
 
cset: --> created cpuset "top_set"
 
cset: --> created cpuset "top_set"
Line 485: Line 432:
 
[zuul:cpuset-trunk]# cset set -c 3 -s /top_set/sub_set
 
[zuul:cpuset-trunk]# cset set -c 3 -s /top_set/sub_set
 
cset: --> created cpuset "/top_set/sub_set"
 
cset: --> created cpuset "/top_set/sub_set"
----------------------------------------------------------
+
</pre>
  
 
These commands created two cpusets: 'top_set'' and ''sub_set''.  The ''top_set'
 
These commands created two cpusets: 'top_set'' and ''sub_set''.  The ''top_set'
Line 493: Line 440:
 
then '''cset''' will complain and ask you to use the path.  For example:
 
then '''cset''' will complain and ask you to use the path.  For example:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -c 1,3 -s sub_set
 
[zuul:cpuset-trunk]# cset set -c 1,3 -s sub_set
 
cset: --> modified cpuset "sub_set
 
cset: --> modified cpuset "sub_set
----------------------------------------------------------
+
</pre>
  
 
This command adds CPU1 to the 'sub_set'' cpuset for it''s use.  Note that using
 
This command adds CPU1 to the 'sub_set'' cpuset for it''s use.  Note that using
Line 502: Line 449:
  
 
If you attempt to destroy a cpuset which has sub-cpusets, '''cset''' will complain
 
If you attempt to destroy a cpuset which has sub-cpusets, '''cset''' will complain
and not do it unless you use the <tt>-r/\--recurse</tt> and the <tt>\--force</tt> options.
+
and not do it unless you use the <tt>-r/--recurse</tt> and the <tt>--force</tt> options.
If you do use <tt>\--force</tt>, then all the tasks running in all subsets of the
+
If you do use <tt>--force</tt>, then all the tasks running in all subsets of the
 
deletion target cpuset will be moved to the target's parent cpuset and all
 
deletion target cpuset will be moved to the target's parent cpuset and all
 
cpusets.
 
cpusets.
Line 510: Line 457:
 
currently not implemented and is slated for a later release of '''cset'''.
 
currently not implemented and is slated for a later release of '''cset'''.
  
 
+
===Listing Cpusets with Set===
Listing Cpusets with Set
+
To list cpusets, use the <tt>set</tt> subcommand with the '-l/--list' option.  For
22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)[[User:Alext|Alext]] 22:51, 11 February 2009 (UTC)
+
To list cpusets, use the <tt>set</tt> subcommand with the '-l/\--list' option.  For
+
 
example:
 
example:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -l
 
[zuul:cpuset-trunk]# cset set -l
 
cset:
 
cset:
Line 523: Line 468:
 
         root        0-3 y      0 y  320    1 /
 
         root        0-3 y      0 y  320    1 /
 
           one          3 n      0 n    0    1 /one
 
           one          3 n      0 n    0    1 /one
----------------------------------------------------------
+
</pre>
  
 
This shows that there is currently one cpuset present called ''one''. (Of course
 
This shows that there is currently one cpuset present called ''one''. (Of course
Line 533: Line 478:
 
specify a cpuset to list with the <tt>set</tt> subcommand as follows.
 
specify a cpuset to list with the <tt>set</tt> subcommand as follows.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -l -s one
 
[zuul:cpuset-trunk]# cset set -l -s one
 
cset:
 
cset:
Line 540: Line 485:
 
           one          3 n      0 n    0    1 /one
 
           one          3 n      0 n    0    1 /one
 
           two          3 n      0 n    0    1 /one/two
 
           two          3 n      0 n    0    1 /one/two
----------------------------------------------------------
+
</pre>
  
 
This output shows that there is a cpuset called ''two'' in cpuset ''one'' and it
 
This output shows that there is a cpuset called ''two'' in cpuset ''one'' and it
 
also has subset.  You can also ask for a recursive listing as follows.
 
also has subset.  You can also ask for a recursive listing as follows.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -l -r
 
[zuul:cpuset-trunk]# cset set -l -r
 
cset:
 
cset:
Line 554: Line 499:
 
           two          3 n      0 n    0    1 /one/two
 
           two          3 n      0 n    0    1 /one/two
 
         three          3 n      0 n    0    0 /one/two/three
 
         three          3 n      0 n    0    0 /one/two/three
----------------------------------------------------------
+
</pre>
  
 
This command lists all cpusets existing on the system since it asks for a
 
This command lists all cpusets existing on the system since it asks for a
Line 561: Line 506:
 
explicitely--just remember that the ''root'' cpuset cannot be deleted or modified.
 
explicitely--just remember that the ''root'' cpuset cannot be deleted or modified.
  
 
+
==The Proc Subcommand==
The Proc Subcommand
+
-------------------
+
 
Now that we know how to create, rename and destroy cpusets with the <tt>set</tt>
 
Now that we know how to create, rename and destroy cpusets with the <tt>set</tt>
 
subcommand, the next step is to manage threads and processes in those
 
subcommand, the next step is to manage threads and processes in those
Line 571: Line 514:
 
assume a cpuset setup of two sets as follows:
 
assume a cpuset setup of two sets as follows:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -l
 
[zuul:cpuset-trunk]# cset set -l
 
cset:
 
cset:
Line 579: Line 522:
 
           two          2 n      0 n    3    0 /two
 
           two          2 n      0 n    3    0 /two
 
         three          3 n      0 n    10    0 /three
 
         three          3 n      0 n    10    0 /three
----------------------------------------------------------
+
</pre>
  
Listing Tasks with Proc
+
===Listing Tasks with Proc===
22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)[[User:Alext|Alext]]
+
 
Operation of the <tt>proc</tt> subcommand follows the same model as the <tt>set</tt>
 
Operation of the <tt>proc</tt> subcommand follows the same model as the <tt>set</tt>
 
subcommand.  For example, to list tasks in a cpuset, you need to use the
 
subcommand.  For example, to list tasks in a cpuset, you need to use the
<tt>-l/\--list</tt> option and specify the cpuset by name or, if the name exists
+
<tt>-l/--list</tt> option and specify the cpuset by name or, if the name exists
 
multiple times in the cpuset hierarchy, by path.  For example:
 
multiple times in the cpuset hierarchy, by path.  For example:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -l -s two
 
[zuul:cpuset-trunk]# cset proc -l -s two
 
cset: "two" cpuset of CPUSPEC(2) with 3 tasks running
 
cset: "two" cpuset of CPUSPEC(2) with 3 tasks running
Line 596: Line 538:
 
  root    16171 16141 Soth bash
 
  root    16171 16141 Soth bash
 
  root    16703 16171 Roth python ./cset proc -l two
 
  root    16703 16171 Roth python ./cset proc -l two
----------------------------------------------------------
+
</pre>
  
 
This output shows us that the cpuset called ''two'' has CPU2 only attached to it
 
This output shows us that the cpuset called ''two'' has CPU2 only attached to it
Line 614: Line 556:
 
number from 1 to 99.  See below for an example.
 
number from 1 to 99.  See below for an example.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -l -s root | head -7
 
[zuul:cpuset-trunk]# cset proc -l -s root | head -7
 
cset: "root" cpuset of CPUSPEC(0-3) with 309 tasks running
 
cset: "root" cpuset of CPUSPEC(0-3) with 309 tasks running
Line 623: Line 565:
 
  root        3    2 Sf99 [migration/0]
 
  root        3    2 Sf99 [migration/0]
 
  root        4    2 Sf99 [posix_cpu_timer]
 
  root        4    2 Sf99 [posix_cpu_timer]
----------------------------------------------------------
+
</pre>
  
 
This output shows the first few tasks in the ''root'' cpuset.  Note that both
 
This output shows the first few tasks in the ''root'' cpuset.  Note that both
Line 635: Line 577:
 
Taking a peek into the third cpuset called ''three'', we see:
 
Taking a peek into the third cpuset called ''three'', we see:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -l -s three
 
[zuul:cpuset-trunk]# cset proc -l -s three
 
cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
 
cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
Line 650: Line 592:
 
  alext    17244    1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 
  alext    17244    1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 
  alext    17265    1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 
  alext    17265    1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
----------------------------------------------------------
+
</pre>
  
 
This output shows that a lot of ''beagled'' tasks are running in this cpuset and
 
This output shows that a lot of ''beagled'' tasks are running in this cpuset and
 
it also shows an ellipsis (...) at the end of their listings.  If you see this
 
it also shows an ellipsis (...) at the end of their listings.  If you see this
 
ellipsis, that means that the command was too long to fit onto an 80 character
 
ellipsis, that means that the command was too long to fit onto an 80 character
screen.  To see the entire commandline, use the <tt>-v/\--verbose</tt> flag, as per
+
screen.  To see the entire commandline, use the <tt>-v/--verbose</tt> flag, as per
 
following.
 
following.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -l -s three -v | head -4
 
[zuul:cpuset-trunk]# cset proc -l -s three -v | head -4
 
cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
 
cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
Line 664: Line 606:
 
  -------- ----- ----- ---- ---------
 
  -------- ----- ----- ---- ---------
 
  alext    16165    1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg --autostarted --indexing-delay 300
 
  alext    16165    1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg --autostarted --indexing-delay 300
----------------------------------------------------------
+
</pre>
  
 
+
===Execing Tasks with Proc===
Execing Tasks with Proc
+
22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)[[User:Alext|Alext]]
+
 
To exec a task into a cpuset, the <tt>proc</tt> subcommand needs to be employed with
 
To exec a task into a cpuset, the <tt>proc</tt> subcommand needs to be employed with
the <tt>-e/\--exec</tt> option.  Let''s exec a shell into the cpuset named ''two' in
+
the <tt>-e/--exec</tt> option.  Let''s exec a shell into the cpuset named ''two' in
 
our set.  First we check to see what is running that set:
 
our set.  First we check to see what is running that set:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -l -s two
 
[zuul:cpuset-trunk]# cset proc -l -s two
 
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
 
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
Line 686: Line 626:
 
  root    20955 19253 Soth bash
 
  root    20955 19253 Soth bash
 
  root    20981 20955 Roth python ./cset proc -l two
 
  root    20981 20955 Roth python ./cset proc -l two
----------------------------------------------------------
+
</pre>
  
 
You can see that initially, ''two'' had nothing running in it.  After the
 
You can see that initially, ''two'' had nothing running in it.  After the
Line 697: Line 637:
 
Let's test that by just running a new shell with no prefixed '''cset''' command.
 
Let's test that by just running a new shell with no prefixed '''cset''' command.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# bash
 
[zuul:cpuset-trunk]# bash
  
Line 707: Line 647:
 
  root    21118 20955 Soth bash
 
  root    21118 20955 Soth bash
 
  root    21147 21118 Roth python ./cset proc -l two
 
  root    21147 21118 Roth python ./cset proc -l two
----------------------------------------------------------
+
</pre>
  
 
Here again we see that the second shell, PID 21118, has a parent PID of 20955
 
Here again we see that the second shell, PID 21118, has a parent PID of 20955
Line 713: Line 653:
 
running in the ''two'' cpuset.
 
running in the ''two'' cpuset.
  
NOTE: '''cset''' follows the tradition of separating the tool options from the
+
{{NOTE|'''cset''' follows the tradition of separating the tool options from the command to be execed options with a double dash (<tt>--</tt>).  This is not shown in this simple example, but if the command you want to exec also takes options, separate them with the double dash like so: +# cset proc -s myset -e mycommand -- -v<tt> The </tt>-v<tt> will be passed to </tt>mycommand+, and not to '''cset'''.}}
command to be execed options with a double dash (<tt>\--</tt>).  This is not shown in
+
this simple example, but if the command you want to exec also takes options,
+
separate them with the double dash like so: +# cset proc -s myset -e mycommand
+
\-- -v<tt> The </tt>-v<tt> will be passed to </tt>mycommand+, and not to '''cset'''.
+
  
TIP: Execing a shell into a cpuset is a useful way to experiment with running
+
{{HINT|Execing a shell into a cpuset is a useful way to experiment with running tasks in that cpuset since all children of the shell will also run in the same cpuset.}}
tasks in that cpuset since all children of the shell will also run in the same
+
cpuset.
+
  
 
Finally, if you misspell the command to be execed, the result may be
 
Finally, if you misspell the command to be execed, the result may be
 
puzzling. For example:
 
puzzling. For example:
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -s two -e blah-blah
 
[zuul:cpuset-trunk]# cset proc -s two -e blah-blah
 
cset: --> last message, executed args into cpuset "/two", new pid is: 21655
 
cset: --> last message, executed args into cpuset "/two", new pid is: 21655
 
cset: ''''''> [Errno 2] No such file or directory
 
cset: ''''''> [Errno 2] No such file or directory
----------------------------------------------------------
+
</pre>
  
 
The result is no new process even though a new PID is output.  The reason for
 
The result is no new process even though a new PID is output.  The reason for
Line 736: Line 670:
 
exec, but the command <tt>blah-blah</tt> was not found in order to exec it.
 
exec, but the command <tt>blah-blah</tt> was not found in order to exec it.
  
 
+
===Moving Tasks with Proc===
Moving Tasks with Proc
+
22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)~~
+
 
Although the ability to exec a task into a cpuset is fundamental, you will
 
Although the ability to exec a task into a cpuset is fundamental, you will
 
most likely be moving tasks between cpusets more often.  Moving tasks is
 
most likely be moving tasks between cpusets more often.  Moving tasks is
accomplished with the <tt>-m/\--move</tt> and <tt>-p/\--pid</tt> options to the <tt>proc</tt>
+
accomplished with the <tt>-m/--move</tt> and <tt>-p/--pid</tt> options to the <tt>proc</tt>
 
subcommand of '''cset'''.  The move option tells the <tt>proc</tt> subcommand that a task
 
subcommand of '''cset'''.  The move option tells the <tt>proc</tt> subcommand that a task
move is requested.  The <tt>-p/\--pid</tt> option takes an argument called a PIDSPEC
+
move is requested.  The <tt>-p/--pid</tt> option takes an argument called a PIDSPEC
 
(PID Specification).  The PIDSPEC defines which tasks get operated on.
 
(PID Specification).  The PIDSPEC defines which tasks get operated on.
  
Line 750: Line 682:
 
example:
 
example:
  
+\--move --pid 1234+::
+
;--move --pid 1234
This PIDSPEC argument specifies that task 1234 be moved.
+
:This PIDSPEC argument specifies that task 1234 be moved.
  
+\--move --pid 1234,42,1934,15000,15001,15002+::
+
;--move --pid 1234,42,1934,15000,15001,15002
This PIDSPEC argument specifies that this list of tasks only be moved.
+
:This PIDSPEC argument specifies that this list of tasks only be moved.
  
+\--move --pid 5000,5100,6010-7000,9232+::
+
;--move --pid 5000,5100,6010-7000,9232
This PIDSPEC argument specifies that tasks 5000,5100 and 9232 be moved along
+
:This PIDSPEC argument specifies that tasks 5000,5100 and 9232 be moved along with any existing task that is in the range 6010 through 7000 inclusive.
with any existing task that is in the range 6010 through 7000 inclusive.
+
  
NOTE: A range in a PIDSPEC does not have to have running tasks for every
+
{{NOTE|A range in a PIDSPEC does not have to have running tasks for every number in that range.  In fact, it is not even an error if there are no tasks running in that range; none will be moved in that case.  The range simply specifies to act on any tasks that have a PID or TID that is within that range.}}
number in that range.  In fact, it is not even an error if there are no tasks
+
running in that range; none will be moved in that case.  The range simply
+
specifies to act on any tasks that have a PID or TID that is within that
+
range.
+
  
 
In the following example, we move the current shell into the cpuset named
 
In the following example, we move the current shell into the cpuset named
Line 770: Line 697:
 
variable for the current PID.
 
variable for the current PID.
  
-----------------------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -l -s two
 
[zuul:cpuset-trunk]# cset proc -l -s two
 
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
 
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
Line 796: Line 723:
 
[zuul:cpuset-trunk]# cset proc -l -s two
 
[zuul:cpuset-trunk]# cset proc -l -s two
 
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
 
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
-----------------------------------------------------------------------
+
</pre>
  
 
Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of
 
Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of
 
tasks.  Additionally, there is one more option that can help with
 
tasks.  Additionally, there is one more option that can help with
multi-threaded processes, and that is the <tt>\--threads</tt> flag.  If this flag is
+
multi-threaded processes, and that is the <tt>--threads</tt> flag.  If this flag is
 
present in a <tt>proc</tt> move command with a PIDSPEC and if any of the task IDs in
 
present in a <tt>proc</tt> move command with a PIDSPEC and if any of the task IDs in
 
the PIDSPEC belongs to a thread in a process container, then '''all''' the sibling
 
the PIDSPEC belongs to a thread in a process container, then '''all''' the sibling
Line 806: Line 733:
 
easy mechanism to move all threads of a process by simply specifying one
 
easy mechanism to move all threads of a process by simply specifying one
 
thread in that process.  In the following example, we move all the threads
 
thread in that process.  In the following example, we move all the threads
running in cpuset ''three'' to cpuset ''two'' by using the <tt>\--threads</tt> flag.
+
running in cpuset ''three'' to cpuset ''two'' by using the <tt>--threads</tt> flag.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set two three
 
[zuul:cpuset-trunk]# cset set two three
 
cset:
 
cset:
Line 843: Line 770:
 
           two          2 n      0 n    10    0 /two
 
           two          2 n      0 n    10    0 /two
 
         three          3 n      0 n    0    0 /three
 
         three          3 n      0 n    0    0 /three
----------------------------------------------------------
+
</pre>
 
+
  
Moving All Tasks from one Cpuset to Another
+
====Moving All Tasks from one Cpuset to Another====
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 
There is a special case for moving all tasks currently running in one cpuset
 
There is a special case for moving all tasks currently running in one cpuset
 
to another.  This can be a common use case, and when you need to do it,
 
to another.  This can be a common use case, and when you need to do it,
 
specifying a PIDSPEC with <tt>-p</tt> is not necessary ''so long as'' you use the  
 
specifying a PIDSPEC with <tt>-p</tt> is not necessary ''so long as'' you use the  
<tt>-f/\--fromset</tt> '''and''' the <tt>-t/\--toset</tt> options.
+
<tt>-f/--fromset</tt> '''and''' the <tt>-t/--toset</tt> options.
  
 
In the following example, we move all 10 <tt>beagled</tt> threads back to cpuset
 
In the following example, we move all 10 <tt>beagled</tt> threads back to cpuset
 
''three'' with this method.
 
''three'' with this method.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -l two three
 
[zuul:cpuset-trunk]# cset proc -l two three
 
cset: "two" cpuset of CPUSPEC(2) with 10 tasks running
 
cset: "two" cpuset of CPUSPEC(2) with 10 tasks running
Line 885: Line 810:
 
           two          2 n      0 n    0    0 /two
 
           two          2 n      0 n    0    0 /two
 
         three          3 n      0 n    10    0 /three
 
         three          3 n      0 n    10    0 /three
----------------------------------------------------------
+
</pre>
  
 
+
====Moving Kernel Threads with Proc====
Moving Kernel Threads with Proc
+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 
Kernel threads are special and '''cset''' detects tasks that are kernel threads
 
Kernel threads are special and '''cset''' detects tasks that are kernel threads
and will refuse to move them unless you also add a <tt>-k/\--kthread</tt> option to
+
and will refuse to move them unless you also add a <tt>-k/--kthread</tt> option to
 
your <tt>proc</tt> move command.  Even if you include <tt>-k</tt>, '''cset''' will ''still''
 
your <tt>proc</tt> move command.  Even if you include <tt>-k</tt>, '''cset''' will ''still''
 
refuse to move the kernel thread if they are bound to specific CPUs.  The
 
refuse to move the kernel thread if they are bound to specific CPUs.  The
Line 904: Line 827:
 
option additionally.
 
option additionally.
  
WARNING: Overriding a task move command with <tt>--force</tt> can have dire
+
{{WARN|Overriding a task move command with <tt>--force</tt> can have '''dire consequences''' for the system.  Please be sure of the command before you force it.}}
consequences for the system.  Please be sure of the command before you force
+
it.
+
  
 
In the following example, we move all unbound kernel threads running in the
 
In the following example, we move all unbound kernel threads running in the
 
''root'' cpuset to the cpuset named ''two'' by using the <tt>-k</tt> option.
 
''root'' cpuset to the cpuset named ''two'' by using the <tt>-k</tt> option.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -k -f root -t two
 
[zuul:cpuset-trunk]# cset proc -k -f root -t two
 
cset: moving all kernel threads from / to /two
 
cset: moving all kernel threads from / to /two
Line 918: Line 839:
 
[==================================================]%
 
[==================================================]%
 
cset: done
 
cset: done
----------------------------------------------------------
+
</pre>
  
 
You will note that we used the fromset->toset facility of the <tt>proc</tt>
 
You will note that we used the fromset->toset facility of the <tt>proc</tt>
Line 928: Line 849:
 
specific CPUs.  Now, let''s move those kernel threads back to ''root'.
 
specific CPUs.  Now, let''s move those kernel threads back to ''root'.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -k -f two -t root
 
[zuul:cpuset-trunk]# cset proc -k -f two -t root
 
cset: moving all kernel threads from /two to /
 
cset: moving all kernel threads from /two to /
cset: '''''' no task matched move criteria
+
cset: **> no task matched move criteria
cset: ''''''> kernel tasks are bound, use --force if ok
+
cset: **> kernel tasks are bound, use --force if ok
  
 
[zuul:cpuset-trunk]# cset set -l -s two
 
[zuul:cpuset-trunk]# cset set -l -s two
Line 939: Line 860:
 
  ------------ ---------- - ------- - ----- ---- ----------
 
  ------------ ---------- - ------- - ----- ---- ----------
 
           two          2 n      0 n    70    0 /two
 
           two          2 n      0 n    70    0 /two
----------------------------------------------------------
+
</pre>
  
 
Ah!  What's this?  '''Cset''' refused to move the kernel threads back to ''root''
 
Ah!  What's this?  '''Cset''' refused to move the kernel threads back to ''root''
Line 945: Line 866:
 
taskset command.
 
taskset command.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -l -s two | head -5
 
[zuul:cpuset-trunk]# cset proc -l -s two | head -5
 
cset: "two" cpuset of CPUSPEC(2) with 70 tasks running
 
cset: "two" cpuset of CPUSPEC(2) with 70 tasks running
Line 961: Line 882:
 
  ------------ ---------- - ------- - ----- ---- ----------
 
  ------------ ---------- - ------- - ----- ---- ----------
 
           two          2 n      0 n    70    0 /two
 
           two          2 n      0 n    70    0 /two
----------------------------------------------------------
+
</pre>
  
 
Of course, since the cpuset named ''two'' only has CPU2 assigned to it, once we
 
Of course, since the cpuset named ''two'' only has CPU2 assigned to it, once we
Line 969: Line 890:
 
''root'', we need to force the move as follows.
 
''root'', we need to force the move as follows.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -k -f two -t root --force
 
[zuul:cpuset-trunk]# cset proc -k -f two -t root --force
 
cset: moving all kernel threads from /two to /
 
cset: moving all kernel threads from /two to /
Line 975: Line 896:
 
[==================================================]%
 
[==================================================]%
 
cset: done
 
cset: done
----------------------------------------------------------
+
</pre>
  
 
+
===Destroying Tasks===
Destroying Tasks
+
22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)22:51, 11 February 2009 (UTC)~
+
 
There actually is no '''cset''' subcommand or option to destroy tasks--it's not
 
There actually is no '''cset''' subcommand or option to destroy tasks--it's not
 
really needed.  Tasks exist and are accessible on the system as normal, even
 
really needed.  Tasks exist and are accessible on the system as normal, even
Line 985: Line 904:
 
the usual ^C method or by using the *kill(1)* command.
 
the usual ^C method or by using the *kill(1)* command.
  
 
+
==Implementing "Shielding" with Set and Proc==
Implementing "Shielding" with Set and Proc
+
------------------------------------------
+
 
With the preceding material on the <tt>set</tt> and <tt>proc</tt> subcommands, we now have
 
With the preceding material on the <tt>set</tt> and <tt>proc</tt> subcommands, we now have
 
the background to implement the basic shielding model, just like the <tt>shield</tt>
 
the background to implement the basic shielding model, just like the <tt>shield</tt>
Line 1,012: Line 929:
 
the rest of the CPUs.
 
the rest of the CPUs.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -c 0 -s system
 
[zuul:cpuset-trunk]# cset set -c 0 -s system
 
cset: --> created cpuset "system"
 
cset: --> created cpuset "system"
Line 1,026: Line 943:
 
         user        1-3 n      0 n    0    0 /user
 
         user        1-3 n      0 n    0    0 /user
 
       system          0 n      0 n    0    0 /system
 
       system          0 n      0 n    0    0 /system
----------------------------------------------------------
+
</pre>
  
 
Now, we need to move all running user processes into the ''system'' cpuset.
 
Now, we need to move all running user processes into the ''system'' cpuset.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -m -f root -t system
 
[zuul:cpuset-trunk]# cset proc -m -f root -t system
 
cset: moving all tasks from root to /system
 
cset: moving all tasks from root to /system
Line 1,044: Line 961:
 
         user        1-3 n      0 n    0    0 /user
 
         user        1-3 n      0 n    0    0 /user
 
       system          0 n      0 n  187    0 /system
 
       system          0 n      0 n  187    0 /system
----------------------------------------------------------
+
</pre>
  
 
We now have the basic shielding set up.  Since all userspace tasks are running
 
We now have the basic shielding set up.  Since all userspace tasks are running
Line 1,054: Line 971:
 
execute the following command as well.
 
execute the following command as well.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -k -f root -t system
 
[zuul:cpuset-trunk]# cset proc -k -f root -t system
 
cset: moving all kernel threads from / to /system
 
cset: moving all kernel threads from / to /system
Line 1,069: Line 986:
 
         user        1-3 n      0 n    0    0 /user
 
         user        1-3 n      0 n    0    0 /user
 
       system          0 n      0 n  257    0 /system
 
       system          0 n      0 n  257    0 /system
----------------------------------------------------------
+
</pre>
  
 
At this point, you have achieved the simple shielding model that the <tt>shield</tt>
 
At this point, you have achieved the simple shielding model that the <tt>shield</tt>
Line 1,075: Line 992:
 
shielding strategy beyond that simple model.
 
shielding strategy beyond that simple model.
  
 
+
==Implementing Hierarchy with Set and Proc==
Implementing Hierarchy with Set and Proc
+
----------------------------------------
+
 
One popular extended "shielding" model is based on hierarchical cpusets, each
 
One popular extended "shielding" model is based on hierarchical cpusets, each
 
with diminishing numbers of CPUs.  This model is used to create "priority
 
with diminishing numbers of CPUs.  This model is used to create "priority
Line 1,097: Line 1,012:
 
CPUs, 'prio_high'' with three CPUs, and ''prio_all' with all CPUs.
 
CPUs, 'prio_high'' with three CPUs, and ''prio_all' with all CPUs.
  
NOTE: One may ask, why create a 'prio_all' with all CPUs when that is
+
{{NOTE|One may ask, why create a 'prio_all' with all CPUs when that is substantially the definition of the ''root'' cpuset?  The answer is that it is best to keep a separation between the ''root'' cpuset and everything else, even if a particular cpuset duplicates ''root'' exactly.  Usually, one builds automation on top of a cpuset strategy.  In these cases, it is best to avoid using invariant names of cpusets, such as ''root'' for example, in this automation.}}
substantially the definition of the ''root'' cpuset?  The answer is that it is
+
best to keep a separation between the ''root'' cpuset and everything else, even
+
if a particular cpuset duplicates ''root'' exactly.  Usually, one builds
+
automation on top of a cpuset strategy.  In these cases, it is best to avoid
+
using invariant names of cpusets, such as ''root'' for example, in this
+
automation.
+
  
 
All of these 'prio_*' cpusets can be created under root, in a flat way;
 
All of these 'prio_*' cpusets can be created under root, in a flat way;
Line 1,121: Line 1,030:
 
follows.
 
follows.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -r
 
[zuul:cpuset-trunk]# cset set -r
 
cset:
 
cset:
Line 1,153: Line 1,062:
 
     prio_med        2-3 n      0 n    0    1 /prio_all/prio_high/prio_med
 
     prio_med        2-3 n      0 n    0    1 /prio_all/prio_high/prio_med
 
     prio_low          3 n      0 n    0    0 /prio_all/pr...rio_med/prio_low
 
     prio_low          3 n      0 n    0    0 /prio_all/pr...rio_med/prio_low
----------------------------------------------------------
+
</pre>
  
NOTE: We used the <tt>-r/\--recurse</tt> switch to list all the sets in the last
+
NOTE: We used the <tt>-r/--recurse</tt> switch to list all the sets in the last
 
command above.  If we had not, then the 'prio_med'' and ''prio_low' cpusets
 
command above.  If we had not, then the 'prio_med'' and ''prio_low' cpusets
 
would not have been listed.
 
would not have been listed.
Line 1,162: Line 1,071:
 
all movable kernel threads into the ''system'' cpuset to activate the shield.
 
all movable kernel threads into the ''system'' cpuset to activate the shield.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset proc -m -k -f root -t system
 
[zuul:cpuset-trunk]# cset proc -m -k -f root -t system
 
cset: moving all tasks from root to /system
 
cset: moving all tasks from root to /system
Line 1,181: Line 1,090:
 
     prio_med        2-3 n      0 n    0    1 /prio_all/prio_high/prio_med
 
     prio_med        2-3 n      0 n    0    1 /prio_all/prio_high/prio_med
 
     prio_low          3 n      0 n    0    0 /prio_all/pr...rio_med/prio_low
 
     prio_low          3 n      0 n    0    0 /prio_all/pr...rio_med/prio_low
----------------------------------------------------------
+
</pre>
  
 
The shield is now active.  Since the 'prio_*' cpuset names are unique, one can
 
The shield is now active.  Since the 'prio_*' cpuset names are unique, one can
Line 1,190: Line 1,099:
 
cpuset in the listing above.  This is done in order to fit the output onto an
 
cpuset in the listing above.  This is done in order to fit the output onto an
 
80 character screen.  If you want to see the entire line, then you need to use
 
80 character screen.  If you want to see the entire line, then you need to use
the <tt>-v/\--verbose</tt> flag as follows.
+
the <tt>-v/--verbose</tt> flag as follows.
  
----------------------------------------------------------
+
<pre>
 
[zuul:cpuset-trunk]# cset set -l -r -v
 
[zuul:cpuset-trunk]# cset set -l -r -v
 
cset:
 
cset:
Line 1,203: Line 1,112:
 
     prio_med        2-3 n      0 n    0    1 /prio_all/prio_high/prio_med
 
     prio_med        2-3 n      0 n    0    1 /prio_all/prio_high/prio_med
 
     prio_low          3 n      0 n    0    0 /prio_all/prio_high/prio_med/prio_low
 
     prio_low          3 n      0 n    0    0 /prio_all/prio_high/prio_med/prio_low
----------------------------------------------------------
+
</pre>
  
 
+
=Using Shortcuts=
Using Shortcuts
+
===============
+
 
The commands listed in the previous sections always used all the required
 
The commands listed in the previous sections always used all the required
 
options.  '''Cset''' however does have a shortcut facility that will execute
 
options.  '''Cset''' however does have a shortcut facility that will execute
Line 1,225: Line 1,132:
 
example, the following commands are identical:
 
example, the following commands are identical:
  
----------------------------------------------------------
+
<pre>
 
# cset shield -s -p 1234        <-->    # cset sh -s -p 1234
 
# cset shield -s -p 1234        <-->    # cset sh -s -p 1234
 
# cset set -c 1,3 -s newset      <-->    # cset se -c 1,3 -s newset
 
# cset set -c 1,3 -s newset      <-->    # cset se -c 1,3 -s newset
 
# cset proc -s newset -e bash    <-->    # cset p -s newset -e bash
 
# cset proc -s newset -e bash    <-->    # cset p -s newset -e bash
----------------------------------------------------------
+
</pre>
  
 
Note that <tt>proc</tt> can be shortened to just <tt>p</tt>, while <tt>shield</tt> and <tt>set</tt> need
 
Note that <tt>proc</tt> can be shortened to just <tt>p</tt>, while <tt>shield</tt> and <tt>set</tt> need
 
two letters to disambiguate.
 
two letters to disambiguate.
  
 
+
==Shield Subcommand Shortcuts==
Shield Subcommand Shortcuts
+
---------------------------
+
 
The <tt>shield</tt> subcommand supports two areas with shortcuts: the case when there
 
The <tt>shield</tt> subcommand supports two areas with shortcuts: the case when there
 
is no options given where to ''shield'' is the common use case, and making the
 
is no options given where to ''shield'' is the common use case, and making the
<tt>-p/\--pid</tt> option ''optional'' for the <tt>-s/\--shield</tt> and <tt>-u/\--unshield</tt>
+
<tt>-p/--pid</tt> option ''optional'' for the <tt>-s/--shield</tt> and <tt>-u/--unshield</tt>
 
options.
 
options.
  
Line 1,245: Line 1,150:
 
command into the shield, the following '''cset''' commands are equivalent.
 
command into the shield, the following '''cset''' commands are equivalent.
  
-----------------------------------------------------------
+
<pre>
 
# cset shield -s -p 1234,500-649  <-->    # cset sh 1234,500-649
 
# cset shield -s -p 1234,500-649  <-->    # cset sh 1234,500-649
 
# cset shield -s -e bash          <-->    # cset sh bash
 
# cset shield -s -e bash          <-->    # cset sh bash
-----------------------------------------------------------
+
</pre>
  
 
When using the <tt>-s</tt> or <tt>-u</tt> shield/unshield options, it is optional to use the
 
When using the <tt>-s</tt> or <tt>-u</tt> shield/unshield options, it is optional to use the
 
<tt>-p</tt> option to specify a PIDSPEC.  For example:
 
<tt>-p</tt> option to specify a PIDSPEC.  For example:
  
-----------------------------------------------------------
+
<pre>
 
# cset shield -s -p 1234    <-->    # cset sh -s 1234
 
# cset shield -s -p 1234    <-->    # cset sh -s 1234
 
# cset shield -u -p 1234    <-->    # cset sh -u 1234
 
# cset shield -u -p 1234    <-->    # cset sh -u 1234
-----------------------------------------------------------
+
</pre>
  
 
+
==Set Subcommand Shortcuts==
Set Subcommand Shortcuts
+
------------------------
+
 
The <tt>set</tt> subcommand has a limited number of shortcuts.  Basically, the
 
The <tt>set</tt> subcommand has a limited number of shortcuts.  Basically, the
<tt>-s/\--set</tt> option is optional in most cases and the <tt>-l/\--list</tt> option is
+
<tt>-s/--set</tt> option is optional in most cases and the <tt>-l/--list</tt> option is
 
also optional if you want to list sets.  For example, these commands are
 
also optional if you want to list sets.  For example, these commands are
 
equivalent.
 
equivalent.
  
-----------------------------------------------------------
+
<pre>
 
# cset set -l -s myset          <-->  # cset se -l myset
 
# cset set -l -s myset          <-->  # cset se -l myset
 
# cset se -l myset              <-->  # cset se myset
 
# cset se -l myset              <-->  # cset se myset
Line 1,274: Line 1,177:
  
 
# cset set -n newname -s oldname <-->  # cset se -n newname oldname
 
# cset set -n newname -s oldname <-->  # cset se -n newname oldname
-----------------------------------------------------------
+
</pre>
  
 
In fact, if you want to apply either the list or the destroy options to
 
In fact, if you want to apply either the list or the destroy options to
Line 1,280: Line 1,183:
 
option.  For example:
 
option.  For example:
  
-----------------------------------------------------------
+
<pre>
 
# cset se -d myset yourset ourset
 
# cset se -d myset yourset ourset
 
   --> destroys cpusets: myset, yourset and ourset
 
   --> destroys cpusets: myset, yourset and ourset
Line 1,287: Line 1,190:
 
   --> lists only cpusets prio_high, prio_med and prio_low
 
   --> lists only cpusets prio_high, prio_med and prio_low
 
   --> the -l is optional in this case since list is default
 
   --> the -l is optional in this case since list is default
-----------------------------------------------------------
+
</pre>
  
Proc Subcommand Shortcuts
+
==Proc Subcommand Shortcuts==
-------------------------
+
 
For the <tt>proc</tt> subcommand, the <tt>-s</tt>, <tt>-t</tt> and <tt>-f</tt> options to specify the
 
For the <tt>proc</tt> subcommand, the <tt>-s</tt>, <tt>-t</tt> and <tt>-f</tt> options to specify the
 
cpuset, the origination cpuset and the destination cpuset, can sometimes be
 
cpuset, the origination cpuset and the destination cpuset, can sometimes be
 
optional.  For example, the following commands are equivalent.
 
optional.  For example, the following commands are equivalent.
  
-----------------------------------------------------------
+
<pre>
 
To list tasks in cpusets:
 
To list tasks in cpusets:
 
# cset proc -l -s myset        \
 
# cset proc -l -s myset        \
Line 1,308: Line 1,210:
 
To exec a process into a cpuset:
 
To exec a process into a cpuset:
 
# cset proc -s myset -e bash  <-->  # cset p myset -e bash
 
# cset proc -s myset -e bash  <-->  # cset p myset -e bash
-----------------------------------------------------------
+
</pre>
  
 
Movement of tasks into and out of cpusets have the following shortcuts.
 
Movement of tasks into and out of cpusets have the following shortcuts.
  
-----------------------------------------------------------
+
<pre>
 
To move a PIDSPEC into a cpuset:
 
To move a PIDSPEC into a cpuset:
 
# cset proc -m -p 4242,4243 -s myset <--> # cset p -m 4242,4243 myset
 
# cset proc -m -p 4242,4243 -s myset <--> # cset p -m 4242,4243 myset
Line 1,321: Line 1,223:
 
# cset proc -m -s set1 -t set2      --> # cset p -m set1 set2
 
# cset proc -m -s set1 -t set2      --> # cset p -m set1 set2
 
# cset proc -m -f set1 -s set2      /
 
# cset proc -m -f set1 -s set2      /
-----------------------------------------------------------
+
</pre>
  
 
+
=What To Do if There are Problems=
What To Do if There are Problems
+
================================
+
 
If you encounter problems with the '''cset''' application, the best option is to
 
If you encounter problems with the '''cset''' application, the best option is to
log a bug with the '''cset''' bugzilla instance found here:
+
log a bug with the '''cset''' issues instance found here:
  
  http://devzilla.novell.com/cpuset
+
  [[http://code.google.com/p/cpuset/issues/list http://code.google.com/p/cpuset/issues/list]]
  
 
If you are using '''cset''' on a supported operating system such as SLES or SLERT
 
If you are using '''cset''' on a supported operating system such as SLES or SLERT
Line 1,337: Line 1,237:
 
fixed quickly.  Also, '''cset''' contains a logging facility that is invaluable
 
fixed quickly.  Also, '''cset''' contains a logging facility that is invaluable
 
for the developers to diagnose problems.  To create a log of a run, use the
 
for the developers to diagnose problems.  To create a log of a run, use the
<tt>-l/\--log</tt> option with a filename as an argument to the main '''cset'''
+
<tt>-l/--log</tt> option with a filename as an argument to the main '''cset'''
 
application.  For example.
 
application.  For example.
  
-------------------------------------------------------------
+
  # cset -l logfile.txt set -n newname oldname
# cset -l logfile.txt set -n newname oldname
+
-------------------------------------------------------------
+
  
 
That command saves a lot of debugging information in the ''logfile.txt'' file.
 
That command saves a lot of debugging information in the ''logfile.txt'' file.
 
Please attach this file to the bug.
 
Please attach this file to the bug.
 +
 +
----
 +
[[Cpuset Management Utility | Back up to cpuset page.]]

Latest revision as of 19:42, 6 September 2011

Back up to cpuset page.

Cpuset (cset) Tutorial

Alex Tsariounov <alext@novell.com>
Copyright (c) 2009 Novell Inc., cset v1.5.0
Verbatim copying and distribution of this entire article are permitted
worldwide, without royalty, in any medium, provided this notice is preserved.

This tutorial describes basic and advanced usage of the cset command to manipulate cpusets on a Linux system. See also the manpages that come with the cset command: cset(1), cset-shield(1), cset-set(1), and cset-proc(1) for more details. Additionally, the cset command has online help.

Contents

[edit] Introduction

In the Linux kernel, the cpuset facility provides a mechanism for creating logical entities called "cpusets" that encompass definitions of CPUs and NUMA Memory Nodes (if NUMA is available). Cpusets constrain the CPU and Memory placement of a task to only the resources defined within that cpuset. These cpusets can then be arranged into a nested hierarchy visible in the "cpuset" virtual filesystem. Sets of tasks can be assigned to these cpusets to constrain the resources that they use. The tasks can be moved from one cpuset to another to utilize other resources defined in those other cpusets.

The cset command is a Python application that provides a command line front end for the Linux cpusets functionality. Working with cpusets directly can be confusing and slightly complex. The cset tool hides that complexity behind an easy-to-use command line interface.

There are two distinct use cases for cset: the basic shielding use case and the "advanced" case of using raw set and proc subcommands. The basic shielding function is accessed with the shield subcommand and described in the next section. Using the raw set and proc subcommands allows one to set up arbitrarily complex cpusets and is described in the later sections.

Note that in general, one either uses the shield subcommand or a combination of the set and proc subcommands. One rarely, if ever, uses all of these subcommands together. Doing so will likely become too confusing. Additionally, the shield subcommand sets up its required cpusets with exclusively marked CPUs. This can interfere with your cpuset strategy. If you find that you need more functionality for your strategy than shield provides, go ahead and transition to using set and proc exclusively. It is straightforward to implement what shield does with a few extra set and proc subcommands.

[edit] Obtaining Online Help

For a full list of cset subcommands:

   # cset help

For in-depth help on individual subcommands:

   # cset help <subcommand>

For options of individual subcommands:

   # cset <subcommand> (-h | --help)

[edit] The Basic Shielding Model

Although any set up of cpusets can really be described as "shielding," there is one prevalent shielding model in use that is so common that cset has a subcommand that is dedicated to its use. This subcommand is called shield.

The concept behind this model is the use of three cpusets. The root cpuset which is always present in all configurations and contains all CPUs. The system cpuset which contains CPUs which are used for system tasks. These are the normal tasks that are not "important," but which need to run on the system. And finally, the user cpuset which contains CPUs which are used for "important" tasks. The user cpuset is the shield. Only those tasks that are somehow important, usually tasks whose performance determines the overall rating for the machine, are run in the user cpuset.

The shield subcommand manages all of these cpusets and lets you define the CPUs and Memory Nodes that are in the shielded and unshielded sets. The subcommand automatically moves all movable tasks on the system into the unshielded cpuset on shield activation, and back into the root cpuset on shield tear down. The subcommand then lets you move tasks into and out of the shield. Additionally, you can move special tasks (kernel threads) which normally run in the root cpuset into the unshielded set so that your shield will have even less disturbance.

The shield subcommand abstracts the management of these cpusets away from you and provides options that drive how the shield is set up, which tasks are to be shielded and which tasks are not, and status of the shield. In fact, you need not be bothered with the naming of the required cpusets or even where the cpuset filesystem is mounted. Cset and the shield subcommand takes care of all that.

If you find yourself needing to define more cpusets for your application, then it is likely that this simple shielding is not a rich enough model for you. In this case, you should transition to using the set and proc subcommands described in a later section.

[edit] A Simple Shielding Example

Assume that we have a 4-way machine that is not NUMA. This means there are 4 CPUs at our disposal and there is only one Memory Node available. On such machines, we do not need to specify any memory node parameters to cset, it sets up the only available memory node by default.

Usually, one wants to dedicate as many CPUs to the shield as possible and leave a minimal set of CPUs for normal system processing. The reasoning for this is because the performance of the important tasks will rule the performance of the installation as a whole and these important tasks need as many resources available to them as possible, exclusive of other, unimportant tasks that are running on the system.

NOTE!
I use the word "task" to represent either a process or a thread that is running on the system.

[edit] Setup and Teardown of the Shield

To set up a shield of 3 CPUs with 1 CPU left for low priority system processing, issue the following command.

[zuul:cpuset-trunk]# cset shield -c 1-3
cset: --> activating shielding:
cset: moving 176 tasks from root into system cpuset...
[==================================================]%
cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running

This command does a number of things. First, a user cpuset is created with what's called a CPUSPEC (CPU specification) from the -c/--cpu option. This CPUSPEC specifies to use CPUs 1 through 3 inclusively. Next, the command creates a system cpuset with a CPUSPEC that is the inverse of the -c option for the current machine. On this machine that cpuset will only contain the first CPU, CPU0. Next, all userspace processes running in the root cpuset are transfered to the system cpuset. This makes all those processes run only on CPU0. The effect of this is that the shield consists of CPUs 1 through 3 and they are now idling.

Note that the command did not move the kernel threads that are running in the root cpuset to the system cpuset. This is because you may want these kernel threads to use all available CPUs. If you do not, the you can use the -k/--kthread option as described below.

The shield setup command above outputs the information of which cpusets were created and how many tasks are running on each. If you want to see the current status of the shield again, issue this command:

[zuul:cpuset-trunk]# cset shield
cset: --> shielding system active with
cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running

Which shows us that the shield is set up and that 176 tasks are running in the system cpuset--the "unshielded" cpuset.

It is important to move all possible tasks from the root cpuset to the unshielded system cpuset because a task's cpuset property is inherited by its children. Since we've moved all running tasks (including init) to the unshielded system cpuset, that means that any new tasks that are spawned will also run in the unshielded system cpuset.

Some kernel threads can be moved into the unshielded system cpuset as well. These are the threads that are not bound to specific CPUs. If a kernel thread is bound to a specific CPU, then it is generally not a good idea to move that thread to the system set because at worst it may hang the system and at best it will slow the system down significantly. These threads are usually the IRQ threads on a real time Linux kernel, for example, and you may want to not move these kernel threads into system. If you leave them in the root cpuset, then they will have access to all CPUs.

However, if your application demands an even "quieter" shield, then you can move all movable kernel threads into the unshielded system set with the following command.

[zuul:cpuset-trunk]# cset shield -k on
cset: --> activating kthread shielding
cset: kthread shield activated, moving 70 tasks into system cpuset...
[==================================================]%
cset: done

You can see that this moved an additional 70 tasks to the unshielded system cpuset. Note that the -k/--kthread on parameter can be given at the shield creation time as well and you do not need to perform these two steps separately if you know that you will want kernel thread shielding as well. Executing cset shield again shows us the current state of the shield.

[zuul:cpuset-trunk]# cset shield
cset: --> shielding system active with
cset: "system" cpuset of CPUSPEC(0) with 246 tasks running
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running

You can get a detailed listing of what is running in the shield by specifying either -s/--shield or -u/--unshield to the shield subcommand and using the verbose flag. You will get output similar to the following.

[zuul:cpuset-trunk]# cset shield --unshield -v
cset: "system" cpuset of CPUSPEC(0) with 251 tasks running
  USER       PID  PPID SPPr TASK NAME
  -------- ----- ----- ---- ---------
  root         1     0 Soth init [5]
  root         2     0 Soth [kthreadd]
  root        84     2 Sf50 [IRQ-9]
...
  alext    31796 31789 Soth less
  root     32653 25222 Roth python ./cset shield --unshield -v

Note that I abbreviated the listing; we do have 251 tasks running in the system set. The output is self-explanatory; however, the "SPPr" field may need a little explanation. "SPPr" stands for State, Policy and Priority. You can see that the initial two tasks are Stopped and running in timeshare priority, marked as "oth" (for "other"). The [IRQ-9] task is also stopped, but marked at real time FIFO policy with a priority of 50. The last task in the listing is the cset command itself and is marked as running. Also note that adding a second -v/--verbose option will not restrict the output to fit into an 80 character screen.

Tear down of the shield, stopping the shield in other words, is done with the -r/--reset option to the shield subcommand. When this command is issued, both the system and user cpusets are deleted and any tasks that are running in both of those cpusets are moved to the root cpuset. Once so moved, all tasks will have access to all resources on the system. For example:

[zuul:cpuset-trunk]# cset shield --reset
cset: --> deactivating/reseting shielding
cset: moving 0 tasks from "/user" user set to root set...
cset: moving 250 tasks from "/system" system set to root set...
[==================================================]%
cset: deleting "/user" and "/system" sets
cset: done

[edit] Moving Interesting Tasks Into and Out of the Shield

Now that we have a shield running, the objective is to run our "important" processes in that shield. These processes can be anything, but usually they are directly related to the purpose of the machine. There are two ways to run tasks in the shield:

  1. Exec a process into the shield
  2. Move an already running task into the shield

[edit] Execing a Process into the Shield

Running a new process in the shield can be done with the -e/--exec option to the shield subcommand. This is the simplest way to get a task to run in the shield. For this example, let's exec a new bash shell into the shield with the following commands.

 [zuul:cpuset-trunk]# cset shield -s
 cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
 cset: done

 [zuul:cpuset-trunk]# cset shield -e bash
 cset: --> last message, executed args into cpuset "/user", new pid is: 13300

 [zuul:cpuset-trunk]# cset shield -s -v
 cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
   USER       PID  PPID SPPr TASK NAME
   -------- ----- ----- ---- ---------
   root     13300  8583 Soth bash
   root     13329 13300 Roth python ./cset shield -s -v

 [zuul:cpuset-trunk]# exit

 [zuul:cpuset-trunk]# cset shield -s
 cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
 cset: done

The first command above lists the status of the shield. We see that the shield is defined as CPUs 1 through 3 inclusive and currently there are no tasks running in it.

The second command execs the bash shell into the shield with the -e option. The last message of cset lists the PID of the new process.

NOTE!
cset follows the tradition of separating the tool options from the command to be execed options with a double dash (--). This is not shown in this simple example, but if the command you want to exec also takes options, separate them with the double dash like so: # cset shield -e mycommand -- -v The -v will be passed to mycommand, and not to cset.

The next command lists the status of the shield again. You will note that there are actually two tasks running shielded: our new shell and the cset status command itself. Remember that the cpuset property of a task is inherited by its children. Since we ran the new shell in the shield, its child, which is the status command, also ran in the shield.

{{HINT:Execing a shell into the shield is a useful way to experiment with running tasks in the shield since all children of the shell will also run in the shield.}}

The last command exits the shell after which we request a shield status again and see that once again, it does not contain any tasks.

You may have noticed in the output above that both the new shell and the status command are running as the root user. This is because cset needs to run as root and so all it's children will also run as root. If you need to run a process under a different user and or group, you may use the --user and --group options for exec as follows.

 [zuul:cpuset-trunk]# cset shield --user=alext --group=users -e bash
 cset: --> last message, executed args into cpuset "/user", new pid is: 14212

 alext@zuul> cset shield -s -v
 cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
   USER       PID  PPID SPPr TASK NAME
   -------- ----- ----- ---- ---------
   alext    14212  8583 Soth bash
   alext    14241 14212 Roth python ./cset shield -s -v

[edit] Moving a Running Task into and out of the Shield

While execing a process into the shield is undoubtedly useful, most of the time, you'll want to move already running tasks into and out of the shield. The cset shield subcommand includes two options for doing this: -s/--shield and -u/--unshield. These options require what's called a PIDSPEC (process specification) to also be specified with the -p/--pid option. The PIDSPEC defines which tasks get operated on. The PIDSPEC can be a single process ID, a list of process IDs separated by commas, and a list of process ID ranges separated by dashes, groups of which are separated by commas. For example:

--shield --pid 1234
This PIDSPEC argument specifies that PID 1234 be shielded.
--shield --pid 1234,42,1934,15000,15001,15002
This PIDSPEC argument specifies that this list of PIDs only be moved into the shield.
--unshield -p 5000,5100,6010-7000,9232
This PIDSPEC argument specifies that PIDs 5000,5100 and 9232 be unshielded (moved out of the shield) along with any existing PID that is in the range 6010 through 7000 inclusive.
NOTE!
A range in a PIDSPEC does not have to have tasks running for every number in that range. In fact, it is not even an error if there are no tasks running in that range; none will be moved in that case. The range simply specifies to act on any tasks that have a PID or TID that is within that range.

Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of tasks into and out of the shield. Additionally, there is one more option that can help with multi-threaded processes, and that is the --threads flag. If this flag is present in a shield or unshield command with a PIDSPEC and if any of the task IDs in the PIDSPEC belong to a thread in a process container, then all the sibling threads in that process container will get shielded or unshielded as well. This flag provides an easy mechanism to shield/unshield all threads of a process by simply specifying one thread in that process.

In the following example, we move the current shell into the shield with a range PIDSPEC and back out with the bash variable for the current PID.

[zuul:cpuset-trunk]# echo $$
22018

[zuul:cpuset-trunk]# cset shield -s -p 22010-22020
cset: --> shielding following pidspec: 22010-22020
cset: done

[zuul:cpuset-trunk]# cset shield -s -v
cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
   USER       PID  PPID SPPr TASK NAME
   -------- ----- ----- ---- ---------
   root      3770 22018 Roth python ./cset shield -s -v
   root     22018  5034 Soth bash
cset: done

[zuul:cpuset-trunk]# cset shield -u -p $$
cset: --> unshielding following pidspec: 22018
cset: done

[zuul:cpuset-trunk]# cset shield -s
cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
cset: done
NOTE!
Ordinarily, the shield option will shield a PIDSPEC only if it is currently running in the system set--the unshielded set. The unshield option will unshield a PIDSPEC only if it is currently running in the user set--the shielded set. If you want to shield/unshield a process that happens to be running in the root set (not common), then use the --force option for these commands.

[edit] Full Featured Cpuset Manipulation Commands

While basic shielding as described above is useful and a common use model for cset, there comes a time when more functionality will be desired to implement your strategy. To implement this, cset provides two subcommands: set, which allows you to manipulate cpusets; and proc, which allows you to manipulate processes within those cpusets.

[edit] The Set Subcommand

In order to do anything with cpusets, you must be able to create, adjust, rename, move and destroy them. The set subcommand allows the management of cpusets in such a manner.

[edit] Creating and Destroying Cpusets with Set

The basic syntax of set for cpuset creation is:

[zuul:cpuset-trunk]# cset set -c 1-3 -s my_cpuset1
cset: --> created cpuset "my_cpuset1"

This creates a cpuset named "my_cpuset1" with a CPUSPEC of CPU1, CPU2 and CPU3. The CPUSPEC is the same concept as described in the "Setup and Teardown of the Shield" section above. The set subcommand also takes a -m/--mem option that lets you specify the memory nodes the set will use as well as flags to make the CPUs and MEMs exclusive to the cpuset. If you are on a non-NUMA machine, just leave the -m option out and the default memory node 0 will be used.

Just like with shield, you can adjust the CPUs and MEMs with subsequent calls to set. If, for example, you wish to adjust the "my_cpuset1" cpuset to only use CPUs 1 and 3 (and omit CPU2), then issue the following command.

[zuul:cpuset-trunk]# cset set -c 1,3 -s my_cpuset1
cset: --> modified cpuset "my_cpuset

cset will then adjust the CPUs that are assigned to the "my_cpuset1" set to only use CPU1 and CPU3.

To rename a cpuset, use the -n/--newname option. For example:

[zuul:cpuset-trunk]# cset set -s my_cpuset1 -n super_set
cset: --> renaming "/cpusets/my_cpuset1" to "super_set"

Renames the cpuset called "my_cpuset1" to "super_set".

To destroy a cpuset, use the -d/--destroy option as follows.

[zuul:cpuset-trunk]# cset set -d super_set
cset: --> processing cpuset "super_set", moving 0 tasks to parent "/"...
cset: --> deleting cpuset "/super_set"
cset: done

This command destroys the newly created cpuset called "super_set". When a cpuset is destroyed, all the tasks running in it are moved to the parent cpuset. The root cpuset, which always exists and always contains all CPUs, can not be destroyed. You may also give the --destroy option a list of cpusets to destroy.

NOTE!
The cset subcommand creates the cpusets based on a mounted cpuset filesystem. You do not need to know where that filesystem is mounted, although it is easy to figure out (by default its on /cpusets'). When you give the set subcommand a name for a new cpuset, it is created wherever the cpuset filesystem is mounted at.

If you want to create a cpuset hierarchy, then you must give a path to the cset set subcommand. This path will always begin with the root cpuset, for which the path is '/'. For example.

[zuul:cpuset-trunk]# cset set -c 1,3 -s top_set
cset: --> created cpuset "top_set"

[zuul:cpuset-trunk]# cset set -c 3 -s /top_set/sub_set
cset: --> created cpuset "/top_set/sub_set"

These commands created two cpusets: 'top_set and sub_set. The top_set' uses CPU1 and CPU3. It has a subset of 'sub_set' which only uses CPU3. Once you have created a subset with a path, then if the name is unique, you do not have to specify the path in order to affect it. If the name is not unique, then cset will complain and ask you to use the path. For example:

[zuul:cpuset-trunk]# cset set -c 1,3 -s sub_set
cset: --> modified cpuset "sub_set

This command adds CPU1 to the 'sub_set cpuset for its use. Note that using the path in this case is optional.

If you attempt to destroy a cpuset which has sub-cpusets, cset will complain and not do it unless you use the -r/--recurse and the --force options. If you do use --force, then all the tasks running in all subsets of the deletion target cpuset will be moved to the target's parent cpuset and all cpusets.

Moving a cpuset from under a certain cpuset to a different location is currently not implemented and is slated for a later release of cset.

[edit] Listing Cpusets with Set

To list cpusets, use the set subcommand with the '-l/--list' option. For example:

[zuul:cpuset-trunk]# cset set -l
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y   320    1 /
          one          3 n       0 n     0    1 /one

This shows that there is currently one cpuset present called one. (Of course that there is also the root set, which is always present.) The output shows that the one cpuset has no tasks running in it. The root cpuset has 320 tasks running. The "-X" for "CPUs" and "MEMs" fields denotes whether the CPUs and MEMs in the cpusets are marked exclusive to those cpusets. Note that the one cpuset has subsets as indicated by a 1 in the Subs field. You can specify a cpuset to list with the set subcommand as follows.

[zuul:cpuset-trunk]# cset set -l -s one
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
          one          3 n       0 n     0    1 /one
          two          3 n       0 n     0    1 /one/two

This output shows that there is a cpuset called two in cpuset one and it also has subset. You can also ask for a recursive listing as follows.

[zuul:cpuset-trunk]# cset set -l -r
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y   320    1 /
          one          3 n       0 n     0    1 /one
          two          3 n       0 n     0    1 /one/two
        three          3 n       0 n     0    0 /one/two/three

This command lists all cpusets existing on the system since it asks for a recursive listing beginning at the root cpuset. Incidentally, should you need to specify the root cpuset you can use either root or / to specify it explicitely--just remember that the root cpuset cannot be deleted or modified.

[edit] The Proc Subcommand

Now that we know how to create, rename and destroy cpusets with the set subcommand, the next step is to manage threads and processes in those cpusets. The subcommand to do this is called proc and it allows you to exec processes into a cpuset, move existing tasks around existing cpusets, and list tasks running in specified cpusets. For the following examples, let us assume a cpuset setup of two sets as follows:

[zuul:cpuset-trunk]# cset set -l
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y   309    2 /
          two          2 n       0 n     3    0 /two
        three          3 n       0 n    10    0 /three

[edit] Listing Tasks with Proc

Operation of the proc subcommand follows the same model as the set subcommand. For example, to list tasks in a cpuset, you need to use the -l/--list option and specify the cpuset by name or, if the name exists multiple times in the cpuset hierarchy, by path. For example:

[zuul:cpuset-trunk]# cset proc -l -s two
cset: "two" cpuset of CPUSPEC(2) with 3 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 root     16141  4300 Soth bash
 root     16171 16141 Soth bash
 root     16703 16171 Roth python ./cset proc -l two

This output shows us that the cpuset called two has CPU2 only attached to it and is running three tasks: two shells and the python command to list it. Note that cpusets are inherited so that if a process is contained in a cpuset, then any children it spawns also run within that set. In this case, the python command to list set two was run from a shell already running in set two. This can be seen by the PPID (parent process ID) of the python command matching the PID of the shell.

Additionally, the "SPPr" field needs explanation. "SPPr" stands for State, Policy and Priority. You can see that the initial two tasks are Stopped and running in timeshare priority, marked as "oth" (for "other"). The last task is marked as running, "R" and also at timeshare priority, "oth." If any of these tasks would have been at real time priority, then the policy would be shown as "f" for FIFO or "r" for round robin, and the priority would be a number from 1 to 99. See below for an example.

[zuul:cpuset-trunk]# cset proc -l -s root | head -7
cset: "root" cpuset of CPUSPEC(0-3) with 309 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 root         1     0 Soth init [5]
 root         2     0 Soth [kthreadd]
 root         3     2 Sf99 [migration/0]
 root         4     2 Sf99 [posix_cpu_timer]

This output shows the first few tasks in the root cpuset. Note that both init and '[kthread] are running at timeshare; however, the [migration/0]' and '[posix_cpu_timer]' kernel threads are running at real time policy of FIFO and priority of 99. Incidentally, this output is from a system running the real time Linux kernel which runs some kernel threads at real time priorities. And finally, note that you can of course use cset as any other Linux tool and include it in pipelines as in the example above.

Taking a peek into the third cpuset called three, we see:

[zuul:cpuset-trunk]# cset proc -l -s three
cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 alext    16165     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16169     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16170     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16237     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16491     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16492     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16493     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    17243     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    17244     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    17265     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...

This output shows that a lot of beagled tasks are running in this cpuset and it also shows an ellipsis (...) at the end of their listings. If you see this ellipsis, that means that the command was too long to fit onto an 80 character screen. To see the entire commandline, use the -v/--verbose flag, as per following.

[zuul:cpuset-trunk]# cset proc -l -s three -v | head -4
cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 alext    16165     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg --autostarted --indexing-delay 300

[edit] Execing Tasks with Proc

To exec a task into a cpuset, the proc subcommand needs to be employed with the -e/--exec option. Lets exec a shell into the cpuset named two' in our set. First we check to see what is running that set:

[zuul:cpuset-trunk]# cset proc -l -s two
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running

[zuul:cpuset-trunk]# cset proc -s two -e bash
cset: --> last message, executed args into cpuset "/two", new pid is: 20955

[zuul:cpuset-trunk]# cset proc -l -s two
cset: "two" cpuset of CPUSPEC(2) with 2 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 root     20955 19253 Soth bash
 root     20981 20955 Roth python ./cset proc -l two

You can see that initially, two had nothing running in it. After the completion of the second command, we list two again and see that there are two tasks running: the shell which we execed and the python cset command that is listing the cpuset. The reason for the second task is that the cpuset property of a running task is inherited by all its children. Since we executed the listing command from the new shell which was bound to cpuset two, the resulting process for the listing is also bound to cpuset two. Let's test that by just running a new shell with no prefixed cset command.

[zuul:cpuset-trunk]# bash

[zuul:cpuset-trunk]# cset proc -l -s two
cset: "two" cpuset of CPUSPEC(2) with 3 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 root     20955 19253 Soth bash
 root     21118 20955 Soth bash
 root     21147 21118 Roth python ./cset proc -l two

Here again we see that the second shell, PID 21118, has a parent PID of 20955 which is the first shell. Both shells as well as the listing command are running in the two cpuset.

NOTE!
cset follows the tradition of separating the tool options from the command to be execed options with a double dash (--). This is not shown in this simple example, but if the command you want to exec also takes options, separate them with the double dash like so: +# cset proc -s myset -e mycommand -- -v The -v will be passed to mycommand+, and not to cset.
Hint:
Execing a shell into a cpuset is a useful way to experiment with running tasks in that cpuset since all children of the shell will also run in the same cpuset.

Finally, if you misspell the command to be execed, the result may be puzzling. For example:

[zuul:cpuset-trunk]# cset proc -s two -e blah-blah
cset: --> last message, executed args into cpuset "/two", new pid is: 21655
cset: ''''''> [Errno 2] No such file or directory

The result is no new process even though a new PID is output. The reason for the message is of course that the cset process forked in preparation for exec, but the command blah-blah was not found in order to exec it.

[edit] Moving Tasks with Proc

Although the ability to exec a task into a cpuset is fundamental, you will most likely be moving tasks between cpusets more often. Moving tasks is accomplished with the -m/--move and -p/--pid options to the proc subcommand of cset. The move option tells the proc subcommand that a task move is requested. The -p/--pid option takes an argument called a PIDSPEC (PID Specification). The PIDSPEC defines which tasks get operated on.

The PIDSPEC can be a single process ID, a list of process IDs separated by commas, and a list of process ID ranges also separated by commas. For example:

--move --pid 1234
This PIDSPEC argument specifies that task 1234 be moved.
--move --pid 1234,42,1934,15000,15001,15002
This PIDSPEC argument specifies that this list of tasks only be moved.
--move --pid 5000,5100,6010-7000,9232
This PIDSPEC argument specifies that tasks 5000,5100 and 9232 be moved along with any existing task that is in the range 6010 through 7000 inclusive.
NOTE!
A range in a PIDSPEC does not have to have running tasks for every number in that range. In fact, it is not even an error if there are no tasks running in that range; none will be moved in that case. The range simply specifies to act on any tasks that have a PID or TID that is within that range.

In the following example, we move the current shell into the cpuset named two with a range PIDSPEC and back out to the root cpuset with the bash variable for the current PID.

[zuul:cpuset-trunk]# cset proc -l -s two
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running

[zuul:cpuset-trunk]# echo $$
19253

[zuul:cpuset-trunk]# cset proc -m -p 19250-19260 -t two
cset: moving following pidspec: 19253
cset: moving 1 userspace tasks to /two
cset: done

[zuul:cpuset-trunk]# cset proc -l -s two
cset: "two" cpuset of CPUSPEC(2) with 2 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 root     19253 16447 Roth bash
 root     29456 19253 Roth python ./cset proc -l -s two

[zuul:cpuset-trunk]# cset proc -m -p $$ -t root
cset: moving following pidspec: 19253
cset: moving 1 userspace tasks to /
cset: done

[zuul:cpuset-trunk]# cset proc -l -s two
cset: "two" cpuset of CPUSPEC(2) with 0 tasks running

Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of tasks. Additionally, there is one more option that can help with multi-threaded processes, and that is the --threads flag. If this flag is present in a proc move command with a PIDSPEC and if any of the task IDs in the PIDSPEC belongs to a thread in a process container, then all the sibling threads in that process container will also get moved. This flag provides an easy mechanism to move all threads of a process by simply specifying one thread in that process. In the following example, we move all the threads running in cpuset three to cpuset two by using the --threads flag.

[zuul:cpuset-trunk]# cset set two three
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
          two          2 n       0 n     0    0 /two
        three          3 n       0 n    10    0 /three

[zuul:cpuset-trunk]# cset proc -l -s three
cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 alext    16165     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16169     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16170     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16237     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16491     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16492     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16493     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    17243     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    17244     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    27133     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...

[zuul:cpuset-trunk]# cset proc -m -p 16165 --threads -t two
cset: moving following pidspec: 16491,16493,16492,16170,16165,16169,27133,17244,17243,16237
cset: moving 10 userspace tasks to /two
[==================================================]%
cset: done

[zuul:cpuset-trunk]# cset set two three
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
          two          2 n       0 n    10    0 /two
        three          3 n       0 n     0    0 /three

[edit] Moving All Tasks from one Cpuset to Another

There is a special case for moving all tasks currently running in one cpuset to another. This can be a common use case, and when you need to do it, specifying a PIDSPEC with -p is not necessary so long as you use the -f/--fromset and the -t/--toset options.

In the following example, we move all 10 beagled threads back to cpuset three with this method.

[zuul:cpuset-trunk]# cset proc -l two three
cset: "two" cpuset of CPUSPEC(2) with 10 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 alext    16165     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16169     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16170     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16237     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16491     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16492     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    16493     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    17243     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    17244     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
 alext    27133     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
cset: "three" cpuset of CPUSPEC(3) with 0 tasks running

[zuul:cpuset-trunk]# cset proc -m -f two -t three
cset: moving all tasks from two to /three
cset: moving 10 userspace tasks to /three
[==================================================]%
cset: done

[zuul:cpuset-trunk]# cset set two three
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
          two          2 n       0 n     0    0 /two
        three          3 n       0 n    10    0 /three

[edit] Moving Kernel Threads with Proc

Kernel threads are special and cset detects tasks that are kernel threads and will refuse to move them unless you also add a -k/--kthread option to your proc move command. Even if you include -k, cset will still refuse to move the kernel thread if they are bound to specific CPUs. The reason for this is system protection.

A number of kernel threads, especially on the real time Linux kernel, are bound to specific CPUs and depend on per-CPU kernel variables. If you move these threads to a different CPU than what they are bound to, you risk at best that the system will become horribly slow, and at worst a system hang. If you still insist to move those threads (after all, cset needs to give the knowledgeable user access to the keys), then you need to use the --force option additionally.

ATTENTION!
Overriding a task move command with --force can have dire consequences for the system. Please be sure of the command before you force it.

In the following example, we move all unbound kernel threads running in the root cpuset to the cpuset named two by using the -k option.

[zuul:cpuset-trunk]# cset proc -k -f root -t two
cset: moving all kernel threads from / to /two
cset: moving 70 kernel threads to: /two
cset: --> not moving 76 threads (not unbound, use --force)
[==================================================]%
cset: done

You will note that we used the fromset->toset facility of the proc subcommand and we only specified the -k option (not the -m option). This has the effect of moving all kernel threads only.

Note that only 70 actual kernel threads were moved and 76 were not. The reason that 76 kernel threads were not moved was because they are bound to specific CPUs. Now, lets move those kernel threads back to root'.

[zuul:cpuset-trunk]# cset proc -k -f two -t root
cset: moving all kernel threads from /two to /
cset: **> no task matched move criteria
cset: **> kernel tasks are bound, use --force if ok

[zuul:cpuset-trunk]# cset set -l -s two
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
          two          2 n       0 n    70    0 /two

Ah! What's this? Cset refused to move the kernel threads back to root because it says that they are "bound." Let's check this with the Linux taskset command.

[zuul:cpuset-trunk]# cset proc -l -s two | head -5
cset: "two" cpuset of CPUSPEC(2) with 70 tasks running
 USER       PID  PPID SPPr TASK NAME
 -------- ----- ----- ---- ---------
 root         2     0 Soth [kthreadd]
 root        55     2 Soth [khelper]

[zuul:cpuset-trunk]# taskset -p 2
pid 2's current affinity mask: 4

[zuul:cpuset-trunk]# cset set -l -s two
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
          two          2 n       0 n    70    0 /two

Of course, since the cpuset named two only has CPU2 assigned to it, once we moved the unbound kernel threads from root to two, their affinity masks got automatically changed to only use CPU2. This is evident from the taskset output which is a hex value. To really move these threads back to root, we need to force the move as follows.

[zuul:cpuset-trunk]# cset proc -k -f two -t root --force
cset: moving all kernel threads from /two to /
cset: moving 70 kernel threads to: /
[==================================================]%
cset: done

[edit] Destroying Tasks

There actually is no cset subcommand or option to destroy tasks--it's not really needed. Tasks exist and are accessible on the system as normal, even if they happen to be running in one cpuset or another. To destroy tasks, use the usual ^C method or by using the *kill(1)* command.

[edit] Implementing "Shielding" with Set and Proc

With the preceding material on the set and proc subcommands, we now have the background to implement the basic shielding model, just like the shield subcommand.

One may pose the question as to why we want to do this, especially since shield already does it? The answer is that sometimes one needs more functionality than shield has to implement one's shielding strategy. In those case you need to first stop using shield since that subcommand will interfere with the further application of set and proc; however, you will still need to implement the functionality of shield in order to implement successful shielding.

Remember from the above sections describing shield, that shielding has at minimum three cpusets: root, which is always present and contains all CPUs; system which is the "non-shielded" set of CPUs and runs unimportant system tasks; and user, which is the "shielded" set of CPUs and runs your important tasks. Remember also that shield moves all movable tasks into system and, optionally, moves unbound kernel threads into system as well.

We start first by creating the system and user cpusets as follows. We assume that the machine is a four-CPU machine without NUMA memory features. The system cpuset should hold only CPU0 while the user cpuset should hold the rest of the CPUs.

[zuul:cpuset-trunk]# cset set -c 0 -s system
cset: --> created cpuset "system"

[zuul:cpuset-trunk]# cset set -c 1-3 -s user
cset: --> created cpuset "user"

[zuul:cpuset-trunk]# cset set -l
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y   333    2 /
         user        1-3 n       0 n     0    0 /user
       system          0 n       0 n     0    0 /system

Now, we need to move all running user processes into the system cpuset.

[zuul:cpuset-trunk]# cset proc -m -f root -t system
cset: moving all tasks from root to /system
cset: moving 188 userspace tasks to /system
[==================================================]%
cset: done

[zuul:cpuset-trunk]# cset set -l
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y   146    2 /
         user        1-3 n       0 n     0    0 /user
       system          0 n       0 n   187    0 /system

We now have the basic shielding set up. Since all userspace tasks are running in system, anything that is spawned from them will also run in system. The user cpuset has nothing running in it unless you put tasks there with the proc subcommand as described above. If you also want to move movable kernel threads from root to system (in order to achieve a form of "interrupt shielding" on a real time Linux kernel for example), you would execute the following command as well.

[zuul:cpuset-trunk]# cset proc -k -f root -t system
cset: moving all kernel threads from / to /system
cset: moving 70 kernel threads to: /system
cset: --> not moving 76 threads (not unbound, use --force)
[==================================================]%
cset: done

[zuul:cpuset-trunk]# cset set -l
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y    76    2 /
         user        1-3 n       0 n     0    0 /user
       system          0 n       0 n   257    0 /system

At this point, you have achieved the simple shielding model that the shield subcommand provides. You can now add other cpuset definitions to expand your shielding strategy beyond that simple model.

[edit] Implementing Hierarchy with Set and Proc

One popular extended "shielding" model is based on hierarchical cpusets, each with diminishing numbers of CPUs. This model is used to create "priority cpusets" that allow assignment of CPU resources to tasks based on some arbitrary priority definition. The idea being that a higher priority task will get access to more CPU resources than a lower priority task.

The example provided here once again assumes a machine with four CPUs and no NUMA memory features. This base serves to illustrate the point well; however, note that if your machine has (many) more CPUs, then strategies such as this and others get more interesting.

We define a shielding set up as in the previous section where we have a system cpuset with just CPU0 that takes care of "unimportant" system tasks. One usually requires this type of cpuset since it forms the basis of shielding. We modify the strategy to not use a user cpuset, instead we create a number of new cpusets each holding one more CPU than the other. These cpusets will be called 'prio_low with one CPU, prio_med' with two CPUs, 'prio_high with three CPUs, and prio_all' with all CPUs.

NOTE!
One may ask, why create a 'prio_all' with all CPUs when that is substantially the definition of the root cpuset? The answer is that it is best to keep a separation between the root cpuset and everything else, even if a particular cpuset duplicates root exactly. Usually, one builds automation on top of a cpuset strategy. In these cases, it is best to avoid using invariant names of cpusets, such as root for example, in this automation.

All of these 'prio_*' cpusets can be created under root, in a flat way; however, it is advantageous to create them as a hierarchy. The reasoning for this is twofold: first, if a cpuset is destroyed, all its tasks are moved to its parent; second, one can use exclusive CPUs in a hierarchy.

There is a planned addition to the proc subcommand that will allow moving a specified PIDSPEC of tasks running in a specified cpuset to its parent. This addition will ease the automation burden.

If a cpuset has CPUs that are exclusive to it, then other cpusets may not make use of those CPUs unless they are children of that cpuset. This has more relevance to machines with many CPUs and more complex strategies.

Now, we start with a clean slate and build the appropriate cpusets as follows.

[zuul:cpuset-trunk]# cset set -r
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y   344    0 /

[zuul:cpuset-trunk]# cset set -c 0-3 prio_all
cset: --> created cpuset "prio_all"

[zuul:cpuset-trunk]# cset set -c 1-3 /prio_all/prio_high
cset: --> created cpuset "/prio_all/prio_high"

[zuul:cpuset-trunk]# cset set -c 2-3 /prio_all/prio_high/prio_med
cset: --> created cpuset "/prio_all/prio_high/prio_med"

[zuul:cpuset-trunk]# cset set -c 3 /prio_all/prio_high/prio_med/prio_low
cset: --> created cpuset "/prio_all/prio_high/prio_med/prio_low"

[zuul:cpuset-trunk]# cset set -c 0 system
cset: --> created cpuset "system"

[zuul:cpuset-trunk]# cset set -l -r
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y   344    2 /
       system          0 n       0 n     0    0 /system
     prio_all        0-3 n       0 n     0    1 /prio_all
    prio_high        1-3 n       0 n     0    1 /prio_all/prio_high
     prio_med        2-3 n       0 n     0    1 /prio_all/prio_high/prio_med
     prio_low          3 n       0 n     0    0 /prio_all/pr...rio_med/prio_low

NOTE: We used the -r/--recurse switch to list all the sets in the last command above. If we had not, then the 'prio_med and prio_low' cpusets would not have been listed.

The strategy is now implemented and we now move all userspace tasks as well as all movable kernel threads into the system cpuset to activate the shield.

[zuul:cpuset-trunk]# cset proc -m -k -f root -t system
cset: moving all tasks from root to /system
cset: moving 198 userspace tasks to /system
cset: moving 70 kernel threads to: /system
cset: --> not moving 76 threads (not unbound, use --force)
[==================================================]%
cset: done

[zuul:cpuset-trunk]# cset set -l -r
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y    76    2 /
       system          0 n       0 n   268    0 /system
     prio_all        0-3 n       0 n     0    1 /prio_all
    prio_high        1-3 n       0 n     0    1 /prio_all/prio_high
     prio_med        2-3 n       0 n     0    1 /prio_all/prio_high/prio_med
     prio_low          3 n       0 n     0    0 /prio_all/pr...rio_med/prio_low

The shield is now active. Since the 'prio_*' cpuset names are unique, one can assign tasks to them either via either their simple name, or their full path (as described in the proc section above).

You may have noted that there is an ellipsis in the path of the 'prio_low' cpuset in the listing above. This is done in order to fit the output onto an 80 character screen. If you want to see the entire line, then you need to use the -v/--verbose flag as follows.

[zuul:cpuset-trunk]# cset set -l -r -v
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
 ------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y    76    2 /
       system          0 n       0 n   268    0 /system
     prio_all        0-3 n       0 n     0    1 /prio_all
    prio_high        1-3 n       0 n     0    1 /prio_all/prio_high
     prio_med        2-3 n       0 n     0    1 /prio_all/prio_high/prio_med
     prio_low          3 n       0 n     0    0 /prio_all/prio_high/prio_med/prio_low

[edit] Using Shortcuts

The commands listed in the previous sections always used all the required options. Cset however does have a shortcut facility that will execute certain commands without specifying all options. An effort has been made to do this with the "principle of least surprise." This means that if you do not specify options, but do specify parameters, then the outcome of the command should be intuitive. As much as possible.

Using shortcuts is of course not necessary. In fact, you can not only not use shortcuts, but you can use long options instead of short, in case you really enjoy typing... All kidding aside, using long options and not using shortcuts does have a use case: when you write a script intended to be self-documenting, or perhaps when you generate cset documentation.

To begin, the subcommands shield, set and proc can themselves be shortened to the fewest number of characters that are unambiguous. For example, the following commands are identical:

# cset shield -s -p 1234         <-->    # cset sh -s -p 1234
# cset set -c 1,3 -s newset      <-->    # cset se -c 1,3 -s newset
# cset proc -s newset -e bash    <-->    # cset p -s newset -e bash

Note that proc can be shortened to just p, while shield and set need two letters to disambiguate.

[edit] Shield Subcommand Shortcuts

The shield subcommand supports two areas with shortcuts: the case when there is no options given where to shield is the common use case, and making the -p/--pid option optional for the -s/--shield and -u/--unshield options.

For the common use case of actually shielding either a PIDSPEC or execing a command into the shield, the following cset commands are equivalent.

# cset shield -s -p 1234,500-649   <-->    # cset sh 1234,500-649
# cset shield -s -e bash           <-->    # cset sh bash

When using the -s or -u shield/unshield options, it is optional to use the -p option to specify a PIDSPEC. For example:

# cset shield -s -p 1234     <-->    # cset sh -s 1234
# cset shield -u -p 1234     <-->    # cset sh -u 1234

[edit] Set Subcommand Shortcuts

The set subcommand has a limited number of shortcuts. Basically, the -s/--set option is optional in most cases and the -l/--list option is also optional if you want to list sets. For example, these commands are equivalent.

# cset set -l -s myset           <-->  # cset se -l myset
# cset se -l myset               <-->  # cset se myset

# cset set -c 1,2,3 -s newset    <-->  # cset se -c 1,2,3 newset
# cset set -d -s newset          <-->  # cset se -d newset

# cset set -n newname -s oldname <-->  # cset se -n newname oldname

In fact, if you want to apply either the list or the destroy options to multiple cpusets with one cset command, you'll need to not use the -s option. For example:

# cset se -d myset yourset ourset
   --> destroys cpusets: myset, yourset and ourset

# cset se -l prio_high prio_med prio_low
   --> lists only cpusets prio_high, prio_med and prio_low
   --> the -l is optional in this case since list is default

[edit] Proc Subcommand Shortcuts

For the proc subcommand, the -s, -t and -f options to specify the cpuset, the origination cpuset and the destination cpuset, can sometimes be optional. For example, the following commands are equivalent.

To list tasks in cpusets:
# cset proc -l -s myset        \
# cset proc -l -f myset         -->  # cset p -l myset
# cset proc -l -t myset        /

# cset p -l myset              <-->  # cset p myset

# cset proc -l -s one two      <-->  # cset p -l one two
# cset p -l one two            <--> # cset p one two

To exec a process into a cpuset:
# cset proc -s myset -e bash   <-->  # cset p myset -e bash

Movement of tasks into and out of cpusets have the following shortcuts.

To move a PIDSPEC into a cpuset:
# cset proc -m -p 4242,4243 -s myset <--> # cset p -m 4242,4243 myset
# cset proc -m -p 12 -t myset        <--> # cset p -m 12 myset

To move all tasks from one cpuset to another:
# cset proc -m -f set1 -t set2      \
# cset proc -m -s set1 -t set2       --> # cset p -m set1 set2
# cset proc -m -f set1 -s set2      /

[edit] What To Do if There are Problems

If you encounter problems with the cset application, the best option is to log a bug with the cset issues instance found here:

  [http://code.google.com/p/cpuset/issues/list]

If you are using cset on a supported operating system such as SLES or SLERT from Novell, then feel free to use that bugzilla instead.

If the problem is repeatable, there is an excellent chance that it will get fixed quickly. Also, cset contains a logging facility that is invaluable for the developers to diagnose problems. To create a log of a run, use the -l/--log option with a filename as an argument to the main cset application. For example.

  # cset -l logfile.txt set -n newname oldname

That command saves a lot of debugging information in the logfile.txt file. Please attach this file to the bug.


Back up to cpuset page.

Personal tools