4 ms·

Tangent, but the 2nd point is intriguing. How does one use cgroups to set up resource limitations? Is there any kind of decent front-end? I've seen the kernel d

Tangent, but the 2nd point is intriguing. How does one use cgroups to set up resource limitations? Is there any kind of decent front-end? I've seen the kernel documentation (https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt), but how do I use it? For example, if I wanted to give some shell users limited accounts where they can't use more than 512 RAM and some CPU quota each, I gather cgroups can do this, but I haven't been able to figure out how to set something like that up. (Yes, I know you could handle that use-case by giving each user their own virtual machine and let VMWare or Xen or Virtualbox handle the RAM/CPU quotas, but that's often not what I want.)

WestCoastJustin 13y ago
You're in luck -- I've done a screencast on cgroups where I explain what they are and how you can use them @ http://sysadmincasts.com/episodes/14-introduction-to-linux-control-groups-cgroups http://sysadmincasts.com/episodes/14-introduction-to-linux-c... Also, it is all command line at this point, but there are some concepts floating around about creating a gui @ http://mairin.wordpress.com/2011/05/13/ideas-for-a-cgroups-ui/ http://mairin.wordpress.com/2011/05/13/ideas-for-a-cgroups-u...
- sandGorgon 13y ago
  could you double check your screencast ? My understanding is limited, but I think Systemd ver 205 changes the cgroup interaction methodology - basically systemd is a single point entry into the cgroup api. You cannot touch cgroup any other way except through systemd. FYI - this is not systemd taking arbitrary control but what the upstream kernel recommended. My understanding is that Fedora 20 (which uses systemd 208 ) will be impacted by this, as will most other distros. However jessie is proposing systemd 204, so your method might still work.
  - nisa 13y ago
    Is there a reason for this? I'm heavily invested in Ubuntu and I'm also making heavy use of the cgroups API in its current form. Does that mean that I can't use cgroups without systemd in newer kernels? Or do I miss cgroups features? I'm quite happy with the mountable filesystem at the moment. It's scriptable and reasonable flexible and still easy to use for solving my problems.
    - _delirium 13y ago
      Found a discussion here of the changes & rationale here: http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/ http://www.freedesktop.org/wiki/Software/systemd/ControlGrou...
      - vidarh 13y ago
        That sounds awful. The idea that systemd vs. another "cgroups owner" changes the API you need to use for interacting with cgroups will effectively impose a compatibility burden for every app that wants to take advantage of it.
  - WestCoastJustin 13y ago
    For the screencast I was using CentOS 6.4 which is not systemd enabled.
eliasmacpherson 13y ago
change their shell to a bash script that calls bash and has a call to ulimit in it(this will affect everybody though I think). there's another program called cpulimit that you could use. here's some cgroups resource limiting "resources": http://dustinoprea.com/2013/11/05/better-resource-throttling-for-processes/ http://dustinoprea.com/2013/11/05/better-resource-throttling... http://www.janoszen.com/2013/02/06/limiting-linux-processes-cgroups-explained/ http://www.janoszen.com/2013/02/06/limiting-linux-processes-...
- _delirium 13y ago
  ulimit only assigns per-process limits though, not per-user. Afaict you need cgroups to limit users' total usage across processes.
icebraining 13y ago
This answer explains how to create a cgroup witht he quota and then assigning the user to it: http://unix.stackexchange.com/a/34335 http://unix.stackexchange.com/a/34335 See also http://manpages.ubuntu.com/manpages/lucid/man5/cgrules.conf.5.html http://manpages.ubuntu.com/manpages/lucid/man5/cgrules.conf....
- _delirium 13y ago
  Excellent, thanks! cgrules/cgconfig seems to be what I was looking for. Fwiw, it looks like those files don't do anything in the default Debian install; you have to install cgroup-bin to pull in the daemons that consult them.
mseepgood 13y ago
> Tangent, but the 2nd point is intriguing. How does one use cgroups to set up resource limitations? Is there any kind of decent front-end? I've seen the kernel documentation, but how do I use it? Use systemd: http://0pointer.de/blog/projects/resources.html http://0pointer.de/blog/projects/resources.html Systemd makes these kernel features finally usable.
- _delirium 13y ago
  That looks interesting for some use-cases, but I don't see a way to use it to assign limits to users, or to processes that aren't started as systemd services. What I have in mind is that I give someone ssh access, they run R or matlab or from the command line, but limited to 512mb (or 1Gb or whatever) of total memory.
  - WestCoastJustin 13y ago
    On RHEL systems you can use the 'cgred' service to classify processes based on the user running it or the command itself. For example, say you have user X who always logs in and runs rsync commands and saturates the link. You can automatically classify this user or rsync process into a cgroup that throttles the network IP (or in your use case, memory) -- problem solved. There is some great documentation @ https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Resource_Management_Guide/index.html#The_cgred_Service https://access.redhat.com/site/documentation/en-US/Red_Hat_E...
    - viraptor 13y ago
      It's not RHEL-specific. Ubuntu and pretty much everything that includes cgroups tools has it.
  - shykes 13y ago
    You could build a matlab docker container and limit memory with 'docker run -m 512M delirium/matlab' :) See http://docker.io http://docker.io
  - 13y ago
sandGorgon 13y ago
I have been investigating this over the past couple of days for a pet project of mine (a local college needs to setup a programming competition - need to resource limit compile jobs). The way that systemd handles it is three-fold: One, in the .service file, you can put in resource constraints [1] - which will be honored by a service on startup. The second way is to create a .slice file which is exactly that - a slice in the cgroup hierarchy and where you can assign some resource limits (where it is bound by the slices above it in hierarchy). All services that are assigned to this slice will SHARE that cgroup resource constraints. For your specific requirement, this is the way to go. Create a user.slice (to affect all user accounts) or a user-<some uid>.slice to affect on an individual basis (still will inherit from user.slice). The third way is to invoke systemctl set-property (with properties defined in [1]) to affect an already running process. What I am attempting to use is systemd-run [3] to start transient processes (not background services) with resource constraints. However, I have filed an enhancement request on some missing features here [4] . Feel free to star it up ! [1] http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html http://www.freedesktop.org/software/systemd/man/systemd.reso... [2] http://www.freedesktop.org/software/systemd/man/systemd.slice http://www.freedesktop.org/software/systemd/man/systemd.slic... [3] http://www.freedesktop.org/software/systemd/man/systemd-run.html http://www.freedesktop.org/software/systemd/man/systemd-run.... [4] https://bugs.freedesktop.org/show_bug.cgi?id=73685 https://bugs.freedesktop.org/show_bug.cgi?id=73685
- sergiosgc 13y ago
  Have a look at online judges created by various universities for the ACM intercollegiate programming contest (ACM ICPC). Back in the day I used the one from the university of Valladolid, but I can't seem to find the source code now.
- mrud 13y ago
  There is runguard [1] from the DOMjudge project. It may be able to do what you are looking for. [1] https://github.com/DOMjudge/domjudge/blob/master/judge/runguard.c https://github.com/DOMjudge/domjudge/blob/master/judge/rungu...
  - 13y ago

WestCoastJustin 13y ago

You're in luck -- I've done a screencast on cgroups where I explain what they are and how you can use them @ http://sysadmincasts.com/episodes/14-introduction-to-linux-control-groups-cgroups http://sysadmincasts.com/episodes/14-introduction-to-linux-c... Also, it is all command line at this point, but there are some concepts floating around about creating a gui @ http://mairin.wordpress.com/2011/05/13/ideas-for-a-cgroups-ui/ http://mairin.wordpress.com/2011/05/13/ideas-for-a-cgroups-u...

sandGorgon 13y ago
could you double check your screencast ? My understanding is limited, but I think Systemd ver 205 changes the cgroup interaction methodology - basically systemd is a single point entry into the cgroup api. You cannot touch cgroup any other way except through systemd. FYI - this is not systemd taking arbitrary control but what the upstream kernel recommended. My understanding is that Fedora 20 (which uses systemd 208 ) will be impacted by this, as will most other distros. However jessie is proposing systemd 204, so your method might still work.
- nisa 13y ago
  Is there a reason for this? I'm heavily invested in Ubuntu and I'm also making heavy use of the cgroups API in its current form. Does that mean that I can't use cgroups without systemd in newer kernels? Or do I miss cgroups features? I'm quite happy with the mountable filesystem at the moment. It's scriptable and reasonable flexible and still easy to use for solving my problems.
  - _delirium 13y ago
    Found a discussion here of the changes & rationale here: http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/ http://www.freedesktop.org/wiki/Software/systemd/ControlGrou...
    - vidarh 13y ago
      That sounds awful. The idea that systemd vs. another "cgroups owner" changes the API you need to use for interacting with cgroups will effectively impose a compatibility burden for every app that wants to take advantage of it.
- WestCoastJustin 13y ago
  For the screencast I was using CentOS 6.4 which is not systemd enabled.

eliasmacpherson 13y ago

change their shell to a bash script that calls bash and has a call to ulimit in it(this will affect everybody though I think). there's another program called cpulimit that you could use. here's some cgroups resource limiting "resources": http://dustinoprea.com/2013/11/05/better-resource-throttling-for-processes/ http://dustinoprea.com/2013/11/05/better-resource-throttling... http://www.janoszen.com/2013/02/06/limiting-linux-processes-cgroups-explained/ http://www.janoszen.com/2013/02/06/limiting-linux-processes-...

_delirium 13y ago
ulimit only assigns per-process limits though, not per-user. Afaict you need cgroups to limit users' total usage across processes.

icebraining 13y ago

This answer explains how to create a cgroup witht he quota and then assigning the user to it: http://unix.stackexchange.com/a/34335 http://unix.stackexchange.com/a/34335 See also http://manpages.ubuntu.com/manpages/lucid/man5/cgrules.conf.5.html http://manpages.ubuntu.com/manpages/lucid/man5/cgrules.conf....

_delirium 13y ago
Excellent, thanks! cgrules/cgconfig seems to be what I was looking for. Fwiw, it looks like those files don't do anything in the default Debian install; you have to install cgroup-bin to pull in the daemons that consult them.

mseepgood 13y ago

> Tangent, but the 2nd point is intriguing. How does one use cgroups to set up resource limitations? Is there any kind of decent front-end? I've seen the kernel documentation, but how do I use it? Use systemd: http://0pointer.de/blog/projects/resources.html http://0pointer.de/blog/projects/resources.html Systemd makes these kernel features finally usable.

_delirium 13y ago
That looks interesting for some use-cases, but I don't see a way to use it to assign limits to users, or to processes that aren't started as systemd services. What I have in mind is that I give someone ssh access, they run R or matlab or from the command line, but limited to 512mb (or 1Gb or whatever) of total memory.
- WestCoastJustin 13y ago
  On RHEL systems you can use the 'cgred' service to classify processes based on the user running it or the command itself. For example, say you have user X who always logs in and runs rsync commands and saturates the link. You can automatically classify this user or rsync process into a cgroup that throttles the network IP (or in your use case, memory) -- problem solved. There is some great documentation @ https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Resource_Management_Guide/index.html#The_cgred_Service https://access.redhat.com/site/documentation/en-US/Red_Hat_E...
  - viraptor 13y ago
    It's not RHEL-specific. Ubuntu and pretty much everything that includes cgroups tools has it.
- shykes 13y ago
  You could build a matlab docker container and limit memory with 'docker run -m 512M delirium/matlab' :) See http://docker.io http://docker.io
- 13y ago

sandGorgon 13y ago

I have been investigating this over the past couple of days for a pet project of mine (a local college needs to setup a programming competition - need to resource limit compile jobs). The way that systemd handles it is three-fold: One, in the .service file, you can put in resource constraints [1] - which will be honored by a service on startup. The second way is to create a .slice file which is exactly that - a slice in the cgroup hierarchy and where you can assign some resource limits (where it is bound by the slices above it in hierarchy). All services that are assigned to this slice will SHARE that cgroup resource constraints. For your specific requirement, this is the way to go. Create a user.slice (to affect all user accounts) or a user-<some uid>.slice to affect on an individual basis (still will inherit from user.slice). The third way is to invoke systemctl set-property (with properties defined in [1]) to affect an already running process. What I am attempting to use is systemd-run [3] to start transient processes (not background services) with resource constraints. However, I have filed an enhancement request on some missing features here [4] . Feel free to star it up ! [1] http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html http://www.freedesktop.org/software/systemd/man/systemd.reso... [2] http://www.freedesktop.org/software/systemd/man/systemd.slice http://www.freedesktop.org/software/systemd/man/systemd.slic... [3] http://www.freedesktop.org/software/systemd/man/systemd-run.html http://www.freedesktop.org/software/systemd/man/systemd-run.... [4] https://bugs.freedesktop.org/show_bug.cgi?id=73685 https://bugs.freedesktop.org/show_bug.cgi?id=73685

sergiosgc 13y ago
Have a look at online judges created by various universities for the ACM intercollegiate programming contest (ACM ICPC). Back in the day I used the one from the university of Valladolid, but I can't seem to find the source code now.
mrud 13y ago
There is runguard [1] from the DOMjudge project. It may be able to do what you are looking for. [1] https://github.com/DOMjudge/domjudge/blob/master/judge/runguard.c https://github.com/DOMjudge/domjudge/blob/master/judge/rungu...
- 13y ago