虽然 CPU 属于可压缩资源，但是任然要避免资源耗尽，如果一个应用程序占用了过多的CPU时间，它可能会导致系统资源耗尽，例如CPU使用率过高、内存不足等。这可能会导致系统变得不稳定、响应缓慢或完全崩溃。通过限制CPU 带宽，可以降低这种风险。
今天和小伙伴分享 通过 cgroups-v2 来实现应用程序 CPU 时间分布的配置，从而控制应用程序的 CPU 消耗。这里的配置主要有三种方法：
设置CPU 亲和性：限定控制组的进程或者服务仅使用分配的CPU
设置CPU带宽：限定控制组的进程每秒可以在CPU上运行多少秒
设置CPU权重：限定控制组的进程在 CPU上多个应用程序使用CPU占比
实验环境
1 2 3 4 5 6 7 8 9 10 11 12 13 14
┌──[root@vms99.liruilongs.github.io]-[~] └─$hostnamectl Static hostname: vms99.liruilongs.github.io Icon name: computer-vm Chassis: vm Machine ID: ea70bf6266cb413c84266d4153276342 Boot ID: 2333fe6d930148d59327b9fd9ce48c37 Virtualization: vmware Operating System: Rocky Linux 8.9 (Green Obsidian) CPE OS Name: cpe:/o:rocky:rocky:8:GA Kernel: Linux 4.18.0-513.9.1.el8_9.x86_64 Architecture: x86-64 ┌──[root@vms99.liruilongs.github.io]-[~] └─$
cgroups-v2 配置
cgroups-v2 设置CPU 亲和性Demo
确认 Cgroup 版本
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
┌──[root@vms99.liruilongs.github.io]-[/sys/fs/cgroup] └─$mount -l | grep cgroup tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,freezer) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpuset) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,pids) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpu,cpuacct) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,hugetlb) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,perf_event) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,devices) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,net_cls,net_prio) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,memory) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,blkio) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,rdma)
┌──[root@vms99.liruilongs.github.io]-[/sys/fs/cgroup] └─$reboot Connection to 192.168.26.99 closed by remote host. Connection to 192.168.26.99 closed. PS C:\Users\Administrator> ssh root@192.168.26.99 root@192.168.26.99's password: Web console: https://vms99.liruilongs.github.io:9090/ or https://192.168.26.99:9090/ Last login: Sat May 11 03:13:30 2024 from 192.168.26.1 ┌──[root@vms99.liruilongs.github.io]-[~] └─$mount -l | grep cgroup cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate) ┌──[root@vms99.liruilongs.github.io]-[~] └─$
┌──[root@vms99.liruilongs.github.io]-[~] └─$cat /sys/fs/cgroup/cgroup.subtree_control /sys/fs/cgroup/Example/cgroup.subtree_control cpuset cpu memory pids cpuset cpu
┌──[root@vms99.liruilongs.github.io]-[~] └─$ab -n 100000 -c 10000 127.0.0.1:80/ This is ApacheBench, Version 2.3 <$Revision: 1843412 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/
┌──[root@vms99.liruilongs.github.io]-[~] └─$ab -n 100000 -c 10000 127.0.0.1:80/ This is ApacheBench, Version 2.3 <$Revision: 1843412 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient) Completed 10000 requests .................. Completed 90000 requests apr_socket_recv: Connection reset by peer (104) Total of 98379 requests completed
┌──[root@vms99.liruilongs.github.io]-[~] └─$echo max max | sudo tee /sys/fs/cgroup/Example/tasks/cpu.max max max ┌──[root@vms99.liruilongs.github.io]-[~] └─$cat /sys/fs/cgroup/Example/tasks/cpu.max max 1000000 ┌──[root@vms99.liruilongs.github.io]-[~] └─$
cgroups-v2 CPU 权重 测试
这里为了Demo 演示，强制所有示例进程运行在单个 CPU 上。当在多个 CPU 上使用时，CPU 权重也会应用同样的原则。
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 6409 root 20 0 362016 27736 5020 S 27.3 0.4 0:04.96 ab 6408 root 20 0 362016 26376 5184 R 11.3 0.4 0:02.79 ab 5588 apache 20 0 2500284 19988 7172 S 5.0 0.3 0:03.69 httpd 1484 apache 20 0 2565820 22380 7172 S 4.7 0.4 0:06.52 httpd 1487 apache 20 0 2500284 19504 7172 S 4.7 0.3 0:06.54 httpd 5047 apache 20 0 2565820 24736 7196 S 4.7 0.4 0:04.81 httpd 5584 apache 20 0 2500284 21788 7260 S 4.7 0.3 0:03.73 httpd 5585 apache 20 0 2500284 20644 7196 S 4.7 0.3 0:03.72 httpd 1483 apache 20 0 2696956 28484 7172 S 4.3 0.5 0:06.64 httpd 5122 apache 20 0 2500284 21264 7196 S 4.3 0.3 0:04.62 httpd 5188 apache 20 0 2500284 21952 7260 S 4.3 0.3 0:04.22 httpd 5189 apache 20 0 2500284 22248 7196 S 4.3 0.4 0:04.20 httpd 5320 apache 20 0 2500284 22376 7196 S 4.3 0.4 0:04.00 httpd 5321 apache 20 0 2500284 22832 7260 S 4.3 0.4 0:03.94 httpd 5322 apache 20 0 2565820 20444 7196 S 4.3 0.3 0:03.81 httpd 5323 apache 20 0 2500284 19432 7196 S 4.3 0.3 0:03.85 httpd 5586 apache 20 0 2500284 19348 7196 S 4.3 0.3 0:03.62 httpd 5587 apache 20 0 2500284 19908 7236 S 4.3 0.3 0:03.68 httpd 5002 nginx 20 0 124896 9220 6424 R 1.7 0.1 5:26.73 nginx 5005 nginx 20 0 124896 9220 6424 R 1.7 0.1 0:51.50 nginx 5007 nginx 20 0 124896 9220 6424 R 1.7 0.1 1:07.06 nginx 5003 nginx 20 0 124896 9220 6424 R 1.3 0.1 0:55.37 nginx 5004 nginx 20 0 124896 9220 6424 R 1.3 0.1 0:52.33 nginx 5006 nginx 20 0 124896 9220 6424 R 1.3 0.1 0:55.32 nginx
Systemd 配置 亲和性，权重，带宽
上面的配置方式直接通过把 服务的所有进程添加到 Cgroup，实际上可以通过 直接通过 服务来限制
在由 systemd 管理的系统上，每个系统服务在其 cgroup 中启动。通过启用对 CPU cgroup 控制器的支持，系统使用对 CPU 资源的服务感知分布，而不是按进程分布。在服务感知型分发中，每个服务收到的 CPU 时间与系统上运行的所有其他服务的 CPU 时间大致相同，无论组成该服务的进程数量如何。
如果特定的服务需要更多 CPU 资源，您可以通过更改该服务的 CPU 时间分配策略来授予它们。
配置流程
1 2 3 4 5 6
# 检查 systemctl show --property <CPU time allocation policy option> <service name> # 配置 systemctl set-property <service name> <CPU time allocation policy option>=<value> # 验证 systemctl show --property <CPU time allocation policy option> <service name>
┌──[root@vms99.liruilongs.github.io]-[~] └─$man systemd.resource-control ................ OPTIONS Units of the types listed above can have settings for resource control configuration:
CPUAccounting= Turn on CPU usage accounting for this unit. Takes a boolean argument. Note that turning on CPU accounting for one unit will also implicitly turn it on for all units contained in the same slice and for all its parent slices and the units contained therein. The system default for this setting may be controlled with DefaultCPUAccounting= in systemd- system.conf(5).
CPUWeight=weight, StartupCPUWeight=weight Assign the specified CPU time weight to the processes executed, if the unified control group hierarchy is used on the system. These options take an integer value and control the "cpu.weight" control group attribute. The allowed range is 1 to 10000. Defaults to 100. For details about this control group attribute, see cgroup-v2.txt[2] and sched-design-CFS.txt[4]. The available CPU time is split up among all units within one slice relative to their CPU time weight.
While StartupCPUWeight= only applies to the startup phase of the system, CPUWeight= applies to normal runtime of the system, and if the former is not set also to the startup phase. Using StartupCPUWeight= allows prioritizing specific services at boot-up differently than during normal runtime.
Implies "CPUAccounting=true".
These settings replace CPUShares= and StartupCPUShares=.
CPUQuota= Assign the specified CPU time quota to the processes executed. Takes a percentage value, suffixed with "%". The percentage specifies how much CPU time the unit shall get at maximum, relative to the total CPU time available on one CPU. Use values > 100% for allotting CPU time on more than one CPU. This controls the "cpu.max" attribute on the unified control group hierarchy and "cpu.cfs_quota_us" on legacy. For details about these control group attributes, see cgroup-v2.txt[2] and sched-design-CFS.txt[4].
Example: CPUQuota=20% ensures that the executed processes will never get more than 20% CPU time on one CPU.
Implies "CPUAccounting=true".
AllowedCPUs= Restrict processes to be executed on specific CPUs. Takes a list of CPU indices or ranges Manual page systemd.resource-control(5) line 63 (press h forhelp or q to quit). .............................
CPUWeight 配置
1 2 3 4 5 6 7 8
┌──[root@vms99.liruilongs.github.io]-[~] └─$systemctl show --property CPUWeight httpd.service CPUWeight=[not set] ┌──[root@vms99.liruilongs.github.io]-[~] └─$systemctl set-property httpd.service CPUWeight=200 ┌──[root@vms99.liruilongs.github.io]-[~] └─$systemctl show --property CPUWeight httpd.service CPUWeight=200
