nagios报警,有一台实机的内存使用率达到极限了,告警信息如下:
MEMORY CRITICAL : Mem used: 99.61%, Swap used: 95.98%
上帝啊,128G的内存居然被用光了!!!
登上这台机器,验证一下,发现确实啥也不剩了,没有任何缓存,而且连交换分区都用了
# free -g
total used free shared buffers cached
Mem: 125 125 0 0 0 0
-/+ buffers/cache: 125 0
Swap: 8 7 0
看看上面跑了什么,居然跑了12台虚机
virsh list --all
Id Name State
----------------------------------------------------
1 vis-16-13-27 running
2 vis-16-13-29 running
3 vis-16-13-74 running
4 vis-16-13-26 running
5 vis-16-13-25 running
6 vis-16-13-30 running
7 vis-16-13-28 running
15 vis-16-13-33 running
41 vis-16-13-75 running
44 vis-16-13-24 running
45 vis-16-13-22 running
46 vis-16-13-23 running
检测一下每个虚机的内存使用情况
virsh dominfo vis-16-13-28
d: 7
Name: vis-16-13-28
UUID: 083bde71-764a-23ad-a593-e81edc7adde1
OS Type: hvm
State: running
CPU(s): 4
CPU time: 2515172.9s
Max memory: 7864320 KiB
Used memory: 7864320 KiB
Persistent: yes
Autostart: enable
Managed save: no
Security model: none
Security DOI: 0
居然也都是全满
于是登录到虚机,用free -g查看
[root@vis-16-13-28 ~]# free -m
total used free shared buffers cached
Mem: 7366 7233 132 0 148 3359
-/+ buffers/cache: 3726 3640
Swap: 0 0 0
分明是只用了4G,空闲3G
调整一下,给虚机分5G内存
virsh setmem vis-16-13-28 5120m
再看实体机的内存
free -g
total used free shared buffers cached
Mem: 125 123 2 0 0 0
-/+ buffers/cache: 122 3
Swap: 8 7 0
放出来了3G,警报消除
看虚机的内存
[root@vis-16-13-28 ~]# free -m
total used free shared buffers cached
Mem: 4806 4129 676 0 71 723
-/+ buffers/cache: 3335 1471
Swap: 0 0 0
也正常。ok,搞定。