Linux Performance Analysis and Tuning talk at PLUG and University of Pune
On this month’s PLUG session I gave talk on Performance Analysis and Tuning on Linux, which has good feedback. Then I got an invitation to do the similar lecture from CMS (Centre for Modeling and Simulation) Department, University of Pune. I accepted it and took a session on 25th of this month. I covered same topics in both sessions.
I decided not to have any slides and just show output of commands like ps, top, sar and different settings in “/proc” and “/sys” filesystems. I tried to give information about differrent fields in the outputs and related that with performance issue,
like “load average” in top output, Average request size in iostat ouput.
To show the different subsystem I referred different figures from http://www.redbooks.ibm.com/redpapers/pdfs/redp4285.pdf
Following are some commands and other details which I share
$ ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.2 61988 5236 ? Ss Aug17 0:02 /usr/lib/systemd/systemd root 2 0.0 0.0 0 0 ? S Aug17 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? S Aug17 0:05 [ksoftirqd/0]
- top Command
From the top command output we discussed :–
- What is load average and what does it high value means?
- What are different fields in line starting with “CpuX”?
- What is nice value and how it effects the priority for a process?
- free command
$ free -m total used free shared buffers cached Mem: 1962 1306 656 0 75 508 -/+ buffers/cache: 722 1240 Swap: 2000 0 2000
With above output I shared the concept of the “buffers” and “cached”. Generally we consider “cached” memory as used memory and does not count it as free, which is not correct. Cache memory can be freed whenever needed. Later I demoed with the drop_cache setting of the VM’s (Virtual Memory) configuration(/proc/sys/vm)
- SAR (System Activity Reporter)
It is used to collect, report, or save system activity information. It can report CPU, Disk, Memory, Network, paging and other information. If configured with cron then it collects information everyday and store the statistics in “/var/log/sa” folder. The “saX” files are binary files and “sarX” file are human readable data, where is day of the month. We can collect all those stats manually as well.
Following command will collect Disk, CPU, Memory, paging, load average stats in every 5 mins and save it in “logfile” file in binary format:-
$ sar -dp -u -r -B -q 5 -o
$ sadf -d — -dp -u -r -B -q 5 >
Then I showed how to configure sysRq and, collect stack trace for each process and memory information of the system
echo 1 > /proc/sys/kernel/sysrq echo t > /proc/sysrq-trigger echo m > /proc/sysrq-trigger