Linux Performance Analysis and Tuning talk at PLUG and University of Pune

by

On this month’s PLUG session I gave talk on Performance Analysis and Tuning on Linux, which has good feedback. Then I got an invitation to do the similar lecture from CMS (Centre for Modeling and Simulation) Department, University of Pune. I accepted it and took a session on 25th of this month. I covered same topics in both sessions.

Notice about the talk in UoP

I decided not to have any slides and just show output of commands like ps, top, sar and different settings in “/proc” and “/sys” filesystems. I tried to give information about differrent fields in the outputs and related that with performance issue,
like “load average” in top output, Average request size in iostat ouput.

To show the different subsystem I referred different figures from http://www.redbooks.ibm.com/redpapers/pdfs/redp4285.pdf

Following are some commands and other details which I share

ps command

[sourcecode language=”text” gutter=”false”]
$ ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.2 61988 5236 ? Ss Aug17 0:02 /usr/lib/systemd/systemd
root 2 0.0 0.0 0 0 ? S Aug17 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? S Aug17 0:05 [ksoftirqd/0]
[/sourcecode]

– top Command

[sourcecode language=”text” gutter=”false”]
$top
[/sourcecode]

From the top command output we discussed :–
– What is load average and what does it high value means?
– What are different fields in line starting with “CpuX”?
– What is nice value and how it effects the priority for a process?

– free command

[sourcecode language=”text” gutter=”false”]
$ free -m

total used  free  shared   buffers cached
Mem:               1962  1306   656    0          75         508
-/+ buffers/cache:        722  1240
Swap:              2000   0  2000

[/sourcecode]

With above output I shared the concept of the “buffers” and “cached”. Generally we consider “cached” memory as used memory and does not count it as free, which is not correct. Cache memory can be freed whenever needed. Later I demoed with the drop_cache setting of the VM’s (Virtual Memory) configuration(/proc/sys/vm)

- SAR (System Activity Reporter)

It is used to collect, report, or save system activity information. It can report CPU, Disk, Memory, Network, paging and other information. If configured with cron then it collects information everyday and store the statistics in “/var/log/sa” folder. The “saX” files are binary files and “sarX” file are human readable data, where is day of the month. We can collect all those stats manually as well.

Following command will collect Disk, CPU, Memory, paging, load average stats in every 5 mins and save it in “logfile” file in binary format:-

[sourcecode language=”text” gutter=”false”]
$ sar -dp -u -r -B -q 5 -o
[/sourcecode]

 And with following the binary file can be coverted into CSV file:-[sourcecode language=”text” gutter=”false”]
$ sadf -d — -dp -u -r -B -q 5 >
[/sourcecode]

Other than above I ran and discussed about output collected from following commands:-
– dmesg
– dmideode
– lspci
– lspci -nv
I also showed content of the some of the files/directories from /proc filesystem and showed different settings. Some of those are:-

–  /proc/meminfo
–  /proc/<pid>/maps
–  /proc/diskstats
–  /proc/sys/net/ipv4
–  /proc/slabinfo
–  /proc/interrupts

Then I showed  how to configure sysRq and, collect stack trace for each process and memory information of the system

[sourcecode language=”text” gutter=”false”]
echo 1 > /proc/sys/kernel/sysrq
echo t > /proc/sysrq-trigger
echo m > /proc/sysrq-trigger
[/sourcecode]

At the end I gave brief info about SystemTap and Ftrace.
2 hours are very short time to talk about Linux performance. I tried to share as much as I can.
Hope it was useful to some of the attendees.

Discover more from Neependra Khare

Subscribe to get the latest posts sent to your email.

Leave a Comment