Questo sito utilizza i cookie per migliorare l'esperienza di navigazione. Continuando la navigazione acconsenti all'utilizzo dei cookie.

A REAL CASE of high CPU consuming analisys by a java process

The scenario is High CPU usage by a java process, It pretty often appens did you know? This is my java process:

$ps -o pid,cmd -e | grep java | grep prod| tr " " "\n"
13605
java
-Xmx8192m
-Xms8192m
-XX:MaxPermSize=1024m
-Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.port=5601
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-XX:+UseMembar
-XX:+UseConcMarkSweepGC
-XX:+UseParNewGC
-XX:+CMSIncrementalMode
-XX:+CMSIncrementalPacing
-Dsun.rmi.dgc.server.gcInterval=3600000
-Dsun.rmi.dgc.client.gcInterval=3600000
-Doracle.jdbc.timezoneAsRegion=false
-Djava.security.egd=file:///dev/urandom
-DgpInstance
-server
-jar
XXXXXXXXX.jar
-n
GC01
-p
production
-m
TEST

The goal is to identify which threads of the process is consuming the most CPU of the host. In this perspective the top command could be very usefull,just add some awk stuff to limit the column displayed ( with -b flag ) and use the -H flag to print lwps.

$ top -b -n 1 -H | awk 'NR>6 && NR<13 {printf "%6s %-4s %-4s %-s\n",$1,$2,$9,$NF}'
   PID USER %CPU COMMAND
 13608 gp   26.5 java
 13607 gp   24.6 java
  5121 gp   13.3 java
  5290 gp   13.3 java
  6060 gp   11.4 java                                                                                                            

As a result we can say that only two thread are consuming CPU:  13608  and 13607 .

The thread id  number is printed in decimal format and we need to convert it in exadecimal

$echo "obase=16; 13608"|bc

3528

the next step is to make a thread dump of the jvm:
$kill -3 13605

the last thing is to find in the standard output of the java process the exadecimal number foud before
$grep 3528 nohup.out
 "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f9ea0017800 nid=0x3528 runnable

In this case we found out that the cpu is consumed by minor collections, So We probably have to tuning the jvm heap.

Share

Aggiungi commento


Codice di sicurezza
Aggiorna