In a production environment, fast and effective troubleshooting is crucial when Java applications experience problems such as performance degradation, slow response, memory leaks or crashes.
As a Java developer, mastering a series of command line tools can help you quickly locate the root cause of your problem and minimize losses.
This article summarizes 50 most commonly used commands for Java developers when troubleshooting problems online, covering multiple aspects such as system resource monitoring, JVM analysis, network detection, and log processing.
System resource monitoring command
1. top - Real-time monitoring of system processes
# Basic usetop # Sort by memorytop -o %MEM # View only the processes of specific userstop -u java_user # Monitor specific processestop -p $(pgrep -d',' java)
Key points for troubleshooting: Pay attention to CPU usage (%CPU), memory usage (%MEM), and run time (TIME+), and can quickly discover processes with abnormal resource occupancy.
2. htop - Enhanced version of system monitoring
htop
Key points for troubleshooting: Provides a more friendly interface than top, supports mouse operation, can view the usage of each CPU core, and is suitable for monitoring of multi-core systems.
3. vmstat - System Resource Statistics
# Output every 2 seconds, 10 times in totalvmstat 2 10
Key points for troubleshooting: Pay attention to r (run queue), b (blocking process), si/so (using switch area), us/sy (user CPU/system CPU), and you can determine whether the system is a CPU-intensive or IO-intensive load.
4. free - Memory usage
# Show readable formatfree -h # Update every 3 secondsfree -h -s 3
Key points for troubleshooting: Check whether the available memory is too low, whether the swap is used too high, and determine whether there is memory pressure.
5. iostat - IO statistics
# Check the device IO statistics, once every 2 seconds, a total of 5 timesiostat -xd 2 5
Key points for troubleshooting: Pay attention to %util (device utilization), r/s and w/s (read and write operands), await (IO waiting time), and determine whether IO has become a bottleneck.
JVM monitoring commands
6. jps - List Java processes
# List simple process informationjps # List the complete package name and process parametersjps -lvm
Key points for troubleshooting: This is the starting point for troubleshooting Java problems. First, you need to determine the PID of the target Java process.
7. jstat - JVM statistics monitoring
# Monitor GC situation, once every 1000 milliseconds, a total of 10 timesjstat -gc [pid] 1000 10 # Check the class loading situationjstat -class [pid] # Check JIT compilation statusjstat -compiler [pid]
Key points for troubleshooting: Pay attention to GC frequency, GC time consumption, and space use in the elderly to determine whether there is a GC problem.
8. jmap - Memory Mapping
# View heap memory usage overviewjmap -heap [pid] # View object statisticsjmap -histo [pid] | head -20 # Generate heap dump filejmap -dump:format=b,file=heap_dump.hprof [pid]
Key points for troubleshooting: Analyze large objects in memory, check whether there is a memory leak, and generate a heap dump file for detailed analysis if necessary.
9. jstack - Thread Stack Analysis
# Generate a thread stack snapshotjstack [pid] > thread_dump.log # Thread dump containing lock informationjstack -l [pid] # Detect deadlockjstack -F [pid] | grep -A 30 "Found.*deadlock"
Key points for troubleshooting: Find threads with BLOCKED status, analyze inter-thread lock competition, and troubleshoot deadlock or thread suspension problems.
10. jinfo - View and modify JVM parameters
# View all JVM parametersjinfo -flags [pid] # View specific parameter valuesjinfo -flag MaxHeapSize [pid] # Dynamically modify certain parametersjinfo -flag +HeapDumpOnOutOfMemoryError [pid]
Key points for troubleshooting: Confirm whether the JVM key parameters are configured correctly and dynamically adjust them if necessary.
11. jcmd - JVM diagnostic command tool
# List all available commandsjcmd [pid] help # View JVM version informationjcmd [pid] # Execute GCjcmd [pid] # Generate heap dumpjcmd [pid] GC.heap_dump filename= # Generate thread dumpjcmd [pid]
Key points for troubleshooting:jcmd is a multi-function tool that can replace multiple single-function JDK tools. Mastering it can simplify the problem-solving process.
12. jhat - Heap Dump Analysis Tool
# Analyze heap dump filesjhat heap_dump.hprof # Then visit http://localhost:7000 View analysis results
Key points for troubleshooting: View object instances and reference relationships in the browser to find possible memory leak points.
Network related commands
13. netstat - Network connection status
# View all TCP connectionsnetstat -ant # Check the listening portnetstat -tnlp # View network connections for specific processesnetstat -anp | grep [pid]
Key points for troubleshooting: Pay attention to the number of ESTABLISHED connections and the number of TIME_WAIT status connections to check the port occupancy.
14.ss - Faster socket statistics
# Show all TCP connectionsss -ta # View connection statisticsss -s # View specific port connectionsss -tn sport = :8080
Key points for troubleshooting: More efficient than netstat, suitable for systems with a large number of connections, and can quickly count various connection status.
15. lsof - List open files
# View all files opened by the processlsof -p [pid] # Check the usage of specific portslsof -i:8080 # View all network connectionslsof -i
Key points for troubleshooting: Check which files and network connections the process has opened, and troubleshoot file descriptor leakage.
16. ping - network connectivity test
# Basic useping # Limit the number of sendsping -c 5
Key points for troubleshooting: Test network connectivity and latency with the target host.
17. curl - HTTP request testing
# Simple GET requestcurl http://localhost:8080/api/status
# Request with timeout controlcurl --connect-timeout 5 -m 10
# POST requestcurl -X POST -d '{"name":"test"}' -H "Content-Type: application/json" http://localhost:8080/api/usersKey points for troubleshooting: Test the HTTP service response status and check the response code, response time and response content.
18. tcpdump - Network packet analysis
# Capture packets on specific portstcpdump -i any port 8080 -n # Capture packets for specific hoststcpdump host 192.168.1.100 # Save the packet to the filetcpdump -i any -w
Key points for troubleshooting: Analyze the content of network packets and troubleshoot network protocol-related issues, especially network delay and connection reset issues.
19. traceroute - Routing Tracking
# Basic usetraceroute
Key points for troubleshooting: Troubleshoot network routing problems and determine which link network delay occurs.
Log and file analysis commands
20. tail - view the end of the file
# View logs in real timetail -f # Show the last 100 linestail -n 100 # Start displaying from line 1000tail -n +1000
Key points for troubleshooting: Real-time monitoring of log output is the basic operation for troubleshooting problems.
21. grep - Text Search
# Search for error loggrep "ERROR" # Show the first and last 5 lines of the matching rowgrep -A 5 -B 5 "OutOfMemoryError" # Recursively search multiple filesgrep -r "Connection refused" /var/log/
Key points for troubleshooting: Quickly locate error information in the log is a powerful tool for analyzing problems.
22. awk - Text processing
# Extract specific columnsawk '{print $1, $4}'
# Statistics HTTP response code distributionawk '{print $9}' | sort | uniq -c | sort -nr
# Calculate the average response timeawk '{sum+=$10; count++} END {print sum/count}' Key points for troubleshooting: Conduct structured analysis of the log, extract key information, and conduct statistical calculations.
23. sed - Stream Editor
# Replace textsed 's/ERROR/CRITICAL/g' # Show only specific row rangessed -n '1000,2000p'
Key points for troubleshooting: Process and convert log text and extract specific lines of content.
24. find - File search
# Find large filesfind / -type f -size +100M
# Find the recently modified filesfind /var/log -mtime -1 -type f -name "*.log"
# Find and delete old log filesfind /var/log -name "*.log" -mtime +30 -exec rm {} ;Key points for troubleshooting: Locate files that take up space, find recently changed configuration files, etc.
25. du - Disk usage statistics
# Check directory sizedu -sh /var/log # Display subdirectories by sizedu -h --max-depth=1 /var | sort -hr
Key points for troubleshooting: Analyze the disk space usage and find the directory that takes up the most space.
Process Management Commands
26. ps - Process Status
# View all processesps aux # View specific user's processesps -u tomcat # Sort by CPU usageps aux --sort=-%cpu | head -10 # Sort by memory usageps aux --sort=-%mem | head -10
Key points for troubleshooting: Check the process status and resource usage, and identify abnormal processes.
27. pstree - Process Tree
# Show process treepstree # Display process tree for a specific processpstree -p [pid]
Key points for troubleshooting: Understand the parent-child relationship between processes, especially when troubleshooting daemons and child processes.
28. kill - Send a signal to the process
# Stop the process normallykill [pid] # Forced terminate processkill -9 [pid] # Generate Java thread dumpkill -3 [java_pid]
Key points for troubleshooting: Terminate the exception process, or send a signal to the Java process to generate diagnostic information.
29. pkill - Terminate process by name
# Terminate process with a specific namepkill java # Send SIGTERM signals to all Java processespkill -15 java
Key points for troubleshooting: Batch operation of specific types of processes and respond quickly in emergencies.
30. pgrep - Find process ID
# Find Java Process IDpgrep java # Find and display the command linepgrep -a java
Key points for troubleshooting: Quickly find the PID of a specific process for subsequent monitoring and analysis.
System performance analysis command
31. sar - System Activity Report
# Check CPU usagesar -u 1 5 # Check memory usagesar -r 1 5 # View disk IOsar -b 1 5 # View network statisticssar -n DEV 1 5
Key points for troubleshooting: Comprehensively analyze the use of system resources and identify performance bottlenecks.
32. mpstat - Multiprocessor Statistics
# Check all CPU core usagempstat -P ALL 2 5
Key points for troubleshooting: Analyze the load distribution of each CPU core and detect whether there is a single core overload.
33. dmesg - kernel message
# View kernel messagesdmesg # View only errors and warningsdmesg --level=err,warn
Key points for troubleshooting: Check system errors and warning information, especially serious problems such as OOM (out of memory).
34. strace - Tracking system calls
# Track the specified processstrace -p [pid] # Track specific system callsstrace -e open,read,write -p [pid] # Statistics the system call time and numberstrace -c -p [pid]
Key points for troubleshooting: Analyze the interaction between the process and the system and troubleshoot the system call level issues.
35. perf - Performance analysis tool
# Record performance dataperf record -F 99 -p [pid] -g -- sleep 30 # Analyze performance dataperf report
Key points for troubleshooting: Position performance bottlenecks through the sampling analysis program's CPU usage hotspots.
Database related commands
36. mysqladmin - MySQL server management
# Check MySQL statusmysqladmin -u root -p status #Process Listmysqladmin -u root -p processlist # Variable statemysqladmin -u root -p variables
Key points for troubleshooting: Check the status and configuration of MySQL server, monitor the number of connections and process status.
37. mysqlshow - Display database information
# Show all databasesmysqlshow -u root -p # Display tables for specific databasesmysqlshow -u root -p database_name
Key points for troubleshooting: Quickly understand database structure and table information.
38. pg_stat_activity - PostgreSQL activity query
# View active connections (execute in psql)SELECT * FROM pg_stat_activity; # View long-running queriesSELECT pid, now() - query_start AS duration, query FROM pg_stat_activity WHERE state = 'active' ORDER BY duration DESC;
Key points for troubleshooting: Check the active connections and executed queries in the PostgreSQL database to find long-running queries.
39. redis-cli - Redis client
# Connect to Redis Serverredis-cli # Monitor Redis commandsredis-cli monitor # Get statisticsredis-cli info
Key points for troubleshooting: Check the status of the Redis server, monitor the execution of commands, and troubleshoot cache-related issues.
Container and microservice related commands
40. docker ps - view container
# View running containersdocker ps # View all containers (including stopped)docker ps -a
Key points for troubleshooting: Understand the currently running container and its status.
41. docker logs - view container logs
# View container logdocker logs [container_id] # View logs in real timedocker logs -f [container_id] # Show the last 100 rows of logsdocker logs --tail 100 [container_id]
Key points for troubleshooting: Check the log output of the application in the container and troubleshoot internal problems of the container.
42. docker stats - container resource usage statistics
# Check the resource usage of all containersdocker stats # View resource usage for specific containersdocker stats [container_id]
Key points for troubleshooting: Monitor the CPU and memory usage of the container and discover resource bottlenecks.
43. kubectl - Kubernetes command line tool
# Get all pod statuskubectl get pods # View Pod detailskubectl describe pod [pod_name] # View Pod logkubectl logs [pod_name] # Enter the Pod to execute the commandkubectl exec -it [pod_name] -- /bin/bash
Key points for troubleshooting: Manage and troubleshoot containerized application problems in Kubernetes environment.
44. kubectl top - Kubernetes resource usage statistics
# Check node resource usagekubectl top nodes # Check Pod resource usagekubectl top pods
Key points for troubleshooting: Monitor the resource usage of each component in the Kubernetes cluster.
Special tool commands
45. arthas - Java diagnostic tools
# Start arthasjava -jar # Use dashboard command to view system overviewdashboard # Tracking method executiontrace method # View JVM informationjvm
Key points for troubleshooting:Arthas is an open source Java diagnostic tool from Alibaba, providing richer Java application analysis capabilities.
46. btrace - Java runtime tracking
# Use BTrace script tracking methodbtrace [pid]
Key points for troubleshooting: Dynamically track the execution of Java methods without restarting the application.
47. jprofiler - Performance Analysis Tool
# Start jprofiler clientjprofiler # Connect to remote JVMjpenable
Key points for troubleshooting: Carry out more in-depth performance analysis, including CPU hotspots, memory leaks, thread analysis, etc.
48. ab - Apache HTTP Server Benchmarking
# Send 1000 requests, and 100 concurrent numbersab -n 1000 -c 100 http://localhost:8080/api/test
Key points for troubleshooting: Test HTTP service performance and check response time and throughput under load.
49. wrk - HTTP benchmarking tool
# Run 30 seconds of test, using 12 threads and 400 connectionswrk -t12 -c400 -d30s http://localhost:8080/api/test
Key points for troubleshooting: A more powerful HTTP pressure testing tool than ab, which can test the performance of applications under high concurrency.
50. java-flight-recorder (JFR) - JVM flight recorder
# Start JFR recordjcmd [pid] duration=60s filename= # Check JFR record statusjcmd [pid] # Stop JFR recordingjcmd [pid]
Key points for troubleshooting: Collect detailed information about the JVM runtime for subsequent analysis and performance optimization.
Online problem investigation ideas and methods
Quickly locate problem types
1. CPU problem troubleshooting path：top → jstack → perf/JFR
2. Memory problem troubleshooting path：free → jmap → jhat/MAT
3. Disk problem troubleshooting path：df → du → lsof/find
4. Network problem troubleshooting path：netstat → tcpdump → wireshark
5. Apply exception troubleshooting path：jps → logs → jstack → arthas
Frequently Asked Questions Tips
1. High CPU usage
# Find processes that occupy high CPUtop -c # If it is a Java process, generate a thread dumpjstack [pid] > thread_dump.log # Find threads that occupy high CPUps -mp [pid] -o THREAD,tid,time | sort -rk 3 # Convert thread ID to hexadecimalprintf "%x\n" [tid] # Find the thread in the thread dumpgrep -A 30 [hex_tid] thread_dump.log
2. Memory leak
# Check the heap memory usagejmap -heap [pid] # Generate multiple heap dumps and compare object growthjmap -dump:format=b,file= [pid] # After waiting for a whilejmap -dump:format=b,file= [pid] # Use MAT or jhat to analyze heap dumpjhat
3. Frequent GC
# Monitor GC statusjstat -gc [pid] 1000 # Enable GC logjava -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc: # Analyze GC logs# useGCViewerorGCEasyPlease help with tools
4. Slow response
# View system loaduptime # Check network connection statusnetstat -ant | grep ESTABLISHED | wc -l # Check thread statusjstack [pid] | grep -c "" # Use Arthas for method trackingtrace slowMethod
5. Insufficient disk space
# Check disk usagedf -h
# Find large filesfind / -type f -size +100M -exec ls -lh {} ;
# Check directory sizedu -sh /*
# Clean the log filesfind /var/log -name "*.log.*" -mtime +7 -deleteSummarize
During the actual investigation process, these commands are usually required to combine and form a systematic problem investigation process.
Remember, successful online problem troubleshooting not only depends on proficient use of commands, but also on in-depth understanding of Java application architecture, JVM principles, and operating system mechanisms.
As a Java developer, it is recommended to be familiar with the use of these commands and practice them in non-production environments to prepare for real online problem investigation. At the same time, establishing a complete monitoring system and log system can make the problem investigation more effective.
The above is the detailed content of the 50 commonly used commands recommended by Java developers for online problem investigation. For more information about Java online problem investigation commands, please pay attention to my other related articles!