Record the troubleshooting, analysis process and solution of a memory overflow problem

cause

This test tool has been developed for some time. Due to the large amount of data and slow writing of data, the execution of the tool takes a long time. Therefore, the implementation scheme is optimized again and the two-stage program development is carried out.

After optimization, 2000 pieces of data are written, which takes 4 seconds. Personally, it's much faster.

Therefore, I want to see how long it takes to execute in batch.

As a result, 10 minutes, 20 minutes and 1 hour passed

The program has been writing data. Wait for me, blue and thin. Think about making a cup of tea.

As a result, NIMA reported an error after receiving the water, as shown in the figure below:

Psychological process

Although, as a test cat, coding ability is not a level at all compared with pure development.

Of course, it can also be solved by turning to development colleagues, but this is not the reason to retreat and avoid solving problems.

This error report is also the first time I have encountered it. It is undoubtedly difficult for a test that does not understand the memory problem. The results are all kinds of searches.

Finally, locate that it is caused by memory overflow.

To tell you the truth, I reported the wrong card for nearly 1.5 days. I tried to find my colleagues to solve it several times, but I still held back and tried to solve it myself.

Several common memory overflows

Trial process

Xiaobian has limited experience. At present, he has only encountered the first and second memory overflow problems.

Let's share my thoughts and process of troubleshooting.

Most blogs and articles on the Internet are written about modifying the running memory. I tried it, but it was not effective at all. The abandonment scheme is as follows:

When I was poor, I suddenly thought of the previous performance test. I saw that the development passed the JDK's own tool jvisualvm for GC tuning, and the results were really used.

How to operate

Find jvisualvm.net in the JDK installation directory bin Exe, double-click to open it, as shown in the figure below.

Run the program you want to monitor. Double click the program on the left to run the program. The tool will display the operation of the program. Only memory is reserved, and others are not selected, as shown in the following figure.

Memory overflow, so we can only focus on memory for the time being. Memory will display the changes of data in the heap at runtime, such as object instances.

Then wait in your heart and check the monitoring memory. The result shows that the maximum memory increases several times instantaneously, as shown in the figure below.

The maximum memory suddenly soared several times, and the program threw the following exceptions at the same time.

After the problem is located, the next step is to reproduce and find the problem.

Adjust the operating parameters to - xmx216m, run the program again, synchronize steps 1 and 2, click dump, enter the monitoring interface, and click to switch to the class tab, as shown in the following figure.

Finally, I found the boy. It turned out that it created 9314 char array objects.

Double click the class name to find the problem, as shown in the figure below.

After finding the problem, the next thing is to optimize the program.

Solution

After the above troubleshooting, the cause of the problem is found, which is the overflow caused by exceeding the maximum allowable number of rows when writing data. Finally, piecewise writing is adopted to solve this problem perfectly.

Although the memory overflow problem was solved, the execution of the program was still very slow. Later, the method of writing big data was found. After re optimization of the program, it took only 54 seconds to write all 260W data. It's really very nice!

The above is the whole process of troubleshooting. Of course, the code of this case is a program written to simulate memory overflow. It is non business code. It is an introductory case for reference only.

Write at the end

Jvisualvm and jprofile are really good tools for memory optimization and troubleshooting. They can be said to be necessary artifacts for writing programs.

There are a lot of articles on memory overflow. For me, there are really few articles that can be used. As a test, meow does not dare to make a chance and comment. Interested students can buy a book to learn more about it.

I would like to thank brother Qiang for his help. He can give me a lot of ideas and inspiration every time, which makes me benefit a lot.

Brother Qiang

Reference articles

Solve the problem of Java heap space in the project [1]

Remember to solve outofmemoryerror once: detailed process and solution of Java heap space [2]

reference material

[1] Solve the problem of Java heap space in the project: https://blog.csdn.net/smh0310/article/details/90664598

[2] Remember the detailed process and solution of outofmemoryerror: Java heap space: https://blog.csdn.net/lyflyyvip/article/details/82288719