Based on elasticsearch5 4. Summary of frequently asked questions

Elasticsearch5 has been used in recent projects 4 (ES) is a relatively new version. There are many problems in the use process, which is a headache, but the problem is finally solved.

Problem 1: esclient acquisition is slow, and client: failed to create a child event loop cannot be obtained

Because of the business needs, we need to add an ES index every time we do not upload a batch of files. Each time we add an index, we need to obtain the connection and then operate. Especially in large quantities, there are obviously a lot of times to obtain it. Moreover, the main reason for this problem is that we operate es frequently. For example, if there are 100 files in a batch, we need to obtain it 100 times, In order to reduce the acquisition time of ES client, a scheme is finally adopted, that is, initialize the connection when the service is started, obtain it at one time, and then call it directly later. After the entire batch of files are uploaded, finally add the ES index instead of adding files one by one. Obviously, this method does not need to obtain connections for each batch, which greatly improves the execution efficiency.

First, we initialize the static es client in the startup class when the service is started:

Then call directly when needed:

This can greatly reduce the connection times of ES client and improve efficiency.

Es codes are as follows:

Problem 2: memory overflow: Java lang.OutOfMemory:unable to create new native thread

During the project development process, memory overflow is a headache. In the process of using ES, it is encountered very frequently, especially in mass stress testing. Many methods have been thought of in terms of JVM memory tuning, but there is no effect. The problem is still not solved. Finally, when looking at the source code, A reason was found. In combination with the error reporting exception, a large number of threads were automatically created during the use of ES, which exceeded the capacity of the system, resulting in memory overflow. When studying the source code, it was found that the number of threads created by ES can be controlled by setting. The following is the default number of ES creation threads:

Our CPU is 10 cores and 40 threads

From the calculation results, if the default value is used, the number of threads that ES can create is a large value, which far exceeds the capacity of the system itself. It is mainly to adjust the setting value. After adjustment, we change the default value of ES as follows:

After testing, ES created a small number of threads and met our development needs. There was no memory overflow problem again.

The above article is based on elasticsearch5 The summary of common problems in 4 is all the content shared by Xiaobian. I hope it can give you a reference and support more programming tips.