Comparison of several common thread pools in Java
1. Why use thread pool
Many server applications, such as web servers, database servers, file servers, or mail servers, are oriented to handle a large number of short tasks from some remote source. Requests arrive at the server in some way, which may be through the network protocol (such as HTTP, FTP or pop), through the JMS queue, or through polling the database. No matter how the requests arrive, it often happens in the server application that the processing time of a single task is very short, but the number of requests is huge.
A simple model for building a server application is to create a new thread every time a request arrives, and then serve the request in the new thread. In fact, this approach works well for prototyping, but if you try to deploy server applications running in this way, the serious shortcomings of this approach are obvious. One of the disadvantages of the thread per request method is that it is very expensive to create a new thread for each request; the server that creates a new thread for each request spends more time and resources on creating and destroying threads than on processing actual user requests.
In addition to the overhead of creating and destroying threads, active threads consume system resources. Creating too many threads in a JVM may cause the system to run out of memory or "over switch" due to excessive memory consumption. To prevent resource shortages, server applications need some way to limit the number of requests processed at any given time.
Thread pool provides a solution to the problem of thread life cycle overhead and insufficient resources. By reusing threads for multiple tasks, the cost of thread creation is allocated to multiple tasks. The advantage is that because the thread already exists when the request arrives, the delay caused by thread creation is inadvertently eliminated. In this way, the request can be served immediately, making the application more responsive. Moreover, by appropriately adjusting the number of threads in the thread pool, that is, when the number of requests exceeds a certain threshold, any other new requests are forced to wait until a thread is obtained to process, so as to prevent insufficient resources.
2. Risks of using thread pool
Although thread pooling is a powerful mechanism for building multithreaded applications, using it is not without risk. An application built with a thread pool is vulnerable to all the concurrency risks that any other multithreaded application is vulnerable to, such as synchronization errors and deadlocks, and it is also vulnerable to a few other risks specific to the thread pool, such as pool related deadlocks, insufficient resources, and thread leaks.
2.1 deadlock
Any multithreaded application is at risk of deadlock. When each process or thread in a group is waiting for an event that can only be caused by another process in the group, we say that the group of processes or threads is deadlocked. The simplest case of deadlock is that thread a holds the exclusive lock of object x and is waiting for the lock of object y, while thread B holds the exclusive lock of object y and is waiting for the lock of object X. Unless there is some way to break the wait for a lock (which is not supported by Java locking), the deadlock thread will wait forever.
Although there is a risk of deadlock in any multithreaded program, thread pool introduces another possibility of deadlock. In that case, all pool threads are executing a task that has been blocked waiting for the execution result of another task in the queue, but this task cannot run because there are no unoccupied threads. This happens when the thread pool is used to simulate many interactive objects, and the simulated objects can send queries to each other. These queries are then executed as queued tasks, and the query objects are synchronously waiting for a response.
2.2 insufficient resources
One advantage of thread pool is that it is compared with other alternative scheduling mechanisms (some of which we have already discussed) they usually perform well. But this is only true if the thread pool size is properly adjusted. Threads consume a lot of resources, including memory and other system resources. In addition to the memory required by thread objects, each thread requires two potentially large execution call stacks. In addition, the JVM may Java threads create a native thread that will consume additional system resources. Finally, although the scheduling overhead of switching between threads is very small, if there are many threads, environment switching may also seriously affect the performance of the program.
If the thread pool is too large, the resources consumed by those threads may seriously affect the system performance. Switching between threads is a waste of time, and using more threads than you actually need may cause resource scarcity because pooled threads are consuming some resources that may be more effectively utilized by other tasks. In addition to the resources used by the thread itself, the work done when the service request may require other resources, such as JDBC connections, sockets, or files. These are also limited resources. Too many concurrent requests may also cause failure. For example, JDBC connections cannot be allocated.
2.3 concurrent errors
Thread pools and other queuing mechanisms rely on the use of wait () and notify () methods, both of which are difficult to use. If the encoding is incorrect, notifications may be lost, causing the thread to remain idle, even though there is work to be processed in the queue. Extreme care must be taken when using these methods. It's best to use an existing implementation that already knows to work, such as util Concurrent package.
2.4 thread leakage
A serious risk in various types of thread pools is thread leakage. This happens when a thread is removed from the pool to execute a task, but the thread does not return to the pool after the task is completed. A thread leak occurs when a task throws a runtimeException or an error. If the pool class does not catch them, the thread will only exit and the size of the thread pool will be permanently reduced by one. When this happens enough times, the thread pool will eventually be empty and the system will stop because there are no threads available to process the task.
Some tasks may always wait for some resources or input from the user, and these resources cannot be guaranteed to become available. The user may have gone home. Such tasks will be permanently stopped, and these stopped tasks will cause the same problem as thread leakage. If a thread is permanently consumed by such a task, it is actually removed from the pool. For such tasks, you should either give them only their own threads or let them wait for a limited time.
2.5 request overload
It is possible that requests alone overwhelm the server. In this case, we may not want to queue every incoming request to our work queue, because the tasks waiting in the queue may consume too many system resources and cause resource shortage. In this case, it's up to you to decide what to do; In some cases, you can simply discard the request and retry the request later by relying on a higher-level protocol. You can also reject the request with a response indicating that the server is temporarily busy.
3. Guidelines for effective use of thread pool
Thread pooling can be an extremely effective way to build server applications as long as you follow a few simple guidelines:
Do not queue tasks that synchronously wait for the results of other tasks. This may lead to a deadlock in the form described above. In that deadlock, all threads are occupied by some tasks, which wait for the results of the queued tasks in turn, and these tasks cannot be executed because all threads are busy.
Be careful when using pooled threads for potentially long operations. If the program must wait for a resource such as I / O completion, specify the maximum wait time and whether the task will then fail or re queue for later execution. This ensures that some progress will eventually be made by releasing a thread to a task that may be completed successfully.
Understand the task. To effectively resize the thread pool, you need to understand the tasks that are queued and what they are doing. Are they CPU bound? Are they I / O-bound? Your answer will affect how you adjust your application. If you have different task classes with different characteristics, it may make sense to set up multiple work queues for different task classes, so that each pool can be adjusted accordingly.
4. Thread pool size setting
Adjusting the size of the thread pool is basically to avoid two types of errors: too few threads or too many threads. Fortunately, for most applications, the gap between too much and too little is quite wide.
Recall that using threads in applications has two main advantages. Although waiting for slow operations such as I / O, it allows processing to continue, and can take advantage of multiprocessors. In an application running on a computing limit with n processor machines, adding additional threads when the number of threads is close to n may improve the total processing capacity, while adding additional threads when the number of threads exceeds n will not work. In fact, too many threads can even degrade performance because it leads to additional environment switching overhead.
The optimal size of the thread pool depends on the number of processors available and the nature of the tasks in the work queue. If there is only one work queue on a system with N processors, all of which are computational tasks, the maximum CPU utilization will generally be obtained when the thread pool has n or N + 1 threads.
For tasks that may need to wait for I / O to complete (for example, tasks that read HTTP requests from sockets), you need to make the pool larger than the number of available processors, because not all threads are working all the time. By using profiling, you can estimate the waiting time (WT) and service time of a typical request (st). If we call this ratio wt / St, for a system with N processors, about n * (1 + wt / st) threads need to be set to keep the processor fully utilized.
Processor utilization is not the only consideration in sizing the thread pool. As the thread pool grows, you may encounter constraints on the scheduler, available memory, or other system resources, such as the number of sockets, open file handles, or database connections.
5. Several common thread pools
5.1 newCachedThreadPool
Create a cacheable thread pool. If the length of the thread pool exceeds the processing needs, you can flexibly recycle idle threads. If there is no recyclable thread, you can create a new thread.
This type of thread pool is characterized by:
• there is almost no limit on the number of worker threads created (in fact, there is also a limit, the number is interger. Max_value), so you can flexibly add threads to the thread pool.
• if a task is not submitted to the thread pool for a long time, that is, if the worker thread is idle for the specified time (1 minute by default), the worker thread will automatically terminate. After termination, if you submit a new task, the thread pool recreates a worker thread.
• when using cachedthreadpool, you must pay attention to controlling the number of tasks. Otherwise, the system will be paralyzed due to a large number of threads running at the same time.
The example code is as follows:
5.1 newFixedThreadPool
Create a thread pool with a specified number of worker threads. Each time a task is submitted, a worker thread is created. If the number of worker threads reaches the initial maximum number of thread pool, the submitted task is stored in the pool queue.
Fixedthreadpool is a typical and excellent thread pool. It has the advantages of improving program efficiency and saving the overhead when creating threads. However, when the thread pool is idle, that is, when there are no runnable tasks in the thread pool, it will not release the working threads and occupy certain system resources.
The example code is as follows:
Because the thread pool size is 3, and the sleep is 2 seconds after each task outputs index, three numbers are printed every two seconds.
The size of the fixed length route pool is best set according to system resources, such as runtime getRuntime(). availableProcessors()。
5.1 newSingleThreadExecutor
Create a single thread executor, that is, only create a unique worker thread to execute tasks. It will only use a unique worker thread to execute tasks to ensure that all tasks are executed in the specified order (FIFO, LIFO, priority). If this thread ends abnormally, another thread will replace it to ensure sequential execution. The biggest feature of a single working thread is that it can ensure the sequential execution of various tasks, and no more than one thread will be active at any given time.
The example code is as follows:
5.1 newScheduleThreadPool
Create a thread pool with a fixed length, and support timed and periodic task execution, as well as timed and periodic task execution.
The execution is delayed for 3 seconds. The example code of delayed execution is as follows:
It means that it is executed every 3 seconds after a delay of 1 second. The example code of regular execution is as follows:
The above article on the comparison of several commonly used thread pools in Java is all the content shared by Xiaobian. I hope it can give you a reference and support more programming tips.