Thread pool tuning
Open Liberty provides a self-tuning algorithm that controls the size of its thread pool. Although you do not need to manually tune the thread pool for most applications, in rare cases you might need to configure thread pool settings in your server.xml file.
All the application code in Open Liberty runs in a single thread pool that is called the default executor. The size of this pool is set by a self-tuning controller, which can manage a wide range of workloads. The default executor pool is the set of threads where your application code runs. Open Liberty uses other threads for tasks like serving the OSGi framework, collecting JVM garbage, and providing Java NIO transport functions. However, these threads are not directly relevant to application performance and most of them are not configurable.
Thread pool tuning behavior
The Open Liberty thread pool self-tuning process runs every 1.5 seconds. The thread pool controller maintains a set of data about the thread pool performance from the time the server started. The throughput is determined by the number of tasks that are completed by the controller each cycle. The controller records the throughput for the various pool sizes that were previously tried. This historical throughput data is then compared to throughput for the current cycle to decide the optimal pool size. At each cycle, the pool size can be incrementally increased or decreased, or left unchanged.
In some cases, no historical data is available to guide the decision. For example, if the pool is growing with each cycle and the current cycle is at the largest size so far, no data exists about throughput for larger pool size. In such a case, the controller decides at random whether to increase the size of the pool. Then, it readjusts for the next cycle based on the results of that decision. This process is analogous to a human thread pool tuner who tries various thread pool sizes to see how they perform and decides on an optimal value for the configuration and workload.
During each 1.5-second cycle, the thread pool controller runs through the following self-tuning operations:
Wakes up and checks whether the threads in the pool are hung. If tasks are in the queue and no tasks were completed in the previous cycle, the controller considers the threads to be hung. In this case, the controller increases the thread pool size as specified by settings and skips to step 5.
Updates the historical data set with the number of tasks that completed in the most recent controller cycle. Performance is recorded as a weighted moving average for each pool size. This performance reflects historical results but adjusts quickly to changing workload characteristics.
Uses historical data to predict whether performance would be better at smaller or larger pool size. If no historical data exists for the smaller or larger pool size, the thread pool controller decides whether to increase or shrink the size of the pool.
Increases or decreases the pool size within the bounds that are specified in settings, or leaves it unchanged, based on predicted performance.
Goes back to sleep.
Various factors other than the thread pool size can affect throughput in the Open Liberty server. The relationship between pool size and observed throughput is not perfectly smooth or continuous. Therefore, to improve the predictions that are derived from the historical throughput data, the controller considers not just the closest larger and smaller pool size, but also several increments in each direction.
Hang resolution
In some application scenarios, all the threads in the pool can become blocked by tasks that must wait for other work to finish before they can run. In these cases, the server can become hung at a certain pool size. To resolve this situation, the thread pool controller enters a hang resolution mode.
Hang resolution adds threads to the pool to allow the server to resume normal operation. Hang resolution also shortens the controller cycle duration to break the deadlock quickly.
When the controller observes that tasks are being completed again, normal operation resumes. The controller cycle returns to its normal duration, and pool size is adjusted based on the usual throughput criteria.
Manual thread pool tuning
In most environments, configurations, and workloads, the Open Liberty thread pool does not require manual configuration or tuning. The thread pool self-tunes to determine how many threads are needed to provide optimal server throughput. The thread pool controller continually adjusts the number of threads in the pool within the defined bounds for the coreThreads and maxThreads attributes. However, in rare cases, you might need to explicitly configure an executor element and adjust the coreThreads and maxThreads attributes to particular values in your server.xml file.
The following sections describe these attributes and provide examples of conditions under which they might need to be manually tuned.
coreThreads
This attribute specifies the minimum number of threads in the pool. The minimum value for this attribute is 4. Open Liberty creates a new thread for each piece of offered work until the number of threads equals the value of this attribute. If thecoreThreadsattribute is not configured, it defaults to a multiple of the number of hardware threads available to the Open Liberty process.If Open Liberty is running in a shared environment, the thread pool controller cannot account for other processes with which it is sharing the available CPUs. In these cases, the default value of the
coreThreadsattribute might cause Open Liberty to create more threads than is optimal, considering the other processes that are competing for CPU resources. In this situation, you can limit thecoreThreadsattribute to a value that reflects only the proportion of the CPU resources that Open Liberty needs to run.maxThreads
This attribute specifies the maximum number of threads in the pool. The default value is -1, which is equal toMAX_INT, or effectively unlimited.Some environments set a hard limit on the number of threads that a process can create. Currently, Open Liberty has no way to know whether such a cap applies, or what the value is. If Open Liberty is running in a thread-limited environment, the operator can configure the
maxThreadsattribute to an acceptable value.
The Open Liberty thread pool controller is designed to handle a wide range of workloads and configurations. In some edge cases, you might need to adjust the coreThreads and maxThreads attributes. However, try the default behavior first to make sure you need to make adjustments.
