Design and practice sharing of dynamic thread pool based on Nacos

1. Preface

In a distributed system architecture, thread pools are an important tool for resource scheduling. Traditional fixed-parameter thread pools perform well in scenarios with stable traffic, but when faced with the tidal traffic characteristics of modern Internet services, there are often problems of wasted resources or insufficient processing capabilities.

For example, visits surged during e-commerce promotions, and were almost idle during normal periods. If the fixed thread pool is too large, it will cause a lot of thread resources to be wasted during the idle period; if it is too small, it will not respond to the request in time during the peak period, resulting in queueing or timeout failure. To this end, in order to ensure the throughput during peak periods and the resource utilization during trough periods, we need a thread pool that can automatically scale and shrink according to business load during operation.

With the help of Nacos Configuration Center, we can distribute the core parameters of the thread pool (such as the number of core threads, maximum number of threads, queue capacity, idle recycling time, etc.) to the client, and achieve hot updates through configuration refresh, so that it can take effect without restarting the application.

2. Background analysis of the use of dynamic thread pools

2.1 Characteristics of fluctuations in request volume

Burst traffic: The business system may receive a large number of concurrent requests in a short time, such as flash sale, group buying and other promotional activities. At this time, the thread pool needs to be expanded quickly to ensure response performance.
Idle resources during idle period: During night or business trough periods, a large number of threads in the thread pool are idle. If not recycled, memory andCPUSwitching overhead;

2.2 Limitations of fixed thread pools

Waste of resources：(n)N threads are maintained at any time and idle threads cannot be automatically recycled;
Response bottleneck: When the number of tasks exceeds n, the excess tasks can only be queued. If the queue queue is configured as bounded, the caller may be discarded or blocked directly;

2.3 Advantages of dynamic thread pool

Automatic capacity expansion: When the task submission rate exceeds the number of core threads and the queue is full, the thread pool will continue to create new threads until the maximum number of threads is reached
Automatic shrinkage: By callingallowCoreThreadTimeOut(true), causing the core thread to exceed the idle timekeepAliveTimeIt can also be recycled later;

3. Nacos dependency and startup configuration

Introducing Spring Cloud Alibaba Nacos dependencies into your project:

<dependency>
  <groupId></groupId>
  <artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
  <version>2023.0.1.0</version>
</dependency>

existMedium configurationNacosServer address and application name:

spring:
  application:
    name: dynamic-threadpool-demo
  cloud:
    nacos:
      config:
        server-addr: 127.0.0.1:8848
        file-extension: yaml
        refresh-enabled: true

existNacosConsole creation Data ID:, content example:

threadpool:
  coreSize: 5
  maxSize: 20
  queueCapacity: 100
  keepAliveSeconds: 60

This file stores various parameters of the thread pool. You can then directly modify and issue application instances in the console in real time.

4. Initialize the thread pool and load the initial parameters

Define the thread pool configuration class and use@ConfigurationPropertiesReadNacosConfiguration:

@Component
@RefreshScope
@ConfigurationProperties(prefix = "threadpool")
public class ThreadPoolProperties {
    private int coreSize;
    private int maxSize;
    private int queueCapacity;
    private long keepAliveSeconds;
    // getters & setters
}

Injection into the factory classThreadPoolProperties, and rebuild or adjust existing thread pool instances when configuration changes:

@Component
public class DynamicThreadPoolManager {
    private volatile ThreadPoolExecutor executor;
    private final ThreadPoolProperties props;

    public DynamicThreadPoolManager(ThreadPoolProperties props) {
         = props;
         = createExecutor(props);
    }

    @NacosConfigListener(dataId = "${}.yaml", timeout = 5000)
    public void onChange(String newContent) throws JsonProcessingException {
        ThreadPoolProperties updated = new ObjectMapper()
            .readValue(newContent, );
        (());
        (());
        ((), );
        // If you need to modify the queue capacity, rebuild the executor    }

    private ThreadPoolExecutor createExecutor(ThreadPoolProperties p) {
        return new ThreadPoolExecutor(
            (), (),
            (), ,
            new LinkedBlockingQueue&lt;&gt;(()),
            r -&gt; new Thread(r, "dyn-pool-" + ()),
            new ()
        );
    }

    public void submit(Runnable task) {
        (task);
    }
}

Start the test and callCommandLineRunnerImplement some initialization operations after the project is started. The code is as follows：

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        (, args);
    }

    @Bean
    public CommandLineRunner demo(DynamicThreadPoolManager manager) {
        return args -> {
            for (int i = 0; i < 50; i++) {
                int id = i;
                (() -> {
                    (().getName() + " - Task " + id);
                });
            }
        };
    }
}

5. Testing and Verification

start upNacos ServerWith this sample project, observe the thread pool parameter initialization information in the log
ReviseNacosParameters in (such ascoreSize、maxSize), click Refresh, the application will automatically trigger the callback and adjust the thread pool settings without restarting
It can be combined with monitoring tools (Prometheus/Grafana)()、getActiveCount()、getQueue().size()Real-time monitoring and comparison verification of other indicators

6. Conclusion

ByNacosConfiguration Center andThreadPoolExecutorIn combination, we have successfully achieved thermal updates and dynamic adjustments of thread pool parameters, meeting the needs of automatic scaling and capacity in high concurrency scenarios. In practice, it has been further extended to more scenarios, such as message queue consumers, asynchronous task execution, etc., bringing higher flexibility and operability to the microservice system.

The above is the detailed content shared by the design and practice of implementing dynamic thread pools based on Nacos. For more information about the implementation of Nacos dynamic thread pools, please pay attention to my other related articles!