Environment: Spring 5.3.23
1. Overview
In Spring Cloud system, circuit breakers include Hystrix, Resilience4j, Sentinel and other components. Their core function is that when a service is unavailable, the circuit breaker will block related faults and return a user-preset fallback.
Specifically, circuit breakers have the following functions:
- Prevent upward delivery of failures: Monitor and protect the health of the service.
- Failure to failures and positively respond to: fallback and elegantly downgrade.
- Three fault tolerance methods are provided to help achieve goals: resource isolation, circuit breaking and downgrade.
In general, the core role of the circuit breaker is to enhance the elasticity of the distributed system and avoid cascading failures to improve the overall availability of the system.
2. Implementation plan
We will implement circuit breaker functionality by using AOP and custom annotations. Add annotations on the key methods according to your needs, then intercept these annotations through AOP at runtime and execute the corresponding circuit breaker logic.
The main function of the circuit breaker is to prevent the spread of faults and protect the stability of the system. When a service fails, the circuit breaker can quickly interrupt the connection to the service and return a preset fallback response, thereby avoiding the impact of the failure on the entire system.
Through the combination of custom annotations and AOP, we can implement the following functions:
Add custom annotations on the required interfaces, which can include configuration information related to the circuit breaker, such as: error times, time window, etc.
Intercept these annotations through AOP and create circuit breakers dynamically at runtime.
When the service is called, the circuit breaker will determine whether the connection needs to be interrupted or the fallback response is returned based on the configured logic.
If the service is normal, the circuit breaker will not perform any operations; if the service fails, the circuit breaker will process according to the preset logic.
3. Code implementation
3.1 Custom annotations
AOP will only intercept the methods or classes of the annotation.
@Target({, })
@Retention()
@Inherited
public @interface PackFuse {
/**Downgrade method*/
String fallback() default "" ;
/**Number of failures*/
int fails() default 5 ;
/**Window time:s*/
int windowSize() default 10 ;
}3.2 Customized circuit breaker status
There are several states of circuit breakers:
- Closed: By default, the circuit breaker is closed, allowing remote service calls to proceed normally.
- Open: When the number of remote service calls fails to reach the preset threshold, the circuit breaker will automatically turn on, interrupt all calls with the service, and return a fallback response.
- Half-Open: After a period of time, the circuit breaker will automatically switch from the Open state to the Half-Open state. In Half-Open state, the circuit breaker attempts a small number of requests to test whether the service has been restored. If the test request is successful, the circuit breaker will automatically close and return to the Closed state; otherwise, the Half-Open state will be maintained and will be converted to the Open state again if the specified number of errors is exceeded.
Status definition
public enum EnumState {
CLOSE, HALF_OPEN, OPEN ;
}Each circuit breaker will have its own state
public class PackFuseState {
/**Current status*/
private EnumState state = ;
/**Number of failures*/
private AtomicInteger failCount = new AtomicInteger(0) ;
/**Maximum number of failures*/
private int maxFailCount = 5 ;
/**Window size; default reset every 10 seconds*/
private int windowTime = 10 ;
private static final ThreadPoolExecutor executor = new ThreadPoolExecutor(2, 2, 60, , new ArrayBlockingQueue<>(1)) ;
private Object lock = new Object() ;
public PackFuseState(int maxFailCount, int windowTime) {
= maxFailCount ;
= windowTime ;
(() -> {
while(true) {
if (state == ) {
try {
(windowTime) ;
if (state == ) {
(0) ;
}
} catch (InterruptedException e) {
() ;
}
} else {
synchronized (lock) {
try {
() ;
} catch (InterruptedException e) {
() ;
}
}
}
}
}) ;
}
public EnumState getState() {
return state;
}
public void setState(EnumState state) {
= state;
}
public AtomicInteger getFailCount() {
return failCount;
}
public void setFailCount(AtomicInteger failCount) {
= failCount;
}
public int getwindowTime() {
return windowTime;
}
public void setwindowTime(int windowTime) {
= windowTime;
}
public PackFuseState addFailCount() {
int count = () ;
if (count >= maxFailCount) {
() ;
(() -> {
try {
(windowTime) ;
setState(EnumState.HALF_OPEN) ;
(0) ;
} catch (InterruptedException e) {
() ;
}
}) ;
}
return this ;
}
public PackFuseState closeState() {
() ;
(0) ;
return this ;
}
public Object getLock() {
return lock;
}
}3.3 Definition of sections
This section intercepts all methods marked with @PackFuse annotation
@Aspect
@Component
public class PackFuseAspect {
private static final Map<String, PackFuseState> META_HOLDER_MAP = new ConcurrentHashMap<>() ;
private static final Map<String, Object> FALLBACK = new ConcurrentHashMap<>() ;
private static final String DEFAULT_RET_DATA = "Service not available" ;
@Pointcut("@annotation(fuse)")
private void fuse(PackFuse fuse) {}
@Around("fuse(fuse)")
public Object packFuse(ProceedingJoinPoint pjp, PackFuse fuse) {
MethodSignature joinPointObject = (MethodSignature) () ;
Class<?> targetType = () ;
Method method = () ;
String targetKey = getKey(targetType, method);
String fallback = () ;
if (!(targetKey)) {
if ((fallback)) {
try {
Method fallbackMethod = (fallback) ;
(targetKey, (())) ;
} catch (Exception e) {
() ;
}
} else {
(targetKey, DEFAULT_RET_DATA) ;
}
}
int fails = () ;
int windowSize = () ;
PackFuseState fuseState = null ;
try {
fuseState = META_HOLDER_MAP.computeIfAbsent(targetKey, key -> new PackFuseState(fails, windowSize)) ;
switch (()) {
case CLOSE:
return () ;
case HALF_OPEN:
Random rd = new Random() ;
int c = (fails) ;
if (c >= (fails / 2)) {
Object ret = () ;
() ;
synchronized (()) {
().notifyAll() ;
}
return ret ;
}
return (targetKey) ;
case OPEN:
return (targetKey) ;
}
} catch (Throwable e) {
() ;
}
return (targetKey) ;
}
private String getKey(Class<?> targetType, Method method) {
StringBuilder builder = new StringBuilder();
(());
('#').append(()).append('(');
if (().length > 0) {
(() - 1);
}
return (')').toString().replaceAll("[^a-zA-Z0-9]", "") ;
}
}The above implements a simple circuit breaker function.
The circuit breaker function was successfully implemented by using AOP+ custom annotations. This method gives us great flexibility and scalability, and can easily isolate specific services to avoid failure spread, and protect the stability of the entire system. At the same time, through custom annotations, we can clearly define the configuration and logic of the circuit breaker, making the code easier to read and maintain.
Here is just a very simple example to give you an implementation idea. You can enrich the functions according to your own ideas or combined with Hystrix implementation.
Summarize
The above is personal experience. I hope you can give you a reference and I hope you can support me more.