Key points of this article:
- An Elegant Solution to the Long GC Time Consumption Problem Caused by False References in MySQL Drivers
- The role of virtual references and usage scenarios
- Analysis of Dummy References in MySQL Driver Source Code
contexts
In a previous article, I wrote about the problem of long JVM GC times due to dummy references in the MySQL JDBC driver (You can see it here.), in the driver code (mysql-connector-java version 5.1.38) the NonRegisteringDriver class has a collection of dummy references, connectionPhantomRefs, which is used to store all the connections to the database, and the method is responsible for placing newly created connections into the collection, which accumulates more and more over time. The virtual references accumulate more and more over time, which leads to a longer time consuming processing of virtual references during GC, affecting the throughput of the service:
public ConnectionImpl(String hostToConnectTo, int portToConnectTo, Properties info, String databaseToConnectTo, String url) throws SQLException {
...
(this);
...
}public class NonRegisteringDriver implements Driver {
...
protected static final ConcurrentHashMap<ConnectionPhantomReference, ConnectionPhantomReference> connectionPhantomRefs = new ConcurrentHashMap();
protected static void trackConnection( newConn) {
ConnectionPhantomReference phantomRef = new ConnectionPhantomReference((ConnectionImpl)newConn, refQueue);
(phantomRef, phantomRef);
}
...
}We tried to reduce the number of virtual references by reducing the speed of database connection generation, but the result was not satisfactory. The final solution is to get the collection of virtual references through reflection, and use a timed task to clean up the collection periodically, so as to avoid the GC taking a long time to process the virtual references.
// Clean up connectionPhantomRefs every two hours to minimize impact on mixed GCs
SCHEDULED_EXECUTOR.scheduleAtFixedRate(() -> {
try {
Field connectionPhantomRefs = ("connectionPhantomRefs");
(true);
Map map = (Map) ();
if (() > 50) {
();
}
} catch (Exception e) {
("connectionPhantomRefs clear error!", e);
}
}, 2, 2, );The effect of using timed tasks to clean up dummy references is immediate, the service mixed GC consumes only 10-30 milliseconds for hundreds of millions of requests per day, and the system is very stable, and has been running online for nearly a year without any problems.
Optimization - from brute force to elegant configuration
Recently, a colleague encountered the same problem, using the same version of mysql-connector-java that we use, and checking the code of the latest version (8.0.32) found that there is a new way of handling virtual references to database connections, unlike the old version (5.1.38) in which virtual references are generated for every connection, but instead you can control whether or not they need to be generated by parameters. be generated. ClassAbandonedConnectionCleanupThread The relevant code for this is as follows:
// Static variables are configured by getting
private static boolean abandonedConnectionCleanupDisabled = ("");
public static boolean getBoolean(String name) {
return parseBoolean((name));
}
protected static void trackConnection(MysqlConnection conn, NetworkResources io) {
// Judge the value of the configured attribute to determine whether a dummy reference needs to be generated or not.
if (!abandonedConnectionCleanupDisabled) {
···
ConnectionFinalizerPhantomReference reference = new ConnectionFinalizerPhantomReference(conn, io, referenceQueue);
(reference);
···
}
}The maintainer of mysql-connector-java should have noticed the impact of dummy references on GC and optimized the code to allow users to customize the generation of dummy references.
With this configuration, it is possible to set properties on the startup parameters:
java -jar -=true
Or set the property in the code:
(PropertyDefinitions.SYSP_disableAbandonedConnectionCleanup,"true");
When =true, no dummy references are generated when generating database connections, and there is no effect on GC.
It is recommended to use the first approach, which is a bit more flexible to configure through the startup parameters.
What is a dummy reference
Some readers reading this know that the dummy references generated by mysql-connector-java have some side effects on GC, but they don't quite understand what dummy references are and what they do, so let's expand a little on dummy references here.
Java A dummy reference (Phantom Reference) is a special type of reference in Java that is the weakest type of reference. Unlike other references, phantom references do not affect the life cycle of an object, nor do they affect the garbage collection of an object. Phantom references are mainly used to receive system notifications when an object is being recycled, so that some necessary cleanup can be performed at the time of recycling.
The above definition of dummy references is still difficult to understand, so let's use code to aid in our understanding:
Let's start by generating a dummy reference:
// Virtual reference queue ReferenceQueue<Object> queue = new ReferenceQueue<>(); //Associated Objects Object o = new Object(); // Call the constructor to generate a dummy reference The first parameter is the association object The second parameter is the association queue PhantomReference<Object> phantomReference = new PhantomReference<>(o, queue); // Perform garbage collection (); //Delay to ensure recovery is complete (300L); // When Object o is reclaimed, the dummy reference associated with it can be retrieved from the dummy reference queue, in this case the phantomReference object. Reference<?> poll = ();
The constructor of a dummy reference requires two inputs, the first is the associated object and the second is the dummy reference queue ReferenceQueue. dummy references need to be used in conjunction with ReferenceQueue, so that when the object Object o is garbage-collected, the dummy reference associated with Object o is placed in the ReferenceQueue. The presence or absence of a dummy reference in the ReferenceQueue determines whether the object is being garbage collected.
Let's understand the above definition of dummy reference again, dummy reference will not affect the life cycle of the object and will not affect the garbage collection of the object. If the phantomReference in the above code were an ordinary object, then Object o would not be recycled when () is executed, because the ordinary object holds a strong reference to Object o, and would not be treated as garbage yet. If the phantomReference is a dummy reference, then Object o will be recycled. Then the associated dummy reference will be put into the queue, which is the mechanism to notify the system when the object associated with the dummy reference is recycled.
Some practiced readers will copy the above code and run it, and find that there are no dummy references in the queue after garbage collection. This is because Object o is still on the stack, which is a type of GC Root and will not be garbage collected. We can rewrite it this way:
static ReferenceQueue<Object> queue = new ReferenceQueue<>();
public static void main(String[] args) throws InterruptedException {
PhantomReference<Object> phantomReference = buildReference();
();(100);
(());
}
public static PhantomReference<Object> buildReference() {
Object o = new Object();
return new PhantomReference<>(o, queue);
}Instead of instantiating the associated object Object o in the main method, a buildReference method is used to instantiate it, so that when garbage collection is performed, Object o is out of the stack, is no longer a GC Root, and will be collected as garbage. This allows the associated virtual reference to be taken out of the virtual reference queue for subsequent processing.
Are Associated Objects Really Recycled
After performing garbage collection, we do get the dummy reference out of the dummy reference queue, and we can think about whether the object associated with that dummy reference has really been recycled.
Use a small experiment to explore the answer:
public static void main(String[] args) {
ReferenceQueue<byte[]> queue = new ReferenceQueue<>();
PhantomReference<byte[]> phantomReference = new PhantomReference<>(
new byte[1024 * 1024 * 2], queue);
();(100L);
(());
byte[] bytes = new byte[1024 * 1024 * 4];
}The code generates a dummy reference, the associated object is an array of 2M, and after garbage collection it tries to instantiate an array of 4M. If we get the dummy reference from the dummy reference queue when the associated object has been reclaimed, then the 4M array will be requested normally. (Set heap memory size to 5M -Xmx5m -Xms5m)
The output of the executed code is as follows:
@533ddba
Exception in thread "main" : Java heap space
at (:15)
As you can see from the output, the memory overflowed when 4M memory was requested, so the answer to the question is obvious, the associated object was not really reclaimed and the memory was not freed.
With a slight modification, the dummy reference is set to null before instantiating the new array, so that the associated object can actually be reclaimed and enough memory can be claimed:
public static void main(String[] args) {
ReferenceQueue<byte[]> queue = new ReferenceQueue<>();
PhantomReference<byte[]> phantomReference = new PhantomReference<>(
new byte[1024 * 1024 * 2], queue);
();(100L);
(());
// Virtual references are set to null.
phantomReference = null;
byte[] bytes = new byte[1024 * 1024 * 4];
}If we use dummy references and don't clean them up in a timely manner, it can lead to memory leaks。
Scenarios for using dummy references - mysql-connector-java Dummy references source code analysis
Reading this I'm sure you've learned some of the basics of virtual references, so where are the scenarios for its use?
The most typical scenario is the under-the-hood logic for handling MySQL connections in mysql-connector-java, which I wrote about at the beginning. MySQL connections are wrapped with dummy references, and if a connection object is reclaimed, it receives a notification from the dummy reference queue, and if some of the connections were not closed correctly, a connection closure is performed before reclaiming them.
The AbandonedConnectionCleanupThread class code in mysql-connector-java does not use the native PhantomReference object, but rather the wrapped ConnectionFinalizerPhantomReference, which adds a property NetworkResources, which is used to get the resources that need to be processed from the virtual references in the virtual reference queue. There is also a finalizeResources method in the wrapper class, which is used to close the network connection:
private static class ConnectionFinalizerPhantomReference extends PhantomReference<MysqlConnection> {
//Placement of network resources that need to be post-processed after GC
private NetworkResources networkResources;
ConnectionFinalizerPhantomReference(MysqlConnection conn, NetworkResources networkResources, ReferenceQueue<? super MysqlConnection> refQueue) {
super(conn, refQueue);
= networkResources;
}
void finalizeResources() {
if ( != null) {
try {
();
} finally {
= null;
}
}
}
}AbandonedConnectionCleanupThread implements the Runnable interface, and in the run method loops through the virtual reference queue referenceQueue, and then calls the finalizeResource method to perform post-processing to avoid connection leaks:
public void run() {
while(true) {
try {
...
Reference<? extends MysqlConnection> reference = (5000L);
if (reference != null) {
// forced to ConnectionFinalizerPhantomReference
finalizeResource((ConnectionFinalizerPhantomReference)reference);
}
...
}
}
}
private static void finalizeResource(ConnectionFinalizerPhantomReference reference) {
try {
// Undercover handling of web resources
();
();
} finally {
//Remove dummy references Avoid possible memory overflow
(reference);
}
}If you want to do some post work when some objects are reclaimed, you can refer to mysql-connector-java for some implementation logic.
summarize
This article briefly describes an elegant solution to the long GC time consuming problem caused by dummy references in the MySQL driver. It also describes the role of dummy references according to your own understanding, and describes the use of dummy references in conjunction with the source code of the MySQL driver, so I hope that it will be helpful to you, and please pay attention to the other related articles for more information about GC time consuming optimization of dummy references in the MySQL driver and the source code analysis!