Thread context class loader contextclassloader memory leak
premise
Today (January 18, 2020), when writing nety related code, we traced two issues related to memory leakage of thread context class loader contextclassloader from threaddeathwatcher and globaleventexecutor in nety source code:
The two issues were raised by two predecessors on December 2017. They described the same kind of problems. Finally, they were adopted by the person in charge of netty, and the corresponding problems were repaired to close the issue. Based on the contents described in these two issues, we will redo the hidden danger of memory leakage of contextclassloader.
Classloader related content
In some scenarios, it is necessary to implement the hot deployment and unloading of classes, such as defining an interface, and then dynamically passing in the implementation of code from the outside.
Since all classes of non JDK class libraries are loaded by appclassloader during application startup, There is no way to load an existing class file with the same name under a non classpath through appclassloader (for a classloader, each class file can only be loaded once to generate a unique class). Therefore, in order to dynamically load classes, you must use a completely different custom classloader instance to load the same class file each time, or use the same custom classloader instance to load different class files. Here is a simple example of class hot deployment:
// 此文件在项目类路径
package club.throwable.loader;
public class DefaultHelloService implements HelloService {
@Override
public String sayHello() {
return "default say hello!";
}
}
// 下面两个文件编译后放在I盘根目录
// I:\\DefaultHelloService1.class
package club.throwable.loader;
public class DefaultHelloService1 implements HelloService {
@Override
public String sayHello() {
return "1 say hello!";
}
}
// I:\\DefaultHelloService2.class
package club.throwable.loader;
public class DefaultHelloService2 implements HelloService {
@Override
public String sayHello() {
return "2 say hello!";
}
}
// 接口和运行方法
public interface HelloService {
String sayHello();
static void main(String[] args) throws Exception {
HelloService helloService = new DefaultHelloService();
System.out.println(helloService.sayHello());
ClassLoader loader = new ClassLoader() {
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
String location = "I:\\DefaultHelloService1.class";
if (name.contains("DefaultHelloService2")) {
location = "I:\\DefaultHelloService2.class";
}
File classFile = new File(location);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
try {
InputStream stream = new FileInputStream(classFile);
int b;
while ((b = stream.read()) != -1) {
outputStream.write(b);
}
} catch (IOException e) {
throw new IllegalArgumentException(e);
}
byte[] bytes = outputStream.toByteArray();
return super.defineClass(name,bytes,bytes.length);
}
};
Class<?> klass = loader.loadClass("club.throwable.loader.DefaultHelloService1");
helloService = (HelloService) klass.newInstance();
System.out.println(helloService.sayHello());
klass = loader.loadClass("club.throwable.loader.DefaultHelloService2");
helloService = (HelloService) klass.newInstance();
System.out.println(helloService.sayHello());
}
}
// 控制台输出
default say hello!
1 say hello!
2 say hello!
If too many classloader instances and class instances are created, it will occupy a lot of memory. If the above conditions cannot be met, that is, these classloader instances and class instances are stacked and cannot be unloaded, This will lead to a memory leak, which has serious consequences and may exhaust the physical memory of the server, because the meta information related to JDK1.8 + class exists in the meta space, and the meta space uses native memory.
Contextclassloader in thread
Contextclassloader actually refers to the thread class java The contextclassloader attribute in lang. thread, which is the classloader type, that is, the classloader instance. In some scenarios, the JDK provides some standard interfaces that need to be implemented by a third-party provider (the most common is SPI, service provider interface, such as java.sql.driver). These standard interface classes are loaded by the boot class loader, but the implementation classes of these interfaces need to be imported from the outside and do not belong to the JDK's native class library, Cannot load with boot class loader. In order to solve this dilemma, thread context classloader is introduced. Thread Java Lang. thread instance will call thread #init() method during initialization. The core code blocks related to thread class and contextclassloader are as follows:
// 线程实例的初始化方法,new Thread()的时候一定会调用
private void init(ThreadGroup g,Runnable target,String name,long stackSize,AccessControlContext acc,boolean inheritThreadLocals) {
// 省略其他代码
Thread parent = currentThread();
// 省略其他代码
if (security == null || isCCLOverridden(parent.getClass()))
this.contextClassLoader = parent.getContextClassLoader();
else
this.contextClassLoader = parent.contextClassLoader;
// 省略其他代码
}
public void setContextClassLoader(ClassLoader cl) {
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
sm.checkPermission(new RuntimePermission("setContextClassLoader"));
}
contextClassLoader = cl;
}
@CallerSensitive
public ClassLoader getContextClassLoader() {
if (contextClassLoader == null)
return null;
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
ClassLoader.checkClassLoaderPermission(contextClassLoader,Reflection.getCallerClass());
}
return contextClassLoader;
}
First, clarify two points:
After analysis, the author only wants to explain one conclusion: the thread context class loader of the descendant thread will inherit the thread context class loader of the parent thread. In fact, the word inheritance is not too accurate here. To be exact, the thread context class loader of the descendant thread should be exactly the same as the context class loader of the parent thread. If both autonomous threads are derived, Then they are all application class loaders. This conclusion can be verified (the following example runs in jdk8):
public class ThreadContextClassLoaderMain {
public static void main(String[] args) throws Exception {
AtomicReference<Thread> grandSonThreadReference = new AtomicReference<>();
Thread sonThread = new Thread(() -> {
Thread thread = new Thread(()-> {},"grand-son-thread");
grandSonThreadReference.set(thread);
},"son-thread");
sonThread.start();
Thread.sleep(100);
Thread main = Thread.currentThread();
Thread grandSonThread = grandSonThreadReference.get();
System.out.println(String.format("ContextClassLoader of [main]:%s",main.getContextClassLoader()));
System.out.println(String.format("ContextClassLoader of [%s]:%s",sonThread.getName(),sonThread.getContextClassLoader()));
System.out.println(String.format("ContextClassLoader of [%s]:%s",grandSonThread.getName(),grandSonThread.getContextClassLoader()));
}
}
The console output is as follows:
ContextClassLoader of [main]:sun.misc.Launcher$AppClassLoader@18b4aac2
ContextClassLoader of [son-thread]:sun.misc.Launcher$AppClassLoader@18b4aac2
ContextClassLoader of [grand-son-thread]:sun.misc.Launcher$AppClassLoader@18b4aac2
It confirms the previous conclusion that the thread context class loaders of main thread, child thread and grandson thread are all of appclassloader type and point to the same instance sun misc. Launcher$ AppClassLoader@18b4aac2 。
The hidden danger of memory leakage caused by improper setting of contextclassloader
As long as there are a large number of scenarios of hot loading and unloading dynamic classes, you need to be vigilant against memory leakage caused by improper setting of descendant thread contextclassloader. Draw a picture to make it clear:
A user-defined class loader is set in the parent thread to load dynamic classes. When a child thread is created, the user-defined class loader of the parent thread is directly used, resulting in that the user-defined class loader has been strongly referenced by the child thread. Combined with the previous analysis of class unloading conditions, all dynamic classes loaded by the user-defined class loader cannot be unloaded, Caused a memory leak. Here is a modification based on the example above:
public interface HelloService {
String sayHello();
BlockingQueue<String> CLASSES = new LinkedBlockingQueue<>();
BlockingQueue<String> EVENTS = new LinkedBlockingQueue<>();
AtomicBoolean START = new AtomicBoolean(false);
static void main(String[] args) throws Exception {
Thread thread = new Thread(() -> {
ClassLoader loader = new ClassLoader() {
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
String location = "I:\\DefaultHelloService1.class";
if (name.contains("DefaultHelloService2")) {
location = "I:\\DefaultHelloService2.class";
}
File classFile = new File(location);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
try {
InputStream stream = new FileInputStream(classFile);
int b;
while ((b = stream.read()) != -1) {
outputStream.write(b);
}
} catch (IOException e) {
throw new IllegalArgumentException(e);
}
byte[] bytes = outputStream.toByteArray();
Class<?> defineClass = super.defineClass(name,bytes.length);
try {
EVENTS.put(String.format("加载类成功,类名:%s",defineClass.getName()));
} catch (Exception ignore) {
}
return defineClass;
}
};
Thread x = new Thread(() -> {
try {
if (START.compareAndSet(false,true)) {
Thread y = new Thread(() -> {
try {
for (; ; ) {
String event = EVENTS.take();
System.out.println("接收到事件,事件内容:" + event);
}
} catch (Exception ignore) {
}
},"Y");
y.setDaemon(true);
y.start();
}
for (; ; ) {
String take = CLASSES.take();
Class<?> klass = loader.loadClass(take);
HelloService helloService = (HelloService) klass.newInstance();
System.out.println(helloService.sayHello());
}
} catch (Exception ignore) {
}
},"X");
x.setContextClassLoader(loader);
x.setDaemon(true);
x.start();
});
thread.start();
CLASSES.put("club.throwable.loader.DefaultHelloService1");
CLASSES.put("club.throwable.loader.DefaultHelloService2");
Thread.sleep(5000);
System.gc();
Thread.sleep(5000);
System.gc();
Thread.sleep(Long.MAX_VALUE);
}
}
Console output:
接收到事件,事件内容:加载类成功,类名:club.throwable.loader.DefaultHelloService1
1 say hello!
接收到事件,类名:club.throwable.loader.DefaultHelloService2
2 say hello!
Open visualvm, dump the memory snapshot of the corresponding process, and execute GC several times. It is found that all dynamic classes have not been unloaded (here, unless thread y is actively terminated to release the custom classloader, it is never possible to release the strong reference). The above conclusion is verified.
Of course, only two dynamic classes are loaded here. Under special scenarios, such as online coding and running code, dynamic compilation and dynamic class loading may be extremely frequent. If a memory leak similar to the above occurs, the server memory may be exhausted.
Solution
Referring to the two issues, there are basically two solutions (or preventive measures):
// ThreadDeathWatcher || GlobalEventExecutor
AccessController.doPrivileged(new PrivilegedAction<Void>() {
@Override
public Void run() {
watcherThread.setContextClassLoader(null);
return null;
}
});
Summary
This article is an in-depth study recently. In the final analysis, the hidden danger of contextclassloader memory leak is that some references that need to be released after the method stack exits cannot be released due to improper use of references. Sometimes this problem is hidden deeply. Once the same problem is hit and in the concurrent scenario, the problem of memory leakage will worsen very quickly. Such problems are classified as performance optimization, which is a very large topic. Similar problems should be encountered in the future. These experiences hope to have a positive effect on the future.
reference material:
My personal blog
(c-2-d e-a-20200119)
The official account of Technology (Throwable Digest), which is not regularly pushed to the original technical article (never copied or copied):
Entertainment official account ("sand sculpture"), select interesting sand sculptures, videos and videos, push them to relieve life and work stress.