1 使用Reference/ReferenceQueue觀察Class和ClassLoader的解除安裝

在java中，存在著強引用(=)，軟引用(SoftReference)，弱引用(WeakReference)，虛引用(PhantomReference)這4種引用型別。如果一個物件強引用可達，就一定不會被GC回收；而如果一個物件只有軟引用可達，則虛機會保證在out of memory之前會回收該物件；如果一個物件只是弱引用或者虛引用可達，則下一次GC時候就會將其回收。弱引用和虛引用的區別是，當一個物件只是弱引用可達時，它在下一次GC來臨之前還有搶救的餘地，也就是說弱引用仍可以獲得該被引用的物件，然後將其賦值給某一個GC Root可達的變數，此時它就“得救”了；而當一個物件只是虛引用可達時候，它已經無法被搶救了，因為虛引用無法得到被引用物件。

Java還提供了ReferenceQueue用於在一個物件被gc回收掉的時候可以進行額外的處理。ReferenceQueue即是這樣的一個佇列，當一個物件被gc回收之後，其相應的包裝類，即ref物件會被放入佇列中。我們可以從queue中獲取到相應的物件資訊，同時進行額外的處理。比如反向操作，資料清理等。

由上面的介紹可以得知，我們可以使用WeakReference或者PhantomReference來觀察ClassLoader和Class的解除安裝動作，而不會影響到他們的生命週期。先上一段程式碼：SimpleMonitorClassLoader.java

public class SimpleMonitorClassLoader {
	public static void main(String args[]) throws Exception{
		final ReferenceQueue<Object> rq = new ReferenceQueue<Object>();
		final Map<Object, Object> map = new HashMap<>();
		Thread thread = new Thread(() -> {
		    try {
		        WeakReference<byte[]> k;
		        while((k = (WeakReference) rq.remove()) != null) {
		            System.out.println("GC回收了:" + map.get(k));
		        }
		    } catch(InterruptedException e) {
		        //結束迴圈
		    }
		});
		thread.setDaemon(true);
		thread.start();
		
		ClassLoader cl = newLoader();
		Class cls = cl.loadClass("classloader.test.Foo");
		Object obj = cls.newInstance();
		
		
		Object value = new Object();
		
		WeakReference<ClassLoader> weakReference = new WeakReference<ClassLoader>(cl, rq);
		map.put(weakReference, "ClassLoader URLClassLoader");
		WeakReference<Class> weakReference1 = new WeakReference<Class>(cls, rq);
		map.put(weakReference1, "Class classloader.test.Foo");
		WeakReference<Object> weakReference2 = new WeakReference<Object>(obj, rq);
		map.put(weakReference2, "Instance of Foo");
		
		obj=null;
		System.out.println("Set instance null and execute gc!");
		System.gc();
		Thread.sleep(3000);
		cls=null;
		System.out.println("Set class null and execute gc!");
		System.gc();
		Thread.sleep(3000);
		cl=null;
		System.out.println("Set classloader null and execute gc!");
		System.gc();
		Thread.sleep(3000);
	}
	
	static URLClassLoader newLoader() throws Exception{
		URL url = new File("/home/wangd/work/test/wangd/target/classes").toURI().toURL();
		URLClassLoader ucl = new URLClassLoader(new URL[] {url}, null);
	    return ucl;
	}
}
 

Set instance null and execute gc!
GC回收了:Instance of Foo
Set class null and execute gc!
Set classloader null and execute gc!
GC回收了:Class classloader.test.Foo
GC回收了:ClassLoader URLClassLoader

2 Guava FinalizableReference/FinalizableReferenceQueue對Reference/ReferenceQueue的封裝

Guava是一種基於開源的Java庫,對Java進行了封裝,谷歌很多專案使用了它的核心庫。這個庫是為了方便編碼,並減少人為的編碼錯誤。Guava中提供了對Reference/ReferenceQueue的封裝，使其使用起來變得非常簡單。上面的例子使用Guava FinalizableReferen實現的程式碼如下：SimpleMonitorClassLoaderByGuava.class

public class SimpleMonitorClassLoaderByGuava {
	public static void main(String args[]) throws Exception{
		final FinalizableReferenceQueue rq = new FinalizableReferenceQueue();
		ClassLoader cl = newLoader();
		Class cls = cl.loadClass("classloader.test.Foo");
		Object obj = cls.newInstance();
		
		Reference<ClassLoader> weakReference = new FinalizableWeakReference<ClassLoader>(cl, rq) {
	    	//在引用物件被GC回收以後執行一些訂製的業務邏輯
			@Override
			public void finalizeReferent() {
				System.out.println("GC回收了:ClassLoader URLClassLoader");
			}
		};
	    
	    Reference<Class> weakReference1 = new FinalizableWeakReference<Class>(cls, rq) {
	    	//在引用物件被GC回收以後執行一些訂製的業務邏輯
			@Override
			public void finalizeReferent() {
				System.out.println("GC回收了:Class classloader.test.Foo");
			}
		};
	    
	    Reference<Object> weakReference2 = new FinalizableWeakReference<Object>(obj, rq) {
	    	//在引用物件被GC回收以後執行一些訂製的業務邏輯
			@Override
			public void finalizeReferent() {
				System.out.println("GC回收了:Instance of Foo");
			}
		};
	
		obj=null;
		System.out.println("Set instance null and execute gc!");
		System.gc();
		Thread.sleep(3000);
		cls=null;
		System.out.println("Set class null and execute gc!");
		System.gc();
		Thread.sleep(3000);
		cl=null;
		System.out.println("Set classloader null and execute gc!");
		System.gc();
		Thread.sleep(3000);
	}
	
	static URLClassLoader newLoader() throws Exception{
		URL url = new File("/home/wangd/work/test/wangd/target/classes").toURI().toURL();
		URLClassLoader ucl = new URLClassLoader(new URL[] {url}, null);
	    return ucl;
	}
}

執行結果如下：

Set instance null and execute gc!
GC回收了:Instance of Foo
Set class null and execute gc!
Set classloader null and execute gc!
GC回收了:ClassLoader URLClassLoader
GC回收了:Class classloader.test.Foo

3 FinalizableReferenceQueue原始碼分析及其引出的Application ClassLoader無法解除安裝的問題

public class FinalizableReferenceQueue implements Closeable {
	.............................
	
  final ReferenceQueue<Object> queue;
  final PhantomReference<Object> frqRef;
  final boolean threadStarted;
  
  public FinalizableReferenceQueue() {
    queue = new ReferenceQueue<Object>();
    frqRef = new PhantomReference<Object>(this, queue);
    boolean threadStarted = false;
    try {
      startFinalizer.invoke(null, FinalizableReference.class, queue, frqRef);
      threadStarted = true;
    } catch (IllegalAccessException impossible) {
      throw new AssertionError(impossible); // startFinalizer() is public
    } catch (Throwable t) {
      logger.log(
          Level.INFO,
          "Failed to start reference finalizer thread."
              + " Reference cleanup will only occur when new references are created.",
          t);
    }

    this.threadStarted = threadStarted;
  }
  ....................
}

public static void startFinalizer(Class<?> finalizableReferenceClass,
      ReferenceQueue<Object> queue,
      PhantomReference<Object> frqReference) {
    ..............
    Finalizer finalizer = new Finalizer(finalizableReferenceClass, queue, frqReference);
    Thread thread = new Thread(finalizer);
    thread.setName(Finalizer.class.getName());
    thread.setDaemon(true);

    try {
      if (inheritableThreadLocals != null) {
        inheritableThreadLocals.set(thread, null);
      }
    } catch (Throwable t) {
      logger.log(
          Level.INFO,
          "Failed to clear thread local values inherited by reference finalizer thread.",
          t);
    }

    thread.start();
  }

private Finalizer(
      Class<?> finalizableReferenceClass,
      ReferenceQueue<Object> queue,
      PhantomReference<Object> frqReference) {
    this.queue = queue;

    this.finalizableReferenceClassReference =
        new WeakReference<Class<?>>(finalizableReferenceClass);

    this.frqReference = frqReference;
  }

public void run() {
    while (true) {
      try {
        if (!cleanUp(queue.remove())) {
          break;
        }
      } catch (InterruptedException e) {
        // ignore
      }
    }
  }

  private boolean cleanUp(Reference<?> reference) {
    Method finalizeReferentMethod = getFinalizeReferentMethod();
    if (finalizeReferentMethod == null) {
      return false;
    }
    do {
      reference.clear();

      if (reference == frqReference) {
        return false;
      }

      try {
        finalizeReferentMethod.invoke(reference);
      } catch (Throwable t) {
        logger.log(Level.SEVERE, "Error cleaning up after reference.", t);
      }

    } while ((reference = queue.poll()) != null);
    return true;
  }

在FinalizableReferenceQueue原始碼的註釋中提到，如果應用客戶端(client)是通過動態載入或者osgi等載入的則其ClassLoader為Application ClassLoader(與預設的System ClassLoader相區別)，在這種情況下假如client有一個static變數指向FinalizableReferenceQueue例項的話，而FinalizableReferenceQueue在啟動Finalizer Thread的時候直接使用預設的Application Classloader，這樣會形成一個Finalizer Thread->Finalizer.class->Application Classloader->ClientClass.class->FinalizableReferenceQueue的引用關係，導致FinalizableReferenceQueue無法被GC回收，從而導致Finalizer Thread無法停止，最終導致Application Classloader無法被GC回收。

/** Reference to Finalizer.startFinalizer(). */
  private static final Method startFinalizer;

  static {
    Class<?> finalizer =
        loadFinalizer(new SystemLoader(), new DecoupledLoader(), new DirectLoader());
    startFinalizer = getStartFinalizer(finalizer);
  }
...............................
  private static Class<?> loadFinalizer(FinalizerLoader... loaders) {
    for (FinalizerLoader loader : loaders) {
      Class<?> finalizer = loader.loadFinalizer();
      if (finalizer != null) {
        return finalizer;
      }
    }

    throw new AssertionError();
  }

public class GuavaQueueUseByDynamicApp {
	public static void main(String args[]) throws Exception{
		final ReferenceQueue<Object> rq = new ReferenceQueue<Object>();
		final Map<Object, Object> map = new HashMap<>();
		Thread thread = new Thread(() -> {
		    try {
		        WeakReference<byte[]> k;
		        while((k = (WeakReference) rq.remove()) != null) {
		            System.out.println("GC回收了:" + map.get(k));
		        }
		    } catch(InterruptedException e) {
		        //結束迴圈
		    }
		});
		thread.setDaemon(true);
		thread.start();
		
		ClassLoader cl = newLoader();
		System.out.println("Set application contextclassloader as: "+cl);
		Thread.currentThread().setContextClassLoader(cl);	//如果到時候不取消contextclassloader對cl的引用，則cl無法被解除安裝
		
		Class cls = cl.loadClass("jdktest.reference.ExecuteSomethingByFinalizableReferenceQueue");
		Object obj = cls.newInstance();
		Method method = cls.getMethod("doSomething");
        method.invoke(obj);
		
		WeakReference<ClassLoader> weakReference = new WeakReference<ClassLoader>(cl, rq);
		map.put(weakReference, "Application ClassLoader");
		WeakReference<Class> weakReference1 = new WeakReference<Class>(cls, rq);
		map.put(weakReference1, "Class(ExecuteSomethingByFinalizableReferenceQueue)");
		WeakReference<Object> weakReference2 = new WeakReference<Object>(obj, rq);
		map.put(weakReference2, "Instance of ExecuteSomethingByFinalizableReferenceQueue");
		
		obj=null;
		method=null;
		cls=null;
		cl=null;
		System.out.println(Thread.currentThread()+":開始執行GC");
		System.gc();
		Thread.sleep(10000);
		
		Thread.currentThread().setContextClassLoader(null);
		
		System.out.println(Thread.currentThread()+":再次執行GC");
		System.gc();
		Thread.sleep(10000);
		System.out.println(Thread.currentThread()+":第三次執行GC");
		System.gc();
		Thread.sleep(600000);
	}
	
	static URLClassLoader newLoader() throws Exception{
		URL url1 = new File("/home/wangd/data/java/guava-23.0.jar").toURI().toURL();
		URL url2 = new File("/home/wangd/work/test/wangd/target/classes").toURI().toURL();
		URLClassLoader ucl = new URLClassLoader(new URL[] {url1, url2}, null);
	      return ucl;
	}
}

public class ExecuteSomethingByFinalizableReferenceQueue{
	public FinalizableReferenceQueue rq = new FinalizableReferenceQueue();
	public void doSomething() throws Exception{
		System.out.println("Just do something!");
	}
}

Set application contextclassloader as: [email protected]
Just do something!
Thread[main,5,main]:開始執行GC
GC回收了:Instance of ExecuteSomethingByFinalizableReferenceQueue
Thread[main,5,main]:再次執行GC
GC回收了:Class(ExecuteSomethingByFinalizableReferenceQueue)
GC回收了:Application ClassLoader
Thread[main,5,main]:第三次執行GC

假如將ExecuteSomethingByFinalizableReferenceQueue中的變數rq新增static修飾，也就是在FinalizableReferenceQueue原始碼註釋中提到的情況時，執行結果如下：

Set application contextclassloader as: [email protected]
Just do something!
Thread[main,5,main]:開始執行GC
GC回收了:Instance of ExecuteSomethingByFinalizableReferenceQueue
Thread[main,5,main]:再次執行GC
Thread[main,5,main]:第三次執行GC

而當FinalizableReferenceQueue rq是類變數的時候，該變數指向的物件不可被GC回收(Class類直接持有靜態變數的引用)，形成了一個Finalizer Thread->Finalizer.class->etc.->Application Classloader->ExecuteSomethingByFinalizableReferenceQueue.class->FinalizableReferenceQueue的引用路徑，阻止了FinalizableReferenceQueue物件被回收，因此也阻止了Finalizer Thread終止。

4 使用MAT分析dump檔案，找出ClassLoader記憶體洩露的可達路徑

首先在程式執行時候dump出記憶體快照：

$ jps
$ jmap -dump:format=b,file=/home/data/testdump.bin <pid>

從結果列表中可以檢視到該ClassLoader的GC Roots可達路徑:

我們發現居然還有三條可達路徑，第一條是Finalizer Thread的contextClassLoader是Application ClassLoader；第二條和第三條略微複雜，都是跟訪問控制上下文（AccessControlContext）相關：其中一條是Finalizer Thread從建立它的執行緒中繼承過來的訪問控制上下文（AccessControllContext）中持有父執行緒呼叫棧對應的保護域（ProtectionDomain）陣列，其中一個保護域對應的ClassLoader就是Application ClassLoader；另外一條是Finalizer.class的類載入器持有的訪問控制上下文（AccessControllContext）中持有的一個保護域的ClassLoader屬性就是Application ClassLoader。

其中，AccessControlContext、ProtectionDomain都是Java內建安全體系中的概念，關於AccessControlContext、ProtectionDomain的介紹，以及線上程和類載入器建立時為何必須繼承它們建立者的AccessControlContext的討論，請參照《Java Security Architecture--Java安全體系技術文件翻譯(四)》4.3，4.4兩節。

5 解決方案

知道了問題所在，我們接下來就考慮解決方案。其實從日常使用的角度講，該極端問題一般不會遇到，另外如果在相關的reference統統被處理掉（finalized）以後顯式的去掉對FinalizableReferenceQueue的強引用，或者執行其close方法就肯定會達到FinalizableReferenceQueue的回收和Finalizer Thread的停止。但是假如就在該極端情況下，又不取消強引用及執行其close方法，有辦法讓GC自動回收掉FinalizableReferenceQueue從而實現Finalizer Thread的停止和Application ClassLoader的解除安裝嗎？

public static void startFinalizer(
	      Class<?> finalizableReferenceClass,
	      ReferenceQueue<Object> queue,
	      PhantomReference<Object> frqReference) {
	    if (!finalizableReferenceClass.getName().equals(FINALIZABLE_REFERENCE)) {
	      throw new IllegalArgumentException("Expected " + FINALIZABLE_REFERENCE + ".");
	    }

	    Finalizer finalizer = new Finalizer(finalizableReferenceClass, queue, frqReference);
	    Thread thread = new Thread(finalizer);
	    thread.setName(Finalizer.class.getName());
	    thread.setDaemon(true);

	    try {
	      if (inheritableThreadLocals != null) {
	        inheritableThreadLocals.set(thread, null);
	      }
	      
	      //消除本執行緒與啟動本執行緒的類的類載入器之間的關聯
	      Field inheritedAccessControlContext = Thread.class.getDeclaredField("inheritedAccessControlContext");
	      inheritedAccessControlContext.setAccessible(true);
	      inheritedAccessControlContext.set(thread, null);
	      
	      
	    } catch (Throwable t) {
	      logger.log(
	          Level.INFO,
	          "Failed to clear thread local values inherited by reference finalizer thread.",
	          t);
	    }
	    //消除本執行緒的contextClassLoader與啟動本執行緒的類的類載入器之間的關聯
	    thread.setContextClassLoader(null);

	    thread.start();
	  }

其中打了註釋的部分為我們新增的程式碼，目的就是切斷上一步找出來的第一條和第二條路徑。我們使用反射機制是因為Thread並沒有提供公開的API來取消繼承父執行緒中訪問控制上下文的行為。

另外，修改FinalizableReferenceQueue的內部靜態類DecoupledLoader中的方法newLoader()，使得新建立的URLClassLoader與Application ClassLoader不再關聯，程式碼如下：

URLClassLoader newLoader(URL base) {
	    	URLClassLoader cl = new URLClassLoader(new URL[] {base}, null);
	    	
	    	try{
	    		//消除新建立的類載入器與本類的類載入器之間的關聯
	    		Field acc = URLClassLoader.class.getDeclaredField("acc");
	    		acc.setAccessible(true);
	    		acc.set(cl, null);
	    	}catch(Exception e){
	    		e.printStackTrace();
	    	}
	      return cl;
	    }

修改完畢使用重新打包後的Guava庫，執行結果如下：

Set application contextclassloader as: [email protected]
----------FinalizableReferenQueue ClassLoader:[email protected]
----------Finalizer ClassLoader:[email protected]
finalizer thread's classloader is:[email protected]
Just do something!
Thread[com.common.finalizablereference.Finalizer,5,main] start run a deamon thread!
Thread[main,5,main]:開始執行GC
GC回收了:Instance of ExecuteSomethingByFinalizableReferenceQueue
Thread[main,5,main]:再次執行GC
GC回收了:Class(ExecuteSomethingByFinalizableReferenceQueue)
Thread[com.common.finalizablereference.Finalizer,5,main] Class has been unloaded!!
Thread[com.common.finalizablereference.Finalizer,5,main] thread loop break;
Thread[com.common.finalizablereference.Finalizer,5,main] run out a deamon thread!
GC回收了:Application ClassLoader
Thread[main,5,main]:第三次執行GC

為了方便檢視，在Finalizer Thread的run方法中添加了一些輸出，這樣可以方便的看到，當第一次執行GC的時候只回收了instance,因為main thread的contextClassLoader還關聯著Application ClassLoader,因此大家都不會被回收，Finalizer Thread也不會停止。而第二次GC的時候，由於執行Thread.currentThread().setContextClassLoader(null)；因此Application ClassLoader，ExecuteSomethingByFinalizableReferenceQueue.class以及static屬性指向的FinalizableReferenceQueue物件都被回收了，因此Finalizer Thread的迴圈也被打破，執行緒也成功關閉了。

6 總結

本文的程式碼及測試結果均為作者在自己的環境中實驗得出，結論也只是一家之言並沒有嚴格考證，若有不當之處，還請大家多多指正。