Java直接内存分配与释放原理

快来打我* 2022-02-25 14:53 362阅读 0赞

**前言**

在Java中分配直接内存大有如下三种主要方式：

1.Unsafe.allocateMemory()  
2.ByteBuffer.allocateDirect()  
3.native方法

**Unsafe类**

> Java提供了Unsafe类用来进行直接内存的分配与释放
> 
> > public native long allocateMemory(long var1); public native void  
> > freeMemory(long var1);
> 
> 示例 public class DirectMemoryMain \{  
> public static void main(String\[\] args) throws InterruptedException \{  
> Unsafe unsafe = getUnsafe();  
> while (true) \{  
> for (int i = 0; i < 10000; i++) \{  
> long address = unsafe.allocateMemory(10000);  
> // System.out.println(address);  
> // unsafe.freeMemory(address);  
> \}  
> Thread.sleep(1);  
> \}  
> \}
> 
>     // Unsafe无法直接使用，需要通过反射来获取
>     private static Unsafe getUnsafe() {
>         try {
>             Class clazz = Unsafe.class;
>             Field field = clazz.getDeclaredField("theUnsafe");
>             field.setAccessible(true);
>             return (Unsafe) field.get(null);
>         } catch (IllegalAccessException | NoSuchFieldException e) {
>             throw new RuntimeException(e);
>         }
>     } }

下面为这段代码的演示效果，其中JVM最大内存设为64M，而真实内存则可以无限增长。  
![在这里插入图片描述][20190402110412453.]  
**DirectByteBuffer类**  
**内存分配**  
虽然Unsafe可以通过反射调用来进行内存分配，但是按照其设计方式，它并不是给开发者来使用的，而且Unsafe里面的方法也十分原始，更像是一个底层设施。而其上层的封装则是DirectByteBuffer，这个才是最终留给开发者使用的。DirectByteBuffer的分配是通过ByteBuffer.allocateDirect(int capacity)方法来实现的。  
DirectByteBuffer申请内存的源码如下：

> DirectByteBuffer(int cap) \{  
> super(-1, 0, cap, cap);
> 
>     // 计算需要分配的内存大小
>     boolean pa = VM.isDirectMemoryPageAligned();
>     int ps = Bits.pageSize();
>     long size = Math.max(1L, (long)cap + (pa ? ps : 0));
>     
>     // 告诉内存管理器要分配内存
>     Bits.reserveMemory(size, cap);
>     
>     // 分配直接内存
>     long base = 0;
>     try {
>         base = unsafe.allocateMemory(size);
>     } catch (OutOfMemoryError x) {
>         Bits.unreserveMemory(size, cap);
>         throw x;
>     }
>     unsafe.setMemory(base, size, (byte) 0);
>     
>     // 计算内存的地址
>     if (pa && (base % ps != 0)) {
>         address = base + ps - (base & (ps - 1));
>     } else {
>         address = base;
>     }
>     
>     // 创建Cleaner
>     cleaner = Cleaner.create(this, new Deallocator(base, size, cap));
>     att = null; }

整个DirectByteBuffer分配过程中，比较需要关注的Bits.reserveMemory()和Cleaner,Deallocator，其中Bits.reserveMemory()与分配相关，Cleaner、Deallocator则与内存释放相关。  
**Bits.reserveMemory()**

> static void reserveMemory(long size, int cap) \{
> 
>     // 初始化maxMemory，就是使用-XX:MaxDirectMemorySize指定的最大直接内存大小
>     if (!memoryLimitSet && VM.isBooted()) {
>         maxMemory = VM.maxDirectMemory();
>         memoryLimitSet = true;
>     }
>     
>     // 第一次先采取最乐观的方式直接尝试告诉Bits要分配内存
>     if (tryReserveMemory(size, cap)) {
>         return;
>     }
>     
>     final JavaLangRefAccess jlra = SharedSecrets.getJavaLangRefAccess();
>     
>     // 尝试执行Cleaner来释放直接内存，直到内存空间足够
>     while (jlra.tryHandlePendingReference()) {
>         if (tryReserveMemory(size, cap)) {
>             return;
>         }
>     }
>     
>     // GC
>     System.gc();
>     
>     // 按照1ms,2ms,4ms,...,256ms的等待间隔尝试9次分配内存
>     boolean interrupted = false;
>     try {
>         long sleepTime = 1;
>         int sleeps = 0;
>         while (true) {
>             if (tryReserveMemory(size, cap)) {
>                 return;
>             }
>             if (sleeps >= MAX_SLEEPS) {
>                 break;
>             }
>             if (!jlra.tryHandlePendingReference()) {
>                 try {
>                     Thread.sleep(sleepTime);
>                     sleepTime <<= 1;
>                     sleeps++;
>                 } catch (InterruptedException e) {
>                     interrupted = true;
>                 }
>             }
>         }
>     
>         throw new OutOfMemoryError("Direct buffer memory");
>     
>     } finally {
>         if (interrupted) {
>             Thread.currentThread().interrupt();
>         }
>     } }
> 
> //  
> \-XX:MaxDirectMemorySize限制的是总cap，而不是真实的内存使用量，(在页对齐的情况下，真实内存使用量和总cap是不同的)  
> private static boolean tryReserveMemory(long size, int cap) \{  
> long totalCap;  
> while (cap <= maxMemory - (totalCap = totalCapacity.get())) \{  
> if (totalCapacity.compareAndSet(totalCap, totalCap + cap)) \{  
> reservedMemory.addAndGet(size);  
> count.incrementAndGet();  
> return true;  
> \}  
> \}
> 
>     return false; }

**内存释放**

内存释放是通过Cleaner和Deallocator来实现的。

**Deallocator**

> private static class Deallocator implements Runnable \{
> 
>     private static Unsafe unsafe = Unsafe.getUnsafe();
>     
>     private long address;
>     private long size;
>     private int capacity;
>     
>     private Deallocator(long address, long size, int capacity) {
>         assert (address != 0);
>         this.address = address;
>         this.size = size;
>         this.capacity = capacity;
>     }
>     
>     public void run() {
>         if (address == 0) {
>             // Paranoia
>             return;
>         }
>         unsafe.freeMemory(address);
>         address = 0;
>         Bits.unreserveMemory(size, capacity);
>     } }

这个类中主要方法为run()，里面的步骤也很简单，包含两步

使用unsafe释放内存  
利用Bits管理内存的释放，就是标记一下该内存已释放

每个DirectByteBuffer都有一个相对应的Deallocator，而Deallocator则是由Cleaner来进行调度。  
**Cleaner**  
Cleaner的数据结构为一个双向链表，如下

> private static Cleaner first = null; // 链表的头节点 private Cleaner next =  
> null; // 下一个节点 private Cleaner prev = null; // 上一个节点 private final  
> Runnable thunk; // 存放Deallocator

Cleaner中主要包含如下操作，add, remove,clean

主要操作

1.  add

> 1.  private static synchronized Cleaner add(Cleaner var0) \{  
>     if (first != null) \{  
>     var0.next = first;  
>     first.prev = var0;  
>     \}
>     
>     first = var0;  
>     return var0; \}

add操作就是不断地将新的Cleaner节点添加在链表头部，之后将头节点指针指向新的Cleaner

1.  remove

> private static synchronized boolean remove(Cleaner var0) \{  
> if (var0.next == var0) \{ // 已经移除，防止重复移除  
> return false;  
> \} else \{  
> if (first == var0) \{  
> if (var0.next != null) \{  
> first = var0.next;  
> \} else \{  
> first = var0.prev;  
> \}  
> \}
> 
>     if (var0.next != null) {
>             var0.next.prev = var0.prev;
>         }
>     
>         if (var0.prev != null) {
>             var0.prev.next = var0.next;
>         }
>     
>         var0.next = var0;
>         var0.prev = var0;
>         return true;
>     } }

remove操作就是将Cleaner节点从链表中删除

1.  clean

> public void clean() \{  
> if (remove(this)) \{  
> try \{  
> this.thunk.run();  
> \} catch (final Throwable var2) \{  
> AccessController.doPrivileged(new PrivilegedAction() \{  
> public Void run() \{  
> if (System.err != null) \{  
> (new Error(“Cleaner terminated abnormally”, var2)).printStackTrace();  
> \}
> 
>     System.exit(1);
>                     return null;
>                 }
>             });
>         }
>     
>     } }

clean操作则是移除Cleaner节点并调用Deallocator的run()方法  
清理过程  
疑问 Cleaner.clean()又是由谁在何时调用的呢？  
仔细观察可以发现，Cleaner继承了PhantomReference，其referent为DirectByteBuffer  
Reference  
在Reference初次加载的过程中会调用一段静态代码

> static \{  
> ThreadGroup tg = Thread.currentThread().getThreadGroup();  
> for (ThreadGroup tgn = tg;  
> tgn != null;  
> tg = tgn, tgn = tg.getParent());  
> Thread handler = new ReferenceHandler(tg, “Reference Handler”);  
> handler.setPriority(Thread.MAX\_PRIORITY);  
> handler.setDaemon(true);  
> handler.start();
> 
>     // provide access in SharedSecrets
>     SharedSecrets.setJavaLangRefAccess(new JavaLangRefAccess() {
>         @Override
>         public boolean tryHandlePendingReference() {
>             return tryHandlePending(false);
>         }
>     }); }

这段代码中包含了两种可以调用Cleaner的方式：

ReferenceHandler，会不停地循环调用tryHandlePending  
SharedSecrets.JavaLangRefAccess，在Bits.reserveMemory()中被调用

事实上直接内存的回收过程也的确是由这两种方式混合组成，这两种方式有一个共同点，他们都会调用Reference.tryHandlePending()方法。

> static boolean tryHandlePending(boolean waitForNotify) \{  
> Reference r;  
> Cleaner c;  
> try \{  
> synchronized (lock) \{  
> if (pending != null) \{  
> r = pending;  
> c = r instanceof Cleaner ? (Cleaner) r : null;  
> pending = r.discovered;  
> r.discovered = null;  
> \} else \{  
> if (waitForNotify) \{  
> lock.wait();  
> \}  
> return waitForNotify;  
> \}  
> \}  
> \} catch (OutOfMemoryError x) \{  
> Thread.yield();  
> return true;  
> \} catch (InterruptedException x) \{  
> return true;  
> \}
> 
>     if (c != null) {
>         c.clean();
>         return true;
>     }
>     
>     ReferenceQueue<? super Object> q = r.queue;
>     if (q != ReferenceQueue.NULL) q.enqueue(r);
>     return true; }

其中pending和discovered由JVM来操作，两个共同组成一个等待队列链表，对于PhantomReference的情况，当对象不存在其他引用，便会直接加入等待队列。每当等待队列中出现Cleaner，就会执行其clean()方法。

**总结**

1.  整个DirectByteBuffer的分配与释放流程如下  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3podWd1YW5nMTA_size_16_color_FFFFFF_t_70]
2.  \-XX:MaxDirectMemorySize参数只对由DirectByteBuffer分配的内存有效，对Unsafe直接分配的内存无效  
    native方法  
    疑问 native方法中分配的内存是否是属于DirectByteBuffer对象呢?  
    这个疑问来自于一次内存泄漏问题的排查，一直没有机会去研究，正好借这次机会寻找一下该问题的答案。  
    demo  
    写了一个简单的demo程序如下

> // java部分 public class NativeMain \{  
> public native void allocateMemory();
> 
>     static {
>         System.setProperty("java.library.path", ".");
>         System.loadLibrary("nativemain");
>     }
>     
>     public static void main(String[] args) throws Exception {
>         NativeMain nativeMain = new NativeMain();
>         while (true) {
>             for (int i = 0; i < 10000; i++) {
>                 nativeMain.allocateMemory();
>             }
>             Thread.sleep(1);
>         }
>     } }
> 
> // c++实现部分  
> \#include “jni.h”  
> \#include “NativeMain.h”  
> \#include <stdlib.h>
> 
> JNIEXPORT void JNICALL Java\_NativeMain\_allocateMemory(JNIEnv \*,  
> jobject) \{  
> char *ptr = (char*)malloc(1000); \}

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3podWd1YW5nMTA_size_16_color_FFFFFF_t_70 1]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3podWd1YW5nMTA_size_16_color_FFFFFF_t_70 2]  
运行发现native方法分配的内存并不会产生DirectByteBuffer对象，同样的也不受-XX:MaxDirectMemorySize影响。

关注我免费送一套Java架构资料，先到先得！需要的可以加小编QQ群798891710，获取！

[20190402110412453.]: /images/20220225/55b9260059f845589f6e33f21e42ff28.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3podWd1YW5nMTA_size_16_color_FFFFFF_t_70]: /images/20220225/c3e65a708e6d470ca5fe2a22126de2ad.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3podWd1YW5nMTA_size_16_color_FFFFFF_t_70 1]: /images/20220225/c32154fb49cf4e25863e401110031d5b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3podWd1YW5nMTA_size_16_color_FFFFFF_t_70 2]: /images/20220225/9a20b8cf58094dbc824101731478f2ba.png