Mycat源码篇 : MyCat线程模型分析

雨点打透心脏的1/2处 2022-07-17 02:18 252阅读 0赞

参考MyCat权威指南，对MyCat-Server里面的线程模型做简要分析:

# 1. 线程模型图 #

根据MyCat权威指南，做出以下线程模型图:

![mycat-thread-model][]

MyCat的线程模型主要分为三部分（: 网络通讯线程、业务线程和定时任务线程，下面分别介绍这些线程的使用:  
**【温馨提示】**  
这里排除JVM自身使用的线程，只关注MyCat服务所使用的线程，如果需要详细了解MyCat里面使用的所有线程，请参考《MyCat权威指南》-> 开发篇 -> MyCat线程模型分析

# 2. 网络通讯线程 #

MyCat-Server使用了原生Java NIO和AIO来构建其最重要的网络通讯层，在MyCat-Server启动之前可以通过配置文件来选择使用NIO网络模型或者AIO网络模型。MyCat官方推荐使用NIO模型。所以，这里仅针对NIO网络通讯模型来分析，网络通讯层线程的使用体现在NIOAcceptor、NIOConnector和NIOReactor。

## 2.1 NIOAcceptor ##

NIOAcceptor线程负责处理客户端连接MyCat的事件（在NIO网络编程中，对应于NIO服务端），包括默认的8066服务端口（Server）和9066管理端口（Manager）。

NIOAcceptor继承Thread类，在run方法里面处理客户端连接MyCat的连接请求，代码如下所示 :

@Override
        public void run() {
            final Selector tSelector = this.selector;
            for (;;) {
                ++acceptCount;
                try {
                    tSelector.select(1000L);
                    Set<SelectionKey> keys = tSelector.selectedKeys();
                    try {
                        for (SelectionKey key : keys) {
                            if (key.isValid() && key.isAcceptable()) {
                                accept();
                            } else {
                                key.cancel();
                            }
                        }
                    } finally {
                        keys.clear();
                    }
                } catch (Exception e) {
                    LOGGER.warn(getName(), e);
                }
            }
        }

对于来自客户端的连接，将被Selector事件选择器捕获，对于成功的连接，将调用accept方法做进一步处理，下面是accept方法的代码 :

private void accept() {
            SocketChannel channel = null;
            try {
                channel = serverChannel.accept();
                channel.configureBlocking(false);
                FrontendConnection c = factory.make(channel);
                c.setAccepted(true);
                c.setId(ID_GENERATOR.getId());
                NIOProcessor processor = (NIOProcessor) MycatServer.getInstance()
                        .nextProcessor();
                c.setProcessor(processor);
    
                NIOReactor reactor = reactorPool.getNextReactor();
                reactor.postRegister(c);
    
            } catch (Exception e) {
                LOGGER.warn(getName(), e);
                closeChannel(channel);
            }
        }

accept方法将客户端连接封装成FrontendConnection对象，分配相关资源，然后将客户端连接注册到NIOReactor上。

**【温馨提示】**  
这里注意到，NIOProcessor（持有处理业务逻辑的线程池）和NIOReactor（负责处理NIO网络通讯的读写事件）都进行了池化，目的是让对象得以共享这些稀缺的资源。

## 2.2 NIOConnector ##

NIOConnector线程负责处理MyCat连接后端MySQL实例的事件（在NIO网络编程中，对应于NIO Client）。与NIOAcceptor类似，该类继承Thread类，在run方法里面处理连接事件，代码如下所示 :

@Override
        public void run() {
            final Selector tSelector = this.selector;
            for (;;) {
                ++connectCount;
                try {
                    //查看有无连接就绪
                    tSelector.select(1000L);
                    connect(tSelector);
                    Set<SelectionKey> keys = tSelector.selectedKeys();
                    try {
                        for (SelectionKey key : keys) {
                            Object att = key.attachment();
                            if (att != null && key.isValid() && key.isConnectable()) {
                                finishConnect(key, att);
                            } else {
                                key.cancel();
                            }
                        }
                    } finally {
                        keys.clear();
                    }
                } catch (Exception e) {
                    LOGGER.warn(name, e);
                }
            }
        }

在connect方法里面会判断连接队列（元素入队列体现在postConnect方法方法里）是否有连接需要进行处理，如果队列有值，那么取得连接对象主动去跟后端的MySQL实例建立连接。代码如下所示 :

private void connect(Selector selector) {
            AbstractConnection c = null;
            while ((c = connectQueue.poll()) != null) {
                try {
                    SocketChannel channel = (SocketChannel) c.getChannel();
                    //注册OP_CONNECT监听与后端连接是否真正建立
                    channel.register(selector, SelectionKey.OP_CONNECT, c);
                    //主动连接
                    channel.connect(new InetSocketAddress(c.host, c.port));
                } catch (Exception e) {
                    c.close(e.toString());
                }
            }
        }

连接建立成功后调用finishConnect方法注册对应的读写事件，然后与NIOAcceptor一样，将读写事件的处理交由NIOReactor处理。代码如下所示 :

private void finishConnect(SelectionKey key, Object att) {
            BackendAIOConnection c = (BackendAIOConnection) att;
            try {
                if (finishConnect(c, (SocketChannel) c.channel)) { //做原生NIO连接是否完成的判断和操作
                    clearSelectionKey(key);
                    c.setId(ID_GENERATOR.getId());
                    //绑定特定的NIOProcessor以作idle清理
                    NIOProcessor processor = MycatServer.getInstance()
                            .nextProcessor();
                    c.setProcessor(processor);
                    //与特定NIOReactor绑定监听读写
                    NIOReactor reactor = reactorPool.getNextReactor();
                    reactor.postRegister(c);
                }
            } catch (Exception e) {
                //如有异常，将key清空
                clearSelectionKey(key);
                c.close(e.toString());
                c.onConnectFailed(e);
            }
        }

## 2.3 NIOReactor ##

一般的NIO网络编程示例，在Server accept客户端连接之后，会直接在该线程里注册读写事件，最后再分发给逻辑处理线程池去处理逻辑，示例代码如下所示 :

for (;;) {
        try {
            tSelector.select();
            Set<SelectionKey> keys = tSelector.selectedKeys();
            try {
                for (SelectionKey key : keys) {
                    if (key.isValid() && key.isAcceptable()) {
                        accept(); // accept register OP_READ and OP_WRITE
                    } else if(key.isReadable) {
                        read(); // read data and do business using thread pool
                    } else if(key.isWritable) {
                        write(); // write data using thread pool
                    }
                }
            } finally {
                keys.clear();
            }
        } catch (Exception e) {
            // 
        }
    }

在高性能IO框架实现中，大多采用Reactor模型，将处理连接和处理读写事件分离，模型图如下所示 :

![high-perf-reactor-model][]

MyCat中采用了reactor模式，将读写事件的注册和接收分离到另外的线程去处理。这个另外的线程指的就是NIOReactor，并且，为了优化并发下频繁进行读写，NIOReactor进行了池化（NIOReactorPool）。

# 3. 业务线程 #

BusinessExecutor采用Java线程池来实现，线程池大小默认计算方式如下所示：

this.processorExecutor = (DEFAULT_PROCESSORS != 1) ? DEFAULT_PROCESSORS * 2 : 4;

其中DEFAULT\_PROCESSORS为系统CPU可用核心数。单核心的线程池大小为4，非单核心线程池大小为系统CPU可用核心数的2倍。

线程池的任务队列使用LinkedTransferQueue。

业务线程主要用在以下方面：

*  负责对读取到的网络字节数据做进一步处理
 *  创建后台mysql连接
 *  多分片节点结果集包返回汇聚后处理

# 4. 定时任务线程 #

TimerExecutor 定时任务线程池，默认线程大小为2。TimerExecutor结合Timer类进行定时任务调度，主要功能为：

*  定时检查前端连接和后端空闲连接，回收不可用的连接。
 *  定期发送心跳包以及心跳检测

**【温馨提示】** 定时逻辑代码入口在MyCatServer.java类里面

# 5. jstack dump线程 #

我们可以使用jdk自带工具jstack来查看MyCat-Server线程的使用情况:

首先使用jps查看MyCat-Server的进程号:

![show-jps][]

然后使用jstack \[mycat-server-pid\]打印出线程使用情况:

C:\WINDOWS\system32>jstack 8648
    2016-08-11 22:34:47
    Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.66-b18 mixed mode):
    
    "Timer1" #31 daemon prio=5 os_prio=0 tid=0x000000001a75a800 nid=0x21dc waiting on condition [0x000000001e0df000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c00730> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "Timer0" #30 daemon prio=5 os_prio=0 tid=0x000000001a759000 nid=0x1ce0 waiting on condition [0x000000001dfde000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c00730> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "BusinessExecutor7" #29 daemon prio=5 os_prio=0 tid=0x000000001a759800 nid=0x13b4 waiting on condition [0x000000001dbde000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "BusinessExecutor6" #28 daemon prio=5 os_prio=0 tid=0x000000001a75b000 nid=0x1090 waiting on condition [0x000000001dade000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "BusinessExecutor5" #27 daemon prio=5 os_prio=0 tid=0x000000001a753800 nid=0xf08 waiting on condition [0x000000001d9de000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "BusinessExecutor4" #26 daemon prio=5 os_prio=0 tid=0x000000001a758000 nid=0x14a0 waiting on condition [0x000000001d8df000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "BusinessExecutor3" #25 daemon prio=5 os_prio=0 tid=0x000000001a756800 nid=0x21c4 waiting on condition [0x000000001d7de000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "BusinessExecutor2" #24 daemon prio=5 os_prio=0 tid=0x000000001a757800 nid=0x21c0 waiting on condition [0x000000001d6de000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "BusinessExecutor1" #23 daemon prio=5 os_prio=0 tid=0x000000001a755000 nid=0x21b4 waiting on condition [0x000000001d5de000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "BusinessExecutor0" #22 daemon prio=5 os_prio=0 tid=0x000000001a4e5000 nid=0x21b0 waiting on condition [0x000000001d4de000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
            at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
            at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    "$_MyCatServer" #21 prio=5 os_prio=0 tid=0x000000001a4ec000 nid=0x21b8 runnable [0x000000001d3de000]
       java.lang.Thread.State: RUNNABLE
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
            at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
            at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
            - locked <0x00000000d7c0bb68> (a sun.nio.ch.Util$2)
            - locked <0x00000000d7c0bb78> (a java.util.Collections$UnmodifiableSet)
            - locked <0x00000000d7c0bae8> (a sun.nio.ch.WindowsSelectorImpl)
            at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
            at org.opencloudb.net.NIOAcceptor.run(NIOAcceptor.java:86)
    
    "$_MyCatManager" #20 prio=5 os_prio=0 tid=0x000000001a4ea800 nid=0x21bc runnable [0x000000001d2df000]
       java.lang.Thread.State: RUNNABLE
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
            at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
            at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
            - locked <0x00000000d7c00de8> (a sun.nio.ch.Util$2)
            - locked <0x00000000d7c00df8> (a java.util.Collections$UnmodifiableSet)
            - locked <0x00000000d7c00d68> (a sun.nio.ch.WindowsSelectorImpl)
            at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
            at org.opencloudb.net.NIOAcceptor.run(NIOAcceptor.java:86)
    
    "$_NIOConnector" #19 prio=5 os_prio=0 tid=0x000000001a4e9000 nid=0x20f0 runnable [0x000000001d1de000]
       java.lang.Thread.State: RUNNABLE
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
            at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
            at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
            - locked <0x00000000d7c1b600> (a sun.nio.ch.Util$2)
            - locked <0x00000000d7c1b610> (a java.util.Collections$UnmodifiableSet)
            - locked <0x00000000d7c1b580> (a sun.nio.ch.WindowsSelectorImpl)
            at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
            at org.opencloudb.net.NIOConnector.run(NIOConnector.java:76)
    
    "$_NIOREACTOR-3-RW" #18 prio=5 os_prio=0 tid=0x000000001a4e8000 nid=0x2314 runnable [0x000000001d0de000]
       java.lang.Thread.State: RUNNABLE
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
            at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
            at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
            - locked <0x00000000d7c13868> (a sun.nio.ch.Util$2)
            - locked <0x00000000d7c13878> (a java.util.Collections$UnmodifiableSet)
            - locked <0x00000000d7c137e8> (a sun.nio.ch.WindowsSelectorImpl)
            at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
            at org.opencloudb.net.NIOReactor$RW.run(NIOReactor.java:89)
            at java.lang.Thread.run(Thread.java:745)
    
    "$_NIOREACTOR-2-RW" #17 prio=5 os_prio=0 tid=0x000000001a4e6800 nid=0x1774 runnable [0x000000001cfdf000]
       java.lang.Thread.State: RUNNABLE
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
            at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
            at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
            - locked <0x00000000d7c1e880> (a sun.nio.ch.Util$2)
            - locked <0x00000000d7c1e890> (a java.util.Collections$UnmodifiableSet)
            - locked <0x00000000d7c1e800> (a sun.nio.ch.WindowsSelectorImpl)
            at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
            at org.opencloudb.net.NIOReactor$RW.run(NIOReactor.java:89)
            at java.lang.Thread.run(Thread.java:745)
    
    "$_NIOREACTOR-1-RW" #16 prio=5 os_prio=0 tid=0x000000001a4e7800 nid=0x2320 runnable [0x000000001cede000]
       java.lang.Thread.State: RUNNABLE
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
            at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
            at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
            - locked <0x00000000d7c07a68> (a sun.nio.ch.Util$2)
            - locked <0x00000000d7c07a78> (a java.util.Collections$UnmodifiableSet)
            - locked <0x00000000d7c079e8> (a sun.nio.ch.WindowsSelectorImpl)
            at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
            at org.opencloudb.net.NIOReactor$RW.run(NIOReactor.java:89)
            at java.lang.Thread.run(Thread.java:745)
    
    "$_NIOREACTOR-0-RW" #15 prio=5 os_prio=0 tid=0x000000001a4e6000 nid=0x20ac runnable [0x000000001cdde000]
       java.lang.Thread.State: RUNNABLE
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
            at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
            at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
            at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
            - locked <0x00000000d7c0f0c0> (a sun.nio.ch.Util$2)
            - locked <0x00000000d7c0f0d0> (a java.util.Collections$UnmodifiableSet)
            - locked <0x00000000d7c0f040> (a sun.nio.ch.WindowsSelectorImpl)
            at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
            at org.opencloudb.net.NIOReactor$RW.run(NIOReactor.java:89)
            at java.lang.Thread.run(Thread.java:745)
    
    "Log4jWatchdog" #14 daemon prio=5 os_prio=0 tid=0x000000001a4e9800 nid=0x16d0 waiting on condition [0x000000001bd0f000]
       java.lang.Thread.State: TIMED_WAITING (sleeping)
            at java.lang.Thread.sleep(Native Method)
            at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104)
    
    "net.sf.ehcache.CacheManager@3abfe836" #12 daemon prio=5 os_prio=0 tid=0x000000001a4eb000 nid=0x2298 in Object.wait() [0x000000001bb0f000]
       java.lang.Thread.State: WAITING (on object monitor)
            at java.lang.Object.wait(Native Method)
            - waiting on <0x00000000d7c454a8> (a java.util.TaskQueue)
            at java.lang.Object.wait(Object.java:502)
            at java.util.TimerThread.mainLoop(Timer.java:526)
            - locked <0x00000000d7c454a8> (a java.util.TaskQueue)
            at java.util.TimerThread.run(Timer.java:505)
    
    "MyCatTimer" #11 daemon prio=5 os_prio=0 tid=0x000000001a372000 nid=0x1f48 in Object.wait() [0x000000001b7bf000]
       java.lang.Thread.State: TIMED_WAITING (on object monitor)
            at java.lang.Object.wait(Native Method)
            at java.util.TimerThread.mainLoop(Timer.java:552)
            - locked <0x00000000d7c46428> (a java.util.TaskQueue)
            at java.util.TimerThread.run(Timer.java:505)
    
    "Thread-0" #10 prio=5 os_prio=0 tid=0x0000000019e9b000 nid=0x2304 waiting on condition [0x000000001a6ee000]
       java.lang.Thread.State: TIMED_WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000d7cda8b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
            at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
            at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
            at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
            at org.opencloudb.route.MyCATSequnceProcessor$ExecuteThread.run(MyCATSequnceProcessor.java:102)
    
    "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x000000001967d000 nid=0x1f5c runnable [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE
    
    "C1 CompilerThread2" #8 daemon prio=9 os_prio=2 tid=0x00000000195f7800 nid=0x213c waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE
    
    "C2 CompilerThread1" #7 daemon prio=9 os_prio=2 tid=0x00000000195f1000 nid=0x20fc waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE
    
    "C2 CompilerThread0" #6 daemon prio=9 os_prio=2 tid=0x0000000017f9a800 nid=0x1cdc waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE
    
    "Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x0000000017f4f000 nid=0x72c waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE
    
    "Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x0000000017f4e000 nid=0x214c runnable [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE
    
    "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x00000000033b7000 nid=0x2150 in Object.wait() [0x000000001928f000]
       java.lang.Thread.State: WAITING (on object monitor)
            at java.lang.Object.wait(Native Method)
            at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
            - locked <0x00000000d7d301b0> (a java.lang.ref.ReferenceQueue$Lock)
            at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
            at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
    
    "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x0000000017f09000 nid=0x1328 in Object.wait() [0x000000001918f000]
       java.lang.Thread.State: WAITING (on object monitor)
            at java.lang.Object.wait(Native Method)
            at java.lang.Object.wait(Object.java:502)
            at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
            - locked <0x00000000d7d303c8> (a java.lang.ref.Reference$Lock)
    
    "main" #1 prio=5 os_prio=0 tid=0x00000000030be000 nid=0x22c8 waiting on condition [0x00000000032bf000]
       java.lang.Thread.State: TIMED_WAITING (sleeping)
            at java.lang.Thread.sleep(Native Method)
            at org.opencloudb.MycatStartup.main(MycatStartup.java:57)
    
    "VM Thread" os_prio=2 tid=0x0000000017f07000 nid=0x1218 runnable
    
    "GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00000000032d7800 nid=0x1d54 runnable
    
    "GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00000000032d9000 nid=0x23f4 runnable
    
    "GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00000000032da800 nid=0x1600 runnable
    
    "GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00000000032dc000 nid=0x1aac runnable
    
    "VM Periodic Task Thread" os_prio=2 tid=0x00000000196b9800 nid=0x2308 waiting on condition
    
    JNI global references: 253

[mycat-thread-model]: http://7xtamf.com1.z0.glb.clouddn.com/image/blog/mycat-thread-model-unstanding/mycat-thread-model.jpg?imageView2/2/w/768
[high-perf-reactor-model]: http://7xtamf.com1.z0.glb.clouddn.com/image/blog/mycat-thread-model-unstanding/hig-perf-reactor-model.png
[show-jps]: http://7xtamf.com1.z0.glb.clouddn.com/image/blog/mycat-thread-model-unstanding/show-jps.png