metaq入门部署到实战

你的名字 2022-08-13 10:58 176阅读 0赞

#  #

##  ##

# 初识metaq #

## zookeeper部署，这里单机zk为例。 ##

wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.5/zookeeper-3.4.6.tar.gz
    tar -zxvf zookeeper-3.4.6.tar.gz
    cd  zookeeper-3.4.6
    mkdir data
    cp conf/zoo_sample.cfg  conf/zoo.cfg

### 修改zk的配置文件，将data路径dataDir的值设置为zookeeper-3.4.6下的data ###

hadoop@yard02:~/bigdata/zookeeper-3.4.6$ cat conf/zoo.cfg 
    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial 
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between 
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    # do not use /tmp for storage, /tmp here is just 
    # example sakes.
    dataDir=/home/hadoop/bigdata/zookeeper-3.4.6/data
    # the port at which the clients will connect
    clientPort=2181
    # the maximum number of client connections.
    # increase this if you need to handle more clients
    #maxClientCnxns=60
    #
    # Be sure to read the maintenance section of the 
    # administrator guide before turning on autopurge.
    #
    # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
    #
    # The number of snapshots to retain in dataDir
    #autopurge.snapRetainCount=3
    # Purge task interval in hours
    # Set to "0" to disable auto purge feature
    #autopurge.purgeInterval=1

## metaq部署 ##

### 这里使用单机多实例测试 ###

wget http://fnil.net/downloads/metaq-server-1.4.6.2.tar.gz
    tar -zxvf metaq-server-1.4.6.2.tar.gz 
    cd  taobao/
    cp -rf  metamorphosis-server-wrapper ../metaq9123
    cp -rf  metamorphosis-server-wrapper ../metaq9124
    cp -rf  metamorphosis-server-wrapper ../metaq9125

### 单机多实例需要解决端口冲突的问题 ###

#### 1，修改JMX端口，修改服务端口，修改dashboardWeb端口,zk地址 ####

我这里保持metaq9123默认，修改metaq9124，metaq9125

vim metaq9124/conf/server.ini

serverPort=8124
    dashboardHttpPort=8121
    zk.zkConnect=192.168.137.2:2181

vim metaq9124/bin/env.sh

export JMX_PORT=9124

metaq9125同样，修改成不被占用的接口就行了 启动：

bin/metaServer.sh start

停止

bin/metaServer.sh stop

状态

bin/metaServer.sh status

### zk上节点 ###

摘自[http://udukwilliam.iteye.com/blog/1931852][http_udukwilliam.iteye.com_blog_1931852]

#### 1) /consumers:存放消费者列表以及消费记录，消费者列表主要是以组的方式存在，结构主要如下： ####

/consumers/xxGroup/ids/xxConsumerId：DATA(“：”后的DATA表示节点xxConsumerId对应的数据) 组内消费者Id;DATA为订阅主题列表，以”,”分隔
           /consumers/xxGroup/offsets/xxTopic/分区N：DATA  组内主题分区N的消费进度；DATA为topic下分区N具体进度值
           /consumers/xxGroup/owners/xxTopic/分区N：DATA 组内主题分区N的的消费者；DATA为消费者ID，表示XXTopic下分区N的数据由指定的消费者进行消费

#### 2) /brokers/ids:存放Broker列表，如果Broker与Zookeeper失去连接，则会自动注销在/brokers/ids下的broker记录，例子如下： ####

/brokers/ids/xxBroker

#### 3) /brokers/topics-pub：存放发布的主题列表以及对应的可发送消息的Broker列表，例子如下： ####

/brokers/topics-pub/xxTopic/xxBroker
        /brokers/topics-pub下记录的是可发送消息到xxTopic的Broker列表，意味着有多少个Broker允许存储Client发送到Topic数据

#### 4) /brokers/topics-sub：存放订阅的主题列表以及对应可订阅的Broker列表，例子如下： ####

/brokers/topics-sub/xxTopic/xxBroker
        /brokers/topics-sub下记录的可订阅xxTopic的Broker列表，意味着有多少个Broker允许被Client订阅topic的数据

这里使用sohu-zookeeper-view来看了下节点信息，其中meta-example是group

![SouthEast][]

![SouthEast 1][]

WEB DASHBOARD

![SouthEast 2][]

[http_udukwilliam.iteye.com_blog_1931852]: http://udukwilliam.iteye.com/blog/1931852
[SouthEast]: /images/20220810/10ea65ee661d45b1bbf8c725c3c0aecd.png
[SouthEast 1]: /images/20220810/0b85520603894a1ca0e032b7d4523420.png
[SouthEast 2]: /images/20220810/0fa1ea18dda94163ad5aeb207cd47f56.png