doris的单节点安装部署(详细)

朴灿烈づ我的快乐病毒、 2024-02-21 12:28 55阅读 0赞

doris的单节点安装部署  
环境准备：  
1.jdk8  
进入官网:https://www.oracle.com/java/technologies/downloads/  
在linux服务上创建jdk安装的目录  
mkdir /itsoftware/jdk \#创建jdk安装路径  
cd /itsoftware/jdk \#进入jdk安装目录  
解压缩包

cd /itsoftware/jdk  #进入jdk安装目录
    tar  -zxvf  jdk-8u212-linux-x64.tar.gz #对压缩包解压

配置环境变量

vim /etc/profile   #编辑配置文件

在文件末尾添加

export JAVA_HOME=/itsoftware/jdk/jdk1.8.0_212
    export JRE_HOME=${JAVA_HOME}/jre
    export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
    export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
    export PATH=$PATH:${JAVA_PATH}

重新加载配置文件

source /etc/profile

验证jdk是否安装成功

javac -version

![\[图片\]][34d82bd4423e42bbb9df7ef47c050c64.png]

修改limits文件

vim /etc/security/limits.conf

在文件的末尾处添加或者修改

* soft nofile 204800
    * hard nofile 204800
    * soft nproc 204800
    * hard nproc 204800

vim /etc/sysctl.conf

\#添加

fs.file-max = 6553560
    vm.max_map_count = 2000000

\#关闭防火墙

systemctl stop firewalld
    systemctl disable firewalld

执行生效

sysctl -p

安装doris  
官网网址：https://doris.apache.org/zh-CN/docs/dev/get-starting/quick-start  
解压 tar.gz 文件

tar -zxvf apache-doris-2.0.2-bin-x64.tar.gz
     mv apache-doris-2.0.2-bin-x64 /opt/doris

配置环境：

[root@10-63-0-181 conf]# vim /etc/profile.d/doris.sh
    export DORIS_HOME=/opt/doris
    export PATH=$PATH:$DORIS_HOME/fe/bin:$DORIS_HOME/be/bin

配置生效：

source /etc/profile.d/doris.sh

FE配置  
进入doris下的fe目录，修改其中conf目录下的fe.conf文件  
![在这里插入图片描述][a56c6ac406e34a7c937c38eff569971a.png]

注意：  
这个参数我们在安装的时候是必须要配置的，特别是当一台机器拥有多个IP地址的时候，我们要为 FE 指定唯一的IP地址。  
这里假设你的节点 IP 是 172.23.16.32，那么我们可以通过掩码的方式配置为 172.23.16.0/24。

[root@10-63-0-181 conf]# pwd
    /opt/doris/fe/conf
    [root@10-63-0-181 conf]# vim fe.conf

更改配置：

meta_dir = /data/doris-meta
    priority_networks = 10.63.0.0/24

启动 FE  
在 FE 安装目录下执行下面的命令，来完成 FE 的启动。

./bin/start_fe.sh --daemon

查看 FE 运行状态  
你可以通过下面的命令来检查 Doris 是否启动成功

[root@10-63-0-181 fe]# curl http://10.63.0.181:8030/api/bootstrap
    {"msg":"success","code":0,"data":{"replayedJournalId":0,"queryPort":0,"rpcPort":0,"version":""},"count":0}[root@10-63-0-181 fe]#
    [root@10-63-0-181 fe]#

![在这里插入图片描述][188d20709e0b4a6dab948968005f9e4a.png]

注意：

1.  这里我们使用 Doris 内置的默认用户 root 进行登录，密码是空
2.  这是一个 Doris 的管理界面，只能拥有管理权限的用户才能登录，普通用户不能登录。

连接 FE  
我们下面通过 MySQL 客户端来连接 Doris FE，下载免安装的 MySQL 客户端  
解压刚才下载的 MySQL 客户端，在 bin/ 目录下可以找到 mysql 命令行工具。然后执行下面的命令连接 Doris。

[root@10-63-0-181 bin]# ./mysql -uroot -P9030 -h10.63.0.181
    Welcome to the MySQL monitor.  Commands end with ; or \g.
    Your MySQL connection id is 0
    Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7
    
    Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
    
    Oracle is a registered trademark of Oracle Corporation and/or its
    affiliates. Other names may be trademarks of their respective
    owners.
    
    Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
    
    mysql> show frontends\G;
    *************************** 1. row ***************************
                 Name: fe_2b76e132_4efe_4425_8919_f95876206eda
                 Host: 10.63.0.181
          EditLogPort: 9010
             HttpPort: 8030
            QueryPort: 9030
              RpcPort: 9020
                 Role: FOLLOWER
             IsMaster: true
            ClusterId: 1092995366
                 Join: true
                Alive: true
    ReplayedJournalId: 319
        LastHeartbeat: 2023-10-23 16:01:08
             IsHelper: true
               ErrMsg:
              Version: doris-2.0.2-rc05-ae923f7
     CurrentConnected: Yes
    1 row in set (0.06 sec)

停止 FE 节点  
Doris FE 的停止可以通过下面的命令完成

./bin/stop_fe.sh

配置 BE  
进入doris下的be目录，修改其中conf目录下的be.conf文件  
\[root@10-63-0-181 conf\]\# pwd

/opt/doris/be/conf

[root@10-63-0-181 conf]# vim be.conf

更改配置：

priority_networks = 10.63.0.0/24
    storage_root_path = /data/doris-data

启动 BE  
在 BE 安装目录下执行下面的命令，来完成 BE 的启动。  
./bin/start\_be.sh --daemon  
添加 BE 节点到集群  
通过MySQL 客户端连接到 FE 之后执行下面的 SQL，将 BE 添加到集群中

ALTER SYSTEM ADD BACKEND "10.60.0.181:9050";
    [root@10-63-0-181 bin]# ./mysql -uroot -P 9030 -h 10.63.0.181
    Welcome to the MySQL monitor.  Commands end with ; or \g.
    Your MySQL connection id is 5
    Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7
    
    Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
    
    Oracle is a registered trademark of Oracle Corporation and/or its
    affiliates. Other names may be trademarks of their respective
    owners.
    
    Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
    
    mysql> SHOW BACKENDS\G;
    *************************** 1. row ***************************
                  BackendId: 10065
                       Host: 10.63.0.181
              HeartbeatPort: 9050
                     BePort: 9060
                   HttpPort: 8040
                   BrpcPort: 8060
              LastStartTime: 2023-10-24 10:40:27
              LastHeartbeat: 2023-10-25 15:44:35
                      Alive: true
       SystemDecommissioned: false
                  TabletNum: 22
           DataUsedCapacity: 1.417 KB
         TrashUsedCapcacity: 0.000
              AvailCapacity: 199.782 GB
              TotalCapacity: 199.902 GB
                    UsedPct: 0.06 %
             MaxDiskUsedPct: 0.06 %
         RemoteUsedCapacity: 0.000
                        Tag: {"location" : "default"}
                     ErrMsg:
                    Version: doris-2.0.2-rc05-ae923f7
                     Status: {"lastSuccessReportTabletsTime":"2023-10-25 15:44:34","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false}

1.  Alive : true表示节点运行正常  
    ![在这里插入图片描述][5826b47904844fdabc505c87ee1dc7ea.png]

验证：  
创建数据表  
创建一个数据库

create database demo;

创建数据表

use demo;
    
    CREATE TABLE IF NOT EXISTS demo.example_tbl
    (
        `user_id` LARGEINT NOT NULL COMMENT "用户id",
        `date` DATE NOT NULL COMMENT "数据灌入日期时间",
        `city` VARCHAR(20) COMMENT "用户所在城市",
        `age` SMALLINT COMMENT "用户年龄",
        `sex` TINYINT COMMENT "用户性别",
        `last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
        `cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
        `max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
        `min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
    )
    AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
    DISTRIBUTED BY HASH(`user_id`) BUCKETS 1
    PROPERTIES (
        "replication_allocation" = "tag.location.default: 1"
    );
    示例数据
    10000,2017-10-01,北京,20,0,2017-10-01 06:00:00,20,10,10
    10000,2017-10-01,北京,20,0,2017-10-01 07:00:00,15,2,2
    10001,2017-10-01,北京,30,1,2017-10-01 17:05:45,2,22,22
    10002,2017-10-02,上海,20,1,2017-10-02 12:59:12,200,5,5
    10003,2017-10-02,广州,32,0,2017-10-02 11:20:00,30,11,11
    10004,2017-10-01,深圳,35,0,2017-10-01 10:00:15,100,3,3
    10004,2017-10-03,深圳,35,0,2017-10-03 10:20:22,11,6,6
    将上面的数据保存在test.csv文件中。
    
    1. 导入数据
    这里我们通过Stream load 方式将上面保存到文件中的数据导入到我们刚才创建的表里。
    [root@10-63-0-181 ~]# vi test.csv
    [root@10-63-0-181 ~]# cat test.csv
    10000,2017-10-01,北京,20,0,2017-10-01 06:00:00,20,10,10
    10000,2017-10-01,北京,20,0,2017-10-01 07:00:00,15,2,2
    10001,2017-10-01,北京,30,1,2017-10-01 17:05:45,2,22,22
    10002,2017-10-02,上海,20,1,2017-10-02 12:59:12,200,5,5
    10003,2017-10-02,广州,32,0,2017-10-02 11:20:00,30,11,11
    10004,2017-10-01,深圳,35,0,2017-10-01 10:00:15,100,3,3
    10004,2017-10-03,深圳,35,0,2017-10-03 10:20:22,11,6,6
    [root@10-63-0-181 ~]# curl  --location-trusted -u root: -T test.csv -H "column_separator:," http://10.63.0.181:8030/api/demo/example_tbl/_stream_load
    {
        "TxnId": 2,
        "Label": "3d320737-a9db-4546-a145-5543e56d44a5",
        "Comment": "",
        "TwoPhaseCommit": "false",
        "Status": "Success",
        "Message": "OK",
        "NumberTotalRows": 7,
        "NumberLoadedRows": 7,
        "NumberFilteredRows": 0,
        "NumberUnselectedRows": 0,
        "LoadBytes": 399,
        "LoadTimeMs": 284,
        "BeginTxnTimeMs": 20,
        "StreamLoadPutTimeMs": 199,
        "ReadDataTimeMs": 0,
        "WriteDataTimeMs": 11,
        "CommitAndPublishTimeMs": 51
    }
    [root@10-63-0-181 bin]# ./mysql -uroot -P 9030 -h 10.63.0.181
    Welcome to the MySQL monitor.  Commands end with ; or \g.
    Your MySQL connection id is 1
    Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7
    
    Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
    
    Oracle is a registered trademark of Oracle Corporation and/or its
    affiliates. Other names may be trademarks of their respective
    owners.
    
    Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
    
    mysql> use demo;
    Reading table information for completion of table and column names
    You can turn off this feature to get a quicker startup with -A
    
    Database changed
    mysql> select * from example_tbl;
    +---------+------------+--------+------+------+---------------------+------+----------------+----------------+
    | user_id | date       | city   | age  | sex  | last_visit_date     | cost | max_dwell_time | min_dwell_time |
    +---------+------------+--------+------+------+---------------------+------+----------------+----------------+
    | 10000   | 2017-10-01 | 北京   |   20 |    0 | 2017-10-01 07:00:00 |   35 |             10 |              2 |
    | 10001   | 2017-10-01 | 北京   |   30 |    1 | 2017-10-01 17:05:45 |    2 |             22 |             22 |
    | 10002   | 2017-10-02 | 上海   |   20 |    1 | 2017-10-02 12:59:12 |  200 |              5 |              5 |
    | 10003   | 2017-10-02 | 广州   |   32 |    0 | 2017-10-02 11:20:00 |   30 |             11 |             11 |
    | 10004   | 2017-10-01 | 深圳   |   35 |    0 | 2017-10-01 10:00:15 |  100 |              3 |              3 |
    | 10004   | 2017-10-03 | 深圳   |   35 |    0 | 2017-10-03 10:20:22 |   11 |              6 |              6 |
    +---------+------------+--------+------+------+---------------------+------+----------------+----------------+
    6 rows in set (0.07 sec)
    
    mysql> select * from example_tbl where city='上海';
    +---------+------------+--------+------+------+---------------------+------+----------------+----------------+
    | user_id | date       | city   | age  | sex  | last_visit_date     | cost | max_dwell_time | min_dwell_time |
    +---------+------------+--------+------+------+---------------------+------+----------------+----------------+
    | 10002   | 2017-10-02 | 上海   |   20 |    1 | 2017-10-02 12:59:12 |  200 |              5 |              5 |
    +---------+------------+--------+------+------+---------------------+------+----------------+----------------+
    1 row in set (0.09 sec)
    
    mysql> select city, sum(cost) as total_cost from example_tbl group by city;
    +--------+------------+
    | city   | total_cost |
    +--------+------------+
    | 上海   |        200 |
    | 深圳   |        111 |
    | 北京   |         37 |
    | 广州   |         30 |
    +--------+------------+
    4 rows in set (0.13 sec)
    
    mysql>

[34d82bd4423e42bbb9df7ef47c050c64.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/21/011eaa9bafa94387877ae5dace76be34.png
[a56c6ac406e34a7c937c38eff569971a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/21/e19cc4ff44404464a5582bb84c4254dc.png
[188d20709e0b4a6dab948968005f9e4a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/21/6d45a35339ba4e5390c66f26a2016af1.png
[5826b47904844fdabc505c87ee1dc7ea.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/21/756d069259914530a1715b3e7269b596.png