Apache Kudu(一) 安装

比眉伴天荒 2023-03-13 03:37 24阅读 0赞

## Kudu 安装 ##

### 快速开始 ###

准备好docker环境，如果没有请自行搜索

#### [Clone the Repository][] ####

git clone https://github.com/apache/kudu
    cd kudu

#### 启动集群 ####

##### 设置IP #####

export KUDU_QUICKSTART_IP=$(ifconfig | grep "inet " | grep -Fv 127.0.0.1 | awk '{print $2}' | tail -1)

##### 启动 #####

Then use `docker-compose` to start a cluster with 3 master servers and 5 tablet servers. When inside the docker network/containers the master addresses will be `kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251` and when on the host machine you can specify the master addresses with `localhost:7051,localhost:7151,localhost:7251`.

docker-compose -f docker/quickstart.yml up -d

> You can include the `-d` flag to run the cluster in the background.

##### 查看kudu控制台界面 #####

Once the cluster is started you can view the master web-ui by visiting localhost:8050.

![image-20200504161514304][]

##### 检查集群健康 #####

Use the command below to get a bash shell in the `kudu-master-1` container:

docker exec -it $(docker ps -aqf "name=kudu-master-1") /bin/bash

You can now run the Kudu `ksck` tool to verify the cluster is healthy:

kudu cluster ksck kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251

Alternatively, if you have a kudu binary available on your host machine, you can run `ksck` there via:

export KUDU_USER_NAME=kudu
    kudu cluster ksck localhost:7051,localhost:7151,localhost:7251

![image-20200504171119034][]

### 生产环境安装 ###

### 命令 ###

*  [cluster][]
    
     *  [ksck][]
     *  [rebalance][]
 *  [diagnose][]
    
     *  [parse\_stacks][parse_stacks]
 *  [fs][]
    
     *  [check][]
     *  [format][]
     *  [list][]
     *  [update\_dirs][update_dirs]
     *  [dump][]
        
         *  [block][]
         *  [cfile][]
         *  [tree][]
         *  [uuid][]
 *  [hms][]
    
     *  [check][check 1]
     *  [downgrade][]
     *  [fix][]
     *  [list][list 1]
     *  [precheck][]
 *  [local\_replica][local_replica]
    
     *  [copy\_from\_remote][copy_from_remote]
     *  [data\_size][data_size]
     *  [delete][]
     *  [list][list 2]
     *  [cmeta][]
        
         *  [print\_replica\_uuids][print_replica_uuids]
         *  [rewrite\_raft\_config][rewrite_raft_config]
         *  [set\_term][set_term]
     *  [dump][dump 1]
        
         *  [block\_ids][block_ids]
         *  [data\_dirs][data_dirs]
         *  [meta][]
         *  [rowset][]
         *  [wals][]
 *  [master][]
    
     *  [dump\_memtrackers][dump_memtrackers]
     *  [get\_flags][get_flags]
     *  [set\_flag][set_flag]
     *  [status][]
     *  [timestamp][]
     *  [list][list 3]
     *  [authz\_cache][authz_cache]
        
         *  [reset][]
 *  [pbc][]
    
     *  [dump][dump 2]
     *  [edit][]
 *  [perf][]
    
     *  [loadgen][]
     *  [table\_scan][table_scan]
     *  [tablet\_scan][tablet_scan]
 *  [remote\_replica][remote_replica]
    
     *  [check][check 2]
     *  [copy][]
     *  [delete][delete 1]
     *  [dump][dump 3]
     *  [list][list 4]
     *  [unsafe\_change\_config][unsafe_change_config]
 *  [table][]
    
     *  [add\_range\_partition][add_range_partition]
     *  [column\_set\_default][column_set_default]
     *  [column\_remove\_default][column_remove_default]
     *  [column\_set\_compression][column_set_compression]
     *  [column\_set\_encoding][column_set_encoding]
     *  [column\_set\_block\_size][column_set_block_size]
     *  [delete\_column][delete_column]
     *  [delete][delete 2]
     *  [describe][]
     *  [drop\_range\_partition][drop_range_partition]
     *  [list][list 5]
     *  [locate\_row][locate_row]
     *  [rename\_column][rename_column]
     *  [rename\_table][rename_table]
     *  [scan][]
     *  [copy][copy 1]
     *  [set\_extra\_config][set_extra_config]
     *  [get\_extra\_configs][get_extra_configs]
     *  [statistics][]
 *  [tablet][]
    
     *  [leader\_step\_down][leader_step_down]
     *  [unsafe\_replace\_tablet][unsafe_replace_tablet]
     *  [change\_config][change_config]
        
         *  [add\_replica][add_replica]
         *  [change\_replica\_type][change_replica_type]
         *  [move\_replica][move_replica]
         *  [remove\_replica][remove_replica]
 *  [test][]
    
     *  [mini\_cluster][mini_cluster]
 *  [tserver][]
    
     *  [dump\_memtrackers][dump_memtrackers 1]
     *  [get\_flags][get_flags 1]
     *  [set\_flag][set_flag 1]
     *  [status][status 1]
     *  [timestamp][timestamp 1]
     *  [list][list 6]
     *  [state][]
        
         *  [enter\_maintenance][enter_maintenance]
         *  [exit\_maintenance][exit_maintenance]
 *  [wal][]
    
     *  [dump][dump 4]

#### 集群 ####

#集群健康：
    kudu cluster ksck kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251

#### 表 ####

# 1.  类似 show tables
    kudu@92b472a0be64:/opt/kudu$ kudu table list kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
    student
    student2
    
    # 2. 扫描表 类似 select * from student2
    kudu@92b472a0be64:/opt/kudu$ kudu table scan kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251 student2
    (int32 id=4, string name="zhangsan-4", int32 age=24, int32 sex=0)
    (int32 id=1, string name="zhangsan-1", int32 age=21, int32 sex=1)
    (int32 id=5, string name="zhangsan-5", int32 age=25, int32 sex=1)
    (int32 id=6, string name="zhangsan-6", int32 age=26, int32 sex=0)
    (int32 id=7, string name="zhangsan-7", int32 age=27, int32 sex=1)
    T b7ffe064c8894abaaf955c12a097f2e4 scanned count 1 cost 0.0124023 seconds
    T 541e387b121643b499a1eb164a864eff scanned count 4 cost 0.0124951 seconds
    T 4199041ed86e45da8c54ed78350fd7f9 scanned count 0 cost 0.0118958 seconds
    (int32 id=3, string name="zhangsan-3", int32 age=23, int32 sex=1)
    (int32 id=10, string name="zhangsan-10", int32 age=30, int32 sex=0)
    T 632e49f6c5874e2c96a8e53b5953a442 scanned count 2 cost 0.0126947 seconds
    (int32 id=8, string name="zhangsan-8", int32 age=28, int32 sex=0)
    (int32 id=9, string name="zhangsan-9", int32 age=29, int32 sex=1)
    T 137406191ebe40d9878c4d197c2afb4f scanned count 2 cost 0.00449864 seconds
    T 851b8c9d8cf645adabaff8e624e27508 scanned count 0 cost 0.00425724 seconds
    Total count 9 cost 0.100472 seconds
    
    # 3. 表分析
    kudu@92b472a0be64:/opt/kudu$ kudu table statistics kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251 student2
    TABLE student2
    on disk size: 50492574
    live row count: 9
    
    # 4. 表详情
    kudu@92b472a0be64:/opt/kudu$ kudu table describe kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251 student2
    TABLE student2 (
        id INT32 NOT NULL,
        name STRING NOT NULL,
        age INT32 NOT NULL,
        sex INT32 NOT NULL,
        PRIMARY KEY (id)
    )
    HASH (id) PARTITIONS 6,
    RANGE (id) (
        PARTITION UNBOUNDED
    )
    REPLICAS 3

> CDH 版本自行搜索

[Clone the Repository]: https://kudu.apache.org/docs/quickstart.html#_clone_the_repository
[image-20200504161514304]: /images/20230312/3545d949d76f4a92895a303835ec46d9.png
[image-20200504171119034]: /images/20230312/efb37dacea0342b4862899b3f8360135.png
[cluster]: https://kudu.apache.org/docs/command_line_tools_reference.html#cluster
[ksck]: https://kudu.apache.org/docs/command_line_tools_reference.html#cluster-ksck
[rebalance]: https://kudu.apache.org/docs/command_line_tools_reference.html#cluster-rebalance
[diagnose]: https://kudu.apache.org/docs/command_line_tools_reference.html#diagnose
[parse_stacks]: https://kudu.apache.org/docs/command_line_tools_reference.html#diagnose-parse_stacks
[fs]: https://kudu.apache.org/docs/command_line_tools_reference.html#fs
[check]: https://kudu.apache.org/docs/command_line_tools_reference.html#fs-check
[format]: https://kudu.apache.org/docs/command_line_tools_reference.html#fs-format
[list]: https://kudu.apache.org/docs/command_line_tools_reference.html#fs-list
[update_dirs]: https://kudu.apache.org/docs/command_line_tools_reference.html#fs-update_dirs
[dump]: https://kudu.apache.org/docs/command_line_tools_reference.html#fs-dump
[block]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-block
[cfile]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-cfile
[tree]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-tree
[uuid]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-uuid
[hms]: https://kudu.apache.org/docs/command_line_tools_reference.html#hms
[check 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#hms-check
[downgrade]: https://kudu.apache.org/docs/command_line_tools_reference.html#hms-downgrade
[fix]: https://kudu.apache.org/docs/command_line_tools_reference.html#hms-fix
[list 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#hms-list
[precheck]: https://kudu.apache.org/docs/command_line_tools_reference.html#hms-precheck
[local_replica]: https://kudu.apache.org/docs/command_line_tools_reference.html#local_replica
[copy_from_remote]: https://kudu.apache.org/docs/command_line_tools_reference.html#local_replica-copy_from_remote
[data_size]: https://kudu.apache.org/docs/command_line_tools_reference.html#local_replica-data_size
[delete]: https://kudu.apache.org/docs/command_line_tools_reference.html#local_replica-delete
[list 2]: https://kudu.apache.org/docs/command_line_tools_reference.html#local_replica-list
[cmeta]: https://kudu.apache.org/docs/command_line_tools_reference.html#local_replica-cmeta
[print_replica_uuids]: https://kudu.apache.org/docs/command_line_tools_reference.html#cmeta-print_replica_uuids
[rewrite_raft_config]: https://kudu.apache.org/docs/command_line_tools_reference.html#cmeta-rewrite_raft_config
[set_term]: https://kudu.apache.org/docs/command_line_tools_reference.html#cmeta-set_term
[dump 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#local_replica-dump
[block_ids]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-block_ids
[data_dirs]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-data_dirs
[meta]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-meta
[rowset]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-rowset
[wals]: https://kudu.apache.org/docs/command_line_tools_reference.html#dump-wals
[master]: https://kudu.apache.org/docs/command_line_tools_reference.html#master
[dump_memtrackers]: https://kudu.apache.org/docs/command_line_tools_reference.html#master-dump_memtrackers
[get_flags]: https://kudu.apache.org/docs/command_line_tools_reference.html#master-get_flags
[set_flag]: https://kudu.apache.org/docs/command_line_tools_reference.html#master-set_flag
[status]: https://kudu.apache.org/docs/command_line_tools_reference.html#master-status
[timestamp]: https://kudu.apache.org/docs/command_line_tools_reference.html#master-timestamp
[list 3]: https://kudu.apache.org/docs/command_line_tools_reference.html#master-list
[authz_cache]: https://kudu.apache.org/docs/command_line_tools_reference.html#master-authz_cache
[reset]: https://kudu.apache.org/docs/command_line_tools_reference.html#authz_cache-reset
[pbc]: https://kudu.apache.org/docs/command_line_tools_reference.html#pbc
[dump 2]: https://kudu.apache.org/docs/command_line_tools_reference.html#pbc-dump
[edit]: https://kudu.apache.org/docs/command_line_tools_reference.html#pbc-edit
[perf]: https://kudu.apache.org/docs/command_line_tools_reference.html#perf
[loadgen]: https://kudu.apache.org/docs/command_line_tools_reference.html#perf-loadgen
[table_scan]: https://kudu.apache.org/docs/command_line_tools_reference.html#perf-table_scan
[tablet_scan]: https://kudu.apache.org/docs/command_line_tools_reference.html#perf-tablet_scan
[remote_replica]: https://kudu.apache.org/docs/command_line_tools_reference.html#remote_replica
[check 2]: https://kudu.apache.org/docs/command_line_tools_reference.html#remote_replica-check
[copy]: https://kudu.apache.org/docs/command_line_tools_reference.html#remote_replica-copy
[delete 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#remote_replica-delete
[dump 3]: https://kudu.apache.org/docs/command_line_tools_reference.html#remote_replica-dump
[list 4]: https://kudu.apache.org/docs/command_line_tools_reference.html#remote_replica-list
[unsafe_change_config]: https://kudu.apache.org/docs/command_line_tools_reference.html#remote_replica-unsafe_change_config
[table]: https://kudu.apache.org/docs/command_line_tools_reference.html#table
[add_range_partition]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-add_range_partition
[column_set_default]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-column_set_default
[column_remove_default]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-column_remove_default
[column_set_compression]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-column_set_compression
[column_set_encoding]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-column_set_encoding
[column_set_block_size]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-column_set_block_size
[delete_column]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-delete_column
[delete 2]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-delete
[describe]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-describe
[drop_range_partition]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-drop_range_partition
[list 5]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-list
[locate_row]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-locate_row
[rename_column]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-rename_column
[rename_table]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-rename_table
[scan]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-scan
[copy 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-copy
[set_extra_config]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-set_extra_config
[get_extra_configs]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-get_extra_configs
[statistics]: https://kudu.apache.org/docs/command_line_tools_reference.html#table-statistics
[tablet]: https://kudu.apache.org/docs/command_line_tools_reference.html#tablet
[leader_step_down]: https://kudu.apache.org/docs/command_line_tools_reference.html#tablet-leader_step_down
[unsafe_replace_tablet]: https://kudu.apache.org/docs/command_line_tools_reference.html#tablet-unsafe_replace_tablet
[change_config]: https://kudu.apache.org/docs/command_line_tools_reference.html#tablet-change_config
[add_replica]: https://kudu.apache.org/docs/command_line_tools_reference.html#change_config-add_replica
[change_replica_type]: https://kudu.apache.org/docs/command_line_tools_reference.html#change_config-change_replica_type
[move_replica]: https://kudu.apache.org/docs/command_line_tools_reference.html#change_config-move_replica
[remove_replica]: https://kudu.apache.org/docs/command_line_tools_reference.html#change_config-remove_replica
[test]: https://kudu.apache.org/docs/command_line_tools_reference.html#test
[mini_cluster]: https://kudu.apache.org/docs/command_line_tools_reference.html#test-mini_cluster
[tserver]: https://kudu.apache.org/docs/command_line_tools_reference.html#tserver
[dump_memtrackers 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#tserver-dump_memtrackers
[get_flags 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#tserver-get_flags
[set_flag 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#tserver-set_flag
[status 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#tserver-status
[timestamp 1]: https://kudu.apache.org/docs/command_line_tools_reference.html#tserver-timestamp
[list 6]: https://kudu.apache.org/docs/command_line_tools_reference.html#tserver-list
[state]: https://kudu.apache.org/docs/command_line_tools_reference.html#tserver-state
[enter_maintenance]: https://kudu.apache.org/docs/command_line_tools_reference.html#state-enter_maintenance
[exit_maintenance]: https://kudu.apache.org/docs/command_line_tools_reference.html#state-exit_maintenance
[wal]: https://kudu.apache.org/docs/command_line_tools_reference.html#wal
[dump 4]: https://kudu.apache.org/docs/command_line_tools_reference.html#wal-dump