centos6.5安装hive-1.2.2

淩亂°似流年 2022-04-13 12:19 204阅读 0赞

### 开始安装 ###

下载地址：[https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-1.2.2/][https_mirrors.tuna.tsinghua.edu.cn_apache_hive_hive-1.2.2]

上传到/opt/soft 目录，然后解压 tar -zxvf apache-hive-1.2.2-bin.tar.gz

配置hive环境变量 vim /etc/profile 添加如下内容

export HIVE_HOME=/opt/soft/apache-hive-1.2.2-bin
    export PATH=$HIVE_HOME/bin:$PATH

刷新环境变量  
source /etc/profile

注意：hive启动之前先启动hdfs和yarn ，命令  start-dfs.sh  和start-yarn.sh

hadoop安装参照另一篇博客[https://blog.csdn.net/xiaobao5214/article/details/84315394][https_blog.csdn.net_xiaobao5214_article_details_84315394]

### 单机模式配置 hive-site.xml ###

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/opt/data/hive/warehouse</value>
    </property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.2.14:3306/hive?createDatabaseIfNotExist=true</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>123456</value>
    </property>
    </configuration>

单机模式启动命令： hive

即可进入hive命令模式

### 远程模式配置hive-site.xml ###

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/opt/data/hive/warehouse</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.2.14:3306/hive?createDatabaseIfNotExist=true</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>123456</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://192.168.2.14:9083</value>
    </property>
    </configuration>

hive远程模式启动时首先启动hive元数据服务   hive --service metastore

然后输入 hive 即可进入远程模式命令行

### hive常见操作 ###

列出所有数据库

hive>  show databases;

切换到 default库

hive>  use default;

列出所有表

hive>  show tables;

创建表范例  person表

hive>  create table person(id  int,name  string,age int,sex string) row format delimited fields terminated by '\t';

编辑要导入的person数据文件

vim /home/data/person.txt
    
    1001    小宝        26    男
    1002    小红        10    女
    1003    小蓝        17    男
    1004    小绿        23    男
    1005    小紫        25    女
    1006    小白        22    女
    1007    小青        23    男
    1008    小黑        19    男

导入person.txt 到hive仓库 （如需导入多个文件且在同一文件夹，没有多余文件，可以不指定具体文件，只写文件夹路径即可）

hive> load data local inpath '/home/data/person.txt' overwrite into table person;

查询person表所有数据按年级升序

hive> select * from person order by age asc;
    Query ID = root_20181129141413_8a4d30e2-81b9-409e-a147-63710b8e37d8
    Total jobs = 1
    Launching Job 1 out of 1
    Number of reduce tasks determined at compile time: 1
    In order to change the average load for a reducer (in bytes):
      set hive.exec.reducers.bytes.per.reducer=<number>
    In order to limit the maximum number of reducers:
      set hive.exec.reducers.max=<number>
    In order to set a constant number of reducers:
      set mapreduce.job.reduces=<number>
    Starting Job = job_1543455989290_0003, Tracking URL = http://hadoop4:8088/proxy/application_1543455989290_0003/
    Kill Command = /opt/soft/hadoop-2.6.0/bin/hadoop job  -kill job_1543455989290_0003
    Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
    2018-11-29 14:14:31,023 Stage-1 map = 0%,  reduce = 0%
    2018-11-29 14:14:42,383 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.52 sec
    2018-11-29 14:14:54,520 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.55 sec
    MapReduce Total cumulative CPU time: 4 seconds 550 msec
    Ended Job = job_1543455989290_0003
    MapReduce Jobs Launched: 
    Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 4.55 sec   HDFS Read: 6488 HDFS Write: 224 SUCCESS
    Total MapReduce CPU Time Spent: 4 seconds 550 msec
    OK
    1002	小红	10	女
    1003	小蓝	17	男
    1008	小黑	19	男
    1006	小白	22	女
    1007	小青	23	男
    1004	小绿	23	男
    1005	小紫	25	女
    1001	小宝	26	男
    Time taken: 42.718 seconds, Fetched: 8 row(s)

显示person表结构

hive>  desc person;
    id                  	int                 	                    
    name                	string              	                    
    age                 	int                 	                    
    sex                 	string

查询数据然后导出到本地文件中第一种方式（hive命令行中执行，需要指定分隔符，这里用\\t分隔）

hive> insert overwrite local directory '/home/data/' row format delimited fields terminated by '\t' select * from person order by age desc;

查询数据然后导出到本地文件中第二种方式 （sql写到命令行，导出结果会自动按 \\t 分隔，）

[root@hadoop4 data]# hive -e 'select * from person order by age desc' > /home/data/person2.txt

查询数据然后导出到本地文件中第三种方式 （sql写到文件中，导出结果会自动按 \\t 分隔，person.sql一定要写绝对路径） person.sql  中的内容： select \* from person order by age desc

[root@hadoop4 data]# hive -f /home/data/person.sql > /home/data/person2.txt

### hive表在hdfs上的存储 ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpYW9iYW81MjE0_size_16_color_FFFFFF_t_70][]

### hive查询时启动mapreduce程序 ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpYW9iYW81MjE0_size_16_color_FFFFFF_t_70 1][]

[https_mirrors.tuna.tsinghua.edu.cn_apache_hive_hive-1.2.2]: https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-1.2.2/
[https_blog.csdn.net_xiaobao5214_article_details_84315394]: https://blog.csdn.net/xiaobao5214/article/details/84315394
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpYW9iYW81MjE0_size_16_color_FFFFFF_t_70]: /images/20220413/36ae1b43541c4facac09f7b8e69776c1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpYW9iYW81MjE0_size_16_color_FFFFFF_t_70 1]: /images/20220413/61c7cb4d53454a30973da8e2e1c1a0cb.png