CDH目录结构

谁践踏了优雅 2023-06-20 09:13 19阅读 0赞

# 一、总结目的 #

因为CDH集群的配置、日志、jar包以及安装文件的目录非常多，而且分散，每次绕进去就有种绕不出来的感觉。所以，下决心总结一下，防止自己后面采坑。

# 二、关键目录 #

### 1.开启应用目录：默认可以直接敲命令行 ###

1.  /opt/cloudera/parcels/CDH/bin
2.  查询
3.  \# ls
4.  avro-tools                kite-dataset   sqoop-create-hive-table
5.  beeline                   kudu           sqoop-eval
6.  bigtop-detect-javahome    llama          sqoop- export
7.  catalogd                  llamaadmin     sqoop-help
8.  cli\_mt                    load\_gen       sqoop- import
9.  cli\_st                    mahout         sqoop- import\-all-tables
10. flume-ng                  mapred         sqoop-job
11. hadoop                    oozie          sqoop- list\-databases
12. hadoop \-0.20               oozie-setup    sqoop- list\-tables
13. hadoop-fuse-dfs           parquet-tools  sqoop-merge
14. hadoop-fuse-dfs.orig      pig            sqoop-metastore
15. hbase                     pyspark        sqoop-version
16. hbase-indexer             sentry         statestored
17. hbase-indexer-sentry      solrctl        whirr
18. hcat                      spark-shell    yarn
19. hdfs                      spark-submit   zookeeper-client
20. hive                      sqoop          zookeeper-server
21. hiveserver2               sqoop2         zookeeper-server-cleanup
22. impala-collect-minidumps  sqoop2-server  zookeeper-server-initialize
23. impalad                   sqoop2-tool
24. impala-shell              sqoop-codegen

（1）zookeeper客户端命令位置

`/opt/cloudera/parcels/CDH/lib/zookeeper/bin/zkCli.sh`

### 2.日志目录（非常关键） ###

1.  /var/ log/下面有对应节点所开启服务的日志
2.  其中
3.  cloudera-scm-agent  CDH客户端日志
4.  cloudera-scm- server CDH服务端日志
5.  hadoop-hdfs hadoop日志目录
6.  hadoop-mapreduce
7.  hadoop-yarn
8.  hbase hbase日志目录
9.  hive hive日志目录
10. zookeeper  zookeeper日志目录

（1）solr日志位置

`tail -f /var/log/solr/solr-cmf-solr-SOLR_SERVER-…(hostname).log.out`

### 3.配置文件目录（非常关键） ###

（1）各个部件配置

1.  /etc/cloudera-scm-agent/config.ini cm agent的配置目录
2.  /etc/cloudera-scm- server/ cm server的配置目录
3.  
4.  /etc/hadoop/conf Hadoop各个组件的配置
5.  /etc/hive/conf hive配置文件目录
6.  /etc/hbase/conf hbase配置文件目录
7.  
8.  /opt/cloudera/parcels/CDH/etc是hadoop集群以及组件的配置文件文件夹

（2）服务运行时所有组件的配置文件目录

1.  / var/run/cloudera-scm-agent/process/
2.  查看内容
3.  \# ls
4.  244\-hdfs-NAMENODE
5.  245\-hdfs-SECONDARYNAMENODE
6.  246\-hdfs-NAMENODE-nnRpcWait
7.  247\-zookeeper-server
8.  250\-yarn-RESOURCEMANAGER
9.  254\-yarn-JOBHISTORY
10. 255\-hive-HIVESERVER2
11. 256\-hive-HIVEMETASTORE
12. 260\-hbase-MASTER
13. 280\-collect-host-statistics
14. 283\-host-inspector
15. 291\-cluster-host-inspector
16. 296\-cluster-host-inspector
17. 302\-hdfs-NAMENODE
18. 303\-hdfs-SECONDARYNAMENODE
19. 304\-hdfs-NAMENODE-nnRpcWait
20. 305\-hive-HIVEMETASTORE
21. 308\-collect-host-statistics
22. 311\-host-inspector
23. 320\-collect-host-statistics
24. 323\-host-inspector
25. 333\-collect-host-statistics
26. 336\-host-inspector
27. 343\-yarn-RESOURCEMANAGER
28. 347\-yarn-JOBHISTORY
29. 348\-hive-HIVESERVER2
30. 349\-hive-HIVEMETASTORE
31. 359\-yarn-RESOURCEMANAGER
32. 363\-yarn-JOBHISTORY
33. 364\-hive-HIVESERVER2
34. 365\-hive-HIVEMETASTORE
35. 377\-collect-host-statistics
36. 380\-host-inspector
37. 391\-collect-host-statistics
38. 394\-host-inspector
39. 403\-collect-host-statistics
40. 406\-host-inspector
41. 415\-solr-SOLR\_SERVER
42. ccdeploy\_hadoop-conf\_etchadoopconf.cloudera.hdfs\_2394207936436282336
43. ccdeploy\_hadoop-conf\_etchadoopconf.cloudera.yarn\_ \-5589955826917190240
44. ccdeploy\_hadoop-conf\_etchadoopconf.cloudera.yarn\_ \-5844902762347635403
45. ccdeploy\_hbase-conf\_etchbaseconf.cloudera.hbase\_1078272019303849382
46. ccdeploy\_hive-conf\_etchiveconf.cloudera.hive\_ \-2148401581528977976
47. ccdeploy\_hive-conf\_etchiveconf.cloudera.hive\_ \-9144129025529935636
48. ccdeploy\_solr-conf\_etcsolrconf.cloudera.solr\_4855637747225843610
49. ccdeploy\_spark2-conf\_etcspark2conf.cloudera.spark2\_on\_yarn\_ \-5389810983416787264
50. ccdeploy\_spark2-conf\_etcspark2conf.cloudera.spark2\_on\_yarn\_ \-8416474417781205702

### 4.安装各个组件目录 ###

`/opt/cloudera/parcels/CDH`

### 5.jar包目录 ###

（1）所有jar包所在目录

`/opt/cloudera/parcels/CDH/jars`

（2）各个服务组件对应的jar包

1.  /opt/cloudera/parcels/CDH/ lib/
2.  
3.  例如sqoop：/opt/cloudera/parcels/CDH/ lib/sqoop/ lib

### 6.Parcels包目录 ###

（1）服务软件包数据(parcels)

`/opt/cloudera/parcel-repo/`

（2）服务软件包缓存数据

`/opt/cloudera/parcel-cache/`