Kylin下构建Cube第一步出错：shell-init: error retrieving current directory

灰太狼 2022-06-06 05:56 611阅读 0赞

问题背景：  
生产环境部署的Kylin-2.1，官方发布的最新安装包并不支持更改hbase存储的namespace，修改源码后重新打包部署过程中，build cube第一步出错

大概错误信息是：

OS command error exit with 5 – hive -e "USE default;
    DROP TABLE IF EXISTS kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000;
    CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000 ( DEFAULT_KYLIN_SALES_PART_DT date ,DEFAULT_KYLIN_SALES_LEAF_CATEG_ID bigint ,DEFAULT_KYLIN_SALES_LSTG_SITE_ID int ,DEFAULT_KYLIN_CATEGORY_GROUPINGS_META_CATEG_NAME string ,DEFAULT_KYLIN_CATEGORY_GROUPINGS_CATEG_LVL2_NAME string ,DEFAULT_KYLIN_CATEGORY_GROUPINGS_CATEG_LVL3_NAME string ,DEFAULT_KYLIN_SALES_LSTG_FORMAT_NAME string ,DEFAULT_KYLIN_SALES_PRICE decimal(19,4) ,DEFAULT_KYLIN_SALES_SELLER_ID bigint ) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\177' STORED AS SEQUENCEFILE LOCATION '/kylin/kylin_metadata/kylin-5020d91f-3114-421b-929b-f733d438411a/kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000';
    SET dfs.replication=2;
    SET dfs.block.size=33554432;
    SET hive.exec.compress.output=true;
    SET hive.auto.convert.join.noconditionaltask=true;
    SET hive.auto.convert.join.noconditionaltask.size=300000000;
    SET mapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;
    SET mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;
    SET hive.merge.mapfiles=true;
    SET hive.merge.mapredfiles=true;
    SET mapred.output.compression.type=BLOCK;
    SET hive.merge.size.per.task=256000000;
    SET hive.support.concurrency=false;
    SET mapreduce.job.split.metainfo.maxsize=-1;
    INSERT OVERWRITE TABLE kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000 SELECT KYLIN_SALES.PART_DT ,KYLIN_SALES.LEAF_CATEG_ID ,KYLIN_SALES.LSTG_SITE_ID ,KYLIN_CATEGORY_GROUPINGS.META_CATEG_NAME ,KYLIN_CATEGORY_GROUPINGS.CATEG_LVL2_NAME ,KYLIN_CATEGORY_GROUPINGS.CATEG_LVL3_NAME ,KYLIN_SALES.LSTG_FORMAT_NAME ,KYLIN_SALES.PRICE ,KYLIN_SALES.SELLER_ID FROM DEFAULT.KYLIN_SALES as KYLIN_SALES INNER JOIN DEFAULT.KYLIN_CAL_DT as KYLIN_CAL_DT ON KYLIN_SALES.PART_DT = KYLIN_CAL_DT.CAL_DT INNER JOIN DEFAULT.KYLIN_CATEGORY_GROUPINGS as KYLIN_CATEGORY_GROUPINGS ON KYLIN_SALES.LEAF_CATEG_ID = KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID AND KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_ID WHERE (KYLIN_SALES.PART_DT >= '2012-01-01' AND KYLIN_SALES.PART_DT < '2016-05-02') ;
    "
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    Unable to determine Hadoop version information.
    'hadoop version' returned:
    shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
    Error occurred during initialization of VM java.lang.Error: Properties init: Could not determine current working directory. at java.lang.System.initProperties(Native Method) at java.lang.System.initializeSystemClass(System.java:1119)

Kylin之前是没有问题的，构建了很多cube都没有问题，即便是出错按理讲也不会出现在第一步，第一步的动作是创建一个宽表，放到hive作为临时表。  
解决思路：  
第一感觉是权限问题，修改了sh脚本的执行权限，修改了hdfs上 kylin 工作目录的权限后，问题仍然存在。  
重新找原因，其中有报工作目录的问题，看上去是本地目录并不是Hdfs上的目录，这才想起可能是上次重新部署Kylin的时候把旧版本的直接删除 ，但并没有停掉进程。

ps aux | grep kylin

查看进程查然有两个Kylin进程在跑，于是Kill掉旧的进程

突然想起之前有人误删了kylin的文件夹，所以去查看了一下，果然某一台机器下面存在两个kylin进程。kill 后重启就好了。

猜测的原因是原进程的安装目录已经不存在了，并行着两个Kylin  
进程产生冲突。