阿里云 blink 相关错误码问题解决

缺乏、安全感 2023-01-05 15:27 174阅读 0赞

1、使用阿里云的flink平台是，有时重启任务会有shell cmd的报错信息，错误代码如下：

Submit job for the [21] times,last error occurred at [Sun Jan 10 18:36:31 CST 2021],lastErrorMessage is as follows:
    
    run instance failed, BLINK error: bayes submit job failed.job name:[test_kafka], brs errorCode:[30013], brs errorMessage:
    
    code:[30013], brief info:[error occur while run app], context info:[details:
    >>>>>
    shell cmd execute failed
    
    2021-01-10 18:36:26,104 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
    2021-01-10 18:36:26,104 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
    2021-01-10 18:36:26,105 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - org.apache.commons.cli.CommandLine@7fd50002
    2021-01-10 18:36:26,105 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - org.apache.commons.cli.CommandLine@7fd50002
    org.apache.commons.cli.CommandLine@7fd50002
    
    ------------------------------------------------------------
     The program finished with the following exception:
    
    org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.
    	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:610)
    	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:469)
    	at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:94)
    	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:81)
    	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:134)
    	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:253)
    	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:217)
    	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1075)
    	at org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1151)
    	at java.security.AccessController.doPrivileged(Native Method)
    	at javax.security.auth.Subject.doAs(Subject.java:422)
    	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
    	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
    	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1151)
    Caused by: com.alibaba.blink.launcher.autoconfig.UnexpectedConfigurationException: 
    ************
    ERR_ID:
         CLI-00000001
    CAUSE:
         AutoConf: an error occurred while applying the resource configuration file.
         Node or edge number in stream graph and json file not matched, please generate a new json.
    ACTION:
         The reason may be that the content of the SQL has been modified or the blink version number has been changed.
         Please regenerate the resource configuration file.
    DETAIL:
    
    ************
    	at com.alibaba.blink.launcher.autoconfig.StreamGraphConfigurer.checkValid(StreamGraphConfigurer.java:58)
    	at com.alibaba.blink.launcher.autoconfig.StreamGraphConfigurer.configure(StreamGraphConfigurer.java:45)
    	at com.alibaba.blink.launcher.JobLauncher.execute(JobLauncher.java:663)
    	at com.alibaba.blink.launcher.JobLauncher.runStream(JobLauncher.java:545)
    	at com.alibaba.blink.launcher.JobLauncher.main(JobLauncher.java:159)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
    	... 13 more
    <<<<<], cause:shell cmd execute failed
    
    2021-01-10 18:36:26,104 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
    2021-01-10 18:36:26,104 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
    2021-01-10 18:36:26,105 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - org.apache.commons.cli.CommandLine@7fd50002
    2021-01-10 18:36:26,105 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                 - org.apache.commons.cli.CommandLine@7fd50002
    org.apache.commons.cli.CommandLine@7fd50002
    
    ------------------------------------------------------------
     The program finished with the following exception:
    
    org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.
    	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:610)
    	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:469)
    	at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:94)
    	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:81)
    	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:134)
    	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:253)
    	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:217)
    	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1075)
    	at org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1151)
    	at java.security.AccessController.doPrivileged(Native Method)
    	at javax.security.auth.Subject.doAs(Subject.java:422)
    	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
    	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
    	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1151)
    Caused by: com.alibaba.blink.launcher.autoconfig.UnexpectedConfigurationException: 
    ************
    ERR_ID:
         CLI-00000001
    CAUSE:
         AutoConf: an error occurred while applying the resource configuration file.
         Node or edge number in stream graph and json file not matched, please generate a new json.
    ACTION:
         The reason may be that the content of the SQL has been modified or the blink version number has been changed.
         Please regenerate the resource configuration file.
    DETAIL:
    
    ************
    	at com.alibaba.blink.launcher.autoconfig.StreamGraphConfigurer.checkValid(StreamGraphConfigurer.java:58)
    	at com.alibaba.blink.launcher.autoconfig.StreamGraphConfigurer.configure(StreamGraphConfigurer.java:45)
    	at com.alibaba.blink.launcher.JobLauncher.execute(JobLauncher.java:663)
    	at com.alibaba.blink.launcher.JobLauncher.runStream(JobLauncher.java:545)
    	at com.alibaba.blink.launcher.JobLauncher.main(JobLauncher.java:159)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593)
    	... 13 more

2、是因为改了执行计划，没有重新获取执行计划导致的，需要重新执行    “ 放弃当前配置” ==》“重新获取配置信息”==》“应用当前配置” 来解决这个问题

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dwcHdwcDE_size_16_color_FFFFFF_t_70][]

3、在执行监控的过程中，也发现某个流任务的停止和启动，也会影响下游流任务的延迟。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dwcHdwcDE_size_16_color_FFFFFF_t_70 1][]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dwcHdwcDE_size_16_color_FFFFFF_t_70]: /images/20221119/601d7df1ddee499880d67936b8941325.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dwcHdwcDE_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20210113113945732.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dwcHdwcDE=,size_16,color_FFFFFF,t_70