elastic-job任务覆盖的异常

不念不忘少年蓝@ 2022-09-11 01:11 195阅读 0赞

**前情提要**：

使用elastic-job-lite，job中对一个对象列表A进行批量计算。

**操作**：使用jmeter或直接调用接口，向对象列表A中增加一个对象，这时，先停止job，然后把对象列表修改完后重启job，JobName都是同一个。

**现象**：新的job没起来，旧的job被shutdown，导致新旧job都shutdown。

**重试**：第一次添加对象，列表中有1个对象，启动job成功；第二次添加对象，先停止，后启动，启动失败；第三次添加对象，因为之前的job已经shutdown，所以启动成功；第四次添加对象，同第二次相同；以此类推。**即：JobName相同，新添加job会导致旧job无法启动且新job也无法启动。**

**异常**问题出现了，具体异常信息为：

com.dangdang.ddframe.job.exception.JobSystemException: java.lang.NullPointerException
    	at com.dangdang.ddframe.job.lite.internal.storage.JobNodeStorage.handleException(JobNodeStorage.java:202)
    	at com.dangdang.ddframe.job.lite.internal.storage.JobNodeStorage.executeInLeader(JobNodeStorage.java:194)
    	at com.dangdang.ddframe.job.lite.internal.election.LeaderService.electLeader(LeaderService.java:54)
    	at com.dangdang.ddframe.job.lite.internal.schedule.SchedulerFacade.registerStartUpInfo(SchedulerFacade.java:114)
    	at com.dangdang.ddframe.job.lite.api.JobScheduler.init(JobScheduler.java:110)
    	at com.emcc.indicator.elasticjob.IndicatorJobStarter.addJob(IndicatorJobStarter.java:232)
    	at com.emcc.indicator.elasticjob.IndicatorJobStarter.addJob(IndicatorJobStarter.java:123)
    	at com.emcc.indicator.elasticjob.ElasticJobUtil.createAndStartJob(ElasticJobUtil.java:93)
    	at com.emcc.indicator.controller.IndicatorController.addIndicator(IndicatorController.java:203)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
    	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
    	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:849)
    	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:760)
    	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
    	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
    	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
    	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
    	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
    	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
    	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507)
    	at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
    	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
    	at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:111)
    	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
    	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
    	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
    	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
    	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
    	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
    	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
    	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
    	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
    	at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103)
    	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
    	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
    	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)
    	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    	at org.eclipse.jetty.server.Server.handle(Server.java:370)
    	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
    	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:984)
    	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1045)
    	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
    	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:236)
    	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
    	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
    	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    	at java.lang.Thread.run(Thread.java:748)
    Caused by: java.lang.NullPointerException: null
    	at com.dangdang.ddframe.job.lite.internal.election.LeaderService$LeaderElectionExecutionCallback.execute(LeaderService.java:109)
    	at com.dangdang.ddframe.job.lite.internal.storage.JobNodeStorage.executeInLeader(JobNodeStorage.java:190)
    	... 69 common frames omitted

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAbWVuZ2d1ZGFva2U_size_20_color_FFFFFF_t_70_g_se_x_16][]

原因：

问题原因是一直在使用相同的JobName，即我停止job之前，先拿到JobName，然后修改完对象列表，又用这个JobName去启动修改后对象列表的job。

猜测：

1.zookeeper中有残留？

2.旧的job有残留，导致启动新的job不成功，然后新的和旧的job都启动失败？

[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAbWVuZ2d1ZGFva2U_size_20_color_FFFFFF_t_70_g_se_x_16]: /images/20220829/a83a85d3b0ca4623bfd3dd0847f7f170.png