KETTLE_内存溢出错误

╰+攻爆jí腚メ 2022-02-20 00:37 1700阅读 0赞

转自：[https://blog.csdn.net/huangyanlong/article/details/42453831][https_blog.csdn.net_huangyanlong_article_details_42453831]

深蓝的blog：http://blog.csdn.net/huangyanlong/article/details/42453831

kettle内存溢出错误解决

环境：  
源端数据库：oracle 10G R2  
目标端数据库：oracle 11G R2  
kettle版本：5.0.1-stable

报错：  
抽取大数据量表时，报错，日志信息如下：

2015/01/05 11:27:42 - Spoon - 转换已经打开.  
2015/01/05 11:27:42 - Spoon - 正在打开转换 \[TABLE\_BWWN\]...  
2015/01/05 11:27:42 - Spoon - 开始执行转换.  
2015/01/05 11:27:42 - TABLE\_BWWN - 为了转换解除补丁开始  \[TABLE\_BWWN\]  
2015/01/05 11:27:44 - max.0 - Finished reading query, closing connection.  
2015/01/05 11:27:44 - max.0 - 完成处理 (I=1, O=0, R=0, W=1, U=0, E=0  
2015/01/05 12:25:08 - 插入 / 更新.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : UnexpectedError:  
2015/01/05 12:25:08 - Spoon - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : 一个未预期的错误发生在Spoon: probable cause:在停止Spoon前，请先关闭其它spoon窗口!   
2015/01/05 12:25:08 - Spoon - Java heap space  
2015/01/05 12:25:08 - 插入 / 更新.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : java.lang.OutOfMemoryError: Java heap space  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at oracle.jdbc.driver.OraclePreparedStatement.bindBytesAsStream(OraclePreparedStatement.java:6713)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at oracle.jdbc.driver.OraclePreparedStatement.setBytesInternal(OraclePreparedStatement.java:6700)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at oracle.jdbc.driver.OraclePreparedStatement.setBytes(OraclePreparedStatement.java:6660)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at oracle.jdbc.driver.OraclePreparedStatementWrapper.setBytes(OraclePreparedStatementWrapper.java:150)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at org.pentaho.di.core.row.value.ValueMetaBase.setPreparedStatementValue(ValueMetaBase.java:4704)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at org.pentaho.di.core.database.Database.setValue(Database.java:928)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at org.pentaho.di.core.database.Database.setValues(Database.java:943)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at org.pentaho.di.core.database.Database.setValuesInsert(Database.java:888)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at org.pentaho.di.trans.steps.insertupdate.InsertUpdate.lookupValues(InsertUpdate.java:113)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at org.pentaho.di.trans.steps.insertupdate.InsertUpdate.processRow(InsertUpdate.java:301)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at org.pentaho.di.trans.step.RunThread.run(RunThread.java:60)  
2015/01/05 12:25:08 - 插入 / 更新.0 -  at java.lang.Thread.run(Unknown Source)  
2015/01/05 12:25:08 - Spoon - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : java.lang.OutOfMemoryError: Java heap space  
2015/01/05 12:25:08 - Spoon -  at java.util.Arrays.copyOf(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at java.lang.StringCoding.safeTrim(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at java.lang.StringCoding.access$100(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at java.lang.StringCoding$StringDecoder.decode(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at java.lang.StringCoding.decode(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at java.lang.StringCoding.decode(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at java.lang.String.<init>(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at java.lang.String.<init>(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.core.row.value.ValueMetaBase.convertBinaryStringToString(ValueMetaBase.java:1105)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.core.row.value.ValueMetaBase.getString(ValueMetaBase.java:1452)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.core.row.RowMeta.getString(RowMeta.java:230)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.ui.spoon.trans.TransPreviewDelegate.showPreviewGrid(TransPreviewDelegate.java:302)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.ui.spoon.trans.TransPreviewDelegate.refreshView(TransPreviewDelegate.java:268)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.ui.spoon.trans.TransPreviewDelegate$1.widgetSelected(TransPreviewDelegate.java:188)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.Widget.notifyListeners(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.custom.CTabFolder.setSelection(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.custom.CTabFolder.onMouse(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.custom.CTabFolder$1.handleEvent(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1227)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7368)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:8673)  
2015/01/05 12:25:08 - Spoon -  at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:625)  
2015/01/05 12:25:08 - Spoon -  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
2015/01/05 12:25:09 - 插入 / 更新.0 - 完成处理 (I=20575, O=20574, R=20575, W=20574, U=0, E=1  
2015/01/05 12:25:09 - TABLE\_BWWN - TABLE\_BWWN  
2015/01/05 12:25:09 - TABLE\_BWWN - TABLE\_BWWN  
2015/01/05 12:25:09 - 表输入.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : Unexpected error  
2015/01/05 12:25:09 - 表输入.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException:   
2015/01/05 12:25:09 - 表输入.0 - Couldn't get row from result set  
2015/01/05 12:25:09 - 表输入.0 - ORA-03111: 通信通道收到中断

2015/01/05 12:25:09 - 表输入.0 -   
2015/01/05 12:25:09 - 表输入.0 -  at org.pentaho.di.core.database.Database.getRow(Database.java:2302)  
2015/01/05 12:25:09 - 表输入.0 -  at org.pentaho.di.core.database.Database.getRow(Database.java:2270)  
2015/01/05 12:25:09 - 表输入.0 -  at org.pentaho.di.trans.steps.tableinput.TableInput.processRow(TableInput.java:153)  
2015/01/05 12:25:09 - 表输入.0 -  at org.pentaho.di.trans.step.RunThread.run(RunThread.java:60)  
2015/01/05 12:25:09 - 表输入.0 -  at java.lang.Thread.run(Unknown Source)  
2015/01/05 12:25:09 - 表输入.0 - Caused by: java.sql.SQLException: ORA-03111: 通信通道收到中断

2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:205)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.T4CPreparedStatement.fetch(T4CPreparedStatement.java:1082)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.OracleResultSetImpl.close\_or\_fetch\_from\_next(OracleResultSetImpl.java:369)  
2015/01/05 12:25:09 - 表输入.0 -  at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:273)  
2015/01/05 12:25:09 - 表输入.0 -  at org.pentaho.di.core.database.Database.getRow(Database.java:2290)  
2015/01/05 12:25:09 - 表输入.0 -  ... 4 more  
2015/01/05 12:25:09 - 表输入.0 - Finished reading query, closing connection.  
2015/01/05 12:25:09 - 表输入.0 - 完成处理 (I=20580, O=0, R=1, W=20579, U=0, E=1  
2015/01/05 12:25:09 - TABLE\_BWWN - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : 错误被检测到!  
2015/01/05 12:25:09 - Spoon - 转换完成!!  
2015/01/05 12:25:09 - TABLE\_BWWN - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : 错误被检测到!  
2015/01/05 12:25:09 - TABLE\_BWWN - ERROR (version 5.0.1-stable, build 1 from 2013-11-15\_16-08-58 by buildguy) : 错误被检测到!  
2015/01/05 12:25:09 - TABLE\_BWWN - TABLE\_BWWN  
2015/01/05 12:25:09 - TABLE\_BWWN - TABLE\_BWWN  
解决方法：  
找到kettle目录下的Spoon.bat文件，找到如下部分：

REM \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
REM \*\* Set java runtime options                                     \*\*  
REM \*\* Change 512m to higher values in case you run out of memory   \*\*  
REM \*\* or set the PENTAHO\_DI\_JAVA\_OPTIONS environment variable      \*\*  
REM \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

if "%PENTAHO\_DI\_JAVA\_OPTIONS%"=="" set PENTAHO\_DI\_JAVA\_OPTIONS="-Xmx512m" "-XX:MaxPermSize=256m"

set OPT=%PENTAHO\_DI\_JAVA\_OPTIONS% "-Djava.library.path=%LIBSPATH%" "-DKETTLE\_HOME=%KETTLE\_HOME%" "-DKETTLE\_REPOSITORY=%KETTLE\_REPOSITORY%" "-DKETTLE\_USER=%KETTLE\_USER%" "-DKETTLE\_PASSWORD=%KETTLE\_PASSWORD%" "-DKETTLE\_PLUGIN\_PACKAGES=%KETTLE\_PLUGIN\_PACKAGES%" "-DKETTLE\_LOG\_SIZE\_LIMIT=%KETTLE\_LOG\_SIZE\_LIMIT%" "-DKETTLE\_JNDI\_ROOT=%KETTLE\_JNDI\_ROOT%"  
修改参数，将内存值调大（256的倍数），调整后如下所示：

if "%PENTAHO\_DI\_JAVA\_OPTIONS%"=="" set PENTAHO\_DI\_JAVA\_OPTIONS="-Xmx1024m" "-XX:MaxPermSize=2048m"

set OPT=%PENTAHO\_DI\_JAVA\_OPTIONS% "-Djava.library.path=%LIBSPATH%" "-DKETTLE\_HOME=%KETTLE\_HOME%" "-DKETTLE\_REPOSITORY=%KETTLE\_REPOSITORY%" "-DKETTLE\_USER=%KETTLE\_USER%" "-DKETTLE\_PASSWORD=%KETTLE\_PASSWORD%" "-DKETTLE\_PLUGIN\_PACKAGES=%KETTLE\_PLUGIN\_PACKAGES%" "-DKETTLE\_LOG\_SIZE\_LIMIT=%KETTLE\_LOG\_SIZE\_LIMIT%" "-DKETTLE\_JNDI\_ROOT=%KETTLE\_JNDI\_ROOT%"  
重新执行kettle抽取数据，不再报内存溢出错误。

小结：  
        使用kettle时，如果遇到数据量大、大对象时很容易出现内存溢出现象，此时需要根据实际的硬件配置对预设的java堆栈大小进行调整。

原创作品，出自 “深蓝的blog” 博客，欢迎转载，转载时请务必注明以下出处，否则追究版权法律责任。

深蓝的blog：http://blog.csdn.net/huangyanlong/article/details/42453831  
\---------------------   
作者：黄炎龙   
来源：CSDN   
原文：https://blog.csdn.net/huangyanlong/article/details/42453831   
版权声明：本文为博主原创文章，转载请附上博文链接！

[https_blog.csdn.net_huangyanlong_article_details_42453831]: https://blog.csdn.net/huangyanlong/article/details/42453831