Sqoop在导入MySQL数据时遇到Timestamp列为0000-00-00 00:00:00报错

分手后的思念是犯贱 2022-01-15 09:17 443阅读 0赞

[为什么80%的码农都做不了架构师？>>> ][80_]  ![hot3.png][]

Sqoop在导入MySQL数据时遇到Timestamp列为'0000-00-00 00:00:00'时报错，解决方法是：在JDBC连接后加上?zeroDateTimeBehavior=convertToNull

16/04/30 12:34:49 INFO mapreduce.Job: Task Id : attempt_1453983885611_376511_m_000000_0, Status : FAILED
    Error: java.io.IOException: SQLException in nextKeyValue
            at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
            at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
            at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
            at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
            at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
            at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
            at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
            at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
            at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:415)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
            at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
    Caused by: java.sql.SQLException: Value '147583120.00782016-04-22 14:32:11�0000-00-00 00:00:00�1�
    
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1094)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:997)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:983)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:928)
            at com.mysql.jdbc.ResultSetRow.getTimestampFast(ResultSetRow.java:1102)
            at com.mysql.jdbc.BufferRow.getTimestampFast(BufferRow.java:576)
            at com.mysql.jdbc.ResultSetImpl.getTimestampInternal(ResultSetImpl.java:6587)
            at com.mysql.jdbc.ResultSetImpl.getTimestamp(ResultSetImpl.java:6187)
            at org.apache.sqoop.lib.JdbcWritableBridge.readTimestamp(JdbcWritableBridge.java:111)
            at com.cloudera.sqoop.lib.JdbcWritableBridge.readTimestamp(JdbcWritableBridge.java:83)
            at QueryResult.readFields(QueryResult.java:215)
            at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:244)
            ... 12 more

举例：

sqoop import --driver com.mysql.jdbc.Driver --connect jdbc:mysql://192.168.4.48:3306/xdgc?zeroDateTimeBehavior=convertToNull --username root --password <...> --table party_org --hive-import --hive-table ori.party_org --incremental lastmodified --check-column last_update_time --last-value '2014-01-01 00:00:00.000'

转载于:https://my.oschina.net/data99/blog/668409

[80_]: https://my.oschina.net/u/2663968/blog/3051541
[hot3.png]: /images/20220115/373d45bc7f0a41d88a23a3901e114b89.png