module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;"/>
om.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;
SparkSQL连接Hive报错:
om.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;
原因这是由于jackson相关包的冲突导致,或者缺少相关的包导致的,在使用spark-sql 连接hive的时候需要假如以下的依赖,将对应的Spark版本和Scala版本换成自己使用的版本即可。
<dependencies><!-- mysql --><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><version>5.1.26</version></dependency><dependency><groupId>org.apache.spark</groupId><artifactId>spark-hive_2.12</artifactId><version>2.4.5</version></dependency><dependency><groupId>org.apache.spark</groupId><artifactId>spark-sql_2.12</artifactId><version>${spark.version}</version></dependency><!-- .fasterxml.jackson.module/jackson-module-scala --><dependency><groupId>com.fasterxml.jackson.module</groupId><artifactId>jackson-module-scala_2.12</artifactId><version>2.10.3</version></dependency><dependency><groupId>com.fasterxml.jackson.module</groupId><artifactId>jackson-module-jaxb-annotations</artifactId><version>2.10.3</version></dependency><dependency><groupId>com.fasterxml.jackson.core</groupId><artifactId>jackson-databind</artifactId><version>2.10.3</version></dependency><dependency><groupId>com.fasterxml.jackson.core</groupId><artifactId>jackson-annotations</artifactId><version>2.10.3</version></dependency></dependencies>
替换之后即可成功运行,注意需要在spark-sql的代码中开启enableHiveSupport()的支持
+------+-------------------+--------+
|uid |time |dt |
+------+-------------------+--------+
|100012|2017-02-02 17:04:02|20170202|
|100011|2017-02-02 17:05:06|20170202|
|100012|2017-02-02 17:07:07|20170202|
|100016|2017-02-02 17:08:04|20170202|
|100011|2017-02-02 18:01:01|20170202|
|100016|2017-02-02 19:03:02|20170202|
+------+-------------------+--------+
更多推荐
om.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Lja
发布评论