Java – using the Jackson module of scalaobjectmapper in spark 1.4 Error running job on 0

I'm running Scala 2.10 4, and runs in spark 1.4 0 cluster (based on HDFS and managed by yarn), and Jackson module 2.6.0 is used in Maven repository one

When I run code locally from the IDE (IntelliJ idea V14), everything applies to the cluster in memory, but when I run a job on my remote cluster (EMR cluster on AWS VPC), I encounter the following exception:

java.lang.AbstractMethodError: com.company.scala.framework.utils.JsonParser$$anon$1.com$fasterxml$jackson$module$scala$experimental$ScalaObjectMapper$_setter_$com$fasterxml$jackson$module$scala$experimental$ScalaObjectMapper$$typeCache_$eq(Lorg/spark-project/guava/cache/LoadingCache;)V
    at com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper$class.$init$(ScalaObjectMapper.scala:50)
    at com.company.scala.framework.utils.JsonParser$$anon$1.<init>(JsonParser.scala:14)
    at com.company.scala.framework.utils.JsonParser$.<init>(JsonParser.scala:14)
    at com.company.scala.framework.utils.JsonParser$.<clinit>(JsonParser.scala)
    at com.company.migration.Migration$.printAllKeys(Migration.scala:21)
    at com.company.migration.Main$$anonfun$main$1.apply(Main.scala:22)
    at com.company.migration.Main$$anonfun$main$1.apply(Main.scala:22)
    at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
    at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:199)
    at org.apache.spark.shuffle.sort.sortShuffleWriter.write(SortShuffleWriter.scala:56)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
    at org.apache.spark.scheduler.Task.run(Task.scala:70)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

I tried to check the abnormal situation on the Internet without luck I also tried to find a similar question here. I found that there was only one thread, no acceptable answer, and no answer helped me there

Hope to find help here,

thank you.

Solution

I'm answering questions about other users' comments

I stopped using scalaobjectmapper and started using regular objectmapper

val jacksonMapper = new ObjectMapper() 
jacksonMapper.registerModule(DefaultScalaModule)

And it works normally for the time being Comments on attached urinals are useful: