Wsrtjtyk

Hi I have a file given to by my teacher. It is about Scala and Spark.
When I run the code it gives me this exception:

  (run-main-0) scala.ScalaReflectionException: class java.sql.Date in 

  JavaMirror with ClasspathFilter 

The file itself looks like this:

import org.apache.spark.ml.feature.Tokenizer

import org.apache.spark.sql.Dataset

import org.apache.spark.sql.SparkSession

import org.apache.spark.sql.types._

object Main {

   type Embedding       = (String, List[Double])

   type ParsedReview    = (Integer, String, Double)

   org.apache.log4j.Logger getLogger "org"  setLevel 

   (org.apache.log4j.Level.WARN)

   org.apache.log4j.Logger getLogger "akka" setLevel 

  (org.apache.log4j.Level.WARN)

   val spark =  SparkSession.builder

     .appName ("Sentiment")

     .master  ("local[9]")

     .getOrCreate



import spark.implicits._



val reviewSchema = StructType(Array(

        StructField ("reviewText", StringType, nullable=false),

        StructField ("overall",    DoubleType, nullable=false),

        StructField ("summary",    StringType, nullable=false)))



// Read file and merge the text abd summary into a single text column



def loadReviews (path: String): Dataset[ParsedReview] =

    spark

        .read

        .schema (reviewSchema)

        .json (path)

        .rdd

        .zipWithUniqueId

        .map[(Integer,String,Double)] { case (row,id) => (id.toInt, s"${row getString 2} ${row getString 0}", row getDouble 1) }

        .toDS

        .withColumnRenamed ("_1", "id" )

        .withColumnRenamed ("_2", "text")

        .withColumnRenamed ("_3", "overall")

        .as[ParsedReview]



 // Load the GLoVe embeddings file



 def loadGlove (path: String): Dataset[Embedding] =

     spark

         .read

         .text (path)

    .map  { _ getString 0 split " " }

    .map  (r => (r.head, r.tail.toList.map (_.toDouble))) // yuck!

         .withColumnRenamed ("_1", "word" )

         .withColumnRenamed ("_2", "vec")

         .as[Embedding]



def main(args: Array[String]) = {



  val glove  = loadGlove ("Data/glove.6B.50d.txt") // take glove 



  val reviews = loadReviews ("Data/Electronics_5.json") // FIXME



  // replace the following with the project code







   glove.show

   reviews.show



        spark.stop

   }



 }

I need to keep the line
import org.apache.spark.sql.Dataset
because some code depends on it but it is exactly because of it I have an exception throw.

My build.sbt file looks like this:

  name := "Sentiment Analysis Project"



  version := "1.1"



  scalaVersion := "2.11.12"



  scalacOptions ++= Seq("-unchecked", "-deprecation")



  initialCommands in console := 

  """

  import Main._

  """



   libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.0"



   libraryDependencies += "org.apache.spark" %% "spark-mllib" % 

   "2.3.0"



    libraryDependencies += "org.scalactic" %% "scalactic" % "3.0.5"



    libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.5" % 

    "test"

I was using OpenJDK 11.0.1. I uninstalled it and it worked.

You can check your current java version by running

java -version

If you have brew installed, you can remove OpenJDK by running:

brew cask uninstall java

And to make sure you have Java 1.8.0 installed run the following:

brew cask install java8