There's a perception that hadoop commands are terribly complex. If you run $spark-shell you can execute (interactively) val file = spark.textFile("hdfs://...

There's a perception that hadoop commands are terribly complex. If you run $spark-shell you can execute (interactively) val file = spark.textFile("hdfs://...") val errors = file.filter(line => line.contains("ERROR")) errors.count() And wordcount a file - ok the wget is not there, but this is really not complex!