Yes you can use IPython with Spark for Pyspark. Spark-notebook and Zeppelin, gives let's you use Scala with Spark,and support all of the other spark libraries.

Yes you can use IPython with Spark for Pyspark. Spark-notebook and Zeppelin, gives let's you use Scala with Spark,and support all of the other spark libraries. There are other projects like spark-kernel https://github.com/ibm-et/spark-kernel https://github.com/ibm-et/spark-kernel that let's you use IPython (Jupyter ) I guess, supporting spark's scala API.