You can use PySpark without ever writing or thinking about Java code. The library handles all the interop for you and even does vectorized serialization with Py

You can use PySpark without ever writing or thinking about Java code. The library handles all the interop for you and even does vectorized serialization with PyArrow. The same way machine learning code is actually mostly C++ or CUDA but you almost never think of that when using Python libraries.