3 ms·
> I tend to work in Python not Scala Fair enough. I’m more comfortable in Scala than Python. Have you tried using a local Jupyter notebook? That would seem to
by nyokodo 5y ago
> I tend to work in Python not Scala
Fair enough. I’m more comfortable in Scala than Python. Have you tried using a local Jupyter notebook? That would seem to tick all the boxes.
https://jupyter.org/install https://jupyter.org/install
- wenc 5y agoI do use Jupyter and the boilerplate code above is what I usually deploy in Jupyter. Spark syntax is still just a touch more complicated than necessary for my needs because it's solving the much more general distributed computing problem. (not to mention the impedance mismatch -- Spark is very much a Scala native app, and it feels that way to a Python user). DuckDB on the other hand seems to provide a quick way to interact with local Parquet files in Python, which I appreciate.