5 ms·
According to the article, the most interesting feature compared to Jupyter is no hidden state - if you delete a cell, the variables it set are gone. Also, you c
by aargh_aargh 7y ago
According to the article, the most interesting feature compared to Jupyter is no hidden state - if you delete a cell, the variables it set are gone. Also, you can mix languages - you'll be able to access variables filled by prevously executed cells in another language.
Personally, I'm looking forward to trying out the SQL support. I haven't seen an elegant solution for SQL notebooks in Jupyter, it was always second-class via Python or some such. Or have I missed something?
- type_enthusiast 7y agoThe SQL support is done through Spark, so it's not particularly novel – Zeppelin for example supports SQL similarly. We've talked about adding a more general SQL interpreter, though. Happy to hear any suggestions about it!
- chrisjc 7y agoDo you know of any generalized SQL interpreter that allows push-downs to the underlying engine where possible, but can also arbitrate compute resources to post-push down operations. Eg: such as merging disparate result-sets or make up for the lack of features from the underlying engines. Closest thing that comes to mind is something like Apache Drill, which coincidentally also uses Apache Calcite as the SQL interpreter. Also wondering why I would use this over Zeppelin which can support other interpreters like Flink?
- capableweb 7y ago> Also, you can mix languages Interesting. Judging by that it seems to be implemented with a JVM language and a screenshot shows "Scala" as a supported language, I'm guessing at least all the JVM languages are supported (personally hope for Clojure) but can't seem to find a list of supported languages anywhere in the post or on the website. What languages are supported by Polynote?
- mch82 7y ago> Currently Scala, Python, and SQL cell types are supported.
- lalaithion 7y agoLooks like just Scala and Python right now: https://github.com/polynote/polynote/blob/08f0751138e2991cf77e711344c8edcd8a976929/polynote-kernel/src/main/scala/polynote/kernel/interpreter/CoreInterpreters.scala https://github.com/polynote/polynote/blob/08f0751138e2991cf7...
- daveFNbuck 7y agoalso SQL
- rhizome 7y ago"Duoglot?"
- type_enthusiast 7y agoCurrently just Scala and Python (via jep). Looking to add more (probably starting with Java and clojure) but haven't had time yet. There's just two of us working on it so far. PRs welcome!
- chrisseaton 7y agoHave you looked at GraalVM to bring in more languages?
- type_enthusiast 7y agoYep, we will be adding a plug-in to support Graal languages (little bit of learning to do on Graal first). We didn't want to make the project depend on Graal, though, because it's a pretty small segment of users (and we're not using it on our team at this point).
- cglong 7y agoTo verify, will this plugin allow external parties to add support to Polynote for more esoteric languages?
- type_enthusiast 7y agoI don't want to over-promise anything given that I still have some reading to do about Graal's inner workings. But even given the interpreter side, there's also some plumbing to do (e.g. Monaco integration) that we haven't thought out yet. It's still in its infancy, and we'll need some help to be able to add stuff like this.
- chrisseaton 7y agoGet in touch with me if you want any help - I have experience in embedding Graal.
- truculent 7y agoR notebooks do a lot of this stuff reasonably well, too. But then you have R as the primary language (other langs are well supported though). As ever, the best answer is the Notebooks Are Bad, Actually