3 ms·

Anyone else work with NLP but unlikely to try this tool simply because it's in Java? Do any Python NLP engineers integrate Java tools in their setup, e.g. via

Anyone else work with NLP but unlikely to try this tool simply because it's in Java? Do any Python NLP engineers integrate Java tools in their setup, e.g. via Jython? How does that work for you?

maga 10y ago
I work with NLP and do not plan to try this tool, though, not because of Java, but because I'm not interested in a summarizer right now. Our current production back-end doing NLP stuff is written in Java and most third party libraries it uses are also written in Java. It was initially written in Python, but at one point we realized that most libraries we use are in Java and Python is just moving data between them. The choice of these Java libraries wasn't driven by any love towards Java either. By the time they were just a fair bit more advanced both in feature set and performance than their Python counterparts. One example is Stanford Parser and CoreNLP toolkit--up until Parsey McParseface it was the most accurate parser, and CoreNLP toolkit had more features (that interested us) than Python's NLTK.
- deleted 10y ago
  [deleted]
- thpalmear 10y ago
  In the end, language and libs don't matter as much as the actual relevancy of the algorithmic methods. This is where the real innovation and invention occurs, in the ability to mimic human cognition as closely as possible. This is also why side-by-side comparisons are the ultimate litmus and why there is now a separation happening between companies that regurgitate voyeuristic ideas (geocities, friendster, myspace, facebook-aol, snapchat, twitter-majordomo) verses companies that truly invent and innovate hard-to-duplicate initial algorithmic solutions (Google, SENS, Buck Inst., Human Longevity, SpaceX). It's like fake AI vs a path toward some semblance of real AI.
sixhobbits 10y ago
I do a lot of NLP stuff and the Python/Java divide is definitely a problem right now. I'm happy with both languages, but generally prefer Python. A lot of the state-of-the-art tools, especially the Stanford stuff, is in Java, while many of the more modern and experimental stuff is in Python. In theory it should be really easy to use both, but in practice I've always had to deal with bugs in the gap between the languages, whether Jython or OS level integration. So currently, yes, there are some new tools (including this one) that I would definitely try if they were written in Python and I could just do a pip install + import, but never get round to testing out because like this one they are more clunky to set up and require Java.
ganeshkrishnan 10y ago
We work with NLP and Java. Stanford NLP and apache Spark both run on Java/Scala and so does hadoop for HDFS. Why the hate against Java?
- amelius 10y ago
  Startup time and memory footprint?
  - ganeshkrishnan 10y ago
    Its hdfs so distributed across machines. Startup time is not a problem for us since the service seldom shutsoff
deepGem 10y ago
Well this code is in Scala, not Java. Python doesn't have a library like the Stanford Datetime parser that can convert a phrase such as 'day after tomorrow' to the actual date. I understand that Writing Java is hard simply because of the amount of code you have to write. Scala is a good compromise.
- sixhobbits 10y ago
  On the contrary, last time I needed to parse natural dates in Python I got confused because there are dozens of libraries that do exactly this. One example: >>> from parsedatetime import Calendar >>> cal = Calendar() >>> cal.parse("The day after tomorrow") (time.struct_time(tm_year=2016, tm_mon=10, tm_mday=3, tm_hour=9, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=277, tm_isdst=-1), 1) Yes I've used Java equivalents before and they're impressive, but I don't think Python is inferior in this regard. For me, the advantage of Python is mainly being able to easily run small sections of project code in an IPython shell while developing that make the development workflow so much faster and more pleasant, rather than the quantity of code.
  - deepGem 10y ago
    Thanks for pointing this out. We missed this during our comparison studies.
    - sixhobbits 10y ago
      Looks like the output I pasted is actually wrong though so maybe I spoke too soon.
computerwizard 10y ago
Would be nice to work with this in JRuby.

Anyone else work with NLP but unlikely to try this tool simply because it's in Java? Do any Python NLP engineers integrate Java tools in their setup, e.g. via Jython? How does that work for you?

maga 10y ago

I work with NLP and do not plan to try this tool, though, not because of Java, but because I'm not interested in a summarizer right now. Our current production back-end doing NLP stuff is written in Java and most third party libraries it uses are also written in Java. It was initially written in Python, but at one point we realized that most libraries we use are in Java and Python is just moving data between them. The choice of these Java libraries wasn't driven by any love towards Java either. By the time they were just a fair bit more advanced both in feature set and performance than their Python counterparts. One example is Stanford Parser and CoreNLP toolkit--up until Parsey McParseface it was the most accurate parser, and CoreNLP toolkit had more features (that interested us) than Python's NLTK.

deleted 10y ago
[deleted]
thpalmear 10y ago
In the end, language and libs don't matter as much as the actual relevancy of the algorithmic methods. This is where the real innovation and invention occurs, in the ability to mimic human cognition as closely as possible. This is also why side-by-side comparisons are the ultimate litmus and why there is now a separation happening between companies that regurgitate voyeuristic ideas (geocities, friendster, myspace, facebook-aol, snapchat, twitter-majordomo) verses companies that truly invent and innovate hard-to-duplicate initial algorithmic solutions (Google, SENS, Buck Inst., Human Longevity, SpaceX). It's like fake AI vs a path toward some semblance of real AI.

sixhobbits 10y ago

I do a lot of NLP stuff and the Python/Java divide is definitely a problem right now. I'm happy with both languages, but generally prefer Python. A lot of the state-of-the-art tools, especially the Stanford stuff, is in Java, while many of the more modern and experimental stuff is in Python. In theory it should be really easy to use both, but in practice I've always had to deal with bugs in the gap between the languages, whether Jython or OS level integration. So currently, yes, there are some new tools (including this one) that I would definitely try if they were written in Python and I could just do a pip install + import, but never get round to testing out because like this one they are more clunky to set up and require Java.

ganeshkrishnan 10y ago

We work with NLP and Java. Stanford NLP and apache Spark both run on Java/Scala and so does hadoop for HDFS. Why the hate against Java?

amelius 10y ago
Startup time and memory footprint?
- ganeshkrishnan 10y ago
  Its hdfs so distributed across machines. Startup time is not a problem for us since the service seldom shutsoff

deepGem 10y ago

Well this code is in Scala, not Java. Python doesn't have a library like the Stanford Datetime parser that can convert a phrase such as 'day after tomorrow' to the actual date. I understand that Writing Java is hard simply because of the amount of code you have to write. Scala is a good compromise.

sixhobbits 10y ago
On the contrary, last time I needed to parse natural dates in Python I got confused because there are dozens of libraries that do exactly this. One example: >>> from parsedatetime import Calendar >>> cal = Calendar() >>> cal.parse("The day after tomorrow") (time.struct_time(tm_year=2016, tm_mon=10, tm_mday=3, tm_hour=9, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=277, tm_isdst=-1), 1) Yes I've used Java equivalents before and they're impressive, but I don't think Python is inferior in this regard. For me, the advantage of Python is mainly being able to easily run small sections of project code in an IPython shell while developing that make the development workflow so much faster and more pleasant, rather than the quantity of code.
- deepGem 10y ago
  Thanks for pointing this out. We missed this during our comparison studies.
  - sixhobbits 10y ago
    Looks like the output I pasted is actually wrong though so maybe I spoke too soon.

computerwizard 10y ago

Would be nice to work with this in JRuby.