4 ms·

Go-Freeling: Natural Language Processing in Go

80 points by cnbuff410 11y ago

nkozyra 11y ago
Excited to try this, although I don't immediately see what model format is used. I've been a bit surprised to see ML and NLP lag a bit on Go compared to rather swift adoption with other languages.
roscopcoltrane 11y ago
Why is it underhanded to let a user know that his government is censoring/surveiling his posts? This should be the freedom-c contest.
- roscopcoltrane 11y ago
  oops. commented on wrong post. sorry
  - Dewie3 11y ago
    You can delete comments if you want.
andrewstuart2 11y ago
I'd suggest making sure it's go-gettable and will build following the instructions in the README when placed in the canonical go path ($GOPATH/github.com/advancedlogic/go-freeling). After fixing the paths, I'm still getting a cgo linking error with the mitie.h file. Also, probably avoid using . imports, as it's considered a Go antipattern. Seems really promising, but if you want it to become a project that the Go community can get behind and maintain, you'll probably want to make sure it's familiar and easy to get running. :-)
- mrkrwtsn 11y ago
  I'm unable to get it to run either. Turning text into tokens is something I've wanted to be able to do in golang before, so it would be super nice if it was easy to install with `go get`. I still need to work through some of the other libraries here: http://biosphere.cc/software-engineering/go-machine-learning-nlp-libraries http://biosphere.cc/software-engineering/go-machine-learning... I agree though, seems promising.
  - mschoch 11y ago
    If you just want to segment larger blocks of text into tokens you can try the segment library (it implements the word boundary portion of unicode annex 29): https://github.com/blevesearch/segment https://github.com/blevesearch/segment If you need more manipulation of tokens after segmentation/tokenization, you could look at the analysis sub-package of bleve. Its intended to be able to be used indepenently of the rest of the library. https://github.com/blevesearch/bleve https://github.com/blevesearch/bleve
frik 11y ago
Very nice, something I have been looking for. Please create a wiki with some more information and examples. Is there a training part involved? Does it support languages other than English, if so which data format or how to train the language model? Is there a WordNet integration? Or can I use the FreeLing website as reference http://nlp.lsi.upc.edu/freeling/index.php?option=com_content&task=view&id=12&Itemid=41 http://nlp.lsi.upc.edu/freeling/index.php?option=com_content... ?
panamafrank 11y ago
There are no unit tests, and the only comments are odd bits of code. How do you know any of it works?
- Dewie3 11y ago
  Maybe he reasoned about his code in his head/on paper?
  - deleted 11y ago
    [deleted]
  - panamafrank 11y ago
    where's the proof that his reasoning is correct?
    - Dewie3 11y ago
      You know as well as I that most programming reasoning is informal and doesn't have anything to do with any kind of proof.
      - panamafrank 11y ago
        http://dilbert.com/strip/2015-06-07 http://dilbert.com/strip/2015-06-07
        Dewie3 11y ago
        Was that last panel supposed to be about you? Anyway nothing I've written here is really angry, or passionate, or inflammatory (just snarky). But since I don't get your usage of the word proof in this context, what do you mean specifically?
hendler 11y ago
The newer versions of Freeling have a socket server you can use from any socket client. I wrote one as my first go project a while back https://github.com/Hendler/goling https://github.com/Hendler/goling
hoare 11y ago
that's awesome! im a little worried about test coverage though. Did you run some kinds of performance tests?

80 points by cnbuff410 11y ago

Excited to try this, although I don't immediately see what model format is used. I've been a bit surprised to see ML and NLP lag a bit on Go compared to rather swift adoption with other languages.

roscopcoltrane 11y ago

Why is it underhanded to let a user know that his government is censoring/surveiling his posts? This should be the freedom-c contest.

roscopcoltrane 11y ago
oops. commented on wrong post. sorry
- Dewie3 11y ago
  You can delete comments if you want.

andrewstuart2 11y ago

I'd suggest making sure it's go-gettable and will build following the instructions in the README when placed in the canonical go path ($GOPATH/github.com/advancedlogic/go-freeling). After fixing the paths, I'm still getting a cgo linking error with the mitie.h file. Also, probably avoid using . imports, as it's considered a Go antipattern. Seems really promising, but if you want it to become a project that the Go community can get behind and maintain, you'll probably want to make sure it's familiar and easy to get running. :-)

mrkrwtsn 11y ago
I'm unable to get it to run either. Turning text into tokens is something I've wanted to be able to do in golang before, so it would be super nice if it was easy to install with `go get`. I still need to work through some of the other libraries here: http://biosphere.cc/software-engineering/go-machine-learning-nlp-libraries http://biosphere.cc/software-engineering/go-machine-learning... I agree though, seems promising.
- mschoch 11y ago
  If you just want to segment larger blocks of text into tokens you can try the segment library (it implements the word boundary portion of unicode annex 29): https://github.com/blevesearch/segment https://github.com/blevesearch/segment If you need more manipulation of tokens after segmentation/tokenization, you could look at the analysis sub-package of bleve. Its intended to be able to be used indepenently of the rest of the library. https://github.com/blevesearch/bleve https://github.com/blevesearch/bleve

frik 11y ago

Very nice, something I have been looking for. Please create a wiki with some more information and examples. Is there a training part involved? Does it support languages other than English, if so which data format or how to train the language model? Is there a WordNet integration? Or can I use the FreeLing website as reference http://nlp.lsi.upc.edu/freeling/index.php?option=com_content&task=view&id=12&Itemid=41 http://nlp.lsi.upc.edu/freeling/index.php?option=com_content... ?

panamafrank 11y ago

There are no unit tests, and the only comments are odd bits of code. How do you know any of it works?

Dewie3 11y ago
Maybe he reasoned about his code in his head/on paper?
- deleted 11y ago
  [deleted]
- panamafrank 11y ago
  where's the proof that his reasoning is correct?
  - Dewie3 11y ago
    You know as well as I that most programming reasoning is informal and doesn't have anything to do with any kind of proof.
    - panamafrank 11y ago
      http://dilbert.com/strip/2015-06-07 http://dilbert.com/strip/2015-06-07
      - Dewie3 11y ago
        Was that last panel supposed to be about you? Anyway nothing I've written here is really angry, or passionate, or inflammatory (just snarky). But since I don't get your usage of the word proof in this context, what do you mean specifically?

hendler 11y ago

The newer versions of Freeling have a socket server you can use from any socket client. I wrote one as my first go project a while back https://github.com/Hendler/goling https://github.com/Hendler/goling

hoare 11y ago

that's awesome! im a little worried about test coverage though. Did you run some kinds of performance tests?