5 ms·

Pretty interesting. Fully automated text summarization certainly isn't new (libots, Classifier4j, OSX's summarization service, Copernic Summarizer, etc.) but I

by naner 15y ago

Pretty interesting. Fully automated text summarization certainly isn't new (libots, Classifier4j, OSX's summarization service, Copernic Summarizer, etc.) but I haven't yet seen any implementations that use a machine learning approach. EDIT: A better integrated summarizer obviously won't "revolutionize the web", though.

marshray 15y ago
Somehow "May summarize the web up to 30% better!" just doesn't have the same ring to it.
- indubitably 15y ago
  I don't really see how 30% better is going to make a qualitative difference in the utility of text on the web.
  - marshray 15y ago
    I can't even think of any time other than raw search results that I read auto-generated summaries now anyway. Of course, maybe if the summaries were just a little better..it'd revolutionize the whole web! :-)
    - ak2012 15y ago
      does this all work based on picking up key sentences or can it rewrite content as well?
      - spicyxtreme 15y ago
        I think 30% more is pretty significant. Especially if applied to email, long articles etc. However, still not sure if it ensures that i wouldn't miss out on anything important which may be fatal if it was an important email. PLUS it doesnt say if it picks up on emotional context which is super important in interpersonal communications?
      - robryan 15y ago
        Doesn't seem like like it rewrites content, also seems to often use a sentence as the first one that refers to a previously introduced concept in the article that has no context in the summary. This doesn't really surprise me as in the time he has been working in ai/ml/nlp I don't think there would have been much time to do more than learn up to and implement some of the current state of the art. The named entity recognition for the keywords seems okay. I did a bit of reading on summarization during my thesis, enough to know it is a big and difficult problem. I hope he keeps at it, seems like a good start but a long way to go.
nl 15y ago
Measurement in summarization is very inexact, and there are no standards for measuring it. I've experimented with machine learning and haven't had great results especially if you want a small summary. I wrote Classifier4J, and the summarizer in that does pretty well - well enough that it has been ported to C# and as a patch for Python NLTK.

by naner 15y ago

marshray 15y ago

Somehow "May summarize the web up to 30% better!" just doesn't have the same ring to it.

indubitably 15y ago
I don't really see how 30% better is going to make a qualitative difference in the utility of text on the web.
- marshray 15y ago
  I can't even think of any time other than raw search results that I read auto-generated summaries now anyway. Of course, maybe if the summaries were just a little better..it'd revolutionize the whole web! :-)
  - ak2012 15y ago
    does this all work based on picking up key sentences or can it rewrite content as well?
    - spicyxtreme 15y ago
      I think 30% more is pretty significant. Especially if applied to email, long articles etc. However, still not sure if it ensures that i wouldn't miss out on anything important which may be fatal if it was an important email. PLUS it doesnt say if it picks up on emotional context which is super important in interpersonal communications?
    - robryan 15y ago
      Doesn't seem like like it rewrites content, also seems to often use a sentence as the first one that refers to a previously introduced concept in the article that has no context in the summary. This doesn't really surprise me as in the time he has been working in ai/ml/nlp I don't think there would have been much time to do more than learn up to and implement some of the current state of the art. The named entity recognition for the keywords seems okay. I did a bit of reading on summarization during my thesis, enough to know it is a big and difficult problem. I hope he keeps at it, seems like a good start but a long way to go.

nl 15y ago

Measurement in summarization is very inexact, and there are no standards for measuring it. I've experimented with machine learning and haven't had great results especially if you want a small summary. I wrote Classifier4J, and the summarizer in that does pretty well - well enough that it has been ported to C# and as a patch for Python NLTK.