4 ms·

That's pretty cool — I had the exact same goals (help professors grade essays faster) with my bookshrink project [0]. Of course, it was the first python code I

That's pretty cool — I had the exact same goals (help professors grade essays faster) with my bookshrink project [0]. Of course, it was the first python code I ever wrote and it uses an extremely naïve algorithm, but the results aren't too bad if you want to try it yourself [1]. [0]: https://github.com/peterldowns/bookshrink https://github.com/peterldowns/bookshrink [1]: http://www.bookshrink.com/ http://www.bookshrink.com/

yid 13y ago
> it uses an extremely naïve algorithm, but the results aren't too bad if you want to try it yourself Don't be so hard on yourself. I review papers for CS conferences and just the fact that you used TF-IDF weighting puts you well above the average.
magicseth 13y ago
You might enjoy my tldr.js [0], you select the text you want to summarize and run the bookmarklet. It could use a little help in the parser department, but it was fun to write. [0]: https://github.com/bumptech/tl-dr.js/blob/master/tldr.js https://github.com/bumptech/tl-dr.js/blob/master/tldr.js
- peter_l_downs 13y ago
  I did enjoy this, it looks like we took a very similar approach! Great idea to make it accessible as a bookmarklet and to let it run easily on certain sites like Wikipedia.
e12e 13y ago
Regarding stuff like: """ SPLIT INPUT INTO SENTENCES""" god_awful_regex = r'''(?<!\d)(?<![A-Z]\.)(?<!\.[a-z]\.)(?<!\.\.\.)(?<!etc\.)(?<![Mm]r\.)(?<![Pp]rof\.)(?<![Dd]r\.)(?<![Mm]rs\.)(?<![Mm]s\.)(?<![Mm]z\.)(?<![Mm]me\.)(?:(?<=[.!?])|(?<=[.!?]['"]))[\s]+?(?=[\S])''' Be advised that one of the nice things about python reg-exes is that they allow in-line comments (and naming of groups), if compiled with the verbose-flag: """ SPLIT INPUT INTO SENTENCES""" verbose_regex = r'''(?<!\d) # I can't actually tell (?<![A-Z]\.)(?<!\.[a-z]\.) # what you're doing here... (?<!\.\.\.)(?<!etc\.) # Is this one big group, or is (?<![Mm]r\.)(?<![Pp]rof\.) # it several groups, with (?<![Dd]r\.)(?<![Mm]rs\.) # different prefixes? (?<![Mm]s\.)(?<![Mm]z\.)(?<![Mm]me\.) # Clearly, it's got something to do (?:(?<=[.!?])|(?<=[.!?]['"]))[\s]+?(?=[\S])''' # with # not matching the dot at the end of Dr. # or as part of an ellipsis as the end of a # sentence? But my point was that if such a # regex is built-up and tested with comments # it can be quite readable god_awful_regex = re.compile(verbose_regex, re.VERBOSE) # continue here... http://www.diveintopython.net/regular_expressions/verbose.html http://www.diveintopython.net/regular_expressions/verbose.ht...
- peter_l_downs 13y ago
  Yeah, I've since learned that feature of Python :) Like I said, this is some of the first Python code I ever wrote; it definitely does not reflect my current knowledge.
  - e12e 13y ago
    Oh, to be clear, it wasn't meant as critique -- I just saw the aptly named variable, and thought it might be useful to highlight this aspect of python to others that might be new to the language. It's one of the few "special" parts of python I have personal experience with, as I worked on a small program that dealt with parsing emails, and being able to fully comment the reg-exp over several lines as I tweaked both it and the groups was very helpful :-)
    - peter_l_downs 13y ago
      You're absolutely right, and what I forgot to say in my earlier comment was "thank you!"

yid 13y ago

> it uses an extremely naïve algorithm, but the results aren't too bad if you want to try it yourself Don't be so hard on yourself. I review papers for CS conferences and just the fact that you used TF-IDF weighting puts you well above the average.

magicseth 13y ago

You might enjoy my tldr.js [0], you select the text you want to summarize and run the bookmarklet. It could use a little help in the parser department, but it was fun to write. [0]: https://github.com/bumptech/tl-dr.js/blob/master/tldr.js https://github.com/bumptech/tl-dr.js/blob/master/tldr.js

peter_l_downs 13y ago
I did enjoy this, it looks like we took a very similar approach! Great idea to make it accessible as a bookmarklet and to let it run easily on certain sites like Wikipedia.

e12e 13y ago

Regarding stuff like: """ SPLIT INPUT INTO SENTENCES""" god_awful_regex = r'''(?<!\d)(?<![A-Z]\.)(?<!\.[a-z]\.)(?<!\.\.\.)(?<!etc\.)(?<![Mm]r\.)(?<![Pp]rof\.)(?<![Dd]r\.)(?<![Mm]rs\.)(?<![Mm]s\.)(?<![Mm]z\.)(?<![Mm]me\.)(?:(?<=[.!?])|(?<=[.!?]['"]))[\s]+?(?=[\S])''' Be advised that one of the nice things about python reg-exes is that they allow in-line comments (and naming of groups), if compiled with the verbose-flag: """ SPLIT INPUT INTO SENTENCES""" verbose_regex = r'''(?<!\d) # I can't actually tell (?<![A-Z]\.)(?<!\.[a-z]\.) # what you're doing here... (?<!\.\.\.)(?<!etc\.) # Is this one big group, or is (?<![Mm]r\.)(?<![Pp]rof\.) # it several groups, with (?<![Dd]r\.)(?<![Mm]rs\.) # different prefixes? (?<![Mm]s\.)(?<![Mm]z\.)(?<![Mm]me\.) # Clearly, it's got something to do (?:(?<=[.!?])|(?<=[.!?]['"]))[\s]+?(?=[\S])''' # with # not matching the dot at the end of Dr. # or as part of an ellipsis as the end of a # sentence? But my point was that if such a # regex is built-up and tested with comments # it can be quite readable god_awful_regex = re.compile(verbose_regex, re.VERBOSE) # continue here... http://www.diveintopython.net/regular_expressions/verbose.html http://www.diveintopython.net/regular_expressions/verbose.ht...

peter_l_downs 13y ago
Yeah, I've since learned that feature of Python :) Like I said, this is some of the first Python code I ever wrote; it definitely does not reflect my current knowledge.
- e12e 13y ago
  Oh, to be clear, it wasn't meant as critique -- I just saw the aptly named variable, and thought it might be useful to highlight this aspect of python to others that might be new to the language. It's one of the few "special" parts of python I have personal experience with, as I worked on a small program that dealt with parsing emails, and being able to fully comment the reg-exp over several lines as I tweaked both it and the groups was very helpful :-)
  - peter_l_downs 13y ago
    You're absolutely right, and what I forgot to say in my earlier comment was "thank you!"