5 ms·

Show HN: DeepQuiz – free NLP question generator

forrestbrazeal 9y ago
Hi, Here is an extremely early stage project that a friend and I have been working on this summer. DeepQuiz generates quiz questions from user-submitted text. Currently we're using a combination of statistical analysis and heuristic rules to do this - no machine learning. The main point in its favor right now is that it's free to use and pretty fast. We would really appreciate your feedback, as well as ideas for how it could be better, and types of generated questions you would like to see. The system does best with English text.
- nessup 9y ago
  This looks cool! As you probably already know, it has some ways to go. Simple English Wikipedia would have plenty of content with sentences that are easier to parse. Also, it'd be pretty cool to see general trivia quizzes autogenerated from Wikipedia using your algo.
  - gattilorenz 9y ago
    The problem of trivia quizzes is selecting what makes a good trivia quiz. You can't simply "quizzify" any part of any Wikipedia page, or the quiz has a very high chance of being too hard or not interesting at all (trivia: how many people live in the German city of Kraiburg am Inn? 600, 4000, 8000 or 16000?. Even someone interested in German geography is probably not going to enjoy this)
- damvigilante 9y ago
  This is a pretty cool project! You should take a look at this paper : https://arxiv.org/abs/1705.02012 https://arxiv.org/abs/1705.02012 . Using your heuristics to select the answers, you maybe able to generate some non fill in the blank questions. (All I saw was fill in the blank, not sure if it does other types of questions)
  - forrestbrazeal 9y ago
    It also does true/false and multiple choice. Right now the text sample has a lot to do with what kind of questions it can find.
- gattilorenz 9y ago
  Nice work! My suggestion at the moment is to avoid excessively long true/false questions (unless you actually plan to modify them, but that can be incredibly hard), and to use Named Entity Recognition/detection and semantic similarity (Word2Vec/Glove) for multiple-choice questions. I guess you are already using the former, but with the latter you should be able to really constrain the options to something more "credible".
- rocketsentence 9y ago
  This is so cool! What kind of tools are you using for the heuristic rules?
badmon 9y ago
Do you by any chance have an API one can use ?
- dhdersch 9y ago
  Yeah, just look at the Web page source to get the URL. Content-Type must be text/plain.
bill_mackenty 9y ago
This is exactly the sort of project I wanted to implement, but linked to more than one data source. I was thinking wikipedia + stack overflow or something like that. My dream is to create a system like this one which dynamically creates questions in a media-wiki article. I think we need to think about categories of question. True / False and multiple choice are only one part of "knowing something". To what extent does someone know something? This is probably a better way to approach asking questions. We also think about "blooms taxonomy of knowing" which is helpful for thinking about different types of assessment tasks. I would imagine generating the questions would be first, and then ranking them according various tiers of knowledge.
- amrrs 9y ago
  Guess, FB Research's DrQA is very close to what your idea is. https://github.com/facebookresearch/DrQA https://github.com/facebookresearch/DrQA
codefined 9y ago
Nice project, a group I worked on did something very similar with retrieving questions from a syllabus. Is this open source? I'd be fascinated to see similarities/differences in our approaches.
- cyborgx7 9y ago
  Is you project open source?
zerop 9y ago
I have lots of text about a particular topic (50K Articles crawled from internet about travel experiences). Now I want to create a Q&A from this given text. Is there any service that can do it? Basically I want to feed my text to this service and it creates a Q&A pairs from this given text. Not sure If I made it clear..
mindhash 9y ago
This could be useful to avoid comment spammers, trolls in blogs.. are you looking into this? A plugin could be a good start

Hi, Here is an extremely early stage project that a friend and I have been working on this summer. DeepQuiz generates quiz questions from user-submitted text. Currently we're using a combination of statistical analysis and heuristic rules to do this - no machine learning. The main point in its favor right now is that it's free to use and pretty fast. We would really appreciate your feedback, as well as ideas for how it could be better, and types of generated questions you would like to see. The system does best with English text.

nessup 9y ago
This looks cool! As you probably already know, it has some ways to go. Simple English Wikipedia would have plenty of content with sentences that are easier to parse. Also, it'd be pretty cool to see general trivia quizzes autogenerated from Wikipedia using your algo.
- gattilorenz 9y ago
  The problem of trivia quizzes is selecting what makes a good trivia quiz. You can't simply "quizzify" any part of any Wikipedia page, or the quiz has a very high chance of being too hard or not interesting at all (trivia: how many people live in the German city of Kraiburg am Inn? 600, 4000, 8000 or 16000?. Even someone interested in German geography is probably not going to enjoy this)
damvigilante 9y ago
This is a pretty cool project! You should take a look at this paper : https://arxiv.org/abs/1705.02012 https://arxiv.org/abs/1705.02012 . Using your heuristics to select the answers, you maybe able to generate some non fill in the blank questions. (All I saw was fill in the blank, not sure if it does other types of questions)
- forrestbrazeal 9y ago
  It also does true/false and multiple choice. Right now the text sample has a lot to do with what kind of questions it can find.
gattilorenz 9y ago
Nice work! My suggestion at the moment is to avoid excessively long true/false questions (unless you actually plan to modify them, but that can be incredibly hard), and to use Named Entity Recognition/detection and semantic similarity (Word2Vec/Glove) for multiple-choice questions. I guess you are already using the former, but with the latter you should be able to really constrain the options to something more "credible".
rocketsentence 9y ago
This is so cool! What kind of tools are you using for the heuristic rules?

badmon 9y ago

Do you by any chance have an API one can use ?

dhdersch 9y ago
Yeah, just look at the Web page source to get the URL. Content-Type must be text/plain.

bill_mackenty 9y ago

This is exactly the sort of project I wanted to implement, but linked to more than one data source. I was thinking wikipedia + stack overflow or something like that. My dream is to create a system like this one which dynamically creates questions in a media-wiki article. I think we need to think about categories of question. True / False and multiple choice are only one part of "knowing something". To what extent does someone know something? This is probably a better way to approach asking questions. We also think about "blooms taxonomy of knowing" which is helpful for thinking about different types of assessment tasks. I would imagine generating the questions would be first, and then ranking them according various tiers of knowledge.

amrrs 9y ago
Guess, FB Research's DrQA is very close to what your idea is. https://github.com/facebookresearch/DrQA https://github.com/facebookresearch/DrQA

codefined 9y ago

Nice project, a group I worked on did something very similar with retrieving questions from a syllabus. Is this open source? I'd be fascinated to see similarities/differences in our approaches.

cyborgx7 9y ago
Is you project open source?

zerop 9y ago

I have lots of text about a particular topic (50K Articles crawled from internet about travel experiences). Now I want to create a Q&A from this given text. Is there any service that can do it? Basically I want to feed my text to this service and it creates a Q&A pairs from this given text. Not sure If I made it clear..

mindhash 9y ago

This could be useful to avoid comment spammers, trolls in blogs.. are you looking into this? A plugin could be a good start