4 ms·

I tried the demo with Cal Ripken Jr. I was surprised by some of the complex questions: >Which MLB player won the Sporting News MLB Rookie of the Year Award as

by Imnimo 3y ago

I tried the demo with Cal Ripken Jr. I was surprised by some of the complex questions: >Which MLB player won the Sporting News MLB Rookie of the Year Award as a pitcher in 1980, and who did Cal Ripken Jr. surpass to hold the record for most home runs hit as a shortstop? >What team did Britt Burns play for in the minor leagues before making his MLB debut, and in what year did Cal Ripken Jr. break the consecutive games played record? >Who was the minor league pitching coordinator for the Houston Astros until 2010, and what significant baseball record did Cal Ripken Jr. break in 1995? All five questions are a combination of a question about a Britt Burns fact and an unrelated Cal Ripken fact. Why is this? Britt Burns doesn't seem to appear on the live Wikipedia page for Ripken. Does he appear on a cached version? Or is it forming complex questions by finding another page in the same category as Ripken and pulling more facts?

maxrmk 3y ago
I was worried people would run into this quirk in the demo. We have several 'advanced' question generation strategies. You correctly guessed the one we're using in the demo; forming complex questions by finding another page in the same category as Ripken and pulling more facts. Normally we pull a ton of related topic and try to pick the best, but to keep the generation fast and cost effective in the demo I limited the number of related pages we pull. So sometimes (like this case) you get something barely related and end up with odd disjointed questions.
- Imnimo 3y ago
  Ah, that does make sense - especially for a category like a baseball player where it may take a lot of other pages to find one that's truly related. Would be expensive for a demo, but not a big deal for a real evaluation.
  - maxrmk 3y ago
    Yep, in real use cases the latency for generating questions doesn't really matter. But in the demo I was really worried about it.

by Imnimo 3y ago

maxrmk 3y ago

I was worried people would run into this quirk in the demo. We have several 'advanced' question generation strategies. You correctly guessed the one we're using in the demo; forming complex questions by finding another page in the same category as Ripken and pulling more facts. Normally we pull a ton of related topic and try to pick the best, but to keep the generation fast and cost effective in the demo I limited the number of related pages we pull. So sometimes (like this case) you get something barely related and end up with odd disjointed questions.

Imnimo 3y ago
Ah, that does make sense - especially for a category like a baseball player where it may take a lot of other pages to find one that's truly related. Would be expensive for a demo, but not a big deal for a real evaluation.
- maxrmk 3y ago
  Yep, in real use cases the latency for generating questions doesn't really matter. But in the demo I was really worried about it.