4 ms·
Thanks Eridrus for this. Your clustering approach looks promising. I've been chipping away at this with a more straightforward heuristic based on parts of spee
by krschacht 7y ago
Thanks Eridrus for this. Your clustering approach looks promising.
I've been chipping away at this with a more straightforward heuristic based on parts of speech, just examining a hundred questions at a time. It turns out that filtering things down to the Direct Object and Noun-Subject (as identified by Google's NLP) filters out a lot of the noise. And then, the vocabulary of children under 10 years old is a finite enough that with a short blacklist, I'm getting quite a good hit rate for the first few hundred questions I've gone through. With a little more testing and refinement, I think this might get me what I want but I'm going to dig into sentence clustering if I hit a dead end.
In case you're curious, here's the last few questions I processed as an example:
How is an eraser made?
> Topics: eraser
Why do rainbows always have the same colors?
> rainbow, color
How do electromagnets work in devices like loudspeakers and microphones?
> Topics: electromagnet, device, loudspeaker, microphone
What is frostbite and why does it make your skin turn black?
> Topics: frostbite, skin
How can you see yourself in a mirror?
> Topics: seeing, mirror