3 ms·
I believe you when you say there is interesting tech running the site, but I couldn't find evidence of it in a few minutes of clicking around. So there's not mu
by tlb 3y ago
I believe you when you say there is interesting tech running the site, but I couldn't find evidence of it in a few minutes of clicking around. So there's not much for me to engage with. A blog post explaining how the tech works might get more interest here.
I tried a demo quiz on studywand, but it seemed worse than typical manually generated quizzes. Here's an example:
Q: "What is the principle behind operant conditioning?"
A... is a process of gradually increasing the intensity of aversive stimuli in order to reduce their impact on a person's emotional state.
B... operant conditioning is the technique used to reinforce positive behaviors and punish unwanted behaviors.
C... is a plan that the therapist and client create together.
You could know nothing about psychology and choose B because it's the only one that's an answer to a question of that form. A good quiz question tests for a more precise understanding.
If the tech is doing something that couldn't be done before, you should draw our attention to it!
- james-revisoai 3y agoThank you, this bug has been fixed. We recently changed from shuffled distractors to Ai-generated distractors, but as you identified, such a bug could occur.
- james-revisoai 3y agoSpecifically, we used semantic similarity for shuffled answers (so they are close and not too easy), completely shuffled answers, and AI-generated distractors. We are now going to choose the most popular default (based on retention during A/B/C testing), and also offer the mode as a choice. The quiz question (though not answer, since overwritten) from the demo you posted was generated in 2020, with T5 not GPT-3. Other and more complex questions were also possible with our finetuned tech on a bespoke carefully collected dataset. It was at the time much more impressive than it is now with GPT models, and we had it out before competitors even launched(the closest thing was old style GOFAI and a few online tutorials on finetuning T5 for similar tasks that had inferior data).