4 ms·
A company I cofounded back in 2013-2014 was trying to do this. Although our plan of attack showed promised, the company crashed and burned for "legal" reasons.
by Denzel 10y ago
A company I cofounded back in 2013-2014 was trying to do this. Although our plan of attack showed promised, the company crashed and burned for "legal" reasons.
For this to work, you need to do three things: (1) get the original exam, (2) tag every question with a series of categories, and (3) get the students' completed exams. You'd think professors/teachers would be excited by the idea of reducing their workload with an automated quiz/exam generator based upon their curriculum. It didn't seem like it.
The professors can't be relied upon to tag their exam questions with categories, so the company has to handle #2. To solve problem #1, we reached out to hundreds of professors at multiple universities, explained what we were doing; and asked if they'd be willing to share their corpus of past (and hopefully present) exams. Nope! They didn't want to do it, and when pressed to give an explanation the lines of communication went cold. Not to mention, if we couldn't convince them to share a blank exam, there's no way they would be willing to share their students' completed exams.
So, we thought of a different route. My cofounder had taken a year off from Vanderbilt University, where he participated in a fraternity. Not only did his frat have a massive test bank that we could use, he knew plenty of other frats that had even more. It was a start. But convincing frats at other schools to upload their exams was a no-go. They slammed their big Greek doors in our face.
There was nowhere else to turn, except the student body itself. But no student wants to give away their exams without an incentive, and we didn't have the money to pay them. So, we setup a marketplace where students could upload their exams and sell them for a split of the sale. Finally, we got some traction. Some students uploaded their exams. Other students bought them. In our first week of release we made almost $1k. We were on track to finally make good on our promise; it looked like we'd be able to hire people to annotate the exams and begin work on our exam generation service.
Until that fateful night, while we sat there trailing the logs, and we saw it. A Vandy professor had signed up and bought one of her own exams! (Just to verify they were authentic.) She immediately reported my cofounder to the university's disciplinary department. And they threatened to expel him with a permanent mark if he didn't immediately take down the website. After 3 years of classes, he'd leave with no credits, no diploma, and no other place to go. Like most young kids, he decided to lay his cards down under such massive pressure.
After that, we shut the website down, and went our separate ways. Never able to achieve our ultimate vision.
Maybe there's a different way to go about it. But without a strategy to compile an extensive corpus of completed exams with tagged questions, I don't know how you would go about it. If you have any ideas, I'd love to hear them. Because not a day goes by where I don't want to take another stab at this.
- germinalphrase 10y agoHigher ed. is likely to be more protective of intellectual property when it comes to assessments. They're less frequent, higher stakes, and are principally used for summative rather than formative purposes. Even in k12, there might be a chicken/egg situation. The obligation to tag content is a barrier (just as tagging is always a barrier). That said, k12 has so many underlying content standards - which are only becoming more entrenched, for instance with standards based grading - that you might want to integrate tagging as a sub feature to a curriculum creation/curriculum performance toolset so teachers don't have to focus on it specifically (only, rather, as fulfilling the pre-existing expectation that they are hitting their standards). But then you'd need a toolset teachers actually want to use.
- Denzel 10y agoYou're absolutely correct with respect to the differences between assessments in higher ed and those in K-12. K-12 is outside my wheelhouse. Although your point of entry is interesting. I'm wondering what the lead time would be before you hit a large enough mass of tagged questions with student performance data. Do you have any links to information on the current market for curriculum creation/curriculum performance toolsets? (Something with a list of companies, their products, total market penetration, and respective market share.)
- germinalphrase 10y agoNo formal list, but seeing market maps like this is typical:www.cbinsights.com/blog/ed-tech-market-map-company-list There is a diversity of approaches to improving schools and education. They often feel as much like features than they do companies. I have to feel that creating the content/performance/assessment backend opens up a lot of progressive improvement. You'd actually be able to see inside the classroom in a broad and detailed manner. You'd actually be able to personalize instruction without diminishing content to paint-by-numbers. And, maybe - eventually - you'd be able to de-school education and let people learn as they can, where they can.
- 10y ago