Of course there are a million-and-one possible problems, but the upside is also massive, especially if an automated grader is used to supplement human graders.

Of course there are a million-and-one possible problems, but the upside is also massive, especially if an automated grader is used to supplement human graders. Some examples: -A student can iterate and improve their essays on their own using an auto-grader. They can get it up to a decent level just through iteration before having to get a human involved. -Human graders are highly subjective -Human graders tend to be strongly affected by factors such as "number of hours since they last ate" -Different humans have different levels of harshness, a machine could help calibrate these -Outside, say, the top 10% of colleges, the vast majority of human graders suck. Especially for standardized tests, they typically get paid near minimum wage and the qualifications are along the lines of "have a degree." While automated graders will probably never be as good as the best graders, they don't have to be. So overall, I'm pretty excited. Even a half-baked solution has a lot of potential value.