4 ms·

Ok, that's good to know. I do have one other question: You supply a "Accuracy" score, describing how accurate your predictions are. However, you supply probabi

by skolor 17y ago

Ok, that's good to know. I do have one other question: You supply a "Accuracy" score, describing how accurate your predictions are. However, you supply probabilistic predictions, how do you get an "Accuracy" rating from a probability like that?

frisco 17y ago
You're right that it's a probabilistic prediction, so accuracy is a little subtle. If we give 10 people chances of 70% and 7 get in and 3 don't, internally we count that we were 'correct' for all of them. For right now, the website is showing simple thresholding above/below 50% to simplify the whole thing for end-users. We're working on ways to keep complexity down while exposing the nuances of the scoring. Also, it looks like the server that computes Edge Score segfaulted this morning corrupting some of the data in memcached: I restarted it and batch re-generated scores for everyone in the system, and a lot of people that had been getting 310 / 387 scores now were showing higher scores, so that might apply to you. Just what you need when launching...
- skolor 17y ago
  Huh. It sounds like you're doing a lot of interesting stats work (especially looking at the analysis stuff a little more), I'd be rather interested in hearing a little more about what is going on behind on the server side of things. As far as working on complexity, and explanation of what probabilistic predictions are would probably help a lot of people. It wouldn't surprise me if you get a lot of unhappy people come Spring who said "BUT! You said I would get in, and I didn't!" While it would be nice to think that anyone looking at going to college is going to have a basic understanding of probability, you can't just assume that. I would also highlight the "Top Action Items" a little more, you have to dig for it a little bit. Especially for Juniors (right now) or Sophomores (come spring), they have plenty of time to bring up grades and/or retake the SAT/ACT several times to bring things up. An aggregate "Adding X to your SAT score raises your chances of admission by Y%" on the college matches page would be good. It doesn't help current seniors, but it will help those planning ahead a little bit. Oh, and you're right. It brought my score up roughly 200 points in all the areas, to more around where I would expect it to be.
  - deleted 17y ago
    [deleted]

by skolor 17y ago

frisco 17y ago

You're right that it's a probabilistic prediction, so accuracy is a little subtle. If we give 10 people chances of 70% and 7 get in and 3 don't, internally we count that we were 'correct' for all of them. For right now, the website is showing simple thresholding above/below 50% to simplify the whole thing for end-users. We're working on ways to keep complexity down while exposing the nuances of the scoring. Also, it looks like the server that computes Edge Score segfaulted this morning corrupting some of the data in memcached: I restarted it and batch re-generated scores for everyone in the system, and a lot of people that had been getting 310 / 387 scores now were showing higher scores, so that might apply to you. Just what you need when launching...

skolor 17y ago
Huh. It sounds like you're doing a lot of interesting stats work (especially looking at the analysis stuff a little more), I'd be rather interested in hearing a little more about what is going on behind on the server side of things. As far as working on complexity, and explanation of what probabilistic predictions are would probably help a lot of people. It wouldn't surprise me if you get a lot of unhappy people come Spring who said "BUT! You said I would get in, and I didn't!" While it would be nice to think that anyone looking at going to college is going to have a basic understanding of probability, you can't just assume that. I would also highlight the "Top Action Items" a little more, you have to dig for it a little bit. Especially for Juniors (right now) or Sophomores (come spring), they have plenty of time to bring up grades and/or retake the SAT/ACT several times to bring things up. An aggregate "Adding X to your SAT score raises your chances of admission by Y%" on the college matches page would be good. It doesn't help current seniors, but it will help those planning ahead a little bit. Oh, and you're right. It brought my score up roughly 200 points in all the areas, to more around where I would expect it to be.
- deleted 17y ago
  [deleted]