9 ms·

I took the AP Chem exam last Thursday, fortunately without incident. One of my friends was unable to submit his AP Literature assessment, facing the same error

I took the AP Chem exam last Thursday, fortunately without incident. One of my friends was unable to submit his AP Literature assessment, facing the same error popup that has plagued so many of the other students. I've heard from several other students in my district that have had a similar problem. My brother had an AWS Lambda error message when he tried to access his exam the first time; CollegeBoard had surpassed their Lambda service limits (Fortunately he was able to access his exam after trying again). Their online tooling has never been great–AP Classroom takes at least 3 tries for me to access, one to log in again which then redirects back to AP Central, one that just redirects back to AP Central, then finally one that takes me to my assignments page. This certainly isn't an easy problem to solve. CollegeBoard definitely should have designed a more stable system to host their exams on, but as with anything digital, there's always an opportunity for error–even moreso since everyone takes the exam at the same time (which is an entirely different can of worms, with students in Guam taking exams at 1a, 3a, and 5a). If I were to design these assessments, I'd have it be an untimed, synthesis-based assessment, more similar to AP Computer Science Principles[0], with their Digital Portfolio. Simply having a deadline for all students to upload to a basic tool (which already exists as CB's Digital Portfolio system). Like the current exam, have the submissions sent back to teachers for validation (there's nothing on the current system that validates your identity, simply paste in your AP ID and enter your name/dob, and you're in the system). I think if a student can write a well-formed essay over a topic, they can get college credit. In science classes, they could design an experiment and simulate a lab report. CS classes can create a program, etc. [0]: https://apcentral.collegeboard.org/courses/ap-computer-science-principles/exam https://apcentral.collegeboard.org/courses/ap-computer-scien...

joncrane 6y ago
I wouldn't be shocked if College Board just signed up AWS ProServe to a big contract. If not, details in profile, College Board!
entee 6y ago
I agree to a certain extent on your revised assessment framework, but I’m not sure it would work very well in some of the basic sciences. As someone who has gone to college for these things and beyond, I can tell you there’s very little as a “practicum” that I could have demonstrated coming out of high school or early college. This is in contrast to CS, where if you build a program, the fact that the program exists and solves a given problem correctly is ipso facto a demonstration of understanding. Further, it’s a demonstration of fairly comprehensive understanding of the coursework (you’d have to grasp >80% to have the program work at all), which I have trouble seeing how to do that in say, chemistry. You propose a lab report, with example data, and I actually really like that idea, but I think it would fail on that last score. Looking back, AP chem is a lot of inorganic reactions (lots of memorization, some principles), acid base work, some organic chem, some lab techniques. It’s fairly scattered. A single (or several) lab reports would not capture all of it, especially given that chemistry doesn’t work perfectly predictably even with a post-graduate understanding. There’s a reason basic science lab courses do boring experiments, it’s because those work reliably. Anyway, I like the idea, and want to hear more on how to do it, but a lab report would have to be supplemented by something else to really evaluate understanding in the sciences.
- jackson1442 6y ago
  AP Chem is definitely an exercise in memorization and general "knowledge of chemistry" for a good portion of the class. Though not a complete solution, a lab report-based assessment could look more similar to the CS Principles exam in another facet: written response. On the APCSP exam, there's actually very few points for your code itself. You get a point for identifying an abstraction and a point for identifying an algorithm, but the rest of the points (there are 8 total) are assigned based on your written responses. Analytical questions could be added to each exam–broad enough that no two students should have too much overlap, but narrow enough that it's clear what is expected of a student response. From there, a few points could be assigned for the quality of the designed experiment, but a majority would be assigned based on the quality of the written response questions. AP Physics follows a similar model on the regular paper exam–you have to prove some theory and design a complete experiment to do so. I believe it also included one light analysis question.
- derefr 6y ago
  > Further, it’s a demonstration of fairly comprehensive understanding of the coursework (you’d have to grasp >80% to have the program work at all), which I have trouble seeing how to do that in say, chemistry. Lock someone in a room with some reagents, and expect them to produce extracted/cleaned output products of a certain testable purity and mass after N time. Make the process of getting from the reagents to the final product really involved, requiring at least one of every kind of basic reaction/setup taught in the course. (NileRed's "aspirin to tylenol" sequence might be a good example challenge: https://www.youtube.com/watch?v=b0Ejuew2riA https://www.youtube.com/watch?v=b0Ejuew2riA) You know this is "the practical approach", because this is essentially how gangs qualify their chemists. "If you can really make [illegal drug], then make us some! Without doing anything that'll get the police called! In four hours!"
  - slavik81 6y ago
    I had excellent knowledge of Chemistry theory, but I could not for the life of me get the expected results in lab. Like, I know that you should not be able to make a battery out of a lead anode and a lead cathode in distilled water, but I did. That sort of practicum may be a reasonable test, but could give very different results than the normal written exam.
acbart 6y ago
I still have yet to hear anyone propose a workable solution for the ultimate problem of remote assessment in CS: hiring someone to just do your work for you. Everyone keeps saying "oh just make it project-based" like I didn't have a student last fall who hired someone to do his final project. I only detected him cause the person he hired sucked and left bread-crumbs; how many students did I not detect because they hired competent people?
- Frondo 6y ago
  Could something like pair programming work to reduce the risk of hiring out the whole project?
  - acbart 6y ago
    Reduces the risk, but doesn't prevent it.
- GhostVII 6y ago
  You could make them give a video presentation about their project to show that they actually know how it works, not foolproof but it helps. I think it is pretty much impossible to completely remove that possibility though. I've had lots of courses where the majority of my mark is assignments and projects, and I'm sure some students pay others to do it for them, and the professors just accept that as a possibility. I don't think it is worth providing a worse experience to the majority of honest students to prevent a small minority of dishonest students from cheating.
- deleted 6y ago
  [deleted]
- vulcan01 6y ago
  An interesting way to assess your students, if you have time and a reasonable amount of students, is to do what the IB organisation required my Spanish teacher to do: a one-on-one oral exam with presentation and follow-up questions from the teacher, all recorded and sent to the IBO grading board.
LeifCarrotson 6y ago
The entire problem seems to be that the College Board (and ACT and SAT) took an initial design that works fine in a proctored classroom and applied it to a global testing scenario where it doesn't fit. > If I were to design these assessments, I'd have it be an untimed, synthesis-based assessment... Yes, but that takes more resources and has more variability among reviewers. It does have the advantages of being more fair for students with time zone issues or closures, and of being durable in the face of temporary website issues. Judging by their decisions, the priorities of the testing agencies seem to have been taking out the cost of the human element of reviewers (multiple choice is easy to grade, synthesis essays need expert reviewers) and the variability of the tests and reviewers to certify their results for schools.
- Gaelan 6y ago
  Normal AP tests are half multiple choice and half "free response." What "free response" means depends heavily on the test, but they are always human-graded. This year, the tests are free-response only.
  - pirocks 6y ago
    Human graded is a generous way of describing what they do with free response. When I took ap tests a large part of the preparation was knowing exactly how they where going to grade the questions, and optimising answers to hit as many points on the answer key.
  - bsder 6y ago
    Yeah, "free-response" is only true if you're unprepared. A significant part of taking AP English when I went through was the teacher saying: "Okay, here is the question. Now, what are the likely literature references the AP reader expects you to use? Okay, what are the likely scoring points the AP reader has on their worksheet? Okay, now write to those. Writing anything else is a waste of time." You could count the number of students out of her class who didn't score a 5 on a single hand.
    - cxseven 6y ago
akhilcacharya 6y ago
> My brother had an AWS Lambda error message when he tried to access his exam the first time; CollegeBoard had surpassed their Lambda service limits (Fortunately he was able to access his exam after trying again). This is absolutely hilarious. Did they not realize the concurrency limitations?

joncrane 6y ago

I wouldn't be shocked if College Board just signed up AWS ProServe to a big contract. If not, details in profile, College Board!

entee 6y ago

I agree to a certain extent on your revised assessment framework, but I’m not sure it would work very well in some of the basic sciences. As someone who has gone to college for these things and beyond, I can tell you there’s very little as a “practicum” that I could have demonstrated coming out of high school or early college. This is in contrast to CS, where if you build a program, the fact that the program exists and solves a given problem correctly is ipso facto a demonstration of understanding. Further, it’s a demonstration of fairly comprehensive understanding of the coursework (you’d have to grasp >80% to have the program work at all), which I have trouble seeing how to do that in say, chemistry. You propose a lab report, with example data, and I actually really like that idea, but I think it would fail on that last score. Looking back, AP chem is a lot of inorganic reactions (lots of memorization, some principles), acid base work, some organic chem, some lab techniques. It’s fairly scattered. A single (or several) lab reports would not capture all of it, especially given that chemistry doesn’t work perfectly predictably even with a post-graduate understanding. There’s a reason basic science lab courses do boring experiments, it’s because those work reliably. Anyway, I like the idea, and want to hear more on how to do it, but a lab report would have to be supplemented by something else to really evaluate understanding in the sciences.

jackson1442 6y ago
AP Chem is definitely an exercise in memorization and general "knowledge of chemistry" for a good portion of the class. Though not a complete solution, a lab report-based assessment could look more similar to the CS Principles exam in another facet: written response. On the APCSP exam, there's actually very few points for your code itself. You get a point for identifying an abstraction and a point for identifying an algorithm, but the rest of the points (there are 8 total) are assigned based on your written responses. Analytical questions could be added to each exam–broad enough that no two students should have too much overlap, but narrow enough that it's clear what is expected of a student response. From there, a few points could be assigned for the quality of the designed experiment, but a majority would be assigned based on the quality of the written response questions. AP Physics follows a similar model on the regular paper exam–you have to prove some theory and design a complete experiment to do so. I believe it also included one light analysis question.
derefr 6y ago
> Further, it’s a demonstration of fairly comprehensive understanding of the coursework (you’d have to grasp >80% to have the program work at all), which I have trouble seeing how to do that in say, chemistry. Lock someone in a room with some reagents, and expect them to produce extracted/cleaned output products of a certain testable purity and mass after N time. Make the process of getting from the reagents to the final product really involved, requiring at least one of every kind of basic reaction/setup taught in the course. (NileRed's "aspirin to tylenol" sequence might be a good example challenge: https://www.youtube.com/watch?v=b0Ejuew2riA https://www.youtube.com/watch?v=b0Ejuew2riA) You know this is "the practical approach", because this is essentially how gangs qualify their chemists. "If you can really make [illegal drug], then make us some! Without doing anything that'll get the police called! In four hours!"
- slavik81 6y ago
  I had excellent knowledge of Chemistry theory, but I could not for the life of me get the expected results in lab. Like, I know that you should not be able to make a battery out of a lead anode and a lead cathode in distilled water, but I did. That sort of practicum may be a reasonable test, but could give very different results than the normal written exam.

acbart 6y ago

I still have yet to hear anyone propose a workable solution for the ultimate problem of remote assessment in CS: hiring someone to just do your work for you. Everyone keeps saying "oh just make it project-based" like I didn't have a student last fall who hired someone to do his final project. I only detected him cause the person he hired sucked and left bread-crumbs; how many students did I not detect because they hired competent people?

Frondo 6y ago
Could something like pair programming work to reduce the risk of hiring out the whole project?
- acbart 6y ago
  Reduces the risk, but doesn't prevent it.
GhostVII 6y ago
You could make them give a video presentation about their project to show that they actually know how it works, not foolproof but it helps. I think it is pretty much impossible to completely remove that possibility though. I've had lots of courses where the majority of my mark is assignments and projects, and I'm sure some students pay others to do it for them, and the professors just accept that as a possibility. I don't think it is worth providing a worse experience to the majority of honest students to prevent a small minority of dishonest students from cheating.
deleted 6y ago
[deleted]
vulcan01 6y ago
An interesting way to assess your students, if you have time and a reasonable amount of students, is to do what the IB organisation required my Spanish teacher to do: a one-on-one oral exam with presentation and follow-up questions from the teacher, all recorded and sent to the IBO grading board.

LeifCarrotson 6y ago

The entire problem seems to be that the College Board (and ACT and SAT) took an initial design that works fine in a proctored classroom and applied it to a global testing scenario where it doesn't fit. > If I were to design these assessments, I'd have it be an untimed, synthesis-based assessment... Yes, but that takes more resources and has more variability among reviewers. It does have the advantages of being more fair for students with time zone issues or closures, and of being durable in the face of temporary website issues. Judging by their decisions, the priorities of the testing agencies seem to have been taking out the cost of the human element of reviewers (multiple choice is easy to grade, synthesis essays need expert reviewers) and the variability of the tests and reviewers to certify their results for schools.

Gaelan 6y ago
Normal AP tests are half multiple choice and half "free response." What "free response" means depends heavily on the test, but they are always human-graded. This year, the tests are free-response only.
- pirocks 6y ago
  Human graded is a generous way of describing what they do with free response. When I took ap tests a large part of the preparation was knowing exactly how they where going to grade the questions, and optimising answers to hit as many points on the answer key.
- bsder 6y ago
  Yeah, "free-response" is only true if you're unprepared. A significant part of taking AP English when I went through was the teacher saying: "Okay, here is the question. Now, what are the likely literature references the AP reader expects you to use? Okay, what are the likely scoring points the AP reader has on their worksheet? Okay, now write to those. Writing anything else is a waste of time." You could count the number of students out of her class who didn't score a 5 on a single hand.
  - cxseven 6y ago

akhilcacharya 6y ago

> My brother had an AWS Lambda error message when he tried to access his exam the first time; CollegeBoard had surpassed their Lambda service limits (Fortunately he was able to access his exam after trying again). This is absolutely hilarious. Did they not realize the concurrency limitations?