4 ms·

An LLM being able to pass your graduate-level exam almost certainly means that your exam is bad. Lots of lazy professors do things like this (true/false and mu

An LLM being able to pass your graduate-level exam almost certainly means that your exam is bad. Lots of lazy professors do things like this (true/false and multiple choice answers, very soft questions, etc.), and the presence of GPTs should help them understand that this is not sufficient for evaluating someone's knowledge of a highly technical topic.

kaesar14 4y ago
Scott Aaronson is one of the best professors and researchers in this field in the world.
- H8crilA 4y ago
  And the test is actually rather easy. It's an undergraduate test.
  - jacquesm 4y ago
    It's probably not 'easy' for the average human out there. I would expect >> 90% of humanity to fail that test.
    - InitialLastName 4y ago
      I have highly technical graduate and undergraduate degrees (in CS-adjacent field) and a decade of experience doing software development part time, but no direct experience with quantum computing (or much beyond an undergraduate physics class and an intro CS class in terms of directly related study), and consider myself a better-than-average test-taker. I could get 1D, 1E, 1F and 1T by intuition, 1J by actual knowledge, 1P by tautology, and had roughly no shot at the open-ended questions.
      - jacquesm 4y ago
        > much beyond an undergraduate physics class and an intro CS class in terms of directly related study That's puts you in < 1% of the population right there.
        InitialLastName 4y ago
        That was my point (in agreement with you): I have no shot, and I'm well into the 99th percentile for this test.
        jacquesm 4y ago
        Right, forgive me I understood it the other way around.
- pclmulqdq 4y ago
  That doesn't mean anything about the quality of his courses or exams. Lots of world-class professors phone it in on their teaching responsibilities because they don't care. I had several of them in school - usually more prestigious professors had worse courses. I can guarantee you that ChatGPT cannot do anything that a quantum computing class prepares you to do, aside from passing this final. That makes it a bad test. ChatGPT, in a sense, is the police on that front.
  - famouswaffles 4y ago
    Oh, you can guarantee, huh? Wow, guess that settles it then.

kaesar14 4y ago

Scott Aaronson is one of the best professors and researchers in this field in the world.

H8crilA 4y ago
And the test is actually rather easy. It's an undergraduate test.
- jacquesm 4y ago
  It's probably not 'easy' for the average human out there. I would expect >> 90% of humanity to fail that test.
  - InitialLastName 4y ago
    I have highly technical graduate and undergraduate degrees (in CS-adjacent field) and a decade of experience doing software development part time, but no direct experience with quantum computing (or much beyond an undergraduate physics class and an intro CS class in terms of directly related study), and consider myself a better-than-average test-taker. I could get 1D, 1E, 1F and 1T by intuition, 1J by actual knowledge, 1P by tautology, and had roughly no shot at the open-ended questions.
    - jacquesm 4y ago
      > much beyond an undergraduate physics class and an intro CS class in terms of directly related study That's puts you in < 1% of the population right there.
      - InitialLastName 4y ago
        That was my point (in agreement with you): I have no shot, and I'm well into the 99th percentile for this test.
        jacquesm 4y ago
        Right, forgive me I understood it the other way around.
pclmulqdq 4y ago
That doesn't mean anything about the quality of his courses or exams. Lots of world-class professors phone it in on their teaching responsibilities because they don't care. I had several of them in school - usually more prestigious professors had worse courses. I can guarantee you that ChatGPT cannot do anything that a quantum computing class prepares you to do, aside from passing this final. That makes it a bad test. ChatGPT, in a sense, is the police on that front.
- famouswaffles 4y ago
  Oh, you can guarantee, huh? Wow, guess that settles it then.