7 ms·

Every undergrad study is, as of now, possible to pass with AI models. Some of them not only pass, but pass with flying colors. The harsh reality is that academ

• runamuck 1y ago
  I hold remote interviews and I can tell when candidates use AI to answer questions, real time with camera on. They repeat my technical question, pause for a few seconds, their voice drops to a monotone and they quickly recite a bulleted list of low level technical details that sounds like a wikipedia page. I worry that candidates will learn to act more subtle, maybe configure their LLM to return with an anecdote around the tech in question, and practice "selling" their vocal communication.
  • dvh 1y ago
    Students cheat when grades are more valuable than knowledge.
    • Biologist123 1y ago
      Perfect.
    • shinycode 1y ago
      Exactly, the problem is not the students but the amount of efforts the education system has to pour in to change and make it better (and take advantage of LLMs as a learning opportunity )
    • jcranmer 1y ago
      Students cheat when they think grades are more valuable than knowledge. We already have a real-world scourge of tech bros who view learning about other subjects as beneath them and who rate their ignorance as more valuable than experts in that subject. This wonderful new world of AI is only going to exacerbate that problem.
    • esafak 1y ago
      He's talking about graduates. There are people out there trying to hold down multiple remote jobs by cheating through interviews and collecting as many paychecks as possible until they get caught.
    • spacemadness 1y ago
      I would say most technical interviewers cheat as they ask questions they themselves couldn’t solve on the spot.
    • const_cast 1y ago
      Grades are how we try to measure knowledge, because knowledge is abstract and hard to measure. Obviously, the measurement of X runs the risk of being optimized as opposed to X - I see this in my job all the time, and we're all professionals. Point is, it's not a problem with academia and it's not easy to solve. If we could just magically peer into people's minds and figure out what they know definitively then we'd solve a lot of world issues. Alas, we must settle for numbers and papers instead.
  • Dracophoenix 1y ago
    > They repeat my technical question, pause for a few seconds, their voice drops to a monotone and they quickly recite a bulleted list of low level technical details that sounds like a wikipedia page. Can you provide an example?
    • runamuck 1y ago
      Me: "Tell me about your favorite tech stack." Them: "(Pause, eyes scan camera, monotone voice) There are many alternatives for a tech stack, each with their positives and negatives. Let's take a look at some of the more popular options..."
    • esafak 1y ago
      https://koliber.com/articles/how-to-avoid-hiring-a-north-korean-spy-on-your-dev-team https://koliber.com/articles/how-to-avoid-hiring-a-north-kor... I encountered this imposter too.
      • akudha 1y ago
        So the very first recommendation is “check their LinkedIn”? What if I, as a candidate, don’t want to be on LinkedIn or any other social media? I understand companies want to protect themselves. They need to understand some people just don’t want to be on social media (HN aside). In other words, there needs to be a middle ground. Job hunting is already an arduous, demoralizing task. Forcing people to be on social media for a just another layer of stress
        • esafak 1y ago
          A work- and education history that can be validated helps. Put yourself in the shoes of a hiring manager. How would you identify imposters?
  • jeffbee 1y ago
    Not even willing to do remote interviews any more.
  • jerf 1y ago
    One of the things I would say I saw coming from a long ways away is that a lot of effort was put into detecting the "default" voice of LLMs, but people would eventually figure out how to kick them out of it, since it is after literally just a matter of asking. It took some months but word does seem to be finally getting out, I'm seeing the idea emerge in more and more places. I gave it the prompt "Suppose you are in a job interview for a front-end web position and someone asks you about how you use the React library and the hardest problem you even had to solve with it. How might you react, along with a somewhat amusing anecdote?"[1] and it did pretty well. I think I'd play with it a bit to see if I can still suppress some of the LLM-isms that came out, but a human could edit them out in real-time with just a bit of practice too... it's not like you can just read it to your interviewer, you will need to Drama Class 101 this up a bit anyhow. It'll be easier to improv a bit over this than a bare Wikipedia list. In other words, as with the question the article title asks, the question isn't about what happens "when" this starts being possible... the capability has run ahead of all but the most fervent AI user's understanding and it is already here. It's just a matter of the word-of-mouth getting around as to how to prompt the AIs to be less obvious. I also anticipate that in the next couple of years, the AI companies will be getting tired of people complaining about the "default LLM voice" and it'll shift to be something less obvious than it is now. Both remote interviews and college writing are really already destroyed, the news just hasn't gotten around to everybody yet. (In fact I suspect that "default LLM voice" will eventually become a sort of cultural touchstone of 2024-2026 and be deliberately used in future cultural references to set stories in this time period. It's a transient quality of current-day LLMs, easy to get them out of even today, and I expect future LLMs to have much different "default voices".) [1]: And in keeping with my own philosophy of "there's not a lot of value of just pasting in LLM responses" if you want to see what comes out you are welcome to play with it yourself. No huge surprises though. It did the job.
    • tempodox 1y ago
      The cynic in me says there's no problem with that. Either their jobs will be just as LLM-heavy and prompt-jockeying was the right skill to learn, or they won't be able to hold any job because they didn't learn anything relevant, which would be just payment for the cheating.
  • msgodel 1y ago
    Most of this stuff isn't that hard to learn and apply. If they put that much effort into setting up an LLM cheating tool they could have just built something with whatever tool you're talking about.
• SecretDreams 1y ago
  So you gave the easy solution. What's the hard solution? Honestly, the pervasiveness of LLMs looks to really erode the critical thinking of entire future generations. Whatever the solution, we need to be taking these existential threats a lot more seriously than how we treated social media (the plague before this current plague).
  • deleted 1y ago
    [deleted]
  • koakuma-chan 1y ago
    Don't forget to ban calculators
    • rescripting 1y ago
      Calculators _are_ banned when students are learning basic arithmetic. They are only allowed once students can do it on their own, because now you have a foundational understanding and the tool just speeds you up.
    • SecretDreams 1y ago
      If we don't get AI correctly regulated, future gens will probably not be able to work a calculator. Calculator producers will then go out of business - so no need to ban 'em. Thanks for the insightful comment!
    • Tadpole9181 1y ago
      Yes, every single math class I've ever had in my life - primary or second education - banned calculators or (in engineering) required us to perform a full memory reset in front of the TA. Using a machine to do the very thing you are supposed to be demonstrating a proficiency in is cheating and harms the legitimacy of the accreditation of the school.
  • futureshock 1y ago
    I think the hard solution is to massively increase expectations. Think Star Trek where the grade schoolers are learning quantum mechanics. If everyone has access to the oracle of all human knowledge, then you should teach and test to the maximum of what a student could do with all that power. Find the frontier where the AI fails and the human adds value and teach there.
• boerseth 1y ago
  It may get resolved on its own. These days people study to get good grades in order to prove to future gatekeepers (like employers, or higher rungs of academia) that they know the material well. Post AGI, however, the gatekeepers may not be so interested in humans anymore, and we might not need grades at all. Studying anything could become something done exclusively for ones own interest, and the only point of a grade would be to give one-self a goal to achieve. Alternatively, if we still want to cling on to this ritual of measuring the performance of students, you could give each and every one of them oral examinations with AI professors.
  • EGreg 1y ago
    Boom, this. Institutions that prepare people for future jobs have an even harder time to justify what they’re doing than the people who are looking for jobs right now. It’s just inertia at this point. Not to mention that AI can educate the people better by solving Bloom’s Two Sigma Problem. So colleges are obsolete except as four year cruises for entertainment and networking.
  • esafak 1y ago
    We are not post AGI. We still need to hire people who did not cheat their way through school and interviews with chatGPT. Even post AGI, we would want to hire qualified people.
  • spacemadness 1y ago
    Ah yes, the AGI that will happen any day now but we can’t even define. Please buy the stock.
• juujian 1y ago
  Don't even need to go that far. Provide a locked-down computer and have students write essays in a dedicated space. I have personally done that and it was a reasonably good experience.
  • deleted 1y ago
    [deleted]
  • m0llusk 1y ago
    Which raises the issue of what education is for. Is it to know things and solve specific problems in a controlled environment, or is it to work with available tools and resources across a range dynamically changing contexts? Does being at a locked down computer in a dedicated space match likely work settings?
    • stvltvs 1y ago
      This assumes that education's sole goal is to prepare students for work. What would be the impact on democratic systems if voters always turn to an LLM for answers because schools didn't require them to think on their own?
      • m0llusk 1y ago
        This is simply raising one specific case of something education is expected to train for. There is already ample evidence that critical thinking skills are essential for political function and social stability.
    • zerkten 1y ago
      You can be sensitive to both, and good institutions have a duty to check instructors who go strongly in a single direction. I think the point is only to exclude the crutch in contexts where it affects learning. Being prepared for work is only one element of education. Its weight varies by course, institution, and many other factors. Further, there are likely situations where the participants avail of AI to different extents based on how they feel about the situation (cf. different degrees of doping in athletics), and students will sometimes be limited by their means in use of AI tools.
• ninetyninenine 1y ago
  There’s a huge volume of revenue coming from remote students.
• slightwinder 1y ago
  > The easy solution would be to revert back to paper only exams, and physical attendance - but that would also exclude a ton students. Which students? If it's just about travel-distance, maybe schools could organize themselves to offer local test-centers where students could attend exams under observation. Reusing existing facilities in this way is pretty common in my countries education-system since decades.
• rich_sasha 1y ago
  Exams were never the pinnacle of what a grad can do. They were an efficient test, under severe time constraints, that correlates well with overall ability in humans. That AI can pass these tests doesn't mean it is as smart and capable as a grad. I mean, it might be, or if not today then in a few years, but not because it can pass exams, having digested past exams and sample solutions into its bellows.
• msgodel 1y ago
  Most of the important ones I took had: 1) Written in person exams that were most of the grade (this includes "blue book" exams where you have to sit in front of the professor and write an essay on whatever topic he writes on the board that morning as well as your typical math/algorithms tests on paper.) 2) Written homework where you have to essentially have a satisfactory discussion on the topic (no word range, you get graded on creative interpretation of the course subject matter.) Language models could maybe help you with 2 but will actually kill your ability to handle 1 if you're cheating on homework with them. If anything language models will mean the end of those retarded make-work cookie cutter graded homework assignments that got in the way of actually studying and learning.
• Balgair 1y ago
  The cheap version of learning is dead, and AI killed it. Not that we were learning all that much to begin with. I mean, walk into any sorority and ask to see the test bank. The students and Profs were phoning it in for a while, by and large. Not all of them were though, and good on yah. But now that the fig leaf is torn away, we're left with the Oxbridge model and not much else - small classes, under 10, likely under 5, with a grad level tutor, social pressure making sure you've done the work. The great thing about this though is that you'll have an AI listening in all the time and helping out, streamlining the busywork and allowing the group to get down to business. But that version is very expensive. You're looking at ~$50k / student year [0] at a baseline Oxbridge model in secondary school on up - ~$400k / student from 9th to university graduation. Assume a 6% loan rate for 30 years (a mortgage, essentially), and you've got ~$2,300 monthly payments for all your working life, ~$46k/year down the drain. How in the hell are you going to manage student loans like that and then try to live a life without a really good job? How the hell is a nation going to be expected to pay for that per kid if you make school free for them? Cheap learning wasn't good, but it sufficed. The new models of education must answer to the fundamental question of education: How much does it cost? [0] 2 hours 3x a week per class; 4 classes per tutor per week. Assume $100k/tutor and 5 students/tutor. So $5k / student / tutor. 4 classes / student. So $20k / student in just raw tutors. At least double that for overhead if not triple.
• wakawaka28 1y ago
  You're greatly exaggerating the problem. Literally any requirement can exclude students. Establishment of a trusted proctor network for administering exams is how you solve this problem. If you're an online student, you'll have to show up a few days per year to prove your knowledge in person. I believe this is how many remote study programs already work, because AI is just the latest way to cheat. You could always pay people to do your work for you before AI, and the solution is the same.
• beAbU 1y ago
  A renowned South African university, UNISA, has done remote learning for decades. Students had to mail in their work every month, and they would set up exam centres all over the country where students can take their tests. This is not an unsolvable problem if handwritten work becomes a requirement.