4 ms·
Why does this blogpost read as it was written a year ago? He speaks at getting 100% on a test if we let him google the answers, while LLMs are now beating stude
by scotty79 1y ago
Why does this blogpost read as it was written a year ago? He speaks at getting 100% on a test if we let him google the answers, while LLMs are now beating students at Math Olympiad and even most adult specialists at various other intellectual competitions. Time to update knowledge cutoff point of your wetware you meatbag.
- AuthAuth 1y agoLLMs are beating math olympiads *after scraping the results from the competitions
- scotty79 1y agoHumans tend to have any chance of winning those also after scraping as much of previous results of those competition as they can. I don't think the most intelligent person that ever lived that haven't seen math could go and win math olimpiad.
- devmor 1y agoWhy are you acting like the digital equivalent of rote memorization is an impressive feat? There are things that are impressive about generative AI, but getting correct questions on what is essentially a school test that is available online is not one of them.
- AuthAuth 1y agoYeah its not impressive to serve questions to an AI that has the answers to those questions within its dataset.
- scotty79 1y agoPick a single question from math olympiad. Solve it using any online resource beyond the solution to this specific problem itself, AI or service of smart people. Let us know how it went.
- devmor 1y agoWhy would they exclude the solution to the problem and give themselves a handicap that the AI does not have?
- scotty79 1y agoBecause AI didn't have solutions for this year's problem in its training materials. Same way that students participating in this year's Math Olympiad didn't. Guys, don't you know anything about how such competitions work? You get limited time and a set of problems to solve. New problems, that weren't available anywhere before they were given to you. That might be the issue. How can you appreciate what the AI achieved if you don't know anything about it beyond the name?
- devmor 1y agoPer Gregor Dolinar, President of the IMO: > “It is very exciting to see progress in the mathematical capabilities of AI models, but we would like to be clear that the IMO cannot validate the methods, including the amount of compute used or whether there was any human involvement, or whether the results can be reproduced. What we can say is that correct mathematical proofs, whether produced by the brightest students or AI models, are valid.”
- fragmede 1y agoWhich is to say, they didn't give the AI model the answer, the AI model produced one without having seen it, and that it's valid. It might have taken extra time or had outside help, or otherwise cheated, but it didn't have the answer given to it to merely reproduce.
- devmor 1y agoNo such stipulation or claim is made. The only verified claim is that the proof is valid.
- thrown-0825 1y agoI think you underestimate how much of what we call intelligence is just memorization of some basic principles and skills.
- devmor 1y agoThere is no “we”, here. What you and other AI hype proponents call intelligence and what I call intelligence are not the same thing.
- thrown-0825 1y agoIntelligence is defined as the ability to acquire, understand, and use knowledge in The American Heritage® Dictionary of the English Language, 5th Edition. Based on that definition I think we are well on out way to building an artificial intelligence that is more capable than the overwhelming majority of the planet.
- devmor 1y agoI don’t believe that you understand what an LLM is - that or you have a problem separating science fiction from reality.
- thrown-0825 1y agoYou should let my employers know, they are paying my salary based on the assumption that I do.
- throwawayoldie 1y agoAnd after all a company has never done anything foolish.
- scotty79 1y agoHow much, do you think, of rote memorization competition level math is? > There are things that are impressive about generative AI, but getting correct questions on what is essentially a school test that is available online is not one of them. Nevermind. Try to solve one. You'll be enlightened.
- devmor 1y agoI don’t need to try to solve one. The answers are available online. That is my entire point.
- scotty79 1y agoYou need to solve one to put yourself in the shoes of the AI model to understand its achievement. When the models and students got the puzzles the answers weren't available online yet.
- vntok 1y agoHow do you think LLMs know the competitions results before the competition takes place? https://deepmind.google/discover/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/ https://deepmind.google/discover/blog/advanced-version-of-ge... A month ago LLMs got golden medal by competing at the same event as the students, on the same problems as the students, during the same period as the students.