4 ms·
Is it just an open secret that the models are currently just being hardcoded for random benchmarks? Seems weird that people would be asking Putnam problems to a
by yifanl 2y ago
Is it just an open secret that the models are currently just being hardcoded for random benchmarks? Seems weird that people would be asking Putnam problems to a chatbot :/
- Trasmatta 2y agoNot hardcoded, I think it's just likely that those problems exist in its training data in some form
- wslh 2y agoIf I temember well this is call overfitting [1]. [1] https://en.wikipedia.org/wiki/Overfitting https://en.wikipedia.org/wiki/Overfitting
- hansworst 2y agoIsn't that just the LLM equivalent of hardcoding though?
- Trasmatta 2y agoI wouldn't call that hardcoding, otherwise you'd have to call everything it does "hardcoded".
- freehorse 2y ago"Overfitting" would be a bit more accurate term if the problem lies in the specific examples existing in its training set in various forms, places, languages etc but with the same values.
- Panzer04 2y agoSeems a bit picky. If the bot has seen the exact problem before it's not really doing anything more than recall to solve it.
- bandrami 2y ago20 years ago in grad school we were doing a very early iteration of this where we built Markov chains with Shakespeare's plays and wanted to produce a plausibly "Shakespearian" clause given a single word to start and a bearish professor said "the more plausible it gets the more I worry people might forget plausibility is all that it promises". (There was also a much earlier piece of software that would generate semi-intelligible Kant or Hegel one sentence at a time, though that was through a series of a priori generation rules and a large at the time dictionary of stock phrases. I wonder what ever happened to that.)
- jeffreygoesto 2y agoIt became a successful consultant...
- sickblastoise 2y agoI think your prof’s worries came true on a massive scale
- marcosdumay 2y agoThat said, a bot with contextual recall can be very useful. The problem is just that people keep insisting that those things are intelligent.
- InkCanon 2y agoI've always assumed they removed it, because it's such a basic and fundamental part of ML training that you separate your test and train data. And yet I never see any papers even mention if/how they do this. And I wonder if they do, how do they guarantee with high reliability that their massive terabytes of data don't contain the answer.
- llm_trw 2y agoImagine you have someone polluting your training data every day. That's what happens when you scrape any tech forum today. The short version is that llm trainign data is the lowest quality data you are likely to see unless you engage in massive potential copyright infringement.
- deleted 2y ago[deleted]
- ryvi 2y ago> unless you engage in massive potential copyright infringement. And nobody is going to do that
- YetAnotherNick 2y agoFirst of all, Putnam is not in the test data, at least I haven't seen OpenAI claiming that publicly. Secondly, removing it from internet data is not 100% accurate. There are translations of the problems and solutions or references and direct match is not enough. MMLU and test set benchmarks show more resilience though in some previous research.
- deleted 2y ago[deleted]
- rst 2y agoOpenAI is extremely cagey about what's in their test data set generally, but absent more specific info, they're widely assumed to be grabbing whatever they can. (Notably including copyrighted information used without explicit authorization -- I'll take no position on legal issues in the New York Times's lawsuit against OpenAI, but at the very least, getting their models to regurgitate NYT articles verbatim demonstrates pretty clearly that those articles are in the training set.)
- jsheard 2y agoIt certainly feels like certain patterns are hardcoded special cases, particularly to do with math. "Solve (1503+5171)*(9494-4823)" reliably gets the correct answer from ChatGPT "Write a poem about the solution to (1503+5171)*(9494-4823)" hallucinates an incorrect answer though That suggests to me that they've papered over the models inability to do basic math, but it's a hack that doesn't generalize beyond the simplest cases.
- whimsicalism 2y agohttps://chatgpt.com/share/67755e6f-bfc8-8010-9aa3-8bcbbd9b264b https://chatgpt.com/share/67755e6f-bfc8-8010-9aa3-8bcbbd9b26...
- deleted 2y ago[deleted]
- jsheard 2y agoTo be clear I was testing with 4o, good to know that o1 has a better grasp of basic arithmetic. Regardless my point was less to do with the models ability to do math and more to do with OpenAI seeming to cover up its lack of ability.
- whimsicalism 2y agoi think it’s mostly that o1 mini can think through the solution before it starts writing the poem. i’m able to reproduce your failure on 4o
- lelandfe 2y ago“a poem about” reads to me at least like the solution need not be in the answer; maybe something like “a poem that includes the answer in the last stanza”
- deleted 2y ago[deleted]
- mlepath 2y agoYea, people have a really hard time dealing with data leakage especially on data sets as large as LLMs need. Basically if something appeared online or was transmitted over the wire should no longer be eligible to evaluate on. D. Sculley had a great talk at NeurIPS 2024 (same conference this paper was in) titled Empirical Rigor at Scale – or, How Not to Fool Yourself Basically no one knows how to properly evaluate LLMs.
- refulgentis 2y agoNo, an absolute massive amount of people do. In fact they have been doing exactly as you recommend, because as you note, it's obvious and required for a basic proper evaluation.
- resoluteteeth 2y ago> Is it just an open secret that the models are currently just being hardcoded for random benchmarks? Seems weird that people would be asking Putnam problems to a chatbot :/ It's because people do keep asking these models math problems and then, when they get them right, citing it as evidence that they can actually do mathematical reasoning. Since it's hard to determine what the models know, it's hard to determine when they're just spitting out something they were specifically trained on.
- strangescript 2y agoThere are tests they are passing that they can't be hardcoded for by design. They still have all kinds of flaws and inconsistency but getting upset they answer "2+2=4" because someone trained them on what the answer to 2+2 is supposed to be is silly.
- bwfan123 2y agothis work is similar to the GSM symbolic paper (applied to putnam) https://arxiv.org/html/2410.05229v1 https://arxiv.org/html/2410.05229v1 going forward, llm performance must be reported on the confounded benchmark as well