9 ms·
I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be
by Validark 6mo ago
I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.
- himata4113 6mo agoIt's less of solving a problem, but trying every single solution until one works. Exhaustive search pretty much. It's pretty much how all the hard problems are solved by AI from my experience.
- lsc4719 6mo agoThat's also the only way how humans solve hard problems.
- jMyles 6mo agoThere have been both inductive and deductive solutions to open math problems by humans in the past decade, including to fairly high-profile problems.
- himata4113 6mo agoNot always, humans are a lot better at poofing a solution into existence without even trying or testing. It's why we have the scientific method: we come up with a process and verify it, but more often than not we already know that it will work. Compared to AI, it thinks of every possible scientific method and tries them all. Not saying that humans never do this as well, but it's mostly reserved for when we just throw mud at a wall and see what sticks.
- virgildotcodes 6mo agoMore often than not, far, far, far more often than not, we do not already know that it will work. For all human endeavors, from the beginning of time. If we get to any sort of confidence it will work it is based on building a history of it, or things related to "it" working consistently over time, out of innumerable other efforts where other "it"s did not work.
- nextaccountic 6mo agoAI can one shot problems too, if they have the necessary tools in their training data, or have the right thing in context, or have access to tools to search relevant data. Not all AI solutions are iterative, trial and error. Also > humans are a lot better at (...) That's maybe true in 2026, but it's hard to make statements about "AI" in a field that is advancing so quickly. For most of 2025 for example, AI doing math like this wouldn't even be possible
- coderenegade 6mo agoThat's just not true at all. There are entire fields that rest pretty heavily on brute force search. Entire theses in biomedical and materials science have been written to the effect of "I ran these tests on this compound, and these are the results", without necessarily any underlying theory more than a hope that it'll yield something useful. As for advances where there is a hypothesis, it rests on the shoulders of those who've come before. You know from observations that putting carbon in iron makes it stronger, and then someone else comes along with a theory of atoms and molecules. You might apply that to figuring out why steel is stronger than iron, and your student takes that and invents a new superalloy with improvements to your model. Remixing is a fundamental part of innovation, because it often teaches you something new. We aren't just alchemying things out of nothing.
- himata4113 6mo agoWell, we know that mixing lead into copper won't make for a strong material. There's a lot of human ingenuity involved. I failed to make my point clear: Humans make the search area way smaller compared to current day AI.
- adventured 6mo agoNo, that's precisely solving a problem. Shotgunning it is an entirely valid approach to solving something. If AI proves to be particularly great at that approach, given the improvement runway that still remains, that's fantastic.
- raincole 6mo agoIn other words, it's solving a problem.
- qsera 6mo agoA random sentence can also generate correct solution to a problem once in a long while...does not mean that it "solved" anything..
- kranner 6mo agoBet you didn't come up with that comment by first discarding a bunch of unsuitable comments.
- bfivyvysj 6mo agoHow often do you self edit before submitting?
- virgildotcodes 6mo agoYou learned what was unsuitable over your entire life until now by making countless mistakes in human interaction. A basic AI chat response also doesn't first discard all other possible responses.
- ivalm 6mo agobecause commenting is easy and solving hard problems is hard
- raincole 6mo agoI hired an artist for an oil painting. The artist drew 10 pencil sketches and said "hmm I think this one works the best" and finished the painting based on it. I said he didn't one shot it and therefore he has no ability to paint, and refused to pay him.
- slg 6mo agoYes, but is it "intelligence" is a valid question. We have known for a long time that computers are a lot faster than humans. Get a dumb person who works fast enough and eventually they'll spit out enough good work to surpass a smart person of average speed. It remains to be seen whether this is genuinely intelligence or an infinite monkeys at infinite typewriters situation. And I'm not sure why this specific example is worthy enough to sway people in one direction or another.
- kelseyfrog 6mo agoHow do you think mathematicians solve problems?
- famouswaffles 6mo agoIf LLMs really solved hard problems by 'trying every single solution until one works', we'd be sitting here waiting until kingdom come for there to be any significant result at all. Instead this is just one of a few that has cropped up in recent months and likely the foretell of many to come.
- jasonfarnon 6mo agoThe link has an entire section on "The infeasibility of finding it by brute force."
- konart 6mo agoBut this is exactly how we do math. We start writing all those formulas etc and if at some point we realise we went th wrong way we start from the begignning (or some point we are sure about).
- storus 6mo agoAI is a remixer; it remixes all known ideas together. It won't come up with new ideas though; the LLMs just predict the most likely next token based on the context. That means the group of characters it outputs must have been quite common in the past. It won't add a new group of characters it has never seen before on its own.
- deleted 6mo ago[deleted]
- maxrmk 6mo agoI don't think this is a correct explanation of how things work these days. RL has really changed things.
- energy123 6mo agoModels based on RL are still just remixers as defined above, but their distribution can cover things that are unknown to humans due to being present in the synthetic training data, but not present in the corpus of human awareness. AlphaGo's move 37 is an example. It appears creative and new to outside observers, and it is creative and new, but it's not because the model is figuring out something new on the spot, it's because similar new things appeared in the synthetic training data used to train the model, and the model is summoning those patterns at inference time.
- trick-or-treat 6mo ago> the model is summoning those patterns at inference time. You can make that claim about anything: "The human isn't being creative when they write a novel, they're just summoning patterns at typing time". AlphaGo taught itself that move, then recalled it later. That's the bar for human creativity and you're holding AlphaGo to a higher standard without realizing it.
- energy123 6mo agoI can't really make that claim about human cognition, because I don't have enough understanding of how human cognition works. But even if I could, why is that relevant? It's still helpful, from both a pedagogical and scientific perspective, to specify precisely why there is seeming novelty in AI outputs. If we understand why, then we can maximize the amount of novelty that AI can produce. AlphaGo didn't teach itself that move. The verifier taught AlphaGo that move. AlphaGo then recalled the same features during inference when faced with similar inputs.
- snemvalts 6mo agoMath and coding competition problems are easier to train because of strict rules and cheap verification. But once you go beyond that to less defined things such as code quality, where even humans have hard time putting down concrete axioms, they start to hallucinate more and become less useful. We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by playing itself. As we have only made progress on unsupervised learning, and RL is constrained as above, I don't see this getting better.
- charcircuit 6mo agoLLMs already do unsupervised learning to get better at creative things. This is possible since LLMs can judge the quality of what is being produced.
- zozbot234 6mo agoThis is not formally verified math so there is no real verifiable-feedback aspect here. The best models for formalized math are still specialized ones. although general purpose models can assist formalization somewhat.
- zar1048576 6mo ago[dead]
- otabdeveloper4 6mo agoLLMs can often guess the final answer, but the intermediate proof steps are always total bunk. When doing math you only ever care about the proof, not the answer itself.
- eru 6mo agoOnce you have a working proof, no matter how bad, you can work towards making it nicer. It's like refactoring in programming. If your proof is machine checkable, that's even easier.
- bigstrat2003 6mo agoThe problem is that the AI industry has been caught lying about their accomplishments and cheating on tests so much that I can't actually trust them when they say they achieved a result. They have burned all credibility in their pursuit of hype.
- parasubvert 6mo agoI'm all for skeptical inquiry, but "burning all credibility" is an overreaction. We are definitely seeing very unexpected levels of performance in frontier models.
- otabdeveloper4 6mo ago> born-again AI believer sigh
- Validark 6mo agoI honestly do think I'm being honest with myself. I have held it in my mind that I'm not impressed until it's innovative. That threshold seems to be getting crossed. I'm not saying, "I used to be an atheist, but then I realized that doesn't explain anything! So glad I'm not as dumb now!"
- otabdeveloper4 6mo agoSomehow people don't need "faith" and "being impressed" to make a hammer or a car work. (This shows that LLMs aren't tools yet.)
- mo7061 6mo agoIt 100% will not be used to make the world better and we all know it will be weaponised first to kill humans like all preceding tech
- tim333 6mo agoMost tech gets used for good and bad.
- catlifeonmars 6mo agoAre the only two options AI doubter and AI believer?
- qsera 6mo agoAsking the right questions...
- sph 6mo agoAll I hear about are AI believers and AI-doubters-just-turned-believers
- Validark 6mo agoHey, I'm a real person. Here's my website. I have YouTube videos up with my real name and face. https://validark.dev https://validark.dev
- Validark 6mo agoPerhaps I should have elaborated more but what I mean is I used to think, "I genuinely don't see the point in even trying to use AI for things I'm trying to solve". Ironically though, I think that because I've repeatedly tried and tested AI and it falls flat on its face over and over. However, this article makes me more hopeful that AI actually could be getting smarter.
- keeda 6mo agoI'm curious as to why you consider this as the benchmark for AI capabilities. Extremely few humans can solve hard problems or do much innovation. The vast majority of knowledge work requires neither of these, and AI has been excelling at that kind of work for a while now. If your definition of AI requires these things, I think -- despite the extreme fuzziness of all these terms -- that it's closer to what most people consider AGI, or maybe even ASI.
- Validark 6mo agoFair point, however I am simply more interested in how AI can advance frontiers than in how it can transcribe a meeting and give a summary or even print out React code. I know the world is heavily in need of the menial labor and AI already has made that stuff way easier and cheaper. However I'm just very interested in innovation and pushing the boundaries as a more powerful force for change. One project I've been super interested in for a while is the Mill CPU architecture. While they haven't (yet) made a real chip to buy, the ideas they have are just super awesome and innovative in a lot of areas involving instruction density & decoding, pipelining, and trying to make CPU cores take 10% of the power. I hope the Mill project comes to fruition, and I hope other people build on it, and I hope that at some point AI could be a tool that prints out innovative ideas that took the Mill folks years to come up with.
- hnfong 6mo agoIt's kind of interesting in your original comment you used the words "doubter" and "believer", as if AI was some kind of messianic event of some sort and you are deciding whether to "believe" in it. I mean, if you step back and think about it, there's nothing that requires faith. As you said, current AI can do a lot of things pretty well (transcribe and summarize meetings, write boilerplate code, etc.) Nobody is doubting this. And AI is definitely helping in innovation to some extent. Not necessarily drive it singlehandedly, but some people working on world-changing innovation find AI useful. So yeah, I think some people are subconsciously not doubting whether AI works, but kinda having conflicted thoughts about AI being our new overlords or something. If you think about it, is having AI that's capable of innovating better than humans really a good thing? Like, even if we manage to make benign AI who won't copy how humans are jerks to each other, it kinda takes away our fun of discovery.
- doctorpangloss 6mo agomost issues at every scale of community and time are political, how do you imagine AI will make that better, not worse? there's no math answer to whether a piece of land in your neighborhood should be apartments, a parking lot or a homeless shelter; whether home prices should go up or down; how much to pay for a new life saving treatment for a child; how much your country should compel fossil fuel emissions even when another country does not... okay, AI isn't going to change anything here, and i've just touched on a bunch of things that can and will affect you personally. math isn't the right answer to everything, not even most questions. every time someone categorizes "problems" as "hard" and "easy" and talks about "problem solving," they are being co-opted into political apathy. it's cringe for a reason. there are hardly any mathematicians who get elected, and it's not because voters are stupid! but math is a great way to make money in America, which is why we are talking about it and not because it solves problems. if you are seeking a simple reason why so many of the "believers" seem to lack integrity, it is because the idea that math is the best solution to everything is an intellectually bankrupt, kind of stupid idea. if you believe that math is the most dangerous thing because it is the best way to solve problems, you are liable to say something really stupid like this: > Imagine, say, [a country of] 50 million people, all of whom are much more capable than any Nobel Prize winner, statesman, or technologist... this is a dangerous situation... Humanity needs to wake up https://www.darioamodei.com/essay/the-adolescence-of-technology https://www.darioamodei.com/essay/the-adolescence-of-technol... Dario Amodei has never won an election. What does he know about countries? (nothing). do you want him running anything? (no). or waking up humanity? In contrast, Barack Obama, who has won elections, thinks education is the best path to less violence and more prosperity. What are you a believer in? ChatGPT has disrupted exactly ONE business: Chegg, because its main use case is cheating on homework. AI, today, only threatens one thing: education. Doesn't bode well for us.
- Validark 6mo agoI agree with what you're saying, and I certainly don't think the one problem facing my country or the world is just that we didn't solve the right math problem yet. I am saddened by the direction the world keeps moving. When I wrote that I hope we use it for good things, I was just putting a hopeful thought out there, not necessarily trying to make realistic predictions. It's more than likely people will do bad things with AI. But it's actually not set in stone yet, it's not guaranteed that it has to go one way. I'm hopeful it works out.
- qsera 6mo ago>it really is a new and exciting world... The point is that from now on, there will be nothing really new, nothing really original, nothing really exciting. Just endless stream of re-hashed old stuff that is just okayish.. Like an AI spotify playlist, it will keep you in chains (aka engaged) without actually making you like really happy or good. It would be like living in a virtual world, but without having anything nice about living in such a world.. We have given up everything nice that human beings used to make and give to each other and to make it worse, we have also multiplied everything bad, that human being used to give each other..
- prox 6mo agoI heard this saying recently “The problem with comfort is that it makes you comfortable.”
- egeozcan 6mo agoOn what do you base your prediction? Is it because the AI is trained with existing data? But, we are also trained with existing data. Do you think that there's something that makes human brain special (other than the hundreds of thousands years of evolution but that's what AI is all trying to emulate)? This may sound hostile (sorry for my lower than average writing skills), but trust me, I'm really trying to understand.
- bogdan 6mo ago> there will be nothing really new How is this the conclusion? Isn't this post about AI solving something new? What am I missing?
- paganel 6mo agoEach solvable problem contains its solution intrinsically, so to speak, it’s only a matter of time and consuming of resources to get to it. There’s nothing creative about it, which is I think what OP was alluding to (the creative part). I’m talking mostly mathematics. There’s also a discussion to be made about maths not being intrinsically creative if AI automatons can “solve” parts of it, which pains me to write down because I had really thought that that wasn’t the case, I genuinely thought that deep down there was still something ethereal about maths, but I’ll leave that discussion for some other time.
- jacquesm 6mo ago> I really hope we use this intelligence resource to make the world better. I wished I had your optimism. I'm not an AI doubter (I can see it works all by myself so I don't think I need such verification). But I do doubt humanity's ability to use these tools for good. The potential for power and wealth concentration is off the scale compared to most of our other inventions so far.
- deleted 6mo ago[deleted]
- keybored 6mo ago> I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. We already have a few years of experience with this. > I really hope we use this intelligence resource to make the world better. We already have a few years of experience with this.
- torginus 6mo agoI remember there was a conversation between two super-duper VCs (dont remember who but famous ones), about how DeepSeek was a super-genius level model because it solved an intro-level (like week 1-2) electrodynamics problem stated in a very convoluted way. While cool and impressive for an LLM, I think they oversold the feat by quite a bit. I don't want to belittle the performance of this model, but I would like for someone with domain expertise (and no dog in the AI race, like a random math PhD) to come forward, and explain exactly what the problem exactly was, and how did the model contribute to the solution.