3 ms·
I suspect the same thing. Rather than LLMs “learning to play chess,” they “learnt” to recognise a chess game and hand over instructions to a chess engine. If th
by scott_w 2y ago
I suspect the same thing. Rather than LLMs “learning to play chess,” they “learnt” to recognise a chess game and hand over instructions to a chess engine. If that’s the case, I don’t feel impressed at all.
- fires10 2y agoRecognize and hand over to a specialist engine? That might be useful for AI. Maybe I am missing something.
- worewood 2y agoIt's because this is standard practice since the early days - there's nothing newsworthy in this at all.
- generic92034 2y agoHow do you think AI are (correctly) solving simple mathematical questions which they have not trained for directly? They hand it over to a specialist maths engine.
- internetter 2y agoThis is a relatively recent development (<3 months), at least for OpenAI, where the model will generate code to solve math and use the response
- cruffle_duffle 2y agoThey’ve been doing that a lot longer than three months. ChatGPT has been handing stuff off to python for a very long time. At least for my paid account anyway.
- nerdponx 2y agoIt is and would be useful, but it would be quite a big lie to the public, but more importantly to paying customers, and even more importantly to investors.
- anon84873628 2y agoThe problem is simply that the company has not been open about how it works, so we're all just speculating here.
- scott_w 2y agoIf I was sold a general AI problem solving system, I’d feel ripped off if I learned that I needed to build my own problem solver and hook it up after I’d paid my money…
- skydhash 2y agoWasn't that the basis of computing and technology in general? Here is one tedious thing, let's have a specific tool that handles it instead of wasting time and efforts. The fact is that properly using the tool takes training and most of current AI marketing are hyping that you don't need that. Instead, hand over the problem to a GPT and it will "magically" solve it.
- Kiro 2y agoThat's something completely different than what the OP suggests and would be a scandal if true (i.e. gpt-3.5-turbo-instruct actually using something else behind the scenes).
- nerdponx 2y agoIronically it's probably a lot closer to what a super-human AGI would look like in practice, compared to just an LLM alone.
- sanderjd 2y agoRight. To me, this is the "agency" thing, that I still feel like is somewhat missing in contemporary AI, despite all the focus on "agents". If I tell an "agent", whether human or artificial, to win at chess, it is a good decision for that agent to decide to delegate that task to a system that is good at chess. This would be obvious to a human agent, so presumably it should be obvious to an AI as well. This isn't useful for AI researchers, I suppose, but it's more useful as a tool. (This may all be a good thing, as giving AIs true agency seems scary.)
- scott_w 2y agoIf this was part of the offering: “we can recognise requests and delegate them to appropriate systems,” I’d understand and be somewhat impressed but the marketing hype is missing this out. Most likely because they want people to think the system is better than it is for hype purposes. I should temper my level of impressed with only if it’s doing this dynamically . Hardcoding recognition of chess moves isn’t exactly a difficult trick to pull given there’s like 3 standard formats…
- Kiro 2y agoYou're speaking like it's confirmed. Do you have any proof? Again, the comment you initially responded to was not talking about faking it by using a chess engine. You were the one introducing that theory.
- gamerDude 2y agoThis is exactly what I feel AI needs. A manager AI that then hands off things to specialized more deterministic algorithms/machines.
- criley2 2y agoBasically what Wolfram Alpha rolled out 15 years ago. It was impressive then, too.
- waffletower 2y agoIt is good to see other people buttressing Stephen Wolfram's ego. It is extraordinarily heavy work and Stephen can't handle it all by himself.
- spiderfarmer 2y agoMulti Agent LLM's are already a thing.
- nine_k 2y agoSomehow they're not in the limelight, and lack a well-known open-source runner implementation (like llama.cpp). Given the potential, they should be winning hands down; where's that?
- waffletower 2y agoWhile deterministic components may be a left-brain default, there is no reason that such delegate services couldn't be more specialized ANN models themselves. It would most likely vastly improve performance if they were evaluated in the same memory space using tensor connectivity. In the specific case of chess, it is helpful to remember that AlphaZero utilizes ANNs as well.
- bigiain 2y agoNext thing, the "manager AIs" start stack ranking the specialized "worker AIs". And the worker AIs "evolve" to meet/exceed expectations only on tasks directly contributing to KPIs the manager AIs measure for - via the mechanism of discarding the "less fit to exceed KPIs". And some of the worker AIs who're trained on recent/polluted internet happen to spit out prompt injection attacks that work against the manager AIs rank stacking metrics and dominate over "less fit" worker AIs. (Congratulations, we've evolved AI cancer!) These manager AIs start performing spectacularly badly compared to other non-cancerous manager AIs, and die or get killed off by the VC's paying for their datacenters. Competing manager AIs get training, perhaps on on newer HN posts discussing this emergent behavior of worker AIs, and start to down rank any exceptionally performing worker AIs. The overall trends towards mediocrity becomes inevitable. Some greybread writes some Perl and regexes that outcompete commercial manager AIs on pretty much every real world task, while running on a 10 year old laptop instead of a cluster of nuclear powered AI datacenters all consuming a city's worth of fresh drinking water. Nobody in powerful positions care. Humanity dies.
- antifa 2y agoTBH I think a good AI would have access to a Swiss army knife of tools and know how to use them. For example a complicated math equation, using a calculator is just smarter than doing it in your head.
- PittleyDunkin 2y agoWe already have the chess "calculator", though. It's called stockfish. I don't know why you'd ask a dictionary how to solve a math problem.
- iamacyborg 2y agoPeople ask LLM’s to do all sorts of things they’re not good at.
- the_af 2y agoA generalist AI with a "chatty" interface that delegates to specialized modules for specific problem-solving seems like a good system to me. "It looks like you're writing a letter" ;)
- datadrivenangel 2y agoLets clip this in the bud before it grows wings.
- nuancebydefault 2y agoIt looks like you have a deja vu
- mkipper 2y agoChess might not be a great example, given that most people interested in analyzing chess moves probably know that chess engines exist. But it's easy to find examples where this approach would be very helpful. If I'm an undergrad doing a math assignment and want to check an answer, I may have no idea that symbolic algebra tools exist or how to use them. But if an all-purpose LLM gets a screenshot of a math equation and knows that its best option is to pass it along to one of those tools, that's valuable to me even if it isn't valuable to a mathematician who would have just cut out of the LLM middle-man and gone straight to the solver. There are probably a billion examples like this. I'd imagine lots of people are clueless that software exists which can help them with some problem they have, so an LLM would be helpful for discovery even if it's just acting as a pass-through.
- kazinator 2y agoThat's not much different from a compiler being rigged to recognize a specific benchmark program and spit out a canned optimization.
- Peteragain 2y ago.. or a Volkswagen recognising an emissions test and turning off power mode...