2 ms·
It is definitely a nice milestone, for researchers. I do think, though, that one of the rewards of making LLMs promising was the discovery that the bar for “wh
by bee_rider 3y ago
It is definitely a nice milestone, for researchers.
I do think, though, that one of the rewards of making LLMs promising was the discovery that the bar for “what is useful” for any task is higher than what the average person can do. That could be seen as moving the goalposts, but another way of looking at it is: we’ve finally bothered to install a scoreboard.
Like, if a person can’t solve a quadratic equation, they probably just aren’t useful for manipulating equations. Which is fine, of course, the average person is also not a very good hammer or screwdriver.
We’re all specialized, the bar for usefulness is probably something more like “how would a person who’s taken an intro college course on this do it.”
- fnordpiglet 3y agoThe goal isn’t to be more or less useful than a person. It’s to expand the capabilities of computers. And computers are exceptionally good at mathematics tasks already. Why do we care if LLMs are good at things that are already exceptionally well done with computers? The point is they’re exceptionally good at things computers to date have been exceptionally bad at. The rest is weird goalpost moving noise - I wouldn’t use ChatGPT to solve a quadratic equation, I would use Mathematica, or any number of numeric libraries and runtimes. But I challenge you to get Mathematica to summarize a book, carry on a conversation, or even call an api based on natural human language without a rigorous specification apriori. These are holy grails of NLP, which is a capability that is enhancing of the existing capabilities offered by computing technologies today.
- cmpalmer52 3y agoWhat we need are LLMs that can use the tools for mathematics (and other disciplines) that already exist and to write and execute new programs to hopefully solve novel problems, or at least to be able to glue the existing tools together with shells.
- fnordpiglet 3y agoWell, this project can use LLMs do decide routes to take then delegates to a path finding algorithm. That’s why it’s interesting IMO. There are increasing number of things that do that. But I would note the LLMs don’t have to use the tools - a new tool can use the LLM, and be the coordinator between tasks to solve. A lot of the challenges folks are struggling with is making a LLM reliably invoke APIs and stuff. I suspect doing that lobotomizes the LLMs just as guard rails and safety does. Better, IMO, is to manage the LLM in an extrinsic framework like this.