4 ms·

I have yet to see LLMs solve any new problems. I think it's pretty clear a lot of the bouncing ball programming demos are specifically trained on to be demoed a

I have yet to see LLMs solve any new problems. I think it's pretty clear a lot of the bouncing ball programming demos are specifically trained on to be demoed at a marketing/advertising thing. Asking AI the most basic logical question about a random video game like, what element synergies with ice spike shield in Dragon Cave Masters and it will make up some nonsense despite it being something you can look up on gamefaqs.org. Now I know it knows the game I'm talking about but in the latent space it's just another set of dimensions that flavor likely next token patterns. Sure, if you train an LLM enough on gamefaqs.org, it will be able to answer my question as accurately as an SQL query, and there's a lot of jobs that are just looking up answers that already exist, but these systems are never going to replace engineering teams. Now, I definitely have seen some novel ideas come out of LLMs, especially in earlier models like GPT-3, where hallucinations were more common and prompts weren't normalized into templates, but now we have "mixtures" of "experts" that really keep LLMs from being general intelligences.

XenophileJKO 1y ago
I don't know, I've had O3 create some surprisingly effective Magic the Gathering decks based on newly released cards it has never seen. It just has to look up what cards are available.
- neuroelectron 1y ago
  I think it's an illusion. It pretends not to know things after what some cut off date in 2024 but then if you actually ask it about new things it does a "search" and then it's reiterating things it just read.
outworlder 1y ago
I don't disagree, but your comment is puzzling. You start talking about a game (which probably lacks a lot of training data) and then extrapolate that to mean AI won't replace engineering teams. What? We do not need AGI to cause massive damage to software engineering jobs. A lot of existing work is glue code, which AI can do pretty well. You don't need 'novel' solutions to problems to have useful AI. They don't need to prove P = NP
- sublinear 1y ago
  Can you give an example of a non-trivial project that is pure glue code?
  - arevno 1y ago
    Parent never said pure glue code, they said "a lot", which is roughly correct. Any nontrivial business application will be on the order of ~60% glue, API, interface/model definition, and CRUD UI code, which LLMs are already quite good at. They're also good at writing tests, with the caveat that a human reviews them. They're pretty decent at emitting documentation from pure code, too. The only way these models don't result in mass unemployment in this industry is if the amount of work required expands to fill the gap. Which is certainly possible! The Jevons Paradox of software development.
  - pjmlp 1y ago
    Current fashion in enterprise projects, getting SaaS products talking to each other. You get a CMS piece, talking to a DAM piece, talking to a ecommerce piece, talking to a payment piece, talking to analytics piece. Currently, the integrations are done via a bunch of serverless functions in providers like Vercel, Nelify, Cloudflare Workers, maybe AWS Lambda or Azure Functions. As of now, some of those SaaS vendors are bringing integrations into house, where you describe, via prompts naturally, how the integration should look like. Example, https://www.sitecore.com/products/sitecore-stream https://www.sitecore.com/products/sitecore-stream So less need for as many devs as usual in such integrations.
flashgordon 1y ago
But what if you could re-solve all of today's problems (solutions) with GenAI. Ie if there was an app or game that you like and just want to clone it, that itself today is quite achievable (there was a game -weewar from about 20 years ago that I loved but went out of business. I was able to "clone" it in about 3 weeks). Even if AI is not solving P=NP even doing the repetetive boring bits feels like a huge boost no?

XenophileJKO 1y ago

I don't know, I've had O3 create some surprisingly effective Magic the Gathering decks based on newly released cards it has never seen. It just has to look up what cards are available.

neuroelectron 1y ago
I think it's an illusion. It pretends not to know things after what some cut off date in 2024 but then if you actually ask it about new things it does a "search" and then it's reiterating things it just read.

outworlder 1y ago

I don't disagree, but your comment is puzzling. You start talking about a game (which probably lacks a lot of training data) and then extrapolate that to mean AI won't replace engineering teams. What? We do not need AGI to cause massive damage to software engineering jobs. A lot of existing work is glue code, which AI can do pretty well. You don't need 'novel' solutions to problems to have useful AI. They don't need to prove P = NP

sublinear 1y ago
Can you give an example of a non-trivial project that is pure glue code?
- arevno 1y ago
  Parent never said pure glue code, they said "a lot", which is roughly correct. Any nontrivial business application will be on the order of ~60% glue, API, interface/model definition, and CRUD UI code, which LLMs are already quite good at. They're also good at writing tests, with the caveat that a human reviews them. They're pretty decent at emitting documentation from pure code, too. The only way these models don't result in mass unemployment in this industry is if the amount of work required expands to fill the gap. Which is certainly possible! The Jevons Paradox of software development.
- pjmlp 1y ago
  Current fashion in enterprise projects, getting SaaS products talking to each other. You get a CMS piece, talking to a DAM piece, talking to a ecommerce piece, talking to a payment piece, talking to analytics piece. Currently, the integrations are done via a bunch of serverless functions in providers like Vercel, Nelify, Cloudflare Workers, maybe AWS Lambda or Azure Functions. As of now, some of those SaaS vendors are bringing integrations into house, where you describe, via prompts naturally, how the integration should look like. Example, https://www.sitecore.com/products/sitecore-stream https://www.sitecore.com/products/sitecore-stream So less need for as many devs as usual in such integrations.

flashgordon 1y ago

But what if you could re-solve all of today's problems (solutions) with GenAI. Ie if there was an app or game that you like and just want to clone it, that itself today is quite achievable (there was a game -weewar from about 20 years ago that I loved but went out of business. I was able to "clone" it in about 3 weeks). Even if AI is not solving P=NP even doing the repetetive boring bits feels like a huge boost no?