4 ms·

> I'll risk sounding antagonistic and ask you this: if LLMs are not "good enough", why are they still around? To quote the parent commenter: > They seem to be

• notashelf 2mo ago
  But that means they're good enough for the short them. This is why I find it such an interesting question to wield, but also, it is why my wording was poor in the post.
  • deleted 2mo ago
    [deleted]
  • deathanatos 2mo ago
    Short term can, IME, be very short. I've seen people generate, say, a bash script with an LLM. It's generated: short term, the problem is "solved": we've generated a bash script. … but does it work? Someone comes along, reviews it, "this is garbage, and does not do what it says it purports to do". Perhaps it even gave an output: the script computed … something, but it's just GIGO. But that "check if this works" friction is the same friction that is what people try to avoid by generating it with an LLM in the first place. If you're too lazy to write the script, you're practically by definition too lazy to verify it.
    • aprdm 2mo ago
      You can have another agent write the tests and verify the former agent ? This is pretty basic stuff. Makes me question if people are actually trying to use AI
      • habinero 2mo ago
        Now you have two problems lol, in that you don't know if the tests are any good or actually test the thing in question. Sooner or later you run out of turtles to put on the stack.
        • ileonichwiesz 2mo ago
          That’s a solved problem, you just add another agent to check if the tests are any good, and one more to oversee the test-checker, and one more…
    • TheOtherHobbes 2mo ago
      If this is your working environment, it sounds like quite an unusual place. I literally can't imagine generating a script with an LLM without testing it at all. Bash is one of those situations where LLMs can do really well. No human on Earth can remember all of the commands and even fewer humans can remember all of the switches for all of the commands. This kind of remembering, searching, and assembling is exactly what LLMs are good at - as long as you're not writing a gigantic build system with hundreds of moving parts, in which case you should probably be using something more streamlined anyway.
      • lmm 2mo ago
        > I literally can't imagine generating a script with an LLM without testing it at all. Then you're extremely unimaginative as well as unusually fastidious. Certainly someone - several someones - are generating lots of scripts and not testing them, given the PRs I'm seeing.
• throwatdem12311 2mo ago
  Breaking down long term problems into small shorter problems we can solve one after another until the long term problem is solved is the essence of engineering.
  • timacles 2mo ago
    No, the "essence" of engineering is having a cohesive vision for a larger plan. No quality large scale project can exist without it. And any seasoned engineer should understand by now that no SOTA LLM can produce quality engineering at scale.
    • aprdm 2mo ago
      Lol. That's not true at all. And not how projects are done on startups or big companies. Execute fast or get nowhere, a series of small wins get you to live long term.
      • dwattttt 2mo ago
        Long term projects that survived on short term wins are still around. But you're going to want to check how big the graveyard is of "project collapsed under tech debt from short term decisions" to see whether it's a good strategy.
      • habinero 2mo ago
        Nah. You can succeed in every single individual thing but fail at the overall project because the parts don't align into anything that makes sense. If you need examples, look at game dev. There's plenty of games that have good execution but aren't fun to play or they're a confusing mess because there wasn't good overall direction.
        • throwatdem12311 2mo ago
          Videogames are a weird comparison because they are basically the ultimate culmination of the combination of both engineering and art. The two can crossover in weird ways but still have distinct aspects that do not really compromise the other. Just because a videogame isn’t fun doesn’t mean there wasnt a cohesive engineering vision. God of War: Ragnarok is a good example of this. Excellent technical execution and (arguably) great direction but a terribly boring game. And many games that are terribly engineered are also incredibly fun. Dark Souls is a good example - runs like ass, looks very rough even by the standards of the time when the game released, with terrible enemy AI in a combat focused game - one might even call it a confusing mess (and I laugh at the notion that it doesn’t have good direction)… despite all this it’s a classic and arguably the most influential game since its release. Elden Ring carries a lot of legacy baggage from years of iteration on the same engine as ~~Dark~~ Demons Souls - and it’s widely considered From Software’s magnum opus and a masterpiece. Yet considering the scope of the game it is a great example of how small wins added up can solve large problems when executed with the skill and judgment gained from experience. So one might still consider it a success in terms of “engineering”. Would From Software have been able to make the same game using Unreal Engine 5 built from scratch with dime a dozen “Unreal experts” brought in off the street? Not a chance. Want to know why western game development studios are suffering right now? Compare American layoff culture and race to the bottom economics to the retention rates of Japanese game studios like Capcom, Nintendo and other heavy hitters. For many of the most marvellous things humans have ever built, if you take a peak behind the curtain you’ll still probably find some amount of duct tape and popsicles sticks somewhere keeping it all together. Fun, however, is a matter of taste and you can’t engineer that.