3 ms·
If you mean Google search, that had an immediate and obvious value from the day it went live. I didn’t call Google or anything else useless. I suggested definin
by gregjor 2y ago
If you mean Google search, that had an immediate and obvious value from the day it went live. I didn’t call Google or anything else useless. I suggested defining and measuring expected benefits, costs, and risks. I would suggest that for any unproven technology, not just LLMs.
We can revisit the graveyard of over-hyped technologies if you want, and catalog the failures and wasted money. Ride the wave if you like. Show the success stories of LLMs in terms of business value or innovation. Don’t just repeat the press releases and anecdotes. You won’t find many successes, though maybe they will come in time.
> I’ve found [an LLM] good at writing SQL queries.
I don’t mean snark, but how do you know it writes good queries? I will grant that it can write queries that execute, but do those give correct results? Will they look good on a large production database where performance starts to matter?
When I played with LLMs writing SQL they worked well enough on toy schemas that resemble those found in tutorials. But faced with a real schema that requires joining tables and applying complex conditions they failed to produce usable SQL. However they did (with prodding) produce SQL that executed. It just gave the wrong results. That’s not a gain along any axis and could cost a lot of money for an organization blindly trusting such tools.
LLMs can write “good” code for some constrained definition of “good.” If you also want correct, testable, maintainable, secure, etc. they don’t perform well enough. Not just my opinion, we’ve had time to study the output of coding LLMs and it’s not looking good.
- uptownfunk 2y agoFor me they provide a good jumping off point. I do find I still need to read it and tweak it. But it’s much easier than having to write the damn thing myself. I am not trying to be an engineer, I am trying to build a business and get an mvp off the ground. And if we get anywhere, then maybe we can have budget for real engineers.
- gregjor 2y agoGood luck. Several of my clients have tried the same thing, using an LLM to build an MVP or something simple. In every case they’ve had to hire programmers and either start over or spend more money getting the AI-generated code to work than it would have cost to hire it out. Look out for security problems, because the bots certainly will. Please report your experience.