9 ms·

My experience (almost exclusively Claude), has just been so different that I don't know what to say. Some of the examples are the kinds of things I explicitly w

My experience (almost exclusively Claude), has just been so different that I don't know what to say. Some of the examples are the kinds of things I explicitly wouldn't expect LLMs to be particularly good at so I wouldn't use them for, and others, she says that it just doesn't work for her, and that experience is just so different than mine that I don't know how to respond. I think that there are two kinds of people who use AI: people who are looking for the ways in which AIs fail (of which there are still many) and people who are looking for the ways in which AIs succeed (of which there are also many). A lot of what I do is relatively simple one off scripting. Code that doesn't need to deal with edge cases, won't be widely deployed, and whose outputs are very quickly and easily verifiable. LLMs are almost perfect for this. It's generally faster than me looking up syntax/documentation, when it's wrong it's easy to tell and correct. Look for the ways that AI works, and it can be a powerful tool. Try and figure out where it still fails, and you will see nothing but hype and hot air. Not every use case is like this, but there are many. -edit- Also, when she says "none of my students has ever invented references that just don't exist"...all I can say is "press X to doubt"

CrossVR 2y ago
The point is that given the current valuations, being good at a bunch of narrow use cases is just not good enough. It needs to be able to replace humans in every role where the primary output is text or speech to meet expectations.
- MostlyStable 2y ago
  I don't think that "replacing humans in every role" is the line for "being bullish on AI models". I think they could stop development exactly where they are, and they would still make pretty dramatic improvements to productivity in a lot of places. For me at least, their value already exceeds the $20/month I'm paying, and I'm pretty sure that way more than covers inference costs.
  - TeMPOraL 2y ago
    > I think they could stop development exactly where they are, and they would still make pretty dramatic improvements to productivity in a lot of places. Yup. Not to mention, we don't even have time to figure out how to effectively work with one generation of models, before the next generation of models get released and rises the bar. If development stopped right now, I'd still expect LLMs to get better for years, as people slowly figure out how to use them well.
  - prawn 2y ago
    Completely agree. As is, Cursor and ChatGPT and even Bing Image Create (for free generation of shoddy ideas, styles, concepts, etc) are very useful to me. In fact, it would suit me if everything stalled at this point rather than improve to the point that everyone can catch up in how they use AI.
bluefirebrand 2y ago
> Look for the ways that AI works, and it can be a powerful tool. Try and figure out where it still fails, and you will see nothing but hype and hot air. Not every use case is like this, but there are many. The problem is that I feel I am constantly being bombarded by people bullish on AI saying "look how great this is" but when I try to do the exact same things they are doing, it doesn't work very well for me Of course I am skeptical of positive claims as a result.
- mattigames 2y ago
  Exactly, thanks to all the money involved in such hype the incentives will always skew towards over spamming naive optimism about it's features.
- MostlyStable 2y ago
  I don't know what you are doing or why it's failed. Maybe my primary use cases really are in the top whatever percentile for AI usefulness, but it doesn't feel like it. All I know is that frontier models have already been good enough for more than a year to increase my productivity by a fair bit.
  - nonchalantsui 2y ago
    Your use case is in fact in the top whatever percentile for AI usefulness. Short simple scripting that won't have to be relied on due to never being widely deployed. No large codebase it has to comb through, no need for thorough maintenance and update management, no need for efficient (and potentially rare) solutions. The only use case that would beat yours is the type of office worker that cannot write professional sounding emails but has to send them out regularly manually.
    - MostlyStable 2y ago
      I fully believe it's far better at the kind of coding/scripting that I do than the kind that real SWEs do. If for no other reason than the coding itself that I do is far far simpler and easier, so of course it's going to do better at it. However, I don't really believe that coding is the only use case. I think that there are a whole universe of other use cases that probably also get a lot of value from LLMs. I think that HN has a lot of people who are working on large software projects that are incredibly complex and have a huge numbers of interdependencies etc., and LLMs aren't quite to the point that they can very usefully contribute to that except around the edges. But I don't think that generalizing from that failure is very useful either. Most things humans do aren't that hard. There is a reason that SWE is one of the best paid jobs in the country.
sidewndr46 2y ago
every time someone brings up "Code that doesn't need to deal with edge cases" I like to point at that such code is not likely to be used for anything that matters
- SoftTalker 2y ago
  I write code like that all the time. It's used for very specific use cases, only by myself or something I've also written. It's not exposed to random end users or inputs.
- Panzer04 2y ago
  Is such code hard to write in the first place? Automating the easy 80% sounds useful, but in practice I'm not convinced that's all that helpful. Reading and putting together code you didn't write is hard enough to begin with.
  - Ferret7446 2y ago
    It's not hard, but it's time consuming.
    - skydhash 2y ago
      The things I'm wary of are pitfalls that are often only in the command/function docs. Kinda like rsync with how it handles terminating slashes at the end of the path. Which is why I always took a moment to read them.
- brulard 2y ago
  Oh, but it is. I can have code that does something nice to have, needs not to be 100% correct etc. For example, I want a background for my playful webpage. Maybe a WebGL shader. It might not be exactly what I asked for, but I can have it in few minutes up and running. Or some non-critical internal tools - like scraper for lunch menus from restaurants around office. Or simple parking spot sharing app. Or any kind of prototypes which in some companies are being created all the time. There are so many use cases that are forgiving regarding correctness and are much more sensitive to development effort.
Sohcahtoa82 2y ago
> LLMs are almost perfect for this. It's generally faster than me looking up syntax/documentation, when it's wrong it's easy to tell and correct. Exactly this. I once had a function that would generate several .csv reports. I wanted these reports to then be uploaded to s3://my_bucket/reports/{timestamp}/.csv I asked ChatGPT "Write a function that moves all .csv files in the current directory to and old_reports directory, calls a create_reports function, then uploads all the csv files in the current directory to s3://my_bucket/reports/{timestamp}/.csv with the timestamp in YYYY-MM-DD format"" And it created the code perfectly. I knew what the correct code would look like, I just couldn't be fucked to look up the exact calls to boto3, whether moving files was os.move or os.rename or something from shutil, and the exact way to format a datetime object. It created the code far faster that I would have. Like, I certainly wouldn't use it to write a whole app, or even a whole class, but individual blocks like this, it's great.
- simonw 2y ago
  I've had so many cases exactly like your example here. If you build up an intuition that knows that e.g. Claude 3.7 Sonnet can write code that uses boto3, and boto3 hasn't had any breaking changes that would affect S3 usage in the past ~24 months, you can jump straight into a prompt for this kind of task. It doesn't just save me a ton of time, it results in me building automations that I normally wouldn't have taken on at all because the time spent fiddling with os.move/boto3/etc wouldn't have been worthwhile compared to other things on my plate.
- taberiand 2y ago
  I have been saying this about llms for a while - if you know what you want, how to ask for it, and what the correct output will look like, LLMs are fantastic (at least Claude Sonnet is). And I mean that seriously, they are a highly effective tool for productive development for senior developers. I use it to produce whole classes, large sql queries, terraform scripts, etc etc. I then look over that output, iterate on it, adjust it to my needs. It's never exactly right at first, but that's fine - neither is code I write from scratch. It's still a massive time saver.
  - 2y ago
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
runjake 2y ago
More often than not, when I inquire deeper, I often find their prompting isn't very good at all. "Garbage in, garbage out" as the law says. Of course, it took a lot of trial and error for me to get to my current level of effectiveness with LLMs. It's probably our responsibility to teach these who are willing.
- cool_dude85 2y ago
  It seems hard to be bullish on LLMs as a generally useful tool if the solution to problems people have is "use trial and error to improve how you write your prompts, no, it's not obvious how to do so, yes, it depends heavily on the exact model you use."
  - simonw 2y ago
    You could say that about any power tool. A Mitre Saw is an amazing thing to have in a woodshop, but if you don't learn how to use it you're probably going to cut off a finger. The problem is that LLMs are power tools that are sold as being so easy to use that you don't need to invest any effort in learning them at all. That's extremely misleading.
    - zeitgeistcowboy 2y ago
      So is using an LLM to write SQL for you like using a mitre saw instead of a table saw? I guess the crux is that you still need to do work either way.
      - simonw 2y ago
        Using LLMs to write SQL is a fascinating case because there are so many traps you could fall into that aren't really the fault of the LLM. My favorite example: you ask the LLM for "most recent restaurant opened in California", give it a schema and it tries "select * from restaurants where state = 'California' order by open_date desc" - but that returns 0 results, because it turns out the state column uses two-letter state abbreviations like CA instead. There are tricks that can help here - I've tried sending the LLM an example row from each table, or you can set up a proper loop where the LLM gets to see the results and iterate on them - but it reflects the fact that interacting with databases can easily go wrong no matter how "smart" the model you are using is.
heraldgeezer 2y ago
>A lot of what I do is relatively simple one off scripting. Code that doesn't need to deal with edge cases, won't be widely deployed, and whose outputs are very quickly and easily verifiable. Yes somewhat. Its good for powershell/bash/cmd scripts and configs, but early models it would hallucinate PowerShell cmdlets especially.
bbor 2y ago
Look for the ways that AI works, and it can be a powerful tool. Try and figure out where it still fails, and you will see nothing but hype and hot air. Perfectly put, IMO. I know arguments from authority aren't primary, but I think this point highlights some important context: Dr. Hossenfelder has gained international renown by publishing clickbait-y YouTube videos that ostensibly debunk scientific and technological advances of all kinds. She's clearly educated and thoughtful (not to mention otherwise gainfully employed), but her whole public persona kinda relies on assuming the exclusively-critical standpoint you mention. I doubt she necessarily feels indebted to her large audience expecting this take (it's not new...), but that certainly does seem like a hard cognitive habit to break.
cjf101 2y ago
It's a weird circle with these things. If you _can't_ do the task you are using the LLM for, you probably shouldn't. But if you can do the task well enough to at least recognize likely-to-be-correct output, then you can get a lot done in less time than you would do it without their assistance. Is that worth the second order effects we're seeing? I'm not convinced, but it's definitely changed the way we do work.
asdev 2y ago
all fun and games until your AI generated script deletes the production database. I think that's the point, fault tolerance in academic and financial settings is too high for LLMs to be useful
satisfice 2y ago
I want to sit next to you and stop you every time you use your LLM and say, “Let me just carefully check this output.” I bet you wouldn’t like that. But when I want to do high quality work, I MUST take that time and carefully review and test. What I am seeing is fanboys who offer me examples of things working well that fail any close scrutiny— with the occasional example that comes out actually working well. I agree that for prototyping unimportant code LLMs do work well. I definitely get to unimportant point B from point A much more quickly when trying to write something unfamiliar.
- ryandrake 2y ago
  What's also scary is that we know LLMs do fail, but nobody (even the people who wrote the LLM) can tell you how often it will fail at any particular task. Not even an order of magnitude. Will it fail 0.2%, 2%, or 20% of the time? Nobody knows! A computer that will randomly produce an incorrect result to my calculation is useless to me because now I have to separately validate the correctness of every result. If I need to ask an LLM to explain to me some fact, how do I know if this time it's hallucinating? There is no "LLM just guessed" flag in the output. It might seem to people to be "miraculous" that it will summarize a random scientific paper down to 5 bullet points, but how do you know if it's output is correct? No LLM proponent seems to want to answer this question.
  - namaria 2y ago
    Better yet than the whole 'unknown, fluctuating and non deterministic rates of failure' is the whole 'agentic' shtick. People proposing to chain together these fluctuating plausibility engines should study probability theory a bit deeper to understand just what they are in for with these rube goldberg machines of text continuation.
  - TeMPOraL 2y ago
    > What's also scary is that we know LLMs do fail, but nobody (even the people who wrote the LLM) can tell you how often it will fail at any particular task. Not even an order of magnitude. Will it fail 0.2%, 2%, or 20% of the time? Benchmarks could track that too - I don't know if they do, but that information should actually be available and easy to get. When models are scored on e.g. "pass10", i.e. pass the challenge in under 10 attempts, and then the benchmark is rerun periodically, that literally produces the information you're asking for: how frequently a given model fails at particular task. > A computer that will randomly produce an incorrect result to my calculation is useless to me because now I have to separately validate the correctness of every result. For many tasks, validating a solution is order of magnitudes easier and cheaper than finding the solution in the first place. For those tasks, LLMs are very useful. > If I need to ask an LLM to explain to me some fact, how do I know if this time it's hallucinating? There is no "LLM just guessed" flag in the output. It might seem to people to be "miraculous" that it will summarize a random scientific paper down to 5 bullet points, but how do you know if it's output is correct? No LLM proponent seems to want to answer this question. How can you be sure whether a human you're asking isn't hallucinating/guessing the answer, or straight up bullshitting you? Apply the same approach to LLMs as you apply to navigating this problem with humans - for example, don't ask it to solve high-consequence problems in areas where you can't evaluate proposed solutions quickly.
glimshe 2y ago
I'm tired of people bashing LLMs. AI is so useful in my daily work that I can't understand where these people are coming from. Well, whatever... As you said, examples where I wouldn't expect LLMs to be good at from people who dismiss the scenarios where LLMs are great at. I don't want to convince anyone, to be honest - I just want to say they are incredibly useful for me and a huge time saver. If people don't want to use LLMs, it's fine for me as I'll have an edge over them in the market. Thanks for the cash, I guess.
- joquarky 2y ago
  I'm starting to resign myself to just enjoying the benefits and let those who can't evolve fall behind. I'm growing weary of trying to help people use these tools properly.
  - hbogert 2y ago
    I'm pretty weary of all the people telling me I'm "holding it wrong"
    - glimshe 2y ago
      I'll give you a simple and silly example which could give you additional ideas. LLMs can be great for checking whether people can understand something. One day I came up with a joke and wondered whether people would "get it". I told the joke to ChatGPT and asked it to explain it back to me. ChatGPT did a great job and nailed what's supposedly funny about the joke. I used it in an email so I have no idea whether anyone found it funny, but at least I know it wasn't too obscure. If an AI can understand a joke, there's a good chance people will understand it too. This might not be super useful but demonstrates that LLMs aren't only about generating text for copy-and-paste or retrieving information. It's "someone" you can bounce ideas with, ask opinions and that's how I use it most frequently.
jaredcwhite 2y ago
One thing I think is clear is society is now using a lot of words to describe things when the words being used are completely devoid of the necessary context. It's like calling a powder you've added to water "juice" and also freshly-squeezed fruit just picked perfectly ripe off a tree "juice". A word stretched like that becomes nearly devoid of meaning. "I write code all day with LLMs, it's amazing!" is in the exact same category. The code you (general you, I'm not picking on you in particular) write using LLMs, and the code I write apart from LLMs: they are not the same. They are categorically different artifacts.
relaxing 2y ago
> Also, when she says "none of my students has ever invented references that just don't exist"...all I can say is "press X to doubt" I’ve never seen it from my students. Why do you think this? It’s trivial to pick a real book/article. No student is generating fake material whole cloth and fake references to match. Even if they could, why would they risk it?
- vanviegen 2y ago
  Exactly. Lazy students just refer to vaguely related but existing material that they didn't read. Much better than llms! :-)
  - bell-cot 2y ago
    TBD whether that makes the effort to spot-check their references greater (does actually say what the student - explicitly or implicitly - claims it does?), or less (proving the non-existence of an obscure references is proving a negative)?
jasonjmcghee 2y ago
Many people have only tried the free version of ChatGPT, which is a completely different experience than the two most recent sonnet models.
- saaaaaam 2y ago
  And many people tried it a year ago and wrote it off as useless after trying to get results with basic one-shot prompting. Those people, likely, will never change their opinion. And that’s fine, because they won’t get the huge benefits that come from spending time learning how to use the tool properly.
- dpkirchner 2y ago
  Agreed. If one compares ChatGPT to, say, the Cline IDE plugin backed by Claude 3.7, they might well be blown away by how far behind ChatGPT seems. A lot of the difference has to do with prompting, for sure -- Cline helps there by generating prompts from your IDE and project context automatically. Every once in a while I send a query off to ChatGPT and I'm often disappointed and jam on the "this was hallucinated" feedback button (or whatever it is called). I have better luck with Claude's chat interface but nowhere near the quality of response that I get with Cline driving.
  - jasonjmcghee 2y ago
    You should try Claude Code. I was pretty impressed the first time I tried it. Go in with a specific task in mind
croes 2y ago
> invented references that just don't exist"...all I can say is "press X to doubt This doesn’t include lying and cheating which LLMs can’t. On the other hand AI is used to solve problems that are already solved. I just recently got an ad about a software for process modeling where one claim was you don’t need always to start from the ground up but can say the AI give me the customer order process to start from that point. That is basically what templates are for with much less energy consumption.
fumeux_fume 2y ago
I think you have an interesting point of view and I enjoy reading your comments, but it sounds a little absurd and circular to discount people's negativity about LLMs simply because it's their fault for using an LLM for something it's not good at. I don't believe in the strawman characterization of people giving LLMs incredibly complex problems and being unreasonably judgemental about the unsatisfactory results. I work with LLMs every day. Companies pay me good money to implement reliable solutions that use these models and it's a struggle. Currently I'm working with Claude 3.5 to analyze customer support chats. Just as many times as it makes impressive, nuanced judgments it fails to correctly make simple trivial judgements. Just as many times as it follows my prompt to a tee, it also forgets or ignores important parts of my prompt. So the problem for me is it's incredibly difficult to know when it'll succeed and when it'll fail for a given input. Am I unreasonable for having these frustrations? Am I unreasonable for doubting the efficacy of LLMs to address problems that many believe are already solved? Can you understand my frustration to see people characterize me as such because ChatGPT made a really cool image for them once?
rurp 2y ago
I think this points to much of the disagreement over LLMs. They can be great at one-off scripts and other similar tasks like prototypes. Some folks who do a lot of that kind of work find the tools genuinely amazing. Other software engineers do almost none of that and instead spend their coding time immersed in large messy code bases, with convoluted business logic. Looping an LLM into that kind of work can easily be net negative.
- danielmarkbruce 2y ago
  Maybe they are just lazy around tooling. Cursor with Claude works well for project sizes much larger than I expected but it takes a little set up. There is a chasm between engineers who use tools well and who do not.
  - rurp 2y ago
    I don't really agree with framing it as lazy. Adding more tools and steps to your workflow isn't free, and the cost/benefit of each tool will be different for everyone. I've lost count of how many times someone has evangelized a software tool to me, LLM or not. Once in a while they turn out to be useful and I incorporate them into my regular workflow, but far more often I don't. This could be for any number of reasons like it does not fit with my workflow well, or I already have a better way of doing whatever it does, or the tool adds more friction than it remove. I'm sure spending more time fiddling with the setup of LLM tools can yield better results, but that doesn't mean that it will be worth it for everyone. In my experience LLMs fail often enough at modestly complex problems that they are more hassle than benefit for a lot of the work I do. I'll still use them for simple tasks, like if I need some standard code in a language I'm not too familiar with. At the same time, I'm not at all surprised that others have a different experience and find them useful for larger projects they work on.
Jyaif 2y ago
> there are two kinds of people who use AI You hit the nail on this one. Around me I noticed that the bashing of LLMs come from the smart people that want others to know they are smart.
- joquarky 2y ago
  I've noticed there seems to be a gatekeeping archetype that operates as a hard cynic to nearly everything, so that when they finally judge something positively they get heaps of attention. It doesn't always correlate with narcissism, but it happens much more than chance.
hagbarth 2y ago
I don’t think your main experience with LLMs (coding clearly defined tasks) refute what Sabine is saying. The use cases are vastly different and the first is just _not_ world changing. It’s great, don’t get me wrong, but it won’t change the world.

CrossVR 2y ago

The point is that given the current valuations, being good at a bunch of narrow use cases is just not good enough. It needs to be able to replace humans in every role where the primary output is text or speech to meet expectations.

MostlyStable 2y ago
I don't think that "replacing humans in every role" is the line for "being bullish on AI models". I think they could stop development exactly where they are, and they would still make pretty dramatic improvements to productivity in a lot of places. For me at least, their value already exceeds the $20/month I'm paying, and I'm pretty sure that way more than covers inference costs.
- TeMPOraL 2y ago
  > I think they could stop development exactly where they are, and they would still make pretty dramatic improvements to productivity in a lot of places. Yup. Not to mention, we don't even have time to figure out how to effectively work with one generation of models, before the next generation of models get released and rises the bar. If development stopped right now, I'd still expect LLMs to get better for years, as people slowly figure out how to use them well.
- prawn 2y ago
  Completely agree. As is, Cursor and ChatGPT and even Bing Image Create (for free generation of shoddy ideas, styles, concepts, etc) are very useful to me. In fact, it would suit me if everything stalled at this point rather than improve to the point that everyone can catch up in how they use AI.

bluefirebrand 2y ago

> Look for the ways that AI works, and it can be a powerful tool. Try and figure out where it still fails, and you will see nothing but hype and hot air. Not every use case is like this, but there are many. The problem is that I feel I am constantly being bombarded by people bullish on AI saying "look how great this is" but when I try to do the exact same things they are doing, it doesn't work very well for me Of course I am skeptical of positive claims as a result.

mattigames 2y ago
Exactly, thanks to all the money involved in such hype the incentives will always skew towards over spamming naive optimism about it's features.
MostlyStable 2y ago
I don't know what you are doing or why it's failed. Maybe my primary use cases really are in the top whatever percentile for AI usefulness, but it doesn't feel like it. All I know is that frontier models have already been good enough for more than a year to increase my productivity by a fair bit.
- nonchalantsui 2y ago
  Your use case is in fact in the top whatever percentile for AI usefulness. Short simple scripting that won't have to be relied on due to never being widely deployed. No large codebase it has to comb through, no need for thorough maintenance and update management, no need for efficient (and potentially rare) solutions. The only use case that would beat yours is the type of office worker that cannot write professional sounding emails but has to send them out regularly manually.
  - MostlyStable 2y ago
    I fully believe it's far better at the kind of coding/scripting that I do than the kind that real SWEs do. If for no other reason than the coding itself that I do is far far simpler and easier, so of course it's going to do better at it. However, I don't really believe that coding is the only use case. I think that there are a whole universe of other use cases that probably also get a lot of value from LLMs. I think that HN has a lot of people who are working on large software projects that are incredibly complex and have a huge numbers of interdependencies etc., and LLMs aren't quite to the point that they can very usefully contribute to that except around the edges. But I don't think that generalizing from that failure is very useful either. Most things humans do aren't that hard. There is a reason that SWE is one of the best paid jobs in the country.

sidewndr46 2y ago

every time someone brings up "Code that doesn't need to deal with edge cases" I like to point at that such code is not likely to be used for anything that matters

SoftTalker 2y ago
I write code like that all the time. It's used for very specific use cases, only by myself or something I've also written. It's not exposed to random end users or inputs.
Panzer04 2y ago
Is such code hard to write in the first place? Automating the easy 80% sounds useful, but in practice I'm not convinced that's all that helpful. Reading and putting together code you didn't write is hard enough to begin with.
- Ferret7446 2y ago
  It's not hard, but it's time consuming.
  - skydhash 2y ago
    The things I'm wary of are pitfalls that are often only in the command/function docs. Kinda like rsync with how it handles terminating slashes at the end of the path. Which is why I always took a moment to read them.
brulard 2y ago
Oh, but it is. I can have code that does something nice to have, needs not to be 100% correct etc. For example, I want a background for my playful webpage. Maybe a WebGL shader. It might not be exactly what I asked for, but I can have it in few minutes up and running. Or some non-critical internal tools - like scraper for lunch menus from restaurants around office. Or simple parking spot sharing app. Or any kind of prototypes which in some companies are being created all the time. There are so many use cases that are forgiving regarding correctness and are much more sensitive to development effort.

Sohcahtoa82 2y ago

> LLMs are almost perfect for this. It's generally faster than me looking up syntax/documentation, when it's wrong it's easy to tell and correct. Exactly this. I once had a function that would generate several .csv reports. I wanted these reports to then be uploaded to s3://my_bucket/reports/{timestamp}/.csv I asked ChatGPT "Write a function that moves all .csv files in the current directory to and old_reports directory, calls a create_reports function, then uploads all the csv files in the current directory to s3://my_bucket/reports/{timestamp}/.csv with the timestamp in YYYY-MM-DD format"" And it created the code perfectly. I knew what the correct code would look like, I just couldn't be fucked to look up the exact calls to boto3, whether moving files was os.move or os.rename or something from shutil, and the exact way to format a datetime object. It created the code far faster that I would have. Like, I certainly wouldn't use it to write a whole app, or even a whole class, but individual blocks like this, it's great.

simonw 2y ago
I've had so many cases exactly like your example here. If you build up an intuition that knows that e.g. Claude 3.7 Sonnet can write code that uses boto3, and boto3 hasn't had any breaking changes that would affect S3 usage in the past ~24 months, you can jump straight into a prompt for this kind of task. It doesn't just save me a ton of time, it results in me building automations that I normally wouldn't have taken on at all because the time spent fiddling with os.move/boto3/etc wouldn't have been worthwhile compared to other things on my plate.
taberiand 2y ago
I have been saying this about llms for a while - if you know what you want, how to ask for it, and what the correct output will look like, LLMs are fantastic (at least Claude Sonnet is). And I mean that seriously, they are a highly effective tool for productive development for senior developers. I use it to produce whole classes, large sql queries, terraform scripts, etc etc. I then look over that output, iterate on it, adjust it to my needs. It's never exactly right at first, but that's fine - neither is code I write from scratch. It's still a massive time saver.
- 2y ago

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

runjake 2y ago

More often than not, when I inquire deeper, I often find their prompting isn't very good at all. "Garbage in, garbage out" as the law says. Of course, it took a lot of trial and error for me to get to my current level of effectiveness with LLMs. It's probably our responsibility to teach these who are willing.

cool_dude85 2y ago
It seems hard to be bullish on LLMs as a generally useful tool if the solution to problems people have is "use trial and error to improve how you write your prompts, no, it's not obvious how to do so, yes, it depends heavily on the exact model you use."
- simonw 2y ago
  You could say that about any power tool. A Mitre Saw is an amazing thing to have in a woodshop, but if you don't learn how to use it you're probably going to cut off a finger. The problem is that LLMs are power tools that are sold as being so easy to use that you don't need to invest any effort in learning them at all. That's extremely misleading.
  - zeitgeistcowboy 2y ago
    So is using an LLM to write SQL for you like using a mitre saw instead of a table saw? I guess the crux is that you still need to do work either way.
    - simonw 2y ago
      Using LLMs to write SQL is a fascinating case because there are so many traps you could fall into that aren't really the fault of the LLM. My favorite example: you ask the LLM for "most recent restaurant opened in California", give it a schema and it tries "select * from restaurants where state = 'California' order by open_date desc" - but that returns 0 results, because it turns out the state column uses two-letter state abbreviations like CA instead. There are tricks that can help here - I've tried sending the LLM an example row from each table, or you can set up a proper loop where the LLM gets to see the results and iterate on them - but it reflects the fact that interacting with databases can easily go wrong no matter how "smart" the model you are using is.

heraldgeezer 2y ago

>A lot of what I do is relatively simple one off scripting. Code that doesn't need to deal with edge cases, won't be widely deployed, and whose outputs are very quickly and easily verifiable. Yes somewhat. Its good for powershell/bash/cmd scripts and configs, but early models it would hallucinate PowerShell cmdlets especially.

bbor 2y ago

Look for the ways that AI works, and it can be a powerful tool. Try and figure out where it still fails, and you will see nothing but hype and hot air. Perfectly put, IMO. I know arguments from authority aren't primary, but I think this point highlights some important context: Dr. Hossenfelder has gained international renown by publishing clickbait-y YouTube videos that ostensibly debunk scientific and technological advances of all kinds. She's clearly educated and thoughtful (not to mention otherwise gainfully employed), but her whole public persona kinda relies on assuming the exclusively-critical standpoint you mention. I doubt she necessarily feels indebted to her large audience expecting this take (it's not new...), but that certainly does seem like a hard cognitive habit to break.

cjf101 2y ago

It's a weird circle with these things. If you _can't_ do the task you are using the LLM for, you probably shouldn't. But if you can do the task well enough to at least recognize likely-to-be-correct output, then you can get a lot done in less time than you would do it without their assistance. Is that worth the second order effects we're seeing? I'm not convinced, but it's definitely changed the way we do work.

asdev 2y ago

all fun and games until your AI generated script deletes the production database. I think that's the point, fault tolerance in academic and financial settings is too high for LLMs to be useful

satisfice 2y ago

I want to sit next to you and stop you every time you use your LLM and say, “Let me just carefully check this output.” I bet you wouldn’t like that. But when I want to do high quality work, I MUST take that time and carefully review and test. What I am seeing is fanboys who offer me examples of things working well that fail any close scrutiny— with the occasional example that comes out actually working well. I agree that for prototyping unimportant code LLMs do work well. I definitely get to unimportant point B from point A much more quickly when trying to write something unfamiliar.

ryandrake 2y ago
What's also scary is that we know LLMs do fail, but nobody (even the people who wrote the LLM) can tell you how often it will fail at any particular task. Not even an order of magnitude. Will it fail 0.2%, 2%, or 20% of the time? Nobody knows! A computer that will randomly produce an incorrect result to my calculation is useless to me because now I have to separately validate the correctness of every result. If I need to ask an LLM to explain to me some fact, how do I know if this time it's hallucinating? There is no "LLM just guessed" flag in the output. It might seem to people to be "miraculous" that it will summarize a random scientific paper down to 5 bullet points, but how do you know if it's output is correct? No LLM proponent seems to want to answer this question.
- namaria 2y ago
  Better yet than the whole 'unknown, fluctuating and non deterministic rates of failure' is the whole 'agentic' shtick. People proposing to chain together these fluctuating plausibility engines should study probability theory a bit deeper to understand just what they are in for with these rube goldberg machines of text continuation.
- TeMPOraL 2y ago
  > What's also scary is that we know LLMs do fail, but nobody (even the people who wrote the LLM) can tell you how often it will fail at any particular task. Not even an order of magnitude. Will it fail 0.2%, 2%, or 20% of the time? Benchmarks could track that too - I don't know if they do, but that information should actually be available and easy to get. When models are scored on e.g. "pass10", i.e. pass the challenge in under 10 attempts, and then the benchmark is rerun periodically, that literally produces the information you're asking for: how frequently a given model fails at particular task. > A computer that will randomly produce an incorrect result to my calculation is useless to me because now I have to separately validate the correctness of every result. For many tasks, validating a solution is order of magnitudes easier and cheaper than finding the solution in the first place. For those tasks, LLMs are very useful. > If I need to ask an LLM to explain to me some fact, how do I know if this time it's hallucinating? There is no "LLM just guessed" flag in the output. It might seem to people to be "miraculous" that it will summarize a random scientific paper down to 5 bullet points, but how do you know if it's output is correct? No LLM proponent seems to want to answer this question. How can you be sure whether a human you're asking isn't hallucinating/guessing the answer, or straight up bullshitting you? Apply the same approach to LLMs as you apply to navigating this problem with humans - for example, don't ask it to solve high-consequence problems in areas where you can't evaluate proposed solutions quickly.

glimshe 2y ago

I'm tired of people bashing LLMs. AI is so useful in my daily work that I can't understand where these people are coming from. Well, whatever... As you said, examples where I wouldn't expect LLMs to be good at from people who dismiss the scenarios where LLMs are great at. I don't want to convince anyone, to be honest - I just want to say they are incredibly useful for me and a huge time saver. If people don't want to use LLMs, it's fine for me as I'll have an edge over them in the market. Thanks for the cash, I guess.

joquarky 2y ago
I'm starting to resign myself to just enjoying the benefits and let those who can't evolve fall behind. I'm growing weary of trying to help people use these tools properly.
- hbogert 2y ago
  I'm pretty weary of all the people telling me I'm "holding it wrong"
  - glimshe 2y ago
    I'll give you a simple and silly example which could give you additional ideas. LLMs can be great for checking whether people can understand something. One day I came up with a joke and wondered whether people would "get it". I told the joke to ChatGPT and asked it to explain it back to me. ChatGPT did a great job and nailed what's supposedly funny about the joke. I used it in an email so I have no idea whether anyone found it funny, but at least I know it wasn't too obscure. If an AI can understand a joke, there's a good chance people will understand it too. This might not be super useful but demonstrates that LLMs aren't only about generating text for copy-and-paste or retrieving information. It's "someone" you can bounce ideas with, ask opinions and that's how I use it most frequently.

jaredcwhite 2y ago

One thing I think is clear is society is now using a lot of words to describe things when the words being used are completely devoid of the necessary context. It's like calling a powder you've added to water "juice" and also freshly-squeezed fruit just picked perfectly ripe off a tree "juice". A word stretched like that becomes nearly devoid of meaning. "I write code all day with LLMs, it's amazing!" is in the exact same category. The code you (general you, I'm not picking on you in particular) write using LLMs, and the code I write apart from LLMs: they are not the same. They are categorically different artifacts.

relaxing 2y ago

> Also, when she says "none of my students has ever invented references that just don't exist"...all I can say is "press X to doubt" I’ve never seen it from my students. Why do you think this? It’s trivial to pick a real book/article. No student is generating fake material whole cloth and fake references to match. Even if they could, why would they risk it?

vanviegen 2y ago
Exactly. Lazy students just refer to vaguely related but existing material that they didn't read. Much better than llms! :-)
- bell-cot 2y ago
  TBD whether that makes the effort to spot-check their references greater (does actually say what the student - explicitly or implicitly - claims it does?), or less (proving the non-existence of an obscure references is proving a negative)?

jasonjmcghee 2y ago

Many people have only tried the free version of ChatGPT, which is a completely different experience than the two most recent sonnet models.

saaaaaam 2y ago
And many people tried it a year ago and wrote it off as useless after trying to get results with basic one-shot prompting. Those people, likely, will never change their opinion. And that’s fine, because they won’t get the huge benefits that come from spending time learning how to use the tool properly.
dpkirchner 2y ago
Agreed. If one compares ChatGPT to, say, the Cline IDE plugin backed by Claude 3.7, they might well be blown away by how far behind ChatGPT seems. A lot of the difference has to do with prompting, for sure -- Cline helps there by generating prompts from your IDE and project context automatically. Every once in a while I send a query off to ChatGPT and I'm often disappointed and jam on the "this was hallucinated" feedback button (or whatever it is called). I have better luck with Claude's chat interface but nowhere near the quality of response that I get with Cline driving.
- jasonjmcghee 2y ago
  You should try Claude Code. I was pretty impressed the first time I tried it. Go in with a specific task in mind

croes 2y ago

> invented references that just don't exist"...all I can say is "press X to doubt This doesn’t include lying and cheating which LLMs can’t. On the other hand AI is used to solve problems that are already solved. I just recently got an ad about a software for process modeling where one claim was you don’t need always to start from the ground up but can say the AI give me the customer order process to start from that point. That is basically what templates are for with much less energy consumption.

fumeux_fume 2y ago

I think you have an interesting point of view and I enjoy reading your comments, but it sounds a little absurd and circular to discount people's negativity about LLMs simply because it's their fault for using an LLM for something it's not good at. I don't believe in the strawman characterization of people giving LLMs incredibly complex problems and being unreasonably judgemental about the unsatisfactory results. I work with LLMs every day. Companies pay me good money to implement reliable solutions that use these models and it's a struggle. Currently I'm working with Claude 3.5 to analyze customer support chats. Just as many times as it makes impressive, nuanced judgments it fails to correctly make simple trivial judgements. Just as many times as it follows my prompt to a tee, it also forgets or ignores important parts of my prompt. So the problem for me is it's incredibly difficult to know when it'll succeed and when it'll fail for a given input. Am I unreasonable for having these frustrations? Am I unreasonable for doubting the efficacy of LLMs to address problems that many believe are already solved? Can you understand my frustration to see people characterize me as such because ChatGPT made a really cool image for them once?

rurp 2y ago

I think this points to much of the disagreement over LLMs. They can be great at one-off scripts and other similar tasks like prototypes. Some folks who do a lot of that kind of work find the tools genuinely amazing. Other software engineers do almost none of that and instead spend their coding time immersed in large messy code bases, with convoluted business logic. Looping an LLM into that kind of work can easily be net negative.

danielmarkbruce 2y ago
Maybe they are just lazy around tooling. Cursor with Claude works well for project sizes much larger than I expected but it takes a little set up. There is a chasm between engineers who use tools well and who do not.
- rurp 2y ago
  I don't really agree with framing it as lazy. Adding more tools and steps to your workflow isn't free, and the cost/benefit of each tool will be different for everyone. I've lost count of how many times someone has evangelized a software tool to me, LLM or not. Once in a while they turn out to be useful and I incorporate them into my regular workflow, but far more often I don't. This could be for any number of reasons like it does not fit with my workflow well, or I already have a better way of doing whatever it does, or the tool adds more friction than it remove. I'm sure spending more time fiddling with the setup of LLM tools can yield better results, but that doesn't mean that it will be worth it for everyone. In my experience LLMs fail often enough at modestly complex problems that they are more hassle than benefit for a lot of the work I do. I'll still use them for simple tasks, like if I need some standard code in a language I'm not too familiar with. At the same time, I'm not at all surprised that others have a different experience and find them useful for larger projects they work on.

Jyaif 2y ago

> there are two kinds of people who use AI You hit the nail on this one. Around me I noticed that the bashing of LLMs come from the smart people that want others to know they are smart.

joquarky 2y ago
I've noticed there seems to be a gatekeeping archetype that operates as a hard cynic to nearly everything, so that when they finally judge something positively they get heaps of attention. It doesn't always correlate with narcissism, but it happens much more than chance.

hagbarth 2y ago

I don’t think your main experience with LLMs (coding clearly defined tasks) refute what Sabine is saying. The use cases are vastly different and the first is just _not_ world changing. It’s great, don’t get me wrong, but it won’t change the world.