5 ms·

I'm continually amazed to find takes like this. Can you explain how you don't find clear utility, at the personal level, from LLMs? I am being 100% genuine her

I'm continually amazed to find takes like this. Can you explain how you don't find clear utility, at the personal level, from LLMs? I am being 100% genuine here, I struggle to understand how the most useful things I've ever encountered are thought of this way and would like to better understand your perspective.

agent_turtle 1y ago
There was a study recently that showed how not only did devs overestimate the time saved using AI, but that they were net negative compared to the control group. Anyway, that about sums up my experience with AI. It may save some time here and there, but on net, you’re better off without it.
- keeda 1y ago
  That study gets mentioned all the time, somehow this one and many of the others it cites don't get much airtime: https://www.stlouisfed.org/on-the-economy/2024/sep/rapid-adoption-generative-ai https://www.stlouisfed.org/on-the-economy/2024/sep/rapid-ado... >This implies that each hour spent using genAI increases the worker’s productivity for that hour by 33%. This is similar in magnitude to the average productivity gain of 27% from several randomized experiments of genAI usage (Cui et al., 2024; Dell’Acqua et al., 2023; Noy and Zhang, 2023; Peng et al., 2023) Our estimated aggregate productivity gain from genAI (1.1%) exceeds the 0.7% estimate by Acemoglu (2024) based on a similar framework. To be clear, they are surmising that GenAI is already having a productivity gain.
  - agent_turtle 1y ago
    The article you gave is derived from a poll, not a study. As for the quote, I can’t find it in the article. Can you point me to it? I did click on one of the studies and it indicated productivity gains specifically on writing tasks. Which reminded me of this recent BBC article about a copywriter making bank fixing expensive mistakes caused by AI: https://www.bbc.com/news/articles/cyvm1dyp9v2o https://www.bbc.com/news/articles/cyvm1dyp9v2o
    - keeda 1y ago
      The quote is from the paper linked in the article: https://s3.amazonaws.com/real.stlouisfed.org/wp/2024/2024-027.pdf https://s3.amazonaws.com/real.stlouisfed.org/wp/2024/2024-02... It's actually based on the results of three surveys conducted by two different parties. While surveys are subject to all kinds of biases and the gains are self-reported, their findings of 25% - 33% producitivity do match the gains shown by at least 3 other randomized studies, one of which was specifically about programming. Those studies are worth looking at as well.
      - foolswisdom 1y ago
        It's worth noting that the METR paper that found decreased productivity also found that many of the developers thought the work was being sped up.
        keeda 1y ago
        Yes, self-reporting has biases and estimating tasks is still a fool's errand, which is why I noted that the estimates from these surveys matched the findings from other RTC studies. However, what doesn't get discussed enough about the METR study is that there was a spike in overall idle time as they waited for the AI to finish. I haven't run the numbers so I don't know how much of the increased completion time it accounts for, but if your cognitive load drops almost to 0, it will of course feel like your work is sped up, even though calendar time has increased. I wonder if that is the more important finding of that paper.
- rockemsockem 1y ago
  I'm not talking about time saving. AI seems to speed up my searching a bit since I can get results quicker without having to find the right query then find a site that actually answers my question, but that's minor, as nice as it is. I use AI in my personal life to learn about things I never would have without it because it makes the cost of finding any basic knowledge basically 0. Diet improvement ideas based on several quick questions about gut functioning, etc, recently learning how to gauge tsunami severity, and tons of other things. Once you have several fundamental terms and phrases for new topics it's easy to then validate the information with some quick googling too. How much have you actually tried using LLMs and did you just use normal chat or some big grand complex tool? I mostly just use chat and prefer to enter my code in artisanally.
  - fzeroracer 1y ago
    Why not just look up the information directly instead of asking a machine that you can never truly validate? If I need information, I can just keyword search wikipedia, then follow the chain there and then validate the sources along with outside information. An LLM would actually cost me time because I would still need to do all of the above anyways, making it a meaningless step. If you don't do the above then it's 'cheaper' but you're implicitly trusting the lying machine to not lie to you.
    - rockemsockem 1y ago
      See my previous statement > Once you have several fundamental terms and phrases for new topics it's easy to then validate the information with some quick googling too. You're practically saying that looking at an index in the back of a book is a meaningless step. It is significantly faster, so much so that I am able to ask it things that would have taken an indeterminate amount of time to research before, for just simple information, not deep understanding. Edit: Also I can truly validate literally any piece of information it gives me. Like I said previously, it makes it very easy to validate via Wikipedia or other places with the right terms, which I may not have known ahead of time.
      - 1y ago
- ares623 1y ago
  Is the "here and there" tasks that were previously so little value that they are always stuck in the backlog? i.e. the parts where it helps have very little value in the first place.
decimalenough 1y ago
I actually do get clear utility, with major caveats, namely that I only ask things where the answer is both well known and verifiable. I still do 10-20x regular Kagi searches for every LLM search, which seems about right in terms of the utility I'm personally getting out of this.
harimau777 1y ago
I think that there's a strong argument to be made that the negatives of having to wade through AI slop outweights the benefits that AI may provide. I also suspect that AI could contribute to enshittification of society; e.g. AI therapy being substituted for real therapy, AI products displacing industrial design, etc.
- rockemsockem 1y ago
  What is this AI slop that you're wading through and where is it? Spam emails are not any worse for being verbose, I don't recognize the sender, I send it straight to spam. The volume seems to be the same. You don't want an AI therapist? Go get a normal therapist. I have not heard of any AI product displacing industrial design, but if anything it'll make it easier to make/design stuff if/when it gets there. Like are these real things you are personally experiencing?
- fc417fc802 1y ago
  > e.g. AI therapy being substituted for real therapy, AI products displacing industrial design, etc. That depends on the quality of the end product and the willingness to invest the resources necessary to achieve a given quality of result. If average quality goes up in practice then I'd chalk that up as a net win. Low quality replacing high quality is categorically different than low quality filling a previously empty void. Therapy in particular is interesting not just because of average quality in practice (therapists are expensive experts) but also because of user behavior. There will be users who exhibit both increased and decreased willingness to share with an LLM versus a human. There's also a very strong privacy angle. Querying a local LLM affords me an expectation of privacy that I don't have when it comes to Google or even Wikipedia. (In the latter case I could maintain a local mirror but that's similar to maintaining a local LLM from a technical perspective making it a moot point.)
kazinator 1y ago
Gemini wasted my time today assuring me that if I want a git bundle that only has the top N commits, yet is cleanly clone-able, I can just make a --depth N clone of the original repo, and and do a git bundle create ... --all. Nope; cloning a bundle created from a depth-limited clone results in error messages about missing commit objects. So I tell the parrot that, and it comes back with: of course, it is well-known that it doesn't work, blah blah. (Then why wasn't it well known one prompt ago, when it was suggested as the definitive answer?) Obviously, I wasn't in the "the right mindset" today. This mindset is one of two things: - the mindset of a complete n00b asking a n00b question that it will nail every time, predicting it out of its training data richly replete with n00b material. - the mindset of a patient data miner, willing to expend all they keystrokes. needed to build up enough context to in effect create a query which zeroes in on the right nugget of information that made an appearance in the training data. It was interesting to go down this #2 rabbit hole when this stuff was new, which it isn't any more. Basically do most of the work, while it looks as if it solved the problem. I had the right mindset for AI, but most of it has worn off. If I don't get something useful in one query with at most one follow up, I quit. The only shills who continue to hype AI are either completely dishonest assholes, or genuine bros bearing weapons-grade confirmation bias. Let's try something else: Q: "What modes of C major are their own reflection?" A: "The Lydian and Phrygian modes are reflections of each other, as are the Ionian and Aeolian modes, and the Dorian and Mixolydian modes. The Locrian mode is its own reflection." Very nice sounding and grammatical, but gapingly wrong in every point. The only mode that is its own reflection is Dorian. Furthermore, Lydian and Phrygian are not mutual reflections. Phrygian reflected around is root is Ionian. The reflection of Lydian is Locrian; and of Aeolian, Mixolydian. I once loaded a NotebookLM with materials about George Russel's concept of the Lydian Chromatic, and Tonal Gravity. It made an incomprehensible mess of explaining the stuff, worse than the original sources. AI performs well on whatever is the focus of its purveyors. When they want to shake down entry-level coding, they beef it up on entry-level coding and let it loose, leaving it unable to tell Mixolydian from mixing console.
- lisbbb 1y ago
  Thank you! This is what I've been trying to tell people about LLMs. They don't hold up. They're like those Western movie set towns that look normal from the front, but when you walk around behind them, you see it is all just scaffolding with false fronts.
bluefirebrand 1y ago
> Can you explain how you don't find clear utility, at the personal level, from LLMs? Sure. They don't meaningfully improve anything in my life personally. They don't improve my search experience, they don't improve my work experience, they don't improve the quality of my online interactions, and I don't think they improve the quality of the society I live in either
- rockemsockem 1y ago
  Have you even tried using them though? Like in earnest? Or do you see yourself as a conscientious objector of sorts?
  - bluefirebrand 1y ago
    I have tried using them frequently. I've tried many things for years now, and while I am impressed I'm not impressed enough to replace any substantial part of my workflow with them At this point I am somewhat of a conscientious objector though Mostly from a stance of "these are not actually as good as people say and we will regret automating away jobs held by competent people in favor of these low quality automations"
  - ruszki 1y ago
    This whole topic makes me remember the argument for vi, and quick typing. I was always baffled because for the 25 years since I can code, typing was never that huge block of my time that it would matter. I have the same feeling with AI. It clearly cannot produce the quality of code, architecture, features which I require from myself. And I also want to understand what’s written, and not saying “it works, it’s fine <inserting dog with coffee image here>”, and not copy-pasting a terrible StackOverflow answer which doesn’t need half of the code in reality, and clearly nobody who answered sat down and tried to understand it. Of course, not everybody wants these, and I’ve seen several people who were fine with not understanding what they were doing. Even before AI. Now they are happy AI users. But it clears to me that it’s not beneficial salary, promotion, and political power wise. So what’s left is that it types faster… but that was never an issue. It can be better however. There was the first case just about a month ago when one of them could answer better to a problem than anything else which I knew or could find via Kagi/Google. But generally speaking it’s not there at all. Yet.
tikhonj 1y ago
Can't speak for anyone else, but for me, AI/LLMs have been firmly in the "nice but forgettable" camp. Like, sometimes it's marginally more convenient to use an LLM than to do a proper web search or to figure out how to write some code—but that's a small time saving at best, it's less of a net impact than Stack Overflow was. I'm already a pretty fast writer and programmer without LLMs. If I hadn't already learned how to write and program quickly, perhaps I would get more use out of LLMs. But the LLMs would be saving me the effort of learning which, ultimately, is an O(1) cost for O(n) benefit. Not super compelling. And what would I even do with a larger volume of text output? I already write more than most folks are willing to read... So, sure, it's not strictly zero utility, but it's far less utility than a long series of other things. On the other hand, trains are fucking amazing. I don't drive, and having real passenger rail is a big chunk of why I want to move to Europe one day. Being able to get places without needing to learn and then operate a big, dangerous machine—one that is statistically much more dangerous for folks with ADHD like me—makes a massive difference in my day-to-day life. Having a language model... doesn't. And that's living in the Bay Area where the trains aren't great. Bart, Caltrain and Amtrak disappearing would have an orders of magnitude larger effect on my life than if LLMs stopped working. And I'm totally ignoring the indirect but substantial value I get out of freight rail. Sure, ships and trucks could probably get us there, but the net increase in costs and pollution should not be underestimated.
- knowitnone2 1y ago
  No matter how good or fast you are, you will never beat the LLM. What you're saying is akin to "your math is faster than a calculator" and I'm willing to bet it's not. LLMs are not perfect and will require intervention and fixing but if it can get you 90% there, that's pretty good. In the coming years, you'll soon find your peers are performing much faster than you (assuming you program for a living) and you will have no choice but you do you.
  - tikhonj 1y ago
    Fun story: when I interned at Jane Street, they gave out worksheets full of put-call parity calculations to do in your head because, when you're trading, being able to do that sort of calculation at a glance is far faster and more fluid than using a calculator or computer. So for some professionals, mental math really is faster. Make of that what you will.
roncesvalles 1y ago
As a dev, I find that the personal utility of LLMs is still very limited. Analyze it this way: Are LLMs enabling something that was impossible before? My answer would be No. Whatever I'm asking of the LLM, I'd have figured it out from googling and RTFMing anyway, and probably have done a better job at it. And guess what, after letting the LLM do it, I probably still need to google and RTFM anyway. You might say "it's enabling the impossible because you can now do things in less time", to which I would say, I don't really think you can do it in less time. It's more like cruise control where it takes the same time to get to your destination but you just need to expend less mental effort. Other elephants in the room: - where is the missing explosion of (non-AI) software startups that should've been enabled by LLM dev efficiency improvements? - why is adoption among big tech SWEs near zero despite intense push from management? You'd think, of all people, you wouldn't have to ask them twice. The emperor has no clothes.
- knowitnone2 1y ago
  do cars enable something that was impossible before? bikes? shoes? clothing? Your answer would be No.
  - roncesvalles 1y ago
    If your implication is that LLM-assisted coding to non-LLM-assisted coding is like motorcar to horse buggy, that is just not the case.
    - ryao 1y ago
      I think he was referring to the ability to go from A to B within a certain amount of time. There is a threshold at which it is possible for a car, yet impossible for a horse and buggy. That said, I recently saw a colleague use a LLM to make a non-trivial UI for electron in HTML/CSS/JS, despite knowing nothing about any of those technologies, in less time than it would have taken me to do it. We had been in the process of devising a set of requirements, he fed his version of them into the LLM, did some back and forth with the LLM, showed me the result, got feedback, fed my feedback back into the LLM and got a good solution. I had suggested that he make a mockup (a drawing in kolourpaint for example) for further discussion, but he had surprised me by using a LLM to make a functional prototype in place of the mockup. It was a huge time saver.
shusaku 1y ago
The concerning thing is that AI contrarianism is being left wing coded. Imagine you’re fighting a war and one side decides “guns are overhyped, let’s stick with swords”. While there is a lot of hype about AI, even the pessimistic take has to admit it’s a game changing tech. If it isn’t doing anything useful for you, that’s because you need to get off your butt and start building tools on top of it. Especially people on the left need to realize how important their vision is to the future if AI. Right now you can see the current US admin having zero concern for AI safety or carbon use. If you keep your head in the dirt saying “bubble!” that’s no problem. But if this is here to stay then you need to get involved.
- rockemsockem 1y ago
  That's a good point and even worse we'll eventually end up with yet another issue where both left and right offer terrible options and there's no nuanced middle ground :/
mopsi 1y ago
Last night, I asked a LLM to produce an /etc/fstab entry for connecting to a network share with specific options. I was too lazy to look up the options from the manual. It gave me the options separated by semicolons, which is invalid because the config file requires commas as separators. I honestly don't see technology that stumbles over trivial problems like these as something that will replace my job, or any job that is not already automatable within ten thousand lines of Python, anytime soon. The gap between hype and actual capabilities is insane. The more I've tried to apply LLMs to real problems, the more disillusioned I've become. There is nothing, absolutely nothing, no matter how small the task, that I can trust LLMs to do correctly.
- ryao 1y ago
  Which one? There are huge variations between LLMs. Was this a frontier thinking model with tool use? Did you ask it review online references before presenting an answer?
danlitt 1y ago
Like other commenters here, when I try to use them to help with my work, they don't. It's that simple. I have tried AI coding assistants, and they just guess incorrectly. If I know the answer, they generally give me the same answer. If I don't know the answer, they give me gibberish that ends up wasting time. I would love to look over the shoulder of an AI booster who had a really good interaction, because it's hard for me to believe they didn't just already know what they were looking for.
ehnto 1y ago
Much work is not "text in, text out", and much work is not digital at all. But even in the digital world, inserting an LLM into a workflow is just not always that useful. In fact much automation, code or otherwise, benefits from or even requires explicit, concise rules. It is far quicker for me to already know, and write, an SQL statement, than it is to explain what I need to an LLM. It is also quite difficult to get LLMs into a lot of processes, and I think big enterprises are going to really struggle with this. I would absolutely love AI to manage some Windows servers that are in my care, but they are three VMs deep in a remote desktop stack that gets me into a DMZ/intranet. There's no interface, and how would an LLM help anyway. What I need is concise, discreet automations. Not a chat bot interface to try and instruct every day. To be clear I do try to use AI most days, I have Claude and I am a software developer so ideally it could be very helpful, but I have far less use for it than say people in the strategy or marketing departments for example. I do a lot of things, but not really all that much writing.
satyrun 1y ago
IMO I think it is a combination of being a really great programmer already and then either not all that intellectually curious or so well read and so intellectually curious that LLMs are a step down from being a voracious reader of books and papers. For me, LLMs are also the most useful thing ever but I was a C student in all my classes. My programming is a joke. I have always been intellectually curious but I am quite lazy. I have always had tons of ideas to explore though and LLMS let me explore these ideas that I either wouldn't be able to otherwise or would be too lazy to bother.
- rockemsockem 1y ago
  I don't totally follow what you're saying. Are you saying that LLMs are most useful if you're not intellectually curious, and therefore most interested in immediate answers, but also that they're very useful if you're a really great programmer for an unstated reason?
deleted 1y ago
[deleted]
m0llusk 1y ago
The worst thing in the information space is bullshit. It's worse than lies because it's slicker. LLMs generate extremely dangerous bullshit that can be difficult to correct. Advice given may apply to old versions, outdated legal precedent, or radical opinions. Fully understanding nuance or mistakes can take a lot of effort and still be off.

agent_turtle 1y ago

There was a study recently that showed how not only did devs overestimate the time saved using AI, but that they were net negative compared to the control group. Anyway, that about sums up my experience with AI. It may save some time here and there, but on net, you’re better off without it.

keeda 1y ago
That study gets mentioned all the time, somehow this one and many of the others it cites don't get much airtime: https://www.stlouisfed.org/on-the-economy/2024/sep/rapid-adoption-generative-ai https://www.stlouisfed.org/on-the-economy/2024/sep/rapid-ado... >This implies that each hour spent using genAI increases the worker’s productivity for that hour by 33%. This is similar in magnitude to the average productivity gain of 27% from several randomized experiments of genAI usage (Cui et al., 2024; Dell’Acqua et al., 2023; Noy and Zhang, 2023; Peng et al., 2023) Our estimated aggregate productivity gain from genAI (1.1%) exceeds the 0.7% estimate by Acemoglu (2024) based on a similar framework. To be clear, they are surmising that GenAI is already having a productivity gain.
- agent_turtle 1y ago
  The article you gave is derived from a poll, not a study. As for the quote, I can’t find it in the article. Can you point me to it? I did click on one of the studies and it indicated productivity gains specifically on writing tasks. Which reminded me of this recent BBC article about a copywriter making bank fixing expensive mistakes caused by AI: https://www.bbc.com/news/articles/cyvm1dyp9v2o https://www.bbc.com/news/articles/cyvm1dyp9v2o
  - keeda 1y ago
    The quote is from the paper linked in the article: https://s3.amazonaws.com/real.stlouisfed.org/wp/2024/2024-027.pdf https://s3.amazonaws.com/real.stlouisfed.org/wp/2024/2024-02... It's actually based on the results of three surveys conducted by two different parties. While surveys are subject to all kinds of biases and the gains are self-reported, their findings of 25% - 33% producitivity do match the gains shown by at least 3 other randomized studies, one of which was specifically about programming. Those studies are worth looking at as well.
    - foolswisdom 1y ago
      It's worth noting that the METR paper that found decreased productivity also found that many of the developers thought the work was being sped up.
      - keeda 1y ago
        Yes, self-reporting has biases and estimating tasks is still a fool's errand, which is why I noted that the estimates from these surveys matched the findings from other RTC studies. However, what doesn't get discussed enough about the METR study is that there was a spike in overall idle time as they waited for the AI to finish. I haven't run the numbers so I don't know how much of the increased completion time it accounts for, but if your cognitive load drops almost to 0, it will of course feel like your work is sped up, even though calendar time has increased. I wonder if that is the more important finding of that paper.
rockemsockem 1y ago
I'm not talking about time saving. AI seems to speed up my searching a bit since I can get results quicker without having to find the right query then find a site that actually answers my question, but that's minor, as nice as it is. I use AI in my personal life to learn about things I never would have without it because it makes the cost of finding any basic knowledge basically 0. Diet improvement ideas based on several quick questions about gut functioning, etc, recently learning how to gauge tsunami severity, and tons of other things. Once you have several fundamental terms and phrases for new topics it's easy to then validate the information with some quick googling too. How much have you actually tried using LLMs and did you just use normal chat or some big grand complex tool? I mostly just use chat and prefer to enter my code in artisanally.
- fzeroracer 1y ago
  Why not just look up the information directly instead of asking a machine that you can never truly validate? If I need information, I can just keyword search wikipedia, then follow the chain there and then validate the sources along with outside information. An LLM would actually cost me time because I would still need to do all of the above anyways, making it a meaningless step. If you don't do the above then it's 'cheaper' but you're implicitly trusting the lying machine to not lie to you.
  - rockemsockem 1y ago
    See my previous statement > Once you have several fundamental terms and phrases for new topics it's easy to then validate the information with some quick googling too. You're practically saying that looking at an index in the back of a book is a meaningless step. It is significantly faster, so much so that I am able to ask it things that would have taken an indeterminate amount of time to research before, for just simple information, not deep understanding. Edit: Also I can truly validate literally any piece of information it gives me. Like I said previously, it makes it very easy to validate via Wikipedia or other places with the right terms, which I may not have known ahead of time.
    - 1y ago
ares623 1y ago
Is the "here and there" tasks that were previously so little value that they are always stuck in the backlog? i.e. the parts where it helps have very little value in the first place.

decimalenough 1y ago

I actually do get clear utility, with major caveats, namely that I only ask things where the answer is both well known and verifiable. I still do 10-20x regular Kagi searches for every LLM search, which seems about right in terms of the utility I'm personally getting out of this.

harimau777 1y ago

I think that there's a strong argument to be made that the negatives of having to wade through AI slop outweights the benefits that AI may provide. I also suspect that AI could contribute to enshittification of society; e.g. AI therapy being substituted for real therapy, AI products displacing industrial design, etc.

rockemsockem 1y ago
What is this AI slop that you're wading through and where is it? Spam emails are not any worse for being verbose, I don't recognize the sender, I send it straight to spam. The volume seems to be the same. You don't want an AI therapist? Go get a normal therapist. I have not heard of any AI product displacing industrial design, but if anything it'll make it easier to make/design stuff if/when it gets there. Like are these real things you are personally experiencing?
fc417fc802 1y ago
> e.g. AI therapy being substituted for real therapy, AI products displacing industrial design, etc. That depends on the quality of the end product and the willingness to invest the resources necessary to achieve a given quality of result. If average quality goes up in practice then I'd chalk that up as a net win. Low quality replacing high quality is categorically different than low quality filling a previously empty void. Therapy in particular is interesting not just because of average quality in practice (therapists are expensive experts) but also because of user behavior. There will be users who exhibit both increased and decreased willingness to share with an LLM versus a human. There's also a very strong privacy angle. Querying a local LLM affords me an expectation of privacy that I don't have when it comes to Google or even Wikipedia. (In the latter case I could maintain a local mirror but that's similar to maintaining a local LLM from a technical perspective making it a moot point.)

kazinator 1y ago

Gemini wasted my time today assuring me that if I want a git bundle that only has the top N commits, yet is cleanly clone-able, I can just make a --depth N clone of the original repo, and and do a git bundle create ... --all. Nope; cloning a bundle created from a depth-limited clone results in error messages about missing commit objects. So I tell the parrot that, and it comes back with: of course, it is well-known that it doesn't work, blah blah. (Then why wasn't it well known one prompt ago, when it was suggested as the definitive answer?) Obviously, I wasn't in the "the right mindset" today. This mindset is one of two things: - the mindset of a complete n00b asking a n00b question that it will nail every time, predicting it out of its training data richly replete with n00b material. - the mindset of a patient data miner, willing to expend all they keystrokes. needed to build up enough context to in effect create a query which zeroes in on the right nugget of information that made an appearance in the training data. It was interesting to go down this #2 rabbit hole when this stuff was new, which it isn't any more. Basically do most of the work, while it looks as if it solved the problem. I had the right mindset for AI, but most of it has worn off. If I don't get something useful in one query with at most one follow up, I quit. The only shills who continue to hype AI are either completely dishonest assholes, or genuine bros bearing weapons-grade confirmation bias. Let's try something else: Q: "What modes of C major are their own reflection?" A: "The Lydian and Phrygian modes are reflections of each other, as are the Ionian and Aeolian modes, and the Dorian and Mixolydian modes. The Locrian mode is its own reflection." Very nice sounding and grammatical, but gapingly wrong in every point. The only mode that is its own reflection is Dorian. Furthermore, Lydian and Phrygian are not mutual reflections. Phrygian reflected around is root is Ionian. The reflection of Lydian is Locrian; and of Aeolian, Mixolydian. I once loaded a NotebookLM with materials about George Russel's concept of the Lydian Chromatic, and Tonal Gravity. It made an incomprehensible mess of explaining the stuff, worse than the original sources. AI performs well on whatever is the focus of its purveyors. When they want to shake down entry-level coding, they beef it up on entry-level coding and let it loose, leaving it unable to tell Mixolydian from mixing console.

lisbbb 1y ago
Thank you! This is what I've been trying to tell people about LLMs. They don't hold up. They're like those Western movie set towns that look normal from the front, but when you walk around behind them, you see it is all just scaffolding with false fronts.

bluefirebrand 1y ago

> Can you explain how you don't find clear utility, at the personal level, from LLMs? Sure. They don't meaningfully improve anything in my life personally. They don't improve my search experience, they don't improve my work experience, they don't improve the quality of my online interactions, and I don't think they improve the quality of the society I live in either

rockemsockem 1y ago
Have you even tried using them though? Like in earnest? Or do you see yourself as a conscientious objector of sorts?
- bluefirebrand 1y ago
  I have tried using them frequently. I've tried many things for years now, and while I am impressed I'm not impressed enough to replace any substantial part of my workflow with them At this point I am somewhat of a conscientious objector though Mostly from a stance of "these are not actually as good as people say and we will regret automating away jobs held by competent people in favor of these low quality automations"
- ruszki 1y ago
  This whole topic makes me remember the argument for vi, and quick typing. I was always baffled because for the 25 years since I can code, typing was never that huge block of my time that it would matter. I have the same feeling with AI. It clearly cannot produce the quality of code, architecture, features which I require from myself. And I also want to understand what’s written, and not saying “it works, it’s fine <inserting dog with coffee image here>”, and not copy-pasting a terrible StackOverflow answer which doesn’t need half of the code in reality, and clearly nobody who answered sat down and tried to understand it. Of course, not everybody wants these, and I’ve seen several people who were fine with not understanding what they were doing. Even before AI. Now they are happy AI users. But it clears to me that it’s not beneficial salary, promotion, and political power wise. So what’s left is that it types faster… but that was never an issue. It can be better however. There was the first case just about a month ago when one of them could answer better to a problem than anything else which I knew or could find via Kagi/Google. But generally speaking it’s not there at all. Yet.

tikhonj 1y ago

Can't speak for anyone else, but for me, AI/LLMs have been firmly in the "nice but forgettable" camp. Like, sometimes it's marginally more convenient to use an LLM than to do a proper web search or to figure out how to write some code—but that's a small time saving at best, it's less of a net impact than Stack Overflow was. I'm already a pretty fast writer and programmer without LLMs. If I hadn't already learned how to write and program quickly, perhaps I would get more use out of LLMs. But the LLMs would be saving me the effort of learning which, ultimately, is an O(1) cost for O(n) benefit. Not super compelling. And what would I even do with a larger volume of text output? I already write more than most folks are willing to read... So, sure, it's not strictly zero utility, but it's far less utility than a long series of other things. On the other hand, trains are fucking amazing. I don't drive, and having real passenger rail is a big chunk of why I want to move to Europe one day. Being able to get places without needing to learn and then operate a big, dangerous machine—one that is statistically much more dangerous for folks with ADHD like me—makes a massive difference in my day-to-day life. Having a language model... doesn't. And that's living in the Bay Area where the trains aren't great. Bart, Caltrain and Amtrak disappearing would have an orders of magnitude larger effect on my life than if LLMs stopped working. And I'm totally ignoring the indirect but substantial value I get out of freight rail. Sure, ships and trucks could probably get us there, but the net increase in costs and pollution should not be underestimated.

knowitnone2 1y ago
No matter how good or fast you are, you will never beat the LLM. What you're saying is akin to "your math is faster than a calculator" and I'm willing to bet it's not. LLMs are not perfect and will require intervention and fixing but if it can get you 90% there, that's pretty good. In the coming years, you'll soon find your peers are performing much faster than you (assuming you program for a living) and you will have no choice but you do you.
- tikhonj 1y ago
  Fun story: when I interned at Jane Street, they gave out worksheets full of put-call parity calculations to do in your head because, when you're trading, being able to do that sort of calculation at a glance is far faster and more fluid than using a calculator or computer. So for some professionals, mental math really is faster. Make of that what you will.

roncesvalles 1y ago

As a dev, I find that the personal utility of LLMs is still very limited. Analyze it this way: Are LLMs enabling something that was impossible before? My answer would be No. Whatever I'm asking of the LLM, I'd have figured it out from googling and RTFMing anyway, and probably have done a better job at it. And guess what, after letting the LLM do it, I probably still need to google and RTFM anyway. You might say "it's enabling the impossible because you can now do things in less time", to which I would say, I don't really think you can do it in less time. It's more like cruise control where it takes the same time to get to your destination but you just need to expend less mental effort. Other elephants in the room: - where is the missing explosion of (non-AI) software startups that should've been enabled by LLM dev efficiency improvements? - why is adoption among big tech SWEs near zero despite intense push from management? You'd think, of all people, you wouldn't have to ask them twice. The emperor has no clothes.

knowitnone2 1y ago
do cars enable something that was impossible before? bikes? shoes? clothing? Your answer would be No.
- roncesvalles 1y ago
  If your implication is that LLM-assisted coding to non-LLM-assisted coding is like motorcar to horse buggy, that is just not the case.
  - ryao 1y ago
    I think he was referring to the ability to go from A to B within a certain amount of time. There is a threshold at which it is possible for a car, yet impossible for a horse and buggy. That said, I recently saw a colleague use a LLM to make a non-trivial UI for electron in HTML/CSS/JS, despite knowing nothing about any of those technologies, in less time than it would have taken me to do it. We had been in the process of devising a set of requirements, he fed his version of them into the LLM, did some back and forth with the LLM, showed me the result, got feedback, fed my feedback back into the LLM and got a good solution. I had suggested that he make a mockup (a drawing in kolourpaint for example) for further discussion, but he had surprised me by using a LLM to make a functional prototype in place of the mockup. It was a huge time saver.

shusaku 1y ago

The concerning thing is that AI contrarianism is being left wing coded. Imagine you’re fighting a war and one side decides “guns are overhyped, let’s stick with swords”. While there is a lot of hype about AI, even the pessimistic take has to admit it’s a game changing tech. If it isn’t doing anything useful for you, that’s because you need to get off your butt and start building tools on top of it. Especially people on the left need to realize how important their vision is to the future if AI. Right now you can see the current US admin having zero concern for AI safety or carbon use. If you keep your head in the dirt saying “bubble!” that’s no problem. But if this is here to stay then you need to get involved.

rockemsockem 1y ago
That's a good point and even worse we'll eventually end up with yet another issue where both left and right offer terrible options and there's no nuanced middle ground :/

mopsi 1y ago

Last night, I asked a LLM to produce an /etc/fstab entry for connecting to a network share with specific options. I was too lazy to look up the options from the manual. It gave me the options separated by semicolons, which is invalid because the config file requires commas as separators. I honestly don't see technology that stumbles over trivial problems like these as something that will replace my job, or any job that is not already automatable within ten thousand lines of Python, anytime soon. The gap between hype and actual capabilities is insane. The more I've tried to apply LLMs to real problems, the more disillusioned I've become. There is nothing, absolutely nothing, no matter how small the task, that I can trust LLMs to do correctly.

ryao 1y ago
Which one? There are huge variations between LLMs. Was this a frontier thinking model with tool use? Did you ask it review online references before presenting an answer?

danlitt 1y ago

Like other commenters here, when I try to use them to help with my work, they don't. It's that simple. I have tried AI coding assistants, and they just guess incorrectly. If I know the answer, they generally give me the same answer. If I don't know the answer, they give me gibberish that ends up wasting time. I would love to look over the shoulder of an AI booster who had a really good interaction, because it's hard for me to believe they didn't just already know what they were looking for.

ehnto 1y ago

Much work is not "text in, text out", and much work is not digital at all. But even in the digital world, inserting an LLM into a workflow is just not always that useful. In fact much automation, code or otherwise, benefits from or even requires explicit, concise rules. It is far quicker for me to already know, and write, an SQL statement, than it is to explain what I need to an LLM. It is also quite difficult to get LLMs into a lot of processes, and I think big enterprises are going to really struggle with this. I would absolutely love AI to manage some Windows servers that are in my care, but they are three VMs deep in a remote desktop stack that gets me into a DMZ/intranet. There's no interface, and how would an LLM help anyway. What I need is concise, discreet automations. Not a chat bot interface to try and instruct every day. To be clear I do try to use AI most days, I have Claude and I am a software developer so ideally it could be very helpful, but I have far less use for it than say people in the strategy or marketing departments for example. I do a lot of things, but not really all that much writing.

satyrun 1y ago

IMO I think it is a combination of being a really great programmer already and then either not all that intellectually curious or so well read and so intellectually curious that LLMs are a step down from being a voracious reader of books and papers. For me, LLMs are also the most useful thing ever but I was a C student in all my classes. My programming is a joke. I have always been intellectually curious but I am quite lazy. I have always had tons of ideas to explore though and LLMS let me explore these ideas that I either wouldn't be able to otherwise or would be too lazy to bother.

rockemsockem 1y ago
I don't totally follow what you're saying. Are you saying that LLMs are most useful if you're not intellectually curious, and therefore most interested in immediate answers, but also that they're very useful if you're a really great programmer for an unstated reason?

deleted 1y ago

[deleted]

m0llusk 1y ago

The worst thing in the information space is bullshit. It's worse than lies because it's slicker. LLMs generate extremely dangerous bullshit that can be difficult to correct. Advice given may apply to old versions, outdated legal precedent, or radical opinions. Fully understanding nuance or mistakes can take a lot of effort and still be off.