7 ms·

I left this comment originally on lobsters: While I understand the fear, I don’t really share it. And if I where to go to the root of it, I think I really most

I left this comment originally on lobsters: While I understand the fear, I don’t really share it. And if I where to go to the root of it, I think I really most take issue with this: > My experiences of genAI are all extremely bad, but that is barely even anecdata. Their experiences are neutral-to-positive. Little scientific data exists. How to resolve this? My experience is astonishingly positive. I would not have imagined how much of a help these tools have become. Deep research and similar tools alone have helped me navigate complex legal matters recently for my incorporation, they have uncovered really useful information that I just would not have found that quickly. First cursor, now Claude Code have really changed how I work. Especially since for the last month or so, I feel myself more and more in the position where I can do things while the machine works. It’s truly liberating and it gives me a lot of joy. So it’s not “neutral-to-positive” to me, it’s exhilarating. And that extends particularly to this part: > Despite this plethora of negative experiences, executives are aggressively mandating the use of AI. It looks like without such mandates, most people will not bother to use such tools, so the executives will need muscular policies to enforce its use. When I was at Sentry the adoption of AI happened by ICs before the company even put money behind it. In fact my memory is that if anything only at the point where an exceeding number of AI invoices showed up from IC expenses did we realize how widespread adoption has been. This was grounds up. For my non techy friends it’s even tricker because some of them work in companies that outright try to prevent the adoption of AI, but they are paying for it themselves to help them with the work. Some of them pay for the expensive ChatGPT package even! None of this should be disregarded, but it stands in a crass contrast to what this post says. That said, I understand where Glyph comes from and I appreciate that point. There is probably a lot of truth in the tail risks of all of that, and I share these. It just does not at all take away from my enjoyment and optimism at all.

oulipo 1y ago
Basically his article says: check the documented issue with the climate, the education, the destruction of trust in society, and your answer is "yes but it makes my nice cuddly programming job better I like it"... don't you understand the issue?
- the_mitsuhiko 1y ago
  I responded to quite specific quotes which I took issues with, and I intentionally did not talk about the rest of it. I did not make a statement about the rest because it's too complex of a topic to which I do not have enough data to contribute meaningfully to the conversation.
  - haswell 1y ago
    > I did not make a statement about the rest because it's too complex of a topic to which I do not have enough data to contribute meaningfully to the conversation. > There is probably a lot of truth in the tail risks of all of that, and I share these. It just does not at all take away from my enjoyment and optimism at all. On the one hand, I get not expanding on things you don't feel you can contribute to meaningfully and respect that. On the other, isn't this a problem? While I agree with you and cannot fully understand the author's conclusion that it's all bad (I've also found value), I wrestle with the downsides on an ongoing basis and have found myself shying away from using these tools as a result, even though I know they are quite capable of some things. I think that if I only responded "Well, I personally find the tools valuable", that this is only a part of the picture and is a net negative to the overall conversation because the "real" conversation that needs to be had IMO is about those harms, not haggling over whether or not the thing can actually do stuff. There's quite a lot of pro-AI content and opinion floating around right now that amounts to: "Yeah, this thing is dangerous, threatens climate goals, is accelerating an education crisis, only exists because of mass theft, creates a massive category of new problems we don't yet know how to solve...but I'm getting value out of it". And the thing that keeps coming to mind is "Don't Look Up".
    - oulipo 1y ago
      > There is probably a lot of truth in the tail risks of all of that, and I share these. It just does not at all take away from my enjoyment and optimism at all. ...is just coded "white rich guy" speak to say "I won't suffer the consequences, so I might as well not bother" shameful
  - oulipo 1y ago
bluefirebrand 1y ago
> I can do things while the machine works I find this workflow makes my life extremely tedious, because reviewing and fixing whatever the machine produces is a slog for me I suppose it would be exhilarating if I just trusted the output but somehow I just can't bring myself to do that How do you reconcile this with your own work? Maybe you just skim the output? Or do you run it and test that way? Please don't tell me you just trust the AI to write automated tests or something...
- the_mitsuhiko 1y ago
  > I find this workflow makes my life extremely tedious, because reviewing and fixing whatever the machine produces is a slog for me I don't actually mind the code review process, it's what I'm used to with open source contributions. Not infrequently did I look at a PR that people left me and decide to rewrite it entirely anyways because I did not quite appreciate the method they chose. That however did not mean the PR was useless, it might have provided the necessary input that made me go down that path ultimately. I also think that what can make code review enjoyable is not necessarily because the person on the other side learns something from it. I actually tend to think that this something that does not necessarily happen in PR review, but in 1:1 conversations in person or though more synchronous collaboration. So really my answer is that I do not mind code review. That said there is an important part here: > How do you reconcile this with your own work? Maybe you just skim the output? Or do you run it and test that way? What makes agentic workflows work is good tooling and fast iteration cyclces. That is also exactly what a human needs for a PR review to be actually thorough. I cannot accept a change when I do not know that it exhibits the intended behavior, and to validate that behavior I usually need a test case. Do I need to run the test case myself? Not necessarily because the agent / CI already does for me. However I do sometimes run it with verbose output to also see the intermediate outputs that might otherwise be hidden from me to give me more confidence. It really is hard to describe, but when you put the necessary effort in to make an agentic workflow work, it's really enjoyable. But it does require some upfront investment for it to work. > Please don't tell me you just trust the AI to write automated tests or something... I do in fact trust the AI a lot to write tests. They have become incredibly good at taking certain requirements definitions and derive something that can be mostly trusted. It does not always work, but you get a pretty good feeling for if it will or not. At the end of the day many of the tasks we do are basic CRUD or data transformation tasks.
vouaobrasil 1y ago
> Deep research and similar tools alone have helped me navigate complex legal matters recently for my incorporation, On the flipside, it will make it even easier for corporations to use the legal system in their favour and corporations will more easily and effectively use GenAI against individuals even if both have it due to the nature of corporations and their ability to invest in better tools than the individual. So it's just an arms race/prisoner's dilemma and while it provides incremental advantages, it makes everything worse because the system becomes more complex and more oppressive.
dyauspitr 1y ago
I don’t understand these people that say they are getting no value from LLM. I’m getting value from them probably every other hour.
- dingnuts 1y ago
  Simple, I bother to subtract all the time LLMs have wasted by giving me fake leads and lying to me from the time it saves when it successfully generates a few lines of code
  - jessoteric 1y ago
    This right here. Every time I talk to someone claiming 2-4x multipliers, after a few rounds of conversation they eventually admit "yeah you have to learn when to stop when it's no longer productive, you get a feel for when it's veering into wasting your time". It can't be both. It can't be 2-4x multiplier _and_ be wasting your time to the extent that you have to "get a feel for when it has been wasting your time".
  - solarwindy 1y ago
    Having intermittently given in to the hype and tried whatever is supposedly latest and greatest, it without fail goes like this. I’ve even on occasion deceived myself into thinking I’ve finally hit on the right way to use these tools, only to realize in retrospect that under a complete accounting, the time spent to reach that productive mode of working in no way generalizes, so must be paid effectively every time.
- recursive 1y ago
  Sometimes I try to do stuff with AI. Results are mixed. Sometimes it's somewhat helpful. Sometimes it's a complete waste of time. Maybe that means I'm holding it wrong. In the end, it's probably a wash for me. Maybe slightly positive. I don't understand these people that are saying they are getting huge value from LLMs. I haven't put a ton of effort into figuring out how to "hold it right" because stuff is changing so fast still. I've had bad experiences with rabbit holes like this where the true believers will tell me the pot of gold is always right around the next corner. If and when I can get some positive value out it using whatever's easily available on the default, then I might investigate deeper. My experience mostly consists of GPT and Copilot, as provided by my employer. I'm part of a pilot program to evaluate Copilot. My feedback is that it's slightly positive in agggregate, but individual results are very mixed. It's not worth much to me.
cdavid 1y ago
The issue about executive mandate is likely coming from the context of large corporations. It creates fatigue, even though the underlying technology can be used very effectively to do "real work". It becomes hard for people to really see where the tech is valuable (reduce cost to test ideas, accelerate getting into a new area, etc.) vs where it is just a BS generator. Those are typical dysfunctions in larger companies w/ weak leadership. They magnified by a few factors: AI is indistinguishable from magic for non tech leadership, demos that can be slapped quickly but that don't actually work w/o actual investments (which was what leadership wanted to avoid in the first place), and ofc the promise to reduce costs. This happens in parallel to people using it to do their own work in a more bottom up manner. My anecdotal observation is that it is overused for low-value/high visibility work. E.g. it replaces "writing as proof of work" by reducing the cost to write bland, low information, documents used by middle management, which increases bureaucratic load.
jmsdnns 1y ago
I'm curious to understand the nature of what you're working on better. Whether or not AI performs well is influenced both by the work you're doing and how experienced you are in it. AI performs better when the work is closer to mainstream work than novel work. It also performs better with lower level instructions, eg. being more specific. As for experience, there are two things: people with less experience get a bigger lift than those with lots of experience, or people wiht lots of experience get a lift by having AI doing the work they don't want to do, which is often unittests and comments or writing the bobdyllionth api endpoint. I read your post the other day and appreciated the optimism, but I wasn't able to work out what kind of work you've been doing since leaving Sentry.

oulipo 1y ago

Basically his article says: check the documented issue with the climate, the education, the destruction of trust in society, and your answer is "yes but it makes my nice cuddly programming job better I like it"... don't you understand the issue?

the_mitsuhiko 1y ago
I responded to quite specific quotes which I took issues with, and I intentionally did not talk about the rest of it. I did not make a statement about the rest because it's too complex of a topic to which I do not have enough data to contribute meaningfully to the conversation.
- haswell 1y ago
  > I did not make a statement about the rest because it's too complex of a topic to which I do not have enough data to contribute meaningfully to the conversation. > There is probably a lot of truth in the tail risks of all of that, and I share these. It just does not at all take away from my enjoyment and optimism at all. On the one hand, I get not expanding on things you don't feel you can contribute to meaningfully and respect that. On the other, isn't this a problem? While I agree with you and cannot fully understand the author's conclusion that it's all bad (I've also found value), I wrestle with the downsides on an ongoing basis and have found myself shying away from using these tools as a result, even though I know they are quite capable of some things. I think that if I only responded "Well, I personally find the tools valuable", that this is only a part of the picture and is a net negative to the overall conversation because the "real" conversation that needs to be had IMO is about those harms, not haggling over whether or not the thing can actually do stuff. There's quite a lot of pro-AI content and opinion floating around right now that amounts to: "Yeah, this thing is dangerous, threatens climate goals, is accelerating an education crisis, only exists because of mass theft, creates a massive category of new problems we don't yet know how to solve...but I'm getting value out of it". And the thing that keeps coming to mind is "Don't Look Up".
  - oulipo 1y ago
    > There is probably a lot of truth in the tail risks of all of that, and I share these. It just does not at all take away from my enjoyment and optimism at all. ...is just coded "white rich guy" speak to say "I won't suffer the consequences, so I might as well not bother" shameful
- oulipo 1y ago

bluefirebrand 1y ago

> I can do things while the machine works I find this workflow makes my life extremely tedious, because reviewing and fixing whatever the machine produces is a slog for me I suppose it would be exhilarating if I just trusted the output but somehow I just can't bring myself to do that How do you reconcile this with your own work? Maybe you just skim the output? Or do you run it and test that way? Please don't tell me you just trust the AI to write automated tests or something...

the_mitsuhiko 1y ago
> I find this workflow makes my life extremely tedious, because reviewing and fixing whatever the machine produces is a slog for me I don't actually mind the code review process, it's what I'm used to with open source contributions. Not infrequently did I look at a PR that people left me and decide to rewrite it entirely anyways because I did not quite appreciate the method they chose. That however did not mean the PR was useless, it might have provided the necessary input that made me go down that path ultimately. I also think that what can make code review enjoyable is not necessarily because the person on the other side learns something from it. I actually tend to think that this something that does not necessarily happen in PR review, but in 1:1 conversations in person or though more synchronous collaboration. So really my answer is that I do not mind code review. That said there is an important part here: > How do you reconcile this with your own work? Maybe you just skim the output? Or do you run it and test that way? What makes agentic workflows work is good tooling and fast iteration cyclces. That is also exactly what a human needs for a PR review to be actually thorough. I cannot accept a change when I do not know that it exhibits the intended behavior, and to validate that behavior I usually need a test case. Do I need to run the test case myself? Not necessarily because the agent / CI already does for me. However I do sometimes run it with verbose output to also see the intermediate outputs that might otherwise be hidden from me to give me more confidence. It really is hard to describe, but when you put the necessary effort in to make an agentic workflow work, it's really enjoyable. But it does require some upfront investment for it to work. > Please don't tell me you just trust the AI to write automated tests or something... I do in fact trust the AI a lot to write tests. They have become incredibly good at taking certain requirements definitions and derive something that can be mostly trusted. It does not always work, but you get a pretty good feeling for if it will or not. At the end of the day many of the tasks we do are basic CRUD or data transformation tasks.

vouaobrasil 1y ago

> Deep research and similar tools alone have helped me navigate complex legal matters recently for my incorporation, On the flipside, it will make it even easier for corporations to use the legal system in their favour and corporations will more easily and effectively use GenAI against individuals even if both have it due to the nature of corporations and their ability to invest in better tools than the individual. So it's just an arms race/prisoner's dilemma and while it provides incremental advantages, it makes everything worse because the system becomes more complex and more oppressive.

dyauspitr 1y ago

I don’t understand these people that say they are getting no value from LLM. I’m getting value from them probably every other hour.

dingnuts 1y ago
Simple, I bother to subtract all the time LLMs have wasted by giving me fake leads and lying to me from the time it saves when it successfully generates a few lines of code
- jessoteric 1y ago
  This right here. Every time I talk to someone claiming 2-4x multipliers, after a few rounds of conversation they eventually admit "yeah you have to learn when to stop when it's no longer productive, you get a feel for when it's veering into wasting your time". It can't be both. It can't be 2-4x multiplier _and_ be wasting your time to the extent that you have to "get a feel for when it has been wasting your time".
- solarwindy 1y ago
  Having intermittently given in to the hype and tried whatever is supposedly latest and greatest, it without fail goes like this. I’ve even on occasion deceived myself into thinking I’ve finally hit on the right way to use these tools, only to realize in retrospect that under a complete accounting, the time spent to reach that productive mode of working in no way generalizes, so must be paid effectively every time.
recursive 1y ago
Sometimes I try to do stuff with AI. Results are mixed. Sometimes it's somewhat helpful. Sometimes it's a complete waste of time. Maybe that means I'm holding it wrong. In the end, it's probably a wash for me. Maybe slightly positive. I don't understand these people that are saying they are getting huge value from LLMs. I haven't put a ton of effort into figuring out how to "hold it right" because stuff is changing so fast still. I've had bad experiences with rabbit holes like this where the true believers will tell me the pot of gold is always right around the next corner. If and when I can get some positive value out it using whatever's easily available on the default, then I might investigate deeper. My experience mostly consists of GPT and Copilot, as provided by my employer. I'm part of a pilot program to evaluate Copilot. My feedback is that it's slightly positive in agggregate, but individual results are very mixed. It's not worth much to me.

cdavid 1y ago

The issue about executive mandate is likely coming from the context of large corporations. It creates fatigue, even though the underlying technology can be used very effectively to do "real work". It becomes hard for people to really see where the tech is valuable (reduce cost to test ideas, accelerate getting into a new area, etc.) vs where it is just a BS generator. Those are typical dysfunctions in larger companies w/ weak leadership. They magnified by a few factors: AI is indistinguishable from magic for non tech leadership, demos that can be slapped quickly but that don't actually work w/o actual investments (which was what leadership wanted to avoid in the first place), and ofc the promise to reduce costs. This happens in parallel to people using it to do their own work in a more bottom up manner. My anecdotal observation is that it is overused for low-value/high visibility work. E.g. it replaces "writing as proof of work" by reducing the cost to write bland, low information, documents used by middle management, which increases bureaucratic load.

jmsdnns 1y ago

I'm curious to understand the nature of what you're working on better. Whether or not AI performs well is influenced both by the work you're doing and how experienced you are in it. AI performs better when the work is closer to mainstream work than novel work. It also performs better with lower level instructions, eg. being more specific. As for experience, there are two things: people with less experience get a bigger lift than those with lots of experience, or people wiht lots of experience get a lift by having AI doing the work they don't want to do, which is often unittests and comments or writing the bobdyllionth api endpoint. I read your post the other day and appreciated the optimism, but I wasn't able to work out what kind of work you've been doing since leaving Sentry.