3 ms·

> I don’t know how much. If you're not willing to measure how it helps you, then it's probably not worth it. I would go even further: if the effort of measuri

> I don’t know how much. If you're not willing to measure how it helps you, then it's probably not worth it. I would go even further: if the effort of measuring is not feasible, then it's probably not worth it. That is more targeted at companies than you specifically, but it also works as an individual reflection. In the individual reflection, it works like this: you should think "how can I prove to myself that I'm not being bamboozled?". Once you acquire that proof, it should be easy to share it with others. If it's not, it's probably not a good proof (like an anecdote). I already said this, and I'll say it again: record yourself using LLMs. Then watch the recording. Is it that good? Notice that I am removing myself from the equation here, I will not judge how good is it, you're going to do it yourself.

aeon_ai 1y ago
I just did it. You were right. It is, in fact, that good.
- alganet 1y ago
  You could have recorded, found it to be good, and didn't shared the news. Only used for your self. But you decided to share only the news, not the recording. That tells me something. To be more clear, I can move this argument further. I promise you that if you share the recording that led you to believe that, I will not judge it. In fact, I will do the opposite and focus on people who judge it, trying my best to make the recording look good and point out whoever is nitpicking.
wongarsu 1y ago
There is a difference between confirming that something is worth it and quantifying the benefit though. One only requires satisfying a lower bound, the other requires an exact number. For example I use a $30/month chatbot subscription for various utility tasks. If I value my time at above $60/hour I need to save half an hour each month (a minute a day) to make the investment worth it. That is absolutely true, just with simple googleable questions and light research tasks I save much more than 7 minutes a week. But how much do I actually save? What exactly is my time actually worth? Those are much more difficult questions to answer
- brailsafe 1y ago
  For me it's less important how much time I think I save on any discrete task, and how much time I net over that time, accounting for how much time the set of tasks I'd be working on would have otherwise taken had I just manually done them. Right now, that means debits and credits in the ledger of time. Sometimes I gain a lot on tasks I probably otherwise wouldn't have done, but I also don't gain much overall by doing, and sometimes I lose a ton of time simply by leaning on a loop of re-doing inaccurate agent work in a way that's actually more time intensive than had I internalized the system in working memory and produced functionality more slowly. If I save an hour, but lose 6, when I'd otherwise have spent 2, then I net -4, but sometimes overall it's positive, so the value is more ambiguous. If my employer didn't pay for the tools, I really don't know whether I would. A good price and conservative usage pattern might net more.
- alganet 1y ago
  The user @brailsafe gave an answer that embodies some things I was going to say. You're accounting for the time wins, not accounting for the time losses. For a human chat user, that's when the LLM fails an answer or answers wrong. For an LLM coder, that's when context rot creeps in and you have to restart your work, and so on. There are people who don't care much if they are being bamboozled for $30/mo, they have nothing to prove nor grand expectations for the thing. To them, cargo culting might be fun and that's what they extract from the bargain. I am directing my answers mostly to people, companies or individuals, who have something to prove (evangelists, AI companies, etc). To those, a series of imperceptible small losses that results in debt in the long run is a big problem. My suggestion (the recording session) also works as a metaphor. That could be, instead of video, metrics about how contexts are discarded. It is, in that sense, also something they can decide to share or not, and the extent to what they share should be a sign of confidence in their product. Makes sense?

aeon_ai 1y ago

I just did it. You were right. It is, in fact, that good.

alganet 1y ago
You could have recorded, found it to be good, and didn't shared the news. Only used for your self. But you decided to share only the news, not the recording. That tells me something. To be more clear, I can move this argument further. I promise you that if you share the recording that led you to believe that, I will not judge it. In fact, I will do the opposite and focus on people who judge it, trying my best to make the recording look good and point out whoever is nitpicking.

wongarsu 1y ago

There is a difference between confirming that something is worth it and quantifying the benefit though. One only requires satisfying a lower bound, the other requires an exact number. For example I use a $30/month chatbot subscription for various utility tasks. If I value my time at above $60/hour I need to save half an hour each month (a minute a day) to make the investment worth it. That is absolutely true, just with simple googleable questions and light research tasks I save much more than 7 minutes a week. But how much do I actually save? What exactly is my time actually worth? Those are much more difficult questions to answer

brailsafe 1y ago
For me it's less important how much time I think I save on any discrete task, and how much time I net over that time, accounting for how much time the set of tasks I'd be working on would have otherwise taken had I just manually done them. Right now, that means debits and credits in the ledger of time. Sometimes I gain a lot on tasks I probably otherwise wouldn't have done, but I also don't gain much overall by doing, and sometimes I lose a ton of time simply by leaning on a loop of re-doing inaccurate agent work in a way that's actually more time intensive than had I internalized the system in working memory and produced functionality more slowly. If I save an hour, but lose 6, when I'd otherwise have spent 2, then I net -4, but sometimes overall it's positive, so the value is more ambiguous. If my employer didn't pay for the tools, I really don't know whether I would. A good price and conservative usage pattern might net more.
alganet 1y ago
The user @brailsafe gave an answer that embodies some things I was going to say. You're accounting for the time wins, not accounting for the time losses. For a human chat user, that's when the LLM fails an answer or answers wrong. For an LLM coder, that's when context rot creeps in and you have to restart your work, and so on. There are people who don't care much if they are being bamboozled for $30/mo, they have nothing to prove nor grand expectations for the thing. To them, cargo culting might be fun and that's what they extract from the bargain. I am directing my answers mostly to people, companies or individuals, who have something to prove (evangelists, AI companies, etc). To those, a series of imperceptible small losses that results in debt in the long run is a big problem. My suggestion (the recording session) also works as a metaphor. That could be, instead of video, metrics about how contexts are discarded. It is, in that sense, also something they can decide to share or not, and the extent to what they share should be a sign of confidence in their product. Makes sense?