9 ms·

I'm with you on this one. If one's experience is just using it as a general chat bot, then yeah, I can see why people are reluctant thinking it's useless. I hav

I'm with you on this one. If one's experience is just using it as a general chat bot, then yeah, I can see why people are reluctant thinking it's useless. I have a feeling that a good chunk of people haven't tried to use latest models on a medium-small project from scratch, where you have to play around with its intricacies to build intuition around it. It becomes some sort of muscle memory, where I can predict whether using LLM would be faster or slower. Or where it's more likely to give bad suggestions or not. Basically treating it as googling skills.

simonw 2y ago
"It becomes some sort of muscle memory, where I can predict whether using LLM would be faster or slower" Yeah, that intuition is so important. You have to use the models a whole bunch to develop it, but eventually you get a sort of sixth sense where you can predict if an LLM is going to be useful or harmful on a problem and be right about it 9/10 times. My frustration is that intuition isn't something I can teach! I'd love to be able to explain why I can tell that problem X is a good fit and problem Y isn't, but often the answer is pretty much just "vibes" based on past experience.
- tokioyoyo 2y ago
  Totally! My current biggest problems: 1) Being up to date with the latest capabilities - this one has slowed down a bit, and my biggest self-learning experience was in August/September, and most of the intuition still works. However, although I had time to do so, it's hard to ask my team to drop 5-6 free weekends of their lives to get up to speed 2) Transition period where not everyone is on the same page about LLM - this I think is much harder, because the expectations from the executives are much different than on the ground developers using LLMs. A lot of people could benefit on alignment of expectations, but once again, it's hard to explain what is possible and not possible if your statements will become nullified a month later with a new AI model/product/feature.
- th0ma5 2y ago
  You're confusing intuition of what works or not with being too close to your problem to make a judgement on the general applicability of your techniques.
  - simonw 2y ago
    I don't understand what you mean.
    - entropicdrifter 2y ago
      They're saying it might work for you, but isn't generally applicable (because most people aren't going to develop this intuition, presumably). Not sure I agree with that. I would say there are classes of problems where LLMs will generally help and a brief training course (1 week, say) would vastly improve the average (non-LLM-trained) engineer's ability to use it productively.
      - th0ma5 2y ago
        No it is more like thinking that my prescribed way of doing stuff must be the way things work in general because it works for me. But you give instructions specifically about everything you did but the person you give them to isn't your exact height or can't read your language, so you can easily assume now that they just don't get it. But with these LLMs you also get this bias hidden from you as you inch closer to the solution at every turn. The result seems "obvious" but the outcomes were never guaranteed and will most likely be different for someone else if one thing at point is different.
        simonw 2y ago
        My whole thing about LLMs is that using them isn't "obvious". I've been banging that drum for over a year now - the single biggest misconception about LLMs is that they are easy to use and you don't need to put a serious amount of effort into learning how to best apply them.

simonw 2y ago

"It becomes some sort of muscle memory, where I can predict whether using LLM would be faster or slower" Yeah, that intuition is so important. You have to use the models a whole bunch to develop it, but eventually you get a sort of sixth sense where you can predict if an LLM is going to be useful or harmful on a problem and be right about it 9/10 times. My frustration is that intuition isn't something I can teach! I'd love to be able to explain why I can tell that problem X is a good fit and problem Y isn't, but often the answer is pretty much just "vibes" based on past experience.

tokioyoyo 2y ago
Totally! My current biggest problems: 1) Being up to date with the latest capabilities - this one has slowed down a bit, and my biggest self-learning experience was in August/September, and most of the intuition still works. However, although I had time to do so, it's hard to ask my team to drop 5-6 free weekends of their lives to get up to speed 2) Transition period where not everyone is on the same page about LLM - this I think is much harder, because the expectations from the executives are much different than on the ground developers using LLMs. A lot of people could benefit on alignment of expectations, but once again, it's hard to explain what is possible and not possible if your statements will become nullified a month later with a new AI model/product/feature.
th0ma5 2y ago
You're confusing intuition of what works or not with being too close to your problem to make a judgement on the general applicability of your techniques.
- simonw 2y ago
  I don't understand what you mean.
  - entropicdrifter 2y ago
    They're saying it might work for you, but isn't generally applicable (because most people aren't going to develop this intuition, presumably). Not sure I agree with that. I would say there are classes of problems where LLMs will generally help and a brief training course (1 week, say) would vastly improve the average (non-LLM-trained) engineer's ability to use it productively.
    - th0ma5 2y ago
      No it is more like thinking that my prescribed way of doing stuff must be the way things work in general because it works for me. But you give instructions specifically about everything you did but the person you give them to isn't your exact height or can't read your language, so you can easily assume now that they just don't get it. But with these LLMs you also get this bias hidden from you as you inch closer to the solution at every turn. The result seems "obvious" but the outcomes were never guaranteed and will most likely be different for someone else if one thing at point is different.
      - simonw 2y ago
        My whole thing about LLMs is that using them isn't "obvious". I've been banging that drum for over a year now - the single biggest misconception about LLMs is that they are easy to use and you don't need to put a serious amount of effort into learning how to best apply them.