5 ms·

I think AI finally is at point where it is. It’s peak hype right now but there is definitely a difference between knowledge workers that use it and one’s that d

I think AI finally is at point where it is. It’s peak hype right now but there is definitely a difference between knowledge workers that use it and one’s that don’t. It’s only going to grow wider in the coming years.

packetlost 3y ago
> there is definitely a difference between knowledge workers that use it and one’s that don’t [citation needed] I tried out Copilot for months and didn't get much value out of it. The majority of my time and effort is not writing code, which is about the only thing an LLM can generate semi-useful output for. It's not more useful than a tailored search engine experience and it doesn't replace deep dives into text books, code, and white papers. Where are you seeing major benefits?
- kimixa 3y ago
  We've also done some initial experiments, and seen no improvements in productivity, and the comments from the people using it were rather negative. Maybe it's sector dependent? Maybe systems programming doesn't have enough reference materials for them to learn from? Certainly not the "Clearly obvious difference" some people seem to be claiming to see in some places online.
  - packetlost 3y ago
    The latency is distracting for me tbh. That's something I think could be fixed by running local models, but the quality of output is just not there. Maybe if you're slinging HTML and JS all day it helps more
  - pydry 3y ago
    I've watched people who claim it is a game changer try to use it and I can see routinely leading them down rabbit holes in real time which they struggle to get out of. Meanwhile, the stuff it is good at filling in - it was never that hard to google it and copy and paste it in the first place. I've seen this like 20 times in tech before. We're peak hype cycle for LLMs and the trough of disillusionment has yet to set in. During peak hype cycle for any tech a lot of people defer to the crowd's excitable opinion, disbelieve the evidence in front of their own eyes and express a level of optimism over future developments that is ludicrous.
  - ben_w 3y ago
    My experiments with 3.5 say that it does the easy 80% solution that takes 20% of my time. The advantage, that actually is a game changer[0], is that it can do languages I can't. I'm an iOS dev, I've technically been paid to write JavaScript at various points, but the best of that was about 20 years ago and just before jQuery got popular. ChatGPT lets me turn my ideas for JavaScript projects into things that almost work and are usually close enough for me to fix — turning me from an iOS dev into a JS project manager. Likewise Stable Diffusion: I've done some game artwork when I tried self employed for a bit, SD can act like a mostly amazing artist that does a few bits (mostly hands) like it was temporarily high on LSD, and I can just highlight those messed up regions and say "do again five times", and the actual images get to me faster than a real human would notice a message on slack or email or whatever, let alone be able to actually attempt a fix. As for music… I actually made a procedural music generator back then (for the games while self-employed), and while neither the music nor the generator is going to win any awards, the output was sufficient for the games it went with. All the various new music AI are way better than what I did. [0] and now I realise the cliché; ugh
    - legendofbrando 3y ago
      This really speaks to me and my experience. It’s not AI’s ability to beat me in my own domain, it’s the ability for me to extend my domain to a different style to achieve the results I want. I think folks that underrate the technology are evaluating on “is it better than me (or the person I know who is great) at this” when what you point out is more: “can it help me do this, like that” — that’s where it shines. Further, I think folks under appreciate how being able to extend what you “know” to a different format (ie different language) enables you to determine the next set of problems that help you either: (a) find the right partners with the right skills or (b) ask the deeper harder questions about your gaps.
  - AussieWog93 3y ago
    For me (solo dev, doing both a project in an unfamiliar framework and a project in on an unfamiliar new microcontroller), GPT-4 has been a lifesaver. Rather than spending potentially hours to sift through documentation or ask around on the forums to find the magic incantation I need, ChatGPT just spits it out instantly and I can move on to solving actual problems. That said, in a team environment with experienced devs, it would probably be better for me to just ask someone more experienced. Learn faster and more in-depth. Also worth pointing out that GPT-3.5 and GPT-4 are completely different beasts. If you were getting the devs to use 3.5, you will get bad results no matter who is using it.
- lordnacho 3y ago
  Main benefit is it does boilerplate really well. You type a few letters and then hit tab, and now your test case is done. I did this as late as yesterday. Type in a case, type in a variant of the case that is more or less symmetric, it fills in what you would have typed. Or you add a member to your object, thus requiring it to be tested. It figures out that you need to check this new thing in all your tests. Other thing it does well is syntax. Some weird c++ template parameter thing, it does it for you. Little things like "what's that thing that makes the sort go the other way" it will know, and save you a minute of googling. What there doesn't seem to be is an interface for that I've found is any kind of refactoring. That's still a form of donkey work that a junior guy can do, but you'd think there would be some way for the AI to suggest DRY candidates and re-orgs.
  - valenterry 3y ago
    I find that to be true as well. It's just that boilerplate is usually an indication of a bad design or a bad/verbose language. Both of it are painpoints that should be fixed. But if that's not possible then Copilot is a good helper.
    - jezzamon 3y ago
      An alternative idea is that some types of "boilerplate" can be annoying to write but actually helpful for people to read, and these tools might make that type of code more acceptable. For example, it's helpful to be able to see all the steps a test case is doing in the one function, so you can read that and see what's going wrong if it fails. If a lot of your test cases are similar you might be tempted to refactor and put common parts in separate methods, often with unhelpful names like "assertResult()", etc, but now the reader has to jump about to understand the logic. Whereas this allows you to write the more repetitive but also more readable code easier
- dasil003 3y ago
  It's early days on the product applications, but the fundamental power is being able to find correlations and connections across a huge corpus and output a large variety of rough ideas that can then be selected and polished by humans. It doesn't replace human expertise and creativity, but it will chip away at the bottom where many jobs are essentially rote tasks with very little human-level judgement needed, and more apropos to the GP, it will provide an increasingly powerful assist to human experts who learn how to harness it.
  - packetlost 3y ago
    I don't see how that's a better position than general automation, which software has been chipping away at the bottom rung of for decades. Most jobs outside of mass produced SEO Buzzfeed garbage are not about generating large amounts of believable sounding prose. I could see it chipping away at the bottom rung of customer support, but that very quickly turns into work where real decisions have to be made and an 80% correct output is not acceptable. I'd be extremely surprised if any half serious company lets an LLM decide whether a refund should be processed or rejected, for example.

packetlost 3y ago

> there is definitely a difference between knowledge workers that use it and one’s that don’t [citation needed] I tried out Copilot for months and didn't get much value out of it. The majority of my time and effort is not writing code, which is about the only thing an LLM can generate semi-useful output for. It's not more useful than a tailored search engine experience and it doesn't replace deep dives into text books, code, and white papers. Where are you seeing major benefits?

kimixa 3y ago
We've also done some initial experiments, and seen no improvements in productivity, and the comments from the people using it were rather negative. Maybe it's sector dependent? Maybe systems programming doesn't have enough reference materials for them to learn from? Certainly not the "Clearly obvious difference" some people seem to be claiming to see in some places online.
- packetlost 3y ago
  The latency is distracting for me tbh. That's something I think could be fixed by running local models, but the quality of output is just not there. Maybe if you're slinging HTML and JS all day it helps more
- pydry 3y ago
  I've watched people who claim it is a game changer try to use it and I can see routinely leading them down rabbit holes in real time which they struggle to get out of. Meanwhile, the stuff it is good at filling in - it was never that hard to google it and copy and paste it in the first place. I've seen this like 20 times in tech before. We're peak hype cycle for LLMs and the trough of disillusionment has yet to set in. During peak hype cycle for any tech a lot of people defer to the crowd's excitable opinion, disbelieve the evidence in front of their own eyes and express a level of optimism over future developments that is ludicrous.
- ben_w 3y ago
  My experiments with 3.5 say that it does the easy 80% solution that takes 20% of my time. The advantage, that actually is a game changer[0], is that it can do languages I can't. I'm an iOS dev, I've technically been paid to write JavaScript at various points, but the best of that was about 20 years ago and just before jQuery got popular. ChatGPT lets me turn my ideas for JavaScript projects into things that almost work and are usually close enough for me to fix — turning me from an iOS dev into a JS project manager. Likewise Stable Diffusion: I've done some game artwork when I tried self employed for a bit, SD can act like a mostly amazing artist that does a few bits (mostly hands) like it was temporarily high on LSD, and I can just highlight those messed up regions and say "do again five times", and the actual images get to me faster than a real human would notice a message on slack or email or whatever, let alone be able to actually attempt a fix. As for music… I actually made a procedural music generator back then (for the games while self-employed), and while neither the music nor the generator is going to win any awards, the output was sufficient for the games it went with. All the various new music AI are way better than what I did. [0] and now I realise the cliché; ugh
  - legendofbrando 3y ago
    This really speaks to me and my experience. It’s not AI’s ability to beat me in my own domain, it’s the ability for me to extend my domain to a different style to achieve the results I want. I think folks that underrate the technology are evaluating on “is it better than me (or the person I know who is great) at this” when what you point out is more: “can it help me do this, like that” — that’s where it shines. Further, I think folks under appreciate how being able to extend what you “know” to a different format (ie different language) enables you to determine the next set of problems that help you either: (a) find the right partners with the right skills or (b) ask the deeper harder questions about your gaps.
- AussieWog93 3y ago
  For me (solo dev, doing both a project in an unfamiliar framework and a project in on an unfamiliar new microcontroller), GPT-4 has been a lifesaver. Rather than spending potentially hours to sift through documentation or ask around on the forums to find the magic incantation I need, ChatGPT just spits it out instantly and I can move on to solving actual problems. That said, in a team environment with experienced devs, it would probably be better for me to just ask someone more experienced. Learn faster and more in-depth. Also worth pointing out that GPT-3.5 and GPT-4 are completely different beasts. If you were getting the devs to use 3.5, you will get bad results no matter who is using it.
lordnacho 3y ago
Main benefit is it does boilerplate really well. You type a few letters and then hit tab, and now your test case is done. I did this as late as yesterday. Type in a case, type in a variant of the case that is more or less symmetric, it fills in what you would have typed. Or you add a member to your object, thus requiring it to be tested. It figures out that you need to check this new thing in all your tests. Other thing it does well is syntax. Some weird c++ template parameter thing, it does it for you. Little things like "what's that thing that makes the sort go the other way" it will know, and save you a minute of googling. What there doesn't seem to be is an interface for that I've found is any kind of refactoring. That's still a form of donkey work that a junior guy can do, but you'd think there would be some way for the AI to suggest DRY candidates and re-orgs.
- valenterry 3y ago
  I find that to be true as well. It's just that boilerplate is usually an indication of a bad design or a bad/verbose language. Both of it are painpoints that should be fixed. But if that's not possible then Copilot is a good helper.
  - jezzamon 3y ago
    An alternative idea is that some types of "boilerplate" can be annoying to write but actually helpful for people to read, and these tools might make that type of code more acceptable. For example, it's helpful to be able to see all the steps a test case is doing in the one function, so you can read that and see what's going wrong if it fails. If a lot of your test cases are similar you might be tempted to refactor and put common parts in separate methods, often with unhelpful names like "assertResult()", etc, but now the reader has to jump about to understand the logic. Whereas this allows you to write the more repetitive but also more readable code easier
dasil003 3y ago
It's early days on the product applications, but the fundamental power is being able to find correlations and connections across a huge corpus and output a large variety of rough ideas that can then be selected and polished by humans. It doesn't replace human expertise and creativity, but it will chip away at the bottom where many jobs are essentially rote tasks with very little human-level judgement needed, and more apropos to the GP, it will provide an increasingly powerful assist to human experts who learn how to harness it.
- packetlost 3y ago
  I don't see how that's a better position than general automation, which software has been chipping away at the bottom rung of for decades. Most jobs outside of mass produced SEO Buzzfeed garbage are not about generating large amounts of believable sounding prose. I could see it chipping away at the bottom rung of customer support, but that very quickly turns into work where real decisions have to be made and an 80% correct output is not acceptable. I'd be extremely surprised if any half serious company lets an LLM decide whether a refund should be processed or rejected, for example.