10 ms·

What i find interesting are the discussions around good vs bad prompts and being adept at “prompt engineering” We’ve had this in the engineering world a long t

What i find interesting are the discussions around good vs bad prompts and being adept at “prompt engineering” We’ve had this in the engineering world a long time in the form of Google and StackOverflow searches. The great engineers i’ve worked with know most stuff they need day-to-day, but when they run into issues it’s their ability to search for and find the right answer that gives them advantage over others who don’t know where to look (or even start).

ChatGTP 4y ago
Using an LLM yesterday to code, there was no way I’d be proficient with the workflow if I wasn’t an experienced programmer. I’ve never seen code look so correct, sort of work but be so strangely crap.
- chillfox 4y ago
  I used to think that LLMs for coding would mostly be a tool to make experienced developers much more productive, but after coming across this video I think it will be used by normal people with no programming experience to make small basic programs. https://www.youtube.com/watch?v=IyKKhxYJ4U4 https://www.youtube.com/watch?v=IyKKhxYJ4U4
  - otabdeveloper4 4y ago
    Normal people with no programming experience can already make small basic programs by copy-pasting code from Github and StackOverflow.
    - lordnacho 4y ago
      This is so much harder than most people think. I would reckon that as soon as a complication shows up, 75% of non coders will get helplessly stuck. Complication meaning something slightly beyond changing variable names: you need a different if condition, the snippet doesn't contain the required import, the version has advanced slightly.
    - chillfox 4y ago
      Most people will have no idea where to even paste the code. The key take away from the video I linked is that an AI can tell people not only what code to copy, but where to copy it and what to change if it doesn't work.
      - otabdeveloper4 4y ago
        > where to copy it and what to change if it doesn't work Google can do that too. (Well, it could if it wasn't permanently broken from SEO spam and monetization efforts.) Which is the key takeaway: the "chat" part of ChatGPT is a stupid smoke and mirrors gimmick. The salient useful parts are the large and well-curated training database. Which is technically quite possible to do without "AI" or text generators; though possibly not realistic from a business sense. There's no business case for "information search without spam and ads".
- jeltz 4y ago
  Indeed, ChatGPT really excels at being confidently incorrect. Ask it anything non-trivial and chances are that it has written code which is subtly wrong, and sometimes very subtly.
  - 8n4vidtmkvmk 4y ago
    it got me today. produced correct output for good input but did not consider a very big edge case that i didn't initially think about. if this is how we're going to write code then there's going to be even more broken edge cases than usual.
    - visarga 4y ago
      Maybe you needed to ask the model to review its own work and especially look for edge cases.
      - 8n4vidtmkvmk 4y ago
        I could. But then I'd feel like a parent that needs to tell his kid to wash his hands and zip up his pants.
- onion2k 4y ago
  I’ve never seen code look so correct, sort of work but be so strangely crap. This is my experience of GPT generated code too. It needs refactoring as soon as it's generated for me to be happy with it. The real question is whether that actually matters though. If an AI tool is writing ugly code that works, and the 'developer' is only interacting with that code through prompts, then what the code looks like is irrevelant. No one will see it. It'd be like worrying about what the inside of a Photoshop file or a Word doc looks like. I think we're some way off using AI to write and modify code like it's an opaque file, but once we're there it won't matter if that code is ugly and terrible. No one will look.
  - rolisz 4y ago
    Yes, I can easily see a future where many abstractions we currently use in code will go away, because we'll just use LLMs to change the code when needed, instead of trying to future proof code.
    - visarga 4y ago
      Refactoring and cleaning up technical debt might see huge improvements in the future.
- akrymski 4y ago
  Sounds like an assembly programmer trying C for the first time. Sure the produced assembly isn't as good as hand written. But with Moore's law, maybe it doesn't matter?
  - funnymony 4y ago
    Not sure if you heard - Gordon Moore just recently passed away. https://en.m.wikipedia.org/wiki/Gordon_Moore https://en.m.wikipedia.org/wiki/Gordon_Moore
    - CyberDildonics 4y ago
      That doesn't have anything to do with the parent poster's point.
reb 4y ago
Another skill that'll be pretty valuable in this era is one many of the best technical leaders possess: the ability to coach desirable outcomes from squishy autonomous black boxes over time. Getting excellent, exact performance out of deterministic systems is an impressive feat, but autonomy means variability, and getting excellent performance out of variable systems (especially illegible ones) is a different game.
- CSSer 4y ago
  I agree 100%. I first had this thought last night when reading “Eight Things to Know about LLMs” (select quote from section 8 below)[0] > Simply prompting a model to “think step by step” can lead it to perform well on entire categories of math and reasoning problems that it would otherwise fail on (Kojima et al., 2022). Similarly, even observing that an LLM consistently fails at some task is far from sufficient evidence that no other LLM can do that task (Bowman, 2022). The eerie thing to me is that this is coaching. I have never coached anything that isn’t alive by any definition. My feelings about this realization are ambivalent. [0]: https://cims.nyu.edu/~sbowman/eightthings.pdf https://cims.nyu.edu/~sbowman/eightthings.pdf
  - kristov 4y ago
    Doesn't prompting "think step by step" simply increase the probability the model will generate its output in the style of places where it saw "step by step" before (eg: tutorial content)? It's not really "deciding" to think step by step. Maybe I am mistaken.
    - hobs 4y ago
      You're not mistaken, its basically perturbing the seed for which the sentence grows.
    - pegasus 4y ago
      It's more than just that. The intermediary steps act as a kind of working memory, ensuring that the model is less likely to get lost in the weeds.
saghm 4y ago
This makes me wonder: will we start to see the reverse like we have in search engines? Will people start finding ways to do SEO for language models to try to get them to output stuff about their products more often?
- spullara 4y ago
  Yes. It is definitely a risk.
- actionfromafar 4y ago
  Sure, I bet they are already using LLMs to poison the web for other LLMs.
  - PaulHoule 4y ago
    see https://arxiv.org/abs/2302.10149 https://arxiv.org/abs/2302.10149
- grumbel 4y ago
  They'll certainly try, but I am somewhat optimistic here, as you can feed the language model with some ground truths that allows it to detect the marketing nonsense. Another nice thing with language models is that they condense the information, it doesn't matter how many webpages there are for a product, if you ask ChatGPT about it, it will list it exactly once. It knows it is the same product each time and the list it produces will be created specifically for your query. Meanwhile Google gets cluttered with duplicate information, since your query is implicitly about webpages mentioning a product, not products themselves. That said, there is still plenty of work to do here. BingChat is completely unusable when it comes to product search, much worse than Google, as it will just pick the first three search results, which tend to be SEO spam, and summarize their content. ChatGPT is much better here, but its lack of direct Web access also handicaps it rather heavily.
ramraj07 4y ago
Exactly, though an addendum is not just their ability to search when the need arises, but also to know WHEN to resort to search, and with what angle. This is not just isolated to coding though, experienced the same in research as well. So much hinges on your ability to know when to search for literature and how.
- visarga 4y ago
  But LLM black boxes are pretty great at surfacing search leads. They might hallucinate, but they are much more precise semantically than search. LLMs understand when you say something in your own words. In fact one trick is to go to LLM to get the "closed-book" answer, and then use that answer to formulate a proper search query for Google. Q: What is the height of Everest. A: The height of Everest is 8800m Search: "The height of Everest is 8800m" Result: 8849m Using the generated answer as search phrase, even if it is factually wrong, works because it has the right structure and good keywords.
alkonaut 4y ago
I always felt my best asset as an engineer was my googling skills. I always also felt like this is almost cheating, but of course in a way (in most ways) it's not. I often solve problems for colleagues by returning a google response from the first page of search results. Meaning, most likely, that I had a better google query than they had. I wonder if this skill can become more useful in the age of "prompt engineering"? Will my googling skills be useless and I need to start over? Or will my ability to google naturally convert into a decent prompt-engineering skill given a reasonable amount of practice?
- bryanrasmussen 4y ago
  >Meaning, most likely, that I had a better google query than they had. there's two parts, good google query, but also figuring out which of the results returned is the most likely to be useful. I've often found results for people that were on the first page of the most obvious query in the world to start with, but they were not the first result but down a bit and maybe looked a bit weird.
- factormeta 4y ago
  Also keep in mind that your google query is bubbled, may be your search query is better than your colleagues because of your search history etc.
- duggan 4y ago
  I used to think this, but I'm not sure google-fu is the real superpower here. I've been surprised to encounter many developers who just don't seem to be able to read and comprehend the output of a program; logs, stack traces, etc. That's the basic — sometimes only — input to a search query. I suspect the ability to comprehend and describe a problem is a skill that translates quite smoothly to a world of prompt-based interaction.

ChatGTP 4y ago

Using an LLM yesterday to code, there was no way I’d be proficient with the workflow if I wasn’t an experienced programmer. I’ve never seen code look so correct, sort of work but be so strangely crap.

chillfox 4y ago
I used to think that LLMs for coding would mostly be a tool to make experienced developers much more productive, but after coming across this video I think it will be used by normal people with no programming experience to make small basic programs. https://www.youtube.com/watch?v=IyKKhxYJ4U4 https://www.youtube.com/watch?v=IyKKhxYJ4U4
- otabdeveloper4 4y ago
  Normal people with no programming experience can already make small basic programs by copy-pasting code from Github and StackOverflow.
  - lordnacho 4y ago
    This is so much harder than most people think. I would reckon that as soon as a complication shows up, 75% of non coders will get helplessly stuck. Complication meaning something slightly beyond changing variable names: you need a different if condition, the snippet doesn't contain the required import, the version has advanced slightly.
  - chillfox 4y ago
    Most people will have no idea where to even paste the code. The key take away from the video I linked is that an AI can tell people not only what code to copy, but where to copy it and what to change if it doesn't work.
    - otabdeveloper4 4y ago
      > where to copy it and what to change if it doesn't work Google can do that too. (Well, it could if it wasn't permanently broken from SEO spam and monetization efforts.) Which is the key takeaway: the "chat" part of ChatGPT is a stupid smoke and mirrors gimmick. The salient useful parts are the large and well-curated training database. Which is technically quite possible to do without "AI" or text generators; though possibly not realistic from a business sense. There's no business case for "information search without spam and ads".
jeltz 4y ago
Indeed, ChatGPT really excels at being confidently incorrect. Ask it anything non-trivial and chances are that it has written code which is subtly wrong, and sometimes very subtly.
- 8n4vidtmkvmk 4y ago
  it got me today. produced correct output for good input but did not consider a very big edge case that i didn't initially think about. if this is how we're going to write code then there's going to be even more broken edge cases than usual.
  - visarga 4y ago
    Maybe you needed to ask the model to review its own work and especially look for edge cases.
    - 8n4vidtmkvmk 4y ago
      I could. But then I'd feel like a parent that needs to tell his kid to wash his hands and zip up his pants.
onion2k 4y ago
I’ve never seen code look so correct, sort of work but be so strangely crap. This is my experience of GPT generated code too. It needs refactoring as soon as it's generated for me to be happy with it. The real question is whether that actually matters though. If an AI tool is writing ugly code that works, and the 'developer' is only interacting with that code through prompts, then what the code looks like is irrevelant. No one will see it. It'd be like worrying about what the inside of a Photoshop file or a Word doc looks like. I think we're some way off using AI to write and modify code like it's an opaque file, but once we're there it won't matter if that code is ugly and terrible. No one will look.
- rolisz 4y ago
  Yes, I can easily see a future where many abstractions we currently use in code will go away, because we'll just use LLMs to change the code when needed, instead of trying to future proof code.
  - visarga 4y ago
    Refactoring and cleaning up technical debt might see huge improvements in the future.
akrymski 4y ago
Sounds like an assembly programmer trying C for the first time. Sure the produced assembly isn't as good as hand written. But with Moore's law, maybe it doesn't matter?
- funnymony 4y ago
  Not sure if you heard - Gordon Moore just recently passed away. https://en.m.wikipedia.org/wiki/Gordon_Moore https://en.m.wikipedia.org/wiki/Gordon_Moore
  - CyberDildonics 4y ago
    That doesn't have anything to do with the parent poster's point.

reb 4y ago

Another skill that'll be pretty valuable in this era is one many of the best technical leaders possess: the ability to coach desirable outcomes from squishy autonomous black boxes over time. Getting excellent, exact performance out of deterministic systems is an impressive feat, but autonomy means variability, and getting excellent performance out of variable systems (especially illegible ones) is a different game.

CSSer 4y ago
I agree 100%. I first had this thought last night when reading “Eight Things to Know about LLMs” (select quote from section 8 below)[0] > Simply prompting a model to “think step by step” can lead it to perform well on entire categories of math and reasoning problems that it would otherwise fail on (Kojima et al., 2022). Similarly, even observing that an LLM consistently fails at some task is far from sufficient evidence that no other LLM can do that task (Bowman, 2022). The eerie thing to me is that this is coaching. I have never coached anything that isn’t alive by any definition. My feelings about this realization are ambivalent. [0]: https://cims.nyu.edu/~sbowman/eightthings.pdf https://cims.nyu.edu/~sbowman/eightthings.pdf
- kristov 4y ago
  Doesn't prompting "think step by step" simply increase the probability the model will generate its output in the style of places where it saw "step by step" before (eg: tutorial content)? It's not really "deciding" to think step by step. Maybe I am mistaken.
  - hobs 4y ago
    You're not mistaken, its basically perturbing the seed for which the sentence grows.
  - pegasus 4y ago
    It's more than just that. The intermediary steps act as a kind of working memory, ensuring that the model is less likely to get lost in the weeds.

saghm 4y ago

This makes me wonder: will we start to see the reverse like we have in search engines? Will people start finding ways to do SEO for language models to try to get them to output stuff about their products more often?

spullara 4y ago
Yes. It is definitely a risk.
actionfromafar 4y ago
Sure, I bet they are already using LLMs to poison the web for other LLMs.
- PaulHoule 4y ago
  see https://arxiv.org/abs/2302.10149 https://arxiv.org/abs/2302.10149
grumbel 4y ago
They'll certainly try, but I am somewhat optimistic here, as you can feed the language model with some ground truths that allows it to detect the marketing nonsense. Another nice thing with language models is that they condense the information, it doesn't matter how many webpages there are for a product, if you ask ChatGPT about it, it will list it exactly once. It knows it is the same product each time and the list it produces will be created specifically for your query. Meanwhile Google gets cluttered with duplicate information, since your query is implicitly about webpages mentioning a product, not products themselves. That said, there is still plenty of work to do here. BingChat is completely unusable when it comes to product search, much worse than Google, as it will just pick the first three search results, which tend to be SEO spam, and summarize their content. ChatGPT is much better here, but its lack of direct Web access also handicaps it rather heavily.

ramraj07 4y ago

Exactly, though an addendum is not just their ability to search when the need arises, but also to know WHEN to resort to search, and with what angle. This is not just isolated to coding though, experienced the same in research as well. So much hinges on your ability to know when to search for literature and how.

visarga 4y ago
But LLM black boxes are pretty great at surfacing search leads. They might hallucinate, but they are much more precise semantically than search. LLMs understand when you say something in your own words. In fact one trick is to go to LLM to get the "closed-book" answer, and then use that answer to formulate a proper search query for Google. Q: What is the height of Everest. A: The height of Everest is 8800m Search: "The height of Everest is 8800m" Result: 8849m Using the generated answer as search phrase, even if it is factually wrong, works because it has the right structure and good keywords.

alkonaut 4y ago

I always felt my best asset as an engineer was my googling skills. I always also felt like this is almost cheating, but of course in a way (in most ways) it's not. I often solve problems for colleagues by returning a google response from the first page of search results. Meaning, most likely, that I had a better google query than they had. I wonder if this skill can become more useful in the age of "prompt engineering"? Will my googling skills be useless and I need to start over? Or will my ability to google naturally convert into a decent prompt-engineering skill given a reasonable amount of practice?

bryanrasmussen 4y ago
>Meaning, most likely, that I had a better google query than they had. there's two parts, good google query, but also figuring out which of the results returned is the most likely to be useful. I've often found results for people that were on the first page of the most obvious query in the world to start with, but they were not the first result but down a bit and maybe looked a bit weird.
factormeta 4y ago
Also keep in mind that your google query is bubbled, may be your search query is better than your colleagues because of your search history etc.
duggan 4y ago
I used to think this, but I'm not sure google-fu is the real superpower here. I've been surprised to encounter many developers who just don't seem to be able to read and comprehend the output of a program; logs, stack traces, etc. That's the basic — sometimes only — input to a search query. I suspect the ability to comprehend and describe a problem is a skill that translates quite smoothly to a world of prompt-based interaction.