4 ms·

Do you have any evidence besides anecdote?

by th0ma5 2y ago

anon291 2y ago
what kind of evidence substantiates creativity? Things I've used chat gpt for: 1. writing songs (couldn't find the generated lyrics online, so assume it's new) 2. Branding ideas (again couldn't find the logos online, so assuming they're new) 3. Recipes (with weird ingredients that I've not found put together online) 4. Vacations with lots of constraints (again, all the information is obviously available online, but it put it together for me and gave recommendations for my family particularly). 5. Theoretical physics explorations where I'm too lazy to write out the math (and why should I... chatgpt will do it for me...) I think perhaps one reason people here do not have the same results is I typically use the API directly and modify the system prompt, which drastically changes the utility of chatgpt. The default prompt is too focused on retrieval and 'truth'. If you want creativity you have to ask it to be an artist.
- th0ma5 2y ago
  No I think they don't have the results you do because they are trying to do those things well ...
  - anon291 2y ago
    The personal insult insinuated here is not appreciated and probably against community guidelines. For what I needed, those things worked very well
    - th0ma5 2y ago
      Anecdotes have equal weight. All of these models frustrate me to no end but I only do things that have never been done before. And it isn't an insult because you have no evidence of quality.
      - anon291 2y ago
        > Anecdotes have equal weight. All of these models frustrate me to no end but I only do things that have never been done before. And it isn't an insult because you have no evidence of quality. You have not specified what evidence would satisfy you. And yes, it was an insult to insinuate I would accept sub par results whereas others would not. EDIT: Chat GPT seems to have a solid understanding of why your comment comes across as insulting: https://chatgpt.com/share/673b95c9-7a98-8010-9f8a-9abf5374bbeb https://chatgpt.com/share/673b95c9-7a98-8010-9f8a-9abf5374bb... Maybe this should be taken as one point of evidence of greater ability?
        th0ma5 2y ago
        I think you lead the result by not providing enough context like saying how there is no objective way to measure the quality of an LLM generation after the fact nor before. Edit I asked ChatGPT with a more proper context: "It’s not inherently insulting to say that an LLM (Large Language Model) cannot guarantee the best quality because it’s a factual statement grounded in the nature of how these models work. LLMs rely on patterns in their training data and probabilistic reasoning rather than subjective or objective judgments about "best quality."
        anon291 2y ago
        I can't criticize how you prompted it because you did not link the transcript :) Zooming out, you seem to be in the wrong conversation. I said: > the LLM can solve a general problem (or tell you why it cannot), while your calculator can only do that which it's been programmed. You said: > Do you have any evidence besides anecdote? I think that -- for both of us now having used chat gpt to generate a response -- we have good evidence that the model can solve a general program (or tell you why it cannot), while a calculator can only do the arithmetic for which it's been programmed. If you want to counter, then a video of your calculator answering the question we just posed would be nice.

Do you have any evidence besides anecdote?

by th0ma5 2y ago

Do you have any evidence besides anecdote?

what kind of evidence substantiates creativity? Things I've used chat gpt for: 1. writing songs (couldn't find the generated lyrics online, so assume it's new) 2. Branding ideas (again couldn't find the logos online, so assuming they're new) 3. Recipes (with weird ingredients that I've not found put together online) 4. Vacations with lots of constraints (again, all the information is obviously available online, but it put it together for me and gave recommendations for my family particularly). 5. Theoretical physics explorations where I'm too lazy to write out the math (and why should I... chatgpt will do it for me...) I think perhaps one reason people here do not have the same results is I typically use the API directly and modify the system prompt, which drastically changes the utility of chatgpt. The default prompt is too focused on retrieval and 'truth'. If you want creativity you have to ask it to be an artist.

th0ma5 2y ago
No I think they don't have the results you do because they are trying to do those things well ...
- anon291 2y ago
  The personal insult insinuated here is not appreciated and probably against community guidelines. For what I needed, those things worked very well
  - th0ma5 2y ago
    Anecdotes have equal weight. All of these models frustrate me to no end but I only do things that have never been done before. And it isn't an insult because you have no evidence of quality.
    - anon291 2y ago
      > Anecdotes have equal weight. All of these models frustrate me to no end but I only do things that have never been done before. And it isn't an insult because you have no evidence of quality. You have not specified what evidence would satisfy you. And yes, it was an insult to insinuate I would accept sub par results whereas others would not. EDIT: Chat GPT seems to have a solid understanding of why your comment comes across as insulting: https://chatgpt.com/share/673b95c9-7a98-8010-9f8a-9abf5374bbeb https://chatgpt.com/share/673b95c9-7a98-8010-9f8a-9abf5374bb... Maybe this should be taken as one point of evidence of greater ability?
      - th0ma5 2y ago
        I think you lead the result by not providing enough context like saying how there is no objective way to measure the quality of an LLM generation after the fact nor before. Edit I asked ChatGPT with a more proper context: "It’s not inherently insulting to say that an LLM (Large Language Model) cannot guarantee the best quality because it’s a factual statement grounded in the nature of how these models work. LLMs rely on patterns in their training data and probabilistic reasoning rather than subjective or objective judgments about "best quality."
        anon291 2y ago
        I can't criticize how you prompted it because you did not link the transcript :) Zooming out, you seem to be in the wrong conversation. I said: > the LLM can solve a general problem (or tell you why it cannot), while your calculator can only do that which it's been programmed. You said: > Do you have any evidence besides anecdote? I think that -- for both of us now having used chat gpt to generate a response -- we have good evidence that the model can solve a general program (or tell you why it cannot), while a calculator can only do the arithmetic for which it's been programmed. If you want to counter, then a video of your calculator answering the question we just posed would be nice.