5 ms·

Is it? I couldn't even get it to figure out an NGINX config, despite feeding it the documentation and URL rewrites it would need, and the prior Apache .htaccess

Is it? I couldn't even get it to figure out an NGINX config, despite feeding it the documentation and URL rewrites it would need, and the prior Apache .htaccess file. Spent days on it. Consulted with people who know more than me about prompt engineering. Nope. Reasoning is not its strong point, IMO. It's a next-word prediction model, why would it be? It's doing what an LLM does. Frequently, nonsensically.

LASR 4y ago
Interesting. Curious as to what specific task you gave it and how it failed. My team has been working on several advanced techniques using reasoning on LLMs. The stacked performance of all of these techniques combined yields is quite impressive.
andai 4y ago
GPT-3 or 4? Re: reasoning, apparently it gives much better performance if you ask it to write out its thought process step by step (because it has no "internal" thought process; it must be externalized).
- simonw 4y ago
  Yeah, that sounds like a GPT-3.5 problem to me. I wouldn't trust 3.5 to write me an Nginx config, but I'd give 4 a shot at it. I just tried using GPT-4 to translate a random htaccess file I found to nginx and it seemed to do a good initial job: https://gist.github.com/simonw/8174f653a0c08c120830c56332054ca3 https://gist.github.com/simonw/8174f653a0c08c120830c56332054...
  - andai 4y ago
    I tried your prompt and it works with 3.5 too, but I'm not qualified to verify the output (75% of it looks the same to me though).
throwaway675309 4y ago
I've actually had great success with the automation of boiler plate DevOps nonsense using GPT-4, including: GitHub actions, Caddy scripts, Docker files, etc. Every second that I didn't have to: - go look up a similar problem on Google - open three or four stack overflow tabs - read through the stack overflow links - copy out the answer - change all the variables to match variable names that I want All of this represents a huge time saver for me. I'm honestly baffled that people lack the ability to use LLMs in a productive and optimal manner.
- consp 4y ago
  Did you learn anything in the process? You'd still have to over the output and understand it, otherwise nothing has changed except the method of producing something you don't understand.
  - rolisz 4y ago
    If you're not DevOps, why bother learning whatever framework is used today, but which will be replaced by something else next year? I just want to get my stuff deployed and that's it.
    - ChatGTP 4y ago
      Yeah that’s true, why bother actually knowing anything about the systems they run your software /s
      - rolisz 4y ago
        I have a finite amount of time. I can choose to learn and know more about the software that I'm developing and make the software better, or I can spend some amount of time to learn the DevOps part, or some mix in between. In some cases, the ratio is 90:10 in favor of software development.
        ChatGTP 4y ago
SanderNL 4y ago
> It's a next-word prediction model, why would it be? If you want to make accurate predictions, you'll need reasoning or at least some process that approximates it. "There a no bananas. Max is looking for bananas. Will he be successful: [yes/no]"
deleted 4y ago
[deleted]
burnished 4y ago
Interesting! I found it super helpful with gitlab-ci stuff and docker, as well as much bash and makefile stuff. I would have assumed it would handle NGINX configurations. Tbh, it isn't surprising that it fails, its surprising that it has subjects where it is very useful.
dmd 4y ago
> It's a next-word prediction model So are you. https://ceoln.wordpress.com/2023/03/31/its-just-predicting-the-next-word-well/ https://ceoln.wordpress.com/2023/03/31/its-just-predicting-t...
- gbjw 4y ago
  Maybe you are, but I'm not :).
- hammyhavoc 4y ago
  And your qualifications and background are what? Why is this on a free WordPress instead of in a scientific paper if it's something to take seriously? Counterpoint: https://the-decoder.com/why-large-ai-language-models-dont-lead-to-human-like-ai/ https://the-decoder.com/why-large-ai-language-models-dont-le...
  - dmd 4y ago
    Do you think I wrote that? And what do qualifications have to do with anything?
    - hammyhavoc 4y ago
      I assumed you did, as it's a very strange post to link to on a very strange blog, and doesn't lend credibility to the idea that a human being is an LLM simply by existing, and if it was such a compelling idea then it wouldn't be languishing on a free WordPress. Qualifications are a great way of figuring out who's worth listening to, but that's not always the case as AI/ML folks prove with regularity, but it's nice to know if someone isn't just an armchair commentator and isn't peddling baseless information.
    - dmd 4y ago
      > a human being is an LLM simply by existing, That's not what it's claiming at all. It's taking umbrage with the word "just" in "just predicting the next word". A gun "just moves some metal from one place to another".
mlboss 4y ago
Are you using GPT 4 ? It is far superior when compared to GPT 3.5
vaidhy 4y ago
I will paraphrase Geoff Hinton here - Predicting next word requires understanding context. Otherwise, you cannot predict the next word. It is a big deal.

LASR 4y ago

Interesting. Curious as to what specific task you gave it and how it failed. My team has been working on several advanced techniques using reasoning on LLMs. The stacked performance of all of these techniques combined yields is quite impressive.

andai 4y ago

GPT-3 or 4? Re: reasoning, apparently it gives much better performance if you ask it to write out its thought process step by step (because it has no "internal" thought process; it must be externalized).

simonw 4y ago
Yeah, that sounds like a GPT-3.5 problem to me. I wouldn't trust 3.5 to write me an Nginx config, but I'd give 4 a shot at it. I just tried using GPT-4 to translate a random htaccess file I found to nginx and it seemed to do a good initial job: https://gist.github.com/simonw/8174f653a0c08c120830c56332054ca3 https://gist.github.com/simonw/8174f653a0c08c120830c56332054...
- andai 4y ago
  I tried your prompt and it works with 3.5 too, but I'm not qualified to verify the output (75% of it looks the same to me though).

throwaway675309 4y ago

I've actually had great success with the automation of boiler plate DevOps nonsense using GPT-4, including: GitHub actions, Caddy scripts, Docker files, etc. Every second that I didn't have to: - go look up a similar problem on Google - open three or four stack overflow tabs - read through the stack overflow links - copy out the answer - change all the variables to match variable names that I want All of this represents a huge time saver for me. I'm honestly baffled that people lack the ability to use LLMs in a productive and optimal manner.

consp 4y ago
Did you learn anything in the process? You'd still have to over the output and understand it, otherwise nothing has changed except the method of producing something you don't understand.
- rolisz 4y ago
  If you're not DevOps, why bother learning whatever framework is used today, but which will be replaced by something else next year? I just want to get my stuff deployed and that's it.
  - ChatGTP 4y ago
    Yeah that’s true, why bother actually knowing anything about the systems they run your software /s
    - rolisz 4y ago
      I have a finite amount of time. I can choose to learn and know more about the software that I'm developing and make the software better, or I can spend some amount of time to learn the DevOps part, or some mix in between. In some cases, the ratio is 90:10 in favor of software development.
      - ChatGTP 4y ago

SanderNL 4y ago

> It's a next-word prediction model, why would it be? If you want to make accurate predictions, you'll need reasoning or at least some process that approximates it. "There a no bananas. Max is looking for bananas. Will he be successful: [yes/no]"

deleted 4y ago

[deleted]

burnished 4y ago

Interesting! I found it super helpful with gitlab-ci stuff and docker, as well as much bash and makefile stuff. I would have assumed it would handle NGINX configurations. Tbh, it isn't surprising that it fails, its surprising that it has subjects where it is very useful.

dmd 4y ago

> It's a next-word prediction model So are you. https://ceoln.wordpress.com/2023/03/31/its-just-predicting-the-next-word-well/ https://ceoln.wordpress.com/2023/03/31/its-just-predicting-t...

gbjw 4y ago
Maybe you are, but I'm not :).
hammyhavoc 4y ago
And your qualifications and background are what? Why is this on a free WordPress instead of in a scientific paper if it's something to take seriously? Counterpoint: https://the-decoder.com/why-large-ai-language-models-dont-lead-to-human-like-ai/ https://the-decoder.com/why-large-ai-language-models-dont-le...
- dmd 4y ago
  Do you think I wrote that? And what do qualifications have to do with anything?
  - hammyhavoc 4y ago
    I assumed you did, as it's a very strange post to link to on a very strange blog, and doesn't lend credibility to the idea that a human being is an LLM simply by existing, and if it was such a compelling idea then it wouldn't be languishing on a free WordPress. Qualifications are a great way of figuring out who's worth listening to, but that's not always the case as AI/ML folks prove with regularity, but it's nice to know if someone isn't just an armchair commentator and isn't peddling baseless information.
  - dmd 4y ago
    > a human being is an LLM simply by existing, That's not what it's claiming at all. It's taking umbrage with the word "just" in "just predicting the next word". A gun "just moves some metal from one place to another".

mlboss 4y ago

Are you using GPT 4 ? It is far superior when compared to GPT 3.5

vaidhy 4y ago

I will paraphrase Geoff Hinton here - Predicting next word requires understanding context. Otherwise, you cannot predict the next word. It is a big deal.