5 ms·

For prototyping Chat-GPT (4.0) is a big deal. I had it write PoCs of pretty complex systems combining different state-of-the art technologies, and it mastered a

For prototyping Chat-GPT (4.0) is a big deal. I had it write PoCs of pretty complex systems combining different state-of-the art technologies, and it mastered all of it brilliantly. What amazes me most is how simple and elegant its solutions are. When you ask something (e.g. build a horizontally scalable data store with a Raft-based consensus mechanism in Rust) it produces code that does just that and nothing else. Lots of programmers, myself included, would struggle with this and include at least some extraneous complexity, but Chat-GPT goes right for what was asked and seems to find the most succint way to achieve it. I haven't felt so excited about programming in a long time, because I can now build PoCs that would take me days or weeks to do in an hour or even less. That's a real game changer, and it helps me overcome this internal friction of "I know this is possible to build but it will take me a long time to figure out an actual approach".

bluejekyll 4y ago
Out of curiosity, how do you know that it’s correct?
- ThePhysicist 4y ago
  Reading the code, and running it. You can ask it for a complete project tree and even test scripts (you have to ask it to continue as it gets interrupted sometimes). I e.g. had it write a simple database engine with an API, and it produced Rust code as well as curl-based examples to test the database. It even correctly added up the data items it sent into the DB via curl to show what the output would probably look like. Pure sci-fi technology.
- bipop5000 4y ago
  Test and reiterate. I don't think you should take whatever is coming out of chatGPT verbatim. It should be used as an interactive tool to amplify your knowledge.
  - mjrpes 4y ago
    GPT-4 for coding is very powerful but can be dangerous. I asked it to write a data retention script that would save X number of backups depending on the number of days/weeks/month/years from the present day. It's first attempt would have worked the first day, but running the script consecutive days would eventually delete all backups. Once I told GPT-4 to take into account the script would be run each day, it "apologized" and produced a script that worked.
  - layer8 4y ago
    Testing does not show correctness, it can only show incorrectness.
- whatever1 4y ago
  Same way we know that any piece of code is correct. Debugging, testing.
- YetAnotherNick 4y ago
  Similar to how you know someone's stackoverflow answer is correct. Read the code, find the documentation of API calls, think of any missed cases, write tests etc.
  - thomastjeffery 4y ago
    So basically the same as writing it yourself: just skipping the writing part. So should we be taking about GPT as an occasional text editor replacement? I honestly think that's more accurate take than most of the ones I have seen.
    - mgfist 4y ago
      Have you seen the GPT4 demo? A few interesting things that make it seem much more than an occasional text editor replacement. One was having it write an entire program. It needed a few prompts to fix some bugs it had, but it worked. The program was then used for the rest of the demo. The program even had an obscure bug that it solved by having the demo-er paste the error + the entire docs page. The second demo was having it write a html page based on a picture drawing on a napkin of a website, and then running the html. It's honestly pretty impressive. I already use chatgpt a lot (doing a lot of google sheets scripting/formula stuff and the docs + syntax are horrendous) and chatgpt helps me find the correct syntax much faster than I'd otherwise be able to
      - thomastjeffery 4y ago
        There is a huge difference between writing a program, and writing the program you want right now. It's tricky to conceptualize because the entire narrative we are familiar with has personified GPT. It may be impressive, but GPT is completely different from humans. If a human is capable of writing a program, that is because they understand conceptually how and why. GPT doesn't. GPT's ability to write a program is entirely dependent on the content of its training corpus: no how, no why, only what. I'll put it another way: If a human is not able to write a program, it is because they don't understand conceptually how. If GPT is not able to write a program, it is because either the desired parts, or the necessary pattern that puts them together, does not exist in the training corpus; or because the prompt didn't yield a continuation that followed that arbitrary pattern. The results are impressive because humans are impressive. GPT doesn't interact with the domain of all possible written text. It deals with the domain of all possible patterns of tokens from the text it was given. It's only given text that was intentionally written by humans. It exists in a world of signal: no noise. It can still only guess, but the guess will always be constructed out of the category of text humans choose to write. Inference models are a completely different approach to language, and confusing them with human behavior is an easy way to make impossible predictions about what they can and cannot accomplish.
- eloff 4y ago
  The author addressed that rather at length in the article. Don’t. Trust. Code. That it originated from ChatGPT instead of any number of other sources doesn’t change that.
Vanclief 4y ago
Could you provide an example of this? What kind of prompts are you using, and what does a complex system of different state-of-the art technology would look like? Sincere question as I don't get how to use it for things larger than simple scripts.
- ThePhysicist 4y ago
  You just talk to it like you would talk to another engineer. I e.g. first describe what I want to achieve and ask it to give me an architecture. I then e.g. ask to break down the work into different smaller tasks, and ask it to do each task, adding to what it wrote before. A bit hard to reproduce all prompts here and the UI doesn't allow to share, but it comes quite natural I find. Sometimes you need to ask it to continue as the output just stops, but other than that it's like talking to a colleague (with superhuman powers).
  - sumtechguy 4y ago
    I usually ask it to describe the concept first. To make sure it is in the right space. Then I say something like "please write a c++ version of that". It will do a decent job at it. Last night a friend who refuses to get an account to use it was having an issue with a cisco router. I put in what he was trying to do. It was not right and gave him an error. I fed that error back in and it realized he had a different version and gave me a better way to do exactly what he wanted. It had kept the context and said 'oh some routers do not have that command here is another way to do it'. He had spent weeks googling around for the answer. I had it in under a half hour and I had never used the interface he was changing before. Then you can turn around and ask it to write a horror story about a monster that devours couches. It will make something up (it is very good at that). Then you can say 'oh put that in the style of the SCP foundation'. It will. This tool is wildly interesting. I think dismissing it is a bad idea. I look forward to using this thing as it gets better.
    - deleted 4y ago
      [deleted]
  - 4y ago
drstewart 4y ago
How is it ChatGPT 4 is like a few days old and people are reviewing its ability to "build complex systems with state of the art technologies" like they've been using it for years? Reminds me of "Seeking developer with 10 years of experience in <brand new> framework"
- raincole 4y ago
  > How is it ChatGPT 4 is like a few days old and people are reviewing its ability to "build complex systems with state of the art technologies" like they've been using it for years? Why... why can't we reviewing its ability to "build complex system" because it's a few days old...? I'm pretty sure here "complex" is relative to what ChatGPT can do before, not relative to, say, what NASA did before. > Reminds me of "Seeking developer with 10 years of experience in <brand new> framework" AI is not human. The anthropomorphism is quite crazy here.
  - drstewart 4y ago
    >AI is not human. The anthropomorphism is quite crazy here. You misunderstand. My point isn't that ChatGPT is the developer here, it's that ChatGPT is the "framework".
yosito 4y ago
I spent half an hour today trying to get ChatGPT to write a simple anchor tag in HTML and didn't get a single correct response. I'd love to see a screencast video from one of you people claiming that ChatGPT is actually returning huge amounts of useful code that is ready to use. I'm incredibly skeptical!
- ThrowawayTestr 4y ago
  Input: Give an example of anchor tags in html Output: Sure, here is an example of how to create an anchor tag in HTML: <a href="https://www.example.com">Click https://www.example.com">Click me!</a> In this example, the text "Click me!" will be displayed as a hyperlink. When a user clicks on the hyperlink, they will be redirected to the website specified in the href attribute, which in this case is https://www.example.com https://www.example.com.
- stnmtn 4y ago
  I would love to see a screencast video from you of what you asked ChatGPT and what it responded with, as I find it hard to believe that it couldn't write an anchor tag.

bluejekyll 4y ago

Out of curiosity, how do you know that it’s correct?

ThePhysicist 4y ago
Reading the code, and running it. You can ask it for a complete project tree and even test scripts (you have to ask it to continue as it gets interrupted sometimes). I e.g. had it write a simple database engine with an API, and it produced Rust code as well as curl-based examples to test the database. It even correctly added up the data items it sent into the DB via curl to show what the output would probably look like. Pure sci-fi technology.
bipop5000 4y ago
Test and reiterate. I don't think you should take whatever is coming out of chatGPT verbatim. It should be used as an interactive tool to amplify your knowledge.
- mjrpes 4y ago
  GPT-4 for coding is very powerful but can be dangerous. I asked it to write a data retention script that would save X number of backups depending on the number of days/weeks/month/years from the present day. It's first attempt would have worked the first day, but running the script consecutive days would eventually delete all backups. Once I told GPT-4 to take into account the script would be run each day, it "apologized" and produced a script that worked.
- layer8 4y ago
  Testing does not show correctness, it can only show incorrectness.
whatever1 4y ago
Same way we know that any piece of code is correct. Debugging, testing.
YetAnotherNick 4y ago
Similar to how you know someone's stackoverflow answer is correct. Read the code, find the documentation of API calls, think of any missed cases, write tests etc.
- thomastjeffery 4y ago
  So basically the same as writing it yourself: just skipping the writing part. So should we be taking about GPT as an occasional text editor replacement? I honestly think that's more accurate take than most of the ones I have seen.
  - mgfist 4y ago
    Have you seen the GPT4 demo? A few interesting things that make it seem much more than an occasional text editor replacement. One was having it write an entire program. It needed a few prompts to fix some bugs it had, but it worked. The program was then used for the rest of the demo. The program even had an obscure bug that it solved by having the demo-er paste the error + the entire docs page. The second demo was having it write a html page based on a picture drawing on a napkin of a website, and then running the html. It's honestly pretty impressive. I already use chatgpt a lot (doing a lot of google sheets scripting/formula stuff and the docs + syntax are horrendous) and chatgpt helps me find the correct syntax much faster than I'd otherwise be able to
    - thomastjeffery 4y ago
      There is a huge difference between writing a program, and writing the program you want right now. It's tricky to conceptualize because the entire narrative we are familiar with has personified GPT. It may be impressive, but GPT is completely different from humans. If a human is capable of writing a program, that is because they understand conceptually how and why. GPT doesn't. GPT's ability to write a program is entirely dependent on the content of its training corpus: no how, no why, only what. I'll put it another way: If a human is not able to write a program, it is because they don't understand conceptually how. If GPT is not able to write a program, it is because either the desired parts, or the necessary pattern that puts them together, does not exist in the training corpus; or because the prompt didn't yield a continuation that followed that arbitrary pattern. The results are impressive because humans are impressive. GPT doesn't interact with the domain of all possible written text. It deals with the domain of all possible patterns of tokens from the text it was given. It's only given text that was intentionally written by humans. It exists in a world of signal: no noise. It can still only guess, but the guess will always be constructed out of the category of text humans choose to write. Inference models are a completely different approach to language, and confusing them with human behavior is an easy way to make impossible predictions about what they can and cannot accomplish.
eloff 4y ago
The author addressed that rather at length in the article. Don’t. Trust. Code. That it originated from ChatGPT instead of any number of other sources doesn’t change that.

Vanclief 4y ago

Could you provide an example of this? What kind of prompts are you using, and what does a complex system of different state-of-the art technology would look like? Sincere question as I don't get how to use it for things larger than simple scripts.

ThePhysicist 4y ago
You just talk to it like you would talk to another engineer. I e.g. first describe what I want to achieve and ask it to give me an architecture. I then e.g. ask to break down the work into different smaller tasks, and ask it to do each task, adding to what it wrote before. A bit hard to reproduce all prompts here and the UI doesn't allow to share, but it comes quite natural I find. Sometimes you need to ask it to continue as the output just stops, but other than that it's like talking to a colleague (with superhuman powers).
- sumtechguy 4y ago
  I usually ask it to describe the concept first. To make sure it is in the right space. Then I say something like "please write a c++ version of that". It will do a decent job at it. Last night a friend who refuses to get an account to use it was having an issue with a cisco router. I put in what he was trying to do. It was not right and gave him an error. I fed that error back in and it realized he had a different version and gave me a better way to do exactly what he wanted. It had kept the context and said 'oh some routers do not have that command here is another way to do it'. He had spent weeks googling around for the answer. I had it in under a half hour and I had never used the interface he was changing before. Then you can turn around and ask it to write a horror story about a monster that devours couches. It will make something up (it is very good at that). Then you can say 'oh put that in the style of the SCP foundation'. It will. This tool is wildly interesting. I think dismissing it is a bad idea. I look forward to using this thing as it gets better.
  - deleted 4y ago
    [deleted]
- 4y ago

drstewart 4y ago

How is it ChatGPT 4 is like a few days old and people are reviewing its ability to "build complex systems with state of the art technologies" like they've been using it for years? Reminds me of "Seeking developer with 10 years of experience in <brand new> framework"

raincole 4y ago
> How is it ChatGPT 4 is like a few days old and people are reviewing its ability to "build complex systems with state of the art technologies" like they've been using it for years? Why... why can't we reviewing its ability to "build complex system" because it's a few days old...? I'm pretty sure here "complex" is relative to what ChatGPT can do before, not relative to, say, what NASA did before. > Reminds me of "Seeking developer with 10 years of experience in <brand new> framework" AI is not human. The anthropomorphism is quite crazy here.
- drstewart 4y ago
  >AI is not human. The anthropomorphism is quite crazy here. You misunderstand. My point isn't that ChatGPT is the developer here, it's that ChatGPT is the "framework".

yosito 4y ago

I spent half an hour today trying to get ChatGPT to write a simple anchor tag in HTML and didn't get a single correct response. I'd love to see a screencast video from one of you people claiming that ChatGPT is actually returning huge amounts of useful code that is ready to use. I'm incredibly skeptical!

ThrowawayTestr 4y ago
Input: Give an example of anchor tags in html Output: Sure, here is an example of how to create an anchor tag in HTML: <a href="https://www.example.com">Click https://www.example.com">Click me!</a> In this example, the text "Click me!" will be displayed as a hyperlink. When a user clicks on the hyperlink, they will be redirected to the website specified in the href attribute, which in this case is https://www.example.com https://www.example.com.
stnmtn 4y ago
I would love to see a screencast video from you of what you asked ChatGPT and what it responded with, as I find it hard to believe that it couldn't write an anchor tag.