4 ms·

Hard to believe. I don't know any programmer who uses ChatGPT on a regular basis for programming. How can anyone rely on a thing that frequently isn't available

Hard to believe. I don't know any programmer who uses ChatGPT on a regular basis for programming. How can anyone rely on a thing that frequently isn't available because high load, and takes tens of seconds to generate a full answer. Secondly, for most of the programming work, to be useful, it has to be able to understand the query but also the existing codebase—"This is my codebase. What do I need to do to paginate the users table?". Would love to hear real-world programming examples of something that would be faster done with ChatGPT vs Google/Github Search. What are they?

Clubber 4y ago
Yes, this is hard to believe for me too. This thing just came out recently didn't it? How can you have a team completely dependent on this already? What did they do before this came out, stand around and stare at the walls?
- Calavar 4y ago
  It's been less than three months. I too have trouble believing that anyone's pre-Chat GPT skills and knowledge could atrophy to that degree in that amount of time.
varispeed 4y ago
It's quicker than search. For instance I have a set of data that I want to make sense of before I can do something with it. I can't remember from top of my head how to do things like calculate standard deviation or how to fit given array to a curve or how to visualise the data in this and that way. I can get these code snippets from ChatGPT in seconds and they 90% of the time work. That's just one of the example, but ChatGPT has reduced my time Googling for something by like 80%.
kunalgupta 4y ago
paying for plus solves most of this
simonw 4y ago
I use ChatGPT all the time to generate quick example code, purely as a faster version of "Google for this API, then this other one, then figure out how to glue that together in a script". I also use it for things like constructing jq recipes, or zsh for loops (I still can't remember how to do those without looking them up). Recent jq example: https://fedi.simonwillison.net/@simon/109882602515468465 https://fedi.simonwillison.net/@simon/109882602515468465 - see also https://til.simonwillison.net/gpt3/jq https://til.simonwillison.net/gpt3/jq Here's an example where I used it as a search engine and it taught me about an entirely new (to me) tool: https://til.simonwillison.net/macos/sips https://til.simonwillison.net/macos/sips
- 28304283409234 4y ago
  Yup. I used to read manpages and skip straight to the examples. Don't see much difference other than less frustration.
- skugeTO 4y ago
  That’s it! Chat GPT excels at reading documentation and writing prototypes of code that uses libraries, APIs, FFIs or all of them together! It saves so much time not to have to figure out how to instantiate that random call that wasn’t mentioned in the documentation and whose code is obscured by objectionable engineering. However I don’t think any line of chatGPTs code has yet made it to my codebase unchanged
_fat_santa 4y ago
Call this a cautionary tale but I tried using ChatGPT yesterday to help me with some work. I was converting code in a an old lambda authorizer and it used a library called "jose" for managing JWTs, JWKs, JWEs, etc. Long story short the existing lambda used version 1.x and I wanted to use version 4.x so I wrote a prompt for ChatGPT and got it to convert the code. At first I was amazed that it could figure it out, until I started to scrutinize the code and realized it was entirely incorrect. It didn't produce a correct answer, just garbage that looks like a correct answer. I assume ChatGPT will be better for more "broad" topics with more training data but in this scenario it totally fell on it's face.
- zamnos 4y ago
  I'll second that. I tested ChatGPT and asked it how I could export my tailscale config and it told me to use the tailscale export command, which all well and good - except it doesn't exist. (You want /var/lib/tailscale/tailscaled.state) Hard to keep using it after that. asked ChatGPT how to export my tailscale config and it told me Whether you say ChatGPT is lying or say it's spewing bullshit or a hallucination, it's just plain wrong.
  - simonw 4y ago
    Part of the challenge of using ChatGPT is developing instincts about whether it will do well on a specific question. Tailscale is too new: ChatGPT's training is unlikely to know much if anything about it. I wouldn't be surprised if Bing could handle this much better, since it has the ability to search for current documentation.
creek567 4y ago
for me chatgpt is 10x more efficient than searching stack overflow. I use it daily and helps speed my dev so much. It's super helpful for tedious stuff like date transforms (pst to utc) or random array filters (ex filter array of objects on this field and remove duplicates) etc. My most recent query from yesterday ways "iterate over a nodejs array, the array has 10 or less elements. The increment should be in chunks of two. Stop when the index is bigger than 10, or when there are no more elements" I can figure this stuff out myself too, but with chatgpt, I can query it and come back in 20 seconds and use whatever the output is. 95% of the time is correct, and the rest I can just quickly fix it myself. Plus i can ask follow on questions and refine it even more. This stuff would have taken me 5+ mins looking over stack overflow and then coding it and testing it. Plus ChatGpt handles edge case so I don't have to worry about them. The efficiency is much much better than SO. I can't imagine going back to Google or SO for any coding questions. I only wish they kept the service up more during the day time. It's available in the morning or at night, but most afternoons it's a hit or miss.
- sidlls 4y ago
  The response time is longer than it would have taken you to just write the code.
  - maze-le 4y ago
    Even writing the statement in natural language takes longer than writing the code...
  - creek567 4y ago
    the point is not the speed, but the brainspace. the brainspace I would have used on this task would have tired me out quicker and prevented me from making progress on other tasks. ChatGPT is literally 0 brain space for these simple tasks. If you had to code it you're at least thinking about edge cases etc. Plus there's 0% chance it would take you 20 seconds for 10+ tasks of these per day. if you're coding at 50 wmp these would take you at least 1 minute to type out, let alone implement correctly. by your logic you should never use google or SO because it's quicker to just implement it instead of opening a browser, typing the question, looking at responses etc.
broast 4y ago
It is pretty simple to summarize the constraints of your codebase and ask for a conceptual solution, I find this much more useful than generating code. Similarly, I've saved hours or days on bug investigations by describing the bug and letting gpt guess why I couldn't reproduce it, and it was right. I can certainly believe a significant productivity drop.
jwestbury 4y ago
I've been using it routinely this week, because I've recently started in a role with a team who leverages pandas extensively, but I have very little pandas experience. I've been asking ChatGPT how to do things as a faster approach than Googling the same information on a number of occasions. I can see myself becoming used to this approach VERY quickly.

Clubber 4y ago

Yes, this is hard to believe for me too. This thing just came out recently didn't it? How can you have a team completely dependent on this already? What did they do before this came out, stand around and stare at the walls?

Calavar 4y ago
It's been less than three months. I too have trouble believing that anyone's pre-Chat GPT skills and knowledge could atrophy to that degree in that amount of time.

varispeed 4y ago

It's quicker than search. For instance I have a set of data that I want to make sense of before I can do something with it. I can't remember from top of my head how to do things like calculate standard deviation or how to fit given array to a curve or how to visualise the data in this and that way. I can get these code snippets from ChatGPT in seconds and they 90% of the time work. That's just one of the example, but ChatGPT has reduced my time Googling for something by like 80%.

kunalgupta 4y ago

paying for plus solves most of this

simonw 4y ago

I use ChatGPT all the time to generate quick example code, purely as a faster version of "Google for this API, then this other one, then figure out how to glue that together in a script". I also use it for things like constructing jq recipes, or zsh for loops (I still can't remember how to do those without looking them up). Recent jq example: https://fedi.simonwillison.net/@simon/109882602515468465 https://fedi.simonwillison.net/@simon/109882602515468465 - see also https://til.simonwillison.net/gpt3/jq https://til.simonwillison.net/gpt3/jq Here's an example where I used it as a search engine and it taught me about an entirely new (to me) tool: https://til.simonwillison.net/macos/sips https://til.simonwillison.net/macos/sips

28304283409234 4y ago
Yup. I used to read manpages and skip straight to the examples. Don't see much difference other than less frustration.
skugeTO 4y ago
That’s it! Chat GPT excels at reading documentation and writing prototypes of code that uses libraries, APIs, FFIs or all of them together! It saves so much time not to have to figure out how to instantiate that random call that wasn’t mentioned in the documentation and whose code is obscured by objectionable engineering. However I don’t think any line of chatGPTs code has yet made it to my codebase unchanged

_fat_santa 4y ago

Call this a cautionary tale but I tried using ChatGPT yesterday to help me with some work. I was converting code in a an old lambda authorizer and it used a library called "jose" for managing JWTs, JWKs, JWEs, etc. Long story short the existing lambda used version 1.x and I wanted to use version 4.x so I wrote a prompt for ChatGPT and got it to convert the code. At first I was amazed that it could figure it out, until I started to scrutinize the code and realized it was entirely incorrect. It didn't produce a correct answer, just garbage that looks like a correct answer. I assume ChatGPT will be better for more "broad" topics with more training data but in this scenario it totally fell on it's face.

zamnos 4y ago
I'll second that. I tested ChatGPT and asked it how I could export my tailscale config and it told me to use the tailscale export command, which all well and good - except it doesn't exist. (You want /var/lib/tailscale/tailscaled.state) Hard to keep using it after that. asked ChatGPT how to export my tailscale config and it told me Whether you say ChatGPT is lying or say it's spewing bullshit or a hallucination, it's just plain wrong.
- simonw 4y ago
  Part of the challenge of using ChatGPT is developing instincts about whether it will do well on a specific question. Tailscale is too new: ChatGPT's training is unlikely to know much if anything about it. I wouldn't be surprised if Bing could handle this much better, since it has the ability to search for current documentation.

creek567 4y ago

for me chatgpt is 10x more efficient than searching stack overflow. I use it daily and helps speed my dev so much. It's super helpful for tedious stuff like date transforms (pst to utc) or random array filters (ex filter array of objects on this field and remove duplicates) etc. My most recent query from yesterday ways "iterate over a nodejs array, the array has 10 or less elements. The increment should be in chunks of two. Stop when the index is bigger than 10, or when there are no more elements" I can figure this stuff out myself too, but with chatgpt, I can query it and come back in 20 seconds and use whatever the output is. 95% of the time is correct, and the rest I can just quickly fix it myself. Plus i can ask follow on questions and refine it even more. This stuff would have taken me 5+ mins looking over stack overflow and then coding it and testing it. Plus ChatGpt handles edge case so I don't have to worry about them. The efficiency is much much better than SO. I can't imagine going back to Google or SO for any coding questions. I only wish they kept the service up more during the day time. It's available in the morning or at night, but most afternoons it's a hit or miss.

sidlls 4y ago
The response time is longer than it would have taken you to just write the code.
- maze-le 4y ago
  Even writing the statement in natural language takes longer than writing the code...
- creek567 4y ago
  the point is not the speed, but the brainspace. the brainspace I would have used on this task would have tired me out quicker and prevented me from making progress on other tasks. ChatGPT is literally 0 brain space for these simple tasks. If you had to code it you're at least thinking about edge cases etc. Plus there's 0% chance it would take you 20 seconds for 10+ tasks of these per day. if you're coding at 50 wmp these would take you at least 1 minute to type out, let alone implement correctly. by your logic you should never use google or SO because it's quicker to just implement it instead of opening a browser, typing the question, looking at responses etc.

broast 4y ago

It is pretty simple to summarize the constraints of your codebase and ask for a conceptual solution, I find this much more useful than generating code. Similarly, I've saved hours or days on bug investigations by describing the bug and letting gpt guess why I couldn't reproduce it, and it was right. I can certainly believe a significant productivity drop.

jwestbury 4y ago

I've been using it routinely this week, because I've recently started in a role with a team who leverages pandas extensively, but I have very little pandas experience. I've been asking ChatGPT how to do things as a faster approach than Googling the same information on a number of occasions. I can see myself becoming used to this approach VERY quickly.