7 ms·

ChatGPT4 writes Stan code so I don’t have to

70 points by luu 4y ago

cs702 4y ago
This is a really useful review of what it's like to use LLMS for writing code. The key insight, for me, is this: > Working with ChatGPT4 for this task was like working with a good intern programmer who is rather forgetful and a bit scattered. And the key take-away -- again, for me -- is this: > But: This has already changed the way I code. I am not sure I am ever going to write code completely by hand again! And I expect these tools to get better fast.
- bobmaxup 4y ago
  I use LLMs in much the same way, but lately I have been wondering how good I will continue to be at thinking, verifying, or communicating in code, if I stop writing as much of it. I suppose time will tell...
  - zarzavat 4y ago
    I’ve been using Copilot since the beta and it does cause dependency, which can be dangerous because you have to be constantly vigilant of errors it might be introducing. It’s a good idea to turn it off sometimes and try to solve problems manually.
- dmix 4y ago
  I highly recommend any programmer seriously using ChatGPT to go through this entire guide: https://learnprompting.org/docs/intermediate/chain_of_thought https://learnprompting.org/docs/intermediate/chain_of_though... Providing context, getting GPT to explain the steps before doing them, creating feedback loops, etc is super helpful when doing this stuff. It's still useful to just go "write this function in x language" but anything beyond a Stack-Overflow type copy/paste script it helps to do a lot more hand-holding. The ROI is worth it.
  - grogenaut 4y ago
    do you have some examples you could share? where y9ou were successful and it was worth the ROI?
    - deleted 4y ago
      [deleted]
- argondonor 4y ago
  It is very nice not having to write code from scratch by hand, but I have noticed that by not doing so, my innate familiarity of the code absolutely craters. Making manual changes to the code suddenly becomes a much more tedious and less confident process, and in many cases I am not sure that it's a net benefit (at least in a situation like mine, where there is no oversight and things go straight to prod.)
  - zainhoda 4y ago
    Same for me. I wonder if this is going to follow a similar pattern that we went through with C vs. Assembly. There was a period where everyone coded in assembly and then C came along but people still hand-coded certain parts of applications in assembly to make them more efficient. Then eventually when C compiler optimizations got good enough and computers got faster, few people bother with assembly.
  - agileAlligator 4y ago
    If it is not doing so already, ask it to comment the code it writes. Worst case scenario, throw the code back into GPT and ask it to make the changes you want to make.
- willbudd 4y ago
  My take-away from your take-away is that LLMs might well take away the one part of my job that I genuinely enjoy doing. Writing code, by hand, myself. Reasoning from first principles. This is also one of the reasons I transitioned to developing AI models myself. At least I'll be safe until they start rewriting themselves from the ground up. And at that point we'll all have bigger worries to deal with anyway.
  - JoshuaDavid 4y ago
    Yeah, you should be safe for at least another few weeks :)
- spaceman_2020 4y ago
  I used to get the same results, but I've lately been using better prompts and the results are substantially better. The key is to a) give it context, and b) give it constraints, and to do it from the very first prompt itself. Most chatGPT failures that I've seen add context and constraints after the initial prompt, which someone sends GPT into a loop. If you have data in a particular format, mention that in the first prompt. Also helpful to tell it to optimize the code for scalability and to use any external libraries if necessary. These tools are essentially software programs, and like any piece of software, you need to know the right commands to get maximum utility out of them.
deleted 4y ago
[deleted]
noobermin 4y ago
What a strange article. The bulk of it tells me it isn't helping you...yet you conclude it's changed the way you code?
- famouswaffles 4y ago
  That's a strange take. He generated working code in less than half the time he did alone. What's not helpful about that ? Or must output be perfect before it's deemed helpful?
  - gwoolhurme 4y ago
    I don't think it is a strange take but can we meet in the middle... there are a lot of things where I would just write them by hand and not look at a 3rd party source too. I think where some people are coming from is extremes on both ends. It seems silly (to me) to allow this thing to take full reigns of your project vs as a helper.
  - chrisco255 4y ago
    Yeah, but he had the experience of building it in the first place, and understanding what he was doing and trying to accomplish. That's not a great speedup given that context. He already knew how to hand-code it, so coaxing the GPT and understanding when it wasn't entirely correct made all the difference. One should expect that even if they were programming a task a second time by hand, that there would be a significant speedup in the same ballpark.
    - famouswaffles 4y ago
      the author himself says he thinks the experience gave him half an hour, maybe an hour tops of time. I'm going to assume he knows what he did to get GPT on track better than you do. and of course, he obviously thought it good enough to want to do it again. Either way, the top comment is just odd. This need to see someone use GPT productively and then butt in and be all "well akshually" just to say something likely unrelated, sometimes not even basing the comment on any sort of experience with the model in comparison is really bizarre. It's not enough to think it's not useful. You just have to "correct" people who found it useful.
milemi 4y ago
This encourages the worst habits, and will make a horrible programmer, and thinker, out of a bad one.
politician 4y ago
Per the article, the technique requires models with a minimum of 100B parameters.
bagels 4y ago
Who is Stan?
- cachvico 4y ago
  He lives here, https://mc-stan.org/ https://mc-stan.org/
- celrod 4y ago
  Stan is a DSL for statistical modeling named after Stanislaw Ulam, the inventor of Monte Carlo methods. Stan's primary method of fitting models is via (Markov Chain) Monte Carlo. https://en.wikipedia.org/wiki/Stanislaw_Ulam https://en.wikipedia.org/wiki/Stanislaw_Ulam
jpollock 4y ago
I use the same programming prompt on both Bard and ChatGPT. "write a function in java to decode a rfc 3588 diameter header" They both need a lot of hand-holding. Too much to make it worth using them. Things they get wrong... 1) They are confident about the structure, but are 100% wrong. 2) They get Java types wrong, and try to put unsigned data into the same size signed type (8 bits into a byte, 24 bits into an int). 3) They marshal data into ints in a way that is correct for C code, but wrong for Java code (missing the 0xFF mask) 4) They forget previous instructions and keep re-introducing bugs. 5) ChatGPT kept thinking that version only had information in bits 6-8. If you know _anything_ about what you're trying to do, it's a waste of time to use a LLM. If the code has anything interesting, a LLM will write broken code. On the plus side, Bard was quoting sources, complete with license. That led to my first pull request to hopefully fix Bard's upstream source, and let me understand what (byte[0]&0x77) << 16 does, why it works and why byte[0]<<16 doesn't. But then, the LLMs didn't help with that, my need to fix it did. http://blog.jason.pollock.ca/2023/04/trying-to-write-code-with-bard-and.html http://blog.jason.pollock.ca/2023/04/trying-to-write-code-wi...
- gwoolhurme 4y ago
  That has been a bit of my experience with LLMs in general. If you know what you are doing before hand it's not too much of a boost, in fact it can be a bit frustrating to wrestle with it to do what you want when you can do it faster. I can see where it is amazing tooling though. Especially for a new project where you don't really have much of an idea or need a working demo.
  - worrycue 4y ago
    But if you don’t know what you are doing, you have to worry about its hallucinations slipping pass you.
    - gwoolhurme 4y ago
      Really good point too, though for the most part I like to think if I audit the code well enough it will work? That said the other day I used it for some string manipulation based on powers of 10, and the code looked correct as per my audit of it. However in reality when I ran the code it was really bad. You might say that I my self would produce buggy code the first time around, that is true... but I also find the LLM equally frustrating because for me (and only me, a sample size of 1 calm down people)? I find I can iterate faster over an idea by hand vs using an LLM for a lot of things.
zippyman55 4y ago
I used GPT to generate some RStan code last month and was very impressed with the initial starting point it gave me. Similar work w LSTM-RNN models. But the number of times it confidentially gives me output and I respond: “are you sure?” and it completely changes its answer. Buyer beware!!
gtirloni 4y ago
<ChatGPT> I reviewed the Stan code again and realized What kind of data structure permits it to keep that code in particular to be reviewed? Is that part of the model? Or is this phrase a hallucination?
- famouswaffles 4y ago
  Every new chat message is the result of a new pass of all the text so far through the model. Mistakes made can be spotted on a new pass.
captnObvious 4y ago
I’ve used GPT-4 as a pair programmer actively for over 40 hours now. Using it to do things you have 90% of the solution for is a bit of a waste of time. Using it to help trouble shoot errors in logs when you’re feeling frustrated, and get a basic framework up from scratch to start modifying toward the your goal are the best use cases I’ve seen. The ability to get yourself unstuck in minutes is life changing. The biggest constraint right now is short term memory loss. It can nearly always recognize on a second pass through what mistake it made. Keep in mind 32kb token requests are already in beta… and 4kb requests are what you have access to now. So… the memory thing is potentially a dead point soon. Though at 6 cents per 1kb if it actually has to volley 32k over to you it’ll almost cost $2.00 based on current API pricing. That 32k option would allow GPT-4 to hold every non-monolithic project I’ve ever worked on in working memory. The pertinent parts anyway, it wouldn’t need to read CSS files or imported modules unless it really gets forced down a specific rabbit hole. This will be one more reason new projects won’t be monolithic architectures. Easier for the AI to help out.
- w4ffl35 4y ago
  It's also good for putting together boiler plate classes and some other things. I had it code most of klotski for example and have used it pretty heavily in this project (and I think that is evident in the code) https://github.com/Capsize-Games/airunner https://github.com/Capsize-Games/airunner
- coder543 4y ago
  $2/request seems a little ridiculous. I checked the pricing and you’re not wrong… I just feel like that kind of pricing will apply even more industry pressure to develop competition. Decent open source options would be nice too.
causality0 4y ago
I'd like to know if there's been any progress in the "memory" area of GPT. As far as I know, when you tell GPT to do something it runs the entire sequence of prompts over again from the beginning. This is obviously not practical for any applications that require ongoing collaboration, like using GPT integrated with your computer to execute commands and policies, or training GPT to be a better coding partner.
- grogenaut 4y ago
  Why is it obviously not practical? Seems to be working in many places. Or is your issue that the memory is too small?
  - causality0 4y ago
    If it has to re-run the entire history every time you interact with it it will quickly slow to non-functionality in days.
    - grogenaut 4y ago
      that's not what happens... basically it is using a ring buffer for the history. It only runs apparently (from another post today) 4k for chatgpt4 right now with 32kb tokens coming in beta. everyotype of memory tho maybe not as limited as this.ne I think has this This is why you see it start getting wierd after a bit, it loses the history and you've basically started a new different conversation. However
Barrin92 4y ago
The funny thing is I've had the exact same experience as the author but come to the completely opposite conclusion. Working with ChatGPT as a daily driver isn't for me precisely because of this: >I have had to be very attentive because it sometimes produces code that runs but does not do what I asked It's like someone putting sand on your toothbrush and in your shoes. The subtle way it messes up are in many ways worse and require more attention than dumber tools. It's even worse than humans because the ways in which it fabricates nonsense aren't the kind of mistakes a human makes, it's not like you're coding with an intern but like you're coding with ALF. The constant context switching between "it's nice" and "it's sabotaging random things" is so taxing if you don't want to mess up. It's like self-driving that way. An automated system that works 95% is worse than one that works 20% of the time.
jimbokun 4y ago
I don't understand the obsession with having Chat GPT4 spit out code for problems like this, where clearly the desired goal would be to have it run the model and display a graph or chart or closed form solution. Like Tony Stark asking Jarvis to run some simulations or calculations and show the solutions in an intuitive way. Is anyone working on something like this? I would be surprised if Wolfram wasn't working on it.
m3kw9 4y ago
ChatGPT is also great at teaching you code. Paste a piece of code you don’t understand, it will explain giving you try it out to make sure. It’s faster scanning docs and on exact match stack overflow answers. At the end of the day you need to know coding to use ChatGPT to help you code, it’s a loop
awinter-py 4y ago
LLM -> DSL codegen is going to be powerful in many cases DSLs do the reasoning that LLMs cannot (though in this case the author works at the place where stan was invented and is probably doing really good prompting) (also to the extent LLMs let us solve everything with boilerplate generation, that's horrible and I weep for the youth)
anabis 4y ago
I too find ChatGPT useful for programming, but: >All told, this took about 2.5 hours. >And all I did was end up with a model very similar to one I wrote by hand several months ago. >But that one took me something like six hours! So 2.5 hours is a huge speedup. It's likely that the attempt being the second time explains 80% of the time reduction. Design and implementation is a learning process, and you being it through once will speed it up massively.
lee101 4y ago
[dead]
poulpy123 4y ago
one thing I found bing AI and phind[1] very useful for was to get a short tutorial of libraries/tools. Google became so bad with it that since several months I was appending "reddit" or worse "youtube"[2] on my search, and now I just ask "how to do x with y" and I get a pretty good answer. [1] I don't know what is used behind the curtains [2] worse because I much prefer to read a text than to watch a video
satisfice 4y ago
The author is playing with fire. I also have used ChatGPT for coding and it hallucinated APIs, twice now, that sent me researching down a rabbit hole. When I have asked it to do things I know how to do and how to evaluate, it has sometimes given very good answers. Other times it misunderstood the problem in a subtle way— and if I hadn’t known enough I would have accepted a program that performed the wrong analysis. If you have to accept the quality of the solution on faith you are playing russian roullette with your reputation. I asked a student to analyze the randomness of set of data. He got a program from ChatGPT without knowing anything about how statistical testing for randomness works. He thus accepted a program from the bot that was hilariously unreliable.

70 points by luu 4y ago

This is a really useful review of what it's like to use LLMS for writing code. The key insight, for me, is this: > Working with ChatGPT4 for this task was like working with a good intern programmer who is rather forgetful and a bit scattered. And the key take-away -- again, for me -- is this: > But: This has already changed the way I code. I am not sure I am ever going to write code completely by hand again! And I expect these tools to get better fast.

bobmaxup 4y ago
I use LLMs in much the same way, but lately I have been wondering how good I will continue to be at thinking, verifying, or communicating in code, if I stop writing as much of it. I suppose time will tell...
- zarzavat 4y ago
  I’ve been using Copilot since the beta and it does cause dependency, which can be dangerous because you have to be constantly vigilant of errors it might be introducing. It’s a good idea to turn it off sometimes and try to solve problems manually.
dmix 4y ago
I highly recommend any programmer seriously using ChatGPT to go through this entire guide: https://learnprompting.org/docs/intermediate/chain_of_thought https://learnprompting.org/docs/intermediate/chain_of_though... Providing context, getting GPT to explain the steps before doing them, creating feedback loops, etc is super helpful when doing this stuff. It's still useful to just go "write this function in x language" but anything beyond a Stack-Overflow type copy/paste script it helps to do a lot more hand-holding. The ROI is worth it.
- grogenaut 4y ago
  do you have some examples you could share? where y9ou were successful and it was worth the ROI?
  - deleted 4y ago
    [deleted]
argondonor 4y ago
It is very nice not having to write code from scratch by hand, but I have noticed that by not doing so, my innate familiarity of the code absolutely craters. Making manual changes to the code suddenly becomes a much more tedious and less confident process, and in many cases I am not sure that it's a net benefit (at least in a situation like mine, where there is no oversight and things go straight to prod.)
- zainhoda 4y ago
  Same for me. I wonder if this is going to follow a similar pattern that we went through with C vs. Assembly. There was a period where everyone coded in assembly and then C came along but people still hand-coded certain parts of applications in assembly to make them more efficient. Then eventually when C compiler optimizations got good enough and computers got faster, few people bother with assembly.
- agileAlligator 4y ago
  If it is not doing so already, ask it to comment the code it writes. Worst case scenario, throw the code back into GPT and ask it to make the changes you want to make.
willbudd 4y ago
My take-away from your take-away is that LLMs might well take away the one part of my job that I genuinely enjoy doing. Writing code, by hand, myself. Reasoning from first principles. This is also one of the reasons I transitioned to developing AI models myself. At least I'll be safe until they start rewriting themselves from the ground up. And at that point we'll all have bigger worries to deal with anyway.
- JoshuaDavid 4y ago
  Yeah, you should be safe for at least another few weeks :)
spaceman_2020 4y ago
I used to get the same results, but I've lately been using better prompts and the results are substantially better. The key is to a) give it context, and b) give it constraints, and to do it from the very first prompt itself. Most chatGPT failures that I've seen add context and constraints after the initial prompt, which someone sends GPT into a loop. If you have data in a particular format, mention that in the first prompt. Also helpful to tell it to optimize the code for scalability and to use any external libraries if necessary. These tools are essentially software programs, and like any piece of software, you need to know the right commands to get maximum utility out of them.

deleted 4y ago

[deleted]

noobermin 4y ago

What a strange article. The bulk of it tells me it isn't helping you...yet you conclude it's changed the way you code?

famouswaffles 4y ago
That's a strange take. He generated working code in less than half the time he did alone. What's not helpful about that ? Or must output be perfect before it's deemed helpful?
- gwoolhurme 4y ago
  I don't think it is a strange take but can we meet in the middle... there are a lot of things where I would just write them by hand and not look at a 3rd party source too. I think where some people are coming from is extremes on both ends. It seems silly (to me) to allow this thing to take full reigns of your project vs as a helper.
- chrisco255 4y ago
  Yeah, but he had the experience of building it in the first place, and understanding what he was doing and trying to accomplish. That's not a great speedup given that context. He already knew how to hand-code it, so coaxing the GPT and understanding when it wasn't entirely correct made all the difference. One should expect that even if they were programming a task a second time by hand, that there would be a significant speedup in the same ballpark.
  - famouswaffles 4y ago
    the author himself says he thinks the experience gave him half an hour, maybe an hour tops of time. I'm going to assume he knows what he did to get GPT on track better than you do. and of course, he obviously thought it good enough to want to do it again. Either way, the top comment is just odd. This need to see someone use GPT productively and then butt in and be all "well akshually" just to say something likely unrelated, sometimes not even basing the comment on any sort of experience with the model in comparison is really bizarre. It's not enough to think it's not useful. You just have to "correct" people who found it useful.

milemi 4y ago

This encourages the worst habits, and will make a horrible programmer, and thinker, out of a bad one.

politician 4y ago

Per the article, the technique requires models with a minimum of 100B parameters.

bagels 4y ago

Who is Stan?

cachvico 4y ago
He lives here, https://mc-stan.org/ https://mc-stan.org/
celrod 4y ago
Stan is a DSL for statistical modeling named after Stanislaw Ulam, the inventor of Monte Carlo methods. Stan's primary method of fitting models is via (Markov Chain) Monte Carlo. https://en.wikipedia.org/wiki/Stanislaw_Ulam https://en.wikipedia.org/wiki/Stanislaw_Ulam

jpollock 4y ago

I use the same programming prompt on both Bard and ChatGPT. "write a function in java to decode a rfc 3588 diameter header" They both need a lot of hand-holding. Too much to make it worth using them. Things they get wrong... 1) They are confident about the structure, but are 100% wrong. 2) They get Java types wrong, and try to put unsigned data into the same size signed type (8 bits into a byte, 24 bits into an int). 3) They marshal data into ints in a way that is correct for C code, but wrong for Java code (missing the 0xFF mask) 4) They forget previous instructions and keep re-introducing bugs. 5) ChatGPT kept thinking that version only had information in bits 6-8. If you know _anything_ about what you're trying to do, it's a waste of time to use a LLM. If the code has anything interesting, a LLM will write broken code. On the plus side, Bard was quoting sources, complete with license. That led to my first pull request to hopefully fix Bard's upstream source, and let me understand what (byte[0]&0x77) << 16 does, why it works and why byte[0]<<16 doesn't. But then, the LLMs didn't help with that, my need to fix it did. http://blog.jason.pollock.ca/2023/04/trying-to-write-code-with-bard-and.html http://blog.jason.pollock.ca/2023/04/trying-to-write-code-wi...

gwoolhurme 4y ago
That has been a bit of my experience with LLMs in general. If you know what you are doing before hand it's not too much of a boost, in fact it can be a bit frustrating to wrestle with it to do what you want when you can do it faster. I can see where it is amazing tooling though. Especially for a new project where you don't really have much of an idea or need a working demo.
- worrycue 4y ago
  But if you don’t know what you are doing, you have to worry about its hallucinations slipping pass you.
  - gwoolhurme 4y ago
    Really good point too, though for the most part I like to think if I audit the code well enough it will work? That said the other day I used it for some string manipulation based on powers of 10, and the code looked correct as per my audit of it. However in reality when I ran the code it was really bad. You might say that I my self would produce buggy code the first time around, that is true... but I also find the LLM equally frustrating because for me (and only me, a sample size of 1 calm down people)? I find I can iterate faster over an idea by hand vs using an LLM for a lot of things.

zippyman55 4y ago

I used GPT to generate some RStan code last month and was very impressed with the initial starting point it gave me. Similar work w LSTM-RNN models. But the number of times it confidentially gives me output and I respond: “are you sure?” and it completely changes its answer. Buyer beware!!

gtirloni 4y ago

<ChatGPT> I reviewed the Stan code again and realized What kind of data structure permits it to keep that code in particular to be reviewed? Is that part of the model? Or is this phrase a hallucination?

famouswaffles 4y ago
Every new chat message is the result of a new pass of all the text so far through the model. Mistakes made can be spotted on a new pass.

captnObvious 4y ago

I’ve used GPT-4 as a pair programmer actively for over 40 hours now. Using it to do things you have 90% of the solution for is a bit of a waste of time. Using it to help trouble shoot errors in logs when you’re feeling frustrated, and get a basic framework up from scratch to start modifying toward the your goal are the best use cases I’ve seen. The ability to get yourself unstuck in minutes is life changing. The biggest constraint right now is short term memory loss. It can nearly always recognize on a second pass through what mistake it made. Keep in mind 32kb token requests are already in beta… and 4kb requests are what you have access to now. So… the memory thing is potentially a dead point soon. Though at 6 cents per 1kb if it actually has to volley 32k over to you it’ll almost cost $2.00 based on current API pricing. That 32k option would allow GPT-4 to hold every non-monolithic project I’ve ever worked on in working memory. The pertinent parts anyway, it wouldn’t need to read CSS files or imported modules unless it really gets forced down a specific rabbit hole. This will be one more reason new projects won’t be monolithic architectures. Easier for the AI to help out.

w4ffl35 4y ago
It's also good for putting together boiler plate classes and some other things. I had it code most of klotski for example and have used it pretty heavily in this project (and I think that is evident in the code) https://github.com/Capsize-Games/airunner https://github.com/Capsize-Games/airunner
coder543 4y ago
$2/request seems a little ridiculous. I checked the pricing and you’re not wrong… I just feel like that kind of pricing will apply even more industry pressure to develop competition. Decent open source options would be nice too.

causality0 4y ago

I'd like to know if there's been any progress in the "memory" area of GPT. As far as I know, when you tell GPT to do something it runs the entire sequence of prompts over again from the beginning. This is obviously not practical for any applications that require ongoing collaboration, like using GPT integrated with your computer to execute commands and policies, or training GPT to be a better coding partner.

grogenaut 4y ago
Why is it obviously not practical? Seems to be working in many places. Or is your issue that the memory is too small?
- causality0 4y ago
  If it has to re-run the entire history every time you interact with it it will quickly slow to non-functionality in days.
  - grogenaut 4y ago
    that's not what happens... basically it is using a ring buffer for the history. It only runs apparently (from another post today) 4k for chatgpt4 right now with 32kb tokens coming in beta. everyotype of memory tho maybe not as limited as this.ne I think has this This is why you see it start getting wierd after a bit, it loses the history and you've basically started a new different conversation. However

Barrin92 4y ago

The funny thing is I've had the exact same experience as the author but come to the completely opposite conclusion. Working with ChatGPT as a daily driver isn't for me precisely because of this: >I have had to be very attentive because it sometimes produces code that runs but does not do what I asked It's like someone putting sand on your toothbrush and in your shoes. The subtle way it messes up are in many ways worse and require more attention than dumber tools. It's even worse than humans because the ways in which it fabricates nonsense aren't the kind of mistakes a human makes, it's not like you're coding with an intern but like you're coding with ALF. The constant context switching between "it's nice" and "it's sabotaging random things" is so taxing if you don't want to mess up. It's like self-driving that way. An automated system that works 95% is worse than one that works 20% of the time.

jimbokun 4y ago

I don't understand the obsession with having Chat GPT4 spit out code for problems like this, where clearly the desired goal would be to have it run the model and display a graph or chart or closed form solution. Like Tony Stark asking Jarvis to run some simulations or calculations and show the solutions in an intuitive way. Is anyone working on something like this? I would be surprised if Wolfram wasn't working on it.

m3kw9 4y ago

ChatGPT is also great at teaching you code. Paste a piece of code you don’t understand, it will explain giving you try it out to make sure. It’s faster scanning docs and on exact match stack overflow answers. At the end of the day you need to know coding to use ChatGPT to help you code, it’s a loop

awinter-py 4y ago

LLM -> DSL codegen is going to be powerful in many cases DSLs do the reasoning that LLMs cannot (though in this case the author works at the place where stan was invented and is probably doing really good prompting) (also to the extent LLMs let us solve everything with boilerplate generation, that's horrible and I weep for the youth)

anabis 4y ago

I too find ChatGPT useful for programming, but: >All told, this took about 2.5 hours. >And all I did was end up with a model very similar to one I wrote by hand several months ago. >But that one took me something like six hours! So 2.5 hours is a huge speedup. It's likely that the attempt being the second time explains 80% of the time reduction. Design and implementation is a learning process, and you being it through once will speed it up massively.

lee101 4y ago

[dead]

poulpy123 4y ago

one thing I found bing AI and phind[1] very useful for was to get a short tutorial of libraries/tools. Google became so bad with it that since several months I was appending "reddit" or worse "youtube"[2] on my search, and now I just ask "how to do x with y" and I get a pretty good answer. [1] I don't know what is used behind the curtains [2] worse because I much prefer to read a text than to watch a video

satisfice 4y ago

The author is playing with fire. I also have used ChatGPT for coding and it hallucinated APIs, twice now, that sent me researching down a rabbit hole. When I have asked it to do things I know how to do and how to evaluate, it has sometimes given very good answers. Other times it misunderstood the problem in a subtle way— and if I hadn’t known enough I would have accepted a program that performed the wrong analysis. If you have to accept the quality of the solution on faith you are playing russian roullette with your reputation. I asked a student to analyze the randomness of set of data. He got a program from ChatGPT without knowing anything about how statistical testing for randomness works. He thus accepted a program from the bot that was hilariously unreliable.