4 ms·

I don't know if it is just me but since this AI "summer" started I have ever pulled a single snippet from any of the available services. I'm absolutely curious

I don't know if it is just me but since this AI "summer" started I have ever pulled a single snippet from any of the available services. I'm absolutely curious what kind of code people are getting from it that is so much more valuable than writting things from scratch and use corresponding documentation, examples or even just source code of the framework/libraries that are needed to accomplish a task.

chx 2y ago
https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.
newswasboring 2y ago
Software development has gone too diverse. Recently I had a case where I was programming in Matlab I had to use some PowerShell commands. I have no idea how PowerShell even works, but I know enough programming that when phind.com spit out a 4 line script I was able to use it. All this happened in like 10mins. I've done something like this before, almost always trying to work around Matlab limitations, but those cases took a whole day and so much frustration. I like the AI experience very valuable just for the mental peace. Edit: before someone accuses me of doing what trovalds is saying, I understand that code now. I was able to probe the model to explain it to me to the same degree as my understanding of my own code. It was the same feeling as when my team innovated over my previous work and I just asked them how they did it.
CHY872 2y ago
'My customer has given me the documentation to arcane antiquated format X (insert pdf, but it includes 24-bit integers, hex encoded data, semantically signficant whitespace). Here is a sample of the message format, and this struct should represent the contents. Please write me a Python parser which takes an input file in the format and provides the output. In particular, given input X, the output should be equivalent to Y. There should be a set of unit tests for important functionality, which should explain to a reader what is complicated about the format'. is something that ChatGPT4 will just drop out a solution to that works in a few seconds. If your job is to make an accurate parser for it, probably you want to hand code it. If your job is to make sense of the data the customer has provided you with, this is merely an impediment to your actual job, and ChatGPT has you covered. Yes, there'll be mistakes. But ChatGPT can do in a few seconds what'd take you hours.
- nine_k 2y ago
  This is a great example of LLM usefulness. It also illustrates how it's half of the solution. Certainly the sense-making part works reasonably well. What it lacks is the formal rigor part. With them both in place, the computer could generate a perfectly correct solution, not just approximately correct. Computers are very good at formal rigor, and we have quite some rigorous methods of program synthesis. Whoever manages to connect these dots will own much of the industry.
- deleted 2y ago
  [deleted]
- probably_wrong 2y ago
  Would you apply the same criteria to code instead of file formats? As in "my customer has given me this code for architecture X and I need it working on architecture Y"? For those who answer "yes" to the above I'd encourage them to read the story of the Therac-25 [1], a machine where hardware mechanisms in model A where replaced by software mechanisms in model B leading to a race condition that would dose patients with massive doses of radiation. > Yes, there'll be mistakes. "Over the following weeks the patient experienced paralysis of the left arm, nausea, vomiting, and ended up being hospitalized for radiation-induced myelitis of the spinal cord. His legs, mid-diaphragm and vocal cords ended up paralyzed. He also had recurrent herpes simplex skin infections. He died five months after the overdose." [1] "But your honor, I specifically wrote 'add unit tests for important functionality'!" [1] https://en.m.wikipedia.org/wiki/Therac-25 https://en.m.wikipedia.org/wiki/Therac-25
  - smugma 2y ago
    The story of the Therac25 is an important one. I first read it in college and re-read it every 5-10 years. That said, I don’t think it’s a strong rebuttal to the arguments here. Context matters. Earlier poster mentioned they’d use it for a React front end but not a backend deletion. Knowing the tool and the appropriateness of the tool in context is one of the lessons to be learned from the Therac 25 tragedy.
  - CHY872 2y ago
keepamovin 2y ago
It’s just faster, less cognitive load and frees you up to think more strategically rather than just focus on implementation
Sunhold 2y ago
I now write most of my personal scripts in English and have ChatGPT compile it to whatever programming language. I don't use it for anything too complex but it makes it easier to write things like grabbing data from an API or website, manipulating it and putting it in a database, etc. Honestly so long as I provide enough detail, the code is more likely to work on the first run than something I wrote by hand.
thom 2y ago
Having a working version of some code to refine is often helpful. I most often just use the output as better documentation that I can interact with. I'll definitely grant you that I never end up using AI generated code directly in production. I find the technology extremely valuable though, dare I say transformative.
rcfox 2y ago
I've recently started writing eslint rules for my project at work. I'll ask ChatGPT how to do it, and it will get the specific implementation wrong every time, but it will give me a decent foundation to start on. It will usually correctly identify the AST node types I need to work with, saving me from having to sift through the possibilities.
reubenmorais 2y ago
Sometimes you're just sketching in a notebook, or working on a shell script, a Makefile, something you don't do often enough to have the syntax and idiosyncrasies at the tip of your fingers. Sometimes you're doing something very mechanistic but not easy to automate, like converting some filtering logic from one DSL to another. Sometimes there's just a lot of typing required to do something conceptually very simple. I find Copilot very useful in these side quests, but when I'm working on the hard stuff, the core functionality, it often just outputs nonsense that takes more time to review/fix than it would take to just write it myself. So it takes some learning of when to use it and when to ignore it but I've come around to appreciate it at times.
- ljm 2y ago
  Copilot for me basically seems like semantic copy/paste. I’m doing some refactoring and make the same style change in a couple of files, then copilot kicks in and understands enough to suggest a legit completion for me. I only use it sparingly though because it can too easily become a crutch where you start to feel a bit lost without it.
7bit 2y ago
I'm a hobby programmer. I mostly ask ChatGPT for stuff that I could do, but want save some time, or I have absolutely no clue how to, for example, get the type annotations right in Rust (still a super noob!). In both cases, even after back and forth with the AI for half an hour, the result is absolute garbage. And never works. I don't understand at all how people get even barely usable stuff out of AI...
- FeepingCreature 2y ago
  Back and forth degrades quality. Try resetting more frequently. And make sure you're using a frontier model, ie. GPT-4, Opus, Gemini etc.
- SunlitCat 2y ago
  Well, ChatGPT surprised me when I asked it to write me a gltf parser. At first it printed out some python code, but after asking it to do the same it same in c++, it printed out a perfectly working parser, listing buffer size and buffer view sizes of a glb file using the nlohmann json library.
FeepingCreature 2y ago
I use it for things I want but don't care enough to learn. I recently wanted to present a bunch of labeled pngs as an HTML overview in a certain order, after grabbing them from a webservice with REST. Took me maybe twenty minutes with Opus, including prettying it up with css. I could have done it myself, sure, but not nearly that quickly.
RivieraKid 2y ago
Today I used it to convert a non-trivial function from Julia to R because I wanted to use a certain R package for a project.
hansvm 2y ago
My main code-adjacent use cases broadly fall into the following categories, and I'll give a few more specific examples afterward: (1) Using LLMs as a learning aid (or docs augmentation) (2) Writing one-off _whatevers_ (3) Finding the appropriate docs/info in the first place (4) As a meta-tool to sniff out whether other people have gone down this path before (5) Cloning any sort of boilerplate "pattern" which requires more than simple text substitution (nvim macros, custom CLI tools applied to a visual selection, ... can get you a long ways, but they're not good for everything) For a few select examples: (1a) You can copy in a snippet of code and ask what _this_ character does _here_, finding appropriate search terms to actually learn about the topic. (1b) Generated examples are fine, even if they're wrong, if you use them as a mental scaffolding to learn more about a library. Analyze them and prove them right or find why they're wrong, and that process of analysis teaches you in a different way than just reading the docs by themselves. (1c) Sometimes the docs are just unwieldy. Ask the LLM for the snippet immediately after the "-B" flag, and then you can search for that more-unique snippet instead of, e.g., 1000 occurrences of other fields referencing "-B" and not defining it. (2a) Say you want to know how your webserver responds when the client sends each byte in a separate syscall, waits a second between each one, and then immediately starts a new pipelined request doing the same shenanigans without waiting for a response to the first one. The LLM can generate exactly what you need, or when it fails it was still faster than typing it yourself. It's a task that's much more succinct to describe in English than code, yet simple enough that the LLM has a high success rate, and a brief scan of the code can confirm it's actually hitting localhost instead of downloading not-a-virus.exe.txt. The stakes are low if it's wrong because you can visually tell if it's doing the right thing. (3a) I recalled reading once about the design of the voting/scoring algorithm used internally in Google's TGIF question ranker and knew there was a brief public blog post going over the more important details in a consumable format. Current search engines mostly can't point me to an old, unmonetized, niche blog post, especially when I don't remember any of the important keywords. LLMs can often point me to the exact blog, and barring that can usually give me the keywords I need to find it myself (similarly with finding things like the AWS SDK C API, which exists but isn't advertised and is hard to search for because the other keywords dominate). In this case, I wanted to read about Wilson's Scoring by Evan Miller [0]. (4a) Ask the same question a few different ways and a lot of times. If you only get garbage hallucinated blog posts, research articles, and keywords not related to your current task, it's suggestive of the specific task you're doing not being popular. If at a high level you know your end goal has been implemented many times, that's suggestive of you going down an incorrect path. (5a) I was writing a program heavily leveraging lazy type instantiation to generate a zero-cost (de-virtualized, inlined where appropriate) iterator library. Anyone can write map and reduce, but it's a little more annoying to have a meta-layer doing that sort of thing in the type system, especially with some other constraints I had in the project. However, showing the LLM a few examples of this type-level way of defining the things I'm doing and then asking for implementations of `peek` or `skip` or whatever worked swimmingly. They're well suited to that task because the boilerplate isn't just simple textual substitution, because the tasks are simple enough that the LLM has a high chance of getting them right, and because the nature of the library is that a couple very simple hand-written tests will catch all the subtle mistakes, and a human reading the code will catch all the non-subtle errors. And so on. I don't use LLMs for everything. If I want to do anything involving infrequently used Python dunders I'll probably go straight to the data model docs [1] and search within that page. I write most code by hand (give or take many custom tools I've integrated into my nvim workflow). Sometimes an LLM makes my life easier though. [0] https://www.evanmiller.org/how-not-to-sort-by-average-rating.html https://www.evanmiller.org/how-not-to-sort-by-average-rating... [1] https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html
coffeebeqn 2y ago
It is helpful as a starting point when I’m switching to a different language I don’t use much. Saves me a lot of googling of how do I do x and y in python. I like a “editing” workflow when I’m not super comfortable with the stack yet - I get that blank page syndrome where I don’t know where to statt. But so far they are not up to snuff for understanding my day to day work or the style of our codebase so I don’t use it for that
- freehorse 2y ago
  It is a good starting point for switching to a language you do not use much when that language is python or js, or maybe a few other languages with a lot of examples in their training set. For many languages (as well as less common libraries etc) it hallucinates too much. My go-to task when trying a new LLM is asking it to import a csv file in julia. You would be surprised (or maybe not) how many fail.

chx 2y ago

https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.

newswasboring 2y ago

Software development has gone too diverse. Recently I had a case where I was programming in Matlab I had to use some PowerShell commands. I have no idea how PowerShell even works, but I know enough programming that when phind.com spit out a 4 line script I was able to use it. All this happened in like 10mins. I've done something like this before, almost always trying to work around Matlab limitations, but those cases took a whole day and so much frustration. I like the AI experience very valuable just for the mental peace. Edit: before someone accuses me of doing what trovalds is saying, I understand that code now. I was able to probe the model to explain it to me to the same degree as my understanding of my own code. It was the same feeling as when my team innovated over my previous work and I just asked them how they did it.

CHY872 2y ago

'My customer has given me the documentation to arcane antiquated format X (insert pdf, but it includes 24-bit integers, hex encoded data, semantically signficant whitespace). Here is a sample of the message format, and this struct should represent the contents. Please write me a Python parser which takes an input file in the format and provides the output. In particular, given input X, the output should be equivalent to Y. There should be a set of unit tests for important functionality, which should explain to a reader what is complicated about the format'. is something that ChatGPT4 will just drop out a solution to that works in a few seconds. If your job is to make an accurate parser for it, probably you want to hand code it. If your job is to make sense of the data the customer has provided you with, this is merely an impediment to your actual job, and ChatGPT has you covered. Yes, there'll be mistakes. But ChatGPT can do in a few seconds what'd take you hours.

nine_k 2y ago
This is a great example of LLM usefulness. It also illustrates how it's half of the solution. Certainly the sense-making part works reasonably well. What it lacks is the formal rigor part. With them both in place, the computer could generate a perfectly correct solution, not just approximately correct. Computers are very good at formal rigor, and we have quite some rigorous methods of program synthesis. Whoever manages to connect these dots will own much of the industry.
deleted 2y ago
[deleted]
probably_wrong 2y ago
Would you apply the same criteria to code instead of file formats? As in "my customer has given me this code for architecture X and I need it working on architecture Y"? For those who answer "yes" to the above I'd encourage them to read the story of the Therac-25 [1], a machine where hardware mechanisms in model A where replaced by software mechanisms in model B leading to a race condition that would dose patients with massive doses of radiation. > Yes, there'll be mistakes. "Over the following weeks the patient experienced paralysis of the left arm, nausea, vomiting, and ended up being hospitalized for radiation-induced myelitis of the spinal cord. His legs, mid-diaphragm and vocal cords ended up paralyzed. He also had recurrent herpes simplex skin infections. He died five months after the overdose." [1] "But your honor, I specifically wrote 'add unit tests for important functionality'!" [1] https://en.m.wikipedia.org/wiki/Therac-25 https://en.m.wikipedia.org/wiki/Therac-25
- smugma 2y ago
  The story of the Therac25 is an important one. I first read it in college and re-read it every 5-10 years. That said, I don’t think it’s a strong rebuttal to the arguments here. Context matters. Earlier poster mentioned they’d use it for a React front end but not a backend deletion. Knowing the tool and the appropriateness of the tool in context is one of the lessons to be learned from the Therac 25 tragedy.
- CHY872 2y ago

keepamovin 2y ago

It’s just faster, less cognitive load and frees you up to think more strategically rather than just focus on implementation

Sunhold 2y ago

I now write most of my personal scripts in English and have ChatGPT compile it to whatever programming language. I don't use it for anything too complex but it makes it easier to write things like grabbing data from an API or website, manipulating it and putting it in a database, etc. Honestly so long as I provide enough detail, the code is more likely to work on the first run than something I wrote by hand.

thom 2y ago

Having a working version of some code to refine is often helpful. I most often just use the output as better documentation that I can interact with. I'll definitely grant you that I never end up using AI generated code directly in production. I find the technology extremely valuable though, dare I say transformative.

rcfox 2y ago

I've recently started writing eslint rules for my project at work. I'll ask ChatGPT how to do it, and it will get the specific implementation wrong every time, but it will give me a decent foundation to start on. It will usually correctly identify the AST node types I need to work with, saving me from having to sift through the possibilities.

reubenmorais 2y ago

Sometimes you're just sketching in a notebook, or working on a shell script, a Makefile, something you don't do often enough to have the syntax and idiosyncrasies at the tip of your fingers. Sometimes you're doing something very mechanistic but not easy to automate, like converting some filtering logic from one DSL to another. Sometimes there's just a lot of typing required to do something conceptually very simple. I find Copilot very useful in these side quests, but when I'm working on the hard stuff, the core functionality, it often just outputs nonsense that takes more time to review/fix than it would take to just write it myself. So it takes some learning of when to use it and when to ignore it but I've come around to appreciate it at times.

ljm 2y ago
Copilot for me basically seems like semantic copy/paste. I’m doing some refactoring and make the same style change in a couple of files, then copilot kicks in and understands enough to suggest a legit completion for me. I only use it sparingly though because it can too easily become a crutch where you start to feel a bit lost without it.

7bit 2y ago

I'm a hobby programmer. I mostly ask ChatGPT for stuff that I could do, but want save some time, or I have absolutely no clue how to, for example, get the type annotations right in Rust (still a super noob!). In both cases, even after back and forth with the AI for half an hour, the result is absolute garbage. And never works. I don't understand at all how people get even barely usable stuff out of AI...

FeepingCreature 2y ago
Back and forth degrades quality. Try resetting more frequently. And make sure you're using a frontier model, ie. GPT-4, Opus, Gemini etc.
SunlitCat 2y ago
Well, ChatGPT surprised me when I asked it to write me a gltf parser. At first it printed out some python code, but after asking it to do the same it same in c++, it printed out a perfectly working parser, listing buffer size and buffer view sizes of a glb file using the nlohmann json library.

FeepingCreature 2y ago

I use it for things I want but don't care enough to learn. I recently wanted to present a bunch of labeled pngs as an HTML overview in a certain order, after grabbing them from a webservice with REST. Took me maybe twenty minutes with Opus, including prettying it up with css. I could have done it myself, sure, but not nearly that quickly.

RivieraKid 2y ago

Today I used it to convert a non-trivial function from Julia to R because I wanted to use a certain R package for a project.

hansvm 2y ago

My main code-adjacent use cases broadly fall into the following categories, and I'll give a few more specific examples afterward: (1) Using LLMs as a learning aid (or docs augmentation) (2) Writing one-off _whatevers_ (3) Finding the appropriate docs/info in the first place (4) As a meta-tool to sniff out whether other people have gone down this path before (5) Cloning any sort of boilerplate "pattern" which requires more than simple text substitution (nvim macros, custom CLI tools applied to a visual selection, ... can get you a long ways, but they're not good for everything) For a few select examples: (1a) You can copy in a snippet of code and ask what _this_ character does _here_, finding appropriate search terms to actually learn about the topic. (1b) Generated examples are fine, even if they're wrong, if you use them as a mental scaffolding to learn more about a library. Analyze them and prove them right or find why they're wrong, and that process of analysis teaches you in a different way than just reading the docs by themselves. (1c) Sometimes the docs are just unwieldy. Ask the LLM for the snippet immediately after the "-B" flag, and then you can search for that more-unique snippet instead of, e.g., 1000 occurrences of other fields referencing "-B" and not defining it. (2a) Say you want to know how your webserver responds when the client sends each byte in a separate syscall, waits a second between each one, and then immediately starts a new pipelined request doing the same shenanigans without waiting for a response to the first one. The LLM can generate exactly what you need, or when it fails it was still faster than typing it yourself. It's a task that's much more succinct to describe in English than code, yet simple enough that the LLM has a high success rate, and a brief scan of the code can confirm it's actually hitting localhost instead of downloading not-a-virus.exe.txt. The stakes are low if it's wrong because you can visually tell if it's doing the right thing. (3a) I recalled reading once about the design of the voting/scoring algorithm used internally in Google's TGIF question ranker and knew there was a brief public blog post going over the more important details in a consumable format. Current search engines mostly can't point me to an old, unmonetized, niche blog post, especially when I don't remember any of the important keywords. LLMs can often point me to the exact blog, and barring that can usually give me the keywords I need to find it myself (similarly with finding things like the AWS SDK C API, which exists but isn't advertised and is hard to search for because the other keywords dominate). In this case, I wanted to read about Wilson's Scoring by Evan Miller [0]. (4a) Ask the same question a few different ways and a lot of times. If you only get garbage hallucinated blog posts, research articles, and keywords not related to your current task, it's suggestive of the specific task you're doing not being popular. If at a high level you know your end goal has been implemented many times, that's suggestive of you going down an incorrect path. (5a) I was writing a program heavily leveraging lazy type instantiation to generate a zero-cost (de-virtualized, inlined where appropriate) iterator library. Anyone can write map and reduce, but it's a little more annoying to have a meta-layer doing that sort of thing in the type system, especially with some other constraints I had in the project. However, showing the LLM a few examples of this type-level way of defining the things I'm doing and then asking for implementations of `peek` or `skip` or whatever worked swimmingly. They're well suited to that task because the boilerplate isn't just simple textual substitution, because the tasks are simple enough that the LLM has a high chance of getting them right, and because the nature of the library is that a couple very simple hand-written tests will catch all the subtle mistakes, and a human reading the code will catch all the non-subtle errors. And so on. I don't use LLMs for everything. If I want to do anything involving infrequently used Python dunders I'll probably go straight to the data model docs [1] and search within that page. I write most code by hand (give or take many custom tools I've integrated into my nvim workflow). Sometimes an LLM makes my life easier though. [0] https://www.evanmiller.org/how-not-to-sort-by-average-rating.html https://www.evanmiller.org/how-not-to-sort-by-average-rating... [1] https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html

coffeebeqn 2y ago

It is helpful as a starting point when I’m switching to a different language I don’t use much. Saves me a lot of googling of how do I do x and y in python. I like a “editing” workflow when I’m not super comfortable with the stack yet - I get that blank page syndrome where I don’t know where to statt. But so far they are not up to snuff for understanding my day to day work or the style of our codebase so I don’t use it for that

freehorse 2y ago
It is a good starting point for switching to a language you do not use much when that language is python or js, or maybe a few other languages with a lot of examples in their training set. For many languages (as well as less common libraries etc) it hallucinates too much. My go-to task when trying a new LLM is asking it to import a csv file in julia. You would be surprised (or maybe not) how many fail.