11 ms·

I’ve never found GPT-4 capable of producing a useful solution in my niche of engineering. When I’m stumped, it’s usually on a complex and very multi-faceted pr

by clnq 3y ago

I’ve never found GPT-4 capable of producing a useful solution in my niche of engineering. When I’m stumped, it’s usually on a complex and very multi-faceted problem where the full scope doesn’t fit into the human brain very well. And for these problems, GPT will produce some borderline unworkable solutions. It’s like a jack of all trades and master of none in code. It’s knowledge seems a mile wide and an inch deep. Granted, it could be different for junior to mid programmers.

danenania 3y ago
What’s your niche? I think much of using it well is understanding what it can and can’t do (though of course this is a moving target). It’s great when the limiting factor is knowledge of APIs, best practices, or common algorithms. When the limiting factor is architectural complexity or understanding how many different components of a system fit together, it’s less useful. Still, I find I can often save time on more difficult tasks by figuring out the structure and then having GPT-4 fill in the blanks. It’s a much better programmer once you get it started down the right path.
- clnq 3y ago
  My niche is in video game programming, and I am very specialized in a specific area. So I might ask things like how would one architect a certain game system with a number of requirements, to meet certain player expectations, and be compatible with a number of things. Unfortunately, it hasn’t been helpful once, and often due to the same reason - when the question gets specific enough, it hallucinates because it doesn’t know, just like in the early days. Moreover, I am a domain expert in my area, so I only ask for help when the problem is really difficult. For example, when it would take me several days to come up with an answer and a few more weeks to refine it. Game development has a lot of enthusiasts online sharing material, but most of this material is at junior to intermediate level. You very quickly run out of resources for questions at a principal level, even if you know the problems you have have been solved in other AAA companies. You have to rely on your industry friends, paid support from middleware providers, rare textbooks, conferences, and, on the off-chance that anything useful got scooped up into the training data set - GPT. But GPT has been more like wishful thinking for me.
  - throwaway4aday 3y ago
    I'm interested to know if you've tried creating a custom GPT with their builder or the API. If you have enough old example code, notes, or those rare textbooks you mention you could add those as files and see if the built in RAG improves the answers it gives.
    - clnq 3y ago
      I tried building a custom GPT but the training data it has is not sufficient, no matter how well it’s steered. Documents and code are confidential in the AAA games industry as they are the money makers. Developers are not free to hand them over to third parties, that would be known as a leak. With textbooks, that would be a pretty grey area use case. So I’ve not experimented with that. I think it could help, but because it’s so infeasible practically, there’s no incentive to try this with synthetic data, too.
  - somestag 3y ago
    Interesting. I also work in game development, and I tend to work on project-specific optimization problems, and I've had the opposite experience. If I have to solve a hairy problem specific to our game's architecture, obviously I'm not going to ask ChatGPT to solve that for me. It's everything else that it works so well for. The stuff that I could do, but it's not really worth my time to actually do it when I can be focusing on the hard stuff. One example: there was a custom protocol our game servers used to communicate with some other service. For reasons, we relied on an open-source tool to handle communication over this protocol, but then we decided we wanted to switch to an in-code solution. Rather than study the open source tool's code, rewrite it in the language we used, write tests for it, generate some test data... I just gave ChatGPT the original source and the protocol spec and spent 10 minutes walking it through the problem. I had a solution (with tests) in under half an hour when doing it all myself would've taken the afternoon. Then I went back to working on the actual hard stuff that my human brain was needed to solve. I can't imagine being so specialized that I only ever work on difficult problems within my niche and nothing else. There's always some extra query to write, some API to interface with, some tests to write... it's not a matter of being able to do it myself, it's a matter of being able to focus primarily on the stuff I need to do myself. Being able to offload the menial work to an AI also just changes the sorts of stuff I'm willing to do with my time. As a standalone software engineer, I will often choose not to write some simple'ish tool or script that might be useful because it might not be worth my time to write it, especially factoring in the cost of context switching. Nothing ground breaking, just something that might not be worth half an hour of my time. But I can just tell AI to write the script for me and I get it in a couple minutes. So instead of doing all my work without access to some convenient small custom tools, now I can do my work with them, with very little change to my workflow.
- girvo 3y ago
  It struggles with (industrial, not hobbyist) embedded firmware a fair bit. I can almost coax decent results for simple tasks out of it, sometimes.
  - clnq 3y ago
    LLMs almost never write good senior quality code at first in niche disciplines. You need to finesse it a lot to have it produce the correct answer. And that makes it unusable for when you genuinely do not know the answer to the question you’re asking, which is kind of the entire point.
chalcolithic 3y ago
How long ago would you have considered this discussion ridiculous? How long till GPT-N will be churning out solutions faster than you can read them? It's useless for me now as well, but I'm pretty sure I'll be doomed professionally in the future.
- jeffreygoesto 3y ago
  Not necessarily. Every hockey stick is just the beginning of an s-curve. It will saturate, probably sooner than you think.
  - hackerlight 3y ago
    Some parts of AI will necessarily asymptote to human-level intelligence because of a fixed corpus of training data. It's hard to think AI will become a better creative writer than the best human creative writers, because the AI is trained on their output and you can't go much further than that. But in areas where there's self-play (e.g. Chess, and to a lesser extent, programming), there is no good reason to think it'll saturate, since there isn't a limit on the amount of training data.
    - borissk 3y ago
      So you think human readers have magical powers to rate say a book that an AI can't replicate?
      - hackerlight 3y ago
        There's a gulf of difference between domains where self-play means we have unlimited training data for free (e.g. Chess) versus domains where there's no known way to generate more training data (e.g. Fine art). It's possible that the latter domains will see unpredictable innovations that allow it to generate more training data beyond what humans have produced, but that's an open question.
    - strken 3y ago
      How does programming have self-play? I'm not sure I understand. Are you going to generate leetcode questions with one AI, have another answer them, and have a third determine whether the answer is correct? I'm struggling to understand how an LLM is meant to answer the questions that come up in day-to-day software engineering, like "Why is the blahblah service occasionally timing out? Here are ten bug reports, most of which are wrong or misleading" or "The foo team and bar team want to be able to configure access to a Project based on the sensitivity_rating field using our access control system, so go and talk to them about implementing ABAC". The discipline of programming might be just a subset of broader software engineering, but it arguably still contains debugging, architecture, and questions which need more context than you can feed into an LLM now. Can't really self-play those things without interacting with the real world.
Zaofy 3y ago
Same here. I'm not a developer. I do engineering and architecture in IAM. I've tested out GPT-4 and it's good for general advice or problem solving. But it can't know the intricascies of the company I work at with all our baggage, legacy systems and us humans sometimes just being straight up illogical and inefficient with what we want. So my usage has mostly been for it to play a more advanced rubber duck to bounce ideas and concepts off of and to do some of the more tedious scripting work (that I still have to double check thoroughly). At some point GPT and other LLMs might be able to replace what I do in large parts. But that's still a while off.
Tainnor 3y ago
Same. Even for technologies that it supposedly should know a lot about (e.g. Kafka), if I prompt it for something slightly non-standard, it just makes up things that aren't supported or is otherwise unhelpful. The one time I've found ChatGPT to be genuinely useful is when I asked it to explain a bash script to me, seeing as bash is notoriously inscrutable. Still, it did get a detail wrong somehow.
- clnq 3y ago
  Yes, it is good at summarizing things and regressing things down to labels. It’s much worse at producing concrete and specific results from its corpus of abstract knowledge. I think that’s the case with every discipline for it, not only programming. Even when everyone was amazed it could make poetry out of everything, if you asked for a specific type of poem and specific imagery in it, it would generally fail.
gwd 3y ago
Well no, you shouldn't use it for your top-end problems, but your bottom-end problems. Aren't there things that you have to do in your job that really could be done by a junior programmer? Don't you ever have one-off (or once-a-year) things you have to do that each time you have to invest a lot of time refreshing in your brain, and then basically forgetting for lack of use? Here's an example I used the other day: Our project had lost access to our YT channel, which had 350+ videos on it (due to someone's untimely passing and a lack of redundancy). I had used yt-dlp to download all the old videos, including descriptions. Our community manager had uploaded all the videos, but wasn't looking forward to copy-and-pasting every description into the new video. So I offered to use GPT-4 to write a python script to use the API to do that for her. I didn't know anything about the YT API, nor am I an expert in python. I wouldn't have invested the time learning the YT API (and trying to work through my rudimentary python knowledge) for a one-off thing like this, but I knew that GPT-4 would be able to help me focus on what to do rather than how to do it. The transcript is here: https://chat.openai.com/share/936e35f9-e500-4a4d-aa76-273f63c9de20 https://chat.openai.com/share/936e35f9-e500-4a4d-aa76-273f63... By contrast, I don't think there's any possible way the current generation could have identified, or helped fix, this problem that I fixed a few years ago: https://xenbits.xenproject.org/xsa/xsa299/0011-x86-mm-Don-t-drop-a-type-ref-unless-you-held-a-ref-t.patch https://xenbits.xenproject.org/xsa/xsa299/0011-x86-mm-Don-t-... (Although it would be interesting to try to ask it about that to see how well it does.) The point of using GPT-4 should be to take over the "low value" work from you, so that you have more time and mental space to focus on the "high value" work.
- ioseph 3y ago
  Perhaps by learning to use the YT API (seriously something that should take 2 hours max if you know how http works) you'll learn something from their design choices, or develop opinions on what makes a good API. And by learning a bit more python you'll get exposed to patterns you could use in your own language.
  - gwd 3y ago
    If anything, using GPT-4 makes a lot of that more efficient. Rather than scrolling through loads of API documentation trying to guess how to do something, writing Python with a "C" accent, I can just read the implementation that GPT-4 spits out, which is almost certainly based on seeing hundreds of examples written by people who are fluent in python, and thus using both to best effect.

by clnq 3y ago

danenania 3y ago

What’s your niche? I think much of using it well is understanding what it can and can’t do (though of course this is a moving target). It’s great when the limiting factor is knowledge of APIs, best practices, or common algorithms. When the limiting factor is architectural complexity or understanding how many different components of a system fit together, it’s less useful. Still, I find I can often save time on more difficult tasks by figuring out the structure and then having GPT-4 fill in the blanks. It’s a much better programmer once you get it started down the right path.

clnq 3y ago
My niche is in video game programming, and I am very specialized in a specific area. So I might ask things like how would one architect a certain game system with a number of requirements, to meet certain player expectations, and be compatible with a number of things. Unfortunately, it hasn’t been helpful once, and often due to the same reason - when the question gets specific enough, it hallucinates because it doesn’t know, just like in the early days. Moreover, I am a domain expert in my area, so I only ask for help when the problem is really difficult. For example, when it would take me several days to come up with an answer and a few more weeks to refine it. Game development has a lot of enthusiasts online sharing material, but most of this material is at junior to intermediate level. You very quickly run out of resources for questions at a principal level, even if you know the problems you have have been solved in other AAA companies. You have to rely on your industry friends, paid support from middleware providers, rare textbooks, conferences, and, on the off-chance that anything useful got scooped up into the training data set - GPT. But GPT has been more like wishful thinking for me.
- throwaway4aday 3y ago
  I'm interested to know if you've tried creating a custom GPT with their builder or the API. If you have enough old example code, notes, or those rare textbooks you mention you could add those as files and see if the built in RAG improves the answers it gives.
  - clnq 3y ago
    I tried building a custom GPT but the training data it has is not sufficient, no matter how well it’s steered. Documents and code are confidential in the AAA games industry as they are the money makers. Developers are not free to hand them over to third parties, that would be known as a leak. With textbooks, that would be a pretty grey area use case. So I’ve not experimented with that. I think it could help, but because it’s so infeasible practically, there’s no incentive to try this with synthetic data, too.
- somestag 3y ago
  Interesting. I also work in game development, and I tend to work on project-specific optimization problems, and I've had the opposite experience. If I have to solve a hairy problem specific to our game's architecture, obviously I'm not going to ask ChatGPT to solve that for me. It's everything else that it works so well for. The stuff that I could do, but it's not really worth my time to actually do it when I can be focusing on the hard stuff. One example: there was a custom protocol our game servers used to communicate with some other service. For reasons, we relied on an open-source tool to handle communication over this protocol, but then we decided we wanted to switch to an in-code solution. Rather than study the open source tool's code, rewrite it in the language we used, write tests for it, generate some test data... I just gave ChatGPT the original source and the protocol spec and spent 10 minutes walking it through the problem. I had a solution (with tests) in under half an hour when doing it all myself would've taken the afternoon. Then I went back to working on the actual hard stuff that my human brain was needed to solve. I can't imagine being so specialized that I only ever work on difficult problems within my niche and nothing else. There's always some extra query to write, some API to interface with, some tests to write... it's not a matter of being able to do it myself, it's a matter of being able to focus primarily on the stuff I need to do myself. Being able to offload the menial work to an AI also just changes the sorts of stuff I'm willing to do with my time. As a standalone software engineer, I will often choose not to write some simple'ish tool or script that might be useful because it might not be worth my time to write it, especially factoring in the cost of context switching. Nothing ground breaking, just something that might not be worth half an hour of my time. But I can just tell AI to write the script for me and I get it in a couple minutes. So instead of doing all my work without access to some convenient small custom tools, now I can do my work with them, with very little change to my workflow.
girvo 3y ago
It struggles with (industrial, not hobbyist) embedded firmware a fair bit. I can almost coax decent results for simple tasks out of it, sometimes.
- clnq 3y ago
  LLMs almost never write good senior quality code at first in niche disciplines. You need to finesse it a lot to have it produce the correct answer. And that makes it unusable for when you genuinely do not know the answer to the question you’re asking, which is kind of the entire point.

chalcolithic 3y ago

How long ago would you have considered this discussion ridiculous? How long till GPT-N will be churning out solutions faster than you can read them? It's useless for me now as well, but I'm pretty sure I'll be doomed professionally in the future.

jeffreygoesto 3y ago
Not necessarily. Every hockey stick is just the beginning of an s-curve. It will saturate, probably sooner than you think.
- hackerlight 3y ago
  Some parts of AI will necessarily asymptote to human-level intelligence because of a fixed corpus of training data. It's hard to think AI will become a better creative writer than the best human creative writers, because the AI is trained on their output and you can't go much further than that. But in areas where there's self-play (e.g. Chess, and to a lesser extent, programming), there is no good reason to think it'll saturate, since there isn't a limit on the amount of training data.
  - borissk 3y ago
    So you think human readers have magical powers to rate say a book that an AI can't replicate?
    - hackerlight 3y ago
      There's a gulf of difference between domains where self-play means we have unlimited training data for free (e.g. Chess) versus domains where there's no known way to generate more training data (e.g. Fine art). It's possible that the latter domains will see unpredictable innovations that allow it to generate more training data beyond what humans have produced, but that's an open question.
  - strken 3y ago
    How does programming have self-play? I'm not sure I understand. Are you going to generate leetcode questions with one AI, have another answer them, and have a third determine whether the answer is correct? I'm struggling to understand how an LLM is meant to answer the questions that come up in day-to-day software engineering, like "Why is the blahblah service occasionally timing out? Here are ten bug reports, most of which are wrong or misleading" or "The foo team and bar team want to be able to configure access to a Project based on the sensitivity_rating field using our access control system, so go and talk to them about implementing ABAC". The discipline of programming might be just a subset of broader software engineering, but it arguably still contains debugging, architecture, and questions which need more context than you can feed into an LLM now. Can't really self-play those things without interacting with the real world.

Zaofy 3y ago

Same here. I'm not a developer. I do engineering and architecture in IAM. I've tested out GPT-4 and it's good for general advice or problem solving. But it can't know the intricascies of the company I work at with all our baggage, legacy systems and us humans sometimes just being straight up illogical and inefficient with what we want. So my usage has mostly been for it to play a more advanced rubber duck to bounce ideas and concepts off of and to do some of the more tedious scripting work (that I still have to double check thoroughly). At some point GPT and other LLMs might be able to replace what I do in large parts. But that's still a while off.

Tainnor 3y ago

Same. Even for technologies that it supposedly should know a lot about (e.g. Kafka), if I prompt it for something slightly non-standard, it just makes up things that aren't supported or is otherwise unhelpful. The one time I've found ChatGPT to be genuinely useful is when I asked it to explain a bash script to me, seeing as bash is notoriously inscrutable. Still, it did get a detail wrong somehow.

clnq 3y ago
Yes, it is good at summarizing things and regressing things down to labels. It’s much worse at producing concrete and specific results from its corpus of abstract knowledge. I think that’s the case with every discipline for it, not only programming. Even when everyone was amazed it could make poetry out of everything, if you asked for a specific type of poem and specific imagery in it, it would generally fail.

gwd 3y ago

Well no, you shouldn't use it for your top-end problems, but your bottom-end problems. Aren't there things that you have to do in your job that really could be done by a junior programmer? Don't you ever have one-off (or once-a-year) things you have to do that each time you have to invest a lot of time refreshing in your brain, and then basically forgetting for lack of use? Here's an example I used the other day: Our project had lost access to our YT channel, which had 350+ videos on it (due to someone's untimely passing and a lack of redundancy). I had used yt-dlp to download all the old videos, including descriptions. Our community manager had uploaded all the videos, but wasn't looking forward to copy-and-pasting every description into the new video. So I offered to use GPT-4 to write a python script to use the API to do that for her. I didn't know anything about the YT API, nor am I an expert in python. I wouldn't have invested the time learning the YT API (and trying to work through my rudimentary python knowledge) for a one-off thing like this, but I knew that GPT-4 would be able to help me focus on what to do rather than how to do it. The transcript is here: https://chat.openai.com/share/936e35f9-e500-4a4d-aa76-273f63c9de20 https://chat.openai.com/share/936e35f9-e500-4a4d-aa76-273f63... By contrast, I don't think there's any possible way the current generation could have identified, or helped fix, this problem that I fixed a few years ago: https://xenbits.xenproject.org/xsa/xsa299/0011-x86-mm-Don-t-drop-a-type-ref-unless-you-held-a-ref-t.patch https://xenbits.xenproject.org/xsa/xsa299/0011-x86-mm-Don-t-... (Although it would be interesting to try to ask it about that to see how well it does.) The point of using GPT-4 should be to take over the "low value" work from you, so that you have more time and mental space to focus on the "high value" work.

ioseph 3y ago
Perhaps by learning to use the YT API (seriously something that should take 2 hours max if you know how http works) you'll learn something from their design choices, or develop opinions on what makes a good API. And by learning a bit more python you'll get exposed to patterns you could use in your own language.
- gwd 3y ago
  If anything, using GPT-4 makes a lot of that more efficient. Rather than scrolling through loads of API documentation trying to guess how to do something, writing Python with a "C" accent, I can just read the implementation that GPT-4 spits out, which is almost certainly based on seeing hundreds of examples written by people who are fluent in python, and thus using both to best effect.