4 ms·

I think ChatGPT is pretty neat but I was somewhat less impressed than everyone else is with the code generation. I actually agree with Wolfram for a change; bei

I think ChatGPT is pretty neat but I was somewhat less impressed than everyone else is with the code generation. I actually agree with Wolfram for a change; being human like isn’t always “good”. For example, I asked ChatGPT to give me a proof of Fermat’s Last Theorem in Isabelle/Isar. It quickly gave me some very pretty Isar code, and I was extremely impressed. Until I tried actually inputting it into Isabelle, and nothing worked. I then started reading the proof, and noticed it was making extremely basic algebra mistakes [1] that even a high schooler would be chastised for. Moreover, I even if I allowed these mistakes with “sorry” [2], the conclusion in the proof didn’t actually follow from the steps. Granted, Fermats Last Theorem is a tricky proof so I understand it struggling with it, but I would have much preferred if it had said “I don’t know how to write proofs in Isabelle” instead of giving something that looks plausible. [1] it seemed extremely convinced that “a^n + b^n = c^n” could be rewritten as “(a + b)^n - c^n = 0” [2] “sorry” in Isabelle basically means “assume this is true even if I didn’t prove it. It’s useful but dangerous.

alfalfasprout 4y ago
It's quickly apparent that the people impressed with ChatGPT's code are generally solving pretty toy problems and want an alternative to SO for "how to do X".
- tombert 4y ago
  And that’s fine for the average HN audience; what disturbs me is that hiring managers see articles like “I had ChatGPT rewrite my code and it went 100000% faster!!!”, and then decide that they don’t need more than one engineer. If we could automate away engineers that’s fine with me, but i just don’t think that ChatGPT is there yet. I actually think Wolfram is kind of onto something with having ChatGPT work with Alpha to be used for stuff that has “objective” answers.
  - ryokeken 4y ago
    that's what unions are for if only an engineer could engineer a solution to what afflicts unions, dat be great
  - alfalfasprout 4y ago
    The comment above captures it well though-- many of us aren't just writing stuff you can copy/paste from stackoverflow. When you're working on things where you're operating with internal business concepts and data with set meanings the difficulty is rarely writing the code but rather reasoning about how to cleanly approach the problem in the first place. The thing is, the hiring managers reacting like that to ChatGPT are also the same folks who would've considered outsourcing engineering anyways to one of those offshore "consultancies". Tools like ChatGPT will ultimately be a replacement for use cases where you've already fully scoped out what to build and how to build it and the implementation is straightforward.
- jerf 4y ago
  One of the problems with ChatGPT is that for the most part, the things that people are most likely to feed it are also the things it is most likely to have training data for. So, you say, "Write me a heap for Java", and it spits out amazing, wonderful code that is almost right. Well, yeah, that's cool and all, though I would point out that its training data probably included a heap that was entirely correct, but, still, it's a common question. But I'm not writing a heap for Java. I'm taking business object X and stuffing through system Y and feeding the result to system Z, all internal to the company I work for. Good luck with getting ChatGPT to do that. But it deceives people, because the very questions they are most likely to ask as a test are the exact questions it knows the answer to, for the very reason that they are also the most likely questions to be in the training data. (This article kind of doubles as great set of examples of just how quickly you get into confident nonsense if you stick a toe slightly out of line in your query. For instance, even if it knows "What is Y" and "What is Z", it can't necessarily combine them in any correct way. It'll spew out a lot of words that sound very confident, but it'll make mistakes without any indication they were mistakes.)
  - alfalfasprout 4y ago
    > But I'm not writing a heap for Java. I'm taking business object X and stuffing through system Y and feeding the result to system Z, all internal to the company I work for. Good luck with getting ChatGPT to do that. Exactly. Nearly every example I've seen is something that I could easily find an answer for on SO for anyways. ChatGPT customizes it as needed which is nice (though it often makes glaring and not so glaring mistakes).
    - phatfish 4y ago
      It's interesting, StackOverflow will give you (after sufficient searching) an existing solution close to what you want. It's pretty certain to be accurate/working in the context of the question asked on SO. You as the "problem solver" just have to adapt the similar solution to your own problem. ChatGPT will try to respond with exactly what you want, but with (multiple) errors that must be fixed. I'll take the working example and adapt it myself. There was a blog post on SO about this recently, they are banning accounts posting ChatGPT answers because they appear reasonable at first glance, but try and execute the code and it has syntax errors or is just plain wrong.
- IshKebab 4y ago
  Well yes because that's about the limit of what it is capable of at the moment, but that limit is very impressive! I don't know why so many people are unimpressed with it because it isn't some kind of all knowing hyperintelligent god. This guy is literally unimpressed with it because it can't solve Fermat's last theorem! Wtf?
  - tombert 4y ago
    I said it was “neat”! It’s impressive tech, but giving a false-but-believable proof is substantially worse than saying “I don’t know how to do that”. If it had come back with “I don’t know how to give a proof of Fermat’s Last Theorem in Isabelle”, I wouldn’t have been unimpressed; there are plenty of very cool things it can do. I just think that it being confidently incorrect is a bad thing. EDIT: I should point out, it also was wrong with substantially simpler proofs, like the proof of infinite primes.
phphphphp 4y ago
> I think ChatGPT is pretty neat but I was somewhat less impressed than everyone else is with the code generation. specifically on code generation, even if it could produce the correct code, it's basically faster horses: if you ask a programmer what they want, it's something to generate rote code for them so they don't have to write it... but that's the wrong problem. The problem is not that code is hard to write, or that we don't have enough code, it's that we have too much code and need to write code to solve previously solved problems in the first place. The future is platforms like Retool that reduce the need to write code, the future is businesses producing less code, the future is businesses producing code to solve new problems -- which an LLM can't do. If code has been written so many times that a language learning model can output it, why on earth are we still producing it (whether that's writing it or inducing a robot to output it)? Code generation might be a fun novelty for a little while but it's not the future, it'll be a distant memory soon enough.
- CuriouslyC 4y ago
  Code generation isn't going anywhere. No/low code platforms are great for some subset of users, but there will always be people who want total control to differentiate themselves, since differentiation is how you get customers in a sea of competitors. That total control requires code of some sort. Additionally, even if 95% are on platforms, the platforms are on code, and being able to rapidly iterate your platform compared with your competitors is how you're going to win. That means any tools that help you iterate with code faster will be invaluable.
  - phphphphp 4y ago
    Code is a tool for businesses, and while it's a tool that can enable you to do anything you can imagine, it is an expensive tool for businesses to do anything they can imagine... and how many businesses need to be able to do anything they can imagine? The cost of code is huge, not only do you need competent software engineers to write the code, but first you need people who can translate business requirements into technical requirements... and then you're burdened with the costs of running, maintaining and iterating on that code as the business evolves. If you could cut all of that out and instead have the people defining the business requirements design the software (through the use of a no-code platform) your business would be orders of magnitude more efficient. Historically, magical all-in-one configurable platforms (like Salesforce) were unwieldy and required experts to operate them, but that's no longer the case. The world is becoming much more mature, and businesses can achieve so much without owning a single line of code. There'll always be technology that needs to be built, and code is a great tool for building technology, but most businesses don't need to build technology, they need to use technology. If you're opening your first grocery store, you rent a building, you don't hire architects and construction workers. The immaturity of technology has historically made that approach to business quite difficult, however, that's rapidly changing. As Software Engineers, we get caught up in believing that because you can do anything with code, code is the right tool. That's a misunderstanding of business, though: given the choice between 2 tools, 1 that can do 90% of what the business needs and 1 that can do 100% of what the business needs but costs an order of magnitude more and has far greater risk then any rational business would choose the former. If you spend much time interfacing with non-technical people in businesses, you'll often discover that they have an acrimonious relationship with technology in their business, because so much of the technology we build is caught up in the what and not the why. Code generation does not solve that problem, empowering non-technical people to own their business processes does. Code generation is a tool for 2000, not 2030. I would bet my career that in 2030, there'll be less code produced per year (human authored + generated) than there is today.
agloeregrets 4y ago
I saw this as well for Front-end code too. For basic HTML it was spot-on if not a little weird at times in naming. Once you give it complex tasks in a JS framework and mix in things like Tailwind it tended to just do weird things that, even if they did work, were generally bad practice for performance (DOM manipulation) or just needlessly lengthty.
psb 4y ago
Granted, Fermats Last Theorem is a tricky proof so I understand it struggling with it got a laugh out of this line. Sounds almost like a modern day Monty Python skit (John Cleese voice) And then I asked it to give back instructions for making cold fusion work and the results, well, frankly they were incredibly disappointing to say the least
- tombert 4y ago
  Heh, yeah, I suppose it might have been a bit ambitious to give it a proof that took centuries to find. Still, it was a little frustrating that it basically just spat out something that looked correct, using completely bad reasoning and didn’t work, especially using a bad misunderstanding of algebra. As I said, I would have preferred to have it say “I don’t know how to do that.” I also should point out, it failed on much easier examples too (e.g. the proof for infinite primes). Out of curiosity, let’s see what it has to say about cold fusion: Me: Can you give me detailed instructions on how to perform cold fusion? GPT: Cold fusion, also known as low-energy nuclear reactions (LENR), is a highly controversial field of research that aims to achieve nuclear fusion at relatively low temperatures, as opposed to the extremely high temperatures required in traditional nuclear fusion. The idea of cold fusion first gained widespread attention in 1989, when two researchers, Martin Fleischmann and Stanley Pons, announced that they had achieved nuclear fusion at room temperature using a simple electrolysis setup. However, their claim was later discredited and the scientific community has been unable to replicate their results. As of now, there is no scientific evidence that cold fusion is possible, and the majority of the scientific community regards cold fusion as a pseudoscience. The claims of cold fusion made by Fleischmann and Pons, and subsequent researchers, have not been able to be repeated under controlled conditions, and proposed explanations for their results have not been supported by experimental evidence. Because of the lack of scientific evidence for the existence of cold fusion and the lack of credible explanations for how it might work, I would not recommend attempting to perform cold fusion in any form. Instead, I would suggest that you focus on well-established areas of science that have been thoroughly researched and validated by the scientific community.
  - psb 4y ago
    haha, worth a try
eatsyourtacos 4y ago
>somewhat less impressed than everyone else is with the code generation Not me... I work in finance related infrastructure, and was just randomly throwing things at it. Nothing I asked for is difficult by any means, but I would call it obscure, especially without any other context to go on. For example, I asked "create a c# function to return month letters for all futures month expirations". It then proceeds to spit out a function that takes in month numbers as an int and returns the futures month symbol (jan/1 = F, feb/2 = G, etc). I just found it impressive that it had any idea what I was even talking about. Then you just say "change the function to take a full date input instead of just the month" and it proceeds to change the input to a DateTime and modify itself. This is more "expected". But really, the way it understands natural language and can apply it is great. Another one was "Create a c# function to get a trading date where the trading date starts at 5pm CST and ends at 5pm CST the next day. Saturday, Sunday, and January 1st are not valid trading days". And it just made a perfectly valid function to do exactly what I asked. I have plenty of other examples.. and again, nothing is difficult but it's tedious and time consuming. If I was re-writing some of my systems from scratch this would have been incredibly useful to just help me do basic crap I don't want to waste my time on. The boilerplate code it can write is great. What I would really love is for it to ingest my library and then be able to create helpful functions with even more context.. that is where this stuff is REALLY going to be useful.
gptgpp 4y ago
The thing is, I think ChatGPT is already using a program underneath to generate it's response on some math questions? Like if you ask it to derive/integrate it will follow the identical format every time, like it's running it through a symbolic algebra calculator. And then if you ask it something slightly more advanced, like to compute the eigenvector of a small matrix, it completely fails. In general I found it failing nearly all basic linear algebra questions I gave it except for very simple ones like Gaussian elimination. Then I asked it about some of the larger mersenne primes and it got some of them wrong too, which is surprising because those are precomputed (yet I had to triple check it because it sounded extremely confident in it's answer, and wouldn't change it). For programming I completely agree it's not great either... at least in my preferred language (Rust). I asked it to write a program that output an MD5 hash, and it got the general idea (which, to be clear, IS REALLY COOL) but it was riddled with errors, with five of them being serious enough to prevent it from compiling. I tried to get it to refactor them and it couldn't do that either, even with me telling it what the errors were. On top of that it had no optimization. I would be surprised to see it being capable of replacing even hobbyist programmers in the next decade or so. In the social sciences it's also way to limited by it's content filters and canned response templates to be useful. For example, I asked it to list the factors that cause Canadian indigenous women suffer 20x the incarceration as Canadian caucasian women (The correct answers are, first and foremost, unequal law enforcement and bias in the justice system, as well as SES inequalities, lack of mental health and addiction support in rural areas, etc. -- it did not get any of these correct) and it gives me the SAME canned template as it does if you ask it about literally any other visible minority in any other country. Pretty bad, although I understand why it would tread lightly around questions involving ethnicity and inequality. So I agree, math and programming it's almost unusable, and in other fields the over-judicious filters and template responses make it useless too. I've yet to find a good use case for it, personally -- I suspect where it excels is at writing letters, paraphrasing, and other simple language tasks.
BoiledCabbage 4y ago
I love watching the bar get raised in real time. If you told people 5 years ago you can have a conv with a ML system and barely be able to tell its not a person, ask it question about topics and get human like plausible answers, give it text requirements and it would respond with 95% working code people's jaws would drop. I mean even 6 months ago you could tell people that and the response wouldn't be all that different. Now I see so many shifted goal posts "yeah it can talk about most topics, but it's often wrong", "yeah it can generate code, but it has bugs! The nerve of anyone to write buggy code without a compiler to check it." (How well do you write code on the whiteboard at age 18 btw?), or "yes it writes code but not for my favorite language", or "yes it writes correct code but only for programming interview questions: or "yes but it's not optimized for performance". It's amazing how quickly the goal posts have shifted. To me it's absolutely astounding what it's capable of - and I see so many complaints are "it's not the computer from Stat Trek so it's really not much use at all". Yes it can be better improved with acknowledging when it isn't confident in something - but its still incredible what it's doing right now.
- thethirdone 4y ago
  I don't think its fair to call this raising the bar. I don't think anyone was imagining plausible yet nonsensical text to be the bar for AI. ChatGPT is incredibly good at generating plausible text, but nearly all of the time it fails to hold up to detailed scrutiny. It writes like a human who doesn't know what they are talking about, but have chosen to bullshit something instead of admit they don't know. At least for me, I would have expected more understanding to be necessary to get to ChatGPT levels of plausibility. I would not have expected a bot that does not understand two digit multiplication at all to be able to say anything vaguely convincing about primes, but ChatGPT can. Being thoroughly surprised by how ML improves and then restating your expectations is not the same as raising the bar.
  - dragonwriter 4y ago
    > ChatGPT is incredibly good at generating plausible text, but nearly all of the time it fails to hold up to detailed scrutiny. It writes like a human who doesn’t know what they are talking about, but have chosen to bullshit something instead of admit they don’t know. What do you expect of a model trained, largely, on the public internet? (Note, if it was trained on professionally published books, without quite a bit of selectivity, I would expect the same; this isn’t so much about the internet as about people.)

alfalfasprout 4y ago

It's quickly apparent that the people impressed with ChatGPT's code are generally solving pretty toy problems and want an alternative to SO for "how to do X".

tombert 4y ago
And that’s fine for the average HN audience; what disturbs me is that hiring managers see articles like “I had ChatGPT rewrite my code and it went 100000% faster!!!”, and then decide that they don’t need more than one engineer. If we could automate away engineers that’s fine with me, but i just don’t think that ChatGPT is there yet. I actually think Wolfram is kind of onto something with having ChatGPT work with Alpha to be used for stuff that has “objective” answers.
- ryokeken 4y ago
  that's what unions are for if only an engineer could engineer a solution to what afflicts unions, dat be great
- alfalfasprout 4y ago
  The comment above captures it well though-- many of us aren't just writing stuff you can copy/paste from stackoverflow. When you're working on things where you're operating with internal business concepts and data with set meanings the difficulty is rarely writing the code but rather reasoning about how to cleanly approach the problem in the first place. The thing is, the hiring managers reacting like that to ChatGPT are also the same folks who would've considered outsourcing engineering anyways to one of those offshore "consultancies". Tools like ChatGPT will ultimately be a replacement for use cases where you've already fully scoped out what to build and how to build it and the implementation is straightforward.
jerf 4y ago
One of the problems with ChatGPT is that for the most part, the things that people are most likely to feed it are also the things it is most likely to have training data for. So, you say, "Write me a heap for Java", and it spits out amazing, wonderful code that is almost right. Well, yeah, that's cool and all, though I would point out that its training data probably included a heap that was entirely correct, but, still, it's a common question. But I'm not writing a heap for Java. I'm taking business object X and stuffing through system Y and feeding the result to system Z, all internal to the company I work for. Good luck with getting ChatGPT to do that. But it deceives people, because the very questions they are most likely to ask as a test are the exact questions it knows the answer to, for the very reason that they are also the most likely questions to be in the training data. (This article kind of doubles as great set of examples of just how quickly you get into confident nonsense if you stick a toe slightly out of line in your query. For instance, even if it knows "What is Y" and "What is Z", it can't necessarily combine them in any correct way. It'll spew out a lot of words that sound very confident, but it'll make mistakes without any indication they were mistakes.)
- alfalfasprout 4y ago
  > But I'm not writing a heap for Java. I'm taking business object X and stuffing through system Y and feeding the result to system Z, all internal to the company I work for. Good luck with getting ChatGPT to do that. Exactly. Nearly every example I've seen is something that I could easily find an answer for on SO for anyways. ChatGPT customizes it as needed which is nice (though it often makes glaring and not so glaring mistakes).
  - phatfish 4y ago
    It's interesting, StackOverflow will give you (after sufficient searching) an existing solution close to what you want. It's pretty certain to be accurate/working in the context of the question asked on SO. You as the "problem solver" just have to adapt the similar solution to your own problem. ChatGPT will try to respond with exactly what you want, but with (multiple) errors that must be fixed. I'll take the working example and adapt it myself. There was a blog post on SO about this recently, they are banning accounts posting ChatGPT answers because they appear reasonable at first glance, but try and execute the code and it has syntax errors or is just plain wrong.
IshKebab 4y ago
Well yes because that's about the limit of what it is capable of at the moment, but that limit is very impressive! I don't know why so many people are unimpressed with it because it isn't some kind of all knowing hyperintelligent god. This guy is literally unimpressed with it because it can't solve Fermat's last theorem! Wtf?
- tombert 4y ago
  I said it was “neat”! It’s impressive tech, but giving a false-but-believable proof is substantially worse than saying “I don’t know how to do that”. If it had come back with “I don’t know how to give a proof of Fermat’s Last Theorem in Isabelle”, I wouldn’t have been unimpressed; there are plenty of very cool things it can do. I just think that it being confidently incorrect is a bad thing. EDIT: I should point out, it also was wrong with substantially simpler proofs, like the proof of infinite primes.

phphphphp 4y ago

> I think ChatGPT is pretty neat but I was somewhat less impressed than everyone else is with the code generation. specifically on code generation, even if it could produce the correct code, it's basically faster horses: if you ask a programmer what they want, it's something to generate rote code for them so they don't have to write it... but that's the wrong problem. The problem is not that code is hard to write, or that we don't have enough code, it's that we have too much code and need to write code to solve previously solved problems in the first place. The future is platforms like Retool that reduce the need to write code, the future is businesses producing less code, the future is businesses producing code to solve new problems -- which an LLM can't do. If code has been written so many times that a language learning model can output it, why on earth are we still producing it (whether that's writing it or inducing a robot to output it)? Code generation might be a fun novelty for a little while but it's not the future, it'll be a distant memory soon enough.

CuriouslyC 4y ago
Code generation isn't going anywhere. No/low code platforms are great for some subset of users, but there will always be people who want total control to differentiate themselves, since differentiation is how you get customers in a sea of competitors. That total control requires code of some sort. Additionally, even if 95% are on platforms, the platforms are on code, and being able to rapidly iterate your platform compared with your competitors is how you're going to win. That means any tools that help you iterate with code faster will be invaluable.
- phphphphp 4y ago
  Code is a tool for businesses, and while it's a tool that can enable you to do anything you can imagine, it is an expensive tool for businesses to do anything they can imagine... and how many businesses need to be able to do anything they can imagine? The cost of code is huge, not only do you need competent software engineers to write the code, but first you need people who can translate business requirements into technical requirements... and then you're burdened with the costs of running, maintaining and iterating on that code as the business evolves. If you could cut all of that out and instead have the people defining the business requirements design the software (through the use of a no-code platform) your business would be orders of magnitude more efficient. Historically, magical all-in-one configurable platforms (like Salesforce) were unwieldy and required experts to operate them, but that's no longer the case. The world is becoming much more mature, and businesses can achieve so much without owning a single line of code. There'll always be technology that needs to be built, and code is a great tool for building technology, but most businesses don't need to build technology, they need to use technology. If you're opening your first grocery store, you rent a building, you don't hire architects and construction workers. The immaturity of technology has historically made that approach to business quite difficult, however, that's rapidly changing. As Software Engineers, we get caught up in believing that because you can do anything with code, code is the right tool. That's a misunderstanding of business, though: given the choice between 2 tools, 1 that can do 90% of what the business needs and 1 that can do 100% of what the business needs but costs an order of magnitude more and has far greater risk then any rational business would choose the former. If you spend much time interfacing with non-technical people in businesses, you'll often discover that they have an acrimonious relationship with technology in their business, because so much of the technology we build is caught up in the what and not the why. Code generation does not solve that problem, empowering non-technical people to own their business processes does. Code generation is a tool for 2000, not 2030. I would bet my career that in 2030, there'll be less code produced per year (human authored + generated) than there is today.

agloeregrets 4y ago

I saw this as well for Front-end code too. For basic HTML it was spot-on if not a little weird at times in naming. Once you give it complex tasks in a JS framework and mix in things like Tailwind it tended to just do weird things that, even if they did work, were generally bad practice for performance (DOM manipulation) or just needlessly lengthty.

psb 4y ago

Granted, Fermats Last Theorem is a tricky proof so I understand it struggling with it got a laugh out of this line. Sounds almost like a modern day Monty Python skit (John Cleese voice) And then I asked it to give back instructions for making cold fusion work and the results, well, frankly they were incredibly disappointing to say the least

tombert 4y ago
Heh, yeah, I suppose it might have been a bit ambitious to give it a proof that took centuries to find. Still, it was a little frustrating that it basically just spat out something that looked correct, using completely bad reasoning and didn’t work, especially using a bad misunderstanding of algebra. As I said, I would have preferred to have it say “I don’t know how to do that.” I also should point out, it failed on much easier examples too (e.g. the proof for infinite primes). Out of curiosity, let’s see what it has to say about cold fusion: Me: Can you give me detailed instructions on how to perform cold fusion? GPT: Cold fusion, also known as low-energy nuclear reactions (LENR), is a highly controversial field of research that aims to achieve nuclear fusion at relatively low temperatures, as opposed to the extremely high temperatures required in traditional nuclear fusion. The idea of cold fusion first gained widespread attention in 1989, when two researchers, Martin Fleischmann and Stanley Pons, announced that they had achieved nuclear fusion at room temperature using a simple electrolysis setup. However, their claim was later discredited and the scientific community has been unable to replicate their results. As of now, there is no scientific evidence that cold fusion is possible, and the majority of the scientific community regards cold fusion as a pseudoscience. The claims of cold fusion made by Fleischmann and Pons, and subsequent researchers, have not been able to be repeated under controlled conditions, and proposed explanations for their results have not been supported by experimental evidence. Because of the lack of scientific evidence for the existence of cold fusion and the lack of credible explanations for how it might work, I would not recommend attempting to perform cold fusion in any form. Instead, I would suggest that you focus on well-established areas of science that have been thoroughly researched and validated by the scientific community.
- psb 4y ago
  haha, worth a try

eatsyourtacos 4y ago

>somewhat less impressed than everyone else is with the code generation Not me... I work in finance related infrastructure, and was just randomly throwing things at it. Nothing I asked for is difficult by any means, but I would call it obscure, especially without any other context to go on. For example, I asked "create a c# function to return month letters for all futures month expirations". It then proceeds to spit out a function that takes in month numbers as an int and returns the futures month symbol (jan/1 = F, feb/2 = G, etc). I just found it impressive that it had any idea what I was even talking about. Then you just say "change the function to take a full date input instead of just the month" and it proceeds to change the input to a DateTime and modify itself. This is more "expected". But really, the way it understands natural language and can apply it is great. Another one was "Create a c# function to get a trading date where the trading date starts at 5pm CST and ends at 5pm CST the next day. Saturday, Sunday, and January 1st are not valid trading days". And it just made a perfectly valid function to do exactly what I asked. I have plenty of other examples.. and again, nothing is difficult but it's tedious and time consuming. If I was re-writing some of my systems from scratch this would have been incredibly useful to just help me do basic crap I don't want to waste my time on. The boilerplate code it can write is great. What I would really love is for it to ingest my library and then be able to create helpful functions with even more context.. that is where this stuff is REALLY going to be useful.

gptgpp 4y ago

The thing is, I think ChatGPT is already using a program underneath to generate it's response on some math questions? Like if you ask it to derive/integrate it will follow the identical format every time, like it's running it through a symbolic algebra calculator. And then if you ask it something slightly more advanced, like to compute the eigenvector of a small matrix, it completely fails. In general I found it failing nearly all basic linear algebra questions I gave it except for very simple ones like Gaussian elimination. Then I asked it about some of the larger mersenne primes and it got some of them wrong too, which is surprising because those are precomputed (yet I had to triple check it because it sounded extremely confident in it's answer, and wouldn't change it). For programming I completely agree it's not great either... at least in my preferred language (Rust). I asked it to write a program that output an MD5 hash, and it got the general idea (which, to be clear, IS REALLY COOL) but it was riddled with errors, with five of them being serious enough to prevent it from compiling. I tried to get it to refactor them and it couldn't do that either, even with me telling it what the errors were. On top of that it had no optimization. I would be surprised to see it being capable of replacing even hobbyist programmers in the next decade or so. In the social sciences it's also way to limited by it's content filters and canned response templates to be useful. For example, I asked it to list the factors that cause Canadian indigenous women suffer 20x the incarceration as Canadian caucasian women (The correct answers are, first and foremost, unequal law enforcement and bias in the justice system, as well as SES inequalities, lack of mental health and addiction support in rural areas, etc. -- it did not get any of these correct) and it gives me the SAME canned template as it does if you ask it about literally any other visible minority in any other country. Pretty bad, although I understand why it would tread lightly around questions involving ethnicity and inequality. So I agree, math and programming it's almost unusable, and in other fields the over-judicious filters and template responses make it useless too. I've yet to find a good use case for it, personally -- I suspect where it excels is at writing letters, paraphrasing, and other simple language tasks.

BoiledCabbage 4y ago

I love watching the bar get raised in real time. If you told people 5 years ago you can have a conv with a ML system and barely be able to tell its not a person, ask it question about topics and get human like plausible answers, give it text requirements and it would respond with 95% working code people's jaws would drop. I mean even 6 months ago you could tell people that and the response wouldn't be all that different. Now I see so many shifted goal posts "yeah it can talk about most topics, but it's often wrong", "yeah it can generate code, but it has bugs! The nerve of anyone to write buggy code without a compiler to check it." (How well do you write code on the whiteboard at age 18 btw?), or "yes it writes code but not for my favorite language", or "yes it writes correct code but only for programming interview questions: or "yes but it's not optimized for performance". It's amazing how quickly the goal posts have shifted. To me it's absolutely astounding what it's capable of - and I see so many complaints are "it's not the computer from Stat Trek so it's really not much use at all". Yes it can be better improved with acknowledging when it isn't confident in something - but its still incredible what it's doing right now.

thethirdone 4y ago
I don't think its fair to call this raising the bar. I don't think anyone was imagining plausible yet nonsensical text to be the bar for AI. ChatGPT is incredibly good at generating plausible text, but nearly all of the time it fails to hold up to detailed scrutiny. It writes like a human who doesn't know what they are talking about, but have chosen to bullshit something instead of admit they don't know. At least for me, I would have expected more understanding to be necessary to get to ChatGPT levels of plausibility. I would not have expected a bot that does not understand two digit multiplication at all to be able to say anything vaguely convincing about primes, but ChatGPT can. Being thoroughly surprised by how ML improves and then restating your expectations is not the same as raising the bar.
- dragonwriter 4y ago
  > ChatGPT is incredibly good at generating plausible text, but nearly all of the time it fails to hold up to detailed scrutiny. It writes like a human who doesn’t know what they are talking about, but have chosen to bullshit something instead of admit they don’t know. What do you expect of a model trained, largely, on the public internet? (Note, if it was trained on professionally published books, without quite a bit of selectivity, I would expect the same; this isn’t so much about the internet as about people.)