4 ms·

My biggest problem with this stuff is that it looks correct, but it’s often subtly wrong. Systems built with stitched together GPT generated code are going to p

My biggest problem with this stuff is that it looks correct, but it’s often subtly wrong. Systems built with stitched together GPT generated code are going to provide the next generation’s buffer overflow exploits. It’s not just code. My wife is a physician and I got her to do a few medical prompts with ChatGPT. The output looked correct the me, and if I read it somewhere I would completely have accepted it. But she could point out numerous severe flaws.

Terretta 4y ago
Agreed, and mitigations are in order if using seriously. For knowledge derived responses, the questions require (a) expertise in formulation, (b) expertise in interpretation. Also turn down the temperature, while keeping repetition minimal and token reuse low, to explore closer to its latent correlations.
saimiam 4y ago
This is super interesting. Can you share some prompts that you recall had believable but wrong responses?
- sarchertech 4y ago
  She asked it to generate a practice emergency medicine board question. It left off some necessary vital signs from the scenario, then in the answer concluded that the patient had severe pneumonia and should be treated with a specific antibiotic. Based on the prompt, the pneumonia probably wasn’t severe, antibiotics weren’t warranted, and even if they were, the antibiotic specified was completely wrong. It looked fine to me—completely plausible. It took an expert to spot the flaws. She also asked it to explain the work up for a febrile neonate. The answer again sounded plausible to me. But she said it sounded like a clueless med student trying to BS their way through an answer.
ly3xqhl8g9 4y ago
Our brain literally wants us to believe that a pen can pass through a window [1] (3:00). Ants, as a collective intelligence, are tricked by the Müller-Lyer illusion [2] [3]. Digital software is merely entering a realm of algorithmic (d)efficiency at least as old as biology, morphogenetic software: so long, be gone abstract truth table resilience unable to detect a shirt without stripes [4], welcome gradient exploration and error minimization able to give the synthetic mind, which, similarly to the carbon-based mind, will make ridiculous errors, just look at a child failing to walk [5]. [1] Ames Window https://www.youtube.com/watch?v=0KrpZMNEDOY https://www.youtube.com/watch?v=0KrpZMNEDOY [2] https://www.researchgate.net/publication/259354882_The_Muller-Lyer_Illusion_in_Ant_Foraging https://www.researchgate.net/publication/259354882_The_Mulle... [3] https://en.wikipedia.org/wiki/M%C3%BCller-Lyer_illusion https://en.wikipedia.org/wiki/M%C3%BCller-Lyer_illusion [4] https://github.com/elsamuko/Shirt-without-Stripes https://github.com/elsamuko/Shirt-without-Stripes [5] https://media.tenor.com/uB5ijGdseFwAAAAC/stumble-haha.gif https://media.tenor.com/uB5ijGdseFwAAAAC/stumble-haha.gif
nerdponx 4y ago
As with many things "AI", the benefit is not to replace humans, but to free them from the tedium of doing the tedious non-creative parts of their job, so that they have more resources to do the parts that require creativity and human reasoning. What the new generation of models is showing us, and what we've been gradually seeing over decades of AI research, is that there is a lot more tedium and a lot less creativity in our work than we thought. Look at how much this repo got right, even if there are bugs and nonsensical sections (I didn't see any upon quick skim, but I've also never implemented my own language before). It would take so much less time to debug this program than it would to type all that code from scratch. AI consistently turns out to be a human effort accelerator, and not a human effort replacer. Maybe in the future it will be good enough to replace humans, but even what is now tried-and-tested technology like machine translation, spam classification, and object detection in images still requires a human in the loop to double-check the results. I'm pessimistic about the AI takeover, but I'm becoming very optimistic about AI precipitating a small industrial revolution. My only concern is about who gets to reap the benefits, but that debate predates the first industrial revolution.
- deleted 4y ago
  [deleted]
gvkhna 4y ago
Just have ChatGPT write Rust code, so at least that’s not a concern. Because it’s so fast, it’s probably faster to iterate with ChatGPT than the majority of people writing Rust because it’s a pretty difficult language.
- ricardobayes 4y ago
  Did anyone try JS/React yet?
- int_19h 4y ago
  This will take care of memory safety and concurrency, but not the logic bugs that just cause the state to become subtly invalid with no diagnostic (and then things break later because of that invalid state). I think we might need to start designing PLs with AI generation and processing specifically in mind. Explicit contracts everywhere might be a good starting point.
ricardobayes 4y ago
For sure, trust is going to be a major thing with this. I can certainly see 'Certified AI' being a thing for certain decision-aiding software. But, even though it's seemingly superior, no one trusts blockchain to store a real estate title. We collectively trust a lawyer with a freaking piece of paper.

Terretta 4y ago

Agreed, and mitigations are in order if using seriously. For knowledge derived responses, the questions require (a) expertise in formulation, (b) expertise in interpretation. Also turn down the temperature, while keeping repetition minimal and token reuse low, to explore closer to its latent correlations.

saimiam 4y ago

This is super interesting. Can you share some prompts that you recall had believable but wrong responses?

sarchertech 4y ago
She asked it to generate a practice emergency medicine board question. It left off some necessary vital signs from the scenario, then in the answer concluded that the patient had severe pneumonia and should be treated with a specific antibiotic. Based on the prompt, the pneumonia probably wasn’t severe, antibiotics weren’t warranted, and even if they were, the antibiotic specified was completely wrong. It looked fine to me—completely plausible. It took an expert to spot the flaws. She also asked it to explain the work up for a febrile neonate. The answer again sounded plausible to me. But she said it sounded like a clueless med student trying to BS their way through an answer.

ly3xqhl8g9 4y ago

Our brain literally wants us to believe that a pen can pass through a window [1] (3:00). Ants, as a collective intelligence, are tricked by the Müller-Lyer illusion [2] [3]. Digital software is merely entering a realm of algorithmic (d)efficiency at least as old as biology, morphogenetic software: so long, be gone abstract truth table resilience unable to detect a shirt without stripes [4], welcome gradient exploration and error minimization able to give the synthetic mind, which, similarly to the carbon-based mind, will make ridiculous errors, just look at a child failing to walk [5]. [1] Ames Window https://www.youtube.com/watch?v=0KrpZMNEDOY https://www.youtube.com/watch?v=0KrpZMNEDOY [2] https://www.researchgate.net/publication/259354882_The_Muller-Lyer_Illusion_in_Ant_Foraging https://www.researchgate.net/publication/259354882_The_Mulle... [3] https://en.wikipedia.org/wiki/M%C3%BCller-Lyer_illusion https://en.wikipedia.org/wiki/M%C3%BCller-Lyer_illusion [4] https://github.com/elsamuko/Shirt-without-Stripes https://github.com/elsamuko/Shirt-without-Stripes [5] https://media.tenor.com/uB5ijGdseFwAAAAC/stumble-haha.gif https://media.tenor.com/uB5ijGdseFwAAAAC/stumble-haha.gif

nerdponx 4y ago

As with many things "AI", the benefit is not to replace humans, but to free them from the tedium of doing the tedious non-creative parts of their job, so that they have more resources to do the parts that require creativity and human reasoning. What the new generation of models is showing us, and what we've been gradually seeing over decades of AI research, is that there is a lot more tedium and a lot less creativity in our work than we thought. Look at how much this repo got right, even if there are bugs and nonsensical sections (I didn't see any upon quick skim, but I've also never implemented my own language before). It would take so much less time to debug this program than it would to type all that code from scratch. AI consistently turns out to be a human effort accelerator, and not a human effort replacer. Maybe in the future it will be good enough to replace humans, but even what is now tried-and-tested technology like machine translation, spam classification, and object detection in images still requires a human in the loop to double-check the results. I'm pessimistic about the AI takeover, but I'm becoming very optimistic about AI precipitating a small industrial revolution. My only concern is about who gets to reap the benefits, but that debate predates the first industrial revolution.

deleted 4y ago
[deleted]

gvkhna 4y ago

Just have ChatGPT write Rust code, so at least that’s not a concern. Because it’s so fast, it’s probably faster to iterate with ChatGPT than the majority of people writing Rust because it’s a pretty difficult language.

ricardobayes 4y ago
Did anyone try JS/React yet?
int_19h 4y ago
This will take care of memory safety and concurrency, but not the logic bugs that just cause the state to become subtly invalid with no diagnostic (and then things break later because of that invalid state). I think we might need to start designing PLs with AI generation and processing specifically in mind. Explicit contracts everywhere might be a good starting point.

ricardobayes 4y ago

For sure, trust is going to be a major thing with this. I can certainly see 'Certified AI' being a thing for certain decision-aiding software. But, even though it's seemingly superior, no one trusts blockchain to store a real estate title. We collectively trust a lawyer with a freaking piece of paper.