6 ms·

OpenAI has solved the XY problem

70 points by nullc 5y ago

humanistbot 5y ago
I can't read this article because of the awful background color.
- chrisco255 5y ago
  This is vintage 90s web aesthetic. I dig it. Only thing it's missing is a view counter and a guest book.
  - YeGoblynQueenne 5y ago
    They're there but they're in the same colour as the background.
- nullc 5y ago
  If you have javascript enabled: Click the "Paint bikeshed" emoji-sequence at the top right until your fitness function has been maximized.
  - refulgentis 5y ago
    I see your point and, being a color nerd, see it as an opportunity to argue that is no bikeshed sir!!! TL;DR: Bump it to #638046. - text is black, #000000. - background is #4A662E. - tone/L* of text is 0 - tone/L* of background is 40 - elements smaller than 18 pts should have a contrast ratio of 4.5 or more - difference in tone for 4.5 contrast ratio is 50 - take background at T40, adjusted to T50: #4A662E turns to #638046
    - nullc 5y ago
      On my decidedly yolo-colors laptop display, I like that ever so slightly less. But not ever-so-slightly enough to fail to defer to a highly actionable and justified display of nerdity. Thank you.
      - refulgentis 5y ago
        Who downvoted this!? We had a fun light-hearted exchange! Thank you for indulging my bike-shedding sir :)
- totetsu 5y ago
  I cant read the article because its blocked by my workplaces proxy.
- Crosseye_Jack 5y ago
  Reader Mode cleaned it up for me.
sooham 5y ago
FYI: The title is a sarcastic take on OpenAI's codex lack of nuance, not that the XY problem has been solved.
version_five 5y ago
If I ask how to write a bash script to get the last 3 letters of a file name, my preferred answer would be the literal answer. If someone wants to pontificate below that answer that maybe I wanted the extension, that's cool, but first answer the question or don't bother participating. People always invoke XY problem as if it's a way of helping someone, when it's mostly just condescension and or showing off when applied to forum answers. If someone is looking for "advice", they can ask for it. If someone is looking for syntax, they don't necessarily want advice, even if you really think they need it. That's my advice.
- nullc 5y ago
  Well there you go, OpenAI codex has solved your problem: technical help that won't bother warning you when you're about to shoot yourself in the foot, if that's really what you want. :) > when it's mostly just condescension and or showing off when applied to forum answers. You might be reading emotion and motivation into text that simple isn't there. I do wonder if codex improves and asks clarifying questions if you'll attribute condescension to its actions? > but first answer the question or don't bother participating Sounds like you might be selfishly confusing other human beings for a coin operated answer vending machine that exists for your personal gratification. Or maybe I'm reading too much into your "just answer or don't participate"? :D (I don't have any reason to believe it of you-- but I would assume that it's no less true than the windbaggy responders replying that way to show off their skills and mock your lack of them) In any case... that is the point: If you want a answer vending machine that doesn't care if what you asked for might not be what you want, an actual machine will do a better job of it. Personally, I'm grateful when someone backs me out of a niche I've reasoned myself into ... and I don't mind having to spend more time on my questions to qualify that I've already tried the obvious. I've probably solved more of my own problems just by stepping back to explain what I'm trying to do and what and then noticing the error on my own, than by any other means. But even if I did mind-- even if the people responding really were just showing off-- if that's the price of free help it still seems like a good deal. I'd much rather get help from someone with a chance of warning me that my whole approach might be confused, in my view that's 95% of the value of learning from another mind as opposed to mashing buttons until the tests pass.
alexjurkiewicz 5y ago
Using Bash examples was a bad idea. Bash scripts are by-and-large hacky things written to solve a simple problem expediently. OpenAI's output code reflects this reality. Expecting OpenAI to write Bash code defensively is like expecting OpenAI to complete SMS messages with scientific english.
- skeaker 5y ago
  I was going to ask a similar thing. Does this AI not pull from a massive pool of countless examples of real human-written code? In the majority of cases, perfect code is far from necessary (put another way, solving Y will generally suffice even if you should be focusing on X), and that seems to be reflected in the sample data that the AI is most likely pulling from. Obviously it isn't perfect, but I never expected it to be perfect, so I'm left more impressed that the AI functions at all than I am upset that it doesn't function absolutely perfectly in every way.
  - nullc 5y ago
    I think it's performance is quite impressive. Consider, the very last example is a better implementation than most here would write (at least without some research). The cheeky tone is not targeted towards the machine. > Does this AI not pull from a massive pool of countless examples of real human-written code Pull from wouldn't be the right way of looking at it-- it's trained on and has presumably memorized significant amounts. But thus far no one has equip this kind of model with something like 'database access', beyond its own quadratic-cost internal self-attention.
    - saurik 5y ago
      And yet only if the prompt includes "using fancy math". I'm still cracking up, and expect to still be cracking up over it tomorrow ;P.
renewiltord 5y ago
Damn, those examples are insanely good.
- nullc 5y ago
  I was pretty impressed by it managing to use fermat's theorem on pythagorean primes.
  - qayxc 5y ago
    Maybe it's just a literal match (Codex is known to quote code verbatim [0]) for existing training data? Given the popularity of the problem in both CS classes and coding competitions, there's a good chance a matching implementation was present in its Github-based training data more than just a few times. [0] https://thenewstack.io/this-week-in-programming-github-copilot-copyright-infringement-and-open-source-licensing/ https://thenewstack.io/this-week-in-programming-github-copil...
    - nullc 5y ago
      If you can find one, I'd be interested in knowing.
flafla2 5y ago
That last example is such a mic drop. I can’t tell if I’m impressed beyond belief or appalled.
- q-big 5y ago
  > That last example is such a mic drop. I can’t tell if I’m impressed beyond belief or appalled. I am appalled, because the code is wrong: consider p=2.
  - tromp 5y ago
    Yes; too bad the prompt wasn't for odd_prime_is_sum_of_two_squares ...

70 points by nullc 5y ago

humanistbot 5y ago

I can't read this article because of the awful background color.

chrisco255 5y ago
This is vintage 90s web aesthetic. I dig it. Only thing it's missing is a view counter and a guest book.
- YeGoblynQueenne 5y ago
  They're there but they're in the same colour as the background.
nullc 5y ago
If you have javascript enabled: Click the "Paint bikeshed" emoji-sequence at the top right until your fitness function has been maximized.
- refulgentis 5y ago
  I see your point and, being a color nerd, see it as an opportunity to argue that is no bikeshed sir!!! TL;DR: Bump it to #638046. - text is black, #000000. - background is #4A662E. - tone/L* of text is 0 - tone/L* of background is 40 - elements smaller than 18 pts should have a contrast ratio of 4.5 or more - difference in tone for 4.5 contrast ratio is 50 - take background at T40, adjusted to T50: #4A662E turns to #638046
  - nullc 5y ago
    On my decidedly yolo-colors laptop display, I like that ever so slightly less. But not ever-so-slightly enough to fail to defer to a highly actionable and justified display of nerdity. Thank you.
    - refulgentis 5y ago
      Who downvoted this!? We had a fun light-hearted exchange! Thank you for indulging my bike-shedding sir :)
totetsu 5y ago
I cant read the article because its blocked by my workplaces proxy.
Crosseye_Jack 5y ago
Reader Mode cleaned it up for me.

sooham 5y ago

FYI: The title is a sarcastic take on OpenAI's codex lack of nuance, not that the XY problem has been solved.

version_five 5y ago

If I ask how to write a bash script to get the last 3 letters of a file name, my preferred answer would be the literal answer. If someone wants to pontificate below that answer that maybe I wanted the extension, that's cool, but first answer the question or don't bother participating. People always invoke XY problem as if it's a way of helping someone, when it's mostly just condescension and or showing off when applied to forum answers. If someone is looking for "advice", they can ask for it. If someone is looking for syntax, they don't necessarily want advice, even if you really think they need it. That's my advice.

nullc 5y ago
Well there you go, OpenAI codex has solved your problem: technical help that won't bother warning you when you're about to shoot yourself in the foot, if that's really what you want. :) > when it's mostly just condescension and or showing off when applied to forum answers. You might be reading emotion and motivation into text that simple isn't there. I do wonder if codex improves and asks clarifying questions if you'll attribute condescension to its actions? > but first answer the question or don't bother participating Sounds like you might be selfishly confusing other human beings for a coin operated answer vending machine that exists for your personal gratification. Or maybe I'm reading too much into your "just answer or don't participate"? :D (I don't have any reason to believe it of you-- but I would assume that it's no less true than the windbaggy responders replying that way to show off their skills and mock your lack of them) In any case... that is the point: If you want a answer vending machine that doesn't care if what you asked for might not be what you want, an actual machine will do a better job of it. Personally, I'm grateful when someone backs me out of a niche I've reasoned myself into ... and I don't mind having to spend more time on my questions to qualify that I've already tried the obvious. I've probably solved more of my own problems just by stepping back to explain what I'm trying to do and what and then noticing the error on my own, than by any other means. But even if I did mind-- even if the people responding really were just showing off-- if that's the price of free help it still seems like a good deal. I'd much rather get help from someone with a chance of warning me that my whole approach might be confused, in my view that's 95% of the value of learning from another mind as opposed to mashing buttons until the tests pass.

alexjurkiewicz 5y ago

Using Bash examples was a bad idea. Bash scripts are by-and-large hacky things written to solve a simple problem expediently. OpenAI's output code reflects this reality. Expecting OpenAI to write Bash code defensively is like expecting OpenAI to complete SMS messages with scientific english.

skeaker 5y ago
I was going to ask a similar thing. Does this AI not pull from a massive pool of countless examples of real human-written code? In the majority of cases, perfect code is far from necessary (put another way, solving Y will generally suffice even if you should be focusing on X), and that seems to be reflected in the sample data that the AI is most likely pulling from. Obviously it isn't perfect, but I never expected it to be perfect, so I'm left more impressed that the AI functions at all than I am upset that it doesn't function absolutely perfectly in every way.
- nullc 5y ago
  I think it's performance is quite impressive. Consider, the very last example is a better implementation than most here would write (at least without some research). The cheeky tone is not targeted towards the machine. > Does this AI not pull from a massive pool of countless examples of real human-written code Pull from wouldn't be the right way of looking at it-- it's trained on and has presumably memorized significant amounts. But thus far no one has equip this kind of model with something like 'database access', beyond its own quadratic-cost internal self-attention.
  - saurik 5y ago
    And yet only if the prompt includes "using fancy math". I'm still cracking up, and expect to still be cracking up over it tomorrow ;P.

renewiltord 5y ago

Damn, those examples are insanely good.

nullc 5y ago
I was pretty impressed by it managing to use fermat's theorem on pythagorean primes.
- qayxc 5y ago
  Maybe it's just a literal match (Codex is known to quote code verbatim [0]) for existing training data? Given the popularity of the problem in both CS classes and coding competitions, there's a good chance a matching implementation was present in its Github-based training data more than just a few times. [0] https://thenewstack.io/this-week-in-programming-github-copilot-copyright-infringement-and-open-source-licensing/ https://thenewstack.io/this-week-in-programming-github-copil...
  - nullc 5y ago
    If you can find one, I'd be interested in knowing.

flafla2 5y ago

That last example is such a mic drop. I can’t tell if I’m impressed beyond belief or appalled.

q-big 5y ago
> That last example is such a mic drop. I can’t tell if I’m impressed beyond belief or appalled. I am appalled, because the code is wrong: consider p=2.
- tromp 5y ago
  Yes; too bad the prompt wasn't for odd_prime_is_sum_of_two_squares ...