3 ms·

We collectively deride leetcoding interviews yet ask AI to flawlessly solve leetcode questions. I bet I'd make more errors on my first try at it.

by senko 2y ago

AnimalMuppet 2y ago
Writing a prime-number factorization function is hardly "leetcode".
- atomic128 2y ago
  Empirical testing (for example: https://news.ycombinator.com/item?id=33293522 https://news.ycombinator.com/item?id=33293522) has established that the people on Hacker News tend to be junior in their skills. Understanding this fact can help you understand why certain opinions and reactions are more likely here. Surprisingly, the more skilled individuals tend to be found on Reddit (same testing performed there).
  - louthy 2y ago
    I’m not sure that’s evidence; I looked at that and saw it was written in Go and just didn’t bother. As someone with 40 years of coding experience and a fundamental dislike of Go, I didn’t feel the need to even try. So the numbers can easily be skewed, surely.
    - atomic128 2y ago
      Only individuals who submitted multiple bad solutions before giving up were counted as failing. If you look but don't bother, or submit a single bad solution, you aren't counted. Thousands of individuals were tested on Hacker News and Reddit, and surprisingly, it's not even close: Reddit is where the hackers are. I mean, at the time of the testing, years ago.
      - louthy 2y ago
        That doesn’t change my point. It didn’t test every dev on all platforms, it tested a subset. That subset may well have different attributes to the ones that didn’t engage. So, it says nothing about the audience for the forums as a whole, just the few thousand that engaged. Perhaps even, there could be fewer Go programmers here and some just took a stab at it even though they don’t know the language. So it could just select for which forum has the most Go programmers. Hardly rigourous. So I’d take that with a pinch of salt personally
        atomic128 2y ago
        Agreed. But remember, this isn't the only time the population has been tested. This is just the test (from two years ago, in 2022) that I happen to have a link to.
        louthy 2y ago
        The population hasn’t been tested. A subset has.
        59nadir 2y ago
        It's also fine to be an outlier. I've been programming for 24 years and have been hanging out on HackerNews on and off for 11. HN was way more relevant to me 11 years ago than it is now, and I don't think that's necessarily only because the subject matter changed, but probably also because I have.
  - Izikiel43 2y ago
    How is that thing testing? Is it expecting a specific solution or actually running the code? I tried some solutions and it complained anyways
    - atomic128 2y ago
      The way the site works is explained in the first puzzle, "Hack This Site". TLDR, it builds and runs your code against a test suite. If your solutions weren't accepted, it's because they're wrong.
  - 0xDEAFBEAD 2y ago
    Where is the data?
  - freilanzer 2y ago
    Yeah, this is useless.
- senko 2y ago
  I didn't say it's hard, but it's most definitely leetcode, as in "pointless algorithmic exercise that will only show you if the candidate recently worked on a similar question". If that doesn't satisfy, here's a similar one at leetcode.com: https://leetcode.com/problems/distinct-prime-factors-of-product-of-array/ https://leetcode.com/problems/distinct-prime-factors-of-prod... I would not expect a programmer of any seniority to churn stuff like that and have it working without testing.
  - AnimalMuppet 2y ago
    > "pointless algorithmic exercise that will only show you if the candidate recently worked on a similar question". I've been able to write one, not from memory but from first principles, any time in the last 40 years.
    - senko 2y ago
      Curious, I would expect a programmer of your age to remember Knuth's "beware of the bugs in above code, I have only proven it's correct but haven't actually run it". I'm happy you know math, but my point before this thread got derailed was that we're holding (coding) AI to a higher standard than actual humans, namely to expect to write bug-free code.
      - Jensson 2y ago
        Humans can actually run the code and knows what it should output. the LLM can't, and putting it in a loop against code output doesn't work well either since the LLM can't navigate that well.
      - smrq 2y ago
        I hold myself to a higher standard than AI tools are capable of, from my experience. (Maybe some people don't, and that's where the disconnect is between the apologists and the naysayers?)
      - 0points 2y ago
        > my point before this thread got derailed was that we're holding (coding) AI to a higher standard than actual humans, namely to expect to write bug-free code This seems like a very layman attitude and I would be surprised to find many devs adhering to this idea. Comments in this thread alone suggests that many devs on HN do not agree.

We collectively deride leetcoding interviews yet ask AI to flawlessly solve leetcode questions. I bet I'd make more errors on my first try at it.

by senko 2y ago

We collectively deride leetcoding interviews yet ask AI to flawlessly solve leetcode questions. I bet I'd make more errors on my first try at it.

AnimalMuppet 2y ago

Writing a prime-number factorization function is hardly "leetcode".

atomic128 2y ago
Empirical testing (for example: https://news.ycombinator.com/item?id=33293522 https://news.ycombinator.com/item?id=33293522) has established that the people on Hacker News tend to be junior in their skills. Understanding this fact can help you understand why certain opinions and reactions are more likely here. Surprisingly, the more skilled individuals tend to be found on Reddit (same testing performed there).
- louthy 2y ago
  I’m not sure that’s evidence; I looked at that and saw it was written in Go and just didn’t bother. As someone with 40 years of coding experience and a fundamental dislike of Go, I didn’t feel the need to even try. So the numbers can easily be skewed, surely.
  - atomic128 2y ago
    Only individuals who submitted multiple bad solutions before giving up were counted as failing. If you look but don't bother, or submit a single bad solution, you aren't counted. Thousands of individuals were tested on Hacker News and Reddit, and surprisingly, it's not even close: Reddit is where the hackers are. I mean, at the time of the testing, years ago.
    - louthy 2y ago
      That doesn’t change my point. It didn’t test every dev on all platforms, it tested a subset. That subset may well have different attributes to the ones that didn’t engage. So, it says nothing about the audience for the forums as a whole, just the few thousand that engaged. Perhaps even, there could be fewer Go programmers here and some just took a stab at it even though they don’t know the language. So it could just select for which forum has the most Go programmers. Hardly rigourous. So I’d take that with a pinch of salt personally
      - atomic128 2y ago
        Agreed. But remember, this isn't the only time the population has been tested. This is just the test (from two years ago, in 2022) that I happen to have a link to.
        louthy 2y ago
        The population hasn’t been tested. A subset has.
        59nadir 2y ago
        It's also fine to be an outlier. I've been programming for 24 years and have been hanging out on HackerNews on and off for 11. HN was way more relevant to me 11 years ago than it is now, and I don't think that's necessarily only because the subject matter changed, but probably also because I have.
- Izikiel43 2y ago
  How is that thing testing? Is it expecting a specific solution or actually running the code? I tried some solutions and it complained anyways
  - atomic128 2y ago
    The way the site works is explained in the first puzzle, "Hack This Site". TLDR, it builds and runs your code against a test suite. If your solutions weren't accepted, it's because they're wrong.
- 0xDEAFBEAD 2y ago
  Where is the data?
- freilanzer 2y ago
  Yeah, this is useless.
senko 2y ago
I didn't say it's hard, but it's most definitely leetcode, as in "pointless algorithmic exercise that will only show you if the candidate recently worked on a similar question". If that doesn't satisfy, here's a similar one at leetcode.com: https://leetcode.com/problems/distinct-prime-factors-of-product-of-array/ https://leetcode.com/problems/distinct-prime-factors-of-prod... I would not expect a programmer of any seniority to churn stuff like that and have it working without testing.
- AnimalMuppet 2y ago
  > "pointless algorithmic exercise that will only show you if the candidate recently worked on a similar question". I've been able to write one, not from memory but from first principles, any time in the last 40 years.
  - senko 2y ago
    Curious, I would expect a programmer of your age to remember Knuth's "beware of the bugs in above code, I have only proven it's correct but haven't actually run it". I'm happy you know math, but my point before this thread got derailed was that we're holding (coding) AI to a higher standard than actual humans, namely to expect to write bug-free code.
    - Jensson 2y ago
      Humans can actually run the code and knows what it should output. the LLM can't, and putting it in a loop against code output doesn't work well either since the LLM can't navigate that well.
    - smrq 2y ago
      I hold myself to a higher standard than AI tools are capable of, from my experience. (Maybe some people don't, and that's where the disconnect is between the apologists and the naysayers?)
    - 0points 2y ago
      > my point before this thread got derailed was that we're holding (coding) AI to a higher standard than actual humans, namely to expect to write bug-free code This seems like a very layman attitude and I would be surprised to find many devs adhering to this idea. Comments in this thread alone suggests that many devs on HN do not agree.