> AI is being sold as the ultimate tool that will replace artists/programmers/writers/whomever I see a potential risk here. This is HN, so let's consider prog

> AI is being sold as the ultimate tool that will replace artists/programmers/writers/whomever I see a potential risk here. This is HN, so let's consider programmers in isolation for a moment. Today, I can issue the following prompt to ChatGPT and get a solution that works: > Using Python, generate an example unit test that uses monkey patching. That gives me a runnable, valid example test. I can then ask it to refactor it to use pytest instead of unittest: > regenerate the above code sample using pytest instead of unittest. Here's the resultant code: # A simple class that we want to test class Calculator: def add(self, a, b): return a + b # Monkey patching the add method to subtract instead def mock_add(self, a, b): return a - b # Test the monkey patched method def test_add_with_monkey_patching(): # Create an instance of the Calculator class calc = Calculator() # Monkey patch the add method of the Calculator instance calc.add = mock_add.__get__(calc, Calculator) # Test the monkey patched method result = calc.add(5, 3) # Assert that the result is what we expect assert result == 2 It _works_, but it completely misses the point. The test case is testing whether or not the monkey patching works as expected - not testing something using monkey patching to isolate a side effect in the method under test, which was my intention. Here's where I think there is a potential major issue: As someone who is currently a "Senior Staff Software Engineer" and has been doing this sort of work for two decades now, if you had given me the same prompt my first response would have been "use monkey patching to do what?". ChatGPT doesn't do that. Instead, it gives you a response that best fits the question. It includes a half dozen paragraphs explaining the code, what it does, and what monkey patching is, sure... but it doesn't even touch on why you'd want to use monkey patching, when it is or is not the best approach, or any alternatives. In short, ChatGPT lacks the intuition that a human programmer will develop over the course of a career. It's getting better. A few months ago I could have asked ChatGPT the same question and probably 30% of the time gotten a result that would even run. In the past couple of months I don't think I've run into that at all. The "intuition" thing is a big deal, though. ChatGPT is a great tool in the hands of a competent engineer who has the experience to examine the output with a critical eye. Today, I could see it replacing a junior engineer in a professional setting. We should not use it in that way. If we do, in a few years we'll get to the point where the current generation of senior engineers are retiring. If we've replaced the majority of our junior engineers with AI, we'll be left without people who can use the AI-generated results effectively. There has to be a way to balance the savings represented by these tools (in terms of both time and money) with the fact that the people who are able to most effectively use it today only got to that point because they've spent much of their adult lives developing the exact skillset that we're proposing to replace with them.