Lately I’ve been doing a lot of very deep discussions with GPT-4 about design issues I’ve had, and brainstormed ideas on how to improve them. In these discussi

Lately I’ve been doing a lot of very deep discussions with GPT-4 about design issues I’ve had, and brainstormed ideas on how to improve them. In these discussions, GPT-4 has been able to really dig into the details and grok the problem space, and has surprised me with the quality and brilliance of its ideas and suggestions. Some of these I have tried, things that I would not have thought of, and it turns out the suggestions really helped solve the challenges I was facing. To me it’s like working with a brilliant person who can only communicate via text, can’t look anything up on the internet, and can’t remember more than a few pages of text at a time, but aside from those limitations is a genius. Lots of people talk about generating code with Chat GPT, but to me its real value is in having deep detailed discussions about design problems. It’s been so successful at this that recently I gave GPT-4 the full interview design skill assessment that I give to engineers when I interview them at Microsoft. GPT 3 wouldn’t be able to handle this, but what GPT-4 did here astonished me. My assessment is that this is a principal level performance. It didn’t have to do other things that normal candidates have to do but for this raw skills assessment of design skills it was spot on. It would have impressed me even more if it figured its final solution out from from the beginning, but that’s what it gets for blurting things out before thinking about them, which people do, too. The important thing I’m communicating here is not that I am impressed because it’s amazing that a computer can do this stuff; I’m impressed at what it has done here compared to almost every human I’ve ever walked through this question with. The approach I used here is the same I use when asking candidates this question, because in addition to testing their ability to code, I’m looking for how well I can understand the candidate’s ideas, and how well they can understand and then apply my own ideas when I ask them to take a different approach on certain things than they were thinking of themselves. This is one area where many great coders struggle; they can code like the wind when it’s their own idea but struggle to work collaboratively. This kind of mental flexibility, ability to think of things in a different sequence or consider other ways to solve the problem after thinking of their own solution is also a required skill often in real-life meetings and other collaborative settings. I’d rate this candidate as an outstanding, top-notch collaborator. My point here is that using it only to write boilerplate code is a waste of its best value. My suggestion is to get GPT-4 with its larger token limit, and talk strategy with it. Tell it all about your biggest challenges at a level of detail that would exhaust a normal human, and talk through ideas of how to improve your world. Talk to it about your people problems too. It’s an astonishingly wise counselor who has a wealth of positive insights and suggestions. It’s also great for elegantly wordsmithing things. Don’t miss out on the chance to collaborate with this endlessly creative and endlessly patient collaborator.