2 ms·

I was absolutely blown away when I saw the demos. Then about 15 minutes later, after some reflection, I thought about what I just saw: -An LLM generating code

I was absolutely blown away when I saw the demos. Then about 15 minutes later, after some reflection, I thought about what I just saw: -An LLM generating code -An LLM Agent taking a request, breaking it down into tasks, and then creating a plan - An LLM using tools (shell, compiler, web browser) Wait a minute, I've seen all these things before. Many times. So why was I so impressed? I thought about it. Well, the UI was slick. But really, the wow factor was due to how much it was able to accomplish with a single prompt like "Benchmark Llama2 on these 3 cloud providers". Then I started wondering whether it actually did all those things from that one prompt or whether there was an entire days worth of prompting, interspersed with manual steps, spliced together into a 2 minute video. Like that staged Gemini demo video from a few months ago. Hmmm.

mattbrewsbytes 3y ago
Your description reminds me of DHH's original "how to build a blog in 15 min" video to demonstrate Rails. Tons of time went into understanding the framework, design patterns in use and general understanding of OO, web code, etc. If you sat a non technical person down and asked them to generate the same blog in 15 minutes it just wouldn't happen. As a Java developer seeing a web app go from zero to working in 15 mins was amazing. So I think the same thing is in effect with any AI/LLM that is outputting code. There is a ton of knowledge of the LLM and trial/error crafting prompts, clarifying things that go into getting any output that will work. People keep focusing on the text output of these tools but the majority of time spent in Software Development is understanding business problems, working with other humans on those and defining a solution that solves the business problem, coding and testing that solution are a minority share of the total time involved from end-to-end of problem/idea inception to working software in production.

mattbrewsbytes 3y ago

Your description reminds me of DHH's original "how to build a blog in 15 min" video to demonstrate Rails. Tons of time went into understanding the framework, design patterns in use and general understanding of OO, web code, etc. If you sat a non technical person down and asked them to generate the same blog in 15 minutes it just wouldn't happen. As a Java developer seeing a web app go from zero to working in 15 mins was amazing. So I think the same thing is in effect with any AI/LLM that is outputting code. There is a ton of knowledge of the LLM and trial/error crafting prompts, clarifying things that go into getting any output that will work. People keep focusing on the text output of these tools but the majority of time spent in Software Development is understanding business problems, working with other humans on those and defining a solution that solves the business problem, coding and testing that solution are a minority share of the total time involved from end-to-end of problem/idea inception to working software in production.