4 ms·

There was an article on HN a few days back on how it’s very hard to convey all the context in your head about a codebase to solve a problem, and that’s partly w

There was an article on HN a few days back on how it’s very hard to convey all the context in your head about a codebase to solve a problem, and that’s partly why it’s actually hard to use AI for non-trivial implementations. That’s not just limited to code. I don’t use AI for most of my product work because it doesn’t know any of the nuances of our product, and just like doing code review for AI is boring and tedious, it’s also boring and tedious to exhaustively explain that stuff in a doc, if it can even be fully conveyed, because it’s a combination of strategy, hearsay from customers, long-standing convos with coworkers… I’d rather just do the product work. Also, I’ve self-selected by survivorship bias to be someone who likes doing the product work too, which means I have even less desire to give it up. Smarter LLMs could solve this maybe. But the difficulty of conveying information seems like a hard thing to solve.

jmtulloss 1y ago
It’s likely that the models don’t need to get much smarter, but rather the ux for providing needed context needs to improve drastically. This is a problem that we’ve worked on for decades with humans, but only single digit years for AI. It will get better and that tediousness will lessen (but isn’t “alignment” always tedious?)
- sbt 1y ago
  > ...context needs to improve drastically. Yes, drastically. This means I'll have to wear Zuck's glasses I think, because the AI currently doesn't know what was discussed at the coffee machine or what management is planning to do with new features. It's like a speed typing goblin living in an isolated basement, always out of the loop.
saratogacx 1y ago
I think architecture needs to evolve as well. Up until now the primary audiences were human comprehension and build tools. We may need to have more context embedded into software architecture and make systematic segmentation easier for an LLM to sus out as part of the concerns involved in our designs. LLM's won't make it on their own because there isn't enough data with their concerns in mind for them to learn from, just a few happy accidents where an LLM can excel in that specific code base.
SAI_Peregrinus 1y ago
LLMs need something like Waterfall development. They require a very complete, detailed specification of the behavior. And probably building the whole system twice, as recommended by Royce's 1970 paper that created the Waterfall process! Agile processes only work when the developers can be self-guiding, which LLMs aren't.

jmtulloss 1y ago

It’s likely that the models don’t need to get much smarter, but rather the ux for providing needed context needs to improve drastically. This is a problem that we’ve worked on for decades with humans, but only single digit years for AI. It will get better and that tediousness will lessen (but isn’t “alignment” always tedious?)

sbt 1y ago
> ...context needs to improve drastically. Yes, drastically. This means I'll have to wear Zuck's glasses I think, because the AI currently doesn't know what was discussed at the coffee machine or what management is planning to do with new features. It's like a speed typing goblin living in an isolated basement, always out of the loop.

saratogacx 1y ago

I think architecture needs to evolve as well. Up until now the primary audiences were human comprehension and build tools. We may need to have more context embedded into software architecture and make systematic segmentation easier for an LLM to sus out as part of the concerns involved in our designs. LLM's won't make it on their own because there isn't enough data with their concerns in mind for them to learn from, just a few happy accidents where an LLM can excel in that specific code base.

SAI_Peregrinus 1y ago

LLMs need something like Waterfall development. They require a very complete, detailed specification of the behavior. And probably building the whole system twice, as recommended by Royce's 1970 paper that created the Waterfall process! Agile processes only work when the developers can be self-guiding, which LLMs aren't.