3 ms·
(author here) i suspect this maybe one of those “feels like a bad idea until you try it” things. Multiple times when i wanted to add a feature or be more specif
by swyx 3y ago
(author here) i suspect this maybe one of those “feels like a bad idea until you try it” things. Multiple times when i wanted to add a feature or be more specific, throwing it into prompt.md knowing the LLM would put it in the right file, or coordinate it across files, for me, was the most delightful experience of the “engineering with prompts” workflow.
that said if you see my future directions notes i do think theres room for file specific .md instructions.
the shared dependencies file is essentially a plan. i didnt realize it at the moment but now looking at it with fresh eyes i can do a no-op `smol plan` command pretty trivially.
- wokwokwok 3y agoLots of different approaches for sure. I certainly see `smol plan` as being at least one step in the right direction. My experience has been that as the amount of code you need to generate scales beyond trivial levels, the prompt-to-code approach from cross file code starts to fail. ie. > the shared dependencies (like filenames and variable names) we have decided on are: {shared_dependencies} Is too trivial of a prompt for large scale code generation. What if you generate a service in one file and expect a controller to call it from another file? How does the model know what the functions in the other file are? Did you put the signature of every public function in shared_dependencies? If you don't have a planning phase, you need a priori knowledge of the order in which files need to be generated, what the functions are called... I just... don't believe what you're trying to do is actually possible. The models will just generate an isolated 'best guess implementation' for each file; and that might hang together for small code blocks... but as the number of inter-dependencies between blocks of generated code increases... There's just no way right? This only works if the code you're generating doesn't call itself, it only calls known library code. shared_dependencies only solves this for the trivial case of like, 'oh, I have these 3 shared functions / types between all these files'; as the shared types and functions increase, your shared_dependencies becomes an unmaintainable monster... you end up having to generate the shared dependencies... split it up so that you have a separate shared_dependencies per file to generate... and suddenly it starts looking a heck of a lot like a .h header file....
- swyx 3y agoi think the underlying assumption you have here is that you think im saying this is meant to generate a program in one shot. i have from the start pitched this as a “human centric” scaffolding tool. what i meant by that is that it doesnt try to replace the human programmer, but merely augment them, and you can just stop using it once you subjectively feel that its no longer adding value more broadly i see the smol dev cycle being dependent on intelligence and latency, both of which will improve over time (https://twitter.com/swyx/status/1657962184935370752 https://twitter.com/swyx/status/1657962184935370752) lets say right now a single run gets you 1% of the way there. you do 10 runs of prompt iteration before you feel its run out of use, and take over, but at least by then youve gotten 20% of the way. as intelligence goes up, the single run % goes up. as latency goes down, more runs become feasible. this is the basic equation for how much smol developer can consume the start phase of any project, especially smol/oneoff/internal apps. this also perhaps opens up a fun definition of agi - when you can one shot 100% an app.
- razodactyl 3y agoI can throw some weight at this statement. I've definitely found that these models work better when you allow them to do things "their own way".