4 ms·

Are there any models that edit or generate file trees?

wokwokwok 4y ago
Like, an entire repository of files? No. How would a model get trained on that? You'd have to pass in the entire repository for each sample. It's prohibitively difficult to create that sort of model. If you want that, you'll have to build tooling on top of a text model (ie. an application that calls a model repeatedly), that takes a prompt and breaks it up into per-file prompts, then incrementally generates the files passing the context of previous files, and the 'context' would be too large, so you'd get large scale consistency errors. Broadly speaking the number of tokens = the size of the text it can generate. With small models, the number is trivial (code fragment), so generally speaking 'generate an entirely application' one-step models currently don't exist. That said, stable diffusion has proved that you can iterate in latent space and use a VAE to upscale to larger sizes to reduce the over all model size while still having output that is ~order of magnitude larger than the latent space. ...so it's not totally out of the question that's coming. However, right now? no.
- stevenhuang 4y ago
  > so generally speaking 'generate an entirely application' one-step models currently don't exist. This already exists. People have created full apps in ChatGPT by doing this. Many examples online.
  - wokwokwok 4y ago
    No, they haven’t. ChatGPT has the same prompt history limitations as other models, and cannot generate entire end to end applications. What you’ve seen is applications running on top of chatGPT and using it iteratively to generate multiple code segments.
    - stevenhuang 4y ago
      But the context window size of ChatGPT is sufficient. And again it does generate a full application, just not all of it at once. Just see https://news.ycombinator.com/item?id=33854638 https://news.ycombinator.com/item?id=33854638 Sure you can't prompt it once and download a zip, but you still get a consistent full app at the end that can be used as a base from this prompting.

Like, an entire repository of files? No. How would a model get trained on that? You'd have to pass in the entire repository for each sample. It's prohibitively difficult to create that sort of model. If you want that, you'll have to build tooling on top of a text model (ie. an application that calls a model repeatedly), that takes a prompt and breaks it up into per-file prompts, then incrementally generates the files passing the context of previous files, and the 'context' would be too large, so you'd get large scale consistency errors. Broadly speaking the number of tokens = the size of the text it can generate. With small models, the number is trivial (code fragment), so generally speaking 'generate an entirely application' one-step models currently don't exist. That said, stable diffusion has proved that you can iterate in latent space and use a VAE to upscale to larger sizes to reduce the over all model size while still having output that is ~order of magnitude larger than the latent space. ...so it's not totally out of the question that's coming. However, right now? no.

stevenhuang 4y ago
> so generally speaking 'generate an entirely application' one-step models currently don't exist. This already exists. People have created full apps in ChatGPT by doing this. Many examples online.
- wokwokwok 4y ago
  No, they haven’t. ChatGPT has the same prompt history limitations as other models, and cannot generate entire end to end applications. What you’ve seen is applications running on top of chatGPT and using it iteratively to generate multiple code segments.
  - stevenhuang 4y ago
    But the context window size of ChatGPT is sufficient. And again it does generate a full application, just not all of it at once. Just see https://news.ycombinator.com/item?id=33854638 https://news.ycombinator.com/item?id=33854638 Sure you can't prompt it once and download a zip, but you still get a consistent full app at the end that can be used as a base from this prompting.