3 ms·
ChatGPT "All Tools" System omni-prompt
- spdustin 3y agoThe "All Tools" omni-prompt takes a whopping 2,756 tokens, but it's also using the GPT-4 32k model, with a 32,767 token context window. It also reduces default DALL•E generations to one 'caption' (with a conflicting instruction later to generate 2 images). Custom Instructions are going to need some work, since all this stuff comes before CIs. The other thing that I found interesting: the built-in RAG accepts a limited number of file types, and uses the same `tools` middleware that the `browser` tool uses. It seems that it renders those files directly in their headless browser container, and captures text from it exactly the way it does when using a the Browse with Bing. It also stores those files in the `/mnt/data` sandbox used by the `python` tool, making the uploads automatically ephemeral. File types supported: "text/plain", "application/pdf", "text/html", "text/x-tex", "application/vnd.openxmlformats-officedocument.presentationml.presentation", "application/json", "application/vnd.openxmlformats-officedocument.wordprocessingml.document", "text/markdown" There are three middlewares (for lack of a better term) used by ChatGPT to talk to external tools. - `tools` (browser) - `tools2` (python/jupyter) - `tools3` (plugins) It's weird that DALL•E clearly uses plugins, but the model metadata endpoint [0] doesn't declare that. [0]: https://github.com/spdustin/ChatGPT-AutoExpert/blob/main/_system-prompts/_backend_api__models.md https://github.com/spdustin/ChatGPT-AutoExpert/blob/main/_sy...