3 ms·
The "All Tools" omni-prompt takes a whopping 2,756 tokens, but it's also using the GPT-4 32k model, with a 32,767 token context window. It also reduces default
by spdustin 3y ago
The "All Tools" omni-prompt takes a whopping 2,756 tokens, but it's also using the GPT-4 32k model, with a 32,767 token context window.
It also reduces default DALL•E generations to one 'caption' (with a conflicting instruction later to generate 2 images).
Custom Instructions are going to need some work, since all this stuff comes before CIs.
The other thing that I found interesting: the built-in RAG accepts a limited number of file types, and uses the same `tools` middleware that the `browser` tool uses. It seems that it renders those files directly in their headless browser container, and captures text from it exactly the way it does when using a the Browse with Bing.
It also stores those files in the `/mnt/data` sandbox used by the `python` tool, making the uploads automatically ephemeral.
File types supported:
"text/plain",
"application/pdf",
"text/html",
"text/x-tex",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
"application/json",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"text/markdown"
There are three middlewares (for lack of a better term) used by ChatGPT to talk to external tools.
- `tools` (browser)
- `tools2` (python/jupyter)
- `tools3` (plugins)
It's weird that DALL•E clearly uses plugins, but the model metadata endpoint [0] doesn't declare that.
[0]: https://github.com/spdustin/ChatGPT-AutoExpert/blob/main/_system-prompts/_backend_api__models.md https://github.com/spdustin/ChatGPT-AutoExpert/blob/main/_sy...