3 ms·
I read all the documentation and I still have no idea what this does. (I make AI agents as my day job, among many other things.)
by otabdeveloper4 1y ago
I read all the documentation and I still have no idea what this does.
(I make AI agents as my day job, among many other things.)
- calebhwin 1y agoDo you build agents that interface with web browsers? BLAST is sort of like vllm for browser+LLM. The motivation for this is that browser+LLM is slow and we can do a lot of optimization with an engine that manages browser+LLM together - e.g. prefix caching, auto-parallelism, data parallelism, request hedging, scheduling policy, and more coming soon. Now the API is what may be throwing folks off. Right now it's an OpenAI-compatible API. We will implement MCP. But really the core thing is abstracting away optimizations required to efficiently run browser+LLM.
- otabdeveloper4 1y agoI have no idea what you mean by "browser+LLM". Image models to process pictures of a webpage? A wrapper around Python's "requests"?
- calebhwin 1y agoA system that does the following given a task_description: while LLM("is <task_description> not done?"): Browser.run(LLM("what should the browser do next given <Browser.get_state()>")) This simple loop turns out to be very powerful, achieving the highest performance on some of OpenAI's latest benchmarks. But it's also heavily unoptimized compared to a system that is just LLM("<task_description>") for which we already have things like vllm. BLAST is a first step towards optimizing this while loop.