calebhwin

5 ms·

31.
Show HN: A1 – An optimizing JIT compiler for AI agents(github.com)
1 points by calebhwin 11mo ago|0 comments
32.
Show HN: a1 - determinism-maxing JIT compiler for AI agents(github.com)
1 points by calebhwin 11mo ago|0 comments
33.
by calebhwin 1y ago
A system that does the following given a task_description: while LLM("is <task_description> not done?"): Browser.run(LLM("what should the browser do next given <Browser.get_state()>")) This simple loop tu
34.
by calebhwin 1y ago
Yes. And you can give BLAST an LLM cost budget or max browser memory usage and BLAST takes care of scheduling.
35.
by calebhwin 1y ago
The use cases are (1) integrating AI automation into my app (2) automating workflows inside web browser (3) personal use. The value is in optimizing for low latency under user-defined constraints such as LLM cost budget or maximum browser m
36.
by calebhwin 1y ago
Do you build agents that interface with web browsers? BLAST is sort of like vllm for browser+LLM. The motivation for this is that browser+LLM is slow and we can do a lot of optimization with an engine that manages browser+LLM together - e.g
37.
by calebhwin 1y ago
I would really think about it as a serving engine like vllm but for browsers+LLMs. It handles caching, parallelism, scheduling, budget constraints for LLM cost and browser memory usage. Yes it currently has an OpenAI-compatible API but we w
38.
by calebhwin 1y ago
Yes, human-in-the-loop is definitely on the roadmap. It's orthogonal to the central goal of low latency but necessary for completeness. Either via VNC or something simpler we have in mind.
39.
by calebhwin 1y ago
Ah you're right, my bad. Hope I didn't sound dismissive because I think some sort of robots.txt needs to exist for AI that's scraping the web both at train or test time. I'm really not excited at all about the "scra
40.
by calebhwin 1y ago
Thank you! It's currently based on task lineage, exact match of task descriptions, and an optional user-provided cache_control argument that can control whether results or plans are cached. One use-case for this is conversations: So fo
41.
by calebhwin 1y ago
Good point, we should probably integrate that. Feel free to submit a PR! BLAST can also be used to add automation to your own site/app FWIW.
42.
by calebhwin 1y ago
IMO it depends on how this tech is deployed. One way I see this being extremely useful is for developers to quickly build AI automation for their own sites. E.g. if I'm the developer of a workforce management app (e.g. https:/&#x
43.
by calebhwin 1y ago
There's definitely opportunities to parallelize. BLAST exploits these with an LLM-planner and tool calls to dynamically spawn/join subtasks (there's also data parallelism and request hedging which further reduce latency). Now
44.
by calebhwin 1y ago
Great point, we are working on an MCP server implementation which should address this. The main benefit of having a serving engine here is to abstract away browser-LLM specific optimizations like parallelism, caching, browser memory managem
45.
by calebhwin 1y ago
Yes! And browser-use is great though I'm hoping at some point we can swap it out for something leaner, maybe one day it'll just be a vision language model. All we'll have to do within BLAST is implement a new Executor and all
46.
by calebhwin 1y ago
Maybe more of a legal than ethical consideration but web browsing AI makes scraping trivial. You could use that for surveillance, profiling (get a full picture of a user's whole online life before they even hit Sign Up), cutting egress
47.
by calebhwin 1y ago
The main sort of parallelism we exploit is across distinct websites. For example "find me the cheapest rental" spawning tasks to look at many different websites. There is another level of parallelism that could be exploited within
48.
by calebhwin 1y ago
Right I figured there isn't a huge overlap of interested communities so hopefully not a point of confusion. I guess that could change!
49.
Show HN: Blast – Fast, multi-threaded serving engine for web browsing AI agents(github.com)
145 points by calebhwin 1y ago|66 comments