9 ms·
The “Docs” link goes back to the homepage. I found it hard to tell what exactly this does - how can serverless reduce overhead (usually it’s the other way aroun
by ricardobeat 2y ago
The “Docs” link goes back to the homepage. I found it hard to tell what exactly this does - how can serverless reduce overhead (usually it’s the other way around), how does it scale the LLM backend and so on.
- bolei 2y agosorry about that. Please try docs.fleak.ai directly? What Fleak offers is a simple way to capture the data processing logic, with or without LLM, and we make deployment to production with one click. That means as a user, no need to write multithread scripts, configuring clusters or create load balancers since Fleak's ingestion engine will optimize the latency across different processing unit (we call them nodes, whether it's llm or simple SQL). On the LLM side, we work with our partner to coordinate the throughput, so if there is a demand for 5000 bps Llm inferencing, we can certainly scale up to that. But for SQL functions, the users can basically get that level of autoscaling out of box.