4 ms·
Local TypeScript Super SDK to Call 200 LLMs
- ArshDilbagi 2y agoThe fully local production-grade Super SDK that provides a simple, unified, and powerful interface for calling more than 200+ LLMs. - Production-ready and used by enterprises. - Fully local and NOT a proxy. You can deploy it anywhere. - Comes with batching, retries, caching, callbacks, and OpenTelemetry support. - Supports custom plugins for caching, logging, HTTP client, and more. You can use it like LEGOs and make it work with your infrastructure. - Supports plug-and-play providers. You can run fully custom providers and still leverage all the benefits of Adaline Gateway. Features Strongly typed in TypeScript Isomorphic - works everywhere 100% local and private and NOT a proxy Tool calling support across all compatible LLMs Batching for all requests with custom queue support Automatic retries with exponential backoff Caching with custom cache plug-in support Callbacks for full custom instrumentation and hooks OpenTelemetry to plug tracing into your existing infrastructure Plug-and-play custom providers for local and custom models
- kazcaptain 2y agoGreat lib, will give it a go in my work
- ArshDilbagi 2y agothank you
- BoorishBears 2y agoCan tools return images to the LLM using this SDK? (where supported by the provider)
- ArshDilbagi 2y agoYes, we support images, tools, response format (for OpenAI) and everything else.
- BoorishBears 2y agoI'm asking specifically about images being returned by a tool call. I didn't see any indication that it's supported skimming through the Zod types
- npace12 2y agothat's a lot of work and it looks nicely done, thanks!
- ArshDilbagi 2y agothank you
- alex_suzuki 2y agowhat‘s a „Super SDK“? A meta sdk, i.e. an SDK wrapping other SDKs/APIs?
- ArshDilbagi 2y agoWe have implemented “super-types” that are strongly typed in TypeScript and compatible with all providers (think a superset in mathematics). Hence the name, Super SDK. Gateway is super light weight. We don’t wrap around other SDKs. We describe the APIs in the provider packages (@adaline/openai) that plugs into @adaline/gateway that has all the features.
- alex_suzuki 2y agoGot it, thanks!
- maeil 2y agoA more fleshed out competitor to llm-interface [1]? Looks great! Does it run in FaaS/serverless environments out of the box? Lambdas, Cloudflare Workers, Vercel functions and the likes? Deno? The README says "Isomorphic - works everywhere", but might be nice to make this more explicit. [1] https://github.com/samestrin/llm-interface https://github.com/samestrin/llm-interface
- vbo 2y agoLooks good, might give it a try. I was looking for something similar to provide a unified interface for gpt and claude and eventually hacked something together myself, as none of the solutions I found could deal with structured output properly across both vendors.
- nip 2y agoIncredible work! I've been meaning to add "intelligence" to my Telegram monitoring bot: it couldn't come at a better time! Thank you for building it!
- Sn0wCoder 2y agoVery neat, bookmarked. Read through the readme.md and would be nice to show an example of using with Ollama or LM Studio running locally. Would guess you just setup an OpenAI model, set the baseUrl to the local instance / API and go from there? For something that is Fully Local connecting to a provider with an API key is iffy at best, unless you are paying for enterprise keys. I guess the first line says used by ‘Enterprises’ but to be fully local that would include running the LLM locally also?
- hirako2000 2y agoWhat do you mean by "no proxy"?
- madarco 2y agoI think it's because the library calls directly the LLM AND saves any debug/trace info. While other tools let you use the standard (eg OpenAI) sdk and uses an https proxy to intercept the requests
- ndndjdjdn 2y agoThat is great. How do you even handle the daily divergence that must happen due to micro decisions made by various APIs. For example I wont mention their name but one OpenAI compatiable provider doesn't fully work with OpenAI client and had various holes so you need to account for that. The other hard thing is just the fast pace of AI. What if the underlying concepts change! Well done!
- joseferben 2y agolooks good! how does it compare to the core of vercel ai sdk?
- yieldcrv 2y agoUse case? ELI5 The LLM itself is still in the cloud right? If using one from anthropic or open ai