3 ms·

What do you suggest instead? Handrolled code with “import openai”? BAML?

by gcr 10mo ago

llmslave2 10mo ago
Oh gosh, not that legacy "hand rolled code"
hhh 10mo ago
yes, in an industry that has rapidly changing features and 7000 of these products that splinter and lose user base so quickly you should write your own orchestration for this stuff. It’s not hard and gives you a very easy path to implementing new features or optimizations.
never_inline 10mo ago
Have you heard of `def`?
- gcr 10mo ago
  It was an earnest question. I didn’t intend to be sarcastic.
  - alzoid 10mo ago
    I went through evaluating a bunch of frameworks. There was Langchain, AG2, Firebase Gen AI / Vertex / whatever Google eventually lands on, Crew AI, Microsoft's stuff etc. It was so early in the game none of those frame works are ready. What they do under the hood when I looked wasn't a lot. I just wanted some sort of abstraction over the model apis and the ability to use the native api if the abstraction wasn't good enough. I ended up using Spring AI. Its working well for me at the moment. I dipped into the native APIS when I needed a new feature (web search). Out of all the others Crew AI was my second choice. All of those frameworks seem parasitic. One your on the platform you are locked in. Some were open source but if you wanted to do anything useful you needed an API key and you could see that features were going to be locked behind some sort of payment. Honestly I think you could get a lot done with one of the CLI's like Claude Code running in a VM.
  - never_inline 10mo ago
    Which abstractions in langchain do you find so useful which require significant code to replicate yourself in functions with OpenAI SDK / LiteLLM?
int_19h 10mo ago
For what LangChain does, yes, handrolled code. There won't be much of it because it doesn't actually do all that much. As for "import openai", that's actually somewhat orthogonal, but if what you want is a common API for different providers then there are many options around that do just that. But then again at that point you probably also want something like OpenRouter, which has its own generic API.
- jmogly 10mo ago
  I’ve been using the following pattern since gpt3, the only other thing I have changed was added another parameter for schema for structured output. People really love to overcomplicate things. class AI(Protocol): def call_llm(prompt: str) -> str: …

What do you suggest instead? Handrolled code with “import openai”? BAML?

by gcr 10mo ago

What do you suggest instead? Handrolled code with “import openai”? BAML?

llmslave2 10mo ago

Oh gosh, not that legacy "hand rolled code"

hhh 10mo ago

yes, in an industry that has rapidly changing features and 7000 of these products that splinter and lose user base so quickly you should write your own orchestration for this stuff. It’s not hard and gives you a very easy path to implementing new features or optimizations.

never_inline 10mo ago

Have you heard of `def`?

gcr 10mo ago
It was an earnest question. I didn’t intend to be sarcastic.
- alzoid 10mo ago
  I went through evaluating a bunch of frameworks. There was Langchain, AG2, Firebase Gen AI / Vertex / whatever Google eventually lands on, Crew AI, Microsoft's stuff etc. It was so early in the game none of those frame works are ready. What they do under the hood when I looked wasn't a lot. I just wanted some sort of abstraction over the model apis and the ability to use the native api if the abstraction wasn't good enough. I ended up using Spring AI. Its working well for me at the moment. I dipped into the native APIS when I needed a new feature (web search). Out of all the others Crew AI was my second choice. All of those frameworks seem parasitic. One your on the platform you are locked in. Some were open source but if you wanted to do anything useful you needed an API key and you could see that features were going to be locked behind some sort of payment. Honestly I think you could get a lot done with one of the CLI's like Claude Code running in a VM.
- never_inline 10mo ago
  Which abstractions in langchain do you find so useful which require significant code to replicate yourself in functions with OpenAI SDK / LiteLLM?

int_19h 10mo ago

For what LangChain does, yes, handrolled code. There won't be much of it because it doesn't actually do all that much. As for "import openai", that's actually somewhat orthogonal, but if what you want is a common API for different providers then there are many options around that do just that. But then again at that point you probably also want something like OpenRouter, which has its own generic API.

jmogly 10mo ago
I’ve been using the following pattern since gpt3, the only other thing I have changed was added another parameter for schema for structured output. People really love to overcomplicate things. class AI(Protocol): def call_llm(prompt: str) -> str: …