4 ms·
That's a big topic :D First thing about building AI or LLM powered apps is that 90% of the time you don't need to train models from scratch. LLMs are models th
by resiros 1y ago
That's a big topic :D
First thing about building AI or LLM powered apps is that 90% of the time you don't need to train models from scratch. LLMs are models that have the world's knowledge, so you can "train them" by prompting them and giving them the right context.
Where to start? I think the first thing is to play around with the OpenAI playground to get an idea of what's possible with prompts.
You'll find tons of frameworks out there: LangChain, LlamaIndex, Agno, and the list goes on. But in my opinion, if you have a use case in mind, start very very simple. Almost a single prompt using the OpenAI SDK or compatible SDK like LiteLLM.
The core challenge isn't technical complexity, it's the stochastic nature of LLMs. Your prompt works perfectly on one example, breaks on another. You fix it for the second case, and it stops working for the first.
The whole trick of building an LLM powered application is finding how to deal with this in an iterative approach. You need to have a workflow that allows you to iterate quickly. For this, start very simple and then add complexity. Don't use frameworks at the beginning.
I think Hamel's approach of error analysis (https://hamel.dev/notes/llm/officehours/erroranalysis.html https://hamel.dev/notes/llm/officehours/erroranalysis.html) is a very good starting point. The idea is that you create a prompt, run it against a test set, then identify clusters of error types and systematically fix them.
What tools do you need? In the beginning, just a playground and a Jupyter Notebook. At some point when you want to look into LLMOps tools. They provide you with observability (so if you have an agent with multiple steps you can see each step), running evals and comparing results easily, creating test sets from prod data... All the stuff that makes your life easier.
There are a lot of these, langsmith, helicone, braintrust, humanloop.. Full disclosure, I'm CEO of one of these tools (Agenta) [one of the reasons I'm answering this post :D].
But I think from your question you're not at the stage where you should look at LLMOps yet. Focus on understanding your use case first and starting a simple repeatable process.