4 ms·
LLMOps / PromptOps, seems to be a new monitoring tool segment which is heating up. And pretty much essential as AI models are constantly being updated "in prod
by pico_creator 3y ago
LLMOps / PromptOps, seems to be a new monitoring tool segment which is heating up.
And pretty much essential as AI models are constantly being updated "in production" these days.
Seeing that you interviewed LangChain, which just launched LangSmith back to back?
How do you think this space will turn out - cause I find it very hard to tell how the two of them differ.
Additionally, what happens when incumbent monitoring tools start entering the scene (DataDog, Elastic, etc) ?
- swyx 3y agothanks! i address that a bit in the essay - basically langchain will try to build a verticalized solution for langchain, humanloop will be more general but for llms, datadog will be offering the same thing but for existing datadog customers. i think all three groups can win for a long time but when they start competing it will come down to customer personal preference and ability to sell and ship compelling features
- pico_creator 3y agomakes sense, considering there lacks an open std to port back and forth. people stick to datadog / elastic / etc passionately, to avoid "redoing their charts" also low hanging fruit to evals prompt and check (does it JSON or not), is significantly more useful then folks give credit for Q_Q
- swyx 3y agoi would love to help define an open standard with the AIEF...