3 ms·
Why does no one measure prompt velocity
- dazzeloid 3y agoLove the concept of prompt velocity. Although it doesn't capture the initial quality of the prompt or whether the changes are effective.
- thepaulthomson 3y agoAny thoughts on how they could improve here? Seems like that would be challenging.
- sp332 3y agoWhat is prompt velocity? It's not mentioned in the blog post.
- casstang 3y agoWhat’s LangSmith's revenue model?
- thepaulthomson 3y agoI have no idea. They're still in beta so probably figuring it out as they go I guess. I could see them charging on tokens or traces most likely though.
- killaJ 3y agoBeen messing with LangSmith playground recently. Pains me to say but it’s actually been a pretty good experience so far.
- thepaulthomson 3y agoThat's awesome. I've been seeing quite a bit of chat about it on X too. Seems like they've hit the mark with playground. What are you using it for specifically?
- fwesss 3y agoI'm curious about LangSmith's 'dynamic datasets'. How does it ensure data integrity, especially when rapidly iterating on AI models?
- thepaulthomson 3y agoFrom the docs it looks like they're fairly explicit with respecting env states for each dataset. I'm not sure how/where contamination would even occur to be honest - regardless of model used.
- finnlobsien1 3y agosuper interesting how it makes "decisions", but nice that they let you tie user feedback directly into LLM refinement, otherwise would be hard to make that info useful
- thepaulthomson 3y agoWhat do you mean by decisions?