4 ms·
Two kinds of LLM responses: Informational vs. Instructional
- deleted 2y ago[deleted]
- js8 2y agoI wish there also was a distinction between truthful and whimsical.
- fuzzy_biscuit 2y agoWhen I was doing SEO full-time, this is one of the ways we used to categorize content - via intent. As a result, my immediate question becomes: how long before those responses start to be subsumed by commercial intent responses? To me, this is an inevitability. A when, not an if.
- deleted 2y ago[deleted]
- rwnspace 2y agoMy experience with them doesn't quite fit either: I've primarily used LLMs for giving me hints when I'm struggling with a leetcode problem or similar. They're surprisingly good at it, providing you regularly remind them to provide little clues only.
- trash_cat 2y agoBut then how do you classify a task that the LMM performed, such as a summary? I think you are onto something here but it really depends on what task you want the LMM to perform, search, how to, summary, extraction etc...
- bbor 2y agoWell put! If you/y’all haven’t heard, there’s a popular breakdown of “technical documentation” into four types, and this is one of the axes: https://nick.groenen.me/posts/the-4-types-of-technical-documentation/ https://nick.groenen.me/posts/the-4-types-of-technical-docum... People’s names for the four types vary, but I’m personally a fan of naming the axes “propositional vs procedural” and “informational vs developmental”, giving us a final four categories (00, 01, 10, 11) of “References”, “Instructions”, “Lessons”, and “Tutorials”. I think the applicability to LLM clearly holds up! Though more so for advanced chatbots than HR widgets TBF, I doubt anyone is looking for developmental content from one of those.
- shabie 2y agoThanks a lot for sharing, I have not heard of this before.
- niobe 2y agoWell put, and separating these would be a good use case for system prompts e.g. llm -m model --save instructional --system "provide the detailed steps to achieve the outcome, using a suitable example if necessary" llm -m model --save informational --system "provide a concise conceptual overview but do not provide implementation steps or detailed examples"
- shabie 2y agoThat's actually a pretty interesting point. Not just evals but other components like system prompt should also be tailored to match the expected outcome.