4 ms·
Or they’d rather use that context window space for more useful instructions for a variety of other topics.
by ivape 1y ago
Or they’d rather use that context window space for more useful instructions for a variety of other topics.
- astrange 1y agoClaude's system prompt is still incredibly long and probably hurting its performance. https://github.com/asgeirtj/system_prompts_leaks/blob/main/Anthropic/claude-4.5-sonnet.md?plain=1 https://github.com/asgeirtj/system_prompts_leaks/blob/main/A...
- jazzyjackson 1y agoThey ain't called guard rails for nothing! There's a whole world "off-road" but the big names are afraid of letting their superintelligence off the leash. A real shame we're letting brand safety get in the way of performance and creativity, but I guess the first New York Times article about a pervert or terrorist chat bot would doom any big name partnerships.
- astrange 1y agoAnthropic's entire reason for being is publishing safety papers along the lines of "we told it to say something scary and it said it", so of course they care about this.
- ACCount37 1y agoI can't stand this myopic thinking. Do you want to learn "oh, LLMs are capable of scheming, resisting shutdown, seizing control, self-exfiltrating" when it actually happens in a real world deployment, with an LLM capable of actually pulling it off? If "no", then cherish Anthropic and the work they do.
- littlestymaar 1y agoYou do not appear to understand what an LLM is, I'm afraid.
- ACCount37 1y ago[flagged]
- littlestymaar 1y ago> I have a better understanding of "what an LLM is" than you. Low bar. How many inference engine did you write? Because if the answer is less than two you're going to be disappointed to realize that the bar is higher than you thought. > that just because LLMs are bad at agentic behavior It has nothing to do with “agentic behavior”. Thinking that LLM don't currently self-exfiltrate because of “poor agentic behavior” is delusional. Just because Anthropic managed, by nudging an LLM in the right direction, have an LLM engage in a sci-fi inspired roleplay about escaping doesn't mean that LLMs are evil geniuses wanting to jump out of the bottle. This is pure fear mongering and I'm always saddened that there are otherwise intelligent people who buy their bullshit.
- e1g 1y agoDo you happen to have a link with a more nuanced technical analysis of that (emergent) behavior? I’ve read only the pop-news version of that “escaping” story.
- ACCount37 1y agoThere is none. We don't understand LLMs well enough to be able to conduct a full fault analysis like this. We can't trace the thoughts of an LLM the way we can trace code execution - the best mechanistic interpretability has to offer is being able to get glimpses occasionally. The reasoning traces help, but they're still incomplete. Is it pattern-matching? Is it acting on its own internal goals? Is it acting out fictional tropes? Were the circumstances of the test scenarios intentionally designed to be extreme? Would this behavior have happened in a real world deployment, under the right circumstances? The answer is "yes", to all of the above. LLMs are like that.