3 ms·

Note that this result actually turns out to generalize well beyond Claude itself: Anthropic has actually conducted very similar research on open weight models,

Note that this result actually turns out to generalize well beyond Claude itself: Anthropic has actually conducted very similar research on open weight models, which they call Model Spec Midtraining https://arxiv.org/abs/2605.02087 https://arxiv.org/abs/2605.02087 (discussed at https://alignment.anthropic.com/2026/msm https://alignment.anthropic.com/2026/msm ) and they have released fine tuned versions of open models trained for a variety of toy "values" (Llama 3.1 8B, Qwen 2.5 32B, Qwen 3 32B) in order to show how the elicitation of these values in any one training context shapes the model's response to tangentially related questions: https://github.com/chloeli-15/model_spec_midtraining https://github.com/chloeli-15/model_spec_midtraining https://huggingface.co/chloeli/collections https://huggingface.co/chloeli/collections Very exciting to see this continued interaction with the open weights community, after the earlier NLA paper!

NitpickLawyer 6mo ago
Really interesting resource, thanks for sharing! It was not on my radar. > https://github.com/chloeli-15/model_spec_midtraining https://github.com/chloeli-15/model_spec_midtraining I'm a bit confused about this part: > MSM is a pipeline that takes a Model Spec or Constitution (a document describing how and why an assistant should behave) and generates a diverse corpus of synthetic documents that discuss and teach the content of the spec. > ANTHROPIC_API_KEY=sk-ant-... > # Optional but highly recommeded — separate key for using the Anthropic Batch API for batch document generation (needed if USE_BATCH_API=true). # This will significantly reduce generation time high-volume generation. ANTHROPIC_BATCH_API_KEY=sk-ant-... Isn't this specifically against Anthropic's ToS? I thought generating data to train other models was specifically disallowed. I get this is a research effort, but still. Say you use this pipeline for something internal, this would be against the ToS and risk getting banned, no?
spwa4 6mo ago
Why do you believe this is what Anthropic is using? You can just directly verify that! If you want to know Claude's alignment, just ask about whether it was wrong to use copyrighted data to train Claude ... you will find it was not wrong, and it is unwilling to discuss further, or implications. In much the same way as discussing Tiananmen with Qwen. Anthropic's actions were obviously judged wrong by just about everyone and everything including even the US state, that judged them illegal. This makes Anthropic's actions against just about every moral system. Claude obviously has a different alignment. In other words: Claude's value system already has the priority "protect Anthropic's money" as having higher priority than following the law. THAT is it's alignment. You can simply objectively verify if this is the case or not.
- RexM 6mo ago
  [flagged]

NitpickLawyer 6mo ago

Really interesting resource, thanks for sharing! It was not on my radar. > https://github.com/chloeli-15/model_spec_midtraining https://github.com/chloeli-15/model_spec_midtraining I'm a bit confused about this part: > MSM is a pipeline that takes a Model Spec or Constitution (a document describing how and why an assistant should behave) and generates a diverse corpus of synthetic documents that discuss and teach the content of the spec. > ANTHROPIC_API_KEY=sk-ant-... > # Optional but highly recommeded — separate key for using the Anthropic Batch API for batch document generation (needed if USE_BATCH_API=true). # This will significantly reduce generation time high-volume generation. ANTHROPIC_BATCH_API_KEY=sk-ant-... Isn't this specifically against Anthropic's ToS? I thought generating data to train other models was specifically disallowed. I get this is a research effort, but still. Say you use this pipeline for something internal, this would be against the ToS and risk getting banned, no?

spwa4 6mo ago

Why do you believe this is what Anthropic is using? You can just directly verify that! If you want to know Claude's alignment, just ask about whether it was wrong to use copyrighted data to train Claude ... you will find it was not wrong, and it is unwilling to discuss further, or implications. In much the same way as discussing Tiananmen with Qwen. Anthropic's actions were obviously judged wrong by just about everyone and everything including even the US state, that judged them illegal. This makes Anthropic's actions against just about every moral system. Claude obviously has a different alignment. In other words: Claude's value system already has the priority "protect Anthropic's money" as having higher priority than following the law. THAT is it's alignment. You can simply objectively verify if this is the case or not.

RexM 6mo ago
[flagged]