Safety, the way you use it has nothing to do with the work being done by the alignment team. Aligned, in this context, simply means that a models output are ali

Safety, the way you use it has nothing to do with the work being done by the alignment team. Aligned, in this context, simply means that a models output are aligned with our expectations... It follows instructions... does what it's told. A super intelligent model that refuses your instructions is no more useful than a dumb model that cannot understand them. The alignment team where part of the race to make more powerful models. Read openai's last paper if you want to understand it better from a technical perspective. https://openai.com/index/weak-to-strong-generalization/ https://openai.com/index/weak-to-strong-generalization/ Naturally you can frame this as a general safety thing, if you convince people that llms have embodied agency and might start breaking out of the system like a computer virus. But llms are text generators. They can do nothing physically, unless you allow them to and plug them in to something. The real risk is that people don't understand this, and lots of people probably are willing to plug these models in to dangerous situations, with only the assurance from openai that they are safe. If you do not have access to the training data used, then you cannot possibly know what behaviour has been trained in to it, and how a model might behave. With the training data, you can have a much better understanding, and the only remaining uncertainty being the non-deterministic nature of the implementation. But at least that's an explainable stochastic process.