3 ms·
What kind of training data did they use? Are the training data and replies censored in any way?
by DeathArrow 3y ago
What kind of training data did they use? Are the training data and replies censored in any way?
- GaggiX 3y agoAre the training data and replies censored in any way? It doesn't seem so.
- donatzsky 3y agoFrom the link: > Mixtral is pre-trained on data extracted from the open Web – we train experts and routers simultaneously. > Note: Mixtral can be gracefully prompted to ban some outputs from constructing applications that require a strong level of moderation, as exemplified here. A proper preference tuning can also serve this purpose. Bear in mind that without such a prompt, the model will just follow whatever instructions are given.