4 ms·

I tried discussing energy drinks with it and found out that the King of Sweden has just issued an edict against Red Bull.

• thomassmith65 3y ago
  That sort of thing drains any enthusiasm I have for the project. Even if it were trained exclusively on old materials, in fact, I would worry about modern footnotes and book introductions tainting the training corpus (ie: by introducing knowledge it should not have). I do love the idea, though. It it were possible to use ideal training material, it would be close to time-travel. As it stands, there's a limit to what one can learn from the text it produces. Much value, imo, would come from what the model does not know. Ideally it would be difficult even to have it comprehend user prompts, due to how different English is today.
  • snovv_crash 3y ago
    I wonder if there's some benefits of reinitialising completely the final layers and then training with the new data. At least for image processing it's common to use a common backbone trained on everything, and then put a final layer or two of task-specific training on top.