5 ms·

> Due to our concerns about malicious applications of the technology, we are not releasing the trained model. Has anyone said when they are intending to releas

> Due to our concerns about malicious applications of the technology, we are not releasing the trained model. Has anyone said when they are intending to release the full model? There's likely a lot of positive applications of this technology as well.

p1esk 7y ago
They said within 6 months, however by that time there will likely be something better.
- marvin 7y ago
  Releasing publicly AI tech that is six months behind the state of the art, might be a reasonable compromise against the fear of nefarious use. That way, more people know that the tech exists before it's available to everyone, and get a 6-month window to adapt. At the same time, the risk is lower that a different team develops similar capabilities and widely deploy it without anyone even knowing that what they do is possible. Wouldn't be surprised if this pattern is followed even more in the future. It's an acknowledgement that there are many angry cats that will (almost unstoppably) be let out of the bag in the next decades, but attempting at releasing the cats in the most responsible manner possible.
- speedplane 7y ago
  > They said within 6 months, however by that time there will likely be something better. There has. BERT has come out and is better.
  - cthalupa 7y ago
    Erm, isn't a big part of this whole story that GPT-2 > BERT -> GPT?
  - gwern 7y ago
    BERT is bidirectional. How do you use that for language generation?
    - visarga 7y ago
      It will generate words for every [UNK] in its input sequence.
      - gwern 7y ago
        It'll generate one token because it's trained to predict one missing UNK, as I understood it. What is the scaffolding? Do you generate random sentences and iterate repeatedly? And how does that get you whole coherent paragraphs? (Has anyone demonstrated that this actually works with BERT?)
        speedplane 7y ago
        BERT can pretty easily be used to generate text. It's intended to be used as a base model and fine-tuned with an additional model on top. The fine-tuning model could then be trained to generate sentences with the underlying language model powered by BERT.

p1esk 7y ago

They said within 6 months, however by that time there will likely be something better.

marvin 7y ago
Releasing publicly AI tech that is six months behind the state of the art, might be a reasonable compromise against the fear of nefarious use. That way, more people know that the tech exists before it's available to everyone, and get a 6-month window to adapt. At the same time, the risk is lower that a different team develops similar capabilities and widely deploy it without anyone even knowing that what they do is possible. Wouldn't be surprised if this pattern is followed even more in the future. It's an acknowledgement that there are many angry cats that will (almost unstoppably) be let out of the bag in the next decades, but attempting at releasing the cats in the most responsible manner possible.
speedplane 7y ago
> They said within 6 months, however by that time there will likely be something better. There has. BERT has come out and is better.
- cthalupa 7y ago
  Erm, isn't a big part of this whole story that GPT-2 > BERT -> GPT?
- gwern 7y ago
  BERT is bidirectional. How do you use that for language generation?
  - visarga 7y ago
    It will generate words for every [UNK] in its input sequence.
    - gwern 7y ago
      It'll generate one token because it's trained to predict one missing UNK, as I understood it. What is the scaffolding? Do you generate random sentences and iterate repeatedly? And how does that get you whole coherent paragraphs? (Has anyone demonstrated that this actually works with BERT?)
      - speedplane 7y ago
        BERT can pretty easily be used to generate text. It's intended to be used as a base model and fine-tuned with an additional model on top. The fine-tuning model could then be trained to generate sentences with the underlying language model powered by BERT.