11 ms·

It seems like OpenAI are finally living up to their name for once with this release? Anything I'm missing? From what I can gather: 1. Includes model weights.

It seems like OpenAI are finally living up to their name for once with this release? Anything I'm missing? From what I can gather: 1. Includes model weights. I can't find the URL, but they reference them enough and have a CLI tool, so I presume I just haven't found them yet. 2. Includes code: https://github.com/openai/whisper https://github.com/openai/whisper 3. Released under MIT License: https://github.com/openai/whisper/blob/main/LICENSE https://github.com/openai/whisper/blob/main/LICENSE

StevenWaterman 4y ago
(Model weights from https://github.com/openai/whisper/blob/main/whisper/__init__.py https://github.com/openai/whisper/blob/main/whisper/__init__... ) "tiny.en": "https://openaipublic.azureedge.net/main/whisper/models/d3dd57d32accea0b295c96e26691aa14d8822fac7d9d27d5dc00b4ca2826dd03/tiny.en.pt https://openaipublic.azureedge.net/main/whisper/models/d3dd5..." "tiny": "https://openaipublic.azureedge.net/main/whisper/models/65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9/tiny.pt https://openaipublic.azureedge.net/main/whisper/models/65147..." "base.en": "https://openaipublic.azureedge.net/main/whisper/models/25a8566e1d0c1e2231d1c762132cd20e0f96a85d16145c3a00adf5d1ac670ead/base.en.pt https://openaipublic.azureedge.net/main/whisper/models/25a85..." "base": "https://openaipublic.azureedge.net/main/whisper/models/ed3a0b6b1c0edf879ad9b11b1af5a0e6ab5db9205f891f668f8b0e6c6326e34e/base.pt https://openaipublic.azureedge.net/main/whisper/models/ed3a0..." "small.en": "https://openaipublic.azureedge.net/main/whisper/models/f953ad0fd29cacd07d5a9eda5624af0f6bcf2258be67c92b79389873d91e0872/small.en.pt https://openaipublic.azureedge.net/main/whisper/models/f953a..." "small": "https://openaipublic.azureedge.net/main/whisper/models/9ecf779972d90ba49c06d968637d720dd632c55bbf19d441fb42bf17a411e794/small.pt https://openaipublic.azureedge.net/main/whisper/models/9ecf7..." "medium.en": "https://openaipublic.azureedge.net/main/whisper/models/d7440d1dc186f76616474e0ff0b3b6b879abc9d1a4926b7adfa41db2d497ab4f/medium.en.pt https://openaipublic.azureedge.net/main/whisper/models/d7440..." "medium": "https://openaipublic.azureedge.net/main/whisper/models/345ae4da62f9b3d59415adc60127b97c714f32e89e936602e85993674d08dcb1/medium.pt https://openaipublic.azureedge.net/main/whisper/models/345ae..." "large": "https://openaipublic.azureedge.net/main/whisper/models/e4b87e7e0bf463eb8e6956e646f1e277e901512310def2c24bf0e11bd3c28e9a/large.pt https://openaipublic.azureedge.net/main/whisper/models/e4b87..."
- mmastrac 4y ago
  Large is 3GB to save everyone a click. Tiny is 72MB.
  - anigbrowl 4y ago
    That's unexpectedly lightweight - enough to run in some phones.
    - yencabulator 4y ago
      However, https://github.com/openai/whisper#available-models-and-languages https://github.com/openai/whisper#available-models-and-langu... says requires ~1 GB VRAM.
solarmist 4y ago
This kind of model is harder to abuse, so I guess it passed their internal checks much more easily. I can understand not releasing GPT-3, even if I disagree with the decision.
- dwohnitmok 4y ago
  > I can understand not releasing GPT-3, even if I disagree with the decision. Why do you disagree?
  - bigyikes 4y ago
    I don’t see how GPT-3 is any more dangerous than Stable Diffusion, Photoshop, that fake news website the crazy person you’re friends with on Facebook really likes, or any of the number of other tools and services that can be used to generate or spread fake information.
    - jfoster 4y ago
      All of your examples are limited in some way, but GPT-3 wouldn't have any meaningful limits. Stable Diffusion: Marks images as AI-generated. (invisible watermark, but still, it's there) Photoshop: Requires time & effort from a human. Fake news website: Requires time & effort from a human.
      - nullc 4y ago
        It would be pretty trivial to have an invisible watermark in GPT3 output-- though you don't really need one: just score text with gpt3 to find out if it was likely gpt3 generated or not.
      - spullara 4y ago
        SD only does that if you don't delete the line of code that does it...
        deleted 4y ago
thesausageking 4y ago
It's one model and in a non-strategic area where there are existing open source projects (Kaldi, DeepSpeech, ...). For a company that raised $1B, that's not exactly living up to their name and original mission.
- whimsicalism 4y ago
  > It's one model and in a non-strategic area where there are existing open source projects (Kaldi, DeepSpeech, ...). I can already tell this is much better than any of the existing open source projects with the exception of the wav2* sequence of projects and potentially nvidia's nemo.
  - thesausageking 4y ago
    Kaldi is an open, pluggable framework and is a ton more flexible and powerful than this. It's used by hundreds of teams, including a number of consumer tech companies you've heard of. They're not going to move to this over it. Especially because ASR is a living organism. You have to constantly update your language model as new people, ideas, and words move into the normal lexicon. As people start talking about "COVID", "metaverse", "king charles", or whatever new things that happen, these need to be added to your language model. You need these updates monthly at a minimum and OpenAI didn't release the raw data which means you can't retrain it even if you wanted to spend the time/resources to. So, this is an interesting research project and helpful for small teams and side projects, but it's unlikely it makes any real impact on the industry.
    - IshKebab 4y ago
      Have you actually tried to use Kaldi though? I have. It's basically impenetrable unless your full time job is working with Kaldi.
    - whimsicalism 4y ago
      Kaldi just is not fast or high quality enough compared to other modern alternatives like wav2letter. I appreciate that it is more flexible than this, it certainly is - but I am not so sure about "powerful."

StevenWaterman 4y ago

(Model weights from https://github.com/openai/whisper/blob/main/whisper/__init__.py https://github.com/openai/whisper/blob/main/whisper/__init__... ) "tiny.en": "https://openaipublic.azureedge.net/main/whisper/models/d3dd57d32accea0b295c96e26691aa14d8822fac7d9d27d5dc00b4ca2826dd03/tiny.en.pt https://openaipublic.azureedge.net/main/whisper/models/d3dd5..." "tiny": "https://openaipublic.azureedge.net/main/whisper/models/65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9/tiny.pt https://openaipublic.azureedge.net/main/whisper/models/65147..." "base.en": "https://openaipublic.azureedge.net/main/whisper/models/25a8566e1d0c1e2231d1c762132cd20e0f96a85d16145c3a00adf5d1ac670ead/base.en.pt https://openaipublic.azureedge.net/main/whisper/models/25a85..." "base": "https://openaipublic.azureedge.net/main/whisper/models/ed3a0b6b1c0edf879ad9b11b1af5a0e6ab5db9205f891f668f8b0e6c6326e34e/base.pt https://openaipublic.azureedge.net/main/whisper/models/ed3a0..." "small.en": "https://openaipublic.azureedge.net/main/whisper/models/f953ad0fd29cacd07d5a9eda5624af0f6bcf2258be67c92b79389873d91e0872/small.en.pt https://openaipublic.azureedge.net/main/whisper/models/f953a..." "small": "https://openaipublic.azureedge.net/main/whisper/models/9ecf779972d90ba49c06d968637d720dd632c55bbf19d441fb42bf17a411e794/small.pt https://openaipublic.azureedge.net/main/whisper/models/9ecf7..." "medium.en": "https://openaipublic.azureedge.net/main/whisper/models/d7440d1dc186f76616474e0ff0b3b6b879abc9d1a4926b7adfa41db2d497ab4f/medium.en.pt https://openaipublic.azureedge.net/main/whisper/models/d7440..." "medium": "https://openaipublic.azureedge.net/main/whisper/models/345ae4da62f9b3d59415adc60127b97c714f32e89e936602e85993674d08dcb1/medium.pt https://openaipublic.azureedge.net/main/whisper/models/345ae..." "large": "https://openaipublic.azureedge.net/main/whisper/models/e4b87e7e0bf463eb8e6956e646f1e277e901512310def2c24bf0e11bd3c28e9a/large.pt https://openaipublic.azureedge.net/main/whisper/models/e4b87..."

mmastrac 4y ago
Large is 3GB to save everyone a click. Tiny is 72MB.
- anigbrowl 4y ago
  That's unexpectedly lightweight - enough to run in some phones.
  - yencabulator 4y ago
    However, https://github.com/openai/whisper#available-models-and-languages https://github.com/openai/whisper#available-models-and-langu... says requires ~1 GB VRAM.

solarmist 4y ago

This kind of model is harder to abuse, so I guess it passed their internal checks much more easily. I can understand not releasing GPT-3, even if I disagree with the decision.

dwohnitmok 4y ago
> I can understand not releasing GPT-3, even if I disagree with the decision. Why do you disagree?
- bigyikes 4y ago
  I don’t see how GPT-3 is any more dangerous than Stable Diffusion, Photoshop, that fake news website the crazy person you’re friends with on Facebook really likes, or any of the number of other tools and services that can be used to generate or spread fake information.
  - jfoster 4y ago
    All of your examples are limited in some way, but GPT-3 wouldn't have any meaningful limits. Stable Diffusion: Marks images as AI-generated. (invisible watermark, but still, it's there) Photoshop: Requires time & effort from a human. Fake news website: Requires time & effort from a human.
    - nullc 4y ago
      It would be pretty trivial to have an invisible watermark in GPT3 output-- though you don't really need one: just score text with gpt3 to find out if it was likely gpt3 generated or not.
    - spullara 4y ago
      SD only does that if you don't delete the line of code that does it...
      - deleted 4y ago

thesausageking 4y ago

It's one model and in a non-strategic area where there are existing open source projects (Kaldi, DeepSpeech, ...). For a company that raised $1B, that's not exactly living up to their name and original mission.

whimsicalism 4y ago
> It's one model and in a non-strategic area where there are existing open source projects (Kaldi, DeepSpeech, ...). I can already tell this is much better than any of the existing open source projects with the exception of the wav2* sequence of projects and potentially nvidia's nemo.
- thesausageking 4y ago
  Kaldi is an open, pluggable framework and is a ton more flexible and powerful than this. It's used by hundreds of teams, including a number of consumer tech companies you've heard of. They're not going to move to this over it. Especially because ASR is a living organism. You have to constantly update your language model as new people, ideas, and words move into the normal lexicon. As people start talking about "COVID", "metaverse", "king charles", or whatever new things that happen, these need to be added to your language model. You need these updates monthly at a minimum and OpenAI didn't release the raw data which means you can't retrain it even if you wanted to spend the time/resources to. So, this is an interesting research project and helpful for small teams and side projects, but it's unlikely it makes any real impact on the industry.
  - IshKebab 4y ago
    Have you actually tried to use Kaldi though? I have. It's basically impenetrable unless your full time job is working with Kaldi.
  - whimsicalism 4y ago
    Kaldi just is not fast or high quality enough compared to other modern alternatives like wav2letter. I appreciate that it is more flexible than this, it certainly is - but I am not so sure about "powerful."