4 ms·

The total data that the page will have to load on startup (probably using Fetch API) is: - 74 MB for the Whisper tiny.en model - 240 MB for the GPT-2 sm

by swyx 4y ago

The total data that the page will have to load on startup (probably using Fetch API) is: - 74 MB for the Whisper tiny.en model - 240 MB for the GPT-2 small model - Web Speech API is built-in in modern browsers cool but im now wondering what it would take to bring this down enough to put this in real apps? anyone talking about this?

arcturus17 4y ago
~314mb is a lot for a web app but small for a desktop or even a mobile app.
- dormento 4y ago
  > ~314mb is a lot for a web app but small for a desktop or even a mobile app. Everyday we stray further from god's light :/
  - tjoff 4y ago
    Those 314 MB are justified though, which can hardly be said for the typical app/homepage.
agolio 4y ago
I really liked how the page tells you the size it is planning to download, and prompts you before downloading. Coming from a limited bandwidth contract, I hate when I click a link and it instantly starts downloading a huge file. Great work OP!
justanotheratom 4y ago
Perhaps it will be built-in to browsers soon
- petercooper 4y ago
  Given Whisper is open source, I'd be surprised if it's not. It would be cool for Web Speech API's SpeechRecognition to simply use it, though that would make browser downloads a little beefier.
  - globalise83 4y ago
    It could easily be downloaded separately in the background once the browser application is already up and running. Would be great to have it in the browser though for sure.
- make3 4y ago
  I don't see why they would ever package GPT2 (the bigger model) in the browser. Speech to text has higher chances though, that's an interesting idea, as they already package text to speech too.
  - neltnerb 4y ago
    To be honest, I expect that in 10 years people will regularly use these sorts of text generation tools in the way text prediction and thesauruses and grammar checkers and spellcheckers are used today but for bigger blocks of text. I can't really see why not anyway, as more things are in the browser it makes sense to me to integrate the ability to "AI check" your text like a grammar or spell checker to improve your writing along some dimensions that you like. It's not honest, but in kind of the same way that a spellchecker isn't honest and since it's going to be possible anyway I don't see what extra harm it causes to make it accessible for everyone so that we can both actually see an upside and also begin to recognize that text we read is at this point likely to be at least partially AI generated and potentially factually incorrect. Even better if things like Firefox reader mode, one of my favorite tools, can also do text summarization. Just imagine the adversarial interaction between a tool designed to generate confident sounding fluff and one to summarize confident sounding fluff. Honestly it seems like a likely inevitable future path. It may as well be part of the browser where it stands a better chance of keeping people's long term attention on the ease of using these tools. Spammers will be able to do it, fake journalists and such will be able to do it, better if we can do it too so that at least we are aware of the potential abuse.
    - visarga 4y ago
      We need much better models in browsers. The main reason is to pass everything through the language model and get polite and helpful responses. You never have to see Google, the website or the ads ever again if you don't want to. The QA model should be able to detect most undesirable parts - spam, ads, fakes, factually incorrect data. Something like chatGPT running locally. This is important for privacy. If we run the model, we have a safe creative space. If they run the model, they get everything spilled out.
make3 4y ago
in real life the models are hosted on a server and you send the text and sound and receive the model's output
CGamesPlay 4y ago
Unfortunately these smaller models are also terrible at performance, particularly the GPT-2 model small model is really unsuitable for the task of generating text. The largest models publicly available, which are nowhere near GPT-3 Da Vinci level, are tens of GBs. We may be able to reduce the size without sacrificing performance, but that's an area of active research still.
fulafel 4y ago
Lots of web based apps load more data than this. The 300 MB is only 3 seconds on a gigabit connection.
addandsubtract 4y ago
We can bring back pre-loading screens for webpages from the Web 2.0 era.
- unnouinceput 4y ago
  Isn't WEB 2.0 era current era? I mean WEB 3.0 era is in relation to blockchains only, not the rest. The proponents of "everything on blockchain" they actually want that for everything (not that will ever work, but that's beyond our discussion)

by swyx 4y ago

arcturus17 4y ago

~314mb is a lot for a web app but small for a desktop or even a mobile app.

dormento 4y ago
> ~314mb is a lot for a web app but small for a desktop or even a mobile app. Everyday we stray further from god's light :/
- tjoff 4y ago
  Those 314 MB are justified though, which can hardly be said for the typical app/homepage.

agolio 4y ago

I really liked how the page tells you the size it is planning to download, and prompts you before downloading. Coming from a limited bandwidth contract, I hate when I click a link and it instantly starts downloading a huge file. Great work OP!

justanotheratom 4y ago

Perhaps it will be built-in to browsers soon

petercooper 4y ago
Given Whisper is open source, I'd be surprised if it's not. It would be cool for Web Speech API's SpeechRecognition to simply use it, though that would make browser downloads a little beefier.
- globalise83 4y ago
  It could easily be downloaded separately in the background once the browser application is already up and running. Would be great to have it in the browser though for sure.
make3 4y ago
I don't see why they would ever package GPT2 (the bigger model) in the browser. Speech to text has higher chances though, that's an interesting idea, as they already package text to speech too.
- neltnerb 4y ago
  To be honest, I expect that in 10 years people will regularly use these sorts of text generation tools in the way text prediction and thesauruses and grammar checkers and spellcheckers are used today but for bigger blocks of text. I can't really see why not anyway, as more things are in the browser it makes sense to me to integrate the ability to "AI check" your text like a grammar or spell checker to improve your writing along some dimensions that you like. It's not honest, but in kind of the same way that a spellchecker isn't honest and since it's going to be possible anyway I don't see what extra harm it causes to make it accessible for everyone so that we can both actually see an upside and also begin to recognize that text we read is at this point likely to be at least partially AI generated and potentially factually incorrect. Even better if things like Firefox reader mode, one of my favorite tools, can also do text summarization. Just imagine the adversarial interaction between a tool designed to generate confident sounding fluff and one to summarize confident sounding fluff. Honestly it seems like a likely inevitable future path. It may as well be part of the browser where it stands a better chance of keeping people's long term attention on the ease of using these tools. Spammers will be able to do it, fake journalists and such will be able to do it, better if we can do it too so that at least we are aware of the potential abuse.
  - visarga 4y ago
    We need much better models in browsers. The main reason is to pass everything through the language model and get polite and helpful responses. You never have to see Google, the website or the ads ever again if you don't want to. The QA model should be able to detect most undesirable parts - spam, ads, fakes, factually incorrect data. Something like chatGPT running locally. This is important for privacy. If we run the model, we have a safe creative space. If they run the model, they get everything spilled out.

make3 4y ago

in real life the models are hosted on a server and you send the text and sound and receive the model's output

CGamesPlay 4y ago

Unfortunately these smaller models are also terrible at performance, particularly the GPT-2 model small model is really unsuitable for the task of generating text. The largest models publicly available, which are nowhere near GPT-3 Da Vinci level, are tens of GBs. We may be able to reduce the size without sacrificing performance, but that's an area of active research still.

fulafel 4y ago

Lots of web based apps load more data than this. The 300 MB is only 3 seconds on a gigabit connection.

addandsubtract 4y ago

We can bring back pre-loading screens for webpages from the Web 2.0 era.

unnouinceput 4y ago
Isn't WEB 2.0 era current era? I mean WEB 3.0 era is in relation to blockchains only, not the rest. The proponents of "everything on blockchain" they actually want that for everything (not that will ever work, but that's beyond our discussion)