4 ms·
Falcon was trained by a research think tank, and this is going to happen more and more as AI compute gets cheaper and the frameworks get better. We aren't going
by brucethemoose2 3y ago
Falcon was trained by a research think tank, and this is going to happen more and more as AI compute gets cheaper and the frameworks get better. We aren't going to be stuck with overpriced A100s running early PyTorch eager mode code for long.
High quality "open" data is an tricky problem, but the community has already come up with some impressive datasets for finetunes, like the roleplaying collection used for Chronos and Pygmalion. I think this is going to get better too, epecially if cheap models can "distill" lower quality human datasets.
- ianbutler 3y ago> We aren't going to be stuck with overpriced A100s running early PyTorch eager mode code for long. I agree and am eagerly awaiting this on the hardware side and trying to participate in this where I can on the software side.
- KRAKRISMOTT 3y ago> Falcon was trained by a research think tank A research think tank subsidized by UAE’s oil wealth. Somebody get Masayoshi on the line, the Saudis must be hungry for deals for their vision fund.
- senko 3y agoUnnecessary ad hominem. Every research tank is subsidized by someone.
- KRAKRISMOTT 3y agoVery true, MIT should disclose their military affiliations too when publishing. Otherwise, they are just dishonest
- meepmorp 3y agoThey do. The grants themselves have requirements for acknowledging the funder in publications; most researchers also mention the funding sources for their work and projects on their website and in their CVs. People who get those grants are proud of the fact and like the money.
- logicchains 3y agoSchmidhuber, the father of deep learning, is already on it: https://cemse.kaust.edu.sa/people/person/jurgen-schmidhuber https://cemse.kaust.edu.sa/people/person/jurgen-schmidhuber .
- hyperbovine 3y ago"The father of deep learning", hah. Sounds like something he would say.