3 ms·
> I just gave Bard a try. It's still nowhere near as good at coding as chatGPT (and GPT4 specifically) Because its a much smaller model IMO. Its like comparing
by amf12 3y ago
> I just gave Bard a try. It's still nowhere near as good at coding as chatGPT (and GPT4 specifically)
Because its a much smaller model IMO. Its like comparing a car with 2.0 engine, with a car with 3.0 engine. Also ChatGPT made a lot of errors in code just a few months ago. Google seems to be catching up.
- verdverm 3y agoWorth noting that Google i/o is next month and many are expecting announcements related to Bard. There is that rumor floating around that they are training a very large (30T) model
- thewataccount 3y agoThe only source I can find is a reddit post from April fool's day, do you have a better one by chance? There's almost no way they make a 30T model that's useful at least? That's over 60TB of memory at 16bit, 15TB at 4 bit. That's 15TB that have to go through either VRAM or TPU Memory (TPUv4 is 32GB I think?). Yes you could distribute it and chunk it, but that would be insanely slow/expensive to run for anything more then a research demo. GPT-4 is most likely 1~2T from what I've heard, and they've been struggling to run that profitably/quickly judging by the crazy pricing and 25messages/3hours if you pay for premium chatgpt. EDIT: The only way this is possible is if they've pulled off some crazy optimization, which is definitely possible, but a lot of the research from them such as their own Chinchilla scaling suggest smaller models that are more fully trained are better. According to their chinchilla paper they would need over 330TB of training data for that, and an absurd amount of training time.
- verdverm 3y agoSee my other comment: https://news.ycombinator.com/item?id=35657567 https://news.ycombinator.com/item?id=35657567