4 ms·

If size was the major factor, then Amazon with it's GPU farms would be the king. Google also would have nothing to worry about. So, just the size is not enough

If size was the major factor, then Amazon with it's GPU farms would be the king. Google also would have nothing to worry about. So, just the size is not enough to explain. I think a) it is really complex thing with many components b) multimodal adds different components. But, I think, it's not limited to text and images. c) they found the way to use 'addons', algorithms, or external models, to improve the results. this would explain why it is good at some tasks, but not the others. d) using 'addons' requires model (re)training. however, some addons can be updated separately. e) they put quite a lot of work in design and implementation, besides the training itself.

ftxbro 3y ago
> If size was the major factor, then Amazon with it's GPU farms would be the king. Google also would have nothing to worry about. It's size (gpt architecture, params, training set, gpus). But there are two complicating factors: first that expertise in training a hundred million dollar transformer LLM model hasn't yet diffused throughout the industry and most of them work at OpenAI, and second is that there is nothing close to equilibrium reached yet. It's the middle of a race right now. You say that Google would have nothing to worry about. Well, maybe you don't know it but Google already has PaLM which is competitive with ChatGPT (although maybe its most recently published version isn't as good as GPT-4). But it's not public. They also have LaMDA the one that Blake Lemoine flagged as having a kind of sentience that is worthy of some rights. Maybe he had a point or maybe he was crazy but in any case LaMDA isn't public either. Their public one is Bard which everyone knows sucks. It's not like Google can't make good AI, it's that they have already made it and they are using it for something else. Presumably they decided it's not worth exposing it publicly. Amazon is joining the race too, and is undoubtedly making some bank by letting others train LLMs on their cloud.
- ChildOfChaos 3y ago
  I've heard that Bard has become very good at coding with a recent update and beats GPT4. I'm sure Google is sitting on a lot, it's about making it cost effective and scalable to the masses. OpenAI have massive investment and are not one of the big tech companies, so in order to get into the field, they have an advantage were they are happy to burn through cash to gain fame, it's working for them currently, but I wouldn't be surprised for some of the big players to pop up and be more relevant when the tech really starts to take off.
- two_in_one 3y ago
  I'm not saying Google isn't doing, or has nothing. So far it looks like it has nothing comparable to GPT-4. That's why they are panicking, reorganizing AI groups. They already had 2 months, and didn't catch up. Meta doesn't have either. I played with LLaMA. Only 65B model can tell how old Joe Biden will be in 2024. They all know his birth date. GPT-4 does it easily. Meta is in better position then Google, they have no competitors with comparable AI or AI team. Twitter is not even thinking about it, under the rule of notorious robophobe. Yes, we are in a race, just a beginning. The winner will be the one with better scientists and business model, IMHO. I'm not taking sides here. Just learning, looking for new ideas. Everything is accelerating, so I expect it to blow up one way or another. Subhuman (near)AGI will be probably first, with some superhuman abilities. That will be a big bang. Soon. And some people still believe there is nothing new since 80-th...

ftxbro 3y ago

> If size was the major factor, then Amazon with it's GPU farms would be the king. Google also would have nothing to worry about. It's size (gpt architecture, params, training set, gpus). But there are two complicating factors: first that expertise in training a hundred million dollar transformer LLM model hasn't yet diffused throughout the industry and most of them work at OpenAI, and second is that there is nothing close to equilibrium reached yet. It's the middle of a race right now. You say that Google would have nothing to worry about. Well, maybe you don't know it but Google already has PaLM which is competitive with ChatGPT (although maybe its most recently published version isn't as good as GPT-4). But it's not public. They also have LaMDA the one that Blake Lemoine flagged as having a kind of sentience that is worthy of some rights. Maybe he had a point or maybe he was crazy but in any case LaMDA isn't public either. Their public one is Bard which everyone knows sucks. It's not like Google can't make good AI, it's that they have already made it and they are using it for something else. Presumably they decided it's not worth exposing it publicly. Amazon is joining the race too, and is undoubtedly making some bank by letting others train LLMs on their cloud.

ChildOfChaos 3y ago
I've heard that Bard has become very good at coding with a recent update and beats GPT4. I'm sure Google is sitting on a lot, it's about making it cost effective and scalable to the masses. OpenAI have massive investment and are not one of the big tech companies, so in order to get into the field, they have an advantage were they are happy to burn through cash to gain fame, it's working for them currently, but I wouldn't be surprised for some of the big players to pop up and be more relevant when the tech really starts to take off.
two_in_one 3y ago
I'm not saying Google isn't doing, or has nothing. So far it looks like it has nothing comparable to GPT-4. That's why they are panicking, reorganizing AI groups. They already had 2 months, and didn't catch up. Meta doesn't have either. I played with LLaMA. Only 65B model can tell how old Joe Biden will be in 2024. They all know his birth date. GPT-4 does it easily. Meta is in better position then Google, they have no competitors with comparable AI or AI team. Twitter is not even thinking about it, under the rule of notorious robophobe. Yes, we are in a race, just a beginning. The winner will be the one with better scientists and business model, IMHO. I'm not taking sides here. Just learning, looking for new ideas. Everything is accelerating, so I expect it to blow up one way or another. Subhuman (near)AGI will be probably first, with some superhuman abilities. That will be a big bang. Soon. And some people still believe there is nothing new since 80-th...