7 ms·
> Yes the team which literally created transformer and almost all the important open research including Bert, T5, imagen, RLHF, ViT don’t have the ability to ex
by dragonwriter 4y ago
> Yes the team which literally created transformer and almost all the important open research including Bert, T5, imagen, RLHF, ViT don’t have the ability to execute on AI /s.
This, but non-sarcastically. Google has spectacularly, so far, failed to execute on products (even of the “selling shovels” kind, much less end-user products) for generative AI, despite both having lots of consumer products to which it is naturally adaptable and a lot of the fundamental research work in generative AI.
The best explanation is that they actually are, institutionally and structurally, bad at execution in this domain, because they have all the pieces and incentives that rule out most of the other potential explanations for that.
> OpenAI bought into the field. They are good at execution but i havent seen anything novel coming out of them.
Right, OpenAI is good at execution (at least, when it comes to selling-shovels tools, I don’t see a lot of evidence beyond that yet), whereas Google is, to all current evidence, not good at execution in this space.
- tomComb 4y agoI’m puzzled that stuff like alpha Fold count for nothing in this discussion (having just browsed through most of it). I saw quotes from independent scientists referring to it as the greatest breakthrough of their lifetime, and I saw similarly strong language used in regard to the potential for good of alpha fold as a product. So they gave it away, but it is still a product they followed through on and continue to. Was it wrong of them that they gave it away, and right, that Microsoft’s primary intent with their open AI technology, seems to be to provoke an arms race with google?
- djtango 4y agoWho cares about protein folding when a hyped up ELIZA can confidently tell you lies /s
- ramraj07 4y agoAlpha Fold is a game changer, but nowhere near the game changer ChatGPT(4) is, even if ChatGPT was only available for the subset of scientists that benefit from Alpha Fold. We are literally arguing semantics if this is AGI, and you're comparing it to a bespoke ML model that solves a highly specific domain problem (as unsolvable and impressive as it was).
- adammarples 4y agoThe domain is the domain of protein structure, something which potentially has gigantic applications to life. Predicting proteins may yet prove more useful than predicting text.
- theragra 4y agoIf so, where are application of this? Is it too early?
- visarga 4y agoCould be interesting to correlate DNA with text produced by people. Both are self replicating, self evolving languages.
- ramraj07 4y ago“Predicting proteins”? I’m a biologist and I can assure you knowing the rough structure of a protein from sequence is nowhere near as important to biology as everyone makes it out to be. It is Nobel prize worthy to be sure but Nobel prizes are awarded once a year not once a century.
- misnome 4y ago> We are literally arguing semantics if this is AGI, And if it isn't? Literally every single argument I've seen towards this being AGI is "We don't know at all how intelligence works, so let's say that this is it!!!!!" > nowhere near the game changer ChatGPT(4) is, even if ChatGPT was only available for the subset of scientists that benefit from Alpha Fold This is utter nonsense. For anyone who actually knows a field, ChatGPT generates unhelpful, plausible-looking nonsense. Conferences are putting up ChatGPT answers about their fields to laugh at because of how misleadingly wrong they are. This is absolutely okay, because it can be a useful tool without being the singularity. I'd sure that in a couple of years time, most of what ChatGPT achieves will be in line with most of the tech industry advances in the past decade - pushing the bottom out of the labor market and actively making the lives of the poorest worse in order to line their own pockets. I really wish people would stop projecting hopes and wishes on top of breathless marketing.
- dragonwriter 4y ago> So they gave it away, but it is still a product Except its not, because they gave it away without any kind of commercialization. Its possible to give something away for free in some context and still have it be a product (Stable Diffusion is doing quite a bit of that, though its very unclear if they’ll be able to do it sustainably), but AlphaFold doesn’t seem to be an example. It seems to be an example of something cool they did that they had no desire to make into a product. Which is great! But isn’t the same as executing on product in a space.
- ChatGTP 4y agoThis is hacker news, AlphaFold doesn’t have an app, some obscure GitHub repo, a hyped up website or a bunch of VC backing, so it’s basically a waste of time.
- neel8986 4y agoGenerative AI at its current state is still a very new area of research with many issues including hallucination, bias and legal baggage. So for the first few version we are looking at many new startups like open ai, stability, anthropic etc. It is yet to be seen if any of the new breed of startups actually starts to make sizeable revenue. But again there is nothing defensible here unless all the major labs stop publishing paper.
- thefourthchime 4y agoNumerous individuals have since transitioned away from Google, with reports suggesting their growing dissatisfaction as the company appeared indecisive about utilizing their technological innovations effectively. Moreover, it has been quite some time since Google successfully developed and sustained a high-quality product without ultimately discontinuing it. The organizational structure at Google seems to inadvertently hinder the creation of exceptional products, exemplifying Conway's Law in practice. Read more about this topic here: https://www.wsj.com/articles/google-ai-chatbot-bard-chatgpt-rival-bing-a4c2d2ad https://www.wsj.com/articles/google-ai-chatbot-bard-chatgpt-...
- nostrademons 4y agoThey're getting Innovator's Dilemma'd, the same way that Bell Labs, DEC, and Xerox did. When you have an exceptionally profitable monopoly, it biases every executive's decision-making toward caution. Things are good; you don't want to upset the golden goose by making any radical moves; and so when your researchers come out with something revolutionary and different you bury it, maybe let them publish a few papers, but certainly don't let it go to market. Then somebody else reads the papers, decides to execute on it, and hires all the researchers who are frustrated at discovering all this cool stuff but never seeing it launch.
- dmix 4y agoThe typical solution to this (assuming there is one internally) is setting up a sub-company and keeping the team isolated from the parent company aka "intrapenuership" but also keeping them well resourced by the parent. It seems like that's what they were doing with DeepMind for the last decade. But it's also possible DeepMind as an institution lacked the pressure/product sense/leadership to produce consumable products/services. Maybe their instincts were more centered around R&D and being isolated left them somewhat directionless? So now that AI suddenly really matters as a business, not just some indefinite future potential, Google wants to bring them inside. They could have created a 3rd entity, their own version of OpenAI, combining DeepMind with some Google management/teams and other acquisitions and spinning it off semi-independently. But this play basically has to be from Google itself for their own reputation's sake - maybe not for practicality's sake but politically/image-wise.
- pclmulqdq 4y agoThe problem with the intrapreneurship idea is that it's really hard to beat desperation as a motivator. I have seen people behave very differently in the context of a startup vs a corporate research lab thanks to this dynamic. Some people thrive in the corporate R&D environment, but the innovator's dilemma eventually gets to their managers. Cisco has done a great job balancing this, actually - they keep contact with engineers who leave to do startups, and then acquire their companies if they become successful enough to prove the product.
- jsnell 4y agoWhen did anyone realize that there generative AI was actually a product with wide consumer appeal? Or how many use cases there were for it as an API service? I'd say it wasn't really obvious until around Q4 last year, maybe Q3 at the earliest. That's a pretty short time ago. So it seems that so far it hasn't really been a failure to execute, but more about problems with product vision or with reading the market right leading to not even attempting to have actual products in this space. That's definitely a problem, but not one that's particularly predictive of how well they'll be able to execute now that they're actually working on products.
- cbzoiav 4y agoThe bigger problem is cost. The hardware costs alone of running something like GPT 3.5 for real time results is 6-7 figures a year. By the time you scale for user numbers and add redundancy... The infra needs to be doing useful work 24/7 to pay for itself. It's more than possible Google knows exactly what it can do, but was waiting for it to be financially viable before acting on that. Meanwhile Microsoft has decided to throw money at it like no tomorrow - if they corner the market and it becomes financially viable before they lose that it could pay off. That is a major gamble...
- nullc 4y ago> The hardware costs alone of running something like GPT 3.5 for real time results is 6-7 figures a year. Can you unpack your thinking there? Even at 5% interest for ownership costs to be six figures a year you're talking about millions of dollars in hardware. Inference is just not that expensive, not even with gigantic models. To the extent that there is operating cost (e.g. energy)-- that isn't generated when the system is offline. I don't know how big GPT 3.5 is, but I can _train_ LLaMA 65B on hardware at home and it is nowhere near that expensive.
- cbzoiav 4y agohttps://twitter.com/tomgoldsteincs/status/1600196981955100694 https://twitter.com/tomgoldsteincs/status/160019698195510069... That's 8 $200k GPUs + all the other hardware + power consumption for one instance. You could run it on cheaper hardware, but then you'll get to nowhere near realtime output which is required for the majority of the use cases not already handled well by much smaller models. Even if Google/Microsoft are getting the hardware at a 50% reduction (bearing in mind these are already not consumer prices) it gets to $1mn in hardware alone - again for a single instance that can handle one user interacting with it at a time. It makes a lot of the bespoke usecases people are getting excited about (i.e. anything with data privacy concerns) far from financially viable. If you want a dedicated instance of full capability ChatGPT for example (32K content) OpenAI are charging $468k for a 3 month commitment / $1,584k for a year.
- kernal 4y ago>This, but non-sarcastically. Google has spectacularly, so far, failed to execute on products Android is the biggest OS in the world Chrome is the biggest browser in the world Gmail is the biggest email service in the world YouTube is the biggest video platform in the world Google is the biggest search engine in the world Google is the biggest digital advertiser in the world and I'm probably missing more things they're #1 in. Not bad for a company that has "spectacularly failed to execute on products"
- tester756 4y agoI'd add Maps
- TheCoelacanth 4y agoWhich of those do you think is a product "for generative AI"?
- Fricken 4y agoThe youngest of those products is 15 years old.
- pb7 4y ago15 years of dominance? Sounds pretty competent at keeping up with market changes. No reason to believe generative AI is an exception.
- dragonwriter 4y agoUh, you snipped in the middle of a clause so you could argue against something it didn’t say. Here’s the whole thing (leaving out a parenthetical that isn’t important here): “Google has spectacularly, so far, failed to execute on products […] for generative AI” You listed a bunch of products in other domains, some of which are the reasons why it has institutional incentives not to push generative AI forward, even if it also stands to lose more if someone else wins in it.
- 29athrowaway 4y agoAll of those are cost centers paid with their adtech business.
- nullc 4y agoIs it just execution or did they gaze deeply into their navels and convince themselves that delivering tools like GPT4 would be 'unethical'?