13 ms·
>> Overall, I am not convinced this will be massively beneficial. I don't trust Google's ability to execute at scale in this area. Yes the team which literally
by neel8986 3y ago
>> Overall, I am not convinced this will be massively beneficial. I don't trust Google's ability to execute at scale in this area.
Yes the team which literally created transformer and almost all the important open research including Bert, T5, imagen, RLHF, ViT don't have the ability to execute on AI /s. Tell me one innovation OpenAI bought into the field. They are good at execution but i havent seen anything novel coming out of them.
- throwntoday 3y agoThere are plenty of products that launch on top tools and frameworks that are worth far more than the underlying will ever be. OpenAI is creating products, DeepMind was creating tools. It's not a matter of skill as much as objective. And DeepMind would still be starting at zero if they decide to pivot to products.
- user3939382 3y agoLook at SIP vs Facetime. We already had SIP av calls forever, few ever heard of it.
- ioblomov 3y agoI certainly haven’t! But the canonical example would be Xerox’s Alto vs the Macintosh.
- mlinsey 3y agoDrawing a bright line between research vs products and considering only the former to be "innovation" is a way to become quickly irrelevant.
- bushbaba 3y ago+1. Bell labs said hi
- neel8986 3y agoProduct based on no fundamental innovation is also a path to irrelevance. If there was even something remotely defensible in GPTs then OpenAI would not have sold 50% of company for 10B dollar. Only a matter of time and large about of human in the loop will bring any large transformer model into the same space as shown by recent models like alpaca and vicuña are showing that. Only thing the whole thing has done is no labs will open source any major breakthroughs anymore
- hot_gril 3y ago> Product based on no fundamental innovation is also a path to irrelevance Microsoft has been doing, like, negative innovation and is still relevant.
- actionfromafar 3y agoYeah, they innovate the frontier for annoying users and developers. How many APIs can one deprecate and ads can one cram down throats?
- ZephyrBlu 3y agoWhat do you mean? Most VC-backed startups sell far more than 50% of their company for far less than 10B.
- neel8986 3y agoThis is not a typical VC based company. According the HN crowd this is the one company who can execute on AI and challenge Google and all other trillion dollar AI labs. In my opinion they themselves are aware of the fact they are one trick pony. Given how astute a VC SamAltman is, if there was any thing remotely innovative and defensible about the product they would have never done that.
- ZephyrBlu 3y agoI don't understand why they wouldn't have raised that money if "there was any thing remotely innovative and defensible about the product"? What would they have done instead? They still need to fund the operation somehow, and raising money is a good way to build partnerships.
- criley2 3y agoThis feels like someone negging Apple in 2007 "Palm invented the smartphone, what has Apple done" lol
- outside1234 3y agowell in early 2007 that would have been fair feedback :)
- KyeRussell 3y agoEven in ‘07, Apple had a track record for doing things right, not doing things first. Current-day Google churns out sterile, uninspiring products, and kills them. If your argument is “this company is going to act out of character and do something innovative!” then…yeah, sure. That’s a good way to be right, sometimes. Just don’t let everyone see the majority of the time where you’ve been wrong.
- tempest_ 3y agoIn the grand scheme and from the outside looking in being `good at execution` might be the most important thing. There were plenty of touch screen phones before the iphone.
- neel8986 3y agoOutside world is looking at AI innovation only in recent times forgetting the entire journey of last decade. If there was any remotely defensible technology in OpenAI they wouldn't have sold 50% of their company for 10B.
- KyeRussell 3y agoYes, we saw your other comment stating the same thing. You are doing a whole lot of tea leave reading with basically zero visibility, which I can’t really reconcile with how absolute you’re being with your language.
- Keyframe 3y agoThere's definitely something amiss. Maybe we're just not seeing the whole picture, but Google has the best potential out there still. Not only vast and fundamental research came out their door (presumably there's more), but they also have their own compute resources and an up-to-date copy of internet.zip and gmail.zip and youtube.zip which they can train on vs what small and stale stuff (compared to Google's data) OpenAI trained their stuff on (like common crawl etc.). What gives, Google? Get on it! edit: I forgot all about google_maps.zip / waze.gz and all the juicy traffic data coming from android.. which probably already relies heavily on AI
- danans 3y ago> gmail.zip Despite what people often write and believe here, the access controls on PII data at Google are incredibly strict. You can't just arbitrarily train on people's personal data. I know, because when I was there, working on search backend data mining, in order to get access to anonymized search and web logs, I had to sign paperwork that essentially said I'd be taken to the cleaners if I abused the access. > What gives, Google? Get on it It's a very difficult decision to intentionally destabilize the space you are the leader in, for all the reasons you can imagine. In a sense, Google needed someone else with nothing to lose to shake up the space. How they execute in the new reality is yet to be seen. The biggest challenge they may have right now isn't technological, but that "ChatGPT" has become a sort of brand, like Kleenex and well, Google.
- hbn 3y ago> They are good at execution You're reiterating their point. Yeah, Google has competent AI people but that means nothing for their own success if they can't execute. OpenAI has proven that.
- why_only_15 3y ago7/8 of the transformer authors are gone, BERT author is at OAI, two first authors of T5 are gone, imagen team left to make their own startup, etc. etc.
- uptownfunk 3y agoThis is I think a caricature of how engineers think. Yes yes that’s right the algorithm is the most important part.
- stingraycharles 3y agoI think in this context “execute” implies “create traction with a real-world product”. Given that even politicians and comedy shows are talking about ChatGPT, I think it’s fair to acknowledge that Google is lacking in this area.
- jerpint 3y agoGPT, CLIP, Dalle, RLHF, these are all novel
- qumpis 3y agoRlhf wasn't introduced by openai. And GPT is a pretty standard transformer, no? Yes they did it at scale and it speaks volumes on their production skills, but OP was asking about research
- jerpint 3y agoGPT is a specific instantiation of a transformer, and doing next token prediction was an openAI thing. Transformers were a big part of it, but GPT was def proposed but openAI
- avereveard 3y agoWell, what better demonstration that engineering without business vision is sterile While PaLM demolished benchmarks scores openai with a chat tune of a sizeable but not unwieldy model took the world by storm.
- dragonwriter 3y ago> Yes the team which literally created transformer and almost all the important open research including Bert, T5, imagen, RLHF, ViT don’t have the ability to execute on AI /s. This, but non-sarcastically. Google has spectacularly, so far, failed to execute on products (even of the “selling shovels” kind, much less end-user products) for generative AI, despite both having lots of consumer products to which it is naturally adaptable and a lot of the fundamental research work in generative AI. The best explanation is that they actually are, institutionally and structurally, bad at execution in this domain, because they have all the pieces and incentives that rule out most of the other potential explanations for that. > OpenAI bought into the field. They are good at execution but i havent seen anything novel coming out of them. Right, OpenAI is good at execution (at least, when it comes to selling-shovels tools, I don’t see a lot of evidence beyond that yet), whereas Google is, to all current evidence, not good at execution in this space.
- tomComb 3y agoI’m puzzled that stuff like alpha Fold count for nothing in this discussion (having just browsed through most of it). I saw quotes from independent scientists referring to it as the greatest breakthrough of their lifetime, and I saw similarly strong language used in regard to the potential for good of alpha fold as a product. So they gave it away, but it is still a product they followed through on and continue to. Was it wrong of them that they gave it away, and right, that Microsoft’s primary intent with their open AI technology, seems to be to provoke an arms race with google?
- djtango 3y agoWho cares about protein folding when a hyped up ELIZA can confidently tell you lies /s
- ramraj07 3y agoAlpha Fold is a game changer, but nowhere near the game changer ChatGPT(4) is, even if ChatGPT was only available for the subset of scientists that benefit from Alpha Fold. We are literally arguing semantics if this is AGI, and you're comparing it to a bespoke ML model that solves a highly specific domain problem (as unsolvable and impressive as it was).
- HyprMusic 3y agoProximal Policy Optimization
- deleted 3y ago[deleted]
- kiratp 3y ago> They are good at execution but i havent seen anything novel coming out of them. Execution is 9/10 of the battle.
- sangnoir 3y agoI suspect recency-bias may be tripping people up: LLMs and ChatGPT are not the final word in AI, and there is no reason for Google to bet the farm on them. I wouldn't bet against Google DeepMind originating the next big thing, at the very least, their odds are higher than OpenAIs. Edit: this may yet turn out to be a Google+ moment, where an upstart spooks Google into thinking it is fighting an existential battle but winds up okay after some major missteps that take years to fix (YouTube comments as a real-name social network. Yuck)
- eigenvalue 3y agoYou could say the same about Xerox in the late 70s. And they conclusively showed that they couldn’t execute and squandered all of their amazing original research. Looking at how laughably bad Bard is, Google has a long way to prove they aren’t Xerox 2.0 at this point. I’m amazed that Sundar hasn’t been pushed out yet by Larry and Sergey.
- gman83 3y agoThis thread is full of people saying that what Xerox did was some terrible mistake, but I think that it was much better that they could afford to do all this research which spawned a massive industry as a result than had they become this massive monopoly which controlled everything. If Google spends billions of it's ad money doing original research that spawns a new industry with thousands of companies, that would seem to be a great result to me.
- eigenvalue 3y agoThat might be true on a societal level, but is small solace to XRX shareholders, not to mention the many researchers who contributed these brilliant creations only to see them exploited by others while their own company just ignored them and let them die on the vine.
- moyix 3y agoRLHF is from Google? The reference I know of is OpenAI: https://arxiv.org/pdf/2009.01325.pdf https://arxiv.org/pdf/2009.01325.pdf CLIP also seems novel?
- neel8986 3y agoOriginal idea for the paper coming from deepmind https://proceedings.neurips.cc/paper_files/paper/2017/file/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf https://proceedings.neurips.cc/paper_files/paper/2017/file/d...
- moyix 3y agoSeems marginal – half of those authors, including the lead author, are from OpenAI!
- hot_gril 3y agoI would go on about how much execution matters, but it's not just about execution, cause ChatGPT is actually a better AI than anything Google has put out so far. So unless Google is hiding something amazing...
- jpeg_hero 3y agoHave you tried Bard?
- hungryforcodes 3y agoAnd yet no one is talking about Google AI. None of it is a house hold name. So OpenAI it is.
- querez 3y agoDeepMind clearly is a household name. Think of AlphaGo or AlphaFold, those were legendary. Google Brain as well is a household name. Think of the Transformer, or BERT. Those are legendary, as well.
- iknowstuff 3y agoI'm sorry, but no. DeepMind might be well known to AI nerds, but it's not a household name the same way ChatGPT/OpenAI has become.
- cdash 3y agoNo one in the public knows what those are but most people have heard of ChatGPT at this point.
- pigscantfly 3y ago95% of those people have left Google because the ethics and safety teams prevented them from releasing any products based on their research. We have those ex-Googlers to thank for ChatGPT, Character.ai, Inceptive, ... which you'll notice are not Google products but rather competitors.
- panarky 3y agoI, for one, appreciate a megacorp purposely sacrificing revenue when they're not confident that the negative externalities of that revenue would be minimized. Google could have built a search engine where paid results were indistinguishable from organic results, but the negative externalities of that were too great. Google could have remained in China, but the negative externalities of developing and managing a censorship engine were too great. Google could have productized AI before the risks were controlled, but they sacrificed revenue and first-mover advantage to be more responsible, and to protect their reputation. This behavior is so rare, it's hard to think of another megacorp that would do that. Google's far from perfect, they've made ethical lapses, which their competitors love to yell and scream about, but their competitors wouldn't hold up well under the same scrutiny.
- hn_throwaway_99 3y ago> Google could have built a search engine where paid results were indistinguishable from organic results, but the negative externalities of that were too great. Have you not used Google search in the past 5 years?
- panarky 3y agoWhen it says "sponsored" in bold text, prominently displayed right at the top of the result, you know it's an ad. Before Google, search engines didn't do this. Paid results were indistinguishable from organic results. Here's an example --> https://imgur.com/a/bSJTBeD https://imgur.com/a/bSJTBeD If you have a counter-example, please share!
- 3y ago
- deleted 3y ago[deleted]
- ianbutler 3y agoBeing able to produce research is a very different skill from being able to produce a very successful product. We have not seen google do that very successfully for over a decade.
- Takennickname 3y agoAre you implying that inventing something is the same as being able to bring it to market?
- crakenzak 3y agoEvery single member of the research team that invented the transformer architecture have left Google to go to OpenAI, or make their own startups (character.ai, anthropic, cohere)
- simonster 3y agoNope, Llion Jones is still at Google.
- kortilla 3y agoProducing research is not executing for a company. Executing is later in the pipeline and google is failing. Reread the comment you are replying to. It explicitly said that the research is good.
- yttribium 3y ago"Sun Microsystems literally invented Java and has done a ton of open research on RISC, how are they not able to execute as those technologies are exploding"
- gorgoiler 3y agoThey are good at execution… It might help to reflect on what the upsides of this have been for OpenAI, re execution. On the face of it, execution is often all that matters. FB v myspace, AMD v Intel (eventually), Uber v Lyft, MS v Apple (pre 2001), Apple v MS (post 2001) etc.
- hintymad 3y ago> Yes the team which literally created transformer and almost all the important open research including Bert, T5, imagen, RLHF, ViT don't have the ability to execute on AI /s Yet Google does not have a slam-dunk product despite so many great research results. This looks a gross failure of the CEO, especially given that he's been chanting AI First in the past few years.
- WanderPanda 3y agoPeople at OpenAI came up with PPO, arguably the most used deep RL algorithm
- richardw 3y agoThat sounds like Google is the the Xerox Parc of AI. Still need to execute.
- candiodari 3y agoThe original transformer team very much has executed on making successful implementations of transformers ... just not for Google. Clearly something went a bit wrong at Google brain in 2017. https://www.linkedin.com/in/ashish-vaswani-99892181/ https://www.linkedin.com/in/ashish-vaswani-99892181/ https://www.linkedin.com/in/noam-shazeer-3b27288/ https://www.linkedin.com/in/noam-shazeer-3b27288/ https://www.linkedin.com/in/nikiparmar/ https://www.linkedin.com/in/nikiparmar/ https://www.linkedin.com/in/jakob-uszkoreit-b238b51/ https://www.linkedin.com/in/jakob-uszkoreit-b238b51/ https://www.linkedin.com/in/aidangomez/ https://www.linkedin.com/in/aidangomez/ https://www.linkedin.com/in/lukaszkaiser/ https://www.linkedin.com/in/lukaszkaiser/ https://www.linkedin.com/in/illia-polosukhin-77b6538/ https://www.linkedin.com/in/illia-polosukhin-77b6538/ Only one remains at Google: https://www.linkedin.com/in/llion-jones-9ab3064b https://www.linkedin.com/in/llion-jones-9ab3064b
- magicalist 3y ago> The original transformer team very much has executed on making successful implementations of transformers ... just not for Google ah yes, stealth startup, my favorite successful product :P
- galactus 3y agoBecause of course Xerox PARC, which literally invented the GUI, desktop computer, the mouse, freaking Ethernet, etc executed the commercialization of all their innovation flawlessly....
- zamnos 3y agoWell, I've heard a lot about a ChatGPT thing in the news, isn't that made by OpenAI?
- chillfox 3y agoI thought that a significant amount of the people who created those at Google are now working at Anthropic, OpenAI and other startups.
- letitgo12345 3y agoPPO for RL. Plus a lot of the people behind the innovations you mention are now at OAI - Lukas who was one of the Transformer authors, Bert paper author, chain of thought prompting author, etc. But ya their strong point is execution and doing the hundreds of little things that make the model do well and it turns out that that's more important than "novel" ideas
- brookst 3y agoYou think the researchers who created transformers are going to become a commercial product team and be good at execution? Google is great at research, one of the best companies in the world. They are also not very good at product. It will not be possible for Google to research their way out of the business problems they’re facing. They may win, but if so it will be because they get good at product, not because the transformers research team comes up with something even more amazing.