7 ms·
I don't think most people realize how much infrastructure separates something like ChatGPT from Google-scale deployment. OpenAI isn't suddenly building transoce
by killjoywashere 4y ago
I don't think most people realize how much infrastructure separates something like ChatGPT from Google-scale deployment. OpenAI isn't suddenly building transoceanic fiber and datacenters near most major population centers. They aren't signing production-ready contract vehicles with most major OEMs and governments. And in the gap it would take a new entrant to acquire 10% of those assets, Google has 100,000 engineers who would iterate on this and 1000 other technologies 1000 times.
- kanwisher 4y agoOpenAI already has the infrastructure Azure and Bing. Microsoft has a bigger public cloud then google
- kennethh 4y agoThe public cloud is probably a small part of the Google Infrastructure.
- dougmwne 4y agoYou are missing the Microsoft partnership. MS has a 48% stake in OpenAI and provides all the infrastructure through Azure, including purpose built machines for model training. Microsoft has also launched GitHub Copilot, summarization features in Teams, and is widely reported to be adding GPT features to Bing.
- scarmig 4y agoGoogle has far more resources for training models and inference. Likely more than all their nearest competitors combined.
- redox99 4y agoSource?
- dougmwne 4y agoThere’s definitely no legitimate source for that info. Details of the exact machines that sit in the Azure and Google clouds are proprietary.
- yunwal 4y agoIf you’re talking about physical hardware, Google pales in comparison to Amazon, and Gcloud is still smaller than Azure. It’s possible google’s private compute makes up for the Azure difference but it’s not like they’re in different leagues in terms of access to hardware.
- alsodumb 4y agoUnlike Azure and Amazon, Google doesn't have to rely on Nvidia GPUs for training and inference, they achieved significant performance gains by using their custom TPUs.
- coder543 4y agoYour comment is simply incorrect. Amazon has had their own TPU equivalents for training and inference for years: https://aws.amazon.com/machine-learning/trainium/ https://aws.amazon.com/machine-learning/trainium/ https://aws.amazon.com/machine-learning/inferentia/ https://aws.amazon.com/machine-learning/inferentia/ I really don't think this would be a limiting factor regardless, even if Amazon didn't already have multiple generations of these products. It's not as if an Amazon or Microsoft sized company is incapable of developing custom silicon to meet an objective, once an objective is identified. TPUs also aren't really that complicated to design, at least compared to GPUs. I'm slightly surprised Microsoft hasn't bothered to release any custom ML chips for Azure yet, but I guess they've run the numbers and decided to focus on other priorities for now.
- scarmig 4y ago> google’s private compute
- the-rc 4y agoDoes it pale? How? AWS is very opaque when it comes to power demand, but you could use that to estimate machine counts and they admitted to Greenpeace that in 2016 they used 500-650MW — a wide range that obviously obfuscates on purpose. See the last Clicking Clean report from 2017. Google/Alphabet used 6514 terawatt hours in 2016 according to data collected from yearly reports at https://www.statista.com/statistics/788540/energy-consumption-of-google/ https://www.statista.com/statistics/788540/energy-consumptio... If my math is right, dividing that by 8760 hours in a year, you get 743MW. Of course, that also includes office space, etc. (did the AWS number, too?) but it should be clear that, cross checking with data center builds, optic fiber and energy purchases as well, for years Google+YT+Apps+GCP were larger than Amazon and all other AWS customers combined. I didn't even factor in efficiency, something that Amazon started focusing on quite a bit later. Someone might be able to extrapolate both numbers to today based on infrastructure, other metrics or other spend in quarterly financial statements (or power procurement, which will be complicated by the non trivial Amazon vs AWS distinction). All of the above to say that Amazon probably has more compute now, but it's a stretch to talk about "paling".
- paganel 4y agoMS cannot deliver successful stuff on the web at scale. They can deliver unsuccessful stuff on the web at scale, they can deliver successful stuff that turns out to be inconsequential for their bottom line on the web in the long-term (AJAX came from MS), but it's just not in their DNA to take over the web. They had lots of chances to do it during the last 20 years or so, they had all the silver bullets at their disposal, they just couldn't deliver what it took.
- moufestaphio 4y agoUh sorry what? Azure is the second largest cloud provider, well ahead of google.. That's like the definition of 'web scale' Not sure what decade you're stuck in here with comments about ajax
- sidibe 4y agoNot agreeing with GP but the capacity of their public clouds is very different from total capacity. Amazon and Microsoft have bigger public clouds, but Google's own workload is probably bigger than anyone else's or even the public clouds by a large factor.
- paganel 4y ago> Azure is the second largest cloud provider, well ahead of google. I explicitly mentioned the "web", as in, what we're doing right now on this website. Leaving aside the fact that Azure is mostly used big corporate/government entities, there's no web-startup that will potentially dominate the web and that would go into Azure just as.
- rezonant 4y agoWhat makes you think anything being discussed in this thread is specific to HTML and web browsers?
- ctvo 4y ago> MS cannot deliver successful stuff on the web at scale. It's this type of confidence that you come back to HN over and over for. Let's ignore Azure, Office 365, and Microsoft's other online properties. In just video games: Microsoft owns and operates the Xbox network with over 100 million active monthly users[1]. That runs on the internet with all the features of a social network and more. I think they can deliver successful stuff at web scale OK, whatever their other shortcomings. 1 - https://hothardware.com/news/xbox-live-surpasses-100-million-active-monthly-users-gamepass https://hothardware.com/news/xbox-live-surpasses-100-million...
- mountainriver 4y agoYeah they also trash up the brand. Seeing ChatGPT be thrown into Teams and Bing let’s me know where it’s headed
- junofan 4y agoBased on the busy mock-up in TFA I don’t think it’ll take 100,000 engineers to beat them.
- AJ007 4y agoAs far as I've read, DuckDuckGo doesn't have 100,000 engineers.
- doctor_eval 4y agoNo but they don’t have a search engine either. They mostly use bing.
- relaxing 4y agoThey’re leveraging Microsoft’s engineering.
- Willish42 4y agoWhat is TFA in this context? Does anyone have a link to to said "busy mock-up"?
- fredoliveira 4y agoTFA would be "the f*cking article", in this case.
- rezonant 4y agoFred's right, it's "the f'ing article", as in RTFA (read the f'ing article) popularized afaik on Slashdot in it's golden age, which came from RTFM (read the f'ing manual), a popular response to a question that can be easily answered by reading the appropriate man(1) page or other such reference material.
- mritchie712 4y agoAs someone using OpenAI in production, I can attest to the lack of stability and consistent performance in the current offering. Depending on the time of the day (and who knows what else), the same calls to GPT3 can take from 500ms to 15 seconds.
- senthilnayagam 4y agochatgpt launched about 2 months back and have over 100 million users, they have secured funding from microsoft so they are going to optimise and scale its infrastructure. go for their paid offering if latency is bothering you
- alvah 4y agoParent is clearly making API calls to GPT3, which isn't really comparable to using ChatGPT (paid or unpaid) via a browser.
- mritchie712 4y agoCorrect.
- HarHarVeryFunny 4y agoYou don't need to own your own datacenters to do this, anymore than someone like Netflix (who run on Amazon cloud) does. I'm sure any of the cloud providers would happily take the money of any company that had the $$$s to pay for it. The barriers to entry really don't seem that great... the tech behind ChatCPT is well understood and multiple similar things already exist from various companies.
- coredog64 4y agoAIUI, Netflix runs a lot of their business in AWS, but content streaming isn’t something they host in AWS.
- rezonant 4y agoWell they have those "last mile" boxes which they give out to ISPs to install close to the end users to improve performance (and reduce overall bandwidth needs), is this what you mean? Would love to read more about this if you have some links.
- acdha 4y agoIt’s called Open Connect if you want to search for it: https://openconnect.netflix.com/ https://openconnect.netflix.com/ Some people who’ve worked on it post around here and they’ve funded things like FreeBSD development which is interesting for seeing the kind of problems you have at that kind of traffic volume.
- rezonant 4y agoRight, I meant other than OpenConnect. As far as I know the rest is AWS, and the OpenConnect boxes need to get their content from somewhere :-)
- doctor_eval 4y agoMaybe it’s changed now, but last time I looked, Google had failed to materially diversify its income, so it is susceptible to competition. All that expensive infrastructure is going to be a weight around Google’s neck if their search revenues start falling. Of course, even if search revenue falls, it won’t happen overnight. But I honestly don’t see how laying all that fibre or owning those data centres is a moat around Google. These things are hugely capital intensive, to be sure, but theres a big market for both.
- deleted 4y ago[deleted]
- Eduard 4y ago> I don't think most people realize how much infrastructure separates something like ChatGPT from Google-scale deployment. OpenAI isn't suddenly building transoceanic fiber and datacenters near most major population centers. This feels like "thinking inside the box" to me. None of these things are necessary requirements for being a "Google killer".
- Mystery-Machine 4y agoGoogle cannot make a good SMS app for Android. They will never be able to launch a simple chat bot. What they will launch will be a monstrosity. The only Google AI product that people will use is any AI embedded within their existing apps and services...which is a great thing, but I wouldn't bet on Google being able to launch any new product that would have a decent UX.
- deleted 4y ago[deleted]
- tarsinge 4y agoI’m sure people were saying the same things regarding IBM in the 80’s, Microsoft or Cisco in the 90’s… Google is a big corporation with inertia and averse to risk regarding its core business, they are not immune to being disrupted.
- literalAardvark 4y agoAnd they literally published the transformer paper back in 2017. I doubt that their next move was to go on vacation for 6 years.