8 ms·
Bing Chat is so hungry for GPUs, Microsoft will rent them from Oracle
- ykonstant 3y agoI mean, making deals with Oracle seems like a great idea!
- Arrath 3y agoSurely even Oracle can't screw up plugging a bunch of GPUs into a rack, right? Right?
- resolutebat 3y agoPlugging GPUs into a rack is not the problem, renewing your license for them next year will be.
- coldbrewed 3y agoOracle's market capitalization is $298 billion, while Microsoft's market capitalization is $2.6 trillion. If there's anyone that can stave off Oracle's lawyers Microsoft is probably one of a handful that can. Also, I'm guessing that Oracle is sufficiently desperate to wring any revenue out of their investment in cloud hardware so this might be a rare occasion where Oracle will just take the money and call it day.
- roschdal 3y ago"In Japan, we believe our competitors are stealing the rice out of the mouths of our children. In Japan, we think anything less than 100 percent market share is not enough. In Japan, we believe it is--and this is a quote that supposedly came from Genghis Khan, but I have heard it attributed La Rochefoucauld and other people--it is not sufficient that I succeed; everyone else must fail. We believe that this is not sufficient. We must destroy our competition. Larry Ellison quote.
- esafak 3y agoMore precisely, it is Larry Ellison quoting a Japanese business executive. Otherwise he would not have begun the sentence with "In Japan, we..." Ellison's opinion is given a moment later: "And we certainly believe that that is absolutely correct, that we have to pursuit market share. Today we have more share than all of our competitors combined. And our primary goal is to improve our share, our market share." https://americanhistory.si.edu/comphist/le1.html https://americanhistory.si.edu/comphist/le1.html
- deleted 3y ago[deleted]
- mike_hearn 3y agoYou said Ellison's opinion is given a moment later but that part you quote isn't about the Japanese executive's story. It's about something said by Jack Welch: Jack Welch over at General Electric: "If you're not one or two in the market, you don't make money. The market leaders make the money." You know, you have to have share. You have to have substantial market share in order to be profitable in most businesses. The Japanese recognize this very, very clearly. And not just the Japanese. The great business leaders in the United States are the same, whether it is the old guard--and Jack Welch forgive me for calling you the old guard--or the young guard like Bill Gates. And we certainly believe that that is absolutely correct, that we have to pursuit market share. Today we have more share than all of our competitors combined. And our primary goal is to improve our share, our market share. Still at least you provided a source. The actual full quote is Ellison saying that he was told this story by a Japanese business executive, and when he repeated this story to a New York Times journalist they took it totally out of context and made it sound like it was him saying it. He even says he's got in trouble for repeating this story before, but it's OK here because it's on video and therefore hard to take out of context! But on HN that's no obstacle apparently. DM: Going back to Japanese culture, one of the quintessential elements of the Japanese approach to competition has been the aggressive pursuit of market share. Was your aggressive pursuit of market share early on something you learned, or was it just the natural outcome of heads-up business? LE: No. I think it was absolutely our primary goal. I was always influenced by Japanese business people. In fact, it was a very famous story that I've gotten in a lot of trouble for, and I'm very hesitant to mention here, but I will, because at least it's on video and it's harder to take things on video out of context. But I was in Japan and I was talking to a Japanese business executive. And he told me that the problem with America is that we just have no stomach for competition. And I was defending America. I was defending my country and my culture. I said, "What do you mean? America, at the turn of the century was the leading edge I mean, we were the laboratory, you know, for free market economics. The industrial revolution reached its zenith in the Eastern States of North America. What do you mean?" And he says to me, "Don't tell me about things that happened almost 100 years ago. Let's talk about today. You listen to your business executives and they say things like, Well, we have great respect for our competition; the market is very large; there is room for all of us to compete." "In Japan, you know, that would be sacrilege," he said. "In Japan, we believe our competitors are stealing the rice out of the mouths of our children. In Japan, we think anything less than 100 percent market share is not enough. In Japan, we believe it is--and this is a quote that supposedly came from Genghis Khan, but I have heard it attributed La Rochefoucauld and other people--it is not sufficient that I succeed; everyone else must fail. We believe that this is not sufficient. We must destroy our competition. And I remember I came back with that story and told it to people at Oracle. And I remember telling a New York Times reporter this story. And the New York Times reporter had a huge picture of me, took the whole thing out of context. It said, Larry Ellison, quote, "It's not sufficient that I succeed; everyone else must fail." With a huge picture. Then I started getting hate mail. (Laughter) I mean, this story stuck with me for a long time.
- cavisne 3y agoI think this has to be something to do with Oracles Mellanox holdings, they must be getting a better price or have a license that means they can deploy GPU clusters cheaper. Otherwise why would nvidia give an allocation to oracle over Microsoft.
- wlesieutre 3y agoOracle’s looking at the 16,896 CUDA cores on an H100 and thinking “is there any way we could sell software licenses with these?”
- lloydatkinson 3y agoThe GPU situation must be dire for this to happen
- bigyikes 3y agoUnless you’re Jensen Huang. The man can certainly afford a few more leather jackets now. I hope all this AI money sloshing around spurs other players to make compelling GPUs.
- latchkey 3y ago> spurs other players to make compelling GPUs AMD MI300
- CamperBob2 3y agoI don't think that's possible. Very few fabs can manufacture these chips, and they are either several years away from completion, or fully booked for about that long.
- latchkey 3y agoIt isn't just GPUs, it is the whole supply chain. Data center companies are having their best years ever and that trickles down the food chain. It is very hard to source the power required now.
- moargpus 3y agoOracle's GPU setup is pretty good - good availability, interconnects, and support. It's the primary selling point of OCI these days and they do it well - unlike the other clouds, they are focused on making NVIDIA chips work well instead of creating their own competing chip. The rest of OCI is a step down, but it's mostly good enough
- bllchmbrs 3y agothat's why I built: https://gpumonger.com/ https://gpumonger.com/ You can see all alternative GPU providers: https://gpumonger.com/cloud-gpu-providers https://gpumonger.com/cloud-gpu-providers as well as the main ones. Oracle has great GPU prices: https://gpumonger.com/cloud-gpu-provider/oracle-cloud https://gpumonger.com/cloud-gpu-provider/oracle-cloud
- bastawhiz 3y agoThis is interesting, but I suspect the real story is that Bing Chat is lower priority than OpenAI. And it should be! If Bing Chat is based on GPT-N, training GPT-N+1 and building the network effects of selling that technology absolutely should be the first priority. If Bing Chat goes down, someone surely gets paged but nobody is going to be ruined. And frankly, most consumers probably won't notice. At the same time, it's Microsoft's flag that they've planted in search. How much is that worth? Apparently (at least) the cost of GPU-hours from Oracle but less than the cost of holding back those GPUs from OpenAI.
- vouaobrasil 3y ago[flagged]
- tananaev 3y agoI think AI is a progress and will benefit all of us at some point. Why do you think it's a waste of energy?
- vouaobrasil 3y agoAI will be a net harm for us: 1. It will take over many creative pursuits, taking away the feeling of usefulness from many people. 2. It will cause humans to be too independent from each other, and remove the sense of community (technology does this to some extent, only AI enhances the effect much more so) 3. It will concentrate wealth into the hands of tech companies, who relentlessly make life worse by continually making consumerism more efficient and thereby destroying our connection with the natural world 4. It will create disinformation and be a tool of manipulation and crime that will be hard to counteract (such as spoofing, etc.) 5. It will foster a world where there is far too much information, which itself plays on our basal instinct to collect information. This instinct was adaptive in the time of information scarcity, but is now maladaptive And many other reasons....
- gaganyaan 3y agoYou're mostly wrong, but more generally, why don't you give a try to not viewing it through the worst possible lens? What positives do you see?
- OkayPhysicist 3y agoYou're confusing technology and the economic system that technology exists under. Your first three points are symptoms of Capitalism, not AI. Your fifth point is largely moot, since there's already more information available to people than anyone could possibly obtain in their lifetime, and the amount is growing faster than anybody could consume it. The problem can't really get any worse. Your fourth point is more interesting, but I'd argue that the tech is already widely available for any type of crime you'd want to commit using AI. Banning it at this point just makes it so that only the criminals have access to it, and ceasing further progress on AI doesn't really make sense because from here, improvements will benefit legitimate users who want consistent, correct responses from their chatbots far moreso than the criminals who can afford to cast wide nets.
- ericfrazier 3y agoI just emailed Bill Gates that I have a gtx1060 up on Marketplace. It's the 8gb version so I expect to hear back soon.
- deepsquirrelnet 3y agoCan it run Crysis?
- subtra3t 3y agoHa, I messaged him (on skype, of course) about my intel UHD 620. I can already imagine him begging for my 128mb card.
- ClassyJacket 3y agoI'm pretty sure I have a Geforce FX 5200 around here somewhere...
- userinanother 3y agoOpen ai is what’s pumping up the stock market right now. Everyone wants more AI so they can juice their bonuses
- threeseed 3y agoOpenAI is on track to reach $1.3b in revenue this year. Everyone wants to get into AI because end users love it and are willing to pay for it.
- hfjjbf 3y ago[dead]
- userinanother 3y agoI mean yeah 1.3b is great but your revenue is coming from everyone else chasing the hype train. It will be a big viable business but it’s probably not that earth shattering
- jrockway 3y agoIt's always hard to tell. Is this "No Wifi. Less space than a Nomad. Lame.", or is it NFTs? You can never tell until the hype train either crashes or takes over the world.
- deleted 3y ago[deleted]
- umbcorp 3y agoIt is earth shattering. This thing is writing code and can do very very simple decision making in a wide variety of domains out of the box. Possibilities are endless.
- brookst 3y agoI envy the certainty.
- samch 3y agoMy guess is that Microsoft’s own resources have to go into supporting enterprise commitments to M365 Copilot (and other Copilots) and their Azure OpenAI service. Bing Chat is easier to carve out from the enterprise / Azure stack.
- moontear 3y agoThis is the answer. Consumer bing chat isn’t that important for MS. Let someone else host it.
- Manheim 3y agoI suspect you are right. However, we should not underestimate the power Open Ai tech has brought to Bing. Combining the GPT4 capabilities with live search data gives you an incredible strong and useful search service. Personally I have entirely shifted my desktop use from Google search to Edge and Bing Chat Enterprise since I got access to it. If my shift is representative for just a few percentage of the population, it represents a major shift in the search market.
- Citizen_Lame 3y agoDid ChatGPT write this? Anyway, bing is still bad even with the help of not-so-smart AI.
- Seanambers 3y agoI use ChatGPT paid for at least 75% of my queries. Only reason to venture out into search engine land is to find updated information which often is done by adding reddit to the end of the search. I'd say Google is in trouble, I mean there is a limit to how many ads you can pack on the first page of results and I'd say they've passed it a long time ago. In that context Bing + GPT would be more of a winner I think.
- Kye 3y ago>> "Only reason to venture out into search engine land is to find updated information which often is done by adding reddit to the end of the search." That's why most implementations add a layer of Retrieval-augmented generation (RAG): https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-customize-rag.html https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-fo...
- pwarner 3y agoOracle cloud may be better than azure at the lowest levels. My prediction is this is part of why MS will buy Oracle. Just my crazy prediction.
- taspeotis 3y agoSeems a bit antitrust-y? Letting Microsoft own SQL Server and Oracle Database, .NET as well as Java?
- jrockway 3y agoMeanwhile, everyone is just using Go and Postgres for free.
- dralley 3y ago>everyone HN is not a representative sample of industry. Postgresql and Go are, of course, well-used, but Java and MySQL / Oracle are still absolutely massive.
- dns_snek 3y agos/everyone/a minority/ Oracle, Java, C#, MS SQL might not be flashy in tech startups and on HN, but they are absolutely dominant in the wild.
- jrockway 3y agoAll I'm saying is that there are viable alternatives. Oracle and SQL Server being successful doesn't really force anyone into using either of those things. Similarly, sure, .NET and Java are popular enough, but they have alternative runtimes available (remember when Oracle sued Google over that). To me, I don't think the industry landscape changes meaningfully if these tools were to become unavailable.
- taspeotis 3y agoOracle restates financial results: Oracle Cloud has doubled its number of customer
- deleted 3y ago[deleted]
- hfjjbf 3y ago[dead]
- trash_cat 3y ago"According to StatCounter, for October 2023, Bing had a 3.1 percent global web search market share for all platforms – that's compared to Google's 91.6 percent, but up from 3 percent the month before." Bing's market share increased by 0.1 percent? What is the point if mentioning this?
- Sharlin 3y ago*0.1 percentage points. About three percent.
- helsinkiandrew 3y agoInstead of the slow decline that’s happened for years, they increased their usage by 3.3% in a month! If I was on the Bing team, I’d take that as a win. This could be the start of the hockey curve uptick!
- omneity 3y agoThis is 0.1 only. At 10% market share MS will just buy Nvidia. (Or merge)
- mensetmanusman 3y agoI love that LLMs really bring to light the material aspect of hardware.
- jrockway 3y agoProbably a problem fixed by time. We laugh a little when we hear the ENIAC took up an entire room and required an entire staff all for 500 floating-point operations per second. Sure, maybe it's difficult to scale current lithography much further, but someone will probably come up with something.
- john2x 3y ago“Hope is not a strategy”
- brookst 3y ago…but betting that the next ten years will play out like the last 75 is a pretty good strategy.
- tekno45 3y agountil it isn't...
- otabdeveloper4 3y agoNot really. Nothing here is a linear process.
- madhatter999 3y agoUpward doesn’t have to be linear
- algobro 3y agoThe OpenAI dev day product manager had a slide with her calendar. Two slots were marked up in red "Find GPUs" . Sending a message there.
- zelly 3y agoShortages lead to gluts. Get ready.
- krasin 3y ago> Shortages lead to gluts. On commodity markets. GPUs are essentially a monopoly, so gluts are only really possible with outdated hardware. My favorite eBay listing of today is a lot of 100 NVIDIA P106-100 6GB GPUs for ~$20 each: https://www.ebay.com/itm/305023042595 https://www.ebay.com/itm/305023042595
- lhl 3y agoJust in case anyone gets tempted, the P106-100's were mining cards and locked to PCIe 1.1, so the bus bandwidth is terrible. Add to that the limited amount of memory (6GB), low memory bandwidth, Pascal (1/64 speed FP16), and that these were all likely being run in extremely shoddy data centers, it's not even worth the power costs to run IMO. For those looking for the cheapest higher memory solutions, 24GB P40s are available (decent amount of VRAM, 3X the memory bandwidth, but requires server or DIY cooling, same bad FP16) or IMO the best bang/buck for hobbyists/home devs atm, used RTX 3090s are going for about $600-700 each. (Note: if you're doing training, unless you have very high utilization/already calculated your costs, you will probably be much better off renting cloud GPUs from Vast.ai, RunPod, etc)
- krasin 3y ago> P106-100's were mining cards and locked to PCIe 1.1 This is not exactly accurate. P106-100 were not specifically made for mining, unlike NVIDIA CMD Hx cards (https://www.nvidia.com/en-us/cmp/ https://www.nvidia.com/en-us/cmp/). P106-100 cards are NOT artificially throttled, but the specs are still very poor compared to modern cards: https://www.techpowerup.com/gpu-specs/p106-100.c2980 https://www.techpowerup.com/gpu-specs/p106-100.c2980 It is true that they are not worth the power costs. I merely made a point that gluts of outdated hardware is a thing even when NVIDIA is a monopoly.
- 3y ago
- neilv 3y agoIs Microsoft also securing GPUs that could otherwise by used by competitors?
- khana 3y ago[dead]
- ks2048 3y agoWhy don’t more tech giants start making their own hardware?
- seanhunter 3y agoI think the takeaway here is a little different. In a world where everyone is desperate for GPUs, Oracle has spare GPUs to rent out to competitors. What does that say about demand for Oracle’s cloud offering?
- mike_hearn 3y agoApparently the demand is large, as Microsoft is not a small customer by any measure. The actual cause here is something else: NVIDIA have been allocating GPUs according to long term corporate priorities rather than just auctioning them off and letting price signals do the work. The reason is twofold: 1. They want a diversity of customers for GPUs, not just one or two LLM mega-players that everyone else invokes via high level cloud APIs. That would put NVIDIA in a bad negotiating position. 2. They don't want GPU supply to be dominated by companies that are simultaneously designing their own silicon. Microsoft is. [1] Oracle has been getting large allocations because they aren't designing their own AI hardware (or at least, if they are, this isn't public), and because they are in the business of selling CUDA and not hardware abstracted model access. [1] https://www.theverge.com/2023/4/18/23687912/microsoft-athena-ai-chips-nvidia https://www.theverge.com/2023/4/18/23687912/microsoft-athena...