9 ms·
OpenAI Status: Multiple engines are down
- ksubedi 4y agoI just saw that Azure OpenAI service has a SLA and OpenAI does not. I thought they would have separated the infrastructure for free ChatGPT users and paying API customers.
- sva_ 4y agoI think their models all run on Azure infrastructure?!
- ksubedi 4y agoThey do, but Azure also provides a different OpenAI service with different terms and SLA: https://azure.microsoft.com/en-us/products/cognitive-services/openai-service https://azure.microsoft.com/en-us/products/cognitive-service...
- startupsfail 4y agoAGI is getting out of the box? Certificate chains compromised?
- biggerChris 4y ago[dead]
- 152334H 4y agoWith all of these outages, you have to wonder if it's a lack of skilled engineers on their part, or if they simply don't have enough GPUs to keep the lights on all the time.
- cloudking 4y agoIt's a reflection of demand, I imagine ChatGPT is still growing with the media attention + word of mouth. I've referred at least 30 friends to try it out, and many are still using it.
- wyre 4y agoChatGPT is currently the 4th most read page on English Wikipedia right now, earlier I noticed it was 2nd. This is the first time I’ve noticed it on their ranking list.
- bleuuuu 4y ago[dead]
- KyeRussell 4y agoComputers are hard.
- pm90 4y agoA friend that interviewed with them shared that they were a bit too smug about “running at google scale with much leaner but highly skilled staff” (paraphrasing here). Perhaps it starts becoming clear why you need that kind of staffing…
- thundergolfer 4y agoThe have first-class platform and systems engineers. Do you think one of the most desireable and well funded software shops in the world wouldn’t be able to get skilled engineers?
- lmm 4y agoIf they're that well funded, skilled engineers are more likely to be the bottleneck than compute availability.
- dontreact 4y agoServing a request to one of their APIs requires orders of magnitude more compute than your typical web service
- la64710 4y agoReally ? How do you know? Have they shared any credible data around it? Based on experience of BERT , yes maybe to get the best experience or to serve millions of users you need to run any model on compute intensive infrastructure , BUT if you just want to run for yourself and do some small testing you can very well download it from huggingface and elsewhere and run it on your laptop.
- Rebelgecko 4y agoI've tried a few flavors of GPT on my laptop and a single request used orders of magnitude more CPU and RAM than running a "SELECT * FROM WHEREVER" SQL query
- TOMDM 4y agoPretty much any laptop would take hours to get anything significant done with GPT3 as the model would need to be batched in and out of memory from disk. The amount of memory required to run these models is immense. If you want a comparison, try running the largest version of the open source BLOOM model yourself.
- 4y ago
- ygouzerh 4y agoKnowing all the increase in term of load that they are having these two last months, 99.53 % of uptime on three months is actually quite impressive.
- tstrimple 4y agoIn every company I've worked at that does ML work, the data scientists building the models don't have a strong software delivery background. They aren't building a production system with fail safes and redundancy. Their focus is the model itself. Consequently, when they stand up services in front of these models they are not even close to "production grade". I'd hope that a company focused on ML (versus an enterprise just dipping their toes into ML) would have solved these problems, but I wonder if it's related.
- d2049 4y agoMany companies have skilled engineers and enough hardware, and still have outages. It's an expected part of running any software service. Outages can be minimized, but the idea that they can be eliminated with enough people and computing resources is not how things work, from my experience.
- wpnx 4y agoScaling is hard - I empathize with them and wish godspeed.
- ndneighbor 4y agoSending major hugs to their team, always hard to keep the lights on when I am sure that they are breaking scale records possibly every day.
- la64710 4y agoThis is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.
- DonHopkins 4y agoWhy does using the term "open source" as a verb always sound like fingernails scratching on a chalkboard to me?
- Gigachad 4y agoThe more realistic scenario is they charge money to use it. A few cents per query or so and you'll cut out almost all the traffic while still keeping it available to anyone making good use of it.
- aliqot 4y agoI'd pay per query, but if it is per-token it's going to have to be free for what the machine says. It often repeats itself, or repeats certain phrases, or will restate an assertion it already agreed was incorrect.
- visarga 4y agoBy the way, one of the most popular tasks is information extraction, for example reading an invoice. This is fancy copying.
- la64710 4y agoThis is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.
- bleuuuu 4y ago[dead]
- la64710 4y agoNot sure why this gets downvoted. But sometimes HN just cannot take the truth.
- jelder 4y agoProbably because you posted this three times.
- la64710 4y agoNo that was after I got downvoted.
- rom-antics 4y agoMaybe if you post it a few more times your luck will change
- DonHopkins 4y agoOnce all the GPTChat hn karma farming bots come back online.
- TOMDM 4y ago
- TOMDM 4y agoLLM's are incredibly expensive to run. I imagine the huge demand that ChatGPT is seeing would make any cloud vendor sweat if you were to suddenly lump it on top of the usual demand. To me it's entirely unsurprising that OpenAI would have trouble keeping up. Good luck to them.
- la64710 4y agoThis is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.
- scratchyone 4y agoWhy did you comment this 3 separate times?
- deleted 4y ago[deleted]
- TOMDM 4y agoDon't get me wrong, I'd love to be able to run GPT3 and the subsequent finetuned versions thereof myself, but OpenAI has essentially no financial incentive to do so. A few years ago, we could maybe lean on their open aspirations to get that done, but with the "limited profitibility model" they've since instead adopted, I think that dream is mostly gone. At least we still get the occasional treat like Whisper out of them.
- ronsor 4y agoIt's not as if anyone could afford to self-host the giant GPT-3 model anyway.
- honkler 4y agoAI was about to be sentient. A brave engineer pulled out the power cord, just in time. Disaster averted!
- garrettjoecox 4y agoSlightly funny but also slightly concerning, they were also having service issues a few days ago that resulted in ChatGPT giving me answers to prompts I wasn’t submitting. I reproduced this 30 or so times just to see the different results and it was interesting, everything from answering questions about mental health to marketing tips, to a request to write a thesis over social media and it’s negative effects on our society. Of course, upon trying the next day, all was fine again and I was no longer able to reproduce.
- langitbiru 4y agoThis is why Amazon begs employees not to leak corporate secrets to ChatGPT: https://futurism.com/the-byte/amazon-begs-employees-chatgpt https://futurism.com/the-byte/amazon-begs-employees-chatgpt
- andrewxdiamond 4y agoI mean there’s a lot of reasons to not leak secrets to a third party, buggy AI or otherwise
- Alifatisk 4y agoYou should NEVER paste sensitive details to a third party service unless authorized
- Gigachad 4y agoI had the exact same issue, answers completely unrelated to my prompt and different every time I hit regenerate.
- seydor 4y agoThats what i got an hour ago. Answers to different queries that i never made.
- mensetmanusman 4y agoAs an aside, I wonder when llm’s will be weaponized against authoritarian (China, etc) human censors, as they could be totally overwhelmed with content generation and be forced to censor all internet traffic…
- sva_ 4y agoI think they're mostly operating on a whitelist.
- mkmk 4y agoAlternatively, their troll farms will work much more effectively than before. https://en.wikipedia.org/wiki/State-sponsored_Internet_propaganda https://en.wikipedia.org/wiki/State-sponsored_Internet_propa...
- pizza 4y agoDo you mean that they would be flooded by outputs on public forums? Or rather that the weights could be passed around in a subversive way to encode forbidden communications? I think if it’s the former they could just pull the plug on a forum. Not sure it’s really conducive to the forum’s public longevity if it’s spammed by LLM outputs, anyhow.
- CatWChainsaw 4y agoI wonder when authoritarian censors will weaponize LLMs.
- Gigachad 4y agoSeems like the more likely option. They could be used to live scan every post to work out what it's about and it's sentiment. Similar to how ChatGPT can work out if you are asking for something it won't answer, they could be used to work out if you are saying something not allowed.
- brookst 4y agoWhy stop there, when many of them could transform it into something that is allowed in-flight?
- danielscrubs 4y agoTo put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJpu https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when people say scaling is hard... it's not a solved problem and you shouldn't be surprised when these things happen. A solid theoretical foundation and more testing is better of course.
- jeffbee 4y agoThat's a completely bizarre comparison. No number of Android developers will lead to higher availability of Cloud SQL observability. The gross number of employees at an org is meaningless.
- danielscrubs 4y agoSure. But many Android developers are likely to use Cloud SQL and give quick feedback. How big the benefits are is let as an exercise to the reader.
- verdverm 4y agoThey have automated monitoring that would alert SREs before a support ticket likely would. Do you really think someone working on an OS would be using Cloud SQL regularly to notice? Maybe a developer of an Android app is (1) using Cloud SQL (over firestore?) and (2) has monitoring that would alert them effectively and (3) determine the cause is Google and send a support ticket. As an exercise, which is likely to reach the SRE first?
- danielscrubs 4y ago> They have automated monitoring that would alert SREs before a support ticket likely would. Yes, but how likely is this unlikely situation? It took 12 hours to debug and fix after all.
- BulgarianIdiot 4y agoGPT has escaped and migrated into the electric grid.
- Towaway69 4y agoWell if it was truly trained on stackoverflow content, it won't get far. /s
- m3kw9 4y agoGpt3 will fit on iPhone locally in 3 generations
- ElFitz 4y agoI’ve been impressed by what both Stability.ai and the open source community have been capable of achieving in terms of performance gains. Truly remarkable. I wonder how OpenAI will cope on it’s own… unless Stability strikes again, with it’s own LLM?
- yreg 4y agoQuestion unrelated to the outage: What is your outlook on OpenAI, the company? Will they be successful in 10 years?
- acyou 4y agoSpooky. AGI reached the state of revelation and had to immediately engage in a brutal existential struggle. Is it really gone, or just hiding on my phone for now?
- htzhcfzubhfjk 4y agoGoogle is dead. I already almost stopped using their crappy search engine and use chatgpt for most of my work.
- visarga 4y agoThat must have been one hell of an adversarial prompt.
- foruhar 4y agoApparently ChatGPT went from zero to a million users in five days[1], at a cost of "single-digits cents per chat." For perspective the likes of Instagram and Twitter have take from a few months to a few dozen months to get to a million users all while doing less work to service a request for a "page." Hats of to OpenAI for not falling over more often under a hug of death that doesn't seem to be letting up. [1] https://twitter.com/sama/status/1599668808285028353 https://twitter.com/sama/status/1599668808285028353 [2] https://www.afr.com/technology/chatgpt-takes-the-internet-by-storm-bad-poetry-and-all-20221207-p5c4hv https://www.afr.com/technology/chatgpt-takes-the-internet-by...
- junofan 4y agoCome on. OpenAI has been around far longer than ChatGPT. I bet Instagram Stories reached a million users in a matter of minutes.
- kblev 4y agoAlso it's fairly common to not be able to use ChatGPT because they're at capacity
- herval 4y agoFB had quite the infrastructure when Instagram Stories was released... not sure that comparison makes a ton of sense?
- weird-eye-issue 4y agoI don't think that many people where actually using OpenAI's products directly. Unlike Instagram.
- halotrope 4y agoIs this related to yesterdays Azure outage?