8 ms·

Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. It

Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. It took years before most companies who now use cloud providers to trust and be willing to bet their operations on them. That gave the cloud providers time to make their systems more robust, and to learn how to resolve issues quickly.

benterix 3y ago
The point is, OpenAI spent a lot of money on training on all these copyrighted materials ordinary individuals/companies don't have access to, so replicating their effort would mean that you either 1) spend a ridiculous amount of money, 2) use Library Genesis (and still pay millions for GPU usage). So we have very little choice now. Open Source LLMs might be getting close to ChatGPT3 (opinions vary), but OpenAI is still far ahead.
- btbuildem 3y ago
  Don't underestimate Big Corp's resistance to using OpenAI's hosted solutions (even on Azure) for anything that's not marketing fluff.
  - phillipcarter 3y ago
    You can say that about anything, though. BigCorps aren't exactly known for adopting useful tech on a reasonable timeline, let alone at all. I don't think anyone is under the impression that orgs who refuse to migrate off of Java 5 will be looking at OpenAI for anything.
    - swagempire 3y ago
      No, this is silly reasoning. A middle manager somewhere has no clue what Java 5 is. But he does know -- or let's say IMAGINES what he knows about ChatGPT. And unlike Java 5-- he just needs to use his departmental budget and instantly mandate that his team now use ChatGPT. Whatever that means you can argue it. But ChatGPT is a front line technology and super accessible. Java 5 is super back end and very specialized. The adoption you say won't happen: it will come from the middle -> up.
      - hashtag-til 3y ago
        Honest question: do you really mean Java 5 when you say Java 5? It sounds a bit 2000s to me.
        swagempire 3y ago
        Parent used "Java 5" as an example. Java 5 somehow in my mind is from like the 200x era. But no. I practically mean any complicated back end technology that takes corporations months or years to migrate off of because its quite complicated and requires an intense amount of technical savoir-faire. My point was that ChatGPT bypasses all this and any middle manager can start using it anywhere for a small hit to his departmental budget.
        eddtries 3y ago
        If you care about the security of OpenAI, you care about the EOL of 14 year old Java 5
        cornel_io 3y ago
        In 2016 I worked on a project with a client who still mandated that all code was written to the Java 1.1 language specification - no generics, no enums, no annotations, etc., not to even mention all the stuff that's come since 1.5 (or Java 5, or whatever you want to call it). They had Reasons(tm), which after filtering through the nonsense mostly boiled down to the CTO being curmudgeonly and unwilling to approve replacing a hand-written code transformer that he had personally written back in the stone ages and that he 1) considered core to their product, and 2) considered too risky to replace, because obviously there were no tests covering any of the core systems...sigh. At least they ran it all on a modern JVM. But no, it would not surprise me to find a decent handful of large companies still writing Java 5 code; it would surprise me a bit more to find many still using that JVM, since you can't even get paid support through Oracle anymore, but I'm sure someone out there is doing it. Never underestimate the "don't touch it, you might break it" sentiment at non-tech companies, even big ones with lots of revenue, they routinely understaff their tech departments and the people who built key systems may have retired 20 years ago at this point so it's really risky to do any sort of big system migration. That's why so many lines of COBOL are still running.
  - outside415 3y ago
    chatgpt will have an on prem solution eventually. in the mean time players like NVIDIA are working on that as well.
  - tsunamifury 3y ago
    Marketing fluff is what 90% of tech is... it amazes me how many people think otherwise on hacker news. Unless you are building utility systems that run power plants, at the end of the day -- you're doing marketing fluff or the tools for it.
    - TeMPOraL 3y ago
      > Unless you are building utility systems that run power plants, at the end of the day -- you're doing marketing fluff or the tools for it. Even when you are building utility systems for critical infrastructure, you'll still be dealing with a disheartening amount of focus on marketing fluff and sales trickery.
- colinsane 3y ago
  the choice is to live 2 years behind (e.g. integrate the open source stuff and ride that wave of improvement). for businesses in a competitive space, that’s perhaps untenable. but for individuals and anywhere else where this stuff is just a “nice to have”, that’s really just the long-term sustainable approach. it reminds me of a choice like “do i host my website on a Windows Server, or a Linux box” at a time when both of these things are new.
  - hfjjbf 3y ago
    [dead]
  - BiteCode_dev 3y ago
    2 years behind in terms of timeline, but what factor in terms of productivity and quality of life? Not to mention openai's lead compounds, so 2 years now and 4 years in 2025 may be 10 times the original prod/qol gain.
    - verdverm 3y ago
      The gap seems to be shrinking, not growing. The OSS models have reach new capabilities faster than most thought
  - TeMPOraL 3y ago
    For individuals, this is a very short window of time where we have cheap access to an actually useful, and relatively unshackled SOTA model[0]. This is the rare time individuals can empower themselves, become briefly better at whatever it is they're doing, expand their skills, cut through tedium, let their creativity bloom. It's only a matter of time before many a corporation and startup parcel it all between themselves, enshittify the living shit out of AI, disenfranchise individuals again and sell them as services what they just took away. No, it's exactly the individuals who can't afford to live "2 years behind". Benefits are too great, and worst that can happen is... going back to where one is now. -- [0] - I'm not talking the political bias and using the idea of alignment to give undue weigh to corporate reputation management issues. I'm talking about gutting the functionality to establish revenue channels. Like, imagine ChatGPT telling you it won't help you with your programming question, until you subscribe to Premium Dev Package for $language, or All Seasons Pass for all languages.
- zarzavat 3y ago
  OpenAI is obviously using libgen. Libgen is necessary but not sufficient for a top AI model. I believe that Google's corporate reluctance to use it is what's holding them back.
  - nvm0n2 3y ago
    How do you know? Has someone created a set of test questions that can definitively prove it one way or another?
    - benterix 3y ago
      I don't believe it's even possible to create such a set of test questions. You would need to specifically test for differences between the versions found on Libgen and those that aren't available there. But even then, the evidence would be inconclusive.
- weinzierl 3y ago
  I won't say I disagree because only time can tell, but what you wrote sounds a lot like what people said before open source software took off. All these companies spend so much money on software development and they hire the best people available, how can a bunch of unorganized volunteers ever compete? We saw how they could and I hope we will see the same in AI.
- giancarlostoro 3y ago
  Kind of hilarious that the vast capabilities of an LLM are held back by copyright infringement.
  - joquarky 3y ago
    The original purpose of copyright was to promote progress, and now it seems to hinder it.
  - TeMPOraL 3y ago
    Intellectual property rights will yet turn out to be one of the Great Filters.
  - zozbot234 3y ago
    I'd love to see a language model that was only trained on public domain and openly available content. It would probably be way too little data to give it ChatGPT-like generality, but even a GPT-2 scale model would be interesting.
    - TeMPOraL 3y ago
      If, hypothetically, libraries in the US - including in particular the Library of Congress - were to scan and OCR every book, newspaper and magazine they have with copyright protection already expired, would that be enough? Is there some estimate for the size of such dataset?
      - Turing_Machine 3y ago
        Much of that material is already available at https://archive.org https://archive.org. It might be good enough for some purposes, but limiting it to stuff before 1928 (in the United Sates) isn't going to be very helpful for (e.g.) coding. Maybe if you added github projects with permissive licenses?
jstummbillig 3y ago
On the one hand, sure, new things take time, but they also benefit from all past developments, and thus compounding effects can speed things along drastically. AI infrastructure problem are cloud infrastructure problems. Expecting it to go as if we were back on square one is a bit pessimistic.
Alifatisk 3y ago
> we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. You said it so well!
- j45 3y ago
  It's possible to include API gateways and API Proxies in between calls to normalize them across multiple providers as they become available.
somsak2 3y ago
I don't think it's so dire. I've gone through this at multiple companies and a startup that's selling B2B only needs one or two of these big outages and then enterprises start demanding SLA guarantees in their contracts. it's a self correcting problem
- logifail 3y ago
  > enterprises start demanding SLA guarantees My experience is that SLA "guarantees" don't actually guarantee anything. Your provider might be really generous and rebate a whole month's fees if they have a really, really, really bad month (perhaps they achieved less than 95% uptime, which is a day and half of downtime). It might not even be that much. How many of them will cover you for the business you lost and/or the reputational damage incurred while their service was down?
  - _jal 3y ago
    It depends entirely on how the SLAs are written. We have some that are garbage, and that's fine, because they really aren't essential services, SLAs are mainly a box-checking exercise. But where it counts, our SLAs have teeth. We have to, because we're offering SLAs with teeth to some of our customers. But that's not something you get "off the shelf", our lawyers negotiate that. You also don't spend that much effort on small contracts, so there's a floor with most vendors for even considering it.
resters 3y ago
To the extend that systems like chat-GPT are valuable, I expect we'll have open source equivalents to GPT-7 within the next five years. The only "moat" will be training on copyrighted content, and OpenAI is not likely to be able to afford to pay copyright owners enough once the value in the context of AI is widely understood. We might see SETI-like distributed training networks and specific permutations of open source licensing (for code and content) intended to address dystopian AI scenarios. It's only been a few years since we as a society learned that LLMs can be useful in this way, and OpenAI is managing to stay in the lead for now, though one could see in his facial countenance that Satya wants to fully own it so I think we can expect a MS acquisition to close within the next year and will be the most Microsoft has ever paid to acquire a company. MS could justify tremendous capital expenditure to get a clear lead over Google both in terms of product and IP related concerns. Also, from the standpoint of LLMs, Microsoft has far, far more proprietary data that would be valuable for training than any other company in the world.
- xeckr 3y ago
  >distributed training networks Now that's an idea. One bottleneck might be a limit on just how much you can parallelize training, though.
  - Aeolos 3y ago
    There's a ton of work in this area, and the reality is... it doesn't work for LLMs. Moving from 900GB/sec GPU memory bandwidth with infiniband interconnects between nodes to 0.01-0.1GB/sec over the internet is brutal (1000x to 10000x slower...) This works for simple image classifiers, but I've never seen anything like a large language model be trained in a meaningful amount of time this way.
    - resters 3y ago
      Maybe there is a way to train a neural network in a distributed way by training subsets of it and then connecting the aggregated weight changes to adjacent network segments. It wouldn't recover 1000x interconnect slowdowns, but might still be useful depending on the topology of the network.
halfcat 3y ago
The app maker can screw the plug-in author at any moment. For general cloud, avoiding screwing might mean multi cloud. But for LLM, there’s only one option at the highest level of quality for now. People tend to over focus on resilience (minimizing probability of breaking) and neglect the plan for recovery when things do break. I can’t tell you how weirdly foreign this is to many people, how many meetings I’ve been in where I ask what the plan is when it fails, and someone starts explaining RAID6 or BGP or something, with no actual plan, other than “it’s really unlikely to fail”, which old dogs know isn’t true. I guess the point is, for now, we’re all de facto plug-in authors.
- dragonwriter 3y ago
  > For general cloud, avoiding screwing might mean multi cloud. But for LLM, there’s only one option at the highest level of quality for now. There's always only one at the highest level of quality at a fine-grained enough resolution. Whether there's only one at sufficient quality for use, and if it is possible to switch between them in realtime without problems caused by the switch (e.g., data locked up in the provider that is down) is the relevant question, and whether the cost of building the multi-provider switching capability is worth it given the cost vs. risk of outage. All those are complicated questions that are application specific, not ones that have an easy answer on a global, uniform basis.
  - TeMPOraL 3y ago
    > There's always only one at the highest level of quality at a fine-grained enough resolution. Of course, but right now, there highest quality level option is an outlier, far ahead of everyone else, so if you need this level of quality (and I struggle to imagine user-facing products where you wouldn't!), there is only one option in the foreseeable future.
IKantRead 3y ago
Provided we can keep riding this hype wave for a while, I think the logical long term solution is most teams will have an in house/alternative LLM they can use as temporary backup. Right now everyone is scrambling to just get some basic products out using LLMs but as people have more breathing room I can't image most teams not having a non-OpenAI LLM that they are using to run experiments on. At the end of the day, OpenAI is just an API, so it's not an incredibly difficult piece of infrastructure to have a back up for.
- j45 3y ago
  I neither agree or disagree, but could you clarify which parts are hype to you? Self-hosting though is useful internally if for no other reason having some amount of fall back architecture. Binding directly only to one API is one oversight that can become a architectural debt issue. I"m spending some time fun time learning about API Proxies and Gateways.
- ilaksh 3y ago
  Except that it is currently impossible to replace GPT-4 with an open model.
  - Rastonbury 3y ago
    Depends on use case if your product has text summarisation, copywriting or translation, you can swap to many when openAI goes down and your users may not even notice
- dragonwriter 3y ago
  > At the end of the day, OpenAI is just an API, so it's not an incredibly difficult piece of infrastructure to have a back up for. The API is easy to reproduce, the functionality of the engines behind it less so. Yes, you can compatibly implement the APIs presented by OpenAI woth open source models hosted elsewhere (including some from OpenAI). And for some applications that can produce tolerable results. But LLMs (and multimodal toolchains centered on an LLM) haven't been commoditized to the point of being easy and mostly functionally-acceptable substitutes to the degree that, say, RDBMS engines are.
m3kw9 3y ago
People that used google and have some technical skill can still survive an OpenAI meltdown
- AlecSchueler 3y ago
  But people who built businesses whose core feature is based on OAI APIs might struggle.
  - m3kw9 3y ago
    Those business should have fall back if they are a serious company if OpenAI goes down. What I would do is have Claude or something or even 2 other models as backups. In the future they may allow on premise model but I don’t how they will secure the weights
thibaut_barrere 3y ago
Not a joke and not everybody is jumping on "AI via API calls", luckily. As more models are released, it becomes possible to integrate directly in some stacks (such as Elixir) without "direct" third-party reliance (except you still depend on a model, of course). For instance, see: - https://www.youtube.com/watch?v=HK38-HIK6NA https://www.youtube.com/watch?v=HK38-HIK6NA (in "LiveBook", but the same code would go inside an app, in a way that is quite easy to adapt) - https://news.livebook.dev/speech-to-text-with-whisper-timestamping-streaming-and-parallelism-oh-my---launch-week-2---day-2-36osSY https://news.livebook.dev/speech-to-text-with-whisper-timest... for the companion blog post I have already seen more than a few people running SaaS app on twitter complaining about AI-downtime :-) Of course, it will also come with a (maintenance) cost (but like external dependencies), as I described here: https://twitter.com/thibaut_barrere/status/1722172915733430770 https://twitter.com/thibaut_barrere/status/17221729157334307...
- j45 3y ago
  The average world and business user doesn't use an API directly. It can be easy to lose sight of that.
- ctrlmeta 3y ago
  Yes, sooner or later this is going to become the future of GPT in applications. The models are going to be embedded directly within the applications. I'm hoping for more progress in the performance of vectorized computing so that both model training and usage can become cheaper. If that happens, I am hopeful we are going to see a lot of open source models that can embedded into the applications.
s3p 3y ago
I mean.. it's a two hour outage. Depending on the severity of the problem that's quite a fast turnaround.
- munksbeer 3y ago
  It has been down for me for longer than two hours, and still not back.
j45 3y ago
The reliance to some degree is what it is until alternatives are available and easy enough to navigate, identify and adopt. Some of the tips in this discussion threads are invaluable and feel good for where I might already be thinking about some things and other new things to think about. Commenting separately on those below.
YetAnotherNick 3y ago
Isn't microsoft azure team working closely with them? There is also azure endpoint which is managed separately.
deleted 3y ago
[deleted]
taf2 3y ago
We were able to failover to Anthropic pretty quickly so limited impact. It'll be harder as we use more of the specialized API features in OpenAI like function calling or now tools...
- zaptrem 3y ago
  What’s your use case? The difference in behavior between the two models seems like it would make failover difficult.
  - taf2 3y ago
    It's really not that different - customers can ask questions about conversations, phone, text, video and typically use that to better understand topics, conversions, sales ops, customer service etc...
    - mlboss 3y ago
      This also shows that OpenAI or other providers does not have a real moat. The interface is very generic and best replaced easily with other provider or even with open model. I think thats why OpenAI is trying to move up the value chain with integration.
    - bmau5 3y ago
      fireflies? We've been looking for a tool like this to analyze customer feedback in aggregate (and have been frustrated with Dovetail's lack of functions here)
HumblyTossed 3y ago
> Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. Gonna be similar (or worse) to what happens when Github goes down. It amazes me how quickly people have come to rely on "AI" to do their work for them.
- sitzkrieg 3y ago
  if github goes down you cannot merge prs etc, literally blocking "work" so this is a stretch
  - Smaug123 3y ago
    Not really true - Git is distributed, after all. During an outage once I just hosted my copy of a certain Git repo somewhere. You can always push the history back up to the golden copy when GitHub comes back.
    - sitzkrieg 3y ago
      i am not talking about git, i am talking about github. lets say i need to merge a PR in GH because use gha pipelines or what have you to deploy a prod fix. this would become severely blocked. where as if openai goes down i can no longer use ai to generate a lame cover letter or whatever i was avoiding actually doing anyway, thats all
      - Smaug123 3y ago
        This is the realm of standard recovery planning though, isn't it? Like, your processes should be able to handle this, because it's routine: GitHub goes down at least once per month for long enough for them to declare an incident, per https://www.githubstatus.com/history https://www.githubstatus.com/history . E.g. one should think carefully before depriving onself of the break-glass ability to do manually what those pipelines do automatically.
        3y ago
toddmorey 3y ago
This is one of the many reasons open source is now more important that ever. Ironically, in the AI space it's now under attack more than ever.
dragonwriter 3y ago
> Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. But...are we? There's a reason that many enterprises that need reliability aren't doing that, but instead... > It took years before most companies who now use cloud providers to trust and be willing to bet their operations on them. That gave the cloud providers time to make their systems more robust, and to learn how to resolve issues quickly. ...to the extent that they are building dependencies on hosted AI services, doing it with traditional cloud providers hosted solutions, not first party hosting by AI development firms that aren't general enterprise cloud providers (e.g., for OpenAI models, using Azure OpenAI rather than OpenAI directly, for a bunch of others, AWS Bedrock.)
hexman 3y ago
Imagine if Apple's or Google's cloud went down and all your apps on iPhone and Android were broken and unavailable. Absolutely all apps on billions of phones. Cloud =! OpenAI Clouds store and process shareable information that multiple participants can access. Otherwise AI agents == new applications. OpenAI is the wrong evolution for the future of AI agents
danvoell 3y ago
Just fail whale it and move on. The dissonance of most folks about how difficult it is to build a product at massive scale from scratch is immense.
deleted 3y ago
[deleted]

benterix 3y ago

The point is, OpenAI spent a lot of money on training on all these copyrighted materials ordinary individuals/companies don't have access to, so replicating their effort would mean that you either 1) spend a ridiculous amount of money, 2) use Library Genesis (and still pay millions for GPU usage). So we have very little choice now. Open Source LLMs might be getting close to ChatGPT3 (opinions vary), but OpenAI is still far ahead.

btbuildem 3y ago
Don't underestimate Big Corp's resistance to using OpenAI's hosted solutions (even on Azure) for anything that's not marketing fluff.
- phillipcarter 3y ago
  You can say that about anything, though. BigCorps aren't exactly known for adopting useful tech on a reasonable timeline, let alone at all. I don't think anyone is under the impression that orgs who refuse to migrate off of Java 5 will be looking at OpenAI for anything.
  - swagempire 3y ago
    No, this is silly reasoning. A middle manager somewhere has no clue what Java 5 is. But he does know -- or let's say IMAGINES what he knows about ChatGPT. And unlike Java 5-- he just needs to use his departmental budget and instantly mandate that his team now use ChatGPT. Whatever that means you can argue it. But ChatGPT is a front line technology and super accessible. Java 5 is super back end and very specialized. The adoption you say won't happen: it will come from the middle -> up.
    - hashtag-til 3y ago
      Honest question: do you really mean Java 5 when you say Java 5? It sounds a bit 2000s to me.
      - swagempire 3y ago
        Parent used "Java 5" as an example. Java 5 somehow in my mind is from like the 200x era. But no. I practically mean any complicated back end technology that takes corporations months or years to migrate off of because its quite complicated and requires an intense amount of technical savoir-faire. My point was that ChatGPT bypasses all this and any middle manager can start using it anywhere for a small hit to his departmental budget.
      - eddtries 3y ago
        If you care about the security of OpenAI, you care about the EOL of 14 year old Java 5
      - cornel_io 3y ago
        In 2016 I worked on a project with a client who still mandated that all code was written to the Java 1.1 language specification - no generics, no enums, no annotations, etc., not to even mention all the stuff that's come since 1.5 (or Java 5, or whatever you want to call it). They had Reasons(tm), which after filtering through the nonsense mostly boiled down to the CTO being curmudgeonly and unwilling to approve replacing a hand-written code transformer that he had personally written back in the stone ages and that he 1) considered core to their product, and 2) considered too risky to replace, because obviously there were no tests covering any of the core systems...sigh. At least they ran it all on a modern JVM. But no, it would not surprise me to find a decent handful of large companies still writing Java 5 code; it would surprise me a bit more to find many still using that JVM, since you can't even get paid support through Oracle anymore, but I'm sure someone out there is doing it. Never underestimate the "don't touch it, you might break it" sentiment at non-tech companies, even big ones with lots of revenue, they routinely understaff their tech departments and the people who built key systems may have retired 20 years ago at this point so it's really risky to do any sort of big system migration. That's why so many lines of COBOL are still running.
- outside415 3y ago
  chatgpt will have an on prem solution eventually. in the mean time players like NVIDIA are working on that as well.
- tsunamifury 3y ago
  Marketing fluff is what 90% of tech is... it amazes me how many people think otherwise on hacker news. Unless you are building utility systems that run power plants, at the end of the day -- you're doing marketing fluff or the tools for it.
  - TeMPOraL 3y ago
    > Unless you are building utility systems that run power plants, at the end of the day -- you're doing marketing fluff or the tools for it. Even when you are building utility systems for critical infrastructure, you'll still be dealing with a disheartening amount of focus on marketing fluff and sales trickery.
colinsane 3y ago
the choice is to live 2 years behind (e.g. integrate the open source stuff and ride that wave of improvement). for businesses in a competitive space, that’s perhaps untenable. but for individuals and anywhere else where this stuff is just a “nice to have”, that’s really just the long-term sustainable approach. it reminds me of a choice like “do i host my website on a Windows Server, or a Linux box” at a time when both of these things are new.
- hfjjbf 3y ago
  [dead]
- BiteCode_dev 3y ago
  2 years behind in terms of timeline, but what factor in terms of productivity and quality of life? Not to mention openai's lead compounds, so 2 years now and 4 years in 2025 may be 10 times the original prod/qol gain.
  - verdverm 3y ago
    The gap seems to be shrinking, not growing. The OSS models have reach new capabilities faster than most thought
- TeMPOraL 3y ago
  For individuals, this is a very short window of time where we have cheap access to an actually useful, and relatively unshackled SOTA model[0]. This is the rare time individuals can empower themselves, become briefly better at whatever it is they're doing, expand their skills, cut through tedium, let their creativity bloom. It's only a matter of time before many a corporation and startup parcel it all between themselves, enshittify the living shit out of AI, disenfranchise individuals again and sell them as services what they just took away. No, it's exactly the individuals who can't afford to live "2 years behind". Benefits are too great, and worst that can happen is... going back to where one is now. -- [0] - I'm not talking the political bias and using the idea of alignment to give undue weigh to corporate reputation management issues. I'm talking about gutting the functionality to establish revenue channels. Like, imagine ChatGPT telling you it won't help you with your programming question, until you subscribe to Premium Dev Package for $language, or All Seasons Pass for all languages.
zarzavat 3y ago
OpenAI is obviously using libgen. Libgen is necessary but not sufficient for a top AI model. I believe that Google's corporate reluctance to use it is what's holding them back.
- nvm0n2 3y ago
  How do you know? Has someone created a set of test questions that can definitively prove it one way or another?
  - benterix 3y ago
    I don't believe it's even possible to create such a set of test questions. You would need to specifically test for differences between the versions found on Libgen and those that aren't available there. But even then, the evidence would be inconclusive.
weinzierl 3y ago
I won't say I disagree because only time can tell, but what you wrote sounds a lot like what people said before open source software took off. All these companies spend so much money on software development and they hire the best people available, how can a bunch of unorganized volunteers ever compete? We saw how they could and I hope we will see the same in AI.
giancarlostoro 3y ago
Kind of hilarious that the vast capabilities of an LLM are held back by copyright infringement.
- joquarky 3y ago
  The original purpose of copyright was to promote progress, and now it seems to hinder it.
- TeMPOraL 3y ago
  Intellectual property rights will yet turn out to be one of the Great Filters.
- zozbot234 3y ago
  I'd love to see a language model that was only trained on public domain and openly available content. It would probably be way too little data to give it ChatGPT-like generality, but even a GPT-2 scale model would be interesting.
  - TeMPOraL 3y ago
    If, hypothetically, libraries in the US - including in particular the Library of Congress - were to scan and OCR every book, newspaper and magazine they have with copyright protection already expired, would that be enough? Is there some estimate for the size of such dataset?
    - Turing_Machine 3y ago
      Much of that material is already available at https://archive.org https://archive.org. It might be good enough for some purposes, but limiting it to stuff before 1928 (in the United Sates) isn't going to be very helpful for (e.g.) coding. Maybe if you added github projects with permissive licenses?

jstummbillig 3y ago

On the one hand, sure, new things take time, but they also benefit from all past developments, and thus compounding effects can speed things along drastically. AI infrastructure problem are cloud infrastructure problems. Expecting it to go as if we were back on square one is a bit pessimistic.

Alifatisk 3y ago

> we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. You said it so well!

j45 3y ago
It's possible to include API gateways and API Proxies in between calls to normalize them across multiple providers as they become available.

somsak2 3y ago

I don't think it's so dire. I've gone through this at multiple companies and a startup that's selling B2B only needs one or two of these big outages and then enterprises start demanding SLA guarantees in their contracts. it's a self correcting problem

logifail 3y ago
> enterprises start demanding SLA guarantees My experience is that SLA "guarantees" don't actually guarantee anything. Your provider might be really generous and rebate a whole month's fees if they have a really, really, really bad month (perhaps they achieved less than 95% uptime, which is a day and half of downtime). It might not even be that much. How many of them will cover you for the business you lost and/or the reputational damage incurred while their service was down?
- _jal 3y ago
  It depends entirely on how the SLAs are written. We have some that are garbage, and that's fine, because they really aren't essential services, SLAs are mainly a box-checking exercise. But where it counts, our SLAs have teeth. We have to, because we're offering SLAs with teeth to some of our customers. But that's not something you get "off the shelf", our lawyers negotiate that. You also don't spend that much effort on small contracts, so there's a floor with most vendors for even considering it.

resters 3y ago

To the extend that systems like chat-GPT are valuable, I expect we'll have open source equivalents to GPT-7 within the next five years. The only "moat" will be training on copyrighted content, and OpenAI is not likely to be able to afford to pay copyright owners enough once the value in the context of AI is widely understood. We might see SETI-like distributed training networks and specific permutations of open source licensing (for code and content) intended to address dystopian AI scenarios. It's only been a few years since we as a society learned that LLMs can be useful in this way, and OpenAI is managing to stay in the lead for now, though one could see in his facial countenance that Satya wants to fully own it so I think we can expect a MS acquisition to close within the next year and will be the most Microsoft has ever paid to acquire a company. MS could justify tremendous capital expenditure to get a clear lead over Google both in terms of product and IP related concerns. Also, from the standpoint of LLMs, Microsoft has far, far more proprietary data that would be valuable for training than any other company in the world.

xeckr 3y ago
>distributed training networks Now that's an idea. One bottleneck might be a limit on just how much you can parallelize training, though.
- Aeolos 3y ago
  There's a ton of work in this area, and the reality is... it doesn't work for LLMs. Moving from 900GB/sec GPU memory bandwidth with infiniband interconnects between nodes to 0.01-0.1GB/sec over the internet is brutal (1000x to 10000x slower...) This works for simple image classifiers, but I've never seen anything like a large language model be trained in a meaningful amount of time this way.
  - resters 3y ago
    Maybe there is a way to train a neural network in a distributed way by training subsets of it and then connecting the aggregated weight changes to adjacent network segments. It wouldn't recover 1000x interconnect slowdowns, but might still be useful depending on the topology of the network.

halfcat 3y ago

The app maker can screw the plug-in author at any moment. For general cloud, avoiding screwing might mean multi cloud. But for LLM, there’s only one option at the highest level of quality for now. People tend to over focus on resilience (minimizing probability of breaking) and neglect the plan for recovery when things do break. I can’t tell you how weirdly foreign this is to many people, how many meetings I’ve been in where I ask what the plan is when it fails, and someone starts explaining RAID6 or BGP or something, with no actual plan, other than “it’s really unlikely to fail”, which old dogs know isn’t true. I guess the point is, for now, we’re all de facto plug-in authors.

dragonwriter 3y ago
> For general cloud, avoiding screwing might mean multi cloud. But for LLM, there’s only one option at the highest level of quality for now. There's always only one at the highest level of quality at a fine-grained enough resolution. Whether there's only one at sufficient quality for use, and if it is possible to switch between them in realtime without problems caused by the switch (e.g., data locked up in the provider that is down) is the relevant question, and whether the cost of building the multi-provider switching capability is worth it given the cost vs. risk of outage. All those are complicated questions that are application specific, not ones that have an easy answer on a global, uniform basis.
- TeMPOraL 3y ago
  > There's always only one at the highest level of quality at a fine-grained enough resolution. Of course, but right now, there highest quality level option is an outlier, far ahead of everyone else, so if you need this level of quality (and I struggle to imagine user-facing products where you wouldn't!), there is only one option in the foreseeable future.

IKantRead 3y ago

Provided we can keep riding this hype wave for a while, I think the logical long term solution is most teams will have an in house/alternative LLM they can use as temporary backup. Right now everyone is scrambling to just get some basic products out using LLMs but as people have more breathing room I can't image most teams not having a non-OpenAI LLM that they are using to run experiments on. At the end of the day, OpenAI is just an API, so it's not an incredibly difficult piece of infrastructure to have a back up for.

j45 3y ago
I neither agree or disagree, but could you clarify which parts are hype to you? Self-hosting though is useful internally if for no other reason having some amount of fall back architecture. Binding directly only to one API is one oversight that can become a architectural debt issue. I"m spending some time fun time learning about API Proxies and Gateways.
ilaksh 3y ago
Except that it is currently impossible to replace GPT-4 with an open model.
- Rastonbury 3y ago
  Depends on use case if your product has text summarisation, copywriting or translation, you can swap to many when openAI goes down and your users may not even notice
dragonwriter 3y ago
> At the end of the day, OpenAI is just an API, so it's not an incredibly difficult piece of infrastructure to have a back up for. The API is easy to reproduce, the functionality of the engines behind it less so. Yes, you can compatibly implement the APIs presented by OpenAI woth open source models hosted elsewhere (including some from OpenAI). And for some applications that can produce tolerable results. But LLMs (and multimodal toolchains centered on an LLM) haven't been commoditized to the point of being easy and mostly functionally-acceptable substitutes to the degree that, say, RDBMS engines are.

m3kw9 3y ago

People that used google and have some technical skill can still survive an OpenAI meltdown

AlecSchueler 3y ago
But people who built businesses whose core feature is based on OAI APIs might struggle.
- m3kw9 3y ago
  Those business should have fall back if they are a serious company if OpenAI goes down. What I would do is have Claude or something or even 2 other models as backups. In the future they may allow on premise model but I don’t how they will secure the weights

thibaut_barrere 3y ago

Not a joke and not everybody is jumping on "AI via API calls", luckily. As more models are released, it becomes possible to integrate directly in some stacks (such as Elixir) without "direct" third-party reliance (except you still depend on a model, of course). For instance, see: - https://www.youtube.com/watch?v=HK38-HIK6NA https://www.youtube.com/watch?v=HK38-HIK6NA (in "LiveBook", but the same code would go inside an app, in a way that is quite easy to adapt) - https://news.livebook.dev/speech-to-text-with-whisper-timestamping-streaming-and-parallelism-oh-my---launch-week-2---day-2-36osSY https://news.livebook.dev/speech-to-text-with-whisper-timest... for the companion blog post I have already seen more than a few people running SaaS app on twitter complaining about AI-downtime :-) Of course, it will also come with a (maintenance) cost (but like external dependencies), as I described here: https://twitter.com/thibaut_barrere/status/1722172915733430770 https://twitter.com/thibaut_barrere/status/17221729157334307...

j45 3y ago
The average world and business user doesn't use an API directly. It can be easy to lose sight of that.
ctrlmeta 3y ago
Yes, sooner or later this is going to become the future of GPT in applications. The models are going to be embedded directly within the applications. I'm hoping for more progress in the performance of vectorized computing so that both model training and usage can become cheaper. If that happens, I am hopeful we are going to see a lot of open source models that can embedded into the applications.

s3p 3y ago

I mean.. it's a two hour outage. Depending on the severity of the problem that's quite a fast turnaround.

munksbeer 3y ago
It has been down for me for longer than two hours, and still not back.

j45 3y ago

The reliance to some degree is what it is until alternatives are available and easy enough to navigate, identify and adopt. Some of the tips in this discussion threads are invaluable and feel good for where I might already be thinking about some things and other new things to think about. Commenting separately on those below.

YetAnotherNick 3y ago

Isn't microsoft azure team working closely with them? There is also azure endpoint which is managed separately.

deleted 3y ago

[deleted]

taf2 3y ago

We were able to failover to Anthropic pretty quickly so limited impact. It'll be harder as we use more of the specialized API features in OpenAI like function calling or now tools...

zaptrem 3y ago
What’s your use case? The difference in behavior between the two models seems like it would make failover difficult.
- taf2 3y ago
  It's really not that different - customers can ask questions about conversations, phone, text, video and typically use that to better understand topics, conversions, sales ops, customer service etc...
  - mlboss 3y ago
    This also shows that OpenAI or other providers does not have a real moat. The interface is very generic and best replaced easily with other provider or even with open model. I think thats why OpenAI is trying to move up the value chain with integration.
  - bmau5 3y ago
    fireflies? We've been looking for a tool like this to analyze customer feedback in aggregate (and have been frustrated with Dovetail's lack of functions here)

HumblyTossed 3y ago

> Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. Gonna be similar (or worse) to what happens when Github goes down. It amazes me how quickly people have come to rely on "AI" to do their work for them.

sitzkrieg 3y ago
if github goes down you cannot merge prs etc, literally blocking "work" so this is a stretch
- Smaug123 3y ago
  Not really true - Git is distributed, after all. During an outage once I just hosted my copy of a certain Git repo somewhere. You can always push the history back up to the golden copy when GitHub comes back.
  - sitzkrieg 3y ago
    i am not talking about git, i am talking about github. lets say i need to merge a PR in GH because use gha pipelines or what have you to deploy a prod fix. this would become severely blocked. where as if openai goes down i can no longer use ai to generate a lame cover letter or whatever i was avoiding actually doing anyway, thats all
    - Smaug123 3y ago
      This is the realm of standard recovery planning though, isn't it? Like, your processes should be able to handle this, because it's routine: GitHub goes down at least once per month for long enough for them to declare an incident, per https://www.githubstatus.com/history https://www.githubstatus.com/history . E.g. one should think carefully before depriving onself of the break-glass ability to do manually what those pipelines do automatically.
      - 3y ago

toddmorey 3y ago

This is one of the many reasons open source is now more important that ever. Ironically, in the AI space it's now under attack more than ever.

dragonwriter 3y ago

> Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. But...are we? There's a reason that many enterprises that need reliability aren't doing that, but instead... > It took years before most companies who now use cloud providers to trust and be willing to bet their operations on them. That gave the cloud providers time to make their systems more robust, and to learn how to resolve issues quickly. ...to the extent that they are building dependencies on hosted AI services, doing it with traditional cloud providers hosted solutions, not first party hosting by AI development firms that aren't general enterprise cloud providers (e.g., for OpenAI models, using Azure OpenAI rather than OpenAI directly, for a bunch of others, AWS Bedrock.)

hexman 3y ago

Imagine if Apple's or Google's cloud went down and all your apps on iPhone and Android were broken and unavailable. Absolutely all apps on billions of phones. Cloud =! OpenAI Clouds store and process shareable information that multiple participants can access. Otherwise AI agents == new applications. OpenAI is the wrong evolution for the future of AI agents

danvoell 3y ago

Just fail whale it and move on. The dissonance of most folks about how difficult it is to build a product at massive scale from scratch is immense.

deleted 3y ago

[deleted]