5 ms·

Makes me wonder whether to keep building upon OpenAI? Given that they have an API and it takes effort to build on that vs. something else. I am small fry but ma

Makes me wonder whether to keep building upon OpenAI? Given that they have an API and it takes effort to build on that vs. something else. I am small fry but maybe other people are wondering the same? Can they give reassurances about their products going into the future?

deleted 3y ago
[deleted]
siva7 3y ago
I am wondering the same. It’s a PR desaster to their dev community and i’m not even sure if Sutskever isn’t secretly happy about this.
- karmasimida 3y ago
  I am at lost. Not fear, just lost. Don't know what to do. Is my investment into their API still worth it? It feels very unstable at this moment.
  - ThrowawayTestr 3y ago
    Microsoft owns it. I honestly can not imagine trying to build a business on an API owned by Microsoft of all companies.
    - karmasimida 3y ago
      Would like to elaborate a bit? Though my investment will be still tiny at the moment, but not other multi-modal model on the market right is as good.
      - reissbaker 3y ago
        I actually wouldn't be worried in the short term for exactly this reason. Microsoft has legal access to GPT-4 and is allowed to host and serve it via Azure. If OpenAI somehow tanks its API in the near term, MS is sitting on a gold mine and will make use of that by continuing to serve it. In the long term I am worried, but less worried if Sam and Greg form a competing co to continue to build.
    - dewey 3y ago
      What makes you say that? Microsoft seems like one of the more reliable partners to build on compared to Google etc. just for the simple reason that their customers are large businesses and not breaking things for them is in their blood. Just like Windows backwards compatibility.
    - jeswin 3y ago
      People have been successfully building on MS APIs for four decades now. I've been for nearly three. What exactly are you saying?
      - ThrowawayTestr 3y ago
        Why would you build a business that Microsoft could sherlock in a second?
    - ahepp 3y ago
      Isn't Microsoft famous for their insane API stability and backwards compatibility?
      - isbvhodnvemrwvn 3y ago
        While I think the sentiment is overblown, if you compare Azure to AWS, Azure's stability is like Google's.
      - amai 3y ago
        Microsoft Azure is famous for its insanely bad security: https://karl-voit.at/cloud/ https://karl-voit.at/cloud/
      - wavewrangler 3y ago
        Microsoft is famous for a lack of adoption DESPITE backwards compatibility.
      - andomiz2 3y ago
        No I would not say that at all. Microsoft has gone through how many Desktop APIs for Windows at this point? At least a dozen I'd think.
    - zx8080 3y ago
      Do you use github?
  - padolsey 3y ago
    If you're just using their completions/chat API, you're gonna be ok. As an ultimate fallback you can spin up H100s in the cloud and run VLLM atop a high param open model like Llama 70B. Such models will catch up and their param counts will increase.. eventually. But initially expect gpt-3.5-esque performance. VLLM will give you an OpenAI-like REST API atop a range of models. Keep making things :))
    - karmasimida 3y ago
      Thx. I will. My current interests mainly lies in benchmarking their vision model. That being said, I might not go further relying on their APIs for something more serious
  - croes 3y ago
    By that logic you could never use a third party API.
  - abm53 3y ago
    If you are building something that is end-user facing that relies on ChatGPT then that was always a huge and risky bet on the future of OpenAI. In addition, it would likely be some time, possibly years, before it would be ready for production. Perhaps recent events have just brought that more clearly into focus for you.
deleted 3y ago
[deleted]
cryptoz 3y ago
Also same here. Actually currently staying up late Friday night hacking on OpenAI API projects (while waiting for SpaceX Starship launch, it's quite a day for high-tech news!) - and wondering if I should even bother. Of course I will keep hacking, but...still it makes you think. Which is a very unexpected feeling. Hugely more interested in the open source models now, even if they are not as good at present. Because at least there is a near-100% guarantee that they will continue to have community support no matter what; the missing problem I suppose is GPUs to run them.
- quickthrower2 3y ago
  Totally. I'll keep going too. I am just putting a nice GUI wrapper around the new Assistant stuff which looks damn cool. Project is half "might make some bucks" and half "see if this is good to use in the day job".
  - Sai_ 3y ago
    Yeah, the assistants api is pretty great. Curious if you’ve faced issues with certain things not working out of the blue forcing you to re-run threads? For example, i have an assistant which is supposed to parse an uploaded file and extract useful info from it. To use this assistant, I create a thread and a run and attach it to the assistant with a different file-id. About half the time, the assistant simply throws up its hands and says it can’t parse the file I supplied with the thread. Retrying a few times seems to do the trick.
    - sbrother 3y ago
      The assistants API is fantastic; I was just getting started with it. This news makes me reconsider -- but I also think it's inevitable that a compatible API will be released with open source underlying LLMs. I've deployed the OpenAI-compatible completions API over Llama2 in production with vLLM, and it works perfectly. Do you know if there are any projects working on this? Even something like a high quality json-tuned base model would go a huge way toward replicating OpenAI's current product.
robbomacrae 3y ago
I'm in this boat. Not for my startup but for side projects I was absolutely pinning my hopes on them unlocking access to tools and relaxing some of their restrictions in the near future.. a future which now seem unlikely.
mebutnotme 3y ago
I’d recommend trying to build out your systems to work across LLMs where you can. Create an interface layer and for now maybe use OpenAI and Vertex as a couple of options. Vertex is handy as while not always as good you may find it works well for some tasks and it can be a lot cheaper for those. If you build out this way then when the next greatest LLM comes out you can plug that into your interface and switch the tasks it’s best at over.
- quickthrower2 3y ago
  The problem is swapping LLMs can require rework of all your prompts, and you may be relying on specific features of OpenAI. If you don't then you are at a disadvantage or at least slowing down your work.
  - bongobingo1 3y ago
    Just ask the LLM to rewrite your prompts for the new model.
    - worldsayshi 3y ago
      Does it really have that kind of self awareness to be able to do that successfully? I feel very sceptical.
      - Roark66 3y ago
        I doubt self awareness has anything to do with it..
        worldsayshi 3y ago
        What else would you call the ability for it to adapt a task for its own capabilities?
        mkl 3y ago
croes 3y ago
>Can they give reassurances about their products going into the future They wouldn't have been able to do that even before Sam's dismissal
bananapub 3y ago
it's business and systems design 101 to actually worry about your dependencies. regardless of this drama, you should have thought about what you'd do if OpenAI shut down, or become your competitor, or gets worse, or is bought by MS or something. > Can they give reassurances about their products going into the future? emotional comfort is not the thing you should be looking for mate.
iamflimflam1 3y ago
Indeed. It was a big enough battle to convince execs that building on top of OpenAI was ok. Now that conversation is pretty much impossible. You have the Microsoft offering, but to most muggles that just looks like them reselling OpenAI. The board of OpenAI should have been replaced by adults a long time ago.
mark_l_watson 3y ago
Take my opinion with some skepticism because I am retired and the massive amount of time I put into LLMs (and deep learning in general) is only for my own understanding and enjoyment: In all three languages I frequently use (Common Lisp, Python, and Racket) it is easy to switch between APIs. You can also use a library like LangChain to make switching easier. For people building startups on OpenAI specific APIs, they can certainly protect themselves by using Azure as an intermediary. Microsoft is in the “stability business.”

deleted 3y ago

[deleted]

siva7 3y ago

I am wondering the same. It’s a PR desaster to their dev community and i’m not even sure if Sutskever isn’t secretly happy about this.

karmasimida 3y ago
I am at lost. Not fear, just lost. Don't know what to do. Is my investment into their API still worth it? It feels very unstable at this moment.
- ThrowawayTestr 3y ago
  Microsoft owns it. I honestly can not imagine trying to build a business on an API owned by Microsoft of all companies.
  - karmasimida 3y ago
    Would like to elaborate a bit? Though my investment will be still tiny at the moment, but not other multi-modal model on the market right is as good.
    - reissbaker 3y ago
      I actually wouldn't be worried in the short term for exactly this reason. Microsoft has legal access to GPT-4 and is allowed to host and serve it via Azure. If OpenAI somehow tanks its API in the near term, MS is sitting on a gold mine and will make use of that by continuing to serve it. In the long term I am worried, but less worried if Sam and Greg form a competing co to continue to build.
  - dewey 3y ago
    What makes you say that? Microsoft seems like one of the more reliable partners to build on compared to Google etc. just for the simple reason that their customers are large businesses and not breaking things for them is in their blood. Just like Windows backwards compatibility.
  - jeswin 3y ago
    People have been successfully building on MS APIs for four decades now. I've been for nearly three. What exactly are you saying?
    - ThrowawayTestr 3y ago
      Why would you build a business that Microsoft could sherlock in a second?
  - ahepp 3y ago
    Isn't Microsoft famous for their insane API stability and backwards compatibility?
    - isbvhodnvemrwvn 3y ago
      While I think the sentiment is overblown, if you compare Azure to AWS, Azure's stability is like Google's.
    - amai 3y ago
      Microsoft Azure is famous for its insanely bad security: https://karl-voit.at/cloud/ https://karl-voit.at/cloud/
    - wavewrangler 3y ago
      Microsoft is famous for a lack of adoption DESPITE backwards compatibility.
    - andomiz2 3y ago
      No I would not say that at all. Microsoft has gone through how many Desktop APIs for Windows at this point? At least a dozen I'd think.
  - zx8080 3y ago
    Do you use github?
- padolsey 3y ago
  If you're just using their completions/chat API, you're gonna be ok. As an ultimate fallback you can spin up H100s in the cloud and run VLLM atop a high param open model like Llama 70B. Such models will catch up and their param counts will increase.. eventually. But initially expect gpt-3.5-esque performance. VLLM will give you an OpenAI-like REST API atop a range of models. Keep making things :))
  - karmasimida 3y ago
    Thx. I will. My current interests mainly lies in benchmarking their vision model. That being said, I might not go further relying on their APIs for something more serious
- croes 3y ago
  By that logic you could never use a third party API.
- abm53 3y ago
  If you are building something that is end-user facing that relies on ChatGPT then that was always a huge and risky bet on the future of OpenAI. In addition, it would likely be some time, possibly years, before it would be ready for production. Perhaps recent events have just brought that more clearly into focus for you.

deleted 3y ago

[deleted]

cryptoz 3y ago

Also same here. Actually currently staying up late Friday night hacking on OpenAI API projects (while waiting for SpaceX Starship launch, it's quite a day for high-tech news!) - and wondering if I should even bother. Of course I will keep hacking, but...still it makes you think. Which is a very unexpected feeling. Hugely more interested in the open source models now, even if they are not as good at present. Because at least there is a near-100% guarantee that they will continue to have community support no matter what; the missing problem I suppose is GPUs to run them.

quickthrower2 3y ago
Totally. I'll keep going too. I am just putting a nice GUI wrapper around the new Assistant stuff which looks damn cool. Project is half "might make some bucks" and half "see if this is good to use in the day job".
- Sai_ 3y ago
  Yeah, the assistants api is pretty great. Curious if you’ve faced issues with certain things not working out of the blue forcing you to re-run threads? For example, i have an assistant which is supposed to parse an uploaded file and extract useful info from it. To use this assistant, I create a thread and a run and attach it to the assistant with a different file-id. About half the time, the assistant simply throws up its hands and says it can’t parse the file I supplied with the thread. Retrying a few times seems to do the trick.
  - sbrother 3y ago
    The assistants API is fantastic; I was just getting started with it. This news makes me reconsider -- but I also think it's inevitable that a compatible API will be released with open source underlying LLMs. I've deployed the OpenAI-compatible completions API over Llama2 in production with vLLM, and it works perfectly. Do you know if there are any projects working on this? Even something like a high quality json-tuned base model would go a huge way toward replicating OpenAI's current product.

robbomacrae 3y ago

I'm in this boat. Not for my startup but for side projects I was absolutely pinning my hopes on them unlocking access to tools and relaxing some of their restrictions in the near future.. a future which now seem unlikely.

mebutnotme 3y ago

I’d recommend trying to build out your systems to work across LLMs where you can. Create an interface layer and for now maybe use OpenAI and Vertex as a couple of options. Vertex is handy as while not always as good you may find it works well for some tasks and it can be a lot cheaper for those. If you build out this way then when the next greatest LLM comes out you can plug that into your interface and switch the tasks it’s best at over.

quickthrower2 3y ago
The problem is swapping LLMs can require rework of all your prompts, and you may be relying on specific features of OpenAI. If you don't then you are at a disadvantage or at least slowing down your work.
- bongobingo1 3y ago
  Just ask the LLM to rewrite your prompts for the new model.
  - worldsayshi 3y ago
    Does it really have that kind of self awareness to be able to do that successfully? I feel very sceptical.
    - Roark66 3y ago
      I doubt self awareness has anything to do with it..
      - worldsayshi 3y ago
        What else would you call the ability for it to adapt a task for its own capabilities?
        mkl 3y ago

croes 3y ago

>Can they give reassurances about their products going into the future They wouldn't have been able to do that even before Sam's dismissal

bananapub 3y ago

it's business and systems design 101 to actually worry about your dependencies. regardless of this drama, you should have thought about what you'd do if OpenAI shut down, or become your competitor, or gets worse, or is bought by MS or something. > Can they give reassurances about their products going into the future? emotional comfort is not the thing you should be looking for mate.

iamflimflam1 3y ago

Indeed. It was a big enough battle to convince execs that building on top of OpenAI was ok. Now that conversation is pretty much impossible. You have the Microsoft offering, but to most muggles that just looks like them reselling OpenAI. The board of OpenAI should have been replaced by adults a long time ago.

mark_l_watson 3y ago

Take my opinion with some skepticism because I am retired and the massive amount of time I put into LLMs (and deep learning in general) is only for my own understanding and enjoyment: In all three languages I frequently use (Common Lisp, Python, and Racket) it is easy to switch between APIs. You can also use a library like LangChain to make switching easier. For people building startups on OpenAI specific APIs, they can certainly protect themselves by using Azure as an intermediary. Microsoft is in the “stability business.”