6 ms·

Unsurprisingly, exactly the same pricing as OpenAI. So what ends up the benefits? I'm currently using GPT-4 for hacking around, and love it if I could get faste

Unsurprisingly, exactly the same pricing as OpenAI. So what ends up the benefits? I'm currently using GPT-4 for hacking around, and love it if I could get faster response times, would make the use case I'm hacking on a lot more attractive. If I use Microsofts API, would it be any faster compared to OpenAI?

lloydatkinson 4y ago
Because it's one less thing to manage outside of Azure. Now you can use IaC such as ARM, Terraform, Bicep, to manage the deployment and configuration.
jimmyechan 4y ago
Your app could be slightly but meaningfully faster (depending on your use case) if the other servers/services your app relies on are close by (also on Azure, in the same region region or datacenter). Otherwise probably not much more speed difference aside. There are other benefits such as managing everything in the same cloud, single billing, and maybe even some security benefits of having everything in the same VPC. This is assuming Azure will allow you to chose a region for the GPT4 service.
- capableweb 4y ago
  When dealing with GPT/LLMs, latency of requests is a minimal part of the time spent, so I don't think geographic location will matter much for that.
  - jimmyechan 4y ago
    Yes, while the GPT API latency will be minimal, whether it's noticeable by users would still depend on the application, use case, and even the location of the client(s). In some cases, the small latencies may still be noticeable for example in gaming, chat apps, or maybe some large enterprise deployment that's tied to specific regions. I agree that for hacking it won't make a significant difference and there's no reason to get on Azure just for GPT - too much overhead and Azure isn't the easiest to figure out to begin with.
chatmasta 4y ago
Maybe you've got some Azure credits and can only spend them on OpenAI when packaged as an Azure service? Too bad ours just expired :( (anyone from microsoft reads this... hmu?)
iknownothow 4y ago
Almost all European companies will be inclined to use Azure in EU regions to get data privacy guarantees than use OpenAI's API directly.

lloydatkinson 4y ago

Because it's one less thing to manage outside of Azure. Now you can use IaC such as ARM, Terraform, Bicep, to manage the deployment and configuration.

jimmyechan 4y ago

Your app could be slightly but meaningfully faster (depending on your use case) if the other servers/services your app relies on are close by (also on Azure, in the same region region or datacenter). Otherwise probably not much more speed difference aside. There are other benefits such as managing everything in the same cloud, single billing, and maybe even some security benefits of having everything in the same VPC. This is assuming Azure will allow you to chose a region for the GPT4 service.

capableweb 4y ago
When dealing with GPT/LLMs, latency of requests is a minimal part of the time spent, so I don't think geographic location will matter much for that.
- jimmyechan 4y ago
  Yes, while the GPT API latency will be minimal, whether it's noticeable by users would still depend on the application, use case, and even the location of the client(s). In some cases, the small latencies may still be noticeable for example in gaming, chat apps, or maybe some large enterprise deployment that's tied to specific regions. I agree that for hacking it won't make a significant difference and there's no reason to get on Azure just for GPT - too much overhead and Azure isn't the easiest to figure out to begin with.

chatmasta 4y ago

Maybe you've got some Azure credits and can only spend them on OpenAI when packaged as an Azure service? Too bad ours just expired :( (anyone from microsoft reads this... hmu?)

iknownothow 4y ago

Almost all European companies will be inclined to use Azure in EU regions to get data privacy guarantees than use OpenAI's API directly.