60 ms·

I'm sure others will comment, but in the meantime, some people have written up their experiences using Cortex in production. I'd point you to AI Dungeon: https:

I'm sure others will comment, but in the meantime, some people have written up their experiences using Cortex in production. I'd point you to AI Dungeon: https://medium.com/@aidungeon/how-we-scaled-ai-dungeon-2-to-support-over-1-000-000-users-d207d5623de9 https://medium.com/@aidungeon/how-we-scaled-ai-dungeon-2-to-... We also have a pretty active Gitter channel: https://gitter.im/cortexlabs/cortex https://gitter.im/cortexlabs/cortex As for your second question, Cortex uses Docker to containerize models. The rest of Cortex's features (deploying models as microservices, orchestrating an inference cluster, autoscaling, prediction monitoring, etc.) are outside Docker's scope.

bobosha 7y ago
>orchestrating an inference cluster, autoscaling, prediction monitoring, Does this approach preclude the need for queuing (a la RabbitMQ) and/or a load balancer?
- calebkaiser 7y ago
  Yep! Cortex deploys load balancers on AWS and manages queueing.
  - bobosha 7y ago
    This is super-exciting! I didn't know it could be this easy! How do you handle API authentication? Is there a module that interfaces with AWS API gateway? or external API authentication?
    - calebkaiser 7y ago
      Right now, users handle API auth by using AWS API gateway in front of Cortex, but incorporating AWS API Gateway into Cortex to automate this is on our short term roadmap.