If your requirements are just to load balance between selhosted AI servers: nginx. If you want a more thorough system with configurability, logging, etc.: Bifro

If your requirements are just to load balance between selhosted AI servers: nginx. If you want a more thorough system with configurability, logging, etc.: Bifrost from MaximAI