4 ms·
If you require consistency for research, I wouldn't recommend relying on hosted inference without validating the provider/inference stack very carefully. For m
by ndr_ 16d ago
If you require consistency for research, I wouldn't recommend relying on hosted inference without validating the provider/inference stack very carefully.
For my contribution to OpenAI's gpt-oss red-teaming competition (published as arXiv:2510.01259), I initially used OpenRouter with DeepInfra and Together AI. The results were much too noisy to draw reliable conclusions from, and markedly different behavior through AWS Bedrock was the last straw. I ended up renting GPUs through vast.ai and running the model myself with vLLM.