3 ms·
This seems super cool if as described, but I'm sure you can understand the skepticism. Do you anticipate having any kind of public accessible chat interface fo
by supern0va 5mo ago
This seems super cool if as described, but I'm sure you can understand the skepticism.
Do you anticipate having any kind of public accessible chat interface for testing in the near future?
Also, what, if any, benefits are there for smaller context windows? Is there still a material improvement in cost to serve under say 256k? I'm curious about the broader implications for the space beyond improvements for very large context windows.
- alexsubq 5mo agoI do, for sure! Yes, we have a few product rollouts lined up. The differentials for latency are posted in our blog post, so that should provide an idea of where the scaling law differentials kick in.
- dvfjsdhgfv 5mo ago> I do, for sure! Yes, we have a few product rollouts lined up. When, more or less?
- alexsubq 5mo agoWe will have a few rollouts in the next two months.