conshama

4 ms·

1. 1.▲
   by conshama 4mo ago

   Somehow I never got the hype around Groq. They served models with fast inference speed - that sounded great in theory - and as a user I was looking forward to use them. But, when I did try, I discovered that they are quantizing the models u
2. 2.▲
   by conshama 2y ago

   Yes.
3. 3.▲
   by conshama 2y ago

   related: https://www.nonbios.ai/post/deploying-large-405b-models-in-f... tldr: uses the latest rocm 6.2 to run full precision inference for llama 405b on a single node 8 x MI300x AMD GPU How mature do you think Rocm 6.
4. 4.▲
   by conshama 2y ago

   Seems to me that the only difference between Anthropic and OpenAI is that Anthropic was for-profit from day one and OpenAI is from day yesterday. I pay for both, and pretty sure they will do everything they can to take as much money from me