2 ms·
A quick play and the UX is great. If you are allowing anyone to use 405B by API then you beat Groq to that :) they have limited the access. Probably need to kn
by langcss 2y ago
A quick play and the UX is great. If you are allowing anyone to use 405B by API then you beat Groq to that :) they have limited the access.
Probably need to know more about you. Main concerns are "can this be used for production" and "will you be around in a year" and "will you likely have 429s if you get too popular?" and of course likely pricing.
- billycao 2y agoThanks for the feedback! We're working on fleshing out ToS, privacy policy, and company specifics, but just to answer your first question, I'm Billy Cao, an ex-Google eng, and Matt Baker is ex-Airbnb, ex-Meta. Re: concerns, our infra will scale relatively well (several qps per model, probably), but we're still in the stages of fleshing things out and getting feedback. :) Feel free to drop us a line at hi@glhf.chat if you wanted to chat specifics! - Billy
- langcss 2y agoThanks Billy! Nothing specific... just wanted to get the feel. Out of interest are you bare metal or using AWS or something? Or both?
- billycao 2y agoAppreciate the feedback! We currently use fly.io as our cloud GPU provider, but we're actively investigating other providers due to various limitations (like NVLink support).