4 ms·
Congrats, I think this will be really successfull and you got a very early food in the door. Do you consider self hosted LLMs a competitor of sorts? I suppose
by kriro 3y ago
Congrats, I think this will be really successfull and you got a very early food in the door.
Do you consider self hosted LLMs a competitor of sorts? I suppose your premise is if a company uses Google Docs they will also likely never host internal LLMs, right?
- r_thambapillai 3y agoThanks so much! So about half of our enterprise customers use Credal in conjunction with either a self hosted LLM or Azure OpenAI (which you can debate, but most companies we've spoken to seem to treat their Azure OpenAI instance as equivalent to self hosted). In practice, you still need to: 1. Manage the permissions of making sure the self hosted LLM is only reading from the documents, slack channels, e.t.c. that the end user should actually have access to 2. Generate an audit log of exactly who did what when So we actually see self hosted LLMs being a big part of how Credal is used! In the long term, we think Credal will actually become a tool for AI app developers to safely request access to data & embeddings from the enterprise on the fly, and make sure the data they get is appropriately controlled and the audit logs exist in a single place for the enterprise to see what data went to whom/when/why etc