3 ms·

Data security and privacy. Our clients (in aviation, finance, etc.) need this due to legal and regulatory reasons. Also, the new Llama 2 models are very powerfu

Data security and privacy. Our clients (in aviation, finance, etc.) need this due to legal and regulatory reasons. Also, the new Llama 2 models are very powerful. In my testing, Llama 2 70b is comparable to GPT-3.5 in capability. (Shameless plug: here's our website: https://www.amw.ai/ https://www.amw.ai/)

euazOn 3y ago
I am also interested in your open-source-first approach, and am a bit confused from the landing page. Could you explain what exactly your product does? Is it an LLM-enhanced document parser?
tab_jockey 3y ago
Looks great my friend
fogx 3y ago
have you considered azure's GPT, or is that not private enough?
- hhh 3y ago
  This assumes you trust Microsoft.
- npsomaratna 3y ago
  We have. This is acceptable for some clients, but not for others. Both groups, however, prefer maintaining complete control over their data, given the chance. Edit: plus, my personal view is that local LLMs are the future. They've already caught up to GPT-3.5 (based on my testing); and they continue to evolve rapidly. Makes sense to focus our limited resources on riding that wave. OpenAI won't go away, but neither will they remain the first choice (or only choice!) for most use-cases.
- MilStdJunkie 3y ago
  There's some . . entrepreneurs . . who have been promising NIST/ITAR-compliant LLM frameworks on Azure, but when you ask around, they have not done all the legwork (AG/AGS). They're working off Azure Public, with "waivers" that they won't show anyone. Also, the history of their leadership is . . questionable. It all feels just a little hinky. Until that's cleared up, I advise anyone fooling with LLMs to do it on-prem, at least for the moment. One thing I'm worried about: doing LLMs with something like GovCloud is going to be absolutely bananas in terms of price-per-compute.
andy99 3y ago
I'm in the same boat. We have customers that need to run models in an environment that have access to the public internet, even if they did trust OpenAI et al. More importantly for me, I don't want to be beholden to a model provider and have to take what they give me. I'd rather host my own model if an API was an option, because then I have control over it and can hack it as I want. I don't want to be just a wrapper on GPT which is sort of what you're stuck with if you just want to use their APIs.
- npsomaratna 3y ago
  Agreed. Worse, you don't want to be in a position where any single company can end your business at the flip of a switch.

euazOn 3y ago

I am also interested in your open-source-first approach, and am a bit confused from the landing page. Could you explain what exactly your product does? Is it an LLM-enhanced document parser?

tab_jockey 3y ago

Looks great my friend

fogx 3y ago

have you considered azure's GPT, or is that not private enough?

hhh 3y ago
This assumes you trust Microsoft.
npsomaratna 3y ago
We have. This is acceptable for some clients, but not for others. Both groups, however, prefer maintaining complete control over their data, given the chance. Edit: plus, my personal view is that local LLMs are the future. They've already caught up to GPT-3.5 (based on my testing); and they continue to evolve rapidly. Makes sense to focus our limited resources on riding that wave. OpenAI won't go away, but neither will they remain the first choice (or only choice!) for most use-cases.
MilStdJunkie 3y ago
There's some . . entrepreneurs . . who have been promising NIST/ITAR-compliant LLM frameworks on Azure, but when you ask around, they have not done all the legwork (AG/AGS). They're working off Azure Public, with "waivers" that they won't show anyone. Also, the history of their leadership is . . questionable. It all feels just a little hinky. Until that's cleared up, I advise anyone fooling with LLMs to do it on-prem, at least for the moment. One thing I'm worried about: doing LLMs with something like GovCloud is going to be absolutely bananas in terms of price-per-compute.

andy99 3y ago

I'm in the same boat. We have customers that need to run models in an environment that have access to the public internet, even if they did trust OpenAI et al. More importantly for me, I don't want to be beholden to a model provider and have to take what they give me. I'd rather host my own model if an API was an option, because then I have control over it and can hack it as I want. I don't want to be just a wrapper on GPT which is sort of what you're stuck with if you just want to use their APIs.

npsomaratna 3y ago
Agreed. Worse, you don't want to be in a position where any single company can end your business at the flip of a switch.