4 ms·

A lot of companies are already using projects like chatbot-ui with Azure's OpenAI for similar local deployments. Given this is as close to local ChatGPT as any

by ajhai 3y ago

A lot of companies are already using projects like chatbot-ui with Azure's OpenAI for similar local deployments. Given this is as close to local ChatGPT as any other project can get, this is a huge deal for all those enterprises looking to maintain control over their data. Shameless plug: Given the sensitivity of the data involved, we believe most companies prefer locally installed solutions to cloud based ones at least in the initial days. To this end, we just open sourced LLMStack (https://github.com/TryPromptly/LLMStack https://github.com/TryPromptly/LLMStack) that we have been working on for a few months now. LLMStack is a platform to build LLM Apps and chatbots by chaining multiple LLMs and connect to user's data. A quick demo at https://www.youtube.com/watch?v=-JeSavSy7GI https://www.youtube.com/watch?v=-JeSavSy7GI. Still early days for the project and there are still a few kinks to iron out but we are very excited for it.

toomuchtodo 3y ago
Can you plug this together with tools like api2ai to create natural language defined workflow automations that interact with external APIs?
- ajhai 3y ago
  There is a generic HTTP API processor that can be used to call APIs as part of the app flow which should help invoke tools. Currently working on improving documentation so it is easy to get started with the project. We also have some features planned around function calling that should make it easy to natively integrate tools into the app flows.
- cosbgn 3y ago
  You can use unfetch.com to make API calls via LLMs and build automations. (I'm building it)
  - scrum-treats 3y ago
    Is it possible to not use Google with unfetch.com?
    - cosbgn 3y ago
      Google is just so easy for login. No need to deal with password forgot, reset, email verification etc. But I'll add login via magic link soon.
gdiamos 3y ago
I find it interesting to see how competitive this space got so quickly. How do these stacks differentiate?
- scrum-treats 3y ago
  Quality and depth of particular types of training data is one difference. Another difference is inference tracking mechanisms within and between single-turn interactions (e.g., what does the human user "mean" with their prompt, what is the "correct" response, and how best can I return the "correct" response for this context; how much information do I cache from the previous turns, and how much if any of it is relevant to this current turn interaction).
- peteradio 3y ago
  [flagged]
  - gdiamos 3y ago
    Thanks that made me smile. Take my upvote
    - omarfarooq 3y ago
      OP shouldn't be flagged.
- lmeyerov 3y ago
  With Louie.ai, there is a lot of work on specialization for the job, and I expect the same for others. We help with data analysis, so connecting enterprise & common data sources & DBs, hooking up data tools (GPU visuals, integrated code interpreter, ...), security controls, and the like, which is different from say a ChatGPT for lawyers or a straight up ChatGPT UI clone. Technically, as soon as the goal is to move beyond just text2gpt2screen, like multistep data wrangling & viz in the middle of a conversation, most tools technically struggle. Query quality also comes up, whether quality of the RAG, the fine tune, prompts, etc: each solves different problems.
bhanu423 3y ago
Interesting project - was trying it out, found an issue in building the image - have opened an issue on github - please take a look. Also do you have plan to support llama over openai models.
- ajhai 3y ago
  Thanks for the issue. Will take a look. In the meantime, you can try the registry image with `cp .env.prod .env && docker compose up` > Also do you have plan to support llama over openai models. Yes, we plan to support llama etc. We currently have support for models from OpenAI, Azure, Google's Vertex AI, Stability and a few others.
robertnishihara 3y ago
> we believe most companies prefer locally installed solutions to cloud based ones We've also seen a strong desire from businesses to manage models and compute on their own machines or in their own cloud accounts. This is often part of a hybrid strategy of using API products like OpenAI for rapid prototyping. The majority of (though not all) businesses we've seen tend to be quite comfortable using hosted API products for rapid prototyping and for proving out an initial version of their AI functionality. But in many cases, they want to complement that with the ability to manage models and compute themselves. The motivation here is often to reduce costs by using smaller / faster / cheaper fine-tuned open models. When we started Anyscale, customer demand led us to run training & inference workloads in our customers' cloud accounts. That way your data and code stays inside of your own cloud account. Now with all the progress in open models and the desire to rapidly prototype, we're complementing that with a fully-managed inference API where you can do inference with the Llama-2 models [1] (like the OpenAI API but for open models). [1] https://app.endpoints.anyscale.com/ https://app.endpoints.anyscale.com/
- deleted 3y ago
  [deleted]

by ajhai 3y ago

toomuchtodo 3y ago

Can you plug this together with tools like api2ai to create natural language defined workflow automations that interact with external APIs?

ajhai 3y ago
There is a generic HTTP API processor that can be used to call APIs as part of the app flow which should help invoke tools. Currently working on improving documentation so it is easy to get started with the project. We also have some features planned around function calling that should make it easy to natively integrate tools into the app flows.
cosbgn 3y ago
You can use unfetch.com to make API calls via LLMs and build automations. (I'm building it)
- scrum-treats 3y ago
  Is it possible to not use Google with unfetch.com?
  - cosbgn 3y ago
    Google is just so easy for login. No need to deal with password forgot, reset, email verification etc. But I'll add login via magic link soon.

gdiamos 3y ago

I find it interesting to see how competitive this space got so quickly. How do these stacks differentiate?

scrum-treats 3y ago
Quality and depth of particular types of training data is one difference. Another difference is inference tracking mechanisms within and between single-turn interactions (e.g., what does the human user "mean" with their prompt, what is the "correct" response, and how best can I return the "correct" response for this context; how much information do I cache from the previous turns, and how much if any of it is relevant to this current turn interaction).
peteradio 3y ago
[flagged]
- gdiamos 3y ago
  Thanks that made me smile. Take my upvote
  - omarfarooq 3y ago
    OP shouldn't be flagged.
lmeyerov 3y ago
With Louie.ai, there is a lot of work on specialization for the job, and I expect the same for others. We help with data analysis, so connecting enterprise & common data sources & DBs, hooking up data tools (GPU visuals, integrated code interpreter, ...), security controls, and the like, which is different from say a ChatGPT for lawyers or a straight up ChatGPT UI clone. Technically, as soon as the goal is to move beyond just text2gpt2screen, like multistep data wrangling & viz in the middle of a conversation, most tools technically struggle. Query quality also comes up, whether quality of the RAG, the fine tune, prompts, etc: each solves different problems.

bhanu423 3y ago

Interesting project - was trying it out, found an issue in building the image - have opened an issue on github - please take a look. Also do you have plan to support llama over openai models.

ajhai 3y ago
Thanks for the issue. Will take a look. In the meantime, you can try the registry image with `cp .env.prod .env && docker compose up` > Also do you have plan to support llama over openai models. Yes, we plan to support llama etc. We currently have support for models from OpenAI, Azure, Google's Vertex AI, Stability and a few others.

robertnishihara 3y ago

> we believe most companies prefer locally installed solutions to cloud based ones We've also seen a strong desire from businesses to manage models and compute on their own machines or in their own cloud accounts. This is often part of a hybrid strategy of using API products like OpenAI for rapid prototyping. The majority of (though not all) businesses we've seen tend to be quite comfortable using hosted API products for rapid prototyping and for proving out an initial version of their AI functionality. But in many cases, they want to complement that with the ability to manage models and compute themselves. The motivation here is often to reduce costs by using smaller / faster / cheaper fine-tuned open models. When we started Anyscale, customer demand led us to run training & inference workloads in our customers' cloud accounts. That way your data and code stays inside of your own cloud account. Now with all the progress in open models and the desire to rapidly prototype, we're complementing that with a fully-managed inference API where you can do inference with the Llama-2 models [1] (like the OpenAI API but for open models). [1] https://app.endpoints.anyscale.com/ https://app.endpoints.anyscale.com/

deleted 3y ago
[deleted]