3 ms·

Offer self-hosted and I would buy. Do not assume that companies are willing to put ALL of their intellectual property into your hands. Even if you would not be

Offer self-hosted and I would buy. Do not assume that companies are willing to put ALL of their intellectual property into your hands. Even if you would not be some startup where any sysadmin could steal and sell my data any time without you even noticing it, you will get hacked just like everyone else that stores interesting data. The data you have access to is absolutely perfect for the global data blackmailing gangs. As soon as you are successful, you will have every black hat hacker and their dog knocking on your doors.

jychang 1y ago
Onyx.app has a self hosted option. I just did the docker setup yesterday. It’s not a great home user option imo but seems like it’s functional for enterprise.
- Fischgericht 1y ago
  Just had a quick look - while they have that self-hosting option, they still assume you will use a cloud LLM. I started digging because I got confused of them not mentioning any GPU when it comes to resource requirements. There is some documentation on using it fully self-hosted including the LLM, but the emphasis here is on "some". To be clear: I am looking at this from a CEO perspective, not a "I will play with it in my spare time" nerd one.
- jxmesth 1y ago
  Going off on a tangent here but, does anyone have a good guide on how to set up an AI/LLM/GPT chatbot/agent for a small business? Not looking to spend millions but a couple thousand are alright. TIA
  - pdksam 1y ago
    Hmm I thought this would be a competitive space already i.e. integrating llm chat bots on any website
  - tinodb 1y ago
    Open-webui works well for us: https://github.com/open-webui/open-webui https://github.com/open-webui/open-webui
    - jxmesth 1y ago
      u/tinodb, thanks
rakeshraghu 1y ago
controlcore.io was brought to the market for the same exact reason. Not AI Powered, but to control AI and its interactions with your Data, APIs,. Applications etc. And yes, we just give our service as a self-hostable solution. However is the encryption and SOC compliance be, we want our clients to know that none of their internal data or interaction transaction leave their control.
eambutu 1y ago
Yep, makes a lot of sense. We architected our system to be easy to self-host & open-source in the future for this very reason, though we decided to launch with hosted because it's easier to improve and iterate.
- jxmesth 1y ago
  u/eambutu, any timeline for the self-hosted version? Also willing to buy.
- Fischgericht 1y ago
  Understood. Not my startup, but I would have started the other way round. Businesses that would be willing to pay (a lot) for such a benefit often will be very conservative. In Germany the majority of medium sized businesses using SAP for example still refuse to be moved to SAP's cloud instead of on-premise. C-Level types typically are not worried putting their email credentials etc into Outlook cloud and getting hacked this way. They are used to "everything is in the cloud". However, as soon as you mention, depending on the type of business "patents", "sales contacts", "production plans" C's will change their mind. In Germany, where I am originally come from, all of these businesses are worried about their trade secrets to end up in China, and rightly so. As self-hosting is very complex you could either make good money with consulting (but this means setting up tech teams in all target markets around the globe, using actual competent humans), or by selling it as a plug&play appliance. With that appliance simply being a rack server with a suitable GPU installed. And again, for your business strategy the long-term risk of pretty much everyone trying to hack you on a daily basis appears too high to me. You might not have on your radar how serious industry spionage is. You will definitely have a fake utility company worker coming into your offices, trying to plug in a USB keylogger into some PC while nobody is looking. As an example, proven strategy: Find targets internet uplink. Cut it. Customer calls ISP for help. You then send a fake ISP technician that arrives before the real one does. You put a data exfiltration dongle between the modem and the LAN. You then fix the cut outdoor line. Customer is happy that you have fixed it. Later the actual ISP guy arrives. Everyone will be a bit confused that the problem was already fixed, but then agree that it's probably just the ISP once again having screwed up their resource management. Works pretty much every time.

jychang 1y ago

Onyx.app has a self hosted option. I just did the docker setup yesterday. It’s not a great home user option imo but seems like it’s functional for enterprise.

Fischgericht 1y ago
Just had a quick look - while they have that self-hosting option, they still assume you will use a cloud LLM. I started digging because I got confused of them not mentioning any GPU when it comes to resource requirements. There is some documentation on using it fully self-hosted including the LLM, but the emphasis here is on "some". To be clear: I am looking at this from a CEO perspective, not a "I will play with it in my spare time" nerd one.
jxmesth 1y ago
Going off on a tangent here but, does anyone have a good guide on how to set up an AI/LLM/GPT chatbot/agent for a small business? Not looking to spend millions but a couple thousand are alright. TIA
- pdksam 1y ago
  Hmm I thought this would be a competitive space already i.e. integrating llm chat bots on any website
- tinodb 1y ago
  Open-webui works well for us: https://github.com/open-webui/open-webui https://github.com/open-webui/open-webui
  - jxmesth 1y ago
    u/tinodb, thanks

rakeshraghu 1y ago

controlcore.io was brought to the market for the same exact reason. Not AI Powered, but to control AI and its interactions with your Data, APIs,. Applications etc. And yes, we just give our service as a self-hostable solution. However is the encryption and SOC compliance be, we want our clients to know that none of their internal data or interaction transaction leave their control.

eambutu 1y ago

Yep, makes a lot of sense. We architected our system to be easy to self-host & open-source in the future for this very reason, though we decided to launch with hosted because it's easier to improve and iterate.

jxmesth 1y ago
u/eambutu, any timeline for the self-hosted version? Also willing to buy.
Fischgericht 1y ago
Understood. Not my startup, but I would have started the other way round. Businesses that would be willing to pay (a lot) for such a benefit often will be very conservative. In Germany the majority of medium sized businesses using SAP for example still refuse to be moved to SAP's cloud instead of on-premise. C-Level types typically are not worried putting their email credentials etc into Outlook cloud and getting hacked this way. They are used to "everything is in the cloud". However, as soon as you mention, depending on the type of business "patents", "sales contacts", "production plans" C's will change their mind. In Germany, where I am originally come from, all of these businesses are worried about their trade secrets to end up in China, and rightly so. As self-hosting is very complex you could either make good money with consulting (but this means setting up tech teams in all target markets around the globe, using actual competent humans), or by selling it as a plug&play appliance. With that appliance simply being a rack server with a suitable GPU installed. And again, for your business strategy the long-term risk of pretty much everyone trying to hack you on a daily basis appears too high to me. You might not have on your radar how serious industry spionage is. You will definitely have a fake utility company worker coming into your offices, trying to plug in a USB keylogger into some PC while nobody is looking. As an example, proven strategy: Find targets internet uplink. Cut it. Customer calls ISP for help. You then send a fake ISP technician that arrives before the real one does. You put a data exfiltration dongle between the modem and the LAN. You then fix the cut outdoor line. Customer is happy that you have fixed it. Later the actual ISP guy arrives. Everyone will be a bit confused that the problem was already fixed, but then agree that it's probably just the ISP once again having screwed up their resource management. Works pretty much every time.