5 ms·

It feels absolutely amazing to build an AI startup right now. It's as if your product automatically becomes cheaper, more reliable, and more scalable with each

It feels absolutely amazing to build an AI startup right now. It's as if your product automatically becomes cheaper, more reliable, and more scalable with each new major model release. - We first struggled with limited context windows [solved] - We had issues with consistent JSON ouput [solved] - We had rate limiting and performance issues for the large 3rd party models [solved] - Hosting our own OSS models for small and medium complex tasks was a pain [solved] Obivously every startup still needs to build up defensibility and focus on differentiating with everything “non-AI”.

paxys 3y ago
We are going to quickly reach the point where most of these AI startups (which do nothing but provide thin wrappers on top of public LLMs) aren't going to be needed at all. The differentiation will need to come from the value of the end product put in front of customers, not the AI backend.
- layble 3y ago
  Sure, in the same way SaaS companies are just thin wrappers on top of databases and the open web.
  - imjonse 3y ago
    You will find that a disproportionately large amount of work and innovation in an AI product is in the backing model (GPT, Mixtral, etc.). While there's a huge amount of work in databases and the open web, SaaS products typically add a lot more than a thin API layer and a shiny website (well some do but you know what I mean)
    - tomrod 3y ago
      I'd argue the comment before you is describing accessibility, features, and services -- yes, the core component has a wrapper, but that wrapper differentiates the use.
- wongarsu 3y ago
  The same happened to image recognition. We have great algorithms for many years now. You can't make a company out of having the best image recognition algorithm, but you absolutely can make a company out of a device that spots defects in the paintjob in a car factory, or that spots concrete cracks in the tunnel segments used by a tunnel boring machine, or by building a wildlife camera that counts wildlife and exports that to a central website. All of them just fine-tune existing algorithms, but the value delivered is vastly different. Or you can continue selling shovels. Still lots of expensive labeling services out there, to stay in the image-recognition parallel
  - pradn 3y ago
    The key thing is AI models are services not products. The real world changes, so you have to change your model. Same goes for new training data (examples, yes/no labels, feedback from production use), updating biases (compliance, changing societal mores). And running models in a highly-available way is also expertise. Not every company wants to be in the ML-ops business.
  - HeatrayEnjoyer 3y ago
    The dynamic does seem to be different with the newer systems. Larger more general systems are better than small specialized models. GPT-4 is SOTA at OCR and sentiment classification, for example.
sleepingreset 3y ago
If you don't mind, I'm trying to experiment w/ local models more. Just now getting into messing w/ these but I'm struggling to come up w/ good use cases. Would you happen to know of any cool OSS model projects that might be good inspiration for a side project? Wondering what most people use these local models for
- sosuke 3y ago
  No ideas about side projects or anything "productive" but for a concrete example look at SillyTavern. Making fictional characters. Finding narratives, stories, role-play for tabletop games. You can even have group chats of AI characters interacting. No good use cases for profit but plenty right now for exploration and fun.
- wing-_-nuts 3y ago
  One idea that I've been mulling over; Given how controllable linux is from the command line, I think it would be somewhat easy to set up a voice to text to a local LLM that could control pretty much everything on command. It would flat out embarass alexa. Imagine 'Hal play a movie', or 'Hal play some music' and it's all running locally, with your content.
  - mikegreenberg 3y ago
    There are a few projects doing this. This one piqued my interest as having a potentially nice UX after some maturity. https://github.com/OpenInterpreter/01 https://github.com/OpenInterpreter/01
  - octobus2021 3y ago
    ...until one day it hallucinates or has existential crisis and does something funny with "dd" or "fdisk/sfdisk"... :)
- ajcp 3y ago
  I've got a couple I've done and it's been really enjoyable. I think the real value in using local models is exposing them to personal/unique information that only you have, thus getting novel and unique outcomes that no public model could provide. 1. Project 1 - Self Knowledge - Download/extract all of my emails and populate into a vector database, like Chroma[0] - For each prompt do a search of the vector store and return N number of matches - Provide both prompt and search result to LLM, instructing it to use the search result as context or in the answer itself. 2. Project 2 - Chat with a Friend - I exported the chat and text history between me and a good friend that passed away - I created a vector store of our chat history in chunks, each consisting of 6 back-and-forth interactions - When I "chat" with the LLM the a search is first conducted for matching chunks from the vector store and then using those as "style" and knowledge context for a response. Optional: You can use SillyTavern[1] for a more "rich" chat experience The above lets me chat, at least superficially, with my friend. It's nice for simple interactions and banter; I've found it to be a positive and reflective experience. 0. https://www.trychroma.com/ https://www.trychroma.com/ 1. https://sillytavernai.com/ https://sillytavernai.com/
deleted 3y ago
[deleted]
milansuk 3y ago
The progress is insane. A few days ago I started being very impressed with LLM coding skills. I wanted Golang code, instead of Python, which you can see in many demos. The prompt was: Write a Golang func, which accepts the path into a .gpx file and outputs a JSON string with points(x=tolal distance in km, y=elevation). Don't use any library.
jasonjmcghee 3y ago
How are you approaching hosting? vLLM?
neillyons 3y ago
> We had issues with consistent JSON ouput [solved] It says the JSON output is constrained via their platform (on la Plateforme). Does that mean JSON output is only available in the hosted version? Are there any small models that can be self hosted that output valid JSON.
- ajcp 3y ago
  > Does that mean JSON output is only available in the [self]-hosted version? I would assume so. They probably constrain JSON output so that the JSON response doesn't bork the front-end/back-end of la Plateforme itself as it moves through their code back to you.
- PhilippGille 3y ago
  > Are there any small models that can be self hosted that output valid JSON. Yes, for example this one is optimized for function calling and JSON output: https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B
yodsanklai 3y ago
> It's as if your product automatically becomes cheaper, more reliable, and more scalable with each new major model release. and so do your competitor's products.
- samus 3y ago
  Any business idea built almost exclusively on AI, without adding much value, is doomed from the start. AI is not good enough to make humans obsolete yet. But a well finetuned model can for sure augment what individual humans can do.

paxys 3y ago

We are going to quickly reach the point where most of these AI startups (which do nothing but provide thin wrappers on top of public LLMs) aren't going to be needed at all. The differentiation will need to come from the value of the end product put in front of customers, not the AI backend.

layble 3y ago
Sure, in the same way SaaS companies are just thin wrappers on top of databases and the open web.
- imjonse 3y ago
  You will find that a disproportionately large amount of work and innovation in an AI product is in the backing model (GPT, Mixtral, etc.). While there's a huge amount of work in databases and the open web, SaaS products typically add a lot more than a thin API layer and a shiny website (well some do but you know what I mean)
  - tomrod 3y ago
    I'd argue the comment before you is describing accessibility, features, and services -- yes, the core component has a wrapper, but that wrapper differentiates the use.
wongarsu 3y ago
The same happened to image recognition. We have great algorithms for many years now. You can't make a company out of having the best image recognition algorithm, but you absolutely can make a company out of a device that spots defects in the paintjob in a car factory, or that spots concrete cracks in the tunnel segments used by a tunnel boring machine, or by building a wildlife camera that counts wildlife and exports that to a central website. All of them just fine-tune existing algorithms, but the value delivered is vastly different. Or you can continue selling shovels. Still lots of expensive labeling services out there, to stay in the image-recognition parallel
- pradn 3y ago
  The key thing is AI models are services not products. The real world changes, so you have to change your model. Same goes for new training data (examples, yes/no labels, feedback from production use), updating biases (compliance, changing societal mores). And running models in a highly-available way is also expertise. Not every company wants to be in the ML-ops business.
- HeatrayEnjoyer 3y ago
  The dynamic does seem to be different with the newer systems. Larger more general systems are better than small specialized models. GPT-4 is SOTA at OCR and sentiment classification, for example.

sleepingreset 3y ago

If you don't mind, I'm trying to experiment w/ local models more. Just now getting into messing w/ these but I'm struggling to come up w/ good use cases. Would you happen to know of any cool OSS model projects that might be good inspiration for a side project? Wondering what most people use these local models for

sosuke 3y ago
No ideas about side projects or anything "productive" but for a concrete example look at SillyTavern. Making fictional characters. Finding narratives, stories, role-play for tabletop games. You can even have group chats of AI characters interacting. No good use cases for profit but plenty right now for exploration and fun.
wing-_-nuts 3y ago
One idea that I've been mulling over; Given how controllable linux is from the command line, I think it would be somewhat easy to set up a voice to text to a local LLM that could control pretty much everything on command. It would flat out embarass alexa. Imagine 'Hal play a movie', or 'Hal play some music' and it's all running locally, with your content.
- mikegreenberg 3y ago
  There are a few projects doing this. This one piqued my interest as having a potentially nice UX after some maturity. https://github.com/OpenInterpreter/01 https://github.com/OpenInterpreter/01
- octobus2021 3y ago
  ...until one day it hallucinates or has existential crisis and does something funny with "dd" or "fdisk/sfdisk"... :)
ajcp 3y ago
I've got a couple I've done and it's been really enjoyable. I think the real value in using local models is exposing them to personal/unique information that only you have, thus getting novel and unique outcomes that no public model could provide. 1. Project 1 - Self Knowledge - Download/extract all of my emails and populate into a vector database, like Chroma[0] - For each prompt do a search of the vector store and return N number of matches - Provide both prompt and search result to LLM, instructing it to use the search result as context or in the answer itself. 2. Project 2 - Chat with a Friend - I exported the chat and text history between me and a good friend that passed away - I created a vector store of our chat history in chunks, each consisting of 6 back-and-forth interactions - When I "chat" with the LLM the a search is first conducted for matching chunks from the vector store and then using those as "style" and knowledge context for a response. Optional: You can use SillyTavern[1] for a more "rich" chat experience The above lets me chat, at least superficially, with my friend. It's nice for simple interactions and banter; I've found it to be a positive and reflective experience. 0. https://www.trychroma.com/ https://www.trychroma.com/ 1. https://sillytavernai.com/ https://sillytavernai.com/

deleted 3y ago

[deleted]

milansuk 3y ago

The progress is insane. A few days ago I started being very impressed with LLM coding skills. I wanted Golang code, instead of Python, which you can see in many demos. The prompt was: Write a Golang func, which accepts the path into a .gpx file and outputs a JSON string with points(x=tolal distance in km, y=elevation). Don't use any library.

jasonjmcghee 3y ago

How are you approaching hosting? vLLM?

neillyons 3y ago

> We had issues with consistent JSON ouput [solved] It says the JSON output is constrained via their platform (on la Plateforme). Does that mean JSON output is only available in the hosted version? Are there any small models that can be self hosted that output valid JSON.

ajcp 3y ago
> Does that mean JSON output is only available in the [self]-hosted version? I would assume so. They probably constrain JSON output so that the JSON response doesn't bork the front-end/back-end of la Plateforme itself as it moves through their code back to you.
PhilippGille 3y ago
> Are there any small models that can be self hosted that output valid JSON. Yes, for example this one is optimized for function calling and JSON output: https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B

yodsanklai 3y ago

> It's as if your product automatically becomes cheaper, more reliable, and more scalable with each new major model release. and so do your competitor's products.

samus 3y ago
Any business idea built almost exclusively on AI, without adding much value, is doomed from the start. AI is not good enough to make humans obsolete yet. But a well finetuned model can for sure augment what individual humans can do.