3 ms·

I'm not sure I'm exactly at the edge of things, but I have 2 companies trying to setup regular meetings with me to be a beta customer. Both have promised I can

by swalsh 2y ago

I'm not sure I'm exactly at the edge of things, but I have 2 companies trying to setup regular meetings with me to be a beta customer. Both have promised I can help define a new product, but when I list my real problems... they aren't in the mission. Everyone wants to solve RAG (that's easy, don't need help) or they want to give me a gui I don't need, or wrap open source software like vllm. Or "solve privacy" (which usually comes in the form of masking... which surprise, that works for PII, but not PHI... I need the protected information). Want to solve a real problem, help me create custom benchmarks, clean my data, get my small parameter model to reason better etc.

benrutter 2y ago
This rings so true! I think it's natural whenever there's a new technology that a lot of start ups spring up with a vibe of "GenAI is cool, let's do something with that!", which is 100% the wrong way to go about building something. Starting by investing yourself fully into a given problem, and fixing it with the most appropriate tool (might be GenAI, might not) is much more likely to end in something people actually want or need. Doing the reverse, and trying to find an existing problem that matches a solution you've already picked is how you end up with hundreds of companies selling thin API wrappers for ChatGPT.
tinco 2y ago
We had the exact same problem before genAI became the next big thing. All the startups were selling generic fine tuning and labeling services both of which are super easy to build, and they didn't even work on our unique super high quality super high resolution 40TB dataset. Our problem was we had a real world problem and real data. All the startups were solving for imaginary problems and had no data.
- llm_trw 2y ago
  Sounds like you need a consultancy and not a startup to solve your problem.
  - BillyTheKing 2y ago
    maybe that's really where the business here is.. working through a whole bunch of custom data-sets and trying to generalise from there. It'll be hard to generalise all of it, but I'm sure there'll be pockets of functionality that can be shared across more than a single data-set. And maybe that's at the core of the issue here, namely that this service in its current form doesn't scale like b2c internet tech
    - hodgesrm 2y ago
      Most startups want to sell you the equivalent of those airline kiosks where you tag your own luggage, because they are cheap to deploy and don't have high labor costs. The problems you describe are labor-intensive and not easy to solve automatically. I think we will go back to tools combined with humans to solve at least some of them. So it's services and software.
  - swalsh 2y ago
    Probably the best play of the cycle if you're not Nvidia
statusfailed 2y ago
I'd love to know what your use case is that makes those things important to you - and what kind of benchmarks and cleaning tasks do you need to run? Also, what kind of evaluations for quality of reasoning do you use?
ozim 2y ago
That is basically every SaaS solution problem. You are not going to want to pay them for your custom development because you would like pay "generic solution price". That is why you have to pay for your own dev team as SaaS vendor is not going to be your custom development team - as your custom problem is not available for easy resell to other customers.
BobbyJo 2y ago
> Want to solve a real problem, help me create custom benchmarks, clean my data, get my small parameter model to reason better etc. I recently started a company with a friend if mine to do exactly this. Ive worked at a few AI startups over the last 8 years, and the problem everyone tackles independently (and poorly) is the long tail of dealing with input data that isn't great. You build a demo with sample data that works well, then you move on to real world uses and the data is suddenly... blehg.
- altdataseller 2y ago
  What are some examples of data theyre trying to use as input?
- ghaff 2y ago
  Ah. Cleaning data. The big trick going back to data warehousing.
  - BobbyJo 2y ago
    When you know the underlying consumer is an AI model, you can do a lot more to make the input data useful.

by swalsh 2y ago

benrutter 2y ago

This rings so true! I think it's natural whenever there's a new technology that a lot of start ups spring up with a vibe of "GenAI is cool, let's do something with that!", which is 100% the wrong way to go about building something. Starting by investing yourself fully into a given problem, and fixing it with the most appropriate tool (might be GenAI, might not) is much more likely to end in something people actually want or need. Doing the reverse, and trying to find an existing problem that matches a solution you've already picked is how you end up with hundreds of companies selling thin API wrappers for ChatGPT.

tinco 2y ago

We had the exact same problem before genAI became the next big thing. All the startups were selling generic fine tuning and labeling services both of which are super easy to build, and they didn't even work on our unique super high quality super high resolution 40TB dataset. Our problem was we had a real world problem and real data. All the startups were solving for imaginary problems and had no data.

llm_trw 2y ago
Sounds like you need a consultancy and not a startup to solve your problem.
- BillyTheKing 2y ago
  maybe that's really where the business here is.. working through a whole bunch of custom data-sets and trying to generalise from there. It'll be hard to generalise all of it, but I'm sure there'll be pockets of functionality that can be shared across more than a single data-set. And maybe that's at the core of the issue here, namely that this service in its current form doesn't scale like b2c internet tech
  - hodgesrm 2y ago
    Most startups want to sell you the equivalent of those airline kiosks where you tag your own luggage, because they are cheap to deploy and don't have high labor costs. The problems you describe are labor-intensive and not easy to solve automatically. I think we will go back to tools combined with humans to solve at least some of them. So it's services and software.
- swalsh 2y ago
  Probably the best play of the cycle if you're not Nvidia

statusfailed 2y ago

I'd love to know what your use case is that makes those things important to you - and what kind of benchmarks and cleaning tasks do you need to run? Also, what kind of evaluations for quality of reasoning do you use?

ozim 2y ago

That is basically every SaaS solution problem. You are not going to want to pay them for your custom development because you would like pay "generic solution price". That is why you have to pay for your own dev team as SaaS vendor is not going to be your custom development team - as your custom problem is not available for easy resell to other customers.

BobbyJo 2y ago

> Want to solve a real problem, help me create custom benchmarks, clean my data, get my small parameter model to reason better etc. I recently started a company with a friend if mine to do exactly this. Ive worked at a few AI startups over the last 8 years, and the problem everyone tackles independently (and poorly) is the long tail of dealing with input data that isn't great. You build a demo with sample data that works well, then you move on to real world uses and the data is suddenly... blehg.

altdataseller 2y ago
What are some examples of data theyre trying to use as input?
ghaff 2y ago
Ah. Cleaning data. The big trick going back to data warehousing.
- BobbyJo 2y ago
  When you know the underlying consumer is an AI model, you can do a lot more to make the input data useful.