5 ms·

Show HN: ServerlessAI – Build, scale, and monetize AI apps without back end

86 points by HeavenFox 2y ago

Hello HN, I’ve always loved building frontend-only apps—those you can prototype over a weekend, host for free on GitHub Pages, and scale to millions of users. Unfortunately, AI-enabled apps complicate things, as exposing your OpenAI key to the world is obviously a no-go. This also means mobile developers often have to run their own servers. That’s why I built ServerlessAI, an API gateway that lets you securely call multiple AI providers directly from client side using OpenAI-compatible APIs. You can authenticate users through any identity provider, like Google or Apple, and set per-user request or spending quotas. You can also define an allowlist for endpoints and models. To monetize, you can apply different quotas for various user tiers. To start, I recommend checking out our tutorials, where we walk you through building a complete, deployment-ready AI app in 5 minutes. We’ve got tutorials for React, Next.js, and iOS: https://serverlessai.dev/docs/tutorials/ https://serverlessai.dev/docs/tutorials/ Our long term vision is to offer the best toolkit for AI developers at every stage of their project’s lifecycle. If OpenAI / Anthropic / etc are AWS, we want to be the Supabase / Upstash / etc. We are building optional out-of-box tools for authentication and payment management, so you can roll out your prototype faster. In the future, we want to provide the best prompt engineering tools for fine-tuning, A/B testing, and backtesting, as well as the best observability tools. We’d love to hear your feedback. Thanks for stopping by!

sf-wy 2y ago
Great idea! I like the ergonomics of this for the developer-side, it's easy to add and puts the onus on the developer to have a robust auth system that avoids users creating 1000s of accounts so they can get unlimited LLM access. One challenge on frontend-only apps is if the prompt is proprietary then this will be exposed unless you will then offer prompt templating or prompt mapping on your side i.e. the frontend says prompt: Template_123 and then this maps to the actual prompt somehow. Prompting is important still and maybe for a while so having the internals externally available could be sensitive.
- ProofHouse 2y ago
  I disagree. These sorts of pricing structures rule out consumer usage quickly due to high numbers of users. Have flat usage fees and get rid of these per user limits. Both doesn’t make sense unless you want to turn away any app that has many users. I’d also recommend they clean up the copy of what they offer (expand on the why). Other than that looks cool
  - HeavenFox 2y ago
    Thanks for the feedback! Pricing is something we are iterating on. Our intention is that we only make money if our customers make money - which is why only authenticated users count towards the limit. If pricing is preventing anyone from using our product, please shoot me an email (in my HN profile) and we'd love to hear about your use case! Good call on the marketing copy! We will do some revisions!
- HeavenFox 2y ago
  Thanks for the feedback! We are building prompt templates right now. Besides the security benefits you mentioned, this can also enable developers to tweak prompts without redeployment, run A/B tests, and evaluate different models. It's an incredibly powerful tool!
friendly_chap 2y ago
Hah! Nice idea! I built something with a similar mindset but instead of calling cloud AI providers my aim is to provide a self-hostable complete AI platform: https://github.com/singulatron/singulatron https://github.com/singulatron/singulatron I know that might sound like putting the server back to serverless. But I would say it's being your own serverless provider - once you have the platform installed on your servers you can build frontend-only AI apps on top. Hope you don't mind the self-plug. Your approach definitely a ton of advantages when starting out (no infra to manage etc).
madamelic 2y ago
To potentially save you some headache, take a look at serverless.com and weigh the likelihood they come after you about that name if you are planning on making this a business. (And yes, I hate their name too. I don't honestly know how defendable an entire technology term actually is. It also results in terrible Googling.)
- hn_throwaway_99 2y ago
  IANAL, but since "Serverless, Inc." chose a generic term for their trademark, usually this means only very specific instances of that mark are enforceable, e.g. "Serverless Framework", "serverless.com" and "Serverless, Inc." But if a company were to say "We make it really easy to use serverless technologies in the cloud!" (note lower case s), that shouldn't be infringing because it's just using serverless as a generic, descriptive term. "ServerlessAI" and "serverlessai.dev" shouldn't be infringing because they incorporate the generic term in their mark, not the Serverless Framework-specific term. Of course, this means that ServerlessAI would have the same issues you point out - a less-defensible mark and poor Google results. Also, in the US pretty much anyone can sue for anything, so even if you're in the right it can be an expensive headache to have to defend yourself.
  - HeavenFox 2y ago
    Thank you both! In a way this could be a nice problem to have - attracting the attention of BigCo means we are big enough right ;) But yes, we will definitely be prepared for that possibility!
    - sleepybrett 2y ago
      Nice problem to have if you happen to be serverlessai's lawyers, if you happen to be serverlessai's investors, probably not.
jdmoreira 2y ago
This is a great idea. You should market to app devs as well. I would also build this on top of firebase marketplace: https://extensions.dev https://extensions.dev
- HeavenFox 2y ago
  Thank you! We do believe app developers will find this valuable, and we are also working on IAP integration. Sadly I am not an app developer, so if anyone have any suggestions on how I can serve this community better, I'm eager to hear from you! My email is in my HN profile. Will look into Firebase Marketplace! That is a great suggestion!
ukuina 2y ago
This is cool, congrats on launching! How is it different from Puter AI, which offers auth + free inference? https://docs.puter.com/AI/chat/ https://docs.puter.com/AI/chat/
- ent101 2y ago
  Founder of Puter here. Thank you very much for mentioning us!
  - simple10 2y ago
    Can you elaborate a bit on the target market and use cases for Puter as it relates to AI? This is the first I've heard of Puter. Looks super interesting.
- HeavenFox 2y ago
  Thanks! Despite my best Google-fu, I didn't find Puter in my market research :) It is a very interesting project for sure! Looking at Puter's offering, at the end of the day, we serve the same goal of making developer's life easier. ServerlessAI is more narrowly focused on AI use cases, while Puter is providing a more generic app runtime. Both have their best use cases!
CuriouslyC 2y ago
If you don't want to pay for this service, keytrustee.org does this for free.
stroupwaffle 2y ago
Please hire a real artist those graphics on the home page are disturbing.
jrm4 2y ago
So, I'll be honest, I don't understand this market. I get that one can be profitable selling shovels during the gold rush, sure. But I have trouble understanding who is knowledgeable/dedicated enough to try to get their AI app going, but would pay to abstract/outsource this part of the chain. (I suppose, relately, I have trouble understanding why anyone would just sort of presume OpenAI would be forever the best backend here as well?)
- HeavenFox 2y ago
  Here's my hypothesis: since we are still in the gold rush phase of AI, going to market early and pivoting quickly is imperative. Therefore, it is worthwhile to focus on your differentiating factor and outsource everything else. Plus we use the OpenAI API format, so there isn't vendor lock-in - though we hope we provide enough value for you to stay! And you are right on the money that OpenAI may not be the best backend forever. That's why we also support Anthropic, Groq, and Mistral.
- bongodongobob 2y ago
  A friend of mine works for a very large company and he was telling me about an AI startup they bought for (he thinks) in the ballpark of 8 digits. After they were on boarded he was able to dig into what it was. It's literally a GPT backend with system prompts and RAG. If you have a good sales team you can print money for doing almost nothing right now.
- __alias 2y ago
  I've seen an AI yearbook photos styled app print money that was serverless. There's lots of novelty apps atm that, as the other commented stated, just want to get to market as soon as possible to validate an idea
- kajecounterhack 2y ago
  Doesn't everyone ask this question about Okta, like what am I doing farming out my user login?
- sailingparrot 2y ago
  Lots of company are doing this exact business, they all have insane churn rate. Biggest use case seems to be for people who want to prototype something quickly, they don’t yet want to bother with managing the infra and don’t mind the extra cost since this will be running at small scale. But if the experiment is successful, then the customer churns as it (most of the time) makes little sense to scale ML on serverless. And if the experiment is not successful, they are obviously also going to churn.
crooked-v 2y ago
So what's supposed to make this service better than OpenRouter (https://openrouter.ai https://openrouter.ai), which has an extensive list of available models with transparent pricing for all of them?
- jstanley 2y ago
  Perhaps: > You can authenticate users through any identity provider, like Google or Apple, and set per-user request or spending quotas. You can also define an allowlist for endpoints and models. To monetize, you can apply different quotas for various user tiers.
jedberg 2y ago
Hey, not sure where you're running your backend for this, but this is the perfect use case for my company's product (check out my profile if you're interested). We make a serverless compute platform that doesn't charge for wait time, so it would be signifigantly cheaper than EC2 or AWS Lambda for example.
davidashe 2y ago
AI plus serverless…without blockchain??
sgarland 2y ago
> Focus on What Matters I really wish companies and people would stop saying things like this. “Focus on what’s important,” “focus on product,” etc. It’s frankly insulting to the admittedly small percentage of tech who deeply understand and care about the infrastructure that makes all of this magic possible. If someone isn’t managing servers, your serverless magic will not work. If someone isn’t managing DBs, your product will not work. There is no getting around this problem; at some level, someone has to know how computers actually work, and to defend them against the abuses levied by people who neither know nor care to learn.
- hinkley 2y ago
  I’m binge watching House MD right now, and the number of distinct ways he shits on people for basically saying, “focus on what’s important” is impressive. The latest one I recall was something like, “we’ll be sure to use the real medicine instead of the placebos.”
echoangle 2y ago
Are we seriously calling someone else’s server „serverless“? Serverless would be local compute for me, not calling an API I’m paying someone else for.
- jedberg 2y ago
  Serverless in today's world means you don't have to manage the infrastructure and you pay nothing if it's idle. It doesn't mean "no servers" and really never did.

86 points by HeavenFox 2y ago

sf-wy 2y ago

Great idea! I like the ergonomics of this for the developer-side, it's easy to add and puts the onus on the developer to have a robust auth system that avoids users creating 1000s of accounts so they can get unlimited LLM access. One challenge on frontend-only apps is if the prompt is proprietary then this will be exposed unless you will then offer prompt templating or prompt mapping on your side i.e. the frontend says prompt: Template_123 and then this maps to the actual prompt somehow. Prompting is important still and maybe for a while so having the internals externally available could be sensitive.

ProofHouse 2y ago
I disagree. These sorts of pricing structures rule out consumer usage quickly due to high numbers of users. Have flat usage fees and get rid of these per user limits. Both doesn’t make sense unless you want to turn away any app that has many users. I’d also recommend they clean up the copy of what they offer (expand on the why). Other than that looks cool
- HeavenFox 2y ago
  Thanks for the feedback! Pricing is something we are iterating on. Our intention is that we only make money if our customers make money - which is why only authenticated users count towards the limit. If pricing is preventing anyone from using our product, please shoot me an email (in my HN profile) and we'd love to hear about your use case! Good call on the marketing copy! We will do some revisions!
HeavenFox 2y ago
Thanks for the feedback! We are building prompt templates right now. Besides the security benefits you mentioned, this can also enable developers to tweak prompts without redeployment, run A/B tests, and evaluate different models. It's an incredibly powerful tool!

friendly_chap 2y ago

Hah! Nice idea! I built something with a similar mindset but instead of calling cloud AI providers my aim is to provide a self-hostable complete AI platform: https://github.com/singulatron/singulatron https://github.com/singulatron/singulatron I know that might sound like putting the server back to serverless. But I would say it's being your own serverless provider - once you have the platform installed on your servers you can build frontend-only AI apps on top. Hope you don't mind the self-plug. Your approach definitely a ton of advantages when starting out (no infra to manage etc).

madamelic 2y ago

To potentially save you some headache, take a look at serverless.com and weigh the likelihood they come after you about that name if you are planning on making this a business. (And yes, I hate their name too. I don't honestly know how defendable an entire technology term actually is. It also results in terrible Googling.)

hn_throwaway_99 2y ago
IANAL, but since "Serverless, Inc." chose a generic term for their trademark, usually this means only very specific instances of that mark are enforceable, e.g. "Serverless Framework", "serverless.com" and "Serverless, Inc." But if a company were to say "We make it really easy to use serverless technologies in the cloud!" (note lower case s), that shouldn't be infringing because it's just using serverless as a generic, descriptive term. "ServerlessAI" and "serverlessai.dev" shouldn't be infringing because they incorporate the generic term in their mark, not the Serverless Framework-specific term. Of course, this means that ServerlessAI would have the same issues you point out - a less-defensible mark and poor Google results. Also, in the US pretty much anyone can sue for anything, so even if you're in the right it can be an expensive headache to have to defend yourself.
- HeavenFox 2y ago
  Thank you both! In a way this could be a nice problem to have - attracting the attention of BigCo means we are big enough right ;) But yes, we will definitely be prepared for that possibility!
  - sleepybrett 2y ago
    Nice problem to have if you happen to be serverlessai's lawyers, if you happen to be serverlessai's investors, probably not.

jdmoreira 2y ago

This is a great idea. You should market to app devs as well. I would also build this on top of firebase marketplace: https://extensions.dev https://extensions.dev

HeavenFox 2y ago
Thank you! We do believe app developers will find this valuable, and we are also working on IAP integration. Sadly I am not an app developer, so if anyone have any suggestions on how I can serve this community better, I'm eager to hear from you! My email is in my HN profile. Will look into Firebase Marketplace! That is a great suggestion!

ukuina 2y ago

This is cool, congrats on launching! How is it different from Puter AI, which offers auth + free inference? https://docs.puter.com/AI/chat/ https://docs.puter.com/AI/chat/

ent101 2y ago
Founder of Puter here. Thank you very much for mentioning us!
- simple10 2y ago
  Can you elaborate a bit on the target market and use cases for Puter as it relates to AI? This is the first I've heard of Puter. Looks super interesting.
HeavenFox 2y ago
Thanks! Despite my best Google-fu, I didn't find Puter in my market research :) It is a very interesting project for sure! Looking at Puter's offering, at the end of the day, we serve the same goal of making developer's life easier. ServerlessAI is more narrowly focused on AI use cases, while Puter is providing a more generic app runtime. Both have their best use cases!

CuriouslyC 2y ago

If you don't want to pay for this service, keytrustee.org does this for free.

stroupwaffle 2y ago

Please hire a real artist those graphics on the home page are disturbing.

jrm4 2y ago

So, I'll be honest, I don't understand this market. I get that one can be profitable selling shovels during the gold rush, sure. But I have trouble understanding who is knowledgeable/dedicated enough to try to get their AI app going, but would pay to abstract/outsource this part of the chain. (I suppose, relately, I have trouble understanding why anyone would just sort of presume OpenAI would be forever the best backend here as well?)

HeavenFox 2y ago
Here's my hypothesis: since we are still in the gold rush phase of AI, going to market early and pivoting quickly is imperative. Therefore, it is worthwhile to focus on your differentiating factor and outsource everything else. Plus we use the OpenAI API format, so there isn't vendor lock-in - though we hope we provide enough value for you to stay! And you are right on the money that OpenAI may not be the best backend forever. That's why we also support Anthropic, Groq, and Mistral.
bongodongobob 2y ago
A friend of mine works for a very large company and he was telling me about an AI startup they bought for (he thinks) in the ballpark of 8 digits. After they were on boarded he was able to dig into what it was. It's literally a GPT backend with system prompts and RAG. If you have a good sales team you can print money for doing almost nothing right now.
__alias 2y ago
I've seen an AI yearbook photos styled app print money that was serverless. There's lots of novelty apps atm that, as the other commented stated, just want to get to market as soon as possible to validate an idea
kajecounterhack 2y ago
Doesn't everyone ask this question about Okta, like what am I doing farming out my user login?
sailingparrot 2y ago
Lots of company are doing this exact business, they all have insane churn rate. Biggest use case seems to be for people who want to prototype something quickly, they don’t yet want to bother with managing the infra and don’t mind the extra cost since this will be running at small scale. But if the experiment is successful, then the customer churns as it (most of the time) makes little sense to scale ML on serverless. And if the experiment is not successful, they are obviously also going to churn.

crooked-v 2y ago

So what's supposed to make this service better than OpenRouter (https://openrouter.ai https://openrouter.ai), which has an extensive list of available models with transparent pricing for all of them?

jstanley 2y ago
Perhaps: > You can authenticate users through any identity provider, like Google or Apple, and set per-user request or spending quotas. You can also define an allowlist for endpoints and models. To monetize, you can apply different quotas for various user tiers.

jedberg 2y ago

Hey, not sure where you're running your backend for this, but this is the perfect use case for my company's product (check out my profile if you're interested). We make a serverless compute platform that doesn't charge for wait time, so it would be signifigantly cheaper than EC2 or AWS Lambda for example.

davidashe 2y ago

AI plus serverless…without blockchain??

sgarland 2y ago

> Focus on What Matters I really wish companies and people would stop saying things like this. “Focus on what’s important,” “focus on product,” etc. It’s frankly insulting to the admittedly small percentage of tech who deeply understand and care about the infrastructure that makes all of this magic possible. If someone isn’t managing servers, your serverless magic will not work. If someone isn’t managing DBs, your product will not work. There is no getting around this problem; at some level, someone has to know how computers actually work, and to defend them against the abuses levied by people who neither know nor care to learn.

hinkley 2y ago
I’m binge watching House MD right now, and the number of distinct ways he shits on people for basically saying, “focus on what’s important” is impressive. The latest one I recall was something like, “we’ll be sure to use the real medicine instead of the placebos.”

echoangle 2y ago

Are we seriously calling someone else’s server „serverless“? Serverless would be local compute for me, not calling an API I’m paying someone else for.

jedberg 2y ago
Serverless in today's world means you don't have to manage the infrastructure and you pay nothing if it's idle. It doesn't mean "no servers" and really never did.