3 ms·

Hi HN, I’m Varun, cofounder of Exafunction. We’re excited about all the use cases of large NLP models like text generation and language comprehension, but found

Hi HN, I’m Varun, cofounder of Exafunction. We’re excited about all the use cases of large NLP models like text generation and language comprehension, but found the existing offerings to be quite expensive. Our goal is to serve these models in a cost effective way at scale. The first model we’re starting with is GPT-J, the recently released open source large language model by EleutherAI. It’s comparable to OpenAI’s GPT-3 Curie in performance. With lots of optimizations, we are able to serve the model for 6x cheaper per token compared to OpenAI. We have a simple HTTPS API that you can try out for free. Here’s a link with the details - https://www.exafunction.com/nlp-api https://www.exafunction.com/nlp-api We’re also looking into supporting custom and fine-tuned models since it’s required to get good performance for a lot of applications. Especially for smaller models like BERT, we’re excited about the cost savings we can deliver for users who use many different models. In this case, we’re 100 - 1000x cheaper than others offering similar inference APIs like Hugging Face. We hope this is interesting to you all and please email me at varun@exafunction.com if there’s a model you’d like to see supported.

GDC7 5y ago
Hi! Please could you explain some use cases? OpenAI is very strict with GPT-3. Say a company wanted to use Exafunction for email interactions or Discord interactions or even SMS interactions. What would be the setup? How would that work with Exafunction? Thank YOU!
- varunkmohan 5y ago
  Unlike OpenAI, which is serving a closed-source model, anyone can validate for themselves (even without using our service) whether open-source models like GPT-J are suitable for a particular application. We'd like to work with the open source community to make this kind of technology more accessible, essentially as an infrastructure provider. If you have some specific use case in mind feel free to shoot us an email at hello@exafunction.com
- robbedpeter 5y ago
  https://nlpcloud.io/effectively-using-gpt-j-gpt-neo-gpt-3-alternatives-few-shot-learning.html https://nlpcloud.io/effectively-using-gpt-j-gpt-neo-gpt-3-al... Here's a wonderfully written breakdown of some functionality that can be achieved with gpt-j and gpt-neo - it's not ExaFunction specific, but the language models are the same. The exafunction api is fairly simple and powerful. Kudos!
d13 5y ago
But how fast? I see other companies advertising 1.5 second response times for GPT-J, but a now assume that’s average per token, as for, say, a 200 word prompt response times can be well over a minute during heavy use periods like weekends when everyone is hitting their side projects.
- varunkmohan 5y ago
  For the public API, we should be under 100ms per token but don't have a strict guarantee. If you have a strict SLA, you can talk to us and we can get it to as low as 20ms per token at high load.

GDC7 5y ago

Hi! Please could you explain some use cases? OpenAI is very strict with GPT-3. Say a company wanted to use Exafunction for email interactions or Discord interactions or even SMS interactions. What would be the setup? How would that work with Exafunction? Thank YOU!

varunkmohan 5y ago
Unlike OpenAI, which is serving a closed-source model, anyone can validate for themselves (even without using our service) whether open-source models like GPT-J are suitable for a particular application. We'd like to work with the open source community to make this kind of technology more accessible, essentially as an infrastructure provider. If you have some specific use case in mind feel free to shoot us an email at hello@exafunction.com
robbedpeter 5y ago
https://nlpcloud.io/effectively-using-gpt-j-gpt-neo-gpt-3-alternatives-few-shot-learning.html https://nlpcloud.io/effectively-using-gpt-j-gpt-neo-gpt-3-al... Here's a wonderfully written breakdown of some functionality that can be achieved with gpt-j and gpt-neo - it's not ExaFunction specific, but the language models are the same. The exafunction api is fairly simple and powerful. Kudos!

d13 5y ago

But how fast? I see other companies advertising 1.5 second response times for GPT-J, but a now assume that’s average per token, as for, say, a 200 word prompt response times can be well over a minute during heavy use periods like weekends when everyone is hitting their side projects.

varunkmohan 5y ago
For the public API, we should be under 100ms per token but don't have a strict guarantee. If you have a strict SLA, you can talk to us and we can get it to as low as 20ms per token at high load.