4 ms·

Yeah I am noticing this as well. GPT enables you to do difficult things really easily, but then it is so expensive you would need to replace it with custom code

Yeah I am noticing this as well. GPT enables you to do difficult things really easily, but then it is so expensive you would need to replace it with custom code for any long term solution. For example: you could use GPT to parse a resume file, pull out work experience and return it as JSON. That would take minutes to setup using the GPT API and it would take weeks to build your own system, but GPT is so expensive that building your own system is totally worth it. Unless they can seriously reduce how expensive it is I don't see it replacing many existing solutions. Using GPT to parse text for a repetitive task is like using a backhoe to plant flowers.

beachy 4y ago
> For example: you could use GPT to parse a resume file, pull out work experience and return it as JSON. That would take minutes to setup using the GPT API and it would take weeks to build your own system, but GPT is so expensive that building your own system is totally worth it. True, but an HR SaaS vendor could use that to put on a compelling demo to a potential customer, stopping them from going to a competitor or otherwise benefiting. And anyway, without churning the numbers, for volumes of say 1M resumes (at which point you've achieved a lot of success) I can't quite believe it would be cheaper to build something when there is such a powerful solution available. Maybe once you are at 1G resumes... My bet is still no though.
- thewataccount 4y ago
  I work for a company with the web development team. We have ~6 software developers. I'd love to be able to just have people submit their resume's and extract the data from there, but instead I'm going to build a form and make applicants fill it out because chatGPT is going to be at least $0.05USD depending on the length of the resume. I'd also love to have mini summeries of order returns summerized in human form, but that also would cost 0.05USD per form. the tl;dr here is that there's a TON of usecases for a LLM outside of your core product (we sell clothes) - but we can't currently justify that cost. Compare that to the rapidly improving self-hosted solutions which don't cost 0.05USD for literally any query (and likely more for anything useful).
  - sitkack 4y ago
    5 cents. Per resume. $500 per 10k. 1-3 hours of a fully loaded engineers salary per year. You are being criminally cheap.
    - ericmcer 4y ago
      For a big company that is nothing but if you are bootstrapping and trying to acquire customers with an MVP racking up a $500 bill is frightening. What if you offer a free trial and blow up and end up with 5k+ bill.
      - beachy 4y ago
        By these maths, the $500 bill is for 10K resumes. To show an MVP to a customer you only need 10 resumes (or 1 in most demos I've been in). So 50c.
    - thewataccount 4y ago
      The problem is that it would take us the same amount of time to just add a form with django. Plus you have to handle failure cases, etc. And yeah I agree this would be a great use-case, and isn't that expensive. I'd like to do this in lots of places, and the problem is I have to convince my boss to pay for something that otherwise would have been free. The conversation would be "We have to add these fields to our model, and we either tell django to add a form for them, which will have 0 ongoing cost and no reliance on a third party, or we send the resume to openai, pay for them to process it, make some mechanism to sanity check what GPT is responding with, alert us if there's issues, and then put it into that model, and pay 5 cents per resume." > 1-3 hours of a fully loaded engineers salary per year. That's assuming 0 time to implement, and because of our framework it would take more hours to implement the openai solution (that's also more like 12 hours where we are). > $500 per 10k. I can't stress this enough - the alternative is 0$ per 10k. My boss wants to know why we would pay any money for a less reliable solution (GPT serialization is not nearly as reliable as a standard django form). I think within the next few years we'll be able to run the model locally and throw dozens of tasks just like this at the LLM, just not yet.
mejutoco 4y ago
You could use those examples to finetune a model only for resume-data extraction.

beachy 4y ago

> For example: you could use GPT to parse a resume file, pull out work experience and return it as JSON. That would take minutes to setup using the GPT API and it would take weeks to build your own system, but GPT is so expensive that building your own system is totally worth it. True, but an HR SaaS vendor could use that to put on a compelling demo to a potential customer, stopping them from going to a competitor or otherwise benefiting. And anyway, without churning the numbers, for volumes of say 1M resumes (at which point you've achieved a lot of success) I can't quite believe it would be cheaper to build something when there is such a powerful solution available. Maybe once you are at 1G resumes... My bet is still no though.

thewataccount 4y ago
I work for a company with the web development team. We have ~6 software developers. I'd love to be able to just have people submit their resume's and extract the data from there, but instead I'm going to build a form and make applicants fill it out because chatGPT is going to be at least $0.05USD depending on the length of the resume. I'd also love to have mini summeries of order returns summerized in human form, but that also would cost 0.05USD per form. the tl;dr here is that there's a TON of usecases for a LLM outside of your core product (we sell clothes) - but we can't currently justify that cost. Compare that to the rapidly improving self-hosted solutions which don't cost 0.05USD for literally any query (and likely more for anything useful).
- sitkack 4y ago
  5 cents. Per resume. $500 per 10k. 1-3 hours of a fully loaded engineers salary per year. You are being criminally cheap.
  - ericmcer 4y ago
    For a big company that is nothing but if you are bootstrapping and trying to acquire customers with an MVP racking up a $500 bill is frightening. What if you offer a free trial and blow up and end up with 5k+ bill.
    - beachy 4y ago
      By these maths, the $500 bill is for 10K resumes. To show an MVP to a customer you only need 10 resumes (or 1 in most demos I've been in). So 50c.
  - thewataccount 4y ago
    The problem is that it would take us the same amount of time to just add a form with django. Plus you have to handle failure cases, etc. And yeah I agree this would be a great use-case, and isn't that expensive. I'd like to do this in lots of places, and the problem is I have to convince my boss to pay for something that otherwise would have been free. The conversation would be "We have to add these fields to our model, and we either tell django to add a form for them, which will have 0 ongoing cost and no reliance on a third party, or we send the resume to openai, pay for them to process it, make some mechanism to sanity check what GPT is responding with, alert us if there's issues, and then put it into that model, and pay 5 cents per resume." > 1-3 hours of a fully loaded engineers salary per year. That's assuming 0 time to implement, and because of our framework it would take more hours to implement the openai solution (that's also more like 12 hours where we are). > $500 per 10k. I can't stress this enough - the alternative is 0$ per 10k. My boss wants to know why we would pay any money for a less reliable solution (GPT serialization is not nearly as reliable as a standard django form). I think within the next few years we'll be able to run the model locally and throw dozens of tasks just like this at the LLM, just not yet.

mejutoco 4y ago

You could use those examples to finetune a model only for resume-data extraction.