8 ms·

> For example: you could use GPT to parse a resume file, pull out work experience and return it as JSON. That would take minutes to setup using the GPT API and

by beachy 4y ago

> For example: you could use GPT to parse a resume file, pull out work experience and return it as JSON. That would take minutes to setup using the GPT API and it would take weeks to build your own system, but GPT is so expensive that building your own system is totally worth it. True, but an HR SaaS vendor could use that to put on a compelling demo to a potential customer, stopping them from going to a competitor or otherwise benefiting. And anyway, without churning the numbers, for volumes of say 1M resumes (at which point you've achieved a lot of success) I can't quite believe it would be cheaper to build something when there is such a powerful solution available. Maybe once you are at 1G resumes... My bet is still no though.

thewataccount 4y ago
I work for a company with the web development team. We have ~6 software developers. I'd love to be able to just have people submit their resume's and extract the data from there, but instead I'm going to build a form and make applicants fill it out because chatGPT is going to be at least $0.05USD depending on the length of the resume. I'd also love to have mini summeries of order returns summerized in human form, but that also would cost 0.05USD per form. the tl;dr here is that there's a TON of usecases for a LLM outside of your core product (we sell clothes) - but we can't currently justify that cost. Compare that to the rapidly improving self-hosted solutions which don't cost 0.05USD for literally any query (and likely more for anything useful).
- sitkack 4y ago
  5 cents. Per resume. $500 per 10k. 1-3 hours of a fully loaded engineers salary per year. You are being criminally cheap.
  - ericmcer 4y ago
    For a big company that is nothing but if you are bootstrapping and trying to acquire customers with an MVP racking up a $500 bill is frightening. What if you offer a free trial and blow up and end up with 5k+ bill.
    - beachy 4y ago
      By these maths, the $500 bill is for 10K resumes. To show an MVP to a customer you only need 10 resumes (or 1 in most demos I've been in). So 50c.
  - thewataccount 4y ago
    The problem is that it would take us the same amount of time to just add a form with django. Plus you have to handle failure cases, etc. And yeah I agree this would be a great use-case, and isn't that expensive. I'd like to do this in lots of places, and the problem is I have to convince my boss to pay for something that otherwise would have been free. The conversation would be "We have to add these fields to our model, and we either tell django to add a form for them, which will have 0 ongoing cost and no reliance on a third party, or we send the resume to openai, pay for them to process it, make some mechanism to sanity check what GPT is responding with, alert us if there's issues, and then put it into that model, and pay 5 cents per resume." > 1-3 hours of a fully loaded engineers salary per year. That's assuming 0 time to implement, and because of our framework it would take more hours to implement the openai solution (that's also more like 12 hours where we are). > $500 per 10k. I can't stress this enough - the alternative is 0$ per 10k. My boss wants to know why we would pay any money for a less reliable solution (GPT serialization is not nearly as reliable as a standard django form). I think within the next few years we'll be able to run the model locally and throw dozens of tasks just like this at the LLM, just not yet.
    - marketerinland 4y ago
      There are excellent commercial AI resume parsers already - Affinda.com being one. Not expensive and takes minutes to implement.
  - yunwal 4y ago
    Also you could likely use GPT3.5 for this and still get near perfect results.
    - thewataccount 4y ago
      > near perfect results. I have tried GPT3.5 and GPT4 for this type of task - the "near perfect results" is really problematic because you need to verify that it's likely correct, notify you if there's issues, and even then you aren't 100% sure that it selected the correct first/last name. This is compared to a standard html form. Which is.... very reliable and (for us) automatically has error handling built in, including alerts to us if there's a 504.

by beachy 4y ago

thewataccount 4y ago

I work for a company with the web development team. We have ~6 software developers. I'd love to be able to just have people submit their resume's and extract the data from there, but instead I'm going to build a form and make applicants fill it out because chatGPT is going to be at least $0.05USD depending on the length of the resume. I'd also love to have mini summeries of order returns summerized in human form, but that also would cost 0.05USD per form. the tl;dr here is that there's a TON of usecases for a LLM outside of your core product (we sell clothes) - but we can't currently justify that cost. Compare that to the rapidly improving self-hosted solutions which don't cost 0.05USD for literally any query (and likely more for anything useful).

sitkack 4y ago
5 cents. Per resume. $500 per 10k. 1-3 hours of a fully loaded engineers salary per year. You are being criminally cheap.
- ericmcer 4y ago
  For a big company that is nothing but if you are bootstrapping and trying to acquire customers with an MVP racking up a $500 bill is frightening. What if you offer a free trial and blow up and end up with 5k+ bill.
  - beachy 4y ago
    By these maths, the $500 bill is for 10K resumes. To show an MVP to a customer you only need 10 resumes (or 1 in most demos I've been in). So 50c.
- thewataccount 4y ago
  The problem is that it would take us the same amount of time to just add a form with django. Plus you have to handle failure cases, etc. And yeah I agree this would be a great use-case, and isn't that expensive. I'd like to do this in lots of places, and the problem is I have to convince my boss to pay for something that otherwise would have been free. The conversation would be "We have to add these fields to our model, and we either tell django to add a form for them, which will have 0 ongoing cost and no reliance on a third party, or we send the resume to openai, pay for them to process it, make some mechanism to sanity check what GPT is responding with, alert us if there's issues, and then put it into that model, and pay 5 cents per resume." > 1-3 hours of a fully loaded engineers salary per year. That's assuming 0 time to implement, and because of our framework it would take more hours to implement the openai solution (that's also more like 12 hours where we are). > $500 per 10k. I can't stress this enough - the alternative is 0$ per 10k. My boss wants to know why we would pay any money for a less reliable solution (GPT serialization is not nearly as reliable as a standard django form). I think within the next few years we'll be able to run the model locally and throw dozens of tasks just like this at the LLM, just not yet.
  - marketerinland 4y ago
    There are excellent commercial AI resume parsers already - Affinda.com being one. Not expensive and takes minutes to implement.
- yunwal 4y ago
  Also you could likely use GPT3.5 for this and still get near perfect results.
  - thewataccount 4y ago
    > near perfect results. I have tried GPT3.5 and GPT4 for this type of task - the "near perfect results" is really problematic because you need to verify that it's likely correct, notify you if there's issues, and even then you aren't 100% sure that it selected the correct first/last name. This is compared to a standard html form. Which is.... very reliable and (for us) automatically has error handling built in, including alerts to us if there's a 504.