4 ms·

I can see a world where "ML Engineer" (or similar) is someone that's hired to solve a known problem (whether it be with classifiers, LLMs, neural nets, etc), wh

by cj 3y ago

I can see a world where "ML Engineer" (or similar) is someone that's hired to solve a known problem (whether it be with classifiers, LLMs, neural nets, etc), whereas a "AI Engineer" (or whatever the title) is hired to figure out how the hell to capitalize on the AI hype, without a specific problem to solve. IMO right now we're entering the "Peak of Inflated Expectations" in Gartner's hype cycle model. https://en.wikipedia.org/wiki/Gartner_hype_cycle#/media/File:Hype-Cycle-General.png https://en.wikipedia.org/wiki/Gartner_hype_cycle#/media/File... Lots of companies want to jump on the AI bandwagon, but they don't really know what to do or how to leverage it. What the LLM community needs now is for companies to leverage and productize these LLM models for truly game changing use cases. If that doesn't happen soon, the hype will start to fade and lose momentum ("trough of disillusionment"). I'd really love to see some killer use cases emerge soon.

yacine_ 3y ago
The right way to leverage AI is to use it to do ye olde engineering, faster. Actually, most AI use cases are in fact that! Ye olde engineering - data pipelines, automation. Stuff we've been doing since before I was born. Everything else - transformative tech, will hit the open domain fairly quickly, is my bet. Similar to databases.
richardw 3y ago
I’ve been thinking the leading edge is already in the trough of disillusionment! At least on HN etc. We see lots of reports of limitations on HN. Those in the know don’t trust it nearly as much as the public and all the CEO’s of businesses drooling to install AI and have the money go up. Going to have to dig our way through those issues to get real value.
- moffkalast 3y ago
  Well you have to consider the inherent bias on HN, in that the trough of disillusionment is HN's default state of mind regarding literally every topic. It's really rare to see a thread where most of the comments aren't negative.
  - staticman2 3y ago
    Ha, when GPT 4 launched the default HN post seemed to be saying LLM's were about to exceed human intelligence and the singularity was upon us. The disillusionment seemed to happen pretty fast.
rvz 3y ago
We are indeed at the "Peak of Inflated Expectations" in the Garter hype cycle. Everyone is screaming that we are out of the AI winter and throwing LLMs at every problem. > What the LLM community needs now is for companies to leverage and productize these LLM models for truly game changing use cases. The killer serious use-case has always been summarization of existing text. That's it. Everything else is a constant flow of creative bull-shitting from a black box AI requiring triple checking everything before using the output, meaning that the output can't be trusted. > If that doesn't happen soon, the hype will start to fade and lose momentum ("trough of disillusionment"). I think they will realize that there is more to "AI" than LLMs, just like the hype with CNNs and the like.
- visarga 3y ago
  Not just summarisation, information extraction in general. LLMs are great data normalisers.
  - z3c0 3y ago
    No. No they are not. Do not keep proliferating this idea, as there are real consequences. Your input only informs the likelihood of the output. There is no model actually extrapolating rules from the information, so the premise of LLMs being extractive is 100% verifiably false. They summarize well due to being handed a roughly correct arrangement of tokens to mimic the order of. This should not be confused with extraction. To put it simply, keep falling for that, and it'll bite you in the ass.
    - visarga 3y ago
      You are talking from a top-down point of view. Of course they can make an error any time, after all they stochastically sample the output. All approaches to information extraction are upper bounded at 90-95% accuracy, I have extensive experience with this task and run many evaluations on invoices, receipts, forms and other doctypes. Human in the loop is still required. But in practice you can rely on good copying and excellent ability to parse names, addresses and various values as good or better than other approaches. LLMs reduce the number of corrections, and they are easier to deploy - instead of labeling hundreds of documents you can just query with the field name.
swyx 3y ago
(author here) There are at least 4 killer apps ($100m/yr revenue potential) so far: 1. Generative Text for writing - Jasper AI going 0 to $75m ARR in 2 years 2. Generative Art for non-artists - Midjourney has by some accounts $80m ARR 3. Copilot for knowledge workers - GitHub’s Copilot has roughly 50-80m ARR as well 4. Conversational AI UX - ChatGPT probably has >$100m ARR by now, Bing Chat has brought $m's worth of attention to Bing (More on agents as the immature #5: https://www.latent.space/p/agents https://www.latent.space/p/agents) what else do you need to see to believe? (geniune question)
- echelon 3y ago
  The longer HN continues to doubt in AI's deliverables, the bigger our revenues and our moats can grow. Don't tell them. (It's not that hard to hit $1M ARR with a good AI product. So many classes of new products and solutions have opened up.)
  - constantly 3y ago
    I’m sure I could find a similar comment about cryptocurrency or NFTs. There’s always snake oil salesmen.
  - _ea1k 3y ago
    I remember thinking this about Tesla too. They built a fairly massive automaker while everyone kept saying their whole approach was fundamentally doomed.
- cj 3y ago
  > what else do you need to see to believe? (geniune question) It's not that I don't believe in LLMs. It's that the level of media attention AI is getting isn't backed by the same level of real world use cases, yet. (All of the use cases you listed are awesome, no denying that, but I don't think those alone justify the amount of hype in mainstream media)
  - swyx 3y ago
    i mean, theres nothing i can do about mainstream media hype, but i guess my main point is this is a growing field with real money and utility behind it, and so will professionalize. if I am correct on that then AI Engineer will be a thing (because it is Least Bad title for the thing)

by cj 3y ago

yacine_ 3y ago

The right way to leverage AI is to use it to do ye olde engineering, faster. Actually, most AI use cases are in fact that! Ye olde engineering - data pipelines, automation. Stuff we've been doing since before I was born. Everything else - transformative tech, will hit the open domain fairly quickly, is my bet. Similar to databases.

richardw 3y ago

I’ve been thinking the leading edge is already in the trough of disillusionment! At least on HN etc. We see lots of reports of limitations on HN. Those in the know don’t trust it nearly as much as the public and all the CEO’s of businesses drooling to install AI and have the money go up. Going to have to dig our way through those issues to get real value.

moffkalast 3y ago
Well you have to consider the inherent bias on HN, in that the trough of disillusionment is HN's default state of mind regarding literally every topic. It's really rare to see a thread where most of the comments aren't negative.
- staticman2 3y ago
  Ha, when GPT 4 launched the default HN post seemed to be saying LLM's were about to exceed human intelligence and the singularity was upon us. The disillusionment seemed to happen pretty fast.

rvz 3y ago

We are indeed at the "Peak of Inflated Expectations" in the Garter hype cycle. Everyone is screaming that we are out of the AI winter and throwing LLMs at every problem. > What the LLM community needs now is for companies to leverage and productize these LLM models for truly game changing use cases. The killer serious use-case has always been summarization of existing text. That's it. Everything else is a constant flow of creative bull-shitting from a black box AI requiring triple checking everything before using the output, meaning that the output can't be trusted. > If that doesn't happen soon, the hype will start to fade and lose momentum ("trough of disillusionment"). I think they will realize that there is more to "AI" than LLMs, just like the hype with CNNs and the like.

visarga 3y ago
Not just summarisation, information extraction in general. LLMs are great data normalisers.
- z3c0 3y ago
  No. No they are not. Do not keep proliferating this idea, as there are real consequences. Your input only informs the likelihood of the output. There is no model actually extrapolating rules from the information, so the premise of LLMs being extractive is 100% verifiably false. They summarize well due to being handed a roughly correct arrangement of tokens to mimic the order of. This should not be confused with extraction. To put it simply, keep falling for that, and it'll bite you in the ass.
  - visarga 3y ago
    You are talking from a top-down point of view. Of course they can make an error any time, after all they stochastically sample the output. All approaches to information extraction are upper bounded at 90-95% accuracy, I have extensive experience with this task and run many evaluations on invoices, receipts, forms and other doctypes. Human in the loop is still required. But in practice you can rely on good copying and excellent ability to parse names, addresses and various values as good or better than other approaches. LLMs reduce the number of corrections, and they are easier to deploy - instead of labeling hundreds of documents you can just query with the field name.

swyx 3y ago

(author here) There are at least 4 killer apps ($100m/yr revenue potential) so far: 1. Generative Text for writing - Jasper AI going 0 to $75m ARR in 2 years 2. Generative Art for non-artists - Midjourney has by some accounts $80m ARR 3. Copilot for knowledge workers - GitHub’s Copilot has roughly 50-80m ARR as well 4. Conversational AI UX - ChatGPT probably has >$100m ARR by now, Bing Chat has brought $m's worth of attention to Bing (More on agents as the immature #5: https://www.latent.space/p/agents https://www.latent.space/p/agents) what else do you need to see to believe? (geniune question)

echelon 3y ago
The longer HN continues to doubt in AI's deliverables, the bigger our revenues and our moats can grow. Don't tell them. (It's not that hard to hit $1M ARR with a good AI product. So many classes of new products and solutions have opened up.)
- constantly 3y ago
  I’m sure I could find a similar comment about cryptocurrency or NFTs. There’s always snake oil salesmen.
- _ea1k 3y ago
  I remember thinking this about Tesla too. They built a fairly massive automaker while everyone kept saying their whole approach was fundamentally doomed.
cj 3y ago
> what else do you need to see to believe? (geniune question) It's not that I don't believe in LLMs. It's that the level of media attention AI is getting isn't backed by the same level of real world use cases, yet. (All of the use cases you listed are awesome, no denying that, but I don't think those alone justify the amount of hype in mainstream media)
- swyx 3y ago
  i mean, theres nothing i can do about mainstream media hype, but i guess my main point is this is a growing field with real money and utility behind it, and so will professionalize. if I am correct on that then AI Engineer will be a thing (because it is Least Bad title for the thing)