4 ms·

There's a bit of snake oil in all of that. By any means ML is a very specialized subfield, where you need solid math basis and a deep understanding of the scie

There's a bit of snake oil in all of that. By any means ML is a very specialized subfield, where you need solid math basis and a deep understanding of the science behind it all. But I struggle to see the same thing for AI. If by "AI engineers" you mean someone who builds an LLM*, then it's very much just ML. If you mean someone who integrates with the LLM someone else built, then it's very much just backend work. Sure, you might need a few days to understand a few concepts, I've integrated with Paypal in the past and that doesn't make me a payment for engineer. * wont even get into the argument of labelling LLMs with AI

fragmede 3y ago
Well, and frontend work. ChatGPT wouldn't been anywhere near the product it is without the webterface. As far as PayPal integration, no but it makes you the subject matter expert (SME) in the room above a room full of people who aren't, and maybe aren't even developers.
senko 3y ago
If you have React engineers, why couldn't you have AI engineers? I think the post is right to try to delineate the roles of "builds LLMs or other ML models" and "uses those models", as (as you noted) the skills required are very different (vs. "creates a JS framework" vs "uses a JS framework" are the same skills, just on another level). "AI engineer" is, as you say, inaccurate, but as swyx says, it's least cringy of the alternatives. > Sure, you might need a few days to understand a few concepts, I've integrated with Paypal in the past and that doesn't make me a payment for engineer. No, but the post explicitly calls out the breadth of knowledge that the future AI engineers will have to have. It's not something you can pick up in a few days, even now, if you want to be on top of things. I can look up some docs and hook up static file hosting on S3 and that doesn't make an AWS engineer. But there are people who full time work on providing solutions using AWS-everything and have built their entire careers (and companies) on top of that specialization. The difference in scale is the difference in job description.
- ignoramous 3y ago
  > If you have React engineers, why couldn't you have AI engineers? We most definitely could. That's one way to prop up a cottage industry, stay relevant, command hefty salaries, and maintain that charade of importance through complexity by creating Frankenstein features in search of problems and promotions.
  - bedobi 3y ago
    like the canned messages that are increasingly finding themselves into any app where you can send messages to people, be it email, whatsapp, tinder, whatever "Hey John! I see you like traveling, I love traveling too! What are some of the most memorable trips you've had?" like, who would want to receive such an obviously canned message lol? soon, it will just be the apps talking to each other, no humans required
    - jackphilson 3y ago
      And that's how you get black mirror - hang the DJ
- 29athrowaway 3y ago
  Because AI as terminology is vague and can mean anything. Graph search is AI, pathfinding is AI, decision trees are AI, fuzzy logic is AI, expert systems are AI, machine learning is AI. Just say LLM analyst or whatever.
- gkvtqcvmcs 3y ago
  Yea i agree, maybe calling them LLM Engineers as against AI Engineers would go down better with folks though? ML enables LLM's that enable AI, So its definitely semantics
selalipop 3y ago
Maybe the term "engineer" did the concept a disservice, but prompt engineering has a lot of parallels to the field of UX. Currently there's a lot intuition involved so it can come across as made up, but there are novel concepts which meaningfully affect the end quality of what you make, and it takes time to learn and/or discover them. As time goes I expect our understanding of what underlies "good prompts" will start to bridge the gap from intuition to science much like how UX bridged into neuroscience and psychology. If you understand things like attention and logits that's already kind of happening: you can use that knowledge to identify gaps in the abilities of LLMs and start to bridge those gaps. - People are convinced that future LLMs will obsolete prompt engineering. To me that'd be like people from the 90s thinking computers are going to obsolete UX because more powerful computers will be better at making user interfaces, and in turn anyone will be able to do it. In some ways they'd be right: Today you don't need a UX expert at PARC to integrate a WYSIWYG interface into your product. Computers got so powerful that in milliseconds we can download libraries that implement the interface and render it across any form factor you can imagine. So now a WYSIWYG on your contact form is nothing. But as computers got more powerful they could do new things, so UX advanced onto improving how we interface with those new things. Things like the Vision Pro will unlock new areas of UX based on novel capabilities they posses. I think people are making a similar mistake with LLMs: they're focused on this idea that we'll just do the current things but better with more powerful models. But the more powerful models will be something we can "prompt engineer" into usecases we haven't even considered yet. (I also built notionsmith.ai and I'd argue it fits into that bucket a bit)
godelski 3y ago
> where you need solid math basis and a deep understanding of the science behind it all. I __REALLY__ really wish this were true. But I'll be honest, I know quite a number of researchers at high level institutions (FAANG and top 10 unis) that don't understand things like probability distributions or the difference between likelihood and probability. There's a lot of "interpretability" left on the table simply through not understanding some basic mathematics, let along advanced (high dimensional statistics, differential geometry, set theory, etc). The AI engineering often "needs" less of an understanding. But I don't think this is a good thing. I specifically have been vocal about how this is going to cause real world harm. Forget the AGI, just look at how people are using models today without any understanding. How people think you can synthesize new data without considering diversity of that data[0], can create "self healing code" that will generate high quality and good code[1,2], how people think LLMs understands causality[3,4], or just how fucking hard evaluation really is[5] (I really cannot stress this last one enough). There is a serious crisis in ML right now, and it is also the thing that made it explode in funding: hype. I don't think this is a bubble in the sense that AI will go away, but I think if we aren't careful with how we deal with this then it isn't unlikely to see heavy governmental restrictions placed on these things. Plus, a lot of us are pretty confident that just learning through data is not enough to get to AGI. It just isn't a high enough level of abstraction, besides being a pain (see the semantic deduplication comments about generation). But academia is even railroaded into SOTA chasing because that's what conferences like. NLP as an entire field right now is almost entirely composed of people just tuning big models instead of developing novel architectures (if you don't win, you struggle to get published despite differing factors). We let big labs spend massive amounts of compute to compare to little labs who can get similar performance with a hundredth, but don't publish those works. It is the curse of benchmarkism and it is maddening. Honestly, a lot of times I feel like a crazy person for bringing this up. Because when I say "ML needs a solid math basis and deep understanding of the science behind it" everyone agrees, but when the rubber hits the road and I suggest mathematical solutions to resolve these, I'm laughed at or told it is unnecessary. [0] https://news.ycombinator.com/item?id=36509816 https://news.ycombinator.com/item?id=36509816 [1] https://news.ycombinator.com/item?id=36297867 https://news.ycombinator.com/item?id=36297867 [2] https://news.ycombinator.com/item?id=35806152 https://news.ycombinator.com/item?id=35806152 [3] https://news.ycombinator.com/item?id=36036859 https://news.ycombinator.com/item?id=36036859 [4] https://www.cs.helsinki.fi/u/ahyvarin/papers/NN99.pdf https://www.cs.helsinki.fi/u/ahyvarin/papers/NN99.pdf [5] https://news.ycombinator.com/item?id=36116939 https://news.ycombinator.com/item?id=36116939
skwirl 3y ago
You could have just read the article to see what the author meant by AI Engineers. You also would have seen that the entire article was addressing a point you tried to make, and you could have responded to the arguments made in the article instead of, ya know, just sounding off based on having read nothing but the headline
- charles_f 3y ago
  I have read the article (though I will admit, quite quickly). The author's definition for AI is by any means not the single one out there, as they outlined themselves. I disagree with the premise that this will be the job of the decade, and I disagree with the premise that it is inherently different from the rest of software as a service. This is a tool, that can be used to solve certain problems, I doubt there is a useful differentiation. That's my point of view, it is antagonistic to what is presented there, and I don't see why you think it departs from usefully commenting on the article.

fragmede 3y ago

Well, and frontend work. ChatGPT wouldn't been anywhere near the product it is without the webterface. As far as PayPal integration, no but it makes you the subject matter expert (SME) in the room above a room full of people who aren't, and maybe aren't even developers.

senko 3y ago

If you have React engineers, why couldn't you have AI engineers? I think the post is right to try to delineate the roles of "builds LLMs or other ML models" and "uses those models", as (as you noted) the skills required are very different (vs. "creates a JS framework" vs "uses a JS framework" are the same skills, just on another level). "AI engineer" is, as you say, inaccurate, but as swyx says, it's least cringy of the alternatives. > Sure, you might need a few days to understand a few concepts, I've integrated with Paypal in the past and that doesn't make me a payment for engineer. No, but the post explicitly calls out the breadth of knowledge that the future AI engineers will have to have. It's not something you can pick up in a few days, even now, if you want to be on top of things. I can look up some docs and hook up static file hosting on S3 and that doesn't make an AWS engineer. But there are people who full time work on providing solutions using AWS-everything and have built their entire careers (and companies) on top of that specialization. The difference in scale is the difference in job description.

ignoramous 3y ago
> If you have React engineers, why couldn't you have AI engineers? We most definitely could. That's one way to prop up a cottage industry, stay relevant, command hefty salaries, and maintain that charade of importance through complexity by creating Frankenstein features in search of problems and promotions.
- bedobi 3y ago
  like the canned messages that are increasingly finding themselves into any app where you can send messages to people, be it email, whatsapp, tinder, whatever "Hey John! I see you like traveling, I love traveling too! What are some of the most memorable trips you've had?" like, who would want to receive such an obviously canned message lol? soon, it will just be the apps talking to each other, no humans required
  - jackphilson 3y ago
    And that's how you get black mirror - hang the DJ
29athrowaway 3y ago
Because AI as terminology is vague and can mean anything. Graph search is AI, pathfinding is AI, decision trees are AI, fuzzy logic is AI, expert systems are AI, machine learning is AI. Just say LLM analyst or whatever.
gkvtqcvmcs 3y ago
Yea i agree, maybe calling them LLM Engineers as against AI Engineers would go down better with folks though? ML enables LLM's that enable AI, So its definitely semantics

selalipop 3y ago

Maybe the term "engineer" did the concept a disservice, but prompt engineering has a lot of parallels to the field of UX. Currently there's a lot intuition involved so it can come across as made up, but there are novel concepts which meaningfully affect the end quality of what you make, and it takes time to learn and/or discover them. As time goes I expect our understanding of what underlies "good prompts" will start to bridge the gap from intuition to science much like how UX bridged into neuroscience and psychology. If you understand things like attention and logits that's already kind of happening: you can use that knowledge to identify gaps in the abilities of LLMs and start to bridge those gaps. - People are convinced that future LLMs will obsolete prompt engineering. To me that'd be like people from the 90s thinking computers are going to obsolete UX because more powerful computers will be better at making user interfaces, and in turn anyone will be able to do it. In some ways they'd be right: Today you don't need a UX expert at PARC to integrate a WYSIWYG interface into your product. Computers got so powerful that in milliseconds we can download libraries that implement the interface and render it across any form factor you can imagine. So now a WYSIWYG on your contact form is nothing. But as computers got more powerful they could do new things, so UX advanced onto improving how we interface with those new things. Things like the Vision Pro will unlock new areas of UX based on novel capabilities they posses. I think people are making a similar mistake with LLMs: they're focused on this idea that we'll just do the current things but better with more powerful models. But the more powerful models will be something we can "prompt engineer" into usecases we haven't even considered yet. (I also built notionsmith.ai and I'd argue it fits into that bucket a bit)

godelski 3y ago

> where you need solid math basis and a deep understanding of the science behind it all. I __REALLY__ really wish this were true. But I'll be honest, I know quite a number of researchers at high level institutions (FAANG and top 10 unis) that don't understand things like probability distributions or the difference between likelihood and probability. There's a lot of "interpretability" left on the table simply through not understanding some basic mathematics, let along advanced (high dimensional statistics, differential geometry, set theory, etc). The AI engineering often "needs" less of an understanding. But I don't think this is a good thing. I specifically have been vocal about how this is going to cause real world harm. Forget the AGI, just look at how people are using models today without any understanding. How people think you can synthesize new data without considering diversity of that data[0], can create "self healing code" that will generate high quality and good code[1,2], how people think LLMs understands causality[3,4], or just how fucking hard evaluation really is[5] (I really cannot stress this last one enough). There is a serious crisis in ML right now, and it is also the thing that made it explode in funding: hype. I don't think this is a bubble in the sense that AI will go away, but I think if we aren't careful with how we deal with this then it isn't unlikely to see heavy governmental restrictions placed on these things. Plus, a lot of us are pretty confident that just learning through data is not enough to get to AGI. It just isn't a high enough level of abstraction, besides being a pain (see the semantic deduplication comments about generation). But academia is even railroaded into SOTA chasing because that's what conferences like. NLP as an entire field right now is almost entirely composed of people just tuning big models instead of developing novel architectures (if you don't win, you struggle to get published despite differing factors). We let big labs spend massive amounts of compute to compare to little labs who can get similar performance with a hundredth, but don't publish those works. It is the curse of benchmarkism and it is maddening. Honestly, a lot of times I feel like a crazy person for bringing this up. Because when I say "ML needs a solid math basis and deep understanding of the science behind it" everyone agrees, but when the rubber hits the road and I suggest mathematical solutions to resolve these, I'm laughed at or told it is unnecessary. [0] https://news.ycombinator.com/item?id=36509816 https://news.ycombinator.com/item?id=36509816 [1] https://news.ycombinator.com/item?id=36297867 https://news.ycombinator.com/item?id=36297867 [2] https://news.ycombinator.com/item?id=35806152 https://news.ycombinator.com/item?id=35806152 [3] https://news.ycombinator.com/item?id=36036859 https://news.ycombinator.com/item?id=36036859 [4] https://www.cs.helsinki.fi/u/ahyvarin/papers/NN99.pdf https://www.cs.helsinki.fi/u/ahyvarin/papers/NN99.pdf [5] https://news.ycombinator.com/item?id=36116939 https://news.ycombinator.com/item?id=36116939

skwirl 3y ago

You could have just read the article to see what the author meant by AI Engineers. You also would have seen that the entire article was addressing a point you tried to make, and you could have responded to the arguments made in the article instead of, ya know, just sounding off based on having read nothing but the headline

charles_f 3y ago
I have read the article (though I will admit, quite quickly). The author's definition for AI is by any means not the single one out there, as they outlined themselves. I disagree with the premise that this will be the job of the decade, and I disagree with the premise that it is inherently different from the rest of software as a service. This is a tool, that can be used to solve certain problems, I doubt there is a useful differentiation. That's my point of view, it is antagonistic to what is presented there, and I don't see why you think it departs from usefully commenting on the article.