15 ms·

The AI research job market

207 points by sebg 3y ago

softwaredoug 3y ago
I just watched The Big Short I felt a bit eery how much the groupthink around mortgages matched todays AI hype. The unwillingness to listen to critical advice. To question the value. Data science depts who measure such things are often vilified. It’s a kind of depressing job landscape in this way. You either go with the, often top down, groupthink against the face of measured evidence, or you’re labeled a cynical naysayer and your career suffers.
- chasd00 3y ago
  just use your skills, i.e. mop, and soak of the cash.
- sillysaurusx 3y ago
  Well, don’t be cynical about it. One thing that I think a lot of us could benefit from is learning how to present a point in a way that people will listen. The key is to harness that hype — use your point to unlock some pent up energy for a new direction, rather than merely try to say it can’t work. A lot of endeavors can work, even if only a little. Find some aspect of it that can, and flip the problem around. Even if the whole thing is mostly bogus, is there a small part that isn’t? Latch onto that. Or go elsewhere. The wonderful part about AI is that the whole world’s problems are up for grabs. Part of why there’s so much unfounded hype is because of how many real advances have recently become possible. This period in history will never come again. It’s also a rare time in history that an individual can make lots of progress. Most of us need to be a part of big groups to do anything worthwhile, in most fields. But in this case lone wolves often have the upper hand over established organizations.
  - x86x87 3y ago
    Yes. Don't tell the emperor that he has no clothes. Instead innovate and introduce a hybrid between his current clothes and old fashion textiles. Who know? In time they may end up covering their junk.
  - softwaredoug 3y ago
    To be clear, I'm mostly excited. But I also think its reasonable to be skeptical and try to educate stakeholders on the realities of the situation. My controversial take on AI is its actually a better time to take things slow, experiment, study, see what works. Not dump TONS of money and cash and get too distracted. Because nobody (besides big tech) has fully figured out how to make a product that makes a profit. Its not clear users want a chatbot (aside from ChatGPT)... But things could change.
    - chung8123 3y ago
      I think the issue is, right now AI is a race. We have the Microsoft, Google, Meta, Apple, and Amazon's of the world with their massive compute and bankrolls racing to see who can build the biggest moat around an AI service. The massive upfront spending is hoping to hit the winning lottery ticket and spending slowly may leave you out of the drawing. As compute costs and requirements come down LLMs will be ubiquitous everywhere.
      - akomtu 3y ago
        It's the Manhattan Project 2.0. The AI, once created, won't be that hard to replicate, but those who fail to create it early, will be sidetracked later. The race among big tech is the american way of doing such projects: fund a few companies, let them compete and pick the winner.
        __loam 3y ago
        This industry is so arrogant lmao. The Manhattan project? Get fucking real.
  - tayo42 3y ago
    > It’s also a rare time in history that an individual can make lots of progress. Most of us need to be a part of big groups to do anything worthwhile, in most fields. But in this case lone wolves often have the upper hand over established organizations. I'm curious why you think this is true? My feeling as a broke individual trying to catch up on ml is that there are some simple demos to do. But scaling up requires a lot of compute and storage for an individual. Acquiring datasets and training are cost prohibitive. I'm only able to play around with some really small stuff because by dumb luck a few years ago I bought a gaming laptop with a nvidia gpu in it. The impressive models that are generating the hype are just a different league. Love to hear to how I am wrong though?
    - sillysaurusx 3y ago
      It’s true that you need compute to do large experiments, but the large experiments grow out of small ones. If you can show promising work in a small way, it’s easier to get compute. You can also apply to TRC to get a bunch of TPUs. They had capacity issues for a long time but I’ve heard it’s improved. Don’t focus on the hype models. Find a niche that you personally like, and do that. If you’re chasing hype you’ll always be skating towards the puck. My original AI interest was to use voice generation to make Dr Kleiner sing about being a modern major general. It went from there, to image gen, to text gen, and kaboom, the whole world blew up. I was the first to show that GPTs can be used for more than just language modeling — in my case, playing chess. Wacky ideas like that are important to play around with, because they won’t seem so wacky in a year.
      - tayo42 3y ago
        Thats interesting, at a glance the TRC thing looks more altruistic and impactful than what I had in mind for learning or making money. I'll have to keep it in mind if I do, do something share worthy ever. Thanks!
    - jebarker 3y ago
      Another area where there's potential for an individual to make lots of progress is in theory and mechanistic interpretation. Although it's not where the money is, it's probably not rapid progress and it's really hard.
  - diogenes4 3y ago
    > The wonderful part about AI is that the whole world’s problems are up for grabs. Most of the world's problems aren't technological, unfortunately for us in tech. There's little it can do against the momentum of capital tearing this globe apart.
- mrtksn 3y ago
  IIRC, the movie ends with a joke on how those responsible were punished only to reveal that they walked away with the loot. You almost never see a group punishment, unless you lost as a group against another organization. So, if all the AI stuff goes bust in a year or two those who benefited from the bubble will keep their benefits. Also, there’s a possibility that it doesn’t go bust and user us into AGI era and win big.
  - gymbeaux 3y ago
    JS devs were/are paid the big bucks because !!"false" == !!"true" and so on.
    - mrtksn 3y ago
      Decided to remove the JS part, lots of people made their fortune on battling its oddities.
    - deleted 3y ago
      [deleted]
- epups 3y ago
  What is the measured evidence that AI is hype right now?
  - 7thaccount 3y ago
    Companies with massive valuations that have no product to sell or aren't making any profit. This is what I've heard anyway (not a stock guy). Some economic schools of thought say that when interest rates are low and money is practically free (historically speaking), you get a lot of bad/risky ideas (boom & bubble) and an inevitable correction of a bust. The bust is accelerated when all of a sudden you have to rise the interest rates to fight inflation and the money is no longer easy to get. All those companies being held afloat by the free money start collapsing. Again, we should make statements on data and I'll just be upfront that I haven't done much research in this area, but considering the sheer amount of AI startups and jobs and conferences and so on with very few transformative products...I would eventually expect a market correction in the form of another AI winter like what happened in the 80s when all the massive government acts defense research dollars dried up. The difference is it'll be far less severe. You'll still have plenty of AI research at the universities and large companies, but maybe not hundreds of questionable startups. This is all just conjecture on my part though.
    - nightski 3y ago
      That and even established companies with other products/services but who have seen massive market cap increases due to AI hype.
    - andrewmutz 3y ago
      > Companies with massive valuations that have no product to sell or aren't making any profit That's not enough to demonstrate that AI is just hype. Every technological breakthrough has opportunists trying to make a buck riding along the hype. In the 90s, Pets.com and webvan didn't prove that the internet was just hype. I am one of the people who is completely bought in to the idea that AI in general (and LLMs in particular) are going to lead to products that are extremely useful to the world. I absolutely think that most of the gen AI startups will fail and that valuations are too high, but I still believe that massively impactful/useful products will also be born.
- deleted 3y ago
  [deleted]
AndrewKemendo 3y ago
Very similar to my experience- modulo my role more in the AI in production, operations side, aka MLOps MLOps leads/lags research depending on your application patterns so it’s an extremely dynamic place to be to see what’s happening I’d argue based on what I’m seeing with implementations, and importantly how FLEXIBLE transformers seem to be, this is the most true part of this article: “we’re going to get way further with the Transformer architecture than most ideas in the past”
- tayo42 3y ago
  > AI in production, operations side, aka MLOps How did you get into this? Seems like a lot of places are stuck on the idea if you didn't do it in the past you cant do it now
  - rg111 3y ago
    This is closer to traditional SWE than AI research. Take some courses and get some certifications. And also make some serious projects where you demonstrate your capabilities with cutting edge tools. This is more focused on tools and use of said tools. Take some trained models, and demonstrate how well you can use them. Some ideas: 1. Take a cats vs. dogs model, deploy it online. Design an API around it. Document the API well. Create a mechanism to show confidence score, and store low confidence score examples in a database that you can later manually label and retrain the model with. 2. Take a smallish LLM, design a VS code extension that documents your functions based on docstring. Just demonstrate your basic knowledge in ML, and really good software engineering skills, learn the vocabulary well, and then start applying for jobs. It's much better if you have a CS/EE degree.
    - alfalfasprout 3y ago
      As someone in this space I could not disagree more. Certifications will do nothing for you. The harsh reality is only real world experience doing this stuff at scale will help you understand all the complexity involved. There are tons of people trying to hop onto this train after taking a few online courses and it's making it hard to filter down candidate pools.
FrustratedMonky 3y ago
Hype doesn't mean it isn't real. There was hype with the Internet, and lots of scams, and naysayer, and bogus money, and real money. I remember reading a Java 1.0 Book, and someone just casually saying "why learn that, that internet thing isn't going to last, it's all hype."
- JohnFen 3y ago
  I think the internet is a poor analogy because by the time it started to enter the awareness of the general public, it had already been around and subjected to refinement for years. Its value and usefulness was proved before the average joe had access to it at all.
  - ska 3y ago
    > awareness of the general public, it had already been around and subjected to refinement for years. And how is that different than "AI" ? It's not like these techniques sprang out of the ether in the 2000s.
  - __rito__ 3y ago
    AI research essentially started in 1940s, and not the 2010s like you might think. I read a ton of AI research papers written in the 80s and 90s. Lack of AI hype is was because the lack of data and compute. The actual field is here for a very long time.
    - JohnFen 3y ago
      I'm well aware -- I've been working with this stuff since the early 90s and have a better than usual understanding of the history of it all. But if we're going to analogize to the internet, you have to count LLMs as "the internet" and the things that came before as just "networking". Networking is very old and laid the basis for the internet, but the internet was something different in kind. My only point is that LLM being used for real things is very recent. There hasn't been a ton of use (relatively speaking) before it was available for public use. That's not true with the internet. The internet was heavily used, and heavily iterated on, before the public had access to it. Being skeptical about LLMs is not irrational, and so you see a lot of it. There's just not a great deal of history with them to provide counterexamples to that skepticism. That's not the case when the internet was opened to the public, which is a large factor in why the skepticism about the internet was much lower than the skepticism about LLMs.
Xcelerate 3y ago
Unlike previous hype cycles, the potential value of this one is extraordinary if it’s actually unlocked (I mean, what was the theoretical upper limit on the benefit of cryptocurrency for the world? Probably not that much.) Previous attempts at AI/AGI have been constrained by computational resources. It’s quite possible that we already have sufficient computational power and the necessary data for AGI—all we need are the right algorithms. Even if for some bizarre reason we’ve already tapped the maximum potential of transformer architectures and all of this money goes nowhere, compared to all the other ways that society wastes money, I would be fine with calling this a big bet for humanity that didn’t pay off. It doesn’t mean that it wasn’t worth the attempt though.
- brucethemoose2 3y ago
  Any correlation between current models and AGI is, IMO, hype. GenAI is a remarkably useful tool, but its not one step away from an AGI.
  - anonyfox 3y ago
    depends on how you define "step". Engineer a 10x/100x version of what we have in terms of LLM (either by being more efficient and/or more/specialized hardware) and let this thing build novel attempts for AGI algorithms 24/7 in a evolutionary setting. I guess the challenge is more to agree on a fitness function to measure the "AGI"-progress" against, but thats a different topic. But in general scaling up the current GenAI tech and parallelize/specialize the models in a multi-generational way _should_ be a safe ticket to AGI, but the time scale is inknown of course (since we can't even agree on the goal definition)
    - staticman2 3y ago
      "depends on how you define "step". Engineer a 10x/100x version of what we have in terms of LLM (either by being more efficient and/or more/specialized hardware) and let this thing build novel attempts for AGI algorithms 24/7 in a evolutionary setting." The current LLM's get stuck in loops when a problem is too hard for it. They just keep doing the wrong thing over and over. It's not obvious this sort of ai can "build novel attempts" at hard problems.
Aurornis 3y ago
From the other side of the table, the machine learning candidate pool is also a clown show right now. I did some hiring for a very real machine learning (AI if you want to call it that) initiative that started even before the LLM explosion. The number of candidates applying with claimed ML/AI experience who haven’t done anything more than follow online tutorials is wild. This was at a company that had a good reputation among tech people and paid above average, so we got a lot of candidates hoping to talk their way into ML jobs after completing some basic courses online. The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. We had people bragging about spending a year or more trying to get an AI model to do simple tasks that were easily solved deterministically with simple math, for example. There was a lot of AI-ification for the sake of using AI. It feels similar to when everyone with a Raspberry Pi started claiming embedded expertise or when people who worked with analytics started branding themselves as Big Data experts.
- jvalencia 3y ago
  100% this. ML has been very hyped for a while now and having it was seen as a badge for the company. To be fair, ML is not also something that was historically central to a degree, so many people wanting to get into AI, even good engineers, did not have the background in it. This too is changing though, but the hype and the lack of an experienced pool doesn't help.
  - mikrl 3y ago
    I don’t think the background is really that important tbh. From physics I have a good theoretical grounding in how ML works (optimizing a cost function over a high dimensional manifold to reconstruct a probability distribution, then using the distribution for some task) but I personally find actually ‘doing ML’ to be rather dull.
- hashtag-til 3y ago
  > The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. I can relate to this a lot. In my company many things you can sell as "AI" can really be solved with traditional data processing.
dwroberts 3y ago
> All of Google DeepMind's headcount is in the first two categories, with most of it being in the first. Does that mean that DM now does no fundamental research - or does it still happen and it has simply been rebranded/hidden away?
- aabhay 3y ago
  In the last year, they have developed what they call a “product” focus. But they still do more basic research, good luck getting a GPU allocation though.
agomez314 3y ago
Jeez it’s kind of amazing to hear the kind of treatment you get if you’re lucky enough to be an AI researcher. Being in the right industry in the right place seems to trump everything else. I’d be happy just being a cog in the machine, work 9-5, and get to have an upper middle class lifestyle with my family the rest of the time. That’s probs better than what 95% of people (in the US) get to experience.
- p1esk 3y ago
  It's definitely not enough to be just an average "AI researcher" to get this treatment. This guy (Nathan Lambert) is from a top RL lab, with a strong RL publication record, an expert in RLHF, which is extremely hot right now, and with a recent experience creating RLHF pipelines at HuggingFace (one of the top AI companies). So he's probably in top-1% of all people looking for AI research jobs currently.
npalli 3y ago
Good chance AI has winner-take-all dynamics. So while the field itself might be very hot and valuable, only a few will make massive bank and rest will get nothing. Like trying to be a basketball or soccer star, much demand and prestige but the average joe is not making millions or on TV.
- aabhay 3y ago
  Hard disagree. My guess is that AI is actually a race-to-the-bottom dynamic. Given the competition across all/every FAANG and tons of startups, my guess is we’ll have a wide range of options for APIs across clouds and providers. On the consumer side we’ll have a range of options for chatbots, API integrations, and more. For most use cases of AI, there is a ceiling to how intelligent it needs to be. I am guessing we’ll be selecting from dozens of models based on various sizes, context lengths, etc. Just like we right-size VMs in the cloud.
  - vagab0nd 3y ago
    "Race-to-the-bottom" might lead to "winner-take-all"? Eventually the profit margin is so thin that only the biggest companies can survive. In other words, once you have a "race-to-the-bottom" situation, it's hard for newcomers to get in the game.
  - waveBidder 3y ago
    ml is entirely dependent on your access to relevant data, which itself has strong network effects.
    - aabhay 3y ago
      Data access has weaker network effects than you would expect. Generated chats / outputs are rarely good enough as training data, the "best"/"cleanest" data is still expert-created.
varelse 3y ago
The biggest tell I got out of this article is OpenAI's willingness to pay $1M per AI researcher but they only want to pay $300K for their new hardware design people. I think we know how that will play out, but narcissists gotta narcissist. As someone who rode the first AI wave to 7 figure comp a decade ago, let moneyball round 2 commence! Let's see who can get to 8 figure comp first! That money isn't doing anyone any good if it's just sitting in a brokerage or bank account. Didn't you hear? The singularity is near! You simply cannot afford to miss out or you're going to get Vernor Vinged! Along the way, don't forget to take some time out of your busy day to drink the delicious bitter tears of venture capitalists and upper management whining that you are paid too much.
lawlessone 3y ago
The below writing is just my opinion ,anecdotal and sour grapes. Have been interested in this stuff for years. Did my CS project with NN just before everyone started using GPU's ,and a short DS course more recently. Seeing all the marketing people move into space with their prompt cheat sheets on LinkedIn while many tech people are ,ironically, locked out by blackbox recruitment algorithms is maddening.(This particular problem goes far beyond tech jobs though). Some also seem to be mixing up DS and DE roles a bit, one of the few times I got an interview I had end it and apologize as what they were looking for was a data engineer. Another was listed as a machine learning role ,when I got the offer it was travelling tech support and paid less. With the promise of undefined ML work later. Some companies are just tacking irrelevant ML and AI stuff onto job descriptions. Also so many live coding tests , and that one weird recruiter asking about "skeletons in closets"
- rg111 3y ago
  > I got an interview I had end it and apologize as what they were looking for was a data engineer. 100% happened to me once. Wasted hours of my time. > Some companies are just tacking irrelevant ML and AI stuff onto job descriptions. Some of them do this deliberately. I have seen this practice in companies targeting junior roles and fresh out of college grads. They hire them with shit pay and promise them ML experience, and then make them do non ML stuff.
  - lawlessone 3y ago
    Luckily in my case it was very short. The first tech question they asked me was about the how to move all a companies data from old gov systems to a data lake. We both got a quick lesson really. All polite. The second bait and switch example though went on a lot longer. I had an off feeling about it from the first call. One guy on call was stifling a laugh the whole time. They made sure to emphasize they we're offering me a lot of experience, doing me favor essentially. When they gave me the offer they also requested I send them over a professional photograph of myself. Maybe that's normal in some countries but to me it was the red flag that finally made me notice all the other red flags.
- 3y ago
spatalo 3y ago
Would you mind sharing your CV? As an AI researcher with PhD in Europe, I am applying to multiple post-grad positions but can't even get interviews. And I'm not even talking about the big companies.
- natolambert 3y ago
  Author here, you can find more on my website: https://natolambert.com/cv https://natolambert.com/cv Have been building RLHF systems at HuggingFace since ChatGPT, with some other experience before.
  - choppaface 3y ago
    Why stay with HF instead of going OpenAI?
- wucaworld 3y ago
  Also work in deep learning for the last many years. We pay people well but the hype about truckloads of cash to researchers is overblown. Unless you happen to be exactly the skill set someone needs, there is a lot of talent. I post this to say there is nothing wrong with you. You hear of the crazy successful few but not the majority of cases.
sad_robot 3y ago
Is it a good idea to pivot into ML/AI? Would the bubble have burst before one can finish a PhD?
- zffr 3y ago
  How do you personally define “good idea”? If this is about your financial outcome, make sure to factor in the opportunity cost of a PhD. It will require 5-7yrs where you will make very little money.
- schott12521 3y ago
  I'll add that I'm at out of work Software Engineer with a BS in Comp Sci and I've been researching to see if a Masters in ML/AI is worth it. I haven't seen anything indicating it's essential, and I still see that with a MS in AI/ML you're still most likely going to be doing Software Dev... I'm sure it's different for a PhD, but as the other commenter said, it's going to take a lot longer.
- j7ake 3y ago
  Let’s just say if you wanted to have hit the LLM bubble today , you would have had to have gone into the unpopular ML field of NLP, while watching all your vision peers getting all the highest paid jobs. So the next hot thing is likely an ML field that is not the current hottest.
- rg111 3y ago
  It's very much like the dotcom bubble. It's going to burst for sure, but people will be using ML/AI anyway. If that's your concern, then you can safely do ML/AI. But, don't hold your breath on getting a presrigious role after PhD. Because, there are very few real AI companies. And their hiring is skewed to Stanford, MIT, UCLA-B, Oxbridge, UToronto, ETH, etc. And getting into these schools in AI was always competitive, but now it is crazy-like because all the prep school kids are eyeing this since basically high-school or even before. So, there is someone like me, who didn't know about modern AI until last year of college in a different major, then learning on my own and getting research jobs in small time companies with shit pay, and then there are people with tiger parents who are white collar or even academics who helps their children get into a really prestigious school, and there they do AI projects with top professors, who write them recommendations, and who also do summer internships in DeepMind, and then they use that to get a job in a proper ML company. At this point they have 2-4 publications in top tier conference. Then they work in BigTech AI lab for 3-5 years and get at least 4/5 more papers (no upper limit). And these are the people who are going to Stanford PhDs. Not people like me. And then these PhDs will be Research Engineers and such in DeepMind, OpenAI, etc. So, before deciding to do a PhD from non elite institutions, think hard. Because of AI hype, the situation is very bad for the rest of us. Because all of the prep school types in STEM want to make it in AI.
dsign 3y ago
It must be hard to get those experts out of the bushes. First, there is the fact that not every expert out there is after the money or working 60 hours a week, or even 40, or work at the office. Second, there is this thing about how much of a shit-show any hiring process is...
slowhadoken 3y ago
The definition for “AI” is being blurred, it’s a black box buzzword lately. Bros will talk my ear off about AI and none of them know basic graph theory.
- constantly 3y ago
  These concepts are orthogonal so that’s probably expected.
  - slowhadoken 3y ago
    orthogonal? Neural networks are a layered graph structure. When they talk about "parameters" they're referring to nodes and edges.
    - constantly 3y ago
      TikTok is backed by complex algorithms to ensure reliability of data transfer and yet I bet you don’t chastise people recording the latest TikTik dance trend for not knowing the fundamentals of TCP/IP. When they talk about sending a video they’re referring to TCP/IP. I bet they can’t even talk about elliptic curve cryptography!
      - slowhadoken 3y ago
        Apples and oranges, bud.
        constantly 3y ago
        Sounds like you’re gatekeeping to be honest. People don’t need to know irrelevant concepts or hear stump the chump questions. If that’s how you approached it, great! There are lots of successful approaches and people shouldn’t be chastised because they took a different path or are interested in a different facet of the topic.
        3y ago
choppaface 3y ago
> It's becoming a bit like transfer news in all of our favorite sports leagues. Except engineers don't have a union nor contracts like athletes, so: * When layoffs / mass firings happen, engineers don't get guaranteed money * The comp of the top 1% doesn't pull up the bottom 99% (not nearly as fast). Much of the bump in SWE salaries in the past 5 years came from the uncovering of the Apple+other co's no-poaches. The $1-5m retainers Google was offering in 2010-2012 to keep G from jumping to FB look now like peanuts. * Engineers at competing companies have no say over each other's comp. It takes actual offers for salaries to rise instead of Eng being able to pool the salary data and stats and determine who's Big Head and who all are the 10,000 underpaid SWEs. Moreover, the glamour of the "transfers" is pretty tightly contained within the author's bubble. A lot of people in AI-adjacent fields still don't even know what Pytorch / Tensorflow are. Important historical context the author leaves out: Hinton accepting $35-40m for his lab to do deep learning at Google is what set most of the initial benchmark. Many stories out there where Hinton broke his NDA, here is one for example: https://www.gq-magazine.co.uk/culture/article/cade-metz-genius-makers-extract https://www.gq-magazine.co.uk/culture/article/cade-metz-geni... It's important context because there's so much arbitrage of private info in AI, you can't take posts like the author's at face value.
ajb117 3y ago
I'm curious about the AI research job market in fields that aren't LLMs and RL. They're all the rage now, but what about researchers in fields like vision or graphs or more fundamental topics like optimization?
- Legend2440 3y ago
  Vision is doing okay. One big focus is how to use these fantastic generative models for perception tasks. Multi-model LLMs and inferring 3D structure from 2D images are also popular topics. Fundamental research tends to be done in academia. Big tech does some of it, but right now they're more focused on making LLMs into products.
- deleted 3y ago
  [deleted]
- DecentRecruiter 3y ago
  An example of Legend2440's reply: A mid-stage Computer Vision software startup just hired its first ML Engineer to work on multi-modal LLM, VLLM, GenAI for image & language-based tasks. Their product is SLAM/Perception-focused and they have many CV Engineers, yet even they've found a need for LLM.
xpe 3y ago
> These are the people who are keeping training and product decisions on track with the big-picture trends, which can change on a dime in a day. If things appear to change that fast, I’d suggest one isn’t well calibrated to the overall environment. There are deeper fundamentals that don’t change very often. The specific industry moves and product offerings can only deviate so much from ground truth. The more they deviate, the more one should be skeptical of their claims.

207 points by sebg 3y ago

I just watched The Big Short I felt a bit eery how much the groupthink around mortgages matched todays AI hype. The unwillingness to listen to critical advice. To question the value. Data science depts who measure such things are often vilified. It’s a kind of depressing job landscape in this way. You either go with the, often top down, groupthink against the face of measured evidence, or you’re labeled a cynical naysayer and your career suffers.

chasd00 3y ago
just use your skills, i.e. mop, and soak of the cash.
sillysaurusx 3y ago
Well, don’t be cynical about it. One thing that I think a lot of us could benefit from is learning how to present a point in a way that people will listen. The key is to harness that hype — use your point to unlock some pent up energy for a new direction, rather than merely try to say it can’t work. A lot of endeavors can work, even if only a little. Find some aspect of it that can, and flip the problem around. Even if the whole thing is mostly bogus, is there a small part that isn’t? Latch onto that. Or go elsewhere. The wonderful part about AI is that the whole world’s problems are up for grabs. Part of why there’s so much unfounded hype is because of how many real advances have recently become possible. This period in history will never come again. It’s also a rare time in history that an individual can make lots of progress. Most of us need to be a part of big groups to do anything worthwhile, in most fields. But in this case lone wolves often have the upper hand over established organizations.
- x86x87 3y ago
  Yes. Don't tell the emperor that he has no clothes. Instead innovate and introduce a hybrid between his current clothes and old fashion textiles. Who know? In time they may end up covering their junk.
- softwaredoug 3y ago
  To be clear, I'm mostly excited. But I also think its reasonable to be skeptical and try to educate stakeholders on the realities of the situation. My controversial take on AI is its actually a better time to take things slow, experiment, study, see what works. Not dump TONS of money and cash and get too distracted. Because nobody (besides big tech) has fully figured out how to make a product that makes a profit. Its not clear users want a chatbot (aside from ChatGPT)... But things could change.
  - chung8123 3y ago
    I think the issue is, right now AI is a race. We have the Microsoft, Google, Meta, Apple, and Amazon's of the world with their massive compute and bankrolls racing to see who can build the biggest moat around an AI service. The massive upfront spending is hoping to hit the winning lottery ticket and spending slowly may leave you out of the drawing. As compute costs and requirements come down LLMs will be ubiquitous everywhere.
    - akomtu 3y ago
      It's the Manhattan Project 2.0. The AI, once created, won't be that hard to replicate, but those who fail to create it early, will be sidetracked later. The race among big tech is the american way of doing such projects: fund a few companies, let them compete and pick the winner.
      - __loam 3y ago
        This industry is so arrogant lmao. The Manhattan project? Get fucking real.
- tayo42 3y ago
  > It’s also a rare time in history that an individual can make lots of progress. Most of us need to be a part of big groups to do anything worthwhile, in most fields. But in this case lone wolves often have the upper hand over established organizations. I'm curious why you think this is true? My feeling as a broke individual trying to catch up on ml is that there are some simple demos to do. But scaling up requires a lot of compute and storage for an individual. Acquiring datasets and training are cost prohibitive. I'm only able to play around with some really small stuff because by dumb luck a few years ago I bought a gaming laptop with a nvidia gpu in it. The impressive models that are generating the hype are just a different league. Love to hear to how I am wrong though?
  - sillysaurusx 3y ago
    It’s true that you need compute to do large experiments, but the large experiments grow out of small ones. If you can show promising work in a small way, it’s easier to get compute. You can also apply to TRC to get a bunch of TPUs. They had capacity issues for a long time but I’ve heard it’s improved. Don’t focus on the hype models. Find a niche that you personally like, and do that. If you’re chasing hype you’ll always be skating towards the puck. My original AI interest was to use voice generation to make Dr Kleiner sing about being a modern major general. It went from there, to image gen, to text gen, and kaboom, the whole world blew up. I was the first to show that GPTs can be used for more than just language modeling — in my case, playing chess. Wacky ideas like that are important to play around with, because they won’t seem so wacky in a year.
    - tayo42 3y ago
      Thats interesting, at a glance the TRC thing looks more altruistic and impactful than what I had in mind for learning or making money. I'll have to keep it in mind if I do, do something share worthy ever. Thanks!
  - jebarker 3y ago
    Another area where there's potential for an individual to make lots of progress is in theory and mechanistic interpretation. Although it's not where the money is, it's probably not rapid progress and it's really hard.
- diogenes4 3y ago
  > The wonderful part about AI is that the whole world’s problems are up for grabs. Most of the world's problems aren't technological, unfortunately for us in tech. There's little it can do against the momentum of capital tearing this globe apart.
mrtksn 3y ago
IIRC, the movie ends with a joke on how those responsible were punished only to reveal that they walked away with the loot. You almost never see a group punishment, unless you lost as a group against another organization. So, if all the AI stuff goes bust in a year or two those who benefited from the bubble will keep their benefits. Also, there’s a possibility that it doesn’t go bust and user us into AGI era and win big.
- gymbeaux 3y ago
  JS devs were/are paid the big bucks because !!"false" == !!"true" and so on.
  - mrtksn 3y ago
    Decided to remove the JS part, lots of people made their fortune on battling its oddities.
  - deleted 3y ago
    [deleted]
epups 3y ago
What is the measured evidence that AI is hype right now?
- 7thaccount 3y ago
  Companies with massive valuations that have no product to sell or aren't making any profit. This is what I've heard anyway (not a stock guy). Some economic schools of thought say that when interest rates are low and money is practically free (historically speaking), you get a lot of bad/risky ideas (boom & bubble) and an inevitable correction of a bust. The bust is accelerated when all of a sudden you have to rise the interest rates to fight inflation and the money is no longer easy to get. All those companies being held afloat by the free money start collapsing. Again, we should make statements on data and I'll just be upfront that I haven't done much research in this area, but considering the sheer amount of AI startups and jobs and conferences and so on with very few transformative products...I would eventually expect a market correction in the form of another AI winter like what happened in the 80s when all the massive government acts defense research dollars dried up. The difference is it'll be far less severe. You'll still have plenty of AI research at the universities and large companies, but maybe not hundreds of questionable startups. This is all just conjecture on my part though.
  - nightski 3y ago
    That and even established companies with other products/services but who have seen massive market cap increases due to AI hype.
  - andrewmutz 3y ago
    > Companies with massive valuations that have no product to sell or aren't making any profit That's not enough to demonstrate that AI is just hype. Every technological breakthrough has opportunists trying to make a buck riding along the hype. In the 90s, Pets.com and webvan didn't prove that the internet was just hype. I am one of the people who is completely bought in to the idea that AI in general (and LLMs in particular) are going to lead to products that are extremely useful to the world. I absolutely think that most of the gen AI startups will fail and that valuations are too high, but I still believe that massively impactful/useful products will also be born.
deleted 3y ago
[deleted]

AndrewKemendo 3y ago

Very similar to my experience- modulo my role more in the AI in production, operations side, aka MLOps MLOps leads/lags research depending on your application patterns so it’s an extremely dynamic place to be to see what’s happening I’d argue based on what I’m seeing with implementations, and importantly how FLEXIBLE transformers seem to be, this is the most true part of this article: “we’re going to get way further with the Transformer architecture than most ideas in the past”

tayo42 3y ago
> AI in production, operations side, aka MLOps How did you get into this? Seems like a lot of places are stuck on the idea if you didn't do it in the past you cant do it now
- rg111 3y ago
  This is closer to traditional SWE than AI research. Take some courses and get some certifications. And also make some serious projects where you demonstrate your capabilities with cutting edge tools. This is more focused on tools and use of said tools. Take some trained models, and demonstrate how well you can use them. Some ideas: 1. Take a cats vs. dogs model, deploy it online. Design an API around it. Document the API well. Create a mechanism to show confidence score, and store low confidence score examples in a database that you can later manually label and retrain the model with. 2. Take a smallish LLM, design a VS code extension that documents your functions based on docstring. Just demonstrate your basic knowledge in ML, and really good software engineering skills, learn the vocabulary well, and then start applying for jobs. It's much better if you have a CS/EE degree.
  - alfalfasprout 3y ago
    As someone in this space I could not disagree more. Certifications will do nothing for you. The harsh reality is only real world experience doing this stuff at scale will help you understand all the complexity involved. There are tons of people trying to hop onto this train after taking a few online courses and it's making it hard to filter down candidate pools.

FrustratedMonky 3y ago

Hype doesn't mean it isn't real. There was hype with the Internet, and lots of scams, and naysayer, and bogus money, and real money. I remember reading a Java 1.0 Book, and someone just casually saying "why learn that, that internet thing isn't going to last, it's all hype."

JohnFen 3y ago
I think the internet is a poor analogy because by the time it started to enter the awareness of the general public, it had already been around and subjected to refinement for years. Its value and usefulness was proved before the average joe had access to it at all.
- ska 3y ago
  > awareness of the general public, it had already been around and subjected to refinement for years. And how is that different than "AI" ? It's not like these techniques sprang out of the ether in the 2000s.
- __rito__ 3y ago
  AI research essentially started in 1940s, and not the 2010s like you might think. I read a ton of AI research papers written in the 80s and 90s. Lack of AI hype is was because the lack of data and compute. The actual field is here for a very long time.
  - JohnFen 3y ago
    I'm well aware -- I've been working with this stuff since the early 90s and have a better than usual understanding of the history of it all. But if we're going to analogize to the internet, you have to count LLMs as "the internet" and the things that came before as just "networking". Networking is very old and laid the basis for the internet, but the internet was something different in kind. My only point is that LLM being used for real things is very recent. There hasn't been a ton of use (relatively speaking) before it was available for public use. That's not true with the internet. The internet was heavily used, and heavily iterated on, before the public had access to it. Being skeptical about LLMs is not irrational, and so you see a lot of it. There's just not a great deal of history with them to provide counterexamples to that skepticism. That's not the case when the internet was opened to the public, which is a large factor in why the skepticism about the internet was much lower than the skepticism about LLMs.

Xcelerate 3y ago

Unlike previous hype cycles, the potential value of this one is extraordinary if it’s actually unlocked (I mean, what was the theoretical upper limit on the benefit of cryptocurrency for the world? Probably not that much.) Previous attempts at AI/AGI have been constrained by computational resources. It’s quite possible that we already have sufficient computational power and the necessary data for AGI—all we need are the right algorithms. Even if for some bizarre reason we’ve already tapped the maximum potential of transformer architectures and all of this money goes nowhere, compared to all the other ways that society wastes money, I would be fine with calling this a big bet for humanity that didn’t pay off. It doesn’t mean that it wasn’t worth the attempt though.

brucethemoose2 3y ago
Any correlation between current models and AGI is, IMO, hype. GenAI is a remarkably useful tool, but its not one step away from an AGI.
- anonyfox 3y ago
  depends on how you define "step". Engineer a 10x/100x version of what we have in terms of LLM (either by being more efficient and/or more/specialized hardware) and let this thing build novel attempts for AGI algorithms 24/7 in a evolutionary setting. I guess the challenge is more to agree on a fitness function to measure the "AGI"-progress" against, but thats a different topic. But in general scaling up the current GenAI tech and parallelize/specialize the models in a multi-generational way _should_ be a safe ticket to AGI, but the time scale is inknown of course (since we can't even agree on the goal definition)
  - staticman2 3y ago
    "depends on how you define "step". Engineer a 10x/100x version of what we have in terms of LLM (either by being more efficient and/or more/specialized hardware) and let this thing build novel attempts for AGI algorithms 24/7 in a evolutionary setting." The current LLM's get stuck in loops when a problem is too hard for it. They just keep doing the wrong thing over and over. It's not obvious this sort of ai can "build novel attempts" at hard problems.

Aurornis 3y ago

From the other side of the table, the machine learning candidate pool is also a clown show right now. I did some hiring for a very real machine learning (AI if you want to call it that) initiative that started even before the LLM explosion. The number of candidates applying with claimed ML/AI experience who haven’t done anything more than follow online tutorials is wild. This was at a company that had a good reputation among tech people and paid above average, so we got a lot of candidates hoping to talk their way into ML jobs after completing some basic courses online. The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. We had people bragging about spending a year or more trying to get an AI model to do simple tasks that were easily solved deterministically with simple math, for example. There was a lot of AI-ification for the sake of using AI. It feels similar to when everyone with a Raspberry Pi started claiming embedded expertise or when people who worked with analytics started branding themselves as Big Data experts.

jvalencia 3y ago
100% this. ML has been very hyped for a while now and having it was seen as a badge for the company. To be fair, ML is not also something that was historically central to a degree, so many people wanting to get into AI, even good engineers, did not have the background in it. This too is changing though, but the hype and the lack of an experienced pool doesn't help.
- mikrl 3y ago
  I don’t think the background is really that important tbh. From physics I have a good theoretical grounding in how ML works (optimizing a cost function over a high dimensional manifold to reconstruct a probability distribution, then using the distribution for some task) but I personally find actually ‘doing ML’ to be rather dull.
hashtag-til 3y ago
> The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. I can relate to this a lot. In my company many things you can sell as "AI" can really be solved with traditional data processing.

dwroberts 3y ago

> All of Google DeepMind's headcount is in the first two categories, with most of it being in the first. Does that mean that DM now does no fundamental research - or does it still happen and it has simply been rebranded/hidden away?

aabhay 3y ago
In the last year, they have developed what they call a “product” focus. But they still do more basic research, good luck getting a GPU allocation though.

agomez314 3y ago

Jeez it’s kind of amazing to hear the kind of treatment you get if you’re lucky enough to be an AI researcher. Being in the right industry in the right place seems to trump everything else. I’d be happy just being a cog in the machine, work 9-5, and get to have an upper middle class lifestyle with my family the rest of the time. That’s probs better than what 95% of people (in the US) get to experience.

p1esk 3y ago
It's definitely not enough to be just an average "AI researcher" to get this treatment. This guy (Nathan Lambert) is from a top RL lab, with a strong RL publication record, an expert in RLHF, which is extremely hot right now, and with a recent experience creating RLHF pipelines at HuggingFace (one of the top AI companies). So he's probably in top-1% of all people looking for AI research jobs currently.

npalli 3y ago

Good chance AI has winner-take-all dynamics. So while the field itself might be very hot and valuable, only a few will make massive bank and rest will get nothing. Like trying to be a basketball or soccer star, much demand and prestige but the average joe is not making millions or on TV.

aabhay 3y ago
Hard disagree. My guess is that AI is actually a race-to-the-bottom dynamic. Given the competition across all/every FAANG and tons of startups, my guess is we’ll have a wide range of options for APIs across clouds and providers. On the consumer side we’ll have a range of options for chatbots, API integrations, and more. For most use cases of AI, there is a ceiling to how intelligent it needs to be. I am guessing we’ll be selecting from dozens of models based on various sizes, context lengths, etc. Just like we right-size VMs in the cloud.
- vagab0nd 3y ago
  "Race-to-the-bottom" might lead to "winner-take-all"? Eventually the profit margin is so thin that only the biggest companies can survive. In other words, once you have a "race-to-the-bottom" situation, it's hard for newcomers to get in the game.
- waveBidder 3y ago
  ml is entirely dependent on your access to relevant data, which itself has strong network effects.
  - aabhay 3y ago
    Data access has weaker network effects than you would expect. Generated chats / outputs are rarely good enough as training data, the "best"/"cleanest" data is still expert-created.

varelse 3y ago

The biggest tell I got out of this article is OpenAI's willingness to pay $1M per AI researcher but they only want to pay $300K for their new hardware design people. I think we know how that will play out, but narcissists gotta narcissist. As someone who rode the first AI wave to 7 figure comp a decade ago, let moneyball round 2 commence! Let's see who can get to 8 figure comp first! That money isn't doing anyone any good if it's just sitting in a brokerage or bank account. Didn't you hear? The singularity is near! You simply cannot afford to miss out or you're going to get Vernor Vinged! Along the way, don't forget to take some time out of your busy day to drink the delicious bitter tears of venture capitalists and upper management whining that you are paid too much.

lawlessone 3y ago

The below writing is just my opinion ,anecdotal and sour grapes. Have been interested in this stuff for years. Did my CS project with NN just before everyone started using GPU's ,and a short DS course more recently. Seeing all the marketing people move into space with their prompt cheat sheets on LinkedIn while many tech people are ,ironically, locked out by blackbox recruitment algorithms is maddening.(This particular problem goes far beyond tech jobs though). Some also seem to be mixing up DS and DE roles a bit, one of the few times I got an interview I had end it and apologize as what they were looking for was a data engineer. Another was listed as a machine learning role ,when I got the offer it was travelling tech support and paid less. With the promise of undefined ML work later. Some companies are just tacking irrelevant ML and AI stuff onto job descriptions. Also so many live coding tests , and that one weird recruiter asking about "skeletons in closets"

rg111 3y ago
> I got an interview I had end it and apologize as what they were looking for was a data engineer. 100% happened to me once. Wasted hours of my time. > Some companies are just tacking irrelevant ML and AI stuff onto job descriptions. Some of them do this deliberately. I have seen this practice in companies targeting junior roles and fresh out of college grads. They hire them with shit pay and promise them ML experience, and then make them do non ML stuff.
- lawlessone 3y ago
  Luckily in my case it was very short. The first tech question they asked me was about the how to move all a companies data from old gov systems to a data lake. We both got a quick lesson really. All polite. The second bait and switch example though went on a lot longer. I had an off feeling about it from the first call. One guy on call was stifling a laugh the whole time. They made sure to emphasize they we're offering me a lot of experience, doing me favor essentially. When they gave me the offer they also requested I send them over a professional photograph of myself. Maybe that's normal in some countries but to me it was the red flag that finally made me notice all the other red flags.
3y ago

spatalo 3y ago

Would you mind sharing your CV? As an AI researcher with PhD in Europe, I am applying to multiple post-grad positions but can't even get interviews. And I'm not even talking about the big companies.

natolambert 3y ago
Author here, you can find more on my website: https://natolambert.com/cv https://natolambert.com/cv Have been building RLHF systems at HuggingFace since ChatGPT, with some other experience before.
- choppaface 3y ago
  Why stay with HF instead of going OpenAI?
wucaworld 3y ago
Also work in deep learning for the last many years. We pay people well but the hype about truckloads of cash to researchers is overblown. Unless you happen to be exactly the skill set someone needs, there is a lot of talent. I post this to say there is nothing wrong with you. You hear of the crazy successful few but not the majority of cases.

sad_robot 3y ago

Is it a good idea to pivot into ML/AI? Would the bubble have burst before one can finish a PhD?

zffr 3y ago
How do you personally define “good idea”? If this is about your financial outcome, make sure to factor in the opportunity cost of a PhD. It will require 5-7yrs where you will make very little money.
schott12521 3y ago
I'll add that I'm at out of work Software Engineer with a BS in Comp Sci and I've been researching to see if a Masters in ML/AI is worth it. I haven't seen anything indicating it's essential, and I still see that with a MS in AI/ML you're still most likely going to be doing Software Dev... I'm sure it's different for a PhD, but as the other commenter said, it's going to take a lot longer.
j7ake 3y ago
Let’s just say if you wanted to have hit the LLM bubble today , you would have had to have gone into the unpopular ML field of NLP, while watching all your vision peers getting all the highest paid jobs. So the next hot thing is likely an ML field that is not the current hottest.
rg111 3y ago
It's very much like the dotcom bubble. It's going to burst for sure, but people will be using ML/AI anyway. If that's your concern, then you can safely do ML/AI. But, don't hold your breath on getting a presrigious role after PhD. Because, there are very few real AI companies. And their hiring is skewed to Stanford, MIT, UCLA-B, Oxbridge, UToronto, ETH, etc. And getting into these schools in AI was always competitive, but now it is crazy-like because all the prep school kids are eyeing this since basically high-school or even before. So, there is someone like me, who didn't know about modern AI until last year of college in a different major, then learning on my own and getting research jobs in small time companies with shit pay, and then there are people with tiger parents who are white collar or even academics who helps their children get into a really prestigious school, and there they do AI projects with top professors, who write them recommendations, and who also do summer internships in DeepMind, and then they use that to get a job in a proper ML company. At this point they have 2-4 publications in top tier conference. Then they work in BigTech AI lab for 3-5 years and get at least 4/5 more papers (no upper limit). And these are the people who are going to Stanford PhDs. Not people like me. And then these PhDs will be Research Engineers and such in DeepMind, OpenAI, etc. So, before deciding to do a PhD from non elite institutions, think hard. Because of AI hype, the situation is very bad for the rest of us. Because all of the prep school types in STEM want to make it in AI.

dsign 3y ago

It must be hard to get those experts out of the bushes. First, there is the fact that not every expert out there is after the money or working 60 hours a week, or even 40, or work at the office. Second, there is this thing about how much of a shit-show any hiring process is...

slowhadoken 3y ago

The definition for “AI” is being blurred, it’s a black box buzzword lately. Bros will talk my ear off about AI and none of them know basic graph theory.

constantly 3y ago
These concepts are orthogonal so that’s probably expected.
- slowhadoken 3y ago
  orthogonal? Neural networks are a layered graph structure. When they talk about "parameters" they're referring to nodes and edges.
  - constantly 3y ago
    TikTok is backed by complex algorithms to ensure reliability of data transfer and yet I bet you don’t chastise people recording the latest TikTik dance trend for not knowing the fundamentals of TCP/IP. When they talk about sending a video they’re referring to TCP/IP. I bet they can’t even talk about elliptic curve cryptography!
    - slowhadoken 3y ago
      Apples and oranges, bud.
      - constantly 3y ago
        Sounds like you’re gatekeeping to be honest. People don’t need to know irrelevant concepts or hear stump the chump questions. If that’s how you approached it, great! There are lots of successful approaches and people shouldn’t be chastised because they took a different path or are interested in a different facet of the topic.
        3y ago

choppaface 3y ago

> It's becoming a bit like transfer news in all of our favorite sports leagues. Except engineers don't have a union nor contracts like athletes, so: * When layoffs / mass firings happen, engineers don't get guaranteed money * The comp of the top 1% doesn't pull up the bottom 99% (not nearly as fast). Much of the bump in SWE salaries in the past 5 years came from the uncovering of the Apple+other co's no-poaches. The $1-5m retainers Google was offering in 2010-2012 to keep G from jumping to FB look now like peanuts. * Engineers at competing companies have no say over each other's comp. It takes actual offers for salaries to rise instead of Eng being able to pool the salary data and stats and determine who's Big Head and who all are the 10,000 underpaid SWEs. Moreover, the glamour of the "transfers" is pretty tightly contained within the author's bubble. A lot of people in AI-adjacent fields still don't even know what Pytorch / Tensorflow are. Important historical context the author leaves out: Hinton accepting $35-40m for his lab to do deep learning at Google is what set most of the initial benchmark. Many stories out there where Hinton broke his NDA, here is one for example: https://www.gq-magazine.co.uk/culture/article/cade-metz-genius-makers-extract https://www.gq-magazine.co.uk/culture/article/cade-metz-geni... It's important context because there's so much arbitrage of private info in AI, you can't take posts like the author's at face value.

ajb117 3y ago

I'm curious about the AI research job market in fields that aren't LLMs and RL. They're all the rage now, but what about researchers in fields like vision or graphs or more fundamental topics like optimization?

Legend2440 3y ago
Vision is doing okay. One big focus is how to use these fantastic generative models for perception tasks. Multi-model LLMs and inferring 3D structure from 2D images are also popular topics. Fundamental research tends to be done in academia. Big tech does some of it, but right now they're more focused on making LLMs into products.
deleted 3y ago
[deleted]
DecentRecruiter 3y ago
An example of Legend2440's reply: A mid-stage Computer Vision software startup just hired its first ML Engineer to work on multi-modal LLM, VLLM, GenAI for image & language-based tasks. Their product is SLAM/Perception-focused and they have many CV Engineers, yet even they've found a need for LLM.

xpe 3y ago

> These are the people who are keeping training and product decisions on track with the big-picture trends, which can change on a dime in a day. If things appear to change that fast, I’d suggest one isn’t well calibrated to the overall environment. There are deeper fundamentals that don’t change very often. The specific industry moves and product offerings can only deviate so much from ground truth. The more they deviate, the more one should be skeptical of their claims.