9 ms·

My personnal experience with data scientist and startups is that they're hired much much too early, when the product has a lot more fundamental issues to solve,

• retbull 9y ago
  How would start ups compete well with those salaries?
  • dsacco 9y ago
    They can’t, and don’t. The secret is that experienced PhDs (mostly) dominate the high end of “AI” hiring, but don’t have much title or departmental differentiation from people who are “only” specialized software engineers in top tech companies. But this isn’t very well known, large companies want to recruit as much talent as they can regardless of role, and startups want to compete on paper - therefore, you have the following effects: 1. Startups give whatever sexy title they want to the people they can afford, which makes titles fairly useless, because they almost never can afford the talent commanding “sky-high” salaries. 2. Within large tech companies like Google and Facebook, it’s hard to immediately tell which of the many data sciencey, machine learning-y titles correspond to the truly stratospheric salaries versus the engineers that work with those roles. For some teams, like DeepMind, Google Brain or FAIR, it’s easier to tell. But for others it’s a mixed bag. For comparison, see the fashionable term of art “quant” in the financial industry, which has similarly devolved into marketing and a bimodal distribution. As a rule of thumb, you generally can’t trust AI titles or salaries at startups unless those startups are really known for their talent; further, you can safely assume that, at companies capable of paying for top talent, the very impressive salaries belong to titles which seem the most exotic and out of reach for general engineers in the job description. Fortunately startups don’t really need to compete for top talent as a genuine technological differentiator, they just need to engage in signaling, so this is mostly a non-problem. Startups almost never have problems usefully improved by the cutting edge of machine learning research, and can instead use off the shelf tools and existing software to accomplish the same things. Frankly, it’s exceptionally rare for a startup to even have the massive data, pipeline and munging infrastructure requisite for actual research.
    • pm90 9y ago
      This actually makes a lot of sense. I have a lot of friends employed as Data Scientist or Data Engineers who don't seem to be able to explain exactly what they do, or describe what "research" they are doing. From what I understand, it seems like they are designing pipelines that ingest data, run an off the shelf AI algorithm and display nice graphs. There is a lot of pipelines you can design for different kinds of data so I imagine they always have something to do.
    • salmonfamine 9y ago
      Is this not the impetus behind the "data engineer" title that's been thrown around lately?
    • mr_toad 9y ago
      It’s not just startups. Big business and government have slapped the ‘data scientist’ label on anyone who can drum up a chart in Excel. As Napoleon said: titles don’t honour men, men honour titles.
  • svantana 9y ago
    In my experience, the best scientists are not motivated by money, at least not beyond enough to provide a comfortable lifestyle. For me it's much more important to work on interesting problems in a good team. Also, startups can provide an exhilarating feeling of 'anything is possible' which you rarely get at a large corporation.
  • Spooky23 9y ago
    You don't. You sell solutions. In most enterprises, 90% of problems can be solved by somebody who can get phone calls answered, a $20/hour intern and Excel.
    • sgt101 9y ago
      I think that's a sign of enterprise dysfunction - problems that can be solved in that way aren't real problems, they are neglect and delinquency.
      • Spooky23 9y ago
        At some level, sure. But... Consider the wide variety of solutions for taking notes. It’s a trivial problem that can be addressed in any number of ways. But it’s a problem that people spend a lot of time solving.
  • robotresearcher 9y ago
    Existing start ups can't. But if you are a well-known AI researcher, or the student of one, you start your own company with students and friends and get your team bought for the talent.
  • s73v3r_ 9y ago
    Stop wasting money on lavish offices and bro-ish perks, and rather put that money toward salaries.
    • vinceguidry 9y ago
      You'd be amazed how little impact spreading that wealth around in the form of salaries will actually make on an individual salary. And on just how much those offices and perks actually contribute to company image. Coders should stop being so mercenary.
      • s73v3r_ 9y ago
        "Coders should stop being so mercenary." Why?
        • vinceguidry 9y ago
          Because it makes everyone less civil.
          • s73v3r_ 9y ago
            I strongly disagree. And given that I'm working for a company who's entire purpose is to make money, I fail to see why I should not do the same. Nothing good comes from me forgoing making all that I can make.
            • vinceguidry 9y ago
              You'll make a lot more money just starting your own business. Like, a ridiculous amount more. Then when you start hiring, you'll understand why attitudes like yours are toxic.
              • s73v3r_ 9y ago
                Again, I'm failing to see why you're advocating for a business to make all the money it can, but why you decry the same for employees. Double standard much?
• searine 9y ago
  >a lot of companies think they need XYZ, but in fact what they need is a good product and a few algorithms requiring high-school level maths. I think this pretty much solves 90% of the problems in the software business.
• rdtsc 9y ago
  > I suspect AI today is like big data ten years ago Exactly. Also as soon as big data came around nobody was doing just data, everyone was doing big data even if they had the same 10GB MySQL database they had from previous years. AI is a bit the same. Doing any analytics? - Now it's AI. Opening and excel spreadsheet and doing a curve fit - I am a data scientist doing AI. Doing any actual ML - not learning anymore but super deep learning.
  • hoosieree 9y ago
    My job as a consultant is to tell you "your data fits in RAM" and charge you $20k. While it may seem expensive, the ROI is about 14 days, because now you don't need to hire that data scientist.
    • aantix 9y ago
      It's amazing how many startups don't re-evaluate their infrastructure requirements every couple of years and get stuck with numbers in their heads reflective of hardware prices five years ago. No need for clever optimizations - Moore's Law will bail you out a lot of times.
    • burger_moon 9y ago
      How does someone get into a position like that? Are you an independent consultant or do you work for a company? Is this exclusively what you consult on?
      • gxs 9y ago
        His comment while accurate was probably a bit dramatized for effect. My guess is he is a consultant hired to do a typical project (i.e., help me move X into the cloud, help me re-architect our data model for big data, help me implement AI for our dog walking app) and at that point he just shows them they aren't ready for it or flat out don't need it. It's just my guess, but it's what consultants do. The bad ones are happy to take on your project and charge you $400 bucks an hour. The good ones unfortunately, deal with the dilemma of turning down lucrative work in the spirit of doing what's right.
• apohn 9y ago
  >My personnal experience with data scientist and startups is that they're hired much much too early, when the product has a lot more fundamental issues to solve, and only because it looks cool to say that you're doing AI. Not just startups, but big established companies as well. At bigger companies not only do you have data and infrastructure issues, you also have business process and political issues. I've seen more than a few cases where fixing a business process would have a much higher ROI than a model, but it's easier and cheaper to hire a data scientist and make a big noise about it than it is to admit your business process (that includes 50+ people) is a mess and do the work to fix it.
• tokai 9y ago
  Winter is comming.
  • varelse 9y ago
    Well then look busy!
• jobu 9y ago
  > a lot of company think they need AI, but in fact what they need is a good product and a few algorithm requiring high-school level maths Yep, AI is the new silver bullet that will solve everyone's problems. It seems much easier to throw a million dollars at someone with the right credentials than to make hard choices and build a better product.
  • john_moscow 9y ago
    That's because you ultimately don't care for building the better product. You care to make an impeccable impression of doing the right thing - in the current market this will get you more capital than your customers will ever pay you.
    • alluro2 9y ago
      Couldn't be more true...
• jihadjihad 9y ago
  This is my experience as well, and I'm wondering if this industry could learn something from another one: video games. In that industry, many specialized technical people work on a large project, but their goals and "infrastructure" (seem to be) aligned. Are the issues we see in many software companies w.r.t. AI/ML/DS an effect of poor role definition/team hierarchy? It seems to me that a five-person team complete with an "engineer", scientist/researcher, a couple of devs for APIs and pretty pictures, and one other "utility" role would totally kill it and create amazing things. But, I've never worked in an environment where the ML people aren't completely segregated into their environment, so I don't know.
  • FLUX-YOU 9y ago
    >Are the issues we see in many software companies w.r.t. AI/ML/DS an effect of poor role definition/team hierarchy? I don't think so, unless they are getting AI/ML/DS people to do regular software features and bug fixes. Front-end/Back-end dev roles merged into full-stack roles at places, but you don't see the same merging between AI/ML/DS roles and front/back/fullstack roles.
• thecortado 9y ago
  >> In practice, they're often frustrated for years by the lack of infrastructure to work on their ideas, but live with it because life is good. Definitely heard this problem for Silicon Valley returnees to some smaller markets.
• iron0012 9y ago
  Hm, I think I disagree with this. The famous statistician and scientist RA Fisher said "To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of." and less-statistically-inclined researchers in academia have often observed this to be true: when statistical/analytical/"data" related considerations are not taken into account during the early stages (design, planning) of a project, it is very difficult (and time and money consuming) to "bolt them on" after the fact. If "AI" (or whatever you want to call it) is going to be a fundamental feature of a product, data scientists (or whatever you want to call them) should be involved right from the very beginning.