6 ms·

In my experience, the title 'Data Scientist' has become to mean data analyst at most companies, meaning working with Excel, Tableau, and SQL. Maybe R if you're

by itg 8y ago

In my experience, the title 'Data Scientist' has become to mean data analyst at most companies, meaning working with Excel, Tableau, and SQL. Maybe R if you're lucky. Companies doing ML/AI will usually have a small team of Research Scientists who mostly hold PhDs, and a team of supporting ML Engineers.

tixocloud 8y ago
I’d be more inclined to think that at a startup, because of the lack of infrastructure, a data scientist will do a lot more than research.
- mlthoughts2018 8y ago
  It’s even worse at established companies because data scientists still only do the data plumbing and simplistic analytics tasks, but not due to anything reasonable, like the “many hats” needs of a startup, but instead because of IT dysfunctiom and bureaucracy.
  - tixocloud 8y ago
    Think it depends but it does happen. We’ve been able to carve ourselves out of IT with our own infrastructure and our data scientists specifically focus on research and analysis. The only issue is when we try to get anything to prod and we hit IT.
    - mlthoughts2018 8y ago
      > “The only issue is when we try to get anything to prod and we hit IT.” Which inevitably means the one person on the data science team who is good with linux and docker suddenly becomes the IT wizard, and their time gets sucked up by having to find ways to go around utterly stupid barriers and tactics used by IT to avoid doing work to help you. Source: I am currently this person for my team.
      - deleted 8y ago
        [deleted]
      - tixocloud 8y ago
        I feel your pain. I am in it too. Would love to get your thoughts some time - I'm building a product to make life a little bit easier (3 step deploy model as API) but with a vision towards more broaded deployment usecases. Your advice will be valuable and much appreciated!
      - logosmonkey 8y ago
        As someone who is about to be tasked with building this sort of infrastructure for a hospital system what would be your dream architecture? I hate IT hurdles and I am hoping I can avoid building them into our infrastructure.
        mlthoughts2018 8y ago
        The number one thing is to make containers a first class deployment and provisioning artifact. As long as dev teams can control their own containers, they will be able to do what they need no matter how arbitrary or assumption-breaking. Do not ever require dev teams to go through IT to get their chosen tools deployed to the right places or with the right resources provisioned. Never. This is the root of all evil with infra teams: if they see themselves or their mandate as being gatekeepers of provisioned resources, then dev teams have lost and you as a data scientist / ML engineer, you’ll never get your work done. As an ML engineer, I want to define the entire runtime and development environments of any analytics artifacts or web services that I create, and to change these environments as needed, as indicated by what’s required to get the job done. Let me define containers, hook them up in whatever CI tools are used, push and pull them from some internal container repository, and describe the configuration for the resources they need. Offer that as the contract to dev teams and then infra’s job is to maintain the underlying data center that physically supports running the containers and occasional hand holding for special exceptions, networking, secrets management, and cost tracking.
        logosmonkey 8y ago
    - telchar 8y ago
      Don't know why you're getting downvoted for this - we have the same pain. We're not allowed to have a sysadmin or administer our own servers but IT is busy supporting thousands of non-computery-users. To them we're a fly in the ointment.
      - tixocloud 8y ago
        I find it quite interesting as technically speaking both IT and data science are technical. Pardon the pun. The real value of data science teams comes from great infrastructure in addition to a solid team so it is a bit of a pity sometimes.
compcoffee 8y ago
>Companies doing ML/AI will usually have a small team of Research Scientists who mostly hold PhDs How many companies are doing genuine AI research, as opposed to applying the research and tooling to their unique business problems?

by itg 8y ago

tixocloud 8y ago

I’d be more inclined to think that at a startup, because of the lack of infrastructure, a data scientist will do a lot more than research.

mlthoughts2018 8y ago
It’s even worse at established companies because data scientists still only do the data plumbing and simplistic analytics tasks, but not due to anything reasonable, like the “many hats” needs of a startup, but instead because of IT dysfunctiom and bureaucracy.
- tixocloud 8y ago
  Think it depends but it does happen. We’ve been able to carve ourselves out of IT with our own infrastructure and our data scientists specifically focus on research and analysis. The only issue is when we try to get anything to prod and we hit IT.
  - mlthoughts2018 8y ago
    > “The only issue is when we try to get anything to prod and we hit IT.” Which inevitably means the one person on the data science team who is good with linux and docker suddenly becomes the IT wizard, and their time gets sucked up by having to find ways to go around utterly stupid barriers and tactics used by IT to avoid doing work to help you. Source: I am currently this person for my team.
    - deleted 8y ago
      [deleted]
    - tixocloud 8y ago
      I feel your pain. I am in it too. Would love to get your thoughts some time - I'm building a product to make life a little bit easier (3 step deploy model as API) but with a vision towards more broaded deployment usecases. Your advice will be valuable and much appreciated!
    - logosmonkey 8y ago
      As someone who is about to be tasked with building this sort of infrastructure for a hospital system what would be your dream architecture? I hate IT hurdles and I am hoping I can avoid building them into our infrastructure.
      - mlthoughts2018 8y ago
        The number one thing is to make containers a first class deployment and provisioning artifact. As long as dev teams can control their own containers, they will be able to do what they need no matter how arbitrary or assumption-breaking. Do not ever require dev teams to go through IT to get their chosen tools deployed to the right places or with the right resources provisioned. Never. This is the root of all evil with infra teams: if they see themselves or their mandate as being gatekeepers of provisioned resources, then dev teams have lost and you as a data scientist / ML engineer, you’ll never get your work done. As an ML engineer, I want to define the entire runtime and development environments of any analytics artifacts or web services that I create, and to change these environments as needed, as indicated by what’s required to get the job done. Let me define containers, hook them up in whatever CI tools are used, push and pull them from some internal container repository, and describe the configuration for the resources they need. Offer that as the contract to dev teams and then infra’s job is to maintain the underlying data center that physically supports running the containers and occasional hand holding for special exceptions, networking, secrets management, and cost tracking.
        logosmonkey 8y ago
  - telchar 8y ago
    Don't know why you're getting downvoted for this - we have the same pain. We're not allowed to have a sysadmin or administer our own servers but IT is busy supporting thousands of non-computery-users. To them we're a fly in the ointment.
    - tixocloud 8y ago
      I find it quite interesting as technically speaking both IT and data science are technical. Pardon the pun. The real value of data science teams comes from great infrastructure in addition to a solid team so it is a bit of a pity sometimes.

compcoffee 8y ago

>Companies doing ML/AI will usually have a small team of Research Scientists who mostly hold PhDs How many companies are doing genuine AI research, as opposed to applying the research and tooling to their unique business problems?