3 ms·
Increasingly data scientists need to know a thing or two about underlying tech. Otherwise you're limiting yourself to stuff that can be built on a single machin
by m0zg 5y ago
Increasingly data scientists need to know a thing or two about underlying tech. Otherwise you're limiting yourself to stuff that can be built on a single machine, and that doesn't get you very far. That said, with that list of qualifications they'll be looking for a very long time, especially if they aren't prepared to hire a $400/hr contractor to do all that stuff. Such people exist, there are just very few of them, and they're booked solid months in advance.
- savin-goyal 5y agoA single machine can take you remarkably far these days, given the availability of high RAM/Disk/CPU machines in the cloud.
- mark_l_watson 5y agoI agree. A huge GCP VPS with a good GPU attached is very inexpensive when you only start it when you are in a work sprint. Just this week I have been experimenting with SageMaker and SageMaker Studio. Too early for a real evaluation, but it looks like SageMaker Studio hits many requirements: good for experimenting, run large distributed jobs, good model and code versioning tools, easy to publish REST APIs, etc. Just yesterday someone asked me to review 3rd party tools, and I look forward to getting a better understanding of how SageMaker Studio stacks up against turn-key systems. I have built my career from standing on the shoulders of giants. I am not shy about just using the results in academic papers, using open source libraries, tools and frameworks, etc. that other people have written. So, I agree with you that so much can be done on a single beefy VPS, but services and frameworks that allow easy use of multiple servers are also important.
- m0zg 5y agoIn plain old data science? Sure. In deep learning? Nope. Gotta be distributed unless you want to wait until the Sun burns out.