3 ms·
Working with a large well known tech company, with surprisingly basic/non-existent ML until only very recently. Using Redshift to do a lot of the heavy lifting
by Simon_O_Rourke 5y ago
Working with a large well known tech company, with surprisingly basic/non-existent ML until only very recently.
Using Redshift to do a lot of the heavy lifting and initial data preparation, then SageMaker for hosting models and scoring, and Tableau for dashboards.
While you can do training within SageMaker, we have a cluster of EC2 instances using H2O libraries (xgboost) to train, then wrap the resulting model as a docker image and deploy it to ECR and link to a SageMaker endpoint.
Clunky and very much human-in-the-loop for training and deployment, but you can't run before you can crawl in this space.
- billsmithwicks 5y agoI found Redshift to be far inferior to Snowflake as a data warehouse for marshalling any tables or views you need for ML work. There's lots of statistical functions available within Snowflake that will speed things up for you if you need pre-calculations on feature sets.