3 ms·
A machine learning model is core to my company, and core to that is pulling in, cleaning, and working with large dataset from a variety of platforms. This means
by audleman 10y ago
A machine learning model is core to my company, and core to that is pulling in, cleaning, and working with large dataset from a variety of platforms. This means:
- excellence in being able to clean data at the column level for millions of data points
- knowing how to work with such large scale data in a time efficient manner. One of our newer hires worked at the Postgres/SSD level to optimize and got it to where he could produce a full set within 5 minutes. Before that it was once every few months.
Being able to do these things is a prerequisite to building even a prototype of a model, and it requires substantial real-world programming experience to deal with those complexities.