4 ms·
Things I always want when looking at something new: - Where does it sit in relation to other data science components? - What does it integrate with, and what
by mooneater 8y ago
Things I always want when looking at something new:
- Where does it sit in relation to other data science components?
- What does it integrate with, and what is it agnostic to?
- Smallest self-contained use case? I see some examples in the repo readme but they are not self-contained so its harder for me to imagine its use
- pplonski86 8y agoI'm also reading the readme and trying to guess why I need this package? One more question, why treating feature engineering and ML models the same way is important?
- asavinov 8y agoBecause they are so much similar that it is much easier to implement them as part of one mechanism. For example: * Apply a (say, SVM) classification model to each object (row) by producing a new column * Generate a new column as a difference between its values and its average over all rows. In both cases, you produce a new column (=feature) by applying some transformation. Also, in both cases, you need to find parameters of this transformation from the data. In the first case, by training SVM model. In the second case, by find the average value. Conclusion: there is no essential difference between defining/training a feature and a ML model.
- pplonski86 8y agoI think it depends on your context. From theoretical point of view you can treat both (feature engineering and ML) as transformation defined by some function with parameters. From practical point of view, they can differ a lot. Imagine, you would like to handle in a different way: feature transformation that can be done on 1M rows and 2 columns in 5 seconds, and training of ML model that can take 7 days to train.
- srean 8y agoYour comment reminds me so much of my PM who asks questions like these to give an appearance that he is putting in some serious effort and energy to understand. Add a few words like 'value', 'leverage' and 'resonate' and the impersonation would be pitch perfect .... Looks up contact detail oops spoke to soon should have guessed.
- mooneater 8y agoHaha :) My comment is precisely aimed at reducing the energy needed to understand the value prop here. Why is that bad? My assumption is that would be helpful to other readers and thus the author. Do you prefer having key information buried?
- mooneater 8y ago> 'value', 'leverage' and 'resonate' You could not be more wrong about my perspective though. My question is from this perspective: I use tensorflow, keras, numpy, scikit-learn, pandas, and Im looking to understand how I would integrate these.
- mcrad 8y agoBy the same token, your comment reminds me of an engineer who puts serious effort into making his work look more important than it is. Even the term "feature engineering" is pretty ridiculous. As if academics found real-world data to be too messy and decided to outsource this little cost to.....a department of expert feature-makers(?)