3 ms·
I really love seeing projects like this. My humble two cents is that in past projects I've often found the process to create the CSV that goes into a model is
by czhu12 6y ago
I really love seeing projects like this.
My humble two cents is that in past projects I've often found the process to create the CSV that goes into a model is often much more time consuming and error prone than actually training a model. I really love the dataset operations that are provided.
Do you have any thoughts about what a gold standard library for data preprocessing would look like? If you have any plans to move further in that direction? Any projects that you find compelling in that space?
- nidhaloff 6y agoHi, thanks for your feedback. hmm any machine learning project has to start with a dataset. Most of the time you will have to construct it (or take an existing one and update it) manually. Sure there are tool that generate a dummy dataset for you but that would be just for playing a world and certainly not for a real world use/production. We are actually working on adding support for text, excel and json format in igel. We already implemented some of the famous preprocessing methods in igel, which you can use by providing them in the yaml file. Now about preprocessing libraries, I personally use numpy, pandas and some of sklearn functionality to preprocess data. Furthermore, I use matplotlib and seaborn for some visualisation & further analysis.