5 ms·
I'm slightly disappointed that the term "data mining" fell out of favour, because I think mining helps convey the idea that you have to sift through a lot of wo
by ploika 6y ago
I'm slightly disappointed that the term "data mining" fell out of favour, because I think mining helps convey the idea that you have to sift through a lot of worthless crap before maybe, if you're lucky, getting something valuable out of your resource. This is not at all intended as snark - it's just that (as you already know) the huge data set might not have enough value in it to start a business.
Regardless of commercial value, it could be a very nice resource to experiment with NLP methods - simple TF-IDF for classification models, topic modelling for unsupervised learning, training/fine-tuning your own BERT model, etc.
To actually answer the question, the best "Big Data" (very much distinct from "Data Science") course I ever did is a now totally outdated course by Cloudera that went through Hadoop in great detail - mappers, reducers, shuffle-and-sort, the works. It really helped me understand what was going on under the hood when I ended up using Hive and then Spark a few years later. It might have merged into Developer Training for Spark and Hadoop"[0], though I'm not completely sure about that.
[0] https://www.cloudera.com/about/training/courses/developer-training-for-spark-and-hadoop.html https://www.cloudera.com/about/training/courses/developer-tr...