4 ms·

This year was quite fruitful. I started off dabbling a bit with basic Neural Networks. Nothing stuck in my mind until I did a mini project. In 2019 I plan to ex

by xkgt 8y ago

This year was quite fruitful. I started off dabbling a bit with basic Neural Networks. Nothing stuck in my mind until I did a mini project. In 2019 I plan to expand my knowledge to cover more algorithms and try some ML engineering tools such as MLFlow or Sagemaker. Next I learnt some Big Data technologies (Hadoop, Hive and Spark) due to job requirements. This is an ongoing process compared to Machine learning which was just a curiosity. Finally I learnt quite a bit of Ansible and little of Jenkins, once again for job requirement. While these are the major bits, I toyed around with few other tools such as OR-tools (operations research), Pandas, Matplotlib, Scrapy. On the whole, 2018 is a year I would like to replicate every year in terms of personal development.

mywrathacademia 8y ago
Any resources to get into Spark?
- xkgt 8y ago
  Quite a few out there but the problem is that of fragmentation. There have been too many API changes making following online resources tricky. Having said that, I got great value out of these resources. Make sure you are working on the right version of Spark. https://jaceklaskowski.gitbooks.io/mastering-apache-spark/ https://jaceklaskowski.gitbooks.io/mastering-apache-spark/ https://pages.databricks.com/definitive-guide-spark.html https://pages.databricks.com/definitive-guide-spark.html Databricks cloud workspace (community edition) offers free interactive spark clusters where you can practice or develop your skills https://community.cloud.databricks.com/ https://community.cloud.databricks.com/ If you want a bit of flexibility, you can launch your own cluster using flinktrock and a pair of free tier t2.micro EC2 instances https://github.com/nchammas/flintrock https://github.com/nchammas/flintrock
  - mywrathacademia 8y ago
    Had a quick look, its really useful and both of the tutorials seem thorough. Hopefully it can get someone with zero Machine Learning experience up to speed. I noticed Scala but not Java. I read that Spark was written in Java, can I use Java instead of Scala? Since you seem quite far along, can you quickly explain how Spark is used in DeepLearning/Machine Learning? Is it just a virtual machine?

by xkgt 8y ago

mywrathacademia 8y ago

Any resources to get into Spark?

xkgt 8y ago
Quite a few out there but the problem is that of fragmentation. There have been too many API changes making following online resources tricky. Having said that, I got great value out of these resources. Make sure you are working on the right version of Spark. https://jaceklaskowski.gitbooks.io/mastering-apache-spark/ https://jaceklaskowski.gitbooks.io/mastering-apache-spark/ https://pages.databricks.com/definitive-guide-spark.html https://pages.databricks.com/definitive-guide-spark.html Databricks cloud workspace (community edition) offers free interactive spark clusters where you can practice or develop your skills https://community.cloud.databricks.com/ https://community.cloud.databricks.com/ If you want a bit of flexibility, you can launch your own cluster using flinktrock and a pair of free tier t2.micro EC2 instances https://github.com/nchammas/flintrock https://github.com/nchammas/flintrock
- mywrathacademia 8y ago
  Had a quick look, its really useful and both of the tutorials seem thorough. Hopefully it can get someone with zero Machine Learning experience up to speed. I noticed Scala but not Java. I read that Spark was written in Java, can I use Java instead of Scala? Since you seem quite far along, can you quickly explain how Spark is used in DeepLearning/Machine Learning? Is it just a virtual machine?