4 ms·
Have you checked out DCOS? Once you have your cluster set up, your path to installing Spark is running "dcos package install spark" on your laptop.
by ryanl-ee 11y ago
Have you checked out DCOS? Once you have your cluster set up, your path to installing Spark is running "dcos package install spark" on your laptop.
- minimaxir 11y agoHuh, no I haven't. Although the package support appears to be more narrow than with Kubernetes/Docker, but DCOS is a possibility too.
- ptrincr 11y agoYou should also check out Ambari, fantastic open source Hadoop/spark cluster management tool.
- mindcrime 11y agoSecond the motion. Ambari is actually one of the key elements of the tech stack we're using for our service. With the "Blueprints" feature and the REST API, it's a really convenient way to automate the provisioning of Spark/Hadoop clusters. Even better, Ambari isn't actually limited to installing just Hadoop/Spark, etc. In principle, you could extend it to take care of installing pretty much anything.
- ryanl-ee 11y agoWell, the included packages in the "dcos package" CLI repo are just the prepackaged frameworks. You can actually deploy Kubernetes on top of DCOS in the same way - it natively supports Docker containers as an execution method.