samkone

8 ms·

31.
Making Apache Kafka Elastic with Apache Mesos – Mesosphere(mesosphere.com)
3 points by samkone 11y ago|1 comments
32.
by samkone 11y ago
Why is it being posted all over again here.
33.
Making HDFS More Reliable on Mesos [Mesosphere Hackweek](mesosphere.com)
3 points by samkone 11y ago|0 comments
34.
by samkone 11y ago
We are using Mesos/Marathon/Chronos/Docker/Bamboo on 200+ nodes on Google compute. Both for Data processing workloads and web applications (intensives advertising real time bidding servers). Using Mesosphere Packages, Sa
35.
by samkone 11y ago
what's your issue with Zk. It might be overweight but so far in ourdeployments it behaves a lot better than etcd. But if you're interested there's a lot being done with etcd and consul. I haven't seen etcd handling the k
36.
by samkone 11y ago
I know two companies running running openshift on Mesos. The reason modern Paas aren't enough, is the factat some scale you're not just running web services. But also rather complex data pipelines involving distributed data syste
37.
by samkone 11y ago
This cracks me up a bit. I don't have anything against whiteboarding, even though I find it meaningless. With this you more often than not have false positives. And if Google had so many resumes they have to filter through, then why do
38.
by samkone 11y ago
Yes opscenter, netlfix's priam, Cassandra cluster manager and the usual monitoring stuff, Datadog in our case. So far so good.
39.
by samkone 11y ago
If a company blames its demise on a piece of tech, then they either need better engineers to change that tech. Or the product had no future at all. And Cassandra has changed a lot since then.
40.
by samkone 11y ago
So are you still running Cassandra? And why rely on counters for analytics when it's wildly known to be a bad idea.
41.
by samkone 11y ago
Google doesn't offer a managed cassandra, rather a manager Bigtable with the HBase Api. But we run a huge Cassandra install in multi dacenters on GCP. And we're good. And yes it requires you to understand distributed storage syste
42.
by samkone 11y ago
Here's laughing. I guess you chose convenience over performance or scalability, which is a valid choice. But We're running a 1000 Cassandra nodes to handle Millions of ops where mongo would simply require too much admin work. the
43.
by samkone 11y ago
which was also started at Airbnb.
44.
by samkone 11y ago
Remember that Borg is an old thing. At the time they probably started out with chroot jails, because cgroups weren't ready yet. But Kubernetes, Mesos use docker and cgroups(Mesos).
45.
by samkone 11y ago
to get an idea look at how Hadoop or spark run on Mesos. You push the data locality into tasks, executors and their caches.
46.
by samkone 11y ago
Try Mesos/Marathon in addition to Kubernetes. Been running Kubernetes on Mesos and it's just great.
47.
by samkone 11y ago
Or you can use the service account to avoid dancing around gcloud and others. But I agree Google has a problem building stuff for us common humans who do not work at the big G.
48.
by samkone 11y ago
Lol He can't because Oracle licensing forbids to do this kind of work and make the results public.
49.
by samkone 11y ago
Yes you can use IPython with Spark for Pyspark. Spark-notebook and Zeppelin, gives let's you use Scala with Spark,and support all of the other spark libraries. There are other projects like spark-kernel https://github.com&#x
50.
by samkone 12y ago
Yarn tries to do the same thing, but is quite different. It's skewed toward running short running jobs, like batch jobs, it was meant for hadoop and co after all. Borg/Omega seems more like a combination of Mesos at the scheduler