vgt

42 ms·

421.
by vgt 11y ago
Just yesterday Quizlet posted a long expose that, in their opinions, shows that GCP is cheaper, more reliable (live migration), more performant than AWS, with details to boot: https://news.ycombinator.com/item?id=11260137 A
422.
by vgt 11y ago
At the end of the article, Quizlet noted their experience with GCP support: "Overall it was a smooth transition and we're very glad that we picked GCP as our provider - we've received excellent support and scaled up our deplo
423.
by vgt 11y ago
Google Cloud platform offers Custom Machine types specifically to help you configure the most optimal CPU/RAM combinations: https://cloud.google.com/custom-machine-types/ Quizlet's post alludes to Google&#x27
424.
by vgt 11y ago
Another great article previously written by Quizlet on their Google Cloud efforts: https://quizlet.com/blog/287-million-events-per-day-and-1-en... TL;DR: One engineer leveraged Google BigQuery's Streaming API to
425.
by vgt 11y ago
And some of Spotify's experiences with Google Cloud: https://labs.spotify.com/2016/03/10/spotifys-event-delivery-...
426.
Why Spotify Migrated from Kafka to Google Pub/Sub for Event Delivery(cloud.google.com)
1 points by vgt 11y ago|0 comments
427.
by vgt 11y ago
It's not so much they "couldn't", it's that things like global presence, reliability, and scalability are just part of the service. With Kafka, you can potentially design a system that has these properties, whereas
428.
by vgt 11y ago
Interesting seeing Spotify's experiences with running Kafka in production at scale.
429.
by vgt 11y ago
Stay tuned for Parts 2 + 3 :)
430.
Spotify's Kafka-Based Event Delivery System(labs.spotify.com)
122 points by vgt 11y ago|14 comments
431.
by vgt 11y ago
ping me and we can help you get this going with BigQuery :)
432.
by vgt 11y ago
yup sorry posted in wrong thread.. but please do :)
433.
by vgt 11y ago
Hey guys it's Tino. When will you contribute to our Big Data blog with your excellent ideas? :) https://cloud.google.com/blog/big-data/
434.
by vgt 11y ago
I hope that you see the irony of your argument in this thread :)
435.
by vgt 11y ago
As someone who's been on Google Cloud for 2.5 years now and has seen the growth and the transformation firsthand, I welcome this bet.
436.
by vgt 11y ago
Where'd you hear that? Not without risk of giving an argument from authority, your sources are inaccurate. As late as November of last year Snapchat was on stage talking about their broad AppEngine use. (I work on Google Cloud).
437.
by vgt 11y ago
In addition to all the excellent comments here. Let's not forget that the massive massive Snapchat runs almost entirely on Google AppEngine. So if AppEngine can handle their scale, it can handle most others :)
438.
by vgt 11y ago
That is true. And, as Spotify found out, what Spotify does can also happen in its own Data Centers :) Google does give an additional level of comfort - seamless scaling, No-Ops, powerful Big Data tools, flexibility. I'm going to shamel
439.
by vgt 11y ago
I think if Spotify dressed down Google Cloud Platform competition, no matter the merit, there would be many more complaints (ex. see comments on this technical post https://news.ycombinator.com/item?id=11029032 ). Spotify di
440.
by vgt 11y ago
Would you prefer testaments like these from Spotify engineers? https://twitter.com/plamere/status/702168809445134336 https://twitter.com/plamere/status/702169733454495745 https:/&#
441.
by vgt 11y ago
Dataproc is pretty significant compared with other managed Hadoop/Spark services. Here's how: - You get a cluster in under 90 seconds and with per-minute billing. - This lets you start with a job, then start a Dataproc cluster as
442.
by vgt 11y ago
"To reduce the impact of stragglers, we used a dynamic sharding technique called reduce subsharding. This is the precursor to fully dynamic sharding used in Dataflow." Fun ways in which internal Google tech makes its way to Google
443.
History of massive-scale sorting experiments at Google(cloud.google.com)
204 points by vgt 11y ago|77 comments
444.
by vgt 11y ago
There's a nice opinion on this from CloudTP: http://www.cloudtp.com/2016/01/21/dont-let-vendor-lock-in-fe... TL;DR: You don't get the girl if you don't ask for a dance :)
445.
Visualizing the mechanics of on-demand pricing in big data technologies(cloud.google.com)
2 points by vgt 11y ago|0 comments
446.
by vgt 11y ago
very nice analysis... I've written on this topic the other week: https://cloud.google.com/blog/big-data/2016/02/understanding... Basically, in VM terms BigQuery lets you scale to thousands of cores
447.
by vgt 11y ago
I agree with you, but re-reading the original commenter's argument, he was saying that with AWS services you don't get seamless scalability, even though there's SQS, Kinesis, Firehose. I don't think he was complaining ab
448.
by vgt 11y ago
Dataflow is a fault-tolerant deterministic processing framework, engine, and service, not a messaging queue, so it doesn't "do that" by definition.. wrong product :) That said, one may order and dedupe the message stream with
449.
by vgt 11y ago
Google Cloud SQL 2.0 is pretty neat
450.
by vgt 11y ago
The latter use case is easily handled by Dataflow (something that AWS lacks. See https://cloud.google.com/blog/big-data/2016/02/comparing-the... ). One may also say that Google has a single Global seamles

More ›