6 ms·

Any inside stories on what twitter discovered when they benchmarked Google with other providers. I imagine they must have done a very exhaustive evaluation

gkoberger 8y ago
Or, Google thought the publicity was worth a gigantic discount. (EDIT: The provided quote from a Twitter employee hints at this directly: "the savings")
- everdev 8y ago
  A comment from a Google Cloud employee seems to suggest otherwise. > Derek’s point about savings is about list price differences that result from total system performance (and not any sort of special discounting).
  - gkoberger 8y ago
    There's a lot of ways that both can be true. For example, metering variables that are friendly to how Google does things.
- WaxProlix 8y ago
  I did a gig for <huge content producer/delivery company> and at the time mostly did AWS implementations (I work at AWS now full time, for what it's worth). The client was really interested in going with Google Cloud because they were promised essentially free storage, with no limit. This is a company with more petabytes of video footage in their data lake than I've seen before and GCP wanted to give them free storage for the prestige (and possible upside, longterm) of having Big Company on their list.
- cheriot 8y ago
  Google may be playing the long game. I've seen an effect on my team after moving from physical hardware and centralized capacity provisioning to GCP with immediate gratification hardware. It's easier to spend money so we ask for more stuff more often.
  - pdelgallego 8y ago
    it is The Jevons paradox [1]. I have seen it as well happening when I migrated a client from a standard cloud architecture to a serverless architecture. >>> In economics, the Jevons paradox (sometimes Jevons effect) occurs when technological progress or government policy increases the efficiency with which a resource is used (reducing the amount necessary for any one use), but the rate of consumption of that resource rises due to increasing demand. [1] https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
- onetimemanytime 8y ago
  This is an interesting case, sorta like Amazon going to NYC. If you give discounts and lower your profit you get something, plus PR. If you don't you loose it all. But if all ask for the Twitter discount, you're screwed. However, by by next year no one can compare the offerings side by side. Better machines, cheaper offerings etc.
- ernsheong 8y ago
  "savings" does not imply discounts, mind you. GCP is actually cheaper on most fronts compared to AWS, Azure. Quote from reply above: "Derek’s point about savings is about list price differences that result from total system performance (and not any sort of special discounting)"
andrewguenther 8y ago
For a company of Twitter's size, benchmarks likely played a small role compared to price negotiations. Being able to say Twitter runs in your cloud is worth a lot to any cloud provider. They're likely operating at steep discounts and probably played off all the major providers to get the best offer.
- deleted 8y ago
  [deleted]
- noahl 8y ago
  Agreed to all of what you said, but also, performance in some circumstances is a discount. One of the Google Cloud employees said "Derek’s point about savings is about list price differences that result from total system performance (and not any sort of special discounting)." I don't have any internal numbers, but imagine if Twitter could meet its needs with an N-node Redshift cluster or an N/2-node Dataproc cluster, because Dataproc is so much faster at flipping the bits or whatever Twitter wants to do (probably network performance really). Google could charge more per node and still come out cheaper because of performance.
StudentStuff 8y ago
Perhaps they did a solid analysis, but Twitter doesn't appear to be a rational actor. Look at Jack Dorsey's recent interview as an example, at one point he names 3 separate things as priority #1: https://www.huffingtonpost.com/entry/jack-dorsey-twitter-interview_us_5c3e2601e4b01c93e00e2a00 https://www.huffingtonpost.com/entry/jack-dorsey-twitter-int... Apparently this is a trend going back years, it doesn't seem to be doing anything good for Twitter or Square: https://sg.finance.yahoo.com/news/big-twitter-investor-defends-company-013248222.html https://sg.finance.yahoo.com/news/big-twitter-investor-defen...
- afarrell 8y ago
  You mean Square. Stripe is led by the Collison brothers but is in the same industry.
  - StudentStuff 8y ago
    Sorry, all the payments companies just kinda blend together for me (having had to deal with too many of 'em). Corrected the post above. On a sidenote, Square, Stripe, Payment Ninja, <insert payments startup> so far seems to be lipstick on the pig that is the underlying debit/credit networks, often with a hefty price for said lipstick. The US badly needs a better version of Zelle that supports business transactions.
    - afarrell 8y ago
      For recurring/invoice transactions, you could use GoCardless: https://gocardless.com/en-ca/pricing/ https://gocardless.com/en-ca/pricing/ We don't use debit/credit card networks but bank account->bank account networks like ACH, SEPA, etc. Not yet available in the US. Coming soon though.
- TheRealDunkirk 8y ago
  People have to come to realize that nothing that gets said at this level should be taken literally. At the C-level of Fortune 500 companies, everything is crafted to create a narrative. People claiming that Twitter or Facebook has a political bias are missing the real point. For example, I think that Zuck really, truly, doesn't give a fink about political bias. How could he? He'd never get any sleep! In his position, all he cares about is the money. If that takes making this or that comment or public statement or testimony to Congress, it doesn't matter. It's all geared towards and carefully crafted towards keeping the money flowing. In that sense, he and the top brass at a place like Facebook are all political whores, so to speak. Selling their convictions for whatever keeps the spigot open.
tanilama 8y ago
It will be simplistic to assume such decision is tech-driven. Per my experience, it is mostly business driven. Who offers the best deal with the lowest price, gets the business. You'd be surprised, if you are a big enough customer, even better if you are a reputable one, how much catering the cloud provider would be willing to offer.
opportune 8y ago
Hadoop is an offline storage mechanism used for analytics, random access benchmarks barely matter compared to the overall cost of storage+compute so long as access time aren't atrocious
derekalyon 8y ago
This is Derek. We shared a bunch of stuff in this video: https://youtu.be/4FLFcWgZdo4 https://youtu.be/4FLFcWgZdo4 One of the interesting benchmarks was GridMix performance at scale. And overall performance for throughput-oriented workloads like Hadoop was generally strong. Most cloud benchmarking I have seen sticks to micro-benchmarks and single-node benchmarks, so they miss this.

gkoberger 8y ago

Or, Google thought the publicity was worth a gigantic discount. (EDIT: The provided quote from a Twitter employee hints at this directly: "the savings")

everdev 8y ago
A comment from a Google Cloud employee seems to suggest otherwise. > Derek’s point about savings is about list price differences that result from total system performance (and not any sort of special discounting).
- gkoberger 8y ago
  There's a lot of ways that both can be true. For example, metering variables that are friendly to how Google does things.
WaxProlix 8y ago
I did a gig for <huge content producer/delivery company> and at the time mostly did AWS implementations (I work at AWS now full time, for what it's worth). The client was really interested in going with Google Cloud because they were promised essentially free storage, with no limit. This is a company with more petabytes of video footage in their data lake than I've seen before and GCP wanted to give them free storage for the prestige (and possible upside, longterm) of having Big Company on their list.
cheriot 8y ago
Google may be playing the long game. I've seen an effect on my team after moving from physical hardware and centralized capacity provisioning to GCP with immediate gratification hardware. It's easier to spend money so we ask for more stuff more often.
- pdelgallego 8y ago
  it is The Jevons paradox [1]. I have seen it as well happening when I migrated a client from a standard cloud architecture to a serverless architecture. >>> In economics, the Jevons paradox (sometimes Jevons effect) occurs when technological progress or government policy increases the efficiency with which a resource is used (reducing the amount necessary for any one use), but the rate of consumption of that resource rises due to increasing demand. [1] https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
onetimemanytime 8y ago
This is an interesting case, sorta like Amazon going to NYC. If you give discounts and lower your profit you get something, plus PR. If you don't you loose it all. But if all ask for the Twitter discount, you're screwed. However, by by next year no one can compare the offerings side by side. Better machines, cheaper offerings etc.
ernsheong 8y ago
"savings" does not imply discounts, mind you. GCP is actually cheaper on most fronts compared to AWS, Azure. Quote from reply above: "Derek’s point about savings is about list price differences that result from total system performance (and not any sort of special discounting)"

andrewguenther 8y ago

For a company of Twitter's size, benchmarks likely played a small role compared to price negotiations. Being able to say Twitter runs in your cloud is worth a lot to any cloud provider. They're likely operating at steep discounts and probably played off all the major providers to get the best offer.

deleted 8y ago
[deleted]
noahl 8y ago
Agreed to all of what you said, but also, performance in some circumstances is a discount. One of the Google Cloud employees said "Derek’s point about savings is about list price differences that result from total system performance (and not any sort of special discounting)." I don't have any internal numbers, but imagine if Twitter could meet its needs with an N-node Redshift cluster or an N/2-node Dataproc cluster, because Dataproc is so much faster at flipping the bits or whatever Twitter wants to do (probably network performance really). Google could charge more per node and still come out cheaper because of performance.

StudentStuff 8y ago

Perhaps they did a solid analysis, but Twitter doesn't appear to be a rational actor. Look at Jack Dorsey's recent interview as an example, at one point he names 3 separate things as priority #1: https://www.huffingtonpost.com/entry/jack-dorsey-twitter-interview_us_5c3e2601e4b01c93e00e2a00 https://www.huffingtonpost.com/entry/jack-dorsey-twitter-int... Apparently this is a trend going back years, it doesn't seem to be doing anything good for Twitter or Square: https://sg.finance.yahoo.com/news/big-twitter-investor-defends-company-013248222.html https://sg.finance.yahoo.com/news/big-twitter-investor-defen...

afarrell 8y ago
You mean Square. Stripe is led by the Collison brothers but is in the same industry.
- StudentStuff 8y ago
  Sorry, all the payments companies just kinda blend together for me (having had to deal with too many of 'em). Corrected the post above. On a sidenote, Square, Stripe, Payment Ninja, <insert payments startup> so far seems to be lipstick on the pig that is the underlying debit/credit networks, often with a hefty price for said lipstick. The US badly needs a better version of Zelle that supports business transactions.
  - afarrell 8y ago
    For recurring/invoice transactions, you could use GoCardless: https://gocardless.com/en-ca/pricing/ https://gocardless.com/en-ca/pricing/ We don't use debit/credit card networks but bank account->bank account networks like ACH, SEPA, etc. Not yet available in the US. Coming soon though.
TheRealDunkirk 8y ago
People have to come to realize that nothing that gets said at this level should be taken literally. At the C-level of Fortune 500 companies, everything is crafted to create a narrative. People claiming that Twitter or Facebook has a political bias are missing the real point. For example, I think that Zuck really, truly, doesn't give a fink about political bias. How could he? He'd never get any sleep! In his position, all he cares about is the money. If that takes making this or that comment or public statement or testimony to Congress, it doesn't matter. It's all geared towards and carefully crafted towards keeping the money flowing. In that sense, he and the top brass at a place like Facebook are all political whores, so to speak. Selling their convictions for whatever keeps the spigot open.

tanilama 8y ago

It will be simplistic to assume such decision is tech-driven. Per my experience, it is mostly business driven. Who offers the best deal with the lowest price, gets the business. You'd be surprised, if you are a big enough customer, even better if you are a reputable one, how much catering the cloud provider would be willing to offer.

opportune 8y ago

Hadoop is an offline storage mechanism used for analytics, random access benchmarks barely matter compared to the overall cost of storage+compute so long as access time aren't atrocious

derekalyon 8y ago

This is Derek. We shared a bunch of stuff in this video: https://youtu.be/4FLFcWgZdo4 https://youtu.be/4FLFcWgZdo4 One of the interesting benchmarks was GridMix performance at scale. And overall performance for throughput-oriented workloads like Hadoop was generally strong. Most cloud benchmarking I have seen sticks to micro-benchmarks and single-node benchmarks, so they miss this.