4 ms·

> virtually infinite auto scaling Ah, the bullshit. When we showed GCS engineers our hadoop workload, their eyes went really big and told us that they don't ha

by pmlnr 6y ago

> virtually infinite auto scaling Ah, the bullshit. When we showed GCS engineers our hadoop workload, their eyes went really big and told us that they don't have that kind of capacity in a single AZ.

sgt101 6y ago
Then you are working at hyperscale yourselves already, and there are maybe 50 organisations like you out there. Maybe. Right now I don't think that there is a single telco that has the scale to break GCP, or a single manufacturing company (exempt Tesla) or a single energy company. Or bank. I agree that pharma, social media and streaming may do this. But since Netflix is on AWS I am doubtful of the last.
- vntok 6y ago
  Total & Aramco come to mind.
  - sgt101 6y ago
    Ok I buy that one. Oil and gas have some big problems..
    - em500 6y ago
      I worked for one of the top-10 ecommerce shops (by transaction value) in the world. It was interesting for me to find out that the aggregate CPU power of our hadoop cluster seems comparable to what an F1 teams uses to simulate air dynamics to tune their cars. There's a big world out there, there are probably tons of big computer users that we are largely unaware of.
- ForHackernews 6y ago
  Plenty of scientific computing applications fall over on the "cloud", too. The LHC produces somewhere between 20-30 petabytes/year[0][1] (at a rate of about 200 Megabyte/second[2] after ignoring many "uninteresting" events) The LHC is a big name, but there are many other applications in disciplines like physics, genomics, meteorology etc for whom "just use the cloud" is a terrible suggestion. [0] https://www.popularmechanics.com/science/a16716/lhc-data-visualization/ https://www.popularmechanics.com/science/a16716/lhc-data-vis... [1] https://home.cern/resources/faqs/facts-and-figures-about-lhc https://home.cern/resources/faqs/facts-and-figures-about-lhc [2] https://www.lhc-closer.es/taking_a_closer_look_at_lhc/0.lhc_data_analysis https://www.lhc-closer.es/taking_a_closer_look_at_lhc/0.lhc_...
  - sgt101 6y ago
    I think that alphafold may indicate that biotech isn't as disjoint from cloud solutions as many suppose. Super computing is also getting eaten as grid solutions go into hyperscale, but yes ..I find you LHC and the mega telescopes.
  - pjmlp 6y ago
    Actually, Grid Computing.
- kazen44 6y ago
  i think you greatly misunderstand the size of some telco's and Infrastructure providers. also, some manufacturing giants are truly enormous.. samsung for instance makes everything from smartphones to artillery installations.. same goes for hyundai, mitsubishi etc..
  - sgt101 6y ago
    I did big data at one of the biggest European telcos. The only thing that kept any of it on prem was capex Vs opex (and we had sunk capex so that wasn't coming back into the budget) AT&T claimed their Hadoop installation was ten times bigger than ours, but it was still will within the parameters of any of the hyperscalers.
- pmlnr 6y ago
  > or a single manufacturing company (exempt Tesla) Tesla is TINY. Try things like Siemens. EDIT Besides, the point is not the whole of AWS of GCS; it's the about availability zones vs singular datacentres. Distributed computing doesn't really like to throw data even within the same AZ - in-DC will always be faster. AZ and the modern idea of distribution suit certain things - streaming, for example, benefits from it, but not everything.

by pmlnr 6y ago

> virtually infinite auto scaling Ah, the bullshit. When we showed GCS engineers our hadoop workload, their eyes went really big and told us that they don't have that kind of capacity in a single AZ.

sgt101 6y ago

Then you are working at hyperscale yourselves already, and there are maybe 50 organisations like you out there. Maybe. Right now I don't think that there is a single telco that has the scale to break GCP, or a single manufacturing company (exempt Tesla) or a single energy company. Or bank. I agree that pharma, social media and streaming may do this. But since Netflix is on AWS I am doubtful of the last.

vntok 6y ago
Total & Aramco come to mind.
- sgt101 6y ago
  Ok I buy that one. Oil and gas have some big problems..
  - em500 6y ago
    I worked for one of the top-10 ecommerce shops (by transaction value) in the world. It was interesting for me to find out that the aggregate CPU power of our hadoop cluster seems comparable to what an F1 teams uses to simulate air dynamics to tune their cars. There's a big world out there, there are probably tons of big computer users that we are largely unaware of.
ForHackernews 6y ago
Plenty of scientific computing applications fall over on the "cloud", too. The LHC produces somewhere between 20-30 petabytes/year[0][1] (at a rate of about 200 Megabyte/second[2] after ignoring many "uninteresting" events) The LHC is a big name, but there are many other applications in disciplines like physics, genomics, meteorology etc for whom "just use the cloud" is a terrible suggestion. [0] https://www.popularmechanics.com/science/a16716/lhc-data-visualization/ https://www.popularmechanics.com/science/a16716/lhc-data-vis... [1] https://home.cern/resources/faqs/facts-and-figures-about-lhc https://home.cern/resources/faqs/facts-and-figures-about-lhc [2] https://www.lhc-closer.es/taking_a_closer_look_at_lhc/0.lhc_data_analysis https://www.lhc-closer.es/taking_a_closer_look_at_lhc/0.lhc_...
- sgt101 6y ago
  I think that alphafold may indicate that biotech isn't as disjoint from cloud solutions as many suppose. Super computing is also getting eaten as grid solutions go into hyperscale, but yes ..I find you LHC and the mega telescopes.
- pjmlp 6y ago
  Actually, Grid Computing.
kazen44 6y ago
i think you greatly misunderstand the size of some telco's and Infrastructure providers. also, some manufacturing giants are truly enormous.. samsung for instance makes everything from smartphones to artillery installations.. same goes for hyundai, mitsubishi etc..
- sgt101 6y ago
  I did big data at one of the biggest European telcos. The only thing that kept any of it on prem was capex Vs opex (and we had sunk capex so that wasn't coming back into the budget) AT&T claimed their Hadoop installation was ten times bigger than ours, but it was still will within the parameters of any of the hyperscalers.
pmlnr 6y ago
> or a single manufacturing company (exempt Tesla) Tesla is TINY. Try things like Siemens. EDIT Besides, the point is not the whole of AWS of GCS; it's the about availability zones vs singular datacentres. Distributed computing doesn't really like to throw data even within the same AZ - in-DC will always be faster. AZ and the modern idea of distribution suit certain things - streaming, for example, benefits from it, but not everything.