4 ms·

I found the Super Cloud bit interesting. 350 employees haven't been able to move an e-commerce store onto Amazon's infrastructure in 27+ months. "For well over

I found the Super Cloud bit interesting. 350 employees haven't been able to move an e-commerce store onto Amazon's infrastructure in 27+ months. "For well over a year, Van Beek wasn't even sure the migration was technically possible ... the difficulty of this effort is almost unfathomable". Are the services Amazon makes available to its own teams substantially different from AWS? If so, why not just use AWS, and figure out the internal billing?

x0x0 11y ago
consider that running a complex piece of software on o&o hardware is very different than amazon. You have to re-architect your app to tolerate (1) hardware just disappearing (yes that happens in a data center but much less often than in amazon), (2) generally lower-powered hardware (no giant db servers running on TB+ ram / fusion-io drives / bonded 1g ethernet hardware), (3) often shitty network and ebs performance, (4) generally a very different HA architecture. Not to mention ops relearning everything they know about how to keep your site up.
dastbe 11y ago
While there are differences in what people use internally vs externally for some operations (see Apollo vs CodeDeploy in a few blog posts and talks), what they're talking about here is migrating Zappos onto/into the retail infrastructure and applications.
ww520 11y ago
It's probably rewriting Zoppos retail app in Amazon's retail platform. That's like a redesign, re-architect, and rewrite of the whole app, plus the whole backend pipeline. That's a huge task. Also retail is driven by huge amount of data: product category, inventory, pricing, customer info, experiment data, marketing, ad, clicks, billing, etc. Migrating those data without interruption with live operation is like changing engine in mid flight.
- BinaryIdiot 11y ago
  You're right it's pretty challenging but 27+ months and 300+ engineers challenging? I would have suspected to take half the time which makes me wonder what their architecture originally looked like and what they've been doing for so long.
  - rorykoehler 11y ago
    And here I am trying to build meaningful apps on my own :/
    - Luyt 11y ago
      I've noticed that often one man can accomplish more than a whole group, especially if the group gets larger.
  - bane 11y ago
    What I think you're hinting at is that a development organization with nobody driving direction devolves into a pile of people working on pet projects rather than doing the terrible boring work of shipping product. https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law
    - AndyNemmity 11y ago
      I don't see how Conway's law relates to what you said. Can you please elaborate? Do you think if no hirearchy is driving direction then there is a lack of communication between groups that work on projects?
      - myhf 11y ago
        If there is no hierarchy driving direction then there are 350 groups.
      - jhou2 11y ago
        Holacracy implies no organization. It is a freeform, organic, shapeless philosophy with possibly no direction. The migration project has taken 27 months with 300 staff, about 8100 man-months, with apparently little to show for progress. Lots of people in a project with no tangible features, like a blob. I would say Conway's law fits perfectly.
- gaius 11y ago
  A huge task? Sure. But product catalogs and shopping carts and payment processing are 90s tech. They could have re-implemented from scratch on AWS with the same graphics assets on the front end and cut over in a fraction of the time. Instead they are trying to force a square peg into a round hole and wondering why no matter hard they push it won't go.
wavefunction 11y ago
I work with a guy who worked on the Digital products side of the Amazon store and he said it was all done in Perl + Mason which sort of sounds like a nightmare, though he says he enjoyed it.
- tamana 11y ago
  Perl+Mason is a frontend tech (not all the way to the database like rails ActiveRecord) and works great.

x0x0 11y ago

consider that running a complex piece of software on o&o hardware is very different than amazon. You have to re-architect your app to tolerate (1) hardware just disappearing (yes that happens in a data center but much less often than in amazon), (2) generally lower-powered hardware (no giant db servers running on TB+ ram / fusion-io drives / bonded 1g ethernet hardware), (3) often shitty network and ebs performance, (4) generally a very different HA architecture. Not to mention ops relearning everything they know about how to keep your site up.

dastbe 11y ago

While there are differences in what people use internally vs externally for some operations (see Apollo vs CodeDeploy in a few blog posts and talks), what they're talking about here is migrating Zappos onto/into the retail infrastructure and applications.

ww520 11y ago

It's probably rewriting Zoppos retail app in Amazon's retail platform. That's like a redesign, re-architect, and rewrite of the whole app, plus the whole backend pipeline. That's a huge task. Also retail is driven by huge amount of data: product category, inventory, pricing, customer info, experiment data, marketing, ad, clicks, billing, etc. Migrating those data without interruption with live operation is like changing engine in mid flight.

BinaryIdiot 11y ago
You're right it's pretty challenging but 27+ months and 300+ engineers challenging? I would have suspected to take half the time which makes me wonder what their architecture originally looked like and what they've been doing for so long.
- rorykoehler 11y ago
  And here I am trying to build meaningful apps on my own :/
  - Luyt 11y ago
    I've noticed that often one man can accomplish more than a whole group, especially if the group gets larger.
- bane 11y ago
  What I think you're hinting at is that a development organization with nobody driving direction devolves into a pile of people working on pet projects rather than doing the terrible boring work of shipping product. https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law
  - AndyNemmity 11y ago
    I don't see how Conway's law relates to what you said. Can you please elaborate? Do you think if no hirearchy is driving direction then there is a lack of communication between groups that work on projects?
    - myhf 11y ago
      If there is no hierarchy driving direction then there are 350 groups.
    - jhou2 11y ago
      Holacracy implies no organization. It is a freeform, organic, shapeless philosophy with possibly no direction. The migration project has taken 27 months with 300 staff, about 8100 man-months, with apparently little to show for progress. Lots of people in a project with no tangible features, like a blob. I would say Conway's law fits perfectly.
gaius 11y ago
A huge task? Sure. But product catalogs and shopping carts and payment processing are 90s tech. They could have re-implemented from scratch on AWS with the same graphics assets on the front end and cut over in a fraction of the time. Instead they are trying to force a square peg into a round hole and wondering why no matter hard they push it won't go.

wavefunction 11y ago

I work with a guy who worked on the Digital products side of the Amazon store and he said it was all done in Perl + Mason which sort of sounds like a nightmare, though he says he enjoyed it.

tamana 11y ago
Perl+Mason is a frontend tech (not all the way to the database like rails ActiveRecord) and works great.