4 ms·

Nike's data engineering is very bad. It's hundreds of temporary contractors, mostly offshore, all with 6-18 month tenures, and everyone reinvents their own squa

Nike's data engineering is very bad. It's hundreds of temporary contractors, mostly offshore, all with 6-18 month tenures, and everyone reinvents their own square wheel. Thousand upon thousands of abandoned confluence pages of documentation. The most convoluted SQL and data architecture you'll ever find. Getting answers to simple questions like "How many shoes did we sell in-store vs ecommerce last week?" is a nearly impossible task.

notyourwork 2y ago
> Getting answers to simple questions like "How many shoes did we sell in-store vs ecommerce last week?" is a nearly impossible task. I find this type of thing scary as an outsider looking in. How a company so large has such immature engineering continues to astonish me.
- shepardrtc 2y ago
  > I find this type of thing scary as an outsider looking in. How a company so large has such immature engineering continues to astonish me. It's management that doesn't want to risk their positions by doing the very difficult business of either starting over or properly simplifying their stack. It's not easy, it's not quick, but if they can't even answer that basic question then they need to do the work.
  - crowcroft 2y ago
    To defend management a little bit, these massive companies have existed through many eras of technology with many different managers. They work with many external companies in many different ways. They have an exceptionally complex, but functioning tech stack, that allows all of these many dependencies to function together. Lastly, they are successful as they are! It's not usually an issue of immaturity, it's just really hard. To make things worse, often people don't really want to do the work because literally any other data engineering job would probably be more enjoyable. Simplifying the tech stack would probably require simplifying their business operations, which probably means less revenue. Starting over is often literally not possible because there are so many interconnected systems that aren't all necessarily owned by the company trying to make the decision...
    - rexpop 2y ago
      > Simplifying the tech stack would probably require simplifying their business operations, which probably means less revenue. I don't understand this perspective. Simplifying the tech stack might mean taking multiple services in multiple languages, and deprecating some in favor of migrating that functionality to the most maintainable codebase. This shouldn't mean "simplifying their business operations", or affecting their business operations in any way.
      - crowcroft 2y ago
        I agree there probably are areas were they can simplify with no impact to operations. But I would imagine there are a lot of pieces of apparent cruft hanging around that is actually there because if you remove it things break. Maybe a large retailer that you rely on requires an integration with an old version of SAP, and then a logistics partner only provides files over FTP, and you need to use OCR to retrieve any data from the files they're sending. Management can't just mandate that you will 'simplify the tech stack'. Even refactoring smaller parts of the tech stack is often a pretty massive job.
        rexpop 2y ago
        "Look, my setup works for me. Just add an option to reenable spacebar heating."[0] > Management can't just mandate that you will 'simplify the tech stack'. In my experience, it's the devs who want to do reactors for love of the craft and satisfaction of a job "well" done, while management seeks to sweep as much technical "debt" as possible under the rug, and they'll burn through as many Romanians' nights' sleeps as it takes to reach their numbers. 0. https://m.xkcd.com/1172/ https://m.xkcd.com/1172/
    - hipadev23 2y ago
      Nike's success in the marketplace has approximately zero to do with their tech stack. That's why their tech literacy is so bad. It just doesn't matter to their business.
      - 2y ago
- Foobar8568 2y ago
  You'd be surprised... I am not...
rootusrootus 2y ago
> How many shoes did we sell in-store vs ecommerce last week? That is perhaps not a great example. My brother is a business analyst at Nike (has been for 15 years or more). I just asked him how hard it would be to answer that question and he said it would be pretty easy. Granted, this is the kind of data he works with routinely, so it may be more difficult for other teams that do not.
- MisterTea 2y ago
  Is he the person running the query or is he reading a blood spattered report pulled from the dead hands of some data engineer who perished battling the system to retrieve said data?
  - AtlasBarfed 2y ago
    https://youtu.be/EF3g4Ua5e7k?si=216W5ZK7AQma2JIR https://youtu.be/EF3g4Ua5e7k?si=216W5ZK7AQma2JIR
  - rootusrootus 2y ago
    A little of both? He's a team lead. Definitely runs queries himself, but probably spends more time interfacing with management and other teams on a day-to-day basis.

notyourwork 2y ago

> Getting answers to simple questions like "How many shoes did we sell in-store vs ecommerce last week?" is a nearly impossible task. I find this type of thing scary as an outsider looking in. How a company so large has such immature engineering continues to astonish me.

shepardrtc 2y ago
> I find this type of thing scary as an outsider looking in. How a company so large has such immature engineering continues to astonish me. It's management that doesn't want to risk their positions by doing the very difficult business of either starting over or properly simplifying their stack. It's not easy, it's not quick, but if they can't even answer that basic question then they need to do the work.
- crowcroft 2y ago
  To defend management a little bit, these massive companies have existed through many eras of technology with many different managers. They work with many external companies in many different ways. They have an exceptionally complex, but functioning tech stack, that allows all of these many dependencies to function together. Lastly, they are successful as they are! It's not usually an issue of immaturity, it's just really hard. To make things worse, often people don't really want to do the work because literally any other data engineering job would probably be more enjoyable. Simplifying the tech stack would probably require simplifying their business operations, which probably means less revenue. Starting over is often literally not possible because there are so many interconnected systems that aren't all necessarily owned by the company trying to make the decision...
  - rexpop 2y ago
    > Simplifying the tech stack would probably require simplifying their business operations, which probably means less revenue. I don't understand this perspective. Simplifying the tech stack might mean taking multiple services in multiple languages, and deprecating some in favor of migrating that functionality to the most maintainable codebase. This shouldn't mean "simplifying their business operations", or affecting their business operations in any way.
    - crowcroft 2y ago
      I agree there probably are areas were they can simplify with no impact to operations. But I would imagine there are a lot of pieces of apparent cruft hanging around that is actually there because if you remove it things break. Maybe a large retailer that you rely on requires an integration with an old version of SAP, and then a logistics partner only provides files over FTP, and you need to use OCR to retrieve any data from the files they're sending. Management can't just mandate that you will 'simplify the tech stack'. Even refactoring smaller parts of the tech stack is often a pretty massive job.
      - rexpop 2y ago
        "Look, my setup works for me. Just add an option to reenable spacebar heating."[0] > Management can't just mandate that you will 'simplify the tech stack'. In my experience, it's the devs who want to do reactors for love of the craft and satisfaction of a job "well" done, while management seeks to sweep as much technical "debt" as possible under the rug, and they'll burn through as many Romanians' nights' sleeps as it takes to reach their numbers. 0. https://m.xkcd.com/1172/ https://m.xkcd.com/1172/
  - hipadev23 2y ago
    Nike's success in the marketplace has approximately zero to do with their tech stack. That's why their tech literacy is so bad. It just doesn't matter to their business.
    - 2y ago
Foobar8568 2y ago
You'd be surprised... I am not...

rootusrootus 2y ago

> How many shoes did we sell in-store vs ecommerce last week? That is perhaps not a great example. My brother is a business analyst at Nike (has been for 15 years or more). I just asked him how hard it would be to answer that question and he said it would be pretty easy. Granted, this is the kind of data he works with routinely, so it may be more difficult for other teams that do not.

MisterTea 2y ago
Is he the person running the query or is he reading a blood spattered report pulled from the dead hands of some data engineer who perished battling the system to retrieve said data?
- AtlasBarfed 2y ago
  https://youtu.be/EF3g4Ua5e7k?si=216W5ZK7AQma2JIR https://youtu.be/EF3g4Ua5e7k?si=216W5ZK7AQma2JIR
- rootusrootus 2y ago
  A little of both? He's a team lead. Definitely runs queries himself, but probably spends more time interfacing with management and other teams on a day-to-day basis.