5 ms·

I haven't come to a personal conclusion on how I feel about the morals of it all, but you have to marvel at the feat of engineering.

femto113 7y ago
Marvel at the feat of operations, not the engineering. The techniques used to process this information over the years have mostly been quite banal. Look behind the scenes and much of it is just batch processes written in old school languages (Perl, Java, even C) reading text files from one folder, filtering or enhancing them, and writing them out to another. I once taught a PM how to use a Perl hash to do a join instead of nested for loops and she spent the next year feeling godlike because she could do this stuff in O(n) instead of O(n^2).
- willberman 7y ago
  Correct I think that would be a better way to put it. None of the individual building blocks seem very technically challenging, but to consistently enforce it at a company wide scale seems quite cool.
- anongraddebt 7y ago
  > I once taught a PM how to use a Perl hash to do a join instead of nested for loops and she spent the next year feeling godlike because she could do this stuff in O(n) instead of O(n^2). Loved this.
- auggierose 7y ago
  What’s a PM, prime minister?
  - capableweb 7y ago
    Because of the context, I'd assume "Project Manager", basically a low-level manager of sorts.
    - femto113 7y ago
      At Amazon "Program Manager", in this case really was a TPM or "Technical Program Manager". TPMs are effectively serial project managers, often charged with shepherding complex projects that require the efforts of more than one business or development group. Dealing with the data produced by those projects sometimes fell on them since it was difficult enough to persuade the developers to write the primary functionality.
krtong 7y ago
i havent come to a personal conclusion on if its even a feat of engineering. Im impressed by amazon’s handling of logistics, and I do think consumer data helps Amazon know when and where products are most likely to sell, perhaps even upsell customers or get customers to agree to a steeper markup, but beyond that, i don’t think actual predictive models of consumer behavior has really been achieved here. Ive never gotten a sense that amazon intuits my consumer habits and ive made a lot of purchases over the years. i dont think amazon has made a dime on storing timestamps for when i turned a page on a kindle or learned anything from the music i listened to. ive never bought a single recommended product. On a scale of 1-10, 10 being Delos incorporated and perfectly modeling user behavior—-accurately predicting how i’ll react before I even see the products they place in front of me—and 1 being Wall-E making blocks of trash, id say this data collection is a 2.5 at best.
- willberman 7y ago
  I would personally define efficacy of predictive modes to be a scientific achievement rather than engineering, but that might just be a vocab discrepancy. Do you think it’s possible to determine how good their modeling is externally? I would assume we’d need an extensive peek behind the curtains to come to a conclusion.
  - krtong 7y ago
    Thinking about your question I just scratch my head and keep returning to the answer "no I don't." I can't think of a single test of Amazon's consumer behavior model better than search results and product recommendations. If you want to know whether you understand my consumer habits you need to interact with me and get me to buy your predictions. Not to mention, content predictions across the board are awful. Amazon, Facebook, Reddit, Instagram, YouTube, Google, Spotify, Soundcloud, you name it they all try to predict what I'm looking for and none of them have figured it out. If one of them was remotely close I might be concerned that there's a formula for how to get me to buy things but so far Im stuck digging through garbage piles for the things I want.

femto113 7y ago

Marvel at the feat of operations, not the engineering. The techniques used to process this information over the years have mostly been quite banal. Look behind the scenes and much of it is just batch processes written in old school languages (Perl, Java, even C) reading text files from one folder, filtering or enhancing them, and writing them out to another. I once taught a PM how to use a Perl hash to do a join instead of nested for loops and she spent the next year feeling godlike because she could do this stuff in O(n) instead of O(n^2).

willberman 7y ago
Correct I think that would be a better way to put it. None of the individual building blocks seem very technically challenging, but to consistently enforce it at a company wide scale seems quite cool.
anongraddebt 7y ago
> I once taught a PM how to use a Perl hash to do a join instead of nested for loops and she spent the next year feeling godlike because she could do this stuff in O(n) instead of O(n^2). Loved this.
auggierose 7y ago
What’s a PM, prime minister?
- capableweb 7y ago
  Because of the context, I'd assume "Project Manager", basically a low-level manager of sorts.
  - femto113 7y ago
    At Amazon "Program Manager", in this case really was a TPM or "Technical Program Manager". TPMs are effectively serial project managers, often charged with shepherding complex projects that require the efforts of more than one business or development group. Dealing with the data produced by those projects sometimes fell on them since it was difficult enough to persuade the developers to write the primary functionality.

krtong 7y ago

i havent come to a personal conclusion on if its even a feat of engineering. Im impressed by amazon’s handling of logistics, and I do think consumer data helps Amazon know when and where products are most likely to sell, perhaps even upsell customers or get customers to agree to a steeper markup, but beyond that, i don’t think actual predictive models of consumer behavior has really been achieved here. Ive never gotten a sense that amazon intuits my consumer habits and ive made a lot of purchases over the years. i dont think amazon has made a dime on storing timestamps for when i turned a page on a kindle or learned anything from the music i listened to. ive never bought a single recommended product. On a scale of 1-10, 10 being Delos incorporated and perfectly modeling user behavior—-accurately predicting how i’ll react before I even see the products they place in front of me—and 1 being Wall-E making blocks of trash, id say this data collection is a 2.5 at best.

willberman 7y ago
I would personally define efficacy of predictive modes to be a scientific achievement rather than engineering, but that might just be a vocab discrepancy. Do you think it’s possible to determine how good their modeling is externally? I would assume we’d need an extensive peek behind the curtains to come to a conclusion.
- krtong 7y ago
  Thinking about your question I just scratch my head and keep returning to the answer "no I don't." I can't think of a single test of Amazon's consumer behavior model better than search results and product recommendations. If you want to know whether you understand my consumer habits you need to interact with me and get me to buy your predictions. Not to mention, content predictions across the board are awful. Amazon, Facebook, Reddit, Instagram, YouTube, Google, Spotify, Soundcloud, you name it they all try to predict what I'm looking for and none of them have figured it out. If one of them was remotely close I might be concerned that there's a formula for how to get me to buy things but so far Im stuck digging through garbage piles for the things I want.