8 ms·

I can't remember if a friend working at Amazon told me this, or if I read it online. Supposedly, Amazon does tons of backend real world optimization on shippin

by GuiA 10y ago

I can't remember if a friend working at Amazon told me this, or if I read it online. Supposedly, Amazon does tons of backend real world optimization on shipping. Say you, a SF resident, look at a book today on Amazon.com. Based on your past actions, Amazon knows very well how likely you are to purchase right away, or tomorrow, etc. Now imagine the book isn't in a warehouse close enough to you for prime, but Amazon's data tells them you are much more likely to buy it when you come back to Amazon tomorrow if Prime is offered. They'll ship the book to the warehouse in case you come back tomorrow to look at that book. Of course there's tons more modeling behind it to make sure they have very accurate estimates of the cost to pre emptively ship vs the payoff of you buying it, but that's the gist of it. That was a few years ago, and it blew my mind at the time to think of all the optimization you can do in shipping to individuals when you're working at scale. Amazon is more and more essentially running a CDN, but for the real world.

jonknee 10y ago
It was a patent application: https://techcrunch.com/2014/01/18/amazon-pre-ships/ https://techcrunch.com/2014/01/18/amazon-pre-ships/
djhworld 10y ago
I find this hard to believe. But there a again, I don't collect patterns about my behaviour so it could be true. Sometimes I look at Amazon and then buy the product elsewhere, say if I spent 3 occasions over a few days looking at a product, does that breach some threshold somewhere for them to say "we have one on the hook, lets start moving this to a warehouse near him just in case"
- noonespecial 10y ago
  Its easy to believe if the marginal cost to ship the item approaches 0. For Amazon, this might actually be close enough to true to matter. Especially after this move.
  - bluGill 10y ago
    It probably does approach 0. The truck is going anyway, most of the cost of a truck is wind resistance so adding more books up to the weight limit is close to zero. Amazon is probably never shipping a single book, they are using analytics to decide if they should ship a case of books. If they think you are 90% likely to buy a book tomorrow but you are the only one who will it isn't worth the risk. If they decide 100 people are 50% likely to buy a book, sending a box of 100 books is a good investment - if 50 people buy tomorrow that implies 40 will buy the next day and 30 the day after and then 45 the next day and so on for a few weeks/months as sales slowly drop to zero.
- ergothus 10y ago
  I have no actual knowledge on the topic, but I presume it's not calculated on a purely individual level. If 100 people live within the range of the local distribution warehouse are interested in this book, and they say "this one is 87% likely to buy, this one is 43% likely to but..." they can work out a model that results in very few over-shippings. But again, I know nothing, other than thinking about it a bit when the patent app became known.
  - djhworld 10y ago
    Calculating it on an aggregate level over a number of consumers makes more sense to me.
  - djhworld 10y ago
    Calculating it on an aggregate level over a number of consumers makes more sense to me.
  - wojt_eu 10y ago
    As a result a book about Computer Science would be pre-emptively shipped to a warehouse near SV and investment book about economy and shipped near NY.
    - Roritharr 10y ago
      The patterns in this would be very exciting to know for people who are looking for their "tribe". It's actually something very hard to find out before moving somewhere thats not the Valley, what is the main intellectual focus of the region, is there none? I missed a lot in my own childhood and upbringing to have people in my environment that we're or we're in contact with people at the top of their field, having deeper understanding of something instead of just struggling at their level.
- yomly 10y ago
  We all talk about Google and Facebook's rich data source for human behaviour. We talk less about Amazon's rich data source on consumer behaviour. As Google themselves once said Amazon is a search engine for products/shopping.[0] They said this some 3~ years ago, Amazon has continued to expand since. It is fairly safe to assume Amazon is the single most largest consolidated reserve of consumer behaviour data on the world now. Facebook can gauge the difference between different types of likes from how you read a page[1], Amazon does too from their harvested clickstream. [0] http://www.bbc.co.uk/news/technology-29609472 http://www.bbc.co.uk/news/technology-29609472 [1] http://www.slate.com/articles/technology/cover_story/2016/01/how_facebook_s_news_feed_algorithm_works.html http://www.slate.com/articles/technology/cover_story/2016/01...
- dalore 10y ago
  Most of the time you won't realise this behaviour.
- mikeash 10y ago
  With a lot of data, the aggregate probabilities will become reasonably accurate. If they predict that there's a 60% chance you'll buy, that they'll make $2 from you if you do, and they'll lose $2 on wasted shipping if you don't, then on average they'll make money by prepositioning your prospective purchase. For Prime orders I can see it making a huge difference. Two-day shipping gets really expensive compared to slower methods when shipping over large distances, but for short distances it can be done entirely on normal ground delivery trucks and is cheap. If prepositioning wastes a few bucks sometimes and saves a bunch of money on expensive two-day shipping other times, it could be a big win.
akg_67 10y ago
Amazon most probably does tons of backend optimization in all its activities. What I have found more amazing is the selection of books in Amazon "brick-and-mortar" bookstore compared to Barnes & Noble (B&N) bookstore in same Seattle shopping complex previously. I typically visit that shopping complex few times a month. Every time, I have visited Amazon bookstore, I tend to pickup couple of books to browse that I find interesting. When I used to visit B&N in same complex previously, it was rare for me to pickup books to browse in repeat visits. Amazon is able to achieve in 1/4th of the one floor what B&N couldn't do with books on two floors, make me pick up books on repeat visits. Amazon seems to refresh books more often in its store. Amazon might also be using the browsing/purchasing habits of users living in the area to determine what books to stock at the store.
- adamnemecek 10y ago
  The amazon bookstore, while nice, has like a non-existent tech section.
  - kchoudhu 10y ago
    Is anyone actually still buying dead wood tech books?
    - GirlsCanCode 10y ago
      Yes!
    - adamnemecek 10y ago
      I get dead wood for trickier tech books where I have to flip back and forth a bunch.
      - kchoudhu 10y ago
        Fair enough! Apologies if I seemed flippant -- things are changing so quickly nowawdays that apart from a few core texts on algorithms and programming languages, I've pretty much stopped expanding my library. It seems to be stackexchange and blog posts across the board now. I suspect I'll pay for this choice sooner or later...
tgcordell 10y ago
The concept of what you are saying is true. The exact details and flags of what drives the proactive optimization is the secret sauce. source: work in Supply Chain for Amazon
- bo1024 10y ago
  Probably the data is the main secret sauce.
  - tgcordell 10y ago
    Everyone has eggs, water, and flour. Not everyone makes incredible bread :)
    - dsfyu404ed 10y ago
      Anyone can sell bread. Not everyone had metadata from a significant chunk of the population's bakery browsing and purchasing habits.
  - AStellersSeaCow 10y ago
    Nah, it's just thousand-island dressing.
cperciva 10y ago
Say you, a SF resident, look at a book today on Amazon.com. Based on your past actions, Amazon knows very well how likely you are to purchase right away, or tomorrow, etc. [...] They'll ship the book to the warehouse in case you come back tomorrow to look at that book. It's unlikely that it would be worth doing this if one person looks at a book; the return-and-buy-later rates wouldn't be high enough. But if ten people look at a book, it makes sense to move one or two copies to a closer warehouse -- and that's where Amazon's scale really wins. With O(N) customers and good historical data they can predict future purchases to within O(sqrt(N)) thanks to the central limit theorem.
- BurningFrog 10y ago
  > It's unlikely that it would be worth doing this if one person looks at a book Yeah... But then again, if a truck is leaving soon from warehouse A to warehouse B, why not put that book on the truck, if no one near A has looked at it recently?
  - kalleboo 10y ago
    Even then if you really have data for everything, you can take into account, "are we overstaffed in warehouse A right now? what are the staffing costs in warehouse B when we arrive?", and do more or less physical work depending on that information
  - sokoloff 10y ago
    The bin pick labor is not (nearly) free.
  - brianbreslin 10y ago
    There IS a marginal cost to touching a book and to moving it, even if the truck is going ANYWAY, it might be pennies of fuel + pennies of labor to move it. This could eat into the margins.
    - BurningFrog 10y ago
      Yeah, there is a small cost. There is also a small benefit. With good data you can make the right small decisions.
kriro 10y ago
Very interesting. "A CDN, but for the real world" is a great way of summarizing it, too.
yomly 10y ago
I say it to all my friends all the time - Amazon is a tech company that happens to do retail, not a retail company that does tech. It's no coincidence that Amazon calls a unit of storage space in its FCs a "bin" and also happens to pseudo-randomly distribute products in the FC. I am convinced that Jeff Bezos' long term plan is to automate everything, and treat every problem like a CS problem so your CDN analogy is spot on. Pricing is automated, ordering is automated, and I'm pretty sure with Kiva storage positioning is dynamically optimized too. [0] The thing is, the further invested Amazon is along this path, the cheaper and more profitable it becomes to continue. Moreover, the harder it becomes for new players to catch up. Will be interesting what the plan is with these planes, I note Amazon also is investigating entering the shipping industry, a space pretty ripe for disruption[1][2] [0] http://m.youtube.com/watch?v=UtBa9yVZBJM http://m.youtube.com/watch?v=UtBa9yVZBJM [1] https://news.ycombinator.com/item?id=10907163 https://news.ycombinator.com/item?id=10907163 [2] http://www.bloomberg.com/news/articles/2016-02-09/amazon-is-building-global-delivery-business-to-take-on-alibaba-ikfhpyes http://www.bloomberg.com/news/articles/2016-02-09/amazon-is-...
- mattmanser 10y ago
  Sorry to burst your bubble, but bin is a standard warehouse term.
  - yomly 10y ago
    Ah, that's disappointing - I did think that was unique to Amazon. I don't suppose other companies call their product classifiers browse nodes and call their most granular classifiers leaf nodes...?
    - mattmanser 10y ago
      I doubt it! I was actually thinking the etymology of the word bin in CS is probably from someone who worked in a warehouse. It can't be a coincidence.
      - chha 10y ago
rohanprabhu 10y ago
I worked NA logistics back when I was with Amazon, and was with the team behind the LA fresh and the team which works with intermediate FCs (they don't stock goods, but do store-and-forward at crucial locations in the logistics pipeline). And you are right, Amazon does an incredible amount of stuff when it comes to logistics. There is an entire building that houses trans & metrics team, and the hyderabad center in India is mostly just the transportation team. Amazon is way ahead in the logistics game, even further than some of your flagship logistics companies.
TorKlingberg 10y ago
> They'll ship the book to the warehouse in case you come back tomorrow to look at that book. That sounds cool, but I doubt it actually happens. At least not for one person.
k__ 10y ago
They don't even give me the right recommendations, how can they estimate what I will buy?!
Thomas_204 10y ago
I was formerly a customs broker for Amazon in the past and found their orders to usually be pretty typical, usually based around volume/time of year. They definitely rotate vendors based off time of year + what would be seasonally important but I think it's mostly just based off what sells. I don't think their algorithm is quite as complex but I don't think it's any bit weak, honestly. But as I was their broker and didn't work internally, I can't comment to specifics, really.

by GuiA 10y ago

jonknee 10y ago

It was a patent application: https://techcrunch.com/2014/01/18/amazon-pre-ships/ https://techcrunch.com/2014/01/18/amazon-pre-ships/

djhworld 10y ago

I find this hard to believe. But there a again, I don't collect patterns about my behaviour so it could be true. Sometimes I look at Amazon and then buy the product elsewhere, say if I spent 3 occasions over a few days looking at a product, does that breach some threshold somewhere for them to say "we have one on the hook, lets start moving this to a warehouse near him just in case"

noonespecial 10y ago
Its easy to believe if the marginal cost to ship the item approaches 0. For Amazon, this might actually be close enough to true to matter. Especially after this move.
- bluGill 10y ago
  It probably does approach 0. The truck is going anyway, most of the cost of a truck is wind resistance so adding more books up to the weight limit is close to zero. Amazon is probably never shipping a single book, they are using analytics to decide if they should ship a case of books. If they think you are 90% likely to buy a book tomorrow but you are the only one who will it isn't worth the risk. If they decide 100 people are 50% likely to buy a book, sending a box of 100 books is a good investment - if 50 people buy tomorrow that implies 40 will buy the next day and 30 the day after and then 45 the next day and so on for a few weeks/months as sales slowly drop to zero.
ergothus 10y ago
I have no actual knowledge on the topic, but I presume it's not calculated on a purely individual level. If 100 people live within the range of the local distribution warehouse are interested in this book, and they say "this one is 87% likely to buy, this one is 43% likely to but..." they can work out a model that results in very few over-shippings. But again, I know nothing, other than thinking about it a bit when the patent app became known.
- djhworld 10y ago
  Calculating it on an aggregate level over a number of consumers makes more sense to me.
- djhworld 10y ago
  Calculating it on an aggregate level over a number of consumers makes more sense to me.
- wojt_eu 10y ago
  As a result a book about Computer Science would be pre-emptively shipped to a warehouse near SV and investment book about economy and shipped near NY.
  - Roritharr 10y ago
    The patterns in this would be very exciting to know for people who are looking for their "tribe". It's actually something very hard to find out before moving somewhere thats not the Valley, what is the main intellectual focus of the region, is there none? I missed a lot in my own childhood and upbringing to have people in my environment that we're or we're in contact with people at the top of their field, having deeper understanding of something instead of just struggling at their level.
yomly 10y ago
We all talk about Google and Facebook's rich data source for human behaviour. We talk less about Amazon's rich data source on consumer behaviour. As Google themselves once said Amazon is a search engine for products/shopping.[0] They said this some 3~ years ago, Amazon has continued to expand since. It is fairly safe to assume Amazon is the single most largest consolidated reserve of consumer behaviour data on the world now. Facebook can gauge the difference between different types of likes from how you read a page[1], Amazon does too from their harvested clickstream. [0] http://www.bbc.co.uk/news/technology-29609472 http://www.bbc.co.uk/news/technology-29609472 [1] http://www.slate.com/articles/technology/cover_story/2016/01/how_facebook_s_news_feed_algorithm_works.html http://www.slate.com/articles/technology/cover_story/2016/01...
dalore 10y ago
Most of the time you won't realise this behaviour.
mikeash 10y ago
With a lot of data, the aggregate probabilities will become reasonably accurate. If they predict that there's a 60% chance you'll buy, that they'll make $2 from you if you do, and they'll lose $2 on wasted shipping if you don't, then on average they'll make money by prepositioning your prospective purchase. For Prime orders I can see it making a huge difference. Two-day shipping gets really expensive compared to slower methods when shipping over large distances, but for short distances it can be done entirely on normal ground delivery trucks and is cheap. If prepositioning wastes a few bucks sometimes and saves a bunch of money on expensive two-day shipping other times, it could be a big win.

akg_67 10y ago

Amazon most probably does tons of backend optimization in all its activities. What I have found more amazing is the selection of books in Amazon "brick-and-mortar" bookstore compared to Barnes & Noble (B&N) bookstore in same Seattle shopping complex previously. I typically visit that shopping complex few times a month. Every time, I have visited Amazon bookstore, I tend to pickup couple of books to browse that I find interesting. When I used to visit B&N in same complex previously, it was rare for me to pickup books to browse in repeat visits. Amazon is able to achieve in 1/4th of the one floor what B&N couldn't do with books on two floors, make me pick up books on repeat visits. Amazon seems to refresh books more often in its store. Amazon might also be using the browsing/purchasing habits of users living in the area to determine what books to stock at the store.

adamnemecek 10y ago
The amazon bookstore, while nice, has like a non-existent tech section.
- kchoudhu 10y ago
  Is anyone actually still buying dead wood tech books?
  - GirlsCanCode 10y ago
    Yes!
  - adamnemecek 10y ago
    I get dead wood for trickier tech books where I have to flip back and forth a bunch.
    - kchoudhu 10y ago
      Fair enough! Apologies if I seemed flippant -- things are changing so quickly nowawdays that apart from a few core texts on algorithms and programming languages, I've pretty much stopped expanding my library. It seems to be stackexchange and blog posts across the board now. I suspect I'll pay for this choice sooner or later...

tgcordell 10y ago

The concept of what you are saying is true. The exact details and flags of what drives the proactive optimization is the secret sauce. source: work in Supply Chain for Amazon

bo1024 10y ago
Probably the data is the main secret sauce.
- tgcordell 10y ago
  Everyone has eggs, water, and flour. Not everyone makes incredible bread :)
  - dsfyu404ed 10y ago
    Anyone can sell bread. Not everyone had metadata from a significant chunk of the population's bakery browsing and purchasing habits.
- AStellersSeaCow 10y ago
  Nah, it's just thousand-island dressing.

cperciva 10y ago

Say you, a SF resident, look at a book today on Amazon.com. Based on your past actions, Amazon knows very well how likely you are to purchase right away, or tomorrow, etc. [...] They'll ship the book to the warehouse in case you come back tomorrow to look at that book. It's unlikely that it would be worth doing this if one person looks at a book; the return-and-buy-later rates wouldn't be high enough. But if ten people look at a book, it makes sense to move one or two copies to a closer warehouse -- and that's where Amazon's scale really wins. With O(N) customers and good historical data they can predict future purchases to within O(sqrt(N)) thanks to the central limit theorem.

BurningFrog 10y ago
> It's unlikely that it would be worth doing this if one person looks at a book Yeah... But then again, if a truck is leaving soon from warehouse A to warehouse B, why not put that book on the truck, if no one near A has looked at it recently?
- kalleboo 10y ago
  Even then if you really have data for everything, you can take into account, "are we overstaffed in warehouse A right now? what are the staffing costs in warehouse B when we arrive?", and do more or less physical work depending on that information
- sokoloff 10y ago
  The bin pick labor is not (nearly) free.
- brianbreslin 10y ago
  There IS a marginal cost to touching a book and to moving it, even if the truck is going ANYWAY, it might be pennies of fuel + pennies of labor to move it. This could eat into the margins.
  - BurningFrog 10y ago
    Yeah, there is a small cost. There is also a small benefit. With good data you can make the right small decisions.

kriro 10y ago

Very interesting. "A CDN, but for the real world" is a great way of summarizing it, too.

yomly 10y ago

I say it to all my friends all the time - Amazon is a tech company that happens to do retail, not a retail company that does tech. It's no coincidence that Amazon calls a unit of storage space in its FCs a "bin" and also happens to pseudo-randomly distribute products in the FC. I am convinced that Jeff Bezos' long term plan is to automate everything, and treat every problem like a CS problem so your CDN analogy is spot on. Pricing is automated, ordering is automated, and I'm pretty sure with Kiva storage positioning is dynamically optimized too. [0] The thing is, the further invested Amazon is along this path, the cheaper and more profitable it becomes to continue. Moreover, the harder it becomes for new players to catch up. Will be interesting what the plan is with these planes, I note Amazon also is investigating entering the shipping industry, a space pretty ripe for disruption[1][2] [0] http://m.youtube.com/watch?v=UtBa9yVZBJM http://m.youtube.com/watch?v=UtBa9yVZBJM [1] https://news.ycombinator.com/item?id=10907163 https://news.ycombinator.com/item?id=10907163 [2] http://www.bloomberg.com/news/articles/2016-02-09/amazon-is-building-global-delivery-business-to-take-on-alibaba-ikfhpyes http://www.bloomberg.com/news/articles/2016-02-09/amazon-is-...

mattmanser 10y ago
Sorry to burst your bubble, but bin is a standard warehouse term.
- yomly 10y ago
  Ah, that's disappointing - I did think that was unique to Amazon. I don't suppose other companies call their product classifiers browse nodes and call their most granular classifiers leaf nodes...?
  - mattmanser 10y ago
    I doubt it! I was actually thinking the etymology of the word bin in CS is probably from someone who worked in a warehouse. It can't be a coincidence.
    - chha 10y ago

rohanprabhu 10y ago

I worked NA logistics back when I was with Amazon, and was with the team behind the LA fresh and the team which works with intermediate FCs (they don't stock goods, but do store-and-forward at crucial locations in the logistics pipeline). And you are right, Amazon does an incredible amount of stuff when it comes to logistics. There is an entire building that houses trans & metrics team, and the hyderabad center in India is mostly just the transportation team. Amazon is way ahead in the logistics game, even further than some of your flagship logistics companies.

TorKlingberg 10y ago

> They'll ship the book to the warehouse in case you come back tomorrow to look at that book. That sounds cool, but I doubt it actually happens. At least not for one person.

k__ 10y ago

They don't even give me the right recommendations, how can they estimate what I will buy?!

Thomas_204 10y ago

I was formerly a customs broker for Amazon in the past and found their orders to usually be pretty typical, usually based around volume/time of year. They definitely rotate vendors based off time of year + what would be seasonally important but I think it's mostly just based off what sells. I don't think their algorithm is quite as complex but I don't think it's any bit weak, honestly. But as I was their broker and didn't work internally, I can't comment to specifics, really.