3 ms·

> That's because in a lot of cases they're optimizing for the wrong metrics, as in maximizing their revenue instead of your utility. What? How do they maximiz

> That's because in a lot of cases they're optimizing for the wrong metrics, as in maximizing their revenue instead of your utility. What? How do they maximize their revenue by spending money on ads the users actually laugh about for how bad their targeting is? Based on what I know about machine learning - it almost always gives great short-term results, and it almost always fails to deliver the expected long-term results. What's worse is this comes with the weirdest most indecipherable bugs that pop up more and more over time. Unless you have a large enough database to show statistical errors that are negligible (something like 99.9999% precision or recall, depending on your metrics), you should assume it will break in ways you cannot possibly predict. And even then, you might be using the wrong training data without even realizing it. I'm not saying ML is bad, although I am saying it is ridiculously overhyped. I'm saying ML is still nascent enough nobody really knows how a lot of edge cases will shake out, simply because there are too many edge cases to test before putting it into production. It's not hard to find examples of ML algorithms gone wrong even for sites like Amazon. https://gizmodo.com/amazon-prime-day-glitch-let-people-buy-13-000-camera-g-1836487919 https://gizmodo.com/amazon-prime-day-glitch-let-people-buy-1...

m_ke 7y ago
Recommending something you bought before is probably a much better bet than showing you random items from their huge catalogue. You can have a system that generates them a ton of cash while making mistakes in some cases, outliers are inevitable and feedback loops in recommender systems can lead to such issues. Amazon wouldn't deploy these systems if they didn't move the needle.
- OnlineGladiator 7y ago
  Nobody is saying the ML algorithm (or any algorithm) is expected to be perfect. > Amazon wouldn't deploy these systems if they didn't move the needle. This is an appeal to authority that Amazon executives are immune to making mistakes. They could have bad metrics. They could have bad incentives encouraging managers to make poor decisions (basically this describes everything wrong with Google today). They could be incompetent. They could be focused on short-term gains at the expense of long-term gains because it maximizes their personal net wealth and they can just jump ship in a few years. Again, nobody is saying ML algorithms are worthless. I just don't believe (and have lots of reasons based on personal experience I won't go into) that they are 10% as useful as the industry wants you to believe.
  - m_ke 7y ago
    I'm saying that they have systems in place to measure the impact of the code that they put in production on their bottom line and if those recommendations didn't move their profit margins in the right direction they wouldn't be using them.
    - OnlineGladiator 7y ago
      I know what you're saying. I don't think you understand what I'm saying, which is those systems can be wrong or they might not be optimizing for the bottom line. Why didn't their systems catch that bug that was discounting expensive equipment for more than 90% that I linked in my previous post? People make mistakes. And for a frame of reference, Google's corporate policies definitely don't optimize for the bottom line, and it took a minimum of 5 years for it to affect them negatively (I'd guess closer to 10+). Anyway, agree to disagree.
    - eitland 7y ago
      > I'm saying that they have systems in place to measure the impact of the code that they put in production on their bottom line There's a story, and I think it was (re)posted here recently about a series of MBAs all optimizing the cost of the burger bums by removing seeds until there are three seeds neatly laid out at the top. It might be measurable all the way but at some point it becomes ridiculous. For me that time was some months ago. For the rest of Internet they might manage to reduce the quality once or twice more before it becomes obvious. This is my way of fighting back. By posting here and on my blog and getting upvoted a lot for pointing out what many can already feel. By letting people who read HN know that yes, that feeling they have that a lot of their ads are wasted because of bad targeting might very well be true.
      - m_ke 7y ago
        That's definitely true, maximizing shareholder value is all about extracting as much value as possible from your customers, until it no longer works. I'm against most forms of recommendations as well, that doesn't mean that they're not valuable to the businesses deploying them. In most cases the end users of these systems are not the real customers of these platforms, and they're there to serve the advertisers who keep these businesses running.
- smsm42 7y ago
  > Recommending something you bought before is probably a much better bet Not if it's something I only buy once a year, for example. That's where "learning" part should come in. You don't need any learning to just parrot me back my inputs. > Amazon wouldn't deploy these systems if they didn't move the needle. I don't know about Amazon, but I've recently read on HN some articles strongly suggesting almost nobody is properly measuring the impact of ads, let alone the impact of "targeting". In most cases, people more or less just stuff money into ads budgets, because that's what you do, and they get customers - because people still need to buy things, regardless of any targeting - but the casual link between the former and the latter is not really very well established.

m_ke 7y ago

Recommending something you bought before is probably a much better bet than showing you random items from their huge catalogue. You can have a system that generates them a ton of cash while making mistakes in some cases, outliers are inevitable and feedback loops in recommender systems can lead to such issues. Amazon wouldn't deploy these systems if they didn't move the needle.

OnlineGladiator 7y ago
Nobody is saying the ML algorithm (or any algorithm) is expected to be perfect. > Amazon wouldn't deploy these systems if they didn't move the needle. This is an appeal to authority that Amazon executives are immune to making mistakes. They could have bad metrics. They could have bad incentives encouraging managers to make poor decisions (basically this describes everything wrong with Google today). They could be incompetent. They could be focused on short-term gains at the expense of long-term gains because it maximizes their personal net wealth and they can just jump ship in a few years. Again, nobody is saying ML algorithms are worthless. I just don't believe (and have lots of reasons based on personal experience I won't go into) that they are 10% as useful as the industry wants you to believe.
- m_ke 7y ago
  I'm saying that they have systems in place to measure the impact of the code that they put in production on their bottom line and if those recommendations didn't move their profit margins in the right direction they wouldn't be using them.
  - OnlineGladiator 7y ago
    I know what you're saying. I don't think you understand what I'm saying, which is those systems can be wrong or they might not be optimizing for the bottom line. Why didn't their systems catch that bug that was discounting expensive equipment for more than 90% that I linked in my previous post? People make mistakes. And for a frame of reference, Google's corporate policies definitely don't optimize for the bottom line, and it took a minimum of 5 years for it to affect them negatively (I'd guess closer to 10+). Anyway, agree to disagree.
  - eitland 7y ago
    > I'm saying that they have systems in place to measure the impact of the code that they put in production on their bottom line There's a story, and I think it was (re)posted here recently about a series of MBAs all optimizing the cost of the burger bums by removing seeds until there are three seeds neatly laid out at the top. It might be measurable all the way but at some point it becomes ridiculous. For me that time was some months ago. For the rest of Internet they might manage to reduce the quality once or twice more before it becomes obvious. This is my way of fighting back. By posting here and on my blog and getting upvoted a lot for pointing out what many can already feel. By letting people who read HN know that yes, that feeling they have that a lot of their ads are wasted because of bad targeting might very well be true.
    - m_ke 7y ago
      That's definitely true, maximizing shareholder value is all about extracting as much value as possible from your customers, until it no longer works. I'm against most forms of recommendations as well, that doesn't mean that they're not valuable to the businesses deploying them. In most cases the end users of these systems are not the real customers of these platforms, and they're there to serve the advertisers who keep these businesses running.
smsm42 7y ago
> Recommending something you bought before is probably a much better bet Not if it's something I only buy once a year, for example. That's where "learning" part should come in. You don't need any learning to just parrot me back my inputs. > Amazon wouldn't deploy these systems if they didn't move the needle. I don't know about Amazon, but I've recently read on HN some articles strongly suggesting almost nobody is properly measuring the impact of ads, let alone the impact of "targeting". In most cases, people more or less just stuff money into ads budgets, because that's what you do, and they get customers - because people still need to buy things, regardless of any targeting - but the casual link between the former and the latter is not really very well established.