7 ms·

To cash in on Kindle Unlimited, a cabal of authors gamed Amazon’s algorithm

118 points by denzil_correa 8y ago

FussyZeus 8y ago
The fight to win over algorithms instead of creating worthwhile content continues to compel all media, books included, in a race to the bottom. I really don't think this is a problem tech can solve.
- snarf21 8y ago
  I would have to say I disagree. I think that things like Patreon and Kickstarter are tech that can help solve this problem. It will take some user behavior modification but I'd rather pay directly to the content creator than give Apple or Amazon 30% or more.
  - FussyZeus 8y ago
    You're still depending on algorithms for exposure though, all you've changed is swapping an Amazon sale for a Patreon subscription. Edit: to be clear I think it's an improvement, just not a solution.
- kanox 8y ago
  I don't see the problem here, algorithms are just helping satisfy the thirst for novels with cocky shirtless males.
  - FussyZeus 8y ago
    Humanity has never been furthered by what the masses were itching to get. In fact it's almost always the exact opposite. I'm not saying there's something intrinsically good about being contrarian, I'm just saying it feels like a whole lot of products, especially but not exclusively in tech, feel so "samey". It's all the same crap just differently branded in different colors, and I feel like algorithms are possibly to blame for that. Going to stores in the old days used to be exciting, you'd always find something new and interesting, especially in Radioshack n such. Now it's just the same old bullshit every single time.
    - avgDev 8y ago
      I think when you refer to going to stores in the "old days" and finding new and exciting stuff basically means things were new for you at the time. The more you learn about world and the older you get things begin to excite you less. The more you know the less excited you get about marketing because you can actually see through the bullshit.
      - FussyZeus 8y ago
        I have to disagree. Products were just more interesting in the 90s and even early 2000s. Tech is the line going through the big-suck right now but cars did the same thing from the 1970's to the 2000's: where once you had different models and styles sporting different looks, different and novel features, etc. you see things slowly boil down to a few good exceptions and the rest of it might as well be all the same thing (and with the example of cars, often is, what with manufacturers selling to each other and re-badging for different markets.) Not long ago, shopping for a laptop was interesting. Did you want the one with the swiveling screen, or the one that was liquid cooled? The one with the super HD screen or the one with the built-in fold-out legs to angle the keyboard better? Thick keys or thin keys? Optical or no? A novel visual design or just as-thin-as-can-be? What were the pros and cons to all this? On and on. Now all laptops practically look identical, even something with a somewhat novel design becomes 2-3 times the price for no additional power, and all of them are designed to overheat and die within a couple of years, with the exception possibly of Macbooks, which cost a King's ransom. There's no novelty or interesting ideas, just the same old things.
        dragonwriter 8y ago
        > Now all laptops practically look identical No, they don't. Most of the options you point to from “not long ago" (and many more) are still available. Okay, I don't see liquid cooled or fold-out legs much, and swivel screens seem to have been displaced with the 360-degree hinges.
- dragonwriter 8y ago
  The problem is tech: that is, if you are using algorithms to drive (ultimately) payments to creators, than creators have an incentive to game the algorithms and will (on average) fail to thrive and be forced to find some other business if they do not. So, the market will be saturated with those gaming the system, because that's what it pays for.
ryandrake 8y ago
If you run a platform and its algorithms can be gamed by “partners” for profit at your’s or the user’s expense you should expect that it is or will be gamed. An important part of designing tech products that scale is sitting down and anticipating all the various ways you are incentivizing cheating and gaming, and then closing the holes as bugs. It’s like doing a security audit, with the attackers being your partners rather than hackers. It’s best to fix the gaming via technical means, but if you can’t, it’s still important to spell out unacceptable behavior in the terms of use. Even if it’s something you can’t enforce at scale (for instance: enforcement requires manual review by humans) you still need the policy so you can do one-off enforcements against egregious cases. Otherwise you sit around in these meetings going, “well we know they are gaming the system but we can’t do anything about it because it’s not against the rules!”
- josephjrobison 8y ago
  I could be wrong but I wonder if they want to leave the loopholes open a bit to see where they get exploited and then slowly roll out updates. Not that they’re intentionally creating bugs but perhaps seeing how they get exploited and where so that the authors are doing the work for them rather than internal engineers.
  - gfo 8y ago
    In a perfect world, maybe. It's hard to tell with Amazon now. Fraud and counterfeiting have become rampant on the platform since they opened up third-party selling to all foreign nations 18 months ago and despite the negative press and angry consumers, they've said and done very little to combat these issues. It's difficult to find other reasons why nothing has done beyond their bottom line numbers growing significantly. Generally speaking a business should find flaws and fix them before releasing. Iterative development is okay, but when issues like this become commonplace and nothing is done to combat it, that's when there's a problem. I have no solutions to offer.
zeristor 8y ago
Do you think Her Cocky Software Developer would be a big seller? This is the real issue
- zeristor 8y ago
  I had to look up how to do italics
- allwein 8y ago
  I know you made this comment as a joke, but there actually is an entire sub-genre of geek-related romance novels.
  - zeristor 8y ago
    These lot seem sweet and charming: http://twincitiesgeek.com/2017/07/24-romance-novels-for-geeks-of-all-kinds/ http://twincitiesgeek.com/2017/07/24-romance-novels-for-geek...
guitarbill 8y ago
The Kindle store is already garbage for any kind of discovery, and Unlimited makes it worse being just more marketing BS to hide the real price and displace good content. Their suggestion algorithms are terrible, and only really work if you want something from the same author. Steam has exactly the same problem, although ar least they‘re trying to solve it in various ways.
- jlarocco 8y ago
  It's interesting how different people can have such different expectations. I personally haven't had any issues finding the books I want, and I find Amazon's suggestion algorithms helpful precisely because it works as you describe. Almost all of my book purchases are made because somebody's recommended a specific book or author to me, or because I'm interested in a specific topic or author and searched for them. In both cases, I'm fine with getting generic suggestions based on the author or topic.
- gfo 8y ago
  Are there examples of businesses out there which have done product recommendation correctly? Amazon is notorious for trying to sell you the same kind of product after you've already purchased it (e.g. I bought a ladder and now all of my recommendations are for more ladders). How hard can it be to uncover which items are more likely to be one-time buys and recommend the products people buy with those items instead (ex. show me painting supplies when I buy a ladder)?
  - virtuabhi 8y ago
    Because there is x% chance that you might return this ladder and get another one. The ladder ads should go away after the expiration of free return period.
  - rhizome 8y ago
    Are there examples of businesses out there which have done product recommendation correctly? I'm not sure what you mean by "correctly," but in general, if you're talking about automation and recommendation engines, my answer is "no." Of course, I don't know the goals of the companies implementing them and whether my sense of "correct" recommendations matches theirs. I don't think any of us do.
rpeden 8y ago
From the article: Valderrama allegedly said that she organized newsletter swaps, in which authors would promote each other’s books to their respective mailing lists. Funnily enough, I've seen this same behavior from developers, freelancers, and authors who are "internet famous" on HN and Reddit. I've signed up for quite a few newsletters from these people because they often provide useful information. But since I signed up for so many, I started noticing that fairly often, they'd recommend each others' products. I realize it's entirely possible that these endorsements are authentic, but after a while, it starts to look like it's an interconnected web of people that met at conferences and agreed to cross-promote each each other on their respective newsletters. And so it ends up feeling like each recommendation isn't really authentic, but is instead more of a quid pro quo in exchange for a reciprocal recommendation. And it's possible that I'm completely wrong and just imagining all of this. It's just one of my pet peeves, so perhaps I'm exaggerating how prevalent it is. And don't even get me started on developers who have resorted to hokey long-form sales copy to sell their books and services... :)
- ageitgey 8y ago
  I'm a person with a mailing list in this space who has recommended other people's products, so I can maybe offer some insight on how this world works. The typical person working in this niche is a solo creator who maybe splits there time creating free content and doing paid consulting projects. The business model is usually something like "write a lot of regular free content to build up a giant mailing list and then occasionally sell something to your mailing list to pay the bills". The problem is that it takes a lot of time and effort to create good content. You are often so drained creating your "free" content that you don't have enough time or energy left to create enough "paid" content regularly enough to pay the bills. So you find friends who have created great stuff that you really like and you help each other out by cross-promoting. In almost every case, it's a direct business relationship. The typical deal in this niche for reselling electronic content is a 50/50 split. So you help your friend sell extra copies (which is just free income to them) while you also make decent money to sustain your work. Of course once you get "magic internet money" coming in this way, it's very tempting to abuse your audience and oversell. But then you lose subscribers quickly. Some people are very cynical about this and literally have spreadsheets to maximize how much they can sell while minimizing user attribution, but I'm not a fan of that. It just feels gross. So yes, it is indeed a small, interconnected web of people who agreed to cross-promote each other. And sometimes the recommendation isn't authentic, but often it is. But in almost case, it's not an exchange of free recommendations, but it's literally an affiliate split between the authors. As far as long-form sales copy to sell books, it is definitely very hokey. But the counterpoint is that most other approaches just don't convert. A cold lead who finds you via Google needs to be convinced that your stuff is worth money and long-form sales copy is one way to do that. I also hate it and completely avoid it, but I understand why people do it.
deleted 8y ago
[deleted]
Sujan 8y ago
Fascinating that Kindle's still don't track which pages actually were "read" (as in "skipping from page 1 to page 100, doesn't meant 2-99 have been read). Basing the author payout system of Kindle Unlimited on that metric is the (technical) root cause here.
- asmosoinio 8y ago
  The OP seems to disagree with you, mentioning that Amazon has some algorithms to determine actually read pages? Or do you have other information?
  - Sujan 8y ago
    What part of the text are you referring to exactly? There was a sentence that Amazon's metric isn't influenced by big fonts or other formatting choices (they probably just use a plain text version of the text to define what a "page" is) and 'that the KENPC system (Kindle Edition Normalized Page Count) recorded pages read with “high precision”' - which doesn't actually mean that much. Did I miss the interesting part? My point was mostly based that I know it was that way for a long time, and that things like stuffing or those newsletter signups or competition in the back of the book wouldn't have any influence if it still wouldn't be this way.
    - asmosoinio 8y ago
      This sentence, mentioning "pages read": A June blog post by the Kindle Direct Publishing Team assured authors that the KENPC system (Kindle Edition Normalized Page Count) recorded pages read with “high precision” and that the company was constantly working to improve its “fidelity.”
      - Sujan 8y ago
        Yeah, as far as I know this has always been the official wording. What makes me skeptical here is the use of "high precision" and what there could be to improve... a page was either viewed for the time necessary to read it or not. Over all readers of a book, it should not be too hard to find out the distribution of valid read times of a page. Unless, of course, you don't have that data, but only some snapshots of which page is opened (e.g. when a sync to the cloud servers happens). Shame the Kindle APIs are undocumented :/
sago 8y ago
I have a feeling that the whole Erom genre is where porn videos were 10 years ago. Low quality, keyword-stuffed, fad-following, repetitive content is so ubiquitous (and all that's needed to get someone off) that someone will come along and be the Erom Hub and swipe the legs out from under the market. The interesting question, that the article raises but does not answer, is how far Erom is demonstrating a pattern that'll be followed by other genres. There is a lot of similar self published genre fiction, but it is still a different order of magnitude to KU contemporary romance.

118 points by denzil_correa 8y ago

The fight to win over algorithms instead of creating worthwhile content continues to compel all media, books included, in a race to the bottom. I really don't think this is a problem tech can solve.

snarf21 8y ago
I would have to say I disagree. I think that things like Patreon and Kickstarter are tech that can help solve this problem. It will take some user behavior modification but I'd rather pay directly to the content creator than give Apple or Amazon 30% or more.
- FussyZeus 8y ago
  You're still depending on algorithms for exposure though, all you've changed is swapping an Amazon sale for a Patreon subscription. Edit: to be clear I think it's an improvement, just not a solution.
kanox 8y ago
I don't see the problem here, algorithms are just helping satisfy the thirst for novels with cocky shirtless males.
- FussyZeus 8y ago
  Humanity has never been furthered by what the masses were itching to get. In fact it's almost always the exact opposite. I'm not saying there's something intrinsically good about being contrarian, I'm just saying it feels like a whole lot of products, especially but not exclusively in tech, feel so "samey". It's all the same crap just differently branded in different colors, and I feel like algorithms are possibly to blame for that. Going to stores in the old days used to be exciting, you'd always find something new and interesting, especially in Radioshack n such. Now it's just the same old bullshit every single time.
  - avgDev 8y ago
    I think when you refer to going to stores in the "old days" and finding new and exciting stuff basically means things were new for you at the time. The more you learn about world and the older you get things begin to excite you less. The more you know the less excited you get about marketing because you can actually see through the bullshit.
    - FussyZeus 8y ago
      I have to disagree. Products were just more interesting in the 90s and even early 2000s. Tech is the line going through the big-suck right now but cars did the same thing from the 1970's to the 2000's: where once you had different models and styles sporting different looks, different and novel features, etc. you see things slowly boil down to a few good exceptions and the rest of it might as well be all the same thing (and with the example of cars, often is, what with manufacturers selling to each other and re-badging for different markets.) Not long ago, shopping for a laptop was interesting. Did you want the one with the swiveling screen, or the one that was liquid cooled? The one with the super HD screen or the one with the built-in fold-out legs to angle the keyboard better? Thick keys or thin keys? Optical or no? A novel visual design or just as-thin-as-can-be? What were the pros and cons to all this? On and on. Now all laptops practically look identical, even something with a somewhat novel design becomes 2-3 times the price for no additional power, and all of them are designed to overheat and die within a couple of years, with the exception possibly of Macbooks, which cost a King's ransom. There's no novelty or interesting ideas, just the same old things.
      - dragonwriter 8y ago
        > Now all laptops practically look identical No, they don't. Most of the options you point to from “not long ago" (and many more) are still available. Okay, I don't see liquid cooled or fold-out legs much, and swivel screens seem to have been displaced with the 360-degree hinges.
dragonwriter 8y ago
The problem is tech: that is, if you are using algorithms to drive (ultimately) payments to creators, than creators have an incentive to game the algorithms and will (on average) fail to thrive and be forced to find some other business if they do not. So, the market will be saturated with those gaming the system, because that's what it pays for.

ryandrake 8y ago

If you run a platform and its algorithms can be gamed by “partners” for profit at your’s or the user’s expense you should expect that it is or will be gamed. An important part of designing tech products that scale is sitting down and anticipating all the various ways you are incentivizing cheating and gaming, and then closing the holes as bugs. It’s like doing a security audit, with the attackers being your partners rather than hackers. It’s best to fix the gaming via technical means, but if you can’t, it’s still important to spell out unacceptable behavior in the terms of use. Even if it’s something you can’t enforce at scale (for instance: enforcement requires manual review by humans) you still need the policy so you can do one-off enforcements against egregious cases. Otherwise you sit around in these meetings going, “well we know they are gaming the system but we can’t do anything about it because it’s not against the rules!”

josephjrobison 8y ago
I could be wrong but I wonder if they want to leave the loopholes open a bit to see where they get exploited and then slowly roll out updates. Not that they’re intentionally creating bugs but perhaps seeing how they get exploited and where so that the authors are doing the work for them rather than internal engineers.
- gfo 8y ago
  In a perfect world, maybe. It's hard to tell with Amazon now. Fraud and counterfeiting have become rampant on the platform since they opened up third-party selling to all foreign nations 18 months ago and despite the negative press and angry consumers, they've said and done very little to combat these issues. It's difficult to find other reasons why nothing has done beyond their bottom line numbers growing significantly. Generally speaking a business should find flaws and fix them before releasing. Iterative development is okay, but when issues like this become commonplace and nothing is done to combat it, that's when there's a problem. I have no solutions to offer.

zeristor 8y ago

Do you think Her Cocky Software Developer would be a big seller? This is the real issue

zeristor 8y ago
I had to look up how to do italics
allwein 8y ago
I know you made this comment as a joke, but there actually is an entire sub-genre of geek-related romance novels.
- zeristor 8y ago
  These lot seem sweet and charming: http://twincitiesgeek.com/2017/07/24-romance-novels-for-geeks-of-all-kinds/ http://twincitiesgeek.com/2017/07/24-romance-novels-for-geek...

guitarbill 8y ago

The Kindle store is already garbage for any kind of discovery, and Unlimited makes it worse being just more marketing BS to hide the real price and displace good content. Their suggestion algorithms are terrible, and only really work if you want something from the same author. Steam has exactly the same problem, although ar least they‘re trying to solve it in various ways.

jlarocco 8y ago
It's interesting how different people can have such different expectations. I personally haven't had any issues finding the books I want, and I find Amazon's suggestion algorithms helpful precisely because it works as you describe. Almost all of my book purchases are made because somebody's recommended a specific book or author to me, or because I'm interested in a specific topic or author and searched for them. In both cases, I'm fine with getting generic suggestions based on the author or topic.
gfo 8y ago
Are there examples of businesses out there which have done product recommendation correctly? Amazon is notorious for trying to sell you the same kind of product after you've already purchased it (e.g. I bought a ladder and now all of my recommendations are for more ladders). How hard can it be to uncover which items are more likely to be one-time buys and recommend the products people buy with those items instead (ex. show me painting supplies when I buy a ladder)?
- virtuabhi 8y ago
  Because there is x% chance that you might return this ladder and get another one. The ladder ads should go away after the expiration of free return period.
- rhizome 8y ago
  Are there examples of businesses out there which have done product recommendation correctly? I'm not sure what you mean by "correctly," but in general, if you're talking about automation and recommendation engines, my answer is "no." Of course, I don't know the goals of the companies implementing them and whether my sense of "correct" recommendations matches theirs. I don't think any of us do.

rpeden 8y ago

From the article: Valderrama allegedly said that she organized newsletter swaps, in which authors would promote each other’s books to their respective mailing lists. Funnily enough, I've seen this same behavior from developers, freelancers, and authors who are "internet famous" on HN and Reddit. I've signed up for quite a few newsletters from these people because they often provide useful information. But since I signed up for so many, I started noticing that fairly often, they'd recommend each others' products. I realize it's entirely possible that these endorsements are authentic, but after a while, it starts to look like it's an interconnected web of people that met at conferences and agreed to cross-promote each each other on their respective newsletters. And so it ends up feeling like each recommendation isn't really authentic, but is instead more of a quid pro quo in exchange for a reciprocal recommendation. And it's possible that I'm completely wrong and just imagining all of this. It's just one of my pet peeves, so perhaps I'm exaggerating how prevalent it is. And don't even get me started on developers who have resorted to hokey long-form sales copy to sell their books and services... :)

ageitgey 8y ago
I'm a person with a mailing list in this space who has recommended other people's products, so I can maybe offer some insight on how this world works. The typical person working in this niche is a solo creator who maybe splits there time creating free content and doing paid consulting projects. The business model is usually something like "write a lot of regular free content to build up a giant mailing list and then occasionally sell something to your mailing list to pay the bills". The problem is that it takes a lot of time and effort to create good content. You are often so drained creating your "free" content that you don't have enough time or energy left to create enough "paid" content regularly enough to pay the bills. So you find friends who have created great stuff that you really like and you help each other out by cross-promoting. In almost every case, it's a direct business relationship. The typical deal in this niche for reselling electronic content is a 50/50 split. So you help your friend sell extra copies (which is just free income to them) while you also make decent money to sustain your work. Of course once you get "magic internet money" coming in this way, it's very tempting to abuse your audience and oversell. But then you lose subscribers quickly. Some people are very cynical about this and literally have spreadsheets to maximize how much they can sell while minimizing user attribution, but I'm not a fan of that. It just feels gross. So yes, it is indeed a small, interconnected web of people who agreed to cross-promote each other. And sometimes the recommendation isn't authentic, but often it is. But in almost case, it's not an exchange of free recommendations, but it's literally an affiliate split between the authors. As far as long-form sales copy to sell books, it is definitely very hokey. But the counterpoint is that most other approaches just don't convert. A cold lead who finds you via Google needs to be convinced that your stuff is worth money and long-form sales copy is one way to do that. I also hate it and completely avoid it, but I understand why people do it.

deleted 8y ago

[deleted]

Sujan 8y ago

Fascinating that Kindle's still don't track which pages actually were "read" (as in "skipping from page 1 to page 100, doesn't meant 2-99 have been read). Basing the author payout system of Kindle Unlimited on that metric is the (technical) root cause here.

asmosoinio 8y ago
The OP seems to disagree with you, mentioning that Amazon has some algorithms to determine actually read pages? Or do you have other information?
- Sujan 8y ago
  What part of the text are you referring to exactly? There was a sentence that Amazon's metric isn't influenced by big fonts or other formatting choices (they probably just use a plain text version of the text to define what a "page" is) and 'that the KENPC system (Kindle Edition Normalized Page Count) recorded pages read with “high precision”' - which doesn't actually mean that much. Did I miss the interesting part? My point was mostly based that I know it was that way for a long time, and that things like stuffing or those newsletter signups or competition in the back of the book wouldn't have any influence if it still wouldn't be this way.
  - asmosoinio 8y ago
    This sentence, mentioning "pages read": A June blog post by the Kindle Direct Publishing Team assured authors that the KENPC system (Kindle Edition Normalized Page Count) recorded pages read with “high precision” and that the company was constantly working to improve its “fidelity.”
    - Sujan 8y ago
      Yeah, as far as I know this has always been the official wording. What makes me skeptical here is the use of "high precision" and what there could be to improve... a page was either viewed for the time necessary to read it or not. Over all readers of a book, it should not be too hard to find out the distribution of valid read times of a page. Unless, of course, you don't have that data, but only some snapshots of which page is opened (e.g. when a sync to the cloud servers happens). Shame the Kindle APIs are undocumented :/

sago 8y ago

I have a feeling that the whole Erom genre is where porn videos were 10 years ago. Low quality, keyword-stuffed, fad-following, repetitive content is so ubiquitous (and all that's needed to get someone off) that someone will come along and be the Erom Hub and swipe the legs out from under the market. The interesting question, that the article raises but does not answer, is how far Erom is demonstrating a pattern that'll be followed by other genres. There is a lot of similar self published genre fiction, but it is still a different order of magnitude to KU contemporary romance.