I’m going to be that guy. The one nobody likes at parties. The one who speaks unpleasant truths. If you don’t want to hear unpleasant truths, stop reading.

If you want to be told which self-help books to buy and which things to do and which gurus will illuminate the shining path to fame and fortune, stop reading.

If you want somebody to hold your hand, and nod at all the right moments and ooh and aah about how your writing has come a long way and you’re “almost there,” stop reading.

It doesn’t matter whether you’ve come a long way. It doesn’t matter whether your writing is almost there, is there, or is beyond there. It doesn’t matter what you’re saying or how you’re saying it. You may have written the most poignant 80,000 words in the English language, or you may have another book of cat photos. None of that matters.

Unless you’re a certain type of person saying a certain type of thing in a certain way, none of it matters. And that certain type of person, that certain type of thing, and that certain way changes all the time. Today it’s one thing, tomorrow it will be another.

Statistically speaking, you’re not it.

“But what about all those success stories,” you argue. “I’m always hearing about Amazon success stories. Success, success, success! This book mentioned them and that blog mentioned them and the 12th cousin of my aunt’s best friend’s roommate had one.”

There are two reasons this doesn’t matter.

Most of those stories are part of a very large industry of selling hope to suckers. Any endeavor which appeals to the masses and appears to be accessible to them spawns such an industry. Business, stock picking, sex, dating, how to get a job, how to get into college, and on and on. Thanks to today’s low barrier to entry, self-publishing is the newest kid on that block.

This isn’t a conspiracy, or some evil corporation with a beak-nosed pin-striped CEO, cackling ominously while rubbing his hands. Self-publishing just attracts a lot of people who see an easy way to make money. When there’s a naive, eager audience, a host of opportunists and charlatans purvey snake oil to any sucker willing to pay. They’re predators, plain and simple. Hopefully, I can dissuade you from being prey. Leave that to others. Others unenlightened by my blog. Cynicism may not always be right, but it’s rarely wrong.

Even seemingly reputable characters have become untrustworthy. The traditional publishing industry has grown very narrow and institutional, and life is hard for everyone associated with it. The temptation to go for the easy money, and cast scruples to the winds, is quite strong. Not that denizens of the publishing industry ever were big on scruples. Many individuals from traditionally respectable roles as agents, editors, and publishers find it increasingly difficult to eke out a living or are growing disillusioned with a rapidly deteriorating industry. It is unsurprising that they are bedazzled by the allure of easy money. Unsurprising, and disappointing. This is especially insidious when agents offer paid services which purport to help improve your chances with other agents. The argument is that they know what their kind wants. Anybody see the problem with this? Anybody, anybody, Bueller? It would be like H.R. employees taking money to teach you how to get a job with them. Oh wait, they do. How could THAT possibly go wrong…

I’m not going to delve into the “selling hope to suckers” angle here. That is fodder for a separate post, in which I analyze a number of things which did or did not work for me. For now, I’ll focus on the second reason your book won’t be an Amazon Success Story. Incidentally, I will resist the temptation to assign an acronym to Amazon Success Story. There! I successfully resisted it.

In this post, I’ll assume that ALL those stories you hear are right. Not that they’re 99% bunk or that most actual successes had some outside catalyst you’re unaware of or were the result of survivorship bias (the old coin-flipping problem to those familiar with Malkiel’s book). To paraphrase the timeless wisdom of Goodfellas, if you have to wait in line like everyone else you’re a schnook. If you’re trying what everyone else tries, making the rounds of getting suckered for a little bit here a little bit there, with nothing to show for it — you’re the schnook.

Don’t feel bad, though. No matter how savvy we are in our own neighborhoods, we’re all schnooks outside it. Hopefully, I can help you avoid paying too much to learn how not to be a schnook.

I can’t show you how to be successful, but I can show you to avoid paying to be unsuccessful. But that’s for another post. We’re not going to deal with the outright lies and deception and rubbish here. Those are obvious pitfalls, if enticing. Like pizza.

In this post, we’re going to assume the success stories are real — as some of them surely are. We’re going to deal with something more subtle than false hope. We’re going to discuss the OTHER reason you won’t be successful on Amazon. It’s not obvious, and it can’t be avoided.

But first, I’m going to make a plea: if you’re the author of one of those breathless, caffeinated “how to be a bzillionaire author like me” books or blogs or podcasts … stop it. Please. Just stop it. Unless you’re cynically selling hope to suckers or mass-producing content-free posts as click-bait. In that case, carry on. I don’t approve of what you do, but I’m not going to waste breathe convincing dirtbags not to be dirtbags. However, if you’re even the least bit well-meaning, stop. Maybe you have some highly popular old posts along these lines. Update them. Maybe you’re writing a new series of posts based on what your friend named John Grisham has to say to self-publishing authors. Don’t.

You’re doing everyone a disservice. People will waste money and time and hope. Best to tell them the truth. You may not be that guy. You may be too nice, tactful, maybe even (dare I say) an optimist. I’m not an optimist. I AM that guy. No false hope sold here.

Maybe you’re still reading this and haven’t sky-dived into a volcano or fatally overdosed on Ben & Jerries, or turned to one of those cheerful, caffeinated blogs. Shame on you. There’s special internet groups for people like you. But you’re still here, and I haven’t driven you away. I must be doing something wrong.

If you’re a true dyed in the wool masochist, I’ll now explain why you won’t be successful. It has to do with a tectonic shift in Amazon’s policies.

Over a year ago, I wrote a post titled “Why NOT to use Amazon Ads for your book,” which many people have written me about. Most found it a useful take on Amazon ads, and one of the few articles which doesn’t regurgitate lobotomized praise for the practice.

I stand by that. Subsequent experiments (to be reported in a future post) have shown that Amazon ads perform even worse now. This led me to wonder why. Why did all the long-tailed keywords and the reviews and the ads make no difference. None of us know the precise inner workings of Amazon ads, but there are strong indications of their behavior.

I now will offer my theory for why there are success stories, why it’s tempting to believe they can be emulated, and why they cannot. To do so, let’s review some basic aspects of Amazon’s algorithms.

There are two algorithms we care about:

(1) The promotion algorithm, which ranks your book. It is responsible for placing it in any top 100 lists, determining its visibility in “customers also bought” entries, when and how it appears in searches, and pretty much any other place where organic (i.e. non-paid) placement is involved.

(2) The ad auction algorithm, which determines whether you win a bid for a given ad placement.

The promotion algorithm determines how much free promotion your book gets, and is critical to success. It has only a couple of basic pieces of information to work with: sales and ratings. The algorithm clearly reflects the timing of sales, and is heavily weighted toward the most recent week. It may reflect the source of those sales — to the extent Amazon can track it — but I have seen no evidence of this. As for ratings, all indications are that the number of ratings or reviews weighs far more heavily than the ratings themselves. This is true for consumers too, as long as the average rating is 3+. Below that, bad ratings can hurt. Buyers don’t care what your exact rating is, as long as there isn’t a big red flag. The number of ratings is seen as a sign of legitimacy, that your book isn’t some piece of schlock that only your grandmother and dad would review — but your mom was too ashamed to attach her name to. Anything from a traditional publisher has 100’s to 1000’s of ratings. A self-published work generally benefits from 15+. More is better.

It makes sense that the promotion algorithm can play a role, but why mention an “ad auction algorithm”. Ad placement should depend on your bid, right? Maybe you can tweak the multipliers and bids for different placements or keywords, but the knobs are yours and yours alone. You might very well think that, but I couldn’t possibly comment. Unlike the ever-diplomatic Mr. Urquhart, I’m too guileless to take this tack. I also don’t use Grey Poupon. I can and will comment. You’re wrong. Amazon’s ad algorithm does a lot more behind the scenes. You may be the highest bidder and still lose, and you may be the lowest bidder and win.

As usual, we must look at incentives to understand why things don’t behave as expected. Amazon does not run ads as a non-profit, nor does it get paid a subscription fee to do so. It only makes money from an ad when that ad is clicked, and it only makes money from a sale when the ad results in a conversion. For sellers, the latter is a commission and for authors it’s the 65% or 30% (depending on whether you chose the 35% or 70% royalty rate) adjusted for costs, etc. In either case, they make money from each sale and they make money from each click.

Amazon loses money if your ad wins lots of impressions, but nobody clicks on it. They would have been happier with a lower bid that actually resulted in clicks. If lots of people click on your ad, but few people buy your book, Amazon would have been happier with a lower bid which resulted in more sales. It’s a trade-off, but there are simple ways of computing these things. When you start fresh, Amazon has no history (though perhaps if you have other books, it uses their performance). It assigns you a set of default parameters representing the average performance of books in that genre. As impressions, clicks, and sales accrue, Amazon adjusts your parameters. This could be done through a simple Bayesian update or periodic regressions or some other method.

When a set of authors bids on an ad, Amazon can compute the expected value of each bid. This looks something like P(click|impression)*ebid + P(sale|click)P(click|impression)*pnl, where P(click|impression) is your predicted click-through-rate for that placement, P(sale|click) is your predicted conversion rate for that placement, ebid is the effective bid (I’ll discuss this momentarily), and pnl is the net income Amazon would make from a sale of your book. This is an oversimplification, but gets the basic idea across.

The ebid quantity is your effective bid, what you actually pay if you win the auction. There actually are two effective bids involved. Amazon’s ad auctions are “second-price,” meaning the winning bidder pays only the 2nd highest bid. Suppose there are 5 bids: 1,2,3,4,5. The bidder who bid 5 wins, but only pays 4. There are game theoretic reasons for preferring this type of auction, as it encourages certain desirable behaviors in bidders. In this case, the effective bid (and what Amazon gets paid) is 4. That is no mystery, and is clearly advertised in their auction rules. What isn’t advertised is the other, hidden effective bid. These effective bids may be 3,2,4,2,3, in which case the third bidder wins. What do they actually pay? I’m not sure, but something less than their actual bid of 3.

Apparently, whatever algorithm Amazon uses guarantees that a bidder never will pay more than their actual bid. It somehow combines the two types of effective bids to ensure this. I am not privy to the precise algorithm (and it constantly changes), so I cannot confirm this. However, I have been informed by an individual with intimate knowledge of the subject that Amazon’s approach provably guarantees no bidder will pay more than their actual bid.

Why would Amazon prefer a lower bid, when they could get 4? As mentioned, they only get paid 4 if the ad of the winning bidder (the 5) gets a click. If the ad makes every reader barf or have a seizure or become a politician, there won’t be a lot of clicks. If it’s the most beautiful ad in human history, but the book’s landing page makes potential buyers weep and tear their hair and gnash their teeth, it probably won’t make many sales. In either case, Amazon would do better with another bidder.

Even without knowing the precise formula, one thing is clear. These algorithms are a big problem for anyone who isn’t already a star.

The problem is that those two algorithms play into one another, generating a feedback loop. If you’re already successful, everything works in your favor. But if you start out unattractive to them, you remain that way. You have few quality ad placements, and get few sales, and this suppresses your organic rank. The organic rank factors into many things which affect P(click|impression) and P(sale|click) — such as the number of reviews, etc. Put simply, once they decide you’re a failure, you become a failure, and remain one. You won’t win quality bids, even if you bid high. If you bid high enough to override the suppression, then you’ll pay an exorbitant fee per click, and it will cost a huge amount to reach the point where success compounds.

I am unsure whether there is cross-pollination between works by a given author, but I strongly suspect so. A new work by a top-ranked author probably starts high and is buoyed by this success. This may be why we see a dozen works by the same author (obviously self-published, and sometimes with very few ratings per book) in the top-100 in a genre.

So how do you get out of this hole? There’s only one accessible way for most people: you cheat. And this is where Amazon’s tectonic policy shift comes into play.

There ARE success stories, like the aforementioned top-ranked self-published authors. But there won’t be any more. To understand why, we must turn to hallowed antiquity before Bezos was revealed to be the latest incarnation of Bchkthmorist the Destroyer, and when Amazon brought to mind a place with trees, snakes, and Sean Connery.

There was a time when the nascent self-publishing industry had really begun to boom, but was poorly regulated. The traditional publishers viewed Amazon, Kindle, and self-publishing as a joke. They relied on their incestuous old-boys network of reviewers from the NY Times, New York Review of Books, and pretty much anything else with New York in the name for promotion. 95% of self-published books were about how to self-publish, and authors who DID self-publish (and were savvy) quickly developed ways to game Amazon.

They COULD pump up their search results, get in top-100 lists, and so on. Usually, this involved getting lots of fake reviews and using keyword tricks to optimize search placement. Once in the top list for a genre, it was easy to stay there — though newcomers with more fake reviews and better keyword antics could displace you. The very top was an unstable equilibrium, but the top 500 or 1000 was not. Once up there, it was easy to keep in that range and then occasionally pop into the very top. Like a cauldron of mediocrity, circulating its vile content into view every now and then. Amazon periodically tweaked its algorithms, but authors kept up.

Then something happened. Amazon decided to crack down on fake reviews. This sounds laudable enough. Fake reviews have the word fake in them, and fake always is bad, right?

There were two problems with HOW Amazon went about it. First, they went way overboard. Overnight, it became well-nigh impossible for an author to get a single new review. If the reviewer had one letter in common with your name, lived in the same hemisphere, or also breathed air, they were deemed connected to you and thus biased.

If this had been applied uniformly, there would be nobody in the top 100 — or it would be random, since nobody would have any tricks they could play. This is where the second problem with Amazon’s approach came in. They didn’t remove legacy fake ratings. Those who cheated before the cutoff got to keep their position. In fact, that position now was secure against all newcomers. A gate had slammed down, and they were firmly on the right side of it. Aside from a few people near the boundary they had nothing to fear. Well, almost nothing to fear.

The only way to break into the top echelon, and thus benefit from the self-reinforcing algorithms which stabilize that position, is to rely on external sources of sales. If you have a million twitter followers who buy your book, or a massive non-amazon advertising campaign, you can break in. They YOU would be very difficult to displace.

Once traditional publishers realized that Amazon is the only de facto bookstore left (outside airport/supermarket sales), they took an interest. THEY have no problem getting a top rank, because they run huge advertising campaigns and have huge existing networks. This is why the top 100 lists are an odd mixture of self-published books you never heard of and traditionally published bestsellers. Eventually it only will be the latter.

So. You. Won’t. Break. In. Amazon created an impenetrable aristocracy, and you’re not it. You won’t be it. You can’t be it. If you use Amazon ads or buy into any of the snake oil sales nonsense, you’ll be the schnook bribing a maitre d’ who knows he’ll never let you in.

Most of those success stories (or at least the real ones) are from before the policy change, as are many of the methods being touted. That path is gone. Amazon ads only work for those who don’t need them, and they work very well for them. They won’t work for you. Becoming a success on Amazon is as unlikely as with a traditional publisher. You’ll always hear stories, but they’re either the few who randomly made it, those with hidden external mechanisms of promotion, or those already entrenched at the top.

That’s the sad truth, or at least my take on it. By all means, waste a few dollars trying. I used to be a statistical trader and know better, but I still buy a lottery ticket when the jackpot’s high enough. It’s entertainment. Two dollars to dream for a day. I just don’t expect to win.

Write what you want, revise, work your butt off, and make it perfect. But do it because you want to, because that’s what makes you happy. Don’t do it expecting success, or hoping for success, or even entertaining the remote possibility of success.

The worst reason to write is for other people. Your work won’t be read, and your work won’t make you money. If you accept that and are happy to write anyway, then write all you want. I urge you to do so. It’s what I do.