Category Archives: Uncategorized

I Finally Got My Patent

Official Patent

My database patent finally has been granted after a long and expensive ordeal. While this is better than not having it granted after a long and expensive ordeal, it still was a truly pathetic reflection on the state of the American patent system. My perception (and, from what I can gather, that of most sensible individuals) is that the American intellectual property system as a whole is broken beyond repair and is one of the primary impediments to real innovation in this country. The system only serves large corporations with deep pockets and large teams of lawyers — and they mostly use it to troll smaller companies or build defensive portfolios to deter competitor lawsuits.

But enough about the sewage dump known as the US Patent system; my thoughts on it are unambiguously expressed here. Instead, this is a happy post about happy things. And what makes people happier than a stream database?

My patent is for a type of stream database that can be used to efficiently manage and scan messages, business transactions, stock data, news, or myriad other types of events. It is the database I wish I had when I was doing high-frequency statistical arbitrage on Wall Street, and which I subsequently developed and refined.

I’ll post a more detailed discussion of it shortly, but here is the basic gist. The idea is based on a sort of indexed multiply-linked-list structure. Ok, maybe that’s too basic a gist, so I’ll elaborate a little.

To use a common example from stock trading, we may wish to query something like the last quote before a trade in a stock. As an individual query, this is easy enough to accomplish in any type of database. However, doing it efficiently and in high volume becomes more challenging. Standard relational and object databases quickly prove unsuitable. Even stream databases prove inadequate. They either require scanning lots of irrelevant events to arrive at the desired one or waste lots of space through sparse storage and/or are constrained to data at fixed intervals. But real data doesn’t work that way. Some stocks have lots of trades and few quotes, others have few quotes and lots of trades. Events happen sporadically and often in clusters.

My approach is to employ a type of multiply-linked list. Each entry has a time stamp, a set of linkages, and a payload. In the stock example, an event would link to the previous and next events overall, the previous and next events in the same stock, and the previous and next events of the same type and stock (ex. quote in IBM or trade in Microsoft). To speed the initial query, an index points to events at periodic intervals in each stock.

For example, to find the last quote before the first trade in IBM after 3:15:08 PM on a given day, we would use the index hash to locate (in logarithmic time) the latest trade prior to 3:15PM in IBM. Then we would scan trades-in-IBM forward (linkage 3) until 3:15:08 to pick the latest. Then we would scan IBM backward (linkage 2) from that latest trade until we encounter a quote.

We also could simulate a trading strategy by playing back historical data in IBM (linkage 2) or all stock data (linkage 1) over some period. This could be done across stocks, for individual stocks, or based on other specific criteria. If there are types of events (example limitbooks) which we do not need for a specific application, they cost us no time since we simply scan the types we care about instead.

This description is vastly oversimplified, and there are other components to the patent as well (such as a flow manager). But more on these another time.

If you’re curious, the patent is US 11593357 B2, titled “Databases And Methods Of Storing, Retrieving, And Processing Data” (originally submitted in 2014). Since the new US Patent Search site is completely unuseable (and they don’t provide permalinks), I’ve attached the relevant document here.

Official Patent

Some Holiday Cheer for Nascent Writers

Readers of my blog know that I’m not given to chatty, optimistic posts. In fact, my typical post is more along the lines of “Not only will you fail at writing, but your cat will run away, your house will burn down, and you’ll spend the rest of your life tweeting from a phone you forgot to take out of airplane mode.” This post is different. While it isn’t quite optimistic, it does offer a perspective you may find uplifting, perhaps even liberating.

I’ve participated in many writing groups over the years and have managed a few as well, including one which technically qualified as Boston’s largest at the time. I have many writer acquaintances and even a few writer friends. I’m not spouting this to toot my own horn, but to lend credence to what I am about to say.

Many writers seem to have a notion of success which I deem unhealthy. I’m not saying that we should redefine "success" so that everybody is a winner or any such happy horseshit. The problem is that writers have two competing, and largely incompatible, goals. I speak here of real writers, not people who simply produce a product. The difference, to my mind, is that a writer wants to be proud of their work. My own standard is that I write what I want to read, and I think many of us implicitly or explicitly have a similar benchmark. We may try to cater to the taste of the crowd or steer toward marketability, but catering is different than pandering and steering is different than veering. At the end of the day, the stories and books which we produce must satisfy us first and foremost. This does not mean we always succeed in meeting that standard, but it is what we strive to aesthetically achieve.

However, American culture imbues us with another standard of success — one that is financial and social. Though success in this regard can be achieved via various avenues, the essential value it embodies — and that which our society most greatly respects — is the ability to sell things. In practice, this often takes the form of selling people stuff they don’t really want — but it need not. We are taught that the "net worth" of an individual is the sum of their possessions, marked to market, and distilled to a number. We are taught that we can order people by importance from lowest to greatest based on that number. We are taught that if one author sells many books and another sells few, the first is much "better" than the second. And we are taught that if a big publisher picks up a book, it is a "better" book than any which are self-published. While many of us may vociferously reject such a simplistic and materialistic outlook, we nonetheless are thralls to it. We may know that fatty foods are bad for us and that consumerism destroys the environment, yet there we all are in front of brand spanking new 100-inch televisions with bags of Fritos in our hands. Knowing and feeling are two different things, as are knowing and doing. We know we shouldn’t adopt the typical American view of success but we do anyway. Understanding and accepting that we are susceptible to such internal contradictions is crucial to avoiding the misery they otherwise can engender.

For a writer, the ability to sell our writing is essential for American-style success. These days, this entails also selling our "own story" as well. I personally find this obsession with the author rather than their work vapid at best and venal at worst, but it’s a fact of the market. The demographics of who reads and how are vastly different than a few years ago, as are the nature of publishers and what they seek. You sell yourself, then your writing. This compounds an already difficult problem for most of us. Good writers are good writers, not necessarily good salesmen. Those who spend their time selling things and have the aptitude to do so rarely also have the time or ability to write a quality book — and those who spend their time writing and have the aptitude to produce a quality book rarely have also have the time or ability to sell it.

This is a practical reality that affects almost any creative or scientific field. Those who can do can’t sell and those who can sell can’t do. But there is a deeper issue as well: a conflict of what we actually deem important with what we imagine we should deem important. What we want as authors and what we have been trained to believe we should want as Americans are largely incompatible. If we achieve only the first, we see ourselves as failures. If we achieve only the second, we see ourselves as hacks. And it is well-nigh impossible to achieve both.

It is not difficult to see why. If you’re like me and have tastes that depart even in the slightest from the mainstream, then ask yourself how many books that you really love are being published by major publishers today. Not books you’re told you’ll love, or books that you’re supposed to love, or books other people tell you they love. But books you love. For me, it’s virtually zero. The type of writing I enjoy simply isn’t published anymore. At least not by big publishers, and probably not by small presses either. It’s still being written. I’m writing it, and I’m sure plenty of other people are too. But it’s at best being self-published, and as a result is very hard to find.

The same is true of the big successes in self-publishing and is the reason neither you nor I ever will be one. The best sellers are in a small set of genres and usually involve the same perennial cast of series and authors. These authors are very good at gaming the system — i.e. at selling their books. However, they are not authors in the sense I described. They view a book as a product and nothing more. They run a business and are very good at it. For them, there is no contradiction in goals because their sole goal is financial success. There is nothing wrong with this, but it is not sufficient (or even attainable) for an author of the type I am addressing this to.

I’ll give you an example of what I mean. When I used to live in New York City, there was a famous camera store I frequented. In the same building, the next storefront was a diving shop. One day, I needed to buy some diving gear and went into that shop. I recognized some of the employees from next door, and it turned out that both stores were owned and run by the same people. The employees in either store knew everything about what they sold. If I asked an obscure question about a camera feature or model, someone knew the answer. But if I asked a subjective question, such as which camera they preferred or which BCD they found comfortable when diving, they were of little help. They could opine about which model customers preferred, and they could rave about one or the other product in a sales-pitch sort of way, but they clearly had no personal experience with the products.

I wondered at this and asked a friend who moved in similar circles about it. He explained that the product didn’t matter. It was all about understanding the market and sourcing the products at low cost. The store employees were generic highly-skilled business people. They could go into any market, learn the jargon and product specs and market layout, source the products at a good price, and then advertise and hard-sell those products very effectively. To them, it didn’t matter what they were selling. The products were widgets. The owners of those stores probably had no especial love of photography or diving but recognized those as markets they could thrive in.

Almost all self-published authors who succeed financially are of the same ilk. The books are products, and they just as happily would produce wicker baskets if that was where the money lay. Such authors have no ambition to write a high-quality literary novel. If their market research says that novels about vampire billionaires who fall for midwestern housewives are the thing, they will pump out dozens of nearly-identical ones. In this regard, such writers are a bit like the big publishers. The main differences are that (1) these self-published writers produce their own products and (2) the big publishers seem to have lost their focus these days and now employ ideological criteria rather than purely market-related ones.

The result is obvious. If I write a book of which I’m proud, it won’t get traditionally published and it won’t sell much when self-published. At a more basic level, this is a problem which affects all "producers", including artists, scientists, and musicians. To succeed in the social/financial sense, you have to spend 100% of your time relentlessly promoting yourself (and even then, the likelihood of success is small), but to produce anything of substance you have to spend all your time developing your craft and then applying it. The product-writers I described are very efficient. They are experts at what they do. After all, even in the world of marketable-schlock there is lots of competition. The winners know how to game the search engines, get in early and stay at the top, spend marketing money efficiently, and expend the minimum time necessary to produce a salable product.

The gist is that the two goals of a real writer are utterly incompatible. Writing a book we are proud of and achieving social/financial success with it are mutually exclusive for most of us. Unless you really love writing crowd-pleasing schlock or happen to be one of the handful of random literary "success" stories, it is impossible.

"But Ken," I can hear you whining, "I thought you said this would be uplifting? That my cat would still love me and my house wouldn’t burn down and I’d remember to turn off airplane mode before tweeting. How the hell is this remotely optimistic? Do you secretly run a razor-blade and cyanide business on Amazon?" Well, yes and no. Since books don’t sell, I do need some side hustles. Please visit my Amazon page for a very special offer.

Ok, fine. Here’s the inspirational bit. It isn’t that we can’t achieve both goals — it’s that we don’t have to adopt both goals. You are in control of your goals, even if your social programming reeeeally wants you to think otherwise. If your goal is to be both successful in the American sense and proud of your work, you’re going to be bitter and miserable. It’s disheartening and you’ll give up as a writer or feel resentful toward the world. But that shouldn’t surprise you. If you demand the impossible, you’ll always be disappointed. If your goal was to be a fantastic high-school teacher and also become rich from it, you’d be miserable too. It’s very hard to succeed financially in any way, let alone one which appeals to you. If I wanted to be a professional basketball player, I’d be disheartened. I’m five-foot-eight. The fault wouldn’t be with the world, it would be with me for demanding the impossible. While it’s admirable to pick difficult but attainable goals, picking wildly implausible ones is a recipe for misery. If you set out to prove the world wrong, all you’ll do is prove yourself a fool. Not because the world is right, but because there’s no point in wasting your life trying to prove anything to nine billion people who won’t notice and couldn’t care less if they did.

I’m not spouting some hippy nonsense about eschewing material possessions. You need money to survive and live comfortably. Money can buy you independence and free time. I’m not saying you don’t need money or shouldn’t pursue it. Just don’t rely on your writing for it, at least not if you want to be proud of that writing. It is perfectly fine to aim for American-style success. It’s difficult, but anything worth striving for is difficult. Nor is it unattainable, assuming your ambition isn’t too extravagant. In that’s your primary aim, do what the successful schlock-producers do and maybe you’ll succeed.

I’m also not saying you should sit in a corner munching a soggy carrot like some dejected rabbit. It is perfectly fine to write books you are proud of and hope for American-style success. I hope that my lottery ticket will win a billion dollars. There’s nothing wrong with that. It may even happen. Hope can be beneficial.

What is not fine is to expect American-style success from your writing. That is toxic. It means you’ll never reward yourself. Even if you write the greatest novel in the world, you still won’t allow yourself a sense of accomplishment. Imagine a small-town artisan who crafts beautiful furniture but demands that each piece be featured on some television show. He’ll be perpetually disappointed. No matter how great his skill and attainment, he never can give himself the slightest praise. There’s always a monkey on his shoulder telling him "So what? You’re not on television." If you’d laugh at such a person, take a good, hard look in the mirror.

To illustrate our biases, here are some scenarios. Suppose you learned that a friend …

  1. Wrote a wonderful book, got rejected by 200 agents, self-published it, and sold 3 copies.
  2. Self-published a vampire-billionaire-loves-midwestern-housewife book and sold 50,000 copies.
  3. Wrote a vapid, self-indulgent novel with elements designed to appeal to certain political sensibilities, which has been picked up by a major publisher.
  4. Self-published a book of pictures of cats with cute little taglines, which went viral and sold 100,000 copies.
  5. Wrote a passable book, though nothing worthy of note, but knew some agents and got picked up by a major publisher.

Most of us automatically would be "impressed" by (2)-(5) but view (1) as a vanity project. That is ridiculous. It’s our subconscious American training at play. Think about it. (2) may be a worthy businessman but isn’t really a writer, (4) produced a little nothing and got lucky, (5) produced tofu but knew the right people, and (3) produced what best could be termed a "vanity project" which ticked the right boxes. Of the five, only (1) produced something actually worthy of praise.

Nor are these contrived scenarios by some bitter rejectee (aka yours truly). Anyone who has contact with the publishing world knows that these are highly-realistic scenarios and that they are way more common and apropos than most of us would care to admit. So why do we view (1) this way? It’s not just our American-success programming. It’s also because of another very common scenario:

  1. Wrote complete trash, self-published it, and sold 3 copies.

(6) is what gives a bad name to self-publishing and constitutes the vast vast vast majority of self-published work. It and (2) are the reason you won’t be able to be heard above the fray or find your niche audience or sell many books.

But that isn’t as awful as it sounds. For most of history, only a privileged few even knew how to write, fewer had the means and leisure to write a book, and fewer still ever got published. Even if you wrote an incredible manuscript, without the money or connections to publish it that manuscript would end up in someone’s fireplace. So what’s different now, you may ask? Isn’t the problem the same, and only the gatekeepers and criteria have changed?

Yes and no. Yes, if you go through the gatekeepers. No, because you don’t have to. You can write a book you are proud of and self-publish it. It will be up forever as print-on-demand (and/or an ebook). You don’t have to build buzz, have a grand launch, and pray you reach critical mass before the rest of your print run gets remaindered and you end up out-of-print forever. Instead, you can put your book out there and point people to it over the years as you see fit. You can market it later when you have time or some opportunity arises. A book you are proud of will be available for anyone to purchase. Your backlist never goes away. Yes, a lot of crap gets self-published today — whereas in yesteryear only a few rich people could self-publish. But that need not bother you. Bad company does not a knave make. You’re not counting on people discovering your book by wading through all that garbage. You’re just making it available. You are the discovery mechanism. When someone asks about your book, you can point them to it. If you so choose, you can spend some money to increase the chance people will buy it. You can do this when and how you want.

And if someone at a cocktail party looks down their nose at you when you mention that you are self-published, just ask them what they’ve done lately. I wouldn’t worry too much about this happening, though. Does anyone even have cocktail parties anymore?

Incidentally, through much of the last three centuries there were no traditional publishers. Everything was self-published. But there was much less of it. Now, everyone can self-publish and everyone does. But just because a lot of other books stink doesn’t mean yours does — or that it will be viewed that way by modern, intelligent people.

In conclusion:

(1) Stop thinking of self-publication as a stigmatizing last-resort and a humiliating proof of failure. It is a tool and an opportunity. Moreover, in today’s world it is both a necessity and a reality for almost any author of substance. (2) If you write a book you are proud of, allow yourself to be proud of it. Feel successful. Decouple this sense of success from guilt or shame or anxiety about it not selling. (3) Write books you are proud of. Hope for American-style success if you wish, but do not expect it. (4) Keep writing. Write what you want to read. Be pleased that you have accomplished something.

If you complete one story you are proud of, you have accomplished more in your life than 99% of people. If you complete one book you are proud of, you have accomplished more than 99.9%. If you spend your life writing books you are proud of and allow yourself to be proud of them, you will have accomplished something almost nobody does: you will have lived a life you are proud of.

Everything everyone does is for naught, "vanity and a striving after wind," to quote Ecclesiastes. Had children? Your genes will dilute out of their progeny after a few generations. Became famous? Nobody will remember you a few years from now, and if they do it will be a mere caricature. Made a lot of money? You will is the last time you get any say how it’s spent. The best you can do is live a life you are proud of. Once you’re gone, the universe ends. It is irrelevant how many people bought your book or whether it lives on or your name is remembered.

And on that uplifting note, I once again refer you to Ken’s razor-blade and cyanide shop on Amazon. Oh, and don’t forget to leave a great review when you’re done…

“The Tale of Rin” serialization is live!

Update (8/8/22): I’ve ditched Vella. The Tale of Rin now exclusively is available on Substack. I’ve amended the post below to reflect this.

My epic fantasy series “The Tale of Rin” now is being serialized on Substack. New episodes come out on Wednesdays and Sundays. Here’s the description:

Just because Rin is indestructible doesn’t mean she can’t be hurt. On her quest to remedy an ancient sin, a single act of casual cruelty sets off an avalanche of events which threaten to destroy everything. Rin must rein in her assistant, a man of fierce attachment and questionable conviction, while avoiding her devious ex-husband, who will stop at nothing to reclaim her. In the balance lies her heart and the fate of the world.

Of the anticipated 6 volumes in the series, the first 2.5 have been written (and the rest mapped out). The first is publication ready and the 2nd close to that state. The first volume, “Protege”, likely will serialize to around 70-80 “episodes”.

The first 10 episodes will be free on Substack. Paid subscriptions are $5 per month (approximately 8 episodes) or $30 per year. Frankly, I would have preferred to charge less, but $5 is the lowest Substack allows. Note that a paid subscription gives you access to previous episodes as well — so don’t hesitate to subscribe at any point!

This is my first serialized novel, so please make some allowance for a few hiccups early on. If you encounter formatting errors or other weirdness please let me know. Such things aren’t always evident to the author, and these platforms are really clunky and buggy to post to.

Disclaimer: This is a work of original fiction, and any resemblance to real people or other fiction is purely coincidental.

Sensitivity Warning: This work may not be appropriate for readers highly sensitive to violence (including the occasional implication of possible sexual violence). Reader discretion is advised. Based on beta reader feedback, I’ve toned down a couple of scenes to make it more palatable to a broad audience. If you’re ok with something like Game of Thrones you should be fine with this.

I hope you enjoy the world I create and have a thrilling journey through it. Whether or not you find Rin an amiable companion, I’m sure you will find her an interesting one.

Trash Talk

These days, every minor institutional faux pas draws a melodramatic fawning apology utterly devoid of a modicum of self-respect and expressed through the metallic insincerity of boilerplate buzzwords . By now, one or more generations have grown up bombarded with such nonsense. We  only can imagine what they must be like at home…

Hey Bob, listen, it’s no big deal, but could you take out the trash when it’s your turn? It’s really been piling up.

I’ve heard you loud and clear.


My top priority has been fostering a community which values inclusiveness, mutual respect, and constructive engagement. A place where all perspectives, values, diverse viewpoints, and lifestyles are cherished.

Um, ok. Sure.

I realize I’ve fallen far short of my high ideals in this regard, and promise to do better.

Great. So … you’ll take out the trash?

But it’s not enough to be sorry. I know there has been an inexcusable breach of trust, and that my actions have caused deep hurt and lasting anguish.

If you feel bad, you could, like, take out the trash.

I can do better. I will not be complacent in the face of such a challenge. This is an opportunity for reflection and learning, to grow into a better version of myself.

Really, it’s not that big a deal. You just take the trash and put it in the bin.

Change is necessary, and the first step toward such change is to understand the scope of the problem.

That’s easy. The schedule is on the fridge. Just, you know, do it.

Toward this end, I have identified several important steps.

There’s really just one.

First, I enrolled in a sixteen-week sensitivity training course, mandatory for me, myself, and I.

Is that the reason you didn’t do any other chores for the last sixteen weeks?

I also hired an outside firm to thoroughly investigate my past behaviors and recommend a path forward. You may have noticed them here and there recently.

You mean that guy who crashed on the couch and ate all my Doritos? I thought he was a friend of yours.

After a rigorous investigation, we have concluded that all policies and procedures were followed and there was no misconduct.

You’re not going to take out the trash, are you?

The repercussions of trashgate are ongoing, and I will not rest on my laurels. I can do better, and I will do better.

Can part of your “not resting” involve moving trash from the kitchen to the bin?

That I did not intend my actions to be offensive is no excuse for the anxiety and pain they have caused.

It doesn’t smell great, and can attract roaches.

Nor do those actions reflect who I am as a person.

Pretty sure they do.

However, in the face of the continuing public reaction, my involvement can only serve to distract from our community’s valuable mission.

I think I know where this is going.

In consultation with myself, I have concluded that the best way for us all to move forward is for me to step down from my trash removal responsibilities.

You know, you could have just refused up front.

Although my formal role has diminished, I will remain active in other aspects of our vibrant and innovative community.

In other words, you’ll continue to use the foosball table.

I only hope these steps can bring some small measure of closure to those who have suffered through my thoughtless actions.

The only closure we need is of the trash bin.

Fun with Voting in Cambridge, MA

My city of Cambridge, MA is one of a few municipalities which employs ranked choice voting for City Council elections. Unlike most cities, the Mayor is chosen by the City Council and is largely a ceremonial position. Most real power resides with the City Manager, who is appointed for an indefinite term by the City Council. This means that City Councils which get to appoint a new City Manager exert an inordinate influence over the future course of the city. One such point is fast approaching. Unfortunately, given the present and probable near-term composition of the City Council, the decision likely will be based on considerations other than aptitude. However, putting aside my city’s somber prognosis, the upcoming City Council election is a good opportunity to discuss an unusual method of voting and some of its shortcomings.

Ordinary winner-takes-all elections dominate the popular consciousness. National elections are of this nature. It would not be inaccurate to observe that such an approach reflects the general weltanschauung of our culture. However, there are many other voting methods. In fact, voting theory is a vibrant field of research. Together with its sibling, auction theory, it forms part of the subject commonly known as “social choice theory”.

As an aside, I recently published a paper, Social Choice using Moral Metrics in that field. It focuses on measuring distances between behaviors, rather than on voting systems per se. Back in 2008, I also wrote a voting theory piece about swing votes and block voting. What I termed “influence” in it is more commonly referred to as “voting power”. Neither are related to what I discuss in this post, but I encourage the interested reader to peruse them.

It may be argued that certain voting methods are fairer than others, by one or another definition of fairness. Particular flavors sometimes are advocated by those disenchanted with an existing method or an agenda to see some particular group gain influence.  Calls for change sometimes arise in response to highly-visible anomalies, election outcomes which appear egregiously unfair even to disinterested eyes.

In elections with a large field of candidates or those in which a number of positions are simultaneously filled (such as the Cambridge City Council election), winner-takes-all voting may not be suitable or may give rise to such anomalies.

California’s recall system is an example. The ballot in that case has 2 questions: (1) whether to recall the governor and (2) who should replace him. The first question is winner-takes-all for the governor alone. If he loses, the 2nd question is winner-takes-all for the other candidates. It is quite possible for a candidate to be chosen who easily would have lost to the recalled governor one-on-one. In 2003, 44.6% of voters voted not to recall Governor Davis. He thus was recalled, and Schwarzenegger then won with 48.58% of the votes for replacement. It is highly unlikely that in a head-to-head gubernatorial election, Republican Schwarzenegger would have beaten Democrat Davis in the heavily blue state. However, Gray was excluded from this 2nd contest and Schwarzenegger was deemed preferable to the alternatives by most voters.

Arrow’s Theorem

It is natural to ask whether any voting system is unimpeachably fair, indicting the use of other systems as anachronistic or disingenuous. Arrow famously proved that, under even a small set of fairness constraints and for a broad class of voting systems, it is impossible to find one. Loosely speaking, when more than 2 candidates are present, no method of aggregating the rankings of candidates by voters into a single outcome ranking can simultaneously satisfy three conditions: (1) if every voter prefers candidate x to candidate y, then x outranks y in the outcome, (2) no single voter’s preference determines the outcome (i.e. no dictator), and (3) if each voter ranks x relative to y (i.e. above or below it) the same way in elections A and B (though the order can differ between voters, of course), then the outcomes of A and B do too. I.e., if voters change their overall ranking of x and y or the relative placement of other candidates, but don’t change whether x is preferred to y or vice versa, then whether x outranks y or vice versa in the outcome is unchanged.

It is quite plausible to add more fairness conditions, but most plausible definitions of fairness would require at least these three conditions to hold. Arrow showed that there is no ranked voting system (including “preponderance of the votes”) in which unfair anomalies cannot arise.

As an aside, if one were to relax a condition, the most palatable clearly would be (3). It is conceivable that a “fair” aggregation method may allow the overall ranking of candidates to affect a pairwise order in the outcome. However, this generally is deemed undesirable.

As with complexity results in computer science (CS) or Godel’s impossibility theorem in logic, the theoretical existence of hard or problematic cases does not necessarily pose a practical obstacle. In CS, an algorithm with worst-case exponential complexity may be far more useful than one with linear complexity in real-world applications. For example, the latter could have a huge constant cost (often referred to as a “galactic algorithm”) and the former could be exponential only in an infinitesimal fraction of cases or under circumstances which never arise in practice. Godel’s theorem does have real-world examples (i.e. non-meta-theorems), but (at this point) they remain rare.

Though nowhere near as profound,  Arrow’s theorem invites similar skepticism.  The impossibility of a preference system which excludes all anomalies does not mean such anomalies arise in practice, or that a system which excludes all realistic anomalies cannot be found.   Unfortunately (or fortunately, depending on one’s perspective), such anomalies do arise in practice.  Worse,  the systems in question often are of significant social import and subject to intense scrutiny.  The anomalies which do arise can be quite visible and politically troublesome.

Social choice theory exhibits another critical difference from CS and logic, one which merits additional caution.  The goal of logic, mathematics, and theoretical computer science generally is to understand which problems are solvable and how best to solve them.  Anomalies are viewed as pathological and undesirable.  They sometimes serve as useful counterexamples, guiding researchers to better understanding and helping them improve their tools.   However, they are to be avoided in real-world applications.   If a pathological case arises in such a context, alternate machinery must be employed or the framework modified to exclude it.

This need not be the case in social choice theory.   Not everyone’s goal is aligned, or social choice would be unnecessary.    With elections, there could be adverse incentives. It may be possible to game an election by identifying and exploiting anomalies endemic to the specific system involved.  There also may be groups who strongly prefer that anomalies arise, either for purposes of fomenting discord or if those anomalies serve them well.  For this reason, dismissing anomalies as almost impossible under some assumed prior may be naive. The prior must incorporate human behavior, and this very well could concentrate probability around the anomalies.  Put another way, if we naively model the probability of anomalies arising using an assumption of ideal behavior we risk ignoring the very real possibility that participants will engineer or utilize anomalies.

This issue is related to Gibbard’s theorem, which loosely states that under even weaker conditions than Arrow’s theorem (at least 3 candidates and no dictator), there is no ideal ballot which reflects a voter’s preferences. Put another way, the voting system can be gamed. In fact, a voter may need to game it (perhaps in response to polls or other information) in order to best reflect their individual preferences. The optimal ballot ranking to enact a voter’s preferences may not be their actual preference ranking of candidates.

The Rules in Cambridge

What does all this have to do with the Cambridge elections? Cambridge employs a particular system of ranked choice voting, which they refer to as “Proportional Representation”. This often is portrayed as fairer, more democratic, and so on. I am going to offer an example of an egregious anomaly which can result. I do this not in the expectation that it will arise or be exploited.  Nor do I hope to change a voting method that is, all things considered, quite reasonable.  Rather, the anomaly serves an illustrative example of the inherent problem with claiming that one voting system is “fairer” than another.

First, I’ll describe the precise rules of the Cambridge election, as best I understand them. See MA Election Laws, section 9 for details.  State law governs the general rules for proportional representation voting in any Massachusetts municipalities which choose to employ it.  Only certain parameters and details of execution are left to local discretion.

The City Council consists of 9 individuals, and the entire body is elected once every 2 years. Voters are presented with a list of candidates and may select a 1st choice, a 2nd choice, and so on.  I do not recall the maximum number of choices which can be made, but let us suppose it is not limited. The anomaly arises whether or not this is the case. Note that a given voter is not required to rank all the candidates. They could select only their top 3 choices, for example. Whether or not a full ranking by each voter is required does not affect the anomaly.

First some definitions. N will denote the total number of ballots (i.e. the number of voters who participate in the election).  At the time of writing, the minimum number of signatures to get on the ballot is 50.  We’ll call this ‘M’, because State law gives it a role in the algorithm. Q=(N/10)+1 will be the “quota”, the minimum number of ballots a candidate needs to win.

Why not choose Q=N/9?  The type of voting system we’re describing is sometimes referred to as “single-transferable-vote” (STV) because of the use of spillovers (described below). There are two common quota methods for determining STV winners:  (1) “Hare” corresponds to Q=N/9, and (2) “Droop” corresponds to Q=(N/10)+1.   In each case, we round up if needed. The two methods generally result in the same outcome or differ only in how the last winner is chosen. Each has benefits and drawbacks vis-a-vis what is deemed fair in terms of proportional representation. Among other things, the Droop quota tends to favor small parties over large. It also is the smallest quota which guarantees no more than 9 winners. As we will see, neither method guarantees a full complement of 9 winners.  Regardless, the Droop quota is that used by Cambridge.

Once the ballots have been collected, a sequence of steps is performed by computer. An order of polling places is determined randomly by the city beforehand. Within each polling place, ballots are sorted by the choice of 1st place candidate (and then presumably randomly within each such cohort).  The ballots then go through a series of stages.  The first stage is special.

Stage 1: Any candidate who reaches Q votes is declared a winner. Subsequent 1st place votes for them are passed to the next ranked candidate on the ballot who has not already been declared a winner. Ex. if a ballot is reached with x, y, and z as the 1st, 2nd, and 3rd candidates, and both x and y already have been declared winners, it would go to z. If no non-winner choice remains on the ballot, it is swapped with a ballot that already was consumed by the winner and has non-winner choices on it. This minimizes the number of discarded ballots. Note that it always pays for a voter to rank a lot of choices, because otherwise some other voter may have their preference registered instead. It’s not clear from the law what order the 1st place candidates’ ballots should be sorted, but we’ll assume randomly. It does not matter for the anomaly we will discuss. As the sorting proceeds, any candidate with Q votes (by spillover from other candidates or by being 1st on their own) is declared a winner, and any remaining votes for them spill over as described.

Once this process has been completed, almost every ballot has been assigned to some candidate (i.e. either consumed by a winner or spilled over to a remaining candidate). Because of the ballot-swapping mechanism described, it unlikely (but still possible) for ballots to have been discarded due to lack of non-winner alternatives. Each winner has consumed precisely Q ballots, and each remaining candidate has less than Q ballots. In what follows we use “higher-ranked” to refer to the preferred candidates on a ballot. In practice, this means they have been assigned a lower number. I.e., the 1st place candidate on a ballot is “higher-ranked” than the 2nd place candidate.

At this point, any candidate with fewer than M ballots (in our case 50) is declared to have lost. Their ballots are transferred in the same manner as before to the remaining candidates. Note that this form of elimination only takes place in this first round, since the number of ballots assigned to a candidate cannot decrease in subsequent rounds.

Stages 2+: If 9 candidates have been declared winners, the process ends. Otherwise, the trailing candidate is declared to have lost, and their votes are transferred (one by one) to the remaining candidates in the same  manner as before, but with one important change. Unlike in the first round, if no remaining non-winner candidates are listed on a ballot, it is discarded rather than swapped with another. As before, any candidate who reaches Q votes is declared a winner and can accrue no more votes. There are some tie-breaker rules associated with determining who is the trailing candidate at the end of a given round, but we won’t go into those. If at any time, the number of winners plus remaining candidates is 9, all remaining candidates are declared winners. The round ends when every ballot in play either has been spilled over (once) or discarded. Those ballots not discarded or consumed by winners and those candidates not eliminated then proceed to the next round.

Note that a spillover never can result in a ballot being assigned to a higher-ranked candidate. For example, suppose a ballot already has been assigned to the 3rd listed candidate on it. This only could happen if there was a reason to skip the top 2. This means they either already were declared winners or already were eliminated. Nor do any swaps (possible only in the 1st round) affect this. Any subsequent spillovers must go to lower-ranked candidates, or the ballot would have been handed to a higher-ranked candidate already.

Note that unless every voter ranks every candidate, it is possible for some ballots to be discarded. This is highly unlikely in the first round, because swapping is allowed. However, in subsequent rounds ballots may be discarded if they list no candidates which remain in play (i.e. that have not already been declared winners or eliminated). Though there is a theoretical bound on the number of possible discarded ballots, it can be high.

It is quite possible for an insufficient number of winners to be declared. This is no surprise. If every voter lists the same three candidates, but no others, then only three candidates will win. Insufficient ranking by voters can lead to inadequate outcomes.

Unless the field of candidates is reduced below 9 in the first round (i.e. too few candidates meet the 50 vote threshold), there ultimately will be 9 winners. However, some may not get many votes. If every voter ranks every candidate, then all winners will meet quota. If not, some candidates may win without meeting quota by dint of being the last ones uneliminated.

A number of obvious anomalies come to mind. For example, if everyone votes for x,y, and z as the top 3 candidates but there is a huge field of candidates for 4th place — so that each gets 51 spillover votes — then the remaining candidates won’t be eliminated in the first round. The remaining 6 winners then will be selected by the tie-breaker procedure (which we didn’t elaborate on).  Fair yes, desirable no. However, such anomalies can be accounted voter-failures. If each voter ranks the whole field of candidates, they won’t arise.

One important thing to note is that the election method described does not obey the conditions of Arrow’s theorem. The procedure is not even deterministic, and certainly does not satisfy the 3rd fairness condition. It is quite possible for a change in the ranking of candidate z on individual ballots to affect the order of x relative to y in the outcome even if the order of x relative to y is unchanged on those individual ballots. As an extreme example, suppose x is 1st and y is 2nd on 50 ballots and y is 1st and x is 2nd on 50 ballots, and suppose z is 3rd on all of these.   If one of the 1st 50 ballots moves z to the top, x will be eliminated in the 1st round.  If one of the 2nd 50 ballots moves z to the top y will be eliminated in the 1st round.  In neither case did the ranking of x relative to y change on any ballots.  Some anomalies arise for similar reasons to those involved in Arrow’s theorem, but others arise for different reasons.

The Anomaly

Let us now consider the specific anomaly we set out to discuss. Suppose there are 10000 ballots and 9 positions to be filled. We require 1001 votes for a candidate to win, but we’ll call it 1000 to simplify calculation. Suppose that candidate x is ranked 1st on all 10000 ballots, candidate y is ranked 3rd on all 10000 ballots, and 100 other candidates (which we’ll call z1-z100) are ranked 2nd on 100 ballots each.

Everyone agrees that candidates x and y should be on the City Council. They both rank in the top 3 choices for everyone. However, candidate y is eliminated in the first round. All the spillover votes from candidate x go to candidates z1-z100. The number could vary for each, depending on the order in which ballots are processed.  For example, it is possible that each of z1-z100 is assigned 90 spillover votes from candidate x.  It also is possible that z1-z90 would accrue 100 spillover votes each, and the rest would get 0 and be eliminated.

At the end of round 1, x is declared a winner and consumes 1000 votes, y has 0 votes, and z1-z100 each have between 0 and 100 votes.  At least 90 of them have enough to survive the 50 vote test.  However, y is eliminated.  The remaining z’s then proceed through a series of elimination and spillover rounds (with possible tie-breakers for the trailing candidate if needed) until only 8 of the z’s remain. These then are declared winners.

The result is 1 winner everyone wants, 8 winners few people agree on, and the conspicuous loss of the 2nd candidate everyone wants.

This is just one fun example of how well-intentioned voting systems can result in highly-undesirable outcomes.

How to Get a Patent in 2 Easy Steps!

1. Expedited Process: [Note: if your name is not Apple, Google, Microsoft, Sony, or Oracle, skip to step 2]:

Scribble a drawing in crayon on a napkin, write ‘for, you know, stuff’ and drop it off at the Patent Commissioner’s house when you have dinner with him and his wife. On the off-chance it isn’t accepted the next day, be polite but firm. The assigned examiner may be new or overworked. Bear in mind, he is NOT your employee. He serves several other large corporations as well.

By the way, don’t forget that the Patent office is running a special this month: you get every 1000th patent free!

2. Standard Process:

(i) spend several months with a team of lawyers (paid out of pocket) carefully researching the state of the art of your field, fleshing out your idea, researching potentially related patents, and constructing unassailable claims of your own. In the course of this, learn a new language called “legalese,” which bears only a superficial resemblance to English — much as its speakers bear only a superficial resemblance to humans.

(ii) assemble a meticulously crafted and airtight application — one which no sane person can find fault with, because it has no fault.

(iii) get rejected by the examiner, who clearly did a sloppy google search for some keywords. He cites several patents which have nothing in common with yours, except for those keywords.

(iv) reply to said patent examiner, patiently explaining why a simple reliance on keyword similarities is insufficient evidence of prior art, and that modern linguistic scholarship has shown different sentences can have words in common.

(v) receive a reply with “final rejection” emblazoned in huge letters, and in what appears to be blood. An attached notice explains that any further communication regarding this patent will result in a late-night visit by three large fellows with Bronx accents. Your lawyers dismiss this as boilerplate, and explain that “final rejection” actually means “we want more patent fees.”

(vi) battle your way through 50 years and $1,000,0000 of appeals and rejections as the examiner displays an almost inhuman level of ineptitude, an apparent failure to grasp rudimentary logic, infantile communication skills, and an astonishing ability to contradict himself hour to hour.

(vii) Suspect your patent examiner is planning to run for Congress, where his skills would be better appreciated. Encourage him to do so. Maybe his replacement will be better equipped, possessing both neurons and synapses.

(viii) Eventually you reach the end of the process. There has been one of two outcomes:

  • You passed away long ago, and no longer care about the patent.
  • Your application finally was accepted. Because an accepted patent is valid from the original date of application, yours expired decades ago. But this does not matter, since the idea is long obsolete anyway.

Either way, you should feel privileged. You have participated in one of the great institutions of American Democracy!

My new monograph is out!

My new math monograph now is available on and, and soon will be available on other venues via Ingram as well.

Amazon US Paperback
Amazon UK/Europe Paperback

The monograph is an attempt to mathematically codify a notion of “moral systems,” and define a sensible measure of distances between them. It delves into a number of related topics, and proposes mathematical proxies for otherwise vague concepts such as hypocrisy, judgment, world-view, and moral trajectory. In addition to detailed derivation of a number of candidate metrics, it offers several examples, including a concrete distance calculation for a simple system. The framework developed is not confined to the analysis of moral systems, and may find use in a wide variety of applications involving decision systems, black box computation, or conditional probability distributions.

Ken Writes a Film

Scroll down for the link to the movie, and to read my original script.

A few months ago, I participated in a 72 hour film contest with some friends. It was a lot of fun, and we actually filmed in my condo — which was quite a blast. Aside from ducking out of the way whenever necessary, my role was to write the script.

The basic premise was that we had to write a horror film in 72 hours with a certain prop, action, and theme. We were given these at 10 PM on the first night, which meant that I had to slam something out relatively quickly. One interesting aspect was that we didn’t actually know who would be available to act, or even how many. So the screenplay had to be easily adaptable. I drafted two ideas by 11:30ish and discussed them with Brian (the director, and a very talented author in his own right). We picked the more promising one, and honed the general idea. About 30 min later, I delivered to Brian the revised script and we decided to go with that.

Below is a link to the film itself, now publicly available. This definitely was a learning experience, and I have to say the actors (David and Elena) were fantastic to work with. Given that they had so little time (filming had to be finished over a mere 30 hour period, from when they first were handed the script), what they accomplished was incredible. One interesting thing I learned was that phrases which read well on paper are not necessarily ones actors find easy to work with. Unusual turns of phrase are enjoyable in literature, but can be difficult to memorize — especially on short order. I imagine an experienced scriptwriter works closely with actors and has a strong sense of what will be executable and what won’t fly.

The thing which surprised me most was post-production. We had a very talented post-production crew, but I had no idea what to expect. Again, there is a vast difference between what is plausible on paper (or seems easily filmed) and what is workable in post-production. As you can see, the final cut is quite different from the script.

This gave me a more forgiving disposition toward Hollywood writers, and a clear understanding that the words (and scenes) set on paper may differ significantly from what audiences ultimately experience. From now on, I’ll be a bit more hesitant to blame screenwriters for the seemingly inane writing which plagues most Hollywood movies. It very well could be due to a confluence of factors which made it difficult or expensive to adhere to the script. Or maybe some idiot executive meddled, or they polled audience sentiment or some such nonsense. We didn’t have any of that, of course — just lots of talented people working performing their roles. So I think such divergences are inevitable. Sadly, no such excuse exists for novel writers.

I still think having a single screenwriter is the best course, however. Having briefly participated in design by committee (or design by pseudo-autocratic democracy in this case), I think the alternative is far worse. Lots of post-its, a chaos of ideas, and most creativity lost in a homogenization driven by sheer exhaustion and a few strong personalities. Writing is best done by a single writer, with feedback at certain key points from the director. In the 2 hours spent “brainstorming,” a good writer could have pumped out 4 draft ideas, the director could have decided on one or two, and the writer could have finalized them. Too many chefs and all that. Then again, what do I know? If I knew what people actually wanted, I’d be rich.

Without further ado, here is the final cut. Presumably it’s available somewhere on Amazon Prime but I couldn’t find the link, so I’m including the unofficial one a friend provided.

Final cut of “A Teachable Moment”

And here’s my original script (with Brian’s formatting reproduced as best I can given the blog limitations):


The whole thing is dialog, interspersed with small cuts to other scenes (no voiceovers). The cuts should be smooth and for a few seconds each. No sudden flashy stuff.


“I’ve been following your work for some time. The unique impact it has.”


[Smiles ingratiatingly]

“I like to think so. Do you know what makes teaching so special? It’s a distillation of the noblest human activity: sharing.”



“Some would take a more cynical view.”


[quietly regards her for a moment]

“I’ll be honest. I’ve had lots of advantages.”

[he laughs light-heartedly]

“Not everybody has those advantages. Sure, I could feel guilty. But isn’t it better to use my strength for others?

When you share…”

[he tenses in poignancy].

“…you can change a life.”



“I don’t think anyone would dispute this, but *how* you share matters too. Not everyone is ready to believe in pure motives.”.


[wry expression]

“To most people sharing involves a trade: part of themselves for virtue, for the right to imagine themselves a better person. That’s foolish. Sharing is not a transaction. It can ennoble both giver and receiver. A teacher can give without losing.”



“A lot of people don’t understand what teachers really do. I mean day in and day out, over and over.”


“I expect it can be quite difficult. Do you ever get tired?”


[pauses, and gives a cautious laugh]

“I don’t have that luxury. That would be letting down the world in a sense.”



“That sounds a bit grandiose.”



“Yes, I suppose it would to someone not conversant with such matters.”



“You definitely sound like a teacher.”

[looks at him slyly]

“So teach me something.”



[wags his finger and smiles]

“I’ll have to charge you. My wisdom doesn’t come free.”


[grins and suggestively slides her chair right up to him. She’s now close to his face and her body quite close to his]

“I’ll have to find some way to repay you.”



[clears his throat, clearly a bit flustered]

“Very well. I’ll teach you something about teaching. The lessons conveyed through sounds we make are the tiniest fraction of how we teach. It is through subtler manipulations that we imprint our thoughts on the mechanism of this world.”



[whispering, sultry]:

“Well, that’s quite a mouthful. I guess I owe you payment.”


[adjusts collar]:

“N-no need.”


“But I insist. I’ll teach you a lesson as well.”

[she lifts her jacket and flashes a badge.]

M hesitates and seems like he’s about to lunge at her but she puts her hand to her hip and shakes her head, smiling in satisfaction..

M slumps back, and W spreads photos of the various cut-scenes.


“You’re here for me, then?”


“In a sense.”

[she smiles and puts her hand on his]

“I’ve been looking for a good teacher.”

Do’s and Don’ts for Modern Authors

Every author has to post about the secrets to authorial success. Well, I’ve got a different take, a special take, a unique take. I HAVE no authorial success. Which means I’m more intimately familiar with what NOT to do. Who wants advice about how to succeed from somebody who HAS succeeded? That’s silly. Obviously they knew somebody, and they’re NOT going to give you that person’s phone number. But I have no such qualms. In fact, here are a few phone numbers which may belong to movers and shakers:

  • 555-1212
  • 000-0000
  • 90210
  • 314159265358979323846
  • 1

The point is that when none of these are willing to give you the time of day, I will. 7:33 PM.

So, without further ado, here is a list of helpful do’s and dont’s for aspiring authors:

  • Don’t … use big words or complex sentences. That makes you posh, elite, pretentious, and altogether hateful. Who reads big words and complex sentences these days? That’s old fashioned, like you know like last decade. Who wants to be OLD? Besides, why would you want your book to be inaccessible? Big words and complex sentences mean you will target a tiny number of people who mostly read things they’re told to read by the N.Y. Times and won’t like your stuff anyway unless you know somebody AT the N.Y. Times.

  • Don’t … employ subtle ideas or twists or anything complicated to grasp. Such books are for privileged old people, those educated in the dark era before people realized that the purpose of school was fashionable political activism. Just remember: ideas are bad. Most people don’t have any, and it’s rude to flaunt what you have and others don’t.

  • Don’t … proofread, spell-check, or worry about style or grammar. These are wasteful. Proofreading and editing take time. Lots of time. Nobody appreciates them, and they’ll just slow you down. All the best books were written on a phone using two thumbs and very few brain cells. How many artisanal craftsmen do you know? Exactly. If you’re not producing beer, it’s not a craft — it’s a waste of time. Just write as many words as you can as fast as you can. To borrow from the bible (Bumperstickers 3:21, 4): write them all and let god sort it out.

  • Don’t … use characters, plot, or dialog. Creativity is bad. You’ll only increase the chance of offending people. The best way to avoid doing that is by writing solely about yourself, but only if you’re not the type of person inherently offensive to others. There are some handy websites which list acceptable types of people and unacceptable ones.

  • Don’t … worry about pesky things like factual accuracy or consistency. A famous director said that when it’s a choice between drama and consistency, drama wins every time. He’s an idiot, but a rich one. What do you want to be: right or rich? Incidentally, it’s ALWAYS a choice between drama and consistency. If you have time to be consistent, spend it writing more drama instead. Your time is finite — which is a plothole that conveniently can be plugged by reversing the polarity of the Quantum Tachyonic Blockchain.

  • Don’t … advertise or pay anybody for anything. Why pay for nobody to buy your book, when you can get that for free?

  • Don’t … ask friends or family to review your book. Not because it’s against the rules, but because they won’t. Then you’ll have fewer friends and family. Only ask people you don’t like and who don’t like you.

  • Don’t … issue a press release. Nobody will read it, nobody will care. Yet another book tossed on the dung heap of human blather. Yawn. “News” must be something which matters to other people. Like journalists. As everyone knows, modern journalism involves complaining about something which happened to the reporter’s BFF, making it sound like a ubiquitous problem, and quoting lots of tweets. Serious journalists won’t have time for you because they always have a BFF in trouble, and curating tweets is a fulltime job.

  • Don’t … submit to agents, magazines, or contests. If you were the type of person who could get accepted, you would know because you would be published, famous, or well-connected. Since you’re not published, famous, or well-connected, you obviously won’t be accepted. Sure, every now and then somebody new accidentally slips in. It’s an accident resulting from their being related to somebody published, famous, or well-connected.

  • Do … copy whatever is popular at the moment. Book, movie, video-game, comic, or meme — it doesn’t matter. People only read what’s popular, otherwise something else would be popular. As a rich person once said: if you want to be rich do what rich people do. Which is giving bad advice to poor people. See? I’m going to be rich. Well, he actually never said you would be rich, just want to be. Look, people want to reread the same book over and over. It’s easier because they already know the words and nothing scary and unexpected can happen. So why not rewrite those very words and partake of the riches?

  • Do … focus on fanfiction. Being original is time-consuming, hard, and terribly unprofitable. Who wants to engage in some new unknown adventure when they can dwell in the comfortable world they’ve come to know. Not the real one; that’s terribly uncomfortable. But one inhabited by loveable characters they somehow feel a personal connection to, and who can’t get a restraining order against them.

  • Do … pretend to be somebody else. Nobody likes your sort. Whatever you are is offensive in all ways imagineable. Choose a name which represents the group favored by the publishing industry at this moment. Just look at who gets published and who doesn’t. Not established authors, but debut novelists. Nobody’s going to dump Stephen King just because the name Stephen is anathema according to the politics of that week. But they probably won’t publish debut novelist Stephen Timingsucks (unless TimingSucks is native American and native Americans are in that week).

  • Do … know somebody. It’s the only way to get an agent or publisher. If you don’t know anybody, then the best way to meet them is a cold approach. Go to buildings inhabited by agents and publishers, and ride the elevators. That’s why it’s called an “elevator pitch”. When somebody important-looking gets in, stand next to them, sideways, and stare at the side of their head. Remember: it doesn’t matter how the conversation gets started, just where it goes. Which isn’t always jail. All you need is one yes, and it really doesn’t matter how you get it.

  • Do … make it political. Your book should bravely embrace the prevailing political sentiments of the publishing industry. Only then will you be recognized for the courage of conformity. The publishing industry regularly offers awards for just that sort of thing.

  • Do … write about you, you, and you. Far more appealing to readers than plot, style, or substance is your commonplace personal struggle and how you specifically overcame it. Nothing is as compelling as minor adversity subjectively related by the one who experienced it. Be sure to make clear that the reason you prevailed was your unique grit, determination, and moral superiority. Like the dictators of old, you thrice refused the world’s entreaties to tell your story. Only when sufficiently importuned by the earnest pleas of the masses did you relent and accept the mantle of greatness.

  • Do … blog, tweet, instagram, post, and youtube. Who wants to read a book by and about somebody they don’t feel a personal connection with? Have you ever heard the names Tolstoy, Dickens, or Proust? Of course not. They didn’t understand the importance of selling the author, not the work. You need to sell yourself. Literally. While actual Roman-style slavery is illegal in most States, a variety of financial instruments can achieve the same affect.

  • Do … spend the vast majority of your time inhabiting an ecosystem of writers. Your time is far better spent blogging, connecting, and advising other writers rather than writing for the lay person. Sure, outreach is fashionable these days, and it does have a few benefits. But one should not spend too much time demonstrating the writing process through novels, stories, or poetry. Best to focus on publishing for one’s peers.

  • Do … workshop, workshop, and workshop. No writer of note ever succeeded without writing courses, workshops, several professional editors, and an emotional support network. How else could they learn to express themselves in precisely the right manner as discovered by modern researchers and taught only through MFA programs? This is why there’s nothing worth reading from before the 1990s. Fortunately we live in enlightened and egalitarian times, and the advantages of an MFA are available to everybody. Which explains why everybody has one.

  • Do … be chatty, shmoozy, and a massive extrovert who attends conferences, sucks up to agents, and shamelessly promotes yourself. If you’re not that way, make yourself that way. There are plenty of blogs and books by chatty, shmoozy, massive extroverts on how to. These explain in clear and practical terms how you should have been born chatty, shmoozy, and a massive extrovert. If that doesn’t work, there is a simple surgical procedure which can help. It’s called a lobotomy, and also will help you blog, tweet, post, and youtube more effectively. Be your audience.

  • Do … consider tried and true techniques when ordinary submission and marketing methods don’t work. These business methodologies have been refined and proven in many domains over many years. Whole enterprises are dedicated to their successful application, and they can be surprisingly inexpensive. Extortion, kidnapping, blackmail, torture, and politics all can work wonders for your book’s advancement. Pick your poison. Literally. I have an excellent book coming out, filled with recommendations and in which I describe my own struggle to find the right poison and the absolutely brilliant way I overcame this adversity. It’s a very compelling read.

  • Do … show, don’t tell. When somebody talks about the aforementioned tactics you’ve used, make a gruesome example of them. This is showing, so that people don’t tell. Most writing coaches emphasize the importance of “show don’t tell,” and you can find some excellent examples in the work of various drug cartels and the Heads of State of certain current allies and trading partners.

  • Do … kill your babies. This is another mainstay of writing wisdom, and a constant refrain in almost any workshop. It can be difficult, especially the first few times. But if that initial instinct can be overcome, it definitely is something worth trying. While it won’t always help, such sacrifices have been known to curry favor with XchXlotbltyl, the dark god of publication (and a major shareholder in most large publishing houses). Details on the appropriate ceremonies for different genres can be found on popular writing blogs. And don’t worry, you always can produce more babies… and thus more success.

  • Do … remember there’s no need to write the ending first — or ever. There has yet to be born a human with a different ending. But entropy and the inevitable degradation wrought by time rarely appeal to modern audiences. Best to throw in a sappy romantic hookup or hint at an improbable revival of the seemingly dead protagonist. Which brings us to…

  • Don’t … hint. Nobody likes ambiguity. That is why TV is so popular. Books are a very primitive technology, and they require a lot of unnecessary work by the reader. Faces, scenes, even actions need be imagined anew by every reader. This is inefficient. Remember, you’re catering to people who don’t have cable or can’t afford it or are allergic. It’s your job to make their entertainment as painless as possible despite their unfortunate circumstance. Anything else would be ablist. So don’t leave anything ambiguous. Make sure you spell out what just happened, over and over, just in case the first few explanantions didn’t work. Remember the first rule of teaching: Keep the kids’ Chromebook software up to date. Well, the 2nd rule: repeat everything 3 times for the people with no attention span, too stupid, or too distracted to have caught the first 2 times. And don’t forget to give them an achievement award for getting it. So repeat every plot point 3 times, and congratulate the reader on finally getting it.

  • Do … make the reader feel smarter than fictional characters. This is the point of revealing things to the reader that characters don’t know. A well written book will have the reader shouting advice to the characters. Because if your readers aren’t better than a nonexistent and contrived character, who are they better than?

  • Do … publish each sentence as you write it. In the old days, writers had to wait a long time. Agents vetted writers’ works, publishers vetted agents’ submissions, editors vetted accepted works, and copy-editors, proofreaders, and countless others meticulously checked things at every stage. That book of cat jokes filled replete with typos would take several years to see print, not counting the time required to hand-deliver manuscripts by stagecoach or the frequent loss of an editor or writer from dropsy. Thank goodness we live in modern times! These days there’s no need to wait years for feedback or abide by the traditional publication timeline. Your brilliance need not be thwarted by the need for reflection or editing. Each sentence you write should be tweeted, posted on Wattpad, and blogged the moment it appears. When you get feedback, incorporate it all. Otherwise somebody might be sad, and we don’t want anybody to be sad while reading your book. That’s for somebody else’s book, somebody poor and unsuccessful who uses big words and doesn’t know the rules. Besides, as Hollywood has shown, design by committee is the best way to create a quality creative product. Call it the democratization of writing. As recent polls showed, nothing’s better than democracy. In an ideal world, every word would be voted on and accepted or rejected accordingly. One day this utopia may be real, but for now you’ll have to settle for releasing on a sentence-by-sentence basis. At least you’ll have the satisfaction of knowing that your final product was vetted by countless strangers with wildly varying aptitudes, motives, and tastes, rather than a few so-called “professionals” who’ve been doing the same boring thing for years. Do you really want the same old boring people reading your work, let alone editing it?

  • Do … setup a botnet to counter the million of bad ratings your book will get on social media sites. In uncivilized times, negative reviews only came from critics who actually read your book but didn’t understand it or found it differed in some small way from what they thought you should have written but never would bother to write themselves because they’re too busy writing negative reviews. That was a slow process. We all know how long it took for Mozart to get meaningful feedback like “too many notes,” and how much his craft improved as a result. Imagine what he could have composed if he learned this earlier! These days we’re much more fortunate. One needn’t wait months or years for a hostile stranger with adverse incentives to read your book and pan it. There are millions of hostile strangers with adverse incentives willing to do so without troubling to read it. This is much more efficient, and we have modern social media to thank for rewarding such behavior with improved social standing. Otherwise, you’d have to wait for some “reputable” critic to actually read your literary novel and comment on it. Instead you’ll generously receive feedback from somebody far more credible who only reads young adult coming-of-age novels about pandas but is willing to step out of their comfort zone and negatively rate your book without having read it. You’re welcome.

  • Don’t … have any faith in humanity. If you did you won’t for long. But you didn’t or you wouldn’t be a writer in the first place. Who but from malice would wish to imprint their thoughts on the world. Or ask of another that they occupy the liminal time between nonexistence and nonexistence with a less poetic, less subtle, and less profound rehash of the same tired ideas. You are, after all, asking people to share your delusion of eloquence. That’s almost like founding a cult. Which incidentally, is an excellent way to promote your book.

  • Do .. buy my book. It won’t make you happy, but you can’t buy happiness so you may as well spend your money on this.


Why NOT to use Amazon Ads for your book

In today’s article, I ask a simple question: does it pay to advertise on amazon for your book? As can be guessed by the exceedingly astute from the title of the post, the answer is no. In addition to explaining how I came to this conclusion, I also will offer a brief review of the basics of Amazon’s online advertising.

I’ll examine the matter purely in terms of tangible cost/benefit, and ignore considerations involving ease of use, time wasted dealing with Amazon’s bureaucracy, and the myriad other intangible costs involved.

First let’s review some of the aspects of Indie publishing which relate to Amazon’s author ad campaigns, as well as how those campaigns work.

Quick Review of Some Relevant Aspects of Indie Publishing

Printer vs Distributor

In general, to sell something on Amazon you need to be designated a “seller” and sign up for a seller account. This can be nontrivial, and at various times Amazon has made it well-nigh impossible to do so. Authors have a special in, however, but only if they publish a version of their book via Amazon. This means producing a Kindle edition and/or (more recently) printing through KDP (formerly Createspace).

When an author publishes a print edition via some other service, one of two things happen, depending on the type of service. Either that service also offers distribution (Ingram Spark/Lightning Source) or it does not (everybody else). There are two major distributors: Ingram-Spark/Lightning-Source and Baker-Taylor. Of these, only Ingram offers Print-on-demand (POD) services to authors. All other POD services (with the exception of Amazon’s own Createspace, now part of KDP) only offer POD.

An ordinary POD company sells books to the author/publisher who then may sell them to bookstores, individuals, etc. The author/publisher is responsible for storage, mailing, returns, invoice management, etc. Ingram, on the other hand, has a catalog that is available to all bookstores and automatically is pushed to them regularly. When you POD through Ingram, your book appears in their catalog — and thus quickly is available for order through almost every bookstore. This doesn’t mean it will appear on their shelves, but an individual who wishes to purchase a copy need only walk into a bookstore and ask to order one. In theory, the author/published need never handle a physical copy of the book!

Why does this matter? It affects how you are treated by Amazon.

Author vs Seller

As far as Ingram is concerned, Amazon is just another bookstore. It too automatically slurps in your entry from Ingram’s catalog. Unlike a physical bookstore it offers it for sale just like it does any other book. A “book page” is created for it (and an “author page” may be created for the author), based on a cover image, blurb, etc, obtained from Ingram. Your book will appear and be treated like any other and show as being fulfilled by Amazon itself. Let’s refer to this as “Amazon proper”. Incidentally, Barnes and Noble will do exactly the same thing online.

Amazon also hosts a seller marketplace (AMS), which includes, among many other vendors, lots of 3rd-party online bookstores. These each slurp in that same info and may offer your book for sale as well, often at a slight discount which they make up for through inflated shipping costs. It’s not uncommon for a new author to see their book appear for sale through myriad sellers immediately after launch and assume those are illicit review copies being resold. They’re not. These just are from mini-bookstore fronts which regurgitate the Ingram catalog. When someone orders from them, the order is relayed to Ingram which then fulfills it. Ditto for an order through Amazon Proper. It’s worth noting that Ingram has special shipping arrangements with Amazon, B&N, etc, and orders from these stores will be prioritized. While it may take 2-4 weeks for an order by the author/publisher themselves to be fulfilled, orders from Amazon or B&N are quickly dispatched.

The information which appears on the Amazon page for a print book is obtained from the Ingram info. They do allow you to declare yourself an author and “claim” books, setup an author page, etc. Almost all authors put out a Kindle version of their book through KDP. In fact, most only do this. Amazon generally attaches this to any existing print page within a week or two. A few emails may be needed to make sure they associate the same author with both, etc, but generally it’s pretty smooth (as far as Amazon processes go).

Independent of whether you are an author or publisher, you may setup a store-front on Amazon. Some publishers do this. In this case, you must register as a seller, setup tax info, etc. In theory, you could sell anything, not just your book. The seller can control the descriptions of products they sell, etc. But authors generally need not go to such lengths — as long as they are using KDP for at least one of their version.

Why all this rigamarole? There is one area where it makes a big difference. Only sellers can run Amazon ad campaigns. If you only have a print edition which has been slurped in, you cannot run an ad campaign. You would have to create a seller store-front, sell the book through that, and then run a campaign as that seller and only for the things sold on that store-front. You couldn’t draw generic traffic to your book on Amazon proper.

There is a trick, however. As mentioned, authors are viewed as an automatic type of seller — but only if they have a version of their book published through Amazon. If you’ve published a Kindle version of the book, then you qualify. In principle, the ads only would be for that version. But since Amazon links all versions of the book on a single page, de facto it is for all of them. No seller account is needed. This is how most author ad campaigns are run.

On a practical note, Amazon used to distinguish author ad campaigns from others, offering tools which were more useful. Recently, they lumped them in with all other sellers, making practical management of ad campaigns much more challenging. Most sellers of any size use the API or 3rd party firms to manage their ad campaigns, but as a single author you will be forced to use Amazon’s own Really Awful Web Interface. Hmm… they should trademark that. Because it describes SO many of their web interfaces. But, that’s not what this article is about. Let’s assume it was the easiest to use interface in the world, a pleasure on par with the greatest of epicurean delights. Is it worth doing?

Before answering that (well, we already answered it in the title, but before explaining that answer), let’s summarize the levels of complexity in managing sales/ads through Amazon:

1. Easiest. Fulfillment via Amazon and can run ad campaign via Amazon as is:

  • Kindle edition, no POD
  • Kindle edition, POD via Amazon KDP
  • No Kindle edition, POD via Amazon KDP
  • Kindle edition, POD via Ingram

2. Some effort. Fulfillment via Amazon but need a seller account to run an ad campaign

  • Kindle edition, POD via somebody other than Amazon or Ingram
  • No Kindle edition, POD via Ingram

3. Messiest. Seller account needed to sell at all

  • No Kindle edition, POD via someone other than Amazon or Ingram

Types of Ad Campaigns

Next, let’s review the types of Amazon ad campaigns. There currently are three types. A given author may run many separate ad campaigns for the same book — but each will be of one and only one type.

1. Sponsored Product Targeting: These ads are in the row of “sponsored products” which appears when you view the relevant product’s Amazon page. In principle you give Amazon a list of specific books, similar in theme or style or subject matter or whose readers are likely to be interested in your own. In practice, you have to be even more specific. You give Amazon a list of “products”, defined by ASINs. There may be many editions or versions of the same book. You’ve got to include ’em all. By hand. Without any helpful “select all” tool. And remember them. Because all you’ll see once your ad campaign is running is a breakdown by ASIN.

2. Keyword Targeting: These ads appear in searches. There are 3 locations they may be placed: the top 2 spots, the middle 2 spots, or the last 2. Each page of results has ads in one or more of these locations, and they’re designated “sponsored”. Try a few searches, and you’ll see the placement. You give Amazon a list of keywords, generally two or more word phrases, and select how specific a match is required for each (exact, containing it, or broadly related). Then your ad will appear in the results when someone searches for those phrases on amazon. Keyword targeting allows negative keywords as well. For example, it may be a good idea to negate words such as “dvd”, “video”, “audio”, etc, especially if the most popular entries are in those categories. Search for your keyword, see what comes up, and negate any undesirable groups that appear toward the top (using -foo in your search). When you’ve negated the relevant keywords, the top entries should be precisely what you’d like to target.

3. Category Targeting: You pick the Amazon categories that best suit your book — and presumably the book appears when somebody clicks the category. My experience is that category targeting is well-nigh useless for authors, and generates very few imprints or clicks. So we’ll ignore it.

Ok, one more piece of review and then we’ll get to the analysis.

How Amazon Ads Work

Although their locations and types differ may differ, all ads are placed via the same process: an auction. In fact, pretty much any ad you see anywhere online has been chosen via an almost identical process.

Every time a web page is served to a user (ex. you browse to a particular product), there are designated slots for ads. This is true of almost any webpage you view anywhere — all that differs is who is selling the ad space. Those slots are termed “impressions” (or more precisely, the placement of an ad in one is called an “impression”). Think of them as very short-lived billboards. To determine which ad is shown, an auction is conducted for each. This all is done very quickly behind the scenes. Well, not *so* quickly. Guess why webpages are so slow to load…


Because of its ubiquity, the auction process is fairly standard by this point. What I describe here holds for most major sites which sell advertising. The auction used by almost everybody is called a “second price auction”. In such an auction, the highest bid wins but only pays the 2nd highest bid. Mathematically, this can be shown to lead to certain desirable behaviors. Specifically, it is optimal for each participant to simply bid their maximum instead of trying to game things. This is important because Amazon will be given a maximum bid by you, and can only act as your proxy if it has a well-defined strategy for using it. Since it’s also acting as everyone else’s proxy, such a strategy must be a truthful one.

[As an aside, what I described technically is called a Vickrey auction. Online services use a generalized version of this in which multiple slots are auctioned at once in order of quality. I.e., all the impressions on a page are auctioned simultaneously to the same bidders. The highest bidder gets the best impression, but pays the 2nd highest bid. The 2nd highest bidder gets the 2nd best impression but pays the 3rd highest bid, etc.]

If you bid 1 and the 2nd highest bid is 0.10, you win and only pay 0.10. So, if you′re a lone risk−taker in a sea of timidity, it pays to bid high. You′ll always win, but you won′t pay much. However if there′s even one other participant with a similar strategy, you may end up paying quite a bit. If both of you bid high, one of you will win, and will pay a lot. For example, if you bid 1 and the other guy bids 0.99, you′ll pay 0.99.

So far, we’ve discussed the second price auction in the abstract. It’s straightforward enough, even if the optimal strategy may require a little thought. The more interesting issue is what precisely you’re bidding on.

In an ad auction, you are *not* bidding on the impression per se. Rather, you effectively are bidding on an option on the impression. Let me explain.

Once every impression on the given web page has been auctioned, the winning ads are displayed. However, the winner of an impression only pays if the user clicks on their ad, regardless of what happens afterwards. To summarize:

  • Win impression, no click: Cost= 0
  • Win impression, click, sale: Cost= 2nd highest bid
  • Win impression, click, no sale: Cost= 2nd highest bid

Amazon gets paid only if your ad is clicked on. If you win a million impression auctions and nobody clicks on your ad, you pay nothing. If every impression you win gets clicked on but nobody buys anything, you pay for all those impressions. In terms of what you pay Amazon, sales mean nothing, impressions mean nothing, only clicks count. But impressions are what you bid on. Financially, this tracks more closely the behavior of an option than a commodity.


Obviously, the bid placement process is automated, so you’re not in direct control of the bidding in each auction. In essence, Amazon acts as your proxy in this regard. We’ll get to how your bids are placed shortly, but first let’s review some terminology.

  • Impression: We already encountered this. It is placement of an ad in a particular slot on a particular web page that is served. It is important to note that this refers to placement one time for one user. If the user refreshes the same page or another user visits it, a fresh auction is conducted.
  • Click-through-rate (CTR): The average fraction of impressions that get clicked on. The context determines precisely which CTR we’re talking about.
  • Conversion Rate: A “conversion” is an instance of the end goal being accomplished. In this case, that end goal is a sale (or order). The “Conversion Rate” is the average fraction of clicks that result in sales.
  • Conversions per Impression (CPI): The average fraction of impressions that result in sales. This is just the CTR * Conversion-Rate.
  • Order vs Sale: For most purposes these are the same. For products which may be bought in bulk, the two may differ (ex. 100 boxes of soap could be 1 order but 100 sales). But this rarely applies to books since customers generally buy only one.
  • Cost Per Click (CPC): The average cost of each click. Basically, the average 2nd highest bid in all auctions won by you and for which a click resulted.
  • Average Cost of Sales (ACOS): Each click may cost a different amount, so this measures the average actual cost of each sale, usually stated as a % of sale price. A 200% ACOS for a 10bookmeansthatitcosts20 of advertising on average to make one sale. The ACOS is the CPC/Conversion-Rate.

Bid Placement

I mentioned that an auction is conducted for each impression, and that it is done very quickly (in theory). If that’s the case, who are the bidders and how are the bids placed?

The pool of potential bidders includes every active ad campaign which hasn’t run out of money that day. This pool is narrowed by the specified ad campaign criteria (product targets, keywords, negative keywords, category, etc). The result is a pool of bidders for the specific auction. In our case, these generally would be authors or publishers — but in principle could be anyone.

Amazon acts as the proxy for all the participants. It determines which ad campaigns should participate in a given auction and it bids based on their instructions. Other than this, it has no discretion.

As a bidder, you have control of the following (for a given ad campaign):

  • Campaign type: product, keyword, or category.
  • A list of products, keywords, negative keywords, and/or categories as appropriate for the campaign type.
  • For each keyword, product, or category, a maximum bid.
  • A “daily” budget. I’ll explain why this is in quotes shortly.
  • Ad text. You can’t control the image (it’s your book cover), but some text can be provided.

Putting aside the campaign type and ad text itself, the salient point is that there is a list of “items” (keywords, products, or categories) which each have a maximum bid specified. There also is an overall daily budget.

It turns out that the “daily” budget isn’t really “daily.” Amazon operates on a monthly cycle, and assigns a monthly budget based on the number of days and the daily budget. On any given day, the daily budget can be exceeeded, though generally not by some huge amount. If Amazon does exceed your monthly budget (which can happen) it will refund the difference. I’ve had this happen. The point is that you’re not really setting a daily budget but a rough guideline. It’s the associated monthly budget which is used.

Once you exceed the budget constraint, that campaign is inactive until the next day (or month, depending on which budget has been exceeded). Obviously, that makes bidding relatively simple — there is none. So let’s assume the budget hasn’t been breached.

For each auction, Amazon must determine whether any of the items in the campaign are a match. It then applies the specified maximum bid for that item. In principle. But nothing’s ever that simple, is it?

Bid Adjustment

By this point you may have noticed a major problem with the auction system as described. Let’s look at is from a transactional standpoint.

You earn revenue through sales, but pay for clicks. The resource you have is money (your budget for advertising) and you need to trade it for sales revenue.

Amazon earns revenue through clicks, but pays in impressions. What do I mean by this? The resource Amazon has is impressions, and they need to trade it for click revenue.

Any scenario that results in lots of clicks per sale (or more precisely, a high ACOS) is detrimental to you. You wish to minimize ACOS. Otherwise, it will cost a lot of ad-money per sale, and that money presumably would have been better spent on other approaches.

Similarly, any scenario which results in lots of impressions per click is detrimental to Amazon. If those impressions had been won by more effective sellers, then people would have clicked on them and Amazon would have been paid.

As an extreme example, suppose Bob’s Offensive Overpriced Craporrium wins every auction on Amazon. Then Amazon will make no money from its ad business. On the other hand, if Sue’s Supertrendy Awesomorrium won, then through hypnosis, telepathy, and blackmail every single user would be compelled to click. This is great for Amazon.

The problem is that you have control over your ad and, in broad strokes, the types of impressions you bid on. But what control does Amazon have? Other than heavy-handed tactics like throwing Bob off the platform, it would seem to have little means of preventing such losses. Obviously, this isn’t the case. Otherwise, how could Jeff Bezos afford a $35 Billion divorce? Amazon actually has 2 powerful tools. It is important to know about these, since you’ll probably perform like Bob when you first start advertising.

First, Amazon has an algorithm which selects which impressions are a good match for you. Sometimes they can tune this based on performance. Amazon has no control when it comes to product-targeting. If you said: sign me up for auctions involving ASIN X, Amazon dutifully will do so. However, for other approaches such as keyword or category targeting, they have discretion and can play games. Bob quickly may find that he somehow isn’t a good fit for anything but books on bankruptcy.

Second, Amazon can reduce your effective bid. In theory, they will bid your stated maximum for the item in question. However, they may throttle this based on performance. Even if your maximum is 3,youmayendupbidding2. It’s unclear whether this affects the amount you (or the other winner) pays upon winning (if a click results), but it probably does. Conducting an auction under other auspices would be very difficult. So, you may end up losing even if the 2nd highest bidder isn’t as high as your maximum.

Ok, now that we have the background material out of the way, let’s get down to brass tacks. Or iron tacks. Brass is expensive.

Why it doesn’t pay to advertise

Now that we’ve reviewed the practice of advertising, let’s look at whether any of this is worth it. Specifically, what would it take to be profitable?

Let us suppose that our book sells for G,ofwhichwekeepP. For example, a 10bookmayyield2.50 in net revenue for an author (where “net” means net of print costs, Amazon’s cut, etc, not net of advertising costs). In practice, things are a bit more complicated because there may be different P and G’s for the print and Kindle editions. For simplicity, let’s assume a single one for now.

Before getting to the formal calculation, let’s look at a real example. Here are some numbers from an ad campaign I ran for my first book, “The Man Who Stands in Line.” I didn’t take it too seriously, because the book is not in a genre most people read. But I viewed the process as a good trial run before my novel (now out) “PACE

Here are the stats. The campaign ran for a little over a year.

  • Impressions: 1,716,984
  • Total sales: $423.99
  • Total ad costs: $931.56
  • CPC: $0.48
  • CTR: 0.11%
  • Total Sales: 77
  • ACOS: 219.7%

While my book didn’t break any records, it did furnish some useful data. Let’s look at these numbers more closely.

On its surface, the ACOS doesn’t look too terrible. After all, I paid a little over twice the amount I made — right? Not quite. I paid a little over twice my gross revenue. The problem is that I only care about net revenue.

As an extreme example, suppose I have two books A and B. Both yield me 2 net revenue per sale as the author, but A costs 1000 and B costs 4. Now suppose I have a pretty darned good ACOS of 50% on 1000 worth of sales. In both scenarios I’ve paid 500 in advertising costs. But in scenario A, I′ve made 2 net revenue. I.e., I have a net loss of 498. In scenario B, I′ve made 500 net revenue and have broken even.

We immediately see 2 things:

  • The same ACOS can correspond to vastly different net revenues depending on the retail price of the book.
  • It’s really hard to advertise at a profit.

Returning to my own book, the first problem in analyzing the numbers if that we can’t easily determine net revenue. There were two book formats. The book was available for 9.99 as a paperback (resulting in net revenue of around 2.50) and a 2.99 Kindle edition (net revenue about 2). Fortunately, the two net revenues per book are close. From the total sales, we can guess a net revenue between $150-200. That paints a much more dismal picture than the ACOS implies.

Let’s next consider a more typical books, and figure out the numbers needed to make advertising profitable. Because the ratio of net to gross revenue per sale will be highest for the Kindle edition, let’s solely focus on that. Any print editions will have even worse ad costs.

A CPC of 0.50 for books is fairly typical from what I′ve seen. Suppose you have a Kindle book priced at 4.99. With the 70% royalty rate (and no large file fees to speak of), you’d make a little under 3.50 per sale. But let′s be liberal. Let′s say your net profit is 4 per book.

As mentioned, ACOS is deceptive. If you have an ACOS of 1, it looks like you’re breaking even. You’re not. It means your gross sales are breaking even. Your net revenue is negative. But it’s much much worse than that if you have a print book. Your net profit may be the same across formats but the gross revenue isn’t. The higher the price of the book and the lower the ratio of net to gross revenue per sale, the more unrepresentative the ACOS becomes.

With the numbers we proposed, we must average 8 or fewer clicks per sale to remain in the black. Otherwise our net revenue for the sale is less than the advertising cost. That is a very optimistic number. Even the most precisely targeted advertising rarely sees such a rate. And that’s just to break even.

Returning to my own book, what sort of ACOS would be required to break even? With the print edition, we would need a 25% ACOS. With the Kindle edition it would be closed to 66%. In my own case I would have required around a 5x lower ACOS than achieved. But that’s just to break even! Presumably we want to do better. The point of advertising isn’t just to break even. In essence, I would need an unattainable ACOS and Conversion rate for advertising to pay off.

From these numbers, it’s clear that advertising on Amazon simply can’t pay off for Indie authors. From an economic standpoint it always will operate at a loss.

But are there any other reasons to advertise?

I’ve heard claims that the real purpose of such ads is exposure, that one nominal sale translates into many through word of mouth, etc, I’ve seen no evidence of this. It may happen, but the scale is very small.

Another argument I’ve encountered is that impressions matter. Having lots of impressions may not translate into immediate sales but it raises awareness. The more times people see a book, the more “validated” it becomes in their mind. Presumably this translates into later sales which can’t be tracked as direct clicks. This is good for the author, since it means sales without any associated click-cost. Unfortunately, I’ve seen no evidence of this either. My real sales closely tracked the 77 listed; there weren’t all sorts of separate ones which didn’t appear as clicks or conversions. True, this wasn’t the world’s most marketable book. But if 1.7MM imprints make no difference then it’s too expensive to reach whatever number would.

My sense is that one or both claims may be true for large, well-known publishers running huge campaigns, and where a friend’s recommendation of a recognizable title tips the scale. But that requires a critical mass and multi-faceted marketing strategy, and way more money than a typical indie author will care to invest.

Like most services associated with indie publishing — agent readings at conferences, query review, marketing and publicity, books on marketing and publicity, etc — Amazon ads is just another piece of a machine designed to separate the naive from their money using the oldest of human failings: hope.

So how should you sell books? If I knew, I’d spend my days basking in luxury and fending off rabid fans rather than writing snarky posts which nobody will read. But until that happens, I’ll keep you posted on the things I try. The simple answer may be the one you don’t want to hear: you don’t. You write if you have the inclination and means to do so, but you should have no expectation of being able to sell your book. If you wish to get people to read it, you may do so at a loss via Amazon ads. But there probably are much more effective ways to pay for readers.

Some Pet Peeves of a Grammar Snob

Language evolves organically, and only a fool would expect the world to remain the same just to accommodate their own inability to move past the life knowledge they happened to acquire during their particular formative years.

But I’m a fool and proud of it. Or more precisely, I’m selective in my folly. I choose to accept changes which arise organically in a sense which meets my arbitrary standards, but have nothing but disdain for those changes effected through the apparent illiteracy and incompetence of celebrities (also known as “influencers”). To me, it’s like corporate-speak but dumber. And that’s saying a lot.

Put in simpler and less pompous terms for those of you who don’t understand big words: if some Hollywood moron screwed up and a bunch of jokers adopted the meme, that’s not “organic” growth of language — it’s a Hollywood moron screwing up and a bunch of jokers adopting the meme. None of these people should be allowed near the language, let alone given power to influence it. As far as I’m concerned, there should be a license required. And since you need a language license to take the written test in the first place, nobody could get one. But that’s ok. The language can’t change if nobody uses it.

So, without further ado (well, there wasn’t really much ado so far, just a lot of whining), here are a few of my favorite things (sung to the dulcet strains of an NWA song):

  1. Same Difference: A difference requires two objects for comparison. To be the same, two differences involve at least 3 objects (and possibly 4) and two comparisons. For example: I’m pedantic and pompous. Same thing (well, not really, but we’ll allow it). I’m pedantic and pompous, and he’s pretentious and self-important. Same difference (well, not really, but a sight better than before). Same thing: 2 items, 1 comparison. Same difference: 3-4 items, 2 comparisons.

  2. Pay the consequences: You pay a penalty or a price. You suffer consequences. I hope that the idiot who birthed this does all three.

  3. Associated to: This one requires a delicate touch. It’s a mistake by my favorite people: mathematicians. And they have oh-so-fragile egos. Sadly, I can’t blame the arch-media-corporate hegemony which secretly controls our brains through alien ultra-quantum-fractal-catchwords. Not that I would anyway. I’m not sure where “associated to” started, but I have an irresistible urge to jump up and scream whenever somebody says it. And since most math articles, books, and even wikipedia articles seem to have adopted it, I basically spend all day standing up and screaming. Which is no different than before, but now I have a plausible explanation when cops, social workers, and concerned-looking parents inquire. I thought of writing an automatic script to change every occurrence in wikipedia, but decided I was too lazy. Besides, every article has a little gatekeeper associated to it who guards it and tends it and flames anybody who tries to change anything. I did read a possible explanation for the phenomenon, however (the “associated to”, not the little folk guarding wikipedia pages). In latinate languages such as Italian, “associare” takes “a” as its preposition, which naively translates to “to” in English. I suspect this is indeed the source, not because I have any knowledge beyond what I read but because of what it would mean if it weren’t true. The only other plausible explanation is that Gonklaxu the Dissatisfier has penetrated the barrier to our galaxy and is sowing discord amongst the mathematicians who pose the greatest threat to his 12-dimensional nonorientable being. Since mathematicians apparently don’t read anything but math books, that strategy would be singularly successful. The thought of Gonklaxu does keep me awake at night, I’ll admit. Because if he is invading, it means he didn’t stop emailing because he was banished to a nonmeasurable corner of the duoverse. Rejection hurts so much. I associate it to the pain of hearing associate to.

I’m sure I’ll think of a few more soon, so stay tuned!

180 Women and Sun Tzu

It is related that Sun Tzu (the elder) of Ch’i was granted an audience with Ho Lu, the King Of Wu, after writing for him a modest monograph which later came to be known as The Art of War. A mere scholar until then (or as much of a theorist as one could be in those volatile times), Sun Tzu clearly aspired to military command.

During the interview, Ho Lu asked whether he could put into practice the military principles he expounded — but using women. Sun Tzu agreed to the test, and 180 palace ladies were summoned. These were divided by him into two companies, with one of the King’s favorite concubines given the command of each.

Sun Tzu ordered his new army to perform a right turn in unison, but was met with a chorus of giggles. He then explained that, “If words of command are not clear and distinct, if orders are not thoroughly understood, then the general is to blame.” He repeated the order, now with a left turn, and the result was the same. He now announced that, “If words of command are not clear and distinct, if orders are not thoroughly understood, then the general is to blame. But if his orders are clear, and the soldiers nevertheless disobey, then it is the fault of their officers,” and promptly ordered the two concubines beheaded.

At this point, Ho Lu intervened and sent down an order to spare the concubines for he would be bereft by their deaths. Sun Tzu replied that, “Having once received His Majesty’s commission to be the general of his forces, there are certain commands of His Majesty which, acting in that capacity, I am unable to accept.” He went ahead and beheaded the two women, promoting others to fill their commands. Subsequent orders were obeyed instantly and silently by the army of women.

Ho Lu was despondent and showed no further interest in the proceedings, for which Sun Tzu rebuked him as a man of words and not deeds. Later he was commissioned a real general by Ho Lu, proceeded to embark on a brilliant campaign of conquest, and is described as eventually “sharing in the might of the king.”

This is a particularly bewildering if unpleasant episode. Putting aside any impression the story may make on modern sensibilities, there are some glaring incongruities. What makes it more indecipherable still is that this is the only reputable tale of Sun Tzu the elder. Apart from this and the words in his book, we know nothing of the man, and therefore cannot place the event in any meaningful context. Let us suppose the specifics of the story are true, and leave speculation on that account to historians. The episode itself raises some very interesting questions about both Sun Tzu and Ho Lu.

It is clear that Sun Tzu knew he would have to execute the King’s two favorite concubines. The only question is whether he knew this before he set out for the interview or only when he acceded to the King’s request. Though according to the tale it was Ho Lu who proposed a drill with the palace women, Sun Tzu must have understood he would have to kill not just two women but these specific women.

Let’s address the broader case first. It was not only natural but inevitable that court ladies would respond to such a summons in precisely the manner they did. Even if we ignore the security they certainly felt in their rank and the affections of the King, the culture demanded it. Earnest participation in such a drill would be deemed unladylike. It would be unfair to think the court ladies silly or foolish. It is reasonable to assume that in their own domain of activity they exhibited the same range of competence and expertise as men did in martial affairs. But their lives were governed by ceremony, and many behaviours were proscribed. There could be no doubt they would view the proceedings as a game and nothing more. Even if they wished to, they could not engage in a serious military drill and behave like men without inviting quiet censure. The penetrating Sun Tzu could not have been unaware of this.

Thus he knew that the commanders would be executed. He may not have entered the King’s presence expecting to kill innocent women, but he clearly was prepared to do so once Ho Lu made his proposal. In fact, Sun Tzu had little choice at that point. Even if the King’s proposal was intended in jest, he still would be judged by the result. Any appearance of frivolity belied the critical proof demanded of him. Sun Tzu’s own fate was in the balance. He would not have been killed, but he likely would have been dismissed, disgraced, and his ambitions irredeemably undermined.

Though the story makes the proposal sound like the whimsical fancy of a King, it very well could have been a considered attempt to dismiss a noisome applicant. Simply refusing an audience could have been impolitic. The man’s connections or family rank may have demanded suitable consideration, or perhaps the king wished to maintain the appearance of munificence. Either way, it is plausible that he deliberately set Sun Tzu an impossible task to be rid of him without the drawbacks of a refusal. The King may not have known what manner of man he dealt with, simply assuming he would be deterred once he encountered the palace ladies.

Or he may have intended it as a true test. One of the central themes of Chinese literature is that the monarch’s will is inviolable. Injustice or folly arises not from a failing in the King but from venal advisers who hide the truth and misguide him. A dutiful subject seeks not to censure or overthrow, but rather remove the putrescence which clouds the King’s judgment with falsehood, and install wise and virtuous advisers. Put simply, the nature of royalty is virtuous but it is bound by the veil of mortality, and thus can be deceived. One consequence of this is that disobedience is a sin, even in service of justice. Any command must be obeyed, however impossible. This is no different from Greek mythology and its treatment of the gods. There, the impossible tasks often only could be accomplished with magical assistance. In Sun Tzu’s case, no magic was needed. Only the will to murder two great ladies.

As for the choice of women to execute, it does not matter whether the King or Sun Tzu chose the disposition of troops and commands. The moment Sun Tzu agreed to the proposal, he knew not only that he would have to execute women but which ones. Since he chose, this decision was made directly. But even if it had been left to the king, there could be no question who would be placed in command and thus executed.

The palace hierarchy was very strict. While the ladies probably weren’t the violent rivals oft depicted in fiction, proximity to the King — or, more precisely, place in his affections, particularly as secured by production of a potential heir — lent rank. No doubt there also was a system of seniority based on age and family, among the women, many of whom probably were neither concubines nor courtesans, but noble-women whose husbands served the King. It was common for ladies to advance their husbands’ (and their own) fortunes through friendship with the King’s concubines. Whatever the precise composition of the group, a strict pecking order existed. At the top of this order were the King’s favorites. There could be no other choice consistent with domestic accord and the rules of precedence. Those two favorite concubines were the only possible commanders of the companies.

To make matters worse, those concubines may already have produced heirs. Possibly they were with child at that very moment. This too must have been clear to Sun Tzu. Thus he knew that he must kill the two most beloved of the King’s concubines, among the most respected and noblest ladies in the land, and possibly the mothers of his children. Sun Tzu even knew he may be aborting potential heirs to the throne. All this is clear as day, and it is impossible to imagine that the man who wrote the Art of War would not immediately discern it.

But there is something even more perplexing in the story. The King did not stop the executions. Though the entire affair took place in his own palace, he did not order his men to intervene, or even belay Sun Tzu’s order. He did not have Sun Tzu arrested, expelled, or executed. Nor did he after the fact. Ho Lu simply lamented his loss, and later hired the man who had effected it.

There are several explanations that come to mind. The simplest is that he indeed was a man of words and not deeds, cowed by the sheer impetuosity of the man before him. However, subsequent events do not support this. Such a man would not engage in aggressive wars of conquest against his neighbors, nor hire the very general who had humiliated and aggrieved him so. Perhaps he feared that Sun Tzu would serve another, turning that prodigious talent against Wu. It would be an understandable concern for a weak ruler who dreaded meeting such a man on the battlefield. But it also was a concern which easily could have been addressed by executing him on the spot. The temperamental Kings of fable certainly would have. Nor did Ho Lu appear to merely dissemble, only to visit some terrible vengeance on the man at a later date. Sun Tzu eventually became his most trusted adviser, described as nearly coequal in power.

It is possible that Ho Lu lacked the power oft conflated with regality, and less commonly attendant upon it. The title of King at the time meant something very different from modern popular imaginings. The event in question took place around 500 BC, well before Qin Shi Huang unified China — briefly — with his final conquest of Qi in 221 BC. In Ho Lu’s time, kingdoms were akin to city-states, and the Kings little more than feudal barons. As in most historical treatises, troop numbers were vastly exaggerated, and 100,000 troops probably translated to a real army of mere thousands.

This said, it seems exceedingly improbable that Ho Lu lacked even the semblance of authority in his own palace. Surely he could execute or countermand Sun Tzu. Nor would there be loss of face in doing so, as the entire exercise could be cast as farcical. Who would object to a King stopping a madman who wanted to murder palace concubines? If Sun Tzu was from a prominent family or widely regarded in his own right (which there is no evidence for), harming him would not have been without consequence. But there is a large difference between executing the man and allowing him to have his way in such a matter. Ho Lu certainly could have dismissed Sun Tzu or proposed a more suitable test using peasants or real soldiers. To imagine that a king would allow his favorite concubines to be executed, contenting himself with a feeble protest, is ludicrous. Nor was Sun Tzu at that point a formidable military figure. A renowned strategist would not have troubled to write an entire treatise just to impress a single potential patron. That is not the action of a man who holds the balance of power.

The conclusion we must draw is that the “favorite concubines” were quite dispensible, and the King’s protest simply the form demanded by propriety. He hardly could not protest the murder of two palace ladies. Most likely, he used Sun Tzu to rid himself of two problems. At the very least, he showed a marked lack of concern for the well-being of his concubines. We can safely assume that his meat and drink did not lose their savour, as he envisioned in his tepid missive before watching Sun Tzu behead the women.

While it is quite possible that he believed Sun Tzu was just making a point and would stop short of the actual execution, this too seems unlikely. The man had just refused a direct order from the King, and unless the entire matter was a tremendous miscommunication there could be little doubt he would not be restrained.

Ho Lu may genuinely have been curious to see the outcome. Even he probably could not command obedience from the palace ladies, and he may have wished to see what Sun Tzu could accomplish. But more than this, the King probably felt Sun Tzu was a valuable potential asset. The matter then takes on a very different aspect.

From this viewpoint, Ho Lu was not the fool he seemed. The test was proposed not in jest, but in deadly earnest, and things went exactly as he had hoped but not expected. He may have had to play the indolent monarch, taking nothing seriously and bereaved by a horrid jest gone awry. It is likely he was engaging in precisely the sort of deception Sun Tzu advocated in his treatise. He appeared weak and foolish, but knew exactly what he wanted and how to obtain it.

This probably was not lost on Sun Tzu, either. Despite his parting admonition, he did later agree to serve Ho Lu. It is quite possible that the king understood precisely the position he was placing Sun Tzu in, and anticipated the possible executions. Even so, he may have been uncertain of the man’s practical talent and the extent of his will. There is a great divide between those who write words and those who heed them. Some may bridge it, most do not. Only in the event did Sun Tzu prove himself.

For this reason, Ho Lu could not be certain of the fate of the women. Nonetheless he placed them in peril. They were disposable, if not to be disposed of. It seems plausible that an apparently frivolous court game actually was a determined contest between two indomitable wills. The only ones who did not grasp this, who could not even recognize the battlefield on which they stepped solely to shed blood, were the concubines.

By this hypothesis, they were regarded as little more than favorite dogs or horses, or perhaps ones which had grown old and tiresome. A King asks an archer to prove his skill by hitting a “best” hound, then sets the dog after a hare, as he has countless times before. The dog quickens to the chase, eagerly performing as always, confident that its master’s love is timeless and true. Of all present, only the dog does not know it is to be sacrificed, to take an arrow to prove something which may or may not be of use one day to its master. If the arrow falls short, it return to its master’s side none the wiser and not one jot less sure of its place in the world or secure in the love of its master, until another day and another archer. This analogy may seem degrading and insulting to the memory of the two ladies, but that does not mean it is inaccurate. It would be foolhardy not to attribute such views to an ancient King and general simply because we do not share them or are horrified by them or wish they weren’t so. In that time and place, the concubines’ lives were nothing more than parchment. The means by which Ho Lu and Sun Tzu communicated, deadly but pure.

The view that Ho Lu was neither a fool nor a bon vivant is lent credence by the manner of his rise to power. He usurped the throne from his uncle, employing an assassin to accomplish the task. This and his subsequent campaign of conquest are not the actions of a dissipated monarch. Nor was he absent from the action, wallowing in luxury back home. In fact, Ho Lu died from a battle wound during his attempted conquest of Yue.

It is of course possible that the true person behind all these moves was Wu Zixu, the King’s main advisor. But by that token, it also is quite possible that the entire exercise was engineered by Wu Zixu — with precisely intended consequences, perhaps ridding himself of two noisome rivals with influence over the King. In that case, the affair would be nothing more than a routine palace assassination.

Whatever the explanation, we should not regard the deaths of the two concubines as a pointless tragedy. The discipline instilled by two deaths could spare an entire army from annihilation on the field. Sun Tzu posited that discipline was one of the key determinants of victory, and in this he was not mistaken. That is no excuse, but history needs none. It simply is.

This said, it certainly is tempting to regard the fate of these ladies as an unadorned loss. Who can read this story and feel anything but sadness for the victims? Who can think Sun Tzu anything but a callous murderer, Ho Lu anything but foolish or complicit? It is easy to imagine the two court concubines looking forward to an evening meal, to poetry with their friends, to time with their beloved husband. They had plans and thoughts, certainly dreams, and perhaps children they left behind. One moment they were invited to play an amusing game, the next a sharp metal blade cut away all they were, while the man they imagined loved them sat idly by though it lay well within his power to save them. Who would not feel commingled sorrow and anger at such a thing? But that is not all that happened.

A great General was discovered that day, one who would take many lives and save many lives. Whether this was for good or ill is pointless to ask and impossible to know. All we can say is that greatness was achieved. 2500 years later and in a distant land we read both his tale and his treatise.

Perhaps those two died not in service to the ambition of one small general in one small kingdom. Perhaps they died so centuries later Cao Cao would, using the principles in Sun Tzu’s book, create a foundation for the eventual unification of China. Or so that many more centuries later a man named Mao would claim spiritual kinship and murder a hundred million to effect a misguided economic policy. Would fewer or more have died if these two women had lived? Would one have given birth to a world-conquering general, or written a romance for the ages?

None of these things. They died like everyone else — because they were born. The axe that felled them was wielded by one man, ordered by another, and sanctioned by a third. Another made it, and yet another dug the ore. Are they all to blame? The affair was one random happening in an infinitude of them, neither better nor worse. A rock rolls one way, but we do not condemn. It rolls another, but we do not praise.

But we do like stories, and this makes a good one.

[Source: The account itself is taken from The Art of War with Commentary, Canterbury Classics edition, which recounted it from a translation of the original in the Records of the Grand Historian. Any wild speculation, ridiculous hypotheses, or rampant mischaracterizations are my own.]