Some Holiday Cheer for Nascent Writers

Readers of my blog know that I’m not given to chatty, optimistic posts. In fact, my typical post is more along the lines of “Not only will you fail at writing, but your cat will run away, your house will burn down, and you’ll spend the rest of your life tweeting from a phone you forgot to take out of airplane mode.” This post is different. While it isn’t quite optimistic, it does offer a perspective you may find uplifting, perhaps even liberating.

I’ve participated in many writing groups over the years and have managed a few as well, including one which technically qualified as Boston’s largest at the time. I have many writer acquaintances and even a few writer friends. I’m not spouting this to toot my own horn, but to lend credence to what I am about to say.

Many writers seem to have a notion of success which I deem unhealthy. I’m not saying that we should redefine "success" so that everybody is a winner or any such happy horseshit. The problem is that writers have two competing, and largely incompatible, goals. I speak here of real writers, not people who simply produce a product. The difference, to my mind, is that a writer wants to be proud of their work. My own standard is that I write what I want to read, and I think many of us implicitly or explicitly have a similar benchmark. We may try to cater to the taste of the crowd or steer toward marketability, but catering is different than pandering and steering is different than veering. At the end of the day, the stories and books which we produce must satisfy us first and foremost. This does not mean we always succeed in meeting that standard, but it is what we strive to aesthetically achieve.

However, American culture imbues us with another standard of success — one that is financial and social. Though success in this regard can be achieved via various avenues, the essential value it embodies — and that which our society most greatly respects — is the ability to sell things. In practice, this often takes the form of selling people stuff they don’t really want — but it need not. We are taught that the "net worth" of an individual is the sum of their possessions, marked to market, and distilled to a number. We are taught that we can order people by importance from lowest to greatest based on that number. We are taught that if one author sells many books and another sells few, the first is much "better" than the second. And we are taught that if a big publisher picks up a book, it is a "better" book than any which are self-published. While many of us may vociferously reject such a simplistic and materialistic outlook, we nonetheless are thralls to it. We may know that fatty foods are bad for us and that consumerism destroys the environment, yet there we all are in front of brand spanking new 100-inch televisions with bags of Fritos in our hands. Knowing and feeling are two different things, as are knowing and doing. We know we shouldn’t adopt the typical American view of success but we do anyway. Understanding and accepting that we are susceptible to such internal contradictions is crucial to avoiding the misery they otherwise can engender.

For a writer, the ability to sell our writing is essential for American-style success. These days, this entails also selling our "own story" as well. I personally find this obsession with the author rather than their work vapid at best and venal at worst, but it’s a fact of the market. The demographics of who reads and how are vastly different than a few years ago, as are the nature of publishers and what they seek. You sell yourself, then your writing. This compounds an already difficult problem for most of us. Good writers are good writers, not necessarily good salesmen. Those who spend their time selling things and have the aptitude to do so rarely also have the time or ability to write a quality book — and those who spend their time writing and have the aptitude to produce a quality book rarely have also have the time or ability to sell it.

This is a practical reality that affects almost any creative or scientific field. Those who can do can’t sell and those who can sell can’t do. But there is a deeper issue as well: a conflict of what we actually deem important with what we imagine we should deem important. What we want as authors and what we have been trained to believe we should want as Americans are largely incompatible. If we achieve only the first, we see ourselves as failures. If we achieve only the second, we see ourselves as hacks. And it is well-nigh impossible to achieve both.

It is not difficult to see why. If you’re like me and have tastes that depart even in the slightest from the mainstream, then ask yourself how many books that you really love are being published by major publishers today. Not books you’re told you’ll love, or books that you’re supposed to love, or books other people tell you they love. But books you love. For me, it’s virtually zero. The type of writing I enjoy simply isn’t published anymore. At least not by big publishers, and probably not by small presses either. It’s still being written. I’m writing it, and I’m sure plenty of other people are too. But it’s at best being self-published, and as a result is very hard to find.

The same is true of the big successes in self-publishing and is the reason neither you nor I ever will be one. The best sellers are in a small set of genres and usually involve the same perennial cast of series and authors. These authors are very good at gaming the system — i.e. at selling their books. However, they are not authors in the sense I described. They view a book as a product and nothing more. They run a business and are very good at it. For them, there is no contradiction in goals because their sole goal is financial success. There is nothing wrong with this, but it is not sufficient (or even attainable) for an author of the type I am addressing this to.

I’ll give you an example of what I mean. When I used to live in New York City, there was a famous camera store I frequented. In the same building, the next storefront was a diving shop. One day, I needed to buy some diving gear and went into that shop. I recognized some of the employees from next door, and it turned out that both stores were owned and run by the same people. The employees in either store knew everything about what they sold. If I asked an obscure question about a camera feature or model, someone knew the answer. But if I asked a subjective question, such as which camera they preferred or which BCD they found comfortable when diving, they were of little help. They could opine about which model customers preferred, and they could rave about one or the other product in a sales-pitch sort of way, but they clearly had no personal experience with the products.

I wondered at this and asked a friend who moved in similar circles about it. He explained that the product didn’t matter. It was all about understanding the market and sourcing the products at low cost. The store employees were generic highly-skilled business people. They could go into any market, learn the jargon and product specs and market layout, source the products at a good price, and then advertise and hard-sell those products very effectively. To them, it didn’t matter what they were selling. The products were widgets. The owners of those stores probably had no especial love of photography or diving but recognized those as markets they could thrive in.

Almost all self-published authors who succeed financially are of the same ilk. The books are products, and they just as happily would produce wicker baskets if that was where the money lay. Such authors have no ambition to write a high-quality literary novel. If their market research says that novels about vampire billionaires who fall for midwestern housewives are the thing, they will pump out dozens of nearly-identical ones. In this regard, such writers are a bit like the big publishers. The main differences are that (1) these self-published writers produce their own products and (2) the big publishers seem to have lost their focus these days and now employ ideological criteria rather than purely market-related ones.

The result is obvious. If I write a book of which I’m proud, it won’t get traditionally published and it won’t sell much when self-published. At a more basic level, this is a problem which affects all "producers", including artists, scientists, and musicians. To succeed in the social/financial sense, you have to spend 100% of your time relentlessly promoting yourself (and even then, the likelihood of success is small), but to produce anything of substance you have to spend all your time developing your craft and then applying it. The product-writers I described are very efficient. They are experts at what they do. After all, even in the world of marketable-schlock there is lots of competition. The winners know how to game the search engines, get in early and stay at the top, spend marketing money efficiently, and expend the minimum time necessary to produce a salable product.

The gist is that the two goals of a real writer are utterly incompatible. Writing a book we are proud of and achieving social/financial success with it are mutually exclusive for most of us. Unless you really love writing crowd-pleasing schlock or happen to be one of the handful of random literary "success" stories, it is impossible.

"But Ken," I can hear you whining, "I thought you said this would be uplifting? That my cat would still love me and my house wouldn’t burn down and I’d remember to turn off airplane mode before tweeting. How the hell is this remotely optimistic? Do you secretly run a razor-blade and cyanide business on Amazon?" Well, yes and no. Since books don’t sell, I do need some side hustles. Please visit my Amazon page for a very special offer.

Ok, fine. Here’s the inspirational bit. It isn’t that we can’t achieve both goals — it’s that we don’t have to adopt both goals. You are in control of your goals, even if your social programming reeeeally wants you to think otherwise. If your goal is to be both successful in the American sense and proud of your work, you’re going to be bitter and miserable. It’s disheartening and you’ll give up as a writer or feel resentful toward the world. But that shouldn’t surprise you. If you demand the impossible, you’ll always be disappointed. If your goal was to be a fantastic high-school teacher and also become rich from it, you’d be miserable too. It’s very hard to succeed financially in any way, let alone one which appeals to you. If I wanted to be a professional basketball player, I’d be disheartened. I’m five-foot-eight. The fault wouldn’t be with the world, it would be with me for demanding the impossible. While it’s admirable to pick difficult but attainable goals, picking wildly implausible ones is a recipe for misery. If you set out to prove the world wrong, all you’ll do is prove yourself a fool. Not because the world is right, but because there’s no point in wasting your life trying to prove anything to nine billion people who won’t notice and couldn’t care less if they did.

I’m not spouting some hippy nonsense about eschewing material possessions. You need money to survive and live comfortably. Money can buy you independence and free time. I’m not saying you don’t need money or shouldn’t pursue it. Just don’t rely on your writing for it, at least not if you want to be proud of that writing. It is perfectly fine to aim for American-style success. It’s difficult, but anything worth striving for is difficult. Nor is it unattainable, assuming your ambition isn’t too extravagant. In that’s your primary aim, do what the successful schlock-producers do and maybe you’ll succeed.

I’m also not saying you should sit in a corner munching a soggy carrot like some dejected rabbit. It is perfectly fine to write books you are proud of and hope for American-style success. I hope that my lottery ticket will win a billion dollars. There’s nothing wrong with that. It may even happen. Hope can be beneficial.

What is not fine is to expect American-style success from your writing. That is toxic. It means you’ll never reward yourself. Even if you write the greatest novel in the world, you still won’t allow yourself a sense of accomplishment. Imagine a small-town artisan who crafts beautiful furniture but demands that each piece be featured on some television show. He’ll be perpetually disappointed. No matter how great his skill and attainment, he never can give himself the slightest praise. There’s always a monkey on his shoulder telling him "So what? You’re not on television." If you’d laugh at such a person, take a good, hard look in the mirror.

To illustrate our biases, here are some scenarios. Suppose you learned that a friend …

  1. Wrote a wonderful book, got rejected by 200 agents, self-published it, and sold 3 copies.
  2. Self-published a vampire-billionaire-loves-midwestern-housewife book and sold 50,000 copies.
  3. Wrote a vapid, self-indulgent novel with elements designed to appeal to certain political sensibilities, which has been picked up by a major publisher.
  4. Self-published a book of pictures of cats with cute little taglines, which went viral and sold 100,000 copies.
  5. Wrote a passable book, though nothing worthy of note, but knew some agents and got picked up by a major publisher.

Most of us automatically would be "impressed" by (2)-(5) but view (1) as a vanity project. That is ridiculous. It’s our subconscious American training at play. Think about it. (2) may be a worthy businessman but isn’t really a writer, (4) produced a little nothing and got lucky, (5) produced tofu but knew the right people, and (3) produced what best could be termed a "vanity project" which ticked the right boxes. Of the five, only (1) produced something actually worthy of praise.

Nor are these contrived scenarios by some bitter rejectee (aka yours truly). Anyone who has contact with the publishing world knows that these are highly-realistic scenarios and that they are way more common and apropos than most of us would care to admit. So why do we view (1) this way? It’s not just our American-success programming. It’s also because of another very common scenario:

  1. Wrote complete trash, self-published it, and sold 3 copies.

(6) is what gives a bad name to self-publishing and constitutes the vast vast vast majority of self-published work. It and (2) are the reason you won’t be able to be heard above the fray or find your niche audience or sell many books.

But that isn’t as awful as it sounds. For most of history, only a privileged few even knew how to write, fewer had the means and leisure to write a book, and fewer still ever got published. Even if you wrote an incredible manuscript, without the money or connections to publish it that manuscript would end up in someone’s fireplace. So what’s different now, you may ask? Isn’t the problem the same, and only the gatekeepers and criteria have changed?

Yes and no. Yes, if you go through the gatekeepers. No, because you don’t have to. You can write a book you are proud of and self-publish it. It will be up forever as print-on-demand (and/or an ebook). You don’t have to build buzz, have a grand launch, and pray you reach critical mass before the rest of your print run gets remaindered and you end up out-of-print forever. Instead, you can put your book out there and point people to it over the years as you see fit. You can market it later when you have time or some opportunity arises. A book you are proud of will be available for anyone to purchase. Your backlist never goes away. Yes, a lot of crap gets self-published today — whereas in yesteryear only a few rich people could self-publish. But that need not bother you. Bad company does not a knave make. You’re not counting on people discovering your book by wading through all that garbage. You’re just making it available. You are the discovery mechanism. When someone asks about your book, you can point them to it. If you so choose, you can spend some money to increase the chance people will buy it. You can do this when and how you want.

And if someone at a cocktail party looks down their nose at you when you mention that you are self-published, just ask them what they’ve done lately. I wouldn’t worry too much about this happening, though. Does anyone even have cocktail parties anymore?

Incidentally, through much of the last three centuries there were no traditional publishers. Everything was self-published. But there was much less of it. Now, everyone can self-publish and everyone does. But just because a lot of other books stink doesn’t mean yours does — or that it will be viewed that way by modern, intelligent people.

In conclusion:

(1) Stop thinking of self-publication as a stigmatizing last-resort and a humiliating proof of failure. It is a tool and an opportunity. Moreover, in today’s world it is both a necessity and a reality for almost any author of substance. (2) If you write a book you are proud of, allow yourself to be proud of it. Feel successful. Decouple this sense of success from guilt or shame or anxiety about it not selling. (3) Write books you are proud of. Hope for American-style success if you wish, but do not expect it. (4) Keep writing. Write what you want to read. Be pleased that you have accomplished something.

If you complete one story you are proud of, you have accomplished more in your life than 99% of people. If you complete one book you are proud of, you have accomplished more than 99.9%. If you spend your life writing books you are proud of and allow yourself to be proud of them, you will have accomplished something almost nobody does: you will have lived a life you are proud of.

Everything everyone does is for naught, "vanity and a striving after wind," to quote Ecclesiastes. Had children? Your genes will dilute out of their progeny after a few generations. Became famous? Nobody will remember you a few years from now, and if they do it will be a mere caricature. Made a lot of money? You will is the last time you get any say how it’s spent. The best you can do is live a life you are proud of. Once you’re gone, the universe ends. It is irrelevant how many people bought your book or whether it lives on or your name is remembered.

And on that uplifting note, I once again refer you to Ken’s razor-blade and cyanide shop on Amazon. Oh, and don’t forget to leave a great review when you’re done…

Kindle Scribe — An Interesting Device Crippled by Bad Software

I’m a big fan of Kindle ereaders. However, since the long-defunct Kindle DX there hasn’t been one with sufficient screen real estate to read scientific papers and pdfs. The Kindle Scribe boasts a (slightly) bigger screen than the Kindle DX and allows writing as well. Couple that with the insanely long battery life of a Kindle ereader and it sounds like a dream machine, right? Wrong. Unfortunately, Amazon decided to follow Apple and Garmin down the path of crippling great hardware. As a result, the writing function is all but worthless to sensible users.

There were warning signs about whom the Kindle Scribe was intended for. I tried to find the screen dimensions (or resolution), and the top few pages of search results showed only the device dimensions or the diagonal (10.2 in). If you’re curious, the screen measures around 6×8 in (6.1×8.2 in to be precise, but there’s probably a tiny border) and probably is 1800×2400 pixels (given their 300 dpi spec), constituting a 1:1.3333 aspect ratio. Such information is pretty basic and useful, but even Amazon’s spec page didn’t list it. It did, however, list the pen’s dimensions, the WIFI security protocols it supports, the wattage of the power adapter, and the (sole) available color. I fail to see how any of those are more relevant than the screen dimensions.

Another warning sign is that finding any useful info about the Amazon Scribe is well-nigh impossible. Granted, it’s only been available for a few days. However, the many sites purporting to review it or to provide helpful tips contain nothing more than caffeinated regurgitation of marketing info. Even the “criticisms” read as thinly-veiled praise. You almost can hear the authors’ voices trembling, though whether from excitement or fear is unclear.

“Pro: Everything about everything. All is for the best in the best of all possible worlds, and Amazon IS that world.”

“Con: Amazon can’t get any more amazing than it currently is. But I’m sure it somehow will!!! Please don’t hurt my family …”

Frankly, I wouldn’t be surprised if they’re mostly shills. That seems to be standard practice in online marketing these days.

Marketing practices and information availability aside, the real problem is that there is no sensible way to extract notebooks from the Kindle Scribe. It is easy to write on the device, and the process is quite pleasant. It’s really a lot like a paper notebook. I’m not a user of existing writeable e-ink tablets or of Wacom devices, but I found the Kindle Scribe’s hardware perfectly suitable for my purposes. Sadly, hardware alone maketh not the product.

As far as I can tell, Kindle Scribe notebooks aren’t stored as pdfs or in any standard format. Rather, they appear to be stored as sequences of strokes in some sort of non-standard Sqlite DB. How can a Sqlite db be “non-standard”? When I try to examine the db schema in Sqlite3, I get a “malformed” error. Most likely, Amazon relies on some Sqlite3 plugin that encrypts the notebooks or otherwise obfuscates them.

None of this is worthy of criticism in itself; in fact, the ability to undo and redo my writing stroke-by-stroke is quite nifty. However, Amazon offers no sensible way to export a pdf from a notebook. The only solution is to email it to yourself as a pdf using their Whispernet.  From what I’ve read,  Whispernet pretty much acts like malware by logging all sorts of activity.  To my mind, the best policy is to keep it off all the time. Why not just fire it up briefly to export a notebook? It is quite possible that the Kindle logs activity locally and then sends the whole log to Amazon the moment Whispernet is activated. I have no evidence that this actually happens, but I wouldn’t be surprised.

However, there is a much bigger issue with Whispernet. Even if you trust Amazon’s intentions, there are a lot of other parties involved and a lot of potential data leaks. Anything you write is uploaded to Amazon’s servers, then converted to a pdf on their end, and finally emailed unencrypted to your email address. I.e., there are multiple insecure steps, not least of which is the receipt of that unencrypted email by your email service. Any security-minded individual would have an issue with this, and it precludes the use of notebooks for anything sensitive or by any professional required to adhere to security protocols.

One of the aforementioned hypercaffeinated blogs gushed over the ability of the author’s law-firm to collaboratively sign NDA’s using Kindle Scribes. My response is simple: “Are you out of your f-ing head?” I’ll mark down the name of that firm to make sure I never do business with it (though frankly, my experience is that most lawyers are terrible with technology and security — which is odd in a profession whose bread and butter is sensitive information). Why is this a bad idea? Let us count the ways. First, it would require that all participants have Amazon Scribes and Amazon accounts. I can count on one finger the number of people I know who have one — and probably ever will. Second, all the info would be passed unencrypted between the various parties and Amazon and whatever email servers are involved for each and every individual. Third, how is this innovative or necessary? Plenty of secure and simple collaborative signature mechanisms exist. Whatever you may think of Docusign, it’s a sight better than passing unencrypted PDFs around like this. I’d guess that iPads have had suitable functionality for some time (and with much better security) as well. And more than one person I know has those.

The inability to directly export PDFs may or may not be an oversight, but I suspect not. I tried a few obvious workarounds and was stymied at every turn. Sometimes it felt quite deliberate.

First, I tried extracting the relevant files. As mentioned, they appear to be in some proprietary variant of Sqlite3. There is no obvious way to extract useful info except by reverse engineering them. Supposedly, it is possible to view notebooks offline using an Amazon reader — but I have had no luck with this. Besides, Amazon doesn’t really support Linux (even though they use it on all their servers AND the Amazon Scribe) — so such functionality probably isn’t available to me without firing up a VM.

Second, I created a PDF notebook of blank pages using pdftk and a free college-ruled template I found online. My plan was to annotate it as a pdf rather than use the official notebook feature. Presumably, PDF annotations provide the same interface. I sideloaded the PDF file via USB, but annotations were unavailable. Apparently, they only are available for PDFs loaded via the “send-to-kindle” email mechanism. I can’t think of any benign reason for such a limitation.

Third, I tracked down the abstruse details of how to “send-to-kindle” and sent the PDF to my Kindle Scribe using the “convert” command in the subject line (as recommended). The Kindle Scribe wouldn’t even open it. Apparently, Amazon’s “convert” machinery is incapable of even converting a blank notebook.

Fourth, I did the same but with a blank subject line. Presumably, this passes the pdf along unchanged. Now, the Kindle Scribe both opened the file and allowed annotations. Success!!! Or not. Like notebooks, annotations apparently are stored as individual strokes in some proprietary Sqlite3 db. And apparently, there also is no mechanism to export them without going through the same email process as for a notebook. Unsurprising, but annoying. You’d figure Amazon at least would allow modified PDFs to be saved as … well … PDFs.

Put simply, Amazon appears to have made it impossible to securely or locally export notebooks or annotations. The writing feature therefore remains largely unusable by anyone with a sensible regard for privacy or security. The year is 2022, and the best solution is to photograph the notebook pages and xfer the photos to your computer. Yay technology! If only I had to send the photos to be developed and then scanned, my jubilation would be complete.

Given the house of horrors which is AWS, the ridiculosity surrounding Kindle Scribe Notebooks should come as no surprise. After all, building your own computer from vacuum tubes and then writing an OS for it is easier than setting up an AWS instance. Sadly, the institutional maladies which plague AWS (and Amazon author services, for anyone unfortunate enough to use those), seem to have spread to one of their few functional and innovative divisions.

“But hey — aren’t you being a bit harsh?” you may ask. “The thing just came out. Give them some time to fix the bugs.”

For anyone hoping that firmware updates will “solve” these problems, don’t count on it. Even for those issues with enormous user support, Amazon has been notoriously slow to provide obvious fixes, and this one is a niche problem that the typical “privacy? what’s privacy?” post-gen-x’er couldn’t care less about. Not to mention, Amazon just laid off much of their Kindle staff. If they followed the standard American corporate playbook, this included all the people who actually knew what they were doing.

Right now I’m debating whether to return the Amazon Scribe. Its size may be useful enough even without the writing features, but I doubt it. There’s also the remote possibility that Amazon will provide local export functionality.  But I’m not holding my breath.