Criticism, fiction and other writing
Maybe 11 or 12 years ago I downloaded and read a 72-page essay in PDF by writer and publisher Michael Allen. Its title is On the Survival of Rats in the Slush Pile. Allen took a gloomy view of the state and prospects of the publishing industry and authorship, and in particular the authorship of fiction. He wrote the essay in 2005, two years before the introduction of the Kindle and long before the ebook self-publishing phenomenon took off. I think it’s fair to say that Allen’s argument had a profound influence (of which I wasn’t always conscious) on my own outlook and assumptions about book publishing. It was there, for example, that I first heard of Nassim Nicholas Taleb’s book Fooled by Randomness, which I subsequently bought and read. I found Taleb’s thesis persuasive but disappointingly unhelpful, for reasons unconnected with the topic of this post but which I may write about in future.
Just last week I rediscovered the PDF of Allen’s essay in a forgotten folder on an external hard drive, so I reread it. Allen argued that publishing was not a business that was conducted rationally. Its profits were dependent on “black swan” events, such as the Harry Potter series: books, authors and series whose runaway success was unpredictable, in many cases highly dependent on chance and so vanishingly rare as to be statistically negligible. For the same reasons, writing (fiction in particular) was not an activity that a rational person would engage in with the expectation of making a living, still less a fortune. It would be interesting to know whether the rise of the ebook self-publisher would have changed Allen’s view. I suspect not: he revived his blog in 2012 after a gap of 5 years, with the avowed purpose of occasionally posting
… shameless plugs for my newly published books in digital form. I am writing some brand-new stuff, it’s true; but more often I’m reviving some of my previously published books in digital forms. Because they’re really rather good and I’m quite proud of them. Worth reviving, imho.
The posts were noticeably sparser than they had been during the earlier life of the blog and there have been none at all since May 2015.
According to Allen, publishers and authors alike — and, to the extent that they rely on their slush piles for the discovery of new talent, literary agents too — indulge in fallacious thinking designed to persuade themselves that they should continue to pursue their accustomed activities in the same way as before. One of the more important ways that writers and publishers fool themselves is by reliance on survivorship bias. This is the idea that people who have survived (or succeeded in their chosen field of activity) will have certain characteristics in common, often including unshakeable self-belief and commitment, persistence, hard work etc. An author who wishes to succeed should therefore emulate these success stories and exhibit, and if necessary develop, the characteristics that have worked for them. The fallacy is that this approach ignores the (presumably many) people who exhibited all these characteristics and have nevertheless seen success and adulation elude them. Survivorship bias has been around for a long time: Francis Bacon noticed it at work in the early seventeenth century, when he was shown portraits of men who had been saved from shipwreck, thought to have been rewarded by God for their holiness and religious observance. Bacon pointed out that there were no portraits of the “drowned worshippers”, those who were not saved despite sharing the same qualities as the ones who were.
As someone who has generally been anything but sanguine about the prospects of the publishing industry and of authors who aim to make a living from writing, I was sympathetic to Allen’s gloomy conclusions. I was surprised, therefore, to find that on rereading his essay, my attitude has changed somewhat. In the years since I had read it for the first time, I’d been imperceptibly coming around to the opinion that things weren’t quite as bad as he depicted them. In particular, I’d begun to think that he was allowing too much weight to the idea of survivorship bias.
The simple fact is that we don’t know how many worshippers were drowned relative to the small number of those who were saved. In the same way, we can’t know how many mute inglorious Miltons are now resting in country churchyards or how many undiscovered Rowlings eventually faced reality and gave up sending their brilliant manuscripts to publishers and agents. Because Milton and Rowling, Stephen King and Kazuo Ishiguro are unique talents, “black swans” in Taleb’s terminology, the literary counterparts of venture capital’s “unicorns”, it’s tempting to draw the conclusion that they’re very rare exceptions in every category to which they belong. Not only do they form a tiny proportion of the population at large, and a tiny (though larger) proportion of the writers who have submitted their work to publishers and agents, but they must also be a small (and unimaginably lucky) proportion of the group of writers of real talent with something significant to say who have submitted their work to those publishers and agents.
This conclusion isn’t warranted and is to some extent contradicted by the evidence that Allen himself presents. He is clear that the slush pile has not historically been the source of profitable new discoveries by agents or publishers. The prominent London agent, Pat Kavanagh, said that she had never found a new author there (p. 26 of the PDF). The success rate reported by other agents and publishers is relatively tiny. Given this history, it’s hardly surprising that the publishers don’t put their more experienced and expert readers on the hunt through the slush pile, a task that is more typically undertaken by interns and the most junior staff. It’s conceivable that these overworked, underpaid readers are missing the occasional potential literary giant, but it’s surely not likely that they’re missing more than a handful. In other words, sustained if none too hopeful attempts are being made to find the drowned worshippers, and it doesn’t seem that there are very many of them after all.
So, if you really think you have the writing skill to become the next Ian McEwan or Curtis Sittenfeld, a new Michael Connelly or Karin Slaughter, you should probably not hold back from submitting your work merely because you’re afraid of being fooled by survivorship bias. Perhaps it’s better to be foolishly hopeful than to bury a black swan out of a foolish fear of acting irrationally.
Posted by Art on 08-Sep-2019.