Picking the newest bestseller with AI

Wouldn’t every publisher like to know what the next bestseller is going to be? With a fractional percentage of books becoming hits, wouldn’t your company want to know what the next hot genre in young adult fiction is going to be (werewolves? centaurs, maybe?) Or the next author people will gravitate to? These days, artificial intelligence (AI) can help you answer these questions. Predictive analysis using AI can give important cues to editorial teams.

This is not to say you can simply hire a computer to pick winners for you. An algorithm alone cannot tell you what is going to sell, though some may posit otherwise. A book titled “The Bestseller Code” claims an algorithm can use text analysis that can predict with 97% certainty whether a fiction manuscript will be a number one bestseller. Digital publishing guru Mike Shatzkin offer a blunt assessment of that claim: “Our verdict on this: absolutely impossible.

Neil Balthaser takes a more measured approach than Shatzkin’s decisive verdict. In a blog post titled “Yes, machine learning can help predict a bestseller,” Balthaser points out machine learning can “identify similar tones, moods, topics and writing styles to books that are topping bestseller lists … and, in this way, better understand the reading audiences’ desires.” That much is true. A system could be trained to look at bestsellers, which represent current market interests, and compare them to any current titles a publisher is soon putting out in order to help identify where to focus marketing efforts. Machine learning can remove the gut feeling or personal bias inherent in business decision making.

By focusing on textual analysis, though, both “The Bestseller Code” and Balthaser are missing the bigger picture. Shatzkin uses Google’s approach to predicting box office receipts as a case against text analysis alone: “They look at all sorts of data: number of screens, box office results from previous movies by the headline actors, search volume for the movie itself, YouTube views of the trailer, genre, seasonality, franchise status, star power, competition, critic and audience ratings of any preview. They don’t try to read the script.”

Machine learning is also working its way into an age-old practice in publishing and media: market research.

AI is Morphing Market Research

Much of the market research done these days still relies on a basic premise: ask your customers what they want. Focus groups, review copies and opinion polling remain very popular methods to inform decision-making. However, if the 2016 presidential election in the United States taught us anything, it’s that polls are not 100% reliable. This is where AI and predictive analysis can help.

As Thomas Baekdal writes, “In the future, polls will likely be a mix of actual poll data with many other data sets, all analysed and weighted by machine learning algorithms.” Combining opinion poll or focus group data with social media trends (i.e., what people are talking about, what they said about similar releases, etc.) enables publishers to forecast market reaction. Shatzkin cosigns on this approach: “The message for publishers is that audience research, with some of it specific to each title you publish, is the key to success in the digital age.”

Bottom line, the machines will not tell you everything you need to know. But they can help.