Science fiction and fantasy

An Explosion Of Discovery Tools

Musings and rants

La Revolution: A Series For Our Time
In the television series La Revolution, French aristocrats are afflicted by a mysterious disease, whilst peasants go missing in suspicious circumstances.

Pandemic Reading
As the Covid pandemic rages, it has affected the way we read in a number of ways.

Reading Resolutions For The New Decade
Here are seven reading resolutions suitable for the 2020s.

Mastodon For SFF Fans
Where to go in the Fediverse to find the best speculative fiction and literary discussions.

Nine Political Books That Change The Conversation
Following news that Simon and Schuster plan to publish an inflammatory commentator, here are nine political books that deserve more attention.

Penguin Random House Withdraws Union Recognition
Penguin Random House have decided not to recognise Unite and the NUJ as a result of staff negotiations, leaving the publisher with a stain on its reputation as an employer.

Authors Support Stop Funding Hate
Some authors have had enough of divisive and xenophobic elements in the British press, and are willing to make an ethical stand.

Women In SFF: Indie Edition
A list of indie and self-published women writing in science fiction, fantasy, horror, and other speculative fiction genres.

Amazon Finally Makes KU Appealing For Novelists
The new per-page payout for the Kindle Unlimited subscription service makes it a much better deal for authors of longer novels.

Thoughts On The Sieghart Report On Libraries
The Sieghart report on libraries missed its mark by miles. Yet the real cause of the decline of the UK library network is depressingly obvious.

A Shout-Out For The Good Guys
When nastiness dominates online conversations about books it is time to appreciate the well-behaved authors.

Critique Circle: Shaping Fabulous Stories
The appeal of a certain writing critique website. Or, why I have neglected this blog.

Where Shall I Point This Pitchfork?
Some thoughts on Jonathan Ross, Loncon, and the twitchfork mob.

Reading Is Not A Race
Why I will be abandoning annual reading challenges in 2014.

What Book Discovery Is Missing
The current state of book discovery is narrowing our reading choices and squeezing out midlist writers. How can it be fixed?

16th September 2013

In autumn 2013, it seems everywhere I look someone else is trying to crack the book discovery problem: how do you introduce readers to their next great read? We've all been in the position of picking up a book that turned out to be less than satisfactory, and until this becomes rare then the book discovery challenge is not solved. According to a recent study by Heathrow Airport, 33% found auto-generated online recommendations to be "stereotypical", and 15% found them "dull".

A growing number of websites are setting out to try to improve this situation, using a variety of approaches.


A company called Oyster (not the card) is launching a subscription-based app that some people have dubbed "the Netflix for books", although it should be a Netflix for books, because this isn't the first innovation to be called that. Oyster features "people-powered book discovery", with personalised recommendations. It claims a catalogue of around 100,000 books, and it's currently at the beta invite stage. At the moment it's only available on the iPhone, but there are plans to put it on the iPad, and eventually on more devices.

However, I think it will suffer from a lack of data because some people will be put off by the price barrier, and others because it's only available as an app. And the fewer people who join, the less information people will add to improve it. 100,000 books sounds like a lot, until you appreciate that the US ISBN agency Bowker assigned 1.608,751 ISBNs in 2011. That's not an accurate estimate of how many books were published that year, because many books get no ISBNs, whilst others are assigned multiple ones for different formats. But it does give some indication of the scale of the problem.

It's what you do with it that counts

Size matters, but it's not the whole story. Amazon has one of the largest catalogues, and Goodreads has both a sizeable catalogue and, often, far more user ratings and reviews for newer books than Amazon. So both of these sites are behemoths. These are the biggest venues for online book discovery, but personally I find them imperfect, and I'm not alone.

Because of their importance for book sales, both Amazon and Goodreads have had problems with sock puppet ratings. There are also users who rate books before they've read them on Goodreads, further compromising the integrity of the data. But fraud and uninformed ratings aren't the biggest problems for these websites, by a long shot.

Take your medicine

The real problem is, aggregate ratings are one of the least useful predictors of how interesting a reader will find a book, in my experience. I've been in various discussions where smart people have been deeply divided over a novel's quality. They may help in outlying cases, when a book is so carelessly written it's likely to find no enthusiastic audience. But that's rare. For other cases, star ratings might as well be random.

Since we're all looking for something different in literature, "good" is too subjective. We all have different experiences that colour what we find obvious and hackneyed, for instance, so one person's meat is another's toxic landfill.

Some people suggest dividing ratings between quality and personal enjoyment. The idea being that a book can be high quality, worthy literature even if it's no fun to read. I consider this rather like insisting that if your medicine tastes horrible it must be doing you some good. I don't agree with this, I think a great book must be one that's hard to put down. But this kind of difference of opinion is exactly my point: no-one can agree on what star ratings really mean.

Other approaches

Bookdigits launched recently. It's a book catalogue in the style of Goodreads or LibraryThing, and users are asked to rate books on a slider for qualities such as literary versus commercial, addictiveness, movie potential, and re-readability. The ratings system is a slightly more granular A-F scale. There's space to leave reviews, and users can also add themes, which can again be given sliding scores.

My feeling is that this adds a new, interesting dimension. But it's still not quite right, because the ratings scale is still in use. It's also unclear what literary versus commercial means. Can't a book be both? Is it merely a euphemism for highbrow versus lowbrow? So the problem of people leaving data that doesn't mean the same thing to everyone remains.

Some of Bookdigit's categories are very specific to certain kinds of fiction, but they don't seem to apply to non-fiction or picture books. Evoke gets round this by focusing on YA fiction, which tends to be fairly character-driven. According to their description, "books and the characters that inhabit them may be browsed based on the emotions they engender". Evoke is the result of a Publishing Hackathon challenge, and unfortunately it isn't open to the public yet.

Expert classification

Evoke's approach reminds me of Whichbook, which has been around for some time. This website ignores the usual genre classifications and suggests books based on factors like beauty, violence levels, unusualness, or optimism. Alternatively, there's the option to pick books based on setting, plot, and character.

Whichbook does focus on lesser-known works, which makes it a little harder to assess how good its recommendations are. The books are all categorised by librarians, with around 70 of them working behind the scenes to create listings.

All of this painstaking classification by experts is in stark contrast to the free-for-all that's going on on Amazon and Goodreads, where anyone can contribute. Insisting on qualified contributors may limit the number of books that they can include, but it has the advantage of ensuring people apply consistent standards. I'm not sure how well that applies to something as subjective as ratings for humour or unusualness, though.

This brings me to Pandora, and its Music Genome Project. In an interview on Ars Technica in 2011 the Chief Musicologist Nolan Gasser explained that the music discovery website categorises 400 factors for classical music alone, and they deal with each genre differently. They also employ analysts with advanced music degrees. So it's not a minor undertaking. Unfortunately Pandora isn't available in the UK, so I can't comment on how well this works.

However, it's hard to say whether Pandora's careful categorisation could apply to books, which tend to resist mathematical analysis. I suppose you could run the text through a machine and sort books by page count, word repetition, and reading grade, and it would be possible to look at things like grammar, tense, and the use of the first or third person. Such automatic analysis could be done in minutes for an ebook, but it only gets you so far. To really understand a book, someone has to read it. The stumbling block is that there are so many, and each one takes several hours to read and evaluate. I don't think there's any getting around the fact that for a discovery system to function, someone has to put the work in, and the evaluations need to be consistent.

Even more tools

Novelry is yet another discovery tool, due to launch on September 16th but delayed, and not for the first time. It claims a reasonably large database of 11 million titles, and it tracks a number of literary awards. Awards are one way of measuring a book's quality, but they're still just the subjective choice of a group of people. The other problem with using awards is that they tend to go to books that have already been discovered by the public.

There's also BookLikes, which is more of a social network and blog platform than an innovative discovery method. The publishing industry-sponsored Bookish also has a tool which recommends books based on four you choose yourself. Its recommendations are based on:
  • Overlapping genres and/or subjects
  • Similar review sentiment and content
  • Expert editorial comparison
  • Shared authors, editors, and/or illustrators
  • Bestseller and award characteristics

In all, there are plenty of alternative discovery engines. I even had a stab at organising my reviews here with my own tool, based on some of the things I look for when deciding what to read. I built that in 2010, but I've recently been expanding it to encompass books that I may not have reviewed but have been able to classify. With only 450 books in the database it's pretty limited, but that's the compromise that all projects of this kind face. If I opened it up to all contributors the information would soon get muddy and meaningless, and if I restrict access too much then it's too small to be useful. Go here and fill in the details if you would like to add your own book to this tool.

And I should also mention Book Matchers, which has a very similar style to my tool, in that you have a page with options to select, and it returns the best match. Book Matchers is different because it has separate forms for fiction and non-fiction, and it has quite a wide range of options to choose from. It doesn't have very many non-fiction books yet, but there's more in the fiction section.

With all of these shiny new toys, readers will have a glut of choice, and it will be interesting to see which style of recommendation proves the most useful and popular. Size will play a vital role, but so will the underlying data, and the technology that brings it all together.