'Personalisation' of online media is one of Silicon Valley's technological obsessions. Christopher Williams reports on how Netflix is relying on its algorithms to take on Sky.
In Todd Yellin’s ideal world, technology can read your mind.
It may seem intrusive, but in an age when Google can gather vast quantities of information about your interests for online advertisers, and Facebook may know more about your family’s social lives than you do, Mr Yellin just wants to know which film you would most like to watch.
A former filmmaker and critic, he works for the American on-demand video service Netflix, developing the firm’s “personalisation” technology. It determines which films and television shows users are presented with when they log in.
“We’ll be finished when I could read your mind and we show one title up there and it’s exactly right every time,” he says. “That’s utopia.”
Such perfect personalisation is a white whale of Silicon Valley. Major web firms, such as Amazon, Google and Facebook, have legions of engineers and mathematicians trying to work out what users want without their systems having to be explicitly told. They analyse online behaviour by designing software algorithms – each one effectively a machine designed to draw conclusions from complex data.
At Netflix, which arrived in Britain in January, personalisation is central, partly because it relies on showing users more obscure or older films and televisions shows. They are cheaper to license than the blockbusters for which the likes of BSkyB pay big. At £5.99 per month for an unlimited service available via the web, connected TVs, games consoles and mobile devices, Netflix can’t afford to do that.
“We are going to pick the best ones we think each person might want to watch and show it to them,” says Mr Yellin. “User’s tolerance for scrolling through lists of content is only a few dozen titles, so we want to get them right away.”
Netflix begins its effort to read your mind from the moment users sign up, with a round of 20 questions. New members are asked what genres they like – action, romance, whatever – to build up an idea of their taste. However, what people say they like and what they actually watch are not always the same, Netflix has discovered, and it is the latter that matters more in the long run.
“I could walk around at a cocktail party and say ‘Oh yeah, I like all these foreign documentaries' and then in the privacy of my own home I’ll hit play on Paul Blart: Mall Cop,” Mr Yellin says.
“So in a matter of weeks of you joining we start using implicit information more and more. We throw information about what you’ve watched, how long you watched it for, how quickly you watched the next episode of something and a whole bunch of other stuff into our algorithms to produce your personalised recommendations, which will change over time.”
Dozens of algorithms are used to produce the various rows of recommendations that Netflix shows its members, such as a personal top 10, content similar to films they have recently watched, and personalised genres. The personalised genres can be absurdly specific, such as “feel-good opposites attract comedies”.
“Personalisation is hard because it involves incredibly big data and it needs to respond incredibly fast,” explains John Ciancutti, who leads the engineers who code the algorithms via a never-ending cycle of incremental improvement, with trial and error testing on unwitting groups of members. “We need to calculate new personalisation for every single user all the time and that’s an immense engineering challenge.”
Mr Ciancutti, himself an engineer, is perhaps less of a dreamer than Mr Yellin, although both agree that perfect personalisation is nowhere near a reality.
“It’ll never be the case where we know exactly what you want to watch. Each of us has different tastes and interests at different times; you might have a boyfriend or girlfriend over, or whatever,” says Mr Ciancutti.
“After 100 years people are still coming up with incredible innovations in car engines. We’re maybe five or 10 years at most into online video in any meaningful way. 100 years from now people will still be working on personalisation because it really is our engine.”
Silicon Valley’s obsession with personalisation technology has attracted critics, who argue that it cuts users off from new types of information and serendipity. The term “filter bubble” has been coined to describe this effect.
Netflix says its new Facebook integration will help combat that. By connecting their accounts, users are able to see what their friends have been watching. It is early days for the system, but Netflix is bullish on the impact that humans, as well as algorithms, can have on viewing rates.
“Ultimately the best metric to measure customer satisfaction is to measure whether people are sticking with us month after month,” says Mr Yellin.
“In that sense we couldn’t care less about what users watch.”