Game Design, Programming and running a one-man games business…

metrics and accidental genius

cliffski

Are metrics killing creativity?

Imagine a future where all TV is either watched digitally, or streamed online, and the TV networks have 100% certain data over what program is watched, when channels are changed, and a good idea of what adverts were shown. The precision of the data would be vastly better than we have now.

Now combine that with the current trend for intensive study of audience reactions. Lets say in 2020, no TV program will get commissioned without the pilot being screened to 500 people in MRI scanners, to observe their emotional response to each line, each character, each event, each word.

This is probably how zynga would make TV, and who can blame them? All the data shows that if you collect extensive metrics on everything, you can fine tune the design of entertainment to maximise the audience figures and the revenue. The problem is,  sometimes the first impressions are just wrong. Sometimes, people think character X sucks, in the pilot, but goes on to be the best thing about the show. Sometimes, series I is basically a bit hit-or-miss, but by series II or III, its an emmy-winning masterpiece.

The first attempt at anything, with a new crew and cast, is normally a bit wobbly. People don’t really know what they are doing, how the whole experience will ‘gel’ and what the character of the program will be. I remember thinking that the first series of QI, and the first series of ‘would I lie to you’ on BBC TV, were both a bit ‘meh’. Apparently, the first series or two of Dragons Den had low audience figures and were dull. All 3 are hugely popular now.

I’m glad sometimes stuff that might seem a bit ‘meh’ is allowed to work out the kinks. The metrics are screaming ‘KILL IT!’, but if there is someone really talented behind a project, who can really see it in their minds eye, those things often go on to be the best things around.

We all know that the beatles got turned down by many record companies and that the sims was turned down by everyone. What if its true that not only are the big money-men often wrong with their first impression, but all of us are often wrong too? Maybe we shouldn’t trust the metrics 100%?

13 thoughts on metrics and accidental genius

  1. There will always be artists. No amount of metric can kill human creativity. Possibly a few more people will understand at that point why mainstream television isn’t shown in art-galleries.

  2. I think you said it yourself – metrics are good to fine tune things, as they can potentially allow you to hit a peak of whatever outcome you are aiming for. However the peak is a local peak. Artistry and craft are obviously still required and by far the most important thing as they create the initial starting point.

    Of course metrics can be over used, and kill off decent potential starting points without putting enough thought into developing or redeveloping a concept that could turn into something great.

  3. Season 1 of Buffy was a little rough, yet the series went on to a very successful 7 year run.

  4. Agreed. Much as the low-immediate-gains, theoretical research of our modern society tends to advance it in leaps, I argue that more experimental and ‘blind’ or accidental creativity is what keeps the entertainment industry so exotic and capable of being original.

  5. I think in a lot of cases, the metrics that are for optimizing a long-running show aren’t the same to getting a popular one. You could use Firefly as an example, but there was a lot more and while there are a ton of rabid fans, they aren’t the majority.

    I’d rather see lower budget, smaller runs to test ideas out, then let them flourish. I’ve been watching a lot of BBC though, with 6 episode seasons and I find that I actually prefer them over watching a show for a few, then seeing it canned.

  6. So you mean to say that the beta of the campaign is a bit ‘meh’ but you’ll work out the kinks and it’ll get better? :)

  8. Pingback: Tweets that mention Cliffski’s Blog » metrics and accidental genius -- Topsy.com
  9. Pingback: DevLinks: Turn of Events « DIYgamer

  10. The Long Tail. The ever plummeting costs of creating content. People such as yourself, who aren’t interested in reaching everyone but in reaching a core audience of a certain variety.

    I think that metrics will remain a useful tool, but we’re moving away from blockbusters and into tailored content. Tailored content by nature can’t be destroyed by metrics.

  11. I think the whole idea of being able to statisticise (is that a word?) human beings is, frankly, silly. You are never going to get a full picture from statistics.

  12. i think the scenario you have described is even worse than you realize. consider:

    http://youarenotsosmart.com/2010/05/26/the-perils-of-introspection/

    the way people evaluate things is dramatically different when they are deliberately evaluating, as opposed to simply experiencing. because the people who will be deciding what does or doesnt get a shot in the market are ACTIVELY deciding and having to explain and support those decisions, their perceptions (and the actual parts of the brain involved in the process) are radically altered. even fMRI doesnt avoid this problem, because the artificial setting and the conscious awareness that they are evaluating vs just experiencing the media will result in the same change in brain processing pathways and yield the same level of faulty results. focus groups are probably the very WORST way to try and determine whether or not you should bring a thing to market.

  13. Well Harry Potter’s first book was refused by several publishers because it was too grim for young children…

    Right…

    Use metrics to back up creativity not to kill it, most of the times metrics are used to understand what’s not working and trying to improve that instead of understanding what is working and improve that only.

Comments are currently closed.