The creators of RobotReplay describe it as a “Cinelytics” (cinema plus analytics) tool. But part of the challenge is that we’re talking silent movie. There is, of course, no sound on Robot Replay recording. That makes a difference. In most of the live usability sessions we’ve been involved in, real “A-ha’s” often come not just from what you see the users do onscreen but also from what you hear the users say. ( Think about how often the moderator steadily reminds to "think out loud") We all want to learn the stories behind our users' behavior. And while a lack of sound didn’t stop silent movies from telling compelling stories, those narratives were helped along by handy title cards to explain motivation ( This Villain has Evil in His Heart) and those great exaggerated facial expressions from the actors.
As site owners begin to parse the data from their RobotReplay sessions, they may find themselves hankering to see what the user’s face looked like when she failed to click, or to hear she was thinking when she lingered so long on page 2 of that form.
Idea: use quick website surveys in tandem with RobotReplay. Craft questions you need answered to validate hypotheses about what you’ve observed and to home in on what to look for the next time you replay sessions.
Beyond the challenges specific to cinelytics, there’s always the general site analytics hurdle of all that data, and the way it will not do much without a real investment in people. Success depends on analysts with time and skill to extract a narrative from the information, and to work with their website teams to make the changes these stories suggest.
None of this is meant to sound overly-curmudgeonly. I'm excited about what the RobotReplay folks are offering and I'm eagerly awaiting my first screenings. It’s great to find another example of site analytics trying to answer questions of "how" and "why" instead of only reporting “how many.”