The power of surveys in a big data world


In his article in the Harvard Business Review titled “Getting More-Granular Data on Customer Journeys,” Jumpshot CEO Deren Baker makes a number of salient points that I can totally get behind:

  • “To effectively target customers, you need to know what they’re doing before, during and after their interactions with you.”
  • “Fragmented snapshots of customers’ online activities are just that: snapshots.”
  • “Mapping a customer journey is much like storyboarding.”

Given that backdrop, and what I’m presuming is a pretty large overlap in the Venn diagram of his beliefs regarding consumer journey mapping and mine, I’ll admit to being a bit gobsmacked (okay, that’s an overstatement, I just totally love the word gobsmacked) when he offers up gems like “To truly connect with customers, you can’t rely on survey data or back-end web analytics” and “Even if surveys were useful…” (with the unspoken point being surveys aren’t useful). And even if he walks back his statements later in the article, the comments at least mildly irked the old-time researcher in me. Not exactly fighting words, but I at least felt the need to respond.

Mr. Baker’s central premise is that the customer journey is no longer linear, but instead “It’s a circuitous web, taking customers from social media to your site to competitors’ sites and back again.”  He uses as evidence research his company has done regarding the dating website Zoosk to drive home his key points (Note to my wife: All the dating websites showing up in my browser history are just research, I swear it!). While the idea that it’s a circuitous web is right on, my issue lies with the premise that we can understand people by merely observing their online activities.

As an example of the power of tracking online behaviors, he highlights that based on consumer journey mapping his company has done Zoosk users tend to be younger, fitter, and 11.4 times more likely to search for other dating sites. And he writes, “It’s crucial insight into who the site’s customers are, what they do on other sites, and— most importantly—the journey they’re taking to Zoosk’s front door.”

I want to spend a few hundred words exploring whether you can really tell the journey Zoosk users are taking just by looking at clicks. Here’s my hot take on a provocative POV offered up by Mr. Baker…

  • Digital life is not entirely real life: According to a 2014 Halverson Group study, 64% of people engage in word of mouth in a consumer journey. From a 2014 article from Marketing News, 90% of brand-related conversations are offline. So yeah, just measuring clicks reveals only a sliver of the journey consumers are taking. The “yellow click road” referenced in the article leads to the man behind the curtain, not necessarily to the truth.
  • Make-believe friends aren’t as important as, you know, real friends: I completely agree with Mr. Baker’s assertion that “People let their guard down around personalities they trust.” I also have no doubt that social media influencers are growing in importance, a key point he makes. But again, here’s where the idea of looking exclusively at clickstream data to understand the consumer journey breaks down. If anyone had a vested interest in proving that relationship consultation occurs online, it would be Relationup, an app that provides 24/7 live relationship advice from professionals via chat (you know, Dear Abby for the next generation—and if you don’t know who Dear Abby is, keep it to yourself).  But they recently sponsored a research study, and it turns out that 80 percent of men sought out advice from close friends; 86 percent of females went the friends and family route. And my assumption is that not all of them are rocking a 75+ Klout score.
  • Just because the eyes see doesn’t mean the heart feels: Before I read Mr. Baker’s article, I’d never actually heard of Zoosk. And maybe good for them, their ads are targeted enough that they aren’t wasting their money on me. But if you looked just at my website behavior, you’d see that I fit their exact profile—I’ve now searched a lot of dating websites, and at least in my own mind I’m still younger and fitter. But it’s clear my motivations and those presupposed by my clicks are different. Just for fun, let’s hypothesize some other reasons why Zoosk users might be 11.4 times more likely to search for other dating websites:
    • Zoosk is terrible, so when people don’t find dates using Zoosk they have no other choice but search other dating sites.
    • Conversely, Zoosk is a hidden gem but not very well advertised, so it takes people a long time to find it.
    • Zoosk primarily appeals to people who are so desperate for a relationship that they sign up on multiple sites.
    • It has a name that only a spelling bee could love, so people type “eharmony” or “farmersonly” or “match” in their search engine before ever getting around to Zoosk.

I could go on with hypotheticals, but the point is this—without some way to actually talk to people, and surveys are one of the main tools for doing so, you may be able to understand WHAT people do, but you’ll never understand WHY.

No I get it, I really do. Jumpshot, Mr. Baker’s company, is in the business of “track[ing] more than 160 billion monthly clicks, helping marketers know what their customers are doing anywhere, anytime they’re online.”  Which is pretty awesome, and maybe 20% creepy (even acknowledging that participants have opted in). And I also get that writing opinionated articles helps you get them published, and Mr. Baker was much more measured in a recent speech when he said, “Surveys are incredibly important for understanding sentiment and intent but they’re not as good for understanding fact.”  (Click the link, scroll down to the video, and watch from about 3:52-3:58—or watch the whole thing, Mr. Baker’s an entertaining speaker). He even writes in another section of his article, “Ask followers’ opinions: Have they bought from you in the past? Did they order online or go to a physical store? What led them to choose you over a competitor?”  Which sounds suspiciously like an endorsement for the behavior that he impugned earlier in the article.

But here’s the deal. Long live surveys! They’re not perfect, but surveys provide information that’s very hard to acquire otherwise (and can be enhanced simply by knowing how to ask good questions). And the information obtained in surveys becomes that much more powerful when coupled with observational and transactional data—we’re talking about an “and” condition, not an “or” condition. Arguing about surveys versus big data is like arguing about peanut butter versus chocolate. Clearly, they are better together.

Maybe as researchers we should all be more like Zoosk users. It’s easy to believe at first glance that they’re looking for love in all the wrong places.  But in hindsight all they’re trying to do is improve their odds of success by leaving no stone unturned.