Small Data vs Big Data - Panel Q&A

A few folks thought this Small Data panel was helpful, so here's my portion. Hopefully it sparks other ideas or discussion. My hypothesis is a great many of us are hitting a point in marketing evolution where we've consumed big data, found it to otherwise not be exactly as expected, and now we're drawing back to the fundamentals before we make a giant leap forward.

Here's my take.

(Thanks to Valerie McCubbin who moderated and asked these questions, and fellow panelists Martha Gonzalez Gorgonio and Don Lane)

What is the difference between big data and little data? 

People understand small data. Only machines (and maybe some data scientists) understand big data. But the data scientists I know are more machine than human, so this statement is still technically true.

But more seriously, my definition of small data is simply "uncomplicated data," or specifically, "foundational data." The deviation between big and small is in the application.

The application of small data is a focus on fundamentals. The application of big data is to build on foundational elements to gain complex understanding and action. I've been guilty of jumping to big data models before ensuring our foundational elements were sound, and it rarely ended well.

What are the rules around when to use little data vs big data?

I believe in the Lowest Hanging Fruit philosophy, which doesn’t sound exciting, but it maximizes return on effort. A rule I have is to exhaust small data until I understand exactly:

1) What data exists

2) How it connects and flows across systems (bi-directional always being the goal), and

3) How that data facilitates customer engagement 

We do this first before building algorithms and machine learning. Or depending on where we are with fundamentals, we concurrently build small/big models, but small always leads.

Personal litmus test: Take a piece of paper and physically draw a basic dataflow chart starting with where your customer information is stored and how it’s accessed. When I realized I wasn’t able to do this, I realized I needed to go back to Step 1.

Philosophical litmus test: How effective is the smartest attribution algorithm in the world if the data it measures is inconsistent, duplicated, missing, and/or unreliable? I've witnessed hundreds of thousands of dollars invested* on attribution modeling, when in hindsight our core customer data wasn't even connected.

*wasted

When should marketers be looking at fewer data points, rather than more?

When I wouldn't bet my career on my understanding of fundamentals. I have to understand: What customer information do I have? How do the marketing systems I employ collect, store, share, and use that data? What about the rest of the company? (POS, sales teams, call center, etc)

Big data models are only as good as its inputs—accuracy and consistency of the data itself, and our understanding of the data and it's context. My success in building big data models is predicated on my understanding the inputs.

Can you share examples of how little data has told a big story within your company? 

Small data was born out of necessity. My default was always big data. 

Ever since we linked Salesforce subscriber IDs with GA client IDs (which everyone should be able to easily do soon as a built-in feature) we gained an integrated view of our customers across our organization. We’re able to understand how many engagements our customers have with us and where, which can help build things like: 

1) A Universal Engagement Cap. Impression capping in Display is a known thing. We can now throttle down in any or all channels based on total engagements with our brand.

2) Universal Negation. If someone calls our call center and has a bad experience, we can negate them from all marketing to not be annoying.

3) Attribution!

Small things like connecting IDs have given us a strong foundation towards engagement automation, which is more than just marketing. Like sending a sales rep with a programmatic email of, "Hey, talk with these customers today about [insert product category or interest topic], and here's some content [built via a linguistic relevancy model] you should read to be more informed and helpful to them. Also here's other content you can send them as a follow up to your conversation to help them more" because based on specific customer engagements with us, we identified signals that hint at them needing help.

This goes without saying, but we need to be exceptionally respectful of PII to be helpful but never annoying nor creepy.

What’s one thing you wish you knew now, that you didn’t know then about little data? Words of wisdom.

I wish I knew how easy and impactful small data was, but also how personally rewarding. An astonishing discovery for me was that big data disconnected me from our customers, whereas our small data approach forced me to better understand them and their needs. The information we looked at moved from ethereal to tangible. What data is available and worthwhile, what do we actually understand about customers, and how do customers want us to engage with them in a way that's meaningful--being helpful to customers, being fiscally responsible, and scalable to any sized organization.

###

Some context: My personal belief is that marketing should primarily serve to help customers, always. That is, customers buying our products or services should be a result of our helpfulness, or our ability to entertain, educate, or otherwise enrich people's lives (my cheesy "3 Es"). Maybe that's naive, but I wouldn't want to work at a place that cares only about selling without improving the well-being of customers in some way. I'm sure others feel the same.