Why “data-driven” isn’t enough

Businesses that truly value a solid user experience should be “evidence-based” rather than “data-driven.”

Photo by UX Indonesia on Unsplash

The other day a very simple post from author Simon Sinek showed up in my LinkedIn feed. It simply said:

It got me thinking about how frequently I see companies looking for UX and product designers to help build and grow products that do very specific things for very specific groups of people (i.e., analytics products). One thing that often stands out to me is how these products and companies seem to be more geared towards machines than people.

These days, at least half of the job postings I come across include the phrase “data-driven.”

“We’re passionate about data!”

“Data is at the center of everything we do!”

“Man, we just can’t get enough of all this data!”

“How many licks does it take to get to the center of a Tootsie Roll Tootsie Pop? DATAAAAAA!!” (A little something for all you Gen-Xers out there.)

Look, I understand what they’re trying to convey: that they make decisions based on hard facts rather than feelings or whims. It’s a distinction that matters, and I appreciate that organizations have increasingly started asking “Why?” when it comes to product strategy.

I get why data is a big deal. It’s powerful and relatively easy to collect. With the right software, we can gather massive amounts of data quickly, in ways that would take a human team weeks or months to accomplish — and, of course, without the cost of salaries.

Once we have this data, software can neatly dissect and categorize it, providing us with insights into user behavior, preferences, and patterns. This information is undeniably valuable and can help us make more informed decisions.

But there’s a barrier — a gap between the data we collect and the deeper meaning we try to derive from it. We can see some of what the data is telling us, but as the Nielsen Norman Group points out in this article on research methods, it’s better-suited to answering questions of “how much,” “how many,” and “how often.”

Product Consultant Saeed Khan published an in-depth article on Bootcamp two years ago entitled “Make ‘Evidence-Based’ Decisions, Not ‘Data-Based’ Decisions,” where he discusses a similar distinction. We differ slightly in our approach, but generally agree on the point:

We can do better than hinging our decision-making solely on “data.”

“Evidence” is a far more comprehensive — and frankly, better — word. All data is a form of evidence, but not all evidence is limited to data points that software can capture in an instant. Truly understanding users requires human connection.

Sometimes this connection is one-way — through reading about human behavior or watching a user interact with an app. Other times, it’s two-way, like when user researchers conduct focus groups or interviews. These interactions allow us to pick up on nuances and cues that no software can capture. Even intelligence agencies, with all their advanced tech, still rely on “HUMINT” (human intelligence) to fill in the detail gaps that machines cannot. These qualitative measures can help us answer questions like “what do our users do?” or “what do our users say?” Pair these together with the quantitative data, and we can get to the “Why.” When you have your “why,” you know which direction to proceed.

Yet, time and again, we see that when company budgets get tight, research is often among the first functions to be cut. It happened during the Dot-Com bust of the early 2000s, again in the 2008 financial crisis, and once more in the early months of the COVID-19 pandemic. This pattern shows that, unfortunately, many companies still view UX research as a luxury rather than a necessity.

I’m not saying data isn’t important — it absolutely is. In practice, data and evidence should coexist — a “Yes-And” scenario.

As I mentioned, data can tell us a lot about what’s happening, but it can’t tell us why. We need humans who can take that information, combine it with empathy and an understanding of human psychology, and interpret it to reveal its deeper meaning. That’s where “data” becomes fully incorporated into “evidence.”

Data is based in action, but evidence, much like Zen Buddhism, is based in experience. It encompasses far more than what a user does; it also tries to measure how they think and feel about what they do. We can get a much fuller picture by studying the whole user rather than just their actions.

As stated in this article published by the Oxford Review,

“When people in an organisation understand this distinction between data and evidence, we tend to find evidence-based practice not only starts to become real, but also it enables people to make better decisions and judgements, and you may be surprised to find that people also become much more adaptable and flexible as well.”

Words matter. When I hear a company say “data-driven,” it often sounds like, “We’ve automated our research process and just need someone to interpret it all into a good user experience.” At worst, it comes across as, “We’re too cheap to hire an experienced UX researcher.”

On the other hand, “evidence-based” suggests they value all types of information — including those messy, hard-to-quantify insights that don’t fit neatly into a box or CSV file. When it comes down to it, a human will always be the one using your product, not a machine. Therefore it behooves us to always remember the person sitting on the other side of the computer screen.

As a UX and product designer who leads with empathy, I always try to think about the people who will be using whatever it is I’m creating. I constantly ask questions like, “Will they find this useful?” “Will this make them feel less stressed?” “Will this help them be more efficient?” These are questions that data alone simply cannot answer, and was never meant to answer.

We need to rethink our language. It’s time to move from being “data-driven” to being truly “evidence-based.” Because the best decisions aren’t made by machines alone — they’re made when human insight and empathy meet data, forming a complete picture.