Building a Data-Obsessed Culture: Why 90% of Experiments Fail (And Why That's OK)

Want the full story? Listen to my complete interview with Sundar Swamanathan, where he shares his journey from managing $19 trillion at the US Treasury to leading Uber’s brand data science team.

Here’s a shocking truth about data-driven companies: 90% of their experiments fail.

That’s not a typo. In my recent conversation with Sundar Swamanathan, former head of US/Canada performance marketing analytics at Uber, he shared that this is actually the norm at truly data-obsessed companies. The difference? They see these failures as victories.

But let’s back up. What does it actually mean to build a data-obsessed culture? And how can companies embrace failure while still driving growth?

The Truth About Data Culture

“I truly think it has to be founder-led,” Sundar told me. “A lot of it is because data-driven data obsession, whatever you want to call it, what it results in is an ego-less decision making.”

This might sound abstract, but Sundar shared a concrete example that demonstrates exactly what this looks like in practice. As head of US/Canada performance marketing analytics, he noticed something odd in their Meta (Facebook) advertising data: their customer acquisition costs were swinging wildly even though spend remained constant.

Digging deeper into Uber’s saturation analysis, he realized their marketing might not be as effective as they thought. So what did he do? He proposed completely shutting off their Facebook advertising - a $35 million annual budget - for a three-month incrementality test.

In many companies, this would be career suicide. But at Uber? “There was not an ounce of resistance,” Sundar said. “I mean, there was hesitation, there is caution, right? Because it’s a big decision, but everything from my manager to my boss, to the CMO at the time… [said] let’s go try it out.”

The test proved the spend was non-incremental, and they turned off a $35 million marketing channel. This wasn’t just about the money saved - it demonstrated how deeply data-driven decision making was embedded in the culture.

Why Most Companies Get Experimentation Wrong

The challenge most companies face isn’t technical - it’s psychological. As Sundar explains, “90% of experiments fail… What that means is you have to be in such a psychologically safe environment that allows you to fail 90% of the time and people still think that’s a good thing.”

This is where most companies stumble. They say they want to be data-driven, but they haven’t built the cultural foundation that allows for failure. They haven’t internalized that experimentation is about learning, not winning.

“You’re basically saying I could be really bad at my job 90% of the time, but that actually makes me good at my job,” Sundar points out. “And that’s actually kind of true with experimentation.”

The Seven-Step Framework for Experimentation

Through his experience at Uber and now advising consumer tech startups, Sundar has developed a seven-step framework for building effective experimentation processes:

1. Start with customer insights - Your job isn’t to prove yourself right; it’s to solve customer problems.
2. Generate multiple hypotheses - Don’t get attached to any single idea.
3. Prioritize experiments - Not all tests are equally valuable.
4. Design rigorous experiments - The scientific method matters.
5. Ensure quality data collection - Garbage in, garbage out.
6. Conduct thorough analysis - Look beyond just whether it “worked.”
7. Create a perpetual cycle - Use learnings to generate new hypotheses.

The key is making this a continuous cycle: idea, experiment, analyze, repeat.

Tools Don’t Matter (As Much As You Think)

One of the most surprising revelations from our conversation was about tools. Despite Uber’s reputation for sophisticated technology, Sundar revealed that for his entire five-year tenure, he primarily used just two tools: SQL and Google Sheets.

“Bankers who are making multi-billion dollar mergers and acquisition deals are just using Excel,” he points out. “That’s enough for them.”

This doesn’t mean tools are irrelevant, but it does suggest that the obsession with finding the perfect tool often misses the point. The magic isn’t in the software - it’s in the process and the culture.

How to Start Building Your Data Culture

If you’re inspired to build a more data-driven culture at your company, here’s where to start:

1. Focus on process over results - Don’t promise success; promise learning.
2. Build psychological safety - Make it clear that failed experiments are valuable.
3. Start small - Begin with one clean experiment to test your process.
4. Use failures to generate ideas - Each “failed” experiment should spawn 3-4 new hypotheses.

Most importantly, remember what Sundar emphasizes: “We don’t need to prove experimentation works. First off, as humans, we’ve been doing it for hundreds of thousands of years, and I think we’re in a pretty good place with that… What you need to prove is do we have the right culture to make experimentation work at our company?”

Want to hear more insights from Sundar about building data-driven cultures, measuring marketing ROI, and learning from Uber’s experimentation framework? Listen to the full interview here.