You don't need data for data-driven development

Everyone I've met in big business over-estimates the value of quantitative data, and under-estimates intuition. When the people trying to intuit product development sit too far from the user, intuition is useless too.

You asked me whether we should implement FullStory or some other user tracking software in our app when we’re just getting started. It’s a great question and one that comes up a lot in the early stages of product development. So, I wanted to share some thoughts on why you don’t always need a ton of data to make data-driven decisions and how a tight feedback loop can often be more valuable.

Apply a Minimalist framework to Modern Product Discovery

When you’re just getting started, it can be tempting to dive into complex data analytics tools and user tracking software. They’re expensive, in both system resources and pricing but here’s the thing: minimalist modern product discovery doesn’t always require massive amounts of data. Instead, fostering a culture of tight feedback loops between engineers, leaders, customer support, and customers can be incredibly effective. Leaving breathing room in your planning to adapt to up to the minute feedback loops will do more for you than trying to unravel the number 1,345 users click data-target:3iIod… every time.

Data belays the human elements of a software product that drives jobs to be done. You can get what you’re looking for with those tools, definately. But you can also get them easier by structuring your team and processes.

The Power of Intuition and Qualitative Data

In my experience, having over a decade in the software industry and specializing in product management and SaaS business development, I’ve found that intuition and qualitative data can lead to powerful insights. With Realbase I fought tooth and nail to get analytics and user metric tools installed and you know what I found, they very rarely provided insights we didn’t already have, they became a backstop/confirmation that were sometimes valuable to argue a point, but rarely resulted in action or development vs. the broader market requirements we reacted to. Instead, we emphasized close communication with our users and business owners.

Building a Feedback Loop

Here’s how you can create a tight feedback loop in your team:

1. Regular Touchpoints: Set up regular meetings between engineers, product leaders, and customer support teams. This ensures that everyone is on the same page and can share their insights. If possible, giving the engineers visibility over the customer support tickets, or atleast the filtered up high-quantity tickets can be hugely valuable. With the caveat that by opening up this can of worms, you’ll also need to train and re-train your engineers on fierce prioritisation, so they don’t get pulled into the orbit of putting out fires, and instead use their magical powers for systemic improvements to broad customer problems.

2. Customer Conversations: Encourage direct conversations with customers. Whether it’s through interviews, surveys, or seeing support tickets, getting firsthand feedback can be more enlightening than data points, where possible having a developer try and do the job the customer is attempting in the platform will usually find you a bunch of ‘easy wins’ that a qualified PM or Engineering lead wouldn’t necessarily be able to find.

3. Intuitive Decision-Making: Don’t be afraid to trust your gut. If you’ve been talking to customers and observing their behavior, your intuition can be way more informative than a spreadsheet or analytics dashboard.

Every observable behaviour these tools open up, not only has to be correctly configured for and massaged out of the data, it can always be observed in other ways, don’t forget to look for this.

The Risks of Relying Solely on Data

While data is undoubtedly valuable, relying solely on it can be risky, especially in the early stages. Here are a few reasons why:

1. Data Overload: Too much data can overwhelm you and lead to analysis paralysis. You might spend more time crunching numbers than actually building and improving your product.

2. Missing the Human Element: Data can tell you what users are doing, but it often doesn’t explain why they’re doing it. Qualitative insights from conversations or sitting and watching a session can fill in these gaps. Yes, the tools can enable you to watch an exact session, but a session where you’re sitting with an actual user, in person or remote will give you the opportunity to ask questions you don’t get with passive viewing.

3. Slow Iteration: In a startup environment, speed is crucial, it’s the lifeblood of inertia, the feeding momentum of growth and it’s value absolutely can’t be overstated. Waiting for significant data to accumulate before making decisions can slow you down, especially when you don’t have that many users. Quick iterations based on qualitative feedback can help you move faster.

From Startup to Scale-up

Transitioning from a startup to a big business involves growth strategies, scaling challenges, team building, maybe even funding rounds, operational efficiency, and strategic planning. Even at scale, the principles of tight feedback loops and intuitive decision-making remain relevant, and they’ll help you fight off distractions.

The number one rule, is make products the customers enjoy using and will pay for. Most corporate level strategy, VC ideas, product strategy sessions will in some way shape or form distract from this need. And unless the data you’re getting from these tools is helping you block out the noise of the BS and you need data backed answers for why you’re not doing what people who don’t talk to the customers think you should be doing, they’re not necessary.

Trying to keep the agility and tight feedback loops during scale up will probably be the hardest thing you have to do, and “Data” platforms like this will make it very easy to take your finger off the pulse and feel like you’re doing the right thing by customers, it’s the easy way out, and it’s usually going to take you in weird directions, or waste time.

Remember, every corporate led, manager heavy business wishes they could react to customer needs as fast as a startup, it’s why they’re always playing defense, and you need to do the things they can’t do to take their market share.

Final Thoughts

So, should you implement FullStory or similar user tracking software right now? Probably not yet. Maybe not ever. Start with building a strong feedback loop within your team and with your customers. Trust your intuition and use qualitative data to guide your product discovery. As you grow and scale, you can layer in more sophisticated data tools to complement your existing insights.

If you’re finding the team isn’t as good at finding the key insights that you’d like, the problem is you and how you’ve trained them. Buying another tool is a bandaid and a dangerous one at that. You’ll feel like you’re doing the right thing, but realistically you’re pouring gasoline on the companies devotion to critical thinking, then holding the match for anyone throw on it at any time they wish.

Remember, data-driven development doesn’t always mean more data. It’s about making informed decisions, and sometimes the best insights come from simply listening and observing.

Hope this helps!

Cheers,
Darrell