Things I wish I knew when I became a founder: Analytics

Experience is what you get when you don’t get what you want. In this series, I’ll share the things I wish I knew when I first founded my company Invincible. I’m by no means an expert in everything but I know enough to be dangerous, and you can too.

You after reading this.

Getting Started with Analytics

I first heard the word “analytics” as a 22-year old, 1st year consultant at Accenture while on a project with a large bank in Chicago. I was asked to understand how users were engaging with the bank’s online system so we could better understand how to prioritize our efforts.

Real-time feed of me as a 1st year consultant getting tasked with analytics.

“Sure but ummm, what exactly are analytics?”

Analytics is making sense of data to help make decisions.

It wasn’t until I started building products that I really started to care about data: building a product is really fun, but what happens when you launch it? And it wasn’t until I launched our app with my own company that I really cared: every week without growth is an increased chance of joining the graveyard of failed startups (not to mention expensive).

This post is a deep-dive into getting comfortable with analytics for your startup / team / organization.

Learning how to use analytics — and why you should care

Even if you’ve raised some funding and can afford to hire some smart people, you need to know your numbers. Reality will almost always be worse than what you think, especially in the beginning.

In bigger organizations, analytics often left to a select few “data people”. Smart people in the room ask lots of questions but have no way (or interest) in self-serving. This leads to great job security for analysts but slow, and usually uninformed, decisions.

I’ve found the best way to get comfortable with data is to just…use data. Analytics helps identify potential issues, show where your product is working (or not), and provides insights that users never can (I can’t tell you how many people have told me what a great idea my app is only to never even sign up).

Designing & Building Your Analytics

If you’ve already got an existing analytics toolset implemented, skip ahead to the next section, smarty pants. If not, carry on.

Note: Most of the steps ahead will show Amplitude-specific information (especially screenshots) but are generalizable to most other analytics toolsets.

Picking the right tool

There are lots of off-the-shelf tools available to build app analytics. My first product engagement dashboard was built with a combination of spreadsheets and Google Data Studio. Whatever works for you.

However, I realized we needed something more, and I wanted something that could easily show user engagement but also didn’t break the bank.

A few of the many potential options from this useful article here.

I elected to use Amplitude. The platform is super robust, pretty user-friendly, and I still don’t pay a dime. They also offer great support with Expo (how we deploy our app), which makes life a lot easier.

Defining your data dictionary

This is probably the most important step of all. There are the key events that happen within your product you want to understand, including:

• When users create an account
• When users log in / log out
• When users access key functionality in the product

Every product is different, so you’ll need to put some thought into what you care about tracking. DO NOT worry about tracking everything and getting every detail perfect (you won’t). DO figure out what you care about most for your product and your users and define a specific set of actions based on the user flow.

For my company, I defined 6 high-level categories of events and a series of events within them that would help us better understand user behavior within the app, including

General Events:

• *Onboarding (e.g., create / verify Account)
• *General (e.g., app open / close)
• *Account (e.g., login/logout).

Specific Events:

• Training (e.g., starting a module, completing a module)
• Team Management (e.g., creating / joining teams)
• Chat (e.g., viewing, sending messages)

The result. Amplitude provides a nice template for doing as part of their Quick Start Guide here.

Implementing analytics in your codebase

Amplitude offers an excellent Quick Start Guide to quickly get up and running that goes into much more detail than I will in this post. It’s easy enough that I was able to do our initial implementation (as a non-coder but through lots of trial and error).

Here are a few things you’ll be doing when you integrate:

• Install in your codebase: The Invincible App is built using Expo.io, which has an API for Amplitude that makes it really easy to get up and running. Many tools will likely offer this level of support which will greatly reduce implementation cost and troubleshooting.
• Configure Events: Once installed, you then breadcrumb the app with specific commands at the identified trigger event points. This include triggering for the event itself and also adding specific properties relevant to the event itself that can later be used in analysis.
• Turn off the Creepy Stuff: Our goal for implementing analytics was to allow us to learn from and support our users, not to track them. We ended up disabling specific tracking mechanisms like IP address.
• Test: It takes a good amount of debugging to get this up and running.

Voila: Data flowing in Amplitude

Making data work for you

Making the data useful

Now you have data coming in, you’ll want to build some charts to make sense of it. Don’t worry about perfection, just start with some open-ended questions and start building (Amplitude also provides starter templates that will give you some ideas).

For Invincible, there are 3 key reasons I use data:

1. Health of the business: (Daily Active Users, Sign-Ups, Retention, Active School District Partners)
2. Product engagement (# unique users / school districts, daily communications by type)
3. Troubleshooting bugs and other issues

Here’s a simple example using our setup in Amplitude. For our app, viewing and sending messages are critical value drivers for the families and school users we support. With analytics, we can answer questions like: how many users engage per day? How many messages are being sent? How long do users spend in the app?

There are many chart views available but most commonly I find myself using the “Segmentation” chart to answer these types of questions. Configuring the chart is pretty straightforward once you understand the data that is available.

After you build and interact with a few charts like this, you’ll soon find which ones you go back to day after day and have a dashboard that provides useful data and insights.

I spend most of my time looking at these top-level dashboards. At a quick glance, they tell me how many users are returning each day and how much each of these users are engaging with the product.

Becoming data-driven

All models are wrong but some are useful (George Box)

While it’s fun to watch data in real-time (and believe me, I check Amplitude more often than I care to admit), for me what’s even more exciting is identifying what’s working and what’s not so you can improve and hone in on the value you’re providing.

For example, one of the most important things you will do with your data is to figure out what separates users who download and stick around vs. leave. This early moment is called an “aha moment” and is what separates great products from products you’ve never heard about.

An Aha moment is where your users realize the value of your product, so if you can effectively push them to this point and if your product offers real value, you’ll have users that are in love with your product.

For Invincible, in-app communication is the initial moment where users experience the value the app provides. The chart below shows users who join a “team” (meaning they are part of a group that communicates about student care) vs. those that don’t: 43% of users who join a team stick around for 4 months, compared to less than 5% of users who don’t join a team.

Joining an Invincible team is a significant predictor of long-term engagement

With this data, one action this led to was product improvements to get more users into a team. We ended up releasing 2 new features to do just that — one to make it easier to join a team (we used to require a 6-digit code) and the other to make the goal more obvious during onboarding.

New features to help get users to their aha moment.

Other Data Stuff to Keep in Mind

• Start at the top (of funnel): In my experience as a PM, all products have a top of funnel problem (especially early stage ones). It doesn’t matter how good your sales and marketing teams are if users who come in the door don’t stick around long enough to actually get any value. {Try: Funnel charts as a way to see how you’re performing}
• Mix in qualitative data: Data will almost never tell you user motivations or provide context behind why users do what they do. I recommend supplementing your analytics with plenty of user research to validate hypotheses and continually learn {Try: Interview potential and active/churned users to find what’s working}
• Make data visible: Not everyone in the organization is willing and able to dive into the data. But finding a way to make data (and problems) visible will ensure everyone is aligned toward shared results {Try: Sharing data at team meetings on a regular cadence}
• Iterate: It can be easy to launch your initial analytics and then forget about them. But your initial implementation will almost always be wrong and definitely won’t tell you how new functionality is performing. {Try: Building data into your user story backlog}
• Trust your gut: Not every decision can (or should) be based on data. Sometimes you need to take a leap based on your gut and just go for it. {Try: Knowing your users better than they know themselves}
• Experiment: As you get better with data, you’ll start experimenting with new features. {Try: Amplitude Experiments}

Now What

Seriously, just go for it. If you’ve got existing dashboards, you can start now. If you’re starting from scratch, it will take a bit longer but nothing more than a couple of days. Before you know it you’ll be cruising.

If I can be helpful, drop me a message at bob@invincibleapp.com.