Feature Utilization

Identifying and tracking what is valuable in your software is one of they key tasks for a product manager.  I see two sides to feature utilization: market and internal.  Obviously, knowing how your product's features compare with your competitors is critical.  But I'm going to talk about the latter of those two: Internal feature analysis.  Essentially: What does your customer use and why?

From https://flic.kr/p/7yXRRH

From https://flic.kr/p/7yXRRH

As a product manager working in a startup, one of my biggest challenges is finding product fit.  Because we don’t have an established customer base and we aren’t simply reproducing a product that is already on the market, determining what features our product should have is a learning process.  As such, it’s important for me to see which features our customers like & use, and which they could do without.

I like to break feature utilization analysis up into two groups: Website Utilization & Application Utilization.  There is SO MUCH to say about website utilization I'm going to save that for another blog post.  Heck, there are books published on Google Analytics alone!   The discussions about heatmaps will simply have to wait for another time.  With that in mind, let's talk about application utilization.  

Tracking Basic Utilization

On the most basic level, anytime a user performs an action in your product you should log it in some way.  Whether it's accessing a report, importing a file, processing an order, etc...  Ideally this should be tracked and correlate back to the data involved.  For example, if someone ships an order from your system, you should process the shipment and then log the event in your feature utilization database.  This gives you the ability to trace a user's actions and activities.

Tracking Advanced Utilization

An added layer on top of basic utilization is the element of possible usages. Instead of just tracking "How many times did they click 'X'?", track "How many times did they click 'X' when they had the chance to do so?" Asking this question will clue you into some interesting data in your application.  In a previous role I had an export function that was never ever used.  It was located under a sub-menu and everyone on the product team thought it was logically placed there.  Unfortunately, when we finally looked at how often our customers expanded that particular sub-menu, we were surprised to see that none of them ever did.  We reorganized the menu a bit and saw feature utilization spike.

    Fig 1. Basic Feature Utilization

    Fig 1. Basic Feature Utilization

Application Utilization

On a basic level, it's helpful to just plot out feature utilization and see what parts of your application your users leverage the most (Fig 1).  When I first roll out a feature, this sort of analysis is useful to measure gross consumption.  However, while this gives you a basic view into user activity, it's lacking and is potentially misleading.  

Segmentation

User segmentation and cohort analysis are powerful tools and help you to delve into your users and their behavior. By getting more granular, you can discover surprising insights into your customers.  In the above example (Fig 1), it looks like Feature D isn't very popular, right?  If you look at Fig 2 though, you'll see that Feature D is used exclusively by one cohort.  If that cohort is of sufficient size, then it could make a great deal of sense to invest in it more.

    Fig 2. Feature Utilization Shown by Cohort

    Fig 2. Feature Utilization Shown by Cohort

The cohorts that work for a product are unique to that product, but here are some you can try: Age of user (group users together that signed up on the same day, week, or month), Type of user (either through an actual "type" or by intuiting their actions to infer a type), Geographic region (particularly interesting if your product has international appeal), or user persona (if you can identify it).  

In my current role I can readily associate user personas with users because of our permissions and role schemas.  As a result, we effectively have at least one version of user segmentation done.  However, even within that model, I find value in doing a second layer of segmentation based around age of user and onboarding experience.  So don't be satisfied with just one round of cohort analysis, try a few and see what works best for your team.

Tracking Journeys

Lastly, please realize that while basic utilization and segmentation are helpful, if you want to really understand your users, you have to look at their journeys and understand intent.  Instead of asking "how many times did users click 'X'", with journeys we ask "How many times did users do Y and Z instead of using X?" and "What else were users doing when they DID click 'X'?"  That's where journeys come in.  

For example, if your application lets a user create new teams and import users, it's important to track each feature (Create a new team, Import the users).  But it's also important to track all of the typical tasks a user goes through to accomplish this journey.  You may find that your users spend a lot of time updating user names after importing them, deleting the users and reimporting them because they left off some field, or some other "odd" behavior. 

Conclusion

When you're evaluating feature utilization, play with the data at various levels: get granular with cohort analysis/segmentation and go macro with journeys.  You never know where the data will lead you until you start digging into it.  Remember, your users and their behaviors can provide a wealth of insight into your product.  Make use of it!