Define ‘Normal.’ The Problem with Norms

We’ve all been there: that moment when study results come back and someone on the team asks, “Do we have any industry norms to compare our performance to?” During my 15 years as a researcher at major pharmaceutical companies, I often dreaded that question – and the subsequent discussion. Not much has changed now that I work on the supplier side of the business.

In order to make a judgment, we often require context, a sufficient framing of what differing levels of success or failure look like. So I agree we should look for relevant comparison points to guide our judgment, but I would argue industry norms are very rarely the right set of comparisons.

The issue with norms is that there are too many variables behind the benchmark. Let’s consider in pharmaceutical research when a company launches a new product and wants to track its awareness in the marketplace as the product rolls out. An industry norm value for awareness might include drugs from different therapeutic areas, with first-to-market drugs and the followers, brands with big dollar spends and small dollar spends. By the time you cut the data to get norms that are “actionable” and realistic – you no longer have norms. You may have one or two studies to compare to and those may or may not help you understand where your product sits on the success continuum.

As a pharma executive, and now as a consultant, I stopped looking to norms to find the right framing context to answer the “What does success look like?” question, and started a different thought process:

1. Start with the end in mind. Long before data gets back, and the marketing team is asking 101 questions, think about what those questions will be. Make sure you’re asking the right questions, and have the right analytical plan. If there is a specific question that you know has been asked a certain way for other brands, include it verbatim in your study.
2. Think about the framing and context of your senior stakeholders. What do they have in mind? The reality is that for most of the people you’re sharing your data with, they have no clue what the “norms” are. That’s why they’re asking! But they do have something in mind that they relate to “success.” Was it the most recent internal brand that launched, so those numbers are fresh in their mind?  Is it the campaign they worked on three years ago for a “Blockbuster” drug in your portfolio?
3. Look to your other company data sets to find context. Take a look at other brands in your portfolio. Discuss with the team what those brands achieved and in what time frame. Why do you think your campaign or brand or whatever you’re measuring would perform better or worse? Set a goal with that information, and compare against the goal. Create a slide with those other brands from within your company – and compare to that.

By adding internal, palatable context, I think you’ll find your results to be much more actionable and realistic than a set of vague and unspecific “industry norms.”  And as an added bonus – in many cases you’ll have access to the “experts” within your own company to help you understand those contextual numbers – and any nuances that they include.