A Proven Formula for More Accurate and Actionable Insights in Quantitative Market Research
By Ken Donaven and Chelsea May
There are certainly situations and use cases in which either quantitative or qualitative research, as a stand-alone methodology, is the more appropriate research vehicle. But where quantitative research is concerned, we usually recommend an approach that “book-ends” the quantitative research instrument with two qualitative research exercises.
This strategy, which we’ll dub “Qual-Quant-Qual,” tends to yield deeper insights and richer context, resulting in intelligence that is both more accurate and more actionable.
Those who follow this methodology have become advocates for the approach going forward. As a Market Insights Manager for an industrial products manufacturer recently relayed to us, “Having qualitative feedback on both ends of the quantitative data helps us turn questions into answers much faster than quantitative data alone.”
Here’s how it works.
Phase One: Qualitative | Design + Discover
A frequent truism expressed when undertaking a quantitative research endeavor is, “We don’t know what we don’t know.” This is especially true when brands are entering new markets, new geographies, or even unfamiliar demographics — “the next generation of buyer,” for example. While at face value this may be considered an admission of ignorance, in reality it is a very sage concession, and often is critical to the design of the survey to come.
There is a danger in making assumptions about the respondents one is preparing to canvas. Mistakes happen when incorrect presumptions are made about the buyer, the user, or the decision-making influencer. Some companies lack a true understanding of the buyer of their products…others misidentify the motivations or desires of the end-user. Sometimes, audience personas are misidentified altogether, and the survey is not addressing the correct set of respondents.
In other instances, there are blind spots in the design of the survey language, which can misalign with the accepted and natural vocabulary or vernacular of the desired respondent. If you’re not speaking the “language” of the market when conducting your survey, your product or brand may run the risk of being considered an “interloper,” which could negatively impact how a respondent participates in the survey.
For these reasons and others, we find it crucial to conduct preliminary qualitative research with potential respondents to identify and account for such blind spots, and to avoid critical disconnects between desired outcomes and potentially misleading response data. The best way to do this, in our view, is to have direct, in-depth conversations with as many potential respondent personas as possible, working diligently to either validate or refute stakeholder assumptions. We may also consider the “tribal knowledge” gained from a client’s internal stakeholders, especially as a headstart to determine who we need to talk to, what we need to learn, and how best to design the survey instrument. These steps help ensure that we authentically capture the true voice of the customer or respondent pool. But we prefer not to rely on those internal inputs alone.
Because brands are often exploring unknowns, it’s generally not the case that a client can simply hand us a call list of existing contacts. This is why, as a company, we have committed to a proven methodology to gain access to the correct respondent pools — especially difficult-to-reach audiences. In addition to the obvious (buyers and users), it is critical to consider the inputs of what might otherwise be omitted, such as distributors, purchasing agents, and other third parties with insights germane to the study being conducted.
Phase Two: Quantitative | Read + React
Once the requisite work has been done to “discover and define” the knowns and unknowns relevant to the study, the survey instrument can be finalized and deployed among appropriate respondents.
Two things to consider during the survey phase that we find critical:
- Data Integrity: As we illustrated in a prior article, quality assurance is of the utmost importance when conducting quantitative research studies — now more than ever, in fact. We stressed in that piece and will reiterate here: “While AI tools show promise and can be exciting in terms of what the future may hold, we believe that the best intelligence is found when artificial intelligence combines with human intelligence to manifest ‘augmented intelligence.’ Human oversight is still critical — probably more so today than ever before, as counterfeit survey respondents proliferate.”
If you missed that piece, take a look at four of the most common bad-actors that can compromise the quality of survey data, and the approaches you should take to prevent them from erroneously influencing the results of your research. - So What and Now What? It is not in our DNA to simply hand over survey results to a client without providing some level of analysis and context. In fact, many companies conducting research will confess to something of a “paralysis” without proper “analysis,” as numbers themselves cannot tell a complete story nor provide direction on how to turn insights into decisive and strategic action. There are often a great many WHYs behind the stated WHATs that a set of survey results reveal. And that is where the true power of the qual-quant-qual methodology lies.
To maximize the power of these deep insights, it is our belief that best practice necessitates the final phase: additional qualitative research.
Phase Three: Qualitative | Sharpen + Activate
When we remind ourselves that a great many research studies begin with a set of unknowns (as well as sometimes incorrect assumptions), it should come as no surprise that survey results often do come as some surprise to the sponsors of the study. Whether the results are surprising or not, in nearly all projects, questions arise regarding how to react and respond to the intelligence gathered.
Clients may want to pursue certain questions or issues more deeply based on the survey findings and responses. Why did this data point or trend reveal itself as important to the respondent? What is motivating certain recurring opinions? How would a respondent persona want a given desire or preference to be addressed by a product or service provider?
This richer context and deeper analysis can best be achieved by having additional meaningful, in-depth, dynamic conversations with specifically identified constituents. The questions that often come from the research study sponsor — whether surprised or not — are often best answered by the target audience themselves. Just as there exists a danger in making unfounded assumptions in the survey design phase, it is equally risky to presume latent motivators and demotivators behind a multiple choice response (or even an open-ended answer).
Slow Down to Speed Up; Learn More to Know More
In a very real sense, it is the initial qual that informs and validates the quant, and it is the final qual that validates and clarifies the quant. Once those embarking on market research experience the benefits gleaned from this Qual-Quant-Qual methodology, they almost never return to doing quantitative research in a vacuum, without book-end qualitative components.
As that same Market Insights Manager said, “The biggest struggle we have with quantitative studies is appropriate context. Sandwiching the qualitative methodology around the quantitative study helps reduce the context gap we would otherwise experience. Starting with hearing open-ended feedback directly from customers informs what and how we need to be asking things in the quantitative stage, and using the qualitative process after a survey helps remove any doubts as to what a customer was thinking when providing specific feedback.”
Such an approach is useful whether a company is entering a situation with an unknown set of unknowns or, conversely, those who hold strong hypotheses about given markets and buying personas. Whether you don’t know what you don’t know or you think you know everything there is to know, the reality is usually some number of unknowns that need to be discovered, defined and accounted for.
As Mike Vance, known as The Dean of Creative Thinking, once said, “Slowing down is sometimes the best way to speed up.” There is more at risk in getting it wrong quickly than there is in taking a measured approach to getting things right the first time. The same could be said of cost — there is a greater cost associated with going to market with incorrect data than there is charging confidently ahead with validated and meaningful insights in your competitive arsenal.
Ken Donaven serves as Senior Director with Martec, and Chelsea May serves as Project Manager.
