Statistics is a science, but there’s an art to it, too. We use both to provide a clear picture of what the data really say.

There are two main types of research surveys — probability and nonprobability. Probability surveys, like those AmeriSpeak conducts, provide findings you can trust. That’s because they survey carefully created panels of respondents who accurately represent the population as a whole. The results of nonprobability surveys—which are typically conducted online—are less reliable because their panels aren’t carefully assembled or fully representative. But businesses often use nonprobability surveys for market research because they’re much cheaper. We’ve developed a statistical process that corrects the errors inherent in nonprobability surveys, bringing their results within the same margins of error as the gold standard AmeriSpeak survey.

From
To
AmeriSpeak sends a survey to its online panel, which is selected to replicate the U.S. Census, including all hard-to-reach groups.
Because they are based on a true subset of the country, the findings become the baseline for less accurate nonprobability surveys.
Because their price points are so low, we order supplemental nonprobability surveys, knowing, however, that they yield substandard results.
TrueNorth’s machine learning algorithm salvages the nonprobability survey results by aligning them with the baseline data from the AmeriSpeak survey.
Our research produces two sets of findings that might vary widely, reducing their value as a guide in setting market strategies.
After fine-tuning the nonprobability survey results, we merge them with the baseline findings to produce a single trustworthy dataset.
From
AmeriSpeak sends a survey to its online panel, which is selected to replicate the U.S. Census, including all hard-to-reach groups.
To
Because they are based on a true subset of the country, the findings become the baseline for less accurate nonprobability surveys.
From
Because their price points are so low, we order supplemental nonprobability surveys, knowing, however, that they yield substandard results.
To
TrueNorth’s machine learning algorithm salvages the nonprobability survey results by aligning them with the baseline data from the AmeriSpeak survey.
From
Our research produces two sets of findings that might vary widely, reducing their value as a guide in setting market strategies.
To
After fine-tuning the nonprobability survey results, we merge them with the baseline findings to produce a single trustworthy dataset.
Slide 1 of 3

The Survey-Taker’s Challenge

We have to get things right every step of the way.

How TrueNorth Solves Challenges

1. Start with AmeriSpeak Probability Panel. These high-quality surveys are taken by AmeriSpeak respondents who are chosen and recruited specifically to be fully representative of the U.S. population. 2. Add Non-probability Panel (Opt-In). These lower quality surveys are taken by groups of respondents who opt-in and don't fully represent the country, resulting in under and overrepresentation of certain groups of people. 3. Run Panel(s) through TrueNorth Calibration Tool. We then calibrate the nonprobability survey data to correct for bias and merge it with the high-quality survey findings. The Outcome: Data you can rely on.
See it In Action

FAQs

Opt-in panels—also called convenience, commercial, internet or nonprobability panels—have been studied intensely in the past decade. Most first-tier, peer-reviewed evaluations of opt-in panels conclude these panels do not produce consistently accurate information and often lead to acutely inaccurate findings. Using such panels is like playing Russian roulette: You will never know for any given benchmark whether it is accurate or not. This might be acceptable for less important research, but not research meant to inform major business or policy decisions. TrueNorth is specifically designed to provide a kind of “research insurance” for this “variance of error” that can make opt-in panel surveys inaccurate.

TrueNorth blends opt-in data with high quality probability data from NORC’s probability-based AmeriSpeak® Panel. It uses highly sophisticated machine learning to combine the samples in a way that minimizes any bias in the opt-in sample, question by question.

TrueNorth takes a bit more time than a simple opt-in sample. This is due to needing to attain a high response rate probability sample and the statistical work needed to combine the samples. Overall a TrueNorth project can take as little as two weeks, but four to five is typical.

The key to all survey representation is randomization. Probability panels essentially stir all addresses together and then sample the mix. Consider a bowl of tomato soup, all the ingredients are mixed together so that every spoonful is the same as the next. Opt-in samples allow anyone to join, and typically reach out to members of partnering organizations. For example, an opt-in survey may invite every member of an airline’s miles program. The thinking is that if you simply add enough people, your sample will start to cover everyone. But in practice, this is never true, and the results show it. For example, questions on travel in particular can be quite skewed because of this leveraging of miles program members to become survey panel members.

Probability sampling is more expensive than opt-in sampling, and our statistical process, while highly automated, still requires skilled people to execute them and evaluate the results. So TrueNorth is more expensive than opt-in sampling, but at the same time, TrueNorth is 25% less expensive than studies using exclusively probability samples.

TrueNorth reduces approximately 2/3 of the bias inherent in opt-in samples, and often as much as eighty percent.

TrueNorth relies on probability data from AmeriSpeak to calibrate the opt-in data. TrueNorth looks at every question in a given survey, assesses the bias in the opt-in data, and creates weights that holistically adjust all data simultaneously. As such, no single metric gets forced to be anything specifically, but rather the opt-in data across all questions gets an overarching best-solution adjustment.