Article by Amplitude Research, Inc.

This article discusses the concept of “incidence rate” and how it is used in the context of online market research surveys. In practice, there is a difference between a “natural incidence rate” and a “practical incidence rate.” For some types of studies, the natural incidence rate may be of interest for analysis purposes by researchers at market research firms or marketing departments of companies or other organizations. For other types of studies, it may not be necessary to estimate “natural incidence,” and research users may become more concerned about ways to increase the “practical incidence” rate in order to reduce costs. It is important to understand when an estimate of natural incidence is needed vs. when it is safe to focus on increasing the practical incidence rate to reduce costs.

Market Research Surveys and Natural Incidence Rate

An example can help to explain these concepts. Imagine that a researcher wants to conduct an online market research survey of people who “regularly” visit large bookstores (e.g., Borders, Barnes & Noble, Books-A-Million). For this study, “regularly” could be defined as “at least once a month.” Many adult consumers would be sampled at random and invited to participate in the survey. For those who respond to the survey, the first question would ask how often they visit large bookstores. Those answering once a month or more often would be considered “qualified” to continue and complete a full survey for the study. Those answering less often than once a month would be considered “disqualified” for the full survey, and their survey would end after this first question. However, the respondents “disqualified” from the full survey (because they do not visit large bookstores “regularly”) would still be tallied.

Although the primary purpose of the study may be to learn about the detailed behavior of those who visit large bookstores regularly, the researcher may also be interested in the “incidence” of these types of consumers. That is, the researcher might ask, what proportion of adult consumers are “regular” visitors of large bookstores? This can be estimated by examining how many “qualified” to complete the full survey vs. how many were “disqualified.” The “incidence” of regular visitors of large bookstores would be estimated using the number qualified for the full survey divided by the total that responded. In other words, the incidence would be the number qualified divided by the sum of those qualified and disqualified.

This incidence may be very useful for the researcher, since it provides an estimate of the proportion of adult consumers who are regular visitors of large bookstores. Sometimes market research firms will use an estimate of “natural incidence” to “size the market.” That is, multiply the incidence by the number in the whole adult population to get the estimated size of the “target population” of regular visitors of large bookstores.

However, the calculation of “natural incidence” assumes that adult consumers were sampled at random to participate in the survey. If this were not the case, then the calculation described above would not yield a valid estimate of natural incidence.

In fact, many studies do not require pure random sampling of all adult consumers. Instead, particular types of consumers may be “targeted.” For example, many market research firms with online panels have extensive information “on file” about their panelists. Let’s assume that the researcher is conducting online market research using a panel in which bookstore behavior is already known. That is, when panelists joined the research panel, they were asked a variety of questions about their shopping behavior, and one of the questions was about frequency of visiting large bookstores. As a result, it would be possible to “target” regular bookstore visitors from the panel and only invite these panelists to participate in the new survey.

Market Research Surveys and Practical Incidence Rate

If regular bookstore visitors were targeted from the panel, then most would be “qualified” for the full survey. To be sure, some would be “disqualified” if the information “on file” about the panelists was sometimes no longer accurate. For example, if someone joined the panel a year ago, they may have been visiting large bookstores regularly at that time, but they may have changed their behavior since then. But, the number “disqualified” in this case would be very low compared to conducting a survey with consumers selected purely at random. Thus, “targeting” particular types of panelists can result in a very high practical incidence rate, but this would also invalidate estimates of “natural incidence.”

At this point, the reader might ask why target panelists based on information already “on file” if this will invalidate estimates of “true” or “natural” incidence? The answer has everything to do with cost. Targeting reduces research costs significantly because fewer survey invitations need to be sent; administrative time for market research firms in managing the project is lower; and, in many cases, the total incentives paid out can decrease. (Many market research panels offer reduced incentives to panelists who respond to a survey invitation even if they don’t qualify for a full survey.) When the true incidence would be very low, targeting can reduce costs significantly. As budgets for market research surveys are often limited, targeting can be an important way to minimize costs (assuming information already exists “on file” that can be used for targeting). This may be no problem for studies that do not require an estimate of the “true” or “natural” incidence rate. But, when the researcher does in fact want to estimate the natural incidence rate, it is important to avoid targeting that would invalidate such an estimate.

The bottom line for researchers interested in conducting a new survey of web panelists: In situations when natural incidence is not a concern, make sure to inquire about options for targeting the online market research so that costs may be reduced. In situations when estimating natural incidence is an important part of the analysis, make sure targeting will not be used that would invalidate your analysis.