Cognitive pretesting
Encyclopedia
Cognitive interviewing is a field research
method used primarily in pre-testing survey instruments developed in collaboration by psychologist
s and survey researchers. It allows survey researchers to collect verbal information regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey instrument to the full sample.
Although survey researchers do not totally agree as to what a cognitive interview entails, it in general collects the following information from participants: evaluations on how the subject constructed his or her answers; explanations on what the subject interprets the questions to mean; reporting of any difficulties the subject had in answering the questions; and anything else that reveals the circumstances to the subject’s answers.
In general, there are two methods practiced when conducting a cognitive interview. The first method, called the think-aloud method, encourages participants participating in the interview to verbalize their thoughts while responding to the survey questions. This method is considered purer as it reduces the possibility of the interviewer introducing any bias into the participants’ answers. In contrast, the disadvantage to this method is that it does require training on the part of the participant on the think-aloud process which can be burdensome to the interviewee. The second method has the interviewer ask detailed probes after subjects answer a survey question (called the probing-method). An example of a probe question is: “In your own words, what is this question asking?” or “How did you arrive at your answer?” Advocates for this method suggest that follow-up probes do not interfere with the actual process of responding to survey questions while requiring very little training on the part of the respondent.
In order to conduct a cognitive interview on a survey instrument, the researcher should recruit a minimum of 10 participants to a maximum of 25 participants. The participants recruited should reflect the diversity in the population being studied.
Cognitive interviewing is regularly practiced by U.S. Federal Agencies, including the Census Bureau
, National Center for Health Statistics
and the Bureau of Labor Statistics
.
Field research
Field research is the collection of raw data in natural settings. It helps to reveal the habits and habitats of various organisms present in their natural surroundings...
method used primarily in pre-testing survey instruments developed in collaboration by psychologist
Psychologist
Psychologist is a professional or academic title used by individuals who are either:* Clinical professionals who work with patients in a variety of therapeutic contexts .* Scientists conducting psychological research or teaching psychology in a college...
s and survey researchers. It allows survey researchers to collect verbal information regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey instrument to the full sample.
Although survey researchers do not totally agree as to what a cognitive interview entails, it in general collects the following information from participants: evaluations on how the subject constructed his or her answers; explanations on what the subject interprets the questions to mean; reporting of any difficulties the subject had in answering the questions; and anything else that reveals the circumstances to the subject’s answers.
In general, there are two methods practiced when conducting a cognitive interview. The first method, called the think-aloud method, encourages participants participating in the interview to verbalize their thoughts while responding to the survey questions. This method is considered purer as it reduces the possibility of the interviewer introducing any bias into the participants’ answers. In contrast, the disadvantage to this method is that it does require training on the part of the participant on the think-aloud process which can be burdensome to the interviewee. The second method has the interviewer ask detailed probes after subjects answer a survey question (called the probing-method). An example of a probe question is: “In your own words, what is this question asking?” or “How did you arrive at your answer?” Advocates for this method suggest that follow-up probes do not interfere with the actual process of responding to survey questions while requiring very little training on the part of the respondent.
In order to conduct a cognitive interview on a survey instrument, the researcher should recruit a minimum of 10 participants to a maximum of 25 participants. The participants recruited should reflect the diversity in the population being studied.
Cognitive interviewing is regularly practiced by U.S. Federal Agencies, including the Census Bureau
United States Census Bureau
The United States Census Bureau is the government agency that is responsible for the United States Census. It also gathers other national demographic and economic data...
, National Center for Health Statistics
National Center for Health Statistics
National Center for Health Statistics is a division of the United States federal agency the Centers for Disease Control and Prevention . As such, NCHS is under the United States Department of Health and Human Services...
and the Bureau of Labor Statistics
Bureau of Labor Statistics
The Bureau of Labor Statistics is a unit of the United States Department of Labor. It is the principal fact-finding agency for the U.S. government in the broad field of labor economics and statistics. The BLS is a governmental statistical agency that collects, processes, analyzes, and...
.