Synthetic Aperture Personality Assessment
Encyclopedia
Synthetic Aperture Personality Assessment (SAPA) is a method used for telemetric assessment of individual differences, primarily in the context of online surveys. The SAPA method uses data collected from the administration of large inventories of personality assessment items to large pools of participants, though it differs from traditional data collection methods in that each participant responds to only a small subset of all available items. In other words, each participant receives a random (or partially random) subset of the items under study. As long as some of the items are overlapping between pairs of participants, the smaller subset is more palatable for individual participants yet can be combined to synthesize large covariance matrices (with considerable data missing at random). In this way, the SAPA methodology is well-suited for assessing personality and individual differences across multiple domains. It is also a highly efficient means for new item prototyping and scale construction.
(and maintained by the PMC Lab at Northwestern University
) has used this technique with approximately 200,000 participants as of 2011. In this example, each participant receives a subset of items equal to approximately 75, though the full set of items being administered at any given time may be as high as 500.
- PISA). Since the 1990s, an increasing number of psychologists have begun to employ web-based data collection techniques as a means of increasing both the size and breadth of samples with little loss of validity (Fraley, 2004; Gosling, Vazire, Srivastava, & John, 2004; Skitka & Sargis, 2006). While several online surveys have collected data from very large samples (e.g., the >300,000 reported by Gosling et al., 2004), most of these studies administer short questionnaires or basic cognitive tasks (Greenwald, Nosek, & Banaji, 2003).
, part of the National Radio Astronomy Observatory
in Socorro, New Mexico, where 27 relatively small (≈ 25 meter) radio telescopes are spread out in a Y-shaped configuration to simulate the resolution of a 36 km telescope. The configuration is adjustable so that the telescope can either emphasize resolution (by maximizing the distance covered) or sensitivity (by concentrating the telescopes close to each other). In optical astronomy similar techniques are used in inferometry at the W. M. Keck Observatory in Hawaii with “outriggers” to supplement the main telescope. Similar techniques are available for "telemetric assessment" of psychological constructs. Rather than combining signals from the same source using different telescopes as is done in astronomy, the structure of personality can be studied by combining the responses of many people across more items than any one person is willing to take. It should be noted that this is not an entirely novel procedure. The Educational Testing Service
, for example, has long used the very large samples available when students take the SAT or GRE to develop new items by randomly giving small subsets of items to much smaller (but still quite large) subsamples of students. The SAPA methodology allows for these techniques to be used by a broader population of researchers by making use of open source and public domain software.
, the California Psychological Inventory
, the Eysenck Personality Questionnaire
, the Freiburger Persönlichkeitsinventar
, the Guilford-Zimmerman Temperament Survey, the Hogan Personality Inventory, the Jackson Personality Research Form, the Myers-Briggs Type Indicator
, the Stanford-Binet Intelligence Scales, the Wechsler Adult Intelligence Scale
, the Wechsler Intelligence Scale for Children
, etc. Each of these tests was carefully developed by research groups and each is protected by copyright. Although some groups will allow non-profit use of the measures for minimal cost, this is the exception. Many of these inventories have similar sounding scales, but given the expense, there are a limited number of studies directly comparing the inventories (Grucza & Goldberg, 2007).
An alternative to the proprietary nature of personality measurement is the International Personality Item Pool (IPIP) developed by Lewis Goldberg (1999). Including more than 2,400 items in the form of sentence stems, the IPIP collaboratory has at least 269 scales targeted at everything from achievement striving to vitality/zest and includes public domain scales meant to reflect constructs found in at least 17 commonly used personality inventories. All of the IPIP items and the common personality inventories have been given to the same community sample in Eugene/Springfield, Oregon, and item statistics are freely available from Goldberg and his associates. Some have questioned the open and free use of the IPIP items with respect to the possibility of the public learning to fake personality tests used in employment settings. Others have worried about whether the freedom to select items will lead to fragmentation of constructs rather than the hoped for integration (Goldberg et al., 2006).
Even more proprietary than non-cognitive personality scales and items are measures of intellectual ability. Items and scales are either under copyright or are completely idiosyncratic to particular labs and yet still not openly published. As a result, there are currently few alternatives for researchers in search of validated, open source measures of cognitive ability.
The SAPA Methodology
The procedure is straightforward. From a large set of personality and ability items , a smaller subset of items are presented to any one subject. With random sampling of the items, all possible pairs of items are eventually presented together. As the number of subjects grows, each item has been given to subjects, and each pair of items has been given to subjects. The online survey developed by William RevelleWilliam Revelle
William Revelle is a psychology professor at Northwestern University working in personality psychology. Revelle studies the biological basis of personality and motivation, psychometric theory, the structure of daily mood, and models of attention and memory. He is a proponent of and contributor to...
(and maintained by the PMC Lab at Northwestern University
Northwestern University
Northwestern University is a private research university in Evanston and Chicago, Illinois, USA. Northwestern has eleven undergraduate, graduate, and professional schools offering 124 undergraduate degrees and 145 graduate and professional degrees....
) has used this technique with approximately 200,000 participants as of 2011. In this example, each participant receives a subset of items equal to approximately 75, though the full set of items being administered at any given time may be as high as 500.
The Traditional Method Used for Internet Surveys
Studies of individual differences in cognitive and non-cognitive aspects of personality are frequently limited by the sample sizes available in the typical university research setting. Small but stable relationships are difficult to detect when one is limited to 50 to 100 subjects, and detecting complex relationships between multiple measures is difficult when participants are limited to short one or two-hour studies. Alternative procedures involve large research groups collecting data across many research sites (e.g., the Programme for International Student AssessmentProgramme for International Student Assessment
The Programme for International Student Assessment is a worldwide evaluation in OECD member countries of 15-year-old school pupils' scholastic performance, performed first in 2000 and repeated every three years...
- PISA). Since the 1990s, an increasing number of psychologists have begun to employ web-based data collection techniques as a means of increasing both the size and breadth of samples with little loss of validity (Fraley, 2004; Gosling, Vazire, Srivastava, & John, 2004; Skitka & Sargis, 2006). While several online surveys have collected data from very large samples (e.g., the >300,000 reported by Gosling et al., 2004), most of these studies administer short questionnaires or basic cognitive tasks (Greenwald, Nosek, & Banaji, 2003).
Explanation of the "Synthetic-Aperture" Analogy
As a variation of standard web-based assessment methods, the SAPA methodology borrows by analogy a technique used in radio and optical astronomy: Synthetic Aperture Measurement. The resolution of a telescope is limited by its diameter which may be functionally increased a great deal by combining input from multiple, linked sites into one coherent image. Effectively, a very large telescope is created by synthesizing the input of many smaller ones. A classic example in radio astronomy is the Very Large ArrayVery Large Array
The Very Large Array is a radio astronomy observatory located on the Plains of San Agustin, between the towns of Magdalena and Datil, some fifty miles west of Socorro, New Mexico, USA...
, part of the National Radio Astronomy Observatory
National Radio Astronomy Observatory
The National Radio Astronomy Observatory is a Federally Funded Research and Development Center of the United States National Science Foundation operated under cooperative agreement by Associated Universities, Inc for the purpose of radio astronomy...
in Socorro, New Mexico, where 27 relatively small (≈ 25 meter) radio telescopes are spread out in a Y-shaped configuration to simulate the resolution of a 36 km telescope. The configuration is adjustable so that the telescope can either emphasize resolution (by maximizing the distance covered) or sensitivity (by concentrating the telescopes close to each other). In optical astronomy similar techniques are used in inferometry at the W. M. Keck Observatory in Hawaii with “outriggers” to supplement the main telescope. Similar techniques are available for "telemetric assessment" of psychological constructs. Rather than combining signals from the same source using different telescopes as is done in astronomy, the structure of personality can be studied by combining the responses of many people across more items than any one person is willing to take. It should be noted that this is not an entirely novel procedure. The Educational Testing Service
Educational Testing Service
Educational Testing Service , founded in 1947, is the world's largest private nonprofit educational testing and assessment organization...
, for example, has long used the very large samples available when students take the SAT or GRE to develop new items by randomly giving small subsets of items to much smaller (but still quite large) subsamples of students. The SAPA methodology allows for these techniques to be used by a broader population of researchers by making use of open source and public domain software.
Public-Domain Item Pools
During the past century, the measurement of personality and ability has tended to be fragmented by separate groups of individuals using proprietary sets of measures. Indeed, the proprietary nature is partly seen in the choice of names for these inventories and tests: the Minnesota Multiphasic Personality InventoryMinnesota Multiphasic Personality Inventory
The Minnesota Multiphasic Personality Inventory is one of the most frequently used personality tests in mental health. The test is used by trained professionals to assist in identifying personality structure and psychopathology....
, the California Psychological Inventory
California Psychological Inventory
The California Psychological Inventory is a self-report inventory created by Harrison Gough and currently published by Consulting Psychologists Press. The test was first published in 1956, and the most recent revision was published in 1987. It was created in a similar manner to the Minnesota...
, the Eysenck Personality Questionnaire
Eysenck Personality Questionnaire
In psychology, Eysenck Personality Questionnaire is a questionnaire to assess the personality traits of a person.It was devised by the psychologists Hans Jürgen Eysenck and his wife Sybil B. G. Eysenck....
, the Freiburger Persönlichkeitsinventar
Freiburger Persönlichkeitsinventar
The Freiburger Persönlichkeitsinventar is a psychological personality test to assess personality. The test is comparable in some aspects to MMPI and more generally to EPI or 16PF and is mainly used in German speaking countries...
, the Guilford-Zimmerman Temperament Survey, the Hogan Personality Inventory, the Jackson Personality Research Form, the Myers-Briggs Type Indicator
Myers-Briggs Type Indicator
The Myers-Briggs Type Indicator assessment is a psychometric questionnaire designed to measure psychological preferences in how people perceive the world and make decisions...
, the Stanford-Binet Intelligence Scales, the Wechsler Adult Intelligence Scale
Wechsler Adult Intelligence Scale
The Wechsler Adult Intelligence Scale intelligence quotient tests are the primary clinical instruments used to measure adult and adolescent intelligence. The original WAIS was published in February 1955 by David Wechsler, as a revision of the Wechsler-Bellevue Intelligence Scale...
, the Wechsler Intelligence Scale for Children
Wechsler Intelligence Scale for Children
The Wechsler Intelligence Scale for Children , developed by Dr. David Wechsler, is an individually administered intelligence test for children between the ages of 6 and 16 inclusive that can be completed without reading or writing...
, etc. Each of these tests was carefully developed by research groups and each is protected by copyright. Although some groups will allow non-profit use of the measures for minimal cost, this is the exception. Many of these inventories have similar sounding scales, but given the expense, there are a limited number of studies directly comparing the inventories (Grucza & Goldberg, 2007).
An alternative to the proprietary nature of personality measurement is the International Personality Item Pool (IPIP) developed by Lewis Goldberg (1999). Including more than 2,400 items in the form of sentence stems, the IPIP collaboratory has at least 269 scales targeted at everything from achievement striving to vitality/zest and includes public domain scales meant to reflect constructs found in at least 17 commonly used personality inventories. All of the IPIP items and the common personality inventories have been given to the same community sample in Eugene/Springfield, Oregon, and item statistics are freely available from Goldberg and his associates. Some have questioned the open and free use of the IPIP items with respect to the possibility of the public learning to fake personality tests used in employment settings. Others have worried about whether the freedom to select items will lead to fragmentation of constructs rather than the hoped for integration (Goldberg et al., 2006).
Even more proprietary than non-cognitive personality scales and items are measures of intellectual ability. Items and scales are either under copyright or are completely idiosyncratic to particular labs and yet still not openly published. As a result, there are currently few alternatives for researchers in search of validated, open source measures of cognitive ability.