Levels of Aggregation and Disaggregation
Tip: Use crosstabs to break down the demographic characteristics of participants to help determine where levels of disaggregation can be done. If almost all of the participants are from one demographic subcategory, for example, the middle class, then it is not necessary nor perhaps appropriate to disaggregate data for analysis by socioeconomic status. However the results could not then be generalized to other socioeconomic groups.
Tip: If there are known or expected differences by subgroup that could skew the overall findings, then disaggregate by those subgroups.
Tip: Be aware that there can be heterogeneity within subgroups. For example, while people who are visually impaired, hearing impaired, and learning disabled are all classified as having disabilities, the differences among them are very large and it might be appropriate to disaggregate by different categories of disability.
Tip: Do preliminary analysis of subgroup differences in areas of importance to the study. This can help inform disaggregation and aggregation decisions.
- Class, race, ethnic, and gender differences in diagnosis of learning disabilities;5
- Race and ethnic differences in retention in undergraduate STEM academic programs;6
- Differences in retention to degree for two year college transfers;7
- Race and ethnic differences in time to STEM degree;8
- Differences in STEM preparation including:
- course taking;
- participation STEM programs;
- work experience.
Tip: Provide a rationale for the decisions made regarding which demographic categories are aggregated and which are disaggregated.
While evaluators must assume responsibilities for capturing and correctly interpreting within-group variability for the groups under study,9 types of disaggregation must be both meaningful and viable. If, for example, there is an interest in trend data, aggregation across years is not appropriate. If there is reason to think there might be different trends for different subgroups it is important to disaggregate by those subgroups. Since there are gender differences in some spatial skills, if you are interested in the impact of a project/program to improve spatial skills, then it is important to disaggregate by gender. Based on the questions to be answered, it might be more appropriate to aggregate across subdisciplines, across institutions, across years, or across some racial/ethnic or disability categories.
1 Mehta, C. R., & Patel, N. R. (2011). IBM SPSS Exact tests.
2 Lachin, J. M. (1981). Introduction to sample size determination and power analysis for clinical trials. Controlled Clinical Trials 2, 93-113.
3 Jolly, E. J. (9/07/12). Personal communication.
4 Metz, S. S., Donohue, S. & Moore, C. (2012). Spatial skills: A focus on gender and engineering. In B. Bogue & E. Cady (Eds.), Apply Research to Practice (ARP) Resources.
5 Center for Disease Control. (2011). Percentage of Children Aged 5-17 years ever receiving a diagnosis of learning disability, by race/ethnicity and family income group - National Health Interview Survey, United States, 2007-2009.
6 Science and Engineering Indicators 2012. (2012). Arlington, VA: National Science Foundation.
National Academy of Sciences (NAS), Expanding Underrepresented Minority Participation: America's Science and Technology Talent at the Crossroads 2011, Washington, DC: National Academies Press.
7 National Student Clearinghouse Research Center (Spring, 2012) Snapshot report: Mobility.
National Student Clearinghouse Research Center (Spring, 2012) Snapshot report: Degree attainment.
8 Bell, N. (2010, March). Research report on data sources: Time-to-degree for doctorate recipients. Communicator, 1-3. Washington, D.C.: Council of Graduate Schools.
Huang, G., Taddese, N., & Walter, E. (2000). Entry and persistence of women and minorities in college science and engineering education (No. NCES 2000601). Washington, DC: National Center for Education Statistics.
9 Nelson-Barber, S., LaFrance, J., Trumbull, E., & Aburto, S. (2005). Culturally-responsive program evaluation. In S. Hood, R. Hopson, & H. Frierson (Eds.), The role of culture and cultural context: A mandate for inclusion, the discovery of truth and understanding in evaluative theory and practice, (pp. 61-85). Greenwich, CT: Information Age Publishing.