Rationale: Cell size is an important factor in determining areas of disaggregation. If you are disaggregating by gender and by race/ethnicity, then the number of White women would be one cell, the number of Black men would be another cell, and so on. If some cell sizes are very small at a specific level of disaggregation, then statistical analysis at that level may not be appropriate. For example, a nonparametric statistical test such as a chi square test requires a minimum expected value of 5 for each cell in order to be valid. For parametric statistical tests such as t-tests, a cell size of 30 is considered the minimum. In addition in parametric statistics, the smaller the sample or cell size, the more difficult it is to get statistical significance.¹ The probability that a significant result will be obtained if a real difference exists (which is called the power of the test) depends largely on the total sample size. Based on the size of the sample, you can compute the power of the test. Alternatively, if you know what power you want the test to have, you can compute the needed sample size to get that power.²

Rationale: There may be some cases where disaggregation is not needed. As Jolly explained, “when you mix ammonia and Clorox in a room, everyone gets sick. At that level it is not necessary to disaggregate.”³ There are other cases where project/program impact may vary for different subgroups and disaggregation is needed. For example, since women students tend to exhibit lower skills in some spatial areas, such as 3-D rotation, that are important to success in engineering,⁴ projects/programs tying improved spatial skills to increased retention in engineering should disaggregate their data by sex to determine if there are statistically significant interactions between sex and impact of program participation in a spatial skills training program. Other possible areas that need to be considered in decisions about disaggregation include:

• Class, race, ethnic, and gender differences in diagnosis of learning disabilities;⁵
• Race and ethnic differences in retention in undergraduate STEM academic programs;⁶
• Differences in retention to degree for two year college transfers;⁷
• Race and ethnic differences in time to STEM degree;⁸
• Differences in STEM preparation including:
  • course taking;
  • achievement;
  • participation STEM programs;
  • work experience.

Rationale: For statistical reasons and often for confidentiality, some aggregation of data needs to be done even though information will be lost in each aggregation. For example, when aggregation is done across disability groups, the ability to determine if a project/program has different impacts on people with learning disabilities and people with mobility impairment is lost. Another example shows the aggregation of Native Americans with other groups because of their small numbers means that little is known about Native Americans and STEM.
While evaluators must assume responsibilities for capturing and correctly interpreting within-group variability for the groups under study,⁹ types of disaggregation must be both meaningful and viable. If, for example, there is an interest in trend data, aggregation across years is not appropriate. If there is reason to think there might be different trends for different subgroups it is important to disaggregate by those subgroups. Since there are gender differences in some spatial skills, if you are interested in the impact of a project/program to improve spatial skills, then it is important to disaggregate by gender. Based on the questions to be answered, it might be more appropriate to aggregate across subdisciplines, across institutions, across years, or across some racial/ethnic or disability categories.