Beyond Rigor: Improving Evaluations With Diverse Populations

Home About Accurate Data Appropriate Analysis Right Data Context Funder Tools Resources PDFs

Money

Money can mean so many things to so many people. In evaluation, money can be an outcome, a confounding variable, or even a risk factor. Money is tied to access to resources and power. It is a key component of context that needs to be considered in evaluations.

Money as defined as annual family income, along with adult educational attainment are the conventional measures of socioeconomic status (SES) or class, although they are not the only ones. Indeed there is no consensus definition of class.¹ To make data as comparable as possible to that collected by others, it would make sense to use the same measure of SES or class as has been used by others. Otherwise, evaluators should consider using a measure that ties most closely to their evaluation questions. If the primary interest is individual or family income, then that is what should be used.

Making assumptions about income based on race, ethnicity or family education is dangerous and should be avoided. While there is a correlation between income and race and ethnicity as well as between income and educational level in the United States, lower income and higher income families come in all colors and from all educational levels. Race, ethnicity and educational level are not proxy indicators of income or SES and should not be used as such. Indeed, evaluators should consider in their analysis disaggregating by race, ethnicity and educational level to tease out interactions.

Asking about income can be sensitive. Many people don't feel comfortable discussing their income and often students don't know their family income. Many evaluators use ranges the participant can choose such as $0-$25,000 or $25,001-$50,000 rather than asking for exact or even approximate numbers.

When working with lower income participants, particularly if they are in programs that provide them with financial support, evaluators should be sensitive to participant fears that if they don't participate in the evaluation or if they raise concerns, they could impact their continued support from the program. This could impact whether their participation in the evaluation is truly voluntary and if their responses are free from pressure.

For more information:

• Measuring Socioeconomic Status. National Institutes of Health: Office of Behavioral & Social Sciences Research.

For more about the role of context in evaluation, click here