Obviousness of Measures

Tip: Have members of the target population review affective and psychosocial measures for clarity. Ask them what concepts they think are being measured. If what is being measured is obvious and there are sex, race, or disability stereotypes associated with the concepts, consider using a less obvious measure, if an equally valid measure is available.

Rationale: People's responses to a measure can be different when the purpose of the measure is obvious:

Answers of White people to multiple choice type survey questions on race were different than their responses to in-depth questions on race.¹
Gender differences were greatest in tests when it was very clear what gender related concepts, like empathy, were being measured.²
Job application ratings were higher when raters thought the candidate had a disability because the researchers felt the raters realized the study was investigating attitudes toward workers with disabilities. ³

When there is concern that skewed responses due to obviousness might happen, evaluators can use more than one measure of a concept and compare or triangulate the data. For example, both students and instructors could rate student skills or interests. Another example would be to have participants respond to racial attitude surveys and to vignettes tied to racially sensitive situations.

¹ Bonilla-Silva, E., & Zuberi, T. (2008). Toward a definition of white logic and white methods. In T. Zuberi & E. Bonilla-Silva (Eds.), White logic, White methods. Racism and methodology. New York, NY: Rowman & Littlefield Publishers, Inc.
² Eisenberg, N., & Lennon, R. (1983). Sex differences in empathy and related capacities. Psychological Bulletin, 94(1), 100-131. doi:10.1037/0033-2909.94.1.100
³ Bell, B. S., & Klein, K. J. (2001). Effects of disability, gender, and job level on ratings of job applicants. Rehabilitation Psychology, 46(3), 229-246. doi:10.1037/0090-5550.46.3.229