A fish labeled 'Accurate Data' swimming in a sea labeled 'Context'.

Participant Interviews and/or Observations

Tip: As appropriate, prior to the data collection, provide the observer or interviewer with as little demographic information as possible about the participants.

Rationale: Observers rate the same behaviors differently based on the perceived characteristics of the subjects. For example, female musicians were more likely to be hired when a “blind” audition process was used, during which the hiring committee is not aware of the sex of any of the auditioning musicians.1 Observers, too, were found to rate the behaviors of those who were of their own sex differently.2 Even accents made a difference. People viewed speakers with accents like theirs as being more knowledgeable than different-accent speakers, even when the different-accent speaker was more knowledgeable.3

Tip: If there are potential barriers to observers understanding what is going on in the setting being observed or to full interviewer/participant communication (e.g., participants are hearing impaired or not fluent in English), this needs to be known and addressed in advance.

Rationale: Because of the potential for bias, the less demographic information known about those being observed/interviewed, the more accurate the objective data. However, there are instances when it is important to have more advanced knowledge about those being observed/interviewed in order to help direct what is being looked for in the observation or the specific questions asked in the interview. When working with deaf populations, it is important to know the communication mode that is used; i.e. American Sign Language, an assistive listening device, lip reading and speech, Mexican Sign Language, etc.

1 Goldin, C., & Rouse, C. (2000). Orchestrating impartiality: The impact of “blind” auditions on female musicians. The American Economic Review, 90(4), 715-741. doi:10.1257/aer.90.4.715
2 Pellegrini, A. D. (2011). “In the eye of the beholder”: Sex bias in observations and ratings of students' aggression. Educational Researcher, 40(6), 281-286. doi:10.3102/0013189X11421983
3 Dahlbäck, N., Wang, Q. Y., Nass, C., & Alwin, J. (2007). Similarity is more important than expertise: Accent effects in speech interfaces. CHI '07 Proceedings of the SIGCHI conference on Human factors in computing systems, 1553-1556. doi:10.1145/1240624.1240859