Selection Issues
What is my sample a sample of?
Selection issues are a part of content analysis studies just as they are a part of most other methods. The key, as with other methods, is to be aware that selection issues exist in your study and very likely affect both your conclusions and implications. For example, you may be interested in how children’s gender is portrayed in parenting magazines, but as you analyze and draw implications from your findings, you must consider both who is creating those portrayals and who is consuming them. Those portrayals may be a reflection of cultural norms, in part, but they are also filtered for publication by a publishing company and its editorial board. And the majority of people consuming or viewing those portrayals will be parents of young children, but, depending on the magazine, they may be mostly white, middle-class mothers, certainly a more select group than just “parents.”
Another example of selection issues has to do with timing of sample selection. You may be interested in gender portrayals of different age groups of adults in television commercials, but the time of day at which you collect your sample will influence the types of commercials you watch (house cleaning products versus adult hotlines), which will likely influence the types of age and gender portrayals you observe. So rather than drawing conclusions about portrayals of older and younger women and men in television commercials in general, it would be more accurate to say you are able to draw conclusions about these portrayals in daytime television commercials knowing that these may be substantively different than commercials shown at other times of day. See Craig (1992) in the Methodological Exemplars for an example.
Studies of Internet content are becoming increasingly common but are also subject to these selection issues, both in terms of who produces and who consumes the content you are analyzing. Producers of Internet content are a highly select group and consumers of Internet content, though growing, are still limited to certain segments of the population. In general, selection issues will not invalidate a study, but they must be acknowledged and speculated on in the write-up. Selection issues are inherent in most analyses of samples. What/who’s in my sample? What/who isn’t? How might this matter? These questions should be carefully considered during both the sampling and analysis phases of the project.
Selection issues are a part of content analysis studies just as they are a part of most other methods. The key, as with other methods, is to be aware that selection issues exist in your study and very likely affect both your conclusions and implications. For example, you may be interested in how children’s gender is portrayed in parenting magazines, but as you analyze and draw implications from your findings, you must consider both who is creating those portrayals and who is consuming them. Those portrayals may be a reflection of cultural norms, in part, but they are also filtered for publication by a publishing company and its editorial board. And the majority of people consuming or viewing those portrayals will be parents of young children, but, depending on the magazine, they may be mostly white, middle-class mothers, certainly a more select group than just “parents.”
Another example of selection issues has to do with timing of sample selection. You may be interested in gender portrayals of different age groups of adults in television commercials, but the time of day at which you collect your sample will influence the types of commercials you watch (house cleaning products versus adult hotlines), which will likely influence the types of age and gender portrayals you observe. So rather than drawing conclusions about portrayals of older and younger women and men in television commercials in general, it would be more accurate to say you are able to draw conclusions about these portrayals in daytime television commercials knowing that these may be substantively different than commercials shown at other times of day. See Craig (1992) in the Methodological Exemplars for an example.
Studies of Internet content are becoming increasingly common but are also subject to these selection issues, both in terms of who produces and who consumes the content you are analyzing. Producers of Internet content are a highly select group and consumers of Internet content, though growing, are still limited to certain segments of the population. In general, selection issues will not invalidate a study, but they must be acknowledged and speculated on in the write-up. Selection issues are inherent in most analyses of samples. What/who’s in my sample? What/who isn’t? How might this matter? These questions should be carefully considered during both the sampling and analysis phases of the project.
