3.10: Practice and Exploration

1. A professor is interested in knowing whether the use of a free tutoring center in a department is helpful with student grades. At the end of the semester the professor gives the students in their class a short survey, and one of the questions asks whether the students used the tutoring center or not. It turns out that the grades for the students who used the tutoring center are higher, on average, than the grades of the students who did not use the center. Explain why this is an observational study and point out some possible confounding problems that could affect how the results of the study are interpreted. Identify the response, the factors studied, and the steps in the scientific method used to make conclusions about the issue. Is there a treatment group and a control group? What are they identified as? Are there other explanations as to why the average grades might be different?

2. Professor Plum is interested in whether the color of the paper used during an exam influences student performance. At the midterm exam in their class, the students who were seated in the first ten rows were given the midterm printed on yellow paper, while the students who were seated in the last ten rows where given the same midterm printed on green paper. The results from the exam showed that the students who took the exam on yellow paper had higher grades on average than the students who took the exam on green paper. Explain why this is or is not an observational study. Whether it is observational or not, suggest some confounding problems that this study might have. Identify the response, the factors studied, and the steps in the scientific method used to make conclusions about the issue. Is there a treatment group and a control group? What are they identified as? Are there ethical implications associated with this study? If so, how could the ethical implications be addressed?

3. One study considered demographic disparities in the nutritional quality of packaged food purchases in the United States between 2008 and 2018. This study used data based on the Nielsen U.S. Homescan Consumer Panel for which households self-report demographic measures and are instructed to scan barcodes on all purchases. The researchers then used the bar codes to match the purchases with specific food items, from which the nutritional value of the purchases could be determined. The study found that nutritional disparities existed between different income and education levels with the largest disparities for education level. Is this an observational study or a designed experiment? Identify the response, the factors studied, and the steps in the scientific method used to make conclusions about the issue. Is there a treatment group and a control group? What are they identified as? Disparities were also found between African American and white households, while no disparities were found between white and Hispanic households. Identify at least three possible confounding effects for this study. For each confounding effect, explain why you think it would occur and if it would be possible to control for the effect with a study designed in a different way.

4. Demographics of high school students have been changing rapidly while the demographics of high school teachers have remained relatively steady. A great deal of research has demonstrated that racial parity between students and teachers increases student performance and success. While it is estimated that by 2029 students of color will make up 60% of the student population, the teacher population has been predominately white. One research article examined how racial parity between students and teachers affects class attendance (Gottfried et al. 2022). In this study the data collection methodology states:

   This study relies on administrative high school data from a high school district in California to examine the influence of a student–teacher racial and ethnic match on absenteeism. The data was obtained directly from the district’s central office.

Is this an observational study or a designed experiment? Identify the response and the factors that are being studied and the steps in the scientific method used to make conclusions about the issue. Is there a treatment group and a control group? What are they identified as? The authors of the study also state that the students’ and teachers' names were coded. Why do you think this was done?

5. Consider a situation where student retention has become an important issue at a large Midwestern university. The problem of interest is that a sizable percentage of those who enroll as first-year students do not complete a degree at the university within seven years. An on-campus researcher hypothesizes that student cultural experiences during their first year enables students to become more connected with the local university community, and that those students would be less likely to leave. Detail how a designed experiment could be used to study this hypothesis. What would be some major problems with implementing this type of study? Would there be possible ethical issues? Detail how this hypothesis could be studied using a traditional observational study and alternatively a case-control study. What are the potential problems with each type of approach? What are some confounding factors that could present a problem?

6. A political scientist is interested in how political affiliates may differ in their response to scientific news reports. To study this idea, they compose a fake scientific report that suggests a common treatment for a contagious disease is just a sugar pill that is used to bolster the profits of large pharmaceutical companies and that the disease normally goes away on its own. The report is then released to several bloggers of different political affiliations, and the response is observed. Is this an observational study or a designed experiment? What are the ethical implications of this study?

7. Using an online news service, find a news article about an issue that is important to you. Does the article use any empirical analysis to convey information about the issue? What was observed, and how were the observations interpreted? Was the empirical investigation based on a designed experiment or an observational study? Identify the response, the factors studied, and the steps in the scientific method used to make conclusions about the issue. Is there a treatment group and a control group? What are they identified as? What possible confounding issues are contained in the empirical analysis? Are there any ethical implications connected with the study? Do you trust the empirical analysis and the conclusions of the article? Why or why not?

8. Using your library’s database research tool, find an academic research article about an issue that is important to you. Does the article use any empirical analysis to convey information about the issue? What was observed, and how were the observations interpreted? Was the empirical investigation based on a designed experiment or an observational study? Identify the response, the factors being studied, and the steps in the scientific method used to make conclusions about the issue. Is there a treatment group and a control group? What are they identified as? What possible confounding issues are contained in the empirical analysis? Are there any ethical implications connected with the study? Do you trust the empirical analysis and the conclusions of the article? Why or why not?