Calculating statistical significance in behaviour change

Question

I have run a study on a group of 158 individuals to see if a teaching intervention can change their perceptions of scientists.

Before the intervention, all the children in the study were asked to choose 6 words to describe a scientist. This data collection was repeated at the end of a 5-week intervention, and then again 3 months after the intervention was complete.

I have begun processing the data and have categorised all the words that the children have used. When I look at the data for the cohort I can see that there has been a change in the types of words used to describe scientists (Pre 128 children used one stereotypical word, post this dropped to 67 and 64 in follow-up).

What statistical tests should I use to demonstrate the significance of these changes? All the data is tracked so I have individual data on the change of language use for each participant, I also have data for the group.

Answer 1

This answer should provide a reasonable discussion of how to compare the two results for statistical significance.

We have n1 = n2 = n3 = 158. We have p1 = 128/158, p2 = 67/158, and p3 = 64/168. Just run the hypothesis test on each pair to see if there is a significant shift.

However, there is a bigger issue here. This question should have been answered before you started collecting data. The first thing you should be doing is coming up with your theory to test. Then you design your experiment. And only then should you be collecting the data, unless your goal is just to collect large amounts of data for future theory testing, which does not seem to be the case here (Best Research Practices).