0
$\begingroup$

I am trying to figure out which participants who took a 18-item multiple choice test scored significantly above chance. Each item has 3 choices, so the probability of getting each question correct is 0.33. To know how many items correct a person would have to get in order to be significantly above chance, does it make sense to use a binomial probability calculator like this: https://stattrek.com/online-calculator/binomial.aspx and try different numbers of successes until the probability is < p=0.05 (assuming this is my significance criteria)?

For 18 items where the probability correct is 0.33, the probability of getting 9 correct is 0.061 and the probability of getting 10 correct is p=0.027, so does it make sense to use "10 items correct" as my criteria for "performed significantly above chance?"

$\endgroup$
0
$\begingroup$

The hypothesis would normally be expressed as a one-sided test as p(X≥10). The online calculator you have selected is fine and the significant result could be expressed in an article as follows: (p=0.33, q=0.66, K=10, n=18, p-value = 0.043).

Another way to think of this is that 9 or fewer correct answers is more likely to be random chance.

At the threshold of 10, it is unlikely that the result could be explained by random chance.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yes, 10 or more. I'm just trying to figure out what the "threshold" for number correct should be for "this participant performed significantly above chance." $\endgroup$ – bernice.anders Jun 15 at 17:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.