4
$\begingroup$

Specific context: A test subject took the same Raven's Standard Progressive Matrices IQ test three times and first scored 115, then 107, and finally 143. The same test was taken three times within 3 and a half hours. Since all three scores are different and appear random, how can one asses these results? i.e. Assuming the individual did not cheat, what can be said about the intelligence of test subject?

More generally:

How should one interpret intelligence test scores for an individual who has taken the same form of the same test multiple times in quick succession?

$\endgroup$
  • $\begingroup$ Edited, because some IQ tests offer parallel froms that can be taken in immediate succession. $\endgroup$ – user3116 Jun 30 '15 at 10:40
4
$\begingroup$

General points about practice effects

Intelligence tests differ in how much they are subject to practice effects.

Practice effects can also be distinguished:

  • Time between taking the test: The shorter the timeframe the more likely you will see practice related improvement.
  • General practice on similar tests and similar items versus practice on the same set of items. In general, practice effects will be greater where the items are identical between test sessions. Adaptive testing where items change each time should show fewer practice effects.
  • Whether feedback is present. In general participants do not get feedback on what was or was not the right answer, but if they do receive such feedback, then practice effects will be much greater, especially for item specific transfer.

Test-retest variation

Another general point is that intelligence tests have a level of reliability such that a score on a test is only an estimate of an individual's true standing on the latent trait being measured. So obviously, some variation between administrations would be expected purely based on the standard error of measurement.

That said, the general model of test-retest reliability assumes reasonably controlled testing conditions and a participant that is applying reasonable effort. When these conditions are not met, you could see much more variation:

  • If a test taker applies minimal effort or is distracted, they may get a much lower score than their true level.
  • If a test taker cheats in some way they may get a much higher score than their true level.

Issues of administration are more likely in an online environment particularly where there is either (a) no consequences for not applying full effort, or (b) an incentive to cheat.

Interpreting scores of someone who has taken a test multiple times

In general, if you are trying to measure intelligence, you should administer the test once and that first measure would be the measure of their intelligence.

Test administrations that follow immediately afterwards would be contaminated by practice effects. Unless you have a very good understanding of how practice effects operate and how they can be controlled for, then it would be difficult to incorporate information from these subsequent administrations.

That said, if you had additional information that said that on the first administration the participant was not concentrating or fundamentally failed to understand the test, then you may wish to tentatively rely on the score from a different administration.

In addition, a pattern of scores that differ by three standard deviations (e.g., 100, 115, 145) may suggest either that the test has reliability issues or that something strange is happening for that particular individual that you want to look into.

$\endgroup$
  • $\begingroup$ If one gets 100 after 115 then Practice effects do not affect much, right? but then 140 is a strong indication of Practice effects since the online test is same. this is what struck me. $\endgroup$ – Jim Jun 30 '15 at 5:46
  • $\begingroup$ To evaluate practice effects, you'd want to examine changes in performance for a large number of individuals. When looking at an individual case, it may be difficult to tease out practice effects from random variation. $\endgroup$ – Jeromy Anglim Jun 30 '15 at 5:49
  • $\begingroup$ Also, I dont get your point on concentration , if one does not concentrate/understand, should not we conclude that person has less IQ since he failed to concentrate /comprehend the test? i am keen to know why concentration should considered, my perception is , intelligent people find problem-solving interesting and do well without much concentration as problem solving is a natural ability.I think this is the ' public perception ' also. $\endgroup$ – Jim Jun 30 '15 at 6:06
  • $\begingroup$ People may not concentrate for transient reasons that presumably could be readily fixed. So no. Low scores for lack of attention suggest a problem with test administration and don't really reflect an individual's latent intelligence. I'm not referring to deeper issues with attentional processes that we might associate with neurocognitive disorders. $\endgroup$ – Jeromy Anglim Jun 30 '15 at 6:09
  • $\begingroup$ Fatigue, drugs, distraction, boredom, diurnal cycle etc. can all lead to lack of concentration, but are not symptoms of low intelligence. $\endgroup$ – user3116 Jun 30 '15 at 10:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.