What is the test-retest improvement in Block Design test?

Question

Background:

My lab is conducting a research on perceptual training. We train participants on different tasks, and assess them pre- and post- training on a large battery of tests. Obviously, people improve in tasks that are very similar to what they trained on, but we are interested in the generalization of training. One of the tests in the pre/post battery is Block Design. We currently do not have a control group that did not train at all. We noticed that all groups improved significantly in Block Design, and we are now wondering if that's only test-retest improvement, but were not able to find data on expected improvement in the literature.

Question:

When testing subjects in the Block Design test twice, with the 2 sessions separated by roughly 4 months, what is the expected improvement?

Answer 1

Ronnlund and Nilsson (2006) studied performance on the WAIS block design test in a large sample of adults both longitudinally and cross-sectionally. The time difference was about 5 years. They wanted to tease out age related changes from practice effects. I believe they estimated the 5-year practice related improvement to be about 0.6 of a T score (i.e., about .06 of a standard deviation).

Benefits of practice presumably decay as a monotonically decreasing function of time between practice and retest. Thus, presumably, any practice effects would be a fair bit greater at two months than they would be at five years.

Dikmen et al (1999) performed a study involving 384 adult participants who completed several test including the WAIS Block Design on two occasions separated by between two and twelve months. Specifically see Table 2 that reports time 1 and time 2 results:

Time 1 M (SD): 11.3 (3.0)
Time 2 M (SD): 12.0 (3.2)

Thus, the standardised increase was $d = 0.7 / 3.0 = 0.23$.

The study also reports in Table 5 practice effects for various test-retest intervals (less than 3 months, 4 to 9 months, and > 10 months). See Table 5 for more information, but in general greater practice effects were observed for WAIS performance IQ for shorter time periods. Thus, you might expect, given your two month period a slightly larger effect than the overall effect (perhaps $d=.30$ or some such).

References

• Dikmen, S.S., Heaton, R.K., Grant, I., Temkin, N.R. & others (1999). Test--retest reliability and practice effects of expanded Halstead--Reitan Neuropsychological Test Battery. Journal of the International Neuropsychological Society, 5, 346-356.
• Ronnlund, M. & Nilsson, L.G. (2006). Adult life-span patterns in WAIS-R Block Design performance: Cross-sectional versus longitudinal age gradients and relations to demographic factors. Intelligence, 34, 63-78.