1
$\begingroup$

How much time between 2 measurements is needed (at least / at most) to be called test-retest reliability?

Concrete casus: I am planning a meta-analysis on reliability of various loneliness scales. We want to determine beforehand what time frame between measures we will consider.

For instance, if the time between 2 measures is 2 years (such as in some longitudinal papers), we would consider this to be an indication of rank-order stability rather than test-retest reliability. This is a clear case, but what about 3 months? Or 1 month? Where should be the limit?

We are looking for good resources to base our decision on, and would be glad with any advise.

| improve this question | |
$\endgroup$
  • 2
    $\begingroup$ This is certainly going to depend on the measure and the time scale on which you care about reliability. It's more related to your specific research question rather than anything about test-retest reliability in general. $\endgroup$ – Bryan Krause Feb 19 '19 at 16:23
  • $\begingroup$ So to be clear: It is for a meta-analysis and I want to select an appropriate time-range. So say: To examine test-retest reliability of each measure, we consider articles that examine the same measure in the same population within a time frame of 2 weeks to 6 weeks. [but then an appropriate timeframe] $\endgroup$ – GerineL Feb 19 '19 at 20:06
  • 1
    $\begingroup$ @AliceD and BryanKrause I would consider these points you raise as answers (post it as such) and move on. $\endgroup$ – Steven Jeuris♦ Feb 23 '19 at 10:34
1
$\begingroup$

Although I don't know the answer, I think it would be important to explain what you exactly wish to do with the re-test? If you solely wish to know how reliable the score is, you could do the test on day 1 and the re-test on day 2? The longer the time, the higher the chance that a person's life is subject to change (e.g., the loss of relatives, or medical intervention), and that it's not a re-test per se, but a different time point in a person's life. Regardless, I think it would be good to have a look at some of the best-cited papers in your arena of research and see what they do, and perhaps contact the authors of why they choose 2 years.

If it is for a review meta analysis paper, as you say in your comments, then the answer to your question is all about your research question. If indeed you wish to minimize that intervening time (t), the best thing might be to map all the papers, organize them in ascending order of t and then decide what the minimum #papers is you wish to include in your review. A kind of reversed approach, but if the #papers is the bottle neck, it may in fact be a proper, and maybe even the only way to go.

P.S. -- for what it's worth -- I don't like to write unreferenced answers, but in this case it seems warranted and in fact I was urged by a long-time user to convert my comments into an answer

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.