2
$\begingroup$

A common test of memory span is to display or utter a sequence of numbers and then request that a candidate being assessed repeat that sequence of numbers in reverse order. Typically, the candidate's score is the length of the longest sequence that he or she repeated correctly.

Thus, to each sequence produced by the candidate it is assigned a score in $\{0,1\}$, where score $0$ denotes incorrect and score $1$ denotes correct. However, what if the candidate does produce a sequence that differs from the correct sequence merely due to, say, two numbers having been swapped? Does this sequence deserve a score of $0$? In my most humble opinion, permuting the reverse of the given sequence should not be a deadly sin, though it should be penalized.

Is there any work on scoring sequences in which the scores are in $[0,1]$ rather than in $\{0,1\}$?

I thought of using the Levenshtein and Damerau-Levenshtein distances and found several papers on Google Scholar. However, since we are working with sequences of numbers rather than sequences of characters, perhaps other metrics would be more appropriate. I am an utter neophyte, am not familiar with the jargon in the field of psychometrics and I am not convinced that number of citations is a great proxy for quality. Thus, I would rather ask the experts (or non-neophytes). I am happy to start a bounty to motivate those who have invested time and effort in learning psychometrics to post an answer. An answer listing good papers would be excellent.

Related

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ As I recall when I administered this test, subjects were given 2 chances at each length, and failing both not only scores 0, but also stops the test. With a partial score, you would also need to decide stopping criteria that can be determined on the fly by the administrator. The current method has the advantage of being very simple to administer and score. $\endgroup$
    – Arnon Weinberg ♦
    Jun 8, 2021 at 20:01
  • $\begingroup$ @ArnonWeinberg Thank you for your comment. If I know very little about psychometrics, I know even less about its history. I assume that some of these tests were developed when computers were much more expensive and much, much slower. Given that computing is now relatively cheap, I am interested in gathering as much data as possible, as discarding data is much easier than acquiring it. $\endgroup$
    – Rodrigo de Azevedo
    Jun 9, 2021 at 13:35

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.