1
$\begingroup$

I know untimed tests exist, but they're still under the constraint of testing people that have lives to get to. People have jobs, loved ones, classes to go to, lives to live, and as such even untimed tests aren't really untimed, since you'd have to be really vain to spend 16 hours on an "untimed test with no rewards" just to get a better score. So the question is, as a mind experiment, if you take the ceiling item of a Raven's test and take an average Joe, then lock them in a room with a bed and nutrition for a week, and say if they don't solve it, they're gonna die akin to a Saw scenario, will they solve it or will they face their doom?

I've found the following relevant research:

  1. Richard P. DeShon, David Chan, Daniel A. Weissbein, Verbal overshadowing effects on Raven's advanced progressive matrices: Evidence for multidimensional performance determinants, Intelligence, Volume 21, Issue 2, 1995,Pages 135-155, ISSN 0160-2896,
  2. Bors, Douglas & Vigneau, François. (2001). The effect of practice on Raven's Advanced Progressive Matrices. Learning and Individual Differences - LEARN INDIVID DIFFER. 13. 291-312. 10.1016/S1041-6080(03)00015-3.
  3. Kunda, Maithilee. (2010). Taking a look (literally!) at the Raven's intelligence test: Two visual solution strategies. 1691-1696.
Improve this question
$\endgroup$
6
  • $\begingroup$ That seems like cheating, since you'd run out of wrong guesses. They cannot know their guesses are wrong. They must arrive to the correct solution through autonomous reasoning, not trial-and-error. But the time given would still be essentially unlimited, as would be their motivation to perform. $\endgroup$
    – AshFromPheonix
    Commented Feb 14, 2022 at 17:24
  • 2
    $\begingroup$ In the SE help, on the don't ask page it says "You should only ask practical, answerable questions based on actual problems that you face" - what's the practical question you're asking/actual problem you seek to solve? $\endgroup$
    – Bryan Krause ♦
    Commented Feb 14, 2022 at 17:30
  • 1
    $\begingroup$ I think you're misunderstanding that page. I'm asking a concrete answerable question, which doesn't presuppose a subjective lens such as ethics, which I'm asking for my own lack of comprehension and struggle with such a question, because the concerned field is psychology. The question is "are all matrix reasoning problems solve-able by anyone given enough time". Clearly culturally-loaded problems aren't solve-able, because they require empirical priors. But matrix reasoning problems have no such priors. Hence the question. $\endgroup$
    – AshFromPheonix
    Commented Feb 14, 2022 at 17:37
  • 3
    $\begingroup$ I agree with @BryanKrause that the thought experiment will likely attract opinion-based answers, so I would focus on a practical question instead. A testable question is "How much does extra time improve performance on Raven's Matrices?" Likely there are diminishing returns, and improvement with extra time is asymptotic to some value. $\endgroup$
    – Arnon Weinberg ♦
    Commented Feb 14, 2022 at 18:18
  • 1
    $\begingroup$ Respectfully, no, I don't think I am misunderstanding it. I think your question needs context as to what goal you are trying to solve. You also need to avoid the XY problem. $\endgroup$
    – Bryan Krause ♦
    Commented Feb 14, 2022 at 19:02

2 Answers 2

Reset to default
1
$\begingroup$

It sounds like you are asking about the functional relationship between time spent on a test item and the probability of answering that item correctly.

In general, the functional form of such relationships for a given individual is sigmoidal. I.e., if you imagine the x-axis is time spent, and the y axis is probability of answering the item correctly for a particular individual, then it will start at the guessing rate (e.g., typically 1 / k where k is the number of response options) and then after a certain period of time it will rise and then plateau.

You could average over individuals to represent a typical time-performance curve for a given item type.

For instance, vocabulary items (e.g., a target word and several response options, one of which has the same meaning) do not take very long to reach the rise and plateau. For the most part, people either know the meaning of the word or they do not know the word. A lot of trivia and general knowledge items would be similar.

In contrast, some items require a somewhat elaborate algorithm to be followed. For instance, some 3d rotation test items require comparison of lots of elements.

Some items are easy given enough time. Perhaps think about certain perceptual speed items, like finding all the "a's" in a grid of letters (e.g., think about Where's Waldo/Wally puzzles). These tests are usually structured as a timed test, in that the ability is really about the speed at which something can be done.

More generally, items can often solved with different strategies that vary in accuracy and in time required. For example, various arithmetic problems (e.g., 37 * 37) can be solved by some quick strategy (involving retrieval and mental manipulation), but spending longer and applying various checks will yield a more accurate answer.

Ultimately, there is a distinction between power and speed tests. Power test don't have a time limit (or the time limit is not meant to interfere with performance substantially), and on speed tests, the time limit is the defining factor. Of course, there are also tests that fall somewhere in between.

From what I know about the Raven's, I imagine having more time would help, but there would also be an upper bound. In some cases, people may think they have found the pattern and be wrong. In other cases, they just would not get it.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Hi Jeromy, good to see you around again. Can you please give some sources to your answer? $\endgroup$
    – AliceD ♦
    Commented Feb 16, 2022 at 1:05
0
$\begingroup$

As time limits are relaxed on the Raven's the correlation between 'working memory' (and conventional IQ score), and raw performance drops drastically. That has been known for over a decade, and yet there are still very strict time limits on fluid IQ tests. Essentially, if you limit time, you force "IQ" to correlate with what can be juggled in short term memory. That is a primary objection to IQ, and a reason why high IQ societies have allowed truly untimed, "Power" tests.

https://www.sciencedirect.com/science/article/pii/S0160289613000445

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.