Duration-independent intelligence tests

Question

Being able to measure intelligence (or the problem-solving aspect of it) quantitatively could be indispensable in fields like cognitive science or artificial intelligence. Obviously, IQ tests aren't culture neutral (questions are stated in an informal language, and often rely on prior knowledge). Ideally I am looking for these characteristics:

1. measure problem solving ability
2. be culture neutral
3. be of unrestricted duration

Raven's Progressive Matrices look almost perfect for the job (variations can be developed for testing animals or even computer programs) but they, like EVERY intelligence test i know of, require test to last not more than a specific amount time, thus, failing to satisfy the third requirement. So I guess the question is whether such a test can be constructed at all. In other words, is there a category of problems that require some minimal intelligence level to solve them no matter how long one might try. Like the problem that could be solved in a minute by a smarter person but is impenetrable to you no matter how many years you spend thinking about it?

Answer 1

So I guess the question is whether such a test can be constructed at all. In other words, is there a category of problems that require some minimal intelligence level to solve them no matter how long one might try. Like the problem that could be solved in a minute by a smarter person but is impenetrable to you no matter how many years you spend thinking about it?

If that is your question, the answer is no. No problems like this are currently known to exist, and there is reason to believe that such problems will never be found.

Intelligence in not like "this person can do A" and "this person can do A and B", but rather more like "this person can do X with medium reliability" and "this person can do X with high readability".

Answer 2

Actually, standard IQ tests, such as Raven's matrices, tend to assess intelligence better if they are not timed. In this paper by Philip Vernon (1988) it was found that the g-factor extracted slightly more variance for the same test if the test was not timed than if it had a time limit. This means if you ask yourself: "What is this test measuring?", you can answer: "Intelligence" with more confidence if it was administered in an untimed setting.

IQ tests measure a host of other things, such as how well you are feeling on the day of the test, whether you are nervous when it comes to test-taking, how aquainted you are with this type of questions, etc. Giving people more time can attenuate some of these factors a bit. Timed variants are more prevalent for reasons of convenience, and also because it's easier to standardize the test situation.

It has been argued that there is little difference between timed and untimed tests because intelligence closely corresponds to working memory capacity (Kyllonen & Christal, 1990):

Confirmatory factor analysis yielded consistently high estimates of the correlation between working-memory capacity and reasoning ability factors (r = .80 to .90). We also found differentiation between the two factors: Reasoning correlated comparatively highly with general knowledge; working-memory capacity correlated comparatively highly with processing speed.

Some IQ tests even have subtests which measure this capacity directly: repeating a row of numbers in the same order or backwards. Giving you additional time will not help you reproduce the row of numbers, if it was initially too long for you to remember.

From the high correlation between working memory capacity and processing speed, it was concluded that IQ tests largely measure the ability to vary features of multiple items at the same time, i.e. to manipulate multiple objects in working memory. Crucially, during administration an IQ test such as Raven's matrices, you are not supposed to use pencil and paper or other cultural crutches: you work on the items 'online' in your head (Engle et al., 1999). The more objects you can hold 'online', the better you will do. Recent research (Colom et al, 2008) goes even further to say that processing speed is not a good predictor on its own, once working memory capacity is removed:

The findings are consistent with the view that simple short-term storage largely accounts for the relationship between working memory and intelligence. Mental speed, updating, and the control of attention are not consistently related to working memory, and they are not genuinely associated with intelligence once the short-term storage component is removed.

And of course, as always with measures of individual differences, it is important to keep in mind that IQ doesn't measure how many questions you completed - it measures how you rank in a group of peers which has done the same test under the same conditions.

References:

• Vernon, P. A. (1989). The generality of< i> g. Personality and Individual Differences, 10(7), 803-804.
• Kyllonen, P. C., & Christal, R. E. (1990). Reasoning ability is (little more than) working-memory capacity?!. Intelligence, 14(4), 389-433.
• Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. (1999). Working memory, short-term memory, and general fluid intelligence: a latent-variable approach. Journal of Experimental Psychology: General, 128(3), 309.
• Colom, R., Abad, F. J., Quiroga, M. Á., Shih, P. C., & Flores-Mendoza, C. (2008). Working memory and intelligence are highly related constructs, but why?. Intelligence, 36(6), 584-606.