In a 2006 paper on Strategic Management Simulation (SMS) it is stated that:

SMS performance data are collected via a simulated real-world equivalent task where the participant is able to engage in various self-generated and sequential actions across several hours to deal with an interrelated flow of concurrent problems. Participants perceive the simulated setting as quite realistic and meaningful. As often paralleled in real life, the simulated task continues over a sustained period of time (for example, to allow for sequencing, planning, development of strategy, etc.). In contrast to several other assessment approaches, the SMS allows participants to respond in relevant, creative, and/or diverse ways to specified contextual demands and to more open-ended challenges that are provided by a computerpresented scenario. Participation in SMS requires a 3- or 4-hour period. Before starting the simulation, participants are given a manual about the situation they are about to encounter. They view a video "newscast type" presentation about current and prior relevant events in their task setting. Maps and relevant props are provided. [...]

Participants can engage in any actions/decisions that they view as relevant. They can initiate actions or continue action sequences at any point in time. In contrast to many other research settings (for example "microworlds" and most paper-and-pencil tests), a participant need not wait for computer-provided information before engaging in an action, and need not respond to each information item. Action/decision texts generated by a participant are entered into the simulation computer (together with received information that led to the action, if any, relevant prior participant actions, and future plans, if any). Entered texts are translated into six digit numerical computer codes that are later used by the system to calculate performance scores.

Performance is computer scored and calculated, eliminating potential sources of bias that are often introduced in other (observer or participant based) simulation measurement systems. The simulation generates data indicating how a participant manages multiple and sequential task components under normal (routine) conditions as well as in emergency situations. Scores on nine measures of functioning allow performance comparison between different simulation participants.

Although the correlation of SMS-generated measures with other tests, including earlier and later WAIS intelligence test formats, are low (.27 or lower) predictions of real-world success [e.g., (a) income at age, (b) job level at age - both corrected for industry and location, (c) number of promotions over a fixed time period, (d) number of supervisees as well as (e) 360 degree success ratings (cf. Streufert et aI., 1988; Streufert & Swezey, 1986)] are four times as high as predictions of success via intelligence measures.

Those are pretty bold claims in the para I've highlighted. (They pretty much claim the golden goose for employment tests, among other things.) Also they claim high test-retest reliability for SMS of 0.92.

I'll note that while this paper is fairly obscure (a handful of citations), and apparently so is an earlier one from [roughly] the same authors (SATISH, U. , STRUFERT, S., & ESLINGER, P. J. (1991). "Simulation-based cognitive rehabilitation following mild-moderate head injury". Journal of the International Neurological Society, 7,174-175.) some subsequent papers applying SMS to detect the effect of environmental effects [instead of head injury] on SMS performance have been fairly cited--like >600 citations for one on CO2).

I honestly hand't heard of SMS style tests until I ran into these [latter] bunch of papers, all from the same group (or at least Satish being a co-author).

I'm really intrigued by the fact that SMS has low correlation with WAIS though. So, do such style of management games/simulations generally correlate poorly with WAIS, and even with [any of] its subscales? Do such management games correlate really not correlate well with any other cognitive tests? The quote essentially claims they are unique and basically "untestable" in any other [known] way... That kind of claim is repeated in more recent papers of Satish et al., e.g. (2019):

The SMS is unique in that it assesses the process of adaptive decision-making (planning, execution, and monitoring), whereas other psychometric tests typically assess individual or more limited sets of executive functions.

Seldom does one find things/tests in psychology that don't correlate with anything else though.

  • $\begingroup$ I've tagged this with IO psychology since the earlier papers (before trying to use SMS for head injury and then environmental effects) were in that area, e.g. onlinelibrary.wiley.com/doi/abs/10.1111/… So maybe SMS-style tests correlate with something in that area. $\endgroup$
    – Fizz
    May 20 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.