I often analyse data that involves participants completing a selection of psychological tests online. A concern with online testing is that some participants may not complete the task conscientiously. For example, they may response options at random or just click the same response option repeatedly.
In general, I like to get response times per item, as I find that they are useful in diagnosing people who are deliberately skipping through the test. However, there are cases where that is not possible.
To make discussion concrete, think about where you have responses to 10 psychological scales. Each scale has 20 items. Some items may be reverse coded.
- What is a good way to identify participants who were not concentrating or not completing the items conscientiously?
- How do you ensure that you don't filter out participants with unusual patterns of responses?