Is there a reasonably valid and reliable self-measure of computer literacy?

Question

Measuring computer literacy or skill is important when grouping respondents for usability studies and other Human Computer Interaction studies. But for the life of me I can't find a good, simple but reliable measure of it.

The most frequent suggestion I see is a terribly complicated and non-standard approach like comparing multiple Likert scales for things like "I feel comfortable with computers". I'm fine with multiple Likert scales if it's a proven measure, but hundreds of people making up their own multi-question Likert scales does not create reproducible research.

I'm a proponent of just asking the darn question as I find it less ridiculous than made up scales, but I'm sure HCI work has been done on this.

Is there an agreed upon or otherwise well tested, reliable and valid self-measure of computer literacy? Ideally one which could be used to easily group participants into relative groups of proficiency, such as a Likert scale. By "self-measure" I mean some test item that can be asked on a survey rather than any form of observation.

Answer 1

I don't know of any well-validated general short scales. Here are a few thoughts:

• A major distinction is between self-report versus ability based measures. Self-report measures will ask the participant to rate their knowledge, skills, and experience. Ability based measures will require demonstration of competence. If you are limited to "self-measures", ability based measures are still possible (e.g., set of multiple choice, or possibly short answer questions). An issue with self-report measures is that people may be inaccurate in their own self-evaluation. Ability-based measures do not have this issue.
• Having more items generally increases the reliability of the test. It also permits you to assess statistically the item properties of the scale.
• Computer literacy is a broad construct. The meaning and standards of computer literacy are probably also changing over time, as computing practices change and as the computers get further integrated into society. I assume this is part of the reason why a lot of people use customised instruments. For example, I once did a study of keyboard-based text-editing. To assess prior experience, I asked a number of general questions about computing experience, but I also assessed various specific measures related to knowledge of shortcut keys.

A quick search revealed a couple of scales (e.g., Wilkinson et al, 2010, Fogarty et al, 2001). None look like entirely what you are after, but I thought I'd mention them. Both followed a formal scale development process.

Wilkinson et al

For example, here's a subset of the items from Wilkinson et al

ICT skills
I know how to create a folder
I feel confident closing down a software program
I do not feel confident attaching a file to an email
I can use a word processing application
I know how to save a document to a folder
I would not recognise a PDF (Portable Document File)
I would have difficulty finding a web site∗
I feel confident making selections from popup menus∗
I would find it difficult to search a database of references to journal papers
I need help with using computers for learninga
I feel confident using a computer to write a letter or essay
I do not know how to find an article in an e-journal

Experience with computers
I have used computers at work/school∗
I feel disadvantaged by my lack of experience with computers
I communicate with people using email
I have used computers to look for information for example: Library catalogue/books
I have used computers to support my learning
I have used computers to find out about careers
I have used computers for leisure

Fogarty et al

Here's a subset of the items from Fogarty et al

The following statements refer to your confidence when using computers. 
1. I have less trouble learning how to use a computer than I do learning other things. 
2. When I have difficulties using a computer I know I can handle them. 
3. I am not what I would call a computer person. 
4. It takes me much longer to understand how to use computers than the average person. 
5. I have never felt myself able to learn how to use computers. 
6. I enjoy trying  new things on a computer. 
7. I find having to use computers frightening. 
8. I find many aspects of using computers interesting and challenging. 
9. I don’t understand how some people can seem to enjoy spending so much time using 
computers. 
10. I have never been very excited about using computers. 
11. I find using computers confusing.
12. I’m nervous that I’m not good enough with computers to be able to use them to learn 
mathematics. 

References

• Wilkinson, A., Roberts, J. & While, A.E. (2010). Construction of an instrument to measure student information and communication technology skills, experience and attitudes to e-learning. Computers in Human Behavior, 26, 1369-1376.
• Fogarty, G., Cretchley, P., Harman, C., Ellerton, N. & Konki, N. (2001). Validation of a questionnaire to measure mathematics confidence, computer confidence, and attitudes towards the use of technology for learning mathematics. Mathematics Education Research Journal, 13, 154-160. PDF