Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Hot answers tagged

6

Here are a couple of suggestions: Inquisit records the timing of each response and can be run online. If you care about timing a lot and you need to run the study online, then this is quite a good option. Qualtrics can record various features of survey delivery time. In particular, you can record time spent on each survey page. If you have one question per ...


6

It's important to distinguish between measures and analyses, because only analyses can be quantitative or qualitative, not measures. Measures are, essentially, systematic processes by which we acquire our data, and analyses are processes we use to look at the data. As a rule of thumb, the difference is not hard to find and is given in the name: ...


6

ANOVA and t-tests are statistical tests for significance and therefore quantitative. The other mentioned items are scales (adding numbers to a certain choice) and therefore they can be considered as ordinal scales, and hence as quantitative as they are based on numbers. The NASA one can be administered by using a sliding scale which can be considered to ...


5

It depends on the types of changes you are looking to make. In general, my experience has been that changes to a response scale can be done with minimal threats to validity (e.g. looking to change the 4 item Likert type scale to a 6 item Likert type scale). On the other hand, changing question wording itself usually can't be done "without losing validity" (...


5

Murphy & Cleveland (1995) mention, that a good way to reduce rater errors in general is to inform raters of the existence and nature of these errors and then to simply urge to avoid them. While this reduces rater errors, it also decreases the accuracy of ratings, though. These findings come from the literature on performance assessment, where halo is ...


5

I know that Helen Fisher, who is a love researcher, has developed a four factor personality model where two of the factors are quite relevant to your question. See (Fisher et al, 2015) for some general information regarding the personality model. The two factors that are relevant to your question are called (1) analytical/tough-minded, and (2) Prosocial/...


4

The best review of experience sampling tools I've found is here. Specifically, to answer you question, check out "MyExperience". To quote the website: MyExperience is a BSD-licensed open source mobile data collection tool developed for Windows Mobile devices (including PDAs and mobile phones) using .NET CF 2 and Microsoft SQL Compact Edition. ...


4

You seem to be concerned with reliability, and more specifically internal reliability. Internal reliability is the degree to which different questions are measuring the same construct. This concept is used often in psychology and is usually measured using Cronbach's alpha. However, it is typically used to measure the reliability of a test, and not the ...


4

You could obtain paid participants from https://www.prolific.ac/ That site allows you filter by country. The prices are reasonable, especially if your survey is short and you don't need too many participants.


4

There is probably a better way of doing this, but you could probably do this using the block feature in Qualtrics. For example, I often do this when I have several multi-item scales. Each scale is a block. You might randomise item order within a block, but you can also randomise the order of blocks. So you could group questions (e.g., 1 and 1a) into their ...


4

It's probably better to see if any validated indexes/tests exist, and some do e.g. https://doi.org/10.1177%2F0013164492052002016 Despite its relevance to a wide variety of situations, competitiveness is a personality characteristic that has not been widely studied. Although some research in need for achievement, sports psychology, experimental social ...


3

One that I used at my university is Qualtrics. It was under the university's license, so I don't know what the costs are for single users. Qualtrics allows you to ask many different types of questions (open ended, likert, option select and more) and can easily export the data to excel files for further processing. Survey Monkey is another tool that is often ...


3

Many construct scales are developed with relatively little attention to the content. Researchers may start with a large set of candidate questions, versions, formats, and phrasings, and then through a validation process, narrow them down to a subset that correlates well with the construct that they are trying to measure. The goal is a balance between ...


3

This is called "demand characteristics": ... an experimental artifact where participants form an interpretation of the experiment's purpose and unconsciously change their behavior to fit that interpretation. An example is: The good-participant role in which the participant attempts to discern the experimenter's hypotheses and to confirm them. ...


3

Self-report methodology was one of my qualifying exam topics as a doctoral student of social and personality psychology, so I've got a ton of references to offer, but I confess I haven't read most of them very thoroughly (if at all), and I've forgotten where exactly I've read some of this. It's really a very broad topic as well, so I won't list most I know ...


3

Preventing random responding: An important first step is to think about ways to prevent random responding from occurring in the first place. A few ideas include: administer the survey face to face; have an experimental invigilator present; communicate the importance of the research to participants and the importance of participants taking the research ...


2

This is probably a bit old but there is a free iOS app called the PIEL Survey. It is in the App store. There is a web site which explains how to set up Experience Sampling surveys.


2

This is just a few thoughts. I agree with @adb that an obvious way to use facebook for conducting surveys is just to provide a link to an external website that runs the survey. There is an interesting article by Tan and colleagues where they talk about using paid advertisements to get targeted participants on Facebook. I thought the following statistics ...


2

So I revisit this question every once in a while, constantly starting an answer with some resources, and then deleting it. Long story short: I think that there are several places where such items could be found, though I think "good" can be hard to define (as it is so situational)! First off, here is a link that provides a few options for most of the ...


2

This is not directly an answer to your question but, in line with my comments to another answer, my main advice would be “don't worry about it”. Jeromy Anglim's tips are all good but I am still unconvinced that this is an important issue for most people. Since you are new to research, there are probably dozens of other things you should worry about. ...


2

Question: if you have used such softwares to make experiments, what did you think about the process? We use Paradigm (www.paradigmexperiments.com) in our lab and like it quite a bit. You can build most experiments just with the drag and drop builder interface. It also supports Python scripting just to give you more flexibility (and piece of mind in case ...


2

First, a minor point is that the TIPI is a very poor measure of individual-level personality. Two item measures of personality are rarely justifiable. Standard errors of measurement are unnecessarily large. In general, you are talking about the topic of norms and standardized scores. Any introductory book on psychological measurement will discuss topics ...


2

Good answer above by Fizz. Also, consider what other well-studied traits are similar. Here's a long list: http://www.sjdm.org/dmidi/All_Scales.html It sounds to me like some items from the Big Five could overlap. For example, in these facets (each would have multiple items): Extraversion: Assertiveness (forceful), Activity (energetic), Excitement-...


2

This is barely on-topic here (as it's more of an economics question), but Kurisu (2016) says This chapter provides an overview of pro-environmental behaviors (PEBs). There is no catchall definition or way to categorize PEBs; therefore, I propose various definitions and ways to categorize PEBs. Two main definitions for PEBs are shown here: purpose oriented ...


1

In my opinion, test-retest method would be suitable to your request. This procedure is quite simple: individuals are asked to take the test and then take the same test again at a later date. The scores are then compared. The closer the scores are, the more reliable the test. This methodology is appropriate for instruments such as IQ tests and surveys ...


1

There are lots of commonly observed biases in surveys, and a whole field of survey methodology and statistics which investigates them. To get an idea of this, take a look at the table of contents of these journals: http://poq.oxfordjournals.org/content/current http://jssam.oxfordjournals.org/content/current Many people in this field are dealing with large ...


1

I made up a SVG version connected to a hidden html form. It's not perfect, but it may be useful anyway. You can find it here. Note that a sufficient screen resolution is required. Please send me feedbacks if you use it so that I can improve it.


1

Both Carol Dweck and Angela Duckworth have fairly brief tests meant to measure student success and by some extension grades. Dweck's work focuses on how having a "fixed mindset" limits a students ability to grow in a subject and is used to explain why students are so apt to give up when met with challenge. Her quiz would have to be formatted for pen and ...


1

The best way to set up a Mechanical Turk longitudinal study is with TurkPrime.com. You can follow-up with users and invite them to take a new survey. As long as they are properly incentivized they will come. I recommend that you inform them at the start that you are running a 10 part study and that those who perform well on each of the 4 minute studies will ...


Only top voted, non community-wiki answers of a minimum length are eligible