22

My research group has gone pure python for coding experiments; we've been burned too many times by glitches and implicit behaviour in boxed experiment-building software to bother trusting it. Moving from a point-and-click experiment design interface to pure code does have a large learning curve, and you want to be careful to model your own code on well ...


16

I'd suggest checking out the Linear Ballistic Accumulator (Donkin et al., 2011) model for a scenario like this. While LBA can be used to model any number of alternatives in a speeded choice task, to model signal detection you'd want to model just two accumulators, one for the "signal" response and one for the "no signal" response. With this scenario, ...


14

PEBL is a free open source programming language to help construct psychological experiments. A wiki and developer community provide some documentation and support. PEBL has a Psychological Test Battery of many common or simple psychological tests so you may be able to use it out of the box without constructing your own test. PEBL includes free versions of ...


14

Sites: The social psychology network has a page for posting online social psychology studies. There are a range of requirements. In particular, it needs to be closely related to social or personality psychology. SampleSize Subreddit is a community dedicated to completing surveys. They like it when results will be provided at study completion. It has close ...


13

For an open source JavaScript/HTML/CSS solution, check out jsPsych: http://www.jspsych.org. It can be used for reaction time measurement and interactive designs. An article describing the library was recently published in Behavior Research Methods. A subsequent article investigated the properties of reaction time distributions collected with JavaScript ...


13

I would recommend Matlab and the Psychophysics Toolbox. It lets you display all sorts of stimuli in full-screen mode, and it lets you capture key strokes and mouse clicks.


13

MANOVA is definitely a bad idea given that one dv is continuous and the other is binomial. After exploring a number of different approaches to combining RT and accuracy data, I've come to conclude that the best current approach is to use linear ballistic accumulator model (e.g., see Donkin et al 2011). The LBA is a simple (structurally and computationally) ...


12

Paper IQ Test: EDIT: There is a list of printable tests including an IQ test at http://personality-testing.info/printable/, it looks to be of standard internet IQ test quality... Here is a printable version of an IQ test developed on this website. It is no where near the quality of a professional IQ test but may be interesting and its internal ...


12

Does the locking refer to the initiation of the measurement with starting cue being being the presentation of stimulus or the response of the subject? More or less, yes. When measuring brain activity, you usually make a long, continuous recording during which you expose your study participants to a task over and over again. There's a lot of noise in ...


12

Antoine Tremblay has just released an advanced analysis toolbox: http://onlinelibrary.wiley.com/doi/10.1111/psyp.12299/abstract It's missing about half the features on your list, although fundamentally, spectral density is a simple task and LORETA is a stand-alone package anyways (although similar approaches, e.g. general CSD estimation, are implemented in ...


12

In speaking to constructs vs. measures, I believe that the difference is clear and implied in your background: constructs are that which cannot be directly measured (but we assume exists), where measures are directly measurable attributes that we assume relate to the construct. The process you seem to be questioning is that of the operational definition, or ...


11

A paper comparing the performance of Inverse Efficiency Scores and diffusion models for the quantification of RT and accuracy can be found here. Rach et al. (2011) "On quantifying multisensory interaction effects in reaction time and detection rate " Psychological Research Volume 75, Number 2, 77-94, DOI , PDF


11

The basic approach that you are describing sounds like inverse efficiency scores (e.g., see Townsend and Ashby, 1978,1983), which are measured as $$\frac{r}{1-e} = \frac{r}{c}$$ where $r$ is reaction time, $e$ is proportion error, and $c$ is proportion correct. John Christie provides a critique of inverse efficiency scores here or see the discussion in ...


10

In addition to Mike's answer, see the Ratcliff diffusion model E.g.: Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two–choice decisions. Psychological Science, 9, 347–356. Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter ...


10

There are a variety of models solving accuracy and RT that have been pretty well tested and LBA is probably fine (I haven't used it). If you don't want to go that far there is a rather simple way to analyze data controlling for SAT that has much better mathematical properties than IE scores (which, as Mike said were named by me, but offhandedly proposed by ...


9

Programs/packages for EEG analysis There are decent MatLab toolboxes with good tutorials for for the analysis of EEG data. The EEGLAB toolbox (tutorial) can be operated by both GUI and command-line (and script). The fieldtrip toolbox (tutorial) is mainly operated by command line / script. Of course there are also (commercial) software packages for EEG ...


9

A consensus within cognitive psychology is that there really is no such thing as "computation", though that depends how you define it. It seems all arithmetic ( + - * / ) is simply fact retrieval from long-term memory. More complex problems simply rely on the fact that we can break them down into simpler problems. For instance, when you learn to solve 364+...


9

I'd like to address important issues that Jeromy Anglim raised in the "Personal thoughts" section of his answer, namely that correlation parameters (i.e., true, population, or infinite-sample correlations) often vary and covary among studies, and this between-studies\interstudy heterogeneity implies heterogeneity in studies' parameters for a structural ...


9

Answer based on your original depression example Note that this answer was originally written based on your initial example, where you asked: Assume, I have developed a new intervention for people with light depression. I want to compare the effectiveness of this intervention (E) with an existing intervention (C). For this, I recruit test subjects ...


9

I have found a list of Python and Matlab packages. I'll summarize them over here. As soon as I have gone through the packages, I'll provide some additional details. Online edaExplorer: Also in Python.* EdaExplorer is a tool that is able to detect noisy data from clean data. Five second epochs are made which will be categorized by a model that is the ...


8

FingerFriendlySoft has created an app for all iOS devices (that is, iPod touch, iPhone and iPad) that is called N-back Suite. This is, as the name suggests, an app which lets you take the n-back test. Included are both the single and the dual n-back test and you can chose different amounts of n (from 1 to 10), five different speeds, and different type of ...


8

OpenSesame is a recent entry that is cross-platform and seems to promote GUI-based design while allowing customization via Python scripting. It can be found at their website (link above). A recent article has references and summarizes 16 other tools as well (including some reported in the other stackexchange responses). I found great video tutorials and ...


8

Bias can be quantified in many different ways. In human memory research carried out in the cognitive psychology tradition, there are simple ways to think about it. One basic measure of cognitive bias is merely called bias, and it's a measure of the absolute accuracy of an individual's probability judgments. You average probability judgments across a given ...


8

If you come to this question from the bayesian tradition, then there is only one place where you can sneak in bias: your prior. This dovetails nicely with the wikipedia definition: a pattern of deviation in judgment that occurs in particular situations, leading to perceptual distortion, inaccurate judgment, illogical interpretation, or what is broadly ...


8

Procedures like this are often called "funnel debriefing" procedures, and they basically consist of an extended version of what you already had in mind. You begin with some very general questions about the nature of the experiment (e.g., "Do you have any initial thoughts or reactions about this experiment?"), then move on to some questions that are slightly ...


8

I don't know of any well-validated general short scales. Here are a few thoughts: A major distinction is between self-report versus ability based measures. Self-report measures will ask the participant to rate their knowledge, skills, and experience. Ability based measures will require demonstration of competence. If you are limited to "self-measures", ...


8

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with ...


Only top voted, non community-wiki answers of a minimum length are eligible