29
$\begingroup$

I've often used Inquisit to run psychological experiments online. The software enables delivery of stimuli (e.g., text, images, etc.) and collection of reaction times.

Obviously general purpose programming languages provide one avenue for delivery of online experiments. There's also a good listing of software for psychological experiments here. I've also seen PsyToolkit which is GNU licensed software for programming psychological experiments, but I'm not aware of any online option.

Are there any open source options for online delivery of psychological experiments, particularly ones that enable reaction time measurement?

Improve this question
$\endgroup$
7
  • $\begingroup$ I'm curious about this too. I use Adobe Flash to write psychological experiments -- the opposite of open source! $\endgroup$
    – Andy DeSoto
    Feb 6, 2012 at 22:27
  • 1
    $\begingroup$ HTML5 is really capable and could be used to construct reaction time experiments. While I have no data on response registration latency, I can't imagine that it is that bad. A plus with using HTML is that you can run pilots (or studies) on Amazon Mechanical turk, which is a very convenient way of rallying participants. See for example: Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data? Perspectives on Psychological Science, 6(1), 3-5. doi:10.1177/1745691610393980 $\endgroup$
    – Rasmus Bååth
    Feb 20, 2012 at 8:43
  • $\begingroup$ I think the question (or answers?) might need a bit refinement in that response time measurements are easy in terms of accuracies in the order of seconds, but it becomes exceedingly difficult if tens-of-milliseconds-accuracy is necessary. $\endgroup$
    – AliceD ♦
    Mar 9, 2015 at 12:28
  • $\begingroup$ @Christiaan that's not accurate. Many web technologies are accurate on the scale of tens-of-milliseconds. $\endgroup$
    – Josh
    Jun 7, 2016 at 21:40
  • $\begingroup$ @Josh - then why did I have to work for months to get my laptop running within the millisecond-range accuracy? Many applications may claim they do well, but it needs thorough validation. For example, a USB port imposes an 8-ms delay on anything you throw in there due to polling. At least on a Windows machine that is. $\endgroup$
    – AliceD ♦
    Jun 7, 2016 at 21:42

12 Answers 12

Reset to default
13
$\begingroup$

For an open source JavaScript/HTML/CSS solution, check out jsPsych. It can be used for reaction time measurement and interactive designs. An article describing the library was recently published in Behavior Research Methods.

A subsequent article investigated the properties of reaction time distributions collected with JavaScript compared to those collected with MATLAB and Psychtoolbox. The main result is that JavaScript was 10-40ms slower, but had equivalent variance across different experimental conditions and equal sensitivity to the experimental manipulation of set size in a visual search task.

There are several good answers to this related question about the validity and accuracy of response time measurements online. Many of those answers discuss findings that are relevant to JavaScript libraries like jsPsych.

de Leeuw, J. R. (2015). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods, 47(1), 1-12.

de Leeuw, J. R., & Motz, B. A. (2015). Psychophysics in a Web browser? Comparing response times collected with JavaScript and Psychophysics Toolbox in a visual search task. Behavior Research Methods, advance online publication.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ thanks - this look really promising and also very current - last update on github github.com/jodeleeuw/jsPsych was only 2 weeks ago $\endgroup$
    – jacanterbury
    Jun 5, 2014 at 14:36
7
$\begingroup$

Another option is ScriptingRT. It's open source and fairly easy to use. The experiments are designed via a script language and then compiled into Flash apps.

ScriptingRT is designed to measure reaction time differences in the millisecond range. Schubert et al. (2013) report comparisons with other response time programs (e.g., DMDX, e-prime, Inquisit, see Studies 1 and 2).

The data suggest that measuring reaction times with ScriptingRT is associated with a little bit more noise than on lab computers. However it is sufficiently precise for the type of questions typically addressed in psychological experiments.

From Study 1:

the SDs of these responses stayed below 7 ms in all three browsers. That value is comparable to many regular keyboards and standard reaction time software. In addition, the constant added by measuring in ScriptingRT was about 60 ms. This result suggests that researchers using ScriptingRT should thus focus primarily on differences between RTs and be cautious when interpreting absolute latencies.

From Study 2:

ScriptingRT resulted in both longer response latencies and a larger standard deviation than all other packages except SuperLab and E-Prime in one configuration. Nevertheless, in absolute terms, the SD of 4.21 is comparable to what was standard for keyboards for a long time [16]. It is thus clear that any test with ScriptingRT should be well powered and used to assess primarily paradigms with a large effect size.

Schubert, T.W., Murteira, C., Collins, E.C., Lopes, D. (2013). ScriptingRT: A software library for collecting response latencies in online studies of cognition. PLoS ONE 8: e67769. doi:10.1371/journal.pone.0067769

Improve this answer
$\endgroup$
0
5
$\begingroup$

We just released a beta version of Tatool Web based on JavaScript and HTML 5 which allows for running web experiments in the browser and measuring reaction times. You can check it out on www.tatool-web.com and of course it's open source.

http://www.tatool-web.com

Improve this answer
$\endgroup$
4
$\begingroup$

I'm not sure if it can measure reaction times but Tatool, developed at the University of Zurich, is an open source experiment platform that can be run from the web:

http://www.tatool.ch/

Improve this answer
$\endgroup$
1
4
$\begingroup$

Limesurvey is worth checking out (more suitable for questionnaire style tasks, but very flexible and with some coding it should be possible to, eg. record RTs)

Wextor could be another possibility - it allows building more complicated designs, has not been developed for a bit, though...

Improve this answer
$\endgroup$
4
$\begingroup$

This is in development within PsychoPy. Porting has begun on the Python lib underlying PsychoPy into a JavaScript lib (PsychoJS) and the Builder graphical interface can now generate the necessary html/js code for that.

The technology on the backend is using WebGL where available (nearly everywhere, including our phone) and this should give good timing in that it synchronises to the screen refresh.

It's early days and there aren't data on RT precision yet but expectations are high. And all this is open-source, of course.

http://www.psychopy.org/online/online.html

Improve this answer
$\endgroup$
3
$\begingroup$

WebExp is a free framework for developing web-based experiments, and the source code is freely available. The client side is a Java applet, so subjects must have Java installed on their computers. Perhaps this causes fewer drop-outs than having to install the executable generated by Inquisit Web. This paper discusses the timing accuracy of WebExp-based experiments.

Improve this answer
$\endgroup$
1
  • $\begingroup$ from a quick look at their website it doesn't look like this product has been worked on since 2009 :( $\endgroup$
    – jacanterbury
    Jun 5, 2014 at 14:33
3
$\begingroup$

To complement the answers, there's also Just Another Tool for Online Studies: JATOS (disclaimer: I'm one of the authors). It's an open source tool that focusses on the server side. It will provide:

  • a secure server
  • a database (MySQL or H2)
  • participant management (optionally prevents repeated access)
  • a graphical user interface to access results
  • run group studies in real-time
  • a few etceteras.

Because JATOS focusses on the server side, it doesn't restrict in -almost- any way what you run in the browser (i.e., what you do in the client side). That means that it's actually complementary to a few of the tools that other answers suggested, that will mostly give you client-side solutions to present stimuli and measure reaction times in the browser. For the latter, I use the performance.now method in JavaScript that runs in most browsers and works pretty well. Probably not for careful priming or multisensory integration-type experiments, though.

Improve this answer
$\endgroup$
1
$\begingroup$

I would highly recommend looking at http://www.qualtrics.com/. You can take a look at free trial accounts and see how its easy to use interface is still extremely sophisticated – e.g. automatic option to record response time – and can be augmented with javascript when you want to.

I'm not sure of pricing because many universities often buy a site license. But Qualtrics has revolutionized the way we do experiments.

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ Hi Joseph. Welcome to the site. It's great to have someone with your background and expertise involved on the site. I guess Qualtrics isn't open source. Is Qualtrics mainly for surveys or can you do experiments involving multimedia, reaction time, randomisation of stimuli, etc? $\endgroup$
    – Jeromy Anglim
    Apr 27, 2012 at 13:10
  • 2
    $\begingroup$ The questions clearly asks (in bold) for Open Source Options and Qualtrics is a commercial package. Also, although I don't think the builtin timing features will have the precision of psychology response time type experiments without writin gyour own Javscript. $\endgroup$
    – jacanterbury
    Jun 5, 2014 at 14:27
1
$\begingroup$

You should have a look at otree, "a Django-based framework for implementing multiplayer decision strategy games".

It is manly designed for experimental economics, but it is a very neat and versatile piece of software, and I am sure you could actually run most psychological experiment with it.

Once the experiment is coded, it only requires internet access and a browser on the part of the participant.

Big plus : otree comprises functionalities to connect with Amazon Mechanical Turk.

Caveat : I don't know if collecting reaction time data is a baseline option, but it should not be too hard to code into the framework.

Improve this answer
$\endgroup$
1
$\begingroup$

Gorilla is cloud based software specifically designed for running reaction time behavioural science experiments online. It's not free, but it is affordable (£0.75 per respondent) and interfaces with popular recruitment services like mTurk and Prolific Academic.

There is a GUI questionnaire builder, task builder and experiment tree builder. So you don't need to do any coding. The vast majority of cognitive science tasks can be created in it very easily.

You can also add snippets of JavaScript to add custom functionality or use the built in code editor tool to create a completely bespoke task.

Disclosure: I built the Gorilla platform.

Improve this answer
$\endgroup$
1
$\begingroup$

I am one of the core developers of https://www.labvanced.com, a very new and probably the most advanced online platform for creating, sharing, and conducting professional behavioral experiments.
There is a complete visual interface, so no need to program a single line of code, but still you can still write if statements, while loops, and asynchronous loadings etc. In fact any arbitrary logic, presenting video, audio, text and many more elements is possible, at the same time ensuring spatially and temporally very precise presentation in fullscreen mode.
From a data-recording perspective we allow, recording of any mouse and keyboard events, voice recording via microphone and eye and face tracking via webcam. We also offer automated text translations to over 50 languages, and have integrated crowdsourcing capabilities (mTurk etc.) to millions of participants worldwide.
Finally all experiments can run online in the browser or locally on your computer / in your lab, and you can simple start by using one of our over 50 experimental templates on https://www.labvanced.com/expLibrary.html
Most importantly there is a free version of our software and we are currently working to open source the source code on github. Just check it out at https://www.labvanced.com.

If you have any more quesiton, feel free to write me message.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.