1
$\begingroup$

In terms of presenting and implementing experiments, which programming language is better?

Note: What we are talking about is not data processing but experiment presentation.

What I need:

  • Experiments can be implemented on many computers at the same time, since the number of participants may be sometimes large.
  • Sometimes I need accurate timing to get parameters such as the reaction time in the millisecond range. Both the magnitude of the reaction time, as well as timing differences between conditions are of interest.
  • I prefer open-source programming environments, yet I have access to MATLAB for instance and finds are available.

What I've considered so far:

  • Python. I've only tried psychopy.

    • pros: Accurate Timing. It is claimed that the timing function is nice.
    • cons: Robustness. The psychopy programs are not robust enough and sometimes I encounter annoying problems and the community is also, although good, but not good enough in my opinion.

  • Html/JS + Backend Server. I've tried the scheme to write html/css/js to present my experiment and send data to the local server which I wrote with flask in Python since js cannot read and write files through the browser.

    • pros:
      • Flexible. With this scheme, I can do almost all I want to do and even more simply with the web app development technique.
      • Apparence. The front-end, i.e. the UI of the experiment can be as pretty as possible since there are many tools to enhance the performance of html/css/js.
      • Online Data Collection.
    • cons: Accurate Timing. ​I guess timing is a disadvantage for this scheme. I don't know how better is the optimization of psychopy or E-prime than simply using javascript, but I guess the measurement of parameters such as reaction time can be more inaccurate. If javascript is enough to do this, please tell me!

  • C#/QT(c++ or python)/VB.NET etc. These are similar to the previous html scheme while the presentation and data collection can be done in a desktop app together, but I have no idea about the timing problem as well.

  • E-prime. I'm not very familiar with E-prime so if there are any mistakes in the description please point them out, thanks!

    • pros:
      • Accurate Timing.
      • Robustness. As a well-known commercial experiment software, I believe e-prime is, to some extent, better than psychopy.
    • cons:
      • Closed Source. I do have money to pay for it and if it's indeed irreplaceable and I'll use it, but if I have other choices, I prefer open-source :)

So, any suggestion on this question? or is there some recommendation? Very much thanks!

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Hey C.K., are you interested in general pros/cons or a recommendation for your particular situation? Also are you interested in languages specifically or are you thinking about software programs? If you want a specific recommendation, more information would be helpful. For example, do you need to collect data online? Do you need precision timing? Prefer open source? Do you have your own server space, or money to pay for services? PS. I think e-prime tasks should run without the dongle. (someone please correct me if that's not the case) $\endgroup$
    – pep
    Jun 20, 2021 at 18:40
  • 2
    $\begingroup$ Welcome to Psychology.SE. I feel this question could lead to an answer which could fill a textbook. Could you please separate your question into several different ones (what are the pros and cons of Python, what are the pros and cons of C#....) This way it provides more focused questions and answers. $\endgroup$
    – Chris Rogers
    Jun 21, 2021 at 3:31
  • $\begingroup$ @P.P. Very much thanks! Sometimes I do need precision timing and indeed I prefer open-source therefore I tried psychopy. I'd directly play with html/js if no precision timing is needed :), and I have money to pay for services, that's no problem. What I'm considering about it only precision timing with JS. However, I found that psychopy is not that robust for me so I'd reconsider this question. As for e-prime, I'm no very familiar with that, so there may be some mistakes in my description and thanks for pointing out them. $\endgroup$
    – C.K.
    Jun 21, 2021 at 3:39

2 Answers 2

Reset to default
3
$\begingroup$

If the magnitude of reaction times in the millisecond range are necessary, the options to do decent experiments are very much limited. However, often reaction times are compared between situations and then relative reaction times are sufficient, which greatly reduces the requirement of accurate measurements, although the precision still needs to be good. The latter, however, is much easier to ascertain even with mediocre systems.

Focusing on the reaction time problem here; the problem is that many platforms based on Windows are not suitable for accurate measurements in the millisecond range, despite processors operating in the Gigahertz range. This is because Windows has its own, illogical, pesky and utterly inefficient internal queue of processes waiting in line to be executed, including executing utterly irrelevant updates while you are doing attempting to do your reaction time experiments. Button presses may be coming in delayed or execution of the next part of your script may be postponed because of some other background process irrelevant to your experiment. MacOS and especially Linux are much better platforms in that regard as they tend to have less of a mind of their own.

Then the programming environment. To be frank, I only have experience with MATLAB, which is not designed for reaction time experiments, although it can be fast. However, running on a Windows machine it will be massively sluggish. To circumvent these issues I have used the (open access) Psychophysics3 toolbox for MATLAB. This toolbox can prioritize the script to put it somewhat on top of the queue to enhance timing. However, it is still far from perfect and in my hands the delay was nonetheless about 40 ms between command execution and (sound) output as measured from the soundcard. So that's not great. It seems that other systems are able to reduce this latency to the single digit millisecond range. I can recommend sifting through the introduction of the PTB3 toolbox and take the time to look at the excellent PTB3 forum which has many excellent posts on reaction time experiments in using PTB3.

From what I understand, C / C++ are great environments for speedy and reliable timing. In addition, letting dedicated hardware take over the job of keyboard or mouse button presses accelerates system delays (USB polling times, especially sluggish with keyboards), such as dedicated response boxes.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thanks for your very nice introduction! I got it! In terms of accurate timing, the better practice may be implementing offline experiments on computers with Linux-based OS and timing function optimized software or packages, but the choice finally depends on the requirement of precision. Is that true? $\endgroup$
    – C.K.
    Jun 22, 2021 at 9:02
  • 1
    $\begingroup$ @C.K. that sums it up pretty nicely. I would seriously suggest sifting through the forum questions, there are excellent and detailed posts of Mario Kleiner in there. $\endgroup$
    – AliceD ♦
    Jun 22, 2021 at 9:08
  • $\begingroup$ Generally I'd recommend going with dedicated hardware if millisecond-precision RTs are needed. $\endgroup$
    – Bryan Krause ♦
    Jun 22, 2021 at 15:15
2
$\begingroup$

All of these are pretty great options, to the point where the answer is probably just 'use the one you're most comfortable with'. (Although open source is a huge plus)

Javascript can do pretty decent timing, (for example here: https://link.springer.com/article/10.3758/s13428-021-01573-x). There's still "losing control of the rest of the environment" if your study is online, but the js itself isn't a strongly limiting factor.

You might enjoy the timing mega-study by Bridges & friends, https://peerj.com/articles/9414/ which goes into this stuff in detail. It has a nice up-to-date collection of presentation platforms, and does the digging into all their timing properties so that you don't have to.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Very much thanks! The papers are really helpful for me! According to the latter paper, it seems javascript can get a pretty good timing despite a relatively large lag at the milliseconds level (less than 50ms). So, relative timing comparison may be good with the same environment whereas absolute timing should be measured in lab-based e-prime, psychtoolbox, psychopy, etc. $\endgroup$
    – C.K.
    Jun 22, 2021 at 0:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.