1
$\begingroup$

There are a number of different interfaces to make an place experiments online, e.g. in questions Software for online psychological experiments that don't require users to download anything or Open source software for running Internet psychological experiments that collect reaction time data.

We have an experimental design that we'd share online, but we've found that one feature may be quite rare - namely counterbalancing. We have 4 conditions, and we'd like participants to be placed in these conditions so that we have more or less equal amount in each by the end. This can be difficult for web studies since it should only count the participants that successfully completed the trial.

Free and open source would be best, we don't have a designated budget to pay for setting up an experiment.

Improve this question
$\endgroup$
2
  • $\begingroup$ Maybe an off suggestion, but I think to add a counterbalancing approach online is going to be challenging; couldn't you simply do that post hoc? $\endgroup$
    – AliceD ♦
    Dec 2, 2017 at 22:14
  • $\begingroup$ Thanks, yea one option would be to have a setup for each type and manually allocate participants. It's not impossible to check the results file for how many there are of each condition and allocate new participants accordingly. It won't be perfect but will be balanced. Simply post-hoc is difficult if we have limited resources and need to make sure we've collected the right conditions with the money. $\endgroup$
    – puslet88
    Dec 3, 2017 at 10:18

2 Answers 2

Reset to default
3
$\begingroup$

You could try www.gorilla.sc It's cloud based software specifically designed for running cognitive psychology experiments online without needing to code. It's not free, but it is affordable - £0.75 per participant.

Counterbalancing of various sorts is built in; randomisers, stimuli set counterbalancers, order counterbalancers. It sounds like the simple randomiser set to balanced will do what you want. As long as you stay on top of rejecting Ns that drop out, you'll get a balance in each condition.

There is a GUI questionnaire builder, task builder and experiment tree builder. So you don't need to do any coding. The vast majority of cognitive science task can be created in it very easily.

Full disclosure, I created Gorilla.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks! This is really useful! We'll have to look at our budget though, we didn't calculate these costs in so far. :) $\endgroup$
    – puslet88
    Dec 4, 2017 at 15:34
  • $\begingroup$ Typically, people use budget set aside for participant fees and commissions or for online hosting of their experiment. From your OP it's sounds like your plan was to use open source software. This usually requires dialing up your own server to host the experiment, which nearly always has a cost. So, with hope that will have been budgeted for. With Gorilla, you won't need to provide your own server as that is all taken care of for you in the 'per respondent' fee. :) $\endgroup$
    – Jo Evershed
    Dec 4, 2017 at 16:58
  • $\begingroup$ Yes, I definitely see the appeal! In this case it's just a small start-up experiment so we haven't planned for anything but the bare essentials. But I know what you mean, and we'll definitely know to keep in mind in the future. :) $\endgroup$
    – puslet88
    Dec 5, 2017 at 19:41
2
$\begingroup$

Qualtrics (survey software) and Inquisit (more sophisticated stimulus presentation software) both run online. Both tools support counterbalancing. Inquisit has extensive support for both randomisation of stimuli in a variety of ways and between-subjects allocations to orders and so on. They both require payment, but some universities have subscriptions to one or both services (especially Qualtrics).

I'm not sure about how you'd solve the issue of drop out to get perfectly even numbers in each group ordering.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks! We'll look into them and our university subscriptions, maybe we find a match! $\endgroup$
    – puslet88
    Dec 4, 2017 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.