I am a Master's student focusing on psycholinguistics and I am preparing for my thesis. What I will be looking at are register effects and for that I will be conducting 2 online experiments using Ibex (participants will hear context sentences, be presented with a set of visual stimuli and then react to the visual stimuli). As far as I know, online RT studies are not as reliable as those which are conducted in the lab in the sense that they are noisier and less accurate. There is a great thread which discusses this topic How valid are reaction times collected from online studies? and where it is claimed that online RTs studies can be reliable if they are well powered. All in all it makes sense but I am confused as to how many participants are too little and how many are just enough. Your help is very much appreciated!
There are four variables of interest in a statistical power analysis: power (often chosen as 0.8 or 0.9 by convention, this is the probability of correctly rejecting the null hypothesis if it is false), tolerated false-positive rate (alpha, often chosen as 0.05 by convention), sample size, and the effect size (which in turn depends on the variability of the observations and the difference between groups).
When you do a power analysis, you can choose any one of those variables to calculate provided you have all the others. In your case, you want to know the sample size you need so you need to choose or estimate the other numbers.
If you know what statistical test you are using, there are numerous online calculators to help you choose a sample size. Personally, I like to use the free software G*Power.
Power and alpha are typically chosen by convention, so what you still need to know to determine your necessary sample size is the effect size you are looking for. If you have an experiment that you expect to have a large impact on reaction times, you don't need as many participants. If you are looking for a subtle effect, you need more. You can choose that threshold based on what is reasonably interesting/clinically relevant (this is ideal but sometimes frustratingly difficult to settle on), or you can base it on pilot data (can be dangerous when pilots are based on small sample sizes).
If you don't have any preliminary data, the best way to estimate the variance in observations would be to look at other similar studies collecting online reaction time data.
Overall, if you want to make sure you have a sufficiently powered study, it's good to use conservative estimates. If you guess that your effect is smaller and the variability is larger than what you hope, then you have a better chance of a sufficiently powered study.