How does one identify relevant variables when performing research on perception?

Question

Suppose, hypothetically, that I wanted to do a blind taste test into preference for Coke or Pepsi. I organize hundreds of volunteers and find that a certain group prefers one and another group the other.

As I work for some marketing company, they ask me to account for this. After all, they reason, preference for Coke vs Pepsi is not random.

Perhaps there's a socioeconomic explanation, perhaps it relates to gender, hormones, age or ethnicity. Who knows! There are hundreds of variables commonly studied in the academic community.

Even if I find a couple of variables that correlate, say hypothetically, BMI and employment status, I have no way to determine if those are "good" variables or if this is just a coincidence.

How are researchers in psych experiments trained to deal with this sort of scenario?

Answer 1

You use the words "experiment" and "correlate" somewhat loosely in your question, but there are various ways that we can address the general idea in a more rigourous way. Let's consider two possible situations:

(1) You randomly put people into two groups and found that group A likes pepsi much more often than group B.

(2) You had an idea that people with, say, higher employment status would be more likely to like pepsi. So you deliberately split people into A (managers) and B (subordinates) and found that group A likes pepsi more often.

If the situation is (1), and the difference in preferences is a large one, then you might want to look at the groups and see in what ways they are different. It would be perfectly fine to observe that the groups were different on, say, BMI, and you could speculate that that is why one group prefers pepsi more often. But it wouldn't be very convincing because there could be many other ways that A and B differ, some of which you probably haven't even thought of or can't measure, and there is no way of being sure which factors align with preferences, let alone what causes them.

The situation (2) is much closer to a "true" experiment. Based on previous research or results you could isolate a particular factor and make sure that the groups differ only in that way. In more refined experiments you could try to exclude potential confounding variables (e.g., by making sure your managers and subordinates were all the same sex, and had equal BMI). Ideally, you could even try to manipulate the factor within the same participants and see if preferences changed (e.g., if a subordinate gets promoted, do they change their preferences when you test them a year later).

Answer 2

Researchers are trained to address this problem by using literature reviews to isolate the specific variables that should be of most interest. At the start of research you generally examine the knowns and unknowns in the particular area of interest, so as to determine the state of the art in that area. Once you know the state of the art you have a better idea of how to proceed with the research.

For example, if you found that something was known about your research area then you could use this literature to justify looking at a specific set of variables when examining the problem. This relevant prior literature need not have examined the exact same area as the problem you are looking at. For example, if you were looking at factors that influence preferences for coke and pepsi you could look at research that examined brand preferences, or even something more fundamental such as the factors that lead to positive beliefs and attitudes. This research would give you an idea as to which particular variables were most deserving of consideration.

Depending on the extent that you could argue that little was known about the particular problem, you could also consider doing exploratory or descriptive research to attempt to identify or categorize the different variables involved. In this case you might use an interview to get some factors that set up your experiment and then use experiment to assess their relative impacts on the behaviour.