What is the distinction between "relationship" and "difference" in behavioural statistics?

Question

Quoting [emphasis added] from D.C. Howell's (2012) Statistical Methods for Psychology, chapter 9 (Correlation and Regression), Pg 252:

Although you should not make too much of the distinction between relationships and differences (if treatments have different means, then means are related to treatments), the distinction is useful in terms of the interests of the experimenter and the structure of the experiment. When we are concerned with differences between means, the experiment usually consists of a few quantitative or qualitative levels of the independent variable (e.g., Treatment A and Treatment B) and the experimenter is interested in showing that the dependent variable differs from one treatment to another. When we are concerned with relationships, however, the independent variable (X) usually has many quantitative levels and the experimenter is interested in showing that the dependent variable is some function of the independent variable.

My question is: What is the distinction between a mere difference of the dependent variable (DV) from one one level to another, and the DV as a function of the independent variable (IV)? In other words, the distinction that Howell explains in the quoted text is unclear to me.

Answer 1

Howell is just making a distinction between types of independent variables (predictors in a regression context): continuous variables versus categorical ones.

In the case of categorical variables, like "Treatment A" and "Treatment B", your regression equation will contain variables that contrast between Treatment A and Treatment B - often this is by "dummy coding" where we consider Treatment A as "0" and Treatment B as "1" though there are other schemes as well. In a two-treatment case, you can consider the coefficient for Treatment to be a difference between A and B. You are likely to plot these sort of data in a box-and-whisker plot (or a bar graph like the infamous 'dynamite plot' that many statisticians abhor) where you have distinct categories on the X-axis and your dependent variable on the Y-axis. You are likely to do statistical inference to determine whether the outcomes for Treatment A are "significantly different" from outcomes for Treatment B.

In the case of a continuous variable, for example, blood pressure, now your regression coefficient for blood pressure is interpreted in terms of units of change in your dependent variable per each unit of difference in your predictor - in this case you are looking at a relationship. For a single linear predictor you are likely to discuss this relationship in terms of a correlation coefficient. You would often plot these sort of data as a scatter plot, with your predictor variable on the X-axis and your outcome variable on the Y-axis, often with a line depicting some sort of fit between the two variables. You are likely to do statistical inference on whether your predictor is significantly related to your outcome variable: that is, whether knowing your predictor gives you better than chance information about the outcome.

From the perspective of linear regression, however, these two types of predictor variables are really not all that different, and it is quite straightforward to combine them in the context of multivariate regression.