3
$\begingroup$

I am reading here, page 5 that d' (d-prime) does not vary with criterion (in contrast to hit rate for instance which does vary with criterion, and which can be a biased measure of a subject's perception).

However, if we think of a subject that always answers "yes" when asked if a stimuli was present, even if not, then clearly the subject has a very liberal criterion (a negative criterion in particular), and it will follow that her hit rate = 1 but also her false alarm rate = 1, leading to a sensitivity of zero.

This also makes me wonder how sensitivity can be inferred independently from criterion. Let's assume that the subject has a d' = 1, i.e. their internal representation of signal and noise are distinct w.r.t. means of the distributions. However if the subject does chose to nevertheless use the very liberal criterion and always answer "signal present", we will be only able to infer that their sensitivity equals zero.

In summary, criterion seems to clearly impact sensitivity. Hence, why are criterion and sensitivity often discussed as being independent?

$\endgroup$
2
$\begingroup$

Mathematically, we can see that d' is independent of criterion, regardless of the criterion, by looking at the definition

$$d' = Z_D - Z_F.$$

It is tempting to argue that: (1) When the criterion is negative infinity, $P_D$ and $P_F$ are equal to unity. (2) When $P_D$ and $P_F$ are equal to unity, that $Z_D$ and $Z_F$ are equal to infinity. (3) Therefore, when the criterion is negative infinity that d' is undefined since $\infty - \infty \ne 0$. A better way of thinking about it is to keep c in the equations a little longer: $Z_D=(c - \mu_{signal})/\sigma$ and $Z_F=c - \mu_{no signal}/\sigma$ and

$$d' = Z_D - Z_F = (c/\sigma - \mu_{signal}/\sigma) - (c/\sigma - \mu_{no signal}/\sigma) = (\mu_{signal} - \mu_{no signal})/\sigma$$

Since the c's cancel out we don't have to worry about the pesky $\infty - \infty$ business.

An alternate way to see what is going on is to consider your statement

However if the subject does chose to nevertheless use the very liberal criterion and always answer "signal present", we will be only able to infer that their sensitivity equals zero.

To see why this is wrong consider an estimation problem where you are trying to estimate upper and lower bounds on d' and c from a set of observations. Consider the case where we have N no signal trials resulting in N false alarms and M signal trials resulting in M hits. The upper bound depends of c depends on N and M as well as d' and we don't need to consider it here to make the point that you can only infer that d' equal zero is wrong. The lower bound on c is the key thing to consider. The lower bound is negative infinity. If c is negative infinity the probability of observing at least N false alarms and and at least M hits, give N no signal trials and M signal trials, is equal to unity for all values of d'. Statistically speaking, there is a greater than 1-alpha chance of observing at least N false alarms and M hits regardless of d' when c is infinity. Therefore we cannot rule out, at any level of statistical confidence, any value of d'.

$\endgroup$
  • $\begingroup$ I understand the argument that we can't be sure about the true underlying distribution. But if we consider the empirical sensitivity calculated from hit and false alarm rates of one particular experiment, wouldn't it follow that when the subject is always answering "yes"/"signal" d' is equal 0, and hence at least within the context of such empirical evaluations the two measures are dependent? $\endgroup$ – user21198 Jan 28 at 17:55
  • 2
    $\begingroup$ @TestGuest no, if the subjects always answer yes, we know nothing about d' and only that they have a very liberal criteria. $\endgroup$ – StrongBad Jan 28 at 19:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy