Assuming a layman with no formal training (person A) says "person B has disorder X". A should not make such diagnosis because they don't have the training. But if A does that because it fits the definition, then wouldn't their conclusion be correct? For example, I know that for a person to have borderline personality disorder they need to have at least 5 of 9 criteria. So if B does have that, then would A be fine to conclude that B has that disorder, in spite of the fact that A isn't trained?
Here are some basics on the diagnostic process:
There are two main sources of disagreement in assessing psychiatric disorder or, as Feinstein (1985) wrote, 'two different phases of the observational process'. First, in the assessment of psychopathology; to transform ' the observed phenomena into certain raw elements of descriptive data'. Second, disagreement arises from the combining of detected psychopathology to yield diagnoses;' In the second phase, these raw descriptive elements are converted into a category of final expression' (Feinstein, 1985). Efforts have been made to standardize both these areas. [...]
And if this needs a quote, here's the obvious fact:
Lay interviewers are unable to exercise expert clinical judgements in making decisions about the presence or absence of symptoms. [...]
There are some particular problems in attempting to standardize assessments of psychiatric disorder. Lack of insight into mental illness, a problem when assessing people with psychotic illness especially, can affect responses to questions. The extreme example is the deluded patient who will say there is nothing wrong. In the training of a clinical psychiatrist much stress is put on obtaining historical details from informants other than the patient because of clinical evidence that those with severe mental illness can be misleading historians. For example, a subject with a delusion of persecution, may state that neighbours are deliberately making loud noises, or moving furniture in order to annoy. Such statements would lead most clinicians to investigate further by means of informants, though others might rely purely on clinical impressions from an interview. Attempts to standardize assessments of psychosis are more difficult and of a different nature from the problems of standardizing minor psychiatric disorder when insight is rarely impaired to an extent which affects the validity of self-report. A similar problem arises when attempting to standardize assessments of conversion hysteria or other physical disorders with a presumed psychological aetiology.
It's not a slam dunk that standazing the interviewing process will eliminate experience as a factor, for example
There are also concerns that lay interviewers are an important source of bias when using the PSE. Lay interviewers tend to score more highly than psychiatrists and so estimate a higher prevalence of psychiatric disorder. For example, Sturt et al. (1981) found that the mean PSE scores for lay interviewers were nearly 30% higher (4-12 v. 3-20) than those of the psychiatrically trained interviewers. Trends in a similar direction and of a similar magnitude have also been described by Rodgers & Mann (1986), Wing et al. (1977) and Cooper et al. (1977).
PSE is the former name of SCAN more or less the WHO-equivalent of the more commonly used SCID. Both are semi-structured interviews in which open-ended questions may be asked, but the SCID allows more freedom, including skips and completely novel questions from the interviewer.
With this long preamble, there has been a lot of research on instruments that would allow a layman to diagnose mental disorders with some level of reliability. These include CIS-R, CIDI, and even the older GHQ. These generally have a short[er] format and don't try to cover an entire diagnostic manual.
Trying to produce something that would beat the SCID (or even the SCAN) with lay interviewers didn't go well, insofar. Probably the most substantive effort was NIMH's DIS; the combination of its complexity (trying to assess the majority of DSM-III diagnoses) and use of lay interviewers didn't go well. E.g.,
Eaton et al., (2000) reported on 349 subjects reinterviewed by psychiatrists using the Schedules for Clinical Assessment in Neuropsychiatry (SCAN) following a DIS interview. Results indicated that the DIS missed many cases judged to meet criteria for diagnosis using the SCAN (29% sensitivity), and agreement between the lay versus clinician administered assessments for major depressive disorder was only fair (k=0.20).
(That's frankly understating it fair bit. A kappa of 0.2 for depression is pretty abysmal.)
Due to its highly structured format, full administration of the DIS-IV typically requires between 90 and 150 minutes. To shorten administration time, the modular format makes it possible to drop evaluation of disorders that are not of interest in a particular study. Another option is to drop further questioning for a particular disorder once it is clear that the threshold number of symptoms needed for diagnosis will not be met. Although designed for use by nonclinician administrators, training for competent administration of the DIS is necessary. Trainees typically attend a 1-week training program at Washington University, during which they review the DIS manual, listen to didactic presentations about the structure and conventions of the DIS, view videotaped vignettes, complete workbook exercises, and conduct several practice interviews, followed by feedback and review. Additional supervised practice is also recommended.
So even this layman-oriented interview is not exactly an exercise where experience and training don't matter. Basically, the obvious conclusion is that the more fine-grained your diagnostic manual is in terms of disorder classification, the more experience and training is required to use it, even with fairly standardized instruments. (Of course that doesn't even bring into discussion whether all diagnostic categories can even be reliably distinguished, even in the best of circumstances, interviewer-wise.)
There's not a lot of research on the use of SCID by non-professionals because its use this way is discouraged. I did find one paper which did attempt this although the contrast was not with another SCID interview by a clinician, but rather with the consensus lifetime best estimate (CLBE) diagnosis (given by professionals) taken to be "gold standard".
The overall agreement between the SCID-derived [they mean SCID applied by a trained non-clinician] and CLBE [consensus lifetime best estimate] DSM-IV diagnoses was very low (kappa 0.18). The SCID tended to diagnose more patients with major depression and fewer with schizophrenia and schizotypal disorder. It was clear that the major source of the diagnostic differences was information variance. A brief, or sometimes monosyllabic response to a question of the structured interview was often short of crucial psychopathological information that might have been elicited in an epistemologically more adequate manner.
Fennig et al showed a high diagnostic concordance for schizophrenia and bipolar disorder between the SCID diagnosis by non-clinician interviewers and the best consensus diagnostic estimates performed by psychiatrists. However, that particular study was limited to patients who were already diagnosed with a psychotic condition. More importantly, the SCID-elicited information also served as the main source of the best consensus estimate. This implies little information variance between psychiatrists and non-psychiatrists, i.e. clinician- and non-clinician raters made their diagnoses upon similar information. Minimizing information variance is exactly one of the cornerstones of the structured interview, but it does not guarantee the quality (validity) of the information.
In conclusion, this study demonstrated a poor diagnostic performance of a structured psychiatric interview conducted by a for-the-purpose trained non-clinician. Such an interview approach cannot be recommended for clinical work, due to a high proportion of misdiagnoses. The high specificity for the diagnosis of schizophrenia (100%) may suggest a limited applicability of such interviews in research with the purpose of a confirmation of an antecedent, clinician’s diagnosis.
What I also find interesting in this study, although not highlighted by its authors, is that the non-clinician categorized correctly all [CLBD] cases of major depression (using SCID). The [sub-]sample was rather small though (14). On other hand, he misdiagnosed a whole bunch of other [CLBD] cases (36=50-14) as being major depression. (High sensitivity, low specificity). That's not too surprising to me because even in primary care depression is generally overdiagnosed "for every 100 unselected cases seen in primary care, there are more false positives (n=15) than either missed (n=10) or identified cases (n=10)"; physician training as well as resorting to reassessments seems to lower this bias.
And that brings me to another comparison/point I wanted to make: if it's hard for primary care physicians to diagnose mental health problems accurately, it's not reasonable to expect laymen to do better (in similar circumstances).
It is tempting to think of medical diagnoses as a recipe. In some cases, it is a lot easier than others to see it that way. It would seem that if you have a broken leg, you have a broken leg. Unfortunately, both physical and mental health diagnoses are rarely that straightforward.
In mental health, when a person comes in complaining of e.g. anxiety, feeling down, prolonged grief, or having an unsatisfying love life, those things are certainly present. But in order to get to the etiology of the presenting concerns takes a good bit of work. Certain diagnoses pop up right away as likely candidates, but then you need to rule them out as there's plenty of overlap. You have to have at least 2 potential conceptualizations of what is going on, and then you rely on your experience and ability to help the patient open up more so you can observe how their mind works, how they see themselves, and how they see the world.
With all that in mind you can formulate what you see as the most likely diagnosis. But even then, as time goes on and you learn more about the patient, you may find yourself changing the diagnosis.
Taking personality disorders in particular, it is actually quite challenging often to differentiate between borderline, dependent, narcissistic, to name a few. You really need a lot of information and a working theory of the person.
As long as the criteria do match, then the diagnosis is correct. Some disorders can have overlap criteria, but a disorder should have a unique set of symptoms. If the symptoms B has match a unique set of a disorder, then we can conclude that B has it, and can conclude that the person making that conclusion correct.
I know this answer has no reference, but since no one explicitly confirms or disagrees with what I really ask, so I put it as an answer so everybody can vote on it. (Downvotes are welcomed.)
On the other hand, I think that this is just basic logic/epistemology, so there is no need for reference either.