2
$\begingroup$

I a fairly solid grasp of Statistics and have taught statistics to both undergraduate and graduate Psychology students. There are, however, sizable gaps in my knowledge and I have attempted to employ a statistical consultant a couple of times. The conversations generally devolved into the consultant suggesting experiments and analyses that did not answer the research questions I was interested in. I am beginning a retrospective epidemiological study which is far enough outside my experience that I want to try a statistical consultant again.

How do I prepare for talking to a statistical consultant such that our discussions can be productive. How much time will it take before I see progress being made on our mutual understanding of the issues (hours, days, weeks, months ...)?

$\endgroup$
  • $\begingroup$ perhaps post on cross-validated? stats.stackexchange.com $\endgroup$ – honi Mar 7 '16 at 19:57
  • $\begingroup$ @honi I thought about it, but I think the answer takes understanding CogSci. $\endgroup$ – StrongBad Mar 7 '16 at 20:06
3
$\begingroup$

This is a challenging question. As a bit of background, I am a cognitive scientist for the most part (by my highest graduate degree), but in addition to that, I have graduate degrees in both math and statistics and I have worked professionally as a consultant in those fields for many years. So I have some access to the problems on both sides of this question.

Statistical consulting is a point of training for some, but certainly not all, graduate students in statistics. What this means is that some people acting as consultants will be well trained in this activity, while others may be picking it up as they go. So you want to talk to either a more senior consultant or work with a statistics consultancy that has more senior people there to make sure your consultant knows what he or she is doing. That said, there is a standard of care in consulting these days, outlined in (among other examples) this ASA document, this 2010 conference paper, and this white paper by Flavia Jolliffe. Also Zahn and Isenberg is a classical paper on this. These may give you an idea of what the consultants who get trained are taught.

Consultants in statistics are well trained (usually!) in methodologies, but the main weakness in their training is usually exposure to the complexities of any given scientific field. In particular, unless they consult for people in your field regularly, the single hardest problem will be communicating in a concise way what it has taken you years to learn yourself: the specifics of your research problems. A solution here is to find someone who specializes in problems related to your research area. While not comprehensive, this page at ASA lets you search for statisticians by speciality, including some content areas (including epidemiology). Also remember that statisticians live in various academic departments (if you are using academic consultants) and these departments often are only vaguely aware of each other: as an example, I used to be at a university where the main campus statistical consultancy was in a school of social sciences, and the actual statistics department did not interact with them at all. Searching broadly is sometimes helpful.

If you cannot find a specialist, then communication is the key, and it will take longer to get to results in your relationship with your consultant. The paper by Jolliffe (above) has good advice that might be useful both to the consultant and to the client in terms of what to do and how to communicate (and there are some references there that go further). But this will be the central problem, so working closely on clarifying the research goals is probably the most important step.

A good preparation is to think about the problem before meeting the consultant and imagine trying to explain it to a good student or colleague from another field. Try to think through the particularly specialist parts of the problem before meeting the consultant; how would you explain these to someone new in your research lab? Likewise, if the consultant is not clear in responding to you, be direct is asking for more clarity. I believe this is covered by item E.2 of the ASA's ethical guidelines for consulting and collaborating for statisticians.

Not that any of this applies to your case, but from the point of view of the consultant, there are a number of things worth noting. First, clients, especially those with statistical knowledge, often come in with strong expectations of what the right answer is, and sometimes grow offended at questions from the consultant that try to get at the real questions being asked. As an example, I used to consult with psychologists who basically wanted me to tell them how to do an ANOVA on their data, even when that was the wrong analysis. So be open to analyses that you are not used to using. Also, a good consultant will ask a lot of questions to clarify the statistical question you want answered. Always assume good faith, and do try to answer the questions even if you don't fully understand them. But be willing to say (and explain) when you think the consultant's questions miss the point. Another issue is when people come into consulting so late that all there is for the statistician to do is conduct a post mortem. If you are serious about consulting, meet with them early on in the process. Finally, work out payments or collaboration rules up front. You would be amazed how often statisticians are asked to do pro bono work, either implicitly or explicitly.

tl;dr Communication, and it may take some time.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.