12

Antoine Tremblay has just released an advanced analysis toolbox: http://onlinelibrary.wiley.com/doi/10.1111/psyp.12299/abstract It's missing about half the features on your list, although fundamentally, spectral density is a simple task and LORETA is a stand-alone package anyways (although similar approaches, e.g. general CSD estimation, are implemented in ...


10

Here's my basic overview of the psychology - statistics textbook market. In general, I think the choice of textbook (where you have a choice) depends on a few factors: Applied versus statistical theory (e.g., do you have immediate needs to analyse data) Whether and which software package you want to use (e.g., SPSS, R, etc.) Which techniques you want learn (...


10

IQ scores in general: An IQ score is a normative score. The norm group is typically defined as the general population, and where the respondent is a child, the norm group is defined in terms of the general population of children of that same age. IQ scores typically have a mean of 100 and a standard deviation of 15. In order to get an estimate of ...


9

Programs/packages for EEG analysis There are decent MatLab toolboxes with good tutorials for for the analysis of EEG data. The EEGLAB toolbox (tutorial) can be operated by both GUI and command-line (and script). The fieldtrip toolbox (tutorial) is mainly operated by command line / script. Of course there are also (commercial) software packages for EEG ...


9

It's not a journal article, but it's a project, called Reproducibility Project. They are re-doing psychological studies to estimate the reproducibility of a sample of studies. They try to stay as close as possible on the original study (they e-mail the author(s) for the exact stimuli etc.). Here's the main site and here's the link to their spreadsheet. This ...


9

I'd like to address important issues that Jeromy Anglim raised in the "Personal thoughts" section of his answer, namely that correlation parameters (i.e., true, population, or infinite-sample correlations) often vary and covary among studies, and this between-studies\interstudy heterogeneity implies heterogeneity in studies' parameters for a structural ...


9

Answer based on your original depression example Note that this answer was originally written based on your initial example, where you asked: Assume, I have developed a new intervention for people with light depression. I want to compare the effectiveness of this intervention (E) with an existing intervention (C). For this, I recruit test subjects ...


9

To calculate $d'$ you need to know two things: the hit rate and the false alarm rate. The hit rate is the proportion of trials where the stimulus was present and the subject responded that the stimulus was present. The false alarm rate is the proportion of trials where the stimulus was not present, and the subject responded that the stimulus was present. ...


8

Experimental psychologists seem quite happy working with single item ad hoc self-report scales, physiological measures, etc. with very little psychometrical assessment so even before talking about a full-fledged latent variable modeling approach, confirmatory factor analysis and the like you might want to wonder why they appear relatively unconcerned with ...


8

You can find an accessible overview of some of the issues in Whelan (2008) which contains further references discussing the issue. Note that from a statistical perspective, the sample mean and median are unbiased estimators of their population equivalents. That said, with outliers, skew and the like, the standard error of the sample mean can be quite a bit ...


8

Some extremely large personality data sets can be downloaded at http://personality-testing.info/_rawdata/ in CSV format. They are not packaged in R, but I post the link because you indicate in your reply to USER:what that you are having trouble finding any data sets what so ever.


7

The following reviews some of the articles that I found discussing and implementing SEM meta-analysis to examine mediation. Cheung and Chan (2005) The authors distinguish three approaches of meta-analytic structural equation modelling (MASEM). Univariate two-stage MASEM: This includes a collection of two step approaches (note that Cheung and Chan call it ...


7

I use the FieldTrip toolbox in Matlab to analyze my own modified auditory MMN experiment :) But I use MEG, so I don't have that many software options. The toolbox is very powerful but it has a steep learning curve and I would recommend it only if you already have both Matlab and EEG data analysis experience. I don't analyze my data in the classical MMN way ...


7

In general, from my extensive experience, using a four or a five point response scale is not going to change much the psychometric properties of a typical psychological self-report scale (e.g., reliability and factor loadings). I also imagine that if you were to measure a multi-item scale with a four point version and a five point version that the ...


7

Short answer IQ scores are distributed normally, because they follow the central limit theorem. Background When we measure IQ scores in sufficiently large populations, they will be normally distributed. This holds for healthy controls, as well as groups of people with ADHD or reading disabilities (Kaplan et al., 2000), and also in people with mild to ...


6

Baayen's book, Analyzing Linguistic Data: A practical introduction to Statistics using R is a little more niche than possibly you'd want, but it does offer some coverage of mixed effects modelling, without which I'd be reticent to call any psych stats text "cutting edge".


6

Using Parameters Estimated from an Individual in a Group Analysis In a way this is exactly what usually happens when we calculate the mean reaction time across all conditions for a group of participants. When we normally calculate mean reaction times we assume that some process (P) takes t milliseconds to complete plus some Gaussian distributed noise. We ...


6

The purpose of reviewing the literature for effect sizes is to form an estimate of what effect size you might expect in your present study. Existing meta-analysis: The principles and techniques of meta-analysis provide a good starting point for generating a predicted effect size. If a meta-analysis has already been conducted, then the estimated population ...


6

It's an interesting question. Here are a bunch of thoughts that came to my mind for why researchers might focus on observed variables. Many researchers report reliability and observed relationships between variables. By adopting a few assumptions, the reader can estimate what the latent relationships would be (see for example, the formula for correction of ...


6

I did eventually find something which may be what I was thinking of, examining the normality & psychometric properties of Likert scales with differing numbers of points. They concluded that the 11-point scale was closest to interval-type data in scaling and normality. Reference: A comparison of psychometric properties and normality in 4-, 5-, 6-, and ...


6

The languageR package contains several interesting real-world datasets relevant to psycholinguistics. These datasets are presumably also described in the accompanying textbook.


6

The F ratio statistic has a numerator and denominator degrees of freedom. Thus, you report: F (numerator_df, denominator_df) = F_value, p = ..., effect size = ... The numerator degrees of freedom relates to the factor of interest; the denominator degrees of freedom corresponds to the degrees of freedom for the error variance. The exact way that these ...


6

In Stevens' levels of measurement framework, the NASA-TLX is an ordinal scale, not an interval scale, because there is no way to know a priori how much "workload" each point corresponds to. In other words, we can only know how much workload a point corresponds to after observing the data from our sample. Indeed, Hart's own review of the NASA-TLX suggests ...


6

ANOVA and t-tests are statistical tests for significance and therefore quantitative. The other mentioned items are scales (adding numbers to a certain choice) and therefore they can be considered as ordinal scales, and hence as quantitative as they are based on numbers. The NASA one can be administered by using a sliding scale which can be considered to ...


6

It's important to distinguish between measures and analyses, because only analyses can be quantitative or qualitative, not measures. Measures are, essentially, systematic processes by which we acquire our data, and analyses are processes we use to look at the data. As a rule of thumb, the difference is not hard to find and is given in the name: ...


6

The APA style manual does not provide specific guidelines for linear mixed models. Additionally, a review of studies using linear mixed models reported that the psychological papers surveyed differed 'substantially' in how they reported on these models (Barr, Levy, Scheepers and Tily, 2013). It depends greatly on your study, in other words. Normatively ...


5

To my knowledge, there is no adjusted RMSD. RMSD, unlike $R^2$, isn't typically used to compare models across the literature. $R^2$ represents the proportion of variance explained by the model, a construct which translates well across different experimental designs. Adjusted $R^2$ distorts this by accounting for the number of parameters in your model, but ...


5

If you're trying to work out a standardized effect size in order to calculate power for your study then it doesn't matter whether any studies like yours have been done. It's important to find studies using the same dependent measure so that you have an estimate of variability but evidence of the size of prior effects is less important. If you have the ...


5

I think your intuitions are correct, and the reason is that Likert scales often suffer from poor construct validity. One example that comes to mind is Alter et al. (2010), who use a mixed design, and note that participants in both conditions gravitate toward the middle of the scale on their first rating. They suggest that participants do not know how to use ...


Only top voted, non community-wiki answers of a minimum length are eligible