Questions on Interpreting Factor Analysis Results and Scores

Question

I am trying to learn factor analysis and I thought it would be a good idea to try and very poorly "mimic" the computation for IQ scores with a dataset of dummy values as a way to "learn by example".

To start off, this is what I intend to do, and I don't know if this methodology is correct or not: I have the loadings for that factor determined. Now that I have the loadings, I want to generate a score for each of the samples. That will leave me with a population of scores that I can then standardize around a mean of 100. From there I would plot a normal distribution. Whenever I get a new sample, I can then generate a score for it and see where it falls on the distribution.

To get my results, I am using Python's Sklearn library, specifically the FactorAnalysis class. I noticed that the FactorAnalysis class has a score_samples() method. The output score for each sample is the log-likelihood of the sample.

Here are some of the questions I have:

• Is my approach in generating a distribution based on the samples' factor scores flawed? How do they do it in practice?

• Is the log-likelihood of a sample even an appropriate score to use? (If not, what alternative ways are there to score a sample?)

• I have gone ahead and generated the scores using the score_samples() method for all the samples, but they range between -4 and -49. Is there a reason they would be negative?

• If you are only looking for 1 latent factor, is it good practice to set the number of factors to 1 or should you leave it unspecified anyways?

Here are the loadings if I leave set the number of factors to 1:

            Factor 1
variable 1  0.082558
variable 2  0.107940
variable 3  0.199645
variable 4  0.612495
variable 5  0.623707

Here are the loadings if I do not specify the number of factors:

             Factor 1   Factor 2   Factor 3  Factor 4  Factor 5       
variable 1   0.263914   0.426346  -0.012893   -0.0       0.0
variable 2   0.297078   0.415269  -0.002193    0.0      -0.0
variable 3   0.243590  -0.005131   0.085178   -0.0      -0.0
variable 4   0.487537  -0.224135  -0.019501   -0.0      -0.0
variable 5   0.484462  -0.248173  -0.008902    0.0       0.0

Answer 1

Is my approach in generating a distribution based on the samples' factor scores flawed? How do they do it in practice?

I found this somewhat difficult to follow. But in general, you should be able to approximate a set of test scores using a multivariate normal distribution where the covariance matrix implies positive correlations between all tests. Some might be larger and some smaller, but the idea is that all ability tests are correlated. And general mental ability can be estimated as the first unrotated factor that results from such tests.

Is the log-likelihood of a sample even an appropriate score to use? (If not, what alternative ways are there to score a sample?)

This sounds more like how you evaluate a model. E.g., how you evaluate a factor analytic solutions. In general, factor saved scores will be a weighted composite of the scores on the component tests.

In R, you can use factanal

factanal(x, factors, data = NULL, covmat = NULL, n.obs = NA,
         subset, na.action, start = NULL,
         scores = c("none", "regression", "Bartlett"),
         rotation = "varimax", control = NULL, ...)

See the scores argument. There are a few different methods.

I have gone ahead and generated the scores using the score_samples() method for all the samples, but they range between -4 and -49. Is there a reason they would be negative?

I don't know Python. But in general, factor saved scores are typically quantified in such a way that they are z-scores (e.g., mean = 0, sd = 1).

If you are only looking for 1 latent factor, is it good practice to set the number of factors to 1 or should you leave it unspecified anyways?

You need to either extract only one factor or ensure that you apply no rotation to the extract factors. Without a rotation, the first factor will be equivalent to just one factor. If you rotate, variation will be partitioned across the extracted factors.