I was thinking last night about the possibility of an experiment that investigates the factors contributing to peoples' judgments of 'stylistic similarity' between two samples of writing. For example, such an experiment might consist of multiple trials for each subject, each of which would present a different set of say 3 short excerpts of writing (text randomly sampled from a corpus of authors' writings... maybe no more than 500 words each). Each participant would be asked to make a judgment of either "Which 2 of these writings are most similar in style?" OR 'Which 2 of these writings were most likely created by the same person?".

Part of the data analysis and interpretation might involve the use of programs similar to this one: http://www.hackerfactor.com/GenderGuesser.php . Researchers could form hypotheses about relatively low-level factors that might be important in peoples' judgments of stylistic similarity, and then write programs that would make judgments based on these low-level factors.

I'm assuming that research resembling or even perfectly matching what I've described above has been conducted in the past. After reading what I've written above, does anyone have any suggestions for relevant readings?

  • $\begingroup$ I think this sounds really interesting, it's almost like "natural" natural language processing. NLP might give you some food for thought about what features could possibly be salient to a human reader based on what works statistically for the computer. $\endgroup$ – Chuck Sherrington Jul 30 '12 at 16:22

It sounds like you're talking about Latent Semantic Analysis. Here's their rundown of what it is.

However, LSA as currently practiced has some additional limitations. It makes no use of word order, thus of syntactic relations or logic, or of morphology. Remarkably, it manages to extract correct reflections of passage and word meanings quite well without these aids, but it must still be suspected of incompleteness or likely error on some occasions.

It's used in a lot of automated grading programs, or systems designed to give a score to paper based on some criteria. However, because order doesn't matter, it makes syntactic and grammatical coherence much harder to measure, so quantifying 'style' might be a little hard, unless by 'style' you mean something like word choice, phrase choice, etc.

Here's a gscholar search. The first three should give a good overview.

| improve this answer | |
  • 1
    $\begingroup$ Welcome to the site. I think the OP is more interested in studies of people's judgements of similarity, not so much the software implementations of style identification (he seems to only mention them in reference to forming a base-line). Can you edit your answer to address the human judgement portion more clearly? $\endgroup$ – Artem Kaznatcheev Aug 4 '12 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.