So I'm doing a research on developing a new text-based CAPTCHA system. I've devised a scheme where characters in a text are broken/split individually and randomly, making it difficult for OCR machines to decode it. (See figure for example) This CAPTCHA is still under development, but it gives a rough idea of what I'm talking here.

CAPTCHA Example which reads "NaAwKgkP"

My main aim here is to understand how the brain reads scrambled characters, as shown in the figure. I was looking up and a term called "filling-in" phenomena of vision came up. I'm not sure if that term is right in this context.

I want to know the terminology and basic understanding of how humans are able to read scrambled text/characters easily.

  • $\begingroup$ Welcome to CogSci. I've tried to clarify your question a bit. Related but probably not duplicate: cogsci.stackexchange.com/q/9992/2868 $\endgroup$ May 10 '15 at 12:09
  • $\begingroup$ Hi, I have read that already, it doesn't specifically answer my question of how a broken text/letter is recognized by human brain easily. For example, the letter N in the above picture, is not displayed in its entirety but parts (being broken from in between); so, how the brain processes it as a whole and concludes it to be letter N? $\endgroup$ May 10 '15 at 12:41
  • 1
    $\begingroup$ i'm sure there's also a neurological basis for it, but a good start would be to look into wertheimer's gestalt theory. some useful references are smashingmagazine.com/2014/03/28/… and scholarpedia.org/article/Gestalt_principles $\endgroup$
    – faustus
    May 10 '15 at 18:06

A human brain recognises letters by their constituent features (sub-letter parts). It is modelized by a pandemonium model where printed information is extracted locally then globally. In the letter recognition literature, this type of feature-based hierarchical model competes with template matching theories (with an advantage to the pandemonium-like models such as deep-learning networks).

enter image description here

Scrambled characters are recognised the same way, with the difference that only part of the visual information is available. It relates to the theory of geons developed by Biedermann (Recognition by components, 1987). He wrote

if an arrangement of two or three sub-components can be recovered from the input, objects can be quickly recognized even when they are occluded or extensively degraded.

Could you identify the following object? Left is the unrecoverable version.

enter image description here


  • $\begingroup$ Is the recognition by components theory and gestalt principle related? Are both applicable to my problem? $\endgroup$ Jun 18 '15 at 20:14
  • 1
    $\begingroup$ You can see components as an extension of gestalt theory and both are applicable to your problem. $\endgroup$
    – marsei
    Jun 19 '15 at 11:41

The cumulative number of versions of a single character that the human brain can recognize is nearly infinite, whereas computers have to be programmed to recognize every single variation. Humans also recognize 'context' in a word, whereas a computer does not recognize context because it does not have an intuitive understanding of language. From wikipedia:

For example, when an individual understands that the first letter of a CAPTCHA is an “a”, that individual also understands where the contours of that “a” are, and also where it melds with the contours of the next letter. Additionally, the human brain is capable of dynamic thinking based upon context. It is able to keep multiple explanations alive and then pick the one that is the best explanation for the whole input based upon contextual clues. This also means it will not be fooled by variations in letters.

Essentially it boils down to two factors: one, humans are preprogrammed with 'object recognition' abilities. Two, humans are able to parse a string of symbols in a way that conforms to the rules of a formal grammar.

Object recognition is the ability to find and recognize and identify human objects in a sequence. Humans naturally have this ability, whereas computers do not. It appears to be a genetic quality, and can be replicated in computers by genetic algorithms that mimic the process of natural selection.

Humans also build up sentences incrementally, and as they do so, they make constant unconscious predictions about what the word or sentence is going to say. This is the explanation behind garden path sentences. We can use event-related potentials to study this phenomenon in the human brain. Certain potentials are activated during specific situations when parsing. From the wikipedia page on garden-path sentences:

Within ERPs, P600 is the most important component. Its activation occurs when the parser comes across a syntactic violation such as The broker persuaded to sell the stock or when parses synthesizes an unsatisfactory disambiguation on an ambiguous string of words such as The Doctor Charged the patient was lying. Hence the activation of P600 marks the parser's attempt to revise the sentence's structural mis-match or ambiguity.

It also appears that the presence of a disfluency in a sentence—caused by filled and long silent parses—does not elicit the P600. Instead, it elicits another ERP component, N400, which gets activated when people try to integrate a new word into the preceding sentence's context.

Based on the above, it appears that the brain has specific event-related potentials that intuitively activate during specific situations while parsing. In other words, a human can recognize familiar sentences and words intuitively, and when it comes across an unrecognized word, it attempts to integrate it based on context. Since computers do not currently have a similar intuitive ability, CAPTCHAs are harder for them to decode.

  • 1
    $\begingroup$ Do you have any references for the information the wikipedia page? Anyone can write on there so peer-reviewed published work would be more reliable $\endgroup$
    – queenslug
    May 11 '15 at 11:11
  • $\begingroup$ There are some concerning aspects about this answer. Computers don't naturally lack any ability that humans do per se. that's a bit meaningless. Also, genetic algorithms don't have much to do with it, they are an optimisation routine, and human's ability to recognise objects in a sequence is not related to any "genetic quality". If you believe these things, I'd advise you to provide references, because I am not familiar wiht any such argument in the literature. $\endgroup$
    – analystic
    May 14 '15 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.