7
$\begingroup$

I read years ago in a pop science book that written and spoken language can be shown to have a high level of redundancy. The speculation was that this served to allow error correction because language is encountered in noisy environments and understanding has a high survival value. The figure I remember is that if 35% of the information remains an attentive listener or reader can still make out the message. This has become a cherished notion in my world view as an indicator that language and human endeavor contains complexity we are not consciously aware of.

I am looking for a knowledgeable confirmation of the idea that spoken or written language contains this redundancy. An impolite smack down due to unexamined confirmation bias is always welcome as well.

|improve this question
$\endgroup$
  • 1
    $\begingroup$ Our smack downs are always polite. ;) Interesting question. Do you recall the name of the popular text? $\endgroup$ – Chuck Sherrington Jan 29 '13 at 5:35
  • $\begingroup$ I think it was called Rules of the Game. It was about self organizing systems. I have it buried in a box. Amazon has many books under that name, can't remember the authors name, couldn't find it. $\endgroup$ – timquinn Jan 29 '13 at 6:27
  • $\begingroup$ It was "Laws of the Game : How the Principles of Nature Govern Chance" by Manfred Eigen and Ruthild Winkler. It is from 30 years ago, so the idea may have come from some other book I read at the time. $\endgroup$ – timquinn Jan 29 '13 at 10:05
  • $\begingroup$ @timquinn Could you please add that extra relevant information into your question if it turns out to be the book you are talking about, also providing a link to it? Thank you. $\endgroup$ – Steven Jeuris♦ Jan 29 '13 at 10:36
  • $\begingroup$ This relates to ambiguity in language (since redundancy often reduces ambiguity). Zipf suggested that the speaker wishes for an ambiguous language where they can use one sound to mean everything, and leave difficulty of disambiguation for the listener. The listener, on the other hand, wishes for a totally unambiguous language, so the difficulty of picking the right words is on the speaker, and the listener doesn't need to spend energy on disambiguation. I would expect redundancy to have similar driving forces. Here is the dual of your question. $\endgroup$ – Artem Kaznatcheev Feb 1 '13 at 4:07
9
$\begingroup$

It seems that for written English, the figure is 50%.

From pages 27 to 28 of The making of cognitive science: Essays in honor of George A. Miller (Cambridge: Cambridge University Press, 1988)

Estimates of redundancy. Shannon (1948, 1951) had himself estimated the redundancy of printed English to be about 50 percent. He had used a technique in which a subject was given a passage of text and then required to guess the next letter until the correct response (i.e., that corresponding to the original text) was given. Redundancy was calculated from the distribution of the numbers of guesses required. Garner and Carson (1960) [...] also estimated the redundancy of printed English to be about 50 percent. Newman and Gerstman (1952) [...] estimated redundancy to be 52 percent.

Uses of redundancy. [...] Chapanis (1954) and Miller and Friedman (1957) both showed that when text was mutilated by deleting different percentages of letters, subjects were able to restore the missing letters with a high degree of accuracy. Such restoration is possible because of redundancy, so these experiments showed that redundancy was useful to humans.

[...]

In summary, printed English is redundant, and thus constrained, both in letter sequences within words and in sequences of words themselves. This redundancy is known to humans, who can use it to reconstruct mutilated text and to recognize and learn words and sequences of words that reflect varying degrees of this constraint. [...]

From page 1086 of A new kind of science by Stephen Wolfram (Wolfram Media, Inc., 2002):

[...] English text typically remains intelligible until about half its characters have been deleted, indicating that it has a redundancy of around 0.5. Most other languages have slightly higher redundancies, making documents in those languages slightly longer than their counterparts in English.

|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Holy wow, Shannon and Wolfram. Nice work. $\endgroup$ – timquinn Jan 29 '13 at 11:23
  • $\begingroup$ I read a little about this experiment of Shannon's. Do you think it would hold up to present day standards? I am sure that Wolfram is referring to Shannon. I wonder if there is any current science on the subject. $\endgroup$ – timquinn Jan 29 '13 at 11:27
  • 1
    $\begingroup$ It would also be interesting to see whether any work is done on the redundancy of spoken English. It probably better reflects realistic scenarios in which the speculation makes sense (noisy environments). $\endgroup$ – Steven Jeuris♦ Jan 29 '13 at 12:20
  • $\begingroup$ Yes, i agree, Steven. It would be more relevant to my question if the research involved spoken English. The Shannon experiment sounds more like solving a crossword puzzle, a very conscious act. $\endgroup$ – timquinn Jan 29 '13 at 13:45
  • $\begingroup$ I am going to give Joel the green check. I am sure you have nailed the source of my initial exposure to this and I thank you for that. That it turns out to be Claude Shannon is very interesting because I realized that the notion has become a sort of received wisdom that has been repeated by a lot of pop and legitimate science writers. His work was about written language, though, so it does not actually support my long held belief in instantaneous error correction. I am going to formulate another question. $\endgroup$ – timquinn Jan 30 '13 at 6:07
0
$\begingroup$

The fact that you can tell when I use "it is" or "its" incorrectly is confirmation of redundancy. If there was no context telling you which it should be, you wouldn't know; and if there is context telling you what it should be, there is no need to write the apostrophe because you already know what the writer intended.

That much is fairly obvious. Further, data compression can give another insight into this. Standard text compression yields a much higher compression ratio than 50%. If I compress your question with "gzip" (a standard compression utility), it goes down from 770 bytes to 440 bytes (57% of the original). This is fairly bad, but this is with zero pre-existing knowledge -- something humans have loads of when processing text.

If I prepend what is currently on Wikipedia's front page (some text about "Suillus salmonicolor", some fungus), compressing your question takes an additional 378 bytes (49% of the original), probably because it can replace words like "and" with a very short code without having to pre-define it.

It doesn't end there either: trying to remove redundancy (e.g. writing words in conjugated form is almost always unnecessary; as are words like "she" in the phrase "Sarah went to the store when she needed something" ("Sarah went to the store when needed something" gets the message across; or at least "he" or "it" (both are a character shorter) would suffice).

I have often wondered how much redundancy is in languages: both in its basic form (sentences as an abstract concept) as well as in its representations (written characters or spoken sounds). I've come to terms with it because it must be for error correction. I probably wouldn't catch half of what my aunt is telling at the Christmas dinner table if not for that.

A related experiment that ran in 2013 looked at how much of a sentence you can hide before it becomes unreadable: https://lucb1e.com/rp/js/read.html For me, more than half can be hidden with no problems at all, meaning the character shapes that we use are >50% redundant as well, at least when combined with the redundancy in words themselves (because an h of which the top is hidden look like an n, but the word probably only makes sense with one of the two).

|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.