Is there a lot of redundancy in written and spoken language?

Question

I read years ago in a pop science book that written and spoken language can be shown to have a high level of redundancy. The speculation was that this served to allow error correction because language is encountered in noisy environments and understanding has a high survival value. The figure I remember is that if 35% of the information remains an attentive listener or reader can still make out the message. This has become a cherished notion in my world view as an indicator that language and human endeavor contains complexity we are not consciously aware of.

I am looking for a knowledgeable confirmation of the idea that spoken or written language contains this redundancy. An impolite smack down due to unexamined confirmation bias is always welcome as well.

Answer 1

It seems that for written English, the figure is 50%.

From pages 27 to 28 of The making of cognitive science: Essays in honor of George A. Miller (Cambridge: Cambridge University Press, 1988)

Estimates of redundancy. Shannon (1948, 1951) had himself estimated the redundancy of printed English to be about 50 percent. He had used a technique in which a subject was given a passage of text and then required to guess the next letter until the correct response (i.e., that corresponding to the original text) was given. Redundancy was calculated from the distribution of the numbers of guesses required. Garner and Carson (1960) [...] also estimated the redundancy of printed English to be about 50 percent. Newman and Gerstman (1952) [...] estimated redundancy to be 52 percent.

Uses of redundancy. [...] Chapanis (1954) and Miller and Friedman (1957) both showed that when text was mutilated by deleting different percentages of letters, subjects were able to restore the missing letters with a high degree of accuracy. Such restoration is possible because of redundancy, so these experiments showed that redundancy was useful to humans.

[...]

In summary, printed English is redundant, and thus constrained, both in letter sequences within words and in sequences of words themselves. This redundancy is known to humans, who can use it to reconstruct mutilated text and to recognize and learn words and sequences of words that reflect varying degrees of this constraint. [...]

From page 1086 of A new kind of science by Stephen Wolfram (Wolfram Media, Inc., 2002):

[...] English text typically remains intelligible until about half its characters have been deleted, indicating that it has a redundancy of around 0.5. Most other languages have slightly higher redundancies, making documents in those languages slightly longer than their counterparts in English.

Answer 2

The fact that you can tell when I use "it is" or "its" incorrectly is confirmation of redundancy. If there was no context telling you which it should be, you wouldn't know; and if there is context telling you what it should be, there is no need to write the apostrophe because you already know what the writer intended.

That much is fairly obvious. Further, data compression can give another insight into this. Standard text compression yields a much higher compression ratio than 50%. If I compress your question with "gzip" (a standard compression utility), it goes down from 770 bytes to 440 bytes (57% of the original). This is fairly bad, but this is with zero pre-existing knowledge -- something humans have loads of when processing text.

If I prepend what is currently on Wikipedia's front page (some text about "Suillus salmonicolor", some fungus), compressing your question takes an additional 378 bytes (49% of the original), probably because it can replace words like "and" with a very short code without having to pre-define it.

It doesn't end there either: trying to remove redundancy (e.g. writing words in conjugated form is almost always unnecessary; as are words like "she" in the phrase "Sarah went to the store when she needed something" ("Sarah went to the store when needed something" gets the message across; or at least "he" or "it" (both are a character shorter) would suffice).

I have often wondered how much redundancy is in languages: both in its basic form (sentences as an abstract concept) as well as in its representations (written characters or spoken sounds). I've come to terms with it because it must be for error correction. I probably wouldn't catch half of what my aunt is telling at the Christmas dinner table if not for that.

A related experiment that ran in 2013 looked at how much of a sentence you can hide before it becomes unreadable: https://lucb1e.com/rp/js/read.html For me, more than half can be hidden with no problems at all, meaning the character shapes that we use are >50% redundant as well, at least when combined with the redundancy in words themselves (because an h of which the top is hidden look like an n, but the word probably only makes sense with one of the two).