Only evolutionary psychology seems to have something to say about the "why" here. And (somewhat predictably) the hypothesis advanced has something to do with social information in ever larger groups:
From Dunbar's evolutionary theories, gossip originated to help bond the groups that were constantly growing in size. To survive, individuals need alliances; but as these alliances grew larger, it was difficult if not impossible to physically connect with everyone. Conversation and language were able to bridge this gap. Gossip became a social interaction that helped the group gain information about other individuals without personally speaking to them. It enabled people to keep up with what was going on in their social network. It also creates a bond between the teller and the hearer, as they share information of mutual interest and spend time together. It also helps the hearer learn about another individual’s behavior and helps them have a more effective approach to their relationship. Dunbar (2004) found that 65% of conversations consist of social topics.
Dunbar (1994) argues that gossip is the equivalent of social grooming often observed in other primate species. Anthropological investigations indicate that gossip is a cross-cultural phenomenon, providing evidence for evolutionary accounts of gossip.
FYI: depending who you ask, it's a social skill and not a character flaw.
When disparaging gossip, we overlook the fact that it’s an essential part of what makes the social world tick; the nasty side of gossip overshadows the more benign ways in which it functions. [...]
In the workplace, studies have shown that harmless gossiping with one’s colleagues can build group cohesiveness and boost morale.
On the flip side, the awareness that others are likely talking about us can keep us in line.
Studies of California cattle ranchers, Maine lobster fishers and college rowing teams confirm that gossip is used in a variety of settings to hold individuals accountable. In each of these groups, individuals who violated expectations about sharing resources or meeting responsibilities became targets of gossip and ostracism. This, in turn, pressured them to become better members of the group.
For example, lobstermen who didn’t respect well-established group norms about when and how lobsters could be harvested were quickly exposed by their colleagues. Their fellow lobstermen temporarily shunned them and, for a while, refused to work with them.
Belgian psychologist Charlotte de Backer makes a distinction between strategy learning gossip and reputation gossip.
Indeed, de Backer discovered that our interest in celebrities may feed off of this [latter] thirst for learning life strategies. For better or for worse, we look to celebrities in the same way that our ancestors looked to role models within their tribes for guidance. [...] From an evolutionary standpoint, “celebrity” is a recent phenomenon, due primarily to the explosion of mass media in the 20th century. Our ancestors, on the other hand, found social importance in the intimate details of everyone‘s private life, since everyone in their small social world mattered. [...]
[Additionally, in a highly mobile, industrial society] Because of the familiarity we feel with celebrities, they can serve an important social function: they may be the only “friends” we have in common with new neighbors and coworkers. They’re shared cultural touchstones that facilitate the types of informal interactions that help people become comfortable in new surroundings. Keeping up with the lives of actors, politicians and athletes can make a person more socially adept during interactions with strangers and even offer inroads into new relationships.
I.e., it's a matter of degree and content whether it greases the wheels of society enough to make them spin well (rather than slip).
A couple of peered-reviewed refs for the latter quote (from that research group):
De Backer, C., Nelissen, M., Vyncke, P., Braeckman, J. & McAndrew, F. (2007). Celebrities: From Teachers to Friends. A Test of Two Hypotheses on the Adaptiveness of Celebrity Gossip. Human Nature, 18(4), 334-354. (IF2007: 1.814)
De Backer, C. J., Larson, C., Fisher, M. L., McAndrew, F. T., & Rudnicki, K. (2016). When Strangers Start to Gossip: Investigating the Effect of Gossip on Cooperation in a Prisoner’s Dilemma Game. Evolutionary Psychological Science, 1-10.
The latter paper concludes that:
gossip may spark initial relations, yet is insufficient to ignite a social bond sustained by cooperative action among complete strangers.
And the former:
The Learning Hypothesis explains interest in celebrity gossip as a by-product of an evolved mechanism useful for acquiring fitness-relevant survival information. The Parasocial Hypothesis sees celebrity gossip as a diversion of this mechanism, which leads individuals to misperceive celebrities as people who are part of their social network. Using two preliminary studies, we tested our predictions. [...] In support of the Learning Hypothesis, age proved to be a strong predictor of interest in celebrities. In partial support of the Parasocial Hypothesis, media exposure, but not social isolation, was a strong predictor of interest in celebrities. The preliminary results support both theories, indicate that across our life span celebrities move from being teachers to being friends