Empirical proof for social network models

Question

I've recently encountered the domain of social network modelling while reading the papers on models of strategic voting (unpublished), opinion dynamics and influence spread.

All of these papers create complex computational models, usually modelling social networks as graphs and inferring that their behaviour match actual social behaviour.

However, I don't understand how these models can be empirically rigorous. It seems to me that the ways to validate a model is to match with empirical data, either via data matching or prediction. This doesn't seem to be done in any of those papers.

Consequently, how are these models validated?

Answer 1

Existing Papers

I found three papers in the same vein with considerably more empirical evidence.

1. Modeling the Size of Wars

In the paper, provinces and conflicts are modeled to justify Richardson's observation that the proportion of conflict severity in relation to their frequency is described by a power law. In other words, the more space there is between each conflict, the more casualties will result. The models uses a lot of detail. A geographical map is created and conflicts (down to technological advancement, political structural change and resource allocation) are simulated in models with dozens of parameters. More importantly, the level of detail in the paper allows parameters to be set to test historical scenarios to disprove or give further evidence to the model.

2. The Dynamics of Polarisation

The focus of the paper is modeling the change of public opinion in the United States as a way to provide explanation for two phenomenon:

1. Polarization of opinion is rare despite being perceived otherwise
2. Opinion homogeneity is rare, despite being perceive otherwise.

To model these phenomenon, a network similar to that of the Opinion Dynamics paper from the question is built, with imposed homophily. However, the paper grounds it's parameters in reality (for example, take-off issues that stimulate much discussion are considered rare). In contrast, the Opinion Dynamics paper creates parameters for skepticism and empathy, without giving much consideration for the mechanisms behind these attributes and how they might change over time.

3. A Psychologically-Motivated Model of Opinion Change with Applications to American Politics

This paper justifies modeling 2D distributed agents using the psychological parameters Influence, Susceptibility and Conformity and their interaction. The model is then validated on American political opinions as they shift over time.

Conclusion

There are papers with empirical basis in the domain of social network models. Typically, this realism is achieved by basing parameters on empirical evidence, rather than simplification of cognitive phenomena.

Reply from author of Social Network papers

The author of two of the papers I cited in the question, Alan Tsang, was kind enough to provide a rebuttal to my scepticism via personal correspondence:

The premise behind both the papers was to examine the effects of a particular psychological phenomenon by examining it in isolation. We base our work on more conventional agent models from economics, which assume completely rational actors. We want to see what happens when the rational behavior is tweaked to incorporate a behavioral component. In particular, we are interested in the qualitative effects that are produced, and the mechanisms by which these are achieved. An agent based simulation is the ideal way of studying this, because it allows us to drill down and take measurements that would be impossible or very difficult to do in an actual community. So what we are doing is more mathematics and less science. The results of the papers provide qualitative insights on what are plausible effects of these behaviors on a larger system. The next step could certainly be to validate the model against real data, but the goal of the paper is not to perform "full stack" science. Rather, we hope it would prove useful component in a more detailed model to be used on data collect in the wild.

That said, we are also interested in validating our model against real data as well, but they are difficult to come by. For instance, Facebook almost certainly has access to data that can be used to detect homophily in networks, and how they might affect opinions over time. But the data is very proprietary and, even if we were to get it, ethical concerns might limit how it can be used. As you've pointed out in the follow-up post, there are a few papers that examine political affiliation, and those could potentially be useful ground truth. But I don't have the background to properly assemble such a data set from scratch (as it would need both a time series of opinions, and the underlying network structure). Moreover, there are sure to be other effects at work as well, which may confound the analysis. We're giving some thought to gathering data for the last Canadian election, since strategic voting was so widespread and successful, but any data we collect will likely be disconnected from social network structure. Maybe some aggregate network properties could be inferred (ex: based on region), but it would be a multi-layered problem.

Answer 2

The paper "Can robots make good models of biological behaviour" by Barbara Webb comes from a slightly different area, i.e. modeling biological organisms with robotics, making artificial cockroaches for example.

It has a brilliant theoretical examination of modeling in terms of epistemology and philosophy of science. For your question it is possible that your skepticism is with regard dimensions 5 and 6 mentioned in the paper, i.e. structural accuracy and performance match.

In terms of structural accuracy for example, if a social network uses random graphs, like Erdős–Rényi, it makes an assumption that is not met by the actual degree distributions of human social networks.

In terms of performance match, the accuracy of the model's prediction can be assessed at various levels of precision. For example, the model may 'capture the trends' qualitatively, which may or may not be good enough for large scale simulations.

On the other extreme one would find precise numeric predictions and examine their fit to real data. I guess that most models fall somewhere in between.