How to measure/predict the difficulty of a conflict?

Question

Question

I want to conduct an experiment and I found that the difficulty of a conflict to resolve could play a factor in this experiment. I now need an easy way of measuring the potential difficulty of a conflict, so that I can control this factor.

Own Proposal

I'm coming from StackOverflow and am used to present own proposals (since psychology is not primarily my field of expertise I wanted to have your opinion for my approach: I thought of a questionnaire that is supposed to measure various factors that could influence the difficulty of a conflict. The factors I thought of are:

• Personal Involvement (PI): The potential change and stress that a given conflict can have on the life of a conflict party. e.g. a colleague is telling you to bring out the trash while you don't feel like doing it.
• Influence of Emotions (IoE): How strong Emotions influence conflicts.
• Objective vs. Subjective (OvS): Are Arguments in the conflict based on evidence or are more testimonial kind.
• Risk (R): What is the probability and damage of an undesired outcome for a conflict party?
• Level of Society (LoS): 1-to-1 vs. 1-to-many vs. many-to-many, also is it a conflict between countries of just a few people. personal-family-inner friend circle-outer friend circle-foreign groups-countries-cultures ... just to give some example to what that could include.
• potential violence (PV): Probability and potential dangerousness of on party using violent measures.
• explicitness of conflict result(s) (ECR): I think some conflicts have explicit results like "Do Aliens exist?" and some don't necessarily have a definite answer like "What is an appropiate outfit for a woman? (which is answered differently in different cultures)"
• conflict resolution skills of the conflict partys (CRS): This also might include negotiating and communication skills as well.
• medium (M): speech, written, social network, face-to-face, etc.

I then thought of measuring those metrics on a scale of 1-7, weight them and add them add for some kind of conflict difficulty score:

Say Risk has a 10% influence on the conflict difficulty the weight, therefore, would be 0,1*(Scale of Risk). I also think this way the result might be 2 different scores depending on the partys point of view.

I also think that some of the metrics I proposed are not needed or cannot be measured using a questionnaire like the medium for example. But before I really go all the way to create a new questionnaire I wanted to know if I'm on the right track, if there is literature on this topic I missed, if I missed other important factors and so on.

Background

I have implemented a chat tool for my bachelors thesis, that displays emotions and handles profanity in messages differently. I know want to evaluate this software. The goal of this software is to improve communication in conflict situations and a new solution for hate speech in social networks. I also have a software engineer background and this is my first question in the psychology sector.

Answer 1

Questionnaires and measurement scales typically take a lot of revision and validation. Otherwise, it is very common that you measure things you don't intend to measure, are missing relevant options, or omit critical factors entirely.

I'd recommend some piloting and to use existing and validated measures whenever possible.

Some of your items sound incredibly broad to me. Are you going to rate "influence of emotions" on a 7-point scale by asking people "On a scale of 1-7, how emotional do you feel about this situation?" That might be fine, but there's also a ton of nuance to emotion that will be missed.

You also have a lot of factors, so it will take a lot of data to resolve their relative influence (potentially hundreds of cases; thousands if you care about interactions between factors) in order to determine proper weights, and you still need to come up with a ground truth "conflict difficulty" outcome measure. Otherwise you are just guessing at weights which isn't very useful.

Make sure you have IRB approval before continuing with human subjects research (I've been surprised how unaware some tech people are about regulatory requirements, especially at Academia.SE).

Given this is a bachelors' thesis, it may be sufficient to start some of these processes rather than complete them. It might be simpler to just do an experiment, test two versions of your software, and compare some satisfaction outcomes. You won't know which conflicts were more difficult, but you won't necessarily have to: you can rely on random assignment to assume that the comparison groups are equal.