0
$\begingroup$

I am trying to make the better user experience for my poll application and one of the ideas I have for this is breaking multi choices polls into binary polls.

Is the statistical result of a question with these 4 options (a or b or c or d) equal with sum of statistical results of these 6 binary questions?:

(a or b), (a or c), (a or d), (b or c), (b or d), (c or d)

assume I ask these questions from separate people and I ask these questions equally

for example: assume we have a society with 1000 members if we ask all of them the first question, people will answer it with this distribution: a:60%, d:20%, b:15%, c:5%

now assume if we ask those six binary questions from all members of that society and then we sum each vote of winners of each of those questions.

we ask 1000 times a or b and the answer is a:600,b:400

we ask 1000 times a or c and the answer is a:500,c:500

we ask 1000 times a or d and the answer is a:100,d:900 ....)

we sum number of votes on a and it is 1300 =600+500+100

we do a similar thing with b,c, and d, is this result similar to the result of the first question? (a:60%, d:20%, b:15%, c:5% ) and is the meaning of this result similar to the first one?

===update-1===

@Bruno mention to the inconsistency of the result of this approach and I think it can be solved if we do not show all combinations to all voters. If voter1 chooses "a" between "a" and "b" we do not show questions with any combination of "b".

===update-2===

In fact, what I wish to know from this community is: "Does anybody do statistical research on how results of these two types of questionnaires are different?" The problem with this research is that when you ask any of these question types from someone it affects the result of the second one (people want to show themselves rationally. and if we do not ask questions with the same items for example (a or b), (a or c) (a or b or c) from the same person we do not certain about the complete answer of this particular person and the data is insufficient for comparing) On the second chance if there is not any already done research on this I want to know what is the proper (psychologically proper) way to do this research?

$\endgroup$
6
  • 6
    $\begingroup$ I'm voting to close this question as off-topic because it is a better fit for CrossValidated.SE $\endgroup$
    – AliceD
    Mar 1, 2016 at 14:30
  • 4
    $\begingroup$ I'm voting to close this question as off-topic because it is a better fit for CrossValidated (stats.SE). $\endgroup$
    – Arnon Weinberg
    Dec 14, 2021 at 2:56
  • $\begingroup$ in fact when I wrote this question I did not know there is such field of science :D $\endgroup$ Dec 16, 2021 at 7:41
  • 1
    $\begingroup$ If it is statistics you are after, the Statistics SE CrossValidated are happy to receive it (got in touch with the mods there). Cross-posting is frowned upon, however, but maybe you can rewrite the question with a more statistical flavor to it? Perhaps remove the edit here and make it into a new question. We may actually want to close it here then, since it is pretty much offtopic here. $\endgroup$
    – AliceD
    Dec 24, 2021 at 14:11
  • 1
    $\begingroup$ Survey design questions, on the other hand, are on topic here. If you rephrase your question here to focus more on what you now only assume, I think it would make it a good question here. But, it's likely best to post that as a new question entirely given the existing answers this question already received. $\endgroup$
    – Steven Jeuris
    Dec 24, 2021 at 14:23

2 Answers 2

2
$\begingroup$

It slightly depends on what you are asking and you might want to post on one of the mathematical stack exchanges instead.

If you are asking "Do I get the same information by asking binary comparisons as by asking the four-option choice?" Then my answer is that you actually get more information, both about the pattern in the group and about a particular individual. Imagine if person X chooses A in the four option case, all you know is that they prefer A the best. If they then complete all 6 binary options, you will still find this out, but you will also know how they feel about B vs. C etc, so you will be able to find the complete ranking for each person. For the whole group, you would certainly be able to find the overall ranking, but I don't think you'd be able to sum the choices in the way you state.

If you are asking "will people respond in the same way?", then the answer is probably "No" because, as Bruno notes, people are not always rational and will be affected by the number of choices. Even if you imagine there is some random "noise" in their decision, then making six independent choices will produce some conflicts and potentially amplify this "error".

At any rate, it doesn't seem like this is going to make the user experience more comfortable! This is not my area, but you could read something about "rational choice theory" to get some background about the math/assumptions. https://en.wikipedia.org/wiki/Rational_choice_theory

$\endgroup$
3
  • $\begingroup$ I do not ask all of 6 questions from one person. I can ask multiple a or b and if he answered "a" then I can ignore all "b or c" and "b or d" questions. $\endgroup$ Feb 29, 2016 at 13:11
  • $\begingroup$ @MohammadRezaEsmaeilzadeh yes, you could. But that still requires 3 separate questions to get essentially the same information as the 4 option case (assuming they keep choosing A over everything else, it will get more complicated if they do not). You might want to randomly simulate some data and see what happens. $\endgroup$
    – splint
    Mar 1, 2016 at 7:37
  • $\begingroup$ the whole point is two option questions are easier to answer users and have better UX. $\endgroup$ Dec 15, 2021 at 16:53
1
$\begingroup$

That's a good question, but... No way, man, they're not equal!! When you ask someone to choose between 4 options, their answer just means about their pick for the 4 options. And this, and only this, is the way to know the preference of people for those 4 options!

The way you're purposing could even generate an inconsistency... Suppose that one person prefer A over B and C, but not over D. It doesn't means that they would necessarily prefer D over B and C, nor means that A or D would be the choice among all. Our choices are not rational, so, if just the order you offer the 4 (or 2) options could induce people to prefer one of the options, asking all vs all could generate a lot of noise!

I'm not saying it's a bad idea, depends on the purpose of your question, of course, and on what you want to know!

I don't know what's your objective, exactly, but maybe you want to ask people to put them in order of importance! e.g., you might be wanting to ask people: "Classify these options in order of importance". This is what does it looks like, by your question, but it's just a guess at all.

It all will depend on what you want to know and the evolved specificity. In some cases, it would be a good idea, in others not.

$\endgroup$
4
  • $\begingroup$ what if I ask these questions by a little intelligence: if I ask a or b and voter picks a i will not ask questions that has "b" as a choice? $\endgroup$ Feb 29, 2016 at 7:31
  • $\begingroup$ I am trying to make user experience of my poll application more comfortable and one of ideas I have for this is breaking multi choices poll to binary polls $\endgroup$ Feb 29, 2016 at 7:33
  • 1
    $\begingroup$ @MohammadRezaEsmaeilzadeh, I understand your idea for making it more comfortable and easy to do. This is why I think asking the user to rank could be a good idea! Anyway, I don't believe the results could be comparable in any way. $\endgroup$
    – Bruno
    Mar 14, 2016 at 14:44
  • $\begingroup$ Hey @MohammadRezaEsmaeilzadeh, I was reading an interesting text on this matter a couple of days ago, while going deeper into my literature investigation on origins of Behavioral Finance. It's Marshack's chapter "Utilities, Psychological Values and the Training of Decision Makers", in Allais&Hagen (1979)'s "EXPECTED UTILITY HYPOTHESES AND THE ALLAIS PARADOX". I think it's worth to read!! I'm editing the answer to include this example. $\endgroup$
    – Bruno
    Dec 17, 2017 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.