1
$\begingroup$

In my layman's experience, I'm vaguely aware there are four base emotions: happy, sad, afraid/surprised, and angry/disgusted.1

Some background: We're training an AI to learn the difference between happy voices and angry voices. We've had some success, by showing it 200 angry audio clips, 200 happy audio clips, and 200 neutral. It can now reasonably tell when we're talking pleasantly or confrontationally... but the accuracy could be better.

Our total training dataset is made up of these audio clips: Happy, angry, neutral, calm, sad, fearful, disgust, and surprised. I think we can be more accurate by including these emotions.

But this is the problem:

Happy/angry/neutral span opposite ends of a spectrum; like binary. It's easy to say:

Happy     1
Neutral   0
Angry    -1

That's the shape of the data we need to train a neural network to recognize 'Happy'.

So the question would be, is there any 'right answer' on filling in these blanks? I've given it my best guesses below, but I'm hoping for something more scientific....

Happy     1
Angry    -1
Neutral   0
Calm      X  (0.5?)
Sad       X  (-1?)
Fearful   X  (-0.5?)
Disgust   X  (-0.75?)
Surprised X  (0.75?)

1: http://www.theatlantic.com/health/archive/2014/02/new-research-says-there-are-only-four-emotions/283560/

$\endgroup$
  • $\begingroup$ I believe this is more of StackOverflow Question but what you could do is have 8 different output nodes in the output layer that can be either 0 or 1. With some sigmoid functions in the hidden layer and supervised learning I believe this may be the best way. What you could also do is output a probability (0 to 1) for each emotion. Then you can select the one with the highest value. I would not recommend going with your approach, because it will probably often confuse surprise with happy e.g., given their proximity. I don't believe you can arrange these emotions so linearly namely. $\endgroup$ – Robin Kramer Jun 21 '16 at 19:49
  • $\begingroup$ What is the evidence (or reasoning) for why happy and angry span opposite ends of a spectrum? Which spectrum? $\endgroup$ – mrt Jun 21 '16 at 19:59
  • $\begingroup$ To clarify further, the emotions you listed can either differ or overlap on several dimensions, including valence, arousal, and conceptual content. What you may be distinguishing in your happy vs. neutral vs. angry paradigm is not the emotions per se, but their valence or arousal. $\endgroup$ – mrt Jun 21 '16 at 22:42
  • $\begingroup$ @mrt The spectrum is customer satisfaction, so dissatisfied seems to translate into angry and satisfied to happy, if you agree? We're also capturing the meaning/sentiment of the words said too, for context, but that's a separate process. $\endgroup$ – RJB Jun 27 '16 at 18:29
  • $\begingroup$ @RobinKramer We're using pyAudioAnalysis for our black box. Not much idea how it sciences the audio. Our hypothesis isn't necessarily to pinpoint an exact emotion, but to be able to tell like a 'good mood' vs a 'bad mood' voice -- by identifying traits of negatively and positively charged emotion $\endgroup$ – RJB Jun 27 '16 at 18:50
1
$\begingroup$

As I mentioned in the comments, the tool we're using is pyAudioAnalysis. The tool's author, Theodoros Giannakopoulos, also wrote an earlier paper "A dimensional approach to emotion recognition of speech from movies"

This "Emotion Wheel" from that paper appears to be exactly what I was looking for: enter image description here

It's also worth mentioning, maybe, that this later paper seems to describe all the specific characteristics that pyAudioAnalysis analyzes: "signal energy, entropy of energy, zero crossing rate, spectral centroid, spectral flux, Mel Frequency Cepstral Coefficients, Chroma-based features, etc."

$\endgroup$
  • 3
    $\begingroup$ BTW, the emotion wheel is called the circumplex model of affect (see James Russell and Lisa Feldman Barrett). Affect is just one feature of emotions (e.g., the wheel doesn't capture conceptual content). Indeed, you aren't really "recognizing" emotions by recognizing valence and arousal. You're recognizing affect. For example, high arousal + positive valence doesn't equal excitement, but the emotion "excitement" (typically, but not always) involves high arousal and positive valence. $\endgroup$ – mrt Jul 6 '16 at 21:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.