# Trying to understand equations in Karl Friston article

I am trying to understand a neuroscience article by Karl Friston. In it he gives three equations that are, as I understand him, equivalent or inter-convertible and refer to both physical and Shannon entropy. They appear as equation (5) in this article (PDF; DOI 10.1007/s00422-010-0364-z). Here they are:

• Energy minus entropy: $F = −{\ln p(\tilde s,Ψ|m)}q + {\ln q(Ψ|μ)}q$
• Divergence plus surprise: = $D(q(Ψ|μ)||p(Ψ|\tilde s,m)) − \ln p (\tilde s|m)$
• Complexity minus accuracy: = $D(q(Ψ|μ)||p(Ψ|m)) − {\ln p(\tilde s|Ψ,m)}q$

The things I am struggling with at this point are:

1. the meaning of the || in the 2nd and 3rd versions of the equations
2. the negative logs

Any help in understanding how these equations are actually what Fristen claims them to be would be greatly appreciated. For example, in the 1st equation, in what sense is the first term energy, etc.?

• Jul 11, 2014 at 23:48
• +1 for also being confused by much of Fristen's writing. Jul 15, 2014 at 17:02
• Cross-posted on Mathematics
– honi
Nov 16, 2015 at 19:52
• Cross-posted on Data Science
– honi
Nov 17, 2015 at 18:59

As mentioned in the definitions section on page 2 of the paper, $D(q||p) = <ln(q/p)>_q$ is the Kullback-Leibler divergence or cross-entropy between two densities.
2. || is a notation used in divergences (or info theory) where the order matters. That is $D(A||B) \neq D(B||A)$ as in KL divergence