mutualInformation {CPTtools} | R Documentation |
Calculates the mutual information for a two-way table of observed counts or a joint probability distribution. The mutual information is a measure of association between two random variables.
mutualInformation(table)
table |
A two way table or probability distribution. Possibly
the output of the |
This is basically the Kullback-Leibler distance between the joint probability distribution and the probability distribution created by assuming the marginal distributions are independent. This is given in the following formula:
I[X;Y] = sum Pr(X=x, Y=y) log [Pr(X=x,Y=y)/Pr(X=x)Pr(Y=y)] where the sum is taken over all possible values of x and y.
Russell Almond
http://planetmath.org/encyclopedia/MutualInformation.html
Shannon (1948) “A Mathematical Theory of Communication.”
## UCBAdmissions is a three way table, so we need to ## make it a two way table. mutualInformation(apply(UCBAdmissions,c(1,2),sum)) apply(UCBAdmissions,3,mutualInformation) apply(UCBAdmissions,2,mutualInformation) apply(UCBAdmissions,1,mutualInformation)