43 Probabilistic Relationships (Standard Structures)

Edge : Conditional dependence (direct cause)

Example

We notice the grass in our garden is wet, which might be caused by rain. Construct a Bayesian network to represent their probabilistic relationship, using the relevant CPT.

Solution : The probabilistic graphical model is :

We can use Binary Random Variables :

• $W\in \{0,1\}$ represents whether the grass is wet (1) or not (0).
• $R\in \{0,1\}$ represents whether it has been raining (1) or not (0). Probabilistic relationship : An edge represents the conditional dependence (“cause-effect” relationship) between the parent node (cause) and the child node (effect) : $P(W|R)$.

The Bayesian network with CPTs is :

Prior Knowledge :

• Probability of raining through the day is : $P(R=1)=0.4\Rightarrow P(R=0)=0.6$
• The probability that the grass gets wet when it rains is : $P(W=1|R=1)=0.9\Rightarrow P(W=0|R=1)=0.1$
• The probability that the grass gets wet without raining (e.g., sprinkler on) is: $P(W=1|R=0)=0.2\Rightarrow P(W=0|R=0)=0.8$ Direct Cause : Inference In the “direct cause” structure, there are 2 random variables (nodes). Inference is the problem of finding out the “cause” variable when we only observed the “effect” variable.
• Observed Variables : The ones we have knowledge about.
• Unobserved (hidden) Variables : The ones we do not observe.

Question

If you observe the grass is wet, what’s the probability it rained ?

Solution : By Bayes’ rule

$$P(R=1\mid W=1)=\frac{P(R=1)P(W=1\mid R=1)}{P(W=1)}$$

Calculate the marginal probability of Wet Grass :

$$\begin{array}{rl}P(W=1)& =\sum _{r\in \{0,1\}}P(W=1,R=r)\\ & =\sum _{r\in \{0,1\}}P(W=1\mid R=r)P(R=r)\\ & =(0.9\times 0.4)+(0.2\times 0.6)\\ & =0.48\end{array}$$

Putting it back into the denominator :

$$\begin{array}{rl}P(R=1\mid W=1)& =\frac{P(R=1)P(W=1\mid R=1)}{P(W=1)}\\ & =\frac{0.9\times 0.4}{0.48}\\ & =0.75\end{array}$$

Interpretation :

• The above inference is called diagnosis i.e., to obtain P(Cause | Effect).
• Conclusion : Knowing that the grass is wet increased the probability it rained from $P(R=1)=0.4$ to $P(R=1|W=1)=0.75$.

Conditional Independence : Why It Matters ?

The Full Joint Distribution :

$$P(A,B,C)=P(C)P(A|C)P(B|A,C)$$

Graphical Representation :

Parameter Count :

• Suppose $A,B\text{and}C$ are binary variables $(0\text{or}1)$.
• Full joint distribution requires $1+2+4=7$ parameters.
• With conditional independence, we have $P(A,B,C)=P(C)P(A|C)P(B|C)$ so only need $1+2+2=5$ parameters.

Indirect cause

Consider the following: Cloudy influences Rain, so we have:

Hence, wetGrass is independent of cloudy give rain. The joint distribution encoded in this graph is :

$$P(C,R,W)=P(C)P(R|C)P(W|R)$$

Common Cause

It is natural to suppose the cloudy weather influence our decision to turn on the sprinkler or not and cloudy weather will also influence the chance of rain, we model this “common cause” probabilistic relationship using :

The joint probability distribution:

$$P(C,S,R)=P(C)P(S|C)P(R|C)$$

If C is observed, then S and R are independent.

Common Effect

Suppose we identify another cause of wet grass, the sprinkler, we can then model the relationship as a converging connection (Common effects):

The joint probability distribution :

$$P(S,R,W)=P(S)P(R)P(W|S,R)$$

If neither W nor any of its descendants are observed, then S and R are independent. Explaining away : If we can confirm one cause (e.g., rain is true) of an observation (e.g., grass is wet), it reduces the need to invoke alternative causes, e.g., sprinkler is on.