Graphical Models

February 21, 2019

Stefano Martina

---

Probabilistic graphical models are a way to express probability models using graph. Each node represents a random variable, while an arc denotes probabilistic relationship. The graph denomination changes if the graph is directed or not:

• Bayesian networks are directed graphical models;
• Markov random fields are undirected graphical models.

Bayesian Networks

To introduce Bayesan networks, consider the probability chain rule where the joint distribution over variables is decomposed as product of conditioned probabilities:

$$\begin{equation} p(a,b,c) = p(c|a,b)\ p(a,b) = p(c|a,b)\ p(b|a)\ p(a) \label{eq:chain} \end{equation}$$

The corresponding graphical model of the rightest term of \eqref{eq:chain} is depicted in Fig 1 .

The arcs represents conditional probabilities, e.g. the random variable $c$ has two incoming arcs because is conditioned by the other two random variables. Note that the ordering of the random variables in the leftmost term of \eqref{eq:chain} is irrelevant, thus we can choose different graphic representation for the same model.

Bayesian networks are only directed acyclic graphs, thus they are oriented graph. For each node $n$, the other nodes that have an outgoing edge to $n$ are denoted as $parents$.

For a generic graph with $K$ nodes $\{x_1,\dots,x_K\}$, the joint distribution is given by:

$$\begin{equation*} p(\vec{x}) = \prod_{k=1}^{K}p(x_k|pa_k) \end{equation*}$$

where $\vec{x}$ is $\{x_1, \dots, x_K\}$ and $pa_k$ is the set of parents of $x_k$.

Polynomial regression

---

References

---

Footnotes