Probabilistic Graphical Models

Machine Learning Probability

Overview

Bayesian networks, Markov random fields, and inference algorithms.

Overview

A joint distribution over \(N\) variables \(\underline{x}\) can permit complicated relationships between the variables. One general principle is that performing inference becomes easier in the absence of relationships between variables. This motivates us to specify relationships between variables as a graphs and then use the graph structure to design efficient inference algorithms.

• History
• Types of Graphical Models
  • Directed Graphs
  • Undirected Graphs
  • Directed vs Undirected
  • Factor Graphs
  • Ancestral Graphs
  • Graph Relations
  • Causal Graphs
• Parameter Estimation
  • Maximum Likelihood
• Types of Inference Problems
  • Marginalization
  • Most Probable Configuration / MAP
• Exact Inference Algorithms
  • Elimination Algorithm
  • Sum-Product (Belief Propagation)
  • Junction Tree Algorithm
• Approximate Inference Algorithms
  • Loopy Belief Propagation
  • Variational Inference
  • Graph Partitioning
• Learning
  • Some variables unobserved, graph is known -> Expectation Maximization
  • All variables observed, graph is unknown Chow Liu Algorithm
  • Structure