Kernel Methods

At a high level, kernel methods are a machine learning approach that works by first transforming any data (images, songs, etc.) into (possibly infinite dimensional) feature spaces, computing the pairwise similarity of each datum through inner products, and then fitting a simple model (i.e. a linear readout) from the feature space.

Background

• Introduction
• Inner product spaces
• Hilbert spaces

Kernels

• Kernels
• Reproducing Kernels
• Mercer’s Theorem
• Nystrom Approximation
• Riesz Representation Theorem

Sources

• Julien Mairal and Jean-Philippe Vert’s INRIA lectures
• Shawe-Taylor and Cristianini Kernel Methods for Pattern Analysis