Streaming Inference for Infinite Feature Models

Rylan Schaeffer, Yilun Du, Gabrielle K Liu, Ila Fiete

International Conference on Machine Learning Accepted

July 2022

Bayesian Nonparametrics Streaming Inference Feature Learning

PDF Poster Tweeprint

Abstract

We derive streaming inference algorithms for infinite feature models based on the Indian Buffet Process, enabling unsupervised feature learning on streaming data where the number of features grows as necessitated by the data.

Summary

Streaming inference algorithms for infinite feature models (Indian Buffet Process).

Summary

Biological intelligence operates in a radically different data regime than AI: data that is unsupervised, streaming, and non-stationary. This work addresses how to perform feature learning on streaming data.

The Problem: Using a preset fixed number of features on streaming data results in an inability to model the data as complexity grows. How can we define a learning algorithm that grows in representational capacity as necessitated by the stream?

Our Solution: We derive a novel recursive form of the Indian Buffet Process designed specifically for streaming data:

Key Results:

1. The Recursive IBP is exact for the stochastic process
2. On synthetic data, it achieves equal or better performance than non-streaming baselines in significantly less time
3. On real tabular data, it again equals or outperforms many baselines
4. On MNIST, it infers features whose class-similarities match those of supervised deep networks