Deep Clustering for Unsupervised Learning of Visual Features
Research Questions
How to adapt clustering in end-to-end training of visual features on large-scale datasets?
Approach
Proposes DeepCluster, a clustering method that learns parameters of neural network as well as cluster assignments of resulting features.
Idea: alternate clustering logits of the network and then training the network via classification, using the cluster identities as targets. Use K-Means to cluster logits.
Questions:
- How to prevent degenerate solutions?
- Cluster labels are arbitrary, whereas class labels are not. How do we ensure cluster labels are stable learning targets?
- Why use classification at all? Why not just backprop on network parameters in K-means objective function?
Heuristics
- When a cluster has no points assigned, delete that cluster and create a new one by splitting a big cluster in two.
- To prevent cluster imbalance, sample points such that each cluster is equally represented
- Use data augmentations (random horizontal flips, crops, aspect ratios)
Results
Left: NMI between ground truth classes and unsupervised cluster labels on ImageNet
