Are Emergent Abilities of Large Language Models a Mirage?

Authors: Rylan Schaeffer, Brando Miranda, Sanmi Koyejo

Venue: NeurIPS 2023 (Outstanding Paper Award).

Quick Links

• Paper
• Poster
• OpenReview
• Tweeprint
• Coverage in Nature
• Citation by US House Committee on Science, Space, and Technology

Summary

We had meant to keep this under wraps for a couple weeks, but it seems that the cat is out of the bag.

Excited to announce our newest preprint: Are Emergent Abilities of Large Language Models a Mirage?

Recent work claims that LLMs display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: (1) sharpness (not present -> present) and (2) unpredictability. We ask whether emergent abilities might be better explained by the researcher’s