Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks

Rylan Schaeffer, Punit Singh Koura, Binh Tang, Ranjan Subramanian, Aaditya K Singh, Todor Mihaylov, Prajjwal Bhargava, Lovish Madaan, Niladri S Chatterji, Vedanuj Goswami

arXiv preprint Under Review

February 2025

Language Models Evaluation Human Evaluation Benchmarks

arXiv

Abstract

Summary

Predicting human evaluations of language models from NLP benchmark scores.