Invalid Logic, Equivalent Gains: The Bizarreness of Reasoning in Language Model Prompting

Rylan Schaeffer, Kateryna Pistunova, Samar Khanna, Sarthak Consul, Sanmi Koyejo

ICML 2023 Workshop: Knowledge and Logical Reasoning in the Era of Data-driven Learning Accepted

July 2023

Language Models Chain-of-Thought Reasoning Prompting

PDF Poster Tweeprint

Abstract

We demonstrate that logically invalid chain-of-thought prompts can be as effective as logically valid ones for improving language model performance, raising questions about the nature of reasoning in LLMs.

Summary

Logically invalid chain-of-thought prompts can be as effective as valid ones - what does this tell us about LLM reasoning?

The Puzzle

Chain-of-thought prompting has become a standard technique for improving language model reasoning. But does the logic in the chain actually matter?

Surprising Finding

We demonstrate that logically invalid chain-of-thought prompts can be as effective as logically valid ones for improving language model performance.

This raises fundamental questions about what “reasoning” means in the context of LLMs and whether these models are truly performing logical inference.

---

See the full research page for more details.