No, Of Course I Can! Refusal Mechanisms Can Be Exploited Using Harmless Data

Joshua Kazdan, Lisa Yu, Rylan Schaeffer, Chris Cundy, Sanmi Koyejo, Krishnamurthy Dj Dvijotham

ICLR 2025 Workshop on Building Trust in Language Models and Applications Accepted

April 2025

Language Models AI Safety Fine-tuning Refusal Mechanisms

Summary

Workshop version: refusal mechanisms can be exploited through harmless fine-tuning data.