Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detection
Jul 15, 2024·,
,,,,,,,·
1 min read
Yuxi Li
Yi Liu
Gelei Deng
Ying Zhang
Wenyuan Song
Liming Shi
Kailong Wang
Yuekang Li
Yang Liu
Haoyu Wang
Abstract
Glitch tokens are anomalous tokens in Large Language Models that can cause unexpected behaviors, including crashes, hallucinations, or security vulnerabilities. This work presents a comprehensive taxonomy for categorizing glitch tokens and proposes effective detection methods to identify and mitigate their impact on LLM systems.
Type
Publication
Proceedings of the ACM on Software Engineering (FSE)
This work provides a systematic study of glitch tokens in Large Language Models, establishing a categorization taxonomy and developing effective detection methods to improve the reliability and security of LLM systems.
