Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detection
A comprehensive taxonomy and effective detection methods for glitch tokens in Large Language Models.
Jul 15, 2024
A Hitchhiker's Guide to Jailbreaking ChatGPT via Prompt Engineering
A comprehensive guide to jailbreaking ChatGPT via prompt engineering techniques.
Apr 20, 2024
MASTERKEY: Automated Jailbreaking of Large Language Model Chatbots
A comprehensive framework for automated jailbreaking of Large Language Model chatbots, featuring novel attack methodologies and systematic analysis of defense mechanisms.
Feb 26, 2024
MASTERKEY
MASTERKEY is a research framework for automated jailbreak attack generation and defense evaluation for LLM chatbots. The framework supports systematic analysis of jailbreak strategies across commercial chatbot systems and was published at NDSS 2024.
Feb 26, 2024
A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
A comprehensive analysis of jailbreak attack and defense techniques for Large Language Models.
Feb 20, 2024
PANDORA: Jailbreak GPTs by Retrieval Augmented Generation Poisoning
Novel attack framework exploiting RAG mechanisms to jailbreak LLMs through retrieval database poisoning. Distinguished Paper Award winner.
Feb 1, 2024
PANDORA
PANDORA studies jailbreak attacks against retrieval-augmented generation systems through retrieval database poisoning. The work received the Distinguished Paper Award at AISCC 2024 and highlights a practical attack surface in RAG-enhanced LLM deployments.
Feb 1, 2024
Digger: Detecting Copyright Content Mis-usage in Large Language Model Training
A novel approach to detecting copyright content mis-usage in Large Language Model training data.
Jan 1, 2024
PentestGPT
An LLM-empowered automatic penetration testing framework with 14k+ GitHub stars and 2.4k+ forks. PentestGPT is designed to automate penetration testing by leveraging the domain knowledge inherent in Large Language Models. It features a three-module architecture (Reasoning, Generation, and Parsing) that emulates human penetration testing workflows. Key Features: Multi-module agent design for reasoning, generation, and parsing Integration with multiple LLM backends and real-world security workflows Evaluation on CTF challenges and practical penetration testing targets 228.6% task completion improvement over baseline GPT models Recognition: Distinguished Artifact Award at USENIX Security 2024 Widely used open-source security research artifact with active community adoption
Aug 1, 2023
Prompt Injection Attack against LLM-integrated Applications
A comprehensive study of prompt injection attacks against LLM-integrated applications.
Jun 9, 2023