What Makes a Good LLM Agent for Real-world Penetration Testing?
A 2026 study of real-world LLM penetration testing agents and Excalibur, a difficulty-aware agent for end-to-end attack planning.
Feb 19, 2026
Excalibur
Excalibur is a difficulty-aware LLM agent design for real-world penetration testing. It couples typed tooling, retrieval-augmented security knowledge, task difficulty assessment, and evidence-guided attack tree search to reduce planning failures in multi-step penetration testing tasks.
Feb 19, 2026
RSafe: Incentivizing Proactive Reasoning to Build Robust and Adaptive LLM Safeguards
NeurIPS 2025 work on adaptive reasoning-based safeguards for robust LLM safety moderation.
Dec 2, 2025
When Audio and Text Disagree: Revealing Text Bias in Large Audio-Language Models
Revealing and analyzing text bias in Large Audio-Language Models when audio and text inputs disagree.
Nov 1, 2025
Oedipus: LLM-enchanced Reasoning CAPTCHA Solver
An LLM-enhanced framework demonstrating vulnerabilities in reasoning-based CAPTCHA systems through AI-powered solving.
Oct 1, 2025
IllusionCAPTCHA: A CAPTCHA based on Visual Illusion
A WWW 2025 CAPTCHA design that uses visual illusions to create human-friendly but AI-hard verification tasks.
Apr 28, 2025
Source Code Summarization in the Era of Large Language Models
A comprehensive study of source code summarization capabilities with Large Language Models.
Apr 27, 2025
Efficient Detection of Toxic Prompts in Large Language Models
ASE 2024 work on efficient greybox detection of toxic prompts for large language models.
Oct 27, 2024
GenderCARE: A Comprehensive Framework for Assessing and Reducing Gender Bias in Large Language Models
CCS 2024 work on assessing and reducing gender bias in large language models.
Oct 14, 2024
PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing
An LLM-empowered automated penetration testing framework that leverages domain knowledge inherent in LLMs, achieving 228.6% task completion improvement over baseline GPT models.
Aug 14, 2024