AI in Penetration Testing: A Systematic Mapping Study AI in Penetration Testing: A Systematic Mapping Study

Problem & Motivation 问题与动机

The paper addresses the lack of a comprehensive systematic mapping of AI techniques applied to penetration testing, investigating what AI methods are used, their effectiveness, the challenges of integration, and future research directions across studies published between 2015 and 2025.

本文解决了将AI技术应用于渗透测试缺乏全面系统映射的问题，调查了2015年至2025年间发表的研究中使用了哪些AI方法、它们的有效性、集成的挑战以及未来的研究方向。

As cyberattacks grow in frequency and sophistication, traditional penetration testing methods are resource-intensive and struggle to keep pace. AI offers the potential to automate, scale, and improve the accuracy of penetration testing, but the landscape of AI-driven approaches is fragmented across diverse techniques (RL, DL, generative AI, supervised ML) with no unified overview of the state of the art, gaps, and future directions.

随着网络攻击频率和复杂程度的增加，传统的渗透测试方法资源密集且难以跟上步伐。AI具有自动化、扩展和提高渗透测试准确性的潜力，但目前AI驱动方法的研究散布在各种技术（强化学习、深度学习、生成式AI、监督学习）中，缺乏对最新技术、研究空白和未来方向的统一概述。

Threat Model 威胁模型

Not explicitly defined; the paper assumes a standard penetration testing threat model where testers simulate real-world attacks to identify and exploit vulnerabilities in target systems and networks.

未明确定义；本文假设了一个标准的渗透测试威胁模型，即测试人员模拟真实世界的攻击，以识别和利用目标系统和网络中的漏洞。

Methodology 核心方法

A systematic mapping study that collected and analyzed 57 primary studies published between 2015 and 2025 from major academic databases (IEEE Xplore, ACM Digital Library, SpringerLink, Elsevier, MDPI, Wiley, Scopus). Papers were selected using structured keyword searches, filtered by inclusion/exclusion criteria, and evaluated against ten quality assessment criteria on a three-point scale. The analysis is organized around four research questions covering AI techniques used (RQ1), challenges and limitations (RQ2), effectiveness of AI-driven techniques (RQ3), and research trends, gaps, and future directions (RQ4).

一项系统映射研究，收集并分析了2015年至2025年间从主要学术数据库（IEEE Xplore, ACM Digital Library, SpringerLink, Elsevier, MDPI, Wiley, Scopus）发表的57项主要研究。论文通过结构化关键词搜索进行选择，根据纳入/排除标准进行筛选，并针对十项质量评估标准在三点量表上进行评估。分析围绕四个研究问题展开，涵盖了所使用的AI技术（RQ1）、挑战和局限性（RQ2）、AI驱动技术的有效性（RQ3）以及研究趋势、空白和未来方向（RQ4）。

Architecture 架构设计

N/A - this is a survey/mapping study, not a system paper.

不适用——这是一项综述/映射研究，而非系统性论文。

Memory Mechanism 记忆机制

none

Attack Phases Covered 覆盖的攻击阶段

Evaluation 评估结果

Reinforcement learning is the most frequently adopted AI technique (20 papers), followed by deep learning (13), generative AI models (7), and supervised machine learning (7). Over 60% of the 57 selected papers were published between 2023 and 2025, indicating rapidly growing research interest. The study identifies six key benefits (automation, efficiency, improved vulnerability detection, scalability, optimization, and training enhancement) and four major challenge areas (scalability, language model limitations and bias, training/convergence/data efficiency, and tool/model integration).

强化学习（RL）是最常被采用的AI技术（20篇论文），其次是深度学习（13篇）、生成式AI模型（7篇）和监督式机器学习（7篇）。在选定的57篇论文中，超过60%发表于2023年至2025年间，表明研究兴趣迅速增长。该研究确定了六大主要益处（自动化、效率、提高漏洞检测、可扩展性、优化和培训增强）以及四个主要挑战领域（可扩展性、语言模型局限性和偏差、训练/收敛/数据效率以及工具/模型集成）。

Metrics 评估指标

Scale 评估规模

57 primary studies reviewed

Contributions 核心贡献

• Systematic mapping of 57 studies on AI in penetration testing published between 2015 and 2025, categorized by AI technique (RL, DL, generative AI, supervised ML)
• Identification and categorization of four main AI technique families used in penetration testing, with reinforcement learning being the most prevalent
• Synthesis of six key benefits of AI integration in penetration testing: automation, efficiency improvement, improved vulnerability/threat detection, scalability and learning improvement, optimization and planning, and cybersecurity training enhancement
• Identification of four major challenge categories: scalability and real-world applicability, language model limitations and AI bias, training convergence and data efficiency, and tool/model integration and deployment
• Roadmap of four future research directions: scalability and real-world deployment improvements, model and algorithm enhancements, automation and methodology development, and security/ethics/bias mitigation

• 对2015年至2025年间发表的57项关于渗透测试中AI的研究进行了系统映射，并按AI技术（RL、深度学习、生成式AI、监督学习）进行了分类
• 识别并分类了渗透测试中使用的四大主要AI技术家族，其中强化学习最为盛行
• 总结了AI集成到渗透测试中的六个关键益处：自动化、效率提升、改进漏洞/威胁检测、可扩展性和学习改进、优化与规划，以及网络安全培训增强
• 识别了四个主要的挑战类别：可扩展性和真实世界的适用性、语言模型局限性和AI偏差、训练收敛和数据效率，以及工具/模型集成与部署
• 规划了四个未来研究方向：可扩展性和真实世界部署改进、模型和算法增强、自动化和方法论发展，以及安全/伦理/偏差缓解

Limitations 局限性

• Limited to English-language papers only, potentially missing relevant work in other languages
• Papers published before 2015 were excluded, omitting earlier foundational work
• Quality assessment threshold of 10 points may have excluded some relevant but lower-quality studies
• As a mapping study rather than a meta-analysis, it does not perform quantitative synthesis of results across studies
• Single author affiliation (student paper from King Saud University), which may limit breadth of perspective
• Does not provide hands-on evaluation or empirical comparison of the surveyed AI techniques

• 仅限于英语论文，可能遗漏了其他语言的相关研究
• 排除了2015年之前发表的论文，忽略了早期的基础性工作
• 10分的质量评估阈值可能排除了一些相关但质量较低的研究
• 作为映射研究而非荟萃分析（meta-analysis），它没有对不同研究的结果进行定量合成
• 单一作者所属机构（沙特国王大学的学生论文），这可能会限制视角的广度
• 未对调查的AI技术提供实际评估或实证比较

Research Gaps 研究空白

• Lack of scalable AI models that can handle large and complex real-world network environments
• Insufficient standardized testbeds and benchmarks for evaluating AI-driven penetration testing tools
• Limited real-world validation of AI-based penetration testing approaches; most work uses simulated environments
• Language models (e.g., PentestGPT, Llama 3.1-405B, GPT-4o) still struggle with complex tasks like privilege escalation and exploitation on medium-to-hard machines and require human intervention
• Training convergence and data efficiency problems in RL-based approaches, including sparse rewards and large state/action spaces
• Ethical concerns around AI bias, misuse potential, and lack of transparency in AI-driven penetration testing tools remain under-explored
• Integration difficulties between AI models and existing heterogeneous cybersecurity tools
• Need for interdisciplinary collaboration between cybersecurity and AI research communities

• 缺乏能够处理大型且复杂的真实网络环境的可扩展AI模型
• 缺乏用于评估AI驱动渗透测试工具的标准化测试床和基准
• 基于AI的渗透测试方法的真实世界验证有限；大多数研究使用模拟环境
• 语言模型（如PentestGPT, Llama 3.1-405B, GPT-4o）在处理中等及以上难度的机器上的权限提升和漏洞利用等复杂任务时仍然吃力，且需要人类干预
• 基于强化学习的方法中存在训练收敛和数据效率问题，包括奖励稀疏以及巨大的状态/动作空间
• 围绕AI驱动渗透测试工具的AI偏差、滥用潜力和缺乏透明度等伦理问题仍缺乏探索
• AI模型与现有的异构网络安全工具之间的集成困难
• 网络安全和AI研究社区之间需要跨学科合作

Novel Techniques 新颖技术

• Categorization of AI-in-pentest landscape into four technique families: RL, DL, generative AI, and supervised ML
• Identification of Deep Q-Learning Networks (DQN) and Deep Reinforcement Learning (DRL) as particularly promising for automating attack path optimization
• Mapping of specific frameworks (Shennina, PenBox, CIPHER, IAPTS, PentestGPT) to their AI techniques and penetration testing phases

• 将渗透测试中的AI图景分类为四个技术家族：RL、深度学习、生成式AI和监督学习
• 识别出深度Q网络（DQN）和深度强化学习（DRL）对于自动化攻击路径优化特别有前景
• 将特定框架（Shennina, PenBox, CIPHER, IAPTS, PentestGPT）与其AI技术和渗透测试阶段进行了映射

Open Questions 开放问题

• How can AI-based penetration testing tools achieve full autonomy without human intervention for complex exploitation tasks?
• What standardized benchmarks and testbeds should the community develop for fair comparison of AI-driven penetration testing approaches?
• How can the balance between exploration and exploitation in RL-based pentest agents be optimized for large-scale networks?
• What governance frameworks are needed to prevent misuse of AI-driven penetration testing tools?
• Can domain-specific LLMs (like CIPHER) match or exceed general-purpose LLMs for penetration testing guidance?

• 基于AI的渗透测试工具如何在无需人类干预的情况下完成复杂的漏洞利用任务，从而实现完全自主？
• 社区应该开发哪些标准化基准和测试床，以便对AI驱动的渗透测试方法进行公平比较？
• 对于大规模网络，如何优化基于强化学习的渗透测试智能体在探索（exploration）与利用（exploitation）之间的平衡？
• 需要什么样的治理框架来防止AI驱动的渗透测试工具被滥用？
• 在渗透测试指导方面，领域特定的LLM（如CIPHER）能否达到或超过通用LLM的表现？

Builds On 基于前人工作

• Garg and Bansal (2021) - systematic review on penetration testing
• McKinnel et al. (2019) - systematic literature review on AI in penetration testing and vulnerability assessment
• Greco et al. (2023) - AI-enabled IoT penetration testing state-of-the-art
• Amalfitano et al. (2023) - AI applied to software testing tertiary study

Open Source 开源信息

No

Tags