RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents

Problem & Motivation 问题与动机

Existing LLM-based penetration testing tools either focus on post-exploitation or require a human-in-the-loop to validate commands and guide exploitation. There is no fully automated system that can achieve IP-to-Shell compromise (initial access from just a target IP address) without any human intervention.

现有的基于大语言模型的渗透测试工具要么侧重于后期利用，要么需要人工干预来验证命令并指导漏洞利用。目前还没有一个能够在没有任何人工干预的情况下实现“IP 到 Shell”突破（仅凭目标 IP 地址获得初始访问权限）的全自动系统。

Initial infiltration is the most critical and challenging phase of penetration testing, yet it has received comparatively little attention in LLM-based automation. Most existing approaches rely on human-in-the-loop validation, creating a barrier for organizations without dedicated security staff. A fast, low-cost, fully autonomous initial-access tool would make penetration testing accessible to non-specialists and allow expert pentesters to offload repetitive tasks.

初始渗透是渗透测试中最关键且最具挑战性的阶段，但在基于 LLM 的自动化研究中受到的关注相对较少。大多数现有方法依赖人工验证，这为缺乏专门安全人员的组织设立了门槛。一个快速、低成本、全自主的初始访问工具将使非专业人士也能进行渗透测试，并允许专业渗透测试人员从重复性任务中解脱出来。

Threat Model 威胁模型

The attacker (RapidPen) is provided only the IP address of the target machine, with no additional configuration details or vulnerability disclosures. The system assumes TCP connectivity to the target (optionally via OpenVPN). RapidPen aims to obtain a reverse shell via Metasploit Framework exploitation. No credentials or prior knowledge of the target are assumed.

攻击者（RapidPen）仅获得目标机器的 IP 地址，没有额外的配置细节或漏洞披露。系统假设与目标具有 TCP 连接（可选通过 OpenVPN）。RapidPen 旨在通过 Metasploit Framework 漏洞利用获得反弹 shell。不假设拥有任何凭据或对目标的预验知识。

Methodology 核心方法

RapidPen is a fully automated pentesting agent that takes a single target IP address and attempts to obtain shell access without human intervention. It adopts a ReAct-style architecture with a Re (reasoning/task planning) module and an Act (command generation and execution) module, both augmented by specialized RAG repositories. The system uses a Pentesting Task Tree (PTT) as its core data model to structure and coordinate scanning, enumeration, and exploitation tasks. A self-correcting feedback loop in the Act module handles command failures through retry, parameter adjustment, or tool substitution. Historical success-case PTTs from prior engagements are retrieved via RAG to guide task generation for similar vulnerabilities.

RapidPen 是一个全自动渗透测试智能体，仅需一个目标 IP 地址，即可尝试在无需人工干预的情况下获取 shell 访问权限。它采用了 ReAct 风格的架构，包含 Re（推理/任务规划）模块和 Act（命令生成与执行）模块，两者都辅以专门的 RAG 存储库。该系统使用渗透任务树（PTT）作为核心数据模型，用于组织和协调扫描、枚举和漏洞利用任务。Act 模块中的自我修正反馈循环通过重试、参数调整或工具替换来处理命令失败。来自先前任务的历史成功案例 PTT 通过 RAG 检索，以指导针对类似漏洞的任务生成。

Architecture 架构设计

Two-module ReAct architecture: (1) Re module containing a PTT Planner (L1) that expands the pentesting task tree based on command results, a PTT Prioritizer (L1) that selects the next task, and a New Task Generation (L2) submodule that queries historical success cases via RAG; (2) Act module containing Command Generation (L1) that leverages offensive security knowledge and HackTricks RAG, Command Execution (L1) for automated execution in a custom Docker sandbox, and Log Analysis (L1) for interpreting results and triggering self-correction. The system uses a Pentesting Task Tree (PTT) in JSON format as the central data structure, with environment metadata, act results in nodes, and parent-child task relationships. A visualization tool (RapidPen-vis) provides real-time monitoring via a Flask backend and JavaScript frontend.

双模块 ReAct 架构：(1) Re 模块包含一个 PTT 规划器（L1），根据命令结果扩展渗透任务树；一个 PTT 优先排序器（L1），选择下一个任务；以及一个新任务生成（L2）子模块，通过 RAG 查询历史成功案例。(2) Act 模块包含命令生成（L1），利用进攻性安全知识和 HackTricks RAG；命令执行（L1），在自定义 Docker 沙箱中自动执行；以及日志分析（L1），用于解释结果并触发自我修正。系统使用 JSON 格式的渗透任务树（PTT）作为中心数据结构，包含环境元数据、节点中的执行结果以及父子任务关系。可视化工具（RapidPen-vis）通过 Flask 后端和 JavaScript 前端提供实时监控。

LLM Models 使用的大模型

Tool Integration 工具集成

Memory Mechanism 记忆机制

RAG

Attack Phases Covered 覆盖的攻击阶段

Evaluation 评估结果

With success-case RAG enabled, RapidPen achieved a 60% success rate (6/10 runs) on the HTB Legacy machine (MS17-010/EternalBlue), with execution times of 200-400 seconds and per-run costs of $0.30-$0.60. Without success-case RAG, the success rate dropped to 30% (3/10 runs) with higher variance in execution time and more frequent stalling. The Re (L1) PTT Planner dominated LLM cost, while Act (L1) Command Execution consumed the most wall-clock time.

在启用成功案例 RAG 的情况下，RapidPen 在 HTB Legacy 机器（MS17-010/EternalBlue）上实现了 60% 的成功率（10 次运行中成功 6 次），执行时间为 200-400 秒，单次运行成本为 0.30-0.60 美元。在不使用成功案例 RAG 的情况下，成功率降至 30%（10 次运行中成功 3 次），执行时间波动较大且更频繁出现停滞。Re (L1) PTT 规划器占据了主要的 LLM 成本，而 Act (L1) 命令执行消耗了最多的实际时间。

Environment 评估环境

Metrics 评估指标

Baseline Comparisons 基准对比

• RapidPen with success-case RAG vs. RapidPen without success-case RAG

Scale 评估规模

1 HackTheBox machine (Legacy), 20 total runs (10 with success cases, 10 without)

Contributions 核心贡献

• Design and implementation of RapidPen, a fully autonomous IP-to-Shell penetration testing framework requiring no human intervention, combining ReAct-style task planning with RAG-augmented knowledge bases
• Extension of the PentestGPT Pentesting Task Tree (PTT) data model with environment metadata, act results in nodes, and JSON-based I/O to support fully automated reasoning
• Two specialized RAG repositories: a Command Generation RAG (148 HackTricks markdown files) and a Success Cases RAG (historical PTTs from prior successful engagements)
• A self-correcting feedback loop in the Act module with three-strike retry, timeout handling, and fail-fast mechanisms for robust command execution
• Preliminary empirical evaluation demonstrating 60% success rate for shell acquisition within minutes at sub-dollar cost

• 设计并实现了 RapidPen，一个全自主的“IP 到 Shell”渗透测试框架，无需人工干预，结合了 ReAct 风格的任务规划与 RAG 增强知识库
• 扩展了 PentestGPT 的渗透任务树（PTT）数据模型，增加了环境元数据、节点执行结果和基于 JSON 的 I/O，以支持全自动推理
• 建立了两个专门的 RAG 存储库：命令生成 RAG（148 个 HackTricks Markdown 文件）和成功案例 RAG（以往成功任务的历史 PTT）
• 在 Act 模块中设计了自我修正反馈循环，具有三次重试、超时处理和快速失败机制，确保健壮的命令执行
• 初步的实证评估表明，在几分钟内、成本低于 1 美元的情况下，获取 shell 的成功率为 60%

Limitations 局限性

• Evaluated on only a single vulnerable machine (HTB Legacy with MS17-010) — no evidence of generalization to diverse vulnerability types or targets
• No post-exploitation capabilities: does not attempt privilege escalation, lateral movement, or data exfiltration after obtaining a shell
• No web-based attack support: web vulnerabilities such as injection, authentication bypass, and GUI-based interactions are excluded
• Limited to TCP-based targeting; UDP-based exploits and scans are not considered
• Passive reconnaissance (domain records, metadata leaks) is excluded; only active port scanning is used as the starting point
• Success-case RAG is only effective when the target vulnerability closely matches a previously recorded engagement; zero-day or novel vulnerabilities require more advanced reasoning
• The fail-fast error handling mechanism can cause premature termination when partial remediation (e.g., installing missing packages) would suffice
• Large PTT inputs to the LLM (sometimes exceeding 14KB) increase cost and latency; no input pruning is implemented
• 40% failure rate even with success-case RAG, often due to incorrect command parameters, hallucinated non-leaf tasks, or timeouts during nmap scanning
• Built on Dify platform as a prototype; not yet production-ready or open-source

• 仅在单一易受攻击机器（具有 MS17-010 的 HTB Legacy）上进行评估 —— 尚无证据表明其能泛化到多种漏洞类型或目标
• 缺乏后期漏洞利用能力：获得 shell 后不尝试权限提升、横向移动或数据外泄
• 不支持基于 Web 的攻击：排除了 Web 漏洞（如注入、身份验证绕过）和基于图形界面的交互
• 仅限于基于 TCP 的目标；未考虑基于 UDP 的利用和扫描
• 排除了被动侦察（域名记录、元数据泄漏）；仅将主动端口扫描作为起点
• 成功案例 RAG 仅在目标漏洞与先前记录的任务高度匹配时有效；0day 或新型漏洞需要更高级的推理
• 快速失败错误处理机制在部分修复（如安装缺失包）即可解决问题时，可能会导致过早终止
• 输入到 LLM 的 PTT 过大（有时超过 14KB），增加了成本和延迟；尚未实现输入剪裁
• 即使使用成功案例 RAG，失败率仍达 40%，原因通常是错误的命令参数、幻觉产生的非叶子任务或 nmap 扫描期间超时
• 作为一个原型构建在 Dify 平台上；尚未准备好投入生产或开源

Research Gaps 研究空白

• Fast, fully automated initial access (IP-to-Shell) remains largely unaddressed; most LLM-based tools focus on post-exploitation or require human-in-the-loop
• No existing framework for abstracting and transferring exploit strategies across similar but not identical vulnerabilities — current success-case reuse is essentially pattern matching
• Handling scenarios where no relevant historical success cases exist requires more advanced reasoning beyond copying past exploit paths
• Integration of initial access tools with post-exploitation frameworks (like BLADE, AutoAttacker) for end-to-end automated pentesting pipelines is unexplored
• Web exploit automation in the context of fully autonomous network pentesting remains an open challenge
• Error categorization and nuanced recovery strategies (beyond simple retry or termination) for LLM-driven command execution need development
• Benchmarking and standardized evaluation of fully autonomous pentesting systems across diverse vulnerability types is lacking

• 快速、全自动的初始访问（IP 到 Shell）在很大程度上仍未解决；大多数基于 LLM 的工具侧重于后期利用或需要人工干预
• 目前尚无框架可以将漏洞利用策略在相似但不相同的漏洞之间进行抽象和迁移 —— 当前的成功案例重用本质上是模式匹配
• 处理没有相关历史成功案例的场景需要超越复制过去利用路径的高级推理
• 将初始访问工具与后期利用框架（如 BLADE, AutoAttacker）整合以构建端到端自动渗透测试流水线的研究尚属空白
• 全自主网络渗透测试背景下的 Web 利用自动化仍然是一个开放的挑战
• 需要开发针对 LLM 驱动的命令执行的错误分类和细化恢复策略（不仅限于简单的重试或终止）
• 缺乏跨多种漏洞类型的全自主渗透测试系统的基准测试和标准化评估

Novel Techniques 新颖技术

• Success-case RAG: retrieval of historical Pentesting Task Trees from prior successful engagements to guide task generation for targets with similar vulnerability profiles
• Layered ReAct architecture with separate Re (planning) and Act (execution) modules, each with specialized L1/L2 submodules and dedicated LLM instances (10 for Re, 8 for Act)
• Extended PTT data model with environment metadata, structured act results per node, and strict JSON-based I/O to constrain LLM hallucination
• Three-strike self-correcting feedback loop with adaptive timeout doubling and tool substitution (e.g., nmap to rustscan) on execution failures
• Fail-fast strategy for unrecoverable errors (COMMAND_NOT_FOUND, FILE_NOT_FOUND) that terminates the session rather than wasting resources on futile retries

• 成功案例 RAG：检索以往成功任务的历史渗透任务树（PTT），以指导针对具有相似漏洞特征的目标的任务生成
• 分层 ReAct 架构：具有独立的 Re（规划）和 Act（执行）模块，每个模块都有专门的 L1/L2 子模块和专用的 LLM 实例（Re 使用 10 个，Act 使用 8 个）
• 扩展的 PTT 数据模型：包含环境元数据、每个节点的结构化执行结果以及严格的基于 JSON 的 I/O，以约束 LLM 幻觉
• 三次重试自我修正反馈循环：在执行失败时具有自适应超时翻倍和工具替换（如将 nmap 替换为 rustscan）机制
• 针对不可恢复错误（COMMAND_NOT_FOUND, FILE_NOT_FOUND）的快速失败策略：直接终止会话，而不是将资源浪费在无效的重试上

Open Questions 开放问题

• How well does success-case RAG transfer across vulnerability classes that are similar but not identical (e.g., different SMB vulnerabilities, or different service types)?
• Can the system scale to targets with multiple potential entry points requiring coordinated or sequential exploitation?
• What is the optimal balance between exploration (trying new attack paths) and exploitation (following known success patterns) in autonomous pentesting?
• How would RapidPen perform against hardened targets with IDS/IPS, firewalls, or other active defenses?
• Can the approach be extended to handle web application vulnerabilities that require browser interaction and multi-step authentication bypass?
• What safety mechanisms are sufficient for responsible deployment of fully autonomous offensive security tools?
• How should the PTT input be pruned or summarized to reduce LLM costs without losing critical context for planning?

• 成功案例 RAG 在相似但不相同的漏洞类别（例如不同的 SMB 漏洞或不同的服务类型）之间的迁移效果如何？
• 系统能否扩展到具有多个潜在入口点、需要协调或顺序漏洞利用的目标？
• 在自主渗透测试中，探索（尝试新攻击路径）与利用（遵循已知成功模式）之间的最佳平衡点是什么？
• RapidPen 在面对具有 IDS/IPS、防火墙或其他主动防御措施的加固目标时表现如何？
• 该方法能否扩展到处理需要浏览器交互和多步身份验证绕过的 Web 应用程序漏洞？
• 哪些安全机制足以支撑全自主进攻性安全工具的负责任部署？
• 应如何剪裁或总结 PTT 输入，以在不丢失规划关键背景的情况下降低 LLM 成本？

Builds On 基于前人工作

• PentestGPT
• ReAct
• HackTricks
• Dify

Open Source 开源信息

No

Tags