Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning

Stephen Lewis; Aikaterini Kanta

Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Downloads (Pure)

Abstract

Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.

Original language	English
Title of host publication	Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.
Publisher	Open Review
Number of pages	9
Publication status	Published - 3 Jun 2026
Event	The Eighteenth Workshop on Adaptive and Learning Agents: ALA 2026 - Paphos, Cyprus Duration: 1 Jan 2026 → …

Conference

Conference	The Eighteenth Workshop on Adaptive and Learning Agents
Country/Territory	Cyprus
City	Paphos
Period	1/01/26 → …

Keywords

LLMs
Agents
Penetration Testing
Cybersecurity
Agentic AI
Lang- Graph
Large Language Models
GAAI

Access to Document

Let it hackFinal published version, 448 KBLicence: CC BY

https://openreview.net/forum?id=6wkbjeOSdR

Cite this

Lewis, S., & Kanta, A. (2026). Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning. In Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security. Open Review. https://openreview.net/forum?id=6wkbjeOSdR

Lewis, Stephen ; Kanta, Aikaterini. / Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning. Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.. Open Review, 2026.

@inproceedings{2064f0e171b646f8b635b199af0b46a1,

title = "Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning",

abstract = "Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.",

keywords = "LLMs, Agents, Penetration Testing, Cybersecurity, Agentic AI, Lang- Graph, Large Language Models, GAAI",

author = "Stephen Lewis and Aikaterini Kanta",

year = "2026",

month = jun,

day = "3",

language = "English",

booktitle = "Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.",

publisher = "Open Review",

note = "The Eighteenth Workshop on Adaptive and Learning Agents : ALA 2026 ; Conference date: 01-01-2026",

}

Lewis, S & Kanta, A 2026, Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning. in Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.. Open Review, The Eighteenth Workshop on Adaptive and Learning Agents, Paphos, Cyprus, 1/01/26. <https://openreview.net/forum?id=6wkbjeOSdR>

Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning. / Lewis, Stephen ; Kanta, Aikaterini.
Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.. Open Review, 2026.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning

AU - Lewis, Stephen

AU - Kanta, Aikaterini

PY - 2026/6/3

Y1 - 2026/6/3

N2 - Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.

AB - Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.

KW - LLMs

KW - Agents

KW - Penetration Testing

KW - Cybersecurity

KW - Agentic AI

KW - Lang- Graph

KW - Large Language Models

KW - GAAI

M3 - Conference contribution

BT - Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.

PB - Open Review

T2 - The Eighteenth Workshop on Adaptive and Learning Agents

Y2 - 1 January 2026

ER -

Lewis S , Kanta A. Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning. In Recent advances in large language models (LLMs) have opened new opportunities for automating complex tasks in Cybersecurity, including offensive operations. However, most existing approaches to LLM-assisted penetration testing rely on human input or scripted interactions. This work explores a fully autonomous, local-agent framework for end-to-end penetration testing, using LangGraph to structure multi-stage tool-augmented reasoning. Without human intervention post-launch, selected open-source LLMs were tasked with scanning, vulnerability analysis, and exploitation against a standard testbed. Results show that models such as Qwen-14B and Qwen-32B can successfully execute multiple real-world exploits, demonstrating that local, API-free LLM agents can move beyond advisory roles into operational offensive security.. Open Review. 2026

Let it hack: autonomous multi-agent penetration testing with LLMs and tool-augmented reasoning

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this