Local LLMs in AppSec: What Open Tools and Models Can Handle in Application Security

The rise of local large language models has opened new possibilities for application security teams that prefer to keep sensitive code and data inside their own infrastructure. Instead of purchasing cloud subscriptions, organizations can now run models on workstations, link them to existing code analysis tools, and delegate selected parts of security research to the AI. Open-source orchestration frameworks make it possible to control exactly which data reaches the model, where processing occurs, and how autonomous the verification process becomes.

Early experiments showed that simply assigning an entire audit to a small local model often produced results that were difficult to interpret. Researchers therefore began breaking security work into discrete tasks of varying complexity: parsing scanner findings, tracing user-controlled data flows, or conducting multi-step application investigations. This decomposition approach, first demonstrated in projects such as PentestGPT, allowed models to maintain context across successive steps rather than losing the thread of an assessment.

By 2024 and 2025 several open-source environments matured enough to support fully local deployments. Ollama and llama.cpp provided efficient model execution on consumer hardware, while agent frameworks including PentAGI, Strix, and Microsoft’s PyRIT added capabilities for tool use, action selection, and result logging. Vulnhuntr demonstrated how models could request specific functions and classes when analyzing Python projects, and Taskflow Agent from GitHub Security Lab introduced reusable task descriptions that could be shared independently of the underlying execution engine.

Three main usage patterns have emerged. In static review, a model receives code fragments flagged by analyzers such as Semgrep and helps determine whether a reported issue is reachable and exploitable. In dynamic testing, agentic systems equipped with browsers, proxies, and terminals allow the model to choose requests, observe responses, and refine hypotheses while respecting role-based access rules. Finally, when applications already contain embedded LLMs, dedicated testing suites such as Garak, Giskard, and DeepTeam enable repeated prompt-injection and capability-abuse checks against both the target model and any supporting evaluation models.

Successful adoption requires careful attention to performance and verification. Agentic workflows accumulate long histories and issue many inference calls, so teams must benchmark configurations on realistic tasks rather than isolated prompts. Model errors early in an investigation can derail subsequent steps, and any verdict produced by a secondary evaluator model still needs expert review against actual executed actions or returned data. Additional components such as embedding models for retrieval must also be assessed for data leakage risks when confidentiality is paramount.