As cybersecurity teams struggle to keep pace with the ever-growing number of vulnerabilities in their applications, many have turned to large language models (LLMs) as a potential solution. However, recent tests have shown that these models are not living up to expectations, with over 60% of flagged vulnerabilities turning out to be false positives or low-severity issues.
This is a major problem for security teams, who are already stretched thin trying to triage and prioritize the many issues they face. As Arshan Dabirsiaghi, chief technology officer and co-founder at Pixee, an AI-powered application-security startup, points out, companies are forced to choose between two unappealing options: either flood their developers with a constant stream of pull requests to update dependencies, or have humans manually review each vulnerability – a task that is simply not scalable.
The issue lies in the way LLMs are being taught and used. These models are often trained on generic datasets and lack the context and nuance needed to accurately identify vulnerabilities. As Dabirsiaghi puts it, “LLMs are dumb” – they can only provide a broad indication of a potential problem without understanding its specific context.
In order to effectively prioritize vulnerabilities, three key types of context are needed: organizational, technical, and code-level. For example, an application that uses MD5 hashing for password storage is likely to have a high-severity vulnerability, while one that uses it for non-security purposes may not be a problem at all.
One area where LLMs do show promise is in identifying vulnerabilities that are actually reachable by attackers. By measuring reachability, security teams can significantly reduce the number of issues they need to patch, as many open-source libraries and code segments are never used at runtime.
The fact remains, however, that LLMs are not yet a reliable solution for vulnerability prioritization. In order to make them work effectively, companies will need to provide more context and create specialized tools that can harness their power.
For security teams, the takeaway is clear: while AI and machine learning hold promise, they are not yet ready to be relied upon as a sole solution. Instead, teams should focus on developing human expertise in vulnerability triage, using techniques such as reachability analysis to reduce the scope of issues they need to address. By striking a balance between technology and human judgment, security teams can make progress towards keeping pace with the ever-growing number of vulnerabilities facing their applications.
Source: Dark Reading — 2026-07-21