Your Scanner Is Lying to You: Building LLM-Assisted Vulnerability Triage

SAST and SCA tools find real vulnerabilities. They also generate enough noise that engineers learn to ignore the queue — false positive rates of 50–80% in production codebases are common, and the ticket backlog becomes a liability in itself.

The hardest problem isn't detection. It's triage.

This talk is a practitioner retrospective on building AVM, an LLM-assisted triage engine that sits between your scanner and your ticket queue. Feed it SARIF output from any scanner; get back structured findings with confidence levels, exploit path narratives, and recommended actions — before a human reviewer ever touches the queue.

I'll walk through what worked (LLMs are genuinely good at common vulnerability patterns), where it struggled (confident false negatives are a real failure mode, and prompt injection via SARIF is weirder than it sounds), and what I'd design differently. Along the way: the threat model for putting an LLM inside your security pipeline, why "inconclusive" is sometimes the most honest answer a machine can give, and what the experience taught me about trusting automated triage.

Working code, real screenshots, and honest lessons from building something that touches your most sensitive findings.