How to start the AI-accelerated defense

Early on in the AI adoption boom, I gained a reputation for just throwing everything at it to see what would stick. That wasn’t the most effective strategy, and my token usage was crazy high. There are a ton of talks and posts on all the cool ways you can use AI for detection engineering, but I didn’t see any that showed you where to begin. So, this isn’t another blog about why you need to use AI in your defensive workflows. It seems most people understand why we need that. My focus is to show how our team got started and realized that providing AI with the necessary context is key to detection engineering successfully adopting AI. This is not just about building detection logic, but that is one of the goals. This foundation helps create the AI Detection Engineering stack: logging pipelines, log onboarding, detection validation, threat modeling, and more. An LLM does not know your stack, so out of the box it has limited value in a security review. In our experience, reliable results depend less on the fanciest model and more on the documentation and context around the workflow. If a human reads your log inventory and still has to ask three people what the ingestion method is, your agent does too. Why cold prompting fails When we first started using AI tooling, we realized prompts alone could get stuff done, but the output was inconsistent. Fields were missed, assumptions were made, and some detection logic was wrong. We saw it write queries that would not work in our SIEM. Usually these were around wildcards. The playbooks it wrote were generic, the tuning was poor, and some detections were just bad. With enough re-prompting, the output would improve, but it always required some massaging. The effort invested in the agent inputs had a noticeable impact on the quality of the outputs. TL;DR: garbage in, garbage out. Where our context came from At the start, this was just internal documentation we built to make our own lives easier. It started with new-hire materials about where logs live, who owns each tool, and what needs protection. That is the bar. If you would hand it to a new hire on day one, it is good enough to onboard the AI (I fear saying this a bit, knowing onboarding isn’t always the best). Just like an intern, if you neglect to set the context, your agent will guess and hand you something that looks right, but isn’t aligned to the team’s actual goals. So every artifact below gets judged against that one test. All of these documents are dual-use. Humans read them, and agents read them. Once again, think of this like onboarding documentation, then modify it to your agents’ needs. The better the data, the better the context, the better the outcome for your agents. The good news is AI can help you make these with a read-only access key. The bad news is you are going to have to read and edit some slop. I ordered these roughly by what to do first. If you have two hours this week, start at the top. Context inventories and libraries Log inventory — what, how, where, format Start with this. A basic log inventory needs a name, category, priority, and owner. It gets more complicated with ingestion methods, vendor or internal contacts, and, to go one step further, detailed notes on what the logs provide. We always include external resources. Blogs, vendor documentation, anything else that will help get someone up to speed on what is being provided. This part does more work than you would think. When an agent has the vendor documentation for a log source, it stops guessing at field names. An agent with a real log inventory can tell you whether a detection idea is even possible before you write a line of logic. That alone is worth the afternoon. Detection library — centralized, mapped, and tagged Creating a single point for all your EDR, SIEM, and CNAPP detections can be hard, but it helps answer that age-old question. Can we detect that? This will require some normalization because not all security tools save the same data. You'll want ATT&CK tagging, log source, and in some cases, origin if you're tracking where the detection ideas came from. Some fields you will need for metrics are “Date Created”, “Date Modified”, and “Dates Tested”. Believe it or not, not all companies track the “Date Modified” field. The last big part is having a description and/or playbook for the detection. Depending on the tool, some limit the number of characters you can put in a description. Our team uses a modified version of the Alerting and Detection Strategies Framework. This gives us a clear understanding of every detection and the next steps for triage when it fires. Having a structured output format defined is also why an agent can generate consistent, high-quality playbooks for us now. Tool inventory — who owns what, contact info These are the tools you'll use to investigate, but you'll also want the tools your counterpart teams use: EDR, SIEM, CNAPP, as well as ticketing, inventory, or internal knowledge sources. These

Excerpt shown under the publisher's RSS terms · 186 words · full text and figures are on the publisher's site