In August, an AI squad submitted a new voice-agent feature for adversarial review. The seven-persona pass came back with nothing above Medium. The feature wasn’t simple – the squad had already run the review against their own code three times and fixed the critical issues themselves before it reached the security team. That told us the model was working.
AI features at Artera reach production faster than a quarterly security review cycle can track. Review everything at the end, gate the release, hope nothing slipped through – that approach stops working once features ship this fast.
So we rebuilt the review itself: security that runs continuously, inside the workflow, at the moment engineers write the code, instead of as a late-stage gate. That’s what produced the August result. Security moved into the hands of the people building the feature, and our team spent its judgment on the handful of things that actually needed it.
Why we built the model ourselves
When we set out to scale security coverage for AI, we chose to build an internal, repeatable capability instead of buying a one-time external engagement. Our new CTO, Damon Lanphear, gave us the mandate: build something we own, that scales, and that improves over time.
It started as a single prompt, then grew into a reusable skill. Today it runs partly through GitHub Actions, with an agent in pilot. Over about 15 weeks, our security team ran roughly 60 adversarial reviews against net-new AI product changes – about four a week – without adding a dedicated application-security headcount.
It’s a shift-left layer on a mature security program, running alongside SAST, DAST, and penetration testing rather than replacing them. It brings an adversary’s mindset to a change before it ships.
What the adversarial review actually is
It’s an adversarial review run against net-new product pull requests through a panel of distinct adversary lenses. Each persona probes a different threat model. Sub-agents then fan out to identify, validate, and steel-man the flaws a single reviewer, human or model, would miss.
We describe it as a seven-persona review because that’s what it is in practice: memorable, repeatable, and not every persona runs on every change.* The lenses:
1. External Attacker
2. Malicious Insider
3. Supply-Chain Attacker
4. Abuse-of-Trust Insider
5. Nation-State Adversary
6. AI Jailbreaker / Prompt-Injection Adversary
7. GRC / Regulatory / Retention / Data-Lifecycle Auditor
A tiered intake sets the right depth for each change, across four tiers:
- Attestation: human sign-off for low-risk changes
- Trip-wire: a lightweight automated pass
- Security Architecture Design Review: for changes that touch how the system is built
- Deep Review: the full adversarial panel
Match the depth to the risk, and you spend the expensive review only where it earns its keep.
How it fits into the workflow
A squad submits an adversarial review, which auto-creates a Jira ticket. An intake skill triages the review type against the four tiers. A human confirms the call and acts on it.
The real shift happens upstream: squads now run the lightweight version against themselves before they submit. That’s how the August voice-agent feature I mentioned earlier came back clean. The squad ran multiple self-run passes, fixed the big issues early, and handed us work with almost nothing left to catch. People getting good at security made that self-sufficiency possible.
Training the humans, not just the models
Application security used to be a narrow conversation between the security team and a handful of software engineers and DevOps staff. Now product AI Builders and CX AI Builders, people who never sat in a traditional AppSec seat, build personas, give feedback, and run reviews themselves.
What I’m proudest of doesn’t show up in a scan report: our people are getting sharper at security. They’re picking up AI security skills by doing the work, faster than a traditional AppSec career path would teach them.
But that didn’t happen on day one. In fact, the first two months had real friction. We worked through an early backlog of flaws and spent serious time coaching non-technical AI Builders through security concepts they’d never had to own before. That coaching wasn’t overhead. It was the point: the friction is where the learning happened.
Keeping it sharp with continuous improvement
We treat our internal squads like customers. Every week or two, we update the skill’s memory and context based on their feedback. One recent example: we tuned the narrative style of the report so findings stay readable and actionable instead of turning into noise builders learn to ignore.
We also keep the coverage current. We add new attack vectors as they emerge and sharpen the security plugins behind each persona.
Where this is headed
We’re moving deeper into automation, further into GitHub Actions and Agent Core. The destination is a human-on-the-loop model, not human-in-the-loop: fully automated, human-initiated, with a person supervising instead of hand-carrying every step. The gating logic:
– Critical / High → alert a human and block the CI/CD pipeline.
– Medium / Low → route to the AI builder to remediate within a defined window.
The payoff is leverage: one security professional covers far more ground by applying judgment only where it changes the outcome. That’s roughly the equivalent of two to three dedicated AppSec engineers’ worth of product-security coverage, without adding headcount or standing up an external engagement.
A Model for Healthcare AI Security
In about two months, we launched this and got it running smoothly across multiple squads, expanding our security work into genuine application security.
The bigger point is the model: a repeatable way for healthcare AI teams to build security that scales with their products instead of trailing behind them. Artera already treats security as foundational: SOC 2 Type 2, HITRUST, HIPAA, and FedRAMP Class D Certification, and we never use identifiable PHI or PII to train our models. This is one more way we hold that line as AI accelerates.
See how we think about responsible AI in healthcare at our Trust & Security Center: https://artera.io/trust-center/.
*We’re running eight personas today – but not all eight run on every change
