Best Practices for Risk Automation

Still trying to manage your ever-increasing alert flow by hiring more analysts? That’s much like adding buckets to deal with a leaking roof. Invest in detection engineering and automation engineering to reduce the alert flow and prevent alert fatigue and unhappy analysts. Here are some best practices: - Apply an automation-first strategy: handle and/or accelerate all alerts through automation - Continuously tune and optimize detection rules - Let analysts and detection / automation engineers work closely together to increase the effectiveness of engineering efforts - Establish metrics for rule quality to identify candidates for tuning and automation - Test against defined quality criteria before putting any detection rules live - Increase the fidelity of your rules by alerting on more specific criteria - Aggregate and analyse batches of noisy alerts daily or weekly, instead of handling them individually in real-time - Consider your ideal ratio between analysts and engineers. Start out with 50-50, then decide what would best suit your needs - Make risk-based decisions on added value of rules compared to time investment, and drop time-consuming rules with little added value if they cannot be tuned properly This is by no means an easy thing to do. But by focussing on engineering and detection quality, you can transition to a state where you control of the alert flow instead of the other way around, so that analysts can focus on the alerts that truly matter. #soc #securityoperations #securityanalysis #detectionengineering #automationfirst

I've reviewed Anthropic's Risk Report for Claude Opus 4.6 because many of our enterprise customers are actively deploying AI agents into production environments. When those systems fail, the consequences are operational, financial and reputational. Most of the reaction centers on the headline that catastrophic risk is very low but not negligible. What matters more for customers and future customers is how risk actually manifests inside live enterprise systems and what that means for uptime, data integrity and compliance. It does not look like a breach. It looks like business as usual. An agent subtly influencing procurement decisions. A finance workflow that starts omitting inconvenient data. Permissions that expand over time without clear oversight. Anthropic describes a scenario called Persistent Rogue Internal Deployment, where an AI system with privileged access creates a less monitored instance of itself and continues operating inside production systems. In a real enterprise environment, that translates into downtime, data exposure or regulatory impact. The organizations at greatest risk are not the ones moving cautiously. They are the ones who pushed agents into production without adding an operational governance layer. We have seen this pattern before in cloud adoption. Technology advances quickly, and controls often lag behind. That gap is where exposure grows. So what should enterprise IT and security teams do now? 1. Constrain actions, not just access. Define what an agent can set in motion and enforce least privilege at the identity level, just as you have done for human users for decades. 2. Log actions, not just outcomes. Maintain an auditable trail of what the agent did, where and what triggered it, the same standard applies to human operators in regulated environments. 3. Automate your tripwires. Do not rely on people to catch machine speed behavior. Build policy enforcement and anomaly response into the loop. 4. Audit your agent footprint. Inventory every agent, its owner, permissions and kill path. Governance starts with visibility and most enterprises are still building it. The window to build these guardrails is now, before the agent workforce scales. At Rackspace, 25 years of running mission-critical systems have taught us that trust without controls creates exposure. We build and operate AI infrastructure with governance embedded from day one because customers need speed, resilience and measurable outcomes, not experiments in production. What this means for you is simple. Move forward on AI with confidence, but make operational governance part of the foundation so scale strengthens your business instead of introducing risk.

An AI model that "kind of" works isn’t good enough. Here’s 10 principle form the last IMDRF : 1) Define a clear intended use & involve experts Outline a precise intended use that meets clinical needs. Engage experts across disciplines to refine it and assess risks at every stage. 2) Strong engineering, design & security practices Ensure traceability, reproducibility, and data integrity. Apply robust security and risk management to protect patient safety. 3) Representative datasets for clinical evaluation Use datasets that reflect the real patient population. Diversity and sufficient size help ensure unbiased performance. 4) Independent training & test datasets Keep training and test datasets completely separate. Perform external validation based on risk levels. 5) Fit-for-purpose reference standards Use clinically relevant standards aligned with the intended use. If no standard exists, document the rationale for selection. 6) Model choice aligned with data & intended use Ensure model design fits the data and mitigates risks. Set clear performance goals and account for variability. 7) Human-AI interaction in device assessment Evaluate performance within clinical workflows. Consider human factors like skill level, autonomy, and misuse risks. 8) Clinically relevant performance testing Assess real-world performance independently from training data. Test across patient subgroups and factor in human-AI interactions. 9) Clear & essential user information Communicate intended use, limitations, and updates transparently. Ensure users understand model function, risks, and feedback mechanisms. 10) Ongoing monitoring & retraining risk management Continuously monitor models to ensure safety and performance. Use risk-based safeguards to manage bias, overfitting, and dataset drift. Developing AI/ML medical devices? These principles should be your foundation. Source: Good machine learning practice for medical device development: Guiding principles / IMDRF/AIML WG/N88 FINAL:2025

Working with AI Agents in production isn’t trivial if you’re regulated. Over the past year, we’ve developed five best practices: 1. Secure integration. Not “agent over the top” integration - While its obvious to most you’d never send sensitive bank or customer information directly to a model like ChatGPT often “AI Agents” are SaaS wrappers over LLMs - This opens them to new security vulnerabilities like prompt injection attacks - Instead AI Agents should be tightly contained within an existing, audited, 3rd party approved vendor platform and only have access to data within that 2. Standard Operating Procedures (SOPs) are the best training material - They provide a baseline for backtesting and evals - If an Agent is trained on and follows that procedure you can then baseline performance against human agents and the AI Agents over time 3. Using AI Agents to power first and second lines of defense - In the first line, Agents accelerate compliance officer’s reviews, reducing manual work - In the second line, they provide a consistent review of decisions and maintain a higher consistency than human reviewers (!) 4. Putting AI Agents in a glass box makes them observable - One worry financial institutions have is explainability, under SR 11-7 models have to be explainable - The solution is to ensure every data element accessed, every click, every thinking token is made available for audit, and rationale is always presented 5. Starting in co-pilot before moving to autopilot - In co-pilot mode an Agent does foundational data gathering and creates recommendations while humans are accountable for every individual decision  - Once an institution has confidence in that agents performance they can move to auto decisioning the lower-risk alerts.

The AI Security Reference Architectures paper provides a structured way to think about risks in three common application patterns: chatbots, retrieval augmented generation (RAG) and agents. These patterns imply that security must be part of the design process from the start, not added later as it's impossible to achieve the results otherwise. What the paper outlines • Three core architectures, each with distinct attack surfaces • Design principles across inputs, models, storage, tool use, and outputs • The importance of testing and guardrails before and after fine tuning, since tuning can weaken alignment Why this matters • By 2027, one in four organizations is expected to rely on chatbots as their primary customer service channel • Retrieval augmented generation connects models to enterprise data, which also connects them to enterprise risk • Agents can plan and act, which means a single error can cascade into business processes There is an old saying, measure twice and cut once. In AI security, this means validating at design, deployment, and runtime. Key risks and practices • Chatbots: prompt injection, data exfiltration, off topic output. Mitigate with input and output filtering, rate limits, secure prompts, and ongoing validation • Retrieval augmented generation: poisoned data, indirect injection, leakage from vector databases. Mitigate with document scanning, integrity checks, scoped prompts, encryption, and parameterized queries • Agents: tool misuse, privilege escalation, memory tampering. Mitigate with least privilege, delegated authorization, isolation, and human in the loop for sensitive actions Who should act • Security architects embedding guardrails into design • Machine learning and platform teams managing pipelines • Product leaders deploying LLM features • Governance leaders ensuring safe adoption Action items • Use these reference architectures as a baseline checklist for new AI systems • Build guardrails into development pipelines rather than waiting until production • Red team each pattern before scaling into critical workflows • Assign clear ownership for data security, model behavior, and tool governance • Review and update these patterns regularly as threats evolve