Detection Engineering Best Practices

OT Detection Use Cases for your OT SOC When it comes to building an OT SOC, there’s a big misconception: many assume success is about collecting every log or integrating every system In reality, the key is focusing on operationally meaningful visibility — the detections that actually help you understand what’s happening inside your control network >> In industrial environments, context defines everything > The same Modbus write command could mean two very different things: > a maintenance engineer performing a scheduled update — or an attacker changing control logic. > Without context, both look identical in your SIEM. >> An OT SOC must speak the language of process, assets, and operations, not just alerts. It should tell you when something changes, who initiated it, and whether it threatens safety, reliability, or integrity >> Below are 10 detection use cases I always recommend as a starting point. They’re mapped to MITRE ATT&CK for ICS and NCA OTCC, but more importantly, they’re grounded in what actually happens inside real plants and industrial networks 1. Unauthorized PLC Programming Detect logic or configuration changes outside scheduled maintenance windows 2. ICS Protocol in IT Zone Flag Modbus, DNP3, or BACnet traffic on IT networks — strong evidence of segmentation drift or misconfiguration 3. PLC Stop or Mode Change Command Detect STOP or PROGRAM mode changes — an event that can halt production and indicate malicious control 4. Remote Access to HMI from Unapproved Source Identify RDP, VNC, or TeamViewer sessions from IT zones targeting OT HMIs — a common lateral movement path 5. New Device in Control VLAN Catch unauthorized or rogue devices joining deterministic control networks where new assets should rarely appear 6. PLC Firmware Downgrade or Version Change Detect unauthorized firmware rollbacks — a subtle but serious method of tampering or hiding malicious code 7. OPC UA Anonymous Session Identify untrusted or anonymous OPC UA sessions that bypass normal authentication or encryption 8. Engineering Software on Non-Engineering Host Detect the execution of TIA Portal, Control Builder, or similar tools on unauthorized systems — often a sign of credential misuse or insider activity 9. PLC Configuration Upload Monitor FTP/TFTP uploads to PLCs — an activity that could replace control logic or inject malicious configuration 10. Abnormal HMI Behavior Spot rapid screen changes, tag edits, or command spamming from operators — signs of misuse, automation, or compromise They aren’t just security detections — they’re process integrity safeguards. Each one gives the SOC visibility into the exact actions adversaries use during real OT incidents — often before physical impact occurs When combined with contextual data (authorized engineers, maintenance schedules, device baselines) and network telemetry , these detections evolve from simple alerts into actionable operational intelligence #OTSOC #OTsecurity #ICSsecurity

Still trying to manage your ever-increasing alert flow by hiring more analysts? That’s much like adding buckets to deal with a leaking roof. Invest in detection engineering and automation engineering to reduce the alert flow and prevent alert fatigue and unhappy analysts. Here are some best practices: - Apply an automation-first strategy: handle and/or accelerate all alerts through automation - Continuously tune and optimize detection rules - Let analysts and detection / automation engineers work closely together to increase the effectiveness of engineering efforts - Establish metrics for rule quality to identify candidates for tuning and automation - Test against defined quality criteria before putting any detection rules live - Increase the fidelity of your rules by alerting on more specific criteria - Aggregate and analyse batches of noisy alerts daily or weekly, instead of handling them individually in real-time - Consider your ideal ratio between analysts and engineers. Start out with 50-50, then decide what would best suit your needs - Make risk-based decisions on added value of rules compared to time investment, and drop time-consuming rules with little added value if they cannot be tuned properly This is by no means an easy thing to do. But by focussing on engineering and detection quality, you can transition to a state where you control of the alert flow instead of the other way around, so that analysts can focus on the alerts that truly matter. #soc #securityoperations #securityanalysis #detectionengineering #automationfirst

🌍International Guidance for Enhanced Cybersecurity: Best Practices for Event Logging and Threat Detection🌍 The Australian Government's Australian Cyber Security Centre (ACSC), in collaboration with global partners like the #NSA, #CISA, the UK's #NCSC, and agencies from Canada, New Zealand, Japan, South Korea, Singapore, and the Netherlands, has released a comprehensive report on best practices for event logging and threat detection. 🚀The report defines a baseline for event logging best practices and emphasizes the importance of robust event logging to enhance security and resilience in the face of evolving cyber threats. Why Event Logging Matters: Event logging isn't just about keeping records—it's about empowering organizations to detect, respond to, and mitigate cyber threats more effectively. The guidance provided in this report aims to bolster an organization’s resilience by enhancing network visibility and enabling timely detection of malicious activities. 🔍 Key Highlights: 🔹Enterprise-Approved Event Logging Policy: Develop and implement a consistent logging policy across all environments to enhance the detection of malicious activities and support incident response. 🔹Centralized Log Collection and Correlation: Utilize a centralized logging facility to aggregate logs, making detecting anomalies and potential security breaches easier. 🔹Secure Storage and Event Log Integrity: Implement secure mechanisms for storing and transporting event logs to prevent unauthorized access, modification, or deletion. 🔹Detection Strategy for Relevant Threats: Leverage behavioral analytics and SIEM tools to detect advanced threats, including "Living off the Land" (LOTL) techniques used by sophisticated threat actors. 📊 Use Case: Detecting "Living Off the Land" Techniques: One highlighted use case involves detecting LOTL techniques, where attackers use legitimate tools available in the environment to carry out malicious activities. The report showcases how the Volt Typhoon group leveraged LOTL techniques, such as using PowerShell and other native tools on compromised Windows systems, to evade detection and conduct espionage. Effective event logging, including process creation events and command-line auditing, was crucial in identifying these activities as abnormal compared to regular operations. Couple this report with the CISA Zero Trust Maturity Model (ZTMM): The report's best practices align with CISA's ZTMM's Visibility and Analytics capability. By following these publications, organizations can progress along their maturity path toward optimal dynamic monitoring and advanced analysis. (Full disclosure: I was co-author of CISA's ZTMM) 💪Implementing these best practices from the Australian Signals Directorate & others is critical to achieving comprehensive visibility and security, aligning with global cybersecurity frameworks. #cybersecurity #zerotrust #digitaltransformation #technology #cloudcomputing #informationsecurity

Dear #DataEngineers, No matter how confident you are in your SQL queries or ETL pipelines, never assume data correctness without validation. ETL is more than just moving data—it’s about ensuring accuracy, completeness, and reliability. That’s why validation should be a mandatory step, making it ETLV (Extract, Transform, Load & Validate). Here are 20 essential data validation checks every data engineer should implement (not all pipeline require all of these, but should follow a checklist like this): 1. Record Count Match – Ensure the number of records in the source and target are the same. 2. Duplicate Check – Identify and remove unintended duplicate records. 3. Null Value Check – Ensure key fields are not missing values, even if counts match. 4. Mandatory Field Validation – Confirm required columns have valid entries. 5. Data Type Consistency – Prevent type mismatches across different systems. 6. Transformation Accuracy – Validate that applied transformations produce expected results. 7. Business Rule Compliance – Ensure data meets predefined business logic and constraints. 8. Aggregate Verification – Validate sum, average, and other computed metrics. 9. Data Truncation & Rounding – Ensure no data is lost due to incorrect truncation or rounding. 10. Encoding Consistency – Prevent issues caused by different character encodings. 11. Schema Drift Detection – Identify unexpected changes in column structure or data types. 12. Referential Integrity Checks – Ensure foreign keys match primary keys across tables. 13. Threshold-Based Anomaly Detection – Flag unexpected spikes or drops in data volume or values. 14. Latency & Freshness Validation – Confirm that data is arriving on time and isn’t stale. 15. Audit Trail & Lineage Tracking – Maintain logs to track data transformations for traceability. 16. Outlier & Distribution Analysis – Identify values that deviate from expected statistical patterns. 17. Historical Trend Comparison – Compare new data against past trends to catch anomalies. 18. Metadata Validation – Ensure timestamps, IDs, and source tags are correct and complete. 19. Error Logging & Handling – Capture and analyze failed records instead of silently dropping them. 20. Performance Validation – Ensure queries and transformations are optimized to prevent bottlenecks. Data validation isn’t just a step—it’s what makes your data trustworthy. What other checks do you use? Drop them in the comments! #ETL #DataEngineering #SQL #DataValidation #BigData #DataQuality #DataGovernance

In addition to threat modeling, we need detection modeling. This is a core part of threat informed defense. Starting with known threats (whether its ATT&CK or bespoke scenarios internally) is a great start, but theres still a lot of work & nuance to get this to a finished analytic or detection. We want to know things like: - how threats specifically manifest in OUR environment - build detections that actually work for OUR tech stack and processes Really cool release from the "Summiting the Pyramid" framework from Center for Threat-Informed Defense to help us bridge this gap: Detection Decomposition Diagrams (D3). These D3 visuals give defenders a view across multiple implementations of a technique to identify analytic and event observables for robust detections. D3 visuals include benign and malicious implementations of the technique. Observables which span across multiple implementations provide higher robustness; that is, resistance to adversary evasion over time. Other observables may be used for better accuracy rates. This coincides with the OpenTide paper released by Amine Besson (Threat Informed Detection Modeling and Engineering as-Code) which is an absolute gold mine of how & why to do this in practice. These approaches connect abstract capabilities to concrete detection opportunities. The real power comes from combining threat modeling WITH detection modeling. This concept is not necessarily new & is the product of a lot of great work already done by folks like Andrew VanVleet as well. Its a whole other level when you can combine TTPs with prevalence, choke point and actionability to the texture of which all detections are written (logs!) with information like core/tiered observables. This is how you create robust & accurate detections. Check out the great work by these folks below: ⛰️ Summit the Pyramid v2 Release: Center for Threat-Informed Defense https://lnkd.in/eb9Cb8Q5 🌊 OpenTide: https://lnkd.in/emcX4rKk 🧱 Improving Threat Identification with Detection Data Models: https://lnkd.in/eZ5HGw-T

Dear SOC Heroes, To detect and respond to any attack correctly, you must make a threat modeling to your business to understand all attacks and identify their attack surface and impact, then you should map each attack to an incident response framework that your organization follows. A well-structured approach that you follow, will enable you to manage and mitigate the impact of any attack. For example, let's map a data exfiltration attack to the NIST incident response framework. 1. Preparation - Establish Baselines: Understand normal data flows and behaviors within your network. - Implement Monitoring Tools: Deploy and configure SIEM, DLP, and IDS/IPS. - Develop Incident Response Plans: Have clear procedures and roles defined for responding to data exfiltration incidents. 2. Detection - Monitor Network Traffic: Look for unusual data transfer volumes, particularly to external IP addresses. - Analyze Logs: Check logs from firewalls, proxies, and network devices for anomalies. - Utilize Behavioral Analytics: Use tools to detect deviations from normal user and system behavior. - Build SIEM Use-Cases: Configure alerts for potential exfiltration activities, such as large data transfers or access to sensitive files. 3. Identification - Correlate Events: Use SIEM to correlate alerts and logs from different sources to identify patterns. - Validate Alerts: Confirm that alerts are not false positives by cross-referencing with known baselines and activities. - Identify Data Sources: Determine which data was accessed and potentially exfiltrated. 4. Containment - Isolate Affected Systems: Disconnect compromised systems from the network to prevent further data loss. - Block Malicious Traffic: Implement firewall rules to block data exfiltration channels. - Reset Credentials: Change passwords and revoke access for compromised accounts. 5. Eradication - Remove Malware: Conduct a thorough scan and clean-up of affected systems to remove any malicious software. - Patch Vulnerabilities: Apply patches and updates to fix exploited vulnerabilities. - Secure Configurations: Ensure systems and network configurations follow best security practices. 6. Recovery - Restore Systems: Rebuild or restore systems from clean backups. - Monitor for Recurrence: Closely watch the affected systems for signs of recurring issues. - Communicate: Inform clients/stakeholders and possibly affected individuals as required by law and policy. 7. Post-Incident Analysis - Conduct a Root Cause Analysis: Determine and document how the exfiltration occurred and why it wasn't detected earlier. - Review and Improve: Update security policies, incident response plans, and monitoring tools based on lessons learned. You must test this procedure/approach with your SOC team to make sure it's well understood and effective and will be followed once you are this type of attack. #SOC #IR #NIST_IR #Data_exfilteration #Cybersecurity

#EDR vs #MDR vs #XDR plus what “good” response looks like in practice ▪︎ EDR (Endpoint Detection & Response) = endpoint #telemetry + detection + containment on #devices (kill process, isolate host, quarantine). Best when the fight is on the endpoint (#ransomware/fileless/zero-day behaviors). ▪︎ MDR (Managed Detection & Response) = EDR (and more) operated by humans. Best when you need 24/7 triage + threat hunting + expert-driven decisions and reduced alert fatigue. ▪︎ XDR (Extended Detection & Response) = correlation across endpoint, identity, email, cloud, network with unified investigations and orchestrated response. Best for attack chains that pivot across domains. ▪︎ What the #SOC playbooks reinforce (repeatable, “always true” takeaways): ○ #Preparation is the multiplier: offline backups + restore testing; centralized logging (EDR/SIEM/cloud logs); hardening (patching, #WAF, #MFA); access governance (least privilege, segmentation); user awareness. ○ #Detect fast, then verify: confirm indicators (e.g., #encryption/ransom note; abnormal uploads; impossible travel/MFA prompts; WAF SQLi/RCE; unusual vendor activity; USB-origin execution; #DDoS surge patterns). ○ #Containment comes before cleanup: isolate hosts/accounts/sessions, block IPs/domains, stop lateral movement, preserve evidence where needed. ○ #Eradication is root-cause driven: remove persistence, patch exploited vectors (RDP/SMB/app vulns), revoke tokens/OAuth, validate software integrity, and update detections to prevent recurrence. ○ #Recovery requires proof of cleanliness: restore only from verified backups; monitor post-recovery for reinfection/reuse; reset credentials where impacted. ▪︎ Close the loop: document timeline + impact, tune SIEM/EDR/XDR rules, and meet regulatory/contractual notifications (e.g., #GDPR/PDPA). ▪︎ Operationalize with targets (examples in the playbooks): detection in minutes, containment in tens of minutes, and defined post-incident monitoring windows, because response without metrics is not manageable. #CyberSecurity #SOC #IncidentResponse #EDR #MDR #XDR #SIEM #DFIR #ThreatHunting Source: https://lnkd.in/d2J33wfk

Everyone talks about what you should do before you push to production, but software engineers, what about after? The job doesn’t end once you’ve deployed; you must monitor, log, and alert. ♠ 1. Logging Logging captures and records events, activities, and data generated by your system, applications, or services. This includes everything from user interactions to system errors. ◄Why do you need it? To capture crucial data that provides insight into system health user behavior and aids in debugging. ◄Best practices • Structured Logging: Use a consistent format for your logs to make it easier to parse and analyze. • Log Levels: Utilize different log levels (info, warning, error, etc.) to differentiate the importance and urgency of logged events. • Sensitive Data: Avoid logging sensitive information like passwords or personal data to maintain security and privacy. • Retention Policy: Implement a log retention policy to manage the storage of logs, ensuring old logs are archived or deleted as needed. ♠ 2.Monitoring It’s observing and analyzing system performance, behavior, and health using the data collected from logs. It involves tracking key metrics and generating insights from real-time and historical data. ◄Why do you need it? To detect real-time issues, monitor trends, and ensure your system runs smoothly. ◄Best practices: • Dashboard Visualization: Use monitoring tools that offer dashboards to present data in a clear, human-readable format, making it easier to spot trends and issues. • Key Metrics: Monitor critical metrics like response times, error rates, CPU/memory usage, and request throughput to ensure overall system health. • Automated Analysis: Implement automated systems to analyze logs and metrics, alerting you to potential issues without constant manual checks. 3. Alerting It’s all about notifying relevant stakeholders when certain conditions or thresholds are met within the monitored system. This ensures that critical issues are addressed as soon as they arise. ◄Why do you need it? To promptly address critical issues like high latency or system failures, preventing downtime. ◄Best practices: •Thresholds: Set clear thresholds for alerts based on what’s acceptable for your system’s performance. For instance, set an alert if latency exceeds 500ms or if error rates rise above 2%. • Alert Fatigue: To prevent desensitization, avoid setting too many alerts. Focus on the most critical metrics to ensure that alerts are meaningful and actionable. • Escalation Policies: Define an escalation path for alerts so that if an issue isn’t resolved promptly, it is automatically escalated to higher levels of support. Without these 3, no one would know there’s a problem until the user calls you themselves. 

Over the past few weeks, I’ve shared a series of posts on the foundations of detection engineering, highlighting the critical role it plays in building a strong SOC. I’ve discussed how solid, purpose-driven detection engineering practices and effective threat research are the backbone of any proactive detection strategy. But, once this foundation is in place, the question becomes: What’s the next step? For me, the answer lies in maturing detection engineering into a process that seamlessly integrates data science, automation, and collaboration across key SOC functions. Here’s how I did it: Instead of having data scientists work with raw telemetry (which creates more noise than signal), I shifted them downstream to work with enriched, context-aware detection outputs and pulled this all together into something I call, The Detection Engineering Escalation & Recommendation (DEER) Framework. What does the framework do in a nutshell? 1. Creates synergy between the threat research team (intelligence backbone), DE team (signal creators), threat hunting team (pattern finders), and data science (insight amplifiers). 2. Leverages data science where it matters most for the SOC with things like: Natural Language Processing (NLP) for entity extractions and embeddings, Learning-to-Rank (LTR) for alert prioritization, LLMs for analysis, escalation & tuning, and clustering for peripheral context. Here’s what I saw happen after implementing this framework: ✓ 𝗕𝗲𝘁𝘁𝗲𝗿 𝗼𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆: With a constant feedback loop and a process for these functions to work together, this reduced the workload across the team and gave them the time to focus on what matters most with our threat priorities. ✓ 𝗘𝗻𝗵𝗮𝗻𝗰𝗲𝗱 𝗗𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻 𝗖𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝗶𝗲𝘀: Behavioral-based detections + NLP and Alert Clustering have provided context-rich alerts, improving the accuracy of detections. ✓ 𝗥𝗲𝗱𝘂𝗰𝗲𝗱 𝗔𝗹𝗲𝗿𝘁 𝗙𝗮𝘁𝗶𝗴𝘂𝗲: Automated rule tuning + real-time feedback with the DEER pipeline = more time for your SOC analysts to focus on genuine threats.    ✓ 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗜𝗺𝗽𝗿𝗼𝘃𝗲𝗺𝗲𝗻𝘁: Embedding data science into the DE process brings automation that will ensure your detections can evolve as quickly as new threats do.  If your detection strategy is starting to feel a bit outdated and you’re considering integrating data science into your practice - this approach might be worth exploring. Curious to hear from others, how are you thinking about the integration of data science into your SOC? You can grab my exact framework, and get more specifics on how we implemented this in my latest blog here: https://lnkd.in/gVYtMJwY

Security data issues rarely begin at the SIEM. They start upstream when no one defines what good telemetry looks like, who owns it, or how it should evolve as threats and infrastructure change. When governance is missing, monitoring agents fail silently, log formats shift without warning, and critical fields vanish, leaving detection teams debugging broken data pipelines instead of stopping real threats. Real-world example? Here are three: ▪ Microsoft lost over two weeks of security logs due to a silent failure in its telemetry agents, leaving customers blind to potential threats during that window. ▪ In a major retail breach, authentication logs were either misconfigured or ignored, and failed login attempts went unmonitored. The attackers used this gap to move laterally into payment systems and steal credit and debit card numbers of millions of customers. ▪ OpenAI overloaded internal infrastructure, causing a widespread outage when resource usage wasn’t properly staged or governed. So what does effective telemetry governance actually involve? ▪ Define Telemetry Expectations Upfront Set clear, use-case-driven standards for what “good” telemetry looks like down to required fields, formats, and frequency. Align logs to specific detection or compliance needs, so that critical fields like device_id, user_agent, or geo_ip are treated as non-negotiable, not best-effort. ▪ Establish Ownership Across the Pipeline Governance starts with clarity on who owns what. Define responsibilities for source selection, enrichment, normalization, validation, and routing ensuring each team knows their role in maintaining telemetry integrity. ▪ Monitor for Drift, Not Just Volume Telemetry should be continuously validated in-flight. Use telemetry pipeline solutions to catch missing fields, schema shifts, malformed events, and time drift before they impact detection. ▪ Align Detection Logic with Telemetry Evolution Threats change, and so do detection rules. Governance ensures telemetry keeps pace by creating structured feedback loops between detection engineers and those managing telemetry. 👉Data Governance means building operational habits into how telemetry is defined, owned and maintained turning telemetry from something you hope is working into something you know is working. #TelemetryGovernance #SecurityData #DetectionEngineering #SIEM #Observability #DataOwnership #SecOps #DataTrust #CyberSecurity #SOC #SecOps #ThreatDetection #Telemetry #DataStrategy #DataQuality #OptimizeLogs #LogReduction #SecurityEfficiency #SIEMOptimization #AlertFatigue #TelemetryPipeline