Understanding AI Systems

I recently received an email titled “An 18-year-old’s dilemma: Too late to contribute to AI?” Its author, who gave me permission to share this, is preparing for college. He is worried that by the time he graduates, AI will be so good there’s no meaningful work left for him to do to contribute to humanity, and he will just live on Universal Basic Income (UBI). I wrote back to reassure him that there will still be plenty of work he can do for decades hence, and encouraged him to work hard and learn to build with AI. But this conversation struck me as an example of how harmful hype about AI is. Yes, AI is amazingly intelligent, and I’m thrilled to be using it every day to build things I couldn’t have built a year ago. At the same time, AI is still incredibly dumb, and I would not trust a frontier LLM by itself to prioritize my calendar, carry out resumé screening, or choose what to order for lunch — tasks that businesses routinely ask junior personnel to do. Yes, we can build AI software to do these tasks. For example, after a lot of customization work, one of my teams now has a decent AI resumé screening assistant. But the point is it took a lot of customization. Even though LLMs can handle a much more general set of tasks than previous iterations of AI technology, compared to what humans can do, they are still highly specialized. They’re much better at working with text than other modalities, still require lots of custom engineering to get it the right context for a particular application, and we have few tools — and only inefficient ones — for getting our systems to learn from feedback and repeated exposure to a specific task (such as screening resumés for a particular role). AI has stark limitations, and despite rapid improvements, it will remain limited compared to humans for a long time. AI is amazing, but it has unfortunately been hyped up to be even more amazing than it is. A pernicious aspect of hype is that it often contains an element of truth, but not to the degree of the hype. This makes it difficult for nontechnical people to discern where the truth really is. Modern AI is a general purpose technology that is enabling many applications, but AI that can do any intellectual tasks that a human can (a popular definition for AGI) is still decades away or longer. This nuanced message that AI is general, but not that general, often is lost in the noise of today's media environment. [Truncated for length. Full text:  https://lnkd.in/gAuQcZ8M ] 

AI is rapidly moving from passive text generators to active decision-makers. To understand where things are headed, it’s important to trace the stages of this evolution. 1. 𝗟𝗟𝗠𝘀: 𝗧𝗵𝗲 𝗘𝗿𝗮 𝗼𝗳 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗙𝗹𝘂𝗲𝗻𝗰𝘆 Large Language Models (LLMs) like GPT-3 and GPT-4 excel at generating human-like text by predicting the next word in a sequence. They can produce coherent and contextually appropriate responses—but their capabilities end there. They don’t retain memory, they don’t take actions, and they don’t understand goals. They are reactive, not proactive. 2. 𝗥𝗔𝗚: 𝗧𝗵𝗲 𝗔𝗴𝗲 𝗼𝗳 𝗖𝗼𝗻𝘁𝗲𝘅𝘁-𝗔𝘄𝗮𝗿𝗲 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 Retrieval-Augmented Generation (RAG) brought a major upgrade by integrating LLMs with external knowledge sources like vector databases or document stores. Now the model could retrieve relevant context and generate more accurate and personalized responses based on that information. This stage introduced the idea of 𝗱𝘆𝗻𝗮𝗺𝗶𝗰 𝗸𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗮𝗰𝗰𝗲𝘀𝘀, but still required orchestration. The system didn’t plan or act—it responded with more relevance. 3. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜: 𝗧𝗼𝘄𝗮𝗿𝗱 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 Agentic AI is a fundamentally different paradigm. Here, systems are built to perceive, reason, and act toward goals—often without constant human prompting. An Agentic system includes: • 𝗠𝗲𝗺𝗼𝗿𝘆: to retain and recall information over time. • 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴: to decide what actions to take and in what order. • 𝗧𝗼𝗼𝗹 𝗨𝘀𝗲: to interact with APIs, databases, code, or software systems. • 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝘆: to loop through perception, decision, and action—iteratively improving performance.    Instead of a single model generating content, we now orchestrate 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝗮𝗴𝗲𝗻𝘁𝘀, each responsible for specific tasks, coordinated by a central controller or planner. This is the architecture behind emerging use cases like autonomous coding assistants, intelligent workflow bots, and AI co-pilots that can operate entire systems. 𝗧𝗵𝗲 𝗦𝗵𝗶𝗳𝘁 𝗶𝗻 𝗧𝗵𝗶𝗻𝗸𝗶𝗻𝗴 We’re no longer designing prompts. We’re designing 𝗺𝗼𝗱𝘂𝗹𝗮𝗿, 𝗴𝗼𝗮𝗹-𝗱𝗿𝗶𝘃𝗲𝗻 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 capable of interacting with the real world. This evolution—LLM → RAG → Agentic AI—marks the transition from 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝘂𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 to 𝗴𝗼𝗮𝗹-𝗱𝗿𝗶𝘃𝗲𝗻 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲.

Microsoft just released a 35-page report on medical AI - and it’s a reality check for healthcare. The paper, “The Illusion of Readiness”, tested six of the most popular models (OpenAI, Gemini, etc)… across six multimodal medical benchmarks. And the verdict? The models scored high on medical exams. But they’re not even close to being real-world ready. Here’s what the stress tests revealed: ▶ 1. Shortcut learning Models often answered correctly even when key information, like medical images, was removed. They weren’t reasoning - they were exploiting statistical shortcuts. That means benchmark wins may hide shallow understanding. ▶ 2. Fragile under small changes Making small tweaks caused big swings in predictions. This fragility shows how unreliable model reasoning becomes under stress. In visual substitution tests, accuracy dropped from 83% to 52% when images were swapped - exposing shallow visual–answer pairings. ▶ 3. Fabricated reasoning Models produced confident, step-by-step medical explanations - but many were medically unsound… or entirely fabricated. Convincing to the eye, dangerous in practice. And more importantly, healthcare isn’t a multiple-choice exam. It’s uncertainty, incomplete data, and high stakes. So Microsoft’s team calls for new standards: - Stress tests that expose fragility - Clinician-guided guidelines that profile benchmarks - Evaluation of robustness and trustworthiness - not just leaderboard scores The takeaway is simple: Medical AI may ace tests today. But until it proves reliable under stress, it’s not ready for the clinic. When do you think popular LLMs will be clinic-ready? #entrepreneurship #healthtech #AI

"With recent advancements in artificial intelligence—particularly, powerful generative models—private and public sector actors have heralded the benefits of incorporating AI more prominently into our daily lives. Frequently cited benefits include increased productivity, efficiency, and personalization. However, the harm caused by AI remains to be more fully understood. As a result of wider AI deployment and use, the number of AI harm incidents has surged in recent years, suggesting that current approaches to harm prevention may be falling short. This report argues that this is due to a limited understanding of how AI risks materialize in practice. Leveraging AI incident reports from the AI Incident Database, it analyzes how AI deployment results in harm and identifies six key mechanisms that describe this process Intentional Harm ● Harm by design ● AI misuse ● Attacks on AI systems Unintentional Harm ● AI failures ● Failures of human oversight ● Integration harm A review of AI incidents associated with these mechanisms leads to several key takeaways that should inform AI governance approaches in the future. A one-size-fits-all approach to harm prevention will fall short. This report illustrates the diverse pathways to AI harm and the wide range of actors involved. Effective mitigation requires an equally diverse response strategy that includes sociotechnical approaches. Adopting model-based approaches alone could especially neglect integration harms and failures of human oversight. To date, risk of harm correlates only weakly with model capabilities. This report illustrates many instances of harm that implicate single-purpose AI systems. Yet many policy approaches use broad model capabilities, often proxied by computing power, as a predictor for the propensity to do harm. This fails to mitigate the significant risk associated with the irresponsible design, development, and deployment of less powerful AI systems. Tracking AI incidents offers invaluable insights into real AI risks and helps build response capacity. Technical innovation, experimentation with new use cases, and novel attack strategies will result in new AI harm incidents in the future. Keeping pace with these developments requires rapid adaptation and agile responses. Comprehensive AI incident reporting allows for learning and adaptation at an accelerated pace, enabling improved mitigation strategies and identification of novel AI risks as they emerge. Incident reporting must be recognized as a critical policy tool to address AI risks." By Mia Hoffmann at Center for Security and Emerging Technology (CSET)

A Manhattan federal judge has delivered a really important ruling on artificial intelligence and legal practice - can you claim legal privilege over AI generated documents? It’s a potential major blind spot for organisations - and a huge responsibility for in-house lawyers to explain the issue to their non-legal colleagues. On 10 February, U.S. District Judge Jed Rakoff ruled in USA v. Heppner that a criminal defendant could not claim attorney-client privilege over documents he had himself prepared using an AI service and then subsequently sent to his lawyers. Bradley Heppner, former chairman of GWG Holdings, faces fraud charges over an alleged $150 million scheme, with trial set for April. But the privilege ruling carries significance beyond any single case. The reasoning rests on a principle that long predates artificial intelligence. Privilege protects confidential communications between lawyer and client made for the purpose of legal advice. It does not automatically attach to materials a client creates independently simply because those materials are later forwarded to counsel. What matters is how the document came to exist, not its destination. What AI changes is the scale of the problem. Generative AI tools now allow any executive to produce polished case narratives, issue summaries, and chronologies that resemble legal work product, all without a lawyer’s involvement. The natural instinct is to assume that once these materials are emailed to counsel, they enter the protected sphere. Judge Rakoff’s ruling suggests otherwise - the court’s focus is on what the document is and how it came to exist, not on the fact that it was subsequently routed to a lawyer. This matters because AI is rapidly becoming the default tool through which businesspeople process complex situations. An executive facing a regulatory investigation who uses a chatbot to organise the facts and draft a summary for their lawyer may be creating discoverable material that sits entirely outside the privileged relationship. Judge Rakoff also noted that the AI-generated materials could prove “problematic” if used at trial. Even where privilege is not the issue, AI-authored documents create genuine evidential difficulties - questions of authorship, accuracy, hearsay characterisation, and the optics of presenting AI-mediated narratives as though they were direct recollection. If you want AI-assisted materials to have any chance of privilege protection, “Client-produced and then forwarded to counsel” is the weak fact pattern, and after this ruling, in the US at least, it may be no fact pattern at all.

BREAKING! The FDA just released this draft guidance, titled Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations, that aims to provide industry and FDA staff with a Total Product Life Cycle (TPLC) approach for developing, validating, and maintaining AI-enabled medical devices. The guidance is important even in its draft stage in providing more detailed, AI-specific instructions on what regulators expect in marketing submissions; and how developers can control AI bias. What’s new in it? 1) It requests clear explanations of how and why AI is used within the device. 2) It requires sponsors to provide adequate instructions, warnings, and limitations so that users understand the model’s outputs and scope (e.g., whether further tests or clinical judgment are needed). 3) Encourages sponsors to follow standard risk-management procedures; and stresses that misunderstanding or incorrect interpretation of the AI’s output is a major risk factor. 4) Recommends analyzing performance across subgroups to detect potential AI bias (e.g., different performance in underrepresented demographics). 5) Recommends robust testing (e.g., sensitivity, specificity, AUC, PPV/NPV) on datasets that match the intended clinical conditions. 6) Recognizes that AI performance may drift (e.g., as clinical practice changes), therefore sponsors are advised to maintain ongoing monitoring, identify performance deterioration, and enact timely mitigations. 7) Discusses AI-specific security threats (e.g., data poisoning, model inversion/stealing, adversarial inputs) and encourages sponsors to adopt threat modeling and testing (fuzz testing, penetration testing). 8) And proposed for public-facing FDA summaries (e.g., 510(k) Summaries, De Novo decision summaries) to foster user trust and better understanding of the model’s capabilities and limits.

🚨 AI Privacy Risks & Mitigations Large Language Models (LLMs), by Isabel Barberá, is the 107-page report about AI & Privacy you were waiting for! [Bookmark & share below]. Topics covered: - Background "This section introduces Large Language Models, how they work, and their common applications. It also discusses performance evaluation measures, helping readers understand the foundational aspects of LLM systems." - Data Flow and Associated Privacy Risks in LLM Systems "Here, we explore how privacy risks emerge across different LLM service models, emphasizing the importance of understanding data flows throughout the AI lifecycle. This section also identifies risks and mitigations and examines roles and responsibilities under the AI Act and the GDPR." - Data Protection and Privacy Risk Assessment: Risk Identification "This section outlines criteria for identifying risks and provides examples of privacy risks specific to LLM systems. Developers and users can use this section as a starting point for identifying risks in their own systems." - Data Protection and Privacy Risk Assessment: Risk Estimation & Evaluation "Guidance on how to analyse, classify and assess privacy risks is provided here, with criteria for evaluating both the probability and severity of risks. This section explains how to derive a final risk evaluation to prioritize mitigation efforts effectively." - Data Protection and Privacy Risk Control "This section details risk treatment strategies, offering practical mitigation measures for common privacy risks in LLM systems. It also discusses residual risk acceptance and the iterative nature of risk management in AI systems." - Residual Risk Evaluation "Evaluating residual risks after mitigation is essential to ensure risks fall within acceptable thresholds and do not require further action. This section outlines how residual risks are evaluated to determine whether additional mitigation is needed or if the model or LLM system is ready for deployment." - Review & Monitor "This section covers the importance of reviewing risk management activities and maintaining a risk register. It also highlights the importance of continuous monitoring to detect emerging risks, assess real-world impact, and refine mitigation strategies." - Examples of LLM Systems’ Risk Assessments "Three detailed use cases are provided to demonstrate the application of the risk management framework in real-world scenarios. These examples illustrate how risks can be identified, assessed, and mitigated across various contexts." - Reference to Tools, Methodologies, Benchmarks, and Guidance "The final section compiles tools, evaluation metrics, benchmarks, methodologies, and standards to support developers and users in managing risks and evaluating the performance of LLM systems." 👉 Download it below. 👉 NEVER MISS my AI governance updates: join my newsletter's 58,500+ subscribers (below). #AI #AIGovernance #Privacy #DataProtection #AIRegulation #EDPB

The next $1T AI data center buildout won’t be about land or servers. It will be defined by heat. Eaton’s $9.5B acquisition of Boyd Thermal signals a shift: power alone no longer drives AI campus economics chip-level cooling does. Why it matters: - Chip-to-Grid Control: Eaton now manages everything from transformers to GPU cold plates, unifying the energy equation. - Liquid Cooling Is Mandatory: AI racks reach 80–500 kW air can’t handle it. Boyd’s leak-free, hyperscaler-grade cooling is a rare strategic moat. - Investor Angle: Eaton becomes a platform for repeatable, de-risked AI infrastructure exposure without direct chip cyclicality. Integration success, Boyd’s roadmap vs. immersion cooling, and deployment across 50–500 MW campuses. These will show who really controls AI energy efficiency. The AI buildout isn’t just about servers or land It’s about who owns the interface between power in and heat out. Eaton just positioned itself at the front of the line. #datacenters

Everyone's talking about LLMs. I went a different direction 🧠 While everyone's building RAG systems with document chunking and vector search, I got curious about something else after Prof Alsayed Algergawy and his assistant Vishvapalsinhji Parmar's Knowledge Graphs seminar. What if the problem isn't just retrieval - but how we structure knowledge itself? 🤔 Traditional RAG's limitation: Chop documents into chunks, embed them, hope semantic search finds the right pieces. But what happens when you need to connect information across chunks? Or when relationships matter more than text similarity? 📄➡️❓ My approach: Instead of chunking, I built a structured knowledge graph from Yelp data (220K+ entities, 555K+ relationships) and trained Graph Neural Networks to reason through connections. 🕸️ The attached visualization shows exactly why this works - see how information naturally exists as interconnected webs, not isolated chunks. 👇🏻 The difference in action: ⚡ Traditional RAG: "Find similar text about Italian restaurants" 🔍 My system: "Traverse user→review→business→category→location→hours and explain why" 🗺️ Result: 94% AUC-ROC performance with explainable reasoning paths. Ask "Find family-friendly Italian restaurants in Philadelphia open Sunday" and get answers that show exactly how the AI connected reviews mentioning kids, atmosphere ratings, location data, and business hours. 🎯 Why this matters: While others optimize chunking strategies, maybe we should question whether chunking is the right approach at all. Sometimes the breakthrough isn't better embeddings - it's fundamentally rethinking how we represent knowledge. 💡 Check my script here 🔗: https://lnkd.in/dwNcS5uM The journey from that seminar to building this alternative has been incredibly rewarding. Excited to continue exploring how structured knowledge can transform AI systems beyond what traditional approaches achieve. ✨ #AI #MachineLearning #RAG #KnowledgeGraphs #GraphNeuralNetworks #NLP #DataScience 

AI is not failing because of bad ideas; it’s "failing" at enterprise scale because of two big gaps: 👉 Workforce Preparation 👉 Data Security for AI While I speak globally on both topics in depth, today I want to educate us on what it takes to secure data for AI—because 70–82% of AI projects pause or get cancelled at POC/MVP stage (source: #Gartner, #MIT). Why? One of the biggest reasons is a lack of readiness at the data layer. So let’s make it simple - there are 7 phases to securing data for AI—and each phase has direct business risk if ignored. 🔹 Phase 1: Data Sourcing Security - Validating the origin, ownership, and licensing rights of all ingested data. Why It Matters: You can’t build scalable AI with data you don’t own or can’t trace. 🔹 Phase 2: Data Infrastructure Security - Ensuring data warehouses, lakes, and pipelines that support your AI models are hardened and access-controlled. Why It Matters: Unsecured data environments are easy targets for bad actors making you exposed to data breaches, IP theft, and model poisoning. 🔹 Phase 3: Data In-Transit Security - Protecting data as it moves across internal or external systems, especially between cloud, APIs, and vendors. Why It Matters: Intercepted training data = compromised models. Think of it as shipping cash across town in an armored truck—or on a bicycle—your choice. 🔹 Phase 4: API Security for Foundational Models - Safeguarding the APIs you use to connect with LLMs and third-party GenAI platforms (OpenAI, Anthropic, etc.). Why It Matters: Unmonitored API calls can leak sensitive data into public models or expose internal IP. This isn’t just tech debt. It’s reputational and regulatory risk. 🔹 Phase 5: Foundational Model Protection - Defending your proprietary models and fine-tunes from external inference, theft, or malicious querying. Why It Matters: Prompt injection attacks are real. And your enterprise-trained model? It’s a business asset. You lock your office at night—do the same with your models. 🔹 Phase 6: Incident Response for AI Data Breaches - Having predefined protocols for breaches, hallucinations, or AI-generated harm—who’s notified, who investigates, how damage is mitigated. Why It Matters: AI-related incidents are happening. Legal needs response plans. Cyber needs escalation tiers. 🔹 Phase 7: CI/CD for Models (with Security Hooks) - Continuous integration and delivery pipelines for models, embedded with testing, governance, and version-control protocols. Why It Matter: Shipping models like software means risk comes faster—and so must detection. Governance must be baked into every deployment sprint. Want your AI strategy to succeed past MVP? Focus and lock down the data. #AI #DataSecurity #AILeadership #Cybersecurity #FutureOfWork #ResponsibleAI #SolRashidi #Data #Leadership