AI Algorithm Transparency

Algorithmic transparency refers to the principle that the operations and decision-making processes of algorithms should be open and understandable to people who interact with or are impacted by them. It’s an aspect of accountability and fairness that seeks to mitigate the ‘black box’ nature of complex AI systems. For high-risk AI systems, strict transparency requirements will apply under the AI Act, such as adequately informing users when they interact with an AI system and making sure that its capabilities and limitations are clearly outlined. The AI Act will also require that users are aware of the AI's decision-making parameters. Companies must not only disclose how the algorithm works but also need to explain the rationale behind these decisions. This is particularly important for high-risk AI systems, where the consequences of error could be catastrophic. Transparency, in this context, evolves from being a mere buzzword to a structural necessity. The AI Act also focuses on transparency in emotion recognition and biometric categorisation, and deepfakes. For the former, the Act requires that people exposed to these AI systems must be informed, except in cases where the technology is used for criminal investigations. This exception raises ethical questions about balancing privacy with security. For the latter, deepfake technology must come with disclosure that the content isn't authentic, though exceptions exist for legal or artistic purposes. These carve-outs have provoked questions about the potential stifling of creative or journalistic endeavours. While the AI Act has taken the spotlight of AI regulation, the Digital Services Act’s provisions on recommender systems echo the AI Act's call for transparency. Recommender systems, a subset of AI technologies, also must outline their main parameters in "plain and intelligible language," echoing the AI Act's push for clear, comprehensible explanations. The DSA even mandates an explanation of why certain parameters are considered more important than others, extending the notion of transparency into the realm of accountability. Both acts show a commitment to user agency. The AI Act ensures that the user retains a degree of control when interacting with high-risk AI systems, including an ‘off switch’. Meanwhile, the DSA promotes user agency by compelling platforms to allow users to modify their preferences. The AI Act introduces obligatory risk assessments for high-risk applications, mirroring the DSA's requirements for platforms to conduct comprehensive risk assessments. Here, we witness two regulatory streams converging into a river of algorithmic accountability, encouraging a more nuanced, ethical approach to AI development and implementation. Laws on algorithmic transparency reflect the a paradigm shift in our approach to the ethical and social implications of AI. The importance of such legislation will only intensify as AI becomes increasingly interwoven into the fabric of our lives.

Medical AI can't earn clinicians' trust if we can't see how it works - this review shows where transparency is breaking down and how to fix it. 1️⃣ Most medical AI systems are "black boxes", trained on private datasets with little visibility into how they work or why they fail. 2️⃣ Transparency spans three stages: data (how it's collected, labeled, and shared), model (how predictions are made), and deployment (how performance is monitored). 3️⃣ Data transparency is hampered by missing demographic details, labeling inconsistencies, and lack of access - limiting reproducibility and fairness. 4️⃣ Explainable AI (XAI) tools like SHAP, LIME, and Grad-CAM can show which features models rely on, but still demand technical skill and may not match clinical reasoning. 5️⃣ Concept-based methods (like TCAV or ProtoPNet) aim to explain predictions in terms clinicians understand - e.g., redness or asymmetry in skin lesions. 6️⃣ Counterfactual tools flip model decisions to show what would need to change, revealing hidden biases like reliance on background skin texture. 7️⃣ Continuous performance monitoring post-deployment is rare but essential - only 2% of FDA-cleared tools showed evidence of it. 8️⃣ Regulatory frameworks (e.g., FDA's Total Product Lifecycle, GMLP) now demand explainability, user-centered design, and ongoing updates. 9️⃣ LLMs (like ChatGPT) add transparency challenges; techniques like retrieval-augmented generation help, but explanations may still lack faithfulness. 🔟 Integrating explainability into EHRs, minimizing cognitive load, and training clinicians on AI's limits are key to real-world adoption. ✍🏻 Chanwoo Kim, Soham U. Gadgil, Su-In Lee. Transparency of medical artificial intelligence systems. Nature Reviews Bioengineering. 2025. DOI: 10.1038/s44222-025-00363-w (behind paywall)

AI explainability is critical for trust and accountability in AI systems. The report “AI Explainability in Practice” highlights key principles and practical steps to ensure AI decisions are transparent, fair, and understandable to diverse stakeholders. Key takeaways: • Explanations in AI can be process-based (how the system was designed and governed) or outcome-based (why a specific decision was made). Both are essential for trust. • Clear, accessible explanations should be tailored to stakeholders’ needs, including non-technical audiences and vulnerable groups such as children. • Transparency and accountability require documenting data sources, model selection, testing, and risk assessments to demonstrate fairness and safety. • Effective AI explainability includes providing rationale, responsibility, safety, fairness, data, and impact explanations. • Use interpretable models where possible, and when black-box models are necessary, supplement with interpretability tools to explain decisions at both local and global levels. • Implementers should be trained to understand AI limitations and risks and to communicate AI-assisted decisions responsibly. • For AI systems involving children, additional care is required for transparent, age-appropriate explanations and protecting their rights throughout the AI lifecycle. This framework helps organizations design and deploy AI that stakeholders can trust and engage with meaningfully. #AIExplainability #ResponsibleAI #HealthcareInnovation Peter Slattery, PhD The Alan Turing Institute

𝗔𝗜 𝗗𝗼𝗲𝘀𝗻'𝘁 𝗡𝗲𝗲𝗱 𝘁𝗼 𝗕𝗲 𝗮 𝗕𝗹𝗮𝗰𝗸 𝗕𝗼𝘅: 𝗪𝗵𝘆 𝗥𝗲𝗴𝘂𝗹𝗮𝘁𝗲𝗱 𝗜𝗻𝗱𝘂𝘀𝘁𝗿𝗶𝗲𝘀 𝗦𝗵𝗼𝘂𝗹𝗱 𝗗𝗲𝗺𝗮𝗻𝗱 𝗧𝗿𝗮𝗻𝘀𝗽𝗮𝗿𝗲𝗻𝘁 𝗔𝗜 𝗢𝘃𝗲𝗿 𝗕𝗹𝗶𝗻𝗱 𝗧𝗿𝘂𝘀𝘁 “Black box” AI is treated as normal. In finance and healthcare, it’s a liability. In this article, I outline what transparent AI actually means (traceability, justification, repeatability), why it matters for audits and accountability, and how to bake it into vendor selection and rollout. If a model can’t explain a decision, you can’t defend it.

Healthcare and Medicine domains are not immune to the "Black Box" challenge of AI systems. Medical artificial intelligence (AI) systems hold promise for transforming healthcare by supporting clinical decision-making in diagnostics and treatment. The effective deployment of medical AI requires trust among key stakeholders, including patients, providers, developers, and regulators. This level of trust can be built by ensuring transparency in medical AI, including in its design, operation, and outcomes. Many AI systems function as opaque "black boxes," making it difficult for clinicians to understand how they reach decisions. This lack of interpretability poses significant risks in healthcare settings where understanding the reasoning behind AI recommendations is crucial for patient safety and clinical decision-making. This paper (https://lnkd.in/gzRs595P) provides a comprehensive overview of transparency requirements for medical AI systems throughout their entire development and deployment lifecycle. An interesting read for those interested in exploring the criticality of transparent AI in an applied context. The paper emphasizes that transparency is not just a technical requirement but a fundamental prerequisite for building trust and ensuring the safe, effective deployment of AI in healthcare settings. Some key areas that have been covered in the paper are: A. Transparency Requirements 1. Data Transparency Clear documentation of training data sources, demographics, and potential biases. Addressing issues like dataset representation and labeling quality. Managing privacy concerns. 2. Model Development Transparency Documenting model architectures, training procedures, and validation methods Using standardized reporting guidelines like CONSORT-AI and MI-CLAIM. 3. Deployment Transparency Continuous monitoring of model performance in real-world clinical settings. Human-in-the-loop systems that maintain clinician oversight. Clear communication of model limitations and appropriate use cases. B. Explainable AI Techniques 1. Feature Attribution Methods: Techniques like SHAP and LIME that highlight which inputs most influenced a prediction. 2. Concept-Based Explanations: Methods that explain AI decisions in terms of human-understandable medical concepts rather than raw features. 3. Counterfactual Explanations: Showing how changes to inputs would alter predictions, helping clinicians understand decision boundaries. C. Regulatory Landscape Clear documentation of intended use and limitations. Evidence of real-world performance validation. Mechanisms for ongoing monitoring and updates. D. Challenges and Future Directions 1. Technical Limitations: Current explanation methods may not always reflect true model reasoning, and some techniques can be manipulated. 2. Clinical Integration 3. Democratization #ai #artificialintelligence #healthcare #lifesciences #explainableai

✨New working paper on the trade-offs involved in AI transparency in news 🤖📝 How does a global news organisation disclose its use of AI? Where, when and how should readers be told when algorithms shape the news they consume? Based on a case study of the Financial Times and led by Liz Lohn we argue that transparency about AI in news is best understood as a spectrum, evolving with tech advancements, commercial, professional and ethical considerations and shifting audience attitudes. 🔗Pre-print: https://lnkd.in/gV3dPXgS 1️⃣ AI‑transparency ≠ a binary. At the FT it’s a hybrid of policy, process and practice. Senior leadership sets explicit principles, cross‑functional panels vet new applications, and AI use is signposted in internal/external tools and reinforced through training. 2️⃣ Disclosure is calibrated to context. Internally, full disclosure aims to reduce frictions and surfaces errors early; externally, labels are scaled with autonomy and oversight. No‑human‑in‑the‑loop features (e.g. Ask FT) get prominent warnings, whereas AI‑assisted, journalist‑edited outputs (e.g. bullet‑point summaries) get lighter labelling. 3️⃣ Nine factors shape what, when & how the FT discloses AI use. These include legal/provider requirements, industry benchmarking, the degree of human oversight, the nature of the task, system novelty, audience expectations & research, perceived risk, commercial sensitivities and design constraints. 4️⃣ Persistent challenges include achieving consistent labelling (especially on mobile), breaking organisational silos, keeping pace with evolving models and norms, guarding against creeping human over‑reliance, and mitigating against “transparency backfire” where disclosures reduce trust. For those of you more academically interested in this, we argue that AI transparency at the FT is shaped by isomorphic pressures – regulations, peer practices and audience expectations – and by intersecting institutional logics. Internally, managerial and commercial logics push for efficient adoption and risk management; externally, professional journalism ethics and commercial imperatives drive an aim to remain trustworthy. Crucially, we argue that AI transparency is best seen as a spectrum: optimising one factor (e.g. maximum disclosure) can undermine others (e.g. perceived trust or revenue). There does not seem to be a one‑size‑fits‑all rule; instead transparency must adapt to org context, audiences and technology. We are very grateful to the team at the Financial Times, particularly Matthew Garrahan, for supporting this study from the outset – and to the participants from the FT who volunteered their precious time to help us in understanding this issue. Feedback welcome, especially on the theoretical section and the discussion as well as literature that we will have missed! So feel free to plug your own or other people’s material, all of which will be appreciated as Liz and I work towards a journal submission.

1,000+ FDA-cleared medical AI devices. Most disclose almost nothing about how they work. We reviewed 1,012 FDA-approved or cleared AI/ML medical devices across five decades and scored each on transparency across 17 basic characteristics — training data, validation, performance metrics, demographics, and update plans. The average score was 3.3 / 17. Here's what surprised us: - Nearly half reported no clinical study. - >50% reported no performance metric. - 93% didn’t disclose where training data came from. - Only 1.5% described how the model would change over time. This was after the FDA released Good Machine Learning Practice guidance in 2021. Transparency improved by less than one point. What stood out most: being more transparent didn’t speed approval. If anything, higher-quality submissions took longer. The system quietly rewards minimal disclosure, safe predicates, and retrospective validation. And clinicians are left deploying tools they can’t meaningfully interrogate. If you’re a clinician, regulator, or building medical AI: what should be non-negotiable before an AI tool reaches patients? Paper link in comments. Viraj Mehta Nigam Shah Kevin Schulman

The European Commission published its first draft of the “Code of Practice on Transparency of AI‑Generated Content” designed as a tool to help organizations demonstrate alignment with the transparency requirements (Art. 50) of the AI Act. Article 50 of the AI Act includes obligations for providers to mark AI-generated or manipulated content in a machine-readable format, and for users who deploy generative AI systems for professional purposes to clearly label deepfakes and AI-text publications on matters of public interest. The document is divided into two sections. The first section covers rules for marking and detecting AI content, applicable to providers of generative AI systems, including to: - Use a Multi‑layered machine-readable marking of AI‑generated content - Use imperceptible watermarks interwoven within content - Adopt a digitally signed “manifest/provenance certificate” for content that can’t securely carry metadata - Offer free detection interfaces/tools, including confidence scoring, and complementary forensic detection that does not rely on active marking - Test against common transformations and adversarial attacks - Use open standards and shared/aggregated verifiers to enable cross-platform detection and lower compliance friction The second section covers labelling deepfakes and certain AI-generated or manipulated text on matters of public interest and is applicable to deployers of generative AI systems, including: - Deepfake labelling - Modality‑specific labelling rules for real-time video, non-real-time video, images, multimodal content, and audio-only - Operational governance: encourages internal compliance documentation, staff training, accessibility measures, and mechanisms to flag and fix missing/incorrect labels.

From Policy to Practice: The Era of Algorithmic Accountability has Arrived in NYC. For years, we’ve talked about “Ethical AI” as a vague goal. In 2026, New York City is proving it’s a legal requirement. With the GUARD Act now in full enforcement, NYC has officially moved past the "Wild West" of algorithmic deployment. As someone following the intersection of tech and civil rights, the shift we are seeing is monumental. It’s no longer just about what the tech can do—it’s about how it accounts for its impact on the 8.3 million people who live here. What does "Algorithmic Accountability" actually look like in NYC right now? ✅ The Public Right to Know: Through the new Office of Algorithmic Data Accountability (OAA), city agencies must now list every AI system in a public directory. Transparency is no longer optional; it’s the baseline. ✅ Mandatory Bias Audits: Whether it’s Local Law 144 governing hiring or the GUARD Act overseeing public services, "black box" algorithms are being opened up for independent review. ✅ Proactive Protection: We are moving from reacting to algorithmic harm to preventing it through pre-deployment risk assessments. New York isn't just regulating AI; we are building a blueprint for how a modern city protects its digital citizens without stifling innovation. The takeaway for tech leaders and policymakers? Compliance is the new competitive advantage. If your AI isn’t transparent, it isn’t ready for New York. #NYTech #AIGovernance #GUARDAct #AlgorithmicAccountability #NYCAI #PrivacyRights #TechPolicy2026

Transparency in AI isn’t just about trust—it’s about survival in regulated industries. The debate around AI "black boxes" is heating up, especially in finance and insurance. Here's why it's crucial to understand how AI thinks: 1. Regulatory compliance: Internal and external regulators will demand transparency. 2. Bias detection: Proving fairness and lack of bias becomes essential. 3. Decision tracing: A perfect lineage of thoughts, observations, and actions (both AI and human) is necessary. 4. Appeal processes: Understanding AI reasoning allows for effective appeals of outcomes. 5. Future-proofing: Building explainability now prepares you for upcoming regulatory requirements. Remember, AI thinks differently from humans. It can be brilliantly right or spectacularly wrong in unexpected ways. That's why chain of thought, observability, and explainability are non-negotiable in regulated industries. If you're not prioritizing this, start now. It's the key to successfully deploying AI agents in production while meeting compliance and regulatory demands. Keep building, but build with transparency in mind.