Ethical Considerations in Biomedical Device Development

The recent development of a “dual-loop” non-invasive brain-computer interface (BCI) system by researchers at Tianjin University and Tsinghua University represents a significant advancement in reciprocal human-machine learning (see: https://lnkd.in/eDrdCF7B). The system, which has demonstrated real-time control of a drone, exemplifies rapid progress in neurotechnology, and while the stated intention is for research and clinical applications, such innovation also raises critical dual-use, neuroethical concerns that must be addressed. Dual-use technologies are those that can be utilized for both beneficial and potentially harmful purposes. The “dual-loop” BCI system, designed to enhance human-machine interactions, holds promise for augmenting human capabilities, which could be purposed for military applications, such as controlling unmanned systems or optimizing warfighter and intelligence operator performance as Rachel Wurzman and I noted some years ago in the journal STEPS (#STEPS). More broadly, this type of BCI system could be employed in other occupational settings to evaluate and affect cognitive capabilities and quality and extent of work output. If viewed through a relatively optimistic lens, this could be seen as positively valent. But this prompts questions of equity and access: such use may exacerbate social inequalities if access is limited to certain groups and widen the divide between those with enhanced capabilities and those without. Moreover, integration of such BCIs into daily life prompts several ethical questions about privacy and consent – namely unauthorized or mandatory monitoring – and influence -of an individual’s cognitive and behavioral patterns. Such engagement can be used to direct neurocognitive processes, with defined risk of controlling individual agency, and diminishing personal autonomy. And as with any emerging technology the longterm use of such a BCI system remains uncertain. To navigate these dual-use, neuroethical challenges, a multifaceted approach is recommended that entails (1) international collaboration – or at least cooperation – to establishing global standards and agreements to regulate responsible development and application of BCI technologies; (2) developing comprehensive ethical guidelines, informed by diverse multinational stakeholders to inform responsible innovation and use; (3) public engagement to enable more informed social awareness and attitudes; and (4) continuous oversight of these cooperatives to monitor – and course correct - BCI research and applications. Thus, while this “dual-loop” non-invasive BCI system offers promising advancements in human-machine interaction, it is imperative to address the associated dual-use and neuroethical issues. Proactive and collaborative efforts are essential to harness the benefits of such technologies while mitigating their potential risks. #dual loop #BCI #dual use #Neurotechnology #neuroethics

🔥 Ethics in AI-enabled medical devices is not an abstract debate. It is governance by design. In AI-enabled medical mobile health devices, ethics constitutes a governance-by-design framework that structures system behaviour and user interaction in domains where legal boundaries are evolving, indeterminate, or insufficiently expressive of the principles they intend to uphold. ⚖️ Even where permissible boundaries are formally defined, they may fail to capture proportionality, fairness, or human impact in adaptive systems. Ethics therefore performs both a pre-regulatory and interpretive function — ensuring that device architecture reflects the spirit as well as the letter of the law. Regulatory silence does not diminish responsibility. Formal compliance does not exhaust it. 🔖 With that lens in mind, I highly recommend "Teaching AI Ethics: A Guide for Educators" by Leon Furze. It is a remarkably practical resource for anyone teaching — or trying to structure thinking around — AI ethics. The book explores key domains including: 🔹Bias 🔹Environment 🔹Truth 🔹Copyright 🔹Privacy 💎 Despite not in the narrower regulatory ethics sense, I found particularly interesting the chapters on social chatbots, power concentration and the hidden workforce. A few reflections particularly resonated: 1️⃣ Copyright We are no longer debating hypotheticals. The Stability AI vs Getty Images case showed how far legal clarity still has to go. Courts may rule that models do not “store” copyrighted works, yet broader consensus questions whether algorithmic weights encode protected material. Copyright is becoming a volatile and imperfect proxy for ethical compliance, especially in multimodal GenAI and mixed authorship contexts. 2️⃣ Privacy Privacy now extends well beyond consent mechanisms. Retroactive use of training data, bystander privacy, national sovereignty, and the tension between GDPR data minimisation and large-scale model training all expose ethical boundaries that law alone does not resolve. 3️⃣ Conversational interfaces In healthcare, conversational components and adaptive interfaces further complicate emotional and relational boundaries — even in certified medical devices where boundaries must be clear and respected. 4️⃣ Power & the hidden workforce Behind AI systems lies invisible labour and increasing concentration of power. The question of alternative development models that distribute capability and accountability more broadly is not theoretical — it is structural. What this guide does exceptionally well is move ethics beyond slogans and into structured inquiry. For those working in adaptive AI, medical devices, digital health governance, or standards development — it is an excellent teaching companion and a useful provocation. Ethics, properly understood, is not about slowing innovation. It is about stabilising it. 📌 We are working to solve in this space. #AIethics #DigitalHealth #MedicalDevices #Governance #AI #Standards

When smart medical devices need to explain themselves 🔬   How do we bridge the gap between the "black box" nature of AI systems and the transparency requirements of European regulations?   A new study from researchers at the University of Zurich and University of Namur addresses this tension by developing a systematic methodology for matching explainable AI (XAI) tools with the specific requirements of GDPR, the AI Act, and Medical Device Regulation.   Medical AI represents one of the largest investment areas globally, nearly $6 billion according to Stanford's 2023 AI Index.   Yet as these systems evolve from simple diagnostic aids to sophisticated closed-loop devices that make autonomous treatment decisions, we're entering uncharted territory where algorithmic opacity meets life-or-death consequences.   The researchers created a framework that categorizes smart biomedical devices by their control mechanisms: (i) open-loop systems where humans interpret data; (ii) closed-loop systems that act autonomously; and (iii) semi-closed-loop systems that blend human and machine decision-making.   Each category triggers different regulatory requirements for explanation.   The study reveals 11 distinct "legal explanatory goals" that EU regulations pursue - from understanding system risks to interpreting specific outputs. A closed-loop epilepsy device that automatically triggers brain stimulation faces the full weight of GDPR's "right to explanation," while semi-closed-loop spinal cord stimulators have different transparency requirements.   The research acknowledges a nuanced reality often overlooked in discussions of AI regulation: simply applying an XAI algorithm doesn't guarantee meaningful explanation or regulatory compliance.   The effectiveness depends on proper implementation, appropriate audience consideration, and recognition that most existing XAI methods rely on imperfect heuristics.   As we embed AI deeper into healthcare, we're asking fundamental questions about trust, autonomy, and the nature of informed consent when the systems making recommendations are too complex for humans to fully comprehend.   The methodology provides a practical framework for developers navigating the complex intersection of innovation and regulation.   It also reveals the inherent tensions: The most transparent systems aren't always the most accurate, and the drive for explainability might sometimes conflict with clinical effectiveness.   This research suggests we need adaptive approaches that can evolve with both technological advancement and regulatory development. The framework they propose is designed to accommodate future XAI methods and emerging legal requirements - recognizing that this intersection of AI and healthcare regulation will continue to evolve. Link to the study in the first comment.

‼️ The UN has set the first global standard for neurotechnology ethics ‼️ - and I was part of it Last year, I received the invitation to join a UNESCO committee to set the first global standard for neurotechnology ethics (there has never been one before!). I then joined neuroscience, ethics, policy and legal experts from every United Nations member country to represent some critical considerations that I have come across my academic career in Harvard University and University of Oxford, as well as in my work with Samphire Neuroscience. Here are 3 things I advocated for that were missed in the early drafts: 1️⃣ Ethics considerations should of course protect vulnerable populations but NOT AT THE EXPENSE OF RESEARCHING THEM. Specifically, there's way too little research on neurotechnology in women in general (in fact we at Samphire Neuroscience are leading some of the largest studies in the field), but especially during pregnancy and teenage years. Ironically, this is when women's mental health is particularly vulnerable, and the current lack of standards in ethics means that very little research is in the pipeline to fix this gap. It's just seen as 'too hard' to run studies on women, who may have irregular cycles, be (or become) pregnant, or under the age of 18 (though spoiler alert - most hormone-driven cases of PMDD and ADHD manifest around menarche, i.e. 10-14 years of age). 2️⃣ Neurotechnology ethics should clearly distinguish between brain 'monitoring' and 'influencing/stimulating' activity. Many people (including some reviewers of ethics applications, in my experience!) don't make a distinction between technologies that merely observe brain activity (think EEG / fNIRS / MRI) and those that can actually influence it (think brain stimulation like TMS, tDCS, tACS). Unless these have special considerations and distinctions in cost/benefit analyses (mainly that observation techniques usually cannot deliver a clinical benefit unless used under very clear and validated neurofeedback protocols that are supervised), we won't get to a place where we can accurately evaluate ethical (and un-ethical) use and study of these technologies. 3️⃣ Everyone should carefully consider 'medical device' status when it comes to consumer-grade neurotechnology for ethical use. Medical devices (which all neurotechnology in Europe but not in the US is) require their evidence to be reviewed by qualified bodies to make claims around 'treating' or 'improving' conditions. In the EU, to my knowledge, there are only three brain stimulation technologies that are approved to make claims to consumers around improving conditions: Flow Neuroscience (for major depressive disorder), Sooma Medical (for major depressive disorder) and Samphire Neuroscience (for mood and pain symptoms linked to menstruation). Unless we agree that the medical device clearance process is the standard of ethics being upheld in neurotechnology development, much deeper revisions will be needed.

In my view, #Neuralink represents an intriguing yet complex intersection of technology and human biology. From an #ethics perspective, it raises significant concerns about consent and privacy. Ensuring that individuals fully comprehend what it means to have a brain-computer interface (BCI) and can provide informed consent is paramount. The potential for BCIs to access, and even manipulate, thoughts and memories is profound, necessitating robust safeguards to protect individual privacy and autonomy. When it comes to governance, establishing effective oversight structures is essential. Clear guidelines and comprehensive oversight mechanisms are needed to ensure that the development and deployment of Neuralink's technology align with societal values. This should involve a multi-stakeholder approach, incorporating input from ethicists, medical professionals, and the public, to create a balanced and transparent governance framework. Risk management is another critical area. The introduction of BCIs brings various risks, including technical failures, cybersecurity threats, and unforeseen long-term health impacts. Developing comprehensive risk management strategies is essential to identify, assess, and mitigate these risks. This involves rigorous testing, continuous monitoring, and having contingency plans in place to address potential issues. Compliance is equally important. Adhering to existing regulations and developing new regulatory frameworks tailored to the unique aspects of BCIs is crucial. This includes compliance with medical device regulations, data protection laws, and standards for clinical trials. Given the global nature of Neuralink's potential impact, achieving harmonized compliance across different jurisdictions will be a significant challenge but one that is necessary for responsible advancement. While I believe Neuralink holds promise for remarkable advancements in neuroscience and human capabilities, it also requires careful consideration and proactive management of ethical, governance, risk, and compliance issues. Ensuring that these aspects are addressed responsibly is essential for realizing the benefits of this groundbreaking technology. What do you think? #NeuroEthics #BCISafety #TechGovernance #RiskManagement #DataPrivacy

This story matters. HeLa cells became foundational to biomedical research, but the origin story is a consent failure with potential racist overtones. Henrietta Lacks remains a clear reminder that scientific progress may be extraordinary. But the originating transaction casts a dark shadow over the work. Governance matters as much as the science. US courts often treat excised tissue as not the patient’s property. Across much of Europe, the framing is less "property title" and more "human-rights, dignity, and stewardship." European norms also place guardrails on commodifying the body (which I wholeheartedly endorse). Consent, provenance, oversight, and benefit-sharing should be explicit, carefully measured, and durable.

𝗧𝗶𝘁𝗹𝗲: The Translation of In-House Imaging AI Research into a Medical Device: Ensuring Ethical and Regulatory Integrity 𝗔𝘂𝘁𝗵𝗼𝗿𝘀: Filippo Pesapane et al. 𝗗𝗢𝗜: https://hubs.li/Q034YQP80 𝗢𝘃𝗲𝗿𝘃𝗶𝗲𝘄: Here is an insightful article that explores the intricate process of transforming in-house AI imaging research into clinically integrated medical devices, emphasizing ethical and regulatory integrity. While the article focuses on navigating the EU Medical Device Regulation (MDR) and the upcoming EU AI Act, it's noteworthy that many device developers are turning to the U.S. market first due to easier regulations and shorter wait times. 𝗞𝗲𝘆 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆𝘀: 1. Identifying Clinical Needs: Begin by addressing specific healthcare challenges that AI can solve. 2. Data Management and Model Training: Use diverse, high-quality datasets to develop unbiased AI models. 3. Regulatory Navigation: Understand and comply with regulatory frameworks like the EU MDR and EU AI Act. However, the U.S. offers a more streamlined regulatory path, attracting developers seeking faster market entry. 4. Ethical Considerations: Prioritize transparency, patient privacy, and equitable access to AI technologies. 5. Interdisciplinary Collaboration: Collaborate across healthcare institutions, data scientists, and industry partners to align innovations with clinical needs. 6. Integration into Clinical Workflows: Ensure AI tools seamlessly fit into existing medical systems without disrupting workflows. 7. Human-AI Interaction: Address psychological factors like automation bias and algorithm aversion to optimize decision-making. 8. Validation and Testing: Rigorously validate AI models with continuous monitoring post-deployment to maintain performance. 9. Comparative Regulatory Insights: The EU's stringent regulations contrast with the U.S.'s more relaxed approach, as evidenced by the FDA's guidance on AI/ML-based Software as a Medical Device (SaMD). 10. Commercialization and Market Entry: Recognize challenges such as intellectual property issues, cost assessments, and the strategic importance of choosing the right market for launching AI devices. 11. Patient-Centric Approach: Ensure AI enhances patient care by improving diagnostics while maintaining the human element in healthcare. 𝗗𝗶𝘀𝗰𝘂𝘀𝘀𝗶𝗼𝗻 𝗣𝗼𝗶𝗻𝘁𝘀: • Regulatory Strategy: How can developers leverage the more accessible U.S. regulatory landscape to bring AI devices to market faster? • Alignment with FDA: What steps are essential for aligning with the FDA's framework for AI/ML-based SaMD? • Ethical Data Management: How can organizations ensure patient data privacy and mitigate biases, especially when operating across different regulatory environments? #AIinHealthcare #MedicalDevices #RegulatoryCompliance #EthicalAI #MedicalImaging #SoftwareasaMedicalDevice #FDA

🛡️ Ethical Considerations in AI for Rehabilitation and Physical Therapy 🤖 As AI technology continues to enhance rehabilitation and physical therapy, it’s crucial to address the ethical considerations that accompany its integration. While the benefits are profound, responsible use demands careful attention to several key areas. 🔒 Data Privacy AI systems handle sensitive patient information, making robust security measures essential to protect against breaches and unauthorized access. 📜 Informed Consent Patients must be fully informed about how their data will be collected, used, and shared. Transparent communication and consent processes are vital to ensure comfort and understanding regarding the technology and its implications. ⚖️ Algorithmic Bias A significant concern in AI-driven rehabilitation is algorithmic bias. AI models trained on biased data can yield skewed results, creating disparities in treatment recommendations and outcomes. Strategies must be developed to identify and mitigate these biases to ensure equitable care for all patients. 🌍 Equitable Access Ensuring that AI-powered rehabilitation is accessible and affordable is crucial. Disparities in access to technology can worsen existing health inequalities, highlighting the need for inclusive solutions that cater to individuals across socioeconomic backgrounds. 🤝 Patient-Therapist Relationship While AI can greatly enhance therapeutic practices, it should complement, not replace, the essential human touch and empathy that therapists provide. By addressing these ethical considerations, we can foster responsible AI use in rehabilitation and physical therapy, promoting fairness, transparency, and equity in healthcare. #EthicsInAI #Rehabilitation #PhysicalTherapy #DataPrivacy #InformedConsent #AlgorithmicBias #EquitableAccess #PatientTherapistRelationship #HealthTech #AIinHealthcare

Impressive paper here for us to consider as neurotechnology advances. "As neural implants become more integrated with human cognition and identity, the risks posed by their sudden discontinuation are unique and demand urgent attention." "Neural implants are transforming the treatment of neurological disorders and reshaping the boundaries between technology and human identity. But although these devices offer immense therapeutic potential, they have also given rise to a crucial ethical and societal challenge, ‘neuroabandonment’ — a term introduced here to describe the premature discontinuation of neural implants and their associated support systems1. Unlike other medical technologies, neural implants integrate deeply with the human nervous system, making their discontinuation both medically and emotionally disruptive. Here we examine the growing risks of neuroabandonment for patient wellbeing, societal trust and industry innovation, and the unique factors that distinguish neural implants from other interventions. We propose strategies to mitigate these risks, including financial safeguards, technical standardization, regulatory reforms and policy-driven interventions." https://lnkd.in/gZ6z9AVF

𝗠𝗜𝗡𝗗 𝗥𝗜𝗚𝗛𝗧𝗦: 𝗘𝗫𝗣𝗟𝗢𝗥𝗜𝗡𝗚 𝗧𝗛𝗘 𝗘𝗧𝗛𝗜𝗖𝗦 𝗢𝗙 𝗡𝗘𝗨𝗥𝗢𝗧𝗘𝗖𝗛𝗡𝗢𝗟𝗢𝗚𝗬 𝗔𝗡𝗗 𝗕𝗥𝗔𝗜𝗡 𝗜𝗡𝗧𝗘𝗥𝗙𝗔𝗖𝗘𝗦 Neurotechnology is unlocking new frontiers in human-machine interaction. Brain-computer interfaces, neural implants, and cognitive enhancement tools are no longer science fiction. They are already reshaping medicine, communication, and performance. From restoring mobility in paralysis to enabling communication through thought, neurotech holds immense promise. Companies like Neuralink, Synchron, and BrainGate are developing interfaces that decode brain signals in real time and translate them into action. But with this power comes an urgent need for ethical guardrails. Who owns brain data when thoughts become digital? What happens if a neurodevice is hacked, manipulated, or misused? How do we protect mental privacy in a world where thoughts can be read or even influenced? As neurotechnologies advance, they raise fundamental questions about autonomy, consent, and cognitive liberty. Invasive and non-invasive technologies alike blur the line between self and system, making mental sovereignty a critical concern. Several academic and policy institutions now advocate for “neurorights,” a framework to safeguard identity, free will, and mental integrity in the face of accelerating innovation. In the Bio-Digital Age, we must not just build smarter interfaces. We must build them ethically. 𝗖𝗼𝗻𝘁𝗿𝗼𝗹 𝘁𝗵𝗲 𝘁𝗲𝗰𝗵, 𝗯𝗲𝗳𝗼𝗿𝗲 𝗶𝘁 𝗰𝗼𝗻𝘁𝗿𝗼𝗹𝘀 𝘁𝗵𝗲 𝗺𝗶𝗻𝗱. Stay tuned for the next post: 𝗟𝗶𝘃𝗶𝗻𝗴 𝗠𝗮𝘁𝗲𝗿𝗶𝗮𝗹𝘀, 𝗵𝗼𝘄 𝗻𝗲𝘅𝘁-𝗴𝗲𝗻 𝗯𝗶𝗼𝗺𝗮𝘁𝗲𝗿𝗶𝗮𝗹𝘀 𝘄𝗶𝗹𝗹 𝗿𝗲𝗱𝗲𝗳𝗶𝗻𝗲 𝗺𝗲𝗱𝗶𝗰𝗶𝗻𝗲 𝗮𝗻𝗱 𝗺𝗮𝗻𝘂𝗳𝗮𝗰𝘁𝘂𝗿𝗶𝗻𝗴. #Neurotechnology #BrainComputerInterface #CognitiveLiberty #Neurorights #BioDigitalAge #EthicalAI #MentalPrivacy #BCIEthics #FutureOfNeuroscience #CosmosRevisits