Addressing User Concerns About AI Data Use

⚠️Privacy Risks in AI Management: Lessons from Italy’s DeepSeek Ban⚠️ Italy’s recent ban on #DeepSeek over privacy concerns underscores the need for organizations to integrate stronger data protection measures into their AI Management System (#AIMS), AI Impact Assessment (#AIIA), and AI Risk Assessment (#AIRA). Ensuring compliance with #ISO42001, #ISO42005 (DIS), #ISO23894, and #ISO27701 (DIS) guidelines is now more material than ever. 1. Strengthening AI Management Systems (AIMS) with Privacy Controls 🔑Key Considerations: 🔸ISO 42001 Clause 6.1.2 (AI Risk Assessment): Organizations must integrate privacy risk evaluations into their AI management framework. 🔸ISO 42001 Clause 6.1.4 (AI System Impact Assessment): Requires assessing AI system risks, including personal data exposure and third-party data handling. 🔸ISO 27701 Clause 5.2 (Privacy Policy): Calls for explicit privacy commitments in AI policies to ensure alignment with global data protection laws. 🪛Implementation Example: Establish an AI Data Protection Policy that incorporates ISO27701 guidelines and explicitly defines how AI models handle user data. 2. Enhancing AI Impact Assessments (AIIA) to Address Privacy Risks 🔑Key Considerations: 🔸ISO 42005 Clause 4.7 (Sensitive Use & Impact Thresholds): Mandates defining thresholds for AI systems handling personal data. 🔸ISO 42005 Clause 5.8 (Potential AI System Harms & Benefits): Identifies risks of data misuse, profiling, and unauthorized access. 🔸ISO 27701 Clause A.1.2.6 (Privacy Impact Assessment): Requires documenting how AI systems process personally identifiable information (#PII). 🪛 Implementation Example: Conduct a Privacy Impact Assessment (#PIA) during AI system design to evaluate data collection, retention policies, and user consent mechanisms. 3. Integrating AI Risk Assessments (AIRA) to Mitigate Regulatory Exposure 🔑Key Considerations: 🔸ISO 23894 Clause 6.4.2 (Risk Identification): Calls for AI models to identify and mitigate privacy risks tied to automated decision-making. 🔸ISO 23894 Clause 6.4.4 (Risk Evaluation): Evaluates the consequences of noncompliance with regulations like #GDPR. 🔸ISO 27701 Clause A.1.3.7 (Access, Correction, & Erasure): Ensures AI systems respect user rights to modify or delete their data. 🪛 Implementation Example: Establish compliance audits that review AI data handling practices against evolving regulatory standards. ➡️ Final Thoughts: Governance Can’t Wait The DeepSeek ban is a clear warning that privacy safeguards in AIMS, AIIA, and AIRA aren’t optional. They’re essential for regulatory compliance, stakeholder trust, and business resilience. 🔑 Key actions: ◻️Adopt AI privacy and governance frameworks (ISO42001 & 27701). ◻️Conduct AI impact assessments to preempt regulatory concerns (ISO 42005). ◻️Align risk assessments with global privacy laws (ISO23894 & 27701).   Privacy-first AI shouldn't be seen just as a cost of doing business, it’s actually your new competitive advantage.

The EDPB recently published a report on AI Privacy Risks and Mitigations in LLMs.   This is one of the most practical and detailed resources I've seen from the EDPB, with extensive guidance for developers and deployers. The report walks through privacy risks associated with LLMs across the AI lifecycle, from data collection and training to deployment and retirement, and offers practical tips for identifying, measuring, and mitigating risks.   Here's a quick summary of some of the key mitigations mentioned in the report:   For providers: • Fine-tune LLMs on curated, high-quality datasets and limit the scope of model outputs to relevant and up-to-date information. • Use robust anonymisation techniques and automated tools to detect and remove personal data from training data. • Apply input filters and user warnings during deployment to discourage users from entering personal data, as well as automated detection methods to flag or anonymise sensitive input data before it is processed. • Clearly inform users about how their data will be processed through privacy policies, instructions, warning or disclaimers in the user interface. • Encrypt user inputs and outputs during transmission and storage to protect data from unauthorized access. • Protect against prompt injection and jailbreaking by validating inputs, monitoring LLMs for abnormal input behaviour, and limiting the amount of text a user can input. • Apply content filtering and human review processes to flag sensitive or inappropriate outputs. • Limit data logging and provide configurable options to deployers regarding log retention. • Offer easy-to-use opt-in/opt-out options for users whose feedback data might be used for retraining.   For deployers: • Enforce strong authentication to restrict access to the input interface and protect session data. • Mitigate adversarial attacks by adding a layer for input sanitization and filtering, monitoring and logging user queries to detect unusual patterns. • Work with providers to ensure they do not retain or misuse sensitive input data. • Guide users to avoid sharing unnecessary personal data through clear instructions, training and warnings. • Educate employees and end users on proper usage, including the appropriate use of outputs and phishing techniques that could trick individuals into revealing sensitive information. • Ensure employees and end users avoid overreliance on LLMs for critical or high-stakes decisions without verification, and ensure outputs are reviewed by humans before implementation or dissemination. • Securely store outputs and restrict access to authorised personnel and systems.   This is a rare example where the EDPB strikes a good balance between practical safeguards and legal expectations. Link to the report included in the comments.   #AIprivacy #LLMs #dataprotection #AIgovernance #EDPB #privacybydesign #GDPR

Using AI in Research? Transparency Isn’t Optional. As more researchers integrate AI tools for transcription, coding, or analysis, we’re also seeing a rise in participant concerns — and, increasingly, refusals — based on misconceptions about what AI actually does with their data. And honestly? Those concerns are valid. AI introduces new questions about privacy, data flow, and security. Participants deserve clarity, not jargon. Here’s the approach I’ve been championing, grounded in the STRESS Framework™ (Sensitivity, Transparency, Responsibility, Ethics, Skepticism, Security): 🔍 Be transparent: Tell participants when AI is used, what it does and doesn’t do, and how long data is stored. 🛡️ Prioritize security: Use vetted tools, encryption, and clear deletion timelines. 🧭 Stay ethical: Participation should always be voluntary — misconceptions are an opportunity to clarify, not persuade. 🤝 Build trust: Explain that AI assists with tasks like transcription, but human researchers still verify and interpret everything. 📄 Document responsibly: Keep clear records of how AI is used, how decisions are made, and how risks are mitigated. When participants understand the process, they’re more empowered — and our research becomes more ethical, transparent, and trustworthy. If you're looking to strengthen your own AI-use statements, consent materials, or research protocols, the STRESS Framework Assistant is an excellent tool to help you structure responsible AI documentation: 👉 https://lnkd.in/esFZEx34

Before I try any new AI tool, whether for my personal use or for work, I ask their customer support the following security-related questions (feel free to copy/paste): 1. Do you use customer data to train, fine-tune, or evaluate AI models beyond my individual account? * Prevent cross-customer learning. 2. If yes, is that data fully de-identified or aggregated? * Reduce re-identification risk. 3. Are AI models trained internally, by third-party providers, or both? * Know who actually touches the data. 4. Is customer data ever used to improve outputs for other customers? * Avoid silent data sharing. 5. Are AI interactions scoped strictly to my account context, or do models learn across customers? * Ensure my data stays mine. 6. Which third-party AI or ML providers process customer data? * Understand the extended trust chain. 7. Do those providers retain, log, or use customer data for their own training? * Avoid backdoor training use. 8. How long is customer data retained for AI or ML purposes? * Limit long-tail exposure. 9. If I request deletion, is my data removed from all downstream systems, including training or evaluation datasets? * Important one - this is nearly impossible to do once the toothpaste is out of the tube. If they say “yes,” then it’s a warning sign that the rest of their answers aren’t accurate. 10. What technical and contractual safeguards prevent misuse of customer data? Verify enforceable controls, not promises. This isn’t paranoia. It’s baseline data and privacy hygiene. AI is moving fast. Trust still has to be earned deliberately. If a vendor can’t answer these clearly, that’s the answer.

This Stanford study examined how six major AI companies (Anthropic, OpenAI, Google, Meta, Microsoft, and Amazon) handle user data from chatbot conversations.  Here are the main privacy concerns. 👀 All six companies use chat data for training by default, though some allow opt-out 👀 Data retention is often indefinite, with personal information stored long-term 👀 Cross-platform data merging occurs at multi-product companies (Google, Meta, Microsoft, Amazon) 👀 Children's data is handled inconsistently, with most companies not adequately protecting minors 👀 Limited transparency in privacy policies, which are complex and hard to understand and often lack crucial details about actual practices Practical Takeaways for Acceptable Use Policy and Training for nonprofits in using generative AI: ✅ Assume anything you share will be used for training - sensitive information, uploaded files, health details, biometric data, etc. ✅ Opt out when possible - proactively disable data collection for training (Meta is the one where you cannot) ✅ Information cascades through ecosystems - your inputs can lead to inferences that affect ads, recommendations, and potentially insurance or other third parties ✅ Special concern for children's data - age verification and consent protections are inconsistent Some questions to consider in acceptable use policies and to incorporate in any training. ❓ What types of sensitive information might your nonprofit staff  share with generative AI?  ❓ Does your nonprofit currently specifically identify what is considered “sensitive information” (beyond PID) and should not be shared with GenerativeAI ? Is this incorporated into training? ❓ Are you working with children, people with health conditions, or others whose data could be particularly harmful if leaked or misused? ❓ What would be the consequences if sensitive information or strategic organizational data ended up being used to train AI models? How might this affect trust, compliance, or your mission? How is this communicated in training and policy? Across the board, the Stanford research points that developers’ privacy policies lack essential information about their practices. They recommend policymakers and developers address data privacy challenges posed by LLM-powered chatbots through comprehensive federal privacy regulation, affirmative opt-in for model training, and filtering personal information from chat inputs by default. “We need to promote innovation in privacy-preserving AI, so that user privacy isn’t an afterthought." How are you advocating for privacy-preserving AI? How are you educating your staff to navigate this challenge? https://lnkd.in/g3RmbEwD

Claude.ai just announced that their Microsoft 365 connector is now available on EVERY plan, including free and personal accounts. That means ANY of your end users with a free Claude account can now connect it directly to your company's Microsoft 365 environment and start pulling in emails, files, spreadsheets, whatever they have access to. That should make you uncomfortable. Because unless your tenant requires admin approval for third-party app connections, any employee can enable this on their own. No ticket, no approval, no one in leadership even knows it happened. And now sensitive client data is sitting inside a platform you didn't evaluate, didn't approve, and don't control. A public AI model is potentially learning from your sensitive data, and almost definitely storing it. This isn't a Claude problem. Every major AI platform is racing to build connectors into your business tools, and every one of them is a potential data exposure event if you're not ready. Here's what I'd recommend doing as soon as possible: - Lock down third-party app permissions. Require admin approval for all app connections in your Microsoft 365 tenant. If you're not sure whether this is on, assume it isn't. - Audit your environment. Do you know where your sensitive data lives and who can access it? Most companies find out the hard way that employees are over-permissioned, and AI makes that exponentially more dangerous because it makes finding and extracting data faster than ever. - Communicate and educate. Most employees aren't being reckless, they just don't know this is a problem. Send a simple message this week: don't connect any AI tools to company systems without approval. Then start building a real AI use policy, even a one-pager. - Review your client agreements. If you handle sensitive client data, your contracts probably don't address AI processing yet. Close that gap before a client asks about it. This isn't about being anti-AI. Every new AI capability is a new governance question, and most businesses aren't asking it fast enough. At the same time, it's imperative that companies start preparing for AI integration because it is inevitable for those that want to move forward with technology in a meaningful way. Have questions? Shoot me a message. Client or not, I'm happy to chat more if I can help!

Is the AI you’re using healthy for you? Kasia Chmielinski argued that just as food products come with nutrition labels detailing their ingredients, AI systems should also have clear labels that inform users about their data sources, algorithms, and decision-making processes. This transparency helps users understand how AI systems function and what influences their outputs. Users can make informed decisions about whether to trust and use a particular AI. This empowerment is crucial in a world where AI increasingly impacts daily life. But design and global standardization of these AI “nutrition labels” are still absent. Calls for global consensus on AI transparency standards are yet to be noticed. Putting it into motion through legislations and reinforcing this practice will be another story. In the meantime, here are 5 practices we can undertake to ensure that we’re using healthy AI systems in our organizations. 1️⃣ Demand transparency from vendors. Understand the training data, the model's decision-making process, and any biases that might exist. 2️⃣ Incorporate ethical considerations into your AI strategy. This will create a culture of ethical AI use in your organization. 3️⃣ Assess your AI system for biases, errors, and vulnerabilities. This confirms that the system is operating as intended and ethically. 4️⃣ Collaborate and create your standards. Engage with industry groups, policymakers, and academic institutions to help shape the development of global standards for AI transparency and ethics. 5️⃣ Invest in Explainable AI (XAI). Develop or choose AI systems that provide clear explanations for their decisions. By taking these steps, we can move towards a future where AI is developed and used responsibly, benefiting society as a whole. How are you ensuring the health and ethical integrity of your AI systems? Share your thoughts and practices in the comments. Let’s lead the way in making AI transparent, fair, and trustworthy. #AI #AIEthics #Tech #Innovation

"What does GPT-3 know about me?", published by MIT Technology Review, discusses the capabilities and limitations of OpenAI's language model GPT-3 in accessing, processing, and generating personal information from the vast data it has been trained on. TLDR; Be mindful of what you share online but also cautious about interacting with LLMs: Be aware that LLMs may generate responses that seem personal, but these responses are based on the data they have been trained on, not on any specific knowledge of you. Educate yourself about LLMs: The more you know about how LLMs work, the better equipped you will be to protect your privacy. 1. GPT-3, a powerful language model, has consumed a massive amount of data from the internet and can generate plausible responses based on patterns it has learned from this data. 2. Some users have observed that GPT-3 seems to know personal details about individuals, such as their hobbies, professions, or family members, raising concerns about privacy. 3. While GPT-3 does not specifically know individuals or have specific knowledge of their personal information, it generates responses based on statistical patterns and correlations in the data it has learned from. 4. The perception of GPT-3 knowing people might stem from the model's ability to generate coherent and plausible narratives, which can be mistaken for genuine knowledge. 5. It is important to recognize the distinction between GPT-3 possessing actual knowledge and its ability to generate convincing and human-like text. Key Recommendations: 1. Educate users about AI language models: Help users understand how AI models like GPT-3 function, their capabilities and limitations, and how they process data. This understanding will help users have realistic expectations and address privacy concerns. 2. Establish AI literacy programs: Encourage educational initiatives that teach individuals about the workings of AI models and promote a better understanding of AI-generated content. 3. Develop transparent AI practices: Encourage AI developers to provide clear and accessible explanations of how AI models work and what type of data they use, helping to build trust and address privacy concerns. 4. Encourage ethical AI development: Support the development of AI systems that prioritize user privacy, data security, and ethical use of personal information. 5. Promote collaboration between AI developers, policymakers, and consumers: Foster open dialogue and collaboration among stakeholders to create guidelines and best practices that ensure responsible and ethical use of AI models like GPT-3. https://lnkd.in/eM89YEtU

AI adoption is accelerating across every part of the enterprise, but governance and data protection must keep pace. The conversation should not solely focus on the speed of deploying AI tools; it also needs to address how responsibly we use them. Every organization should be asking a few fundamental questions: - What data is being entered into AI tools? - Where is that data stored? - Who has access to it? - How is it being retained, protected, and governed? - Are employees clear on what is and is not acceptable to use? AI can create real value through productivity, automation, decision support, and improved customer and employee experiences. However, without the right guardrails, it can introduce unnecessary risks around sensitive data, intellectual property, privacy, compliance, and security. Strong AI adoption requires more than enthusiasm; it requires clear governance, defined policies, data classification, access controls, vendor due diligence, monitoring, and workforce education. Often, the biggest risk is not malicious intent but well-meaning employees using powerful tools without understanding the downstream implications. The companies that will benefit most from AI will not be those that move recklessly but those that act with speed, discipline, and accountability. Adopt AI aggressively, but govern it intentionally. Innovation and protection must move together. #AI #AIGovernance #Cybersecurity #DataSecurity #DigitalTransformation #TechnologyLeadership

Your data is being used to train AI—unless you know this. 60% of professionals hesitate to use AI because of privacy concerns. If that’s you, here’s the truth: Your prompts might be training someone else’s model—unless you take control. Here’s a quick breakdown of how leading LLMs handle your data: 🧠 ChatGPT by OpenAI – Can you opt out? Yes – in Settings → Data Controls, turn off "Improve the model for everyone". – Temporary Chats aren’t saved or used for training. – Enterprise users? Your data is never used for training by default. 🛡️ Claude by Anthropic – No opt-out needed. Claude doesn’t train on your data unless you explicitly opt in (via feedback). – Deleted chats are wiped from their systems in 30 days. – Feedback (thumbs up/down) may still be used for improvement. 🔍 Gemini by Google – By default, chats are saved for 18 months and used to train models. – Turn off “Gemini Apps Activity” in Google Account settings to stop this. – Human reviewers may still see a portion of anonymized chats—for up to 3 years. – Pro tip: Never enter confidential info into Gemini. 🧭 Perplexity AI – Go to Settings → AI Data Usage and toggle off model training. – With that off, your chats won’t be used to improve anything. – Use Incognito Mode for fully private sessions (no memory, no retention). 💼 Microsoft Copilot – Enterprise data? Safe. It’s excluded from model training by default. – Consumer accounts? Training is on by default—but opt-outs are available in your profile’s privacy settings. – You can also disable memory and personalization at any time. 🔒 The takeaway: You don’t need to stop using AI. You just need to know how each platform handles your data—and how to take control of it. #AI #Privacy #LLM #DataSecurity #ChatGPT #Claude #Gemini #Perplexity #MicrosoftCopilot #ArtificialIntelligence #PromptEngineering